text:
# TURBO18: Boost WABT Performance using JitBuilder

In this tutorial you will learn how to integrate a just-in-time (JIT) compiler into
a language runtime.  The WebAssembly Binary Toolkit (WABT) provides a number of tools
for working with WebAssembly (Wasm) files, and the objective of this tutorial is to
provide a JIT compiler for the WABT interpreter.  The JIT compiler is based on
Eclipse OMR, which is a toolkit of runtime technology components that can be extended
and customized for the needs of different runtimes.  This tutorial uses a simplified
interface to the compiler component called JitBuilder, which makes it easy for a runtime
to describe methods that can be input into the JIT for compilation.

The concepts you will learn here are the same general steps you need to consider when
building a dynamic compiler for any language runtime.

You will not be building a complete Wasm JIT from scratch.  Rather, you will be building
upon (and in some cases re-implementing) the functionality already provided by an existing Wasm
JIT for the WABT interpreter.  This is mainly so that you can continuously build as you follow
the exercises, watch the evolution of your changes, and run a real workload that demonstrates
the effect of JITing code.

The workload is an implementation of a Mandelbrot viewer that will plot the result.
The speed at which the set is computed and the plot completed is a function of the speed
of execution of the underlying Wasm functions.  The faster you can make your Wasm functions
(i.e., by compiling them) the faster the plot will complete.  You will use this Mandelbrot
viewer as a measure of your progress.

## How to Complete the Tutorial

The means by which you complete this tutorial is up to you!  You may set your own pace
and complete the exercises as you choose.  You should be able to build after each exercise
is complete to validate your work.

The best way to make the most of this tutorial is to go through each exercise and
complete the code yourself by following the hints provided and diving into and exploring
the JitBuilder and wasm-jit codebases.

The source code has been annotated with comments `// YOUR CODE HERE` to guide you
to the right place in the source code to implement your solution.

Alternatively, solutions for each exercise are provided at the end of this tutorial booklet that
you can either cut-and-paste or type in directly.  Links to the GitHub repository are also provided.
Typing the solutions in yourself
may give you more pause to think about the details of the solution.  A cut-and-paste
approach will allow you to complete this tutorial very quickly and allow you to get
a high-level overview of the steps in involved in integrating a JIT compiler in a
runtime.

## Getting Started

Before providing you with an overview of the technologies you will be working with in this
tutorial, you may wish to begin installing and building the components you will need
to complete the tutorial.  Because building may take several minutes, you can read the
background material while the installation is proceeding.

But feel free to to skip ahead and then come back to this step when you're ready.

### Prerequisites

To complete this tutorial you will need:

* A laptop running Linux or macOS (Windows is not yet supported in this tutorial)
* a C++11 toolchain (gcc, Clang)
* CMake 2.6+, and a supported backend build system (make, Ninja)
* Python 2.7+ (WABT requirement)
* git

### Cloning, Building, and Running the Project

Start by cloning the repository.
The `turbo` branch will be checked out by default.
Depending on your network connection, this step should take approximately 25 seconds.

```sh
git clone --recursive https://github.com/omr-turbo/wasmjit-omr.git
cd wasmjit-omr
```

Next, build the project.  This step typically takes around 7 minutes depending
on your build machine and development environment.  If you are building in a
virtual machine then building may take significantly longer.

```sh
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release ..     # optionally with -GNinja
make libc-interp                        # or ninja libc-interp
env time ./libc-interp mandelbrot.wasm
```

Run and time the execution of the `mandelbrot.wasm` example using the
`libc-interp` tool. You should see output similar to:

```
$ env time ./libc-interp mandelbrot.wasm

                                                       *
                                                     ****
                                                     ****
                                             *        **
                                             **  ************
                                             ********************
                                             *******************
                                            *********************
                                          *************************
                                  *       ************************
                               ********  *************************
                              ********** *************************
                             *********** ************************
            ***************************************************
                             *********** ************************
                              ********** *************************
                               ********  *************************
                                  *       ************************
                                          *************************
                                            *********************
                                             *******************
                                             ********************
                                             **  ************
                                             *        **
                                                     ****
                                                     ****
                                                       *

exit_group(0) called
_start() => error: host function trapped
  at $_Exit [@7383]
  at $exit [@7947]
  at $__libc_start_main [@7202]
  at $_start_c [@436]
  at $_start [@127]
21.35user 0.01system 0:21.37elapsed 99%CPU (0avgtext+0avgdata 3172maxresident)k
0inputs+0outputs (0major+916minor)pagefaults 0swaps
```

Note that the error status reported is expected!

* * *

## WABT Overview

The [WebAssembly Binary Toolkit](https://github.com/WebAssembly/wabt) (WABT) is a suite
of tools for working with WebAssembly.  The tools are written in C/C++ and are designed
for easy integration into other projects.

This tutorial builds upon the [wasmjit-omr](https://github.com/wasmjit-omr/wasmjit-omr)
project, which provides a rudimentary JIT for the Wasm interpreter library in WABT.

The tool we will be focused on for this tutorial is **libc-interp**.  This is a
stack-based interpreter that can decode and interpret the Wasm instructions in a
Wasm binary file.  It also provides a library of utility functions that can be called at
runtime from Wasm functions, similar to the C runtime for C applications.

The bulk of the source for the interpreter can be found in `src/` with JIT specializations
in `src/jit/`.  The main interpreter module, Wasm runtime, and Mandelbrot demo file for
this tutorial can be found in `libc-interp/`.

## JitBuilder Overview

JitBuilder provides a simplified interface to the compiler technology that allows you to
integrate a compiler into your runtime more quickly.  It abstracts away many of the low-level
compilation details that you would normally need to consider when building an Eclipse OMR
based compiler behind a much simpler API.

JitBuilder packages the guts of the compiler into a static library that you can then link
against in your language runtime using APIs, which are described in just a few header files.

There are a few structures that are particularly important when using JitBuilder:

`TR::MethodBuilder` is an object that represents the ABI (parameters, return value) and
the operations that should be performed when a compiled method is called. To compile your
own methods, you subclass `OMR::MethodBuilder` and implement its constructor to specify
the ABI and then specify the operations the compiled method should perform by implementing
a `MethodBuilder` member function called `buildIL()`.  After passing a `TR::MethodBuilder`
object to `compileMethodBuilder()`, you get back a pointer to a function that you can call
by casting that pointer to a C function prototype. Each `TR::MethodBuilder` has a hash map
of names (C strings) representing the method’s local variables, and you can define them all
at once or make them up as you go.

`TR::TypeDictionary` is an object that you can use to describe the shape of structures, unions,
and other types so that JitBuilder knows how to access their fields and values.

`TR::IlBuilder` is an object that represents the operations to perform when a particular
control flow path is reached. The method entry point is one particular control flow path,
and so a `TR::MethodBuilder` object is also a `TR::IlBuilder` object. But you can create
arbitrary `TR::IlBuilder` objects and connect them together (even nest them) in arbitrary
ways using an ever-growing list of services like `IfThen` or `ForLoopUp` or `WhileDoLoop`.
Each service you call on a `TR::IlBuilder` appends operations to the control flow path and
so describes the intended order of operations to the compiler. There is no restriction on
how you add operations to different `TR::IlBuilder` objects. You can create all the
`TR::IlBuilder` objects up front and then add operations to them individually, or you can
start from the `TR::MethodBuilder` object (function entry) and create `TR::IlBuilder`
objects as you need them to represent the different paths of execution as you require them.

`TR::BytecodeBuilder` is a special kind of IlBuilder that’s designed to simplify writing
JIT compilers for bytecode based languages. You allocate a `TR::BytecodeBuilder` object for
each bytecode in a method and translate the operations needed for that bytecode using that
object. You have to specify every control flow edge (even fall through edges) between bytecode
builders. `TR::BytecodeBuilder` also taps into a handy and mostly automatic built-in
worklist algorithm provided by `TR::MethodBuilder` that can traverse all flow edges in a
control flow graph, visiting each bytecode at most once. Any bytecodes in the method that
aren’t reachable (many parsers generate unreachable code to simplify the process of initial
code generation from a parse tree) will not be visited.

`TR::IlValue` is an object that represents the values that are created and consumed by
expressions. If you load a local variable by name, the value that’s loaded is a
`TR::IlValue`. If you create a constant integer, that’s a `TR::IlValue`. If you pass both
of those `TR::IlValue`s into the Add operation, you’ll get back another `TR::IlValue` that
represents their sum. There is an ever growing list of operations you can apply to create
arbitrarily complex `TR::IlValue` expressions.

These key structures can be found in `third_party/omr/compiler/ilgen` and can be extended
by your language runtime.

## Further resources

After the tutorial, you may wish to review the following resources for a deeper understanding
of the wasmjt-omr and JitBuilder content.

* VIDEO (30 minutes) [Performant WebAssembly outside the browser using Eclipse OMR JitBuilder](https://www.youtube.com/watch?v=wWhGlNVg2O8)

* [JitBuilder Library and Eclipse OMR: Just-in-time compilers made easy](https://developer.ibm.com/code/2016/07/19/jitbuilder-library-and-eclipse-omr-just-in-time-compilers-made-easy/)

* [Build more complicated methods using the JitBuilder library](https://developer.ibm.com/code/2017/03/08/build-more-complicated-methods-using-the-jitbuilder-library/)

* [Building the JitBuilder library from source code](https://developer.ibm.com/code/2017/04/14/building-jitbuilder-library-source-code/)

* * *

## Part 1: Dispatching the JIT

### Exercise 1: Calling the OMR Compiler from JitBuilder

#### Background

The main task involved in creating a JIT compiler using JitBuilder is to
translate the VM/interpreter's representation of functions into the OMR
compiler's Intermediate Language (known as TR IL), a process known as IL generation. In
JitBuilder, IL generation is implemented by subclassing `OMR::MethodBuilder` ([`third_party/omr/compiler/ilgen/OMRMethodBuilder.hpp`](https://github.com/eclipse/omr/blob/3123e6d913318d74e0f0ecacd9de96533874e1fc/compiler/ilgen/OMRMethodBuilder.hpp#L47)) and
overriding the required functions. To actually compile a function/method,
`compileMethodBuilder()` is called with an instance of the `OMR::MethodBuilder`
subclass, an instance of a `TypeDictionary` (an object used to describe
non-primitive types to JitBuilder), and a pointer to a variable in which the
entry point to a function will be stored, if compilation is successful.
`compileMethodBuilder()` returns 0 if compilation succeeds and some non-0 value
otherwise.

In wasmjit-omr, the `OMR::MethodBuilder` subclass is `wabt::jit::FunctionBuilder` ([`src/jit/function-builder.h`](https://github.com/wasmjit-omr/wasmjit-omr/blob/2f7c7ba59fa36f7b5beed916d9b0e444c9dc2da8/src/jit/function-builder.h#L32)).
A subclass of `TypeDictionary` is also implemented as `wabt::jit::TypeDictionary` ([`src/jit/type-dictionary.h`](https://github.com/wasmjit-omr/wasmjit-omr/blob/2f7c7ba59fa36f7b5beed916d9b0e444c9dc2da8/src/jit/type-dictionary.h#L25)).
In later sections, you will complete parts of
`FunctionBuilder` to practice generating IL using JitBuilder.

#### Your Task

In [`src/jit/wabtjit.cc`](https://github.com/wasmjit-omr/wasmjit-omr/blob/42b8ae72308581eaff882626496ec1cf8dadff8f/src/jit/wabtjit.cc#L26), complete the implementation
of `wabt::jit::compile(interp::Thread* thread, interp::DefinedFunc* fn)`.

```c++
JITedFunction compile(interp::Thread* thread, interp::DefinedFunc* fn) {
  TypeDictionary types;
  FunctionBuilder builder(thread, fn, &types);

  // YOUR CODE HERE

  return nullptr;
}
```

Given an interpreter `thread` and a defined function object `fn`, `compile()` should
(unconditionally) invoke the JIT compiler and return the entry point to the
generated body. If JIT compilation fails, `nullptr` should be returned instead.

#### What To Do

Complete `compile()` by:

- defining a variable of type `uint8_t *` to store the entry point to the JIT
compiled body
- calling `compileMethodBuilder()` with `types`, `builder`, and the address
of the variable used to store the entry point as arguments
- returning the entry point cast to `JITedFunction` if `compileMethodBuilder()`
returns `0`, `nullptr` (or `NULL`) otherwise

#### Solution

The solution to this exercise is available in this document [here](#solution-exercise-1-calling-the-omr-compiler-from-jitbuilder)
or from the GitHub repository [here](https://github.com/wasmjit-omr/wasmjit-omr/blob/2f7c7ba59fa36f7b5beed916d9b0e444c9dc2da8/src/jit/wabtjit.cc#L26).

### Exercise 2: Deciding When To Compile

#### Background

A key aspect of maximizing performance using a JIT compiler is to control when
the JIT compiler is invoked. Compiling too often can mean a VM spends more time
compiling than executing application code.

A simple way to limit what/when functions get compiled is to only compile
functions if they have been called a fixed number of times--a so-called "counted
compilation".

Another thing that VMs must handle is the case when JIT compilation fails. The
JIT may fail to compile a function if, for example, the function uses a
language feature not supported by the compiler. In such cases the function
must always be interpreted.

`wabt::interp::JitMeta` ([`src/interp.h`](https://github.com/wasmjit-omr/wasmjit-omr/blob/2f7c7ba59fa36f7b5beed916d9b0e444c9dc2da8/src/interp.h#L476))
is a WABT struct used to keep track of information about WebAssembly
functions that is useful for controlling JIT compilation:

```c++
struct JitMeta {
  DefinedFunc* wasm_fn;            // pointer to the description object for the Wasm function
  uint32_t num_calls = 0;          // the number of times the function has been called

  bool tried_jit = false;          // whether we've already tried JITing this function
  JITedFunction jit_fn = nullptr;  // pointer to the entry point of the compiled body

  JitMeta(DefinedFunc* wasm_fn) : wasm_fn(wasm_fn) {}
};
```

#### Your Task

In [`src/interp.cc`](https://github.com/wasmjit-omr/wasmjit-omr/blob/42b8ae72308581eaff882626496ec1cf8dadff8f/src/interp.cc#L1189), complete the implementation
of `Environment::TryJit()`.

```c++
bool Environment::TryJit(Thread* t, IstreamOffset offset, Environment::JITedFunction* fn) {
  if (!enable_jit) {
    *fn = nullptr;
    return false;
  }

  auto meta_it = jit_meta_.find(offset);

  if (meta_it != jit_meta_.end()) {
    JitMeta* meta = &meta_it->second;

    // YOUR CODE HERE

    *fn = meta->jit_fn;
    return trap_on_failed_comp || *fn;
  } else {
    *fn = nullptr;
    return trap_on_failed_comp;
  }
}
```

Given an interpreter thread `t` ([`src/interp.h`](https://github.com/wasmjit-omr/wasmjit-omr/blob/42b8ae72308581eaff882626496ec1cf8dadff8f/src/interp.h#L502)), an instruction offset `offset`, and a
pointer to a pointer to a function `fn` (used as an in-out parameter) ([`src/jit/wabjit.h`](https://github.com/wasmjit-omr/wasmjit-omr/blob/42b8ae72308581eaff882626496ec1cf8dadff8f/src/jit/wabtjit.h#L26)),
`TryJit()` should try to JIT compile the function at `offset` if:

1. the JIT compiler is enabled
2. there is an entry for it in the meta-data table
3. there has never been a previous attempt to compile the function (preventing
recompilation of functions for which a previous compilation failed)
4. the function has been called at least the number of times set in `jit_threshold`
(a member of the `Environment` class)

If all conditions for compilation are met, the JIT compiler is invoked by
calling `jit::compile()` and the `tried_jit` field of the function's metadata should be
set to `true`.

If compilation succeeds, the `jit_fn` field of the function's metadata and the
variable pointed to by the `fn` parameter are set to the entry point returned by
`jit::compile()`.

#### What To Do

- check if `meta->tried_jit` is false, indicating that we have never tried to
compile this function before
- increment `meta->num_calls` (the number of times this function has been called)
- check if `meta->num_calls` is greater than `jit_threshold`
- if it is
    - call `jit::compile(t, meta->wasm_fn)`
    - assign the returned value to `meta->jit_fn`
    - set `meta->tried_jit` to true
- if not
    - set the variable pointed to be `fn` to `nullptr`
    - return false to indicate compilation failure

#### Solution

The solution to this exercise is available in this document [here](#solution-exercise-2-deciding-when-to-compile)
or from the GitHub repository [here](https://github.com/wasmjit-omr/wasmjit-omr/blob/2f7c7ba59fa36f7b5beed916d9b0e444c9dc2da8/src/interp.cc#L1189).

### Exercise 3: Where To Compile

#### Background

The last piece of the puzzle is to actually call the JIT compiler (`TryJit()`)
and to call the JIT compiled function. In many VMs, it makes sense to simply do
these calls where the interpreter handles calls. When a program is about to
call a function, the interpreter will invoke the JIT and, if compilation
succeeds, call the entry point returned by the JIT.

To correctly manage calls, the VM must keep track of the call stack at all times.
For convenience, the `PushCall(const uint8_t* pc)` and `PopCall()` function are
provided. `PushCall()` takes as an argument the `pc` of the call instruction and
returns a status to indicate possible trap conditions due to a call stack
overflow. `PopCall()` simply resets the interpreter's pc and takes no arguments
and produces no return value.

When the interpreter calls a JIT compiled function body, it is important to call
both of the above functions before and after the JIT compiled function call.
More precisely, the sequence of events must be:

- push a new stack by calling `PushCall(pc)`, checking for a trap condition in the
  returned value (`CHECK_TRAP()` macro can be used for this purpose)
- call the JIT compiled function
- check that the JIT compiled function returns `interp::Result::Ok`
- pop the stack frame by calling `PopCall()`

#### Your Task

In [`src/interp.cc`](https://github.com/wasmjit-omr/wasmjit-omr/blob/42b8ae72308581eaff882626496ec1cf8dadff8f/src/interp.cc#L1371),
complete the WABT interpreter's handling of the
`Call` opcode to call `TryJit()` and to call the entry point to the compiled
body when successful.

```c++
      case Opcode::Call: {
        IstreamOffset offset = ReadU32(&pc);

        // YOUR CODE HERE

        // REPLACE `false` WITH CHECK OF `TryJit()` RETURN VALUE
        if (false) {

          // REPLACE `true WITH CHECK THAT VALUE RETURNED BY COMPILED BODY IS NOT `Result::Ok`
          if (true) {
            // **DO NOT CHANGE ANYTHING HERE**

            // We don't want to overwrite the pc of the JITted function if it traps
            tpc.Reload();

            return result;
          }

          // POP CALL STACK

        } else {
          CHECK_TRAP(PushCall(pc));
          GOTO(offset);
        }
        break;
      }
```

When the interpreter encounters a `Call` instruction, it proceeds as follows:

- get the offset of the function's first instruction
- try to JIT compile the function by calling `TryJit()`
- if compilation succeeds
    - emit `TrapFailedJITCompilation` if the entry point is null
    - push the current `pc` on the call stack, check for any trap conditions
    - call the JIT compiled body
    - if the compiled function *does not* return `interp::Result::Ok`
        - call `tpc.Reload()`
        - return the returned value
    - pop the pc off the call stack
- otherwise
    - push the current `pc` on the call stack, check for any trap conditions
    - jump to the offset
- break out of the case

#### What To Do

- Add a local variable of type `Environment::JITedFunction` to store the entry
point to the JIT compiled function body
- call `TryJit()` on the current environment (stored in the `env_` member
variable) passing `this` as the thread argument, the offset of the function, and
the address of the variable to store the entry point to the JIT compiled body
- replace the `if (false)` with a check that `TryJit()` returned `true`
- in the `if` body
    - use `TRAP_IF(<check entry point is null>, FailedJITCompilation);` to emit a
    trap if the entry point is null
    - push the current pc onto the call stack with `CHECK_TRAP(PushCall(pc));`
    - call the entry point (a JIT compiled function body takes no arguments)
    - replace the `if (true)` to check that JIT compiled function *did not*
    return `interp::Result::Ok` (`interp::` is optional)
        - *(no modifications to the body of the `if` needed)*
    - after the `if`, call `PopCall()` to pop the pc from the call stack

#### Solution

The solution to this exercise is available in this document [here](#solution-exercise-3-where-to-compile)
or from the GitHub repository [here](https://github.com/wasmjit-omr/wasmjit-omr/blob/2f7c7ba59fa36f7b5beed916d9b0e444c9dc2da8/src/interp.cc#L1380).

* * *

## Part 2: Generate TR IL

### Exercise 4: Implement `buildIL()`

#### Background

`buildIL()` is the function called by the OMR compiler to generate IL. It should
return `true` when IL generation succeeds, `false` otherwise. A JitBuilder JIT
overrides this method and implements their own IL generation.

`VirtualMachineState` is used to simulates state changes that would happen if
the function being compiled were to run int the VM.

`BytecodeBuilder`s are used to generate the IL corresponding to a specific
instruction/bytecode in a function. The call to `workItems_.emplace_back()`
inserts an instance corresponding to the first bytecode in the function being
compiled.

The work list keeps track of which opcodes from the function still need to be
handled. Internally, the JIT will call `AppendBuilder()` every time a new
instruction is encountered that requires IL generation.

#### Your Task

In [`src/jit/function-builder.cc`](https://github.com/wasmjit-omr/wasmjit-omr/blob/42b8ae72308581eaff882626496ec1cf8dadff8f/src/jit/function-builder.cc#L245),
complete the implementation of `FunctionBuilder::buildIL()`.

```c++
bool FunctionBuilder::buildIL() {
  setVMState(new TR::VirtualMachineState());

  const uint8_t* istream = thread_->GetIstream();

  workItems_.emplace_back(OrphanBytecodeBuilder(0, const_cast<char*>(ReadOpcodeAt(&istream[fn_->offset]).GetName())),
                          &istream[fn_->offset]);
  AppendBuilder(workItems_[0].builder);

  // YOUR CODE HERE

  return false;
}
```

When invoked, `buildIL()` will generate IL for the WebAssembly function
corresponding to the current `FunctionBuilder` object.

It first sets an instance of `VirtualMachineState` on the current object. Since
the current JIT implementation does not use this particular JitBuilder feature,
using an instance of the base class is sufficient.

It then inserts a single element into the `workItems_` list. Each element of the
list contains an instance of `BytecodeBuilder` and the address pointing to a
corresponding instruction in the function. The first element inserted in the
list corresponds to the first instruction of the function and is therefore given
the index 0.

Next, the newly created `BytecodeBuilder` instance is added to the internal
JitBuilder.

In a loop, `GetNextBytecodeFromWorklist()` is called and the index is verified
to be different than -1 (which indicates the end of the work list). The work
item at the given index in `workItems_` is retrieved and IL is generated for
the corresponding instruction, using the contained `builder` object. If IL
generation for an instruction fails, the function returns false.

Once IL is successfully generated for all instructions, `buildIL()` returns
true.

#### What To Do

- in a loop, call `GetNextBytecodeFromWorklist()`
- break out of the loop if the returned index is -1
- get the work item at the returned index in `workItems_`
- call `Emit()`, passing as arguments:
    - the `BytecodeBuilder` instance for the instruction (`.builder` member of
    the work item object)
    - the instruction stream `istream`
    - a pointer to the instruction (`.pc` member of the the work item object)
- if `Emit()` returns false, `buildIL()` should also return false
- change the `return false` at the end of `buildIL()` to `return true`

#### Solution

The solution to this exercise is available in this document [here](#solution-exercise-4-implement-buildil)
or from the GitHub repository [here](https://github.com/wasmjit-omr/wasmjit-omr/blob/2f7c7ba59fa36f7b5beed916d9b0e444c9dc2da8/src/jit/function-builder.cc#L245).

* * *

## Part 3: Implement a few Wasm opcodes

### Exercise 5: Implement `Return`

#### Background

To generate IL, JitBuilder provides "services" that must be called on an `OMR::IlBuilder`
instance. Sub-classes of `OMR::IlBuilder` include `OMR::BytecodeBuilder`, `OMR::MethodBuilder`,
and sub-classes of these that are implemented by JitBuilder users (e.g.
`wabt::jit::FunctionBuilder`).

Each JitBuilder service generates IL that represents a particular action. For
example, the `Return()` service generates IL representing "returning from a
function". When no arguments are passed, the generated IL represents a simple
return, without a return value. An argument, of type `TR::IlValue`, can be passed to
represent a returned value.

`TR::IlValue` ([`third_party/omr/compiler/ilgen/OMRIlValue.hpp`](https://github.com/eclipse/omr/blob/a3d48e4713fa0078d0f34a5e901b9c2b84ad3c6d/compiler/ilgen/OMRIlValue.hpp#L41))
is a class used by JitBuilder to represent values that are "computed" by
the generated IL or, more precisely, values computed by the code *generated*
from the IL by the compiler. JitBuilder provides various services for generating
`TR::IlValue` instances. For instance, `ConstInt32(int32_t)` generates the
representation of a constant, 32-bit integer value. All the arithmetic
operation services take as arguments and return instances of `TR::IlValue`. Memory
load operations produce `TR::IlValue` instances while memory store operations take
an instance as argument.

#### Your Task

In [`src/jit/function-builder.cc`](https://github.com/wasmjit-omr/wasmjit-omr/blob/42b8ae72308581eaff882626496ec1cf8dadff8f/src/jit/function-builder.cc#L678),
implement IL generation for the `Return` opcode.

```c++
case Opcode::Return:
  return false;
```

The generated IL must represent "return the value `interp::Result::Ok`".

Because no other instructions from the function are expected to be executed,
`Emit()` can simply return true after successfully generating IL for `Return`.

#### What To Do

- static cast the enum value `interp::Result::Ok` to the underlying integer type
`Result_t`
- generate a `TR::IlValue` for the constant by calling `Const()` on the current
`BytecodeBuilder` instance `b`, passing the constant itself as an argument
- generate the IL for the return by calling `Return()` on the builder `b`,
passing the `TR::IlValue` instance as an argument
- change `return false` to `return true` :)

#### Solution

The solution to this exercise is available in this document [here](#solution-exercise-5-implement-return)
or from the GitHub repository [here](https://github.com/wasmjit-omr/wasmjit-omr/blob/2f7c7ba59fa36f7b5beed916d9b0e444c9dc2da8/src/jit/function-builder.cc#L683).

### Exercise 6: Implement `i32.sub` and `i32.mul`

#### Background

Being a stack-based VM, the WABT interpreter performs pushes and pops to the
operand stack every time an operation is performed. For convenience,
`FunctionBuilder` provides some helpers to generate IL for operand stack pushes
and pops.

`PopI32(TR::IlBuilder*)` generates IL corresponding to a stack pop. The first
argument is a pointer to the `TR::IlBuilder` object that should be used to
generate the IL. It returns a `TR::IlValue` instance representing the value
popped from the stack.

`PushI32(TR::IlBuilder*, TR::IlValue*, const uint8_t*)` generates IL representing
a stack push. The first argument is the `TR::IlBuilder` object to be used to
generate IL. The second argument is the `TR::IlValue` instance representing the
value being pushed. The final argument is the `pc` pointing to the instruction
performing the push. It is used to generate a trap if the stack overflows
because of the push.

#### Your Task

In [`src/jit/function-builder.cc`](https://github.com/wasmjit-omr/wasmjit-omr/blob/42b8ae72308581eaff882626496ec1cf8dadff8f/src/jit/function-builder.cc#L993),
implement the 32-bit integer `Sub` and `Mul` opcodes.

```c++
case Opcode::I32Sub:
  return false;
```

The IL generated for every binary arithmetic operation (`Add`, `Sub`, `Mul`, etc.)
must:

- pop the RHS from the operand stack
- pop the LHS from the operand stack
- compute the operation
- push the result onto the operand stack

Because the instructions that follow the arithmetic operations must also be
processed, once IL is successfully generated, each `case` must `break`
out of the `switch` statement instead of `return`.

Use the implementation of `i32.add` as an example to guide you:

```c++
case Opcode::I32Add: {
  auto rhs = PopI32(b);
  auto lhs = PopI32(b);
  PushI32(b, b->Add(lhs, rhs), pc);
  break;
}
```

#### What To Do

- pop the RHS by calling `PopI32()` with `b` as argument
- pop the LHS following the style used to pop the RHS
- generate the computation by calling the appropriate service
  (`b->Sub()` or `b->Mul()`) with the LHS and RHS as arguments
- push the resulting `TR::IlValue` using `PushI32()` with `b`, the `TR::IlValue`
  instance, and `pc` as arguments
- `break` out of the `switch`

#### Solution

The solution to this exercise is available in this document [here](#solution-exercise-6-implement-i32-sub-and-i32-mul)
or from the GitHub repository [here](https://github.com/wasmjit-omr/wasmjit-omr/blob/2f7c7ba59fa36f7b5beed916d9b0e444c9dc2da8/src/jit/function-builder.cc#L1007).

### Exercise 7: Implement `f32.sub` and `f32.mul`

Because most arithmetic opcode implementations follow a similar pattern, it is
useful to abstract the commonalities into a helper function. In wasmjit-omr,
the `EmitBinaryOp` templated function can be used for this purpose:

```c++
/**
 * \tparam    `T` : the type of operation being emitted (e.g., `int32_t` for 32-bit
 *           integer binary operations)
 * \tparam    `TResult` : the type of the result of the operation (same as `T` by
 *           default)
 * \tparam    `TOpHandler` : the type of the callable object that generates IL for
 *           the operation
 * \param[in] `builder` : a pointer to the builder object on which pushes and pops
 *           should be generated
 * \param[in] `pc` : the current ("virtual") pc pointing to the instruction for which
 *           IL is being generated
 * \param[in] `operation` : a lambda (or other callable object) that generates only
 *           the IL for the operation.  The lambda's arguments are the TR::IlValues
 *           corresponding to the operation operands and is expected to return the
 *           IlValue corresponding to the result:
 *              `TR::IlValue * lambda(TR::IlValue* lhs, TR::IlValue* rhs)`
 */
template <typename T, typename TResult = T, typename TOpHandler>
void EmitBinaryOp(TR::IlBuilder* builder, const uint8_t* pc, TOpHandler operation);
```

#### Your Task

In [`src/jit/function-builder.cc`](https://github.com/wasmjit-omr/wasmjit-omr/blob/42b8ae72308581eaff882626496ec1cf8dadff8f/src/jit/function-builder.cc#L1328),
implement the 32-bit floating-point `Sub` and `Mul` opcodes using the
`EmitBinaryOp` template function.

```c++
case Opcode::F32Sub:
  return false;

case Opcode::F32Mul:
  return false;
```

Use the implementation of `f32.add` as an example to guide you:

```c++
case Opcode::F32Add:
  EmitBinaryOp<float>(b, pc, [&](TR::IlValue* lhs, TR::IlValue* rhs) {
    return b->Add(lhs, rhs);
  });
  break;
```

#### What To Do

- call `EmitBinaryOp<>()` with `float` as the template argument, `b` and `pc`
as second and third arguments, respectively, and a lambda function that:
    - captures by reference
    - calls `Sub()` or `Mul()` on `b`, forwarding the lambda's arguments
    - returns the resulting `TR::IlValue` instance
- `break` out of the `switch`

#### Solution

The solution to this exercise is available in this document [here](#solution-exercise-7-implement-f32-sub-and-f32-mul)
or from the GitHub repository [here](https://github.com/wasmjit-omr/wasmjit-omr/blob/2f7c7ba59fa36f7b5beed916d9b0e444c9dc2da8/src/jit/function-builder.cc#L1330).

### Exercise 8: Implement `Call`

#### Background

Both the interpreter and JIT compiled code must be capable of calling the
following entities:

- interpreted (not yet JIT compiled) functions
- JIT compiled function
- the JIT compiler itself

To call an interpreted function from JIT compiled code, a common strategy is
for the JIT compiled code to simply call the interpreter, pointing its pc to the
first instruction in the function.

To avoid having to generate IL that represents all this logic, we can instead
generate a call to a so-called *runtime helper* that will take care of all the
complexity.

In WABT, the following is already implemented for you [`src/jit/function-builder.cc`](https://github.com/wasmjit-omr/wasmjit-omr/blob/42b8ae72308581eaff882626496ec1cf8dadff8f/src/jit/function-builder.cc#L95):

```c++
Result_t CallHelper(interp::Thread* th, interp::IstreamOffset offset, uint8_t* current_pc)
```

Runtime helpers are "registered" with JitBuilder by calling the `DefineFunction`
service, normally in the constructor of the `MethodBuilder` subclass.

```c++
DefineFunction("CallHelper", __FILE__, "0",          // the name of the helper (with dummy file and line number)
               reinterpret_cast<void*>(CallHelper),  // pointer to the function implementation
               types->toIlType<Result_t>(),          // JitBuilder representation of the returnt type
               3,                                    // number of parameters
               types->toIlType<void*>(),             // JitBuilder representation of the parameter types
               types->toIlType<wabt::interp::IstreamOffset>(),
               types->PointerTo(Int8));
```

Registered functions can then be called using the `Call()` services.

#### Your Task

In [`src/jit/function-builder.cc`](https://github.com/wasmjit-omr/wasmjit-omr/blob/42b8ae72308581eaff882626496ec1cf8dadff8f/src/jit/function-builder.cc#L764),
complete IL generation for the `Call` opcode.

```c++
case Opcode::Call: {
  auto th_addr = b->ConstAddress(thread_);
  auto offset = b->ConstInt32(ReadU32(&pc));
  auto current_pc = b->Const(pc);

  // YOUR CODE HERE

  return false;
}
```

The IL generated for `Call` must generate IL for a `Call` to the `CallHelper`
runtime helper. The arguments to the call must be:

- a pointer to the current thread instance (`thread_` member of `FunctionBuilder`)
- the offset of the function to be called (offset of the function's first instruction
- the pc of the of the current `Call` instruction

The `Call()` service returns a `TR::IlValue` instance representing the value
returned by the function call. It takes as arguments:

- the name of the function to be called (must match the name used in `DefineFunction`)
- the number of arguments to be passed
- `TR::IlValue` instances representing the values of the arguments (as a vararg)

The value returned by the function must then be checked for a trap value. If
it is a trap, then the value must be propagated back by returning from the
Wasm function that IL is being generated for. The `EmitCheckTrap()` can be used
to generate IL representing the required trap handling. As arguments, it takes
a builder object, `TR::IlValue` representing the value to be checked (return value of
`CallHelper` in this case), and a pointer to the pc that must be updated if a
trap condition is detected.

Note: in the generated code, because the called function has *already returned*
by this point, there is no need to update the pc, which can be indicated to
`EmitCheckTrap()` by passing a null pointer in place of a pointer to the pc.

Finally, the code must break out of the `switch` statement to handle execution
of the next instruction.

#### What To Do

- call the `Call()` JitBuilder service, saving the returned value, and passing
as arguments:
    - `"CallHelper"` to call the runtime helper that has been registered for you
    - `3` for the 3 arguments that must be passed
    - `th_addr`, which represents a pointer to the current `interp::Thread`
    instance
    - `offset`, which represents the instruction offset of the function to be
    called
    - `current_pc`, which represents an instruction pointer to the current
    `Call` instruction
- call `EmitCheckTrap`, passing as arguments:
    - the current builder object `b`
    - an IlValue representing the value returned by the call to `CallHelper`
    - `nullptr` since the pc does not need to be updated
- change the `return false` to `break`

#### Solution

The solution to this exercise is available in this document [here](#solution-exercise-8-implement-call)
or from the GitHub repository [here](https://github.com/wasmjit-omr/wasmjit-omr/blob/2f7c7ba59fa36f7b5beed916d9b0e444c9dc2da8/src/jit/function-builder.cc#L770).

* * *

## BONUS Part 4: Set a different JIT compilation threshold

As was previously discussed, the JIT compiler will only compile a function after
it has been called a fixed number of times. By default, the current
implementation uses a threshold of 1, so functions will only be compiled after
being called once. In many applications, it's useful to be able to configure
the threshold. For example, the call-count threshold for JIT compilation can
be configured for `libc-interp` using the `--jit-threshold` option. For example:

```sh
env time ./libc-interp mandelbrot.wasm --jit-threshold 2
```

will compile functions after they have been called twice.

Try varying the threshold and see if you can find the "optimal" value for the
Mandelbrot example we've been using.

You will find that the "optimal" value for one application may not be the "optimal"
value for another application.  Choosing a general compilation threshold that behaves
reasonably well for different applications is a challenge for JIT compilers.  More
complex heuristics (such as method sampling) may be needed.

## BONUS Part 5: Generate a verbose log

A useful feature of the OMR compiler is its ability to log what it's compiling.
The functionality can be triggered in JitBuilder-based compilers by setting
(or exporting) the `TR_Options` environment variable to `verbose`. For example:

```sh
TR_Options=verbose env time ./libc-interp mandelbrot.wasm
```

Try experimenting with different JIT thresholds and see how the output changes.
What different functions get compiled?

Note: `TR_Options` is a general way of passing options to the OMR compiler.

## BONUS Part 6: Generate a trace log

In addition to tracking what is being compiled, the OMR compiler also logs what
the compiler is doing *while compiling* each function. As an example, run
the Mandelbrot example with `TR_Options` set as follows:

```sh
TR_Options=traceIlGen,traceFull,log=trtrace.log ./libc-interp mandelbrot.wasm
```

The `traceIlGen` option will cause the OMR compiler (and JitBuilder) to trace
IL generation. `traceFull` (which, un-intuitively, doesn't actually trace
everything) causes a few important steps of compilation to be traced. Among
other things, it provides information such as what optimizations are executed
and how the IL is transformed by each pass. Finally, `log=trtrace.log` will
cause the OMR compiler to dump the trace log to a file called `trtrace.log`.
Try opening the file in a text editor. **(Warning: it is possible for this file
to contain a few *million* lines!)**

For information about what the content of the log file means, take a look at the
[`Compilation Log`](https://github.com/eclipse/omr/blob/master/doc/compiler/ProblemDetermination.md#compilation-log)
section of `third_party/omr/doc/compiler/ProblemDetermination.md`.

* * *

## Exercise Solutions

### Solution Exercise 1: Calling the OMR Compiler from JitBuilder

```c++
JITedFunction compile(interp::Thread* thread, interp::DefinedFunc* fn) {
  TypeDictionary types;
  FunctionBuilder builder(thread, fn, &types);
  uint8_t* function = nullptr;

  if (compileMethodBuilder(&builder, &function) == 0) {
    return reinterpret_cast<JITedFunction>(function);
  } else {
    return nullptr;
  }
}
```

### Solution Exercise 2: Deciding When To Compile

```c++
bool Environment::TryJit(Thread* t, IstreamOffset offset, Environment::JITedFunction* fn) {
  if (!enable_jit) {
    *fn = nullptr;
    return false;
  }

  auto meta_it = jit_meta_.find(offset);

  if (meta_it != jit_meta_.end()) {
    JitMeta* meta = &meta_it->second;
    if (!meta->tried_jit) {
      meta->num_calls++;

      if (meta->num_calls > jit_threshold) {
        meta->jit_fn = jit::compile(t, meta->wasm_fn);
        meta->tried_jit = true;
      } else {
        *fn = nullptr;
        return false;
      }
    }

    *fn = meta->jit_fn;
    return trap_on_failed_comp || *fn;
  } else {
    *fn = nullptr;
    return trap_on_failed_comp;
  }
}
```

### Solution Exercise 3: Where To Compile

```c++
       case Opcode::Call: {
         IstreamOffset offset = ReadU32(&pc);
         Environment::JITedFunction jit_fn;

         if (env_->TryJit(this, offset, &jit_fn)) {
           TRAP_IF(!jit_fn, FailedJITCompilation);
           CHECK_TRAP(PushCall(pc));

           auto result = jit_fn();
           if (result != Result::Ok) {
             // We don't want to overwrite the pc of the JITted function if it traps
             tpc.Reload();

             return result;
           }

           PopCall();
         } else {
           CHECK_TRAP(PushCall(pc));
           GOTO(offset);
         }
         break;
       }

```

### Solution Exercise 4: Implement `buildIL`

```c++
bool FunctionBuilder::buildIL() {
  setVMState(new TR::VirtualMachineState());

  const uint8_t* istream = thread_->GetIstream();

  workItems_.emplace_back(OrphanBytecodeBuilder(0, const_cast<char*>(ReadOpcodeAt(&istream[fn_->offset]).GetName())),
                          &istream[fn_->offset]);
  AppendBuilder(workItems_[0].builder);

  int32_t next_index;

  while ((next_index = GetNextBytecodeFromWorklist()) != -1) {
    auto& work_item = workItems_[next_index];

    if (!Emit(work_item.builder, istream, work_item.pc))
      return false;
  }

  return true;
}
```

### Solution Exercise 5: Implement `Return`

```c++
case Opcode::Return:
  b->Return(b->Const(static_cast<Result_t>(interp::Result::Ok)));
  return true;
```

### Solution Exercise 6: Implement `i32 sub` and `i32 mul`

```c++
case Opcode::I32Sub: {
  auto rhs = PopI32(b);
  auto lhs = PopI32(b);
  PushI32(b, b->Sub(lhs, rhs), pc);
  break;
}

case Opcode::I32Mul: {
  auto rhs = PopI32(b);
  auto lhs = PopI32(b);
  PushI32(b, b->Mul(lhs, rhs), pc);
  break;
}
```

### Solution Exercise 7: Implement `f32 sub` and `f32 mul`

```c++

case Opcode::F32Sub:
  EmitBinaryOp<float>(b, pc, [&](TR::IlValue* lhs, TR::IlValue* rhs) {
    return b->Sub(lhs, rhs);
  });
  break;

case Opcode::F32Mul:
  EmitBinaryOp<float>(b, pc, [&](TR::IlValue* lhs, TR::IlValue* rhs) {
    return b->Mul(lhs, rhs);
  });
  break;
```

### Solution Exercise 8: Implement `Call`

```c++
case Opcode::Call: {
  auto th_addr = b->ConstAddress(thread_);
  auto offset = b->ConstInt32(ReadU32(&pc));
  auto current_pc = b->Const(pc);

  b->Store("result",
  b->      Call("CallHelper", 3, th_addr, offset, current_pc));

  // Don't pass the pc since a trap in a called function should not update the thread's pc
  EmitCheckTrap(b, b->Load("result"), nullptr);

  break;
}
```

repository: omr-turbo/wasmjit-omr

branch_name: refs/heads/turbo

repo_name: omr-turbo/wasmjit-omr


text:
# WSO2 Enterprise Integrator
 For Integration, Broker and Business Process (BPS)
 Use-cases With Analytics Support

![alt tag](deployment-diagram.png)

## Prerequisites

  * [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git), [Docker](https://www.docker.com/get-docker) and [Docker Compose](https://docs.docker.com/compose/install/#install-compose) are required for running this Docker Compose template.
  * In order to run the Docker Compose setup, you will need an active subscription from WSO2 since the
    Docker images hosted at docker.wso2.com contains the latest updates and fixes to WSO2 Enterprise Integrator. You
    can sign up for a Free Trial Subscription [here](https://wso2.com/free-trial-subscription).
  * If you wish to run the Docker Compose setup using Docker images built locally, build Integrator, Broker, Business Process,
   Analytics Images using [Dockerfiles](../../dockerfiles/README.md) and remove the `docker.wso2.com/` prefix from the `image` name In the `docker-compose.yml`

## How to Run

  1. Clone WSO2 Enterprise Integrator Docker git repository.
     ```
     git clone https://github.com/wso2/docker-ei
     ```
     > Note that the local copy of `docker-ei` repository will be referred to as `[docker-ei]` from this point onwards.

  2. Switch to docker-compose/integrator-broker-bps-analytics folder :
     ```
     cd [docker-ei]/docker-compose/integrator-broker-bps-analytics
     ```

  3. Before to start deployment process, add a host entry pointing to the Docker host machine IP address.

     For an example if the Docker host is accessible via <IP_ADDRESS> on a Linux or Mac machine, add

     following entry to /etc/hosts file :
     ```
     <IP_ADDRESS> wso2ei-integrator wso2ei-analytics wso2ei-broker wso2ei-business-process
     ```

  4. Execute following Docker Compose command to start the deployment :
     ```
     docker-compose up
     ```

  5. Access management console via a web browser :

     ```
     For Integrator - https://wso2ei-integrator:9443/carbon
     For Analytics - https://wso2ei-analytics:9444/carbon
     For Business Process - https://wso2ei-business-process:9445/carbon
     For Broker - https://wso2ei-broker:9446/carbon
     ```

repository: ThilinaManamgoda/docker-ei

branch_name: refs/heads/master

extension: md

license_type: permissive

repo_name: ThilinaManamgoda/docker-ei


text:
# TRIANGLE TRACKER
#### The application is for testing which type of triangle can be formed by the inputs entered, {29/3/2019 version 1}
#### By **Authors: Elvis Makuthi and assisted by class members 2019 moringa-sfc academy**
## Description
This application can be used to test whether the inputs entered by a user on the interface is valid to form a triangle.User input will be tested whether its a number or a string . if a number is entered, the programme will convert the input into an integer and parse it to the next step where all inputs will be tested whether they can for a triangle and which type of triangle.
if the inputs cannot form a triangle a message will be displayed to notify the user that "the values can not for a triangle with the values entered displayed being part of the message.
If the use enters a valid input then the message will be displayed showing the type of triangle and the values entered. in addition a brief description will be displayed as a picture.
## Setup/Installation Requirements

* run the index.html using a browser preferably chrome
* make sure all the files and sub-folders are on the same folder.

## Known Bugs
A bug has been noted of the application allowing any input to be entered. but this will be soorted in the upcomming update of this application that is version 2
## Technologies Used
This application was developed using css, javascript and html and can be opened using any browser by running the index.html. but i prefer using chrome to get the full experience.
## Support and contact details
For support and more<EMAIL_ADDRESS><<<<<<< HEAD
visit: https://emakuthi.github.io/triangle_tracker/
=======
visit :https://emakuthi.github.io/triangle_tracker/
>>>>>>> 2d378b4de9c36efee7c89f784090ad2b7718b07f
### License
*licenced by MIT*
Copyright (c) 2019 **Authored by Elvis Makuthi**

license: MIT


text:
---
layout: kanji-remain
v4: 2894
kanji: 麿
keyword: Utamaro
strokes: 18
on-yomi: 0
permalink: /rtk/麿/
---

## Koohii stories:

1) [Katsuo] 9-2-2007(48): Bending over constantly to cut all those woodblocks has given Utamaro a bad back. Now he smokes hemp while he works to ease the pain in in his spine.

2) [Fishface] 25-3-2011(2): 歌麿 うた-まろ: The poor guy with leukaemia smokes hemp to ease the pain from all the bone マロ that has to be injected into his spine. [Note: probably not medically accurate.].

3) [mantixen] 31-7-2009(1): A soft Mashimaro doll made with a hemp spine for some semblance of posture.

4) [lisbet] 7-4-2013(): -I- , utaマロ have tried several thingns to ease the pain in my spine from working too long in the caves; singing an uta, and smoking hemp included.


text:
# Overview

Kafka has many moving pieces, but also has a ton of helpful resources to learn available online. In this homework, your
challenge is to write answers that make sense to you, and most importantly, **in your own words!**
Two of the best skills you can get from this class are to find answers to your questions using any means possible, and to
reword confusing descriptions in a way that makes sense to you.

### Tips
* You don't need to write novels, just write enough that you feel like you've fully answered the question
* Use the helpful resources that we post next to the questions as a starting point, but carve your own path by searching on Google, YouTube, books in a library, etc to get answers!
* We're here if you need us. Reach out anytime if you want to ask deeper questions about a topic
* This file is a markdown file. We don't expect you to do any fancy markdown, but you're welcome to format however you like

### Your Challenge
1. Create a new branch for your answers
2. Complete all of the questions below by writing your answers under each question
3. Commit your changes and push to your forked repository

## Questions
#### What problem does Kafka help solve? Use a specific use case in your answer
* Helpful resource: [Confluent Motivations and Use Cases](https://youtu.be/BsojaA1XnpM)

#### What is Kafka?
* Helpful resource: [Kafka in 6 minutes](https://youtu.be/Ch5VhJzaoaI)

#### Describe each of the following with an example of how they all fit together:
 * Topic
 * Producer
 * Consumer
 * Broker
 * Partition

#### Describe Kafka Producers and Consumers

#### How are consumers and consumer groups different in Kafka?
* Helpful resource: [Consumers](https://youtu.be/lAdG16KaHLs)
* Helpful resource: [Confluent Consumer Overview](https://youtu.be/Z9g4jMQwog0)

#### How are Kafka offsets different than partitions?

#### How is data assigned to a specific partition in Kafka?

#### Describe immutability - Is data on a Kafka topic immutable?

#### How is data replicated across brokers in kafka? If you have a replication factor of 3 and 3 brokers, explain how data is spread across brokers
* Helpful resource [Brokers and Replication factors](https://youtu.be/ZOU7PJWZU9w)

#### What was the most fascinating aspect of Kafka to you while learning?

repository: kitmenke/streaming-data-pipeline

branch_name: refs/heads/main

gha_license_id: MIT

repo_name: kitmenke/streaming-data-pipeline


text:
# matterbridge-plus

Simple bridge between mattermost and IRC. (Uses the mattermost API instead of webhooks)

* Relays public channel messages between mattermost and IRC.
* Supports multiple mattermost and irc channels.
* Matterbridge-plus also works with private groups on your mattermost.

## Requirements:
* [Mattermost] (https://github.com/mattermost/platform/) 2.1.0 (stable, not a dev build)
* A dedicated user(bot) on your mattermost instance.

There is also a version with webhooks that doesn't need a dedicated user. See [matterbridge] (https://github.com/42wim/matterbridge/)

If you want to test with mattermost development builds, you also need to use the develop branch of matterbridge-plus.

## binaries
Binaries can be found [here] (https://github.com/42wim/matterbridge-plus/releases/tag/v0.1)

## building
Go 1.6 is required (or go1.5 with GO15VENDOREXPERIMENT=1)
Make sure you have [Go](https://golang.org/doc/install) properly installed, including setting up your [GOPATH] (https://golang.org/doc/code.html#GOPATH)

```
cd $GOPATH
go get github.com/42wim/matterbridge-plus
```

You should now have matterbridge binary in the bin directory:

```
$ ls bin/
matterbridge-plus
```

## running
1) Copy the matterbridge.conf.sample to matterbridge.conf in the same directory as the matterbridge binary.
2) Edit matterbridge.conf with the settings for your environment. See below for more config information.
3) Now you can run matterbridge-plus.

```
Usage of matterbridge-plus:
  -conf="matterbridge.conf": config file
```

Matterbridge will:
* connect to specified irc server and channel.
* send messages from mattermost to irc and vice versa, messages in mattermost will appear with irc-nick

## config
### matterbridge-plus
matterbridge-plus looks for matterbridge.conf in current directory.

Look at matterbridge.conf.sample for an example

```
[IRC]
server="irc.freenode.net"
port=6667
UseTLS=false
SkipTLSVerify=true
nick="matterbot"
channel="#matterbridge"
UseSlackCircumfix=false

[mattermost]
server="yourmattermostserver.domain"
team="yourteam"
#login/pass of your bot. Use a dedicated user for this and not your own!
login="yourlogin"
password="yourpass"
showjoinpart=true
channel="thechannel"
#whether to prefix messages from IRC to mattermost with the sender's nick. Useful if username overrides for incoming webhooks isn't enabled on the mattermost server
PrefixMessagesWithNick=false
#how to format the list of IRC nicks when displayed in mattermost. Possible options are "table" and "plain"
NickFormatter=plain
#how many nicks to list per row for formatters that support this
NicksPerRow=4
#Freenode nickserv
NickServNick="nickserv"
#Password for nickserv
NickServPassword="secret"

[general]
#request your API key on https://github.com/giphy/GiphyAPI. This is a public beta key
GiphyApiKey="dc6zaTOxFJmzC"

#multiple channel config
[channel "our testing channel"]
irc="#bottesting"
mattermost="testing"

[channel "random channel"]
irc="#random"
mattermost="random"
```

repository: sk-unikent/matterbridge-plus

repo_name: sk-unikent/matterbridge-plus


text:
# SubFinder Documentation

:+1::tada: First off, Many Thanks for taking the time to contribute! :tada::+1:

The follwing is a document describing the project structure of SubFinder subdomain enumeration tool and teaching you how to contribute to it.

### Project Structure

SubFinder is written in golang. The author has tried to keep the code as simple and modular as possible. There can be flaws in the golang implementation or there may be many things which can be done better. Please create an issue with the details and we would be more than happy to implement your suggestions. Pull Requests are more appreciated :+1:.

Comments can be found throughout the majority of the code base explaining functionality of usage. The codebase is pretty simple and modular and you can easily understand the structure and contribute to the codebase.

### Adding new Passive Sources

Passive Sources are data sources which can be used to find subdomains without directly connecting to the site. It can easily fetch many subdomains which cannot be found through plain bruteforcing and alterations. SubFinder uses multiple passive sources which are organized in `github.com/ice3man543/subfinder/libsubfinder/sources`. Each passive source has it's own directory which contains a subpackage under the package sources. Each passive source must export a query function. This allows the code to retain a similar structure throughout the tool.

- Create a directory `github.com/ice3man543/subfinder/libsubfinder/sources/example`
- Create a file called example.go in `example` directory.

Here is a boilerplate code for a passive package called example.

```golang
package example 	// The first line contains package name, like example

import (
	"github.com/ice3man543/subfinder/libsubfinder/helper" 	// Obligatory, Contains helper class
)

// Contains all subdomains found
var subdomains []string

//
// Query : Each passive package exports a Query function
//  which returns all results found.
//
// @param state : Current application state
// @param ch : Channel for our result structure
func Query(state *helper.State, ch chan helper.Result) {
	// Perform all the operations here
	...

	// In case of error handling, return error
	if err != nil {
		result.Error = err
		ch <- result
		return
	}

	for _, found := range block.Dns_names {
			if state.Verbose == true {
				if state.Color == true {
					fmt.Printf("\n[%sEXAMPLE%s] %s", helper.Red, helper.Reset, found)
				} else {
					fmt.Printf("\n[EXAMPLE] %s", found)
				}
			}
			subdomains = append(subdomains, found)
		}
	}

	// Return success result
	result.Subdomains = subdomains
	result.Error = nil
	ch <-result
}

```

Some notes :
1. A Package can have as many functions as you want. But all functionality must be exported by the Query function itself.
2. The helper class exports a function `helper.GetHTTPResponse("https://example.com?domain="+state.Domain, state.Timeout)` which can help with HTTP requests.
3. We have been keeping Package count almost to none. Please use `Regexp` package or native Golang HTML parser.

After you have created your package, it's time to import it. All logic for passive sources is in `github.com/ice3man543/subfinder/libsubfinder/engines/passive/passive.go`.

- Import your custom package in passive.go.
```golang
package passive

import (
	...

	// Load different Passive data sources
	...

	"github.com/ice3man543/subfinder/libsubfinder/sources/example"
)
```

- Add your data source to Source struct at the last of all sources before NoOfSources.

```golang
type Source struct {
	...
	Example 		bool 		// Your example data source

	NoOfSources		int
}

func PassiveDiscovery(state *helper.State) (finalPassiveSubdomains []string) {
```

- Add a false statement to your sourceConfig object at the last before NoOfSources.

```golang
func PassiveDiscovery(state *helper.State) (finalPassiveSubdomains []string) {
	sourceConfig := Source{... false, 0}
```

- Add a print statement to all data sources if-block. Also add a true statement to sourceConfig assignment and also increment NoOfSources field by 1.
```golang
if state.Sources == "all" {
		// Search all data sources

		...
		fmt.Printf("\n[-] Searching For Subdomains in Example")
		...

		sourceConfig = Source{... true, 13}
```

- Add an else-if block to the custom source config block.
```golang
else {
		// Check data sources and create a source configuration structure

		dataSources := strings.Split(state.Sources, ",")
		for _, source := range dataSources {
			if source == "crtsh" {
				fmt.Printf("\n[-] Searching For Subdomains in Crt.sh")
				sourceConfig.Crtsh = true
				sourceConfig.NoOfSources = sourceConfig.NoOfSources + 1

			...

			} else if source == "example" {
				fmt.Printf("\n[-] Searching For Subdomains in Example")
				sourceConfig.Example = true
				sourceConfig.NoOfSources = sourceConfig.NoOfSources + 1
			}
		}
```

- Add an if-block for main goroutine execution.
```golang
if sourceConfig.Virustotal == true { go virustotal.Query(state, ch) }
if sourceConfig.Example == true { go example.Query(state, ch) }
```

And finally, you have created your own custom passive source.

### Notes

This document is currently in development like this project. If there are any mistakes or questions, feel free to create an issue or contact me directly on twitter or email. Have fun developing it. If you do create something awesome, do share it and we will add your name to the Acknowledgements section.

---

Written by : @Ice3man

repository: ehsahil/subfinder

repo_name: ehsahil/subfinder


text:
---
# You don't need to edit this file, it's empty on purpose.
# Edit theme's home layout instead if you wanna make some changes
# See: https://jekyllrb.com/docs/themes/#overriding-theme-defaults
layout: default
---

Supported by the King's Together Multi & Interdisciplinary Research Scheme, the project 'Taming the complexity of the law: modelling and visualization of dynamically interacting legal systems' (April-November 2018) aimed at exploring the complexity of the legal system from a unique range of perspectives, from user interface design to network analysis, from legal expertise to geography and informatics.

repository: kingsdigitallab/tcl

gha_language: JavaScript

repo_name: kingsdigitallab/tcl


text:
## AutoGluon: AutoML Toolkit for Deep Learning

[![Build Status](http://ci.mxnet.io/view/all/job/autogluon/job/master/badge/icon)](http://ci.mxnet.io/view/all/job/autogluon/job/master/)
[![Pypi Version](https://img.shields.io/pypi/v/autogluon.svg)](https://pypi.org/project/autogluon/#history)

AutoGluon automates machine learning tasks enabling you to easily achieve strong predictive performance in your applications.  With just a few lines of code, you can train and deploy high-accuracy deep learning models on tabular, image, and text data.  Get started with:

```python
# First install package from terminal:  pip install mxnet autogluon

from autogluon import TabularPrediction as task
train_data = task.Dataset(file_path='https://autogluon.s3.amazonaws.com/datasets/Inc/train.csv')
test_data = task.Dataset(file_path='https://autogluon.s3.amazonaws.com/datasets/Inc/test.csv')
predictor = task.fit(train_data=train_data, label='class')
performance = predictor.evaluate(test_data)
```

See the [AutoGluon Website](http://autogluon.mxnet.io/index.html) for instructions on:
- [Installing AutoGluon](http://autogluon.mxnet.io/index.html#installation)
- [Learning with tabular data](http://autogluon.mxnet.io/tutorials/tabular_prediction/tabular-quickstart.html)
- [Learning with image data](http://autogluon.mxnet.io/tutorials/image_classification/beginner.html)
- [Learning with text data](http://autogluon.mxnet.io/tutorials/text_classification/beginner.html)
- More advanced topics such as [Neural Architecture Search](http://autogluon.mxnet.io/tutorials/nas/index.html)

## License

This library is licensed under the Apache 2.0 License.

repository: johnjdailey/autogluon

repo_name: johnjdailey/autogluon


text:
Rest.ts express adapter
=======================

Serve [Rest.ts](https://code.hmil.fr/rest.ts) typed HTTP APIs from the comfort of express.

Please refer to the usage doc on the [project homepage](https://code.hmil.fr/rest.ts).


text:
# BannerlordTemplate
Simple template visual studio solution for building modules for Bannerlord.

You **must edit** BannerlordTemplate.csproj GameFolder tag to match your game install folder.

Debug mode builds the files to \bin folder inside the solution.
Release mode builds the files to the \Modules game folder.

Modding documentation: https://docs.bannerlordmodding.com/

repository: Arthur-Neto/BannerlordTemplate

repo_name: Arthur-Neto/BannerlordTemplate


text:
# About

* Easy time tracking plugin
* Allow users to add time directly from calendar view
  * Week view
  * Working week view
  * Month view
  * Month working view

## Installation

* connect to redmine installation

```
cd $REDMINE_DIR/plugins/
git clone https://github.com/iotbzh/iotbzh-redmine-plugin
```

* Enable plugin in Redmine webinterface and restart web server

```
systemctl restart httpd/apache2/nginx
```

* Plugin is available from top banner : EasyTT

## Specific for wiki

* The plugin had a wiki shortcut in topbar.
* If you want to edit or delete the link, edit  **init.rb** file and change

```
menu :top_menu, :wiki, { :controller => 'wiki', :action =>  'index', :project_id => 'iot-team'  }, :caption => 'WIKI'
```

repository: fulljackz/iotbzh-redmine-plugin

repo_name: fulljackz/iotbzh-redmine-plugin


text:
# iruby-notebook-docker
This is iruby notebook with docker, so you can start sciruby without much configuration

## Requirements
* docker

## How to run
* docker build -t rubytues/iruby .
* docker run -p 8888:8888 -v $(pwd)/workspace:/home/sciruby/workspace -it rubytues/iruby /bin/bash
* jupyter notebook --NotebookApp.token='' --ip=<IP_ADDRESS> --port=8888 --no-browser

## Note
* The workspace folder in the repo is mounted to the docker container, so anything that you do in the container will be saved to that folder and you still able to see it after the container being shut down.
* Make sure the folder is correctly mounted by creating a new file in jupyter notebook, then see if the file exist in the host folder.

Happy learning!


text:
#### NEWS HIGHLIGHTS.
### By Canssidle

## Description
In this News Highlights website the user is able to search the website on their browser.The user is able to access different categories of the latest news .On the home page the user is free to choose the category he or she is interested in from business news to entertainment news to sports.The user selects the preferred choice is able to view all the latest news in details with regards to the choosen option.

## Behaviour of the application
* The user is able to get the latest business news
* The user is able to get the latest entertainment news
* The user is able to get the latest sports news.

## Development
* There are two classes namely user and credentials to create instances of the classes
* Tests to see the functionality of the classes

## Test Driven Development
* Test done using python(unittest)

## Setup/Installation Requirements
* Git clone or visit the python documentation:https://docs.python.org/3/

## Technologies Used
* python

## Support and contact details
 If any complications occur email me at<EMAIL_ADDRESS>### License

* MIT Copyright (c) 2018 **Canssidle

repository: canssidle/News

repo_name: canssidle/News


text:
Title: Haskell Wikibook: Pattern Matching
Date: 2016-04-24 08:33
Author: Tony
Slug: haskell-wikibook-pattern-matching
Status: published

Here are my answers to the exercises in the [Haskell Wikibook: Pattern Matching](https://en.wikibooks.org/wiki/Haskell/Pattern_matching) chapter:

> 1.  Test the flawed `h` function above in GHCi, with arguments equal to and different from 1. Then, explain what goes wrong.
>
>     The following code:
>
>
>         k = 1
>
>         h :: Int -> Bool
>         h k = True
>         h _ = False
>
>
>     won't compile because the `k` that's defined to equal 1 is different from the `k` in the pattern. The two patterns for `h` overlap, they both match everything, one is anonymous and one is bound to `k`.
>
>
> 2.  In this section about pattern matching with literal values, we made no mention of the boolean values `True` and `False`, but we can do pattern matching with them as well, as demonstrated in the [Next steps](https://en.wikibooks.org/wiki/Haskell/Next_steps#Introducing_pattern_matching "Haskell/Next steps") chapter. Can you guess why we omitted them? (Hint: is there anything distinctive about the way we write boolean values?)
>
>     The literal values True and False begin with capital letters, and so could be confused with Types or Constructors.

> Implement `scanr`, as in the exercise in [Lists III](https://en.wikibooks.org/wiki/Haskell/Lists_III#Scans "Haskell/Lists III"), but this time using an as-pattern.
>
>
>     myScanr :: (a -> b -> b) -> b -> [a] -> [b]
>     myScanr f n l = foldr g [n] l
>             where
>             g x list@(y:ys) = (f x y) : list

</p>

repository: tlocke/inconsistent-dedent

repo_name: tlocke/inconsistent-dedent


text:
---
published: true
layout: post

title: Sending Apache Logs to LogZilla
subtitle: How to send apache logs to syslog-ng
author: tomd
tags: [Apache, Syslog, LogZilla, syslog-ng]
summary: This method is not limited to Apache...
headline-bg: /assets/images/blog/post_images/sending-apache-logs-to-logzilla/apache-chopper.jpg
redirect_from:
 - /documentation/sending-your-apache-logs-to-logzilla

---

This method is not limited to Apache, but will work for any Common Log Format log. Each line in a file stored in the Common Log Format has the following syntax:

	host ident authuser date request status bytes

The first step is to add a new source to your syslog-ng configuration. In the /etc/syslog-ng/conf.d directory, we'll create a file and name it apache.conf.

	cd /etc/syslog-ng/conf.d
	 vi apache.conf

Once the file is open in the editor, we'll first add the source.

	source s_apache {
	 file("/var/log/apache2/access.log");
	 file("/var/log/apache2/error.log");
	 };

You can also add the ssl-access.log if you have enabled that on your web server. In the same file, we'll need to add a destination.

	log { source(s_apache);
	 destination(d_tls);
	 };

In this example, the destination is a TLS tunnel created in a previous tutorial. Save the file and quit, then restart syslog-ng.

	service syslog-ng restart

You should now be receiving apache events on your Logzilla server, but they'll look a little off. That's because they haven't been formatted yet. To do that, we'll need to edit the apache configuration.  This step will only work for Apache.  For other Common Log Format sources, each will have it's own solution for formatting.

	cd /etc/apache2
	 vi apache2.conf

In that file, you'll find a line like this:

	LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\" %v" combined

It will need a bit added to it (it will ignore the pre-set date).

	LogFormat "Jan 12 12:12:12 %v apache[666]: %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\" %v" combined

Save the file and restart Apache, and your logs should look like this:

	0 www user notice apache None <IP_ADDRESS> - - [07/Nov/2013:15:14:41 -0500] "GET /highslide/highslide.css HTTP/1.1" 304 209
	"http://www.yourserver.com/" "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0; EIE10;ENUSMSN)"
	www.yourserver.com

You can create custom reports to make further use of this data, or just [contact us](/contact.html) us to give you a hand with it.


text:
# Epicodus Track Selector Tool

#### This form helps a prospective student decide which course at Epicodus might be best for them , 02/17/17

#### By **Xi Xia**

## Description

This is form-based project that determines which course at Epicodus you should take based on your answers to the 10 questions.

Some questions directly and only affect your "preference score" for one single course.  Other questions affect multiple preference scores.  Your one answer to these questions will produce different scoring effects. While one score may increase by, say 1, another score affected by that question may only increase by 0.5.

This repo includes index.html, CSS, JS, and IMG folders.

## Setup/Installation Requirements

* _Clone trackSelector folder from github onto Desktop_
* _Open trackSelector folder_
* _Double Click index.html_
* _Start Answering Questions!_
* _link to github repo_

Internet connection is needed while using the tool. The JavaScript and Bootstrap resources are made available via CDN.

Ensure that there is a browser installed, ensure you have a link to github portfolio

## Known Bugs

None known at this time.

## Support and contact details

If there are issues/defects please contact<EMAIL_ADDRESS>
## Technologies Used

HTML, CSS, Bootstrap, JavaScript, Jquery

### License

This software is licensed under the MIT license

Copyright (c) 2017 **Xi Xia**

repository: XiXiaPdx/trackSelector

repo_name: XiXiaPdx/trackSelector


text:
# Robotics Series of Courses

- :dollar: _Udacity_: [Robotics Nanodegree][1]
- :dollar: _Udacity_: [Self-Driving Car Nanodegree][2]
- :dollar: _Udacity_: [Flying Car Nanodegree](https://www.udacity.com/course/flying-car-nanodegree--nd787)
- :dollar: _Coursera_: [Robotics Specialization][3], _University of Pennsylvania_
- _EdX_: [Robotics MicroMasters][4], _University of Pennsylvania_
- [Robot Academy][5], _Queensland University of Technology_

# Individual Robotics Courses

### EdX
- [Autonomous Mobile Robots][21], _ETHZurich_
- [Autonomous Navigation for Flying Robots][22], _Technische Universitat Munchen_
- [Underactuated Robotics][23], _Massachusetts Institute of Technology_
- [Robotics][24], _Columbia University in the city of New York_
- Robot Mechanics and Control [Part I][25] and [Part II][26], _Seoul National University_

### Coursera
- :dollar: [Control of Mobile Robots][27], _Georgia Institute of technology_

### Others
- [Autonomous Intelligent Systems][28], _University of Freiburg_
- [Introduction to Robotics][31], _Stanford Engineering Everywhere_
- [Introduction to Robotics][33], _MIT OCW_
- [Introduction to Aerial Robotics][43], _Dr Kostas Alexis, University of Nevada, Reno_
- [Deep-learning for Self-Driving Cars][48], _MIT 6.S094: Deep Learning for Self-Driving Cars_
- [SLAM playlist][62], _Cyrill Stachniss_

# Hands-on
- [MIT Race Car][29], _Massachusetts Institute of Technology_
- [SLAM Lectures][30], _Clause Brenne_
- [F1/10 by Penn Engineering ][59]

# Useful Concepts
- [Artifical Intelligence and Machine-Learning](https://github.com/mithi/ai-playground)
- [Kalman Filters and Bayesian Thinking][55]
- **Control Systems**: [Control Bootcamp][19], _Steve Brunton_
- [Structure and Interpretation of Computer Programs][39]
- **Parallel Programming, Cuda**: [Udacity Introduction to Parallel Programming][37]
- **Functional Programming Paradigm**: [Haskell][45] + [Real World Haskell][47], [Scheme][44], and [Common Lisp][46]

# Useful Tools
- [C++ Resources][65] and [Python][53]
- :dollar: **ROS**: [Learn ROS Fast][14], _Robot Ignite Academy_
- **Source Control**: [GitKraken][15], and [GIT SCM][42]
- [The Unix Shell][57] and [Shell Scripting][56]
- **Fancy Text Editors**: [Sublime](https://www.sublimetext.com/) and [Atom](atom.io)
- [MATLAB][54] and [LaTex Tutorials][32], _Michelle Krummel_

# Blogs
- [MIT RaceCar Team 5 Documentation](https://mit-racecar.github.io/6.141-spring-2016-team-5-documentation/)
- [Jetson Hacks][60]
- [Reality Bytes by Andrew Dahdouh](https://realitybytes.blog/)

# Algorithms and Technical Interviews
- [Andreis Interview Repository][51]
- [Algorithm Playground][64]

# Related Lists
* [Awesome Artificial Intelligence](https://github.com/owainlewis/awesome-artificial-intelligence)
* [Awesome Computer Vision](https://github.com/jbhuang0604/awesome-computer-vision)
* [Awesome Machine Learning](https://github.com/josephmisiti/awesome-machine-learning)
* [Awesome Deep Learning](https://github.com/ChristosChristofidis/awesome-deep-learning)
* [Awesome Deep Vision](https://github.com/kjw0612/awesome-deep-vision)
* [Awesome Reinforcement Learning](https://github.com/aikorea/awesome-rl/)
* [Awesome Robotics](https://github.com/ahundt/awesome-robotics)
* [Awesome Robotics Libraries](https://github.com/jslee02/awesome-robotics-libraries)
* David Silver: [Related online courses][41]

[1]: https://www.udacity.com/robotics
[2]: https://www.udacity.com/drive
[3]: https://www.coursera.org/specializations/robotics
[4]: https://www.edx.org/micromasters/pennx-robotics
[5]: http://robotacademy.net.au
[6]: https://www.coursera.org/specializations/algorithms
[7]: https://www.coursera.org/learn/algorithms-part1
[8]: https://www.coursera.org/learn/algorithms-part2
[9]: https://www.youtube.com/watch?v=8mYfZeHtdNc&list=PLxc4gS-_A5VDXUIOPkJkwQKYiT2T1t0I8
[10]: https://www.youtube.com/watch?v=zDUo8HTFwLo&list=PLxc4gS-_A5VDvP_9W8JJ04zk6m1qTolzG
[11]: https://www.youtube.com/watch?v=3fwKlU9AwSY&list=PLfVsf4Bjg79Cu5MYkyJ-u4SyQmMhFeC1C
[12]: http://shop.oreilly.com/product/0636920049814.do
[13]: https://github.com/google/googletest/blob/master/googletest/docs/Primer.md
[14]: http://www.theconstructsim.com/construct-learn-develop-robots-using-ros/robotigniteacademy_learnros/
[15]: https://www.youtube.com/channel/UCp06FAzrFalo3txskS1gCfA/playlists

[18]: https://www.udemy.com/the-complete-elixir-and-phoenix-bootcamp-and-tutorial/
[19]: https://youtu.be/Pi7l8mMjYVE?list=PLMrJAkhIeNNR20Mz-VpzgfQs5zrYi085m
[20]: https://www.edx.org/course/analytics-edge-mitx-15-071x-3
[21]: https://www.edx.org/course/autonomous-mobile-robots-ethx-amrx-2
[22]: https://www.edx.org/course/autonomous-navigation-flying-robots-tumx-autonavx-0
[23]: https://www.edx.org/course/underactuated-robotics-mitx-6-832x-0
[24]: https://www.edx.org/course/robotics-columbiax-csmm-103x#!
[25]: https://www.edx.org/course/robot-mechanics-control-part-i-snux-snu446-345-1x
[26]: https://www.edx.org/course/robot-mechanics-control-part-ii-snux-snu446-345-2x
[27]: https://www.coursera.org/learn/mobile-robot
[28]: http://ais.informatik.uni-freiburg.de/teaching/ss16/robotics/index_en.php
[29]: https://mit-racecar.github.io
[30]: https://www.youtube.com/watch?v=B2qzYCeT9oQ&list=PLpUPoM7Rgzi_7YWn14Va2FODh7LzADBSm
[31]: https://see.stanford.edu/Course/CS223A
[32]: https://www.youtube.com/watch?v=SoDv0qhyysQ&index=1&list=PL1D4EAB31D3EBC449
[33]: https://ocw.mit.edu/courses/mechanical-engineering/2-12-introduction-to-robotics-fall-2005/lecture-notes/
[34]: https://bigmachine.io/products/the-imposters-handbook/
[35]: http://www.redblobgames.com/
[36]: http://leetcode.com
[37]: https://classroom.udacity.com/courses/cs344]
[38]: https://www.coursera.org/specializations/data-structures-algorithms
[39]: https://groups.csail.mit.edu/mac/classes/6.001/abelson-sussman-lectures/
[40]: https://medium.com/@paysa/self-driving-car-engineer-deep-dive-89b814f3ff04
[41]: https://medium.com/self-driving-cars/how-to-land-an-autonomous-vehicle-job-coursework-e7acc2bfe740
[42]: https://git-scm.com/book/en/v2
[43]: http://www.kostasalexis.com/introduction-to-aerial-robotics.html
[44]: http://htdp.org/2003-09-26/Book/curriculum.html
[45]: http://learnyouahaskell.com
[46]: http://gigamonkeys.com/book/
[47]: http://book.realworldhaskell.org/read/
[48]: http://selfdrivingcars.mit.edu/
[49]: https://people.eecs.berkeley.edu/~pabbeel/
[50]: https://deeplearning.ai
[51]: https://github.com/andreis/interview
[52]: https://www.labri.fr/perso/nrougier/teaching/c++-crash-course/index.html
[53]: http://docs.python-guide.org/en/latest/intro/learning/
[54]: https://matlabacademy.mathworks.com/
[55]: https://github.com/rlabbe/Kalman-and-Bayesian-Filters-in-Python/
[56]: https://www.shellscript.sh/
[57]: https://swcarpentry.github.io/shell-novice/
[58]: http://www.fast.ai/
[59]: http://f1tenth.org/lectures
[60]: http://www.jetsonhacks.com/
[61]: https://agi.mit.edu/
[62]: https://www.youtube.com/watch?v=V9qQc5X7O0k&list=PLgnQpQtFTOGQECnBvZSV61oxTrkPut-nc
[63]: https://developers.google.com/machine-learning/crash-course/
[64]: https://github.com/mithi/algorithm-playground
[65]: https://github.com/mithi/cpp-resources

license: Unlicense

gha_license_id: Unlicense


text:
# Diff to Html Node Module by [rtfpessoa](https://github.com/rtfpessoa)

Diff to Html generates pretty HTML diffs.

### Inspiration

This project uses [diff2html](https://github.com/rtfpessoa/diff2html) as base.

## Features

* `line-by-line` and `side-by-side` diff

* new and old line numbers

* inserted and removed lines

* GitHub like style

## Online Example

> Go to [Diff2HTML](http://rtfpessoa.github.io/diff2html/)

## Other Distributions

* [WebJar](http://www.webjars.org/)

* Manually download and import [diff2html.js](https://github.com/rtfpessoa/diff2html) into your page

* Node Package CLI [diff2html-cli](https://github.com/rtfpessoa/diff2html-nodejs-cli)

## Setup

    sudo npm install -g diff2html

## Usage

Require the library using:

```
var Diff2Html = require('diff2html').Diff2Html;
```

> Pretty Line-by-Line Html From Git Word Diff Output

    Diff2Html.getPrettyHtmlFromDiff(exInput)

> Pretty Side-by-Side Html From Git Word Diff Output

    Diff2Html.getPrettySideBySideHtmlFromDiff(exInput)

> Intermediate Json From Git Word Diff Output

    Diff2Html.getJsonFromDiff(exInput)

> Pretty Line-by-Line Html From Json

    Diff2Html.getPrettyHtmlFromJson(exInput)

> Pretty Side-by-Side Html From Json

    Diff2Html.getPrettySideBySideHtmlFromJson(exInput)

## Contribution

All contributions are welcome.

To contribute just send a pull request with your feature,fix,... and it will be reviewed asap.

## License

Copyright 2014 Rodrigo Fernandes. Released under the terms of the MIT license.

---


text:
In the next couple of posts, we're going to learn all about `promise`. In order to get the most out of these lessons, you need to know two things.

First of all, `promise`s are often used when you're fetching a JSON API and doing some AJAX work. As an example, take a look at my [blog API](http://wesbos.com/wp-json/wp/v2/posts). You can see that it lists my blog posts as JSON.

Secondly, I'm going to be using a thing called `fetch` to be able to `fetch` in this JSON API. It's not a library, it's actually something built right into the browser. It returns a `promise`.

If you haven't heard of `fetch` before, it's very similar to `$.getJSON', if you've used jquery before, or '$.ajax', or any of these other AJAX libraries. Except, rather than having to load an external library, it's going to be built right into the browser.

A `promise` is something that will happen between now and the end of time. A really nice way to think of that is something that will happen in the future, but probably not immediately.

In order to understand this, we need to be reminded that JavaScript is almost entirely asynchronous, which means that if we take some code here...

```js
console.log('Going to fetch the latest posts from Wes\' Blog');
const posts = fetch('http://wesbos.com/wp-json/wp/v2/posts');
console.log('Done!');
console.log(posts);
```

Using `console.log`, we're logging that we're fetching the latest posts from my blog, and attempt to store them in a variable called `posts` using `fetch` on the API URL, then in the console, we'll say that we're done, and then see the posts.

What happens when we refresh this? We see going to `fetch`, done immediately. How can we see both of these immediately?

If we call `posts` in the console, we don't see the data inside of it.

That's really important because `fetch` queues up the search. It goes over to wesbos.com and tries to `fetch` the posts immediately, but it does not just store them in your `post` variable.

That might be different if you're coming from another language like PHP, where it would stop everything from running. This will just queue it up and put inside of `posts`, not the post, but it puts what's called a `promise`. Which means like, "Hey, I don't have the data from it just yet, but here's a little IOU the posts from Wes's blog. When they do come back, I'll be sure to let you know about that."

Let's take a look at a different example, but first a quick refresher about callbacks. Let's say we're going to create a callback function for a link...

```js
$('a').on('click',function() {
    alert('hey');

})
```

In that example, the alert saying 'hey' is waiting for the user to click the link, which is called a callback. The callback will run only when someone clicks it.

So if we look at a different example here...

```js
const postPromise = fetch('http://wesbos.com/wp-json/wp/v2/posts');

postPromise.then()
```

It's the same thing here, where we have a `then` function that will only run when the data comes back. The `then` function will give us the data.

If we expand this a little further with an arrow function to log our `data`...

```js
const postPromise = fetch('http://wesbos.com/wp-json/wp/v2/posts');

postPromise.then(data => {
    console.log(data);
})
```

If we look at the console, it looks like we have a response here. After two seconds the data actually comes back. The console shows some good data, some headers and the body. That's probably what we want. If we open up our body, though, we don't get the actual posts. What we have here is the actual raw data, but it doesn't yet know that we're expecting JSON, because you could `fetch` any type of data, not just JSON.

So let's fix that. We could probably just do an implicit return like the first posts:

```js
const postPromise = fetch('http://wesbos.com/wp-json/wp/v2/posts');

postPromise.then(data => data.json()).then(data => {console.log(data)}

)
```

If we open it up in the console, we have the actual data that comes back from our blog. You see there's the content, excerpt, id, Wordpress slug, and all that stuff.

We have some good data here. Now, a couple of things we can do here is, probably you want to format it a little bit:

```js
const postPromise = fetch('http://wesbos.com/wp-json/wp/v2/posts');

postPromise
    .then(data => data.json())
    .then(data => {console.log(data)})
    .catch((err) => {
        console.error(err);
    })
```

Often people will put a `then` on its own line, just for readability's sake, but I've also added a `catch` function.

Remember, our `then` will only fire when the promise successfully comes back. If there is an error with the data that comes back, `catch` runs. Generally in your function, you'll have all of the `thens` that we need, it might be one, it might be four, but if we have a `catch` on the end, that will just catch any errors that happen anywhere along the way.

So let's pass some broken code, like a typo:

```js
const postPromise = fetch('http://wessboss.com/wp-jsan/wp/v2/posts');

postPromise
    .then(data => data.json())
    .then(data => {console.log(data)})
    .catch((err) => {
        console.error(err);
    })
```

In the console, we see our error here, along with some other errors that happened. But the `TypeError` gets thrown by this catch block here, which reports that it failed to `fetch` anything.

That's `fetch` at a very high level.

license: ISC

path: /34 - Promises.md

filename: 34 - Promises.md


text:
# dojo-crypto

The crypto package provides cryptographic utilities including hashing and signing functions.

## Features

Coming soon!

## How do I use this package?

Users will need to download and compile directly from this repository and
[dojo/core](https://github.com/dojo/core) for the time being. Precompiled
AMD/CommonJS modules will be provided in the future as our release tools are
improved.

Once you've downloaded `dojo-core` and `dojo-crypto`, perform the following
steps:

```sh
cd dojo-core
grunt dist
cd dist
npm link
cd ../../dojo-crypto
npm install
npm link dojo-core
```

## How do I contribute?

We appreciate your interest!  Please see the [Guidelines Repository](https://github.com/dojo/guidelines#readme) for the
Contributing Guidelines and Style Guide.

## Testing

Test cases MUST be written using Intern using the Object test interface and
Assert assertion interface.

90% branch coverage MUST be provided for all code submitted to this repository,
as reported by istanbul’s combined coverage results for all supported
platforms.

## Licensing information

© 2017 JS Foundation & contributors. [New BSD](http://opensource.org/licenses/BSD-3-Clause) license.


text:
<!--- Provide a general summary of your changes in the Title above -->

<!-- If you don't mind add a fun gif or meme, but no pressure -->
![A GIF or MEME to give some spice of the internet](url)

## Description
<!--- Describe your changes in detail -->

## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here. -->
[issue](https://github.com/BrainJS/brain.js/issues/###)

## How Has This Been Tested?
<!--- Please describe in detail how you tested your changes. -->
<!--- Include details of your testing environment, tests ran to see how -->
<!--- your change affects other areas of the code, etc. -->

## Screenshots (if appropriate):

## Types of changes
<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

## Author's Checklist:
<!--- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [ ] My code focuses on the main motivation and avoids scope creep.
- [ ] My code passes current tests and adds new tests where possible.
- [ ] My code is [SOLID](https://en.wikipedia.org/wiki/SOLID_(object-oriented_design)) and [DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself).
- [ ] I have updated the documentation as needed.

## Reviewer's Checklist:
- [ ] I kept my comments to the author positive, specific, and productive.
- [ ] I tested the code and didn't find any new problems.
- [ ] I think the motivation is good for the project.
- [ ] I think the code works to satisfies the motivation.

<!-- Thanks to the folks of brain.js for this Template! -->

repository: mvrahden/recurrent-js

gha_language: TypeScript

repo_name: mvrahden/recurrent-js


text:
# Exam requirements

## Backend

- You have a starter code for NodeJS backend that follows MVC pattern
- Controllers are ready to be used on a way that you should provide the logic for it
- Model and a seed function that are ready to use
- server.js established for express backend with all the needed middlewares and imports

## Requirements

- Create a home endpoint that can be requested via GET method just to check everthing is working fine

- Create an endpoint that requests data from [this](https://coffeepedias.herokuapp.com/coffee-list/) api and return it as a response. your endpoint should be requested using GET method
and the provided api also should be requested using axios.get()

- The api will return to you a list of coffee types, each coffee type has id, title, description, img and ingredients

- Create an endpoint that when requested, it should recieve data of a coffee type and store it to the database as a favorite type.

- Create an endpoint that when requested it will return the list of favorite coffee list that is saved in the MongoDB

- Create an endpoint when requested, receives an id of a specific favorite coffee stored in MongoDB, and it is responsible for updating the requested favorite coffee.

- Create an endpoint when requested, receives an id of specific favorite coffee stored in MongoDB and deletes it from there.

- Test your endpoints in thunder client before using them in the frontend.

## Frontend

- Refer to the final result demo video that is provided to you to make sure you meet the requirements.

- Create a header component that has a navbar.
- the navbar should have two links/list items: Home and Favorite list.
- Create a component called home, it should show coffee types that are requested from your backend api once the component is displayed on screen, the items should be displayed in cards.

- Create a component called coffeecard, that will display coffee title, image, ingredients on the card with one button that is responsible for adding it to the favorite list.

- Create favlist component that once loaded, it will display the list of favorite coffee that are coming from your backend, items should be displayed in cards, each card must show : title, image and ingredients of the coffee, each card must also have two buttons: update and delete.

- When update button is clicked, it should show a modal that contains a form the form must have title field that has the coffee title as default value, ingredients field that also have the coffee ingredients as default value, the user should be able to update those fields, and a submit button should also be there to submit the updated data to your update endpoint that will be responsible for updating the favorite coffee in the database.

## Stretch goal

- Create a search functionality that enables the user to search for a coffee type by name using the coffee api that is provided for you.

- Sample request: `https://coffeepedias.herokuapp.com/coffee-list/?query=Black`


text:
# discord-webhooks
Simple wrapper for building and sending webhooks with Discord's webhook API

Allows for both simple small uses (single Webhook object to single Hook (url)) and mass dispatchers for multiple channels or with multiple Webhook object.

Built with simple end-user OOP approach in mind (method chaining for example).

@todo README & documentation

```php
// dispatcher manages building and sending webhooks
$dispatch = \Forien\WebhookDispatcher::getInstance();

// to create new Webhook object you can use either:
$webhook = $dispatch->createWebhook();
// or:
$webhook = new \Forien\DiscordHelpers\Webhook();

$webhook->setContent('some basic text');
$embed = $webhook->addEmbed();
$embed->addAuthor(['name' => 'Forien', 'icon_url' => '/*...*/']);
$embed->addField()->setName('name')->setValue('value');
$embed->addField()->setName('name2')->setValue('value2');
$embed->setColor('#cba');

// you can either add Hooks (Discord webhook urls) through either:
$dispatch->addHook('https://...');
$dispatch->setHooks(['https://...', ...])

// you can send attach Webhook object to Dispatcher if you created it independly:
$dispatch->addWebhook($webhook);

// you can send all attached Webhook objects at once (every will be sent to every Hook):
$dispatch->send()

// you can also specify single webhook-hook:
$dispatch->send($webhook, $hook);
```

repository: Forien/discord-webhooks

repo_name: Forien/discord-webhooks


text:
<IP_ADDRESS>.1 Server Name Indication http://www.ietf.org/internet-drafts/draft-ietf-tls-rfc4346-bis-01.txt

   [TLS1.1] does not provide a mechanism for a client to tell a server
   the name of the server it is contacting.  It may be desirable for
   clients to provide this information to facilitate secure connections
   to servers that host multiple 'virtual' servers at a single
   underlying network address.

   In order to provide the server name, clients MAY include an extension
   of type "server_name" in the (extended) client hello.  The
   "extension_data" field of this extension SHALL contain
   "ServerNameList" where:

         struct {
             NameType name_type;
             select (name_type) {
                 case host_name: HostName;
             } name;
         } ServerName;

         enum {
             host_name(0), (255)
         } NameType;

         opaque HostName<1..2^16-1>;

         struct {
             ServerName server_name_list<1..2^16-1>
         } ServerNameList;

   Currently the only server names supported are DNS hostnames, however

Dierks & Rescorla            Standards Track                    [Page 42]draft-ietf-tls-rfc4346-bis-01.txt  TLS                         June 2006

   this does not imply any dependency of TLS on DNS, and other name
   types may be added in the future (by an RFC that Updates this
   document).  TLS MAY treat provided server names as opaque data and
   pass the names and types to the application.

   "HostName" contains the fully qualified DNS hostname of the server,
   as understood by the client. The hostname is represented as a byte
   string using UTF-8 encoding [UTF8], without a trailing dot.

   If the hostname labels contain only US-ASCII characters, then the
   client MUST ensure that labels are separated only by the byte 0x2E,
   representing the dot character U+002E (requirement 1 in section 3.1
   of [IDNA] notwithstanding). If the server needs to match the HostName
   against names that contain non-US-ASCII characters, it MUST perform
   the conversion operation described in section 4 of [IDNA], treating
   the HostName as a "query string" (i.e. the AllowUnassigned flag MUST
   be set). Note that IDNA allows labels to be separated by any of the
   Unicode characters U+002E, U+3002, U+FF0E, and U+FF61, therefore
   servers MUST accept any of these characters as a label separator.  If
   the server only needs to match the HostName against names containing
   exclusively ASCII characters, it MUST compare ASCII names case-
   insensitively.

   Literal IPv4 and IPv6 addresses are not permitted in "HostName".  It
   is RECOMMENDED that clients include an extension of type
   "server_name" in the client hello whenever they locate a server by a
   supported name type.

   A server that receives a client hello containing the "server_name"
   extension, MAY use the information contained in the extension to
   guide its selection of an appropriate certificate to return to the
   client, and/or other aspects of security policy.  In this event, the
   server SHALL include an extension of type "server_name" in the
   (extended) server hello.  The "extension_data" field of this
   extension SHALL be empty.

   If the server understood the client hello extension but does not
   recognize the server name, it SHOULD send an "unrecognized_name"
   alert (which MAY be fatal).

   If an application negotiates a server name using an application
   protocol, then upgrades to TLS, and a server_name extension is sent,
   then the extension SHOULD contain the same name that was negotiated
   in the application protocol.  If the server_name is established in
   the TLS session handshake, the client SHOULD NOT attempt to request a
   different server name at the application layer.

repository: GsDevKit/Cryptography

gha_language: Smalltalk

repo_name: GsDevKit/Cryptography