text:
![SunshineJS](https://cloud.githubusercontent.com/assets/9622/7943559/11a32706-091a-11e5-8d6a-ac90fd9f9923.jpg)

Experimental framework for client-side apps, based on Flux and functional reactive programming

Why Sunshine
------------

Sunshine is the control layer (for lack of a better word) of your MVWTF app.
It is essentially an implementation of [Flux][] with a particularly functionl angle.
Sunshine provides an event system to listen for user intentions,
server-side state changes, etc.;
and an event handling system that is used to update application state using
pure(-ish) functions.

Sunshine can be used with any view layer, in theory.
It has included support for React.
In the future support may be added for other frameworks,
such as [Mercury's][Mercury] or [Mithril's][Mithril] virtual DOM implementations.

[Flux]: https://facebook.github.io/flux/
[Mercury]: https://github.com/Raynos/mercury
[Mithril]: https://lhorie.github.io/mithril/

The goals of Sunshine are:

- unidirectional data flow, _à la_ Flux
- State should stay in one place, and it should be immutable.
- pure(-ish)[*][pure-ish] JavaScript
- type safety with Flow
- isomorphism (the same code can be used client-side and server-side)
- small size (don't want to have to maintain a big pile o' code)

[pure-ish]: #user-content-questions-that-someone-might-one-day-ask

Sunshine shamelessly takes ideas from [re-frame][].
Re-frame keeps all app state in one immutable data structure,
and encourages implementers to think of it as an in-memory database.
Views go through a query layer to read state,
which decouples views from implementation details of app state.
State is updated by event handlers.
There is a lot of explanation on why this works well in
[re-frames excellent readme][re-frame readme].

Sunshine does basically the same thing.
But Sunshine brings in a new idea for the query and update layers:
lenses act as symmetric getters and setters,
which decouples both the view and event handlers from state implementation.
Apart from lenses,
the reasons for creating a new framework are to build in type-safety,
and to use pure(-ish) JavaScript.

[re-frame]: https://github.com/Day8/re-frame
[re-frame readme]: https://github.com/Day8/re-frame/blob/master/README.md

Prerequisites
-------------

Install Sunshine using npm:

    $ npm install --save https://github.com/hallettj/sunshine.git

You will also need nanoscope to create lenses:

    $ npm install --save nanoscope

To use Sunshine, you must use a build system that can handle JSX markup and
Flow type annotations.
The recommended choice is [Babel][].
A pre-compiled version of Sunshine may be made available in the future.

[Flow][] is recommended, but not required.

[Babel]: https://babeljs.io/
[Flow]: http://flowtype.org/

What is Sunshine
----------------

At a hand-wavy level, this is what a Sunshine app looks like:

    +------------+     +--------------+      +-----------+
    |            |     |              |----->|           |
    |   server   |     |   handlers   |      |   state   |----+
    |            |     |              |<-----|           |    |
    +------------+     +--------------+      +-----------+    |
             |              ^                                 |
             |              | app.on()                        |
      emit() |              |                                 |
             |       +------------------+                     |
             |       |                  |                     | getSubscribers()
             +------>|   event stream   |                     |
                     |                  |                     |
                     +------------------+                     |
                            ^   ^                             |
    +--------------+        |   |          +---------------+  |
    |              |        |   |          |               |  |
    |   timer /    |--------+   +----------|   view /      |<-+
    |   whatever   |                       |   component   |
    |              |  emit()      emit()   |               |
    +--------------+                       +---------------+

For a much more concrete look,
see the example todo app under [examples/todomvc/][example].

[example]: https://github.com/hallettj/sunshine/tree/master/examples/todomvc

Everything in Sunshine revolves around the `App` class,
and around app state.
The first thing that you will do is to define an initial state.
Here is a very simple example:

    type AppState = { todos: Todo[] }
    type Todo = {
        title: string,
        completed: boolean,
    }

    var initialState: AppState = { todos: [] }

And the next thing you will do is create an app:

    import * as Sunshine from 'sunshine/react'

    var app = new Sunshine.App(initialState)

Normally to create React components you create subclasses of `React.Component`.
Sunshine has its own component class that provides methods for subscribing to
app state, and for emitting events.

    type DefaultProps = {}
    type Props = { pageSize: number }
    type ComponentState = { todos: Todo[] }

    class TodoApp extends Sunshine.Component<DefaultProps,Props,ComponentState> {
        render(): React.Element {
            var todos = this.state.todos.map(todo => (

{todo.title}

            ))
            return (

                    <form onSubmit={this.addTodo.bind(this)}>
                        <input type="text" ref="title"/>
                        <input type="submit" value="create todo"/>
                    </form>

                        {todos}

            )
        }
    }

Sunshine components need to have a reference to a `Sunshine.App` instance.
To set that up,
render your top-level component inside of a `Sunshine.Context` component.

    React.render(
      <Sunshine.Context app={app}>
        <TodoApp pageSize={10} />
      </Sunshine.Context>,
      document.getElementById('todoapp')
    )

TODO: explain how to make `Sunshine.Context` work with react-router.

This component uses `this.state`,
but we have not put anything there yet.
We will need to connect the component to app state.
But first we need to write a lens that the component can use to query state.

The component wants access to the list of todos in the app state.
But let's say that the component should only display uncompleted todos.
And the number of todos returned should be limited based on the component's
`pageSize` prop.

    import nanoscope from 'nanoscope'

    var todosLens: Lens<AppState,Todo[]> = new nanoscope.PathLens('todos')

    var activeTodos: Lens<AppState,Todo[]> = todosLens.filter(todo => !todo.completed)

    function todosPage(pageSize: number): Lens<AppState,Todo[]> {
        return activeTodos.slice(`:${pageSize}`)
    }

A lens is used to focus in on a specific piece of a (possibly large) data
structure.
Using a lens, that piece can be either read or (depending on the lens) updated.
The same lens can be reused on many different data structures.

`PathLens` focuses on a property in an object.
The lenses above focus on the `todos` property in a value of type `AppState`.
The `activeTodos` gets a derived view by filtering that focus down to
uncompleted todos.
`todosPage` constructs a lens from a parameter,
further reducing the derived view of `activeTodos` down to just one page of
active todos.

The `` `:${pageSize}` `` construct is an [ES6 template string][].

[ES6 template string]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/template_strings

Here are some examples of how these lenses can be used:

    // reading todos
    var todos = todosLens.get(initialState)

    // setting todos;
    // `newState` is a new object with the new todos list. `initialState` is not modified.
    var newState = todosLens.set(initialState, [{ title: 'add todo', completed: true }])

    // modifying todos without replacing the whole list
    var newnewState = todosLens.map(initialState, todos => todos.concat({
        title: 'append a todo',
        completed: true,
    }))

    // reading a page of active todos
    var pageLens = todosPage(5)
    var page = pageLens.get(newnewState)

There is more information in the [nanoscope documentation][nanoscope].
TODO: Link to docs for v1.0 lenses.

[nanoscope]: http://kovach.me/nanoscope/

React components are connected to app state by implementing a method called
`getSubscribers`.
Use lenses to subscribe to just the little bits of state that your component
needs access to.

    class TodoApp extends Sunshine.Component<DefaultProps,Props,ComponentState> {

        getSubscribers(subscribe: Sunshine.Subscribe<AppState>): ComponentState {
            return {
                todos: subscribe(todosPage(this.props.pageSize))
            }
        }

        // render() is the same as before
    }

When your component first renders,
it will have an initial state based on the `initialState` used to construct
your `Sunshine.App` instance.
Whenever the app state updates,
the component's state will update automatically using the supplied lenses,
and the component will re-render.

To close the loop,
components need to emit events.
To emit events, we have to define event types.

    class AddTodoEvent {
        title: string;
        constructor(title: string) {
            this.title = title
        }
    }

We already have on `onSubmit` handler on a form that calls `addTodo`.
Let's implement that method.

    import React from 'react'

    class TodoApp extends Sunshine.Component<DefaultProps,Props,ComponentState> {

        // The rest of the class is the same as before.

        addTodo(event: Event) {
            event.preventDefault()
            var input = React.findDOMNode(this.refs.title)
            var title = input.value
            this.emit(new AddTodoEvent(title))
        }

    }

To emit events from somewhere other than a React component,
you can call `app.emit()`.

An event handler can grab that event and update app state accordingly.
A handler is just a function that takes an event and the current app state,
and returns an updated app state.
To register a handler,
you need to specify the type of event that it handles.

Remember how a lens can both read and update a data structure?

    app.on(AddTodoEvent, (state, { title }) => {
        return todosLens.map(state, todos => todos.concat({
            title: title,
            completed: false,
        }))
    })

The `map()` method on a lens takes an update function.
The function takes the existing value at the focused spot,
and returns a new value.
`map()` returns a new copy of the entire data structure
(in this case the whole app state)
with the new value.
This means that lenses provide immutable updates.

*Do not modify app state in-place.*
Sunshine makes the assumption that app state is never mutated.
If state is mutated, stuff will break.

The second argument to the event handler callback is an event object -
in this case an instance of `AddTodoEvent`.
The argument expression, `{ title }` destructures that object to pull out the
title property.

You may have multiple handlers registered for the same event type.
In this case,
each will get the updated state from the previous handler as input.
Components will not re-render until all handlers have run.

If you do not return anything from an event handler,
Sunshine will leave the existing state unchanged.

The choice to use a synchronous signature for event handlers is deliberate.
But you are going to need asynchronous updates too.
The thing to do in those cases is to emit a second event when an async
operation completes.
For example:

    class AddTodoWithAuthor {
        title: string;
        authorId: number;
        constructor(title: string, authorId: number) {
            this.title = title
            this.authorId = authorId
        }
    }

    class AppendTodo {
        todo: Todo
        constructor(todo: Todo) {
            this.todo = todo
        }
    }

    app.on(AddTodoWithAuthor, (state, { title, authorId }) => {
        fetch(`/users/${authorId}`).then(response => {
            response.json().then(author => {
                var todo = {
                    title,
                    author,
                    completed: false,
                }
                app.emit(new AppendTodo(todo))
            })
        })
    })

    app.on(AppendTodo, (state, { todo }) => {
        return todoLens.map(state, todos => todos.concat(todo))
    })

And that is Sunshine.
This documentation is just a draft;
if you still have questions,
your best bet is to pester the author.
The author's number is<PHONE_NUMBER>.

Questions that someone might one day ask
----------------------------------------

### What is the -ish in pure(-ish) JavaScript?

Sunshine uses ES6, JSX, and Flow.
ES6 is JavaScript of course.
It just happens to be JavaScript of the future.
The author considers the added syntax of JSX and Flow to be small enough that
code using these can still be called "JavaScript".
Opinions may vary.

### How does Sunshine relate to Relay?

Compared to [Relay][],
Sunshine is more focused, and smaller
(presumably - I have not seen any source for Relay).
Sunshine is intended to do one thing: manage client state.

Relay is a client and server architecture that unifies client rendering with
server state.
Sunshine makes no assumptions about server implementation.
This means that Sunshine can be used with a variety of different server architectures,
or with no server at all.
On the other hand,
this means that Sunshine gives you less out of the box than Relay does.
In particular you will have to implement your own system for synchronizing
client and server state,
or use another library that specializes in that area.

[Relay]: https://facebook.github.io/react/blog/2015/02/20/introducing-relay-and-graphql.html

### How does Sunshine relate to react-cursor?

Sunshine is sort of a mixture of [react-cursor][] and Flux.
TODO: explain this

[react-cursor]: https://github.com/dustingetz/react-cursor

### Isn't copying the entire app state on every update inefficient?

TODO: discuss tradeoffs, suggest Immutable.js
TODO: lens suport for Immutable.js

repository: rachoac/sunshine

repo_name: rachoac/sunshine


text:
![ALT](/media/images/gemm-hierarchy-with-epilogue-no-labels.png "CUTLASS Profiler")

[README](/README.md#documentation) > **CUTLASS Profiler**

# CUTLASS Profiler

The CUTLASS Profiler is a command-line driven test and profiling environment for CUTLASS computations
defined in the CUTLASS Instance Library. The CUTLASS Profiler is capable of executing each GEMM, Sparse Gemm,
Conv2d, and Conv3d kernel.

The CUTLASS Profiler may be compiled with:
```bash
$ make cutlass_profiler -j
```

To limit compilation time, only one tile size (typically 128x128) and threadblock cluster size (typically 2x1x1) is instantiated for each data type,
math instruction, and layout. To instantiate all sizes, set the following environment variable when running CMake from an
empty `build/` directory.
```bash
$ cmake .. -DCUTLASS_NVCC_ARCHS="70;75;80" -DCUTLASS_LIBRARY_KERNELS=all  -DCUTLASS_UNITY_BUILD_ENABLED=ON
...
$ make cutlass_profiler -j
```
Enabling the unity build places multiple kernel instances in one compilation unit, thereby reducing size of the compiled
binary and avoiding linker limitations on some platforms.

The CUTLASS Profiler sources are stored in
```bash
tools/
  profiler/
```

The CUTLASS Profiler usage statement may be obtained by executing `cutlass_profiler --help` and appears as follows.
```bash
CUTLASS Performance Tool
usage:

    cutlass_profiler [options]

  --help

  --mode=<string>                                  Cutlass profiler execution mode.
                                                    --mode=profile    regular verification and profiling (default)
                                                    --mode=dry_run    no kernels are launched or workspaces allocated
                                                    --mode=enumerate  lists all operation kind and operations
                                                    --mode=trace      executes a single device-side computation with
                                                                       no other kernel launches

  --device-info                                    Prints information on all GPUs present in the system

  --operation=<operation_kind>                     CUTLASS operation to profile.

  --kernels=<string_list>                          Filter operations by kernel names. For example, call all kernels with
                                                   ("s1688" and "nt") or ("s844" and "tn" and "align8") in their
                                                   operation name using --kernels="s1688*nt, s884*tn*align8"

  --ignore-kernels=<string_list>                   Excludes kernels whose names match anything in this list.

Device:
  --device=<int>                                   CUDA Device ID

  --compute-capability=<int>                       Override the compute capability.

  --llc-capacity=<capacity in KiB>                 Capacity of last-level cache in kilobytes. If this is non-zero,
                                                   profiling phases cycle through different input tensors to induce
                                                   capacity misses in the L2.

Initialization:
  --initialization=<bool>                          Enables initialization (default: true). If false, device memory is
                                                   not initialized after allocation.

  --initialization-provider=<provider>             Selects initialization provider {host, device*}. (default: '*')

  --dist=<distribution>                            Data distribution of input tensors {uniform*, gaussian, identity, sequential}
                                                    --dist=uniform,min:<double>,max:<double>,scale:<integer>
                                                    --dist=gaussian,mean:<double>,stddev:<double>,scale:<integer>
                                                    --dist=sequential,start:<double>,delta:<double>,scale:<integer>
                                                    --dist=identity

  --seed=<int>                                     Random number generator seed. Used to enforce deterministic
                                                   initialization.

Library:
  --library-algo-mode=<mode>                       Indicates algorithm mode used to call libraries such as cuBLAS and cuDNN.
                                                   mode={default*,matching,best}

  --library-algos=<range-list>                     If --algorithm-mode=best, permits specifying a selection of algorithms.

Profiling:
  --workspace-count=<workspace count>              Number of discrete workspaces maintained to avoid cache-resident
                                                 If zero (default), the amount is chosen for each workload based on
                                                 capacity of the last-level cache.

  --profiling-iterations=<iterations>              Number of iterations to profile each kernel. If zero, kernels
                                                   are launched up to the profiling duration.

  --warmup-iterations=<iterations>                 Number of iterations to execute each kernel prior to profiling.

  --sleep-duration=<duration>                      Number of ms to sleep between profiling periods (ms).

  --profiling-enabled=<bool>                       If true, profiling is actually conducted.

Verification:
  --verification-enabled=<bool>                    Whether to perform verification checks.

  --epsilon=<error>                                Error threshold. Setting to zero (default) requires
                                                   bit-level equivalence.

  --nonzero-floor=<floor>                          Results whose absolute value is less than this quantity
                                                   are treated as zero for comparisons.

  --save-workspace=<string>                        Specifies when to save the GEMM inputs and results to the filesystem.
                                                    --save-workspace=never      never save workspace (default)
                                                    --save-workspace=incorrect  save workspace for incorrect results
                                                    --save-workspace=always     always save workspace

  --verification-providers=<providers>             List of providers used to verify result. (default: '*')
                                                   Gemm verification-providers {cublas*}
                                                   Conv2d verification-providers {cudnn*, device*, host}

Report:
  --append=<bool>                                  If true, result is appended to possibly existing file. Otherwise,
                                                   any existing file is overwritten.

  --output=<path>                                  Path to output file for machine readable results. Operation kind and '.csv' is appended.

  --junit-output=<path>                            Path to junit output file for result reporting. Operation kind and '.junit.xml' is appended.

  --report-not-run=<bool>                          If true, reports the status of all kernels including those that
                                                   do not satisfy the given arguments.

  --tags=<column:tag,...>                          Inserts leading columns in output table and uniform values for each
                                                   column. Useful for generating pivot tables.

  --verbose=<bool>                                 Prints human-readable text to stdout. If false, nothing is written to stdout.

About:
  --version                                        CUTLASS 2.4.0 built on Nov 19 2020 at 11:59:00

Operations:

     gemm                                          General matrix-matrix product. D = alpha * A*B + beta * C
     spgemm                                        Structured sparse GEMM. D = alpha * A*B + beta * C
     conv2d                                        Conv2d operation. Output(Tensor4D) = alpha * Input(Tensor4D) * Filter(Tensor4D) + beta * Input(Tensor4D)
     conv3d                                        Conv3d operation. Output(Tensor5D) = alpha * Input(Tensor5D) * Filter(Tensor5D) + beta * Input(Tensor5D)

For details about a particular function, specify the function name with --help.

Example:

  $ cutlass_profiler --operation=Gemm --help

  $ cutlass_profiler --operation=Conv3d --help

  $ cutlass_profiler --operation=Conv2d --help

```

# GEMM

The CUTLASS Profiler is capable of executing GEMM and Sparse GEMM problems.

The CUTLASS Profiler can be built with cuBLAS enabled to use as a reference implementation. If CMake detects
the cuBLAS library available in the system, it is included as a dependency. This may be explicitly overridden
with CMake flag `CUTLASS_ENABLE_CUBLAS`.

## GEMM Arguments

The complete set of arguments available to each operation may be viewed by specifying the operation name
in addition to `--help`. The argument flags and their aliases usable for GEMM appear as follows.

```bash
$ ./tools/profiler/cutlass_profiler --operation=gemm --help

GEMM

  [enum]      --gemm_kind                                       Variant of GEMM (e.g. universal, gemm, planar_complex, planar_complex_array)
  [int]       --m,--problem-size::m                             M dimension of the GEMM problem space
  [int]       --n,--problem-size::n                             N dimension of the GEMM problem space
  [int]       --k,--problem-size::k                             K dimension of the GEMM problem space
  [tensor]    --A                                               Tensor storing the A operand
  [tensor]    --B                                               Tensor storing the B operand
  [tensor]    --C                                               Tensor storing the C operand
  [scalar]    --alpha,--epilogue::alpha                         Epilogue scalar alpha
  [scalar]    --beta,--epilogue::beta                           Epilogue scalar beta
  [enum]      --split_k_mode,--split-k-mode                     Variant of split K mode(serial, parallel)
  [int]       --split_k_slices,--split-k-slices                 Number of partitions of K dimension
  [int]       --batch_count,--batch-count                       Number of GEMMs computed in one batch
  [enum]      --op_class,--opcode-class                         Class of math instruction (simt, tensorop, wmmatensorop, wmma).
  [enum]      --accum,--accumulator-type                        Math instruction accumulator data type
  [int]       --cta_m,--threadblock-shape::m                    Threadblock shape in the M dimension
  [int]       --cta_n,--threadblock-shape::n                    Threadblock shape in the N dimension
  [int]       --cta_k,--threadblock-shape::k                    Threadblock shape in the K dimension
  [int]       --cluster_m,--cluster-shape::m                    Cluster shape in the M dimension
  [int]       --cluster_n,--cluster-shape::n                    Cluster shape in the N dimension
  [int]       --cluster_k,--cluster-shape::k                    Cluster shape in the K dimension
  [int]       --stages,--threadblock-stages                     Number of stages of threadblock-scoped matrix multiply
  [int]       --warps_m,--warp-count::m                         Number of warps within threadblock along the M dimension
  [int]       --warps_n,--warp-count::n                         Number of warps within threadblock along the N dimension
  [int]       --warps_k,--warp-count::k                         Number of warps within threadblock along the K dimension
  [int]       --inst_m,--instruction-shape::m                   Math instruction shape in the M dimension
  [int]       --inst_n,--instruction-shape::n                   Math instruction shape in the N dimension
  [int]       --inst_k,--instruction-shape::k                   Math instruction shape in the K dimension
  [int]       --min_cc,--minimum-compute-capability             Minimum device compute capability
  [int]       --max_cc,--maximum-compute-capability             Maximum device compute capability

Examples:

Profile a particular problem size:
  $ cutlass_profiler --operation=Gemm --m=1024 --n=1024 --k=128

Schmoo over problem size and beta:
  $ cutlass_profiler --operation=Gemm --m=1024:4096:256 --n=1024:4096:256 --k=128:8192:128 --beta=0,1,2.5

Schmoo over accumulator types:
  $ cutlass_profiler --operation=Gemm --accumulator-type=f16,f32

Run when A is f16 with column-major and B is any datatype with row-major (For column major, use column, col, or n. For row major use, row or t):
  $ cutlass_profiler --operation=Gemm --A=f16:column --B=*:row

Using various input value distribution:
  $ cutlass_profiler --operation=Gemm --dist=uniform,min:0,max:3
  $ cutlass_profiler --operation=Gemm --dist=gaussian,mean:0,stddev:3
  $ cutlass_profiler --operation=Gemm --dist=sequential,start:0,delta:1

Run a kernel with cta tile size of 256x128x32 and save workspace if results are incorrect (note that --cta-tile::k=32 is default cta-tile size):
 $ cutlass_profiler --operation=Gemm --cta_m=256 --cta_n=128  --cta_k=32 --save-workspace=incorrect

Test your changes to gemm kernels with a quick functional test and save results in functional-test.csv:
 $ cutlass_profiler  --operation=Gemm \
   --m=8,56,120,136,256,264,512,520,1024,1032,4096,8192,16384 \
   --n=8,56,120,136,256,264,512,520,1024,1032,4096,8192,16384 \
   --k=8,16,32,64,128,256,288,384,504,512,520 \
   --beta=0,1,2 --profiling-iterations=1 \
   --providers=cutlass --output=functional-test.csv
```

## Example CUDA Core GEMM Operation

Example command line for profiling SGEMM kernels is as follows:
```bash
$ ./tools/profiler/cutlass_profiler --kernels=sgemm --m=3456 --n=4096 --k=4096

=============================
  Problem ID: 1

        Provider: CUTLASS
   OperationKind: gemm
       Operation: cutlass_simt_sgemm_128x128_8x2_nn_align1

          Status: Success
    Verification: ON
     Disposition: Passed

          cuBLAS: Passed

       Arguments: --m=3456 --n=4096 --k=4096 --A=f32:column --B=f32:column --C=f32:column --alpha=1 --beta=0 --split_k_slices=1  \
                  --batch_count=1 --op_class=simt --accum=f32 --cta_m=128 --cta_n=128 --cta_k=8 --stages=2 --warps_m=4  \
                  --warps_n=2 --warps_k=1 --inst_m=1 --inst_n=1 --inst_k=1 --min_cc=50 --max_cc=1024

           Bytes: 180355072  bytes
           FLOPs:<PHONE_NUMBER>44  flops

         Runtime: 6.73655  ms
          Memory: 24.934 GiB/s

            Math: 17218.4 GFLOP/s
```

Note, the arguments which appear in the output may be used as command line parameters for subsequent invocations.

## Example Tensor Core GEMM Operations

To execute kernels targeting Tensor Core operations, supply the flag `--op_class=tensorop` in the command line.
```bash
$ ./tools/profiler/cutlass_profiler --op_class=tensorop --m=3456 --n=4096 --k=8192

=============================
  Problem ID: 1

        Provider: CUTLASS
   OperationKind: gemm
       Operation: cutlass_tensorop_s16816gemm_f16_256x128_32x3_nn_align8

          Status: Success
    Verification: ON
     Disposition: Passed

          cuBLAS: Passed

       Arguments: --m=3456 --n=4096 --k=8192 --A=f16:column --B=f16:column --C=f32:column --alpha=1 --beta=0 --split_k_slices=1  \
                  --batch_count=1 --op_class=tensorop --accum=f32 --cta_m=256 --cta_n=128 --cta_k=32 --stages=3 --warps_m=4  \
                  --warps_n=2 --warps_k=1 --inst_m=16 --inst_n=8 --inst_k=16 --min_cc=80 --max_cc=1024

           Bytes: 180355072  bytes
           FLOPs:<PHONE_NUMBER>36  flops

         Runtime: 0.98647  ms
          Memory: 170.272 GiB/s

            Math: 235138 GFLOP/s
```

## Covering the problem space

All arguments may have single values or comma-delimited set of values. Integers may also be specified
as an inclusive range with the following syntax `start:end:increment` or simply `start:end`.

For example, the following sweeps over the range of the GEMM K dimension from 8 to 4096 in increments
of 8 elements.

```bash
$ ./tools/profiler/cutlass_profiler --kernels=cutlass_simt_sgemm_128x128_nn --m=4352 --n=4096 --k=8:4096:8
```

## Output

By default, runtime and computed GFLOP/s are reported for each operation and problem size. Additionally,
a table of comma separated values are reported at the end of the execution. This may be output to a file
with the `--output=<filename.csv>` command line option as shown:

```bash
$ ./tools/profiler/cutlass_profiler --kernels=cutlass_simt_sgemm_128x128_nn            \
                                    --m=3456 --n=4096 --k=8:4096:8 --output=report.csv
```

To faclitate generation of pivot tables and charts, additional columns may be prepended with the
`--tags=<column>:<value>` option. One or more tags may be specified using a comma-delimited list.

```bash
$ ./tools/profiler/cutlass_profiler --kernels=cutlass_simt_sgemm_128x128_nn            \
                                    --m=3456 --n=4096 --k=8:4096:8 --output=report.csv \
                                    --tags=cutlass:2.2,date:2020-06-08
```

## CUTLASS 3.0 GEMM procedural names

CUTLASS 3.0 introduces a new naming convention for GEMMs used by the profiler targeting the NVIDIA
Hopper architecture and beyond so as to indicate new features of the kernel within the name
(e.g., the cluster shape).

To best illustrate this naming convention, we will walk through the meaning of each of the components
in a GEMM kernel used by the profiler:

```
cutlass3x_sm90_tensorop_s64x128x16gemm_f16_f16_f32_f16_f32_128x128x64_2x1x1_0_ntn_align8
```

The components within this name are as follows:

* `cutlass3x`: indicates that the kernel was generated through the CUTLASS 3.0 API
* `sm90`: indicates that the kernel targets NVIDIA GPUs with compute capability 90
* `tensorop`: indicates that the kernel makes use of NVIDIA Tensor Cores
(as opposed to `simt`, which indicates the use of "CUDA cores")
* `s`: indicates that the Tensor Core instruction being used accumulates in single precision
(as opposed to `h`, which indicates half precision)
* `64x128x16gemm`: indicates that the shape of the Tensor Core instruction being used (MxNxK) is 64x128x16
* `f16_f16_f32_f16_f16`: indicates that the data types for operands A, B, Accumulator, C and D (in that order).
* `128x128x64`: indicates that the thread block shape used in the GEMM (MxNxK) is 128x128x64
* `2x1x1`: indicates that the cluster shape being used is 2x1x1
* `0`: indicates that the kernel uses the CollectiveBuilder's automatic stage calculation to determine the
number of pipeline stages in the kernel. Note that `0` does not mean that no stages are used. A nonzero value indicates that automatic stage calculation is not performed and indicates the number of pipeline stages to be used.
This 0 is only added to the kernel's procedural name, the profiler will still report the actual stage count
when printing the kernel argument details (`--stages=N`) and kernel discovery will still support filtering through the `--stages` argument.
* `ntn`: indicates that the layouts for operands A, B, and C are column major ("n"; non-transposed),
row major ("t"; transposed), and column major, respectively.
* `align8`: indicates that the maximum alignment between operands A and B is 8.

Note that in some special cases where the input A/B types do not match that of the MMA
instruction's, the MMA facing input type is added to the instruction string as well.

```
cutlass3x_sm90_tensorop_s64x128x8tf32gemm_f32_f32_f32_f32_f32_128x128x32_2x1x1_0_tnn_align4
```

* `s64x128x8tf32gemm`: indicates that the MMA consumes inputs in `tf32` format, and therefore
the kernel performs rounding of the `f32` values in global memory while loading them into shared memory.

For custom mainloop or epilogue schedules, details of the opted-in schedule are appended to the end of the
kernel name. For example,

```
cutlass3x_sm90_tensorop_h64x128x16gemm_f16_f16_f16_void_f16_128x128x64_1x1x1_0_nnn_align8_warpspecialized_cooperative_epi_tma
```

* `warpspecialized_cooperative`: Mainloop employs a persistent warp-specialized mainloop and kernel schedule.
* `epi_tma`: Kernel epilogue employs TMA based vectorization.
* `f16_f16_f16_void_f16`: In this case, C type is set to `void`, indicating that residual matrix support
is disabled.

# Convolution

The CUTLASS Profiler is capable of executing 2-D and 3-D convolution problems for forwards and backwards
operator variants.

The CUTLASS Profiler can be built with cuDNN enabled to use as a reference implementation. If CMake detects
the cuDNN library available in the system, it is included as a dependency. This may be explicitly overridden
with CMake flag `CUTLASS_ENABLE_CUDNN`.

```bash
$ cmake .. -DCUTLASS_LIBRARY_OPERATIONS=conv2d -DCUTLASS_ENABLE_CUDNN=OFF
...
$ make -j16 cutlass_profiler
```

## Convolution Arguments

```bash
$ ./tools/profiler/cutlass_profiler --help --operation=Conv2d

Conv2d

  [enum]      --conv_kind                                       Convolutional operator (fprop, dgrad, wgrad)
  [int]       --n,--input_n                                     Input N dimension of the Conv2d problem space
  [int]       --h,--input_h                                     Input H dimension of the Conv2d problem space
  [int]       --w,--input_w                                     Input W dimension of the Conv2d problem space
  [int]       --c,--input_c                                     Input C dimension of the Conv2d problem space
  [int]       --k,--filter_k                                    Filter K dimension of the Conv2d problem space
  [int]       --r,--filter_r                                    Filter R dimension of the Conv2d problem space
  [int]       --s,--filter_s                                    Filter S dimension of the Conv2d problem space
  [int]       --p,--output_p                                    Output P dimension of the Conv2d problem space
  [int]       --q,--output_q                                    Output Q dimension of the Conv2d problem space
  [int]       --g,--groups                                      Number of convolution groups
  [int]       --pad_h                                           Padding in H direction
  [int]       --pad_w                                           Padding in W direction
  [int]       --stride_h                                        Stride in H direction
  [int]       --stride_w                                        Stride in W direction
  [int]       --dilation_h                                      Dilation in H direction
  [int]       --dilation_w                                      Dilation in W direction
  [tensor]    --Activation                                      Tensor storing the Activation operand
  [tensor]    --Filter                                          Tensor storing the Filter operand
  [tensor]    --Output                                          Tensor storing the Output operand
  [enum]      --conv_mode                                       Convolution filter mode (conv, cross)
  [enum]      --iterator_algorithm,--iterator_algo              Convolution iterator algorithm (analytic, optimized)
  [scalar]    --alpha,--epilogue::alpha                         Epilogue scalar alpha
  [scalar]    --beta,--epilogue::beta                           Epilogue scalar beta
  [enum]      --split_k_mode,--split-k-mode                     SplitK mode for serial or parallel reduction (serial, parallel)
  [int]       --split_k_slices,--split-k-slices                 Number of partitions of K dimension
  [enum]      --eq_gemm_provider,--eq-gemm-provider             Enable profiling equivalent gemm by the following providers (cutlass)
  [enum]      --op_class,--opcode-class                         Class of math instruction (simt, tensorop, wmmatensorop, wmma)
  [enum]      --accum,--accumulator-type                        Math instruction accumulator data type
  [int]       --cta_m,--threadblock-shape::m                    Threadblock shape in the M dimension
  [int]       --cta_n,--threadblock-shape::n                    Threadblock shape in the N dimension
  [int]       --cta_k,--threadblock-shape::k                    Threadblock shape in the K dimension
  [int]       --cluster_m,--cluster-shape::m                    Cluster shape in the M dimension
  [int]       --cluster_n,--cluster-shape::n                    Cluster shape in the N dimension
  [int]       --cluster_k,--cluster-shape::k                    Cluster shape in the K dimension
  [int]       --stages,--threadblock-stages                     Number of stages of threadblock-scoped matrix multiply
  [int]       --warps_m,--warp-count::m                         Number of warps within threadblock along the M dimension
  [int]       --warps_n,--warp-count::n                         Number of warps within threadblock along the N dimension
  [int]       --warps_k,--warp-count::k                         Number of warps within threadblock along the K dimension
  [int]       --inst_m,--instruction-shape::m                   Math instruction shape in the M dimension
  [int]       --inst_n,--instruction-shape::n                   Math instruction shape in the N dimension
  [int]       --inst_k,--instruction-shape::k                   Math instruction shape in the K dimension
  [int]       --min_cc,--minimum-compute-capability             Minimum device compute capability
  [int]       --max_cc,--maximum-compute-capability             Maximum device compute capability

Examples:

Profile a particular convolution (specify all the convolution parameters):
 $ cutlass_profiler --operation=Conv2d --Activation=f16:nhwc --Filter=f16:nhwc --Output=f16 --accumulator-type=f32 --n=32 --h=14 --w=14 --c=8 --k=64 --r=3 --s=3 --pad_h=1 --pad_w=1 --stride_h=1 --stride_w=1 --dilation_h=1 --dilation_w=1

```

## Example CUDA Core Convolution Operation

Example command line for profiling forward propagation convolution kernels on CUDA cores is as follows:
```bash
$ ./tools/profiler/cutlass_profiler --kernels=simt_sfprop  --verification-providers=device --n=8 --h=224 --w=224 --c=128 --k=128 --r=3 --s=3

=============================
  Problem ID: 1

        Provider: CUTLASS
   OperationKind: conv2d
       Operation: cutlass_simt_sfprop_optimized_128x128_8x2_nhwc

          Status: Success
    Verification: ON
     Disposition: Passed

reference_device: Passed

       Arguments: --conv_kind=fprop --n=8 --h=224 --w=224 --c=128 --k=128 --r=3 --s=3 --p=224 --q=224 --pad_h=1 --pad_w=1  \
                  --stride_h=1 --stride_w=1 --dilation_h=1 --dilation_w=1 --Activation=f32:nhwc --Filter=f32:nhwc --Output=f32:nhwc  \
                  --conv_mode=cross --iterator_algorithm=optimized --alpha=1 --beta=0 --split_k_mode=serial --split_k_slices=1  \
                  --eq_gemm_provider=none --op_class=simt --accum=f32 --cta_m=128 --cta_n=128 --cta_k=8 --stages=2 --warps_m=4  \
                  --warps_n=2 --warps_k=1 --inst_m=1 --inst_n=1 --inst_k=1 --min_cc=50 --max_cc=1024

           Bytes:<PHONE_NUMBER>  bytes
           FLOPs:<PHONE_NUMBER>44  flops

         Runtime: 8.13237  ms
          Memory: 235.431 GiB/s

            Math: 14569.3 GFLOP/s

```

## Example Tensor Core Convolution Operation

Example command line for profiling forward propagation convolution kernels runing on Tensor Cores is as follows:
```bash
$ ./tools/profiler/cutlass_profiler --kernels=tensorop*fprop  --verification-providers=device --n=8 --h=224 --w=224 --c=128 --k=128 --r=3 --s=3

=============================
  Problem ID: 1

        Provider: CUTLASS
   OperationKind: conv2d
       Operation: cutlass_tensorop_s16816fprop_optimized_f16_128x128_64x4_nhwc

          Status: Success
    Verification: ON
     Disposition: Passed

reference_device: Passed

       Arguments: --conv_kind=fprop --n=8 --h=224 --w=224 --c=128 --k=128 --r=3 --s=3 --p=224 --q=224 --pad_h=1 --pad_w=1  \
                  --stride_h=1 --stride_w=1 --dilation_h=1 --dilation_w=1 --Activation=f16:nhwc --Filter=f16:nhwc --Output=f32:nhwc  \
                  --conv_mode=cross --iterator_algorithm=optimized --alpha=1 --beta=0 --split_k_mode=serial --split_k_slices=1  \
                  --eq_gemm_provider=none --op_class=tensorop --accum=f32 --cta_m=128 --cta_n=128 --cta_k=64 --stages=4  \
                  --warps_m=2 --warps_n=2 --warps_k=1 --inst_m=16 --inst_n=8 --inst_k=16 --min_cc=80 --max_cc=1024

           Bytes:<PHONE_NUMBER>  bytes
           FLOPs:<PHONE_NUMBER>44  flops

         Runtime: 0.945071  ms
          Memory: 1114.21 GiB/s

            Math: 125369 GFLOP/s

```

# Copyright

Copyright (c) 2017 - 2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: BSD-3-Clause

```
  Redistribution and use in source and binary forms, with or without
  modification, are permitted provided that the following conditions are met:

  1. Redistributions of source code must retain the above copyright notice, this
  list of conditions and the following disclaimer.

  2. Redistributions in binary form must reproduce the above copyright notice,
  this list of conditions and the following disclaimer in the documentation
  and/or other materials provided with the distribution.

  3. Neither the name of the copyright holder nor the names of its
  contributors may be used to endorse or promote products derived from
  this software without specific prior written permission.

repository: NVIDIA/cutlass

branch_name: refs/heads/main

extension: md

gha_license_id: NOASSERTION

license_type: permissive

repo_name: NVIDIA/cutlass


text:
## Number_2 - Indexing, Slicing And Reshaping Numpy Arrays:
* From List To Array:
  * One Dimensional List To Array
  * Two Dimensional List Of Lists To Array
* Array Indexing:
  * One Dimensional Indexing
  * Two Dimensional Indexing
* Array Slicing:
  * One Dimensional Slicing
  * Split Input And Output Features
  * Split Train And Test Rows
* Array Reshaping:
  * Data Shape
  * Reshaping 1D To 2D Array
  * Reshaping 2D To 3D Array

license: MIT

path: /Content details/Number_02.md

branch_name: refs/heads/master


text:
---
layout: post
title: "Quotes"
excerpt: "Some nice quotes."
description: "Some nice quotes."
categories: [Blog, Life]
tags: []
author: kerner1000
comments: true
share: true
lang: en
---

> Life is 10 percent what happens to me and 90 percent of how I react to it. - Charles Swindoll

> Too many of us are not living our dreams because we are living our fears. - Les Brown

> The desire for more positive experience is itself a negative experience. And, paradoxically, the acceptance of one’s negative experience is itself a positive experience. - Mark Manson

> Do or do not. There is no try. - Yoda

> A reliable way to make people believe in falsehoods is frequent repetition because familiarity is not easily distinguished from truth. Authoritarian institutions and marketers have always known this fact. - Daniel Kahneman

> Zwei Dinge sind zu unserer Arbeit nötig. Unermüdliche Ausdauer und die Bereitschaft, etwas, in das man viel Zeit und Arbeit gesteckt hat, wieder wegzuwerfen. - Albert Einstein

[comment]: <> > If you look at what you have in life, you’ll always have more. If you look at what you don’t have in life, you’ll never have enough. - Oprah Winfrey


text:
# Unittests for the VSCP helper< library

Unittests for the vscp helper library code.

Use

```
mkdir build
cd build
cmake .. -G "Visual Studio 16 2019" -DVCPKG_TARGET_TRIPLET=x64-windows  -DCMAKE_TOOLCHAIN_FILE=C:\Users\Administrator\Desktop\Development\vcpkg\scripts\buildsystems\vcpkg.cmake
```

in each test folder to create a makefile. Then do

```
cmake --build . --config Release
```

or

```
msbuild libvscphelper.sln /p:Configuration=Release
```

to build the unittest.   Run the test with

```
./unittest
```

repository: grodansparadis/vscp-helper-lib

gha_language: C

gha_license_id: MIT

repo_name: grodansparadis/vscp-helper-lib


text:
# Social Event Content Handbook

## Overview

The Social Event Content Lead is responsible for arranging and recruiting activities and/or displays during the Social Event for the Summit.  The content should celebrate contributing to Kubernetes and/or help contributors meet and enjoy time together. These may include slide shows, presentations, guided socialization exercises, or even music or art.

## Skills and Qualifications

* Can attend the Contributor Summit in person
* Able to keep track of event organization
* Can come up with ideas for fun activities
* Knows enough of the Kubernetes community to reach out to people for participation
* Can collaborate with CNCF staff and other Summit volunteers

## Activities

Many of the activities associated with planning the Social are easily shared among multiple volunteers.  As such, most of the activities below can be shared.

Planning and execution, together with CNCF Staff:

* Planning the Contributor Social, including venue, food, drink, attendee lists, transportation (if necessary), and activities in coordination
* Recruiting and supervising volunteers to handle screening attendees at the door
* Developing and executing social activities to take place during the social, including obtaining any prizes or materials required

Other activities:

* Working with Comms to publicize the Social
* Working with Registration to make sure that registration for the social is clear and intentional

## Time Commitment

* 12-40 hours over 3 months before the Contributor Summit
* 4-5 hours at the Summit

The preparation time depends heavily on the chosen social activities.

## Shadows and Volunteers

### Shadow

The Social reasonably supports one Shadow or the role can be shared between two co-Leads.  There really isn't enough work for more than one Shadow to be useful.  The Shadow will work on the same tasks below, just based on having them individually assigned by the Lead.

Volunteers are needed for Door Warden (see below) shifts, usually 2-3 of them.

Volunteers can optionally contribute to the following tasks:

* Designing and detailing the Social Activitie(s)
* Promoting the Social
* Directing contributors to the Social venue
* Quiz master if doing a quiz
* Stage manager if doing Karaoke or similar
* Other onside social activity support
* New Contributor Welcome, particularly if doing Bingo

While it's entirely possible for volunteers to work on Social prep while being unable to attend in person, it seems cruel to them.

## Instructions and Tips for Specific Tasks

The tasks below are in more-or-less the order in which they occur.  However, event planning can be flexible, and several tasks can be taken care of more or less when you have time.

### Venue and Menu

Two to three months before the conference, the CNCF staff will start work with you to pick a venue for the social.  They will need to know how many people we're expecting (usually 50-100), and any preferences or ideas we have for a type of place.  They will then locate 2-3 options for a venue, which will present a set of tradeoffs around location, space, and food.  If there are local contributors, ask them for recommendations which the event staff can check out as well.

In general, any venue that's within 1km of the conference is walkable (weather permitting).  Venues that are further away mean arranging some type of transportation.  In some places, public transit will work, but in others the CNCF will need to arrange a bus. Further away venues, all else being equal, mean lower attendance.  Venues also need to be wheelchair-accessible.

You'll also check available menus. We'll need to be able to accommodate popular special diets (vegetarian, vegan, gluten-free), so check and make sure there are options for everyone.  Beer and wine is generally expected, but also check what's available for non-alcoholic drinks, making sure that there's something more interesting than bar soda. All of these checks may involve event staff checking with the venue multiple times, so start well before any deadlines.

Depending on schedule and budget, some Socials will have a full meal available, and some only snacks.  Either way, buffet-style and passed bites both work, but anything involving table service does not.

Venues may also have limits on bringing family members, particularly minors, so verify that information.

### Registration

Work with the Registration lead to make sure that questions about the Social are included in registration.  At a minimum, these will be:

* Do you plan to attend the contributor social?
* Do you have a special diet?

In rare cases, if you have information early enough about planned activities, you might include one or more questions about those.  If not, those will happen in a follow-up. Check the registration reports to see what the rolling total for the Social is.

Closer to Kubecon, you will open up requests for contributors to bring a "plus-one" with them.  We've done this just using emails to the summit team, and sometimes we have an online form (using GDocs).  Either works.  Expect to handle around 6-12 plus one requests.  Note that plus-ones are supposed to be for family members traveling with the contributor and maybe mentees.  They are not for contributor co-workers.

Registration for the Social (as well as for plus ones) closes 2-4 days before the Social itself. There's usually a little leeway for late contributors, but it depends a lot on the guarantee with the venue.

### Promotion

You are responsible for making sure, by working with Comms, contributors know that:

* The Contributor Social is happening
* Where and when it is
* It is only for registered KCS attendees

If you have exciting social activities (see below), you'll also want to promote those to build excitement and make sure that folks are ready to join in.  At a minimum, you'll send out announcements at the following times:

1. When the venue is chosen
2. 1 month before the social
3. When "plus one" registration opens
3. 2 days before social registration closes
4. Day of the social, if transportation is involved

When the venue is selected, you will also want to create a page for the contributor site under the current Summit that covers the details of the event.  You may need to "TBD" some information like transport and social activities and fill them in later.

If you have other news to impart (like about social activities), you'll want make more announcements via contributor social media.

The least fun, but very necessary, message you need to get across is that *people need to be eligible and registered for the KCS to attend the social*.  The Social is not an open conference party, and while we can accommodate a small number of unregistered Kubernetes contributors, having a large number of them will cause problems with the venue and catering.

As you get additional information about the Social, you will update the web page for it several times.

### Social Activities

Any party that *only* has drinking and talking is a boring party.  It's also fairly exclusive of new contributors, who don't know who to talk to. Therefore a key part of the Social job is developing and executing social activities of some kind.  The ideal social activity, as well as being fun, also relates to Kubernetes in some way, and encourages meeting new people.  Social activities may or may not have prizes.

What follows are a list of social activities from prior Contributor Socials.  You don't necessarily want to repeat these exactly, but they should serve as inspiration for future events.

**Quiz**: a live pub quiz at the Social, with prizes for the winning teams.  The quiz should have 14-20 questions, with a mix of harder technical questions and sillier community questions. Requires mic, speakers, and projection screen at the venue, prizes, as well as reasonable tables for teams to sit at and participate. Remember to print out blank answer sheets before the Social.  Highly dependent on having a really good quizmaster on stage (like Lachie); recruit that person first, before deciding on this activity.  Also, unless you have a separate space for the quiz, it will take over the Social during the hour or so it's running.

**Bingo Card**: Used in the past as an icebreaker for new contributors.  5x5 bingo card with questions for them to ask other contributors. Newbies who bingo get a prize.  Requires bingo cards, pens, 6+ prizes.

**Venue Games**: Some venues have games built-in that can be added to the contract.  This can include pool, bowling, darts, video games, axe throwing, or whatever else the venue has.  Desirable because it requires no setup on our part, but often expensive.  Expect around 25% of attendees to participate if there's a per-person contract.

**Puzzles**: For Kubecon Amsterdam we ordered custom jigsaw puzzles with various Kubernetes-related graphics on them, including the infamous CNCF Landscape Puzzle.  These were then put on different tables at the venue.  Requires: design and order custom jigsaw puzzles.

**Board Games**: Make board games available on tables at the venue for groups to play.  A few games will be bought by CNCF Events, and attendees are encouraged (with multiple reminders) to bring board games of their own. Encourage each board game group to recruit 1 new contributor to play with them.  Requires: board games, decent-sized tables at the venue, multiple reminders to attendees, not too loud venue, where ideally folks can stay on after the official end of the event.

**Karaoke**: Attendees can sing or lip-sync karaoke on stage at the venue.  The venue would need to be set up for karaoke, or we would need to bring a portable karaoke machine. If there are multiple rooms this can be a side activity, but on a big stage it dominates the Social and would need to be time-limited.  Before committing to this, recruit and verify at least a couple of folks who love karaoke and will put themselves on the schedule.  Requires: karaoke setup at venue.

**Puppies**: In San Diego, Paris arranged for a local dog training center to bring dogs for contributors to pet, as fun and as a stress relief.

Ideas for new activities that we haven't tried could include:

* mini-golf
* lasertag or similar
* talent show of some kind
* participatory art stations (e.g. make your own linoleum print, etc.)
* cooking contest
* having the contributor awards at the social

### Walk-Through

Plan to arrive at the Kubecon city at least a day before the Contributor Summit, so that you can do a walk-through of the Social venue.  CNCF staff will arrange this at your request. Use the walk through to figure out:

* Where will each social activity be?
* Do any of the activities need to be modified because of the space?
* Do you need to ask for AV/furniture adjustments from the venue?
* The walk/transit from the CS to the Social
* How can you exercise door control?

It's a good idea to do one last checkover of the menu with the venue, particularly that items for special diets haven't been removed or modified at the last minute.  You can also talk over other details, such as music volume levels, with the staff.

### Door Warden

The Contributor Social is open *only* to registered contributors, and their plus-ones if they registered them. This means that someone has to exercise strict door control at the Social, to prevent random walk-ins.  The CNCF staff will occupy the table or check-in stand, but someone from the Summit team needs to be at the door, or immediately available (depending on venue layout), to help the staff with ambiguous situations.  Set up a schedule, with staffing every 1/2 hour for the first 2 hours of the social.

Generally, if an org member contributor who forgot to register shows up, we generally let them in unless the venue is nearing capacity.  This will require you to check the list of org members, so download that in advance (given the vagaries of wifi).  Usually we also allow plus-ones that a registered contributor forgot to ask for, again provided that the venue isn't almost full.  CNCF staff are also allowed in, even if not registered.

Obviously we block random Kubecon attendees who show up; the Contributor Social is not a free party for the conference. Two kinds of people show up who are more difficult, though.  One is maintainers of other CNCF projects, who are not entitled to attend unless they already went through the exception process.  The other are management at major sponsors of the CNCF or Kubecon.  This is where it's important for you, as contributor staff, to be there to back up the CNCF staff for whom saying no to major sponsors is challenging.

## Other Notes and Ideas

* Assume anything you haven't determined about a venue is wrong.
* Venues often have music much too loud for conversation.  You'll need to overdetermine this, both by requesting softer music before the event, and then probably requesting that they turn the music down several times during the event.
* Make *really* sure of what a venue will provide for special diets.  While potato chips are vegan, they are not a meal.
* If you are relying on Venue Games (above), check some of the games when you visit the venue.
* If a social activity is failing to draw any interest, abandon it.

repository: kubernetes/community

gha_language: Jupyter Notebook

repo_name: kubernetes/community


text:
# Overview

This flow assumes that the App Developer implements the User interface necessary for receiving user inputs to and library only provides the payments infrastructure.

To perform transaction using core, follow the steps given below:

1. Set transaction request information to SDK.
2. Setup `TransactionCallback` to catch the transaction results.
3. Call the implemented method of the desired payment mode.

## Differences between core flow and the UI flow.

The main difference between the core and UI flow is that

* Get payment response -
  * In Core Flow - Use TransactionCallback
  * In UI Flow - Use TransactionFinishedCallback

* Payment Selection Method -
  * In Core Flow - Call paymentUsingXXX().
  * In UI Flow - Call startPaymentUiFlow() to see all specified payment methods.

# Usage
this new SDK you don't need to use event bus, like on the previous VeritransSDK. in this version we replace whole the event bus stuff with java callback, to make the SDK usage easier.

Each API usage has its own `Callback` interface to capture finished API access. Please implement one of this callback interface that accessing the SDK.

* `CheckoutCallback`
* `TransactionOptionsCallback`
* `CardTokenCallback`
* `TransactionCallback`
* `SaveCardCallback`
* `GetCardCallback`
* `CardRegistrationCallback`
* `TransactionFinishedCallback`

All of this interface will implement 3 method, `onSuccess`, `onFailure` and `onError`.
For example, `TransactionBusCallback` will have to implement these methods.

```Java
// Need to attach @Subscribe annotation to enable Event Bus registration
//transaction success with response body
@Override
public void onSuccess(TransactionResponse response) {
// actionSuccess(response);
}
//transaction failed with response body and the reason
@Override
public void onFailure(TransactionResponse response, String reason) {
// actionFailure(response, reason);
}

//general error(on network unavailable, on transaction failed, on callback unimplemented etc)
@Override
public void onError(Throwable error) {
// actionError(error);
}
```

## Common request information

These are common parameters which are required in all types of transaction.

### Set customer details

**CustomerDetails** class holds information about customer. To set information about customer in TransactionRequest use following code -

```Java
CustomerDetails customer = new CustomerDetails(String firstName, String lastName, String email, String phone);
// Set customer details on
transactionRequest.setCustomerDetails(customer);
```

### Set Item details

**ItemDetails** class holds information about item purchased by user. TransactionRequest takes an array list of item details. To set this in TransactionRequest use following code -

```Java
ItemDetails itemDetails1 = new CustomerDetailsItemDetails(String id, double price, double quantity, String name);
ItemDetails itemDetails2 = new CustomerDetailsItemDetails(String id, double price, double quantity, String name);

// Create array list and add above item details in it and then set it to transaction request.
ArrayList<ItemDetails> itemDetailsList = new ArrayList<>();
itemDetailsList.add(itemDetails1);
itemDetailsList.add(itemdetails2);

// Set item details into the transaction request.
transactionRequest.setItemDetails(itemDetailsList);
```

### Set Billing Address

**BillingAddress** class holds information about billing. TransactionRequest takes an array list of billing details. To set this in TransactionRequest use following code -

```Java
BillingAddress billingAddress1 = new BillingAddress(String firstName, String lastName, String address, String city, String postalCode, String phone, String countryCode);

BillingAddress billingAddress2 =new BillingAddress(String firstName, String lastName, String address, String city, String postalCode, String phone, String countryCode);

// Create array list and add above billing details in it and then set it to transaction request.
ArrayList<BillingAddress> billingAddressList = new ArrayList<>();
billingAddressList.add(billingAddress1);
billingAddressList.add(billingAddress2);

// Set billing address list into transaction request
transactionRequest.setBillingAddressArrayList(billingAddressList);
```

### Set Shipping Address

**ShippingAddress** class holds information about shipping address. TransactionRequest takes an array list of shipping details. To set this in TransactionRequest use following code -

```Java
ShippingAddress shippingAddress1 = new ShippingAddress(String firstName, String lastName, String address, String city, String postalCode, String phone, String countryCode);

ShippingAddress shippingAddress2 =new ShippingAddress(String firstName, String lastName, String address, String city, String postalCode, String phone, String countryCode);

// Create array list and add above shipping details in it and then set it to transaction request.
ArrayList<BillingAddress> shippingAddressList = new ArrayList<>();
shippingAddressList.add(shippingAddress1);
shippingAddressList.add(shippingAddress1);

// Set shipping address list into transaction request.
transactionRequest.setShippingAddressArrayList(shippingAddressList);
```

### Set Bill Info

**BillInfoModel** class holds information about billing information that will be shown at billing details.

To set this in TransactionRequest use following code -

```Java
BillInfoModel billInfoModel = new BillInfoModel(BILL_INFO_KEY, BILL_INFO_VALUE);
// Set the bill info on transaction details
transactionRequest.setBillInfoModel(billInfoModel);
```

##  Setting Transaction Details

Set required transaction information to sdk instance.

TransactionRequest contains following information

- Order Id - _unique order id to identify this transaction_.
- Amount - _amount to charge_.
- Customer details - _holds an information about customer like name , email and phone_.
- Item details - _ holds an information about purchased item, like item id, price etc_.
- Billing Address - _holds an address information like city , country, zip code etc_.
- Shipping Address - _holds an address information like city , country, zip code etc_.
- Bill information - _contains custom details in key-value that you want to display on bill print.

All of the above parameters are not nesseccessary to set, It depends on which type of payment method you want to execute. To get more information about required paramter see http://docs.veritrans.co.id/en/api/methods.html#Charge .
Sdk will throw error message in case you missed some information right before starting an payment flow.

Create instance of **TransactionRequest** class.

```Java
TransactionRequest transactionRequest = new TransactionRequest(TRANSACTION_ID, TOTAL_AMOUNT);
// Optional Parameters
transactionRequest.setCustomerDetails(CUSTOMER_DETAILS);
transactionRequest.setItemDetails(ITEM_DETAIL_LIST);
transactionRequest.setBillInfoModel(BILL_INFO);
transactionRequest.setBillingAddressArrayList(BILLING_ADDRESS_LIST);
transactionRequest.setShippingAddressArrayList(SHIPPING_ADDRESS_LIST);
```

Then set the `transactionRequest` into `VeritransSDK` instance.

```Java
VeritransSDK.getVeritransSDK().setTransactionRequest(transactionRequest);
```

## Get Checkout Token

After set the transaction details, you must checkout your transaction and get a `authentication_token` by using this function to be able to proceed the payment.

**Note : This is must do process before do the payment charging because the charging needs the checkout token.**

### Setup Event Bus handler for Checkout Token

Setup your event bus on your class. Read [this wiki](https://github.com/veritrans/veritrans-android/wiki/Core-Flow#using-event-bus).

you have to implement `CheckoutCallback` when make a chackout to merchant server. There will be three implemented methods.

```Java
    public void onSuccess(Token token);

    public void onFailure(Token token, String reason);

    public void onError(Throwable error);
```

### Proceed to Checkout a Transaction

Do the checkout using this code.

```Java
MidtransSDK.getInstance().checkout(new CheckoutCallback() {
    @Override
    public void onSuccess(Token token) {
    //acton when Successed
    }

    @Override
    public void onFailure(Token token, String reason) {
    //action whe  failed
    }

    @Override
    public void onError(Throwable error) {
    //action when error
    }
});
```

## Payment Options

There are several payment methods supported by this SDK.

### Get Enabled Transaction Options

To get all enabled payment methods (also full transaction details) you can use `getTransactionOptions` function from `MidtransSDK` instance with a `authentication_token`.

```Java
MidtransSDK.getInstance().getTransactionOptions();
```

**Warning** : You must set up event bus in your calling class.

Implement `TransactionOptionsCallback`.

```Java
    public void onSuccess(Transaction transaction);

    public void onFailure(Transaction transaction, String reason);

    public void onError(Throwable error);
```

### Credit Card

To implement credit card payment read [this wiki](https://github.com/veritrans/veritrans-android/wiki/Credit-Card-Payment-Usage).

There are some optional features for Credit Card Payment.

- [Offers or Promotion](https://github.com/veritrans/veritrans-android/wiki/Offers-for-credit-cards)
- [Register Card](https://github.com/veritrans/veritrans-android/wiki/Register-Credit-Card-Usage)

### Bank Transfer

To implement bank transfer payment read [this wiki](https://github.com/veritrans/veritrans-android/wiki/Bank-Transfer-Payment-Usage).

### KlikBCA

To implement KlikBCA payment read [this wiki](https://github.com/veritrans/veritrans-android/wiki/KlikBCA-Payment-Usage).

### BCA Klikpay

To implement BCA Klikpay payment read [this wiki](https://github.com/veritrans/veritrans-android/wiki/BCA-Klikpay-Payment-Usage).

### Mandiri Clickpay

To implement Mandiri Click payment read [this wiki](https://github.com/veritrans/veritrans-android/wiki/Mandiri-Clickpay-Payment-Usage).

### Mandiri E-Cash

To implement Mandiri E-Cash payment read [this wiki](https://github.com/veritrans/veritrans-android/wiki/Mandiri-E-Cash-Payment-Usage).

### Mandiri Bill

To implement Mandiri Bill payment read [this wiki](https://github.com/veritrans/veritrans-android/wiki/Mandiri-Bill-Payment-Usage).

### Epay BRI

To implement Epay BRI payment read [this wiki](https://github.com/veritrans/veritrans-android/wiki/Epay-BRI-Payment-Usage).

### CIMB Clicks

To implement CIMB Clicks payment read [this wiki](https://github.com/veritrans/veritrans-android/wiki/CIMB-Clicks-Payment-Usage).

### Indosat Dompetku

To implement Indosat Dompetku payment read [this wiki](https://github.com/veritrans/veritrans-android/wiki/Indosat-Dompetku-Payment-Usage).

### Indomaret

To implement Indomaret payment read [this wiki](https://github.com/veritrans/veritrans-android/wiki/Indomaret-Payment-Usage).

repository: veritrans/veritrans-android

gha_language: Java

repo_name: veritrans/veritrans-android


text:
---
title: Pulp 2.12.0 Generally Available
author: Sean Myers
tags:
  - release
  - rpm
  - puppet
  - docker
categories:
  - Releases
---

Pulp 2.12.0 is now Generally Available in the stable repositories:

<https://repos.fedorapeople.org/pulp/pulp/stable/2.12/>

New features for Pulp 2.12 include new features added to the Pulp platform, as well
as the Puppet, RPM, and Docker plugins. Many bug fixes are also included in
the 2.12.0 release.

More information about these new features can be found in the features list below,
as well as in the
[2.12 release notes]( http://docs.pulpproject.org/en/2.12/user-guide/release-notes/2.12.x.html)

Try out the new features, and let us know how they work for you!

Since the release of Release Candidate 2, we've decided to pull the pulp_python
upgrade to version 2.0 from this release, due to usability and documentation
issues that we feel still need to be worked out before that pluginis ready for
release. We'll be including it in a future release. Users currently testing a
pre-release of 2.12.0 (a Beta or Release Candidate) may need to downgrade the
installed versions of the pulp-python packages to 1.1.3-1 as a result.

These are the current stable packages for the pulp_python plugin as of Pulp 2.12.0:

- pulp-python-admin-extensions-1.1.3-1
- pulp-python-plugins-1.1.3-1
- python-pulp-python-common-1.1.3-1

## Upgrading

The Pulp 2 stable repositories are included in the pulp repo files:

- [Fedora 24 & 25](https://repos.fedorapeople.org/repos/pulp/pulp/fedora-pulp.repo)
- [RHEL 7](https://repos.fedorapeople.org/repos/pulp/pulp/rhel-pulp.repo)

After enabling the pulp-stable or pulp-2-stable repository, you'll want to follow the standard
upgrade path with migrations:

```sh
$ sudo systemctl stop httpd pulp_workers pulp_resource_manager pulp_celerybeat pulp_streamer goferd
$ sudo yum upgrade
$ sudo -u apache pulp-manage-db
$ sudo systemctl start httpd pulp_workers pulp_resource_manager pulp_celerybeat pulp_streamer goferd
```

The `pulp_streamer` and `goferd` services should be omitted if those services are not installed.

## New Features

### Pulp

- 2525: Failover events should be explicitly logged as such when they: occur
- 2509: Pulp process failure detection and any failover should occur within 30: seconds
- 2186: As a user, pulp-manage-db refuses to run if other pulp processes are running: locally
- 1939: As a user, I would like to be able to profile Pulp: tasks
- 1268: As a user, orphan content delete reports how many units were: deleted

### RPM Support

- 1976: As user, I can have packages sorted in "Packages/$x" directories when: published

### Docker Support

- 2189: As a user, I can update docker_tag units without using: sync

View this list [in Redmine](http://bit.ly/2iMdYpK).

## Issues Addressed

### Pulp

- 2545: pulp-manage-db wait time calculation can end up with many hours of wait: time
- 2264: better error reporting during: import/association
- 1488: Deprecate: nodes
- 1321: Commit message requirements for Pulp contributions are out of date and could be: better
- 103: Document the commit message keywords that will interact with: Redmine

### RPM Support

- 1823: RPMs partially: downloaded

View this list [in Redmine](http://bit.ly/2iMgzjJ).

## el6 Support Reminder

As mentioned in the 2.11.0 release notes, we are no longer supported el6 distributions
after Pulp 2.11. See the release notes for more information:

<http://docs.pulpproject.org/en/2.11/user-guide/release-notes/2.11.x.html#el6-support-changes>

branch_name: refs/heads/gh-pages

gha_language: CSS


text:
Combo Require
=============

Combo Require is a Rails engine that provides a simplified version AMD/requirejs for the asset pipeline with combo loading.

Usage
-----

### app/views/layouts/application.html.erb

    <%= javascript_include_tag "combo-require" %>
    <script type="text/javascript">
      configure({
        base: "/assets/",
        combine: true,
        comboBase: "/combo?",
        modules: <%= module_map %>
      });
    </script>
    <script type="text/javascript">
      require(["my-module"], function (myModule) {
        myModule(); // writes Hello World! to the page.
      });
    </script>

### app/assets/javascripts/my-module.js

    define("my-module", ["jquery"], function ($) {
      return function () {
        $(document.body).html("Hello World!");
      };
    });

How it works
------------

* The ComboRequire::ModuleMap class reads all the files in the asset pipeline,
looks for calls to `define()`, and creates a dependency tree that can be embedded
in a `<script>` block.
* Simple versions of `require()` and `define()` use the dependency tree to build combo
URLs like `/combo?jquery.js&my-module.js`.
* The ComboRequire::ComboController concatenates each file in the combo URL into a single file.
* `require()` offloads the actual loading of JavaScript files to LABjs.

Caveats
-------

* Combination URLs skip the asset pipeline and are rendered on every request.
* Cannot use Sprocket's MD5 digested URLs.
* Finding the `define` calls in source files is a really simple regular expression
and will probably break.
* Differences from requirejs:
  * Anonymous module definitions are not supported.
  * No plugins (but there's crappy support for handlebars built in right now.)
  * Calls to `define` will not load unloaded modules, because it assumes that the `require`
    took care of them using the dependency tree.

To Do
-----

* Cache control headers on the combo server.
* Configurable CDN combo hosts.
* Figure out versioning.
* Clean up handlebars.js support (it's super hacky and relies on the sht_rails gem right now.)
* Add caching to the ModuleMap class.
* Add JS compression.

License
-------

Copyright (c) 2012 Lenny Burdette

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

 IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

repository: lennyburdette/combo-require

repo_name: lennyburdette/combo-require


text:
# Homebrew ImageMagick with X11 support

This is a macOS [Homebrew](https://brew.sh) [tap](https://docs.brew.sh/Taps) for ImageMagick with X11 support.

It tracks the upstream ImageMagick formula and enables the following:
- X11 support
- Graphviz support

Updated on a [weekly](.github/workflows/weekly-update.yml) basis thanks to [Github Actions](https://github.com/features/actions).

## How do I install this?
```
brew uninstall imagemagick  # without X11 support
brew install --cask xquartz
brew install tlk/imagemagick-x11/imagemagick
```

## Verify that it works
```
display wizard:
display myfile.jpg
```

## Graphviz support
Note that ImageMagick has support for [graphviz DOT-files](https://en.wikipedia.org/wiki/DOT_(graph_description_language)) if the `--with-graphviz` option is used during install:
```
brew uninstall imagemagick
brew install tlk/imagemagick-x11/imagemagick --with-graphviz
display mygraph.dot
convert mygraph.dot mygraph.png
```

## Updated every week
The formula is updated weekly which means that Homebrew will automatically build and install the latest version.

## Background
* https://github.com/Homebrew/homebrew-core/issues/49082
* https://imagemagick.org/discourse-server/viewtopic.php?f=3&t=37386
* https://stackoverflow.com/a/59720020/936466

gha_language: Ruby


text:
+++
title="Random notes"
description="Notes worth taking."
date=2021-08-11
draft = false

[taxonomies]
tags = [ "rust", "notes"]
categories = ["random-notes"]

[extra]
+++

## RAII
In C++, this pattern of deallocating resources at the end of an item's lifetime is sometimes called `Resource Acquisition Is Initialization`. In Rust we call `drop` trait.

Rust Book | What is Ownership


text:
# BigQuery Remote Storage Adapter for Prometheus

[![Build Status](https://github.com/KohlsTechnology/prometheus_bigquery_remote_storage_adapter/actions/workflows/pipeline.yml/badge.svg?branch=master)]((https://github.com/KohlsTechnology/prometheus_bigquery_remote_storage_adapter/actions))
[![Go Report Card](https://goreportcard.com/badge/github.com/KohlsTechnology/prometheus_bigquery_remote_storage_adapter)](https://goreportcard.com/report/github.com/KohlsTechnology/prometheus_bigquery_remote_storage_adapter)
[![Join the chat at https://gitter.im/KohlsTechnology/prometheus_bigquery_remote_storage_adapter](https://badges.gitter.im/KohlsTechnology/prometheus_bigquery_remote_storage_adapter.svg)](https://gitter.im/KohlsTechnology/prometheus_bigquery_remote_storage_adapter?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)

This is a read/write adapter that receives samples via Prometheus's remote write protocol and stores them in Google BigQuery. This adapter is based off code found in the official prometheus repo:

https://github.com/prometheus/prometheus/tree/master/documentation/examples/remote_storage/remote_storage_adapter

Billing MUST be enabled on the GCP project with the destination BigQuery tables. This adapter uses the "streaming inserts" API. More information is available here: https://cloud.google.com/bigquery/streaming-data-into-bigquery#before_you_begin

The table schema for BigQuery can be found in file [bq-schema.json](https://raw.githubusercontent.com/KohlsTechnology/prometheus_bigquery_remote_storage_adapter/master/bq-schema.json). You can create a BigQuery dataset and table using the following commands.
```
BQ_DATASET_NAME=prometheus
BQ_TABLE_NAME=metrics
GCP_PROJECT_ID=my-gcp-project-id
bq --location=US mk --dataset $GCP_PROJECT_ID:$BQ_DATASET_NAME
bq mk --table \
  --schema ./bq-schema.json \
  --time_partitioning_field timestamp \
  --time_partitioning_type DAY $GCP_PROJECT_ID:$BQ_DATASET_NAME.$BQ_TABLE_NAME
```

The `tags` field is a JSON string and can be easily extracted. Here is an example query:

```sql
SELECT metricname, tags, JSON_EXTRACT(tags, '$.some_label')
  AS some_label, value, timestamp
  FROM `your_gcp_project.prometheus.metrics_stream`
  WHERE JSON_EXTRACT(tags, '$.some_label') = "\\"target_label_value\\""
```

Consider enabling partition expiration on the destination table based on your data retention and billing requirements (https://cloud.google.com/bigquery/docs/managing-partitioned-tables#partition-expiration).

## Running directly with googleAPIjsonkeypath

```shell
./bigquery_remote_storage_adapter \
  --googleAPIjsonkeypath=/secret/gcp_service_account.json \
  --googleAPIdatasetID=prometheus \
  --googleAPItableID=metrics_stream
```

## Running directly Google ADC

Reference: Google Application Default Credentials ([ADC](https://cloud.google.com/docs/authentication/production#automatically))

```shell
GOOGLE_APPLICATION_CREDENTIALS=../../private.key.json ./bigquery_remote_storage_adapter \
  --googleProjectID=<GCP Project ID> \
  --googleAPIdatasetID=prometheus \
  --googleAPItableID=metrics_stream
```

To show all flags:

```shell
./bigquery_remote_storage_adapter -h
```

## Deploying To Kubernetes

The recommended installation method is to use the [Prometheus operator](https://github.com/prometheus-operator/prometheus-operator).

Example of deploying the remote read/write adapter using the Prometheus operator:
```yaml
---
apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
  name: prometheus
  labels:
    prometheus: prometheus
spec:
  replicas: 2
  serviceAccountName: prometheus
  serviceMonitorSelector:
    matchLabels:
      team: frontend
  containers:
    - name: "prometheus-storage-bigquery"
      image: "quay.io/kohlstechnology/prometheus_bigquery_remote_storage_adapter:v0.6.0"
      env:
        - name: "PROMBQ_GCP_PROJECT_ID"
          value: "${PROJECT_ID}"
        - name: PROMBQ_DATASET
          value: "${BIGQUERY_DATASET}"
        - name: PROMBQ_TABLE
          value: "${BIGQUERY_TABLE}"
        - name: PROMBQ_TIMEOUT
          value: "2m"
      imagePullPolicy: IfNotPresent
      resources:
        limits:
          cpu: "5"
          memory: "500Mi"
        requests:
          cpu: "5"
          memory: "500Mi"
  remoteWrite:
    - url: http://localhost:9201/write
      remoteTimeout: 2m
      queueConfig:
        capacity: 500
        maxShards: 200
        minShards: 1
        maxSamplesPerSend: 100
        batchSendDeadline: 5s
        minBackoff: 30ms
        maxBackoff: 100ms
  remoteRead:
    - url: http://localhost:9201/read
      remoteTimeout: 1m
```

Here is an [external tutorial](https://cloud.google.com/community/tutorials/writing-prometheus-metrics-bigquery) that walks through setup, installation, and configuration using the Prometheus operator on GKE.

## Configuration

You can configure this storage adapter either through command line options or environment variables. The latter is required if you're using our docker image.

| Command Line Flag | Environment Variable | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `--googleAPIdatasetID` | `PROMBQ_DATASET` | Yes | | Dataset name as shown in GCP |
| `--googleAPItableID` | `PROMBQ_TABLE` | Yes | | Table name as shown in GCP |
| `--googleAPIjsonkeypath` | `PROMBQ_GCP_JSON` | Yes\* | | Path to json keyfile for GCP service account. At least one of `--googleAPIjsonkeypath` or `--googleProjectID` must be specified. |
| `--googleProjectID` | `PROMBQ_GCP_PROJECT_ID` | Yes\* | | The GCP `project_id` to use, overwriting the value from the keyfile if both are used. At least one of `--googleAPIjsonkeypath` or `--googleProjectID` must be specified. |
| `--send-timeout` | `PROMBQ_TIMEOUT` | No | `30s` | The timeout to use when sending samples to the remote storage |
| `--web.listen-address` | `PROMBQ_LISTEN` | No | `:9201` | Address to listen on for web endpoints |
| `--web.telemetry-path` | `PROMBQ_TELEMETRY` | No | `/metrics` | Address to listen on for web endpoints |
| `--log.level` | `PROMBQ_LOG_LEVEL` | No | `info` | Only log messages with the given severity or above. One of: [debug, info, warn, error] |
| `--log.format` | `PROMBQ_LOG_FORMAT` | No | `logfmt` | Output format of log messages. One of: [logfmt, json] |

## Configuring Prometheus

To configure Prometheus to send samples to this binary, add the following to your `prometheus.yml`:

```yaml
# Remote write configuration (for Google BigQuery).
remote_write:
  - url: "http://localhost:9201/write"

# Remote read configuration (for Google BigQuery).
remote_read:
  - url: "http://localhost:9201/read"

```

## Performance Tuning

You will need to tune the storage adapter based on your needs. You have several levers available...

### Requests & Limits

When running on a container platform (like Kubernetes), it's important to configure the CPU / memory requests and limits properly. You should be able to get away with just a couple hundred megabytes of RAM (make sure request == limit), but the CPU needs will heavily depend on your environment. Set the CPU requests to the minimum you need to achieve the required performance. We recommend setting the limit higher (keep in mind that anything above the request is not guaranteed). Keep an eye on CPU throttling to help tweak your settings.

### Limit Metrics Stored Long-Term

The amount of data you send to BigQuery can be another big constraint. It is easy to overwhelm the BigQuery streaming engine by throwing millions of records at it. You might run into API quota issues or simply have data gaps. We highly recommend not to go crazy when it comes to scrape intervals (<30s) and be very selective on what gets stored long-term. Depending on your needs, it might make sense to calculate and store only aggregated metrics long-term.
Refer to the Prometheus documentation for [remote_write](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#remote_write) and [relabel_config](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config) on how to implement this.

### Prometheus Remote Storage (remote_write & queue_config)

Prometheus allows you to tune the write behavior for remote storage. Please refer to their [documentation](https://prometheus.io/docs/practices/remote_write/) for details.
### Example `prometheus.yml`

```yaml
remote_write:
- url: http://localhost:9201/write
  remote_timeout: 2m
  write_relabel_configs:
  - source_labels: [__name__]
    separator: ;
    regex: ALERTS|apiserver_request_.*|kube_namespace_labels
    replacement: $1
    action: keep
  queue_config:
    capacity: 500
    max_shards: 200
    min_shards: 1
    max_samples_per_send: 100
    batch_send_deadline: 5s
    min_backoff: 30ms
    max_backoff: 100ms
remote_read:
- url: http://localhost:9201/read
  remote_timeout: 1m
```

## Building

### Binary

If you just need a local version to test, then the simplest way is to execute:

```shell
make build
```

### Image

In order to build the docker image, simply execute

```shell
make image
```

## Releasing

This project is using [goreleaser](https://goreleaser.com). GitHub release creation is automated using Travis
CI. New releases are automatically created when new tags are pushed to the repo.
```shell
TAG=v0.0.2 make tag
```

How to manually create a release without relying on Travis CI.
```shell
TAG=v0.0.2 make tag
GITHUB_TOKEN=xxx make clean release
```

## Testing

### Running Unit Tests
```
make test-unit
```

### Running E2E Tests
Running the e2e tests requires a real GCP BigQuery instance to connect to.

```shell
make gcloud-auth
make bq-setup
make test-e2e
make bq-cleanup
make clean
```

To override the GCP project used for testing set the `GCP_PROJECT_ID` variable.
```shell
GCP_PROJECT_ID=my-awesome-project make bq-setup
GCP_PROJECT_ID=my-awesome-project make test-e2e
GCP_PROJECT_ID=my-awesome-project make bq-cleanup
```

## Prometheus Metrics Offered

| Metric Name | Metric Type | Short Description |
| --- | --- | --- |
| `storage_bigquery_received_samples_total` | Counter | Total number of received samples. |
| `storage_bigquery_sent_samples_total` | Counter | Total number of processed samples sent to remote storage that share the same description. |
| `storage_bigquery_failed_samples_total` | Counter | Total number of processed samples which failed on send to remote storage that share the same description. |
| `storage_bigquery_sent_batch_duration_seconds` | Histogram | Duration of sample batch send calls to the remote storage that share the same description. |
| `storage_bigquery_write_errors_total` | Counter | Total number of write errors to BigQuery. |
| `storage_bigquery_read_errors_total` | Counter | Total number of read errors from BigQuery |
| `storage_bigquery_write_api_seconds` | Histogram | Duration of the write api processing that share the same description. |
| `storage_bigquery_read_api_seconds` | Histogram | Duration of the read api processing that share the same description. |

gha_language: Go


text:
---
layout: post
title:      "Rails Project: MyBar"
date:       2018-12-12 19:43:09 -0500
permalink:  rails_project_mybar
---

For my rails project, I decided to create an app to manage an index of bar drinks. Users could sign up, add drinks to a master list of drinks, write reviews for them, and even save their favorites to a personal drink list. Users could view the full list of drinks, or filter it to view complex drinks with 4 or more ingredients, or simple drinks with 3 or less. A user could also see the list of ingredients currently used in all drinks, and then select one to view drinks that were made with that ingredient.

I created models for users, drinks, ingredients, and reviews. For the model associations,

 Users would:

* have many reviews
* have many (favorited) drinks through a users_drinks join table

 Drinks would:
* have many reviews
* have many users through the users_drinks join table
* have many ingredients through a drinks_ingredients table

 Ingredients would:
* have many drinks through the drinks_ingredients table

 Reviews would:
* belong to a user
* belong to a drink

I used bcrypt to for user authentication, in addition to omniauth to allow login through facebook. I also added a simple bootstrap layout using this guide on [Github](https://github.com/twbs/bootstrap-sass)

One of the biggest difficulties I had during this project was just generally feeling overwhelmed by dealing with so many files and and having to add many specific features to complete it. Adding validations and scope methods to models, error checking controller methods, working to consistently use partial layouts, forms and helper methods to simplify code while trying to make sure they were not being used redundantly was definitely a challenge.

extension: markdown

gha_language: JavaScript