text:
### [CVE-2020-28967](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-28967)
![](https://img.shields.io/static/v1?label=Product&message=n%2Fa&color=blue)
![](https://img.shields.io/static/v1?label=Version&message=n%2Fa&color=blue)
![](https://img.shields.io/static/v1?label=Vulnerability&message=n%2Fa&color=brighgreen)

### Description

FlashGet v1.9.6 was discovered to contain a buffer overflow in the 'current path directory' function. This vulnerability allows attackers to elevate local process privileges via overwriting the registers.

### POC

#### Reference
- https://www.vulnerability-lab.com/get_content.php?id=2248

#### Github
No PoCs found on GitHub currently.

repository: trickest/cve

repo_name: trickest/cve


text:
[![Documentation Status](https://readthedocs.org/projects/ees-project/badge/?version=latest)](https://ees-project.readthedocs.io/en/latest/?badge=latest)
[![Build Status](https://travis-ci.com/geektype/EATON-EES-2019.svg?branch=master)](https://travis-ci.com/geektype/EATON-EES-2019)

# EES Project

## v.0.01
A communication interface between an Arduino and a host machine over a serial connection.

## Features

 - Establishes communication over  serial interface
 - Able to send and receive data encoded in ASCII format
 - Arduino can handle up to 6 LDRs and can return their reading
 - Each sensor is identified using a unique identifier to send and receive data
 - Program contains database of sensors and can handle incorrect UIDs

 ## Usage
 Import the file containing the models
 ```python
 from models import Arduino, Sensor
 ```

Create an instance of a serial connection to the Arduino
```python
arduino = Arduino()
```
This creates an object with the following attributes:
|attr|  value|
|--|--|
|port|`/dev/ttyACM0` |
|baudrate|`9600`|
|bytesize|`8`|
|timeout |`2`|
|stopbits | `serial.STOPBITS_ONE`|

If any of the attributes need to be changed change the instable variable. Eg. to change the port:
```python
arduino.port = "/dev/ttyACM0"
```

Create and add to the array storing all of the sensors
```python
ldrs = []
```

To initialise a sensor:

`main.py`
```python
ldr = Sensor(1, arduino)
```
`1` is the unique id given to this sensor. This will be used to identify the sensor. This **must** be a unique id.

`arduino` is the `Arduino()` object created earlier. This tells the program where to communicate about the sensor.

Say for example the sensor is attached to pin A5.

`arduino_code.ino`

```c++
int sensors[2] = {A4,A5}; # add A5 to the array

...

switch (serial_string.toInt())
    {
    case 1:
      Serial.print(analogRead(sensors[0]));
    break;
    // Add your own case statement
    case 2:
      Serial.print(analogRead(sensors[1]));
    break;

...
```

To get a reading from a sensor

```python
print(ldr.getReading())
```

To read what is currently in the serial buffer simply do
```python
arudino.readBuffer()
```
Close the connection:
```python
arduino.closeConnection()
```

branch_name: refs/heads/master

extension: md

gha_language: Python

license_type: permissive


text:
# Pipe Transport

Pipe Transport allows communication through a pipe program to a remote shell. An example on Linux would be `ssh`.

## How-To

We have added `"pipeTransport"` as an option within the **launch.json** file. The structure looks as follows:
```json
"pipeTransport": {
    "pipeCwd": "/usr/bin",
    "pipeProgram": "/usr/bin/ssh",
    "pipeArgs": [
        "-pw",
        "<password>",
<EMAIL_ADDRESS>    ],
    "debuggerPath": "/usr/bin/gdb"
},
```
The `pipeArgs` can be any set of arguments necessary to setup and authenticate the pipe connection. In the example, a password is used but you can also use an ssh key.

You may also need to add a `sourceFileMap` to map the path of where the code exists on the remote shell to where it is locally:
```json
"sourceFileMap": {
    // "remote": "local"
    "/home/user/src": "/src/projectA/src"
}
```

## Attach

You can also use the above `pipeTransport` block to attach to a remote process. In the attach case, you will need to specify a `processId`. We have added the ability to query processes from the remote machine. To do this, change `"processId": "${command.pickProcess}"` to `"processId": "${command.pickRemoteProcess}"`. The `pipeTransport` settings will be used to query the processes on the remote machine. Then select the process from the drop down list. As with `launch`, you may need to configure `sourceFileMap`.

## Docker example

The `pipeTransport` can also be used to debug a process in a Docker container. For an attach, **launch.json** will include:

```json
"pipeTransport": {
    "pipeCwd": "${workspaceRoot}",
    "pipeProgram": "docker",
    "pipeArgs": [
        "exec",
        "-i",
        "hello_gdb",
        "sh",
        "-c"
    ],
    "debuggerPath": "/usr/bin/gdb"
},
```
Where `hello_gdb` is the name of your container.

Launching a process is accomplished by starting a container and then using the same `pipeTransport` launch additional processes in the container. See this [**launch.json**](https://github.com/andyneff/hello-world-gdb/blob/master/.vscode/launch.json) for a [full example](https://github.com/andyneff/hello-world-gdb/).

license: MIT

branch_name: refs/heads/Default

gha_language: TypeScript

gha_license_id: NOASSERTION


text:
# Baseline decimal field

![Screenshot](extras/baseline-decimal.jpg)

## Description

A simple replacement for the default *decimal* field, plus a few improvements (see below). You can either download and use this as-is, or you can use it as a starting template when creating your own field plug-in for a *decimal* field.

[![Download now](extras/download-button.png)](https://github.com/surveycto/baseline-decimal/raw/master/baseline-decimal.fieldplugin.zip)

### Features

In addition to supporting the default SurveyCTO features listed below, this field plug-in offers the following expanded functionality:

1. **Support for RTL languages**
    If your *label* or *hint* is in a language that uses a right-to-left alphabet (like Arabic), it will be right-justified.
1. **Support for HTML in field references**
    If you reference another field's value in either the field *label* or field *hint*, and that referenced value contains HTML, the HTML will be correctly rendered.

### Data format

This field plug-in requires the `decimal` field type.

## How to use

### Getting started

**To use this plug-in as-is**, just download the [baseline-decimal.fieldplugin.zip](https://github.com/surveycto/baseline-decimal/raw/master/baseline-decimal.fieldplugin.zip) file from this repo, specify this field plug-in as a custom field appearance in the form design (like in the [test form](https://github.com/surveycto/baseline-decimal/raw/master/extras/test-form/test-form-package.zip)), and attach it to your form. For more details about using field plug-ins, please read the [user documentation](https://docs.surveycto.com/02-designing-forms/03-advanced-topics/06.using-field-plug-ins.html).

**To create your own** field plug-in using this as a template, follow these steps:

1. Fork this repo
1. Make changes to the files in the `source` directory.
    * **Note:** be sure to update the `manifest.json` file as well.
1. Zip the updated contents of the `source` directory.
1. Rename the .zip file to *yourpluginname*.fieldplugin.zip (replace *yourpluginname* with the name you want to use for your plug-in).
1. You may then attach your new .fieldplugin.zip file to your form as normal.

For more information about developing your own field plug-ins, please read the [developer documentation](https://github.com/surveycto/Field-plug-in-resources).

### Default SurveyCTO feature support

| Feature / Property | Support |
| --- | --- |
| Supported field type(s) | `decimal`|
| Default values | Yes |
| Constraint message | Uses default behavior |
| Required message | Uses default behavior |
| Read only | Yes *(shows the current value, if present)* |
| media:image | Yes |
| media:audio | Yes |
| media:video | Yes |
| `show-formatted` appearance | No |

### Parameters

**Note about the keyboard on iOS:**
iOS has a less-consistent way of determining which on-screen keyboard to display than Android. If you plan on using this field plug-in on iOS, you should test your form on the actual iOS device that you plan on using. If the wrong keyboard appears (for example, if there is no minus button available), you may supply an optional parameter to the plug-in to set the keyboard type manually.

| Parameter key | Parameter value |
| --- | --- |
| `inputmode-ios`, `inputmode-android`, `inputmode-web` (all optional) | Possible values: `none`, `text`, `decimal`, `numeric`, `tel`, `search`, `email`, `url`. [Click here for more information about the inputmode attribute.](https://css-tricks.com/everything-you-ever-wanted-to-know-about-inputmode)|

While the `inputmode-ios` parameter is the most important, you can also use the `inputmode-android` and `inputmode-web` parameters to change the keyboard for that field in Android and web forms, respectively.

## More resources

* **Test form**
This form will help you explore the differences between this field plug-in and the default text field.
[Download test form package](https://github.com/surveycto/baseline-decimal/raw/master/extras/test-form/test-form-package.zip)
[Instructions for test form](/extras/test-form/README.md)

* **Developer documentation**
Instructions and resources for developing your own field plug-ins.
[https://github.com/surveycto/Field-plug-in-resources](https://github.com/surveycto/Field-plug-in-resources)

* **User documentation**
How to get started using field plug-ins in your SurveyCTO form.
[https://docs.surveycto.com/02-designing-forms/03-advanced-topics/06.using-field-plug-ins.html](https://docs.surveycto.com/02-designing-forms/03-advanced-topics/06.using-field-plug-ins.html)


text:
# Raspberry Pi battery monitor

This project monitors the voltage and, indirectly, charge status of a 12 volt
lead-acid battery, e.g. a car starter battery or a marine deep-cycle battery.
The voltage readings are sent to a [Raspberry Pi](https://www.raspberrypi.org/)
where you can plot them, set up alerts, send them over the network or do
anything else you want. This sort of thing has been done many times before, but
you might find my approach interesting.

## The hardware

Because the Raspberry Pi doesn't have an analog-to-digital converter, we need
some kind of external component to take voltage readings.  Many people use
dedicated SPI or I²C ADC chips for this. I elected to use an
[ATtiny13](http://www.mouser.com/ds/2/268/doc8126-1066157.pdf) AVR
microcontroller because it's what I had in my parts bin. Here's a schematic:

![battery monitor circuit schematic](assets/schematic.png)

The load is just whatever you're powering from the battery. In my case I'm
powering the RPi itself from the battery, through a car-style USB power
adapter.  But that's not essential to the operation of the circuit, as long as
all the grounds are tied together.

The positive terminal of the battery is connected to an analog input pin on the
ATtiny13, through two resistors functioning as a [voltage
divider](https://learn.sparkfun.com/tutorials/voltage-dividers).  I measured
these resistors myself which is why they're slightly off of the standard
values. You can use any pair of resistors in approximately the same ratio.
Just be sure to adjust the `R1` and `R2` values in
[`src/main.rs`](src/main.rs). However, if you make the resistors too small, the
system will waste more power.  At these values, the divider will consume about
20 μA, which isn't much to an 85 amp-hour battery!

The maximum analog reading on the ATtiny13 is equal to its supply voltage, 3.3 V
here. So the maximum battery voltage we can measure is

> 3.3 V × (447 kΩ + 118 kΩ) / 118 kΩ
>
> = 15.8 V.

Since the ADC reads 10 bits, the voltage resolution is 15.8 V / 1024 = 15.4 mV.

Here's the full setup:

![Battery power setup with a large battery, USB power to a Raspberry Pi, and
sensor hardware.](assets/setup.jpg)

And a close-up of the sensor circuit:

![A custom home-made circuit on perfboard, with a DIP-8 microcontroller, two
resistors, and some wires held on with hot glue.](assets/closeup.jpg)

Parts list, including powering the Pi:

* Marine deep cycle battery from AutoZone
* [12V fused plug socket with eyelet terminals](https://www.amazon.com/gp/product/B00G8WLW2Y)
* [33W USB car charger](https://www.amazon.com/gp/product/B00OZ5OIFE)
* [USB power meter](https://www.adafruit.com/product/2690)
* [USB A to Micro-B cable](https://www.amazon.com/gp/product/B00NH13O7K)
* [Raspberry Pi 3 Model B](https://www.adafruit.com/product/3055)
* [Raspberry Pi case](https://www.adafruit.com/product/2253) with [lid](https://www.adafruit.com/product/2257)
* 32 GB MicroSD card
* 64 GB USB flash drive (much faster than MicroSD!)
* Basic perfboard
* [ATtiny13a in PDIP-8 package](http://www.mouser.com/search/ProductDetail.aspx?R=0virtualkey0virtualkeyATTINY13A-PU)
* Two resistors
* Miscellaneous wire

## The software

The ATtiny13 sends an ADC sample to the RPi every 100 ms. There is no need for
bidirectional communication, so I used just a single data line. However the
ATtiny13's internal RC oscillator does not provide enough clock stability for
asynchronous serial transmission. I solved this problem using a [Manchester
code](https://en.wikipedia.org/wiki/Manchester_code), which allows the receiver
to recover the sender's bit clock. Each bit transmission takes 1 ms, divided
into two halves for the Manchester code.  So the data rate is 1000 bps.

Each transmission consists of 17 bits: a start bit which is always 1, a 10-bit
ADC sample, and a 6-bit "signature". The signature provides some ability to
detect garbled transmissions, and also ensures that the signal includes the
necessary waveforms for clock recovery. The ADC sample is sent with the least
significant bit first. Between transmissions, the bus is at 0 V, a logic low
state.

The software on the RPi side is in [`src/`](src/). Most of it is written in
[Rust](https://www.rust-lang.org), including the Manchester decoding and the
computation of battery voltage. Low-level access to the Raspberry Pi GPIO pins
is coded in C, based on a [code sample from the eLinux wiki](http://elinux.org/RPi_GPIO_Code_Samples#Direct_register_access).
Rust could do this as well, but the C code contains some gnarly macros and I
didn't want the hassle of porting them to Rust.

For speed, I am using direct access to the memory-mapped GPIO registers,
rather than the Linux `sysfs` interface. The era of [`PEEK` and `POKE`](https://en.wikipedia.org/wiki/PEEK_and_POKE)
lives on!

I've tested this on a Raspberry Pi 3, however it's likely to work on other
models as well.

## Build and install

The ATtiny13 code is very simple and can be found in the [`avr/`](avr/) directory.
To get the tools in Debian:

```
sudo apt-get install gcc-avr avr-libc avrdude
```

Then run `./build-and-upload.sh`. The script is configured to use an [AVRISP
mkII](http://www.atmel.com/tools/avrispmkii.aspx) programmer over USB. If you
have a different programmer, change the `-c` argument to `avrdude`. If you're
using a different AVR microcontroller, e.g. an Arduino Uno, you will need to
edit the script as well as make changes to the code in
[`avr/monitor.c`](avr/monitor.c).

For the RPi side of things, you can install the Rust toolchain on your RPi
[in the usual way](https://rustup.rs/). I've tested this with [Raspbian jessie](https://www.raspberrypi.org/downloads/raspbian/).
You'll also need `gcc`, which is installed by default. The RPi is fairly slow;
a clean build from scratch takes almost 40 seconds. I haven't tried
cross-compiling from a more powerful system. Once you have built dependencies,
a build takes only about 6 seconds.

After you've installed the tools, you can build the code by running

```
cargo build --release
```

in the root directory of this repository. You will get a binary at
`./target/release/rpi-battery-monitor`. Run it with no arguments
to get a battery voltage reading.

This program uses two system features which require it to run as root:

* Direct access to GPIO registers through `/dev/mem`
* Realtime scheduler priority while reading data

## Munin setup

`rpi-battery-monitor` can be used as a [Munin](http://munin-monitoring.org/)
plugin to get pretty graphs. The plugin must run as root, and the
simplest way to accomplish this is by marking it [setuid](https://en.wikipedia.org/wiki/Setuid).
This is **a major security hazard** because, although simple, this program
has not been carefully audited for setuid safety. Thus I recommend you
**only do this on a single-user machine**.

```
# Install the plugin.
sudo cp target/release/rpi-battery-monitor /usr/share/munin/plugins
sudo chown root:root /usr/share/munin/plugins/rpi-battery-monitor
sudo chmod u+s /usr/share/munin/plugins/rpi-battery-monitor
sudo ln -f -s /usr/share/munin/plugins/rpi-battery-monitor /etc/munin/plugins/battery_voltage

# Test run.
sudo munin-run battery_voltage
# Expect output like this: battery_voltage.value 12.197

# Restart Munin so it starts graphing.
sudo service munin-node restart

# Regenerate graphs immediately.
# If you get a lockfile error, try again in a minute.
sudo -u munin /usr/share/munin/munin-html --debug
```

## Configuration

If you want to put the sensor on a different RPi GPIO pin, edit `static const
int pin` in [`src/cbits.c`](src/cbits.c). This uses the Broadcom GPIO numbering,
*not* the RPi GPIO header pin numbering. You can find a [helpful chart here](http://www.raspberrypi-spy.co.uk/2012/06/simple-guide-to-the-rpi-gpio-header-and-pins/).
You will also need to change `BCM2708_PERI_BASE` to `0x20000000` if you're
using a Raspberry Pi 1.

There are many other configuration variables documented in
[`src/main.rs`](src/main.rs). This includes the values for the voltage divider
and the input voltage to the ATtiny13.

Configuration for the Munin graphs is stored in [`src/munin.cfg`](src/munin.cfg).
This file is baked into the binary and therefore you must rebuild after changing
it.

Happy hacking!

repository: dhootha/rpi-battery-monitor

repo_name: dhootha/rpi-battery-monitor


text:
# Electronic Note Taker

  [![license](https://img.shields.io/badge/license-MIT-blue.svg)](https://choosealicense.com/licenses/mit/)

***

  ## Description:
  This application gives the user the ability to take notes electronically on a computer or mobile device.  The notes are saved and can be referenced at any time.

  ## Table of Contents:
  1.  [Description](#description)
  2.  [Demostration](#demonstration)
  3.  [Screenshots](#screenshots)
  4.  [Installation](#installation)
  5.  [Technologies](#technologies)
  6.  [Usage](#usage)
  7.  [License](#license)
  8.  [Contributing](#contributing)
  9.  [Tests](#tests)
  10.  [Questions](#questions)

***
  ## Demonstration:
  [Deployed Application](https://mysterious-chamber-75890.herokuapp.com/)
***
  ## Screenshots
  ### (Application Deployed on Heroku)
  ![Application_Deployed](./public/assets/images/Application_Deployed_on_Heroku.jpeg)

  ### (Entering Notes)
  ![Entering_Notes](./public/assets/images/Adding_Jazz_Artist.jpeg)

  ### (Deleting Notes)
  ![Deleting_Notes](./public/assets/images/Country_Artist_Deleted.jpeg)

***
  ## Installation:
  There are no installation instructions that must be followed for a user to use this application.  The user just needs to click the link under deployed application.  Cloning this application to a local server will require the user to run `npm init` and then `npm express path uuid`.

***
  ## Technologies:
  - HTML
  - CSS
  - JavaScript
  - Node.js
  - Express.js
  - Github
  - Heroku

***
  ## Usage:
  This application is capable of allowing a user to write the title and text of a note.  The note is saved in an array and this makes it persistent.  The user is able to later refer to the notes and even delete them if needed.

***
  ## License:
  This project falls under the MIT License.  The full documentation for this license can be found at [MIT Full Documentation](https://choosealicense.com/licenses/mit).

  Below is an excerpt of the MIT License.

  Copyright <YEAR> <COPYRIGHT HOLDER>

    Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
    The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
     IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

  ## Contributing:
  The contributors for this project are: D. Javan Worthy.

  To contribute to this project, please submit a push request and the author will review and take it into consideration..

***
  ## Tests:
  There are no special tests that need to be run for this application.

***
  ## Questions:
  - The GitHub profile for this project is https://github.com/djavanw/note_taker.

  - If there are any questions, the contributor can be reached at<EMAIL_ADDRESS>

branch_name: refs/heads/main


text:
# Psylo

## Description

### MITRE Description

> [Psylo](https://attack.mitre.org/software/S0078) is a shellcode-based Trojan that has been used by [Scarlet Mimic](https://attack.mitre.org/groups/G0029). It has similar characteristics as [FakeM](https://attack.mitre.org/software/S0076). (Citation: Scarlet Mimic Jan 2016)

## Aliases

```
Psylo
```

## Platforms

```
['Windows']
``

# Techniques

* [Web Protocols](../techniques/Web-Protocols.md)

* [Ingress Tool Transfer](../techniques/Ingress-Tool-Transfer.md)

* [Exfiltration Over C2 Channel](../techniques/Exfiltration-Over-C2-Channel.md)

* [File and Directory Discovery](../techniques/File-and-Directory-Discovery.md)

* [Timestomp](../techniques/Timestomp.md)

# Actors

* [Scarlet Mimic](../actors/Scarlet-Mimic.md)


text:
# uptime-reporter

`uptime-reporter` is a Go package and a set of command-line tools for reporting on website check data from the [Uptime.com](https://uptime.com) monitoring service.

## What does it do?

If you'd like to know how many outages each of your site checks had, and how much downtime each of them had in a given period, for example, `uptime-reporter` can produce that information for you.

It can also produce statistics, such as the average number of outages (both mean and median), average downtime, and so on. If your checks have tags (to classify them by site or business area, for example), `uptime-reporter` can produce statistics for each tag, as well as overall.

## How do I use it?

First, you'll need an Uptime.com account (if you don't have one yet, [sign up for a free trial](https://uptime.com)). Then, you'll need an [API token](https://uptime.com/api/tokens).

Clone this repository if you haven't already. Then run the following command to set your API token, substituting your token for `XXX`:

```sh
export UPTIME_API_TOKEN="XXX"
```

## Downloading data

Now you're ready to start downloading your check data. The downloader will fetch data for the period you specify, so you will need to specify the exact start and end times of this period in the following format: `YYYY-MM-DDTHH:MM:SSZ`.

Run this command, substituting the start and end times of the period you're interested in:

```sh
go run cmd/download/main.go 2019-01-01T00:00:00Z 2020-01-01T00:00:00Z |tee data.csv
```

If all is well, you'll start to see CSV data printed out, one line for each of your checks:

```
My Main Website,https://example.com/,Sites,0,0
...
```

Occasionally you may see a message that the downloader is being rate-limited by the Uptime.com API. That's okay, just wait a few seconds and it will keep trying:

```
rate-limited; sleeping 5s before retry
My Secondary Website,https://subdomain.example.com,Sites,0,0
```

## Resuming interrupted downloads

If you have a large number of checks in your account, and something interrupts your statistics download, don't worry—you don't have to start again from scratch. Just specify the ID of the last check you successfully downloaded, as an extra parameter on the command line:

```
go run cmd/download/main.go 2019-01-01T00:00:00Z 2020-01-01T00:00:00Z 21999 |tee -a data.csv
```

The downloader will skip all the checks up to and including ID 21999, and begin downloading data from the next check onwards.

## Analysing data

Once the downloader has finished running, your data will be stored in the file `data.csv`. To produce statistics on it, run:

```
go run cmd/analyse/main.go <data.csv
```

You should see some output like this:

```
Sector: All (160 sites)
Outages: Total 60.0 Min 0.0 Max 19.0 Median 0.0 Mean 0.4 Standard deviation 1.6
Downtimes: Total 1831856.0 Min 0.0 Max 1160496.0 Median 0.0 Mean 11449.1 Standard deviation 98558.7

Sites with most downtime:
Rank  Name     Outages  Downtime
1     Site A   19       322h21m36s
2     Site B   2        131h30m18s
3     Site C   2        14h4m1s
...
```

## Filtering excessive downtime

The downtime figure for some sites may not be meaningful. For example, if you're analysing 90 days of data and a given site only existed or was only monitored for the last day of that period, it would appear to have 89 days of downtime, which is incorrect.

To eliminate these outliers, you can specify a 'maximum downtime' argument to the analyser:

```
go run cmd/analyse/main.go 100h <data.csv
```

The `100h` here represents a maximum downtime of 100 hours, and sites with more downtime than this will not be included in the statistics or league tables. Try different numbers to find the best one to eliminate the noise from the dataset you're working with.

You can specify the maximum downtime duration as, for example, `5h10m`, or `90s`, or any combination of units supported by [Go's `time` library](https://golang.org/pkg/time/#ParseDuration).

## Using the Go package

If you want to do your own analysis, or download additional data, you can write your own Go programs which use the `uptime-reporter` library. This provides some useful functions such as a rate-limit-aware API client, CSV input and output, data types, and so on.

```go
import reporter "github.com/bitfield/uptime-reporter"
```

repository: bitfield/uptime-reporter

gha_language: Go

gha_license_id: MIT

repo_name: bitfield/uptime-reporter


text:
---
layout: post
title:      "CSS - Shapes"
date:       2019-10-24 19:29:57 +0000
permalink:  css_-_shapes
---

So we are back for some more CSS, and we will be discussing shapes.

## Flexible Ellipses

You may have noticed that any square created in css and with a sufficiently large enough border-radius can turn into a circle. Just like this:

```
background: #fb3;
width: 200px;
height: 200px;
border-radius: 100px; /* >= half the side */
```

On top of that we can also specify any radius larger than 100px and it will still result in a circle. This happens because when the sum of any two adjacent border radii exceeds the size of the border box, user agents must proportionally reduce the used values of all border radii until none of them overlap.

However, we often cannot provide a specific width and height of an element, because of the need to adjust to its content, which may not be known. The question then becomes, can we use border-radius to make an ellipse, let alone a flexible one?

To start off, it is not often known that border-radius accepts different horizontal and vertical radii, if you use a slash(/) to seperate the two. This little bit of syntax allows us to create elliptical rounding at the corners. However, this results in a major flaw. If the dimensions change, the border-radius values need to change as well. When the dimensions vary depending on content, it seems we may have a problem. Fortunately, another lesser known feature of border-radius is that it accepts percentages, not just lengths. This feature means the same percentage can compute to different horizontal and vertical radii. Check out the codepen below for an example.

> Flexible Ellipses -
> //codepen.io/monksandninjas/embed/NWWjXxL?height=265&theme-id=0&default-tab=css,result
> https://codepen.io/monksandninjas/pen/NWWjXxL

## Half Ellipses

Armed with a descent understanding of making an ellipse we can now tackle making half ellipses or half circles. Just like before we can use border-radius, but this time we will seperate out each corner of the square so we can control the shape and then make the code DRY by using shorthand. For example, border-radius value of 10px/ 5px 20px is equivalent to 10px 10px 10px 10px/ 5px 20px 5px 20px.

Before writing code, let's make some observations.

* The shape is symmetrical horizontally => top left and top right radii should be the same. Also bottom left and bottom right radii should also match.
* There are no straight horizontal edges at the top => the top left and top right radii together should total 100% of the shape's width.
* Horizontal left and horizontal right radii should be 50%.
* The vertical part of the border-radius would be 100% 100% 0 0, since the height of the shape is in the top half.
* Lastly bottom vertical rounding is zero.

All together, our possibilities are shown below.

> Half Ellipses -
>  //codepen.io/monksandninjas/embed/OJJmzgw?height=265&theme-id=0&default-tab=css,result
>  https://codepen.io/monksandninjas/pen/OJJmzgw

## Quarter Ellipses
We end this post with making quarter ellipses. What we need to know is that one of the corners needs to have a 100% radius both horizontally and vertially, and the other four will have no rounding.

> Quarter Ellipses -
> //codepen.io/monksandninjas/embed/QWWvaOP?height=265&theme-id=0&default-tab=css,result
> https://codepen.io/monksandninjas/pen/QWWvaOP

That concludes this topic and stay tuned for another installment.

extension: markdown


text:
# Recipe2Vec: Multi-modal Recipe Representation Learning with Graph Neural Networks

```
@inproceedings{recipe2vec_ijcai22,
title = {Recipe2Vec: Multi-modal Recipe Representation Learning with Graph Neural Networks},
author = {Tian, Yijun and Zhang, Chuxu and Guo, Zhichun and Ma, Yihong and Metoyer, Ronald and Chawla, Nitesh V.},
booktitle = {Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence (IJCAI)},
pages = {3473--3479},
year = {2022}
}
```

links
- [ijcai](https://www.ijcai.org/Proceedings/2022/482)
- [arxiv](https://aps.arxiv.org/abs/2205.12396)
- [ code](https://github.com/meettyj/Recipe2Vec)

gha_language: Jupyter Notebook


text:
### Known issues

Can not find experiments in frontend if namespace is not kubeflow
https://github.com/kubeflow/katib/issues/774

If you submit experiment job in notebook directly, it will use same namespace `kubeflow-anonymous` but it won't show in the UI so we recommend you copy yaml files and change `namespace` to `kubeflow` to make visualization available.

repository: sublimotion/eks-kubeflow-workshop

repo_name: sublimotion/eks-kubeflow-workshop


text:
{{{
  "title": "Monitors for Managed Operating Systems",
  "date": "02-12-2015",
  "author": "Ben Swoboda",
  "attachments": [],
  "contentIsHTML": false,
  "sticky": true
}}}

### Overview
This document provides an overview of the monitors that are enabled with Managed Operating Systems and offers some details about the monitoring process.

### Audience

Users employed by companies that have agreed to terms with [Lumen Sales](http://www.centurylink.com/) for the Lumen Cloud product.

### Prerequisites
* An understanding of the process for [making newly created servers managed](../Managed Services/created-a-managed-server-now-what.md) or for [converting pre-existing servers to managed](../Managed Services/converting-unmanaged-virtual-machines-to-managed.md)
* Review of the general platform [Services Guide's](https://www.ctl.io/legal/serviceguide/) Managed Services section, specifically Managed Operating Systems.
* Review of the [Services Guide](http://service-guides.centurylink.com/Default.aspx?ReturnUrl=%2fHandlers%2fDownloadPDF.ashx%3fid%3d{55962F17-2DA3-446D-9ED3-67DC3CF30F6F}&id={55962F17-2DA3-446D-9ED3-67DC3CF30F6F) for Managed Operating Systems

### Details

Monitoring alerts from Managed, CLC servers are enriched with your company's data, including your primary technical point of contact, and presented to the Global Operations team. After reviewing the issue, the Operations team will either resolve the issue or reach out to your primary technical contact.

You are encouraged to set up a primary technical contact by contacting Support as soon as possible. If you have not set up a primary technical contact, the default contact will be an arbitrary contact associated with your account. **The primary technical contact is NOT retrieved from the notification contacts you may have identified in Control Portal.** The notification users in the Control Portal will be recipients of alerts directly from the portal.

If you have specific or detailed contact strategies and/or remediation requirements for your servers, you are encouraged to contact Support. Your requirements will be documented and presented to Operations whenever they review alerts coming from your servers.

If you wish to set up maintenance activity, such as an update, please contact Support. During the time you schedule, alerts will be suppressed and prevent unnecessary and confusing contacts for you.

The following monitors have default settings at the time the servers are made managed, but can be adjusted by making a request with operations. If you have any questions about what other monitors are available, you are encouraged to contact support.

**Monitor** | **Description** | **Threshold** | **Frequency of Check**
---- | ---- | ---- | ---
CPU | Percentage of total CPU used | 100% average for 15 minutes (VM so can go over 100%) | 1 minute
Memory | Percentage of total Memory used | 97% (Win) 95% (Unix) average for 15 minutes | 1 minute
Disk Usage - Priority 3 | Percentage of total disk used | 90% | 1 minute
Disk Usage - Priority 2 | Percentage of total disk used | 95% | 1 minute
Port Reachable | Is it possible to reach the port? |Not reachable | 1 minute
Heartbeat | The monitoring agent on your server sends a heartbeat to a secondary system so that if there was ever a fault with the agent, we are notified. | No heartbeat | 1 minute checks (alert at 7 minutes without response)

repository: AnthonyHakim/PublicKB

repo_name: AnthonyHakim/PublicKB


text:
# Limitations
<br/>

- Only GitHub webhooks are currently supported.
- Webhooks in GitHub are sometimes left behind after deletion (details further below).
- Only `push` and `pull_request` events are currently supported, these are the events defined on the webhook.
- A limited number of pipelines will currently work with this implementation.

## Webhook not deleted

Due to a bug in the Knative codebase, the deletion of the webhook does not alway occur in GitHub.  All artifacts required for the webhook to run a pipeline are successfully removed from the cluster but, the webhook in GitHub may need to be manually deleted.  Until the webhook is manually deleted, it will attempt to send event details to a non-existent service.  This problem will cease to exist once [issue 247](https://github.com/tektoncd/experimental/issues/247) is addressed and the webhooks extension moves away from its dependency on Knative.

## Deleted webhooks can still be rendered until a refresh occurs

The Webhooks Extension component does not currently work with all `Pipelines`, it very specifically creates the following when the webhook is triggered:
Due to a bug in *our* codebase, a scenario exists whereby deleted webhooks can appear on the webhooks display table. This scenario is known to occur under the following circumstance.

- Create a webhook named `-`
- Create a webhook named `--`
- Create a webhook with an appropriate name e.g. `mywebhook`
- Attempt to delete all three webhooks

An error is displayed mentioning that problems occurred deleting webhooks (the ones named - and -), but `mywebhook` has actually been deleted. It is only until you refresh the page that this webhook will no longer be displayed.

## Pipeline limitations

The Webhooks Extension component does not currently work with all Tekton Pipelines, it very specifically creates the following when the webhook is triggered:

#### Git PipelineResource

A PipelineResource of type `git` is created with:

  - `revision` set to the short commit ID from the webhook payload.
  - `url` set to the repository URL from the webhook payload.

#### Image PipelineResource

A PipelineResource of type `image` is created with:

  - `url` set to `${REGISTRY}/${REPOSITORY-NAME}:${SHORT-COMMITID}` where, `REGISTRY` is the value set when creating the webhook, other values are taken from the webhook payload.

#### PipelineRuns Parameters/Resources

For a PipelineRun for your chosen Pipeline, in the namespace specified when your webhook was created, the values assigned to parameters on the PipelineRun are taken from values provided when configuring the webhook or from the webhook payload itself.

It is important to note the names of the parameters and resources, should you wish to use the extension with your own Pipelines and make use of these values.

PipelineRun parameters and resources made available:

```
params:
- name: image-tag
  description: The short commit ID
- name: image-name
  description: Image name formatted as: <REGISTRY>/<REPOSITORY-NAME>
- name: repository-name
  description: Repository name formatted as: <REPOSITORY-NAME>
- name: release-name
  description: Repository name formatted as: <REPOSITORY-NAME>
- name: target-namespace
  description: The PipelineRun target namespace
- name: event-payload
  description: The JSON event payload/body
- name: event-headers
  description: The JSON headers
- name: branch
  description: The repository branch

resources:
- name: docker-image
  description: The Name of the ImageResource
- name: git-source
  description: The Name of the GitResource
```

In particular, the `event-headers` and `event-payload` parameters can be especially useful when creating `Conditions` for `Pipelines`. For example, [this](https://github.com/pipeline-hotel/example-pipelines/blob/master/config/pipeline.yaml) `Pipeline` uses [this](https://github.com/pipeline-hotel/example-pipelines/blob/master/config/deployment-condition.yaml) `Condition` to skip `Tasks` if the event type is a `pull_request`.

repository: carlos-logro/experimental

repo_name: carlos-logro/experimental


text:
# Automated E-mail Reminder (AER)

## A sender bot based in Python

![gif](https://github.com/Meluiscruz/Automated_Email_Reminder/blob/master/Images/email_bot.gif)

## Project metrics

![Stars](https://img.shields.io/github/stars/Meluiscruz/Automated_Email_Reminder.svg)
![Forks](https://img.shields.io/github/forks/Meluiscruz/Automated_Email_Reminder.svg)
![Issues](https://img.shields.io/github/issues/Meluiscruz/Automated_Email_Reminder.svg)
![Tags](https://img.shields.io/github/tag/Meluiscruz/Automated_Email_Reminder.svg)

## Scope of the project

The main purpose of this project is to build messages and attachment files from some Excel files and send them to the clients as invoicing reminders.

This project is the first part of an RPA system developed in Python and uses an inner file manager for sorting pending files from submitted ones. The outputs of this script are the inputs of another automated process.

## Table of Contents

- [Python_Scripts](https://github.com/Meluiscruz/Automated_Email_Reminder/tree/master/Python_Scripts "Python_Scripts"):
  - [credentials.py](https://github.com/Meluiscruz/Automated_Email_Reminder/blob/master/Python_Scripts/credentials.py "credentials.py"): Edit this file with the sender data (email and password account).
  - [main.py](https://github.com/Meluiscruz/Automated_Email_Reminder/tree/master/Python_Scripts "main.py"): This is the head pice of code of the project.
- [input_files](https://github.com/Meluiscruz/Automated_Email_Reminder/tree/master/input_files "input_files"):
  - [providers](https://github.com/Meluiscruz/Automated_Email_Reminder/tree/master/input_files/providers "providers"): Where directories of clients (receivers information) are.
  - [pending_base_file](https://github.com/Meluiscruz/Automated_Email_Reminder/tree/master/input_files/pending_base_file "pending_base_file"): Where base files (patients, services and clinics information) are.
- [output_files](https://github.com/Meluiscruz/Automated_Email_Reminder/tree/master/output_files "output_files"):
  - [Reports](https://github.com/Meluiscruz/Automated_Email_Reminder/tree/master/output_files/Reports "Reports"): Where the summaries created by the bot are.
  - [Submitted](https://github.com/Meluiscruz/Automated_Email_Reminder/tree/master/input_files/pending_base_file "pending_base_file"):
    - [E1P1_base_file](https://github.com/Meluiscruz/Automated_Email_Reminder/tree/master/output_files/Submitted/E1P1_base_file "E1P1_base_file"): Where base files are moved after the process.
    - [E1P2_Submitted_files](https://github.com/Meluiscruz/Automated_Email_Reminder/tree/master/output_files/Submitted/E1P2_Submitted_files "E1P2_Submitted_files"): Where files created by the bot and used for the next process are.
  - [Tables](https://github.com/Meluiscruz/Automated_Email_Reminder/tree/master/output_files/Tables "Tables"): Where attachments (html and csv files created by the bot) are.
- [Images](https://github.com/Meluiscruz/Automated_Email_Reminder/tree/master/Images "Images"): Where files used in README.md are.

## What things are pending to develop?

- A script that installs all the required modules for the project.
- A script that asks the user for sender credentials (e-mail address, password and port).
- A script that sets the absolute path of the project (enviroment variable DEFAULT_DIR in main.py, for Windows and Linux).
- Handle the most of excepetions in main.py.
- The main.py script should work as 'lazy'. So, files must not be created if all the messages are not sent.

## Technical information

I used VSCode as source-code editor. Also, I have employed the following modules listed in [main.py](https://github.com/Meluiscruz/Automated_Email_Reminder/tree/master/Python_Scripts "main.py") header:

- ```numpy==1.20.1```
- ```openpyxl==3.0.6```
- ```pandas==1.2.2```
- ```xlrd==2.0.1```


text:
---
layout: post
title:  "January Upcoming Events"
date:   2018-01-08 11:32:00
---

Here's a list of all the upcoming events (click title for details):

We will be participating in the Charlotte MLK Jr Parade on Saturday, 01/13/18.
Meet on 10th street between Tryon and Church by 9:15 AM.
Parade starts at 10 AM and lasts until noon.

Reading Group will be this Wednesday, 01/17/18, from 6 PM to 8 PM
It will be located at "The Last Word"
Address: 5744 Tryon St, Charlotte, NC 28213

We will be participating in the Women's March on Saturday, 01/20/18.
The parade starts at First Ward Park.
It begins at 10 AM and lasts until noon.

Happy Hour will be next Wednesday, 01/24/2018, starting at 6:30 PM
Location is TDB. Check the "Upcoming Events" link to our calendar for up-to-date details.


text:
# tic-tac-toe with react hooks

[Reactjs](https://reactjs.org/tutorial/tutorial.html) official tutorial redone using react hooks.

This great [tutorial](https://scrimba.com/learn/reactgame) from [scrimba](https://scrimba.com) came in handy.

repository: devKiratu/tic-tac-toe-vhooks

repo_name: devKiratu/tic-tac-toe-vhooks


text:
# MockRobot_Device_Driver

## Description

The MockRobot is a robotic arm that moves objects to and from various locations around a system.  The robot can be controlled remotely by sending commands from a UI, implemented by the MocKRObot_Device_Driver.

Extra information:
- [Device Driver Interface Details](https://drive.google.com/file/d/1Hmib_VlGv-B80Fw2MoMsBSeRfoPw23OI/view?usp=sharing)
- [MockRobot Device API](https://drive.google.com/file/d/18cUXNJ3LKrwjKRp7UfoA_x9PXySLczRu/view?usp=sharing)

## Documentation

[Link to github folder containing documentation]()

## License

[Apache Licence 2.0](https://github.com/CristinaRiveraV/MockRobot_Device_Driver/blob/main/LICENSE)

## Contact

cristina.rivera.valdez@gmail.com


text:
# XOS

XOS is a framework for operationalizing a collection of
disaggregated services. It is packaged as a project in
the Open CORD initiative, with source code managed through
`https://gerrit.opencord.org`. It is also mirrored at
`https://github.com/opencord`.

You can download and use XOS either as part of CORD (see the
[CORD Guide](https://guide.opencord.org/) for details)
or as a standalone component (see the
[XOS Guide](https://guide.xosproject.org/) for details).
Also visit the
[XOS Wiki Page](https://wiki.opencord.org/display/CORD/XOS+and+the+CORD+Controller)
for additional information.

repository: iecedge/xos

repo_name: iecedge/xos


text:
---
output:
  html_document: default
  pdf_document: default
---

# References

Burkov, Andriy.  2019.  *The Hundred-Page Machine Learning Book.*  http://themlbook.com/

Goldburd, Mark et al. 2016.*Generalized Linear Models for Insurance Rating: CAS Monograph Series Number 5*. https://contentpreview.s3.us-east-2.amazonaws.com/CAS+Monograph+5+-+Generalized+Linear+Models+for+Insurance+Ratemaking.pdf

Hastie, Trevor, et al. 2002.  *The Elements of Statistical Learning*. Print.

James, Gareth, et al.  2017.  *An Introduction to Statistical Learning*.
http://faculty.marshall.usc.edu/gareth-james/ISL/ISLR%20Seventh%20Printing.pdf

Piech, Chris and Ng, Andrew.  2019.  Stanford CS221.  Course Notes.  https://stanford.edu/~cpiech/cs221/handouts/kmeans.html

Rigollet, Philippe (2017).  *Lecture 21: Generalized Linear Models*.  Video.  https://www.youtube.com/watch?v=X-ix97pw0xY&t=899s

Wickham, Hadley. 2019.  *R for Data Science*.  https://r4ds.had.co.nz/

"Complimentary Log Log Model".  University of Alberta.  Accessed 2020.  http://www.stat.ualberta.ca/~kcarrier/STAT562/comp_log_log.pdf

repository: dfsp-spirit/PA-R-Study-Manual

repo_name: dfsp-spirit/PA-R-Study-Manual


text:
## PID-Controller-Project
[![Udacity - Self-Driving Car NanoDegree](https://s3.amazonaws.com/udacity-sdc/github/shield-carnd.svg)](http://www.udacity.com/drive)

Project - 8: PID Controller Project.
---

In this project we will implement a PID Controller in C++ to maneuver the vehicle around the lake race track.

Overview
---
In this project we try to be in lane center by generating vehicle's steering angle, and drive around the race track, using PID Controller

Goal
---
The car will try to be in lane. No tire will leave the drivable portion of the track. The car will not pop up onto ledges or roll over any surfaces that would otherwise be considered

## Output

![Output_gif](./Results/result.gif)

## Implementation Details
Refer to [this](./Model_Documentation-PID_Controller_Project.pdf) for detailed Implementation strategy.

#### Simulator Details

Download Simulator from [here](https://github.com/udacity/self-driving-car-sim/releases).

To run the simulator on Mac/Linux, first make the binary file executable with the following command:
```shell
sudo chmod u+x {simulator_file_name}
```
The simulator will provide you the cross track error (CTE) and the velocity (mph) in order to compute the appropriate steering angle.

---

## Dependencies

* cmake >= 3.5
  * All OSes: [click here for installation instructions](https://cmake.org/install/)
* make >= 4.1
  * Linux: make is installed by default on most Linux distros
  * Mac: [install Xcode command line tools to get make](https://developer.apple.com/xcode/features/)
  * Windows: [Click here for installation instructions](http://gnuwin32.sourceforge.net/packages/make.htm)
* gcc/g++ >= 5.4
  * Linux: gcc / g++ is installed by default on most Linux distros
  * Mac: same deal as make - [install Xcode command line tools]((https://developer.apple.com/xcode/features/)
  * Windows: recommend using [MinGW](http://www.mingw.org/)
* [uWebSockets](https://github.com/uWebSockets/uWebSockets)
  * Run either `install-mac.sh` or `install-ubuntu.sh`.
  * If you install from source, checkout to commit `e94b6e1`, i.e.
    ```
    git clone https://github.com/uWebSockets/uWebSockets
    cd uWebSockets
    git checkout e94b6e1
    ```


text:
---
title: Overview
page_title: Button Overview
description: Discover the Blazor Button and explore the examples.
slug: components/button/overview
tags: telerik,blazor,button,overview
published: True
position: 0
---

# Blazor Button Overview

This article introduces the Blazor Button component, shows how to start using it, and lists its core features.

Buttons convey user actions and can display text, images and HTML.

The Blazor Button provides a variety of styling options through the [built-in themes]({%slug general-information/themes%}) and the [button type]({%slug button-type%}). It supports [font icons and images]({%slug button-icons%}) and fires click [events]({%slug button-events%}).

## Creating Blazor Button

1. Use the `<TelerikButton>` tag to add the component to your razor page.

1. Handle the `OnClick` event to respond to the user action.

>caption Basic Blazor Button with `OnClick` event handler

````CSHTML
@result
<br />
<TelerikButton OnClick="@OnClickHandler">Hello!</TelerikButton>

@code {
    private string result;

    private async Task OnClickHandler()
    {
        result = DateTime.Now.ToString();
    }
}
````

## Icons

To visually communicate the purpose of a button, you can add an image, sprite, or font icon. You can choose between a wide range of built-in font icons or use your custom font icons. [Read more about the Blazor Button icons...]({%slug button-icons%})

## Type

To control the submit behavior of the Blazor Button, use the `Type` attribute. Select from the following button types: `Submit`, `Reset`, and `Button`. The component also provides a `Form` parameter, which allows the user to submit a form from an external button. [Read more about the Blazor Button Type...]({%slug button-type%})

## Events

The Blazor Button fires events that you can handle and respond to user actions. [Read more about the Blazor Button events...]({%slug button-events%}).

## Disabled State

To prevent user interaction with a Button, disable it. [Read more about the disabled state of the Blazor Button...]({%slug button-disabled%}).

## Styling

To customize the style and the appearance of the Blazor Button, you can use the [built-in themes]({%slug general-information/themes%}). Additionally, set the `Class` attribute to implement custom CSS rules or [configure the built-in appearance options]({%slug button-styling%}).

>tip To learn more about the appearance, anatomy, and accessibility of the Button, visit the [Progress Design System documentation](https://www.telerik.com/design-system/docs/components/button/)—an information portal offering rich component usage guidelines, descriptions of the available style variables, and globalization support details.

## Button Parameters

The Blazor Button provides various parameters that allow you to configure the component. Also check the [Button's public API](/blazor-ui/api/Telerik.Blazor.Components.TelerikButton).

@[template](/_contentTemplates/common/parameters-table-styles.md#table-layout)

| Parameter | Type and Default Value | Description |
|----------|----------|----------|
| `ButtonType` | `ButtonType` enum
 (`ButtonType.Submit`)  | The `type` attribute of the Button. |
|`Class` | `string` | The CSS class that will be rendered on the main wrapping element of the Button (`<button class="k-button>`). |
| `Enabled` | `bool`
 (`true`) | Whether the Button is enabled. |
| `Form` | `string` | The ID of the associated form. Allows using a submit button outside a form. |
| `Id` | `string` | The `id` attribute of the Button. |
| `Icon` | `object` | The [icon rendered in the Button]({%slug button-icons%}). Can be set to a predefined Telerik icon or a custom one. |
| `Title` | `string` | The `title` attribute of the Button. |
| `Visible` | `bool`
 (`true`) | Whether the Button is visible. |

## Button Reference and Methods

Add a reference to the component instance to use the [Button methods](/blazor-ui/api/Telerik.Blazor.Components.TelerikButton). Be aware of the Blazor life cycle if you want to [focus the component on page load]({%slug inputs-kb-focus%}#notes).

@[template](/_contentTemplates/common/parameters-table-styles.md#table-layout)

| Method | Description |
| --- | --- |
| `FocusAsync` | Focuses the Button component. Always call with `await`. |

````CSHTML
<TelerikButton @ref="ButtonRef">Hello!</TelerikButton>

@code {
    private Telerik.Blazor.Components.TelerikButton ButtonRef { get; set; }

    protected override async Task OnAfterRenderAsync(bool firstRender)
    {
        await ButtonRef.FocusAsync();

        await base.OnAfterRenderAsync(firstRender);
    }
}
````

## Next Steps

* [Styling the Blazor Button]({%slug button-styling%})

* [Using Button Icons]({%slug button-icons%})

## See Also

* [Live Demo: Button](https://demos.telerik.com/blazor-ui/button/index)
* [Events]({%slug button-events%})
* [Type]({%slug button-type%})
* [Button API Reference](/blazor-ui/api/Telerik.Blazor.Components.TelerikButton)

repository: telerik/blazor-docs

gha_language: JavaScript

repo_name: telerik/blazor-docs


text:
---
layout: engineering-education
status: publish
published: true
url: /agora-express-token-server/
title: Generating Authentication Token for Agora Applications
description: This tutorial will give readers a detailed guide on how they can implement authentication for the Agora RTC SDK and RTM SDK.
author: mohan-raj
date: 2020-12-02T00:00:00-16:00
topics: [Node.js]
excerpt_separator: <!--more-->
images:

  - url: /engineering-education/agora-express-token-server/hero.jpg
    alt: Agora express token server
---
In this tutorial, we will be building a server using Node.js and Express to generate authentication tokens for the Agora SDKs. Authentication tokens are used in Agora SDKs to improve security. There are three authentication mechanisms available in Agora that we will discuss further below.
<!--more-->
### Goals
By the end of this tutorial, you’ll know:

- How Agora's authentication works.

- How to generate a temporary token for testing.

- How to set up an Express server to dispatch authentication tokens for the Agora SDKs.

### Agora
Agora.io was founded in 2014. It's a service provider for real-time voice and video. Its core technology is real-time communication (RTC).

Agora provides SDKs to build apps that require real-time engagement like:

- Audio/Video Calls.

- Interactive Livestreaming (Audio/ Video).

- Interactive Gaming.

- Real-Time Messaging (which is in BETA at the time of writing this article).

Agora is a paid service, but the first 10,000 minutes are free every month. You can check their pricing [here](https://www.agora.io/en/pricing/).

If you'd like to learn more about Agora, visit their [website](https://www.agora.io/en/) or read [this article](https://equalocean.com/analysis/201904121773).

### Overview
We'll be going through these steps in this article:

1. Authentication in Agora.
2. Creating a project in Agora.
3. Generating a temporary token.
4. Setting up the server.
5. Generating authentication tokens.
6. Recap.

> If you want to take a look at the final code, check out the [GitHub Repo](https://github.com/zolomohan/agora-express-token-server).

### Prerequisites
This tutorial is for applications that use [Agora](https://www.agora.io/) and want to implement authentication for its users. If you are not using Agora in your application, then this tutorial is not for you.

If you'd like to learn how to build a React Native application with Agora, refer to these articles,

- [Livestream App](/engineering-education/react-native-agora-livestreaming-app/).

- [Video Conference App](/engineering-education/react-native-agora-livestreaming-app/).

The fundamentals of Node.js and Express will not be covered in this tutorial. If you are not comfortable with the fundamentals, this is a [helpful<EMAIL_ADDRESS>that you can go through before beginning with this project.

### Authentication in Agora
We can provide authentication tokens for the Agora SDKs to improve security. There are three authentication mechanisms available in Agora.

- App ID: All clients in a channel use app ID for authentication. (Low Security)

- Token:  All clients in a channel use Token for authentication. (High Security)

- App ID or Token:  In a channel, some clients use an app ID for authentication, while others use a token. This is used in a transition scenario when you upgrade your project from App ID authentication to Token authentication.

> It's recommended to upgrade all your projects to use tokens for authentication since Agora is phasing out the support for App ID authentication.

#### App ID mechanism
An App ID is the unique identity of the project. If your app ID gets compromised, others can use it in their projects. So, it's not recommended to use an app ID for authentication.

#### Token mechanism
A token is a dynamic key generated by using App ID, App Certificate, User ID, Channel name, Token expiration timestamp, and other information. It's recommended to use a token for authentication as it's more secure than App ID authentication.

Tokens are generated on your server. When a user joins a channel the client requests the server for a token. The server will generate a token and pass it back to the client as the response.

You can refer to the documentation for Agora's authentication [here](https://docs.agora.io/en/Agora%20Platform/token?platform=Android).

### Creating a project in Agora
Head to Agora and create an account. You can reach the signup page from [here](https://sso.agora.io/en/v2/signup).

Fill in the details and create an account or you can signup with Google, GitHub, or Cocos. Once you've signed up, you'll see the dashboard.

![Agora Dashboard](/engineering-education/agora-express-token-server/agora_dashboard.png)

Click on New Project.

You'll see this form. Fill out the Project Name and set the Authentication Mechanism to Secure Mode.

![Agora New Project Form](/engineering-education/agora-express-token-server/agora_new_project.png)

Once you hit on submit, it'll create a new project and you should see it on the Project Management Console.

Now, click on the closed eye icon near the app ID to reveal it and copy that app ID. We'll need this ID to generate the authentication tokens.

![Agora Project Management Console App ID](/engineering-education/agora-express-token-server/project_management_console_app_id.png)

Now, click on the Edit action.

![Agora Project Management Console Edit Button](/engineering-education/agora-express-token-server/project_management_console_edit.png)

This will take you to the Edit Project screen. Here, you can copy the primary certificate that is required to generate the tokens.

![Primary Certificate](/engineering-education/agora-express-token-server/primary_certificate.png)

> The app ID and the app certificates must be kept secret. If the primary certificate gets compromised, you can enable a secondary app certificate and replace the current certificate with the secondary certificate. Later, you can delete the compromised app certificate. Make sure you update the token server with the new app certificate.

If you've set the Authentication Mode to Testing while creating the project, the primary certificate will be disabled. You can enable the primary certificate by clicking on Enable.

![Disabled Primary Certificate](/engineering-education/agora-express-token-server/disabled_primary_certificate.png)

### Generating a temporary token
For testing purposes, you can generate a temporary token from the project management console. Click on the temporary token button next to the app ID in the project management console.

![Agora Project Management Console Temp Token](/engineering-education/agora-express-token-server/project_management_console_temp_token.png)

This will take you to generate a temporary token form. You need to fill in the channel name and then click on the Generate Temp token button. You'll get a temporary token that you can use in your app for testing.

![Agora Project Management Console Temp Token](/engineering-education/agora-express-token-server/temp_token_form.png)

### Setting up the server
> You'll need Node.js to set up an Express server. You can download Node.js from [here](https://nodejs.org/en/).
> To test the server, I'll be using [Postman](https://www.postman.com/) to make requests to this server. You can download it from [here](https://www.postman.com/downloads/).

Let's install `Express` using `NPM`.

```bash
npm install express
```

Now, we can import `express` in our code to create a simple server module that'll listen on port 3000.

```JavaScript
const express = require("express");
const app = express();
app.use(express.json());

app.get("/", (req, res) => res.send("Agora Auth Token Server"));

const port = process.env.PORT || 3000;
app.listen(port, () => console.log(`Agora Auth Token Server listening at Port ${port}`));
```

You can start the server by running:

```bash
node index.js
```

This server will be listening on port 3000 and when you hit the `'/'` endpoint, it'll send `"Agora Auth Token Server"`.

### Generating authentication tokens
Agora provides the Real-time Communication (RTC) SDK and the Real-time Messaging (RTM) SDK. The `agora-access-token` package is used to generate tokens for both the SDKs.

The RTC SDK is used to enable real-time audio and video communications. With the help of the RTC SDK, developers can add audio/video call and audio/video broadcast functions in their projects. You can learn more about the RTC SDK [here](https://docs.agora.io/en/Agora%20Platform/term_agora_rtc_sdk?platform=Android).

The RTM SDK is used for real-time messaging scenarios. The RTM SDK is in beta at the time of writing this article. You can learn more about the RTM SDK [here](https://docs.agora.io/en/Real-time-Messaging/landing-page?platform=Android).

Let's install the `agora-access-token` package.

```bash
npm install agora-access-token
```

Let's import this package into our code.

```JavaScript
const Agora = require("agora-access-token");
```

You can refer to the documentation to generate a token [here](https://docs.agora.io/en/Video/token_server?platform=Android).

### RTC token
Let's add a POST handler for a new endpoint called `'/rtctoken'` to generate authentication tokens for RTC Channels.

```JavaScript
app.post("/rtctoken", (req, res) => {
  // Generate Token Here
});
```

When someone requests this endpoint, we'll generate an RTC authentication token and send it back to them.

The `agora-access-token` provides a class called `RtcTokenBuilder` that has a method called `buildTokenWithUid`.

The `buildTokenWithUid` method requires 6 arguments.

1. App ID.
2. App Certificate.
3. Channel ID.
4. User ID.
5. User Role.
6. Expiration Time.

We have the app ID and the app certificate from the project management console.

We'll get the channel ID and the user role from the request. You can learn more about the channel [here](https://docs.agora.io/en/Agora%20Platform/channel?platform=Android). You can learn more about user roles [here](https://docs.agora.io/en/Agora%20Platform/user_role?platform=Android).

We'll generate a random user ID and set the expiration time for the token in the server.

> Make sure the user ID you generate here is the same as the optional UID you pass to the SDK in the front-end while joining the channel.

```JavaScript
const appID = "<-- Your app ID here -->";
const appCertificate = "<-- Your app certificate here -->";
```

Let's generate a random User ID. Agora accepts a 32-bit unsigned integer with a value ranging from 1 to (2^32-1) as the User ID.
To make it simple, let's limit the range of the user ID to 100,000.

```JavaScript
const uid = Math.floor(Math.random() * 100000);
```

Let's set the expiration time of the token. You need to provide a timestamp for the expiration time. We'll add the time (in seconds) to the current timestamp to set when we want the token to expire.

```JavaScript
const expirationTimeInSeconds = 3600;
const currentTimestamp = Math.floor(Date.now() / 1000);
const privilegeExpiredTs = currentTimestamp + expirationTimeInSeconds;
```

There are two types of roles in a channel, a Publisher and a Subscriber.

- A Publisher is someone who streams audio/video in the channel and also receives audio/video from other publishers.

- A Subscriber will not be able to stream audio/video but can receive audio/video streams from publishers.

For example, in the case of a video conference, everyone is a publisher. But in the case of a livestream, only some users will be publishers while the others will be subscribers.

We should get whether the user is a publisher or subscriber in the request body. The `agora-access-token` provides constants for publisher and subscriber.

```JavaScript
const role = req.body.isPublisher ? Agora.RtcRole.PUBLISHER : Agora.RtcRole.SUBSCRIBER;
```

We should also get the Channel ID from the request body.

```JavaScript
const channel = req.body.channel;
```

Let's pass these data to `buildTokenWithUid` to generate the token.

```JavaScript
const token = Agora.RtcTokenBuilder.buildTokenWithUid(appID, appCertificate, channel, uid, role, privilegeExpiredTs);
```

Now that we have the token, Let's return the user ID and the token as the response.

```JavaScript
res.send({ uid, token });
```

Final Code:

```JavaScript
app.post("/rtctoken", (req, res) => {
  const appID = "<-- Your app ID here -->";
  const appCertificate = "<-- Your app certificate here -->";
  const expirationTimeInSeconds = 3600;
  const uid = Math.floor(Math.random() * 100000);
  const role = req.body.isPublisher ? Agora.RtcRole.PUBLISHER : Agora.RtcRole.SUBSCRIBER;
  const channel = req.body.channel;
  const currentTimestamp = Math.floor(Date.now() / 1000);
  const expirationTimestamp = currentTimestamp + expirationTimeInSeconds;

  const token = Agora.RtcTokenBuilder.buildTokenWithUid(appID, appCertificate, channel, uid, role, expirationTimestamp);
  res.send({ uid, token });
});
```

Let's request this endpoint to get the RTC token now.

Body of the request:

```json
{
  "channel": "<-- Channel ID here -->",
  "isPublisher": "<-- True or False here -->"
}
```

Request & Response:

![RTC Token Request and Response](/engineering-education/agora-express-token-server/rtc_postman.png)

> If you are using User Accounts instead of integer UIDs, you can use the `buildTokenWithAccount` function instead of the `buildTokenWithUid`.
> In `buildTokenWithAccount`, you will pass the user account instead of the UID. Everything else remains the same. You can learn more about user accounts [here](https://docs.agora.io/en/All/faq/string).

```JavaScript
Agora.RtcTokenBuilder.buildTokenWithAccount(appID, appCertificate, channel, userAccount, role, expirationTimestamp);
```

### RTM token
Let's add a POST handler for a new endpoint called `'/rtmtoken'` to generate authentication tokens for the RTM SDK.

```JavaScript
app.post("/rtmtoken", (req, res) => {
  // Generate Token Here
});
```

When someone requests this endpoint, we'll generate an RTM authentication token and send it as the response.

The `agora-access-token` provides a class called `RtmTokenBuilder` that has a method called `buildToken`.

The `buildToken` method requires 5 arguments.

1. App ID.
2. App Certificate.
3. User Account.
4. User Role.
5. Expiration Time.

We have the app ID and the App Certificate from the project management console.

```JavaScript
const appID = "<-- Your app ID Here -->";
const appCertificate = "<-- Your App certificate Here -->";
```

We should get the user account from the request. You can learn more about User Accounts [here](https://docs.agora.io/en/All/faq/string).

```JavaScript
const userAccount = req.body.user;
```

Let's set the expiration time of the token. You need to provide a timestamp for the expiration time. We'll add the time (in seconds) to the current timestamp to set when we want the token to expire.

```JavaScript
const expirationTimeInSeconds = 3600;
const currentTimestamp = Math.floor(Date.now() / 1000);
const privilegeExpiredTs = currentTimestamp + expirationTimeInSeconds;
```

There's only one role in RTM, that is an RTM User.

```JavaScript
const role = Agora.RtmRole.Rtm_User;
```

Let's pass these data to `buildToken` to generate the token.

```JavaScript
const token = Agora.RtmTokenBuilder.buildToken(appID, appCertificate, userAccount, role, privilegeExpiredTs);
```

Now that we have the token, let's return the token as the response.

```JavaScript
res.send({ token });
```

Final Code:

```JavaScript
app.post("/rtmtoken", (req, res) => {
  const appID = "<-- Your app ID here -->";
  const appCertificate = "<-- Your app certificate here -->";
  const user = req.body.user;
  const role = Agora.RtmRole.Rtm_User;
  const expirationTimeInSeconds = 3600;
  const currentTimestamp = Math.floor(Date.now() / 1000);
  const expirationTimestamp = currentTimestamp + expirationTimeInSeconds;

  const token = Agora.RtmTokenBuilder.buildToken(appID, appCertificate, userAccount, role, expirationTimestamp);
  res.send({ token });
});
```

Let's request this endpoint to get the RTM token now.

Body of the request:

```json
{
  "user": "<-- User Account here -->",
}
```

Request & Response:

![RTM Token Request and Response](/engineering-education/agora-express-token-server/rtm_postman.png)

### Let's Recap
1. We learned about authentication mechanisms in Agora.

2. We created a project in Agora and copied the app ID and the app certificate.

3. We set up a simple Express server.

4. We added a POST handler to generate RTC authentication tokens using the `buildTokenWithUid` method.

4. We added a POST handler to generate RTM authentication tokens using the `buildToken` method.

Congratulations, :partying_face: You did it.

Thanks for reading!

---
Peer Review Contributions by: [Adrian Murage](/engineering-education/authors/adrian-murage/)

repository: mbuguac/engineering-education

gha_language: SCSS

repo_name: mbuguac/engineering-education


text:
### Welcome to my Portfolio

Hi visitor, this is my portfolio.

### About me
 * I am a Software Engineer passionate about mobile development, Coffe and Dota 2.

### Code challenges
#### March-2019
 * [Adidas code challenge](https://github.com/therealandroid/adidas-code-challenge-android-master)
 * [Skyscanner code challenge](https://github.com/therealandroid/SkyScannerTest)

### Android Projects

 - [Sqlite Model Mapper](https://github.com/therealandroid/SqliteModelMapper)

    *Description:*

    *Language:* `Java`

 - [Mentions](https://github.com/therealandroid/Mentions)

    *Description:*

    *Language:* `Java`

 - [Header and Footer Adapter](https://github.com/therealandroid/HeaderFooterAdapter/blob/master/README.md)

     *Description:*

     *Language:* `Java`

 - [Header and Footer Adapter Sample](https://github.com/therealandroid/HeaderFooterAdapter/blob/master/README.md)

     *Description:*

     *Language:* `Java`

 - [Sqlite Table Generator](https://github.com/therealandroid/SqliteTableGenerator)

     *Description:*

     *Language:* `Java`

 - [Unit tests using Room Database](UnityTests-Room-Database)

     *Description:*

     *Language:* `Kotlin`

repository: therealandroid/mywebsite

repo_name: therealandroid/mywebsite


text:
+++
# A Demo section created with the Blank widget.
# Any elements can be added in the body: https://sourcethemes.com/academic/docs/writing-markdown-latex/
# Add more sections by duplicating this file and customizing to your requirements.

widget = "blank"  # See https://sourcethemes.com/academic/docs/page-builder/
headless = true  # This file represents a page section.
active = true  # Activate this widget? true/false
weight = 15  # Order that this section will appear.
share = false
profile = false
date = "2019-06-23"

title = "ACMUS-MIR DATASET"
subtitle = ""

[design]
  # Choose how many columns the section has. Valid values: 1 or 2.
  columns = "1"

[design.background]
  # Apply a background color, gradient, or image.
  #   Uncomment (by removing `#`) an option to apply it.
  #   Choose a light or dark text color by setting `text_color_light`.
  #   Any HTML color name or Hex value is valid.

  # Background color.
  # color = "white"

  # Background gradient.
  #gradient_start = "DarkGreen"
  # gradient_end = "ForestGreen"

  # Background image.
  # image = "image.jpg"  # Name of image in `static/img/`.
  # image_darken = 0.6  # Darken the image? Range 0-1 where 0 is transparent and 1 is opaque.

  # Text color (true=light or false=dark).
  # text_color_light = true

[design.spacing]
  # Customize the section spacing. Order is top, right, bottom, left.
  padding = ["20px", "0", "20px", "0"]

[advanced]
 # Custom CSS.
 css_style = ""

 # CSS class.
 css_class = ""

+++

![Andes region of Colombia](/img/andes.png#floatright)
The ACMUS-MIR datas set is a growing collection of annotated music from the [Andes region of Colombia]({{< ref "/andes-music/index.md" >}}).
The data set was compiled with the goal of supporting the different MIR task addressed in our project: [meter extraction]({{< ref "/project/meterextraction.md" >}}),
[instrumental format recognition]({{< ref "/project/ensemblesize.md" >}}), [speech/music discrimination]({{< ref "/project/speechmusic.md" >}}) , and [scale detection in music]({{< ref "/project/scale.md" >}}).
For each of these tasks, a dedicated collection of music was compiled and annotated by musicologist in Colombia.

The ACMUS-MIR data set has been made publicly available for the wider research community as a Zenodo repository[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.3965447.svg)](https://doi.org/10.5281/zenodo.3965447)

For a detailed description of the dataset, please refer to our DLfM 2019 [publication.](https://dlsi.ua.es/gent/drizo/dlfm2019/mora.pdf) For information about the different versions of the ACMUS-MIR dataset, see the links below:
- [v1.0]({{< ref "/acmus-log/v10.md" >}})
- [v1.1]({{< ref "/acmus-log/v11.md" >}})

## Rhtyhm Set
Bambuco - 6/8

{{< audio src="rh_0001.wav" >}}

Bambuco - 6/8

{{< audio src="rh_0002.wav" >}}

Bambuco - 6/8

{{< audio src="rh_0003.wav" >}}

Guabina - 3/4

{{< audio src="rh_0004.wav" >}}

 Pasillo - 3/4

 {{< audio src="rh_0005.wav" >}}

## Instrumental Format Set
Solo tiple

{{< audio src="if_0112.wav" >}}

Solo bandola

{{< audio src="if_0134.wav" >}}

Solo guitar

{{< audio src="if_0015.wav" >}}

Duet - tiple, guitar

{{< audio src="if_0056.wav" >}}

Trio - bandola, tiple, guitar

{{< audio src="if_0162.wav" >}}

Quartet -  2 bandolas, tiple, guitar

{{< audio src="if_0167.wav" >}}

Quintet - 2 bandolas, tiple, guitar, bass

{{< audio src="if_0103.wav" >}}

Large ensemble

{{< audio src="if_0113.wav" >}}

## Scale Set
Ab - Major - 4 instruments

{{< audio src="sc_0001.wav" >}}

Eb - Minor - 4 instruments

{{< audio src="sc_0002.wav" >}}

F - Major - 4 instruments

{{< audio src="sc_0003.wav" >}}

Bb - Major - 4 instruments

{{< audio src="sc_0004.wav" >}}

Eb - Minor - 4 instruments

{{< audio src="sc_0005.wav" >}}

repository: ACMUS-MIR/academic-kickstart

repo_name: ACMUS-MIR/academic-kickstart


text:
---
title: On-the-fly encoding and decoding of bitmaps
description: The importance of bitmaps is irrefutable, this is why I recently started investigating which are the most effective techniques used to compress them. Being able to reduce their memory usage means being able to store more data or, possibly, fit it in a lower level of the cache hierarchy which immediately translates to faster access.
layout: post
tags:
    - compression
    - bitmaps
    - bitvector
image: /uploads/binary-code.png
---
![binary-code]({{ site.url }}/uploads/binary-code.png){:class="img-responsive"}

A bitmap, also referred to as bit-vector or bit-array, is a sequence of *0s* and *1s* which typically encodes a more complex object.

A common example of this is a set of numbers where each of the elements are indicated as set bits in a bitmap of length equal to the greatest element plus one (as we count from zero), also commonly referred as the *universe*. As an example, the set `{3,5,21,4,23,12}` can be represented as 101000000001000000111000, where - counting right to left - we have the 1-bit at the corresponding positions of the elements in the initial set.

The importance of bitmaps is irrefutable, this is why I recently started investigating which are the most effective techniques used to compress them.
Being able to reduce their memory usage means being able to store more data or, possibly, fit it in a lower level of the cache hierarchy which immediately translates to faster access.

The technique I would like to discuss sets the base for more complex ones, which I will try to cover in a future blog post. The most important property of the following compression algorithm is the ability to query the bitmap without fully decompressing it. Considering the set we saw in the previous example, this would be extremely appealing, as we would be able to tell if an element *i* is present or not just by looking at the bit at position *i*.
The compression I am going to present falls into the category of data structures called **succinct data structures**, which allow efficient query operations while using an amount of space that is close to the information-theoretic lower bound.

Now we split the bitmap into fixed-length blocks. In the previous example, the bitmap was 24-bit long, if we split it into blocks of 3-bits each we obtain four distinct blocks.

101 000 000 001 000 000 111 000

 The idea is to code each block independently from the the others, using a pair of values *<Ci,Oi>* for the *i-th* block.
 The first element of the pair is the **cardinality** of the block, also referred as population count or just *popcount*; while the latter is the **offset** in the table that contains all the distinct permutations (so combinations) of the bits in that block [^fn1].

Let's say we want to encode the first block 101. Calculating *C* is trivial as we need to count the number of bits set to 1. This can also be done in hardware by most of the modern CPUs (I will come back to this topic again in the future), but for now, we can rely on the following naive implementation.

{% highlight cpp %}
size_t popcount(uint64_t n)
{
    size_t count = 0;
    for(size_t i = 0; n > 0; ++i )
    {
        count += n & 1;
        n >>= 1;
    }
    return count;
}
{% endhighlight %}

Now lets imagine we have a table containing all the $$\binom{3}{2} = 3$$ ordered permutations of the previous block. If we iterated over the rows of this table and stop when we reach the entry that matches our block, then we would have computed the offset for that block.
In our example, the offset of the block in the following table would be 2.

|-----------------|:-----------------:|
| 0               | 011 |
| 1               | 101 |
| 2  | 110 |
{:style="margin: auto;"}

In this way we can encode our block with the two integers *C = 2* and *O = 2*.

Whenever we would like to decode the block from the given *C* and *O* we need to select the appropriate table of combinations using *C* and then move to the index *O* of that table to retrieve the original representation.

In this environment if we were only interested in the *i-th* bit, we would have decoded the entire block and applied a proper mask to filter it out.

## Cost analysis

So far we realized we need to store a pair for integers for each block, so for $$m/b$$ blocks where *m* is the original bitmap length and b is the fixed block size. We know that the population count of the block cannot be greater then the block size its-self, since there cannot be more than *b* set bits in a block. Then we can state that the C coefficients can be stored in $$\log_2(b)$$ bits.
Regarding the offsets we know that they are indexes in a table, but the table size depends on two factors: the blocks size and the number of set bits in it.
We know that the former is the same for each block, but the population count can vary.

There are two lucky cases where the cardinality gives us enough information to infer the offset:

1. if *C* is 0, as we know then that the only permutation is then one where all the bits are unset.
2. if *C* is equal to the size of the block, where the block as all the bits set.

In these two cases we can store the offset implicitly and so we would not sacrifice any extra space. For all the other possibilities we can always store the offset in $$\log_2\binom{b}{C}$$ bits.

For instance, the original bitmap we used in the previous example used 24 bits of actual data in its uncompressed form. To store the cardinality of each block we would need 2 bits per block, for a total of 16 bits. Then, the blocks containing all-zeros or all-ones is encoded implicitly, while the blocks with *C = 1* and *C = 2* need 2 additional bits each. This sums up to 20 bits used to represent our uncompressed 24-bit bitmap, with a saving of 2 bits or ~16% of the initial size.

## On-the-fly generation of ordered binary permutations

At this point we know how to encode and decode a block, what is actually missing is the way to generate the lookup table of the permutations. The answer is that we don't do it, but instead ordered binary permutations are generated on-the-fly [^fn2].

For small blocks it would actually be doable and probably also convenient, but if the block gets bigger then it is just not feasible. What is needed is an algorithm to compute offsets for a given block and being able to go back from the offset to the original block  representation in a reasonably effective way.

### Compute offsets

The aim of the computation of the offset is to find the index of a block, given its size and population count, in a table listing all the possible permutations. Moreover, it needs to be deterministic and without the overhead of an actual table.
Basically what the following algorithm does is iterating over every bit in the block, if is unset it moves to the next bit (so the block size decreases but the cardinality does not vary), else if it is set then we increase the offset by a quantity equal to $$\binom{n-1}{count}$$ (now both block size and cardinality decrease by one) where *n* is the position of the bit and *count* the missing set bits to encounter.

{% highlight cpp %}
uint64_t compute_offset(uint64_t block_size, uint64_t count, uint64_t block) {
    uint64_t offset = 0;
    uint64_t n = block_size - 1;
    while (count > 0) {
        if (block & (1ULL << n)) {
            if (count <= n)
                offset += boost::math::binomial_coefficient<double>(n, count);
            --count;
        }
        --n;
    }
    return offset;
}
{% endhighlight %}

### Decode offsets on the fly

Now we need to reverse the encoding process.
If $$offset \geq \binom{n-1}{count}$$, then the first bit of the block was a 1 and we decrement both *n* and *count* and subtract $$\binom{t-1}{count}$$ from the offset; otherwise it was a 0 and we decrement only *n*.
Every iteration the block size decreases by one. We can stop when we have processed the whole block or when the count reaches 0 as it means that the remaining bits are all unset.

{% highlight cpp %}
uint64_t decode_offset(uint64_t block_size, uint64_t count, uint64_t offset) {
  uint64_t block = 0;
  uint64_t n = block_size - 1;
  while (count > 0 && n > 0) {
    if (count <= n &&
        offset >= boost::math::binomial_coefficient<double>(n, count)) {
      offset -= boost::math::binomial_coefficient<double>(n, count);
      block |= (1ULL << n);
      --count;
    }
    --n;
  }
  if (count > 0)
    block |= (1ULL << count) - 1;
  return block;
}
{% endhighlight %}

Since most of the times we are interested in a single bit of the block and blocks can be quite long and so slower to decode, we can perform two forms of optimization. The former is to stop the iteration when we reach the given position, the latter is to perform a binary search instead of the linear scan we already described. The combination of the two solutions is ideal, indeed a linear scan is still faster when the position we are interested in is within the first $$block\_size \cdot \log_2(block\_size)$$ elements.

## Conclusion

I feel this is a nice and elegant way to compress a bitmap while keeping the ability to decode a block in constant time as it on-the-fly decoding only depends on the the block size which is fixed. I am also sure that further improvements for faster decoding can be possible with the use of SIMD instructions.

Feel free to get in touch if you want to share any feedback or have any ideas about the topic and would like to dig more into it.

## References

[^fn1]: Rajeev Raman, Venkatesh Raman, and S. Srinivasa Rao. 2002. Succinct indexable dictionaries with applications to encoding k-ary trees and multisets. In Proceedings of the thirteenth annual ACM-SIAM symposium on Discrete algorithms (SODA '02). Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, 233-242.

[^fn2]: Gonzalo Navarro and Eliana Providel. 2012. Fast, small, simple rank/select on bitmaps. In Proceedings of the 11th international conference on Experimental Algorithms (SEA'12), Ralf Klasing (Ed.). Springer-Verlag, Berlin, Heidelberg, 295-306.


text:
# Warden HttpApi Integration

![Warden](http://spetz.github.io/img/warden_logo.png)

**OPEN SOURCE & CROSS-PLATFORM TOOL FOR SIMPLIFIED MONITORING**

**[getwarden.net](http://getwarden.net)**

|Branch             |Build status
|-------------------|-----------------------------------------------------
|master             |[![master branch build status](https://api.travis-ci.org/warden-stack/Warden.Integrations.HttpApi.svg?branch=master)](https://travis-ci.org/warden-stack/Warden.Integrations.HttpApi)
|develop            |[![develop branch build status](https://api.travis-ci.org/warden-stack/Warden.Integrations.HttpApi.svg?branch=develop)](https://travis-ci.org/warden-stack/Warden.Integrations.HttpApi/branches)

**HttpApiIntegration** can be used for making a POST request to the own/custom API or the Warden Web Panel via the provided helper methods. You may include any data while making the POST request and for example create your own webhooks that will be invoked under specific conditions.

### Installation:

Available as a **[NuGet package](https://www.nuget.org/packages/Warden.Integrations.HttpApi)**.
```
dotnet add package Warden.Integrations.HttpApi
```

### Configuration:

 - **WithTimeout()** - timeout of the HTTP request.
 - **WithHeaders()** - collection of the HTTP request headers.
 - **WithJsonSerializerSettings()** - custom JSON serializer settings of the Newtonsoft.Json.
 - **FailFast()** - default parameters of the transactional template.
 - **WithHttpServiceProvider()** - flag determining whether an exception should be thrown on if PostAsync() returns invalid reponse (false by default).

**HttpApiIntegration** can be configured by using the **HttpApiIntegrationConfiguration** class or via the lambda expression passed to a specialized constructor.

### Initialization:

In order to register and resolve **HttpApiIntegration** make use of the available extension methods while configuring the **Warden**:

```csharp
var wardenConfiguration = WardenConfiguration
    .Create()
    .IntegrateWithHttpApi("http://my-api.com")
    .SetGlobalWatcherHooks((hooks, integrations) =>
    {
        hooks.OnStart(check => GlobalHookOnStart(check))
             .OnFailureAsync(result => integrations.HttpApi().PostAsync(new {failure = true}))
    })
    //Configure watchers, hooks etc..
```

You may also provide the API key (which will set the X-Api-Key request header). On top of that, you may set the organization id (created within the Warden Web Panel) and POST the iteration data directly to the Warden API.
Please note that you may either use the **[Web Panel](https://github.com/spetz/Warden/wiki/Web-Panel)** in **[Azure](https://panel.getwarden.net/)** cloud (https://panel.getwarden.net/api) or your own API e.g. http://localhost/api.

```csharp
var wardenConfiguration = WardenConfiguration
    .Create()
    .IntegrateWithHttpApi("https://panel.getwarden.net/api",
                          "my-api-key", "my-organization-id")
    .SetHooks((hooks, integrations) =>
    {
        .OnIterationCompletedAsync(iteration => integrations.HttpApi()
                                   .PostIterationToWardenPanelAsync(iteration));
    })
    //Configure watchers, hooks etc..
```

### Custom interfaces:
```csharp
public interface IHttpService
{
   Task<IHttpResponse> ExecuteAsync(string baseUrl, IHttpRequest request, TimeSpan? timeout = null)
}
```

**IHttpService** is responsible for making a POST request. It can be configured via the *WithHttpServiceProvider()* method. By default, it is based on the HttpClient instance.


text:
---
title: "Talk, Christmas and next year | Readings"
date: "2023-01-02"
---

![Photo by Hiroko Yoshii on Unsplash](./reef.avif)

For the last couple of weeks, I've slowed down a bit and been reading a bit more high fantasy and enjoying the festivities. I've given the dbt in a data mesh world talk at Data Porto meetup which was nice and hope to work a bit more on relevant work to do other talks on more technical stuff.

With the close of the year, it's time to look back at what I've accomplished.

## Programming languages

My initial focus of the year had been on Python and Scala. However, things changed and I've got now to add javascript/Typescript. I'll do some somethings in Python but I've decided to learn Rust as my "fast" language for when TS isn't enough. My goal will be to finish the advent of code of 2022, read the book and help launch a data-intensive application I've got in the pipeline. This leads me to the next section.

## Side projects

It's funny how looking at [last year review](https://cabeda.dev/2021/20210101-2020-review/#programming-languages-and-frameworks) I got things done by helping solve daily tasks with my software. Nothing major. I guess I'll keep on doing this as I have a Rust project already in the works and I'll additionally be looking for ways to improve the tooling at my current place of work.

## Organization

My calendar was indeed a good friend of mine as I saw last year. I'm doing a mixture of writing my tasks on Notion but I look into blocking some time on the calendar for achieving this. If the task is too big for say 2 hours, I try to break it down. There's not a perfect tool although having a sync between tasks and the calendar would be great. Google tasks can do 90% of this but I guess I like to set a duration so I'd still need to

## Writing

I've come this year to like to write the weekly update. I've published 24 articles in total and the last one went to a [medium publication](https://medium.com/better-programming/duckdb-vs-porto-buses-a-small-case-for-a-new-olap-engine-1c04b898d293)). Small but important steps. The updates are a retrospective of what I've accomplished and what I plan to do next. I don't set hard dates but it has helped me define and fulfill loose goals. Instead of setting blogging goals, I'll be looking to write around 500 daily words. It can be anything from personal notes, knowledge, or blogging. For this simple article, I was able to fulfill this already so I think it's an achievable one. Hopefully, I'll come up with something interesting to share but, if not, at least I'll feel [my head is more organized](http://paulgraham.com/words.html).

## Entertainment, Social & Fitness

The remote work is great but I've had to find mechanisms to keep my head fresh and ready. I read a lot and my usual plan was to mix technical books with light reading. I don't think this works anymore as I usually need to balance entertainment with studying. So now I'm switching to have always something to read or watch. Technical reading is not something to be entertained (unless it's that interesting 😅).

Through the past year, I've started to balance my life between soccer (1X per week) and swimming (2X per week). I'm set on this and with the flexible schedule, it seems to be something I can keep for the rest of my life. Just need to be more careful with what I eat to not spoil it 🤐.

Regarding socializing this is something that keeps on turning as very very important with remote work. I want to keep in touch with my friends and I have a good idea to help me in this task by reminding of who I haven't been talking to. Additionally, I'd love to set up or join a monthly board game group. This has been the year I learned about D&D but I'm guessing that is still a bit hardcore. One can dream though...

## Summary

So here is the TLDR of actionable actions to do/keep:

- Keep on scheduling the tasks on the calendar
- Daily 500 words
- Read Rust book
- Finish advent of code in Rust
- Develop a research program in Rust
- Develop app to remind who to keep in touch
- Setup/join a monthly board game

## Readings of the week

- [Data Catalog - A Broken Promise](https://www.dataengineeringweekly.com/p/data-catalog-a-broken-promise)
- [Extending Python with Rust](https://maxwellrules.com/programming/rusty-python.html)
- [Why I'm Glad I Lack Passion to BE a Programmer](https://blog.miris.design/not-a-programmer)
- [Open-Source is the Way Out of a Tech Downturn](https://www.plural.sh/blog/open-source-is-the-way-out-of-a-tech-downturn/)
- [How HomeToGo has connected Superset Dashboards to dbt Exposures to improve Data Discoverability](https://engineering.hometogo.com/how-hometogo-has-connected-superset-dashboards-to-dbt-exposures-to-improve-data-discoverability-3d0add162e4a)
- [Staging is dead: The rise of preview environments](https://www.withcoherence.com/post/staging-is-dead-the-rise-of-preview-environments)
- [AWS re:Invent 2022 - [NEW] Democratize data w/governance: People, data & tools together (ANT344)](https://www.youtube.com/watch?v=t3LTunhYys4)
- [Programming as play](https://austinhenley.com/blog/programmingasplay.html)
- [12 Days of PostgreSQL Tools and Projects](https://www.timescale.com/blog/12-days-of-postgresql-tools-and-projects/)
- [How Twitter Automate Data Quality Check Process](https://www.infoq.com/news/2022/12/twitter-data-quality/)
- [WebAssembly: Docker without containers!](https://wasmlabs.dev/articles/docker-without-containers/)
- [Next, Nest, Nuxt… Nust?](https://www.twilio.com/blog/comparing-nextjs-nestjs-nuxt-gatsby)
- [Lichess & Scala 3](https://lichess.org/@/thibault/blog/lichess--scala-3/y1sbYzJX)
- [Programs are dead](https://pouria.dev/programs-are-dead/)
- [Highlights from Git 2.39](https://github.blog/2022-12-12-highlights-from-git-2-39/)

repository: Cabeda/jecabeda-blog

repo_name: Cabeda/jecabeda-blog


text:
# GFL Framework

---------------------------------------------------

English | [简体中文](./README_cn.md)

**Galaxy Federated Learning Framework(GFL)** is a decentralized federated learning framework based on blockchain. GFL builds a decentralized communication network based on Ethereum, and executes key operations that requires credibility in FL through smart contracts.

## Quick Start

### 1. System Envs Required

a) GFL only supports Python3， please make sure your python version is no less than `3.4`.

b) GFL is based on `Pytorch`, so, `torch>=1.4.0` and `torchvision>=0.5.0` is required before using GFL. [Pytorch installation tutorial](https://pytorch.org/get-started/locally/)

### 2. Install

```shell
pip install gfl
```

### 3. Usage

The available commands of GFL.

```
usage: GFL [-h] {init,app,attach} ...

optional arguments:
  -h, --help         show this help message and exit

actions:
  {init,app,attach}
    init             init gfl env
    run              startup gfl
    attach           connect to gfl node
```

Init GFL node in `datadir` directory.

```shell
python -m gfl init --home datadir
```

Start GFL node(start in standalone mode by default). If you need to open console when starting node, use the `--console`` argument.

```shell
python -m gfl run --home datadir
```

Open console for operating GFL node. The following three methods can be used to connect to the node started in the previous step.

```
python -m gfl attach						# connect to http://localhost:9434 in default
python -m gfl attach -H <IP_ADDRESS> -P 9434
python -m gfl attach --home datadir
```

## GFL base design

![image-20210903165315547](./assets/GFL-base-framework.png)

The GFL framework is divided into two parts:

**Job Generator**

> Used to create a job that can be executed in the GFL network. Developers can use the interface provided by GFL to generate a Job for various configuration parameters and distribute them to the network for training.

**Run-Time Network**

> Several running nodes build GFL's decentralized training network, and each GFL node is also a node in the blockchain. These nodes continuously process the jobs to be trained in the network according to user commands.

## GFL core arch

![image-20210903213928765](./assets/GFL-core-framework.png)

+ **Manager Layer**

  + The start/stop/status operation of node
  + Provide communication interface for nodes
  + Sync job

+ **Scheduler Layer**

  + Manage the execution process of each job
  + Synchronize parameter files among nodes
  + Schedule the execution order of multiple jobs on the node

+ **FL Layer**

  + Configure the running environment of the job
  + Perform training/aggregation tasks
  + Provide the interfaces of user-defined action

repository: qianjilong/GFL

repo_name: qianjilong/GFL


text:
# Publishing gretty artifacts

Gretty sources are configured for publishing gretty artifacts to sonatype and bintray.
Publishing typically consists of two steps:

1. Define publishing-specific properties
2. Invoke publishing task.

Below are the detailed instructions.

## Publishing to sonatype

1. Create GPG signing key, add it to your public key ring.
2. Publish public GPG key to public keyserver, for example, to keyserver.ubuntu.com
3. Create file "gradle.properties" in ~/.gradle, insert the following content to it:
  ```gradle
  signing.keyId={gpg-key-id}
  signing.password={gpg-key-passphrase}
  signing.secretKeyRingFile=/home/yourUserName/.gnupg/secring.gpg
  sonatypeUsername={sonatype-username}
  sonatypePassword={sonatype-password}
  ```
  (Please substitute real values instead of curly braces).

4. Run from command line:
```shell
gradle uploadArchives -PpublishToSonatype=true
```

## Publishing to bintray

1. Upload private and public armored GPG key at https://bintray.com/profile/edit, section "GPG Signing".

2. Tick the checkbox "Sign this repository's files with key from: ..." at https://bintray.com/yourUserName/maven/edit?tab=general

3. Create file "gradle.properties" in ~/.gradle, insert the following content to it:
  ```groovy
  bintrayUser={bintray-user-name}
  bintrayKey={bintray-API-key}
  ```
  (Please substitute real values instead of curly braces).

4. Run from command line:
```shell
gradle bintrayUpload
```