text:
---
title: HXP 2017 pdf.pdf
authors: Lense
date: 2017-11-19
categories: forensics
---

>RTFM! and a link to [pdf.pdf]({{ site.baseurl }}/assets/pdf.pdf).

The PDF contained a bunch of different types of content, like text, a vector
drawing, a bitmap image, embedded fonts, internal and external links, annotated
rectangles around the links, table of contents metadata, and presumably the
flag. It ended with a link to the
[PDF spec](http://www.adobe.com/content/dam/acom/en/devnet/pdf/pdfs/PDF32000_2008.pdf),
and we took that to mean that we should write a PDF parser.

### PDF file format overview

It turns out that while rendering PDFs is extremely painful, just parsing a PDF
is only moderately painful, and the spec isn't as bad as I was expecting.

#### File sections

##### Header
The first line is a comment (starting with `%`) with the PDF version number.
The challenge used version 1.5.
The second line is a comment with 4 non-ascii bytes, designed to keep text
viewers from trying to render PDFs as plaintext.

##### Body

The body is a series of objects. They aren't rendered implicitly, but they can
be referenced by other objects and the XRef table at the end.

##### Footer

The footer can have a whole bunch of different types that render the PDF
differently, but since we were just trying to parse the PDF, we didn't deal
with it much. At a high level, the footer contains a cross-reference table that
points to byte-offsets in the file of objects.

#### Object types

In this PDF, the body consisted entirely of indirect objects. Those indirect
objects contained streams of compressed data, which contained one or more
direct objects of arbitrary type. I'm sure there are many other ways that PDFs
can be structured, but we just had to deal with this.

##### Indirect objects

Format:

```
<object number> <generation number> obj
<contents (stream)>
endobj
```

In our PDF, generation number was always 0 and the contents were always
streams. To point to the stream, other objects can use a reference:
`<object number> <generation number> R`.

##### Dictionaries

Format: `<< Key Value Key Value ... >>`

Most of the objects in the PDF were dictionaries. Among other things, these are
used for content hierarchies, like defining what pages there were and what
contents were on each page.

##### Streams

Format:

```
<dictionary>
stream
<data>
endstream
```

Streams consist of a dictionary of metadata and stream data. The streams in
this PDF set `/Filter` to `/FlateDecode` in their dictionaries, which meant
that the streams were compressed with zlib-flate. Some streams also had more
complicated compression methods that we looked at and decided didn't contain
the flag.

##### Hex strings

Format: `<hex...>`

In this PDF, they were all UTF-16.

##### Other objects

PDFs also have ints, floats, arrays, strings, and more, but they're more
self-explanatory and not as important for the challenge.

#### But where's the flag?

We wrote a parser for almost all the objects of the binary, but it didn't
render anything, and so we didn't see the flag just lying there. After spinning
our wheels for a while, we had the idea to look for something hidden. We made a
list of all the references and looked for an object that was never referenced.

Object 30 was never referenced:

```
<< /Contents [ 83 0 R ] /Parent 58 0 R /Resources 43 0 R /Type /Page >>
```

It was a hidden page with page contents of object 83 and resource object 43.
Object 43 was a reference to XObject 85, a vector image. Hawkheart wrote a
renderer for it, and we got this nice picture of a flag:

![vector image of a flag]({{ site.baseurl }}/assets/hxp_pdf_vector_flag.png)

That's pretty, but not very useful. Object 83 was trickier:

```
q 1 0 0 1 72 769.89 cm q .5489 0 0 .5489 70.2 -62.656 cm q 1 0 0 1 155.62 -138.895 cm /Fm1 Do Q Q BT /F3 11.9552 Tf -1.134 -335.582 Td[<0075>81<0032001c003f0035>-433<0075>81<0051006d>-325<004b001c002f0032>-326<00420069>-326<001c004d002f>-326<00370051006d004d002f>-326<0069003f0032>-326<004b001c003b0042002b>-326<007e001c003b002c>]TJ ET BT /F6 17.2154 Tf -1.134 -371.697 Td[<004e005a0044005e004500580057004e001d00540048>17<005b005d0054004b005b005d004c004d00540055004e>31<0058005400510049004a004a0049002c00460054004400570046001600540049002c0054005b005d004700540056005b005b005300540044004d004b0060>]TJ ET q .5489 0 0 .5489 70.2 -315.33 cm q 1 0 0 1 155.62 -699.024 cm /Fm1 Do Q Q BT /F3 11.9552 Tf 222.709 -728.912 Td[<0039>]TJ ET Q
```

It's a graphics object (or something, not exactly sure) containing operators
and operands in postfix order.
[Table 51](https://www.adobe.com/content/dam/acom/en/devnet/pdf/pdfs/PDF32000_2008.pdf#G7.3952560)
of the spec was very useful for figuring out what everything was.

Going through it:

- `q` does something about graphics state. ignored
- `# # # # # # cm` also does something about graphics state. ignored
- `<name> Do` renders the name object. This is the flag image
- `Q` does the opposite of `q`. ignored
- `BT` begins a text object. This is what we're interested in
- `<font name> <font size> Tf` sets the current font. We ignored this at first,
  but it turned out to be very important
- `# # Td` sets the text position. ignored
- `<text array> TJ` renders the hex strings in the array as text, adjusting the
  position of the chatacter by the numbers. We extracted the hex strings and
  ignored the positioning data

So we have 3 hex strings, but these don't translate to ascii. After some time,
we looked into how the fonts render. The first hex string uses font F3:

```
<< /BaseFont /MXUXTP+LMSans12-Regular-Identity-H /DescendantFonts [ 14 0 R ] /Encoding /Identity-H /Subtype /Type0 /ToUnicode 76 0 R /Type /Font >>
```

The Encoding Identity-H means that the encoding is 1-1, so we don't need to
worry about that. The ToUnicode reference, however is something we need to
worry about. Object 76 is some kind of font PDF metadata thing containing

```
71 beginbfchar
<001B> <0041>
<001C> <0061>
<0022> <0042>
<0023> <0062>
<0026> <007B>
<0027> <007D>
<002A> <0043>
...
```

When we mapped the characters in the hex string from the left column to the
right column for both of te fonts, we got:

```
Yeah!Youmadeitandfoundthemagicflag:kwa{butk:qexzqhxzijqrkuqnfggfIcqatc3qfIqxzdqsxxpqajh}
```

Well... That's something... The readable part was in font F3 and the unreadable
part was in font F6. After a couple hours staring at this and trying other
character replacements, we noticed a line in the metadata for font F6:

```
/CMapName /.-DejaVuSansCondensedmod.ttf,000-UTF16 def
```

All the other fonts had lines that looked like:

```
/CMapName /-usr-share-texlive-texmf-dist-fonts-opentype-public-lm-lmsans10-bold.otf,000-UTF16 def
```

The relative path and having it called "mod" seemed suspicious, so we extracted
the ttf file (object 99) and opened it in FontForge.

![glyphs that look like a rearranged flag]({{ site.baseurl }}/assets/hxp_pdf_fontforge.png)

That doesn't look like the alphabet. Since this was 6am, we decoded the flag by
hand, getting

```
hxp{Yeah!_y0u_f0und_The_missing_pag3_in_0ur_c00l_pdf}
```

## Code

### PDF parser

This code will parse the top-level objects and dump streams to files. It did a
lot more parsing and formatting, but none of that was relevant to the
challenge.

``` python
#!/usr/bin/python3
"""
PDF top-level parser

Don't expect this to work on anything except pdf.pdf
"""

import binascii
import zlib

WHITESPACE = [0, 0x9, 0xa, 0xc, 0xd, 0x20]

def count_whitespace(pdf):
    for pointer in range(len(pdf)):
        if pdf[pointer] not in WHITESPACE:
            return pointer

def parse_line(pdf):
    line_len = pdf.index(b'\n')
    return line_len+1, pdf[:line_len]

def parse_int(pdf):
    for pointer in range(len(pdf)):
        if pdf[pointer] in WHITESPACE:
            return pointer+1, int(pdf[:pointer])

def parse_float(pdf):
    for pointer in range(len(pdf)):
        if pdf[pointer] in WHITESPACE:
            return pointer+1, float(pdf[:pointer])

def parse_hex(pdf):
    if pdf[0:1] != b'<':
        raise ValueError('invalid hex')
    line_len = pdf.index(b'>')
    hex_data = binascii.unhexlify(pdf[1:line_len])
    return line_len+1, hex_data

def parse_name(pdf):
    if pdf[0:1] != b'/':
        raise ValueError('invalid name')
    for pointer in range(len(pdf)):
        if pdf[pointer] in WHITESPACE:
            return pointer+1, pdf[1:pointer]

def parse_obj(pdf):
    print(f'Parsing "{pdf[:16]}..."')
    # Comment
    if pdf[0:1] == b'%':
        comment_len, comment = parse_line(pdf)
        return comment_len, {'type': 'comment', 'contents': comment}

    # Name
    if pdf[0:1] == b'/':
        return parse_name(pdf)

    # startxref <<< not standard
    if pdf[1:10] == b'startxref':
        # TODO optional Trailer
        startxref_len, startxref = parse_int(pdf[11:])
        return 11 + startxref_len + 6, \
            {'type': 'startxref', 'contents': startxref}

    # Array
    if pdf[0:1] == b'[':
        pointer = 1 + count_whitespace(pdf[1:])
        d = list()
        while pdf[pointer:pointer+1] != b']':
            obj_len, obj = parse_obj(pdf[pointer:])
            d.append(obj)
            pointer += obj_len
        return pointer+2, {'type': 'array', 'contents': d}

    # Dictionary
    if pdf[0:2] == b'<<':
        pointer = 3
        d = dict()
        while pdf[pointer:pointer+2] != b'>>':
            key_len, key = parse_name(pdf[pointer:])
            value_len, value = parse_obj(pdf[pointer+key_len:])
            d[key] = value
            pointer += key_len + value_len
        pointer += 3
        if pdf[pointer:pointer+6] == b'stream':
            stream = pdf[pointer+7:pointer+7+d[b'Length']]
            pointer += 7 + d[b'Length'] + 10
            d['*stream'] = stream
        return pointer, {'type': 'dictionary', 'contents': d}

    # Reference
    try:
        len1, object_number = parse_int(pdf)
        len2, generation_number = parse_int(pdf[len1:])
        if pdf[len1+len2:len1+len2+1] != b'R':
            raise ValueError
        return len1+len2+2, \
            {
                'type': 'reference',
                'contents': None,
                'object_number': object_number,
                'generation_number': generation_number
            }
    except ValueError:
        pass

    # Indirect object
    try:
        len1, object_number = parse_int(pdf)
        len2, generation_number = parse_int(pdf[len1:])
        if pdf[len1+len2:len1+len2+3] != b'obj':
            raise ValueError
        inner_len, inner_obj = parse_obj(pdf[len1+len2+4:])
        return len1+len2+3+inner_len+8, \
            {
                'type': 'indirect object',
                'contents': inner_obj,
                'object_number': object_number,
                'generation_number': generation_number
            }
    except ValueError:
        pass

    # Hex string
    if pdf[0:1] == b'<':
        return parse_hex(pdf)

    # Integer
    try:
        return parse_int(pdf)
    except ValueError:
        pass

    # Float
    try:
        return parse_float(pdf)
    except ValueError:
        pass

    print('!!!Failed to parse!!!')
    raise Exception

def parse_pdf(pdf):
    # Parse PDF and extract objects into data
    data = []
    pointer = 0
    while pointer < len(pdf):
        try:
            obj_len, obj = parse_obj(pdf[pointer:])
        except Exception:
            print(f'Failed at {pointer}, "{pdf[pointer:pointer+16]}..."')
            raise
        pointer += obj_len
        data.append(obj)
    return data

def dump_objects(data):
    # Go through and dump parsed objects
    for obj in data:
        # We don't care if it's not an indirect object
        if obj['type'] != 'indirect object':
            print('not dumping:', obj)
            continue

        # Assume indirect objects are streams
        print('dumping indirect object:', obj['object_number'])
        assert obj['contents']['type'] == 'dictionary'
        assert '*stream' in obj['contents']['contents']
        obj_dict = obj['contents']['contents']
        stream = obj_dict['*stream']

        # De-Flate
        if b'Filter' in obj_dict and b'FlateDecode' in obj_dict[b'Filter']:
            stream = zlib.decompress(stream)

        # Dump to file
        with open(
                f'{obj["object_number"]}_{obj["generation_number"]}.bin',
                'wb') as f:
            f.write(stream)

if __name__ == '__main__':
    with open('pdf.pdf', 'rb') as f:
        pdf = f.read()
    data = parse_pdf(pdf)
    dump_objects(data)
```

### Vector image painter

Hawkheart wrote this to render line drawings from object 85. It shows a nice
flag, but not the flag we wanted :(

```python
import matplotlib.path as mpath
import matplotlib.patches as mpatches
import matplotlib.pyplot as plt

def cmykToRgb(c, m, y, k) :
    # Thanks to Stack Overflow for this function :)
    r = round(255.0 - ((min(1.0, c * (1.0 - k) + k)) * 255.0))
    g = round(255.0 - ((min(1.0, m * (1.0 - k) + k)) * 255.0))
    b = round(255.0 - ((min(1.0, y * (1.0 - k) + k)) * 255.0))
    return (r,g,b)

Path = mpath.Path

with open("85_0.bin") as f:
    s = f.read()

fig, ax = plt.subplots()

current_path = []
current_path_actions = []
current_color = (0, 0, 0)
linewidth = 1

for line in s.split('\n'):
    line = line.strip()
    line = line.rstrip()
    if not line:
        continue
    if line[-2:] == ' m':
        current_path_actions.append(Path.MOVETO)
        x,y,m = line.split()
        current_path.append((float(x), float(y)))
    elif line[-1] == "w":
        width, _ = line.split()
        linewidth = float(width)/2
    elif line[-2:] == 're':
        x, y, width, height, _ = line.split()
        x = float(x)
        y = float(y)
        width = float(width)
        height = float(height)
        current_path.extend([(x,y), (x+width,y),(x+width,y+height), (x, y+height), (x, y)])
        current_path_actions.extend([Path.MOVETO,Path.LINETO,Path.LINETO,Path.LINETO,Path.LINETO])
    elif line[-1] == 'k':
        c,m,y,k,_ = line.split()
        c = float(c)
        m = float(m)
        y = float(y)
        k = float(k)
        r,g,b =  cmykToRgb(c,m,y,k)
        current_color = (r/255.0, g/255.0, b/255.0)
    elif line[-1] == 'c':
        x1, y1, x2, y2, x3, y3, c = line.split()
        x1 = float(x1)
        x2 = float(x2)
        x3 = float(x3)
        y1 = float(y1)
        y2 = float(y2)
        y3 = float(y3)
        current_path.append((x1, y1))
        current_path.append((x2, y2))
        current_path.append((x3, y3))
        current_path_actions.extend([Path.CURVE4]*3)
    elif line[-1] == "l":
        x, y, l = line.split()
        x = float(x)
        y = float(y)
        current_path.append((x,y))
        current_path_actions.append(Path.LINETO)
    if line[0] in ["W", "S"]:
        current_path.append(current_path[0])
        current_path_actions.append(Path.CLOSEPOLY)
        patch = mpatches.PathPatch(Path(current_path, current_path_actions), transform=ax.transData, linewidth=linewidth)
        ax.add_patch(patch)
        current_path = []
        current_path_actions = []
    elif line[0] == 'f':
        current_path.append(current_path[0])
        current_path_actions.append(Path.CLOSEPOLY)
        patch = mpatches.PathPatch(Path(current_path, current_path_actions), fc=current_color,transform=ax.transData, fill=True,linewidth=0)
        ax.add_patch(patch)
        current_path = []
        current_path_actions = []

ax.plot()
plt.show()
```

repository: RPISEC/website

gha_language: Verilog

repo_name: RPISEC/website


text:
# SSDP Chapter Activity Tracker
## v3.0.0 - Flask Rebuild

> A web application to track activity in the [Students For Sensible Drug Policy](https://ssdp.org) network
> The Flask CAT was built with Flask v1.0 and Python 3.

## Setup
These instructions apply to our setup on Dreamhost and may vary depending on your web hosting services.

1. Set up and fully host a new domain with Dreamhost. Make sure the server can run Python apps. For DH, just check off the "Passenger" setting.
2. Clone the repository into the directory that looks like something.domain.com. I set mine up as `cat.ssdp.org/cat`.

  ```
  git clone https://github.com/SSDP-Dev/flask-cat.git cat
  ```

3. [Install a custom version of Python 3.](https://help.dreamhost.com/hc/en-us/articles/115000702772-Installing-a-custom-version-of-Python-3)
4. [Set up a new virtualenv](https://help.dreamhost.com/hc/en-us/articles/115000695551-Installing-and-using-virtualenv-with-Python-3) in a separate directory inside something.domain.com/cat. I set mine up as `cat.ssdp.org/cat/venv`.
5. With your virtualenv activated (refer to step 4 for help), install [Flask](http://flask.pocoo.org/) with `pip3 install Flask`
6. You'll want to create a `passenger_wsgi.py` file. There's an example in the root of this repo, which you should be able to move into `something.domain.com/`. You may need to change some of the settings in there, like the INTERP variable (set it to the python3 interpreter you want to use - most likely `cat/venv/bin/python3`) and the systempath (make sure you append it with the app name, most likely `cat`).
7. In that same directory, do the following

  ```
  mkdir tmp
  touch tmp/restart.txt
  ```

  - Every time you make changes to your Passenger config, touch `tmp/restart.txt`

That should do the trick.

**Important Notes**
This repo ships with a standard database. It includes only one entry: a user with username 'admin' and password 'admin'. For privacy reasons, we don't list anything else in there. You'll have to fill in the rest, but you'll definitely want to change your password.

## Development
You'll want to set up a Flask development environment to work on this. You can learn more about the steps to do so in the [Flask Documentation](http://flask.pocoo.org/docs/1.0/installation/#installation).

```
├── admin.py
├── auth.py
├── blog.py
├── chapters.py
├── db.py
├── __init__.py
├── schema.sql
├── static
│   ├── bootstrap.css
│   ├── bootstrap.min.js
│   ├── custom.css
│   ├── logo-transparent.png
│   └── SSDP20-logo-transparent.png
└── templates
    ├── admin
    │   ├── activities.html
    │   ├── categories.html
    │   ├── index.html
    │   ├── spending.html
    │   ├── stats.html
    │   ├── user-edit.html
    │   ├── user-list.html
    │   └── users.html
    ├── auth
    │   ├── login.html
    │   └── register.html
    ├── base.html
    ├── blog
    │   ├── available-activities.html
    │   ├── create.html
    │   ├── faq.html
    │   ├── index.html
    │   ├── leaderboard.html
    │   ├── store.html
    │   └── update.html
    └── chapters
        ├── chapter.html
        └── index.html
```
## Python Classes

### admin.py
This is where all the administrative functions and pages live. The index, at /admin, is a landing page for all those pages and functions. It should just be a column of buttons that include links and fuctions for users to control their CAT experience.

Users with the permissions 'Admin' can see all of the controls. 'Staffer' gets fewer. 'Chapter' gets the least.

This class controls:
- Creating unique URLs for user pages with `makeURL()`
- Generating the stats page with `stats()`
- Adding users with `users()`
- Adding point categories with `categories()`
- Logging points with `activities()`
- Spending points with `spending()`
- One-off commands with `command()`
  - `RESET` will reset the stats in the user table
  - `BACKUP` will create a backup of the database. There's notes in the documentation in this class, but for good measure: you need to have a folder called `/backups` in your root directory for this to work. Otherwise it won't. Planning some funcationality to hook into a Google Drive or DropBox or something
- Editing user information through the `userList()` and `userEdit()` methods

### auth.py
This class controls logging in and logging out. It's almost a direct duplicate of the Flaskr tutorial login system.

I pretty much didn't change anything from the tutorial [here](http://flask.pocoo.org/docs/1.0/tutorial/) - so not much to say about it. If you want to dig in to how things are working, check out the tutorial and feel free to make changes there. Login systems are beyond my abilities at the moment, so I won't make many changes here.

### blog.py
This class came from the [Flaskr tutorial](http://flask.pocoo.org/docs/1.0/tutorial/) as well.

The tutorial gave me a few things like creating 'blog posts' and editing them, so I kept the bulk of that to use with actions.

I never changed it from `blog` because I didn't want to create any weird bugs. Maybe later on down the road we'll clean it up in refactoring. For now, here's what we do:

- Control the homepage for the entire site with `index()`
- Control the creation of action with `create()`
  - This is **not** how we're creating actions. We could, but we're doing it better in admin.activities(). Leaving this in for now, but will come back and remove it during refactoring and cleanup
- Retrieve actions through `get_action()`
- Update actions with `update()`
- Delete actions with `delete()`
- Render a leaderboard with `leaderboard()`
- Render the available activities page through `availableActivities()`
- Render the FAQ with `faq()`
- Render the store with `store()`
  - This may need to get merged or changed with the spending function in the admin class

Lots of these methods are good boilerplates for manipulating the database in other ways, so I've kept a lot of it pretty close to the tutorial.

### chapters.py
This class provides methods to work with chapter data. It's a bit more front facing.

Here's what this class does:

- Displays all of the chapters in a page with `index()`
- Return a chapter from the database with `get_chapter()`
- Displays a chapter page with `chapter()`

### db.py
This class was also provided by the [Flaskr tutorial](http://flask.pocoo.org/docs/1.0/tutorial/). In the same fashion, I really didn't make any changes here.

The biggest thing to note is that I was having trouble getting click to work and couldn't make new CLI commands. That was a hassle during development, so I just overrode the one CLI command that did work.

There are some comments in `init_db()` that go over those overrides and how I used them.

### init.py
Another class that came from the [Flaskr tutorial](http://flask.pocoo.org/docs/1.0/tutorial/). My understanding is this class initializes the Flask app and registers blueprints and URL rules. I've only added here what I needed for Blueprints and URLs.

## SQL files
### schema.sql
This is the SQL schema for the SQLite database. It's the first database I've ever designed from the ground up, so could be refined. I'm somewhat hopeful I built it in such a way that it's at least easy to extend.

## Static assets
We've got the bootstrap CSS and JS, along with a custom CSS file here and other assets. Pretty straightforward. Any other required assets should live in this directory.

## Templates
Flask uses the Jinja2 templating language for HTML templates.

### base.html
This template provides the `<head>`, `<header>`, and `<html>` tags, along with the nav bar.

In the future, we might pull out some of those components for better structure.

### admin/activities.html
This template is where we log activities.

### admin/categories.html
This page allows us to create new types of actions

### admin/index.html
Landing page for the control panel for all types of users.

### admin/spending.html
This template allows us to create new spending.

### admin/stats.html
Renders a stats page. Most of the interesting work is done on the Python side for this one, and the HTML just renders the values.

### admin/user-edit.html
Allows us to edit some values for each user.

### admin/user-list.html
Renders a list of users and their user-edit links. In the future this will include a search function. For now, ctrl+f will have to do.

### admin/users.html
Allows us to add new users.

### auth/login.html
Login page - provided by Flask tutorial

### auth/register.html
Register page - disabled in our app for now. Will likely need to remove or consider enabling this in the future.

### blog/available-activities.html
Lists the available activities

### blog/create.html
Used to create blog posts/actions. Disabled in our app, and should consider re-enabling it or removing it in a refactor.

### blog/faq.html
Displays the FAQ page.

### blog/index.html
Default home page of the site. Right now displays an activity feed. In the future, it could display more. One TODO here is adding links to the chapter pages and links to the activity descriptions in the feed.

### blog/leaderboard.html
Displays the leaderboard

### blog/store.html
Displays the store. As stated in the control documentation for this - we have duplicative code here and could probably stand to remove either this or the control panel option.

### blog/update.html
Allows us to update action information.

### chapters/chapter.html
Single chapter page, displays stats and an activity feed for that chapter.

### chapters/index.html
Displays a list of all the chapters and includes links to their chapter pages.

## TODO:
- PEP8 standards
- WCAG standards
- Email notifications for staffers, chapters, etc.
- Twitter integration for activity feed
- Mozilla Observatory changes
- Automate points to Master List and/or NationBuilder
- Consistent login reqs across pages
- Learn D3 and use it to render cool stats and stuff
- Added Flask-SSLify requirement. May need to add that in setup instructions
- Clean up the database that we ship with, figure out good way to distribute venv, etc.
- Document new templates, methods, etc. that haven't made it in since the first documentation session.
- Add note about Google Analytics

repository: SSDP-Dev/flask-cat

branch_name: refs/heads/master

extension: md

gha_language: HTML

license_type: permissive

repo_name: SSDP-Dev/flask-cat


text:
# Lophiostoma acrostichi (K.D. Hyde) Aptroot & K.D. Hyde, 2002 SPECIES

#### Status
ACCEPTED

#### According to
The Catalogue of Life, 3rd January 2011

#### Published in
in Hyde, Wong & Aptroot, Fungal Diversity Res. Ser. (Hong Kong) 7: 106 (2002)

#### Original name
Massarina acrostichi K.D. Hyde, 1989

### Remarks
null

repository: mdoering/backbone

repo_name: mdoering/backbone


text:
# Domain Event Bundle

![Build Status](https://github.com/headsnet/domain-events-bundle/actions/workflows/ci.yml/badge.svg)
[![Latest Stable Version](https://poser.pugx.org/headsnet/domain-events-bundle/v)](//packagist.org/packages/headsnet/domain-events-bundle)
[![Total Downloads](https://poser.pugx.org/headsnet/domain-events-bundle/downloads)](//packagist.org/packages/headsnet/domain-events-bundle)
[![License](https://poser.pugx.org/headsnet/domain-events-bundle/license)](//packagist.org/packages/headsnet/domain-events-bundle)

DDD Domain Events for Symfony, with a Doctrine based event store.

This package allows you to dispatch domain events from within your domain
model, so that they are persisted in the same transaction as your aggregate.

These events are then published using a Symfony event listener in the
`kernel.TERMINATE` event.

This ensures transactional consistency and guaranteed delivery via the Outbox
pattern.

_Requires Symfony 4.4 or higher_

### Installation

```bash
composer require headsnet/domain-events-bundle
```

(see [Messenger Component](#messenger-component) below for prerequisites)

### The Domain Event Class

A domain event class must be instantiated with an aggregate root ID.

You can add other parameters to the constructor as required.

```php
use Headsnet\DomainEventsBundle\Domain\Model\DomainEvent;
use Headsnet\DomainEventsBundle\Domain\Model\Traits\DomainEventTrait;

final class DiscountWasApplied implements DomainEvent
{
    use DomainEventTrait;

    public function __construct(string $aggregateRootId)
    {
        $this->aggregateRootId = $aggregateRootId;
        $this->occurredOn = (new \DateTimeImmutable)->format(DateTime::ATOM);
    }
}
```

### Recording Events

Domain events should be dispatched from within your domain model - i.e. from
directly inside your entities.

Here we record a domain event for entity creation. It is then automatically
persisted to the Doctrine `event`
database table in the same database transaction as the main entity is persisted.

```php
use Headsnet\DomainEventsBundle\Domain\Model\ContainsEvents;
use Headsnet\DomainEventsBundle\Domain\Model\RecordsEvents;
use Headsnet\DomainEventsBundle\Domain\Model\Traits\EventRecorderTrait;

class MyEntity implements ContainsEvents, RecordsEvents
{
	use EventRecorderTrait;

	public function __construct(PropertyId $uuid)
    	{
    	    $this->uuid = $uuid;

    	    // Record a domain event
    	    $this->record(
    		    new DiscountWasApplied($uuid->asString())
    	    );
    	}
}
```

Then, in `kernel.TERMINATE` event, a listener automatically publishes the domain
event on to the `messenger.bus.event` event bus for consumption elsewhere.

### Amending domain events

Even though events should be treated as immutable, it might be convenient
to add or change meta data before adding them to the event store.

Before a domain event is appended to the event store,
the standard Doctrine event store emits a `PreAppendEvent` Symfony event,
which can be used e.g. to set the actor ID as in the following example:

```php
use App\Entity\User;
use Headsnet\DomainEventsBundle\Doctrine\Event\PreAppendEvent;
use Symfony\Component\EventDispatcher\EventSubscriberInterface;
use Symfony\Component\Security\Core\Security;

final class AssignDomainEventUser implements EventSubscriberInterface
{
    private Security $security;

    public function __construct(Security $security)
    {
        $this->security = $security;
    }

    public static function getSubscribedEvents(): array
    {
        return [
            PreAppendEvent::class => 'onPreAppend'
        ];
    }

    public function onPreAppend(PreAppendEvent $event): void
    {
        $domainEvent = $event->getDomainEvent();
        if (null === $domainEvent->getActorId()) {
            $user = $this->security->getUser();
            if ($user instanceof User) {
                $domainEvent->setActorId($user->getId());
            }
        }
    }
}
```

### Deferring Events Into The Future

If you specify a future date for the `DomainEvent::occurredOn` the event will
not be published until this date.

This allows scheduling of tasks directly from within the domain model.

#### Replaceable Future Events

If an event implements `ReplaceableDomainEvent` instead of `DomainEvent`,
recording multiple instances of the same event for the same aggregate root will
overwrite previous recordings of the event, as long as it is not yet published.

For example, say you have an aggregate _Booking_, which has a future
_ReminderDue_ event. If the booking is then modified to have a different
date/time, the reminder must also be modified. By implementing
`ReplaceableDomainEvent`, you can simply record a new _ReminderDue_ event, and
providing that the previous _ReminderDue_ event had not been published, it will
be removed and superseded by the new _ReminderDue_ event.

### Event dispatching

By default only the DomainEvent is dispatched to the configured event bus.

You can overwrite the default event dispatcher with your own implementation to
annotate the message before dispatching it, e.g. to add an envelope with custom stamps.

Example:

```yaml
services:
    headsnet_domain_events.domain_event_dispatcher_service:
        class: App\Infrastructure\DomainEventDispatcher
```

```php
class PersonCreated implements DomainEvent, AuditableEvent
{
    …
}
```

```php
class DomainEventDispatcher implements \Headsnet\DomainEventsBundle\EventSubscriber\DomainEventDispatcher
{
    private MessageBusInterface  $eventBus;

    public function __construct(MessageBusInterface $eventBus)
    {
        $this->eventBus = $eventBus;
    }

    public function dispatch(DomainEvent $event): void
    {
        if ($event instanceof AuditableEvent) {
            $this->eventBus->dispatch(
                new Envelope($event, [new AuditStamp()])
            );
        } else {
            $this->eventBus->dispatch($event);
        }
    }
}
```

### Messenger Component

By default, the bundle expects a message bus called `messenger.bus.event` to be
available.
This can be configured using the bundle configuration - see
[Default Configuration](#default-configuration).

```yaml
framework:
    messenger:
        …

        buses:
            messenger.bus.event:
                # Optional
                default_middleware: allow_no_handlers
```

[Symfony Messenger/Multiple Buses](https://symfony.com/doc/current/messenger/multiple_buses.html)

### Doctrine

The bundle will create a database table called `event` to persist the events
before dispatch. This allows a permanent record of all events raised.

The database table name can be configured - see *Default Configuration* below.

The `StoredEvent` entity also tracks whether each event has been published to
the bus or not.

Finally, a Doctrine DBAL custom type called `datetime_immutable_microseconds` is
automatically registered. This allows the StoredEvent entity to persist events
with microsecond accuracy. This ensures that events are published in the exact
same order they are recorded.

### Legacy Events Classes

During refactorings, you may well move or rename event classes. This will
result in legacy class names being stored in the database.

There is a console command, which will report on these legacy event classes
that do not match an existing, current class in the codebase (based on the
Composer autoloading).

```
bin/console headsnet:domain-events:name-check
```

You can then define the `legacy_map` configuration parameter, to map old,
legacy event class names to their new replacements.

```yaml
headsnet_domain_events:
    legacy_map:
        App\Namespace\Event\YourLegacyEvent1: App\Namespace\Event\YourNewEvent1
        App\Namespace\Event\YourLegacyEvent2: App\Namespace\Event\YourNewEvent2
```

Then you can re-run the console command with the `--fix` option. This will
then update the legacy class names in the database with their new references.

There is also a `--delete` option which will remove all legacy events from
the database if they are not found in the legacy map.

### Default Configuration

```yaml
headsnet_domain_events:
    message_bus:
        name: messenger.bus.event
    persistence:
        table_name: event
    legacy_map: []
```

### Contributing

Contributions are welcome. Please submit pull requests with one fix/feature per
pull request.

Composer scripts are configured for your convenience:

```
> composer test       # Run test suite
> composer cs         # Run coding standards checks
> composer cs-fix     # Fix coding standards violations
> composer static     # Run static analysis with Phpstan
```

license: MIT

repository: headsnet/domain-events-bundle

gha_language: PHP

gha_license_id: MIT

repo_name: headsnet/domain-events-bundle


text:
# geostyler-shapefile-parser

[![Greenkeeper badge](https://badges.greenkeeper.io/geostyler/geostyler-shapefile-parser.svg)](https://greenkeeper.io/)
[![Build Status](https://travis-ci.com/geostyler/geostyler-shapefile-parser.svg?branch=master)](https://travis-ci.com/geostyler/geostyler-geojson-parser)
[![Coverage Status](https://coveralls.io/repos/github/geostyler/geostyler-shapefile-parser/badge.svg?branch=master)](https://coveralls.io/github/geostyler/geostyler-shapefile-parser?branch=master)

[GeoStyler](https://github.com/geostyler/geostyler/) Data Parser implementation for Shapefile

## Funding & financial sponsorship

Maintenance and further development of this code can be funded through the
[GeoStyler Open Collective](https://opencollective.com/geostyler). All contributions and
expenses can transparently be reviewed by anyone; you see what we use the donated money for.
Thank you for any financial support you give the GeoStyler project 💞

repository: geostyler/geostyler-shapefile-parser

gha_language: TypeScript

repo_name: geostyler/geostyler-shapefile-parser


text:
*Quick links :*
***
[Home](/README.md) - [**Part 1**](/part1/README.md) - [Part 2](/part2/README.md) - [Part 3](/part3/README.md) - [Resources](/additionalResources/README.md)
***
**Part 1** - [Mesh Networks](MESH.md) - [**Build a Mesh network**](PIMESH.md) - [Network access](ROUTE.md)
***

# Part 1 - Mesh Networks

In this part of the workshop you will build your own mesh network over WiFi using Raspberry Pi computers.  You need a minimum of 2 Raspberry Pi computers, but can optionally add as many Pis as you want to make a larger mesh network.

## batman-adv

To create the mesh network on the Raspberry Pi we are using [batman-adv](https://www.open-mesh.org/projects/open-mesh/wiki), which is part of the standard linux kernel.  We are going to configure the batman-adv kernel module to take control of the WiFi interface **wlan0** and create a mesh network over WiFi.  Batman-adv will then create a new interface **bat0** to allow the Pi to send network traffic over the mesh network.  This will be explored more in the section on [network access](ROUTE.md)

You need to complete the following steps on all the Raspberry Pis that you want to be part of the mesh network, including the gateway and bridge nodes.  You can choose to use [headerless setup](/additionalResources/HEADERLESS_SETUP.md) or a keyboard, mouse and monitor to access the Pi command line.

## Create the SD card and perform initial setup

1. Download the latest Raspbian image from [https://www.raspberrypi.org/downloads/raspbian/](https://www.raspberrypi.org/downloads/raspbian/).  Choose the Raspbian Buster Lite version.
2. Flash the image to an SD card suitable for your Raspberry Pi.  Instructions are available [here](https://www.raspberrypi.org/documentation/installation/installing-images/README.md) if needed.
3. If you are doing a [headerless setup](/additionalResources/HEADERLESS_SETUP.md) insert the SD card back into your computer.  Open a command line or terminal on your computer and change to the directory where the SD card boot partition is mounted and create an empty file in the boot partition called ssh:
    - *Linux* : ```touch ssh```
    - *MacOS* : ```touch ssh```
    - *Windows command prompt* : ```type NUL >> ssh```
    - *Windows PowerShell* : ```echo $null >> ssh```

        *optionally, create the wpa_supplicant.conf file or modify the cmdline.txt and config.txt files if you need OTG mode or WiFi connectivity*

    Eject the SD card from your operating system and remove the card from your computer.
4. Insert the SD card into the Raspberry Pi and then power on the Raspberry Pi.
5. Login to the pi with user **pi** and password **raspberry**.  If using headerless setup then connect via [ssh](/additionalResources/COMMAND_LINE_ACCESS.md).  The hostname on first boot is **raspberrypi.local**.
6. On the Raspberry Pi command line issue the command

    ```sudo raspi-config```

    and then go through and change the following settings:
    - Change the user password (don't forget it, as you will need it every time you remotely connect to the Pi)
    - Network Options - Hostname
    - Localisation Options - set Locale, Timezone and WiFi country to match your location
    - Network Option - WiFi.  If your pi is not connected to the internet already, use this option to setup WiFi connectivity to ensure your Pi has access to the internet
    - interfacing Options - SSH, ensure SSH server is enabled

    Exit raspi-config, don't reboot yet.
7. Issue command

    ```sudo apt-get update && sudo apt-get upgrade -y```
8. Reboot the Raspberry Pi with command

    ```sudo reboot -n```

## Setup batman-adv

Once the Pi has rebooted, get to the command line (remember your pi now has a new hostname and the pi user has a new password that you set in the previous section).  If connecting via ssh, the ssh command line command is:

```ssh<EMAIL_ADDRESS>
replacing **hostname** with the name you specified.

Perform the following on the pi command line:

1. To manage the mesh network, a utility called **batctl** needs to be installed.  This can be done using command

    ```sudo apt-get install -y batctl```

2. Using your preferred editor create a file ~/start-batman-adv.sh

    e.g.
    - ```vi ~/start-batman-adv.sh```
    - ```nano ~/start-batman-adv.sh```

    the file should contain the following:

    ```text
    #!/bin/bash
    # batman-adv interface to use
    sudo batctl if add wlan0
    sudo ifconfig bat0 mtu 1468

    # Tell batman-adv this is a gateway client
    sudo batctl gw_mode client

    # Activates batman-adv interfaces
    sudo ifconfig wlan0 up
    sudo ifconfig bat0 up
    ```

3. Make the start-batman-adv.sh file executable with command :

    ```text
    chmod +x ~/start-batman-adv.sh
    ```

4. Create the network interface definition for the wlan0 interface by creating a file as root user e.g.

    - ```sudo vi /etc/network/interfaces.d/wlan0```
    - ```sudo nano /etc/network/interfaces.d/wlan0```

    then add the following content:

    ```text
    auto wlan0
    iface wlan0 inet manual
        wireless-channel 1
        wireless-essid call-code-mesh
        wireless-mode ad-hoc
    ```

    You can replace:

    - the channel number with a [valid 2.4 GHz WiFi channel number for your region](https://en.wikipedia.org/wiki/List_of_WLAN_channels) (most regions support channels 1 to 11)
    - the essid with a network name of your choosing

    However, these values must be the same on ALL devices that will form your mesh network.

5. Ensure the batman-adv kernel module is loaded at boot time by issuing the following command :

    ```text
    echo 'batman-adv' | sudo tee --append /etc/modules
    ```

6. Stop the DHCP process from trying to manage the wireless lan interface by issuing the following command :

    ```text
    echo 'denyinterfaces wlan0' | sudo tee --append /etc/dhcpcd.conf
    ```

7. Make sure the startup script gets called by editing file **/etc/rc.local** as root user, e.g.

    - ```sudo vi /etc/rc.local```
    - ```sudo nano /etc/rc.local```

    and insert:

    ```text
    /home/pi/start-batman-adv.sh &
    ```

    before the last line: **exit 0**
8. If this pi will not be a bridge or gateway node then shut it down using command :

    ```text
    sudo shutdown -h now
    ```

You now have all the raspberry pi systems configured to join the mesh, so proceed to the [next section](ROUTE.md) to setup access to the Internet and also enable a bridge.

***
*Quick links :*
***
**Part 1** - [Mesh Networks](MESH.md) - [**Build a Mesh network**](PIMESH.md) - [Network access](ROUTE.md)
***
[Home](/README.md) - [**Part 1**](/part1/README.md) - [Part 2](/part2/README.md) - [Part 3](/part3/README.md) - [Resources**](/additionalResources/README.md)

repository: binnes/WiFiMeshRaspberryPi

repo_name: binnes/WiFiMeshRaspberryPi


text:
---
layout: post
title: "Self-Supervised Monocular Image Depth Learning and Confidence Estimation"
date: 2018-03-14 22:59:01
categories: arXiv_CV
tags: arXiv_CV CNN Relation
author: Long Chen, Wen Tang, Nigel John
mathjax: true
---

* content
{:toc}

##### Abstract
Convolutional Neural Networks (CNNs) need large amounts of data with ground truth annotation, which is a challenging problem that has limited the development and fast deployment of CNNs for many computer vision tasks. We propose a novel framework for depth estimation from monocular images with corresponding confidence in a self-supervised manner. A fully differential patch-based cost function is proposed by using the Zero-Mean Normalized Cross Correlation (ZNCC) that takes multi-scale patches as a matching strategy. This approach greatly increases the accuracy and robustness of the depth learning. In addition, the proposed patch-based cost function can provide a 0 to 1 confidence, which is then used to supervise the training of a parallel network for confidence map learning and estimation. Evaluation on KITTI dataset shows that our method outperforms the state-of-the-art results.

##### Abstract (translated by Google)
卷积神经网络（CNN）需要大量的具有地面真实注释的数据，这是一个具有挑战性的问题，它限制了许多计算机视觉任务的CNNs的开发和快速部署。我们提出了一种新颖的基于单目图像的深度估计框架，并且具有相应的自我监督方式的信心。基于零均值归一化互相关（ZNCC）提出了一种全差分基于块的成本函数，该算法将多尺度贴片作为匹配策略。这种方法大大提高了深度学习的准确性和鲁棒性。此外，所提出的基于补丁的成本函数可以提供0到1的置信度，然后用它来监督并行网络的训练以进行置信度学习和估计。对KITTI数据集的评估表明，我们的方法胜过了最先进的结果。

##### URL
[https://arxiv.org/abs/1803.05530](https://arxiv.org/abs/1803.05530)

##### PDF
[https://arxiv.org/pdf/1803.05530](https://arxiv.org/pdf/1803.05530)


text:
binary-clock
============
A LED matrix binary clock built with a Raspberry Pi. The python code uses MAX7219 driver by Richard Hull https://github.com/rm-hull/max7219 and forecast.io wrapper by Ze'ev Gilovitz https://github.com/ZeevG/python-forecast.io

The binary clock displays weather information under the following conditions:

* Weather alerts are active for the area
* Current weather condition is rain, snow, sleet, wind or fog
* If precipitation is expected in the next hour

![Raspberry PI binary clock](https://raw.githubusercontent.com/sharjeelaziz/binary-clock/master/0.jpg)

Pre-requisites
--------------
Ensure that the SPI kernel driver is enabled:

	pi@binclock ~/binary_clock $ dmesg | grep spi
	[    9.443124] bcm2708_spi bcm2708_spi.0: master is unqueued, this is deprecated
	[    9.593357] bcm2708_spi bcm2708_spi.0: SPI Controller at 0x20204000 (irq 80)

And that the devices are successfully installed in /dev:

	pi@binclock ~/binary_clock $ ls -l /dev/spi*
	crw-rw---T 1 root spi 153, 0 Dec 31  1969 /dev/spidev0.0
	crw-rw---T 1 root spi 153, 1 Dec 31  1969 /dev/spidev0.1

Refer to http://www.raspberrypi.org/documentation/hardware/raspberrypi/spi/README.md if the devices do not appear before proceeding.

GPIO pin-outs
-------------
The breakout board has an two headers to allow daisy-chaining:

| Board Pin | Name | Remarks | RPi Pin | RPi Function |
|--------:|:-----|:--------|--------:|--------------|
| 1 | VCC | +5V Power | 2 | 5V0 |
| 2 | GND | Ground | 6 | GND |
| 3 | DIN | Data In | 19 | GPIO 10 (MOSI) |
| 4 | CS | Chip Select | 24 | GPIO 8 (SPI CS0) |
| 5 | CLK | Clock | 23 | GPIO 11 (SPI CLK) |

Weather
-------
You need an API key to use it (http://developer.forecast.io). The key is free for 1000 calls per day. To use the key rename config.json.sample to config.json and update the api_key and location with your values.

References
----------
* https://github.com/rm-hull/max7219
* https://github.com/ZeevG/python-forecast.io
* http://www.raspberrypi.org/documentation/hardware/raspberrypi/spi/README.md
* http://developer.forecast.io
* https://developer.forecast.io/docs/v2#data-points

License
-------
Copyright 2014 Sharjeel Aziz (Shaji)

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

	http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

repository: sharjeelaziz/binary-clock

repo_name: sharjeelaziz/binary-clock


text:
# Retrieve office 365 user details and give permission thru c# using EWS and Pow
## Requires
* Visual Studio 2012
## License
* Apache License, Version 2.0
## Technologies
* scripting
* Windows PowerShell
## Topics
* C#
* Powershell
* Office 365
* EWS API
## IsPublished
* True
## ModifiedDate
* 2013-07-05 02:50:20
## Description

O365 Delegates (CSEWSSol)

Introduction

Office 365 Delegates Tool has been developed to help Office 365 Exchange administrators to perform get mailbox and the calendar delegate task for the mailboxes in the cloud. Also developers and IT Professional will get benefit out of
 the sample.

The tool can be used to:

ü
Find existing distribution list members.

ü
Allow others to manage your mailbox and calendar by giving delegate permission.

ü
List delegate permission.

ü
Remove delegate permission.

ü
Modify delegate permission level.

ü
Grant & Revoke "Full Mailbox" and "Send As" Permission.

ü
Run various Office 365 PowerShell commands to retrieve user/mailbox details.

ü
It is provisioned to get LDP dump of the on-premise user through PowerShell.

As the tool can be used to perform the task that can be done with other application and thru manual PowerShell, but with less effort, it is named as Office 365 Delegates.

Delegates' synonyms: The Person authorized to act as representative for another; a deputy or an agent.

Building the Sample

Pre-Requisites:

Operating system: Use Windows 7, Windows 8, Windows Server 2008 R2, or Windows Server 2012.

Microsoft .NET Framework: By Default, the latest OS windows includes .Net Framework. If you do not have, you must install the Microsoft .NET Framework 4.0 feature and above.

Related Link:
http://www.microsoft.com/en-us/download/details.aspx?id=17851

Install the Windows Management Framework:

By Default, the latest OS Windows includes WinRM and PowerShell. If you do not have, you should download and install the Windows Management Framework. Choose the package that includes
 Windows PowerShell v2 and WinRM 2.0, and that apply to your operating system, system architecture, and language. After you install WinRM and Windows PowerShell, configure the software to work correctly as described in the next steps.
Note If your local computer is protected by a Microsoft Internet Security and Acceleration (ISA) server, you may have to install the Windows Firewall Client or configure a proxy server on your local computer to connect Windows PowerShell to the cloud-based
 service. For more information, see Windows PowerShell: FAQs for Administrators.

Install Microsoft Online Services Sign-in Assistant: You must install the appropriate version of the Microsoft Online Services Sign-in Assistant for your operating system from the Microsoft Download Center. Microsoft Online Services
 Sign-In Assistant for IT Professionals RTW.

Install the Windows Azure AD Module for Windows PowerShell: You must install the appropriate version of the Windows Azure AD Module for Windows PowerShell for your operating system from the Microsoft Download Center:

o
Windows Azure Active Directory Module for Windows PowerShell (32-bit version)

o
Windows Azure Active Directory Module for Windows PowerShell (64-bit version)

Running the Sample

•
How to Use the Tool

It is a stand-alone tool and is multi-instance capable. The tool cannot be run in DOS/Command Prompt as it is GUI based. It creates a Desktop shortcut after installation. The tool tested in Windows 7, Windows 8, Windows 2008 R2 and Windows
 2012.

When you double-click the tool short-cut or the application file, the tool presents you with Disclaimer. If you are fine with the disclaimer, click "I Accept". Then it will display the
Authentication screen as shown in Picture 1.

Picture 1: Authentication screen

The Screen let you input logon details and it is self-explanatory. There are two buttons along with logon input options,

Buttons are,

AutoDiscover (WS):
It will use the logon details on the screen to makes autodiscover call to find the Exchange Web Service (EWS) URL and establish EWS session. Once the session established, you can use option in tab titled with (WS). As only
 EWS session is setup, you will not be able to use the option in tab titled with (PS).
8

PowerShell (PS): It will use the logon details on the screen to establish PowerShell Session and Import Exchange and
MSOnline Modules. Once the session established, you can use option in tab titled with (PS). As only PS session is setup, you will not be able to use the option in tab titled with (WS).

Note: Depending on the internet throughput, the tool takes time to create Remote PowerShell / EWS Session.

In order to discover EWS endpoint, you have to provide the Email ID of any cloud mailbox user with
Impersonation permission, the password and the tenant domain then click buttons.

You can refer to the following TechNet link for configuring Exchange Impersonation for all users:
http://msdn.microsoft.com/en-us/library/bb204095(EXCHG.140).aspx.

The task buttons in each tab will be enabled only if the EWS endpoints discover process succeeds or the Remote PowerShell Session gets through, otherwise you will not be able to use the tool as EWS Endpoint/Remote PowerShell are crucial
 to the functionality of the tool.

During the discover/ Remote PowerShell process, the button title changes to "Processing…" You have to wait until it changes back to its original title. Once the sessions are successfully created,
 you can use the tool to perform various tasks.

•
How Tool Works

As an Office 365 Exchange Administrator, if you want to perform any activity on the mailboxes that are in the cloud, you will use
Remote PowerShell or Online Portal. Both connect to the remote server over the internet.

Similarly, the Office 365 Delegates tool requires internet connection to connect to Office 365 servers.

The tool has to connect to EWS or Remote
Powershell to perform the tasks. It uses the Autodiscover
feature of Exchange to return EWS endpoint to communicate with the Exchange server. By default, direct EWS access is enabled for all Exchange Online plans except for the
Kiosk plan. So, any mailbox user in the cloud can discover EWS URL using
Autodiscover. You do not need any additional permission. However, if you wish to do other tasks in the tool thru EWS, you need to have
Impersonation permission. When you want to perform a delegate task, the tool makes EWS call to the Exchange Server. So it is crucial to use account with
Impersonation permission.

You can refer to the following TechNet link for configuring Exchange Impersonation for all users:
http://msdn.microsoft.com/en-us/library/bb204095(EXCHG.140).aspx.

Secondly the tool also uses Remote PowerShell to perform various SET & GET tasks. It is expected that the machine that runs the tool installed with pre-requisites for the commands to function. For Example: Get-msoluser
 command would require additional module to run. So please refer the pre-requisite section in the documentation before using the tool.

Note: I have titled the fields/options as per Exchange Terminology used with Exchange Shell for the easier understanding.

The options in various tabs are un-uniform. It was done intentionally as I wanted to display the different coding style to help IT Pros/Developers.

Note: The EWS tasks being done may take time on the mailboxes on the cloud. Even if the tool completed the task, the replication on the cloud may be still working. When you attempt to perform the activity on the mailbox for which
 the replication on the cloud, you may get message as "Previous action on the mailbox is still in process. Please try after sometime".

Note: The tool duplicates PowerShell behavior while executing the commands. That means, if there is no output shown to any of the command executed, you will have no detail shown in output file.

Using the Code

The code used in the sample can be used for developing applications/tool using EWS API or PowerShell to work on Office 365 mailboxes. I intentionally did not follow the uniformity in the tool to help the developers to get
 an idea on sample code for different requirement.

Although there are many ways to call the PowerShell commands, I used the method to build the PS1 file thru C#, and then execute it.

C#

Edit|Remove

csharp

if (!File.Exists(psPath))
    {
        using (StreamWriter writeFile = File.CreateText(psPath))
        {
            writeFile.WriteLine("$error.clear()");
            writeFile.WriteLine(@"Add-MailboxPermission -Identity  $usr -AccessRights FullAccess -User $tst -InheritanceType All -Confirm:$False ");
            writeFile.WriteLine(@"Get-MailboxPermission -identity $usr | FL | out-file $outPath");
            writeFile.WriteLine("if($error -ne $null){$error > $outPath }");
            writeFile.Close();
        }
    }
    else
    {
        TextWriter writeFile = new StreamWriter(psPath);
        writeFile.WriteLine("$error.clear()");
        writeFile.WriteLine(@"Add-MailboxPermission -Identity  $usr -AccessRights FullAccess -User $tst -InheritanceType All -Confirm:$False ");
        writeFile.WriteLine(@"Get-MailboxPermission -identity $usr | FL | out-file $outPath");
        writeFile.WriteLine("if($error -ne $null){$error > $outPath }");
        writeFile.Close();
    }

if (!File.Exists(psPath))
    {
        using (StreamWriter writeFile = File.CreateText(psPath))
        {
            writeFile.WriteLine("$error.clear()");
            writeFile.WriteLine(@"Add-MailboxPermission -Identity  $usr -AccessRights FullAccess -User $tst -InheritanceType All -Confirm:$False ");
            writeFile.WriteLine(@"Get-MailboxPermission -identity $usr | FL | out-file $outPath");
            writeFile.WriteLine("if($error -ne $null){$error > $outPath }");
            writeFile.Close();
        }
    }
    else
    {
        TextWriter writeFile = new StreamWriter(psPath);
        writeFile.WriteLine("$error.clear()");
        writeFile.WriteLine(@"Add-MailboxPermission -Identity  $usr -AccessRights FullAccess -User $tst -InheritanceType All -Confirm:$False ");
        writeFile.WriteLine(@"Get-MailboxPermission -identity $usr | FL | out-file $outPath");
        writeFile.WriteLine("if($error -ne $null){$error > $outPath }");
        writeFile.Close();
    }

C#

Edit|Remove

csharp

//Function runSetCMDlet
                    public void runSetCMDlet(string PSPath, string usr, string tst, string OutPath)
                    {
                       System.Collections.ObjectModel.Collection<PSObject> outPUT = new System.Collections.ObjectModel.Collection<PSObject>();
                       powershell = PowerShell.Create();
                       powershell.Runspace = runspace;
                       System.IO.StreamReader sr = new System.IO.StreamReader(PSPath);
                       powershell.AddScript(sr.ReadToEnd());
                       powershell.Runspace.SessionStateProxy.SetVariable("usr", usr);
                       powershell.Runspace.SessionStateProxy.SetVariable("tst", tst);
                       powershell.Runspace.SessionStateProxy.SetVariable("outPath", OutPath);
                       outPUT = powershell.Invoke();
                       sr.Close();
                     }

//Function runSetCMDlet
                    public void runSetCMDlet(string PSPath, string usr, string tst, string OutPath)
                    {
                       System.Collections.ObjectModel.Collection<PSObject> outPUT = new System.Collections.ObjectModel.Collection<PSObject>();
                       powershell = PowerShell.Create();
                       powershell.Runspace = runspace;
                       System.IO.StreamReader sr = new System.IO.StreamReader(PSPath);
                       powershell.AddScript(sr.ReadToEnd());
                       powershell.Runspace.SessionStateProxy.SetVariable("usr", usr);
                       powershell.Runspace.SessionStateProxy.SetVariable("tst", tst);
                       powershell.Runspace.SessionStateProxy.SetVariable("outPath", OutPath);
                       outPUT = powershell.Invoke();
                       sr.Close();
                     }

More Information

You may contact me for any further details thru blog:

 http://blogs.technet.com/b/sukum/archive/2013/01/23/coming-soon-office-365-delegates-tool.aspx

For more details on EWS Managed API, refer

http://msdn.microsoft.com/en-us/library/exchange/dd633709(v=exchg.80).aspx

repository: cnsuhao/Samples-archive-msdn-OneCode

repo_name: cnsuhao/Samples-archive-msdn-OneCode


text:
#The Stability of n-Body Choreographed Orbits
###PHYS 4250 Project Proposal by Nathan Jewsbury and Scott Osadchuk

In an effort to better understand exoctic choreographies of celestial objects, a simulation will be created to find, solve and display various stable orbits with n-bodies.  The proposed project will consist of three parts:

##Choreographed Orbit Renderer
This will be a very simple project that will take an input set of initial conditions of n-bodies and display them.  Users will be allowed to modify the display to view the selected orbit and various properties of the orbit will be displayed as the simulation is played through.

##Choreographed Orbit Engine
The engine will be the workhorse for the renderer. Once the selected orbit is displayed and the user requests to watch the orbit, the engine steps in to calculate all the required positions of each body. The overall energy of the solution will also be displayed to prove that the given solution conserves energy.  The only set of energies that will be considered are potential energy and kinetic energy.  The engine will have two different integration methods available; Euler steps and Symplectic integration. A comparison of the two methods will be included in the final report, but is not the focus of this study.

##Choreographed Orbit Solver
The solver will be the root part of this project. Through the use of a genetic algorithm a variety of control points will be generated and a cubic spline run through them to create a smooth curve.  An attempt will be made to find the minimum action of the curve such that n-bodies of similar mass produce a stable orbit. Once a minimum is found, it will be tested with more control points and once valid initial conditions are found they will be added to a database for use in the renderer.  This portion of the project should be independent of the other two and as such could be parallelized or distributed.

##Project References

Casselman, Bill. "A New Solution to the Three Body Problem." Feature Column from the AMS. American Mathematical Society, n.d. Web. 21 Feb. 2015. <http://www.ams.org/samplings/feature-column/fcarc-orbits5>.

Simo, Carles. "New Families of Solutions in N-Body Problems." Progress in Mathematics 201 (2001): 101-15. Web.

Sowinski, Arthur. A Study of Symplectic Integrators. Physics 4250 - Computational Physics. University of Manitoba, n.d.


text:
# Technologies Used
In this section, the technologies used for implementing the pizza-menu application will be described.

## Persistence Layer
A relational model is enough to satisfy all the requirements given for the application. The following image describes the relationship between each entity.
![alt text](images/er_model.png)

Regarding the Relational Database Management System, MySQL was used. However, since an ORM was used, another RDBMS can be easily implemented

## Backend
 - The API was implemented using **NodeJS** and **Express** as it allows the creation of web services in a flexible, simple and scalable way. Moreover, it is quite easy to set up a development environment. It can handle multiple requests since all the I/O operations are processed in thread pools.
 - As it was mentioned before, an ORM was implemented ([Sequelize](https://sequelize.org/)) in order to use other RDBMS.
 - Endpoints are protected through an API key although the initial logic for using json web tokens has been implemented.
 - [Swagger](https://swagger.io/) was used to document the endpoints.

 ## Frontend
  - ReactJs was used to develop the UI as it makes it user to implement and re-use web components. It has a good performance regarding rendering.
  - Redux was used to manage the sate.
  - An interceptor was created for future requirements regarding to session management and user roles.

path: /Technologies Used.md

filename: Technologies Used.md

gha_language: JavaScript


text:
Instasubmit
===========
A userscript which auto-submits the Instapaper Add form when loaded via Google Reader's Send To Instapaper action.

Installation
------------
[Install from the script page at userscripts.org](http://userscripts.org/scripts/show/98861) or simply [browse the raw script source on github](https://github.com/jasonkarns/userscripts/raw/master/instasubmit/instasubmit.user.js).

Requirements
------------
Firefox with [Greasemonkey](https://addons.mozilla.org/en-US/firefox/addon/748)

repository: jasonkarns/userscripts

repo_name: jasonkarns/userscripts


text:
---
title: OR Element
icon: or
tags: ["Element"]
layout: "page.njk"
description: Calculate when at least one of the input values is true.
excerpt: >
  The OR Element combines multiple logic input values to a single output value.
  The output value will be true only when at least one of the input values is true.
---

Actions can be send to this element using the properties value[n].
Every time the value changes the onValue event is emitted.

## Web UI for the OR Element

There is a dedicated card for this element available that will be used for the web server config and landing pages:

![OR Web UI](/elements/orui.png "w400")

This card shows the current output computed from the last logical input values.
It is updated every time the status of the device is polled by the page.

## Element Configuration

<object data="/element.svg?or" type="image/svg+xml"></object>

The up to 8 input values can be preset and the output can be inverted.

When no input value is given the output value is 0.

The non-inverted output value is on HIGH level when all input values are on HIGH level.

{% include "./calcproperties.md" %}

{% include "./elementproperties.md" %}

### Configuration Example

``` json
{
  "or": {
    "o": {
      "value": [ "true", "false"],
      "invert": "false",
      "onValue": "device/0?log=or-output: $v"
    }
  }
}
```

## Element State

The following properties are available with the current values at runtime

**active** - Is set to true when the element is active.

**value** - Current output value of the element.

## Implementation Details

When input values have changed event is not created immediately but when the next loop function is called.
This avoids flickering values on the output for a certain degree.

You can set values using a browser by sending actions to the element by requesting for the URLs like:

* <http://homeding/$board/or/a?value[0]=1> to set the first input value to true
* <http://homeding/$board/or/a?value[1]=1> to set the second input value to true
* <http://homeding/$board/or/a?value[1]=0> to set the second input value to false

## See also

* [Elements](/elements/index.md)
* [Calc Element](/elements/calc.md)
* [AND Element](/elements/and.md)
* [ADD Element](/elements/add.md)

gha_license_id: NOASSERTION


text:
# airfield_gui

## Contribute
### Build
To build airfieldFX you need my small signing library
[Latest release](https://github.com/Kaito23/panda-signer/releases/latest "Here you find the latest release of the signing library").
Or you can add the imported project in eclipse to the airfieldFX project.


text:
---
layout: page
title: Code
tags: [about]
modified: 2014-08-08T20:53:07.573882-04:00
comments: false
---
For my thesis I learnt to code in Python. [John Stachurski](http://johnstachurski.net/) and Thomas Sargent's [quant-econ](http://quant-econ.net/) site provides a great introduction to Python for economists. Python is a joy to use, but it's a bit slow. Here I rely on [Cython](http://cython.org/) which is basically a Python to C translator, see [this post]({% post_url 2014-10-30-cython1 %}).

Recently I've been working on adapting some machine learning methods to economic problems...

econlearn
---------

[econlearn](https://github.com/nealbob/econlearn) is a python [machine learning](http://en.wikipedia.org/wiki/Machine_learning) toolkit, based on the methods I developed for my thesis. `econlearn` provides some [reinforcement learning](http://en.wikipedia.org/wiki/Reinforcement_learning) (i.e., approximate dynamic programming) and [supervised learning](http://en.wikipedia.org/wiki/Supervised_learning) (i.e., non-parametric regression) algorithms suited to economic problems. That is noisy problems, requiring large sample sizes, but involving relatively few dimensions. In particular, it implements a batch version of [Q-learning](http://en.wikipedia.org/wiki/Q-learning) using [tilecoding](http://en.wikipedia.org/wiki/Cerebellar_Model_Articulation_Controller) for function approximation.

You can download `econlearn` from github [here](https://github.com/nealbob/econlearn). The code and documentation remain a work in progress. If you have any problems with it please let me know.


text:
This command starts the Web server and routes requests to the specified files.
For example:

    sencha web start

This will "mount" the current directory as the web root at the default port.
The port can be specified if needed:

    sencha web -port 8000 start

To stop the server, press CTRL+C or you can use these commands (from another
terminal), respectively:

    sencha web stop

    sencha web -port 8000 stop

## The Web Root

By default, `sencha web start` mounts the current directory so that all files
and folders are available at the root of the web server's URL. Sometimes you
may need to connect various folders into a common web root. To do this, use
the `-map` switch like so:

    sencha web start -map foo=/path/to/foo,bar=/another/path

Given the above, the following URL entered in a browser will display the files
in `"/path/to/foo"`:

    http://localhost:8000/foo

And this URL will display the files in `"/another/path"`:

    http://localhost:8000/bar

For more details regarding the `Sencha Cmd` web server, run this command:

    sencha help web

*NOTE:* These are low-level commands that do not relate to the current
application. For applications, consider the `web-start` target using
`sencha ant web-start` and `sencha ant web-stop`.

repository: bradyhouse/house

repo_name: bradyhouse/house


text:
---
name: 🐞 Bug report
about: Mobile Telemetry Bug Report
title: "\U0001F41E Bug Summary"
labels: "bug"
assignees:

---

**Configuration**
 - SDK Version [e.g. 1.9.x]
 - Android OS Version [ e.g. 8.0.0]
 - Device model(s): [e.g. Google Pixel XL etc]
 - App: [e.g. Mapbox Studio]

**Steps to Reproduce**
How did you get there?

**Expected**
What did you expect?

**Logging**
Please attach logs or error messages

repository: mapbox/mapbox-events-android

branch_name: refs/heads/main

gha_language: Java

repo_name: mapbox/mapbox-events-android


text:
# Azure IoT C# End-to-end test prerequisites

The E2E tests require Azure services set up and configured with test devices (IoT Hub, Provisioning).
Please copy the iot_config.* scripts to a secure location, remove the `_template` suffix and follow the comments to fill in the missing information.

To generate required test certificates you can use the tools available [here](https://github.com/Azure/azure-iot-sdk-c/blob/master/tools/CACertificates/CACertificateOverview.md).

The tests require the following docker containers to run locally:

### TPM Simulator

Docker image: https://hub.docker.com/r/aziotbld/testtpm/

```
docker pull aziotbld/testtpm
docker run -d --restart unless-stopped --name azure-iot-tpmsim -p <IP_ADDRESS>:2321:2321 -p <IP_ADDRESS>:2322:2322 aziotbld/testtpm
```

Alternatives:
 - Stand-alone executable for Windows: https://www.microsoft.com/en-us/download/details.aspx?id=52507

### Proxy Server

Docker image: https://hub.docker.com/r/aziotbld/testproxy/

```
docker pull aziotbld/testproxy
docker run -d --restart unless-stopped --name azure-iot-tinyproxy -p <IP_ADDRESS>:8888:8888 aziotbld/testproxy
```

Alternatives:
 - TinyProxy for Linux/Mac: https://tinyproxy.github.io/
 - Squid for Windows: https://chocolatey.org/packages/squid

### Far away hub
In order to test reprovisioning, your dps instance under test must be linked to two different iot hubs. The first hub should already be set under the
IOTHUB_CONN_STRING_CSHARP environment variable, and the other linked hub should be set as the value to the FAR_AWAY_IOTHUB_HOSTNAME environment variable

While the name of this variable indicates that the hub is far away, the tests currently do not require the hub to be far away. The tests
only require it to be a separate hub from the IOTHUB_CONN_STRING_CSHARP hub. It can be deployed to any region.

### Custom allocation policy webhook

Follow these instructions to setup a custom allocation webhook to test against:
https://docs.microsoft.com/en-us/azure/iot-dps/how-to-use-custom-allocation-policies

For the tests currently in this repo, your azure fucntion, which picks what iot hub to provision to, should always return the iot hub with the
longest hostname. Not all custom allocation policies need to work this way, but this is an easy-to-test functionality.

In this folder, there is a run.csx file that is the code the azure function should run in order to achieve this behavior.

The actual webhook url can be found through the azure portal, under your function, where there is a "Get Function URL" button that gives you a url
such as: "https://someazurefunction.azurewebsites.net/api/SomeTriggerName?code=XXXXXXXX" and this is the value that should be configured to the
CUSTOM_ALLOCATION_POLICY_WEBHOOK environment name

repository: MakaWeli/azure-iot-sdk-csharp

repo_name: MakaWeli/azure-iot-sdk-csharp


text:
---
name: Documentation request
about: Request documentation
title: ''
labels: 'documentation'
assignees: ''
---

### Documentation Request

<!-- What scenario or tool do you feel is missing documentation? Any particular characteristic of the feature that you believe deserves to be explained further?-->

### Previous documentation

<!-- Is there any documentation that already exists for this? If so, please provide links so we can update those sources. -->

<!-- Thanks for taking the time to report this! -->

repository: dotnet/diagnostics

repo_name: dotnet/diagnostics


text:
# SESSION 2: COMPUTER VISION AND TRANSFER LEARNING

## Learning Goals
- Understand what the convolution and related operations are and why they are useful for neural networks to understand image data
- Understand the basic building blocks of convolutional neural network architectures (layer patterns) and terminologies used
- Learn what transfer learning is, why and how you should use it
- Build your own CNN image classifier from scratch like for german traffic signs as an example
- Use transfer-learning to classify a very small set of training images (like maybe 120 images of birds and bees)

# Videos
- [(Ava Soleimany) Deep Computer Vision 2019 MIT Lecture](https://www.youtube.com/watch?v=H-HVZJ7kGI0&index=1&list=PLtBw6njQRU-rwp5__7C0oIVt26ZgjG9NI)
- [(Fernanda Viegas) Data Visualization for Machine Learning](https://www.youtube.com/watch?v=ulLx2iPTIcs&index=1&list=PLtBw6njQRU-rwp5__7C0oIVt26ZgjG9NI)

# Code
- [PYTORCH: Train a CNN classifier for CIFAR10](https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html)
- [(Sasank Chilamkurthy) PYTORCH: Transfer Learning Tutorial](https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html)

## Going beyond
- Be familiarized with data augmentation techniques
- Be familiarized on some techniques on how to visualize what and how CNNs learn and see
- Be familiarized about the idea of adversarial examples and how it can fool image classifiers
- You might also be interested to try [Object detection](http://course18.fast.ai/lessons/lesson8.html)

> Object detection, which means getting a model to draw a box around every key object in an image, and label each one correctly. - Fast AI

> I like to summarize this point as “don’t be a hero”: Instead of rolling your own architecture for a problem,
you should look at whatever architecture currently works best on ImageNet, download a pretrained model and finetune
it on your data. You should rarely ever have to train a ConvNet from scratch or design one from scratch. - Andrej Karpathy

## References

- CS231n: Convolutional Neural Networks for Visual Recognition Course Notes by Andrej Karpathy
  - [Convolutional Neural Networks: Architectures, Convolution / Pooling Layers ](http://cs231n.github.io/convolutional-networks/)
  - Visualizing what ConvNets learn (draft)
  - Transfer learning (draft)
- [Daniel Nouri 2014, Using convolutional neural nets to detect facial keypoints tutorial](http://danielnouri.org/notes/2014/12/17/using-convolutional-neural-nets-to-detect-facial-keypoints-tutorial/)
- [(Olah) Conv Nets: A Modular Perspective Posted on July 8, 2014](http://colah.github.io/posts/2014-07-Conv-Nets-Modular/)


text:
couchdb
=======

Lightweight straight forward streaming couchdb client implementation in java, with little dependencies.

Access to a couchdb view is implemented as an Iterator. Just feed it an input stream and than you can iterate the view
as Jackson JsonNode's:
```java
 InputStream inputStream = CouchdbViewIterator.class.getResourceAsStream("/alldocs.json");
 CouchdbViewIterator iterator = new CouchdbViewIterator(inputStream);
 while(iterator.hasNext()) {
    JsonNode node = iterator.next();
    ....
 }
```
This can be used on huge views, and won't consume much memory.

This example is from a test case. Normally you would open the stream using HTTP. To help constructing the URL for that,
a 'CouchdbOptions' object is present too (which is copied from org.jcouchdb.db.Options, but with removed dependencies),
and also a 'CouchdView' class.

Some pseudo code:
```java

   private static final CouchdbView view = new CouchdbView("media", "by-parent-and-type");

   public Iterator<MyObject> getMyObjects(String id) {
        CouchdbOptions options = new CouchdbOptions()
                    .reduce(false)
                    .includeDocs(true)
                    .startKey(new Object[]{id, null})
                    .endKey(new Object[]{id, new Object[]{}});
        URL url = new URL(couchdbUrlprovider + database + "/" + view.toString() + options.toQuery());
        return new JsonNodeToMyObjectAdapter(new CouchdViewIterator(url.openStream()));

   }

```

repository: vpro/vpro-shared

repo_name: vpro/vpro-shared


text:
# express-sed

Express middleware to transform response body

[![Node.js CI](https://github.com/kawanet/express-sed/workflows/Node.js%20CI/badge.svg?branch=master)](https://github.com/kawanet/express-sed/actions/)
[![npm version](https://badge.fury.io/js/express-sed.svg)](https://www.npmjs.com/package/express-sed)

## SYNOPSIS

```js
const express = require("express");
const sed = require("express-sed").sed;
const app = express();

// replace with string pair
app.use(sed(body => body.replace("Copyright (c) [year]", "Copyright (c) 2020")));

// replace by function
const vars = {year: "2020"};
app.use(sed(body => body.replace(/\[(\w+)\]/g, (match, $1) => vars[$1])));

// replace with `sed` transform definition for HTML contents only
app.use(sed("s/©/(c)/g", {contentType: /html/}));

app.use(express.static("htdocs"));
app.listen(3000);
```

See TypeScript declaration
[express-sed.d.ts](https://github.com/kawanet/express-sed/blob/master/types/express-sed.d.ts)
for more detail.

## SEE ALSO

- https://github.com/kawanet/sed-lite
- https://github.com/kawanet/express-intercept
- https://github.com/kawanet/express-sed

## LICENSE

The MIT License (MIT)

Copyright (c) 2020-2022 Yusuke Kawasaki

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

 IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

repository: kawanet/express-sed

repo_name: kawanet/express-sed


text:
# Raft Consensus Algorithm

Raft is a consensus algorithm designed as an alternative to Paxos. It was meant to be more understandable than Paxos by means of separation of logic, but it is also formally proven safe and offers some additional features.

## The Basics

### All Servers:

* If commitIndex > lastApplied: increment lastApplied, apply log[lastApplied] to state machine.
* If RPC request or response contains term T > currentTerm: set currentTerm = T, convert to follower.

### Followers:

* Respond to RPCs from candidates and leaders
* If election timeout elapses without receiving AppendEntries RPC from current leader or granting vote to candidate: convert to candidate

### Candidates:

* On conversion to candidate, start election:
    * Increment currentTerm
    * Vote for self
    * Reset election timer
    * Send RequestVote RPCs to all other servers
* If votes received from majority of servers: become leader
* If AppendEntries RPC received from new leader: convert to follower
* If election timeout elapses: start new election

### Leaders:

* Upon election: send initial empty AppendEntries RPCs (heartbeat) to each server; repeat during idle periods to prevent election timeouts
* If command received from client: append entry to local log, respond after entry applied to state machine
* If last log index ≥ nextIndex for a follower: send AppendEntries RPC with log entries starting at nextIndex
    * If successful: update nextIndex and matchIndex for follower
    * If AppendEntries fails because of log inconsistency: decrement nextIndex and retry
* If there exists an N such that N > commitIndex, a majority of matchIndex[i] ≥ N, and log[N].term == currentTerm: set commitIndex = N.

## Where is the Raft Consensus Algorithm Used?

* [Redis](https://redis.io/topics/cluster-spec)
* [Consul](https://www.consul.io/docs/internals/consensus.html)

## Sources:

[In Search of an Understandable Consensus Algorithm
](https://raft.github.io/raft.pdf)

repository: barend-erasmus/raft-consensus-algorithm

repo_name: barend-erasmus/raft-consensus-algorithm


text:
# Developer Machine Setup

## Install Visual Studio

- Visual Studio 2017 ([download](https://www.visualstudio.com/downloads)).
  - Make sure the following is selected:
    - VC++ 2017 v141 toolset (x86, x64)
    - Visual C++ compilers and libraries for ARM.
    - Visual C++ 2017 redistributable update.
    - Visual Studio C++ Core Features
    - Visual C++ ATL support
    - Windows 10 SDK (10.0.16299.0) for Desktop

## Install CMake

  - Install [v3.11](https://cmake.org/files/v3.11/cmake-3.11.0-win64-x64.msi) or the [latest](https://cmake.org/download/).
  - Add CMake to the PATH variable

## Install DeviceExplorer

- Clone the Azure CSharp SDK

git clone --recursive --branch master https://github.com/Azure/azure-iot-sdk-csharp.git azure-iot-sdk-csharp

- Open `tools\DeviceExplorer\DeviceExplorer.sln` in Visual Studio, and build and run it.

## Cloning the Azure DM Repo

  - Clone the master branch of the device management repo recursively:

cd /d d:\git
git clone --recursive --branch master https://github.com/ms-iot/windows-iot-azure-dm-standalone-client.git dm

----

[Development Scenario](../development-scenario.md)

repository: v-jush/azure-client-tools

repo_name: v-jush/azure-client-tools


text:
# contextvars
rspec inspired context variables for Jest

## Scoped assignment to context variables

```js
const contextVars = require('@nks/contextvars');
const EmailSender = ... // fake unit under test

describe('EmailSender#send', () => {

  // sets up overridable context variables
  const ctx = contextVars({
    to<EMAIL_ADDRESS>    from<EMAIL_ADDRESS>    subject: 'Hey check it out',
  });

  // You can use them in a beforeEach block to set up complex objects.
  // This object will be instantiated with the context set up at each
  // scoping level the way you expect it to be
  let sender;
  beforeEach(() => {
    sender = new EmailSender({
      to: ctx.to,
      from: ctx.from,
      subject: ctx.subject,
    });
  });

  it('sends an email', () => {
    return expect(sender.send()).resolves.to({ sent: true });
  });

  describe('when the subject is blank', () => {

    // You can override variables within a nested scope
    ctx.assign({ subject: '' });

    // Or add entirely new ones
    ctx.assign({ cc<EMAIL_ADDRESS>});

    it('throws', () => {
      return expect(() => sender.send()).rejects.toThrow(/empty subject/);
    });
  });
});
```

## Caveats

This library uses Proxy objects (originally defined in ECMAScript 2015), which
don't have universal support.

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Proxy#Browser_compatibility

## Related Projects

*[Given2](https://github.com/tatyshev/given2):* A more mature project using a
similar beforeAll/beforeEach approach, but requires that variables be defined
with functions. It also has broader test runner support, but uses some tricker
tools to make the raw property access syntax work.

repository: noahsilas/contextvars

repo_name: noahsilas/contextvars


text:
## Java Microbenchmark Harness

This module contains articles about the Java Microbenchmark Harness (JMH).

### Relevant articles:

- [Microbenchmarking with Java](https://www.baeldung.com/java-microbenchmark-harness)
- [A Guide to False Sharing and @Contended](https://www.baeldung.com/java-false-sharing-contended)
- [Performance Comparison of boolean[] vs BitSet](https://www.baeldung.com/java-boolean-array-bitset-performance)


text:
# sparkjava-facebook-oauth-demo

Running on Heroku at: https://sparkjava-facebook-oauth-demo.herokuapp.com/

Requires Java 1.8, and Maven (`mvn` command)

To deploy

One time steps:

1.  *One time step*: Register a Facebook OAuth application at <http://developers.facebook.com>
   * Also see: <https://developers.facebook.com/apps/>

TODO: Change stuff below to FACEBOOK

ALSO Add code from: http://facebook4j.github.io/en/code-examples.html

2.  *One time step*:
   Copy `env.sh.SAMPLE` to `env.sh`, and edit `env.sh`
   to provide correct values
   for GITHUB_CLIENT_ID and GITHUB_CLIENT_SECRET.  For APPLICATION_SALT,
   enter a long random string of letters and digits (20 or more).
   For GITHUB_CALLBACK_URL, enter `http://localhost:4567/callback`.
   Substitute the actual hostname/port on the server on which you are
   running, if appropriate.   Note that `env.sh` is ignored in the `.gitignore`
   for the repo, because these values should be kept secret.

3.  *Each time you are running*: Use the following command to load the
   environment variables into your process:

   ```
   . env.sh
   ```

4.  *To compile code* each time you make changes, run:

   ```
   mvn package
   ```

5.  *To start server at command line*, after `mvn package`, use:

   ```
   java -jar target/github-org-utility-webapp-1.0.jar
   ```

The command `mvn clean` can be used if needed (equivalent of `make clean`
or `ant clean`) to remove temporary files and reset the project to get a clean
compile.

# References:

Some helpful references can be found here:

* https://github.com/pac4j/spark-pac4j
* https://github.com/pac4j/spark-pac4j-demo
* https://github.com/pconrad-webapps/sparkjava-pac4j-github-oauth-demo
* https://github.com/kohsuke/github-api


text:
**All Time Low双手简谱** 和五线谱完全对应。

_All Time Low_ 是美国创作型歌手Jon
Bellion演唱的歌曲，最初版本发行于2015年4月14日，之后重新制作修改的版本于2016年5月13日发行。歌曲被收录在Jon
Bellion首张录音室专辑《The Human Condition》中。

新版本的All Time
Low与原始版本相比有多处修改，包括移除了部分原始音效，在最后一段副歌增加人声合唱元素等。同时，新的单曲封面作为新专辑美术设计的一部分，也于当日发行。

歌词下方是 _All Time Low钢琴谱_ ，大家可以免费下载学习。

### All Time Low歌词：

I was the knight in shining armour in your movie
Would put your lips on mine and love the aftertaste
Now I'm a ghost, I call your name, you look right through me
You're the reason I'm alone and masturbate
I (I, I), I've been trying to fix my pride
But that shit's broken, that shit's broken
Lie (lie, lie), lie, l-lie, I try to hide
But now you know it
That I'm at an all time
[x4:]
Low, low, low, low, low, low, low, low, low
Low, low, low, low, low, low, low, low
I was the prototype like 3 stacks on that CD
An example of the perfect candidate
Now all your girlfriends say that you don't want to see me
You're the reason that I just can't concentrate
I (Oh! I, I, I...)

path: /流行/All Time Low-Jon Bellion/README.md


text:
## MR1006

  TypeName
  MR1006OperatorsMustHaveXMLComment

  CheckId
  MR1006

  Category
  Documentation Rules

## Cause

Operators must have an XML comment.

## Rule description

The violation of this rule occurs when a operator does not have an XML comment.

## How to fix violations

To fix a violation of this rule, ensure that the operator has a valid XML comment.

## How to suppress violations

```csharp
[SuppressMessage("CommentCop.CSharp", "MR1006:OperatorsMustHaveXMLComment", Justification = "Reviewed.")]
```

```csharp
#pragma warning disable MR1006 // OperatorsMustHaveXMLComment
#pragma warning restore MR1006 // OperatorsMustHaveXMLComment
```

license: MS-PL

repository: mreu/CommentCop

repo_name: mreu/CommentCop


text:
---
name: "🛠 🐞 Dev Tools Bug Report"
about: Report a reproducible bug or regression related to dev tools (CLI, publishing, commands like Manage Extension or Create Extension, etc)
title: '[Dev Tools Bug] '
labels: 'dev tools, bug'

---

<!--
  Please provide a clear and concise description of what the bug is. Include
  screenshots if needed. Please test using the latest version of Raycast and API.
-->

Raycast version:

## Description

## Steps To Reproduce

1.
2.

<!--
  Your bug will get fixed much faster if we can easily reproduce the bug. Issues without reproduction steps may be immediately closed as not actionable.
-->

## The current behavior

## The expected behavior

repository: danigoldberg/extensions

repo_name: danigoldberg/extensions


text:
## Course Outline
The outline is subject to change.

###  1: Source Version Control (git)
* Development Environment Setup
* Source Version Control Using Git/Github
* **Assignment:** Online Portfolio
* **Quiz 1:** Syllabus
* **Quiz 2:** Source Version Control

###  2: JavaScript Fundamentals
* Web History/Architecture
* JavaScript Fundamentals
* **Assignment:** JS Exercise
* **Quiz:** JS Fundamentals

###  3: DOM and Forms
* DOM
* jQuery
* Web forms
* Input validation
* Regular Expressions
* **Assignment:** Rock Paper Scissors
* **Quiz:** DOM and Forms

###  4: JSON and Networking
* JavaScript Object Notation
* JSON methods
* Asynchronous JavaScript and XML
* XMLHttpRequest
* fetch
* WebRTC
* **Assignment:** Chatting Application (Lab)
* **Quiz:** JSON and Networking
* **Check-in Survey1**

###  5: Web Storage
* LocalStorage
* WebSQL
* IndexedDB
* PouchDB
* localForage
* **Quiz:** Web Storage
<!-- * **Assignment:** Sudoku -->

###  6: Canvas and Animation
* Canvas
* CSS Animation
* WebGL
* **Quiz:** Canvas API and Animation
* **Assignment:** Hangman

###  7: Web APIs
* Navigator
* Geolocation
* WebVR
* Web Workers
* Notification
* **Quiz:** Web APIs
* **Extra Credit Assignment:** (up to 10 points)

###  8: Phaser - Game Engine
* What is Phaser?
* What does Phaser offer
* How to get set up
* Build your first game
* **Assignment:** Phaser Tutorial (Lab)
* **Final Project:** Teams and Ideas

###  9: Phaser Continued
* Colliders
* Input Options
* Audio
* Physics
* **Assignment:** Phaser Game
* **Final Project:** Final Project Proposal

###  10: Debugging, Tooling and Testing
* Debugging applications with the Chrome Developer Tools
* Testing applications with Jest
* Automatically browser testing
* Tooling (Webpack and Babel)
<!-- * **Assignment:** Hangman Tests -->

###  11: Mobile Application Development
* What is Cordova?
* Building your first Cordova application
* Native Plugins
* Application Deployment
* Ionic Framework

###  12: Framework Comparison
* Framework vs Library
* Model-View-Controller
* Angular.js
* Angular
* React

###  13: Where to go from here?
* Server-side programming
* Desktop Application Development
    * Electron
* Mobile Application Development
* Virtual Reality in JavaScript
* Machine Learning in JavaScript
* IoT in JavaScript
* TypeScript


text:
---
title: Ethereum Improvement Proposals (EIPs)
description: The basic information you need to understand EIPs
lang: en
---

# Introduction to Ethereum Improvement Proposals (EIPs) {#introduction-to-ethereum-improvement-proposals}

## What are EIPs? {#what-are-eips}

[Ethereum Improvement Proposals (EIPs)](https://eips.ethereum.org/) are standards specifying potential new features or processes for Ethereum. EIPs contain technical specifications for the proposed changes and act as the “source of truth” for the community. Network upgrades and application standards for Ethereum are discussed and developed through the EIP process.

Anyone within the Ethereum community has the ability to create an EIP. Guidelines for writing EIPs are included in [EIP-1](https://eips.ethereum.org/EIPS/eip-1). An EIP should primarily provide a concise technical specification with a small amount of motivation. The EIP author is responsible for reaching consensus within the community and documenting alternative opinions. Given the high technical barrier for submitting a well-formed EIP, historically, most EIP authors are typically application or protocol developers.

## Why do EIPs matter? {#why-do-eips-matter}

EIPs play a central role in how changes happen and are documented on Ethereum. They are the way for people to propose, debate, and adopt changes. There are [different types of EIPs](https://eips.ethereum.org/EIPS/eip-1#eip-types), including core EIPs for low-level protocol changes that affect consensus and require a network upgrade like [EIP-1559](https://eips.ethereum.org/EIPS/eip-1559), and ERCs for application standards like [EIP-20](https://eips.ethereum.org/EIPS/eip-20) and [EIP-721](https://eips.ethereum.org/EIPS/eip-721).

Every network upgrade consists of a set of EIPs that need to be implemented by each [Ethereum client](/learn/#clients-and-nodes) on the network. This means that to stay in consensus with other clients on the Ethereum Mainnet, client developers need to make sure they have all implemented the required EIPs.

Along with providing a technical specification for changes, EIPs are the unit around which governance happens in Ethereum: anyone is free to propose one, and then various stakeholders in the community will debate to determine if it should be adopted as a standard or included in a network upgrade. Because non-core EIPs don't have to be adopted by all applications (for example, it is possible to create a fungible token that doesn't implement EIP-20), but core EIPs must be widely adopted (because all nodes must upgrade to stay part of the same network), core EIPs require a broader consensus within the community than non-core EIPs.

## History of EIPs {#history-of-eips}

The [Ethereum Improvement Proposals (EIPs) GitHub repository](https://github.com/ethereum/EIPs) was created in October 2015. The EIP process is based on the [Bitcoin Improvement Proposals (BIPs)](https://github.com/bitcoin/bips) process, which itself is based on the [Python Enhancement Proposals (PEPs)](https://www.python.org/dev/peps/) process.

EIP editors are tasked with process of reviewing EIPs for technical soundness, formatting issues, and correcting spelling, grammar, and code style. Martin Becze, Vitalik Buterin, Gavin Wood, and a few others were the original EIP editors from 2015 to late 2016.

The current EIP editors are

- Alex Beregszaszi (@axic)
- Gavin John (@Pandapip1)
- Greg Colvin (@gcolvin)
- Matt Garnett (@lightclient)
- Sam Wilson (@SamWilsn)

Emeritus EIP editors are

- Casey Detrio (@cdetrio)
- Hudson Jameson (@Souptacular)
- Martin Becze (@wanderer)
- Micah Zoltu (@MicahZoltu)
- Nick Johnson (@arachnid)
- Nick Savers (@nicksavers)
- Vitalik Buterin (@vbuterin)

If you would like to become an EIP editor, please check [EIP-5069](https://eips.ethereum.org/EIPS/eip-5069).

EIP editors decide when a proposal is ready to become an EIP, and help EIP authors move their proposals forward. [Ethereum Cat Herders](https://www.ethereumcatherders.com/) help organize meetings between the EIP editors and the community (see [EIPIP](https://github.com/ethereum-cat-herders/EIPIP)).

Full standardization process alongside with chart is described in [EIP-1](https://eips.ethereum.org/EIPS/eip-1)

## Learn more {#learn-more}

If you’re interested to read more about EIPs, check out the [EIPs website](https://eips.ethereum.org/) and [EIP-1](https://eips.ethereum.org/EIPS/eip-1). Here are some useful links:

- [A list of every EIP](https://eips.ethereum.org/all)
- [A description of all EIP types](https://eips.ethereum.org/EIPS/eip-1#eip-types)
- [A description of all EIP statuses](https://eips.ethereum.org/EIPS/eip-1#eip-process)

## Participate {#participate}

Anyone can create an EIP. Before submitting a proposal, one must read [EIP-1](https://eips.ethereum.org/EIPS/eip-1) which outlines the EIP process and how to write an EIP, and solicit feedback on [Ethereum Magicians](https://ethereum-magicians.org/), where proposals are first discussed with the community before a draft is submitted.

## References {#references}

<cite class="citation">

Page content provided in part from [Ethereum Protocol Development Governance and Network Upgrade Coordination](https://hudsonjameson.com/2020-03-23-ethereum-protocol-development-governance-and-network-upgrade-coordination/) by Hudson Jameson

</cite>

repository: ethereum/ethereum-org-website

branch_name: refs/heads/dev

gha_language: Markdown

repo_name: ethereum/ethereum-org-website