text:
## Synopsis

Full nodes are procedures which process a list of messages, `[Message]`, according to the
`Consensus` algorithm. The lightclient `Verifier` does not process all messages, but
it uses artifacts produced by execution of the `Consensus` algorithm, along with well-forumated security assumptions, to verify parts of a consensus execution transcript
process. The `Verifier` MUST work identically to a full node, given that the
security assumptions of `Consensus` are preserved. This means that if and only
if a full node accepts the new `Header` given a `ConsensusState` and
`[Message]`, will the `Verifier` then also accept it.

### Definitions

#### Consensus

`Consensus` is a `Header` generating function which takes the previous
`ConsensusState` with the messages and returns the result.

```typescript
type Consensus = (ConsensusState, [Message]) => Header
```

### Blockchain

Defined as blockchain consensus algorithm which generates valid `Header`s.
It generates a unique list of headers starting from a genesis `ConsensusState` with arbitrary
messages.

`Blockchain` is defined as
```typescript
interface Blockchain {
  genesis: ConsensusState
  consensus: Consensus
}
```
where
  * `Genesis` is the genesis `ConsensusState`
  * `Consensus` is the header generating function

### Desired Properties

#### Blockchain

The headers generated from the `Blockchain` are expected to satisfy the
followings:

1. The `Header`s MUST NOT have more than one direct child

* Satisfied if: deterministic safety
* Possible violation scenario: validator double signing, chain reorganization (Nakamoto consensus)

2. The `Header`s MUST eventually have at least one direct child

* Satisfied if: liveness, light-client verifier continuity
* Possible violation scenario: synchronised halt, incompatible hard fork

3. The `Header`s MUST be generated from the `Consensus`, which ensures valid transition of the state

* Satisfied if: correct block generation & state machine
* Possible violation scenario: invariant break, supermajor validator cartel

If the blockchain does not satisfy any of the above then the IBC protocol
may not work as intended; the chain can receive multiple conflicting
packets, the chain cannot recover from the timeout event, the chain can
steal the user's asset, etc.

#### Verifier

The validity of `Verifier` is dependent on the security model of the
`Consensus`. For example, the `Consensus` can be a proof of authority with
a trusted operator without legal binding, or the a proof of stake but with
insufficient value of stake. In such cases, it is possible that the
security assumptions break, the correspondence between `Consensus` and
`Verifier` no longer exists, and the behaviour of `Verifier` becomes
undefined. Also, the `Blockchain` may not longer satisfy
the requirements above, which leads the chain to be incompatible with IBC
protocol. The equivocation proof have to be generated and submitted to the
chain storing the `LightClient`, as defined in
[ICS ?](https://github.com/cosmos/ics/issues/53), to safely disconnect the
light client.

repository: katelynsills/ics

repo_name: katelynsills/ics


text:
# ProtoDef

ProtoDef compiler written in Elixir.

(mostly) Compatible with https://github.com/ProtoDef-io/ProtoDef.

## Installation

elixir-protodef is availible in [Hex](https://hex.pm/packages/proto_def). The package can be installed as:

  1. Add proto_def to your list of dependencies in `mix.exs`:

        def deps do
          [{:proto_def, "~> 0.0.1"}]
        end

  2. Ensure proto_def is started before your application:

        def application do
          [applications: [:proto_def]]
        end

license: MIT

repository: ProtoDef-io/elixir-protodef

branch_name: refs/heads/master

extension: md

license_type: permissive

repo_name: ProtoDef-io/elixir-protodef


text:
# Installing itch on Gentoo

At the time of this writing, there is no PKGBUILD available for itch in
any overlay, that we know of.

However, I (Amos) hold a special place in my heart for Gentoo, having used it
for several years during my college days, and although I don't have the time
to maintain one, if someone feels like maintaining a PKGBUILD for itch, I would
be happy to provide support.

If you're interested, please [open a new issue](https://github.com/itchio/itch/issues/new)
to start a discussion!

## Git installation

We ship portable archives of itch at <https://github.com/itchio/itch/releases>
which you should be able to unpack and run without much trouble, provided
you have the few X and gtk libraries required.

Alternatively, you can always follow the [Getting started](developing/getting-started.md) guide to run it as if in dev.

repository: adtyn/itch

repo_name: adtyn/itch


text:
# Partitioning and composing intermediate results of aggregations

Suppose that you want to calculate the count of distinct users over the last seven days every day. One way to do it would be to run "summarize dcount(user)" once a day with a span filtered to the last seven days. This is inefficient, because each time the calculation is run there's a six-days overlap with the previous calculation. Another option is to calculate some aggregate for each day, and then combine these aggregates in an efficient way. This option requires you to "remember" the last six results, but is much more efficient.

Partitioning queries like that would be easy for simple aggregates, such as count() and sum(). It is possible, however, to also perform them for more complex aggregates such as dcount() and percentiles(). This topic explains how Kusto supports such calculations.

Here are few examples which shows how to use hll/tdigest and that using these in some scenarios may be much more performant than other ways:

**Example**

Assuming that we have the table PageViewsHllTDigest which has the hll values over Pages viewed in each hour.
now we are interested in getting these values but binned to `12h` so we may merge the hll values using hll_merge() aggregate function by timestamp binned to `12h` and then call the function dcount_hll to get the final dcount value:

```
PageViewsHllTDigest
| summarize merged_hll = hll_merge(hllPage) by bin(Timestamp, 12h)
| project Timestamp , dcount_hll(merged_hll)

```

|Timestamp|dcount_hll_merged_hll|
|---|---|
|2016-05-01 12:00:00.0000000|20056275|
|2016-05-02 00:00:00.0000000|38797623|
|2016-05-02 12:00:00.0000000|39316056|
|2016-05-03 00:00:00.0000000|13685621|

Or even for binned timestamp for `1d` :

```
PageViewsHllTDigest
| summarize merged_hll = hll_merge(hllPage) by bin(Timestamp, 1d)
| project Timestamp , dcount_hll(merged_hll)

```

|Timestamp|dcount_hll_merged_hll|
|---|---|
|2016-05-01 00:00:00.0000000|20056275|
|2016-05-02 00:00:00.0000000|64135183|
|2016-05-03 00:00:00.0000000|13685621|

 The samething may be done over the values of tdigest which represents the BytesDelivered in each hour:

```
PageViewsHllTDigest
| summarize merged_tdigests = merge_tdigests(tdigestBytesDel) by bin(Timestamp, 12h)
| project Timestamp , percentile_tdigest(merged_tdigests, 95, typeof(long))

```

|Timestamp|percentile_tdigest_merged_tdigests|
|---|---|
|2016-05-01 12:00:00.0000000|170200|
|2016-05-02 00:00:00.0000000|152975|
|2016-05-02 12:00:00.0000000|181315|
|2016-05-03 00:00:00.0000000|146817|

**Example**

When having a too large datasets where we need to run a periodic queries over this dataset but running the regular queries to calculate [`percentile()`](percentiles-aggfunction.md) or [`dcount()`](dcount-aggfunction.md) over these big dataset hits kusto limits.

::: zone pivot="azuredataexplorer"

To solve this problem, Newly added data may be added to a temp table as hll or tdigest values using [`hll()`](hll-aggfunction.md) when the required operation is dcount or [`tdigest()`](tdigest-aggfunction.md) when the required operation is percentile using [`set/append`](../management/data-ingestion/index.md) or [`update policy`](../concepts/updatepolicy.md), In this case, the intermediate results of dcount or tdigest are saved into another dataset which should be smaller than the target big one.

::: zone-end

::: zone pivot="azuremonitor"

To solve this problem, Newly added data may be added to a temp table as hll or tdigest values using [`hll()`](hll-aggfunction.md) when the required operation is dcount, In this case, the intermediate results of dcount are saved into another dataset which should be smaller than the target big one.

::: zone-end

Then when we need to get the final results of these values, the queries may use hll/tdigest mergers: [`hll-merge()`](hll-merge-aggfunction.md)/[`tdigest_merge()`](tdigest-merge-aggfunction.md), Then, After getting the merged values, [`percentile_tdigest()`](percentile-tdigestfunction.md) / [`dcount_hll()`](dcount-hllfunction.md) may be invoked on these merged values to get the final result of dcount or percentiles.

Assuming that we have a table PageViews where each day we ingest data, each day we want to calculate the distinct count of pages viewed per minuite later than date = datetime(2016-05-01 18:00:00.0000000).

Running the following query :

```
PageViews
| where Timestamp > datetime(2016-05-01 18:00:00.0000000)
| summarize percentile(BytesDelivered, 90), dcount(Page,2) by bin(Timestamp, 1d)

```

|Timestamp|percentile_BytesDelivered_90|dcount_Page|
|---|---|---|
|2016-05-01 00:00:00.0000000|83634|20056275|
|2016-05-02 00:00:00.0000000|82770|64135183|
|2016-05-03 00:00:00.0000000|72920|13685621|

This query will aggregate all the values each time we run this query (e.g, if we are interested in running it many times a day).

if we save the hll and tdigest values (which are the intermediate results of dcount and percentile) into a temp table PageViewsHllTDigest using update policy or set/append commands so we may only merge the values and then use dcount_hll/percentile_tdigest using the following query :

```
PageViewsHllTDigest
| summarize  percentile_tdigest(merge_tdigests(tdigestBytesDel), 90), dcount_hll(hll_merge(hllPage)) by bin(Timestamp, 1d)

```

|Timestamp|percentile_tdigest_merge_tdigests_tdigestBytesDel|dcount_hll_hll_merge_hllPage|
|---|---|---|
|2016-05-01 00:00:00.0000000|84224|20056275|
|2016-05-02 00:00:00.0000000|83486|64135183|
|2016-05-03 00:00:00.0000000|72247|13685621|

This should be more performant and the query runs over a smaller table (in this example, The first query runs over ~215M records while the second one runs over 32 records only).

**Example**

The Retention Query.
Assuming that we have a table which summarizes when each wikipedia page was viewed (Sample size is 10M), And we are interested in finding foreach date1 date2
what percent of pages reviewed in both date1 and date2 relatively to the pages viewed on date1 (date1 < date2).

The trivial way which uses join and summarize operators :

```
// get the total pages viewed in each day
let totalPagesPerDay = PageViewsSample
| summarize by Page, Day = startofday(Timestamp)
| summarize count() by Day;
// joining the table to itself to get a grid where
// each row shows foreach page1 , in which two dates
// it was viewed.
// then counting the pages between each two dates to
// get how many pages viewed between date1 and date2.
PageViewsSample
| summarize by Page, Day1 = startofday(Timestamp)
| join kind = inner
(
    PageViewsSample
    | summarize by Page, Day2 = startofday(Timestamp)
)
on Page
| where Day2 > Day1
| summarize count() by Day1, Day2
| join kind = inner
    totalPagesPerDay
on $left.Day1 == $right.Day
| project Day1, Day2, Percentage = count_*100.0/count_1

```

|Day1|Day2|Percentage|
|---|---|---|
|2016-05-01 00:00:00.0000000|2016-05-02 00:00:00.0000000|34.0645725975255|
|2016-05-01 00:00:00.0000000|2016-05-03 00:00:00.0000000|16.618368960101|
|2016-05-02 00:00:00.0000000|2016-05-03 00:00:00.0000000|14.6291376489636|

The query above took ~18 seconds.

When using the functions of [`hll()`](hll-aggfunction.md), [`hll_merge()`](hll-merge-aggfunction.md) and [`dcount_hll()`](dcount-hllfunction.md), the equivalent query for above will end after ~1.3 seconds and shows that hll functions speeds up
the query above by ~14 times:

```
let Stats=PageViewsSample | summarize pagehll=hll(Page, 2) by day=startofday(Timestamp); // saving the hll values (intermediate results of the dcount values)
let day0=toscalar(Stats | summarize min(day)); // finding the min date over all dates.
let dayn=toscalar(Stats | summarize max(day)); // finding the max date over all dates.
let daycount=tolong((dayn-day0)/1d); // finding the range between max and min
Stats
| project idx=tolong((day-day0)/1d), day, pagehll
| mv-expand pidx=range(0, daycount) to typeof(long)
// extending column to get the dcount value from hll'ed values for each date (same as totalPagesPerDay from above query)
| extend key1=iff(idx < pidx, idx, pidx), key2=iff(idx < pidx, pidx, idx), pages=dcount_hll(pagehll)
// foreach two dates , merge the hll'ed values to get the total dcount over each two dates,
// this will help us to get the pages viewed in both date1 and date2 (see describtion below about the intersection_size)
| summarize (day1, pages1)=arg_min(day, pages), (day2, pages2)=arg_max(day, pages), union_size=dcount_hll(hll_merge(pagehll)) by key1, key2
| where day2 > day1
// to get pages viewed in date1 and also date2, we look at the merged dcount of date1 and date2, subtract it from pages of date1 + pages on date2.
| project pages1, day1,day2, intersection_size=(pages1 + pages2 - union_size)
| project day1, day2, Percentage = intersection_size*100.0 / pages1

```

|day1|day2|Percentage|
|---|---|---|
|2016-05-01 00:00:00.0000000|2016-05-02 00:00:00.0000000|33.2298494510578|
|2016-05-01 00:00:00.0000000|2016-05-03 00:00:00.0000000|16.9773830213667|
|2016-05-02 00:00:00.0000000|2016-05-03 00:00:00.0000000|14.5160020350006|

Note: the results of the queries are not 100% accurate due to the error of the hll functions.(see [`dcount()`](dcount-aggfunction.md) for further information about the errors).

repository: vijayraavi/Kusto-Query-Language

repo_name: vijayraavi/Kusto-Query-Language


text:
C# Assorted code
=======================

Miscellaneous C# programs, experiments and code samples.

* `\SQLHelper\` : Small SQL Server helper to quickly operate with stored procedures for performing typical CRUD operations. Works with DataTables, DataRows and scalars to minimize CPU/memory footprint, uses generics to automatically map .NET types with MSSQL types (not all are implemented), and while not really decoupled from the SQL engine the goal was to forget about SQL types outside of the Storage layer (instead of building an ODBC like abstraction).
Tests will probably be added in the future.
* `\Delegate_AI_Ex\` : Example of how C# Delegates (function pointers) can easily be used in a videogame's Artificial Intelligence code, by easily switching behaviours.

license: Unlicense

repository: Kartones/CSharpAssorted

repo_name: Kartones/CSharpAssorted


text:
---
layout: sidebar-right
title: Teenage library books – or how we made a mostly static collection more vibrant and popular
date: 2015-08-20 12:12:53+00:00
author: kate-ashton
category: news
excerpt: Kate shares how dispersing teenage books throughout the library has made them more popular among teenagers and adults.
breadcrumb: news
---
**Kate shares how dispersing teenage books throughout the library has made them more popular among teenagers and adults.**

Here at Haverhill library we have always had a rather static teenage library collection. We’ve had a dedicated teenage section “T-Zone”, but we rarely see any teenagers in the section browsing the stock.

We know that some of the stock goes out on loan as it reappears on our returns trolleys to be shelved, but we rarely see our borrowers and, in reality, a small, niche collection is borrowed. Our teenager borrowers are known as ninja teens because we rarely see them but we know they’ve been there!

It’s become something of a personal crusade to get our teenage books issuing more. There are some great titles but they just don’t seem to be read (except by me and a couple of my colleagues) and we end up weeding loads of books which never seem to have been loaned. I’m not averse to a good stock weed but it seemed such a pity.

When our stock unit’s Jo Dixon visited to weed our children’s stock, we had a chat about our teenage library and how we could get more stock issuing. She suggested the radical approach of breaking up the teenage library and moving the stock around. So we did.

I appreciate that in a relatively large library we’ve got the luxury of a bit more space to play around with than in a smaller library, but it’s worth considering. It has also played to our advantage as we recently changed our layout and used the teenage library space for the relocation of some of our IT equipment and our newspaper and reading area.

![New computers](/images/article/haverhill-library-new-computers.jpg)

*New computers*

We’ve moved our Younger Teen stock on to its own unit to the edge of our children’s library and our Older Teen stock is at the end of our adult fiction.

They’re still near neighbours so you’ve got that natural progression from Younger Teen to Older Teen, and you can move between the two more easily. Our teenage non-fiction, which was always half a shelf sitting awkwardly at the end of the teenage library, has been carefully moved (with much deliberation on our part) in to either the adult non-fiction section or the children’s non-fiction.

![Teen fiction integrated into the rest of the stock](/images/article/haverhill-library-new-teen-bookshelf.jpg)

*Teen fiction integrated into the rest of the stock*

It was a slow start and we didn’t really notice a change at first, but slowly and surely readers are borrowing more teenage books. There are more gaps on the shelves in both the Younger and Older Teen sections and borrowers are asking us where we’ve moved the stock. We know it’s being used.

I think it’s been great for the adults who like to dip into the Older Teen stock; they feel less conspicuous browsing at the end of the adult fiction, and it’s been a boon for those older children who are moving on from the junior fiction but find the step into the teenage library a step too far!

repository: suffolklibraries/sljekyll

repo_name: suffolklibraries/sljekyll


text:
# tutorial-osgi
Examples on OSGI

## Set Up

Set up Git, Ant and Maven

Checkout the examples.

	$ git clone https://github.com/danidemi/tutorial-osgi.git
	$ cd tutorial-osgi

Just run `ant` in the project folder, this will instruct Ant to download some additional libraries.

	$ ant

Then, you can run whatever example you want. Run `ant -p` to discover the available examples.

	$ ant -p
	Buildfile: tutorial-osgi/build.xml
    	Launching various OSGI examples.

	Main targets:

	 example-felix-standard-bundles  Run bare Felix with Gogo console.
	 setup                           Set up the environment. You need to run this before any other example.
	Default target: setup

For instance, to run the `example-felix-standard-bundles` example, just run the following command.

	$ ant example-felix-standard-bundles

## Do you need more examples ?

Would you like me to include more examples ? Please, feel free to open an issue.

repository: danidemi/tutorial-osgi

repo_name: danidemi/tutorial-osgi


text:
Device Commands
===============

A quick reference of the currently supported device types and subtypes, and the commands that can sent to each
(i.e. transmitted by the RFXtrx433). Note that not every device subtype in a given type will support
all the available commands. Invalid command/device combinations, address codes and parameters will throw an error.

For the full details see the RFXtrx433 user manual. Some of these devices may only be supported by recent versions of
the RFXCOM firmware.

Curtain1
--------

Devices:

    HARRISON

Commands:

    Curtain1.open()
    Curtain1.close()
    Curtain1.stop()
    Curtain1.program()

Lighting1
---------

Devices:

    X10
    ARC
    ELRO
    WAVEMAN
    CHACON
    IMPULS
    RISING_SUN
    PHILIPS_SBC
    ENERGENIE_ENER010
    ENERGENIE_5_GANG
    COCO

Commands:

    Lighting1.switchOn()
    Lighting1.switchOff()
    Lighting1.increaseLevel()
    Lighting1.decreaseLevel()
    Lighting1.chime()

Lighting2
---------

Devices:

    AC
    HOMEEASY_EU
    ANSLUT
    KAMBROOK

Commands:

    Lighting2.switchOn()
    Lighting2.switchOff()
    Lighting2.setLevel()

Lighting3
---------

Devices:

    KOPPLA

Commands:

    Lighting3.switchOn()
    Lighting3.switchOff()
    Lighting3.setLevel()
    Lighting3.increaseLevel()
    Lighting3.decreaseLevel()
    Lighting3.program()

Lighting4
---------

Devices:

    PT2262

Commands:

    Lighting4.sendData()

Lighting5
---------

Devices:

    LIGHTWAVERF
    EMW100
    BBSB
    MDREMOTE
    CONRAD
    LIVOLO
    TRC02

Commands:

    Lighting5.switchOn()
    Lighting5.switchOff()
    Lighting5.setLevel()
    Lighting5.setMood()
    Lighting5.increaseLevel()
    Lighting5.decreaseLevel()
    Lighting5.toggleOnOff()
    Lighting5.program()

Lighting6
---------

Devices:

    BLYSS

Commands:

    Lighting6.switchOn()
    Lighting6.switchOff()

Chime1
---------

Devices:

    BYRON_SX
    BYRON_MP001
    SELECT_PLUS
    ENVIVO

Commands:

    Chime1.chime()

repository: gizimoto/node-rfxcom

repo_name: gizimoto/node-rfxcom


text:
---
layout: default
---

# 1998-03-02 - Accept Checks By Phone

## Header Data

From<EMAIL_ADDRESS>To<EMAIL_ADDRESS>Message Hash: 583fe78428844b9a4d68a9691113ca753dbab7c85aec0fe3d066b520b699e279

Message ID<EMAIL_ADDRESS>Reply To: _N/A_

UTC Datetime: 1998-03-02 21:42:21 UTC

Raw Date: Mon, 2 Mar 1998 13:42:21 -0800 (PST)

## Raw message

```
{% raw  %}From<EMAIL_ADDRESS>Date: Mon, 2 Mar 1998 13:42:21 -0800 (PST)
To<EMAIL_ADDRESS>Subject: Accept Checks By Phone
Message-ID<EMAIL_ADDRESS>MIME-Version: 1.0
Content-Type: text/plain

Accept Checks By Fax or Telephone

Accept checks by telephone, fax or e-mail just like credit cards.  The
customers signature is not necessary.  Verbal authorization over the telephone
is all that is needed.  Increase your sales and collection results by 200% and
more.  Don't lose that all important impulse sale because your customer
doesn't have credit card or has no available credit to make a purchase.
Eliminate credit card chargebacks that can cause you to lose your credit card
merchant status!  Collect on that problem customer in advance to save you time
and money.  Do away with the check is in the mail syndrome.  It is fast and
simple to accept checks over the phone.

Donations

Collections

Sales

Monthly Payments

Post Dated Checks

Here's How It Works!

Your telephone sales or collections people simply collect the check
information right there over the phone, Telephone Check Systems supplies all
the necessary forms.  This information can either be faxed into the National
Headquarters 24 hours per day, or called in to them between 9 am and 10 PM EST
on any business day.  As soon as National Headquarters receives the
information they will immediately prepare a check against the customers bank
account.  The checks can be shipped overnight express
so that you have them in your hands ready to deposit into your bank account
within 24 hours.  You can still withhold shipment for bank clearance just as
you might normally do with checks received in the mail or you can telephone
your customers bank to see if funds are currently available to cover the check
if that is your normal procedure.

Your Cost...As Low as $1 Per Check

Now...enjoy increased sales, payments, collections and donations at a very
economical cost.  Telephone Check Payment Systems' rates are only 1 3/4% of
the amount of the check, with a $1 minimum.  That means that a check for $57
cost you only $1.  A check for $100 cost you only $1.75...less than a credit
card order and less than the charges of any of our competitors that we know
of.  The one-time set up fee is $295 and you can use the service as much or as
little with no monthly penalties, dues or statement fees like there is with
credit card merchant accounts.

There are over 65 million consumers who have checking accounts but do not have
credit cards...plus millions of credit card holders with little or no usable
credit currently available on their cards.  You can dramatically increase your
sales by tapping into this huge market with your ability to take check
payments over the telephone.  To order your check by phone system complete the
order form below and fax to<PHONE_NUMBER> along with your check for $295 or
credit card information.

CHECK BY PHONE ORDER FORM
(Print and fax to<PHONE_NUMBER>)

Company Name:

Shipping Address:

City/State/Zip:

Telephone Number:

Fax Number:

Your Name:

Title:

Home Phone:

TAPE YOUR CHECK HERE

or

Complete Credit Card Information Below

Circle One:

Visa   MasterCard   Amex   Discover
Card holders Name:

Card holders Address:

City/State/Zip:

Card Number:

Expiration Date:

____________________________
Signature

{% endraw %}
```

## Thread

+ Return to [March 1998](/archive/1998/03)

+ Return to "[way@utz.com](/authors/way_at_utz_com)"

+ 1998-03-02 (Mon, 2 Mar 1998 13:42:21 -0800 (PST)) - Accept Checks By Phone -<EMAIL_ADDRESS>

gha_license_id: MIT


text:
$LOAD_PATH

files

Dir.glob("lib/**/*")

gem build magic_set.gemspec && gem uninstall magic_set && gem install magic_set-0.1.0.gem && irb

require 'magic_set'

# Set the echo off
irb_context.echo = false

# A Magician can't do anything without a hat
class Hat; end
hat = Hat.new

# The first trick I mastered as a kid was putting cups under balls
cups = ["","ball",""]

# Give them a shuffle

cups.shuffle!

# Guess a cup
puts cups[0]
puts cups[1]
puts cups[2]

# Try again
cups.shuffle!

# Guess again
puts cups[1]
puts cups[2]
puts cups[0]

#-----

# Now magicians always putting things in boxes to see what happens

# First we'll need a box to put stuff in

class Box
  attr_accessor :contents
end

b = Box.new

# And then lets put something inside it, perhaps a glass of water

b.contents = "glass of water"

# As you can see, there is water in the box
puts b.contents

# wave wand
puts b.contents

# Now lets put something else in the box
b.contents = "dove"
puts b.contents

# wave wand again
puts b.contents

# And it's disappeared

#-------------

# Another trick that magicians like to do, is sawing a women in half.  I feel that's a bit sexist, so I'll saw a man in half instead.  Specifically, Nigel Farage.

# Now I'll represent Nigel as a good old English, beer-drinking String

irb_context.echo = false

man = "Nigel Farage"

# In ruby, it's quite easy to cut things up

man_parts = man.split " "
puts man_parts

# Now I know some of you would like it if I left Nigel that way, but I should really put him back together before the Daily Mail find out.

new_person = man_parts.join " "
puts new_person

  🍷


text:
# Conversation Sample Application with Amazon Alexa

This Node.js app demonstrates the Conversation service in a simple chat interface simulating a cognitive car dashboard and Amazon Alexa integration.
Alexa is the voice service behind products, like the Amazon Echo.

That code is based two different projects designed by colleagues.
* [Conversation Simple Sample](https://github.com/watson-developer-cloud/conversation-simple)
* [Alexa Skill with Servless & Watson Conversation](https://github.com/IBM/alexa-skill-watson-conversation?cm_sp=Developer-_-create-an-alexa-skill-with-serverless-and-a-conversation-_-Get-the-Code) --- Credit goes to [Niklas Heidloff](http://heidloff.net/) for creating the original project.

## Configuration
That code requires Watson Conversation & Redis service (available on IBM Cloud), and the configuration of Amazon Alexa Skills.

repository: vperrinfr/conversation-simple

repo_name: vperrinfr/conversation-simple


text:
# Restful GIN
Rest API with Golang, MongoDB

# Feature
* CRUD API
* Authentication
* Authorization
    * Branch `master` for role authorization
    * Branch `rbac` for dynamic role base access control authorization
* CORS
* Auto generate Swagger Docs with annotations

# Technologies
* [Gin](https://github.com/gin-gonic/gin)
* [MongoDB](https://www.mongodb.com)

# Set up
* Create file .env
* Set MongoDB URI and DB
  - PORT = "8585" or your port
  - MONGO_HOST = "your host/ localhost:27017"
  - MONGO_DB_NAME = "your db name"

* If you want to use real-time firebase's database. Replace with your serviceAccountKey.json. Then, add these variable into .env
  - FIREBASE_DATABASE = "your database url"
  - FIREBASE_STORAGE = "your firebase storage"

# Run
* `go mod download` for download dependencies
* `go run main.go`

# Swagger
* `localhost:8585/swagger/index.html`

branch_name: refs/heads/main


text:
---
id: body
title: Body
---

`Body` is a domain to model content for `Request` and `Response`. ZIO HTTP uses Netty at its core and Netty handles content as `ByteBuf`. `Body` helps you decode and encode this content into simpler, easier to use data types while creating a Request or Response.

## Server-side usage of `Body`

On the server-side, `ZIO-HTTP` models content in `Request` and `Response` as `Body` with `Body.empty` as the default value. To add content while creating a `Response` you can use the `Response` constructor:

```scala mdoc:silent
  import zio._
  import zio.http._
  import zio.stream._

  val res: Response = Response(body = Body.fromString("Some String"))
```

To add content while creating a `Request` for unit tests, you can use the `Request` constructor:

```scala mdoc:silent
  val req: Request = Request.post(URL(Root / "save"), Body.fromString("Some String"))
```

## Client-side usage of `Body`

On the client-side, `ZIO-HTTP` models content in `Client` as `Body` with `Body.Empty` as the default value.

To add content while making a request using ZIO HTTP you can use the `Client.request` method:

```scala mdoc:silent
  val actual: ZIO[Client with Scope, Throwable, Response] =
    Client.request(Request.post("https://localhost:8073/success", Body.fromString("Some string")))
```

## Creating a Body

### Creating a Body from a `String`

To create an `Body` that encodes a String you can use `Body.fromString`:

```scala mdoc:silent
  val textHttpData: Body = Body.fromString("any string", Charsets.Http)
```

### Creating a Body from `Chunk of Bytes`

To create an `Body` that encodes chunk of bytes you can use `Body.fromChunk`:

```scala mdoc:silent
  val chunkHttpData: Body = Body.fromChunk(Chunk.fromArray("Some Sting".getBytes(Charsets.Http)))
```

### Creating a Body from a `Stream`

To create an `Body` that encodes a Stream you can use `Body.fromStream`.

- Using a Stream of Bytes

```scala mdoc:silent
  val streamHttpData1: Body = Body.fromStream(ZStream.fromChunk(Chunk.fromArray("Some String".getBytes(Charsets.Http))))
```

- Using a Stream of String

```scala mdoc:silent
  val streamHttpData2: Body = Body.fromStream(ZStream("a", "b", "c"), Charsets.Http)
```

### Creating a Body from a `File`

To create an `Body` that encodes a File you can use `Body.fromFile`:

```scala mdoc:silent:crash
  val fileHttpData: Body = Body.fromFile(new java.io.File(getClass.getResource("/fileName.txt").getPath))
```

repository: zio/zio-http

gha_language: Scala

repo_name: zio/zio-http


text:
---
date: 2017-11-22 00:00:00 +0000
title: Slaughterbots, A Terrifying Vision of the Future if Bots are Allowed to Kill.
type: media
layout: youtube
videos:
  - slug: 9CO6M2HsoIA
    name: Slaughterbots
    channel: Stop Autonomous Weapons
tags:
- Technology
- Stop Autonomous Weapons
- America
- War
- Cyber War
- Automation
- Artificial Intelligence
aliases:
- "/2017/11/22-22-slaughterbots-terrifying-vision-future/"
featured: true
---

The rate of technological evolution is increasing exponentially, devices which we couldn't imagine existing 20 years ago are now a mainstream part of life. Consumer product development is only one area of advancement though, and it's hardly the cutting edge.

All technological innovations of the past 50 years have been driven primarily by military need, this includes but is not limited to the [Internet](https://en.wikipedia.org/wiki/History_of_the_Internet), [Encryption](https://en.wikipedia.org/wiki/History_of_cryptography), [Drones](https://en.wikipedia.org/wiki/History_of_unmanned_aerial_vehicles), [Artificial Intelligence](https://en.wikipedia.org/wiki/History_of_artificial_intelligence), and most obviously [Weapons](https://en.wikipedia.org/wiki/History_of_weapons).

As these different technologies become more mature their shared capabilities are becoming increasingly powerful, so much so that humans are no longer as capable as machines in [many](https://www.polygon.com/2017/8/12/16138942/dota-2-bot-vs-human-international-2017) [different](https://www.theverge.com/2017/1/25/14358246/ai-poker-tournament-cmu-libratus-vs-human-losing) [fields](http://time.com/4960778/computers-smarter-than-humans/).

Realizing the potentially catastrophic impact of smart machines with the ability to kill at their own discretion, experts across all these industries have teamed up to educate the public of the importance of regulating and limiting the capabilities of these tools.

> Many of the world's leading AI researchers and humanitarian organizations are concerned about the potentially catastrophic consequences of allowing lethal autonomous weapons to be developed. Encourage your country’s leaders to support an international treaty limiting lethal autonomous weapons:
> <br/>
> **[Ban Lethal Autonomous Weapons](http://autonomousweapons.org/)**

This video is a fictional representation of what our world could look like if we continue down the unregulated, unlimited path of creating machines that kill.

repository: dirkkelly/dirkkelly

gha_language: HTML

gha_license_id: NOASSERTION

repo_name: dirkkelly/dirkkelly


text:
xGherkin.net
============

xGherkin.net is a xspec flavored BDD framework written in c#, that tries to mimic Gherkin syntax as closely as possible.

It is implemented as an extension to xunit.net

Let's look at an typical example of a feature explained in Gherkin syntax.

```gherkin
@Issue-345
Feature: Password management

    An authenticated user must be able to change password, by providing new and old password.
    If user isn't authenticated yet, then providing email (that user had registered with),
    s/he shoudld be able to reset password. Resetting password will cause an email to be sent to
    the user with a new system generated password.

Background:
    Given that following users exist
        |username |password|
        -------------------
        |bob@123  |pass1   |
        |sam@123  |pass2   |

@PBI-32160 @Bug-42150 @Sprint-1
Scenario: Successfull password change
    Given that I have logged in with 'bob@123'
        And I have set oldpassword to 'pass1' and new password to 'pass2'
    When I call reset password
    Then it should be successful
        And calling GetByCredentials with username: 'bob@123' and with password: 'pass1' should return null
        And calling GetByCredentials with username: 'bob@123' and password: 'pass2'
            should return a user with username 'bob@123'

@PBI-32160 @Bug-42150 @Sprint-1
Scenario: Unsuccessfull password change, due to wrong old password
    Given that I have logged in with 'bob@123'
        And I have set oldpassword to 'abc' and new password to 'pass2'
    When I call reset password
    Then it should not succeed
        And calling GetByCredentials with username: 'bob@123' and password: 'abc' should return null
        And calling GetByCredentials with username: 'bob@123' and password: 'pass1' should user
```

Now lets look at xGherkin.net syntax

```csharp
using Moq;
using Ploeh.AutoFixture;
using Ploeh.AutoFixture.AutoMoq;
using System.Linq;
using xGherkin;
using Xunit.Should;

namespace xGherkinTests
{
    [Issue("345"),
    Feature("Password management",
               @"An authenticated user must be able to change password, by providing new and old password.
                 If user isn't authenticated yet, then providing email (that user had registered with),
                 s/he shoudld be able to reset password. Resetting password will cause an email to be sent to
                 the user with a new system generated password.
    ")]
    public class PasswordManagement
    {
        public void Background()
        {
            "Given that following users exist".Do(new GherkinTable("username", "password")
            {
                {"bob@123", "pass1"},
                {"sam@123", "pass2"}
            }, SetupUserService);
        }

        [PBI("32160"), Bug("42150"), Sprint("1"),
        Scenario("Successfull password change")]
        public void Scenario1()
        {
            string oldpassword = "";
            string newpassword = "";
            bool success = false;

            "Given that I have logged in with 'bob@123'".Do(_ =>
            {
                _authenticatedUser = new User { Username = "bob@123", Password = "pass1" };
            });

            "And I have set oldpassword to 'pass1' and new password to 'pass2'".Do(_ =>
            {
                oldpassword = "pass1";
                newpassword = "pass2";
            });

            "When I call reset password".Do(_ =>
            {
                success = _userService.ResetPassword(_authenticatedUser, oldpassword, newpassword);
            });

            "Then it should be successful".Do(_ =>
            {
                success.ShouldBeTrue();
            });

            "And calling GetByCredentials with username: 'bob@123' and with password: 'pass1' should return null".Do(_ =>
            {
                _userService.GetByCredentials("bob@123", "pass1").ShouldBeNull();
            });

            @"And calling GetByCredentials with username: 'bob@123' and password: 'pass2'
        should return a user with username 'bob@123'".Do(_ =>
            {
                var user = _userService.GetByCredentials("bob@123", "pass2");

                user.ShouldNotBeNull();
                user.Username.ShouldBe("bob@123");
            });

        }

        [PBI("32160"), Bug("42150"), Sprint("1"),
        Scenario("Unsuccessfull password change, due to wrong old password")]
        public void Scenario2()
        {
            string oldpassword = "";
            string newpassword = "";
            bool success = false;

            "Given that I have logged in with 'bob@123'".Do(_ =>
            {
                _authenticatedUser = new User { Username = "bob@123", Password = "pass1" };
            });

            "And I have set oldpassword to 'abc' and new password to 'pass2'".Do(_ =>
            {
                oldpassword = "abc";
                newpassword = "pass2";
            });

            "When I call reset password".Do(_ =>
            {
                success = _userService.ResetPassword(_authenticatedUser, oldpassword, newpassword);
            });

            "Then it should not succeed".Do(_ =>
            {
                success.ShouldBeFalse();
            });

            @"And calling GetByCredentials with username: 'bob@123' and password: 'abc' should return null".Do(_ =>
            {
                _userService.GetByCredentials("bob@123", "abc").ShouldBeNull();
            });

            @"And calling GetByCredentials with username: 'bob@123' and password: 'pass1' should user".Do(_ =>
            {
                _userService.GetByCredentials("bob@123", "pass1").ShouldNotBeNull();
            });

        }

        IUserService _userService;

        User _authenticatedUser;

        private void SetupUserService(GherkinTable table)
        {
            var userFromTable = table.Rows.Select(x => new User { Username = x[0] as string, Password = x[1] as string }).ToList();

            IFixture fixture = new Fixture().Customize(new AutoMoqCustomization());
            var serviceMock = fixture.Freeze<Mock<IUserService>>();

            serviceMock.Setup(rep => rep.GetByCredentials(It.IsAny<string>(), It.IsAny<string>())).Returns<string, string>((username, password) =>
            {
                return userFromTable.FirstOrDefault(x => x.Username == username && x.Password == password);
            });

            serviceMock.Setup(rep => rep.ResetPassword(It.IsAny<User>(), It.IsAny<string>(), It.IsAny<string>())).Returns<User, string, string>((user, oldpassword, newpassword) =>
            {
                var svcUser = userFromTable.FirstOrDefault(x => x.Username == user.Username && x.Password == oldpassword);
                if (svcUser != null)
                {
                    svcUser.Password = newpassword;
                    return true;
                }

                return false;
            });

            _userService = serviceMock.Object;
        }
    }
}
```

Notice how similar xGherkin.net syntax is to plain Gherkin. Just remove the c# noise and what remains will be pure Gherkin.

There is support for much of Gherkin syntax such as:

* Feature with description
* Scenario with Given-When-Then steps
* Scenario outline, that can parameterize the Scenario steps
* Example table
* Fixture table in given step is also supported.
* Background
* Tags

xGherkin.net also support some built in scrum tags such as:

* PBI
* Task
* Sprint
* Bug

Also there is an 'Issue' tag to represent Github issue.

License
============

This library has been released under the term of very open [MIT license](https://github.com/nripendra/xGherkin.net/blob/master/LICENSE), so feel free to extend or modify it any way you like.
xGherkin.net is built upon [xunit.net](https://github.com/xunit/xunit) testing framework, which is released under [Apache V2](https://github.com/xunit/xunit/blob/master/license.txt), which itself is quite open and flexible
license too.

Test project, and sample codes uses following libraries:

| Library                                                   | License                                                                    |
|-----------------------------------------------------------|----------------------------------------------------------------------------|
| [xunit.should](https://github.com/cprieto/xunit.should)   | Unsure                                                                     |
| [AutoFixture](https://github.com/AutoFixture/AutoFixture) | [MIT](https://github.com/AutoFixture/AutoFixture/blob/master/LICENCE.txt)  |
| [Moq](https://github.com/Moq/moq4)                        | [BSD](https://github.com/Moq/moq4/blob/master/License.txt)                 |

Each of these libraries have their own licenses, so be careful that xGherkin.net's MIT license doesn't cover third party libraries.

*Disclaimer:*

There has no violations of any license in the process of building this software that I'm aware of. Please notify me about such license violation if found, I will investigate and rectify the issue promptly.

Building from source
====================
If you prefer to compile xGherkin.net from it's source yourself, you'll need:

- Visual Studio 2013 (I haven't tested in other versions)
- Windows 8 or higher (I haven't tested in other versions)
- Git or [Github for windows](https://windows.github.com/)

To get the source code you can clone it locally, by clicking the "Clone in Desktop" button above.

If you are more of a command line hacker, here are the sequence of commands you'd be interested in:

```
git clone https://github.com/nripendra/xGherkin.net.git xGherkin.net
cd xGherkin.net
.\build.cmd
```

The build.cmd is setup to update nuget (if available), install Fake.Core package (if necessary) and then run build.fsx

**Working with source:**

Once the repository is cloned, open the solution file in visual studio.

- Make any change you want.
- Open package manager console (TOOLS > Nuget Package Manager > Package Manager Console).
- And type ".\build.cmd" inside the Package Manager Console.


text:
# rl-agents

A collection of Reinforcement Learning agents

[![Build Status](https://travis-ci.org/eleurent/rl-agents.svg?branch=master)](https://travis-ci.org/eleurent/rl-agents/)

* [Installation](#installation)
* [Usage](#usage)
* [Agents](#agents)
  * Planning
    * [Value Iteration](#value-iteration)
    * [Monte-Carlo Tree Search](#monte-carlo-tree-search)
    * [Deterministic Optimistic Planning](#deterministic-optimistic-planning)
    * [Open Loop Optimistic Planning](#open-loop-optimistic-planning)
    * [Trailblazer](#trailblazer)
  * Robust planning
    * [Robust Value Iteration](#robust-value-iteration)
    * [Discrete Robust Optimistic Planning](#discrete-robust-optimistic-planning)
    * [Interval-based Robust Planning](#interval-based-robust-planning)
  * Value-based
    * [DQN](#dqn)

# Installation

`pip install --user git+https://github.com/eleurent/rl-agents`

# Usage

Most experiments can be run from `scripts/experiments.py`

```
Usage:
  experiments evaluate <environment> <agent> (--train|--test)
                                             [--episodes <count>]
                                             [--seed <str>]
                                             [--analyze]
  experiments benchmark <benchmark> (--train|--test)
                                    [--processes <count>]
                                    [--episodes <count>]
                                    [--seed <str>]
  experiments -h | --help

Options:
  -h --help            Show this screen.
  --analyze            Automatically analyze the experiment results.
  --episodes <count>   Number of episodes [default: 5].
  --processes <count>  Number of running processes [default: 4].
  --seed <str>         Seed the environments and agents.
  --train              Train the agent.
  --test               Test the agent.
```

The `evaluate` command allows to evaluate a given agent on a given environment. For instance,

```bash
# Train a DQN agent on the CartPole-v0 environment
$ python3 experiments.py evaluate envs/cartpole.json agents/dqn.json --train --episodes=200
```

The environments are described by their [gym](https://github.com/openai/gym) registration `id`
```JSON
{
    "id":"CartPole-v0"
}
```

And the agents by their class, and configuration dictionary.

```JSON
{
    "__class__": "<class 'rl_agents.agents.dqn.pytorch.DQNAgent'>",
    "model": {
        "type": "DuelingNetwork",
        "layers": [512, 512]
    },
    "gamma": 0.99,
    "n_steps": 1,
    "batch_size": 32,
    "memory_capacity": 50000,
    "target_update": 1,
    "exploration": {
        "method": "EpsilonGreedy",
        "tau": 50000,
        "temperature": 1.0,
        "final_temperature": 0.1
    }
}
```

If keys are missing from these configurations, default values will be used instead.

Finally, a batch of experiments can be scheduled in a _benchmark_.
All experiments are then executed in parallel on several processes.

```bash
# Run a benchmark of several agents interacting with environments
$ python3 experiments.py benchmark cartpole_benchmark.json --test --processes=4
```

A benchmark configuration files contains a list of environment configurations and a list of agent configurations.

```JSON
{
    "environments": ["envs/cartpole.json"],
    "agents":["agents/dqn.json", "agents/mcts.json"]
}
```

# Agents

The following agents are currently implemented:

## Planning

### [Value Iteration](rl_agents/agents/dynamics_programming/value_iteration.py)

Perform a Value Iteration to compute the state-action value, and acts greedily with respect to it.

Only compatible with [finite-mdp](https://github.com/eleurent/finite-mdp) environments, or environments that handle an `env.to_finite_mdp()` conversion method.

Reference: [Dynamic Programming](https://press.princeton.edu/titles/9234.html), Bellman R., Princeton University Press (1957).

### [Monte-Carlo Tree Search](rl_agents/agents/tree_search/mcts.py)

Implemented as Upper Confidence Trees (UCT). A world transition model is leveraged for trajectory search. A search tree is expanded though a selection rule tailored to focus the search around the most promising moves, and leaves are evaluated by sampling.

References:
* [Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search](https://hal.inria.fr/inria-00116992/document), Coulom R., 2006.
* [Bandit based Monte-Carlo Planning](http://ggp.stanford.edu/readings/uct.pdf), Kocsis L., Szepesvári C. (2006).

### [Deterministic Optimistic Planning](rl_agents/agents/tree_search/deterministic.py)

Reference: [Optimistic Planning for Deterministic Systems](https://hal.inria.fr/hal-00830182), Hren J., Munos R. (2008).

### [Open Loop Optimistic Planning](rl_agents/agents/tree_search/olop.py)

Reference: [Open Loop Optimistic Planning](http://sbubeck.com/COLT10_BM.pdf), Bubeck S., Munos R. (2010).

### [Trailblazer](rl_agents/agents/tree_search/trailblazer.py)

Reference: [Blazing the trails before beating the path: Sample-efficient Monte-Carlo planning](http://researchers.lille.inria.fr/~valko/hp/serve.php?what=publications/grill2016blazing.pdf), Grill J. B., Valko M., Munos R. (2017).

## Robust planning

### [Robust Value Iteration](rl_agents/agents/dynamics_programming/robust_value_iteration.py)

A list of possible [finite-mdp](https://github.com/eleurent/finite-mdp) models is provided in the agent configuration. The MDP ambiguity set is constrained to be rectangular: different models can be selected at every transition.The corresponding robust state-action value is computed so as to maximize the worst-case total reward.

References:
* [Robust Control of Markov Decision Processes with Uncertain Transition Matrices](https://people.eecs.berkeley.edu/~elghaoui/pdffiles/rmdp_erl.pdf), Nilim A., El Ghaoui L. (2005).
* [Robust Dynamic Programming](http://www.corc.ieor.columbia.edu/reports/techreports/tr-2002-07.pdf), Iyengar G. (2005).
* [Robust Markov Decision Processes](http://www.optimization-online.org/DB_FILE/2010/05/2610.pdf), Wiesemann W. et al. (2012).

### [Discrete Robust Optimistic Planning](rl_agents/agents/tree_search/robust.py)

The MDP ambiguity set is assumed to be finite, and is constructed from a list of modifiers to the true environment.
The corresponding robust value is approximately computed by [Deterministic Optimistic Planning](#deterministic-optimistic-planning) so as to maximize the worst-case total reward.

### [Interval-based Robust Planning]((rl_agents/agents/tree_search/robust.py))

TODO

## Value-based

### [DQN](rl_agents/agents/dqn)

A neural-network model is used to estimate the state-action value function and produce a greedy optimal policy.

Implemented variants:
* Double DQN
* Dueling architecture
* N-step targets

References:
* [Playing Atari with Deep Reinforcement Learning](https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf), Mnih V. et al (2013).
* [Deep Reinforcement Learning with Double Q-learning](https://arxiv.org/abs/1509.06461), van Hasselt H. et al. (2015).
* [Dueling Network Architectures for Deep Reinforcement Learning](https://arxiv.org/abs/1511.06581), Wang Z. et al. (2015).

repository: itismouad/rl-agents

repo_name: itismouad/rl-agents


text:
---
title: "Search Extraction"
linkTitle: "Search Extraction"
weight: 30
date: 2022-02-18
tags:
  - extraction
description: >
  The process of combining data extracted from events with the data stored in an index.

---

When indexing data it is possible to store (see [Stored Fields]({{< relref "/docs/user-guide/indexing/lucene" >}}) all data in the index.
This comes with a storage cost as the data is then held in two places; the event; and the index document.

Stroom has the capability of doing Search Extraction at query time.
This involves combining the data stored in the index document with data extracted using a search extraction pipeline.
Extracting data in this way is slower but reduces the data stored in the index, so it is a trade off between performance and storage space consumed.

Search Extraction relies on the StreamId and EventId being stored in the Index.
Stroom can then used these two fields to locate the event in the stream store and process it with the search extraction pipeline.

{{% todo %}}
Add more detail
{{% /todo %}}

repository: burnalting/stroom-docs

repo_name: burnalting/stroom-docs


text:
## Portfolio - Data Science Projects

---

### Machine Learning

#### [Fire Hydrant Inspection Prioritisation Scheme using Predicted Latent Fire Risk](https://pennmusa.github.io/MUSA_801.io/project_12/index.html)

Capstone project for the MUSA800 course in the School of Design, Upenn. The project was to build a inspection prioritization scheme for the Philadelphia Fire Department.

*Project with Ran Xin and Li Zi Qun*

Predicting Food Establishment Inspection Failures in Chicago

Distribution of various risk factors across Chicago.

 ROC Curve - A measure of performance of the classification model.

*Project with Yap Yun Qi*

Predicting Geospatial Risk - Sexual Assault Risk in Chicago

Plot shows the predictions of counts of sexual assault cases of various models compared to the true count of sexual assault cases (right).

Predicting Home Prices in San Francisco

The plot shows the distribution of predicted home prices compared to observed home prices.

*Project with Yap Yun Qi*

Predicting Take Up Rates of Housing Credit Programme in Philadelphia to Optimise Cost

 Plot shows fraction of true positive, false positives, true negatives and false negatives as a result of model predictions at different thresholds.

#### [Predicting Severity of Traffic Accidents in Philadelphia](/pdf/CIS_519_Project_Report.pdf)

[Summary Sldies](/pdf/cis519_summary_slides.pdf)

---
### Data Visualisation

- [Visualising Gentrification in New York City](https://leannechan.github.io/Gentrification-Trends-In-NYC/.)

The following plot shows the price to size for each NYC borough over recent years.

*Project with Yap Yun Qi*

- Handling Big Data - 3 Million Points, 1 GIF
![GIF](./images/permits_89_19.gif)

---
### Webmapping

- [Hydrant Inspector Application](https://njxinran95.github.io/PhillyFire_App/)

Application built as part of the machine learning project to build a fire hydrant inspection scheme for the Philadelphia Fire Department.

- [Recycling Points in Singapore](https://leannechan.github.io/MUSA611_Midterm/)

---

Page template forked from evanca


text:
# Entity Model

## static members

* db
    * connector - Getter
    * createNewConnector - Create a new connector, usually used for transaction
* meta - Metadata about the enttiy
    * knowledge
        * dependsOnExisting
* i18n - I18n object

## operation context

There are predefined context properties which can be accessed in an entity operation as listed below.

* operation - 'create/retrieve/update/delete'
* raw - Raw input data.
* latest - Validated and sanitized data.
* existing - Existing data from database.
* i18n - I18n object.
* connector - Existing connector for chained operation.
* result - Operation result.
* return - Data to return, if retrieveCreated or retrieveUpdated or retrieveDeleted is true, return will be the just create/updated/deleted data.
* entities - Access other entity models in the same schema
* schemas - Access other schema models in the same application
* state - Current request state

## opertion helper

queryFunction
queryBinExpr
queryColumn

## operation options

* connector - Transaction connector.
* $projection
* $association
* $relationships
* $query - Query condition
* $variables - Variables to interpolate into query condition
* $features - Custom feature options override
* $orderBy - Order by condition, map of column to ascend?
* $groupBy - Group by condition
* $offset
* $limit
* $totalCount - Returns total record count when used with $limit
* $includeDeleted - {boolean}
* $skipOrm - {boolean}
* $custom - User defined operation control data
* $retrieveCreated
* $retrieveUpdated
* $retrieveDeleted
* $retrieveExisting
* $bypassReadOnly
* $physicalDeletion - {boolean}
* $existing
* $requireSplitColumns
* $bypassEnsureUnique
* $toDictionary
* $migration - {boolean}

## operation sequence

1. prepare query & context
2. sub-class before hooks
3. wrap in transaction-safe closure
4. pre-process data
5. features before hooks
6. driver-specific pre-process
7. execute the operation
8. driver-specific post-process
9. store query key
10. features after hooks
11. end transaction-safe closure
12. sub-class after hooks

## semantic symbols

* supported symbols
    * @@now - Current datetime value

## special tokens

* oolType

    Oolong Language Syntax Types (design time)

    * General Value Types
        * ObjectReference
        * ConstReference
        * StringTemplate
        * PipedValue
        * FunctionCall
        * RegExp
        * JavaScript

    * Modifiers
        * Validator - |~, read as "Ensure"
        * Processor - |>, read as "Transform to"
        * Activator - |=, read as "Set to"

    * Data Operations
        * findOne
        * DoStatement

    * Statements & Expressions
        * cases
        * ConditionalStatement
        * ReturnExpression
        * ThrowExpression
        * UnaryExpression
        * ValidateExpression
        * BinaryExpression
        * LogicalExpression

* oorType

    Oolong Runtime Types (run time)

    * SymbolToken
    * SessionVariable
    * QueryVariable

## fieldDependency

* fieldName: ifAnyExist: [ a1, a2, ... ] - If any of a1, a2, ... exists, all become dependency (need existing values)

    For activator that references to other fields
    When creating

* fieldName:

    For processor that references to other fields and

* dependsOnExisting: [ a1, a2 ] -

## known issues

* retrieveUpdated - The previous query maybe affected by updating

repository: genx-tech/gx-model

repo_name: genx-tech/gx-model


text:
## [`@architect/functions`][npm] [![Travis Build Status](https://travis-ci.com/architect/functions.svg?branch=master)](https://travis-ci.com/architect/functions) [![Appveyor Build Status](https://ci.appveyor.com/api/projects/status/k1ct9sv8xv9pbgg2/branch/master?svg=true)](https://ci.appveyor.com/project/ArchitectCI/functions/branch/master) [![codecov](https://codecov.io/gh/architect/functions/branch/master/graph/badge.svg)](https://codecov.io/gh/architect/functions)

> Helper library for serverless apps built with [Architect](https://www.npmjs.com/package/@architect/architect)

Check out the docs: [https://arc.codes](https://arc.codes)

[npm]: https://www.npmjs.com/package/@architect/functions


text:
# GitHub Trending (2023-06-06)

![](https://img.shields.io/badge/Python-New%20193-green?style=flat-square&logo=appveyor)
- [](https://github.comundefined): Enable everyone to develop, optimize and deploy AI models natively on everyone's devices.

![](https://img.shields.io/badge/Blade-New%20382-green?style=flat-square&logo=appveyor)
- [](https://github.comundefined): Run and create custom ChatGPT-like bots with OpenChat, embed and share these bots anywhere, the open-source chatbot console.

![](https://img.shields.io/badge/Python-New%20782-green?style=flat-square&logo=appveyor)
- [](https://github.comundefined): <⚡> SuperAGI - A dev-first open source autonomous AI agent framework. Enabling developers to build, manage & run useful autonomous agents quickly and reliably.

![](https://img.shields.io/badge/Rust-New%20143-green?style=flat-square&logo=appveyor)
- [](https://github.comundefined): Pure Rust implementation of a minimal Generative Pretrained Transformer

![](https://img.shields.io/badge/none-New%20227-green?style=flat-square&logo=appveyor)
- [](https://github.comundefined): 📚 Freely available programming books

![](https://img.shields.io/badge/Python-New%20166-green?style=flat-square&logo=appveyor)
- [](https://github.comundefined): CodeTF: One-stop Transformer Library for State-of-the-art Code LLM

![](https://img.shields.io/badge/C-New%20431-green?style=flat-square&logo=appveyor)
- [](https://github.comundefined): tiniest x86-64-linux emulator

![](https://img.shields.io/badge/Python-New%20740-green?style=flat-square&logo=appveyor)
- [](https://github.comundefined): Chat with your documents on your local device using GPT models. No data leaves your device and 100% private.

![](https://img.shields.io/badge/C-New%20211-green?style=flat-square&logo=appveyor)
- [](https://github.comundefined): Port of Facebook's LLaMA model in C/C++

![](https://img.shields.io/badge/C%23-New%20196-green?style=flat-square&logo=appveyor)
- [](https://github.comundefined): Realtime Diffusion, using Automatic1111 Stable Diffusion API

![](https://img.shields.io/badge/none-New%2091-green?style=flat-square&logo=appveyor)
- [](https://github.comundefined): Collection of Summer 2023 & Summer 2024 tech internships!

![](https://img.shields.io/badge/HTML-New%20155-green?style=flat-square&logo=appveyor)
- [](https://github.comundefined): A list of StableDiffusion styles and some notes for offline use. Pure HTML, CSS and a bit of JS.

![](https://img.shields.io/badge/TypeScript-New%2076-green?style=flat-square&logo=appveyor)
- [](https://github.comundefined): Build Python LLM apps in minutes ⚡

![](https://img.shields.io/badge/none-New%20483-green?style=flat-square&logo=appveyor)
- [](https://github.comundefined): Curated list of project-based tutorials

![](https://img.shields.io/badge/none-New%209-green?style=flat-square&logo=appveyor)
- [](https://github.comundefined): WWDC Community: Learning and sharing together

![](https://img.shields.io/badge/Python-New%2011-green?style=flat-square&logo=appveyor)
- [](https://github.comundefined): Binder Trace is a tool for intercepting and parsing Android Binder messages. Think of it as "Wireshark for Binder".

![](https://img.shields.io/badge/Shell-New%2028-green?style=flat-square&logo=appveyor)
- [](https://github.comundefined): Node Version Manager - POSIX-compliant bash script to manage multiple active node.js versions

![](https://img.shields.io/badge/Swift-New%2065-green?style=flat-square&logo=appveyor)
- [](https://github.comundefined): Mirador makes it easy to build impressive “Point of Interest” AR experiences, on Apple's new RealityKit framework.

![](https://img.shields.io/badge/Jupyter%20Notebook-New%20118-green?style=flat-square&logo=appveyor)
- [](https://github.comundefined): 🧑🏫 59 Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠

![](https://img.shields.io/badge/Jupyter%20Notebook-New%2038-green?style=flat-square&logo=appveyor)
- [](https://github.comundefined): 吴恩达大模型系列课程中文版，包括《Prompt Engineering》、《Building System》和《LangChain》

![](https://img.shields.io/badge/TypeScript-New%20101-green?style=flat-square&logo=appveyor)
- [](https://github.comundefined): Dump all your files and thoughts into your private GenerativeAI Second Brain and chat with it

![](https://img.shields.io/badge/none-New%20217-green?style=flat-square&logo=appveyor)
- [](https://github.comundefined): The Mojo Programming Language

![](https://img.shields.io/badge/none-New%2030-green?style=flat-square&logo=appveyor)
- [](https://github.comundefined): A collection of New Grad full time roles in SWE, Quant, and PM.

![](https://img.shields.io/badge/Go-New%209-green?style=flat-square&logo=appveyor)
- [](https://github.comundefined): The Go language implementation of gRPC. HTTP/2 based RPC

![](https://img.shields.io/badge/Python-New%20416-green?style=flat-square&logo=appveyor)
- [](https://github.comundefined): Interact privately with your documents using the power of GPT, 100% privately, no data leaks

repository: Hsins/Daily-GitHub-Trending

gha_language: JavaScript

repo_name: Hsins/Daily-GitHub-Trending


text:
Preface
-------

This lab is a
[literate](https://en.wikipedia.org/wiki/Literate_programming) Cryptol
document—that is, it can be loaded directly into the Cryptol
interpreter. Load this module from within the Cryptol interpreter
running in the `cryptol-course` directory with:

```shell
cryptol> :m labs::Language::Basics
```

Basic Use of the Cryptol Language
=================================

Introduction
------------

In you've programmed in a variety of languages (not just different
[procedural languages](https://en.wikipedia.org/wiki/Procedural_programming)
descending from C), you'll find that, for the most part, Cryptol is
just another language with a different vocabulary and a funky type
system. After a little study you'll be able to do most anything
computationally that you could otherwise, especially in the
cryptographic realm. But once you are accustomed to Cryptol you will
find that it is much easier to write correct cryptographic programs
than with conventional languages. That's 'cause it's been tuned for
such! To throw out the buzzwords:
* Cryptol is a
  [domain-specific language](https://en.wikipedia.org/wiki/Domain-specific_language). Not
  only does it have things to support its domain, but also it elides a
  lot of junk that makes programming and analyzing the programs
  difficult.
* Cryptol has been designed with automated reasoning about its code as
  a priority, so that we can leverage it for verification. Some things
  are harder to do in Cryptol, but they pay off in code that can be
  proven correct!

In some ways this requires a new mindset:
* Write properties about your functions.
* `:check` them.
* Invest in `:prove` when your function's definition has settled down.

Enjoy getting addicted to this level of assurance!

Preliminaries
-------------

The following code declares the module name of this literate Cryptol
document.

```
module labs::Language::Basics where
```

For examples in this lab I have turned off the warning messages you
get when not specifying bit sizes of numbers. This is **not**
something you should do when you're new at Cryptol. (In fact, I don't
do it except when teaching.)

```shell
labs::Language::Basics> :set warnDefaulting = off
```

Also some examples have octets as outputs that are easier to see as
characters so I use:

```shell
labs::Language::Basics> :set ascii = on
```

That makes any sequence of octets be displayed as the corresponding
[ASCII](https://en.wikipedia.org/wiki/ASCII) string in double quotes
(`"`) and an octet outside a sequence be displayed as the
corresponding ASCII character in single quotes (`'`). (This is mostly
useful as a pedagogical aid.)

```shell
labs::Language::Basics> [0x63, 0x61, 0x74]
"cat"
labs::Language::Basics> 0x78
'x'
```

The Cryptol interpreter parses `"abc"` and `[0x61, 0x62, 0x63]` into
the exact same internal representation. `:set ascii = on` just causes
the display of output to be ASCII strings or characters when
appropriate, not unlike using `:set base = 10` to see numbers in
base 10.

Comments
--------

* `//` to end of line
* `/*` ... `*/` block comment

There is also a [docstring](https://en.wikipedia.org/wiki/Docstring)
comment facility:

```
/**
  * A totally made up identifier for pedagogical purposes. It is
  * used elsewhere for demonstration of something or other.
  */
mask = 7
```

Now when issuing `:help mask`, the above comments are displayed along
with other information about `mask`.

Identifiers
-----------

Cryptol identifiers consist of alphanumeric characters plus `'`
(apostrophe, but read "prime") and `_` (underscore). They must begin
with an alphabetic character or an underscore. The notational
convention for `'` is to indicate a related definition, while
underscore is mostly used to separate words or as a catch all for
characters we'd like to use but are
forbidden. [Camel case](https://en.wikipedia.org/wiki/Camel_case) is
often used when other naming constraints aren't mandated.

```
fooBar = 15
fooBar' = fooBar && mask // mask defined elsewhere
```

Technically, Cryptol supports
[Unicode](https://en.wikipedia.org/wiki/Unicode), but for simplicity I
am pretending that it doesn't.

Data
----

Cryptol's "basic" data type is an _n_-dimensional array whose base
type is bits.
* 0-d: `False : Bit` and `True: Bit`
* 1-d: Think bytes, words, nibbles, etc., i.e., a sequence of bits of
  any length usually thought of as a number. E.g., `0x2a : [8]`,
  `0b101010 : [6]` and
  `[False, True, False, True, False, True, False] : [7]`. These all
  compare as 42 in type appropriate contexts.
* 2-d: Think sequences of 1-d objects all of the same size. E.g.,
  `[42, 0b010101010101, 0xa5a, 0o5757] : [4][12]`
* 3-d: Sequences of 2-d objects all of the same size. E.g.,
  `[[0, 1], [1, 2], [3, 5], [8, 13]] : [4][2][4]`
* ...

Things to note:
* There are no privileged bit widths in Cryptol. `[13]` is just as
  good a type as `[8]`, `[16]`, `[32]` or `[64]`.
* There's `0b...` for binary, `0o...` for octal and `0x...` for
  hexadecimal.
* Lengths of sequences may be zero. Zero length sequences act as an
  identity for concatenation and are useful in padding.
* The possible values by type:
  * `[0]`—`0`
  * `[1]`—`0` and `1`
  * `[2]`—`0`, `1`, `2` and `3`
  * ...
  * `[n]`—`0` through `2^^n - 1`

  There are 2_n_ values of type `[n]`. There are
  2_mn_ values of type `[m][n]`, etc.
* 1-d sequences of bits are treated as numbers by arithmetic and
  comparison operators. So for instance, `[False, True] == (1 : [2])`
  and `[True, False, True] > 4` both hold.
* Cryptol distinguishes between the different dimensions. In
  particular, `True` and `[True]` are type incompatible.
* The number of bracket pairs in a type gives its dimension.

Other data types include:
* Arbitrary-precision integers: E.g., `2^^1023 - 347835479 : Integer`
* Heterogeneous tuples: E.g.: `(False, 0b11) : (Bit, [2])` and
  `(True, [1, 0],<PHONE_NUMBER>987) : (Bit, [2][1], Integer)`
  * Elements of tuples are accessed by `.0`, `.1`, ...
    ```shell
    labs::Language::Basics> (False, 0b11).0
    False
    ```
* Records with named fields: E.g.,
  `{flag = True, x = 2} : {flag : Bit, x : [4]}`
  * Elements of records are accessed by `.` followed by the field name.
    ```shell
    labs::Language::Basics> {flag = True, x = 2}.flag
    True
    ```
* Integers modulo _n_: Types of the form `[n]` already provide
  [least residue systems](https://en.wikipedia.org/wiki/Modular_arithmetic#Residue_systems)
  for
  [integers modulo 2n](https://en.wikipedia.org/wiki/Modular_arithmetic#Integers_modulo_n).
  Types of the form `Z n` provide that for any positive _n_. E.g.,
  `4 + 4 : Z 7` evaluates to `1`.

Operators
---------

Cryptol's `:help` command will provide a brief description of the
operators in this section by issuing `:help ` followed
by the name of the operator in parentheses. For example: `:help (@)`

Many languages differentiate signed and unsigned numbers at the type
level (e.g. C's `uint32` and `int32`). Cryptol has separate operators
for signed operations which are indicated by a suffixed `$`. Most of
the time you don't need them as cryptography tends to use nonnegative
numbers.

Where appropriate, operators act elementwise (or "blast through")
typing constructs like sequences, tuples and records.

```shell
labs::Language::Basics> [[0, 1], [1, 2]] + [[3, 5], [8, 13]]
[[3, 6], [9, 15]]
labs::Language::Basics> (3, (1, 4)) + (1, (5, 9))
(4, (6, 13))
labs::Language::Basics> {x = 1, y = 3} + {y = 6, x = 10}
{x = 11, y = 9}
labs::Language::Basics> [(0, 1), (4, 9), (16, 25)].1
[1, 9, 25]
labs::Language::Basics> [{x = 1, y = 3}, {y = 6, x = 10}].y
[3, 6]
```

Following are some really quick examples of operators to remind you
and show some tricks of Cryptol.

### Arithmetic: `+`, `-`, `*`, `/`, `%` and `^^`
#### Signed versions: `/$` and `%$`

```shell
labs::Language::Basics> 1 + 1
2
labs::Language::Basics> 1 + 1 : [1]
0x0
labs::Language::Basics> 2^^127 - 1 // a 33 digit Mersenne prime
170141183460469231731687303715884105727
```

The first example defaults to type `Integer`. In the second, I
explicitly state that I want 1 bit addition and so the computation is
modular addition. The third shows that `^^` is exponentiation.

The division (`/`) operation is not what a mathematician imagines in
modular arithmetic. For instance `3 * 3 == 1 : [3]` so a mathematician
would expect `1 / 3 == 3 : [3]` since division is the inverse of
multiplication. However, in Cryptol `1 / 3 == 0 :
[3]`. Mathematically, `/` and `%` yield the quotient and remainder,
respectively, in
[Euclidean division](https://en.wikipedia.org/wiki/Euclidean_division).

### Bitwise logical: `~`, `&&`, `||` and `^`

```shell
labs::Language::Basics> ~0b000011001101 && 0o4115 || 0x0d0 ^ 9
0x8d9
```

### Comparison:`==`, `!=`, `<` , `<=`, `>` and `>=`
#### Signed versions: `<$`, `<=$`, `>$` and `>=$`

```shell
labs::Language::Basics> [~1, 1] == [6 : [3], 3 * 3]
True
labs::Language::Basics> [~1, 1] == [6 : [4], 3 * 3]
False
```

In the first example, `6` is the literal needing the most bits and is
given type `[3]`. That makes both sides of the equality test have type
`[2][3]` (two elements of three bits each). Now `~1 : [3]` is `0b110`
or `6` in decimal and `3 * 3 : [3]` is `1 : [3]` so `[~1, 1] == [6, 1]
== [6, 3 * 3]`.

In the second example, `6` is given type `[4]`, so both sides have
type `[2][4]`. Now `~1 : [4]` is `0b1110` or `14` in decimal and
`3 * 3 : [4]` is `9` in decimal. We have `[~1, 1] == [14, 1]`
while `[6, 3 * 3] == [6, 9]` so equality fails.

_**It can be crucially important to be precise about the widths of
things!**_

Comparisons are lexicographic on sequences of numbers.

```shell
labs::Language::Basics> [1, 2] < [1, 3]
True
labs::Language::Basics> [1, 2] < [1, 2]
False
```

### Shifts: `<<`, `>>`, `<<<` and `>>>`
#### Signed version: `>>$`

```shell
labs::Language::Basics> 0xa5a << 4
0x5a0
labs::Language::Basics> 0xa5a << 12
0x000
labs::Language::Basics> 0xa5a <<< 16
0x5aa
```

### Indexing and slicing: `@`, `!`, `@@` and `!!`

```shell
labs::Language::Basics> "cat" @ 0
'c'
labs::Language::Basics> "dog" @@ [2, 1, 1, 0, 0, 1, 2]
"gooddog"
labs::Language::Basics> "cow" ! 0
'w'
```

Notice that these operators all use 0-based indexing: `@` and `@@`
from the beginning of the sequence and `!` and `!!` from the end.

### Concatenation: `#`

```shell
labs::Language::Basics> "dog" # "cow" // Moof!
"dogcow"
```

### Shortcutting logical: `/\`, `\/` and `==>`

These are most often used in property statements. `/\` is "and", `\/`
is "or" and `==>` is "implies". They have very low precedence.

```shell
labs::Language::Basics> 1 == 5 \/ 5 == 5
True
labs::Language::Basics> False ==> 1 == 5 /\ 1 != 5
True
```

Common Primitives
-----------------

Cryptol's `:help` command will provide a brief description of the
primitives in the section by issuing `:help ` followed
by the name of the primitive.

### Collections of all `False` or all `True` bits

* `0` is a sequence of `False` bits whose type is determined by the
context.
  ```shell
  labs::Language::Basics> 0 : [12]
  0x000
  ```
* `zero` is an arbitrary collection of `False` bits whose type
is determined by the context.
  ```shell
  labs::Language::Basics> zero: ([8], [4])
  (0x00, 0x0)
  ```
  Here we produce an order pair of a 0 octet and a 0 nibble.
* `~0` and `~zero` produce all `True` bits correspondingly.

### List manipulation: `take`, `drop`, `tail`, `last` and `reverse`
```
labs::Language::Basics> take "dogcow" : [3][8]
"dog"
labs::Language::Basics> drop [2, 3, 5, 7, 11] : [3]Integer
[5, 7, 11]
labs::Language::Basics> tail [0, 1, 1]
[1, 1]
labs::Language::Basics> last [2, 3, 5, 7, 11]
11
labs::Language::Basics> reverse [0, 0, 1]
[1, 0, 0]
```

Of course the sizes of lists have to be big enough. Also, notice that
`last` (which is equivalent to `! 0`) returns an element while the
others return lists.

Often in a Cryptol program, the context will determine the shapes of
sequences, so that the type annotations (`: [3][8]` and `: [3]Integer`
above) will be unnecessary.

### List shape manipulation: `split`, `join`, `transpose`
```shell
labs::Language::Basics> split 0xdeadbeef : [8][4]
[0xd, 0xe, 0xa, 0xd, 0xb, 0xe, 0xe, 0xf]
labs::Language::Basics> join [0xca, 0xfe]
0xcafe
labs::Language::Basics> transpose [[1, 2], [3, 4]]
[[1, 3], [2, 4]]
```

In most Cryptol programs, the context will enforce the size of things,
so the type annotations shown in these examples need not be present.

The Types of Functions
----------------------

The Cryptol interpreter command `:type` is very useful for helping you
understand types. For instance the type of the `abs` function which we
will define later is displayed by:

```shell
labs::Language::Basics> :type abs
abs : Integer -> Integer
```

indicating that it takes an integer and returns an integer.

Curried and Uncurried Style
---------------------------

Cryptol functions are often written in the
[curried](https://en.wikipedia.org/wiki/Currying) style:

```
gcdCurried: Integer -> Integer -> Integer
```

rather than:

```
gcdUncurried: (Integer, Integer) -> Integer
```

These two functions would be applied as shown:

```shell
labs::Language::Basics> gcdCurried 20 28
4
labs::Language::Basics> gcdUncurried (20, 28)
4
```

These two styles are equivalent at some level. The former is preferred
as it affords
[partial application](https://en.wikipedia.org/wiki/Partial_application),
but the latter can be useful for explicating the correspondence to
functions from other languages or documents.

* If it helps you, mentally read curried functions like this: input
  argument types are all prior to the last arrow and the
  result type follows the last arrow. Pictorially: `in -> in -> ... ->
  in -> out`
* Partial application lets one form a new function from an old one
  where an argument is fixed.  For instance, `gcdCurried 10` is a
  function itself!
  ```shell
  labs::Language::Basics> :type gcdCurried 10
  gcdCurried 10 : Integer -> Integer
  ```
  `gcdCurried 10` takes an integer and returns an integer. When it
  is applied to an integer it computes the gcd of 10 and that
  integer. Other examples to illustrate partial application:
  * Incrementing is addition partially applied to 1. Notionally:
    `inc x = (add 1) x`
  * The reciprocal is division partially applied to 1. Notionally:
    `recip x = (div 1) x`
* Really `gcdUncurried` is a function of one argument. That argument
  is an ordered pair which makes `gcdUncurried (28, 20)` look just
  like a two argument function in many languages.

Small Functions
---------------

Cryptol programs are just sequences of appropriate functions applied
in the correct order. Good Cryptol features small, easy to understand
functions composed into conceptually bigger ones. This is good
computer science in general, but in Cryptol it is even more
advantageous:
* Easy to test—Cryptol's interpreter makes it very cheap to try your
  functions out.
* Encourages programming with properties—Properties can be tested
  easily and, as we'll see, proven to provide guarantees about
  code. Moreover, properties serve as another kind of documentation!

### Examples

```
abs : Integer -> Integer
abs n = if n >= 0 then n else -n

absNonnegative : Integer -> Bit
property absNonnegative x = abs x >= 0
```

* `abs : Integer -> Integer` is the type signature for `abs`.
* `abs n = if n >= 0 then n else -n` is the definition for `abs` (or function body).
* `property absNonnegative ...` is a property we expect the function to have.
* `:check absNonnegative` checks this property with
    random tests. It's super cheap unit testing!
  ```shell
  labs::Language::Basics> :check absNonnegative
  Using random testing.
  Passed 100 tests.
  ```
* Cryptol's `if ... then ... else` is much like C's ternary operator
  `?`...`:`. It is not like the `if ... then ... else` control structure.
* The reserved word `property` documents that definition's intention.
* We can go a step further and `:prove` this property:
  ```shell
  labs::Language::Basics> :prove absNonnegative
  Q.E.D.
  (Total Elapsed Time: 0.032s, using Z3)
  ```
* Also Cryptol's `:check` will check all functions marked as
  properties in one go and, you guessed it, `:prove` works similarly.

A little more involved example follows.

```
gcd : Integer -> Integer -> Integer
gcd m n = gcd' (abs m) (abs n)
  where
    gcd' : Integer -> Integer -> Integer
    gcd' x y = if y == 0 then x else gcd' y (x % y)

/* This property states that gcd x y is a divisor of both x and y */
gcdDividesBoth' : Integer -> Integer -> Bit
property gcdDividesBoth' x y
    =  x % (gcd x y) == 0
    /\ y % (gcd x y) == 0
```

* `where` introduces locally scoped definitions. (Mathematicians use
  the word "where" in a similar fashion.)
* The function `gcd'` is scoped within `gcd`.
* The function `gcd'` is recursive.
* Let's check `gcdDividesBoth'`:
  ```shell
  labs::Language::Basics> :check gcdDividesBoth'
  Using random testing.
  Passed 100 tests.
  ```
* It seems okay, yet `gcdDividesBoth' 0 0` gives a division by 0 error.
  ```shell
  labs::Language::Basics> gcdDividesBoth' 0 0
  division by 0
  ```
* We could perhaps have found that with more testing:
  ```shell
  labs::Language::Basics> :set tests=1000
  labs::Language::Basics> :check gcdDividesBoth'
  Using random testing.
  ERROR for the following inputs:
  0
  0
  division by 0
  ```
* Since `:check` uses randomly generated tests the failing result may
  be intermittent.
* Properties are useful and sometimes may be proven, but you must
  remember that _**properties that pass `:check` are not
  guarantees!**_ That is: _**`:check` is evidence, `:prove` is
  proof!**_
* Properties may be partially applied: `:check gcdDividesBoth' 0`
  finds the problem faster since it only is using random values for
  the second argument.
* **Warning**: `:prove gcdDividesBoth'` will never complete. If you
  issue that command, you'll need to issue the abort sequence (often
  `Control-C`) once or twice to interrupt and regain control. The
  reason this proof won't complete is too technical for the moment.

Let's patch up that property. (You surely noticed the prime (`'`) in
the property name which is a giveaway that is not quite the property
I have in mind.)

```
gcdDividesBoth : Integer -> Integer -> Bit
property gcdDividesBoth x y
    = if z == 0
       then True
       else x % z == 0 /\ y % z == 0
  where
    z = gcd x y
```

* Functions may have many associated properties.
* Properties can have locally scoped definitions.
* Property corner-cases are handled with `if ... then ... else`.
* Another way to write the conditional part of the property that's a
  bit cooler: `z != 0 ==> x % z == 0 /\ y % z == 0`. You can read that
  as, "nonzero _z_ implies _z_ divides both _x_ and _y_".
* WARNING: `:prove gcdDividesBoth` will never complete. Don't try it
  unless your favorite key combination is the abort sequence (often
  `Control-C`) on your computer.

Writing Loops
-------------

### Or not...

* Many of Cryptol's operators naturally extend elementwise over nested
  sequences to any depth.
```shell
labs::Language::Basics> [[[2, 3], [5, 7]], [[11, 13], [17, 19]]] + [[[0, 1], [1, 2]], [[3, 5], [8, 13]]]
[[[2, 4], [6, 9]], [[14, 18], [25, 32]]]
```

* So we don't have to write loops within loops to process these sorts
of multidimensional arrays.
* All the arithmetic, bitwise logical and comparison operators work
  elementwise over nested sequences!

### Loop indices

Enumerations serve to provide the indices to loops.

```shell
labs::Language::Basics> [1..10]
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
labs::Language::Basics> [1, 3..10]
[1, 3, 5, 7, 9]
```

#### Infinite indices

You can have "infinite" enumerations with `...`.

```shell
labs::Language::Basics> [1...]
[1, 2, 3, 4, 5, ...]
```

So long as only a finite prefix of any "infinite" calculation is needed we're fine.

### Loops to accumulate a value

Loops to accumulate a value are often simple calculations over indices.

```shell
labs::Language::Basics> sum [1..100]
5050
```

### Loops with functions on the indices

Loops with functions on the indices are written as sequence comprehensions.

```shell
labs::Language::Basics> [n^^3 | n <- [0..10]]
[0, 1, 8, 27, 64, 125, 216, 343, 512, 729, 1000]
```

Star Trek's (T.O.S.) warp factor light speed multipliers!

### Loops that modify an accumulator in place

Loops that modify an accumulator in place become self-referential
sequence comprehensions. The following example illustrates this.

Simple Block Encryption Example
-------------------------------

```
keyExpand : [32] -> [10][32]
keyExpand key = take roundKeys // take leverages the type signature
  where
    roundKeys : [inf][32]  // a conceptually infinite list
    roundKeys = [key] # [roundKey <<< 1 | roundKey <- roundKeys]

encrypt : [32] -> [32] -> [32]
encrypt key plainText = cipherText
  where
    roundKeys = keyExpand key
    roundResults = [plainText] # [ roundResult ^ roundKey
                                 | roundResult <- roundResults
                                 | roundKey <- roundKeys
                                 ]
    cipherText = last roundResults
```

Many block ciphers are just variations of the above theme. Here's a sample of it in action:

```shell
labs::Language::Basics> encrypt 0x1337c0de 0xdabbad00
0x6157c571
labs::Language::Basics> encrypt 0 0xdabbad00
0xdabbad00
```

The latter shows that you can still write bad crypto with Cryptol! 😉

Notice that both `roundKeys` in `keyExpand` and `roundResults` in
`encrypt` are self-referential sequences, a paradigm that will often
occur when coding up cryptography.

Laziness
--------

Cryptol's evaluation strategy is
[lazy](https://en.wikipedia.org/wiki/Lazy_evaluation)
a.k.a. "call-by-need". I.e., computations are not performed until
necessary. So
```
lazyAbsMin : Integer -> Integer -> Integer
lazyAbsMin x y = if x == 0 then 0 else min (abs x) (abs y)
```
Does not produce an error when `x` is zero, regardless of the value of `y`. For instance:
```shell
labs::Language::Basics> lazyAbsMin 1 (0/0)

division by 0
labs::Language::Basics> lazyAbsMin 0 (0/0)
0
```

Less Common Operators
---------------------

Function equality: `===` and `!==`. These are mostly used to state
properties about functions over a finite domain.

```
add8 : [4] -> [4]
add8 x = x + 8
sub8 : [4] -> [4]
sub8 x = x - 8
```

```shell
labs::Language::Basics> :prove add8 === sub8
Q.E.D.
(Total Elapsed Time: 0.014s, using Z3)
```

There is also an `:exhaust` command for finite domains. On occasion
the machinery behind `:prove` gets overwhelmed and, on small enough
domains, exhausting works in a reasonable time.

```shell
labs::Language::Basics> :exhaust add8 === sub8
Using exhaustive testing.
Passed 16 tests.
Q.E.D.
```

The `:check` command is smart enough to notice small enough domains
and switch to exhaustion automagically:

```shell
labs::Language::Basics> :check add8 === sub8
Using exhaustive testing.
Passed 16 tests.
Q.E.D.
```

Judicious Type System Usage
---------------------------

### Don't let the type system do your work

Cryptol's type system tries to infer the types of functions lacking a
type signature. Sometimes it comes up with a more general type than
you were imagining. This causes problems:
* Perhaps you only want your function to be applicable on a smaller
  set of types (usually minor, but occasionally major).
* Error messages can become even more incomprehensible! (Major)

### Do let the type system work for you

Type signatures for functions are wonderful bits of documentation. It
is much easier to see what's going on if you use type synonyms and
signatures.

* cleaner code
* easier for other tools to consume/reason about

### Provide additional types to aid in debugging

Many of the errors in coding Cryptol will be instances of type
mismatching. If you can't see your problem based on the error message,
try adding more type annotations. This:
* makes the interpreter do less work trying alternative possibilities
and, consequently, can make error messages more comprehensible
* reduces the body of code to examine for bugs (a sort of binary bug
search)
* can get you to notice where you blew it

Here Abide Monsters
-------------------

Following is some code that is needed to make the Cryptol within this
document valid, but is not discussed for pedagogical reasons. You may
read on, but you have been warned...

The following code does serve to illustrate the type signature and
function definitions can be separated within a file. A practice that
is _**strongly**_ discouraged.

```
gcdCurried = gcd
gcdUncurried = uncurry gcdCurried
```

These also illustrate higher order functions. We define the _function_
`gcdCurried` in terms of the _function_ `gcd` without mentioning
arguments. We use the built in `uncurry` higher-order function which
takes a two argument curried function and returns an uncurried version
(a one argument function operating on an ordered pair).

Postface
--------

Go forth and write correct cryptographic algorithms!

repository: gavsaint/cryptol-course

repo_name: gavsaint/cryptol-course


text:
---
layout: post
keywords: AngularJS, NodeJS, Couchbase, retrospective, Rafael Audy Glanzner
---

Since I have been working for almost an year in a big project built with AngularJS, NodeJS and Couchbase I thought it would be a good idea to do a small review and register the coolest things I learned and some of the pitfalls my team encountered.

![Gandalf thinking about the project](http://mulubinba.typepad.com/.a/6a00d8341dd88553ef014e5f56216c970c-pi)

The application was targeted to be globally available and had to handle a lot of data so there was no shortage of challenges.

I'll be writing about some of those topics in the followings months, this post is a kind of a backlog.
So please shout at the comments if you want me to prioritize one of these subjects and thank you for your patience. :)

If you disagree with the way we handled one of these challenges let me know, I would love the feedback!

# AngularJS and other UI bits

## Things I enjoyed doing:

#### IE8 Support

If you are supporting IE8 and using AngularJS you have to use version 1.2. Which is a shame since 1.3 and beyond have some major improvements in performance. You are also probably need to add the es5 [shims and shams](https://github.com/es-shims/es5-shim).

#### Tests

I may write a post about this soon. :)
<br>I will talk about how to write unit and end to end tests, how to mock services and how to add test coverage.

#### SASS

#### AngularJS

One or more post about the following may be written soon:

* Interesting techniques around the __resolve method and caching__;
* How to enable __localization__;
* The best way to __structure__ 90% of your AngularJS __apps__;
* How to add __validation banners__ with scroll, focus and highlights;
* Implementing modals;
* Controller as syntax and __John Papa's style guide__;
* How to convert from __View model to DTO__ and vice versa;
* How to use __factories__ in AngularJS.

#### Dev process

I may also write one or more posts about the following:

* How to run your app in __dev and prod__ environments;
* How to set up __live browser update__ and __run the tests after changes__;
* Improving __grunt__, creating internal variables and __separating its content in different files__;
* Implementing __feature switch__;
* Adding __static analysis__;
* How to concatenate files for deploy, minify them, add version to generated assets, cach them, create a distribution folder and more.

## Challenges we faced:

![Spider man does not want to support IE8](http://cdn.meme.am/instances/500x/55274752.jpg)

#### Time to run the tests

We have loads of unit and especially end to end tests and they can take up to 9 minutes to run without the API mocks. With the mocks and different optimization techniques we managed to drop this number to 3 minutes, which is still a relevant wait.

I have some ideas to improve that in the following projects and I will discuss that in a future post.

#### Performance

We encountered some performance issues because of our UX guidelines, which made us add too many bindable controls visible on the screen at the same time. We also encountered problems optimizing directives.

I will talk about the solutions we encountered in another post.

#### IE8

Do I really have to say something? :) Supporting old versions of IE is a pain. This means not being able to migrate to newer versions of Angular and also dealing with weird CSS and Javascript problems.

#### Phantom

I was impressed by the amount of times we had e2e tests failing just on phantom for a variety of weird reasons. After we discovered that we could also have headless browser testing with chrome we completely abandoned Phantom and our lives got much better ever since :)

#### Unit tests

I think we didn't achieve a healthy ratio of unit X e2e tests. Although our code coverage was above 90% of the app a small percentage of that number came from unit tests. That means our test suite was not as fast as we would like. I will talk more about this in a future post.

#### Build process

Our build process was quite slow in the beginning due to some inefficacies and the test replication across different branches. We now changed that and the process is much quicker. I will also talk more about this in another post.

# NodeJS

![NodeJS is awesome](http://www.quickmeme.com/img/00/006fd811b42bb561542996b7ffb15bb36f25449f5f063eacac886919da848ce6.jpg)

## Things I enjoyed doing:

#### Project setup
* Using logstash, Kibana and Winston for logging;
* Using nconf to organize configuration files;
* Code organization and folder structure.

#### API
* Using JSON schema validation to validate incoming json objects;
* Converting XML to json and vice versa;
* Using Request extensively to connect to other APIs;
* Using Bluebird to transform callbacks into promises;
* Organizing Express routes;
* Restarting Node.js process on errors;
* Creating a variety of Express middlewares.

#### Tests
* Adding code coverage to Mocha tests.

#### Couchbase
* Implementing a small wrapper to the Couchbase API;
* Setting up views / documents.

# Other

#### Sharing data between javascript apps
#### Get a MAC
Seriously, get a MAC if you don't have one :)
I started with a Windows box, then changed to Linux and finally got a MAC.
Although I like Linux for programming purposes a MAC is much better is all other regards.

__Cheers :)__


text:
The data reported here come exclusively from an ongoing experiment in my house.

See the **Wiki** section for more examples apart from this readme.

This is a primer, see folder for more a more interesting dataset.

# RPi Temperature data

I got a new aluminium heat sink case for my Raspberry Pi 2 Model B. Then, I installed a DHT22 temperature and humidity sensor. Now, every minute for 5 days some readings were recorded:

* room's temperature (Celsius), sensed by the DHT22,
* relative humidity (%), also sensed by the DHT22,
* internal CPU temperature (Celsius).

![Time series plot for the three variables in the dataset](timeseries.png)

Feel free to experiment any method you like, in order to practice or to make other people practice. Can you predict the room's temperature in the short term, so that I can lower the sampling frequency by a significant amount, say, down to just 1 reading per hour? Can you explain your solution in simple terms?

 Around 10 a.m. every day, I open my room's window, so both temperature and humidity will shift a bit. Is there a way to automatically detect these outliers? Which smoothing methods can work reliably even without removing the outliers?

Also, there are very few variables. Can you do enough with so little? Choose whether to use all the variables or just the time and the room's temperature.

(Maybe it's not important, but the readings are scheduled at times HH:MM:00, yet it takes few seconds to sense and record data. This amount of time may depend on concurrent activity, which may cause heating?)

Have fun.

## Baseline model

The baseline, the placebo-level model, is the linear model for the room's temperature versus just the CPU's temperature. You see the fit is pretty good:

![Time series for room's temperature, with linear model's prediction](lm.png)

![Room's temperature prediction error](lmerr.png)

So the linear model makes a pretty decent work: it gets the room's temperature up to a *substantially negligible* error, as personally I just cannot notice a temperature shift about 2 degrees.

Moreover, the linear model, though not inherently robust, suffices here in filtering out all morning outliers.

But you see, the prediction error persists over time, as positive errors lead to subsequent positive errors, the same for negative errors, due to a likely auto-regressive process. So, can a time series model reduce the error further? This can be an exercise for practitioners.

repository: mlambardi/rpidata

repo_name: mlambardi/rpidata


text:
# Init Pipeline Jenkins Plugin

## License

This software is licensed under the MIT open source license

see https://opensource.org/licenses/MIT

## Architecture

This plugin was derived from the Jenkins Git Plugin, specifically the GitStatus.java file.

This file was adapted to catch the case(s) where the notifyCommit url references a repo for which the Jenkins server
does not have a pipeline job configured.

The Git Plugin drops the notifyCommit on the floor with a single log line to /var/log/jenkins/jenkins.log in this case.

We listen alongside and catch that case, calling an external golang binary to configure and build a new pipeline.


text:
# Fundamentals of Machine Learning Course at DSR

## Local Environment

To have these notebooks running locally, you need to follow these steps:

1) Clone the repo: `git clone https://github.com/dvgodoy/ML_Fundamentals.git`
2) Inside the folder of the clone repo, create a conda environment: `conda env create -f environment.yml -n fund`
3) Activate the virtual environment: `conda activate fund`
4) Run Jupyter: `jupyter notebook`

Obs.: environment.yml is conda's equivalent of requirements.txt
Obs.: you need to have [GraphViz](https://www.graphviz.org/) installed as well

## Binder

You can use [Binder](https://mybinder.org/) to open the notebooks on an executable environment - just click on the badge: [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/dvgodoy/ML_Fundamentals/master)

After it starts, it will open a familiar ***Jupyter notebook interface***! Just run ***all cells***, wait for the ***interactive plots*** to load and you're good to go!


text:
# Sphaerotheca parietariae (Schwarzman) U. Braun & Simonyan, 1985 SPECIES

#### Status
SYNONYM

#### According to
The Catalogue of Life, 3rd January 2011

#### Published in
in Braun, Zentbl. Mikrobiol. 140: 163 (1985)

#### Original name
Sphaerotheca macularis f. parietariae Schwarzman, 1961

### Remarks
null

repository: mdoering/backbone

path: /life/Fungi/Ascomycota/Leotiomycetes/Erysiphales/Erysiphaceae/Podosphaera/Podosphaera parietariae/ Syn. Sphaerotheca parietariae/README.md

repo_name: mdoering/backbone