text:
# VAMPIRE <img src="figures/bat.png" width="60">

VAriational Methods for Pretraining In Resource-limited Environments

Read paper [here](https://arxiv.org/abs/1906.02242).

## Citation

```
@inproceedings{vampire,
 author = {Suchin Gururangan and Tam Dang and Dallas Card and Noah A. Smith},
 title = {Variational Pretraining for Semi-supervised Text Classification},
 year = {2019},
 booktitle = {Proceedings of ACL},
}
```

## Installation

Install necessary dependencies via `requirements.txt`, which will include the latest unreleased install of `allennlp` (from the `master` branch).

```
pip install -r requirements.txt
```

Install the spacy english model with:

```
python -m spacy download en
```

Verify your installation by running:

```
SEED=42 pytest -v --color=yes vampire
```

All tests should pass.

## Install from Docker

Alternatively, you can install the repository with Docker.

First, build the container:

```
docker build -f Dockerfile --tag vampire/vampire:latest .
```

Then, run the container:

```
docker run -it vampire/vampire:latest
```

This will open a shell in a docker container that has all the dependencies installed.

## Download Data

Download your dataset of interest, and make sure it is made up of json files, where each line of each file corresponds to a separate instance. Each line must contain a `text` field, and optionally a `label` field.

In this tutorial we use the AG News dataset hosted on AllenNLP. Download it using the following script:

```
sh scripts/download_ag.sh
```

This will make an `examples/ag` directory with train, dev, test files from the AG News corpus.

## Preprocess data

To make pretraining fast, we precompute fixed bag-of-words representations of the data.

```
python -m scripts.preprocess_data \
            --train-path examples/ag/train.jsonl \
            --dev-path examples/ag/dev.jsonl \
            --tokenize \
            --tokenizer-type spacy \
            --vocab-size 30000 \
            --serialization-dir examples/ag
```

This script will tokenize your data, and save the resulting output into the specified `serialization-dir`.

Alternatively, under `https://s3-us-west-2.amazonaws.com/allennlp/datasets/ag-news/preprocessed.tar", we have a tar file containing a pre-processed AG news data (with vocab size set to 30K).

Run

```
curl -Lo examples/ag/ag.tar https://s3-us-west-2.amazonaws.com/allennlp/datasets/ag-news/vampire_preprocessed_example.tar
tar -xvf examples/ag/ag.tar -C examples/
```

to access its contents.

In `examples/ag` (after running the `preprocess_data` module or unpacking `ag.tar`), you should see:

* `train.npz` - pre-computed bag of word representations of the training data
* `dev.npz` - pre-computed bag of word representations of the dev data
* `vampire.bgfreq` - background word frequencies
* `vocabulary/` - AllenNLP vocabulary directory

This script also creates a reference corpus to calcuate NPMI (normalized pointwise mutual information), a measure of topical coherence that we use for early stopping. By default, we use the validation data as our reference corpus. You can supply a `--reference-corpus-path` to the preprocessing script to use your own reference corpus.

In `examples/ag/reference`, you should see:

* `ref.npz` - pre-computed bag of word representations of the reference corpus (the dev data)
* `ref.vocab.json` - the reference corpus vocabulary

## Pretrain VAMPIRE

Set your data directory and vocabulary size as environment variables:

```
export DATA_DIR="$(pwd)/examples/ag"
export VOCAB_SIZE=30000
```

If you're training on a dataset that's to large to fit into RAM, run VAMPIRE in lazy mode by additionally exporting:

```
export LAZY=1
```

Then train VAMPIRE:

```
python -m scripts.train \
            --config training_config/vampire.jsonnet \
            --serialization-dir model_logs/vampire \
            --environment VAMPIRE \
            --device -1
```

This model can be run on a CPU (`--device -1`). To run on a GPU instead, run with `--device 0` (or any other available CUDA device number).

This command will output training logs at `model_logs/vampire`.

For convenience, we include the `--override` flag to remove the previous experiment at the same serialization directory.

## Inspect topics learned

During training, we output the learned topics after each epoch in the serialization directory, under `model_logs/vampire`.

After your model is finished training, check out the `best_epoch` field in `model_logs/vampire/metrics.json`, which corresponds to the training epoch at which NPMI is highest.

Then open up the corresponding epoch's file in `model_logs/vampire/topics/`.

## Use VAMPIRE with a downstream classifier

Using VAMPIRE with a downstream classifier is essentially the same as using regular ELMo. See [this documentation](https://github.com/allenai/allennlp/blob/master/docs/tutorials/how_to/elmo.md#using-elmo-with-existing-allennlp-models) for details on how to do that.

This library has some convenience functions for including VAMPIRE with a downstream classifier.

First, set some environment variables:
* `VAMPIRE_DIR`: path to newly trained VAMPIRE
* `VAMPIRE_DIM`: dimensionality of the newly trained VAMPIRE (the token embedder needs it explicitly)
* `THROTTLE`: the sample size of the data we want to train on.
* `EVALUATE_ON_TEST`: whether or not you would like to evaluate on test

```
export VAMPIRE_DIR="$(pwd)/model_logs/vampire"
export VAMPIRE_DIM=81
export THROTTLE=200
export EVALUATE_ON_TEST=0
```

Then, you can run the classifier:

```
python -m scripts.train \
            --config training_config/classifier.jsonnet \
            --serialization-dir model_logs/clf \
            --environment CLASSIFIER \
            --device -1
```

As with VAMPIRE, this model can be run on a CPU (`--device -1`). To run on a GPU instead, run with `--device 0` (or any other available CUDA device number)

This command will output training logs at `model_logs/clf`.

The dataset sample (specified by `THROTTLE`) is governed by the global seed supplied to the trainer; the same seed will result in the same subsampling of training data. You can set an explicit seed by passing the additional flag `--seed` to the `train` module.

With 200 examples, we report a test accuracy of `83.9 +- 0.9` over 5 random seeds on the AG dataset. Note that your results may vary beyond these bounds under the low-resource setting.

## Troubleshooting

If you're running into issues during training (e.g. NaN losses), checkout the [troubleshooting](TROUBLESHOOTING.md) file.

repository: heddayam/vampire

repo_name: heddayam/vampire


text:
# Hyperledger Fabric Network boilerplate
This repo is a snippet of the fabcar [fabric sample](https://github.com/hyperledger/fabric-samples) with the basic network. It contains a fabric network with 1 peer and 1 CA.

## Starting
Before starting, you will need to pull all the images of Hyperledger fabric to your desktop and tag them as latest. We included a script to do this. By default it will try to pull in `1.3.0` but you can pull a custom version by adding the version as a parameter.
```bash
./scripts/bootstrap.sh [optional_custom_version]
```
Starting this network requires you to run following command. This will automatically setup your docker network using docker-compose and install your chaincode.
```bash
./scripts/startFabric.sh
```

license: MIT

repository: meiremans/hypercoin

branch_name: refs/heads/master

extension: md

license_type: permissive

repo_name: meiremans/hypercoin


text:
---
layout: post
status: publish
published: true
title: How to use the new Memoji feature on iOS 12
author:
  display_name: Charlotte Grainger
  login: LottieBetty
  email<EMAIL_ADDRESS>  url: ''
author_login: LottieBetty
author_email<EMAIL_ADDRESS>wordpress_id: 18996
wordpress_url: https://blog.buymobiles.net/?p=18996
date: '2018-09-27 12:32:01 +0100'
date_gmt: '2018-09-27 11:32:01 +0100'
categories:
- Features
tags:
- apple
- iphone
- iPhone X
- iOS 12
- Animoji
- Memoji
- iPhone Xs
---

Turn yourself into an animated emoji, using Memoji on iOS 12.

The release of iOS 12 has brought with it some cool updates to iPhone. So, if you’ve got an iPhone X, or even a brand-spanking-new iPhone Xs or Xs Max, there’s one fresh feature you mustn’t overlook.

Here’s what you need to know about Memoji.

First of All, What is Memoji?

If you were a teen in the early 2000s, this particular iOS 12 feature is going to give you some serious nostalgia pangs. Prepare yourself. Let me take you back to a time when we all spent far too much of our youth on a little PC game known as The Sims.

Does it ring a bell? Oh yes, many of us would whittle away the hours on this game. Without a shadow of a doubt (and I dare you to contradict me here!), the greatest feature on The Sims was at the beginning when you created your own character. Manipulating the face shape, hairstyle, and other features was oddly satisfying, as you tried to achieve personal resemblance.

Well, Apple is tapping into that feeling once again. Say hello to Memoji.

As the name suggests, the iOS 12 feature is a type of Animoji that looks just like you, or any way you choose. Rather than using premade emoji, like the ever-popular tiger, you can design your very own character.

When you’ve created your Memoji, you can use it just like an Animoji… But I’m getting ahead of myself here. Let’s talk about how you can get started.

How to Create Your Memoji

To create your own Memoji, you need to start by heading to the Messages app on your iPhone. Open a chat with someone and then look for the little Animoji icon. Next, tap on the + symbol to start crafting your Memoji.

This is where the fun begins. You’ll be presented with a creation deck much like the one you used on The Sims back in the day. You can control the facial features, hair, skin, eye-wear, and head-wear of your Memoji, to create an animated version of yourself, or a wild interpretation of your inner character.

What to Do With Your Memoji

When your Memoji is ready, you’re going to want to start using it pronto. There are two main things you can do.

First, you can use the True Depth camera to record 30-second clips of you as a Memoji, which you can send as messages. You might want to make a short video of you singing or chatting away in character.

That’s not all… With iOS 12, comes a brand-new trick you can try.  Give it a whirl – you know you want to.


text:
---
layout: post
title: Hacknews2018-11-09新闻
category: Hacknews
tags: hacknews
keywords: hacknews
---

- [Show HN: Graph-based notebook for data scientists and researchers](https://www.amie.ai)
- `显示HN:数据科学家和研究人员的基于图形的笔记本`
- [Redo, buildroot, and serializing parallel logs](https://apenwarr.ca/log/20181106)
- `重做、buildroot和序列化并行日志`
- [Ground: A Data Context Service [pdf]](https://rise.cs.berkeley.edu/wp-content/uploads/2017/03/CIDR17.pdf)
- `地面:数据上下文服务[pdf]`
- [I've compiled the best SaaS Landing pages and broke down all their secrets](https://www.cortes.design/post/best-saas-landing-page-examples-inspiration)
- `我已经编辑了最好的SaaS登录页面，并破解了它们的所有秘密`
- [Logo Adventure for C64 Terrapin Logo](https://medium.com/@donhopkins/logo-adventure-for-c64-terrapin-logo-4c684a240b53)
- `标志冒险为C64 Terrapin标志`
- [Heritability of Human Longevity Substantially Inflated Due to Assortative Mating](http://www.genetics.org/content/210/3/1109)
- `人类寿命的遗传性因选型交配而大大提高`
- [Application for Fixed Satellite Service by SpaceX](https://fcc.report/IBFS/SAT-MOD-20181108-00083)
- `SpaceX公司申请固定卫星服务`
- [Why We Need Innovative Nuclear Power](https://blogs.scientificamerican.com/observations/why-we-need-innovative-nuclear-power/?fbclid=IwAR0-yCiE67h-OS5tujbjQSqxhQqUXGDSKZV3eZLDEj0T6VpbSkvrxivOFe8)
- `为什么我们需要创新的核能`
- [Now 2.0](https://zeit.co/blog/now-2)
- `现在2.0`
- [Ford Buying San Francisco-Area E-Scooter Startup Spin](https://www.nytimes.com/reuters/2018/11/08/business/08reuters-ford-scooter.html)
- `福特收购旧金山地区的电动滑板车创业公司Spin`
- [Design Review: Key-Value Storage](https://mozilla.github.io/firefox-browser-architecture/text/0015-rkv.html)
- `设计回顾:键值存储`
- [Attack Directories, Not Caches: Side-Channel Attacks in a Non-Inclusive World [pdf]](http://iacoma.cs.uiuc.edu/iacoma-papers/ssp19.pdf)
- `攻击目录，而非缓存:非包容世界中的侧通道攻击[pdf]`
- [Making an Ancient Roman Murderer](https://www.laphamsquarterly.org/roundtable/making-ancient-roman-murderer)
- `制造一个古罗马杀人犯`
- [Linux on Dex](https://www.linuxondex.com/)
- `Linux on敏捷`
- [New discovery throws light on mystery of pyramids' construction](https://www.theguardian.com/world/2018/nov/06/new-discovery-throws-light-on-mystery-of-pyramids-construction)
- `新的发现揭开了金字塔建造的神秘面纱`
- [Tell HN: Amazon now owns <IP_ADDRESS>/8](item?id=18407173)
- `告诉HN:亚马逊现在拥有<IP_ADDRESS>/8`
- [A time capsule in an Austrian salt mine](https://www.gq.com/story/memory-of-mankind-time-capsule)
- `奥地利盐矿里的时间胶囊`
- [Brain activity pattern may be early sign of schizophrenia](http://news.mit.edu/2018/brain-activity-pattern-sign-schizophrenia-1108)
- `大脑活动模式可能是精神分裂症的早期征兆`
- [Show HN: Vyne – Automated API Orchestration](https://vyne.co)
- `显示HN: Vyne -自动API编排`
- [How Biologists Are Creating Life-Like Cells from Scratch](https://www.nature.com/articles/d41586-018-07289-x)
- `生物学家是如何从零开始创造类生命细胞的`
- [How to Design Software Good](https://www.haiku-os.org/docs/HIG/index.xml)
- `如何设计好软件`
- [Key Papers in Deep Reinforcement Learning](https://spinningup.openai.com/en/latest/spinningup/keypapers.html)
- `深度强化学习的关键论文`
- [At Doomed Flight’s Helm, Pilots May Have Been Overwhelmed in Seconds](https://www.nytimes.com/2018/11/08/world/asia/indonesia-plane-crash-last-moments.html)
- `在注定失败的飞行舵轮上，飞行员可能在几秒钟内就被压垮了`
- [Why We Need Innovative Nuclear Power](https://blogs.scientificamerican.com/observations/why-we-need-innovative-nuclear-power/)
- `为什么我们需要创新的核能`
- [Why do all websites look the same?](https://medium.com/s/story/on-the-visual-weariness-of-the-web-8af1c969ce73)
- `为什么所有的网站看起来都一样?`
- [Mac mini Late 2018 Teardown](https://www.ifixit.com/Teardown/Mac+mini+Late+2018+Teardown/115210)
- `Mac mini在2018年末拆卸`
- [The making of an opioid epidemic](https://www.theguardian.com/news/2018/nov/08/the-making-of-an-opioid-epidemic)
- `阿片类药物的流行`
- [To John Wheeler, the race to explain time was personal](http://nautil.us/issue/66/clockwork/haunted-by-his-brother-he-revolutionized-physics-rp)
- `对约翰·惠勒(John Wheeler)来说，解释时间的竞赛是私人的`
- [U.S. Secret Service Warns ID Thieves Are Abusing USPS’s Mail Scanning Service](https://krebsonsecurity.com/2018/11/u-s-secret-service-warns-id-thieves-are-abusing-uspss-mail-scanning-service/)
- `美国特勤局警告身份窃贼正在滥用美国邮政的邮件扫描服务`
- [Optimal Shard Placement in a Petabyte Scale Elasticsearch Cluster](https://underthehood.meltwater.com/blog/2018/11/05/optimal-shard-placement-in-a-petabyte-scale-elasticsearch-cluster/)
- `在Petabyte尺度弹性搜索集群中最优的碎片放置`
- [How We Built a Cost-Based SQL Optimizer](https://www.cockroachlabs.com/blog/building-cost-based-sql-optimizer/)
- `我们如何构建基于成本的SQL优化器`
- [Sprint Is Throttling Skype, Study Finds](https://www.bloomberg.com/news/articles/2018-11-08/sprint-is-throttling-microsoft-s-skype-service-study-finds)
- `研究发现，Sprint正在扼杀Skype`
- [Ruby 2.x Universal RCE Deserialization Gadget Chain](https://www.elttam.com.au/blog/ruby-deserialization/)
- `Ruby 2。xuniversal RCE反序列化小工具链`
- [Golden Thoughts for a New Nuclear Age – from a new biography of René Girard](http://churchlife.nd.edu/2018/02/28/golden-thoughts-for-a-new-nuclear-age/)
- `新核时代的黄金思想——来自雷内吉拉尔的新传记`
- [8×8 has acquired the Jitsi team and technology from Atlassian](https://jitsi.org/news/we-have-a-new-home-and-it-wont-impact-you/)
- `8×8获得了从Atlassian Jitsi团队和技术`
- [The Human Diagnosis Project Is Looking for a Design Lead](https://www.humandx.org/team?gh_jid=4097301002)
- `人类诊断项目正在寻找设计线索`
- [$vau: the ultimate abstraction (2010) [pdf]](https://web.wpi.edu/Pubs/ETD/Available/etd-090110-124904/unrestricted/jshutt.pdf)
- `$vau:终极抽象(2010)[pdf]`
- [Predicting stock prices with deep learning: A Keras LSTM model](https://heartbeat.fritz.ai/using-a-keras-long-shortterm-memory-lstm-model-to-predict-stock-prices-a08c9f69aa74)
- `深度学习预测股票价格:Keras LSTM模型`
- [Open-sourcing FBGEMM for state-of-the-art server-side inference](https://code.fb.com/ml-applications/fbgemm/)
- `开源FBGEMM，用于最先进的服务器端推理`
- [Erlang/OTP on OpenBSD](http://blog.obligd.com/posts/erlang-otp-on-openbsd.html)
- `Erlang / OTP OpenBSD`
- [Phenomenally saddening: Inside the sordid world of America's for-profit colleges](https://www.theguardian.com/film/2018/nov/09/fail-state-documentary-for-profit-colleges)
- `令人悲哀的是:在美国盈利性大学肮脏的世界里`
- [The weird and wonderful world of constructive mathematics (2017) [pdf]](https://home.sandiego.edu/~shulman/papers/rabbithole.pdf)
- `《构造数学的奇妙世界》(2017)[pdf]`
- [Spinning Up in Deep RL](https://blog.openai.com/spinning-up-in-deep-rl/)
- `在RL深处旋转`
- [What Makes .NET Core So Special?](https://www.blog.jamesmichaelhickey.com/What-Makes-NET-Core-So-Special-Why-You-Should-Use-NET-Core/)
- `是什么让。net Core如此特别?`
- [Application-Layer DDoS Attack Protection with HAProxy](https://www.haproxy.com/blog/application-layer-ddos-attack-protection-with-haproxy/)
- `使用HAProxy的应用层DDoS攻击保护`
- [Utah man dies from rabies, the first in the state since 1944](https://www.deseretnews.com/article/900041016/utahn-dies-from-rabies-a-first-since-1944.html)
- `犹他州一名男子死于狂犬病，这是该州1944年以来的首例狂犬病病例`
- [Chart of the Decade: Why You Shouldn’t Trust Every Scientific Study You See](https://www.motherjones.com/kevin-drum/2018/11/chart-of-the-decade-why-you-shouldnt-trust-every-scientific-study-you-see/)
- `十年图表:为什么你不应该相信你看到的每一项科学研究`
- [Herman Hesse: Outside Man](https://www.weeklystandard.com/christoph-irmscher/herman-hesse-outside-man)
- `赫尔曼·黑塞:外面的人`
- [Against software development](http://www.rntz.net/post/against-software-development.html)
- `对软件开发`
- [Specification gaming examples in AI](https://docs.google.com/spreadsheets/u/1/d/e/2PACX-1vRPiprOaC3HsCf5Tuum8bRfzYUiKLRqJmbOoC-32JorNdfyTiRRsR7Ea5eWtvsWzuxo8bjOxCG84dAg/pubhtml)
- `说明游戏例子在AI`
- [First FAA waiver issued to fly a large drone beyond line of sight](https://www.aviationtoday.com/2018/10/19/ge-owned-avitas-granted-first-large-commercial-bvlos-waiver/)
- `美国联邦航空局首次发布豁免令，允许大型无人机飞出视线之外`
- [Production Rules (Read-Eval-Print-λove)](https://leanpub.com/readevalprintlove004/read)
- `生产规则(Read-Eval-Print-λove)`
- [China's AI news anchors](https://www.inkstonenews.com/tech/xinhua-and-sogou-show-news-anchors-powered-artificial-intelligence/article/2172460)
- `中国人工智能新闻主播`
- [BigInt: Arbitrary precision integers in JavaScript](https://github.com/tc39/proposal-bigint)
- `BigInt: JavaScript中的任意精度整数`
- [Real time numbers recognition (MNIST) on an iPhone with CoreML](https://www.liip.ch/en/blog/numbers-recognition-mnist-on-an-iphone-with-coreml-from-a-to-z)
- `带有CoreML的iPhone上的实时数字识别(MNIST)`
- [China has developed a virtual anchor to deliver the news](https://www.cnn.com/2018/11/09/media/china-xinhua-ai-anchor/index.html)
- `中国开发了一个虚拟主播来播报新闻`
- [At China’s Internet Conference, a Darker Side of Tech Emerges](https://www.nytimes.com/2018/11/08/technology/china-world-internet-conference.html)
- `在中国互联网大会上，科技的阴暗面浮现`
- [96-core ARM supercomputer using the NanoPi-Fire3](https://climbers.net/sbc/nanopi-fire3-arm-supercomputer/)
- `使用NanoPi-Fire3的96核ARM超级计算机`
- [Micron Kicks Off Mass Production of 12 Gb LPDDR4X DRAM Chips](https://www.anandtech.com/show/13569/micron-kicks-off-mass-production-of-12-gb-lpddr4x-dram-chip)
- `美光开始大规模生产12 Gb LPDDR4X DRAM芯片`
- [Amazon's AbeBooks backs down after booksellers stage global protest](https://www.theguardian.com/books/2018/nov/08/amazon-abebooks-backs-down-after-booksellers-stage-global-protest?CMP=twt_gu)
- `在全球书商发起抗议后，亚马逊旗下的AbeBooks做出了让步`


text:
# Static Social Share Buttons

Simple static socials buttons with share count!
I've developed these buttons for my blog: [www.savjee.be](http://savjee.be/2015/01/Creating-static-social-share-buttons/).

Support for Twitter and Facebook buttons. The square version also supports Reddit.

## Demo
There are two versions: traditional and square. This is what they look like:

**Traditional ([Live demo](http://savjee.github.io/static-social-buttons/traditional/index.html)):**

![](http://savjee.github.io/static-social-buttons/screenshots/traditional.png)

**Square ([Live demo](http://savjee.github.io/static-social-buttons/square/index.html)):**

![](http://savjee.github.io/static-social-buttons/screenshots/square.png)

repository: hansonj-codes/static-social-buttons

repo_name: hansonj-codes/static-social-buttons


text:
# word2graph2vec

Representation learning for words and Labelled Documents by jointly representing words, labels and documents as nodes of a graph and then learning low dimensional representation of it.

The method is based on the paper by Tang J. et. al (http://research.microsoft.com/pubs/255567/fp292-Tang.pdf).

# Current state of project

Current task is sentiment analysis on IMDB dataset with two labels (sentiments - positive, negative). It constructs word-word, word-doc and word-label graph from data and trains a skip-gram based model for learning low dimensional representation of nodes in graph.

Currently it trains model only on word-word graph. Training jointly on all three graphs is in development phase. We are running experiments.

# Results

Current implementaion has quite poor results. It gives ~55% accuracy on test.

# State - Development

The project is still in development state. We will change the state of the project to alpha when it gives decent performance of dataset

# Authors

For any queries related to project you can contact the authors:

Shashank Gupta -<EMAIL_ADDRESS>
Karan Chandnani -<EMAIL_ADDRESS>
Nishant Prateek -<EMAIL_ADDRESS>


text:
#PERKT

##Overview
Developed for HackIllinois Spring 2015, PERKT is a prototype of a security platform that utilizes facial recognition instead of the typical text-based password. Here's the website!

##Usage
This script takes a picture with the webcam and first identifies the faces present. Then it takes these faces and compares them to a directory of authorized faces already loaded within the script. If a match is found, the person is authorized and can move on, however, if the person is not authorized then a security system goes off triggering a rapid firing flashlight and a 4X4X4 led cube. There is a website to complement the implementation of this the PERKT security system with a list of the developers and a video tutorial demonstrating its function.

##Modules Included
- Facial Recogonition software created by Abishek Verma
- Back-End development done with python by Timothy Ong
- Website done by Chris Jang
- Arduino code and Python compliment done by Akshay Ivatury

###Hardware Included
-  Arduino
-  Redboard
-  Servo motor
-  hacked flashlight
-  Proto Board
-  64 LED's
-  Jumper Cables


text:
# NRCAN's Energuide API

CircleCI Status: [![CircleCI](https://circleci.com/gh/cds-snc/nrcan_api.svg?style=svg)](https://circleci.com/gh/cds-snc/nrcan_api)

This is the API for NRCAN's Energuide data.

## Running the server

#### Quickstart

```sh
# make sure to set your NRCAN_ENGINE_API_KEY environment variable
export NRCAN_ENGINE_API_KEY=your_apollo_engine_api_key

# build and run the API
make watch
```

`make watch` will do a few things for you
- install all missing dependencies
- build the app
- serve the API locally
- rebuild and restart if you make any changes

Once it's running, check it out at [http://localhost:3000/](http://localhost:3000/).
Or [here's an example query](http://localhost:3000/graphiql?query=%7B%0A%20%20dwellings(%0A%20%20%20%20filters%3A%20%5B%0A%20%20%20%20%20%20%7Bfield%3A%20dwellingForwardSortationArea%20comparator%3A%20eq%20value%3A%20%22C1A%22%7D%0A%20%20%20%20%5D%0A%20%20)%20%7B%0A%20%20%20%20results%20%7B%0A%20%20%20%20%20%20yearBuilt%0A%20%20%20%20%20%20city%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D) to get you going.

### Manually starting the server

Running the sever requires a MongoDB server running somewhere and the following environmental variables defined:

* NRCAN_ENGINE_API_KEY
* NRCAN_DB_CONNECTION_STRING
* NRCAN_DB_NAME
* NRCAN_COLLECTION_NAME

With those defined you can run the server like this:

```sh
> yarn && yarn run build
> NRCAN_DB_CONNECTION_STRING="mongodb://localhost:27017" \
	NRCAN_DB_NAME="energuide" \
	NRCAN_COLLECTION_NAME="dwellings" \
	NRCAN_ENGINE_API_KEY="your_apollo_engine_api_key" yarn start
```

The API server is available as a Docker container and can be run with:
```
docker run -it -e "NRCAN_ENGINE_API_KEY=your_apollo_engine_api_key" -e "NRCAN_DB_CONNECTION_STRING=mongodb://localhost:27017" -e "NRCAN_DB_NAME=energuide" -e "NRCAN_COLLECTION_NAME=dwellings" --net="host" -p 3000:3000 cdssnc/nrcan_api

```

The container will need network connectivity to the database (obviously) so
ensure that docker networking is setup up in such a way to allow that. The
above command uses the `--net="host"` option to connect the container to the
SQLServer database installed on the host machine. Adjust as needed.
Monitoring and caching is done with [Apollo Engine](https://engine.apollographql.com), hence the API key.

Assuming the DB credentials/connectivity is correct, the command above will
start a graphql endpoint that will respond to the following `curl` commands:

```sh
curl -s -H "Content-Type: application/json" -d '{"query": "{dwellings(filters: [{field: dwellingForwardSortationArea comparator: eq value: \"C1A\"}]){ results { yearBuilt city }}}"}'  "localhost:3000/graphql"
```

It is also possible to access the API via `/graphiql` which will serve up a Graphical IDE to allow you to explore the functionality the API offers and run queries against it.

## Running the tests

Run the unit tests

```sh
yarn test
```

Run the integration tests

```sh
yarn integration
```

gha_language: Python

gha_license_id: MIT


text:
# Data Warehouse

## Summary
The purpose of this Redshift database is to understand what, where and how each user is listening to the songs in Sparkify. The analytial goals is to find out what is making the free tier users switch to paid tier and why paid users are downgrading to free tier through their listening habits.

1. Build an ETL pipeline that extracts data from S3
1. Stages song and log data in Redshift
1. Transforms data into a set of dimensional tables and fact table for analytics team to continue finding insights in what songs their users are listening to

## Project Datasets
### Song Dataset
**S3 Song Data Source** - `s3://udacity-dend/song_data`
- Subset of the real data from [Million Song Dataset](http://millionsongdataset.com/)
- Each file is in JSON format and contains metadata about a song and the artist of that song

### Log Dataset
**S3 Log Data Source** - `s3://udacity-dend/log_data`
- Log files are in JSON format generated by this [event simulator](https://github.com/Interana/eventsim) based on the songs in the dataset above
- These simulate activity logs from a music streaming app based on specified configurations
- The dataset are partitioned by year and month

## How to run the Python scripts
1. `create_tables.py` will create staging, fact, and dimension tables, dropping any tables with conflicting names beforehand.
1. `etl.py` loads data from S3 to staging tables, and transforms data into a set of dimensional tables.

## Database schema design for Song Play Analysis
![](https://raw.githubusercontent.com/gyhou/millionsongs/master/img/Song_ERD.png)

- Create a star schema optimized for queries on song play analysis
- 2 staging tables (log_data, song_data), 1 fact table (songplays), and 4 dimension tables (users, songs, artists, time).

### Staging Tables
Load S3 bucket data into staging tables first then transform the data into smaller tables.

Data and Column type are modified as needed from `stl_load_errors` on Redshift query editor.
<br>
`select * from stl_load_errors`
- VARCHAR: string values
- FLOAT: floating-point numbers with decimals
- INTEGER: four-byte numbers (integers)
- BIGINT: eight-byte integers
No constraints on staging tables as they should be a clean copy of the source, so no filtering happening at this stage.
- staging_events: Stores entire data from `'s3://udacity-dend/log_data'`
- staging_songs: Stores entire data from `'s3://udacity-dend/song_data'`

### Fact Table
- songplay: Stores information of each song played in the logs data
  - Aggregates the crucial information of what and when the user is listening
  - songplay_id as `IDENTITY PRIMARY KEY`, for each record the id will increment automatically as each record is unique

### Dimension Tables
- users: Stores the information of each unique users using the app
  - user_id as `PRIMARY KEY`, each user_id is associated with only one user
- songs: Stores the information of each unique songs in the music database
  - song_id as `PRIMARY KEY`, each song_id is associated with only one song
- artists: Stores the information of each unique artists in the music database
  - artist_id as `PRIMARY KEY`, each artist_id is associated with only one artist
- time: timestamps of records in songplays broken down into specific units
  - This allows us to better understand what period of the day/week/month/year do people listen to music
  - start_time as `PRIMARY KEY`, each record requires a start_time to initialize the date time record

### Dealing with duplicate records
Since Redshift does not enforce uniqueness for any column, we have to implement additional queries to achieve distinct rows.
1. Locate and delete the duplicate rows
1. Create a new table to hold the distinct rows from old table
    - Delete new table data from old table
    - Append new table back to old table

## ETL pipeline
- Extract data from the S3 bucket
- Transform the data into `artists` and `songs`
- Load the data into its respective tables

- Extract data from `data/log_data`
- Transform the data to show additional information regarding date and time
- Load the data into its respective tables

- Combine data from `artists`, `songs`, `users`, `time` to `songsplay` table

## Files in the repository
- create_tables.py: run this file to reset the tables before each time the ETL script runs
- etl.py: extracts data from S3, stages them in Redshift, and transforms data into a set of dimensional tables
- sql_queries.py: contains all sql queries, and is imported into the last two files above

## Example queries and results for song play analysis

```SQL
SELECT user_id, count(*)
FROM songplays
WHERE level = 'paid'
GROUP BY user_id
ORDER BY count DESC
LIMIT 5;
```

| **user_id | **count** |
|---------|-------|
| 49      | 650   |
| 80      | 648   |
| 97      | 557   |
| 15      | 462   |
| 44      | 397   |

repository: gyhou/millionsongs

gha_language: Jupyter Notebook

repo_name: gyhou/millionsongs


text:
# Selenoid UI
[![Build Status](https://travis-ci.org/aerokube/selenoid-ui.svg?branch=master)](https://travis-ci.org/aerokube/selenoid-ui)
[![Coverage](https://codecov.io/github/aerokube/selenoid-ui/coverage.svg)](https://codecov.io/gh/aerokube/selenoid-ui)
[![Release](https://img.shields.io/github/release/aerokube/selenoid-ui.svg)](https://github.com/aerokube/selenoid-ui/releases/latest)
[![Docker Pulls](https://img.shields.io/docker/pulls/aerokube/selenoid-ui.svg)](https://hub.docker.com/r/aerokube/selenoid-ui)

Simple status page with UI updates by SSE,
backed by constant polling of status handle
of [selenoid](https://github.com/aerokube/selenoid) on small go backend.

![ui](docs/img/stats-sessions.png)

## Usage

We distribute UI as a lightweight [Docker](http://docker.com/) container. To run it type:
```
$ docker run -d --name selenoid-ui  \
    --link selenoid                 \
    -p 8080:8080                    \
    aerokube/selenoid-ui --selenoid-uri=http://selenoid:4444
```

where `--link selenoid` links with running container named `selenoid` with selenoid inside

Then access the UI on port 8080:
```
http://localhost:8080/
```
The following flags are supported:

- `--listen` - host and port to listen (e.g. `:1234`)
- `--period` - data refresh period (e.g. `5s` or `1m`)
- `--selenoid-uri` - selenoid uri to fetch data from (e.g. `http://selenoid.example.com:4444/`)

## Features, Screenshots and Complete Guide

Can be found at http://aerokube.com/selenoid-ui/latest/

## Usage note

This UI is designed for debug purposes for one selenoid node. If you need
monitoring capabilities on more than one selenoid, consider
to use [external monitoring system](http://aerokube.com/selenoid/latest/#_sending_statistics_to_external_systems)

repository: vania-pooh/selenoid-ui

gha_language: JavaScript

repo_name: vania-pooh/selenoid-ui


text:
# How to generate clients from Smithy model

## Prerequisite

- Node.js version >= 12.
- [Build and publish Smithy-TypeScript locally](https://github.com/awslabs/smithy-typescript#steps-to-build).

## Options:

```
  -m, --models  The directory of models.                                [string]
  -g, --globs   A list of smithy model globs                             [array]
  -o, --output  The output directory for built clients
           [string] [default: "../../clients"]
  --help        Show help                                              [boolean]
```

## Examples:

- Generate clients from folder with models:

  `yarn generate-clients -m path/to/model`

- Generate clients from a glob (e.g. generating S3, S3Control and DynamoDB clients):

  `yarn generate-clients -g path/to/models/s3*.json path/to/another/dynamodb*.json`

- Generate all known AWS clients:

  `yarn generate-clients`

branch_name: refs/heads/main

gha_language: TypeScript


text:
## GLV-islands
Jupyter Notebook accompanying the article "Island Species Area Relationships and the assembly of complex ecological communities" by Ankit Vikrant and Martin Nilsson Jacobi.

## Dependencies
Requires python libraries scipy, numpy, matplotlib, networkx.

## Use:
The notebook (.ipynb file) implements the simulations described in the article for a particular choice of parameters. Other cases can be simulated using parameter choices that accompany the figures within the article.

repository: ankitvi/GLV-islands

repo_name: ankitvi/GLV-islands


text:
# emoji-reader

A simple tool to recognize Emoji in string.

[![](https://jitpack.io/v/YvesCheung/EmojiReader.svg)](https://jitpack.io/#YvesCheung/EmojiReader) [![npm version](https://badge.fury.io/js/emoji-reader.svg)](https://badge.fury.io/js/emoji-reader)

> [Preview: https://emoji-reader.netlify.app](https://emoji-reader.netlify.app)

## Feature

- Support **Unicode 12** specification, [click here to view] [1]

- The 'Emoji' judgment based on **EBNF** state machine, which is easier to maintain than regular expressions

- Accurately judge the length of the string containing 'Emoji'

- Exact cut string safely without breaking 'Emoji'

## String Length

| Emoji | String.length | EmojiReader.getTextLength |
| :----:| :----: | :----: |
| ♀ | 1 | 1 |
| 🙂 | 2 | 1 |
|👱♂|5|1|
|🏳🌈|6|1|
| 👨👩👦👦 | 11| 1 |

Within a string, an 'Emoji' consists of either one or more Unicode code points.
A code point may consist of multiple characters (depending on whether the code point is greater than 0x010000).
Therefore, an 'Emoji' may consist of several characters.

Many businesses need to judge the number of words, such as the user's nickname can not be too long,
the speech content has the word limit and so on.
If you do not make special treatment for 'Emoji', it will not meet the user's expectation.

Use ``EmojiReader.getTextLength`` to obtain the length of the visual symbol of the text.
The length of an 'Emoji' is 1.

```java
//Java
String strWithEmoji = "我是一个😃";
int error = strWithEmoji.length(); //6
int correct = EmojiReader.getTextLength(strWithEmoji); //5
```

```javascript
//JavaScript
const strWithEmoji = '我是一个😃';
const error = strWithEmoji.length; //6
const correct = require('emoji-reader').getTextLength(strWithEmoji); //5
```

## SubString

When the display text is too long, we usually omit the text at the end and add an ellipsis.

However, if the string contains 'Emoji', it is likely to cut the 'Emoji' into segments and turn it into garbled code.

For example, the following string:

> "I am 🙂😐😎💏"

After invoke `String.subString(0, 8)` will become:

> "I am 🙂?"

Because the display of 'Emoji' is completed by the combination of multiple Unicode code points.
After `subString`, the remaining Unicode code points will show mess symbol that cannot be displayed normally.

Using `EmojiReader.subSequence`, the String can be cropped correctly according to the visual expectation.

```
//JavaScript
import EmojiReader from 'emoji-reader';
//Java
import com.yy.mobile.emoji.EmojiReader;

EmojiReader.subSequence("I am 🙂😐😎💏", 0, 8) == "I am 🙂😐😎"
```

## Install (for Js)

```
npm install --save emoji-reader
```

## Install (for Java/Android)

Add in `build.gradle` :

```Groovy
allprojects {
    repositories {
        ...
        maven { url 'https://jitpack.io' }
    }
}

dependencies {
    api 'com.github.YvesCheung.EmojiReade:lib-jvm:x.y.z'
}
```

## Source Code

[https://github.com/YvesCheung/EmojiReader][2]

  [1]: https://www.unicode.org/reports/tr51/
  [2]: https://github.com/YvesCheung/EmojiReader

repository: YvesCheung/EmojiReader

gha_language: Kotlin

repo_name: YvesCheung/EmojiReader


text:
# monster-mash-crud-practice
This application is a simple crud scaffold using full stack javascript to allow users to interact with the database
from the DOM. The focus of this practice was to build out an efficient model/controller/route structure on the server,
which is why the user interface has been left plain and boring.
 # dependancies
This appliation is dependant on jQuery, node.js, express.js, body-parser module, mongoose module.

repository: haileemiu/monster-mash-crud-practice

repo_name: haileemiu/monster-mash-crud-practice


text:
---
title: Quick Start
---

Welcome to KubeVela! In this guide, we'll walk you through how to install KubeVela, and deploy your first simple application.

## Step 1: Install

Make sure you have finished and verified the installation following [this guide](./install).

## Step 2: Deploy Your First Application

```bash
$ kubectl apply -f https://raw.githubusercontent.com/oam-dev/kubevela/master/docs/examples/vela-app.yaml
application.core.oam.dev/first-vela-app created
```

Check the status until we see `status` is `running` and services are `healthy`:

```bash
$  kubectl get application first-vela-app -o yaml
apiVersion: core.oam.dev/v1alpha2
kind: Application
metadata:
  generation: 1
  name: first-vela-app
  ...
  namespace: default
spec:
  components:
  - name: express-server
    type: webservice
    settings:
      image: crccheck/hello-world
      port: 8000
    traits:
    - name: ingress
      properties:
        domain: testsvc.example.com
        http:
          /: 8000
status:
  ...
  services:
  - healthy: true
    name: express-server
    traits:
    - healthy: true
      message: 'Visiting URL: testsvc.example.com, IP: your ip address'
      type: ingress
  status: running
```

Under the neath, the K8s resources was created:

```bash
$ kubectl get deployment
NAME                READY   UP-TO-DATE   AVAILABLE   AGE
express-server-v1   1/1     1            1           8m
$ kubectl get svc
NAME             TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
express-server   ClusterIP   <IP_ADDRESS>   <none>        8000/TCP   7m43s
kubernetes       ClusterIP   <IP_ADDRESS>      <none>        443/TCP    116d
$ kubectl get ingress
NAME             CLASS    HOSTS                 ADDRESS          PORTS   AGE
express-server   <none>   testsvc.example.com   <your ip address>   80      7m47s
```

If your cluster has a working ingress, you can visit the service.

```
$ curl -H "Host:testsvc.example.com" http://<your ip address>/
<xmp>
Hello World

                                       ##         .
                                 ## ## ##        ==
                              ## ## ## ## ##    ===
                           /""""""""""""""""\___/ ===
                      ~~~ {~~ ~~~~ ~~~ ~~~~ ~~ ~ /  ===- ~~~
                           \______ o          _,/
                            \      \       _,'
                             `'--.._\..--''
</xmp>
```
**Voila!** You are all set to go.

## What's Next

Here are some recommended next steps:

- Learn KubeVela starting from its [core concepts](../overview/concepts)
- Learn more details about [`Application`](../platform-builder-guide/design-abstraction/application) and understand how it works.
- Join `#kubevela` channel in CNCF [Slack](https://cloud-native.slack.com) and/or [Gitter](https://gitter.im/oam-dev/community)

Welcome onboard and sail Vela!


text:
---
description: Blackpoint compensation. Specifies whether blackpoint compensation should be applied to color conversions when no explicit selection is made with icc=.
solution: Experience Manager
title: IccBlackPointCompensation
feature: Dynamic Media Classic,SDK/API
role: Developer,User
exl-id: 9cf6a415-cc82-40e9-a8b9-a687ca95560e
---
# IccBlackPointCompensation{#iccblackpointcompensation}

Blackpoint compensation. Specifies whether blackpoint compensation should be applied to color conversions when no explicit selection is made with icc=.

## Properties {#section-ea27b8089b89468bbc38e9e7154ea413}

Flag. Set to 0 to disable or to 1 to enable blackpoint compensation.

## Default {#section-0d79b203be4c434f927b7c03c7a0062d}

Inherited from `default::IccBlackPointCompensation` if not defined or if empty.

## See also {#section-c6ae25de4b5b45dcb068340acfee91be}

[attribute::IccProfile](../../../../../is-api/image-catalog/image-serving-api-ref/c-image-catalog-reference/c-attributes-reference/r-iccprofilecmyk.md#reference-db89f9dac33e447cadb359ec1ba27ee0) *, [attribute::IccRenderIntent](../../../../../is-api/image-catalog/image-serving-api-ref/c-image-catalog-reference/c-attributes-reference/r-iccrenderintent.md#reference-012f207f28bd4406a5368d23ed95a51f), [icc=](../../../../../is-api/http-ref/image-serving-api-ref/c-http-protocol-reference/c-command-reference/r-icc.md#reference-182b5679e21e4df3b4d330535a5a7517)


text:
[![Integration Tests](https://github.com/phac-nml/irida/workflows/Integration%20Tests/badge.svg?branch=development&event=schedule)](https://github.com/phac-nml/irida/actions?query=branch%3Adevelopment)
[![GitHub release](https://img.shields.io/github/release/phac-nml/irida.svg)](https://github.com/phac-nml/irida/releases/latest)
[![Gitter](https://img.shields.io/gitter/room/phac-nml/irida.svg)](https://gitter.im/irida-project/Lobby)

IRIDA
=====

IRIDA is Canada’s Integrated Rapid Infectious Disease Analysis Platform for Genomic Epidemiology.  More information can be found in the following links.

Important Links
---------------

* Main Site: <http://www.irida.ca/>
* Documentation
    * Main: <https://phac-nml.github.io/irida-documentation/>
    * Download: <https://phac-nml.github.io/irida-documentation/downloads/index.html>
* Available Pipeline Plugins: <https://github.com/phac-nml/irida-pipeline-plugins>

Contributing to IRIDA
---------------------
IRIDA is primarily developed by the IRIDA consortium, but the development team is happy to accept contributions from the community.  See our [contributing guide](CONTRIBUTING.md) for more information.

Contact the IRIDA Team
----------------------
* To ask general or usage questions, email the IRIDA team at<EMAIL_ADDRESS>* Report bugs, suggest features, or contribute code on [GitHub](http://github.com/phac-nml/irida)
* Chat with the developers on [Gitter](https://gitter.im/irida-project/)

Citation
--------

The IRIDA manuscript is in preparation. A preprint is available on bioRxiv:

Thomas C Matthews, Franklin R Bristow, Emma J Griffiths, Aaron Petkau, Josh Adam, Damion Dooley, Peter Kruczkiewicz, John Curatcha, Jennifer Cabral, Dan Fornika, Geoffrey L. Winsor, Melanie Courtot, Claire Bertelli, Ataollah Roudgar, Pedro Feijao, Philip Mabon, Eric Enns, Joel Thiessen, Alexander Keddy, Judith Isaac-Renton, Jennifer L. Gardy, Patrick Tang, The IRIDA Consortium, João A Carriço, Leonid Chindelevitch, Cedric Chauve, Morag R Graham, Andrew G McArthur, Eduardo N Taboada, Robert G Beiko, Fiona SL Brinkman, William WL Hsiao, Gary Van Domselaar. 2018. **The Integrated Rapid Infectious Disease Analysis (IRIDA) Platform**. bioRxiv 381830; doi: <https://doi.org/10.1101/381830>.

Additional related publications are available on <https://www.irida.ca/related-publications/>.

repository: phac-nml/irida

branch_name: refs/heads/development

gha_language: Java

repo_name: phac-nml/irida


text:
---
layout: post
title:  "More on Goalsetting"
date:   2016-08-28
---

My friend said something interesting that caught my attention the other day. We were talking about our career choices and plans for the coming few years, which is very natural for people in their 20's. Specifically, what he mentioned was the *sacrifices* we would have to make for each possible career we could choose. Now, for some reason, this stuck to me really hard over the next couple of days. For some reason, I had never truly thought about the consequences of success, though I'd understood that success was difficult, and entailed many hard days.

It was interesting, because I've failed to accomplish a lot of my goals in the past. When thinking about it, I can confidently say that whilst setting my goals, I didn't realize what I had to sacrifice in order to properly accomplish what I'd set out to do. I tried to have everything - to have my cake and eat it too. An easy example is that I tried to go on a low-carb diet without understanding that the sacrifices I would have to make are enduring feelings of hunger, many long tiring days, and a general lack of energy throughout. These things are by nature characteristic of the low-carb diet (at least when you start out). Yet, by being unaware that they were sacrifices I had to make, I was constantly looking for ways around the inconveniences that the diet brought. I made excuses to "eat carbs on days I exercies," or "allow milk because it's healthy."

When you don't understand the sacrifices you need to make to advance toward you try to find cheats that offset the hard work, pain, and effort required to improve yourself. Yet, if there was truly an easy way around weight loss, why would anyone ever be fat?

Hard work is meant to be hard. You can make it easy on yourself and do easy work, but you'll also stagnate exactly where you are.

Personally, my goal for the summer was weight loss. Although I've shed some weight off, I'm nowhere near where I'd planned to be. So, as of right now, I'm going to explicitly state what my goals are, and what sacrifices I'll be making.

Current Weight: 185.6lb
Target Weight: 170lb
Time: By the start of December (3 months, 12 weeks. Approx 1.3lb/week)

Fundamentally weight loss is about increasing exercise and decreasing calories. It's that simple. There are several sacrifices that I'll have to make in order to succeed.

1) I'll have to properly track my calories, even though it's a huge hassle. This means that after every snack or meal, I'll have to pull out my phone and properly track the foods that I've eaten.

2) When I'm out of calories for the day, I MUST REFUSE ALL OFFERS AND OPPORTUNITIES FOR FOOD. I'll also have to endure whatever hunger I may have afterwards.

3) There are days where I'm extremely tired, and will want to eat a ton of food at breakfast, lunch, and dinner. On these days, I will also have to endure the hunger and tiredness.

4) I'll also have to spend some time to develop useful eye-ball metrics for food quantities. Such as translating my fist into a number of grams.

5) I will have to endure tiring training days at Muay Thai. I will have to endure tiring days at the gym. I *cannot* not go.

extension: markdown


text:
# Old Apple-page

> This is the fourth collaborative project from Microverse program.
> A mock-up of an Old Apple page.

![screenshot](./project-SC.png)

## Built With :

 - **Html** & **CSS**
 - Using background and gradients.
 - -   StyleLinters.
   - W3 HTML Validation Service.

## Demo Link:
[You can find the project live here](https://raw.githack.com/Fanger53/apple-page/features/index.html)

## Authors

👤 Author1

Github: @elasfarc

👤 Author2

Github: https://github.com/Fanger53

Twitter: https://twitter.com/DavidLe97005129

Linkedin: https://www.linkedin.com/in/david-castillo-61ba10b8/

## 🤝 Contributing
Contributions, issues and feature requests are welcome!
Feel free to check the issues page.

## Show your support ⭐
Give a ⭐ if you like this project!

## Acknowledgments
Hat tip to anyone whose code was used Inspiration etc 📝 License This project is MIT licensed.


text:
---
title: "Classes"
---

We have presented a few classes on network concepts and training. Here is a list of slides and videos we have made for reference.

* Networking 101 version 1, by Zach Giles, Early 2017 [Slides](https://goo.gl/i9Asx4)

* Networking 101 version 2, by Zach Giles, December 2018 [Slides](https://goo.gl/dwXSwo), [Video](https://www.youtube.com/watch?v=_Vi-wlEMWOc)

repository: nycmeshnet/docs

gha_language: HTML

repo_name: nycmeshnet/docs


text:
# Borg s3/gcloud storage home backup

## What

This is a script to backup your home folder using borg into a local borg repository which is synced
into an AWS S3 or Google Cloud Storage bucket.

## Introduction

Having enough of overcomplicated or crappy commercial backup tools for Linux, I wrote this script to
back up my home folder into s3 on a cron schedule using borg backup:

  * Borg is quick, saves a great deal of space compared to traditional incremental backup solutions
  * s3 and gcloud storage are both cheap compared to commercial backup solutions, and also reliable
  * And cron is always easy to get going on linux

This script will backup your home folder using borg. The backup will be both compressed and encrypted.
Since borg does not natively support these cloud storage solutions, and the fuse/mount solutions are
slow as heck. We'll be using the rsync-like capabilities of the cli tools for each cloud provider,
which actually work pretty well.

This also prunes your backups to keep 7 dailies and 4 end-of-weeks.

The side effect however is you'll have two backups, one wherever borg has its backup repo, and another
on the bucket. This is good though, always a good idea to keep several backups even if one is on the same
computer.

## Compatibility

  * Linux for sure.
  * MacOS likely.
  * Linux on Windows 10, long shot.

## Requirements

### Tools

  * [Borg backup.](https://www.borgbackup.org/) - your distro will probably have this on its repos
  * The cloud provider's cli tool: either `awscli` or `gsutil` (part of `google-cloud-sdk`). Same as above.
  * A bucket

### Environment variables

These are borg-standard, as per [borg's documentation](https://borgbackup.readthedocs.io/en/stable/usage.html#environment-variables):

  * **BORG_REPO** (mandatory): location where all your backups will go into as they're made (NOT s3).
  Whereas borg supports ssh paths here as well as any mounted folder (eg s3 via fuse), I would recommend
  this to be a local folder on a real drive, if you can afford the space, for speed.
  * **BORG_PASSPHRASE** (optional, recommended): borg will encrypt your backups using this passphrase. You should.
  Make sure you keep this somewhere safe, other than your backup as you'll also need it to restore your files.

These are required by the script to function:

  * **BORG_S3_BACKUP_GOOGLE**: set to `true` if you want to use gcloud storage instead of AWS
  * **BORG_S3_BACKUP_BUCKET** (for both cloud providers): put in here the bucket name only. The naming with the `s3`
        word is a backwards compatibility measure from when this script supported AWS s3 only.
  * **BORG_S3_BACKUP_AWS_PROFILE** (s3 only): put in here the aws cli profile that has access to that bucket (eg `default`).

## How to use

  * Git clone this repo somewhere in your computer.
  * Copy [excludes.txt.dist](excludes.txt.dist) into `excludes.txt` and check its contents. The file is
  mandatory, but it can be empty. Have a look at
  [borg's documentation on the subject of excludes](https://borgbackup.readthedocs.io/en/stable/usage/help.html#borg-help-patterns) for more info.
  [excludes.txt.dist](excludes.txt.dist) has a few examples, indeed these are for my specific use case. If not a regular expression, paths must be
  absolute.
  * Install borg backup according to your platform. Possibly already on your distro's software repositories.
  * For S3:
      * Install `awscli`. Possibly on your distro's repos already.
      * Generate programmatic credentials
      * Configure aws cli with these credentials, eg `aws configure`.
  * For Gcloud Storage:
      * [Install gsutil](https://cloud.google.com/storage/docs/gsutil_install). Comes with the `google-cloud-sdk`.
      * [Authenticate with the `gcloud` tool](https://cloud.google.com/storage/docs/authentication)
  * Make yourself a bucket in your cloud provider.
  * Give full access to the bucket the account you're authenticating the cli tool with.
  * Set up the environment variables discussed above (eg on your `~/.profile`). I do recommend you also set `BORG_PASSPHRASE`.
  * Create a borg repo. Eg: `borg init -e repokey-blake2` - see [borg quickstart guide](https://borgbackup.readthedocs.io/en/stable/quickstart.html) and [borg init docs](https://borgbackup.readthedocs.io/en/stable/usage/init.html) for more info
  * Run [borg-backup.sh](borg-backup.sh)!

### Cron

By default, cron won't load any environment as it runs - make sure you source the file holding your environment
variables before running the script!.

Example crontab (assuming you added your environment variables into `~/.profile` that runs the backup every
working weekday at 17:30, on low priority, piping the output into a log file in your home folder overwritting any
previous logs:

```cron
30 17 * * MON-FRI . $HOME/.profile && nice -n19 $HOME/Projects/borg-s3-home-backup/borg-backup.sh > $HOME/backup.log 2>&1
```

#### Failed backups

Sometimes borg will report a failed backup. In my experience, this is typically due to files owned by another user.
I for instance work a lot with docker and sometimes files owned by root are created in my home folder. You can
set up a crontab for your root user to fix this:

```shell script
~ sudo crontab -e
```

then add

```cron
28 17 * * MON-FRI nice -n19 /usr/bin/find /home/YOURUSERNAME \! -user YOURUSERNAME -print | /usr/bin/xargs /usr/bin/chown YOURUSERNAME:YOURUSERNAME -Rf
```

Quadruple-check the folder `find` is searching on. You don't want to be running this in `/` by accident.

Make sure this cronjob is run (and has time to finish) before the cronjob that backs up your files.

### Note: borg backup locking

We lock the borg backup repository during bucket sync to ensure it doesn't change during uploads. Borg achieves locks using lock files within the repo,
therefore these files will also be backed up to the bucket during sync. If you ever need to download your backup from the bucket it will thus be locked and will need
unlocking with `borg break-lock`, after you've restored it to the correct repo location.

## Restoring backups

We provide with a script to download your backup from the cloud provider. It has the same requirements for env vars as the
backup script:

```shell script
~ ./download-backup.sh /path/to/folder
```

Then move that folder to the expected location of your borg repo.

If your borg repo is not present on your computer, you can of course simply:

```shell script
~ ./download-backup.sh $BORG_REPO
```

After the backup is present at `$BORG_REPO`:
  * CD into the folder and run `borg break-lock` to unlock your backup repo. See [note above](#note-borg-backup-locking)
        for explanation on why your backup is locked.
  * `borg list` will show you available backups
  * CD into some folder then `borg extract ::backup-name`. This will extract your backup to the current folder. See
        [borg's documentation](https://borgbackup.readthedocs.io/en/stable/usage.html#borg-extract) for more info.
  * Move extracted files where they're meant to be.

Example for a typical desktop computer - total restore of your home folder:
  * Make an administrative user whichever way you'd like, make sure they can `sudo` (for instance, on ubuntu
  they must be on the `adm` group). You'll be using this user to restore your data. Do this even if your user
  is already sudo-able to avoid issues when running the commands below.
  * Log out of your desktop back to the log in screen.
  * `CTRL+ALT+F1` to switch to tty1.
  * Log in as said user.

Then:

```bash
# Move your current home folder out of the way
~ cd /
/ ~ mv $HOME $HOME-old

# Make yourself a new one belonging to you
/ ~ sudo mkdir $HOME
sudo chown $(whoami):$(getent group $(whoami) | awk 'BEGIN { FS = ":" } ; { print $1 }') $HOME

# Work out available backups and extract!
/ ~ borg list
home-2022-10-05T17.30          Weds, 2022-10-05 17:30:36 [1234567890abcdef1234567890abcdef]
home-2022-10-06T17.30          Thu, 2022-10-06 17:30:36  [1234567890abcdef1234567890abcdef]

/ ~ borg extract ::home-2022-10-05T17.30

[... check your restored home folder ...]
/ ~ reboot
```

gha_language: Shell


text:
# Starting Plasma5

You can start Plasma5 from a TTY, using xinit-1.4.1.

To start Plasma 5 using xinit-1.4.1, run the following commands:

```bash
cat > ~/.xinitrc << "EOF"
dbus-launch --exit-with-session /usr/bin/startplasma-x11
EOF

startx
```

The X session starts on the first unused virtual terminal, normally vt7. You can switch to another vtn simultaneously pressing the keys Ctrl-Alt-Fn (n=1, 2, ...). To switch back to the X session, normally started at vt7, use Ctrl-Alt-F7. The vt where the command startx was executed will display many messages, including X starting messages, applications automatically started with the session, and eventually, some warning and error messages. You may prefer to redirect those messages to a log file, which not only will keep the initial vt uncluttered, but can also be used for debugging purposes. This can be done starting X with:

```bash
startx &> ~/x-session-errors
```

When shutting down or rebooting, the shutdown messages appear on the vt where X was running. If you wish to see those messages, simultaneously press keys Alt-F7 (assuming that X was running on vt7).

If you intend to start Plasma using a display manager such as lightdm-1.30.0, there will be two entries for Plasma, one for use with Xorg, and another for Wayland. Modify the Xorg entry with the following command, as the root user, so that you can differentiate between the two:

```bash
sed '/^Name=/s/Plasma/Plasma on Xorg/' -i /usr/share/xsessions/plasma.desktop
```

repository: iagapie/lfs-ports

repo_name: iagapie/lfs-ports


text:
# bakego

bakego bakes files into a go file.

## Install

Install Go, then

```
go get github.com/kybin/bakego
```

## Usage Example

bakego created for github.com/kybin/whisky at first.
It has template files in tmpl directory like any usual server program.

In this case you can,

```
$ cd whisky
$ bakego -d tmpl
```

It will generate gen_bakego.go.
The generated code has a global variable named `bakego`, itself has two methods `Extract` and `Ensure`.
So you can add code yourself like,

```
// add init flag to program, then in somewhere of main function

if initFlag {
	bakego.Extract()
} else {
	bakego.Ensure()
}
```

After `go install`, `whisky -init` will create tmpl directory in the run directory.

license: Unlicense

repository: kybin/bakego

repo_name: kybin/bakego


text:
---
title: How to Tune Guzheng to Diatonic Scales
image: /img/white-background.jpeg
sections:
  - heading: ''
    text: >-
      Guzheng are usually tuned to pentatonic scales. The open strings are scale
      degrees 1, 2, 3, 5, and 6. You reach 4 and 7 by bending scale degrees 3
      and 6. Some western songs may call for articulations on 4 and 7 which you
      can’t play while bending. What do you do? Tune diatonically of course! If
      you want to play traditional Eastern music use the Pentatonic Tunings
      instead.

      There are two ways to tune a guzheng diatonically. Both require you buy
      more strings:
  - heading: '1: Type C Strings'
    text: >-
      Buy a set of diatonic strings. These are known as “Type C” strings and are
      calibrated to work for the balance of thickness, tension, and length
      you’ll need. **This is the recommended method.** But if you don’t have
      access to Type C strings:
  - heading: '2: Manually tune with duplicate strings'
    text: >-
      Get duplicate strings (Type A or B) and tune your instrument manually.
      You'll need about 5 strings depending on the range you choose. Below is an
      example that gives a centered range from G2 to F5. This requires duplicate
      #17, #14, #12, #9, and #7 strings.

      **WARNING**: Do not try to tune diatonically without duplicate strings.
      You WILL snap strings or otherwise damage your instrument.

      Guzheng strings are not meant to be swapped frequently. If you are reusing
      your strings be careful - it is common for strings to snap off their ends
      when re-used. Ideally you’d have two instruments, one diatonic, one
      pentatonic. If you have just one instrument, don’t swap too frequently or
      you’ll be buying a lot of strings.
  - heading: Manual Process
    text: >-
      The following process was adapted from instructions by Carol Chang.

      First decide what range of notes you want. A guzheng in the pentatonic Key
      of G will range from D2 to D6, 4 octaves and 1 extra string. A diatonic
      tuning will only span 3 octaves so you have to pick: high, low, or middle?
      Use the songs you want to play as a guide. Write the lowest and highest
      notes down. If they span more than 3 octaves you will have to leave some
      out. In our example we’ll use a range from G2 to F5. (You can use an
      online tone generator to help.)

      Once you have your notes you’ll need to figure out which string
      thicknesses you can use. A guzheng string is designed to be tuned about
      one half step higher or lower than what they are set to in the Key of G.
      Strings may tolerate up to one whole step.  Following our example, a #21 string WILL BREAK or damage the
      guzheng if we tuned it to G5. That’s 2 and a half steps or 5x higher than
      the expected range. Instead we’ll use a #19 string. #19 is tuned to G2 in
      the pentatonic key of G so that’s perfect. Put a #19 string in position
      21.

      The next higher note is A2. Great, #18 strings are tuned to A2 in key of
      G. Put a #18 string in position 20. Position 19 would be B2, which is
      normally a #17 string. Excellent. That brings us to position 18.

      This diatonic scale calls for C3 in position 18. If we go to our Key Table
      you’ll see the #17 string can be tuned to C3 in the Key of C. So, we’ll
      put a second # 17 string in position 18. See the full writeup below. We
      also duplicate strings for F and G.
  - heading: Full Diatonic Example
    text: >-
      **Position**: the location on the instrument.

      **String #**: thickness. See your string’s packaging to record what string
      thickness you are using. On pentatonic tunings the string # is the same as
      the position.

      **Distance**: The distance between the right fixed bridge and the movable
      bridge. The conversion from centimeters to inches is rounded for your
      convenience.

      Position = Note = String # = Distance

      * Position 21 = G2 = String #19 = 73.5cm (29in)

      * Position 20 = A2 = String #18 = 71cm (28in)

      * Position 19 = B2  = String #17 = 68.5cm (27in)

      * Position 18 = C3  = String #17 = 66cm (26in)

      * Position 17 = D3  = String #16 = 63.5cm (25in)

      * Position 16 = E3  = String #15 = 61cm (24in)

      * Position 15 = F3  = String #14 = 58.5cm (23in)

      * Position 14 = G3  = String #14 = 56cm (22in)

      * Position 13 = A3  = String #13 = 53.5cm (21in)

      * Position 12 = B3  = String #12 = 51cm (20in)

      * Position 11 = C4  = String #12 = 48.5cm (19in)

      * Position 10 = D4  = String #11 = 46cm (18in)

      * Position 9 = E4  = String #10 = 43.5cm (17in)

      * Position 8 = F4  = String #9 = 41cm (16in)

      * Position 7 = G4  = String #9 = 38.5cm (15in)

      * Position 6 = A4  = String #8 = 36cm (14in)

      * Position 5 = B4  = String #7 = 33.5cm (13in)

      * Position 4 = C5  = String #7 = 31cm (12in)

      * Position 3 = D5  = String #6 = 28.5cm (11in)

      * Position 2 = E5  = String #5 = 26cm (10in)

      * Position 1 = F5  = String #4 = 23.5cm (9in)
  - heading: Bridge Placement
    text: >-
      _This spacing has not been tested. It is based on theory. Use your
      judgement._

      Bridges are normally positioned in sets of 2 and 3 to allow shifting
      between pentatonic keys. If you are tuned diatonically you have less need
      of this. So, consider spacing the bridges equally. See the table above for
      where you may want them. If you find these placements seem to demand too
      high of a tension, shift your bridges closer to the right. You may need to
      experiment a bit to find the right balance. If you did even spacing you
      want roughly 2.5 cm between bridges, which is about one inch.

      Place your Position 1 bridge about 23.5cm or 9.25 inches from the right
      fixed bridge. Place your position 2 bridge 2.5cm or 1 inch to the left.
      keep doing this for all your movable bridges. Your final bridge should be
      about 73 cm from the right fixed bridge (about 29 inches).

      Bridge Position 1 and Position 21 are based on the distance that string #
      would normally have if it was tuned to that pitch on a pentatonic
      instrument.

      Note: This spacing has not been tested with key changes (shifting strings
      to flats or sharps). You may not have enough room to move your bridges far
      enough. You can either change your string tensions or redo the spacing to
      try and find that space.
---

repository: GuzhengAlive/gza-rebuild

repo_name: GuzhengAlive/gza-rebuild


text:
# Minesweeper in Pygame

Originally written in MonkeyX for my A level project, now recreated in Python with the Pygame library.

![Screenshot of game](https://marleysudbury.github.io/minesweeper/screen1.png)

## Improvements on the original

* The leaderboard actually works
* You can right click twice to put a question mark
* Cross platform (MonkeyX is supposed to be cross-platform, but it was always glitchy/broken on MacOS)

## Requires

* Python 3.6.1
* Pygame 2.0.1

## Installation

Once Python 3.6.1 is installed onto your system, navigate to the directory where you want to install the game, then clone the repoistory with `git clone https://github.com/marleysudbury/minesweeper-pygame` (if you don't have git, you can download the repository as a .zip). Then go into the folder (`cd minesweeper-pygame`) and create virtual environment (venv) with `python3 -m venv venv`.

Activate the venv with `source venv/bin/activate` on MacOS or `venv\Scripts\activate.bat` on Windows. Install Pygame in the venv with `pip install pygame`.

## Running

Whilst in the venv, run the command `python3 game.py`. For quickstart, try `python3 game.py -quick`, `python3 game.py -quick medium` or `python3 game.py -quick hard`. For touchscreen compatability (WIP), try `python3 game.py -touch`.

## Other

To see the page about this on my website, click [here](https://marleysudbury.github.io/minesweeper).

For original click [here](https://github.com/marleysudbury/Minesweeper).

This version is not yet fully complete (see [issues](https://github.com/marleysudbury/minesweeper-pygame/issues)), but should be soon.

repository: marleysudbury/minesweeper-pygame

repo_name: marleysudbury/minesweeper-pygame


text:
#### Project Repository
https://github.com/AbhayAysola/tnb-go

#### Issue
https://github.com/thenewboston-developers/Projects/issues/212

#### Project Lead
Github: https://github.com/AbhayAysola
Discord: Abhay#6969

#### How would you describe your project?
tnbGo is a project which looks to provide an SDK written in Go to interact with thenewboston network.

#### What problem does this project solve?
Many programmers who use languages other than Javascript do not have an easy and simple way to acccess the network.

#### Who is the target market?
Any developers interested in interacting with the chain through Golang

#### How will this project benefit the network and/or the community?
More apps and projects will pop up written in Golang which will be helpful because of Golang's unique features.

#### How is this project centered around thenewboston?
It is an SDK for thenewboston network.

#### What is the estimated completion date for this project? (rough estimates are acceptable)
2-4 weeks from approval

#### Please include a detailed roadmap for your project below. This must include a list of all milestones where payments will be awarded upon completion of each.
**Milestones are subject to change due to change in the beta API**
1. Golang Implementation

    * Account
        * Tests
        * Documentation
        * createAccount
            * From signingKey
            * Random new account
        * createSignature
        * verifySignature
        * verifyKeyPair
    * Node
        * Tests
        * Documentation
        * getAccount
            * balance
            * balance_lock
            * transactions
        * getBlocks
        * postBlocks
        * addTransferRequest
        * getNodes
        * postNodes
    * Block
        * coinTransfer
        * coinLock
        * nodeRegistration
        * nodeBoost
        * schedule
        * usernameRegistration
        * governerApplication
        * votePurchase
        * governerVote
    * Others (Voting, Governance, Lockup Coins)
        * Tests
        * Documentation
        * Not sure about this part yet
    * Build Example App to show how the sdk works.

I am planning to split this into two packages, one as a payment library for developers and the other as a library to maintain the node. (nodeRegistration and such)

#### What is your Discord username?
Abhay#6969

#### Additional files and documentation including architectural diagrams, wireframes, research documents, visuals, etc…
- I will be following the https://github.com/thenewboston-developers/thenewboston-js/ repo and using their implementation as a base.

# Milestone Details

Please include a detailed breakdown of the first milestone including a list of all quantifiable objectives. Upon
completion of the milestone, each objective will be reviewed by thenewboston team and once all objectives have been
verified as completed, coins will then be rewarded.

Note that all future detailed milestone breakdowns will be written as a separate GitHub issue. Only the first milestone
is included in this project proposal.

---

#### First Milestone Description
Thenewboston account implementation in Golang using crypto libraries.

#### First Milestone Objectives

| # | Objective             | How can auditors verify milestone?     | Estimated Value in coins | Estimated time in Hours |
| - | --------------------- | -------------------------------------- | ------- | ---------------------------------------------- |
| 1 | createAccount | Check code in Github and verify using the network | 3000 | 3 |
| 2 | createSignature and verifySignature | Check code in Github and verify using the network | 2500 | 3 |
| 3 | verifyKeyPair | Check code in Github and verify using the network | 1500 | 1.5 |
| 4 | Tests | Run the test command in the repo and see if it passes | 3000 | 3 |
| 5 | Documentation| Read the docs | 2500 | 2.5 |

#### How long do you estimate this first milestone will take to complete?
< 1 week

#### Project Wallet
81a1f1f5d167011f96a7e919997c5aca3f79f9389ba865901ab12c7ef1ade556

repository: sheraazsyed/Projects

repo_name: sheraazsyed/Projects


text:
spirit-core
===========

The core module of the Spirit Engine. This contains the base functionality required to run any Spirit Engine project.

Note
===

This is a work-in-progress repo, nothing will be added here until other projects have been finished.

Test
===

This was edited by Figglewatts.

repository: CreationSoftworks/spirit-core

repo_name: CreationSoftworks/spirit-core


text:
# Open Sourced Self-Supervised Representation Learning From Electroencephalography Signals

[![ML@B](https://github.com/rhotter/eeg-ssl/blob/master/images/mlab_logo.png)](https://ml.berkeley.edu)

This implementation is based on the [paper](https://arxiv.org/pdf/1911.05419.pdf) by Banville et al.

# Directories
* `experiments` : Improvement experiments folder
* `images` : Folder for images for README and results
* `model` : Folder for model scripts
* `preprocessing` : Folder for preprocessing scripts
* `ssl` : Folder for SSL scripts

# Installation and tutorial

## Installation
```sh
$ python eeg_ssl.py data_folder T_pos_RP T_neg_RP T_pos_TS T_neg_TS
```
Inputs
- data_folder: a folder containing EEG .edf files
- T_pos_RP: an integer representing the positive limit for relative positioning.
- T_neg_RP: an integer representing the negative limit for relative positioning.
- T_pos_TS: an integer representing the positive limit for temporal shuffling.
- T_neg_TS: an integer representing the negative limit for temporal shuffling.

Outputs
- RP_dataset: pairs of 30 second normalized EEG time windows
- RP_labels:
  - +1 if the distance between the two windows is T_pos_RP
  - -1 if the distance between the two windows is T_neg_RP
- TS_dataset:  triples of 30 second normalized EEG time windows
- TS_labels:
  - +1 if the distance between the two windows is T_pos_TS
  - -1 if the distance between the two windows is T_neg_TS

License
----

MIT

repository: bkxcyu/eeg-ssl

repo_name: bkxcyu/eeg-ssl


text:
---
title: Training Tutorial Template in Analysis Workspace
description: The Analysis Workspace Training Tutorial walks users through common terminology and steps for building their first analysis in Workspace.
feature: Panels
topics:
activity: use
doc-type: feature video
team: Technical Marketing
kt: 5032

role: User
level: Beginner
exl-id: 52f800b6-f184-4a82-8729-e6edd478331c
---
# Training Tutorial Template in Analysis Workspace

The Analysis Workspace Training Tutorial walks users through common terminology and steps for building their first analysis in Workspace.

>[!VIDEO](https://video.tv.adobe.com/v/33773/?quality=12&learn=on)

This template is available as a standard template in the New Project modal. From this template, you can create your own tutorials, tailoring them to your organization's unique use cases and data structure (eVars, custom events).


text:
# Generating Protocols

## How to Run

First make sure you are inside your AEA's folder (see here on how to create a new agent).

Then run

``` bash
aea generate protocol <path-to-protocol-specification>
```

where `<path-to-protocol-specification>` is the path to a protocol specification file.

If there are no errors, this command will generate the protocol and place it in your AEA project. The name of the protocol's directory will match the protocol name given in the specification. The author will match the registered author in the CLI. The generator currently produces the following files (assuming the name of the protocol in the specification is `sample`):

1. `message.py`: defines messages valid under the `sample` protocol
2. `serialisation.py`: defines how messages are serialized/deserialized
3. `__init__.py`: makes the directory a package
4. `protocol.yaml`: contains package information about the `sample` protocol
5. `sample.proto` protocol buffer schema file
6. `sample_pb2.py`: the generated protocol buffer implementation
7. `custom_types.py`: stub implementations for custom types (created only if the specification contains custom types)

### Full Mode vs Protobuf Only Mode

Currently, the generator can operate in _full mode_ for Python, creating a complete protocol package (files 1 to 7 above) from a protocol specification. The generator also has a _protobuf only mode_ which only creates the protocol buffer schema and implementation files (files 5 and 6 above). The languages supported in the _protobuf only mode_ and their respective ids are below:

- go: `go`
- c++: `cpp`
- java: `java`
- c#: `csharp`
- ruby: `ruby`
- objective-c: `objc`
- javascript: `js`

To use the generator in protobuf only mode for any of the above languages:

``` bash
aea generate protocol --l <language> <path-to-protocol-specification>
```

where `<language>` is a language id.

The protocol buffer compiler requires a plugin to generate Go code. Install it with:

!!! note
    Note the protocol buffer compiler `protoc` that the generator uses requires a plugin to produce `go` code. Follow this instruction.

## Protocol Specification

A protocol can be described in a YAML file. This is called a _protocol specification_. The following is an example protocol specification:

``` yaml
---
name: two_party_negotiation
author: fetchai
version: 0.1.0
description: An example of a protocol specification that describes a protocol for bilateral negotiation.
license: Apache-2.0
aea_version: '>=1.0.0, <2.0.0'
speech_acts:
  cfp:
    query: ct:Query
  propose:
    price: pt:float
    proposal: pt:dict[pt:str, pt:str]
    conditions: pt:optional[pt:union[pt:str, pt:dict[pt:str,pt:str], pt:set[pt:str]]]
    resources: pt:list[pt:bytes]
  accept: {}
  decline: {}
...
---
ct:Query: |
  bytes query_bytes = 1;
...
---
initiation: [cfp]
reply:
  cfp: [propose, decline]
  propose: [propose, accept, decline]
  accept: []
  decline: []
termination: [accept, decline]
roles: {buyer, seller}
end_states: [agreement_reached, agreement_unreached]
keep_terminal_state_dialogues: true
...
```

Each protocol specification must follow the YAML format, and have a minimum of one and a maximum of three YAML documents (each YAML document is enclosed within --- and ...).

### Basic Protocol Detail and Messages Syntax

The first YAML document is mandatory in any protocol specification. It contains some basic information about the protocol and describes the syntax of communicative messages allowed under this protocol.

The allowed fields and what they represent are:

- `name`: The name of the protocol (written in snake_case)
- `author`: The creator of the protocol
- `version`: The current version of the protocol
- `license`: Licensing information
- `aea_version`: The version(s) of the framework that support this protocol. The format is described here.
- `description`: A short description of the protocol
- `protocol_specification_id`: The id which identifies the protocol for over-the-wire transport. This id is decoupled from the `protocol_id` (`{author}/{name}:{version}`) which is tied to the Python implementation.

All of the above fields are mandatory and each is a key/value pair, where both key and value are YAML strings.

Additionally, the first YAML document of a protocol specification must describe the syntax of valid messages according to this protocol. Therefore, it must contain another mandatory `speech-acts` field which defines the set of _performatives_ valid under this protocol, and a set of _contents_ for each performative.

A _performative_ defines the _type_ of a message (e.g. propose, accept) and has a set of _contents_ (or parameters) of varying types.

The format of the `speech-act` is as follows: `speech-act` is a dictionary, where each key is a **unique** _performative_ (YAML string), and the value is a _content_ dictionary. If a performative does not have any content, then its content dictionary is empty, for instance `accept` and `decline` in the specification above.

A content dictionary in turn has key/value pairs, where each key is the name of a content (YAML string) and the value is its type (YAML string). For example, the `cfp` (short for 'call for proposal') performative has one content whose name is `query` and whose type is `ct:Query`.

#### Types

The specific types which could be assigned to contents in a protocol specification are described in the table below.

Types are either user defined (i.e. custom types) or primitive:

- Custom types are prepended with `ct:` and their format is described using regular expression in the table below.
- Primitive types are prepended with `pt:`. There are different categories of primitive types. For example, `<PT>` such as integers and booleans, `<PCT>` such as sets and lists, and so on. Primitive types are compositional:
    - For example, consider `pt:set[...]` under `<PCT>`, i.e. an unordered collection of elements without duplicates. A `pt:set[...]` describes the type of its elements (called "sub-type") in square brackets. The subtype of a `pt:set[...]` must be a `<PT>` (e.g. `pt:int`, `pt:bool`).
    - In describing the format of types, `/` between two subtypes should be treated as "or". For example, the subtype of a `pt:optional[...]` is either a `<PT>`, `<CT>`, `<PCT>`, `<PMT>` or `<MT>`.

A multi type denotes an "or" separated set of subtypes. For example, a content whose type is specified as `pt:union[pt:str, pt:int]` should either be `pt:int` or `pt:float`.

An optional type `pt:optional[...]` assigned to a content means the content's existence is optional, but if it is present, its type must match `pt:optional[...]`'s subtype.

| Type                                | Code    | Format                                                        | Example                                  | In Python                          |
|-------------------------------------|---------|---------------------------------------------------------------|------------------------------------------|------------------------------------|
| Custom types1            | `<CT>`  | `ct:RegExp(^[A-Z][a-zA-Z0-9]*$)`                              | `ct:DataModel`                           | Custom Class                       |
| Primitive types                     | `<PT>`  | `pt:bytes`                                                    | `pt:bytes`                               | `bytes`                            |
|                                     |         | `pt:int`                                                      | `pt:int`                                 | `int`                              |
|                                     |         | `pt:float`                                                    | `pt:float`                               | `float`                            |
|                                     |         | `pt:bool`                                                     | `pt:bool`                                | `bool`                             |
|                                     |         | `pt:str`                                                      | `pt:str`                                 | `str`                              |
| Primitive collection types          | `<PCT>` | `pt:set[<PT>]`                                                | `pt:set[pt:str]`                         | `FrozenSet[str]`                   |
|                                     |         | `pt:list[<PT>]`                                               | `pt:list[pt:int]`                        | `Tuple[int, ...]`*      |
| Primitive mapping types2 | `<PMT>` | `pt:dict[<PT>, <PT>]`                                         | `pt:dict[pt:str, pt:bool]`               | `Dict[str, bool]`                  |
| Multi types                         | `<MT>`  | `pt:union[<PT>/<CT>/<PCT>/<PMT>, ..., <PT>/<CT>/<PCT>/<PMT>]` | `pt:union[ct:DataModel, pt:set[pt:str]]` | `Union[DataModel, FrozenSet[str]]` |
| Optional types                      | `<O>`   | `pt:optional[<MT>/<PMT>/<PCT>/<PT>/<CT>]`                     | `pt:optional[pt:bool]`                   | `Optional[bool]`                   |

* This is how variable length tuples containing elements of the same type are declared in Python; see here.

### Protocol Buffer Schema

Currently, the AEA framework does not officially support describing custom types in a programming language independent format. This means that if a protocol specification includes custom types, the required serialisation logic must be provided manually.

Therefore, if any of the contents declared in `speech-acts` is of a custom type, the specification must then have a second YAML document, containing the protocol buffer schema code for each custom type.

You can see an example of the second YAML document in the above protocol specification.

### Dialogues

You can optionally specify the structure of dialogues conforming to your protocol in a third YAML document in the specification.

The allowed fields and what they represent are:

- `initiation`: The list of initial performatives
- `reply`: The reply structure of speech-acts
- `termination`: The list of terminal performatives
- `roles`: The roles of players participating in a dialogue
- `end_states`: The possible outcomes a terminated dialogue.
- `keep_terminal_state_dialogues`: whether to keep or drop a terminated dialogue. When a storage backend is configured, the dialogues will be persisted in storage when kept.

All of the above fields are mandatory.

`initiation` is a YAML list, containing the performatives which can be used to start a dialogue.

`reply` specifies for every performative, what its valid replies are. If a performative `per_1` is a valid reply to another `per_2`, this means a message with performative `per_1` can target a message whose performative is `per_2`.

`reply` is a YAML dictionary, where the keys are the performatives (YAML string) defined in `speech-acts`. For each performative key, its value is a list of performatives which are defined to be a valid reply.
For example, valid replies to `cfp` are `propose` and `decline`.

`termination` is a YAML list, containing the performatives which terminate a dialogue. Once any of these performatives are used in a dialogue, the dialogue is terminated and no other messages may be added to it.

`roles` is a YAML set, containing the roles  players participating in dialogues can take. `roles` may contain one or two roles, each role being a YAML string. If there are two roles, each participant has a distinguished role in the dialogue (e.g. buyer and seller in the above specification). If there is only one role, then both participants in a dialogue have this same role.

`end_states` lists the final states a terminated dialogue may have. `end_states` is a YAML list of strings.

`keep_terminal_state_dialogues` has a boolean value and specifies whether the terminated dialogues of this protocol are to be kept or discarded.

## Design Guidelines

1. `initiation` and `termination` cannot be empty.

2. Make sure that when defining `reply`, you include every speech-act you specified under `speech_acts`. If any of the speech-acts does not have a reply, indicate that with an empty list `[]` similar to `accept` and `decline` in the specification above.

3. If a speech-act is listed in `termination`, it must not have any replies in `reply`. The reason is simple: a terminal speech-act terminates a dialogue and so its reply can never be used.

4. If a speech-act replies to no other speech-acts, it should be listed in `initiation` otherwise it could never be used in a dialogue (neither to a start a dialogue with, nor as a reply to another speech-act).

### Notes

1. Currently, there is no way to describe custom types in a programming language independent format. This means that if a protocol specification includes custom types, the required implementations must be provided manually.
    _ Before generating the protocol, the protocol buffer schema code for every custom type must be provided in the protocol specification.
    - Once the generator is called, it produces a `custom_types` module containing stub implementations for every custom type in the specification. The user must then modify this module and add implementations for every custom type in the specification. This includes implementations of how an object of a custom type can be encoded and decoded using protocol buffer.
    - Note, currently the way custom types are dealt with in the generator is admittedly inconvenient. The reason is, the generator does not know the structure of custom types and how they may be serialized/deserialized. Although this approach works, it is only a temporary solution until further work on a programming language-independent type description language is finished (similar to how the generator is designed to be a programming language-independent protocol description language).
2. Currently, the first element in `pt:dict` cannot be a `<CT>`, `pt:float` or  `pt:bytes`. This is because of a constraint in protocol buffer version 3 which is the framework's underlying serialisation mechanism. In a future version, we may address this limitation, in which case we will relax this constraint.
3. In protocol buffer version 3, which is the version used by the generator, there is no way to check whether an optional field (i.e. contents of type `pt:optional[...]`) has been set or not (see discussion here). In proto3, all optional fields are assigned a default value (e.g. `0` for integers types, `false` for boolean types, etc). Therefore, given an optional field whose value is the default value, there is no way to know from the optional field itself, whether it is not set, or in fact is set but its value happens to be the default value. Because of this, in the generated protocol schema file (the `.proto` file), for every optional content there is a second field that declares whether this field is set or not. We will maintain this temporary solution until a cleaner alternative is found.
4. Be aware that currently, using the generated protocols in python, there might be some rounding errors when serialising and then deserializing values of `pt:float` contents.

## Demo Instructions

First, create a new AEA project:

``` bash
aea create my_aea
cd my_aea
```

Second, run the generator on the sample specification:

``` bash
aea generate protocol ../examples/protocol_specification_ex/sample.yaml
```

This will generate the protocol and place it in your AEA project.

Third, try generating other protocols by first defining a specification, then running the generator.

repository: fetchai/agents-aea

repo_name: fetchai/agents-aea


text:
Business Rules Engine Requirements
==================================

*The following text originates directly from the book on page 90.*
*Refer to the README.md at the top-level of the project for publication information.*

>Before you start, let’s think about what is it you want to achieve. You’d like to enable nonprogrammers to add or
>change business logic in their own workflow. For example, a marketing executive may wish to apply a special
>discount when a prospect is making an inquiry about one of your products and fits certain criteria. An accounting
>executive may wish to create an alert if expenses are unusually high. These are examples of what you can achieve
>with a Business Rules Engine. It’s essentially software that executes one or more business rules that are often
>declared using a simple bespoke language. A Business Rules Engine can support multiple different components:
>
>- Facts
>   - The available information to which rules have access
>
>- Actions
>    - The operation you want to perform
>
>- Conditions
>    - These specify when an action should be triggered
>
>- Rules
>   - These specify the business logic you want to execute, essentially grouping facts, conditions, and actions together

repository: CBrobsto/RealWorldSoftwareDevelopment

repo_name: CBrobsto/RealWorldSoftwareDevelopment


text:
virtual-development-server
==========================

Repository with information on how to create a solid development server in a short amount of time

# TOC
<!-- MarkdownTOC depth=2 -->

- [Development Server Installation](#development-server-installation)
    - [Requirements](#requirements)
    - [Install VirtualBox](#install-virtualbox)
    - [Create virtual machine (VirtualBox)](#create-virtual-machine-virtualbox)
    - [Ubuntu installation](#ubuntu-installation)
    - [Apache](#apache)
    - [Memcached](#memcached)
    - [/etc/hosts on Host](#etchosts-on-host)
    - [Need to know](#need-to-know)
    - [Todo / wishlist](#todo--wishlist)

<!-- /MarkdownTOC -->

# Development Server Installation

## Requirements
- [VirtualBox](https://www.virtualbox.org/wiki/Downloads)
- [Ubuntu 14.04 LTS image](http://www.ubuntu.com/download/server)

## Install VirtualBox
Just install VirtualBox like you install other software. If this is not enough information, please read the [VirtualBox manual](https://www.virtualbox.org/manual/UserManual.html)

### Create Host-only network
Before creating a virtual machine, open the VirtualBox settings and go to the Network tab. There you can create NAT networks as well as Host-only networks. For my virtual machines I use a Host-only network, so create one (default named 'vboxnet0').

## Create virtual machine (VirtualBox)
Below are some typical settings I use. Please feel free to use alternate values.

### General
- Type: Linux
- Version: Ubuntu (64 bit)

### System
- Memory: 1024MB

### Storage
- HDD: 16GB
- ISO: mount Ubuntu image

## Ubuntu installation
Install Ubuntu by following the installation wizard and use defaults on most questions. When wizard ask what packages to (pre-)install, select the following:
- OpenSSH
- LAMP

Afterwards, login into system and install any updates and nfs-common (for the NFS share):
```Shell
$ sudo aptitude update
$ sudo aptitude upgrade
$ sudo aptitude install nfs-common
```

### Add NFS share
Add the NFS share to the guest by connecting to the host:
```Shell
$ sudo mkdir /mnt/share-name
```

Show which mounts are exposed on the host:
```Shell
# this IP is the default IP of the Host. This may vary depending on your Host-only network configuration in VirtualBox
$ showmount -e <IP_ADDRESS>
```

Mount by adding to /etc/fstab:
```Shell
$ sudo vi /etc/fstab
# add line like:
<IP_ADDRESS>:/path/to/share /mnt/share-name nfs defaults 0 0
```

## Apache

### Enabled modules
- rewrite
- vhost_alias

### Use Dynamic Virtual hosts
See [this article](http://eosrei.net/articles/2012/08/create-dynamic-virtual-hosts-apache-http-vhostalias)

My current vhost:
```ApacheConf
<VirtualHost *:80>
    ServerAdmin<EMAIL_ADDRESS>    ServerName dev.ubuntu1404
    ServerAlias *.dev.ubuntu1404

    <Directory /data/*/http>
        Options Indexes FollowSymLinks MultiViews
        AllowOverride All
        Order allow,deny
        allow from all
        Require all granted
    </Directory>

    LogFormat "%V %h %l %u %t \"%r\" %s %b" vcommon
    CustomLog /data/logs/access.log vcommon
    ErrorLog /data/logs/error.log

    #URL structure project.dev.ubuntu1404
    UseCanonicalName Off
    VirtualDocumentRoot /data/%1/http/
</VirtualHost>
```

### ServerName
Open /etc/apache2/apache2.conf and add to the bottom of the file
```ApacheConf
ServerName ubuntu1404 # or anything else you like
```

Needless to say, you'll have to reload Apache to make changes effective: `$ sudo service apache2 reload`

## Memcached
To install [Memcached](http://memcached.org/) use the following guide:
```Shell
sudo apt-get install php5 php5-memcache php-pear memcached build-essential
sudo pecl install memcache
```
More information here: [How to install and use Memcache](https://www.digitalocean.com/community/tutorials/how-to-install-and-use-memcache-on-ubuntu-12-04)

## /etc/hosts on Host
Edit your /etc/hosts on your Host machine and add the following lines:
```Shell
<IP_ADDRESS> ubuntu1404
<IP_ADDRESS> dev.ubuntu1404
<IP_ADDRESS> <project>.dev.ubuntu1404
```

## Need to know
- added 2 Network Interfaces
  - Host-Only (create one within VirtualBox): this one is used for local communication. Has static IP, so no worry when your IP changes while using multiple networks (home, office, etc.)
  - Bridged: this one is for internet communication (updates, API's, etc.)
- enable NFS on NAT on Mac Host: http://gathering.tweakers.net/forum/list_message/39054666#39054666
- Beware of DNS Rebinding trap: http://xs4all.ipv6.narkive.com/GOzl3Gdl/de-dns-van-fritzbox-7390-vindt-hosts-met-eigen-ipv6-nummers-niet-leuk
- On host: add `umask 002` to ~/.bash_profile. With this fix the files you create on your host are also Group writable, so you'll only have to share the group on the guest to be able to write to files.
- On guest: add user to www-data group and group of NFS share (could be 'dialout'), also added www-data to group of NFS share

## Todo / wishlist
- nginx
- varnish
- nodejs

repository: marcobalk/virtual-development-server

repo_name: marcobalk/virtual-development-server


text:
# Mongo Dump and copy to S3

A simple docker image for mongo dump.

## License
mongo-dump is Copyright © 2015-2019 Codica. It is released under the [MIT License](https://opensource.org/licenses/MIT).

## About Codica

[![Codica logo](https://www.codica.com/assets/images/logo/logo.svg)](https://www.codica.com)

We love open source software! See [our other projects](https://github.com/codica2) or [hire us](https://www.codica.com/) to design, develop, and grow your product.

repository: andrewlubenets/mongo-dump

repo_name: andrewlubenets/mongo-dump


text:
# tborg

This is a very simple Docker Image based on Alpine that creates periodic backups of [our Terraria image](https://github.com/objpatrish/terraria) using [borgbackup](https://borgbackup.org) and [cron](https://en.wikipedia.org/wiki/Cron).

## Usage

If you want to change the frequency of the backups make sure to edit the [crontab](src/crontab.txt) before building the container.

### Docker

```bash
docker run \
    -name=terraria_tborg \
    --restart unless-stopped \
    --detach \
    -v "<backup_target_volume>:/repo"
    -v "<terraria_data_volume>:/archive"
    docker.pkg.github.com/objpatrish/tborg/tborg:latest
```

### Docker-Compose

```yaml
---
version: "3.8"
services:
  backup:
      image: "docker.pkg.github.com/objpatrish/tborg/tborg:latest"
      restart: unless-stopped
      volumes:
        - "terraria_backup:/repo"
        - "terraria_game_data:/archive"

volumes:
  terraria_game_data:
    external: true
  terraria_backup:
    external: true
```

repository: objpatrish/tborg

repo_name: objpatrish/tborg