text:
# My dotfiles

This repo contains my dotfiles, terminal configs, a "manifest" of apps that should be installed on my system and an installation script to make the process of setting up a new computer as seamless as possible.

## Before installation

If you have a working computer, make sure you first go through this checklist before reinstalling everything from scratch...

1. commit and push changes/branches to your git repos
1. save/backup all important documents from non-iCloud directories
1. save all your work from apps which aren't synced through iCloud
1. export important data from your local databases
1. update [mackup](https://github.com/lra/mackup) to the latest version and run `mackup backup`

## Installation

```bash
# 1. Update macOS to the latest version with the App Store
sudo softwareupdate -i -a

# 2. Install macOS Command Line Tools
xcode-select --install

# 3. Copy your public and private SSH keys to ~/.ssh and make sure they're set to 600

# 4. Clone this repo to ~/.dotfiles
cd ~; git clone git@github.com:marcaube/dotfiles.git .dotfiles

# 5. Run install.sh to start the installation
sh .dotfiles/install.sh

# 6. Restore preferences
mackup restore

# 8. Check that homebrew installations are A1
brew doctor

# 9. Restart your computer to finalize the process
shutdown -r now
```

## Update

To pull the latest changes from the repo and update your setup.

```bash
# 1. Move into the dotfiles directory
dotfiles

# 2. Pull the changes from master
git pull

# 3. Reload the configs
reload

# 4. Restore preferences
mackup restore
```

## Tweaking the setup

To add git credentials, custom commands, or private aliases, you should create an `extra.zsh` file. This file is going to be picked up by ZSH but ignored by git. You can use this file to add stuff you don't want to commit to a public repository.

For example, that's where I define my `GIT_AUTHOR_NAME` and `GIT_AUTHOR_EMAIL`.

## Features

- sensible [`.macos`](./.macos) configs, see [this repo](https://github.com/kevinSuttle/MacOS-Defaults) for more options
- a [`Brewfile`](./Brewfile) with a list of binaries and apps I use all the time, see [the caskroom](https://caskroom.github.io/search) for more
    - binaries are installed with [Homebrew](http://brew.sh/)
    - apps are either installed with Homebrew Cask or [MAS](https://github.com/mas-cli/mas)
- an [`aliases.zsh`](./aliases.zsh) file with a bunch of useful aliases and functions
    - CLI app launchers
    - encoding and crypto utility functions
    - an `update` alias to update macOS, mac app store apps, Homebrew binaries, Cask apps and globally installed composer packages
    - a `weather` function to get weather reports from [wttr.in](https://github.com/chubin/wttr.in)
    - and much more...
- a [`.zshrc`](./.zshrc) file to tweak my Oh My Zsh setup, see the [docs](https://github.com/robbyrussell/oh-my-zsh/wiki/Customization) for more
- most app settings synced with iCloud via [Mackup](https://github.com/lra/mackup)

## The files

- [`.gitconfig`](./.gitconfig) is my git configurations
- [`.gitignore_global`](./.gitignore_global) is a list of files and extensions that should be globally ignored by git
- [`.mackup.cfg`](./.mackup.cfg) is the [Mackup](https://github.com/lra/mackup) configuration, to sync app settings using iCloud instead of Dropbox
- [`.macos`](./.macos)  contains my macOS preferences
- [`.zshrc`](./.zshrc)  contains my Oh My ZSH preferences
- [`aliases.zsh`](./aliases.zsh) defines a list of useful CLI aliases, shortcuts and functions
- [`Brewfile`](./Brewfile) contains the list of binaries (homebrew) and apps (cask and mas) that I want installed on my system
- [`com.googlecode.iterm2.plist`](./com.googlecode.iterm2.plist) is my iTerm2 config
- [`exports.zsh`](./exports.zsh) contains the `PATH` env variable configs
- extra.zsh is a file ignored by git where you can add your git credentials, custom commands, private aliases, etc.
- [`install.sh`](./install.sh) is the installation script to make this all work "automagically"

## Inspired by...

- Mathias Bynens' [dotfiles](https://github.com/mathiasbynens/dotfiles)
- Dries Vints' [dotfiles](https://github.com/driesvints/dotfiles)
- Sourabh Bajaj's [macOS Setup Guide](https://sourabhbajaj.com/mac-setup/)
- [A practical guide to securing macOS](https://github.com/drduh/macOS-Security-and-Privacy-Guide)

repository: adrienrusso/dotfiles

repo_name: adrienrusso/dotfiles


text:
# secret_messages
Python Secret Messages - Cipher Game

Cipher game to encrypt and decrypt user text into 3 different cipher methods

Affine Cipher:
  https://en.wikipedia.org/wiki/Affine_cipher

Keyword Cipher:
  https://en.wikipedia.org/wiki/Keyword_cipher

Polybius Square Cipher:
  https://en.wikipedia.org/wiki/Polybius_square

OneTimePad Cipher is applied to text before it is encrytped again:
  https://en.wikipedia.org/wiki/One-time_pad

# Using Secret Messages
Main file to run is menu.py.  To start playing secret messages game:
`python menu.py`

A menu will appear which asks the user for different options to encrypt or decrypt text

license: MIT

branch_name: refs/heads/master

extension: md

license_type: permissive


text:
---
templateKey: 'product-page'
path: /products
title: Come and stay
image: /img/exterior-back-low.JPG
heading: Forest, pond, and surf
description: >-
  This open-plan three-bedroom house looks out on beautiful Gull Pond. Surrounded by pine
  and sycamore, this restful setting is a perfect retreat from city life. Just steps from
  the spacious deck is the pond, great for swimming. Newcomb Hollow Beach is a 1/4 mile walk; the town of Wellfleet, a short drive.
intro:
  blurbs:
    - image: /img/lifeguard-chair.PNG
      text: >
        The Cape Cod National Seashore (CCNS). Created on August 7th, 1961 by President
        John F. Kennedy, the seashore encompasses 43,500 acres (176 km²) of ponds, woods
        and beachfront on Cape Cod, Massachusetts. The CCNS includes nearly 40 miles (60
        km) of seashore along the Atlantic-facing eastern edge of Cape Cod, in the towns
        of Provincetown, Truro, Wellfleet, Eastham, Orleans and Chatham.
    - image: /img/lotus.PNG
      text: >
        The house is positioned between freshwater and salt. Sitting on the deck you will
        hear ocean waves breaking in the distance and Gull Pond lapping at the canoe
        landing thirty steps away.
    - image: /img/frog.PNG
      text: >
        Hike to the beach, around the pond, or through the woods to town.
    - image: /img/lobster.PNG
      text: >
        Wellfleet has many good restaurants. Try Ceraldi for Diner Italian, Mac's
        Shack for local seafood, CShore Kitchen and Bar for Patio Dining.
  heading: What we offer
  description: >
    The Town of Wellfleet is known for its many fine art galleries, scenic beauty and
    historic charm. Wellfleet Harbor is a constantly changing panorama of fishing boats,
    sail boats, yachts and trawlers. It also has several clear, spring fed ponds (or
    kettle holes, as they are commonly called). Golfers and cyclists have great
    opportunities to enjoy their respective sports. Beach enthusiasts can pick from
    several beautiful public beaches, including Marconi, Indian Neck and Cahoons Hollow.
    Other sites of interest include the Samuel Rider House and the Wellfleet Bay Wildlife
    Sanctuary.
main:
  heading: A great vacation, no compromises
  description: >
    tell us MORE! ...
  image1:
    alt: Master bedroom
    image: /img/interior-master.JPG
  image2:
    alt: Your lunch and dinner table
    image: /img/interior-table.JPG
  image3:
    alt: The kitchen
    image: /img/kitchen.JPG
testimonials:
  - author: Elisabeth Kaurismäki
    quote: >-
      We had soo muvh fun!
  - author: Philipp Trommler
    quote: >-
      I learned how to surf at Newcomb Hollow.
full_image: /img/trees-pond.JPG
pricing:
  heading: Rates
  description: >-
    Come for the week or a month
  plans:
    - description: Contact us!
      items:
        - The whole season could be yours
        - Housekeeping can be arranged
      plan: Summer
      price: ' [Contact us](/contact)'
---

repository: douglasgoodwin/gullpondretreat

repo_name: douglasgoodwin/gullpondretreat


text:
## vcgencmd Screensaver for Kodi

The Raspberry Pi requires a special command to enable DPMS (thereby allowing a display to enter power saving mode). This plugin brings that functionality to Kodi.


text:
---
title: "Cypress Chains of Commands"
date: "2020-02-10"
description: This is the big secret of Cypress that Commands are Promises.
category: Cypress
---

[Cypress Chains of Commands](https://docs.cypress.io/guides/core-concepts/introduction-to-cypress.html#Chains-of-Commands)

It’s very important to understand the mechanism Cypress uses to chain commands together. It manages a **Promise chain** on your behalf, with each command yielding a ‘subject’ to the next command, until the chain ends or an error is encountered. The developer should not need to use Promises directly, but understanding how they work is helpful!

#### Interacting With Elements

Cypress allows you to click on and type into elements on the page by using **.click()** and **.type()** commands with a **cy.get()** or **cy.contains()** command. This is a great example of chaining in action. Let’s see it again:
```
cy.get('textarea.post-body')
  .type('This is an excellent post.')
```
We’re chaining the .type() onto the cy.get(), telling it to type into the subject yielded from the cy.get() command, which will be a DOM element.

Here are even more action commands Cypress provides to interact with your app:
•	.blur() - Make a focused DOM element blur.
•	.focus() - Focus on a DOM element.
•	.clear() - Clear the value of an input or textarea.
•	.check() - Check checkbox(es) or radio(s).
•	.uncheck() - Uncheck checkbox(es).
•	.select() - Select an <option\> within a <select\>.
•	.dblclick() - Double-click a DOM element.
•	.rightclick() - Right-click a DOM element.

These commands ensure some guarantees about what the state of the elements should be prior to performing their actions.

For example, when writing a .click() command, **Cypress ensures that the element is able to be interacted with** (like a real user would). It will automatically wait until the element reaches an “actionable” state by:
•	Not being hidden
•	Not being covered
•	Not being disabled
•	Not animating

This also helps prevent flake when interacting with your application in tests. You can usually override this behavior with a force option.

Cypress provides a simple but powerful algorithm when interacting with elements.

#### Asserting About Elements

Assertions let you do things like ensuring an element is visible or has a particular attribute, CSS class, or state. Assertions are commands that enable you to describe the desired state of your application. **Cypress will automatically wait until your elements reach this state**, or fail the test if the assertions don’t pass. Here’s a quick look at assertions in action:
```
cy.get(':checkbox').should('be.disabled')

cy.get('form').should('have.class', 'form-horizontal')

cy.get('input').should('not.have.value', 'US')
```
In each of these examples, it’s important to note that Cypress will automatically wait until these assertions pass. This prevents you from having to know or care about the precise moment your elements eventually do reach this state.

#### Subject Management

A new Cypress chain always starts with cy.\[command\], where what is yielded by the command establishes what other commands can be called **next** (chained).

Some methods yield null and thus cannot be chained, such as **cy.clearCookies()**.

Some methods, such as cy.get() or cy.contains(), yield a **DOM element**, allowing further commands to be chained onto them (assuming they expect a DOM subject) like .click() or even cy.contains() again.
```
cy.clearCookies()         // Done: 'null' was yielded, no chaining possible

cy.get('.main-container') // Yields an array of matching DOM elements
  .contains('Headlines')  // Yields the first DOM element containing content
  .click()                // Yields same DOM element from previous command
```
Cypress commands do not return their subjects, they **yield** them. Remember: **Cypress commands are asynchronous and get queued for execution at a later time**. During execution, subjects are yielded from one command to the next, and a lot of helpful Cypress code runs between each command to ensure everything is in order.

To work around the need to reference elements, Cypress has a feature known as **aliasing**. Aliasing helps you to store and save element references for future use.

#### Using .then() To Act On A Subject

Want to jump into the command flow and get your hands on the subject directly? No problem, add a .then() to your command chain. When the previous command **resolves**, it will call your callback function with the yielded subject as the first argument.

If you wish to continue chaining commands after your .then(), you’ll need to specify the subject you want to yield to those commands, which you can achieve with a return value other than null or undefined. Cypress will yield that to the next command for you.
```
cy
  // Find the el with id 'some-link'
  .get('#some-link')

  .then(($myElement) => {
    // ...massage the subject with some arbitrary code

    // grab its href property
    const href = $myElement.prop('href')

    // strip out the 'hash' character and everything after it
    return href.replace(/(#.*)/, '')
  })
  .then((href) => {
    // href is now the new subject
    // which we can work with now
  })
```

#### Using Aliases to Refer to Previous Subjects

Cypress has some added functionality for quickly referring back to past subjects called **Aliases**. It looks something like this:
```
cy
  .get('.my-selector')
  .as('myElement') // sets the alias
  .click()

/* many more actions */

cy
  .get('@myElement') // re-queries the DOM as before (only if necessary)
  .click()
```
This lets us reuse our DOM queries for faster tests when the element is still in the DOM, and it automatically handles re-querying the DOM for us when it is not immediately found in the DOM. **This is particularly helpful when dealing with front end frameworks that do a lot of re-rendering**!

#### Commands Are Asynchronous

It is very important to understand that **Cypress commands don’t do anything at the moment they are invoked, but rather enqueue themselves to be run later**. This is what we mean when we say Cypress commands are asynchronous.
```
it('changes the URL when "awesome" is clicked', function() {
  cy.visit('/my/resource/path') // Nothing happens yet

  cy.get('.awesome-selector')   // Still nothing happening
    .click()                    // Nope, nothing

  cy.url()                      // Nothing to see, yet
    .should('include', '/my/resource/path#awesomeness') // Nada.
})

// Ok, the test function has finished executing...
// We've queued all of these commands and now
// Cypress will begin running them in order!
```

Each Cypress command (and chain of commands) returns immediately, having only been appended to a queue of commands to be executed at a later time. You purposefully cannot do anything useful with the return value from a command. **Commands are enqueued** and managed entirely behind the scenes.

We’ve designed our API this way because the DOM is a highly mutable object that constantly goes stale. For Cypress to prevent flake, and know when to proceed, we manage commands in a highly controlled deterministic way.

#### Why can’t I use async / await?
If you’re a modern JS programmer you might hear “asynchronous” and think: why can’t I just use async/await instead of learning some proprietary API?

Cypress’s APIs are built very differently from what you’re likely used to: but these design patterns are incredibly intentional. We’ll go into more detail later.

#### Commands Run Serially

After a test function is finished running, Cypress goes to work executing the commands that were enqueued using the cy.* command chains. The test above would cause an execution in this order:

1.	Visit a URL.
2.	Find an element by its selector.
3.	Perform a click action on that element.
4.	Grab the URL.
5.	Assert the URL to include a specific string.

These actions will always happen **serially** (one after the other), never in parallel (at the same time). Why?
To illustrate this, let’s revisit that list of actions and expose some of the hidden ✨ magic ✨ Cypress does for us at each step:
1.	Visit a URL
✨ and wait for the page load event to fire after all external resources have loaded✨
2.	Find an element by its selector
✨ and retry until it is found in the DOM ✨
3.	Perform a click action on that element
✨ after we wait for the element to reach an actionable state ✨
4.	Grab the URL and…
5.	Assert the URL to include a specific string
✨ and retry until the assertion passes ✨

As you can see, Cypress does a lot of extra work to ensure the state of the application matches what our commands expect about it. Each command may resolve quickly (so fast you won’t see them in a pending state) but others may take seconds, or even dozens of seconds to resolve.

While most commands time out after a few seconds, other specialized commands that expect particular things to take much longer like **cy.visit()** will naturally wait longer before timing out.

These commands have their own particular timeout values which are documented in our configuration.

Any waiting or retrying that is necessary to ensure a step was successful must complete before the next step begins. If they don’t complete successfully before the timeout is reached, the test will fail.

#### Commands Are Promises

**This is the big secret of Cypress**: we’ve taken our favorite pattern for composing JavaScript code, Promises, and built them right into the fabric of Cypress. Above, when we say we’re enqueuing actions to be taken later, we could restate that as “**adding Promises to a chain of Promises**”.
```
it('changes the URL when "awesome" is clicked', function() {
  cy.visit('/my/resource/path')

  cy.get('.awesome-selector')
    .click()

  cy.url()
    .should('include', '/my/resource/path#awesomeness')
})
```
Cypress is built using Promises that come from [Bluebird](http://bluebirdjs.com/docs/getting-started.html). However, Cypress commands do not return these typical Promise instances.  For this reason **you cannot ever return or assign anything useful from Cypress commands**.

If you’d like to learn more about handling asynchronous Cypress Commands please read our [Core Concept Guide](https://docs.cypress.io/guides/core-concepts/variables-and-aliases.html#Return-Values).

#### Commands Are Not Promises

The Cypress API is not an exact 1:1 implementation of Promises. They have Promise like qualities and yet there are important differences you should be aware of.
1.	You cannot race or run multiple commands at the same time (in parallel).
2.	You cannot ‘accidentally’ forget to return or chain a command.
3.	**You cannot add a .catch error handler to a failed command**.

There are very specific reasons these limitations are built into the Cypress API.
The whole intention of Cypress (and what makes it very different from other testing tools) is to create consistent, non-flaky tests that perform identically from one run to the next. Making this happen isn’t free - there are some trade-offs we make that may initially seem unfamiliar to developers accustomed to working with Promises.

#### You cannot race or run multiple commands at the same time

Cypress guarantees that it will execute all of its commands deterministically and identically every time they are run.

A lot of Cypress commands **mutate the state of the browser** in some way.
•	**cy.request()** automatically gets + sets cookies to and from the remote server.
•	**cy.clearCookies()** clears all of the browser cookies.
•	**.click()** causes your application to react to click events.

None of the above commands are idempotent; they all cause side effects. Racing commands is not possible because commands must be run in a controlled, serial manner in order to create consistency. **Because integration and e2e tests primarily mimic the actions of a real user, Cypress models its command execution model after a real user working step by step**.

repository: taislu/mobilelu-starter-blog

gha_language: JavaScript

gha_license_id: MIT

repo_name: taislu/mobilelu-starter-blog


text:
# [882. Reachable Nodes In Subdivided Graph](https://leetcode-cn.com/problems/reachable-nodes-in-subdivided-graph/)

## 题目

Starting with anundirected graph (the "original graph") with nodes from 0 to N-1, subdivisions are made to some of the edges.

The graph is given as follows: edges[k] is a list of integer pairs (i, j, n) such that (i, j) is an edge of the original graph,

and n is the total number of new nodes on that edge.

Then, the edge (i, j) is deleted from the original graph,nnew nodes (x_1, x_2, ..., x_n) are added to the original graph,

and n+1 newedges (i, x_1), (x_1, x_2), (x_2, x_3), ..., (x_{n-1}, x_n), (x_n, j)are added to the originalgraph.

Now, you start at node 0from the original graph, and in each move, you travel along oneedge.

Return how many nodes you can reach in at most M moves.

Example 1:

```text
Input: edges = [[0,1,10],[0,2,1],[1,2,2]], M = 6, N = 3
Output: 13
Explanation:
The nodes that are reachable in the final graph after M = 6 moves are indicated below.
```

![original-to-final](origfinal.png)

Example 2:

```text
Input: edges = [[0,1,4],[1,2,6],[0,2,8],[1,3,1]], M = 10, N = 4
Output: 23
```

Note:

1. 0 <= edges.length <= 10000
1. 0 <= edges[i][0] <edges[i][1] < N
1. There does not exist anyi != j for which edges[i][0] == edges[j][0] and edges[i][1] == edges[j][1].
1. The original graphhas no parallel edges.
1. 0 <= edges[i][2] <= 10000
1. 0 <= M <= 10^9
1. 1 <= N <= 3000
1. A reachable node is a node that can be travelled tousing at mostM moves starting fromnode 0.

## 解题思路

见程序注释

repository: imgoogege/LeetCode-in-Go

repo_name: imgoogege/LeetCode-in-Go


text:
---
layout:     post                    # 使用的布局（不需要改）
title:      1- Sequence to Sequence Learning with Neural Networks              # 标题
subtitle:   seq2seq PyTorch复现代码 #副标题
date:       2019-08-28              # 时间
author:     甜果果                      # 作者
header-img<EMAIL_ADDRESS>  #这篇文章标题背景图片
catalog: true                       # 是否归档
tags:                               #标签
    - 学习日记
    - paper
    - pytorch
    - python
    - nlp
    - 机器翻译
---

# 1 - Sequence to Sequence Learning with Neural Networks

In this series we'll be building a machine learning model to go from once sequence to another, using PyTorch and TorchText. This will be done on German to English translations, but the models can be applied to any problem that involves going from one sequence to another, such as summarization.

In this first notebook, we'll start simple to understand the general concepts by implementing the model from the [Sequence to Sequence Learning with Neural Networks](https://arxiv.org/abs/1409.3215) paper.

## Introduction

The most common sequence-to-sequence (seq2seq) models are *encoder-decoder* models, which (commonly) use a *recurrent neural network* (RNN) to *encode* the source (input) sentence into a single vector. In this notebook, we'll refer to this single vector as a *context vector*. You can think of the context vector as being an abstract representation of the entire input sentence. This vector is then *decoded* by a second RNN which learns to output the target (output) sentence by generating it one word at a time.

<EMAIL_ADDRESS>
The above image shows an example translation. The input/source sentence, "guten morgen", is input into the encoder (green) one word at a time. We also append a *start of sequence* (`<sos>`) and *end of sequence* (`<eos>`) token to the start and end of sentence, respectively. At each time-step, the input to the encoder RNN is both the current word, $x_t$, as well as the hidden state from the previous time-step, $h_{t-1}$, and the encoder RNN outputs a new hidden state $h_t$. You can think of the hidden state as a vector representation of the sentence so far. The RNN can be represented as a function of both of $x_t$ and $h_{t-1}$:

$$h_t = \text{EncoderRNN}(x_t, h_{t-1})$$

We're using the term RNN generally here, it could be any recurrent architecture, such as an *LSTM* (Long Short-Term Memory) or a *GRU* (Gated Recurrent Unit).

Here, we have $X = \{x_1, x_2, ..., x_T\}$ , where $$x_1 = \text{<sos>}, x_2 = \text{guten}$$ , etc. The initial hidden state, $h_0$, is usually either initialized to zeros or a learned parameter.

Once the final word, $x_T$, has been passed into the RNN, we use the final hidden state, $h_T$, as the context vector, i.e. $h_T = z$. This is a vector representation of the entire source sentence.

Now we have our context vector, $z$, we can start decoding it to get the target sentence, "good morning". Again, we append start and end of sequence tokens to the target sentence. At each time-step, the input to the decoder RNN (blue) is the current word, $y_t$, as well as the hidden state from the previous time-step, $s_{t-1}$, where the initial decoder hidden state, $s_0$, is the context vector, $s_0 = z = h_T$, i.e. the initial decoder hidden state is the final encoder hidden state. Thus, similar to the encoder, we can represent the decoder as:

$$s_t = \text{DecoderRNN}(y_t, s_{t-1})$$

In the decoder, we need to go from the hidden state to an actual word, therefore at each time-step we use $s_t$ to predict (by passing it through a `Linear` layer, shown in purple) what we think is the next word in the sequence, $\hat{y}_t$.

$$\hat{y}_t = f(s_t)$$

We always use `<sos>` for the first input to the decoder, $y_1$, but for subsequent inputs, $y_{t>1}$, we will sometimes use the actual, ground truth next word in the sequence, $y_t$ and sometimes use the word predicted by our decoder, $\hat{y}_{t-1}$. This is called *teacher forcing*, and you can read about it more [here](https://machinelearningmastery.com/teacher-forcing-for-recurrent-neural-networks/).

When training/testing our model, we always know how many words are in our target sentence, so we stop generating words once we hit that many. During inference (i.e. real world usage) it is common to keep generating words until the model outputs an `<eos>` token or after a certain amount of words have been generated.

Once we have our predicted target sentence, $\hat{Y} = \{ \hat{y}_1, \hat{y}_2, ..., \hat{y}_T \}$, we compare it against our actual target sentence, $Y = \{ y_1, y_2, ..., y_T \}$, to calculate our loss. We then use this loss to update all of the parameters in our model.

## Preparing Data

We'll be coding up the models in PyTorch and using TorchText to help us do all of the pre-processing required. We'll also be using spaCy to assist in the tokenization of the data.

```python
import torch
import torch.nn as nn
import torch.optim as optim

from torchtext.datasets import TranslationDataset, Multi30k
from torchtext.data import Field, BucketIterator

import spacy

import random
import math
import time
```

We'll set the random seeds for deterministic results.

```python
SEED = 1234

random.seed(SEED)
torch.manual_seed(SEED)
torch.backends.cudnn.deterministic = True
```

Next, we'll create the tokenizers. A tokenizer is used to turn a string containing a sentence into a list of individual tokens that make up that string, e.g. "good morning!" becomes ["good", "morning", "!"]. We'll start talking about the sentences being a sequence of tokens from now, instead of saying they're a sequence of words. What's the difference? Well, "good" and "morning" are both words and tokens, but "!" is a token, not a word.

spaCy has model for each language ("de" for German and "en" for English) which need to be loaded so we can access the tokenizer of each model.

**Note**: the models must first be downloaded using the following on the command line:

```
python -m spacy download en
python -m spacy download de
```

We load the models as such:

```python
spacy_de = spacy.load('de')
spacy_en = spacy.load('en')
```

Next, we create the tokenizer functions. These can be passed to TorchText and will take in the sentence as a string and return the sentence as a list of tokens.

In the paper we are implementing, they find it beneficial to reverse the order of the input which they believe "introduces many short term dependencies in the data that make the optimization problem much easier".

```python
def tokenize_de(text):
    """
    Tokenizes German text from a string into a list of strings (tokens) and reverses it
    """
    return [tok.text for tok in spacy_de.tokenizer(text)][::-1]

def tokenize_en(text):
    """
    Tokenizes English text from a string into a list of strings (tokens)
    """
    return [tok.text for tok in spacy_en.tokenizer(text)]
```

TorchText's `Field`s handle how data should be processed. You can read all of the possible arguments [here](https://github.com/pytorch/text/blob/master/torchtext/data/field.py#L61).

We set the `tokenize` argument to the correct tokenization function for each, with German being the `SRC` (source) field and English being the `TRG` (target) field. The field also appends the "start of sequence" and "end of sequence" tokens via the `init_token` and `eos_token` arguments, and converts all words to lowercase.

```python
SRC = Field(tokenize = tokenize_de,
            init_token = '<sos>',
            eos_token = '<eos>',
            lower = True)

TRG = Field(tokenize = tokenize_en,
            init_token = '<sos>',
            eos_token = '<eos>',
            lower = True)
```

Next, we download and load the train, validation and test data.

The dataset we'll be using is the [Multi30k dataset](https://github.com/multi30k/dataset). This is a dataset with ~30,000 parallel English, German and French sentences, each with ~12 words per sentence.

`exts` specifies which languages to use as the source and target (source goes first) and `fields` specifies which field to use for the source and target.

```python
train_data, valid_data, test_data = Multi30k.splits(exts = ('.de', '.en'),
                                                    fields = (SRC, TRG))
```

We can double check that we've loaded the right number of examples:

```python
print(f"Number of training examples: {len(train_data.examples)}")
print(f"Number of validation examples: {len(valid_data.examples)}")
print(f"Number of testing examples: {len(test_data.examples)}")
```

```
Number of training examples: 29000
Number of validation examples: 1014
Number of testing examples: 1000
```

We can also print out an example, making sure the source sentence is reversed:

```python
print(vars(train_data.examples[0]))
```

```
{'src': ['.', 'büsche', 'vieler', 'nähe', 'der', 'in', 'freien', 'im', 'sind', 'männer', 'weiße', 'junge', 'zwei'], 'trg': ['two', 'young', ',', 'white', 'males', 'are', 'outside', 'near', 'many', 'bushes', '.']}
```

The period is at the beginning of the German (src) sentence, so looks good!

Next, we'll build the *vocabulary* for the source and target languages. The vocabulary is used to associate each unique token with an index (an integer) and this is used to build a one-hot encoding for each token (a vector of all zeros except for the position represented by the index, which is 1). The vocabularies of the source and target languages are distinct.

Using the `min_freq` argument, we only allow tokens that appear at least 2 times to appear in our vocabulary. Tokens that appear only once are converted into an `<unk>` (unknown) token.

It is important to note that your vocabulary should only be built from the training set and not the validation/test set. This prevents "information leakage" into your model, giving you artifically inflated validation/test scores.

```python
SRC.build_vocab(train_data, min_freq = 2)
TRG.build_vocab(train_data, min_freq = 2)
```

```python
print(f"Unique tokens in source (de) vocabulary: {len(SRC.vocab)}")
print(f"Unique tokens in target (en) vocabulary: {len(TRG.vocab)}")
```

```
Unique tokens in source (de) vocabulary: 7855
Unique tokens in target (en) vocabulary: 5893
```

The final step of preparing the data is to create the iterators. These can be iterated on to return a batch of data which will have a `src` attribute (the PyTorch tensors containing a batch of numericalized source sentences) and a `trg` attribute (the PyTorch tensors containing a batch of numericalized target sentences). Numericalized is just a fancy way of saying they have been converted from a sequence of readable tokens to a sequence of corresponding indexes, using the vocabulary.

We also need to define a `torch.device`. This is used to tell TorchText to put the tensors on the GPU or not. We use the `torch.cuda.is_available()` function, which will return `True` if a GPU is detected on our computer. We pass this `device` to the iterator.

When we get a batch of examples using an iterator we need to make sure that all of the source sentences are padded to the same length, the same with the target sentences. Luckily, TorchText iterators handle this for us!

We use a `BucketIterator` instead of the standard `Iterator` as it creates batches in such a way that it minimizes the amount of padding in both the source and target sentences.

```python
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
```

```python
BATCH_SIZE = 128

train_iterator, valid_iterator, test_iterator = BucketIterator.splits(
    (train_data, valid_data, test_data),
    batch_size = BATCH_SIZE,
    device = device)
```

## Building the Seq2Seq Model

We'll be building our model in three parts. The encoder, the decoder and a seq2seq model that encapsulates the encoder and decoder and will provide a way to interface with each.

### Encoder

First, the encoder, a 2 layer LSTM. The paper we are implementing uses a 4-layer LSTM, but in the interest of training time we cut this down to 2-layers. The concept of multi-layer RNNs is easy to expand from 2 to 4 layers.

For a multi-layer RNN, the input sentence, $X$, goes into the first (bottom) layer of the RNN and hidden states, $H=\{h_1, h_2, ..., h_T\}$, output by this layer are used as inputs to the RNN in the layer above. Thus, representing each layer with a superscript, the hidden states in the first layer are given by:

$$h_t^1 = \text{EncoderRNN}^1(x_t, h_{t-1}^1)$$

The hidden states in the second layer are given by:

$$h_t^2 = \text{EncoderRNN}^2(h_t^1, h_{t-1}^2)$$

Using a multi-layer RNN also means we'll also need an initial hidden state as input per layer, $h_0^l$, and we will also output a context vector per layer, $z^l$.

Without going into too much detail about LSTMs (see [this](https://colah.github.io/posts/2015-08-Understanding-LSTMs/) blog post if you want to learn more about them), all we need to know is that they're a type of RNN which instead of just taking in a hidden state and returning a new hidden state per time-step, also take in and return a *cell state*, $c_t$, per time-step.

$$\begin{align*}
h_t &= \text{RNN}(x_t, h_{t-1})\\
(h_t, c_t) &= \text{LSTM}(x_t, (h_{t-1}, c_{t-1}))
\end{align*}$$

You can just think of $c_t$ as another type of hidden state. Similar to $h_0^l$, $c_0^l$ will be initialized to a tensor of all zeros. Also, our context vector will now be both the final hidden state and the final cell state, i.e. $z^l = (h_T^l, c_T^l)$.

Extending our multi-layer equations to LSTMs, we get:

$$\begin{align*}
(h_t^1, c_t^1) &= \text{EncoderLSTM}^1(x_t, (h_{t-1}^1, c_{t-1}^1))\\
(h_t^2, c_t^2) &= \text{EncoderLSTM}^2(h_t^1, (h_{t-1}^2, c_{t-1}^2))
\end{align*}$$

Note how only our hidden state from the first layer is passed as input to the second layer, and not the cell state.

So our encoder looks something like this:

<EMAIL_ADDRESS>
We create this in code by making an `Encoder` module, which requires we inherit from `torch.nn.Module` and use the `super().__init__()` as some boilerplate code. The encoder takes the following arguments:

- `input_dim` is the size/dimensionality of the one-hot vectors that will be input to the encoder. This is equal to the input (source) vocabulary size.
- `emb_dim` is the dimensionality of the embedding layer. This layer converts the one-hot vectors into dense vectors with `emb_dim` dimensions.
- `hid_dim` is the dimensionality of the hidden and cell states.
- `n_layers` is the number of layers in the RNN.
- `dropout` is the amount of dropout to use. This is a regularization parameter to prevent overfitting. Check out [this](https://www.coursera.org/lecture/deep-neural-network/understanding-dropout-YaGbR) for more details about dropout.

We aren't going to discuss the embedding layer in these tutorials. All you need to know is that there is a step before the words - technically, the indexes of the words - are passed into the RNN, where the words are transformed into vectors. If you would like to know about word embeddings, I'd recommend these articles [1](https://monkeylearn.com/blog/word-embeddings-transform-text-numbers/), [2](http://p.migdal.pl/2017/01/06/king-man-woman-queen-why.html), [3](http://mccormickml.com/2016/04/19/word2vec-tutorial-the-skip-gram-model/), [4](http://mccormickml.com/2017/01/11/word2vec-tutorial-part-2-negative-sampling/).

The embedding layer is created using `nn.Embedding`, the LSTM with `nn.LSTM` and a dropout layer with `nn.Dropout`. Check the PyTorch [documentation](https://pytorch.org/docs/stable/nn.html) for more about these.

One thing to note is that the `dropout` argument to the LSTM is how much dropout to apply between the layers of a multi-layer RNN, i.e. between the hidden states output from layer $l$ and those same hidden states being used for the input of layer $l+1$.

In the `forward` method, we pass in the source sentence, $X$, which is converted into dense vectors using the `embedding` layer, and then dropout is applied. These embeddings are then passed into the RNN. As we pass a whole sequence to the RNN, it will automatically do the recurrent calculation of the hidden states over the whole sequence for us! You may notice that we do not pass an initial hidden or cell state to the RNN. This is because, as noted in the [documentation](https://pytorch.org/docs/stable/nn.html#torch.nn.LSTM), that if no hidden/cell state is passed to the RNN, it will automatically create an initial hidden/cell state as a tensor of all zeros.

The RNN returns: `outputs` (the top-layer hidden state for each time-step), `hidden` (the final hidden state for each layer, $h_T$, stacked on top of each other) and `cell` (the final cell state for each layer, $c_T$, stacked on top of each other).

As we only need the final hidden and cell states (to make our context vector), `forward` only returns `hidden` and `cell`.

The sizes of each of the tensors is left as comments in the code. In this implementation `n_directions` will always be 1, however note that bidirectional RNNs (covered in tutorial 3) will have `n_directions` as 2.

```python
class Encoder(nn.Module):
    def __init__(self, input_dim, emb_dim, hid_dim, n_layers, dropout):
        super().__init__()

        self.input_dim = input_dim
        self.emb_dim = emb_dim
        self.hid_dim = hid_dim
        self.n_layers = n_layers
        self.dropout = dropout

        self.embedding = nn.Embedding(input_dim, emb_dim)

        self.rnn = nn.LSTM(emb_dim, hid_dim, n_layers, dropout = dropout)

        self.dropout = nn.Dropout(dropout)

    def forward(self, src):

        #src = [src sent len, batch size]

        embedded = self.dropout(self.embedding(src))

        #embedded = [src sent len, batch size, emb dim]

        outputs, (hidden, cell) = self.rnn(embedded)

        #outputs = [src sent len, batch size, hid dim * n directions]
        #hidden = [n layers * n directions, batch size, hid dim]
        #cell = [n layers * n directions, batch size, hid dim]

        #outputs are always from the top hidden layer

        return hidden, cell

```

### Decoder

Next, we'll build our decoder, which will also be a 2-layer (4 in the paper) LSTM.

<EMAIL_ADDRESS>
The `Decoder` class does a single step of decoding. The first layer will receive a hidden and cell state from the previous time-step, $(s_{t-1}^1, c_{t-1}^1)$, and feed it through the LSTM with the current token, $y_t$, to produce a new hidden and cell state, $(s_t^1, c_t^1)$. The subsequent layers will use the hidden state from the layer below, $s_t^{l-1}$, and the previous hidden and cell states from their layer, $(s_{t-1}^l, c_{t-1}^l)$. This provides equations very similar to those in the encoder.

$$\begin{align*}
(s_t^1, c_t^1) = \text{DecoderLSTM}^1(y_t, (s_{t-1}^1, c_{t-1}^1))\\
(s_t^2, c_t^2) = \text{DecoderLSTM}^2(s_t^1, (s_{t-1}^2, c_{t-1}^2))
\end{align*}$$

Remember that the initial hidden and cell states to our decoder are our context vectors, which are the final hidden and cell states of our encoder from the same layer, i.e. $(s_0^l,c_0^l)=z^l=(h_T^l,c_T^l)$.

We then pass the hidden state from the top layer of the RNN, $s_t^L$, through a linear layer, $f$, to make a prediction of what the next token in the target (output) sequence should be, $\hat{y}_{t+1}$.

$$\hat{y}_{t+1} = f(s_t^L)$$

The arguments and initialization are similar to the `Encoder` class, except we now have an `output_dim` which is the size of the one-hot vectors that will be input to the decoder. These are equal to the vocabulary size of the output/target. There is also the addition of the `Linear` layer, used to make the predictions from the top layer hidden state.

Within the `forward` method, we accept a batch of input tokens, previous hidden states and previous cell states. We `unsqueeze` the input tokens to add a sentence length dimension of 1. Then, similar to the encoder, we pass through an embedding layer and apply dropout. This batch of embedded tokens is then passed into the RNN with the previous hidden and cell states. This produces an `output` (hidden state from the top layer of the RNN), a new `hidden` state (one for each layer, stacked on top of each other) and a new `cell` state (also one per layer, stacked on top of each other). We then pass the `output` (after getting rid of the sentence length dimension) through the linear layer to receive our `prediction`. We then return the `prediction`, the new `hidden` state and the new `cell` state.

```python
class Decoder(nn.Module):
    def __init__(self, output_dim, emb_dim, hid_dim, n_layers, dropout):
        super().__init__()

        self.emb_dim = emb_dim
        self.hid_dim = hid_dim
        self.output_dim = output_dim
        self.n_layers = n_layers
        self.dropout = dropout

        self.embedding = nn.Embedding(output_dim, emb_dim)

        self.rnn = nn.LSTM(emb_dim, hid_dim, n_layers, dropout = dropout)

        self.out = nn.Linear(hid_dim, output_dim)

        self.dropout = nn.Dropout(dropout)

    def forward(self, input, hidden, cell):

        #input = [batch size]
        #hidden = [n layers * n directions, batch size, hid dim]
        #cell = [n layers * n directions, batch size, hid dim]

        #n directions in the decoder will both always be 1, therefore:
        #hidden = [n layers, batch size, hid dim]
        #context = [n layers, batch size, hid dim]

        input = input.unsqueeze(0)

        #input = [1, batch size]

        embedded = self.dropout(self.embedding(input))

        #embedded = [1, batch size, emb dim]

        output, (hidden, cell) = self.rnn(embedded, (hidden, cell))

        #output = [sent len, batch size, hid dim * n directions]
        #hidden = [n layers * n directions, batch size, hid dim]
        #cell = [n layers * n directions, batch size, hid dim]

        #sent len and n directions will always be 1 in the decoder, therefore:
        #output = [1, batch size, hid dim]
        #hidden = [n layers, batch size, hid dim]
        #cell = [n layers, batch size, hid dim]

        prediction = self.out(output.squeeze(0))

        #prediction = [batch size, output dim]

        return prediction, hidden, cell
```

### Seq2Seq

For the final part of the implemenetation, we'll implement the seq2seq model. This will handle:

- receiving the input/source sentence
- using the encoder to produce the context vectors
- using the decoder to produce the predicted output/target sentence

Our full model will look like this:

<EMAIL_ADDRESS>
The `Seq2Seq` model takes in an `Encoder`, `Decoder`, and a `device` (used to place tensors on the GPU, if it exists).

For this implementation, we have to ensure that the number of layers and the hidden (and cell) dimensions are equal in the `Encoder` and `Decoder`. This is not always the case, you do not necessarily need the same number of layers or the same hidden dimension sizes in a sequence-to-sequence model. However, if you do something like having a different number of layers you will need to make decisions about how this is handled. For example, if your encoder has 2 layers and your decoder only has 1, how is this handled? Do you average the two context vectors output by the decoder? Do you pass both through a linear layer? Do you only use the context vector from the highest layer? Etc.

Our `forward` method takes the source sentence, target sentence and a teacher-forcing ratio. The teacher forcing ratio is used when training our model. When decoding, at each time-step we will predict what the next token in the target sequence will be from the previous tokens decoded, $\hat{y}_{t+1}=f(s_t^L)$. With probability equal to the teaching forcing ratio (`teacher_forcing_ratio`) we will use the actual ground-truth next token in the sequence as the input to the decoder during the next time-step. However, with probability `1 - teacher_forcing_ratio`, we will use the token that the model predicted as the next input to the model, even if it doesn't match the actual next token in the sequence.

The first thing we do in the `forward` method is to create an `outputs` tensor that will store all of our predictions, $\hat{Y}$.

We then feed the input/source sentence, $X$/`src`, into the encoder and receive out final hidden and cell states.

The first input to the decoder is the start of sequence (`<sos>`) token. As our `trg` tensor already has the `<sos>` token appended (all the way back when we defined the `init_token` in our `TRG` field) we get our $y_1$ by slicing into it. We know how long our target sentences should be (`max_len`), so we loop that many times. During each iteration of the loop, we:

- pass the input, previous hidden and previous cell states ($y_t, s_{t-1}, c_{t-1}$) into the decoder
- receive a prediction, next hidden state and next cell state ($\hat{y}_{t+1}, s_{t}, c_{t}$) from the decoder
- place our prediction, $\hat{y}_{t+1}$/`output` in our tensor of predictions, $\hat{Y}$/`outputs`
- decide if we are going to "teacher force" or not
  - if we do, the next `input` is the ground-truth next token in the sequence, $y_{t+1}$/`trg[t]`
  - if we don't, the next `input` is the predicted next token in the sequence, $\hat{y}_{t+1}$/`top1`

Once we've made all of our predictions, we return our tensor full of predictions, $\hat{Y}$/`outputs`.

```python
class Seq2Seq(nn.Module):
    def __init__(self, encoder, decoder, device):
        super().__init__()

        self.encoder = encoder
        self.decoder = decoder
        self.device = device

        assert encoder.hid_dim == decoder.hid_dim, \
            "Hidden dimensions of encoder and decoder must be equal!"
        assert encoder.n_layers == decoder.n_layers, \
            "Encoder and decoder must have equal number of layers!"

    def forward(self, src, trg, teacher_forcing_ratio = 0.5):

        #src = [src sent len, batch size]
        #trg = [trg sent len, batch size]
        #teacher_forcing_ratio is probability to use teacher forcing
        #e.g. if teacher_forcing_ratio is 0.75 we use ground-truth inputs 75% of the time

        batch_size = trg.shape[1]
        max_len = trg.shape[0]
        trg_vocab_size = self.decoder.output_dim

        #tensor to store decoder outputs
        outputs = torch.zeros(max_len, batch_size, trg_vocab_size).to(self.device)

        #last hidden state of the encoder is used as the initial hidden state of the decoder
        hidden, cell = self.encoder(src)

        #first input to the decoder is the <sos> tokens
        input = trg[0,:]

        for t in range(1, max_len):

            output, hidden, cell = self.decoder(input, hidden, cell)
            outputs[t] = output
            teacher_force = random.random() < teacher_forcing_ratio
            top1 = output.max(1)[1]
            input = (trg[t] if teacher_force else top1)

        return outputs

```

# Training the Seq2Seq Model

Now we have our model implemented, we can begin training it.

First, we'll initialize our model. As mentioned before, the input and output dimensions are defined by the size of the vocabulary. The embedding dimesions and dropout for the encoder and decoder can be different, but the number of layers and the size of the hidden/cell states must be the same.

We then define the encoder, decoder and then our Seq2Seq model, which we place on the `device`.

```python
INPUT_DIM = len(SRC.vocab)
OUTPUT_DIM = len(TRG.vocab)
ENC_EMB_DIM = 256
DEC_EMB_DIM = 256
HID_DIM = 512
N_LAYERS = 2
ENC_DROPOUT = 0.5
DEC_DROPOUT = 0.5

enc = Encoder(INPUT_DIM, ENC_EMB_DIM, HID_DIM, N_LAYERS, ENC_DROPOUT)
dec = Decoder(OUTPUT_DIM, DEC_EMB_DIM, HID_DIM, N_LAYERS, DEC_DROPOUT)

model = Seq2Seq(enc, dec, device).to(device)

```

Next up is initializing the weights of our model. In the paper they state they initialize all weights from a uniform distribution between -0.08 and +0.08, i.e. $\mathcal{U}(-0.08, 0.08)$.

We initialize weights in PyTorch by creating a function which we `apply` to our model. When using `apply`, the `init_weights` function will be called on every module and sub-module within our model. For each module we loop through all of the parameters and sample them from a uniform distribution with `nn.init.uniform_`.

```python
def init_weights(m):
    for name, param in m.named_parameters():
        nn.init.uniform_(param.data, -0.08, 0.08)

model.apply(init_weights)

```

```
Seq2Seq(
  (encoder): Encoder(
    (embedding): Embedding(7855, 256)
    (rnn): LSTM(256, 512, num_layers=2, dropout=0.5)
    (dropout): Dropout(p=0.5)
  )
  (decoder): Decoder(
    (embedding): Embedding(5893, 256)
    (rnn): LSTM(256, 512, num_layers=2, dropout=0.5)
    (out): Linear(in_features=512, out_features=5893, bias=True)
    (dropout): Dropout(p=0.5)
  )
)

```

We also define a function that will calculate the number of trainable parameters in the model.

```python
def count_parameters(model):
    return sum(p.numel() for p in model.parameters() if p.requires_grad)

print(f'The model has {count_parameters(model):,} trainable parameters')

```

```
The model has 13,899,013 trainable parameters

```

We define our optimizer, which we use to update our parameters in the training loop. Check out [this](http://ruder.io/optimizing-gradient-descent/) post for information about different optimizers. Here, we'll use Adam.

```python
optimizer = optim.Adam(model.parameters())

```

Next, we define our loss function. The `CrossEntropyLoss` function calculates both the log softmax as well as the negative log-likelihood of our predictions.

Our loss function calculates the average loss per token, however by passing the index of the `<pad>` token as the `ignore_index` argument we ignore the loss whenever the target token is a padding token.

```python
PAD_IDX = TRG.vocab.stoi['<pad>']

criterion = nn.CrossEntropyLoss(ignore_index = PAD_IDX)

```

Next, we'll define our training loop.

First, we'll set the model into "training mode" with `model.train()`. This will turn on dropout (and batch normalization, which we aren't using) and then iterate through our data iterator.

At each iteration:

- get the source and target sentences from the batch, $X$ and $Y$
- zero the gradients calculated from the last batch
- feed the source and target into the model to get the output, $\hat{Y}$
- as the loss function only works on 2d inputs with 1d targets we need to flatten each of them with `.view`
  - we also don't want to measure the loss of the `<sos>` token, hence we slice off the first column of the output and target tensors
- calculate the gradients with `loss.backward()`
- clip the gradients to prevent them from exploding (a common issue in RNNs)
- update the parameters of our model by doing an optimizer step
- sum the loss value to a running total

Finally, we return the loss that is averaged over all batches.

```python
def train(model, iterator, optimizer, criterion, clip):

    model.train()

    epoch_loss = 0

    for i, batch in enumerate(iterator):

        src = batch.src
        trg = batch.trg

        optimizer.zero_grad()

        output = model(src, trg)

        #trg = [trg sent len, batch size]
        #output = [trg sent len, batch size, output dim]

        output = output[1:].view(-1, output.shape[-1])
        trg = trg[1:].view(-1)

        #trg = [(trg sent len - 1) * batch size]
        #output = [(trg sent len - 1) * batch size, output dim]

        loss = criterion(output, trg)

        loss.backward()

        torch.nn.utils.clip_grad_norm_(model.parameters(), clip)

        optimizer.step()

        epoch_loss += loss.item()

    return epoch_loss / len(iterator)

```

Our evaluation loop is similar to our training loop, however as we aren't updating any parameters we don't need to pass an optimizer or a clip value.

We must remember to set the model to evaluation mode with `model.eval()`. This will turn off dropout (and batch normalization, if used).

We use the `with torch.no_grad()` block to ensure no gradients are calculated within the block. This reduces memory consumption and speeds things up.

The iteration loop is similar (without the parameter updates), however we must ensure we turn teacher forcing off for evaluation. This will cause the model to only use it's own predictions to make further predictions within a sentence, which mirrors how it would be used in deployment.

```python
def evaluate(model, iterator, criterion):

    model.eval()

    epoch_loss = 0

    with torch.no_grad():

        for i, batch in enumerate(iterator):

            src = batch.src
            trg = batch.trg

            output = model(src, trg, 0) #turn off teacher forcing

            #trg = [trg sent len, batch size]
            #output = [trg sent len, batch size, output dim]

            output = output[1:].view(-1, output.shape[-1])
            trg = trg[1:].view(-1)

            #trg = [(trg sent len - 1) * batch size]
            #output = [(trg sent len - 1) * batch size, output dim]

            loss = criterion(output, trg)

            epoch_loss += loss.item()

    return epoch_loss / len(iterator)

```

Next, we'll create a function that we'll use to tell us how long an epoch takes.

```python
def epoch_time(start_time, end_time):
    elapsed_time = end_time - start_time
    elapsed_mins = int(elapsed_time / 60)
    elapsed_secs = int(elapsed_time - (elapsed_mins * 60))
    return elapsed_mins, elapsed_secs

```

We can finally start training our model!

At each epoch, we'll be checking if our model has achieved the best validation loss so far. If it has, we'll update our best validation loss and save the parameters of our model (called `state_dict` in PyTorch). Then, when we come to test our model, we'll use the saved parameters used to achieve the best validation loss.

We'll be printing out both the loss and the perplexity at each epoch. It is easier to see a change in perplexity than a change in loss as the numbers are much bigger.

```python
N_EPOCHS = 10
CLIP = 1

best_valid_loss = float('inf')

for epoch in range(N_EPOCHS):

    start_time = time.time()

    train_loss = train(model, train_iterator, optimizer, criterion, CLIP)
    valid_loss = evaluate(model, valid_iterator, criterion)

    end_time = time.time()

    epoch_mins, epoch_secs = epoch_time(start_time, end_time)

    if valid_loss < best_valid_loss:
        best_valid_loss = valid_loss
        torch.save(model.state_dict(), 'tut1-model.pt')

    print(f'Epoch: {epoch+1:02} | Time: {epoch_mins}m {epoch_secs}s')
    print(f'\tTrain Loss: {train_loss:.3f} | Train PPL: {math.exp(train_loss):7.3f}')
    print(f'\t Val. Loss: {valid_loss:.3f} |  Val. PPL: {math.exp(valid_loss):7.3f}')

```

```
Epoch: 01 | Time: 0m 24s
	Train Loss: 5.065 | Train PPL: 158.383
	 Val. Loss: 4.971 |  Val. PPL: 144.200
Epoch: 02 | Time: 0m 24s
	Train Loss: 4.543 | Train PPL:  93.968
	 Val. Loss: 4.928 |  Val. PPL: 138.160
Epoch: 03 | Time: 0m 24s
	Train Loss: 4.263 | Train PPL:  71.039
	 Val. Loss: 4.650 |  Val. PPL: 104.627
Epoch: 04 | Time: 0m 24s
	Train Loss: 4.039 | Train PPL:  56.745
	 Val. Loss: 4.554 |  Val. PPL:  95.007
Epoch: 05 | Time: 0m 23s
	Train Loss: 3.889 | Train PPL:  48.884
	 Val. Loss: 4.495 |  Val. PPL:  89.537
Epoch: 06 | Time: 0m 24s
	Train Loss: 3.773 | Train PPL:  43.514
	 Val. Loss: 4.390 |  Val. PPL:  80.644
Epoch: 07 | Time: 0m 23s
	Train Loss: 3.668 | Train PPL:  39.164
	 Val. Loss: 4.246 |  Val. PPL:  69.815
Epoch: 08 | Time: 0m 23s
	Train Loss: 3.555 | Train PPL:  34.990
	 Val. Loss: 4.235 |  Val. PPL:  69.030
Epoch: 09 | Time: 0m 23s
	Train Loss: 3.445 | Train PPL:  31.351
	 Val. Loss: 4.142 |  Val. PPL:  62.907
Epoch: 10 | Time: 0m 24s
	Train Loss: 3.370 | Train PPL:  29.074
	 Val. Loss: 4.145 |  Val. PPL:  63.089

```

We'll load the parameters (`state_dict`) that gave our model the best validation loss and run it the model on the test set.

```python
model.load_state_dict(torch.load('tut1-model.pt'))

test_loss = evaluate(model, test_iterator, criterion)

print(f'| Test Loss: {test_loss:.3f} | Test PPL: {math.exp(test_loss):7.3f} |')

```

```
| Test Loss: 4.178 | Test PPL:  65.215 |

```

In the following notebook we'll implement a model that achieves improved test perplexity, but only uses a single layer in the encoder and the decoder.

url: https://raw.githubusercontent.com/tian-guo-guo/tian-guo-guo.github.io/179a0646232128b39ea12156ff292b394e9f24a3/_posts/2019-08-28-1-Sequence to Sequence Learning with Neural Networks.md


text:
---
"name": "Kanbana 1",
"picture": "http://alvaro.ga/images/header.png"
---

# Headers
* [UI] Labels
* [1h] Timers
* [25m] [UI] Labels and timers

# Markdown
* **Bold**
* *Italic*
* `code`

---
"name": "Kanbana 2",
"picture": "http://alvaro.ga/images/header.png"
---

# More Headers!
* [ ] Uncompleted task
* [x] Completed task

repository: ThangaAyyanar/kanbana

repo_name: ThangaAyyanar/kanbana


text:
# Discrete isolation
PostgreSQL/PostGIS implementation of the discrete isolation. The resulting value can be used for point selection purposes. It needs a single-point geometry and a numerical value. For usage, you need the data in a PostgreSQL database with the PostGIS extension. It works well with around 10,000 points; you have to consider a long execution time with more points. If you have less than 1,000 points, take a look at the free  [QGIS Plugin](https://github.com/MathiasGroebe/point_selection/).

## Usage

Here some sample data using peaks. The geometry is just for the illustration shown as Well-known text.

| id | name              | elevation | isolation | geom::geometry(Point, 3857)            |
|----|-------------------|-----------|-----------|----------------------------------------|
| 1  | Cottaer Spitzberg | 391       |           | POINT (1554915.030049 6603385.479223)  |
| 2  | Mt. Everest       | 8848.86   |           | POINT (9676427.650365 3247440.479570)  |
| 3  | Zugspitze         | 2962      |           | POINT (1222988.989970 6011100.150710)  |
| 4  | Dürrer Berg       | 437       |           | POINT (1553173.319730 6597213.489256)  |

The syntax for the function is the follwing:

    discrete_isolation(table_name::text, name_of_geometry_column::text, name_of_elevation_column::text,
                        geometry_of_point::geometry, elevation_of_point::numeric (, max_search_radius::numeric))

All calculations are made within the given spatial reference system; also, the isolation is measured in this system. For best results, project first the data and afterward calculate the isolation. For a better performance create an index on the elevation and geometry column.

For calculating the discrete isolation in our example, run the following query:

    UPDATE peaks
    SET isolation = discrete_isolation('peaks', 'geom', 'elevation', geom, elevation)

It will fill the isolation column in the sample table called "peaks". If you want to limit the search radius, you can use the following query.

    UPDATE peaks
    SET isolation = discrete_isolation('peaks', 'geom', 'elevation', geom, elevation, 100000)

branch_name: refs/heads/main


text:
# Azure Functions with TypeScript

## Visual Studio Code extensions

* Azure Functions - ms-azuretools.vscode-azurefunctions
* Remote - Containers - ms-vscode-remote.remote-containers

## Azure resources

* Run this app locally in the container provided:
    * all environment and JS dependencies are installed
    * there isn't a problem with the version of Node.js you have locally installed. This allows you to use the same version of Node.js as the Azure Function runtime environment.

## Azure Functions

* 200 - Post with valid body value
* 404 - Post without body or without body.tweetText

## HTTP Request

Function is configured to use HTTP Post and expects the POST Body to look like:

```JSON
{ tweetText: "This is my tweet" }
```

You can see this used in the test.

## HTTP Response

```JSON
{
  "textReturn": "16 / 280",
  "isValid": true
}
```
## Local Function runtime

Use [Azure Function core tools](https://www.npmjs.com/package/azure-functions-core-tools) to run local function environment. If you run the project in the Docker container, core tools is already installed with the `./devcontainer.json/DockerFile`.

## CORS

## Function authentication

Create the function (not the function app) with function-level authentication. This requires the requestor to pass an `x-functions-key` header with the key value (found in the Azure portal) to validate the requestor has proper auth to call the function.

## Test local functions

The boilerplate for the project was created with the Azure Functions VSCode extension.


text:
Bash CLI
--------------

**A command line framework built using nothing but Bash and compatible with anything**

REF: https://github.com/SierraSoftworks/bash-cli

<!-- TOC -->

- [create command](#create-command)
- [define command](#define-command)
- [usage](#usage)
    - [project operation](#project-operation)
        - [create project](#create-project)
        - [refresh project](#refresh-project)
        - [get project](#get-project)
        - [delete project](#delete-project)
    - [container operation](#container-operation)
        - [run container](#run-container)
        - [refresh container](#refresh-container)
        - [get container](#get-container)
        - [rm container](#rm-container)

<!-- /TOC -->

# create command
```
//create a soft link /usr/local/bin/my-pi
$ sudo ./cli install my-pi

//create customized command
$ ./cli command create project create
$ ./cli command create project refresh
$ ./cli command create project get
$ ./cli command create project delete

$ ./cli command create run
$ ./cli command create refresh
$ ./cli command create get
$ ./cli command create rm
```

# define command

- app/project/create
- app/project/get
- app/project/refresh
- app/project/delete
- app/run
- app/refresh
- app/get
- app/rm

# usage

## project operation

AWS resources: VPC,DHCPOPtion,Subnet,IGW,RouteTable,NetworkAcl

### create project
```
$ my-pi project create --type aws demo
Start to create project 'demo' in cloud 'aws'
<skip refresh dure to without --refresh>
 terraform apply -auto-approve -refresh=false -var PROJECT_NAME=demo sucessed.
apply duration: 87 (seconds)
```

### refresh project
```
$ my-pi project refresh --type aws demo
Start to refresh project 'demo' in cloud 'aws'
 terraform refresh -var PROJECT_NAME=demo sucessed.
refresh duration: 52 (seconds)
```

### get project
```
$ my-pi project get --type aws demo
Start to get project 'demo' in cloud 'aws'
<skip refresh dure to without --refresh>
==============================
project_name = demo
vpc_id = vpc-1e431979
==============================
```

### delete project
```
$ my-pi project delete --type aws demo
Start to delete project 'demo' in cloud 'aws'
<skip refresh dure to without --refresh>
 terraform destroy -auto-approve -refresh=false -var PROJECT_NAME=demo sucessed.
destroy duration: 46 (seconds)
```

## container operation

AWS resources: EC2 Instance, SG

### run container
```
$ my-pi run --type aws demo
Start to run container 'demo' in cloud 'aws'
<skip refresh dure to without --refresh>
 terraform apply -auto-approve refresh=false -var PROJECT_NAME=demo sucessed.
apply duration: 87 (seconds)
```

### refresh container
```
$ my-pi refresh --type aws demo
Start to refresh container 'demo' in cloud 'aws'
 terraform refresh -var PROJECT_NAME=demo sucessed.
refresh duration: 23 (seconds)
```

### get container
```
$ my-pi get --type aws demo
Start to get container 'demo' in cloud 'aws'
<skip refresh dure to without --refresh>
==============================
ContainerVM_ip = [
    <IP_ADDRESS>
]
SG_id = sg-a1c57ad9
SUBNET_id = [
    subnet-90e7d8d9
]
VPC_id = vpc-1e431979
==============================
```

### rm container
```
$ my-pi rm --type aws demo
Start to rm container 'demo' in cloud 'aws'
<skip refresh dure to without --refresh>
 terraform destroy -auto-approve refresh=false -var PROJECT_NAME=demo sucessed.
destroy duration: 57 (seconds)
```

repository: Jimmy-Xu/learn-terraform

repo_name: Jimmy-Xu/learn-terraform


text:
## Haskell for Clojure Programmers

#### Why might we be interested in Haskell? (A few random reasons)

* It represents an extension to the lambda calculus, which Lisp is based on, and therefore Clojure as well. <!-- .element: class="fragment" data-fragment-index="1" -->
* A Clojure programmer may find the variety of persistent data structures provided natural and fun to use, if appearing verbose at first. <!-- .element: class="fragment" data-fragment-index="2" -->
* The flexibity provided in leveraging the type system to generate structure can be akin to how macros are used in Lisp, but with more logical rigor. <!-- .element: class="fragment" data-fragment-index="3" -->

#### Most importantly?

Thinking about the types of values (and types of types!) can help provide a strong foundation for structuring programs in a safe, rigorous, consistent, and _simple_ way.

### What is Haskell's lineage?

Committee-driven, created to provide a lazy, functional language as a basis for academic research, as the widely-used language Miranda was proprietary.

### What is Haskell's lineage?

* Lisp (1958) - lambda calculus <!-- .element: class="fragment" data-fragment-index="1" -->
* ML - "Meta Language" (1973): (Damas) Hindley Milner type inference, garbage collection, pattern matching, currying <!-- .element: class="fragment" data-fragment-index="2" -->
* NPL (-> Hope) - algebraic data types, list comprehension <!-- .element: class="fragment" data-fragment-index="3" -->
* Miranda - laziness and purity, syntax <!-- .element: class="fragment" data-fragment-index="4" -->

Unique to Haskell: monadic effects, typeclasses for ad-hoc polymorphism <!-- .element: class="fragment" data-fragment-index="5" -->

#### What is already familiar to us?

* Functions are first-class <!-- .element: class="fragment" data-fragment-index="1" -->
* Immutable Data (maps, lists, vectors, arrays, etc.) <!-- .element: class="fragment" data-fragment-index="2" -->
* A lot of thought around managing state <!-- .element: class="fragment" data-fragment-index="3" -->
* "How is data shaped" <!-- .element: class="fragment" data-fragment-index="4" -->
* Mapping, Reducing, etc. (Generic operations/Parametric Polymorphism) <!-- .element: class="fragment" data-fragment-index="5" -->
* Type Classes are reasonably similar to Protocols (Ad-hoc Polymorphism) <!-- .element: class="fragment" data-fragment-index="6" -->
* Laziness* <!-- .element: class="fragment" data-fragment-index="7" -->

(*Kinda sorta) <!-- .element: class="fragment haskell-talk-small" data-fragment-index="8" -->

#### What is new to us?

* The type system! <!-- .element: class="fragment" data-fragment-index="1" -->
* Tons of algebraic structures/category theoretic concepts in core libraries <!-- .element: class="fragment" data-fragment-index="2" -->
* "Built-in" Recursion <!-- .element: class="fragment" data-fragment-index="3" -->
* Pattern Matching* <!-- .element: class="fragment" data-fragment-index="4" -->
* Laziness* <!-- .element: class="fragment" data-fragment-index="5" -->

(*Kinda sorta) <!-- .element: class="fragment haskell-talk-small" data-fragment-index="6" -->

### Types

> The fundamental purpose of a type system is to prevent the occurrence of execution errors during the running of a program.

Luca Cardelli, Microsoft Research <!-- .element: class="haskell-talk-small" -->

> The purpose of the type system is to rule out bad programs without ruling out too many good programs.

Edward Kmett, Extremely Prolific Haskell Programmer and Haskell Core Libraries Committee Chair <!-- .element: class="haskell-talk-small" -->

#### Static vs. Dynamic

> Types are an alternative semantics to runtime evaluation. Thus, asking for types is asking for greater power to analyze and comes at a cost of power to construct.
>
> From here, an argument must move on to tradeoffs between power to analyze and power to construct. Why wouldn’t you want to just side entirely with power to construct?
>
> ...

> ...
> Most arguments of the static typing apologist end up being arguments for the value of power to analyze.

 (That I know about)

* Strings <!-- .element: class="fragment" data-fragment-index="1" -->
* Records <!-- .element: class="fragment" data-fragment-index="2" -->
* (Consistency of) Documentation <!-- .element: class="fragment" data-fragment-index="3" -->
* Laziness* <!-- .element: class="fragment" data-fragment-index="5" -->

(*Kinda sorta) <!-- .element: class="fragment" data-fragment-index="6" -->

### OMG Just show me some code already

![judge judy says hurry it up](https://i.imgur.com/XQzGQr5.gif)

### Values and Their Types

```Haskell
-- a typical type function definition with a type signature
foo :: (a -> b) -> a -> b
foo = id

-- 'a' and 'b' are type variables
-- '->' represents function application

-- a function from 'a' to 'b'
(a -> b)

-- and given an 'a', will return a 'b'
-> a -> b
```

### Values and Their Types

```Haskell
λ :type show
show :: Show a => a -> String
λ :t foo
foo :: (a -> b) -> a -> b
λ :t foo show
foo show :: Show a => a -> String
λ :t foo show 1
foo show 1 :: String
λ foo show 1
"1"
```

### Values and Their Types

```Haskell
-- The default strings are lists ([]) of Chars
λ :t "hi" :: [Char]
"hi" :: [Char] :: [Char]
λ :t "hi" ++ " world"
"hi" ++ " world" :: [Char]

-- We can use an infix operator in the prefix position using parens
-- (Note how we get partial application for free):
λ :t (++) "hi"
(++) "hi" :: [Char] -> [Char]
```

### Values and Their Types

```Haskell
λ :t (++)
-- a is a type variable - so this is a list ([]) of 'a'
(++) :: [a] -> [a] -> [a]

-- But look--once we apply a value, we force our 'a' to be a String!
λ :t (++) "hi"
(++) "hi" :: [Char] -> [Char]

-- ...but we can also do this, because all we need is a list of 'a':
λ :t (++) [1,2]
(++) [1,2] :: Num a => [a] -> [a]
λ (++) [1,2] [3]
[1,2,3]
λ
```

### Numbers

```Haskell
λ :type 1
1 :: Num p => p
λ :type 1 :: Int
1 :: Int :: Int
λ :t 2.45 :: Int

<interactive>:1:1: warning: [-Wdeferred-type-errors]
    • No instance for (Fractional Int) arising from the literal ‘2.45’
    • In the expression: 2.45 :: Int
2.45 :: Int :: Int
λ :t 2.45 :: Double
2.45 :: Double :: Double
λ
```

### Strings

```Haskell
λ :t "foo"
"foo" :: [Char]
λ :set -XOverloadedStrings
λ :t "foo"
"foo" :: Data.String.IsString p => p
λ :t "foo" :: String
"foo" :: String :: String
λ :m +Data.Text
λ :t "foo" :: Text
"foo" :: Text :: Text
λ
```

### Lists and Tuples

```Haskell
λ :t [1,2,3]
[1,2,3] :: Num a => [a]
λ [1,2,3] !! 0
1
λ [1,2,3] !! 3
*** Exception: Prelude.!!: index too large
λ
λ :t (1,2)
(1,2) :: (Num b, Num a) => (a, b)
λ fst (1,2)
1
λ snd (1,2)
2
λ
```

### Algebraic Data Types

A Sum type (think "or")

```Haskell
data PossiblyAn a = ThisHereIsAn a | GotNada
  deriving (Show)

checkIt (ThisHereIsAn n) = "We got a " ++ show n
checkIt GotNada = "Oh sorry we got no values today"

λ checkIt $ ThisHereIsAn "Foo"
"We got a \"Foo\""
λ checkIt $ GotNada
"Oh sorry we got no values today"
```

### Algebraic Data Types

A Product type (think "and")

```Haskell
data Pair a b = MakePair a b deriving Show

firstInPair (MakePair x y) = x
secondInPair (MakePair x y) = y

λ firstInPair $ MakePair 1 2
1
λ secondInPair $ MakePair 1 2
2
λ
```

### Algebraic Data Types

Records

```Haskell
data Color = Red | Blue | Yellow | White | Black
  deriving (Show)

data Person = Person { name :: String
                     , age :: Int
                     , favoriteColor :: Color
                     }
  deriving (Show)

λ Person "Bob" 35 Blue
Person {name = "Bob", age = 35, favoriteColor = Blue}
λ
```

### Parametric Polymorphism

```Haskell
λ :t (++)
(++) :: [a] -> [a] -> [a]
λ :t id
id :: a -> a
λ :t const
const :: a -> b -> a
λ
```

### Ad-hoc Polymorphism: Type Classes

```Haskell
λ :t (+)
(+) :: Num a => a -> a -> a
λ 1.0 + 1.0
2.0
λ 1 + 1
2
λ :t 1.0 + 1.0
1.0 + 1.0 :: Fractional a => a
λ :t 1 + 1
1 + 1 :: Num a => a
λ
```

### Another Ad-hoc Polymorphism Example

```Haskell
λ :t (>)
(>) :: Ord a => a -> a -> Bool
λ :t (==)
(==) :: Eq a => a -> a -> Bool
λ data Food = Sushi | Pizza | TurkeySandwich | JustSomeCrackers
    | Natto | LiterallyJustPaper deriving (Show, Eq, Ord)
λ Pizza > LiterallyJustPaper
True
λ Pizza > JustSomeCrackers
True
λ Pizza > Sushi
False
```

### Another Ad-hoc Polymorphism Example

```Haskell
λ :t fmap
fmap :: Functor f => (a -> b) -> f a -> f b
λ fmap (+1) [1,2,3]
[2,3,4]
λ fmap (+1) $ Just 1
Just 2
λ fmap (+1) (1,2)
(1,3)
λ
```

### Let's make our own typeclass

```Haskell
data PossiblyAn a = ThisHereIsAn a | GotNada

class ShowGoofy a where
  showGoofy :: a -> String

instance (Show a) => ShowGoofy (PossiblyAn a) where
  showGoofy (ThisHereIsAn a) = "Gwarsh you got an " ++ (show a) ++ "!"
  showGoofy GotNada = "Well shucks you got nothin'"

```

## Okay but what's all this monad stuff about!?

### Be patient, we're getting there

![patience](https://i.imgur.com/XSZ7kCr.gif)

### First of all, a quote

> In the end, however, a Functor is simply what it is defined to be; doubtless there are many examples of Functor instances that don’t exactly fit either of the above intuitions. The wise student will focus their attention on definitions and examples, without leaning too heavily on any particular metaphor. Intuition will come, in time, on its own. <!-- .element: class="haskell-talk-small" -->

-- Prof. Brent Yorgey, from the Typeclassopedia <!-- .element: class="haskell-talk-small" -->
https://wiki.haskell.org/Typeclassopedia

#### Functor, Monad, Applicative, Monad...

Haskell structures a lot of its core libraries around some basic algebraic structures. And to a large extent typeclasses are how these structures are defined and extended.

Typeclassopedia is the definitive reference: https://wiki.haskell.org/Typeclassopedia

#### Functor

Lifts a function into a computational context. Think `map`--the only reason that name wasn't used in Haskell is because it was already taken, for lists. Whoops.

```Haskell
λ :t fmap
fmap :: Functor f => (a -> b) -> f a -> f b
λ fmap (+1) [1,2,3]
[2,3,4]
λ fmap (+ 1) $ Just 1
Just 2
λ (fmap.fmap) (+1) [[1],[2],[3]]
[[2],[3],[4]]
λ (fmap.fmap.fmap) (+1) [[Just 1],[Just 2],[Just 3]]
[[Just 2],[Just 3],[Just 4]]
λ
```

#### Applicative Functor

Adds some more structure on top of `Functor`.  For `Monad`, we must define `return` and `>>=` ("bind").

```Haskell
λ :t (>>=)
(>>=) :: Monad m => m a -> (a -> m b) -> m b
λ :t return
return :: Monad m => a -> m a
λ (Just 1) >>= (\x -> return (x + 1))
Just 2
λ [1,2,3] >>= (\x -> return (x + 1))
[2,3,4]
```

(...)

```Haskell
λ :t (\x -> return (x + 1))
(\x -> return (x + 1)) :: (Num a, Monad m) => a -> m a
λ :t (flip (>>=)) (\x -> return (x + 1))
(flip (>>=)) (\x -> return (x + 1))
  :: (Num b, Monad m) => m b -> m b
λ (flip (>>=)) (\x -> return (x + 1)) $ Just 1
Just 2
λ doIt v = do { x <- v; return $ x + 1 }
λ :t doIt
doIt :: (Monad m, Num b) => m b -> m b
λ doIt $ Just 1
Just 2
λ doIt $ [1,2,3]
[2,3,4]
```

Monad is famous 'cause it is the main mechanism Haskell provides for interacting with effects.

```Haskell
main :: IO ()
main = do
  putStrLn "What's your name?"
  name <- getLine
  putStrLn $ "hi " ++ name

-- equivalent to
altMain =
  putStrLn "What's your Name" >> getLine >>= \n -> putStrLn $ "hi " ++ n
```

#### Monad Transformers

A.k.a. "How do I compose these things?"

```Haskell
type MyMonad = StateT Integer IO

doStuff :: MyMonad String
doStuff = do
  liftIO $ putStrLn "What's your name?"
  name <- liftIO getLine
  modify (+1)
  liftIO $ putStrLn $ "hi " ++ name ++ ", what's your favorite color?"
  color <- liftIO getLine
  modify (+1)
  return $ "name: " ++ name ++ ", color: " ++ color
```

### Laziness in Haskell vs. Clojure

```Clojure
(defn test-lazy
  [b]
  (let [x (do (println "Calculating.") (+ 1 1))]
    (if b x 1)))

> (test-lazy true)
Calculating.
2
> (test-lazy false)
Calculating.
1
>
```

```Haskell
testLazy b = let x = trace "Calculating." 1 + 1
             in if b then x else 1

λ testLazy True
Calculating.
2
λ testLazy False
1
λ
```

### Data Structures!

Built-ins:

```Haskell
-- Lists!
λ take 10 [1..]
[1,2,3,4,5,6,7,8,9,10]

-- Tuples!
λ true = fst (True,False)
λ false = snd (True,False)
```

### Data Structures!

https://hackage.haskell.org/package/containers

Graphs, sets, trees, maps, some specific variations for efficiency.

https://hackage.haskell.org/package/array
https://hackage.haskell.org/package/vector

Efficient mutable and immutable indexed arrays

### Kinds: Types of Types!

```Haskell
λ data Foo a b = Foo { runFoo :: a -> b }
λ :t Foo
Foo :: (a -> b) -> Foo a b
λ :k Foo
Foo :: * -> * -> *
λ :t Foo id
Foo id :: Foo b b
λ :t Foo const
Foo const :: Foo a (b -> a)
λ (runFoo $ Foo const) 1 2
1
λ
```


text:
# bracket-terminal

`bracket-terminal` is part of the `bracket-lib` family. It provides a virtual ASCII/Codepage-437 terminal (with optional tile graphic support and layers), and a game loop. This frees you up from implementation difficulties, making it easy to write grid-based games (Roguelikes are a great fit, but any grid/tile-based game can work). It also provides assistance with keyboard and mouse input.

Bracket-terminal supports multiple back-ends:

* The default is `OpenGL`, which works on just about everything. The GL back-end supports all features, including post-processing (retro screen effects) and layers.
* The `WebGL` (WASM) back-end works in Web Assembly, allowing you to compile your `bracket-terminal`-based game for the web.
* The `Amethyst` back-ends provide rendering in `Vulkan` and `Metal`. It currently supports everything except the post-processing effects.
* The `crossterm` back-end runs natively in your existing terminal. Graphical features are not supported.
* The `curses` back-end runs natively in *NIX terminals, or in a `pdcurses` terminal emulator on Windows. Graphical features are not supported.

## Why `bracket-terminal` and not direct console rendering?

Bracket-terminal can do terminal rendering, but if that is your only target you may be better off using `crossterm`. Bracket-terminal gets you a few features you don't find elsewhere:

* It is game-loop based, so it is ideal for frame-oriented game programming.
* Codepage-437 emulation is sprite-based on graphical back-ends. You can be absolutely sure that your game will look the same on all platforms, using *exactly* the font(s) you specify.
* It provides multiple layers, which can use different font/sprite files.
* There are some retro post-processing effects available if you like them.
* `bracket-terminal` works hard to be simple and straightforward, making for a great learning environment.

## Minimal example

The following code is enough to put `Hello Minimal Bracket World` on the screen:

```rust
use bracket_terminal::prelude::*;

struct State {}

impl GameState for State {
    fn tick(&mut self, ctx: &mut BTerm) {
        ctx.print(1, 1, "Hello Bracket World");
    }
}

fn main() {
    let context = BTermBuilder::simple80x50()
        .with_title("Hello Minimal Bracket World")
        .build();

    let gs: State = State {};
    main_loop(context, gs);
}
```

It's worth noting that `(0,0)` in `bracket-terminal` is the top-left of the screen.

## Examples

* `cargo run --example hello_minimal` puts "Hello Minimal Bracket World" on the screen.
* `cargo run --example hello_terminal` puts a bouncing "Hello World" on the screen in color, with frames-per-second [FPS] counting, and frame-rate limiting.
* `cargo run --example sparse` is the same demo, but with a second layer in a VGA 8x16 font on a second layer, no frame-rate limiting, and utilizing batched command submission.
* `cargo run --example walking` lets you use your keyboard to walk an `@` symbol around a random map.
* `cargo run --example astar-mouse` lets you use your mouse to move around a random map, using A-Star pathing (from the `bracket-pathfinding` crate) to avoid obstacles.
* `cargo run --example tiles` is similar to the `walking` demo, but uses two layers of graphical tiles (graphical back-ends only).
* `cargo run --example rex` demonstrates loading a sprite from [REX Paint](https://www.gridsagegames.com/rexpaint/) and rendering it to the terminal.
* `cargo run --example postprocess` demonstrates the library's post-processing effects - scan lines and screen burn.
* `cargo run --example textblock` demonstrates the `TextBlock` system, giving you a "builder" approach to constructing larger blocks of text with word-wrapping and formatting.
* `cargo run --example dwarfmap` demonstrates using the terminal with `Algorithm3D` to provide a Dwarf Fortress style 3D map (2D "slices" of a 3D world). It uses the `bracket-noise` library for terrain generation.
* `cargo run --example keyboard` demonstrates keyboard scan-code input. It's mostly useful for debugging.
* `cargo run --example textsprites` demonstrates multi-tile sprites.

repository: spenserblack/bracket-terminal

repo_name: spenserblack/bracket-terminal


text:
---
layout: default
---

# 1994-05-18 - Re: So PGP2.5 is becoming clearing...

## Header Data

From<EMAIL_ADDRESS>(John E. Kreznar)

To<EMAIL_ADDRESS>Message Hash: feaea3a30ff2a17b1c9aa2dbda35db398dc82f5c59852cdf47745e992badf524

Message ID: \<9405181043.AA24163@ininx\>

Reply To<EMAIL_ADDRESS>UTC Datetime: 1994-05-18 10:44:08 UTC

Raw Date: Wed, 18 May 94 03:44:08 PDT

## Raw message

```
{% raw  %}From<EMAIL_ADDRESS>(John E. Kreznar)
Date: Wed, 18 May 94 03:44:08 PDT
To<EMAIL_ADDRESS>Subject: Re: So PGP2.5 is becoming clearing...
In-Reply-To<EMAIL_ADDRESS>Message-ID: <9405181043.AA24163@ininx>
MIME-Version: 1.0
Content-Type: text/plain

-----BEGIN PGP SIGNED MESSAGE-----

> Lile Elam posted the RSA licensing agreement.  He thought it
> was bad.  I think it is great.  Maybe I do not understand it.

> Am I missing something?  This sounds like the war is over
> and we won!

You would have to consent to be a national person (United States of
America or Canada) in order to have it.  You wouldn't want to give up
your freeman status to do that, would you?

	John E. Kreznar		| Relations among people to be by
<EMAIL_ADDRESS>| mutual consent, or not at all.

-----BEGIN PGP SIGNATURE-----
Version: 2.3a

iQCVAgUBLdnwysDhz44ugybJAQHZfQQA0gLlkVbTOG72NR2FyFoKOzFSIPv/AG9k
+BoPCZjMqbDexVvWnftlUXizEVoUsM7qJHCN3oOurzntsJvRy0WVVd7HmomkV57l
8JC7yFBUI9Ogw/txa/I9+sVWymcKfTC6s7exIO7NDCX7pWu+nLuKDS+xZ4xfgaSN
MVdryFgx3Ww=
=XQNM
-----END PGP SIGNATURE-----

{% endraw %}
```

## Thread

+ Return to [May 1994](/archive/1994/05)

+ Return to<EMAIL_ADDRESS>(David Taffs)](/authors/dat_at_spock_ebt_com_david_taffs_)"
+ Return to<EMAIL_ADDRESS>(James A. Donald)](/authors/jamesd_at_netcom_com_james_a_donald_)"
+ Return to<EMAIL_ADDRESS>(John E. Kreznar)](/authors/jkreznar_at_ininx_com_john_e_kreznar_)"
+ Return to<EMAIL_ADDRESS>(Lile Elam)](/authors/lile_at_netcom_com_lile_elam_)"
+ Return to "["Perry E. Metzger" <perry@imsi.com>](/authors/perry_e_metzger_perry_at_imsi_com_)"
+ Return to "[Richard Johnson <Richard.Johnson@Colorado.EDU>](/authors/richard_johnson_richard_johnson_at_colorado_edu_)"

+ 1994-05-18 (Tue, 17 May 94 18:18:03 PDT) - [So PGP2.5 is becoming clearing...](/archive/1994/05/f7b7170b93c72e9594224c0f098e204c3dc79e7118b09011dd51a9949cb3fbf2) -<EMAIL_ADDRESS>(Lile Elam)_
  + 1994-05-18 (Tue, 17 May 94 18:47:56 PDT) - [Re: So PGP2.5 is becoming clearing...](/archive/1994/05/5202525192899a7f1cb031f2e638c578b92e6432496e9bc78c26a0db5f56bfd3) -<EMAIL_ADDRESS>(James A. Donald)_
    + 1994-05-18 (Wed, 18 May 94 03:44:08 PDT) - Re: So PGP2.5 is becoming clearing... -<EMAIL_ADDRESS>(John E. Kreznar)_
      + 1994-05-18 (Wed, 18 May 94 08:33:34 PDT) - [Re: So PGP2.5 is becoming clearing...](/archive/1994/05/569246a06a47fa3d0d93b27005d1536ef841ee0b0cb32dbc660306b86a9cf07d) - _"Perry E. Metzger"<EMAIL_ADDRESS>        + 1994-05-18 (Wed, 18 May 94 15:08:05 PDT) - [So PGP2.5 is becoming clearing...](/archive/1994/05/79e801c7c79e0b380494b8d73af4110ce8e6776d244c3c4e72dfd0561dd10b5e) -<EMAIL_ADDRESS>(John E. Kreznar)_
  + 1994-05-18 (Tue, 17 May 94 19:51:45 PDT) - [Re: So PGP2.5 is becoming clearing...](/archive/1994/05/4b6dd8b5185a3d47cd9ba7dad9bc96cbff89f8f1730fdf074527ca5e4003e9e7) - _Richard Johnson<EMAIL_ADDRESS>    + 1994-05-18 (Wed, 18 May 94 11:28:34 PDT) - [Re: So PGP2.5 is becoming clearing...](/archive/1994/05/8919068ed60df8e57baaa9363ff413ef5474bd032e686982f13d16810bdd516b) -<EMAIL_ADDRESS>(David Taffs)_


text:
Example of functions implemented in Python
==================================

In this example, we define two functions `Foo` and `Bar` in `func_config.json`.
Both functions are implemented in Python (`main.py` includes handlers of both functions).
Foo will invoke Bar with Nightcore's API for internal function calls.

This example requires Python 3.7+ to run (we have tested Python 3.8.5).
First run `make` within `$REPO_ROOT/worker/python` to build Nightcore's Python module.

Then run `run_stack.sh` to start all Nightcore components. Finally, run
```
curl -X POST -d "Hello" http://<IP_ADDRESS>:8080/function/Foo
```
to invoke function Foo, and you will see the output
```
From function Bar: Hello, World
```

repository: mjaysonnn/nightcore

branch_name: refs/heads/asplos-release

repo_name: mjaysonnn/nightcore