text:
# Cratetorrent design

This document contains notes on the *current* design of the implementation of
both the cratetorrent library as well as the CLI cratetorrent binary. It serves
as a high-level documentation of the code and documenting design decisions that
may not be self-evident. It is continuously expanded as more features are added.
It is important to note that this document only reflects the current state of
the code and does not include future features, unless otherwise noted.

The library can be found in the [`cratetorrent`](./cratetorrent) folder, while
the binary is in [`cratetorrent-cli`](./cratetorrent-cli). The rationale for
developing a library and a binary simultaneously is because big emphasis is
placed on the usability of the cratetorrent library API, and testing the API, as
it is developed, in a near real world scenario helps to integrate feedback
immediately.

## Documentation

This document is focused on the high-level design of cratetorrent, but the code
has ample of inline documentation (beyond the documentation of public APIs). To
view it, you need to run the following command from the repo root:
```bash
cargo doc --document-private-items --workspace --exclude cratetorrent-cli --open
```

## Sources

- The official protocol specification: http://bittorrent.org/beps/bep_0003.html
- My previous torrent engine implementation (in C++), Tide: (https://github.com/mandreyel/tide)
- The libtorrent blog: https://blog.libtorrent.org/

**NOTE**: This document assumes familiarity with the official protocol.

## Overview

Currently the "engine" is simplified to perform a download from seed-only
connections. Still, such a simple goal requires quite a few components (each
detailed in its own section):

- [metainfo](#metainfo), which contains necessary information to start the
  torrent;
- [engine](#engine), which orchestrates all components of the torrent engine,
- [torrent](#torrent), which coordinates the download of a torrent;
- a [piece picker](#piece-picker) per torrent, that selects the next piece to
  download;
- [peer connection](#peer-connection) in a torrent, which implement the peer
  protocol and perform the actual download (any number may exist);
- in progress [piece download](#piece-download), which track the state of an
  ongoing piece download;
- [disk IO](#disk-io), which saves downloaded file pieces.

At the end of the document, there is an [overview](#anatomy-of-a-block-fetch) of
a block fetch, incorporating the above key components.

The binary takes as command line arguments the IP address and port pairs of all
seeds to download from (must be at least one), the torrent metainfo, and sets up
a single torrent, piece picker, and peer connections to the seeds. All
IO is done asynchronously, and the torrent has a periodic update function, run
every second.

## Metainfo

A torrent's metadata is described in the metainfo file. Starting a download or
  upload requires feeding such a metainfo file to the torrent client, which will
  extract relevant information from it about the torrent and start the
  upload/download. The contents of the metainfo file must be UTF-8 encoded.

There is an easy way to parse metainfo files using the [`serde` bencode
extension](https://github.com/toby/serde-bencode).

The metainfo is a dictionary of key values. It has the following two top-level
keys:
- **`announce`**: The URL of the torrent's tracker.
- **`info`**: The torrent information, described below.

### Announce

Trackers are not implemented so this is disregarded for now.

### Info

- **`name`**: UTF-8 encoded string of the name of the torrent, which in the case
  of a single file download is the name of the downloaded file, or in the case
  of a multi-file download the name of the downloaded directory. However, when
  downloading, the user should be able to set a different name.
- **`piece length`**: Files are split up into _pieces_, and the length of a
  piece is the number of bytes in a piece, except for possibly the last piece,
  which may be shorter. This value is usually a power of two, and usually some
  multiple of 16 KiB. File pieces are indexed from zero.
- **`pieces`**: A list of SHA1 hashes, that represent the expected value of
  hashing each of the file pieces.
- **`length`** exclusive or **`files`**:
  - **`length`**: In case of a single file download, this is the length of the
    file, and the name of the file is the download name.
  - **`files`**: In case of multiple files, this is a list of files representing
    the directory structure of the torrent archive, and each element is a
    dictionary:
    - **`length`**: The length of the file.
    - **`path`**: The UTF-8 encoded full path of the file, relative to the
      download root.

### Torrent info hash

The hash of the torrent is the SHA1 hash of the raw (bencoded) string of the
metainfo's `info` key's value.

Note that this can be problematic as some metainfo files define fields not
supported or used by cratetorrent, and currently the way we generate this hash
is by encoding the parsed metainfo struct again into its raw bencoded
representation, and this way we may not serialize fields originally present in
the source metainfo.
A better approach would be to keep the underlying raw source (as in Tide), but
  this is not currently possible with the current solution of using `serde`.

## Engine

Currently there is no explicit entity, but simply an engine module that provides
a public method to start a torrent until completion.

## Torrent

- Contains the piece picker and connections to all seed sfrom which to download
  the torrent.
- Also contains other metadata relevant to the torrent, such as its info hash,
  the files it needs to download, the destination directory, and others.
- Torrent tick: periodically (currently set to 1 second) loops through all its
  peer connections and performs actions like stats collections, later
  choking/unchoking, resume state saving, requesting peers from tracker(s) if
  needed, and others.

### Trackers

Only HTTP trackers are supported. These are used to request peers to download
from, as well as to announce our download or upload statistics.

This is handled in torrent's event loop. The tracker has an interval in which we
are allowed to request peers to not overwhelm the tracker, which may only be
overridden if the torrent has no peers to download from.

Each tracker HTTP request runs asynchronously and is polled by the torrent
event loop's select call.

Periodically each torrent also sends progress updates to the tracker. The
periodicity is defined by the tracker, but it may be configurable in the future.

### Peer sessions

A peer session is spawned on a new
[task](https://docs.rs/tokio/0.2.13/tokio/task), and torrent and the peer
session communicate with each other via async `mpsc` channels. This is
necessary as a peer connection is spawned on another task, which from the
point of view of the borrow checker is as though it were spawned on another
thread (which it may well be), so we can't just call methods of peer session
without synchronization.

It would be possible to spawn the peer task
[locally](https://docs.rs/tokio/0.2.20/tokio/task/fn.spawn_local.html), but we'd
still have to use channels for communication as running the peer session and
calling its methods on the torrent task would cause lifetime issues, due to the
peer session's run method taking a mutable reference to self, thus preventing
torrent from calling any of its other methods.

As to whether running all peer sessions and torrent on the same local task
(since a peer session doesn't do much other than IO) would be more performant
over running each on a separate task is an open question and more research is
needed.

### Stats collection

Torrent aggregates stats from its peer sessions as well as itself. This
information is mostly session metadata: whether the session is running or
stopped, whether a peer is a seed or a leech, its throughput rates, piece
availability, etc. These are needed either simply for stats reporting, but
crucially to make informed decisions about the management of the torrent: such
as which peers to unchoke, which connections to close, etc.

These need to happen frequently, about once a second. As mentioned above, `mpsc` channels are
used to communicate these state changes. Each peer session, when its
[tick](#session-tick) routine runs, sends a "state change" message to its
torrent from which torrent calculates its own state and performs various
bookkeeping. See more info [below](#sending-stats-changes).

## Piece picker

Each torrent has a piece picker, which is the entity that collects information
about the torrent swarm's piece availability in order to make a more optimal
decision on what piece to pick next. For now though, pieces are downloaded
sequentially and more complex algorithms will be implemented later (see research
notes).

The piece picker holds a vector pre-allocated to the number of pieces in the
torrent and each element in this vector contains metadata about the piece:
whether we have it or not and its frequency in the swarm.

Later the internal data structures of the piece picker will most likely be
changed to be more optimal for the rarest-pieces-first algorithm (the default
defined by the standard).

## Peer connection

This represents an open or closed connection to another BitTorrent peer. It
implements the protocol defined in specification, under "peer protocol". The
specification defines connections over TCP or uTP (uTorrent protocol), but
currently only TCP is supported.

- Peer connections are symmetrical.
- Peers request from each other file pieces by their indices.
- Each side of the connection could be in one of two states (from the POV of the
  protocol): choked or unchoked. When a peer is choked, it means that it may not
  request pieces.
- Thus transfer of data only takes place when one side is interested and is not
  choked. Peers need to express interest.
- Connections start out choked and not interested.
- Download pipeline: downloaders should keep several piece requests queued up
  for optimal throughput rates. This is achieved by sending several piece
    requests at a time and always keeping the optimal number of piece requests
    outstanding until choked or the download is complete. See the [download
    pipeline section](#download-pipeline).
- The first exchanged message _must_ be the handshake.
- Apart from the handshake length prefix, all integers are encoded as four bytes
  in big-endian (i.e. multi-byte integers are sent in network order).
- Each peer sends empty keep-alive messages, but timeouts could occur sooner
  when data is requested but not received in time.
- While a file is split into pieces, peers usually never download the entire
  piece. Rather, downloads happen for _blocks_, which are the equal size units
  within pieces. This is almost always **2^14 bytes**, or **16 KiB**, some
  clients even rejecting to service requests for piece blocks larger than this
  value.
- All IO is asynchronous, and the "session loop" (that runs the peer session) is
  going to multiplex several event sources (futures): receiving of messages from
  peer, saving of a block to disk, updates from owning torrent.

### Startup

1. Connect to TCP socket of peer.
2. We're in the handshake exchange state.
3. If this is an outbound connection, start by sending a handshake, otherwise
   just start receiving and wait for incoming handshake.
4. Receive handshake, and if it is valid and the torrent info hash checks out,
   send our own handshake.
5. The connected peers may now optionally exchange their piece availability.
6. After this step, peers start exchanging normal messages.

### Current session algorithm

A simplified version of the peer session algorithm follows.

1. Connect to the TCP socket of peer and send the BitTorrent handshake.
2. Receive and verify peer's handshake.
3. Start receiving messages.
4. Receive peer's bitfield. If the peer didn't send a bitfield, or it doesn't
   contain all pieces (i.e.  peer is not a seed), we abort the connection as
   only downloading from seed's are supported.
5. Tell peer we're interested.
6. Wait for peer to unchoke us.
7. From this point on we can start making block requests:
   1. Fill the request pipeline with the optimal number of requests for blocks
      from existing and/or new piece downloads.
   2. Wait for requested blocks.
   3. Mark requested blocks as received with their corresponding piece download.
   4. If the piece is complete, mark it as finished with the piece picker.
   5. After each complete piece, check that we have all pieces. If so, conclude
      the download, otherwise make more requests and restart.

### Download pipeline

After receiving an unchoke message from our seed, we can start making requests.
In order to download blocks in the fastest way, we always need to keep the
number of requests outstanding that saturates the link to the peer we're
downloading from.

From a relevant [libtorrent
blog post](https://blog.libtorrent.org/2011/11/requesting-pieces/):

> Deciding how many outstanding requests to keep to peers typically is based on
  the bandwidth delay product, or a simplified model thereof.

> The bandwidth delay product is the bandwidth capacity of a link multiplied by
  the latency of the link. It represents the number of bytes that fit in the
  wire and buffers along the path. In order to fully utilize a link, one needs
  to keep at least the bandwidth delay product worth of bytes outstanding
  requests at any given time. If fewer bytes than that are kept in outstanding
  requests, the other end will satisfy all the requests and then have to wait
  (wasting bandwidth) until it receives more requests.

> If many more requests than the bandwidth delay product is kept outstanding,
  more blocks than necessary will be marked as requested, and waste potential to
  request them from other peers, completing a piece sooner. Typically, keeping
  too few outstanding requests has a much more severe impact on performance than
  keeping too many.

> On machines running at max disk capacity, the actual disk-job that a request
  result in may take longer to be satisfied by the drive than the network
  latency. It’s important to take the disk latency into account, as to not under
  estimate the latency of the link. i.e. the link could be considered to go all
  the way down to the disk [...] This is another reason to rather
  over-estimate the latency of the network connection than to under-estimate it.

> A simple way to determine the number of outstanding block requests to keep to
  a peer is to take the current download rate (in bytes per second) we see from
  that peer and divide it by the block size (16 kiB). That assumes the latency
  of the link to be 1 second.
> num_pending = download_rate / 0x4000;
> It might make sense to increase the assumed latency to 1.5 or 2 seconds, in
  order to take into account the case where the other ends disk latency is the
  dominant factor. Keep in mind that over-estimating is cheaper than
  under-estimating.

Based on the above, each peer session needs to maintain an "optimal request
queue size" value (approximately the bandwidth-delay product), which is the
number of block requests it keeps outstanding to fully saturate the link.

This value is derived by collecting a running average of the downloaded bytes
per second, as well as the average request latency, to arrive at the
bandwidth-delay product B x D. This value is recalculated every time we receive
a block, in order to always keep the link fully saturated. See
[wikipedia](https://en.wikipedia.org/wiki/Bandwidth-delay_product) for more.

So the best request queue size can be arrived at using the following formula:

```
Q = B * D / 16 KiB
```

Where B is the current download rate, D is the delay (latency) of the link, and
16 KiB is the standard block size. The download rate is continuously measured
and used to adjust this value. The latency for now is hard-coded to 1 second,
but this may be probed and dynamically adjusted too
(https://github.com/mandreyel/cratetorrent/issues/44).

To emphasize the value of this optimization, let's see a visual example of
comparing two connections each downloading two blocks on a link with the same
capacity, where one pair of peers' link is not kept saturated and another pair
of peers whose is:
```
Legend
------
R: request
B: block
{A,B,C,D}: peers

  A      B        C      D
R |      |      R |      |
  |\     |        |\     |
  | \    |      R | \    |
  |  \   |        |\ \   |
  |   \  |        | \ \  |
  |    \ |        |  \ \ |
  |     \|        |   \ \|
  |      | B      |    \ | B
  |     /|        |     /|
  |    / |        |    / | B
  |   /  |        |   / /|
  |  /   |        |  / / |
  | /    |        | / /  |
  |/     |        |/ /   |
R |      |        | /    |
  |\     |        |/     |
  | \    |
  |  \   |
  |   \  |
  |    \ |
  |     \|
  |      | B
  |     /|
  |    / |
  |   /  |
  |  /   |
  | /    |
  |/     |
```

And even from such a simple example as this, it can be concluded as a feature
and not premature optimization.

#### Slow start

Initially the request queue size is going to start from a very low value, but it
in case of fast seeders we want to max out the available bandwidth as quickly as
possible.

TCP has already solved this: it uses the [slow start
algorithm](https://blog.libtorrent.org/2011/11/requesting-pieces/) to discover
the link capacity as quickly as possible, and then enter congestion avoidance
mode. Introducing this on the BitTorrent level allows us to keep up with the
underlying TCP slow start algorithm and waste as little bandwidth as possible.

The mechanism is similar: in the beginning the session is in slow start
mode and increases the target request queue size by one with every block it
receives (doubling the queue size with each complete round trip). While in TCP
the slow start mode is left when the first timeout or packet loss occurs, at the
application layer this is less obvious and one mechanism employed by
[libtorrent](https://blog.libtorrent.org/2015/07/slow-start/) is to leave slow
start when the download rate is not increasing (significantly) anymore.
Libtorrent leaves slow start when the download rate inceases by less than 10
kB/s. Currently cratetorrent does the same.

#### Timing out requests

Each peer session needs to be able to time out requests, for two reasons:
- A peer may never send the request. Without timing out a request, we would not
  be able to attempt the download from another peer. This would cause the
  whole download to get stuck. This is the most crucial issue.
- It is also a form of optimization: if one peer doesn't send it within some
  allowed window, we can attempt to request it from another peer, which may
  result in the piece completing faster.

Each peer session has its own timeout mechanism, as each peer will have unique
download characteristics and so the timeout value needs to be dynamically
adjusted for each peer. A weighed running average is used to measure round trip
times, with new samples having moderate amount of weight (currently 1/20). This
guards against jitters. This average is used to derive the timeout value.

We don't want to time out peers instantly. For fast seeders the long tail of
request round trip times will be a few milliseconds, so any occasional hiccup
will be relatively large, which would time out the peer instantly if we didn't
use a smoothed running average. This is not desirable, as the overall round trip
time will still be low. For this reason, timeouts are minimum 2 seconds. This
number comes from tide, and further experiments are needed to prove its
viability.

### Session tick

Every second a peer session runs the update loop, which performs periodic
updates.

Each round the current download and upload rates are collected, and the session
tick collects each rounds value, calculates a running average, and resets the
per round counter. Using this counter, it is also checked whether the session
needs to escape slow start.

Since the minimum timeout granularity is measured in seconds, the timeout
procedure is also part of the session tick.

#### Sending stats changes

Further, state change updates are sent to the torrent in the session tick. For
more info, see [above](#stats-collection).

It is not known whether using mostly `mpsc` channels for most inter-task control
flow is too heavy. Let's do some napkin math:
- assume this is a fully utilized engine, say 20 torrents, half of them seeds,
  half of them leeches;
- each torrent has about 50 connected peers;
- we only seed to about 10 peers in total;
- the leech torrents download from 10 peers each;
- average piece size is 128 KiB (8 16 KiB blocks).
- We also assume ideal but realistic throughput rates:
  - each seed uploads at 5 MBps
  - and each leech downloads at 10 MBps.

Then, based on this, the following rough metrics hold:
- each leech receives 640 16 KiB blocks a second, or 64 per peer session (10 MiB
  throughput / 16 KiB),
- which corresponds to 640 block write messages and 80 piece completion messages
  a second, per downloader.
- Thus 10 * (640 + 80) = 7200 disk write related `mpsc` messages per second for
  all downloaders;
- Then, we also have 4 uploaders: each seed uploads 320 blocks to its peer per
  second;
- so 4 * 320 = 1280 `mpsc` round-trips (so twice that many messages) are made by
  all uploaders.
- For disk IO only, we have roughly 7200 + 2 * 1280 = 9760 such messages across
  all torrents.

To these come the periodic stats reporting functions, once a second for each
peer: 20 * 50 = 1000 messages. We can probably slightly optimize this for a peer
session to only send a message if there has been some change. In that case, we
need only send 10 * 10 + 5 = 105 messages (100 downloader peer sessions and 5
uploader ones). The former is about 10% of all internal messages sent, while the
latter would be about 1%.

Assuming the overall number of messages have a measurable overhead, the
unoptimized variant is perhaps not acceptable, but the latter is within
calculation error.

### Seeding

Seeding is relatively uncomplicated, on the peer session level, anyway. The peer
sends us requests for a block and we check if 1) the block is correct, 2) aren't
choked and 3) aren't already requesting the same block. If these checks pass,
the peer session sends a request to the disk task to [fetch the
block](#reading-from-disk).

### Messages

The messages a peer can exchange is detailed [here](./PEER_MESSAGES.md).

## Piece download

A piece download tracks the piece completion of an ongoing piece download.

It plays an important role in optimizing download performance:
- sharing block requests in piece among peers,
- timing out peer requests,
- and end game mode to speed up the last part of the
  download.

Currently the last two are not implemented.

Piece downloads are stored in the torrent and shared with all peers in the
torrent. When a peer starts a new download, it places the download instance in
the shared torrent object. This way other peers may join this download.

## Disk IO

This entity (called `Disk` in the code, in the [`disk`
module](cratetorrent/src/disk.rs]) is responsible for everything disk storage
related. This includes:
- allocating a torrent's files at download start;
- hashing downloaded file pieces and saving them to disk;
- once seeding functionality is implemented: reading blocks from disk.

### High level architecture

All disk related activity is spawned on a separate task by the torrent engine,
and communication with it happens through `mpsc` channels. The alternative is
for `Disk` to not be on its own task but simply be referenced (through an `Arc`)
by all entities using it. In this case, we would have the following issues:
- While disk IO itself is spawned on blocking tasks (see below), we need to
  await its result. This means that for e.g. block writes initiated from a peer
  session task, the peer session task would be blocked (since await calls in the
  same `async` block are sequential). This is the most crucial one.
- Increased complexity due to lifetime and synchronization issues with multiple
  peer session tasks referring to the same `Disk`, whereas with the task based
  solution `Disk` is the only one accessing its own internal state.
- Worse separation of concerns, leading to code that is more difficult to reason
  about and thus potentially have more bugs. Whereas having `Disk` on a separate
  task allows for a sequential model of execution, which is clearer and less
  error-prone.

Spawning the disk task returns a command channel (aka disk handle) and a global
alert port (aka receiver): the former is used for sending commands to disk and
the latter is used to receive back the results of certain commands. However,
because `mpsc` channels can only have a single receiver and because each torrent
needs its own alert port, we use a second layer of alert channels for torrents.

Thus, when `Disk` receives a command to create a torrent, it creates the torrent
specific alert channel's send and receive halves, and the latter is returned in
the torrent allocation command result via the global alert channel, so engine
can hand it down to the new torrent. A step by step example follows.

#### Example workflow

1. `Engine` creates a new torrent, and part of this routine allocates the torrent
   on disk via the disk command channel.
2. `Disk` creates a `Torrent` in-memory entry and allocates torrent and creates a
   new alert channel pair and sends the receiving half via the global alert
   port.
3. `Engine` listens on the alert port for the allocation result containing the
   torrent alert port (or an error).
4. `Engine` creates a `Torrent`, which spawns `PeerSession`s with a copy of the disk
   handle, and starts downloading blocks.
5. Each peer session saves blocks to disk by sending block write commands to
   `Disk` via the handle.
6. Blocks are usually buffered until their piece is complete, at which point the
   blocks are written to disk. The result of the disk write is advertised to
   torrent via its alert channel.
7. Torrent receives and handles the block write result.

Caveat emptor: the performance implications of this indirection needs to be
tested, `mpsc` is not guaranteed to be the good choice here.

To summarize, sending alerts from the disk task is two-tiered:
- global (e.g. the result of allocating a new torrent),
- per torrent (e.g. the result of writing blocks to disk).

### Torrent allocation
Since a torrent may contain multiple files, they will be downloaded in a
directory named after the torrent. There may be additional subdirectories in a
torrent, so the whole torrent's file system structure needs to be set up before
the download is begun.

For single file downloads support the file allocation is very simple: the
to-be-downloaded file is checked for existence and the first write creates the
file.

Late we will want to pre-allocate sparse files truncated to the download length.

### Saving to disk

#### Buffering

The primary purpose of `Disk` right now is to buffer downloaded blocks in memory
until the piece is complete, hash the piece, notify torrent of the result, and,
if the resulting hash matches the expected one, save to disk.

Efficiency is key here; a torrent application is twofold IO bound: on disk
and on the network. To achieve the fastest download given a fixed network
bandwidth is to buffer the whole download in memory and write a file to disk in
one go, i.e. to not produce disk backpressure for the download.

However, this is not practical in most cases. For one, the download host may be
shut down mid-download, which would lead to the loss of the existing downloaded
data. The other reason is the desire to constrain memory usage as the user may
not have enough RAM to buffer an entire download, or even if they do, they would
likely wish that other programs running in parallel would also have sufficient
RAM.

For these reasons cratetorrent will be highly configurable to serve all needs,
but will aim to provide sane, good enough defaults for most use cases. For now,
though, the implementation is very simple: we download sequentially, and buffer
blocks of a piece until the piece is complete, after which that piece is flushed
to disk. The rationale for saving a piece to disk as soon as it is complete is
twofold:
- being defensive: so that in an event of shutdown, as much data is persisted as
  possible;
- simplicity of the implementation of the MVP. (Later it will be possible to
  configure cratetorrent to buffer several pieces before flushing them to disk,
  or buffer at most `n` blocks, which may be fewer than is in a torrent's
  piece.)

#### Mapping pieces to files

When writing a piece to disk, we need to determine which file(s) the piece
belongs to. Further, a piece may span several files. This way, every time we
wish to save a piece, we need to look for its file boundaries and thus
"partition" the piece, saving the partitions to their corresponding files.

To find the files quickly, if we imagine the torrent as a single contiguous byte
stream, we can pre-compute each file's offset in the entire torrent, and compute
each piece's offset in the torrent, and use this information to check which
pieces intersect the files.

More concretely, there are two options:
- compute what pieces a file intersects with  at the beginning of a torrent, and
  store it with the file information,
- or compute what files the piece intersects with when starting a piece
  download, storing it with the in-progress piece.

The second approach is chosen for several reasons:
- the way the code is currently laid out it is easier to implement and test;
- in the first approach, to find the matching piece indices, we'd have to loop
  through all files anyway, so we don't save much computation;
- and also, there is slightly less memory overhead as we're only storing this
  data when a piece is being downloaded, not with all files in a torrent at all
  times, which is wasteful if their pieces are already downloaded.

The concrete algorithm is this:
- find the first file where
  `[file.start_offset, file.end_offset).contains(piece.start_offset)`
- if none is found, return an empty range (`[0..0)`)
- otherwise initialize the resulting file index range with
  `[found_index, found_index + 1)`
- then loop through files after the above found one, while
  `[piece.start_offset..piece.end_offset].contains(file.start_offset)` returns
  true and record the file's index in our resulting range's end
- return the resulting range

Unless the piece is very large and there are many small files, the second part
(finding the last file intersecting with piece) should be very quick, with only
a few iterations. (This is the part that could be optimized by storing which
pieces a file intersects with in the file information.)

Then, getting the file indices is not sufficient: when writing to each file, we
also need to know where exactly in the file we need to write the part of the
piece. For each file to be written to, we query the slice of the file that
overlaps with the piece, and write to file at that position.

##### Example

Let's illustrate with an example. The first row contains 3 pieces, each 16 units
long (the precise unit doesn't matter now), and indicates their indices and
offsets within the whole torrent. The second row contains the files in the
torrent, of various length, indicating their names and their first and last byte
offsets in the torrent.

```
--------------------------------------------------------------
|0:0               |1:16                |2:32                |
--------------------------------------------------------------
|a:0,8      |b:9,19    |c:20,26  |d:27,36  |e:37,50          |
--------------------------------------------------------------
```

Then, let's assume we got piece 1 and wish to save it to disk and need to
determine which file(s) the piece belongs to.
`files_intersecting_piece(1)` should yield the files: `b`, `c`, `d`.

Another example:

```
-------------------------------------------------------
|0:0               |1:16                |2:32         |
-------------------------------------------------------
|a:0,15            |b:16,19 |c:20,23    |d:32,44      |
-------------------------------------------------------
```

Here, `files_intersecting_piece(1)` should yield the files: `b`, `c`.

#### Vectored IO

A peer downloads pieces in 16 KiB blocks and to save overhead of concatenating
these buffers these blocks are stored in the disk write buffer as is, i.e the
write buffer is a vector of byte vectors.
Writing each block to disk as a separate system call would incur tremendous
overhead (relative to the other operations in most of cratetorrent), especially
that context switches into kernel space have become more expensive lately to
mitigate risks of speculative execution, so these buffers are flushed to disk in
one system call, using [vectored
IO](http://man7.org/linux/man-pages/man2/readv.2.html).

While in most of the code asynchrony is provided for by `tokio`, all disk IO is
synchronous because `tokio_fs`'s vectored write implementation is just a wrapper
around repeatedly calling `write` for each buffer, defeating any performance
gains. For these instance of blocking IO, we make use of `tokio`'s builtin
threadpool and spawn the blocking IO
[tasks](https://docs.rs/tokio/0.2.17/tokio/task/fn.spawn_blocking.html) on it.

The [`pwritev`](https://linux.die.net/man/2/pwritev) syscall is used for the
concrete IO operation. This syscall allows one to write a vector of byte arrays
(referred to as "iovec") at a specified position in the file, without having to
make a separate [`seek`](https://linux.die.net/man/2/lseek) syscall. This means
that we do not need to write to the file in sequential order.

##### Writing blocks spanning multiple files

There is an additional complication to this: a block in a piece may span
multiple files. Therefore, we can't just pass all blocks in a piece to
`pwritev`, as that could write more than is supposed to be in the file (the
syscall writes as long as there is data in the passed in iovecs, potentially
truncating the file to a larger than desired size).

Thus, when the blocks in a piece would extend beyond a file, we need to shrink
the slice of iovecs passed to `pwritev`. To illustrate:
```
------------------
| file1 | file 2 |
--------.---------
| block .        |
--------.---------
        ^
  file boundary
```
On the first write, to `file1`, we need to trim the second part of the block.
Conversely, on the second write, to `file2`, we need to trim the first part.

There is a problem here: trimming the iovec during the first write makes us lose
the second part of the block (since we're working with slices here to
minimize allocations). There are two solutions:
- When we detect that the file boundary is in the middle of an iovec, we
  construct a new vector of iovecs, that is bounded by the length of the file
  slice that we're writing to (we can use
  [copy-on-write](https://doc.rust-lang.org/std/borrow/enum.Cow.html) to
  seamlessly handle the resulting iovecs). This avoids overwriting the original
  iovecs, at a slight cost.
- Keep metadata about the trimmed iovec, if any, and restore the trimmed part
  after the IO, [like in
  tide](https://github.com/mandreyel/tide/blob/master/src/file.cpp#L403).

The second part is chosen for performance (as it is one of the main goals of
cratetorrent), as well as for the joy of optimizing such small and
self-contained code (this is a spare time project after all). The hairy logic is
encapsulated in the [`iovecs` module](cratetorrent/src/iovecs.rs), providing a
simple and safe abstraction.

Finally, after each write, we must trim the front of the slice of iovecs by the
number of bytes written, so that for the next file we don't write the wrong
data (otherwise we'd be writing out-of-order, i.e. wrong data to the file). This
is also taken care of by `IoVecs`.

#### Anatomy of a piece write

1. Create a new in-progress piece entry and calculate which files it spans, as
   per [the above step](#mapping-pieces-to-files).
2. Buffer all downloaded blocks in piece's write buffer.
3. When the piece is complete, remove its entry from the torrent and spawn a new
   blocking task for the hashing and write.
4. Hash the piece, and if the result is good, continue. Otherwise let torrent
   know via the alert channel of the bad piece.
5. Allocate a vector of iovecs pointing to the blocks in the piece.
6. Create a slice for this vector of iovecs; this will be adjusted with each
   write.
7. For each file that the piece spans:
  8. Get the slice of the file where we need to write the piece.
  9. Bound the iovec by the length of the file slice, optionally allocating a
     new vector of iovecs if the provided iovecs exceed the file slice's length.
  10. Write to the file at the position and for the length of the slice.
  11. Trim the slice of iovecs by the number of bytes that were written to disk.
  12. Step 10 and 11 may need to be repeated if not all bytes could be written
      to disk (as the syscall doesn't guarantee that all provided bytes can be
      written).

### Reading from disk

Reading from disk is somewhat similar to writing: we make use of a read buffer
(or cache), we use vectored IO over potentially multiple files, and use almost
the same `mpsc` infrastructure to communicate results. Therefore this is going
to be an overview of the differences.

#### Communicating results

The major difference is how results are communicated: while for block writes the
piece result is communicated to the torrent, the result of a disk read is
communicated directly to a peer. This is for two reasons:
- The torrent doesn't need to know about disk reads, so sending the read block
  first to torrent and then from torrent to the peer would not only be
  redundant, it would introduce some complexity.
- Although profiling is needed, it's likely more optimal to directly send the
  block to peer.

Now the reason I say likely is because there is a trade-off: currently peer
needs to pass a clone of its command sender with each disk read to the disk
task (otherwise the disk task wouldn't know to whom to return the block). This
incurs two updates to the atomic reference count (creation and drop), so this
solution is not entirely optimal either. For now it suffices though.

However, an _error_ is communicated to the torrent, rather than a peer session.
This is because the peer session is the wrong layer to deal with such global
errors. So the torrent is chosen. In the future it may be the engine that
handles such errors, but there is currently no real engine so this is kept in
torrent for now.

#### Read cache

The read cache is brutally simple at this step of the alpha version. For each
block request, we read in the _entire_ piece from disk, and then store it in a
hashmap in memory. That's it. Subsequent reads to blocks in the same piece will
hit the cache.

The problem here of course is that there is no eviction: so memory use could
blow up here very fast. This is one of the reasons why cratetorrent is still a
toy project (besides being very obviously in alpha).

In the future the read cache size will be configurable, as well as various other
parameters, such as how many blocks to read in when caching a piece (read cache
line size).

## Anatomy of a block fetch

1. `PeerSession` requests `n` blocks from peer.
2. For each received block, `DiskHandle::write_block` is called which sends a
   message via the command channel to instruct `Disk` to place the block in the
   write buffer.
3. Once the corresponding piece has all blocks in it, it is hashed and, if the
   hash matches the expected hash, saved to disk.
4. The written blocks and the hash result is communicated back to torrent via a
   channel (or the IO error if any occurred).
5. `Torrent` receives this message, processes it, and forwards it to the peer
   session.

## Anatomy of seeding a block

1. `PeerSession` receives a request for a block from peer.
2. Various conditions are [checked](#seeding) and a block read request is sent
   to the disk task, along with a copy of the peer session's command sender.
3. Check if the block's piece is already in the read cache.
4. If not, the entire piece for which the block was requested is read from disk
   and placed in the read cache.
5. Return the block via the provided sender.
6. Peer eventually receives the block from the disk task and sends it to the
   peer if there hadn't been any cancel messages during this time.

## Error handling

Particular emphasis is placed on correct error handling; both internally and
the way the API consumer may handle errors.

Errors are distinguished as **fatal** and **fallible**. Fatal errors cause the
system to stop, while fallible errors are such as would occur on a time-to-time
basis (more frequently network IO failure, less frequently disk IO failure)
which should not be grounds for stopping the engine.

The distinction is made due to the ability to effortlessly propagate errors via
Rust's try (`?`) operator, as otherwise with a single `Error` type, encompassing
all possible errors of the crate (as is common in the Rust ecosystem), it would
be all too easy to forget to handle errors in critical places. For this reason
not only are the two types of errors distinct types, they don't provide
automatic conversion either.

As an example, take disk IO errors: they are generally not expected to occur,
but it could be that the user has moved the download file while the download was
ongoing, or that disk storage has been used up. This, however, should not bring
the entire engine to halt and instead these errors should be logged, routed to
the responsible entity, and the operation re-attempted at a later time point.

Now recovery is not implemented in cratetorrent yet, but the error distinction
is: there is a general `Error` type, and a `WriteError`. The disk task is
run until a fatal error is encountered (e.g. `mpsc` channel failure) and so all
it's internal methods only return `Error` on fatal error; non-fatal errors are
communicated via the alert channels. Because of this we know that in the
following code a disk write won't abort the event loop because of IO failure,
only because of channel sending failure (which is for now the desired behavior):
```rust
while let Some(cmd) = self.cmd_rx.recv().await {
    match cmd {
        Command::BlockWrite(block) => {
            self.write_block(block)?;
        }
    }
}
```


text:
# User Stories
User Stories for the Bus Mall Project

## Marketing-Research Team
As a marketing and research team, I want to collect user interest in my products so that I can learn about potential markets in bus-riders.

As a marketing and research team, I want to see how users react to different products side-by-side so that I can learn how to best present my products.

As a marketing and research team, I want each user to be tested a limited number of times to maintain healthy data in my results.

As a marketing and research team, I want each user to see a wide variety of products so that I can gather the most information.

## Focus Group
As a focus-group member, I want to see interesting products in an aesthetic environment so that I maintain my focus throughout the survey.

As a focus-group member, I want to see the range of products that Bus Mall will be offering so that I can gauge my own interest in the company.

As a focus-group member, I want to see how my answers compare to others, because it continues my interest in the company.

As a focus-group member, I want to see a variety of products, and never two of the same product, so that I stay interested in the experience.

## Developer
As a developer, I want to collect all neccesary data at the time of each selection, so that I can easiy and accurately store this data.

As a developer, I want to create an attractive website so that users have a positive brand experience with Bus Mall.

As a developer, I want to prevent a product from displaying twice during the same prompt, so a user compares each item to two others.

As a developer, I want the user to be able to easily click on a product in order to cast a vote for it, so that it is simple and easy for a user to provide accurate data.

As a developer, I want new images to appear instantly after a user casts their vote in order to collect multiple votes from the user.

As a devloper, I want to prompt the user 25 times and then stop allowing votes, so that each user is asked the same number of questions.

license: MIT

repository: jkliptonia/bus-mall

branch_name: refs/heads/master

extension: md

license_type: permissive

repo_name: jkliptonia/bus-mall


text:
# advanced 2 way binding

In hello world of 2-way binding we discussed how to directly interact with a state-item using `IS v-model`. Although handy in prototyping phase, directly mutating the state is not the way to make use of a state manager, you miss out the power that comes with creating mutations.

If you wanna create a two way binding with a mutation there are two ways you could work:

::: tip codesandbox
[advanced two way binding](https://codesandbox.io/s/manual-advanced-2-way-binding-wkcny)
:::

## create a generic getter and setter in your component

```vue{8,9}
// src/App.vue

<template>
  <div id="app">
    <img width="10%" src="./assets/logo.png">

        <ExampleComponentWithGetterAndSetter
            asr-bind-state="message"
            asr-bind-mutations="setMessage"
        />

  </div>
</template>
```

Now the component has a state binding with `message` and a mutation binding with `setMessage`

Now in the

```vue{11,18}
<template>
  <div class="example-component-with-getter-and-setter">
    <p class="message">{{ message }}</p>
    <input v-model="message" />
  </div>
</template>

<script>
export default {
  computed: {
    message: {
      get() {
        return this.message;
      },
      set(value) {
        this.setMessage(value);
      }
    }
  }
};
</script>
```

## work with an event-handler in your component

repository: vuex-asr/vuex-asr

gha_language: JavaScript

gha_license_id: MIT

repo_name: vuex-asr/vuex-asr


text:
---
layout: article
titles:
  # @start locale config
  en      : &EN       Suggests
  en-GB   : *EN
  en-US   : *EN
  en-CA   : *EN
  en-AU   : *EN
  zh-Hans : &ZH_HANS  书签
  zh      : *ZH_HANS
  zh-CN   : *ZH_HANS
  zh-SG   : *ZH_HANS
  zh-Hant : &ZH_HANT  書籤
  zh-TW   : *ZH_HANT
  zh-HK   : *ZH_HANT
  ko      : &KO       북마크
  ko-KR   : *KO
  fr      : &FR       signets
  fr-BE   : *FR
  fr-CA   : *FR
  fr-CH   : *FR
  fr-FR   : *FR
  fr-LU   : *FR
  # @end locale config
mathjax: true
mathjax_autoNumber: false
mermaid: true
key: page-suggests
show_title: false
---

# Suggested websites

## Drug and substance abuse

* 449 Recovery, <https://449recovery.org/>
* Addiction Resources in Northern Illinois, <https://www.northernillinoisrecovery.com/resources/>
- Alcohol Rehab Help, <https://alcoholrehabhelp.org/> ([Effects of Alcohol - Cancer](https://alcoholrehabhelp.org/addiction/effects/cancer/))
* Amethyst Recovery DeTox, <https://www.amethystrecovery.org/>
* AToN Center, <https://www.atoncenter.com/>
* Bayview Recovery Center, <https://www.bayviewrecovery.com/>
+ Blue Star Recovery, <https://bluestarrecovery.com/>
- Casco Bay Recovery, The Leading Addiction Treatment Center for Portland, Maine, <https://cascobayrecovery.com/>
* Crestview Recovery, <https://www.crestviewrecovery.com/>
* Discovery Institute, <https://www.discoverynj.org/>
* Drug and Alcohol Addiction Treatment Services, <https://addiction-treatment-services.com/>
* Eagle Creek Ranch Recovery, <https://idahorecoverycenter.com/>
- First City Recovery Center, <https://firstcityrecoverycenter.com/programs/>
- FreebytheSea Sunset View Drug & Alcohol Recovery Center, <https://freebythesea.com>
* IOP Addiction Treatment for Men, <https://www.recoverintegrity.com/>
* Jay Walker Lodge, <https://jaywalkerlodge.com/>
* Legends Recovery Center LLC, <https://legendsrecovery.com/>
* Live Free Recovery Services -- Drug Rehab Program,  <https://livefreerecoverynh.com/programs/>
- Magnolia City Detox, <https://magnoliacitydetox.com/programs/>
* MPower Wellness Exton, <https://mpowerwellness.com/>
* Orlando Treatment Solutions, <https://orlandotreatmentsolutions.com/>
+ Path To Recovery Detox, <https://pathtorecoverydetox.com/>
* Sana Lake Recovery Behavioral Wellness Center, <https://sanalake.com/service-areas/maryland-heights/>
* Smoke free UK, <https://smokefreeuk.co.uk/>
* The Freedom Center, <https://www.thefreedomcenter.com/>
* The Villa Kali Ma holistic addiction and mental health recovery centers, <https://villakalima.com/>
* Westwind Behavioral Health Treatment, <https://www.westwindrecovery.com/>

## Physical, mental and behavioral Health

* Adler Health, <https://adlerhealth.com/>
* Evolve Wellness, <https://www.evolvewellnessinc.com/>
* Innercept, <https://www.innercept.com/>
* Ironwood Maine, <https://www.ironwoodmaine.com/>
* Paradigm Treatment, <https://paradigmtreatment.com/>
* Shortridge Academcy, <https://www.shortridgeacademy.com>
+ The Ridge, <https://theridgertc.com/>
* Tusk Kratom, <https://tuskkratom.com/>
* West Valley Detox, <https://westvalleydetox.com/>

## Social care

* Shepherd Premier Senior Living, <https://shepherdpremierseniorliving.com/>

---

## Contributors

- Eliza Hadley<EMAIL_ADDRESS>- Dominique Legget<EMAIL_ADDRESS>- Rachel Parker<EMAIL_ADDRESS>- Emma Tressler<EMAIL_ADDRESS>
***Created on** 12/1/2023 and **last changed** 5/7/2023*


text:
#Use RackHD to Install CentOS On Your Server
This document is describing How to use RackHD service to install CentOS on your Server.
###1. Unpacking the CentOS ISO file
1. Download the ISO file to `~/RackHD/example`

		$ wget http://mirrors.ustc.edu.cn/centos/7.2.1511/isos/x86_64/CentOS-7-x86_64-DVD-1511.iso
2. Unpack the ISO file in Vagrant

		$ vagrant ssh
		$ sudo mkdir /var/mirrors
		$ sudo python ~/src/on-http/data/templates/setup_iso.py /vagrant/CentOS-7-x86_64-DVD-1511.iso /var/mirrors --link=/home/vagrant/src
	**Note:** The VM has mounted shared folders:/vagrant => ~/RackHD/example

###2. Install a default workflow for your Server and a SKUs definition
Here is the example for CLS server. And how it work, please refer to http://rackhd.readthedocs.org/en/latest/how_it_works.html

1. Write a default workflow for CLS Server install CentOS by using IPMI OBM

		$ vi ~/RackHD/example/samples/CLS_install_centos.json

		{
		    "friendlyName": "CLS Server Install CentOS",
		    "injectableName": "Graph.CLS.InstallCentOS",
		    "tasks": [
		        {
		            "label": "create-ipmi-obm-settings",
		            "taskDefinition": {
		                "friendlyName": "Create IPMI OBM settings",
		                "injectableName": "Task.Obm.Ipmi.CreateSettings",
		                "implementsTask": "Task.Base.Obm.Ipmi.CreateSettings",
		                "options": {
		                    "impichanel": "lan",
		                    "user": "<user>",
		                    "password": "<passwd>"
		                },
		                "properties": {}
		            }
		        },
		        {
		            "label": "Install CentOS",
		            "taskDefinition": {
		                "friendlyName": "Install CentOS",
		                "injectableName": "Task.Os.Install.CentOS",
		                "implementsTask": "Task.Base.Os.Install",
		                "options": {
		                    "osType": "linux",
		                    "profile": "install-centos.ipxe",
		                    "hostname": "localhost",
		                    "comport": "ttyS0",
		                    "domain": "rackhd.github.com",
		                    "completionUri": "renasar-ansible.pub",
		                    "version": "7.0",
		                    "repo": "{{api.server}}/Centos/{{options.version}}",
		                    "rootPassword": "root",
		                    "installDisk": "sda"
		                },
		                "properties": {
		                    "os": {
		                        "linux": {
		                            "distribution": "centos"
		                        }
		                    }
		                }
		            }
		        }
		    ]
		}

		$ curl -H "Content-Type: application/json" -X PUT --data @samples/CLS_install_centos.json http://localhost:9090/api/1.1/workflows
2. To enable that workflow, we also need to include a SKU definition that includes the option of another workflow to run once the SKU has been identified. This takes advantage of the `Graph.SKU.Discovery` workflow, which will attempt to identify a SKU and run another workflow if specified.

		$ vi  ~/RackHD/example/samples/CLS_sku.json

		{
		    "name": "IPMI OBM settings for CLS nodes",
		    "discoveryGraphName": "Graph.CLS.InstallCentOS",
		    "discoveryGraphOptions": {},
		    "rules": [
		        {
		            "path": "dmi.System Information.Manufacturer",
		            "equals": "CLS"
		        }
		    ]
		}

		$ curl -H "Content-Type: application/json" -X POST --data @samples/CLS_sku.json http://localhost:9090/api/1.1/skus
3. View the SKU definitions:

	$ curl http://localhost:9090/api/1.1/skus | python -m json.tool

		[
			{
		        "createdAt": "2016-03-02T02:27:18.623Z",
		        "discoveryGraphName": "Graph.CLS.InstallCentOS",
		        "discoveryGraphOptions": {},
		        "id": "56d64f862dda5258076fd9ff",
		        "name": "IPMI OBM settings for CLS nodes",
		        "rules": [
		            {
		                "equals": "CLS",
		                "path": "dmi.System Information.Manufacturer"
		            }
		        ],
		        "updatedAt": "2016-03-02T02:27:18.623Z"
		    }
		]

###3.Startup your Machine through PXE

**Note:** In the process of the installation, You may probably meet the issue that you can not get the "centos.ks" from HTTP server. https://github.com/RackHD/RackHD/issues/68. Or maybe not. That would be decided by your code version.

And the log file should be in /tmp/* . If there don't have it, please check this PR https://github.com/RackHD/RackHD/pull/66/files to make sure you have these code in your server.


text:
---
title: "Specification"
date: 2017-11-29T15:42:32-05:00
description: ""
categories: []
keywords: []
slug: ""
aliases: []
toc: false
draft: false
reviewed: true
---

Light*4j microservices frameworks encourage Design Driven API building. For the REST
style of APIs we support the Swagger 2.0 specification and OpenAPI 3.0 specification. In
this tutorial, we are going to use the Swagger 2.0 specification as the central piece to
drive the runtime for security and validation. Also, the specification can be used to
scaffold a running server project the first time so that developers can focus their
efforts on the domain business logic implementation without worrying about how each
component are wired together.

During the service implementation phase, specification might be changed. You can
regenerate the service codebase again without overwriting your handlers and test
cases for handlers. The regeneration is also useful if you want to upgrade to the
latest version of the frameworks for your project.

To create Swagger 2.0 specification, the best tool is [swagger-editor][] and I have an
[article][] in tool section to describe how to use it.

By following the instructions on how to use the editor, let's create four API specifications
in model-config repository.

API A will call API B, API B will call API C, API C will call API D

```
API A -> API B -> API C -> API D
```

Here is the API A swagger.yaml and others can be found at [model-config][] or
model-config/rest/swagger folder in your workspace.

```
swagger: '2.0'

info:
  version: "1.0.0"
  title: API A for microservices demo
  description: API A is called by consumer directly and it will call API B and API C to get data
  contact:
    email<EMAIL_ADDRESS>  license:
    name: "Apache 2.0"
    url: "http://www.apache.org/licenses/LICENSE-2.0.html"
host: a.networknt.com
schemes:
  - http
basePath: /v1

consumes:
  - application/json
produces:
  - application/json

paths:
  /data:
    get:
      description: Return an array of strings collected from down stream APIs
      operationId: listData
      responses:
        200:
          description: Successful response
          schema:
            title: ArrayOfStrings
            type: array
            items:
              type: string
          examples: {
            "application/json": ["Message 1","Message 2"]
          }
      security:
        - a_auth:
          - api_a.w
          - api_a.r

securityDefinitions:
  a_auth:
    type: oauth2
    authorizationUrl: http://localhost:8080/oauth2/code
    flow: implicit
    scopes:
      api_a.w: write access
      api_a.r: read access
```

As defined in the specification, API A will return a list of strings and it requires
scope api_a.r or scope api_a.w to access the endpoint /data.

The next step, we will build the generator and generate the projects based on the specs.

[Swagger 2.0 specification]: https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md
[OpenAPI 3.0 specification]: https://swagger.io/specification/
[swagger-editor]: http://swagger.io/swagger-editor/
[article]: /tool/swagger-editor/
[model-config]: https://github.com/networknt/model-config/tree/master/rest/swagger

repository: networknt/light-doc

gha_language: CSS

repo_name: networknt/light-doc


text:
# cli-doc

A cli tool developed using GoLang to instantly look up documentation for libraries.

It might only save you two clicks but if you're looking at documentation 100 times a day then you're saving 200 clicks.

## How to use

Currently, it only supports few python libraries but the aim is to add all the common libraries as well as eventually add support for other languages.

Use `doc` to get basic information.
`doc help [command]` to get information. For example, `doc help numpy`.

We have also added `doc google [search query]`.

## How to install

Installation is very easy if you have go installed. Start by cloning the repository `git clone https://github.com/ParthS28/cli-doc`, open a terminal and navigate to the repository and type `go install`. As far as I know, this works perfectly on linux. We still need to check on other OS.

Become a contributor by checking the installation.

## Contributing
The code underlying for this is very simple. This is a beginner friendly community. Do not feel overwhelmed and contribute.

## TODO
- [x] ability to use it without go installation
- [ ] adding support for other OS
- [ ] sklearn
- [ ] opencv
- [ ] sympy


text:
websafe-michelf-markdown-parser-psr
================================================================================

`websafe-michelf-markdown-parser-psr` is a [PSR-0] compliant version of
[Michel Fortin]'s **[PHP Markdown]** library ([branch: lib]), automatically
generated by a buildscript, enhanced with a [PHPUnit] test suite, [Composer]
ready, with fixed PHP code and Travis CI integration and more.

Current build status.

    Branch
    Build status
    Based on commit

    master

        a10d48919913c2b3efdc3637e5494e4e57c04d5e

    develop

Table of contents
--------------------------------------------------------------------------------

 1. [Features](#features)
 2. [Quickstart](#quickstart)
 5. [Class and namespace hierarchy diagram](#class-and-namespace-hierarchy-diagram)
 6. [Milestones](#milestones)

Features
--------------------------------------------------------------------------------

 + [PSR-0] compliant - the migration process from [Michel Fortin]'s
   [PHP Markdown] [branch: extra] to a PSR-0 compliant library is done by
   a buildscript (not a part of this repo) in an fully automated way.
 + [PHPUnit] test suite for Markdown parsers. Easy creation of test cases
   without PHPUnit knowledge.
 + [Composer] ready/compliant package - `composer.json` shipped.
 + Fixed PHP code with [Fabien Potencier]'s [PHP-CS-Fixer].
 + [PHP_CodeSniffer] reports for [Michel Fortin] and the
   [PHP Markdown community].
 + [Continuous integration] with [Travis CI]. See project's [Build History].
 + Very handy `build.xml` for [Ant].

Quickstart for developers and Markdown hackers ;)
--------------------------------------------------------------------------------

Clone this repos master branch:

~~~~
git clone https://github.com/websafe/test-a.git
cd test-a
~~~~

Now create a `composer.json` file:

~~~~
ant setup-dev-env
~~~~

Now You should have all required tools and dependecies installed in `vendor/`,
accessible via `vendor/bin/*`, or better [Ant].

~~~~
ant phpdoc
~~~~

PHP Documentation is created in `doc/phpdoc/`.

Now run some tests:

~~~~
ant phpunit
~~~~

You think any of the dependencies may have changed? Update Your dev environment

~~~~
ant update-dev-env
~~~~

this will update [Composer] with all dependencies and recreate missing dirs.

Class and namespace hierarchy diagram
--------------------------------------------------------------------------------

This diagram was generated using [phpDocumentor2]. You can browse the full
PHP documentation for this library https://docs.websafe.pl/websafe-michelf-markdown-parser-psr/phpdoc/

[![Class hierarchy diagram](https://docs.websafe.pl/websafe-michelf-markdown-parser-psr/phpdoc/classes.svg)](https://docs.websafe.pl/websafe-michelf-markdown-parser-psr/phpdoc/ "PHP documentation")

Milestones
--------------------------------------------------------------------------------

### 0.0.0

 + [PSR-0] autoloader interoperability.
 + [Composer] ready.
 + [phpDocumentor2].
 + Handy [Ant] `build.xml`.
 + Easy extendable [PHPUnit] test suite.
 + [Continuous integration] with [Travis CI].

### 0.0.1

 + xxx

### 0.1.0

 + PSR-1

### 0.2.0

 + PSR-2

--------------------------------------------------------------------------------
[Michel Fortin]: http://michelf.ca/
[PHP Markdown]: https://github.com/michelf/php-markdown/
[branch: lib]: https://github.com/michelf/php-markdown/tree/lib
[PSR-0]: https://github.com/php-fig/fig-standards/blob/master/accepted/PSR-0.md
[PSR-1]: https://github.com/php-fig/fig-standards/blob/master/accepted/PSR-1-basic-coding-standard.md
[PSR-2]: https://github.com/php-fig/fig-standards/blob/master/accepted/PSR-2-coding-style-guide.md
[Continuous integration]: http://en.wikipedia.org/wiki/Continuous_integration
[Travis CI]: https://travis-ci.org/
[Build History]: https://travis-ci.org/websafe/test-a/builds
[PHP Markdown community]: https://github.com/michelf/php-markdown/issues?state=open
[Fabien Potencier]: http://fabien.potencier.org/
[PHP-CS-Fixer]: https://github.com/fabpot/PHP-CS-Fixer
[Sebastian Bergmann]: http://sebastian-bergmann.de/
[PHPUnit]: https://github.com/sebastianbergmann/phpunit
[Composer]: http://getcomposer.org/
[PHP_CodeSniffer]: https://github.com/squizlabs/PHP_CodeSniffer
[phpDocumentor2]: http://www.phpdoc.org/

repository: websafe/test-a

repo_name: websafe/test-a


text:
## fe-starter
A skeleton front-end application

### Instructions
- Install [nodejs](https://nodejs.org)
- Install dependencies: `npm install`
- Install gulp: `npm install -g gulp`
- Run dev server: `gulp dev`

### Tests
You're recommended to test your React components. There are a couple examples in the `test` folder.
To run tests: `gulp test`. Alternatively, you can run `npm test`.

### ES6
You can develop modules and write tests using [ES6](https://github.com/lukehoban/es6features) syntax. In the background, [Babel](https://babeljs.io) is used to transpile to web-friendly ES5.

### Styles
[Bootstrap](http://getbootstrap.com) is included by default but it's very easy to remove it. Simply go to `public/index.html` and remove the Bootstrap references.

In general, you can add new stylesheets (CSS or SCSS) in the `app/styles` folder. Each component can require these stylesheets individually. See example in `app/components/auth/login.js`. At build time, all required styles will be extracted and put in `dist/main.css`.

repository: ndnhat/fe-starter

repo_name: ndnhat/fe-starter


text:
---
layout: work_page
title: Missile Command
data: archived_work
permalink: /archived_work/missile
---

Missile Command is a 1980 classic arcade game that involves protecting buildings by shooting incoming missiles. I recreated Missile Command by using components in a [Mbed kit](https://www.sparkfun.com/products/14458) and programming the logic using its corresponding C++ compiler.

The electrical components consisted of the following:

- 1 Mbed microcontroller to program game logic
- LCD screen to display the game in color
- 4 push buttons to move and shoot missiles
- 1 speaker to play music
- 1 SD card to store game data and files

The games features included:

- Multiple levels of increasing difficulty
- Multi-colored explosions
- Cool sound effects
- High score rankings
- Level skipping

gha_language: SCSS


text:
# brudi

When it comes to backup-creation there are several solutions to use.
In general everybody's doing some sort of `dump` or `tar` and backing up the results incremental with [`restic`](https://github.com/restic/restic) or similar programs.

This is why `brudi` was born. `brudi` supports several backup-methods and is configurable by a simple `yaml` file.
The advantage of `brudi` is, that you can create a backup of a source of your choice and save it with `restic` afterwards in one step.
Under the hood, `brudi` uses the given binaries like `mysqldump`, `mongodump`, `pg_dump`, `tar` or `restic`.

Using `brudi` will save you from finding yourself writing bash-scripts to create your backups.

Besides creating backups, `brudi` can also be used to restore your data from backup in an emergency.

## Table of contents

 - [Usage](#usage)
   - [CLI](#cli)
   - [Docker](#docker)
   - [Configuration](#configuration)
      - [Sources](#sources)
         - [Tar](#tar)
         - [MySQLDump](#mysqldump)
         - [MongoDump](#mongodump)
         - [PgDump](#pgdump)
            - [Limitations](#limitations)
         - [Redis](#redis)
      - [Restic](#restic)
         - [Forget](#forget)
      - [Sensitive data: Environment variables](#sensitive-data-environment-variables)
      - [Gzip support for binaries without native gzip support](#gzip-support-for-binaries-without-native-gzip-support)
      - [Restoring from backup](#restoring-from-backup)
         - [TarRestore](#tarrestore)
         - [MongoRestore](#mongorestore)
         - [MySQLRestore](#mysqlrestore)
         - [PgRestore](#pgrestore)
           - [Restore using pg_restore](#restore-using-pg_restore)
           - [Restore using psql](#restore-using-psql)
         - [Restoring using restic](#restoring-using-restic)
 - [Featurestate](#featurestate)
     - [Source backup methods](#source-backup-methods)
     - [Restore backup methods](#restore-backup-methods)
     - [Incremental backup of the source backups](#incremental-backup-of-the-source-backups)

## Usage

### CLI

In order to use the `brudi`-binary on your local machine or a remote server of your choice, ensure you have the required tools installed.

- `mongodump` (required when running `brudi mongodump`)
- `mysqldump` (required when running `brudi mysqldump`)
- `tar` (required when running `brudi tar`)
- `redis-cli` (required when running `brudi redisdump`)
- `restic` (required when running `brudi --restic`)

```shell
$ brudi --help

Easy, incremental and encrypted backup creation for different backends (file, mongoDB, mysql, etc.)
After creating your desired tar- or dump-file, brudi backs up the result with restic - if you want to

Usage:
  brudi [command]

Available Commands:
  help           Help about any command
  mongodump      Creates a mongodump of your desired server
  mongorestore   Restores a server from a mongodump
  mysqldump      Creates a mysqldump of your desired server
  mysqlrestore   Restores a database from an sqldump
  pgdump         Creates a pg_dump of your desired postgresql-server
  pgrestore      Restores a database from a pgdump using pg_restore
  psql           Restores a database from a plain-text pgdump using psql
  redisdump      Creates an rdb dump of your desired server
  tar            Creates a tar archive of your desired
  tarrestore     Restores files from a tar archive
  version        Print the version number of brudi

Flags:
      --cleanup         cleanup backup files afterwards
  -c, --config string   config file (default is ${HOME}/.brudi.yaml)
  -h, --help            help for brudi
      --restic          backup result with 'restic backup'
      --restic-forget   executes 'restic forget' after backing up things with restic
      --version         version for brudi

Use "brudi [command] --help" for more information about a command.
```

### Docker

In case you don't want to install additional tools, you can also use `brudi` inside docker:

`docker run --rm -v ${HOME}/.brudi.yml:/home/brudi/.brudi.yml quay.io/mittwald/brudi mongodump --restic --cleanup`

The docker-image comes with all required binaries.

### Configuration

As already mentioned, `brudi` is configured via `.yaml`. The default path for this file is `${HOME}/.brudi.yaml`, but it's adjustable via `-c` or `--config`.
The config file itself can include environment-variables via `go-template`:

```yaml
restic:
  global:
    flags:
      repo: "{{ .Env.RESTIC_REPOSITORY }}"
```

Since the configuration provided by the `.yaml`-file is mapped to the corresponding CLI-flags, you can adjust literally every parameter of your source backup.
Therefore you can simply refer to the official documentation for explanations on the available flags:

- [`restic`](https://restic.readthedocs.io/en/latest/manual_rest.html)
- [`tar`](https://www.gnu.org/software/tar/manual/html_section/tar_22.html)
- [`mongodump`](https://docs.mongodb.com/manual/reference/program/mongodump/#options)
- [`mysqldump`](https://dev.mysql.com/doc/refman/8.0/en/mysqldump.html#mysqldump-option-summary)
- [`pg_dump`](https://www.postgresql.org/docs/12/app-pgdump.html)

Every source has a an `additionalArgs`-key which's value is an array of strings. The value of this key is appended to the command, generated by `brudi`.
Even though `brudi` should support all cli-flags to be configured via the `.yaml`-file, there may be flags which are not.
In this case, use the `additionalArgs`-key.

It is also possible to provide more than one configuration file, for example `-c mongodump.yaml -c restic.yaml`. These configs get merged at runtime.
If available, the default config will always be laoded first and then overwritten with any values from user-specified files.
In case the same config file has been provided more than once, only the first instance will be taken into account.

#### Sources

##### Tar

```yaml
tar:
  options:
    flags:
      create: true
      gzip: true
      file: /tmp/test.tar.gz
    additionalArgs: []
    paths:
      - /tmp/testfile
  hostName: autoGeneratedIfEmpty
```

Running: `brudi tar -c ${HOME}/.brudi.yml --cleanup`

Becomes the following command:
`tar -c -z -f /tmp/test.tar.gz /tmp/testfile`

All available flags to be set in the `.yaml`-configuration can be found [here](pkg/source/tar/cli.go#L7).

##### MySQLDump

```yaml
mysqldump:
  options:
    flags:
      host: <IP_ADDRESS>
      port: 3306
      password: mysqlroot
      user: root
      opt: true
      allDatabases: true
      resultFile: /tmp/test.sqldump
    additionalArgs: []
```

Running: `brudi mysqldump -c ${HOME}/.brudi.yml --cleanup`

Becomes the following command:
`mysqldump --all-databases --host=<IP_ADDRESS> --opt --password=mysqlroot --port=3306 --result-file=/tmp/test.sqldump --user=root`

All available flags to be set in the `.yaml`-configuration can be found [here](pkg/source/mysqldump/cli.go#L7).

##### MongoDump

```yaml
mongodump:
  options:
    flags:
      host: <IP_ADDRESS>
      port: 27017
      username: root
      password: mongodbroot
      gzip: true
      archive: /tmp/dump.tar.gz
    additionalArgs: []
```

Running: `brudi mongodump -c ${HOME}/.brudi.yml --cleanup`

Becomes the following command:
`mongodump --host=<IP_ADDRESS> --port=27017 --username=root --password=mongodbroot --gzip --archive=/tmp/dump.tar.gz`

All available flags to be set in the `.yaml`-configuration can be found [here](pkg/source/mongodump/cli.go#L7).

##### PgDump

```yaml
pgdump:
  options:
    flags:
      host: <IP_ADDRESS>
      port: 5432
      password: postgresroot
      username: postgresuser
      dbName: postgres
      file: /tmp/postgres.dump
    additionalArgs: []
```

Running: `brudi pgdump -c ${HOME}/.brudi.yml --cleanup`

Becomes the following command:
`pg_dump --file=/tmp/postgres.dump --dbname=postgres --host=<IP_ADDRESS> --port=5432 --username=postgresuser`

All available flags to be set in the `.yaml`-configuration can be found [here](pkg/source/pgdump/cli.go#L7).

###### Limitations

Unfortunately `PostgreSQL` is very strict when it comes to version-compatibility.
Therefore your `pg_dump`-binary requires the exact same version your `PostgreSQL`-server is running.

The Docker-image of `brudi` always has the latest version available for the corresponding alpine-version installed.

##### Redis

```yaml
redisdump:
  options:
    flags:
      host: <IP_ADDRESS>
      password: redisdb
      rdb: /tmp/redisdump.rdb
    additionalArgs: []
```

Running: `brudi redisdump -c ${HOME}/.brudi.yml`

Becomes the following command:
`redis-cli -h <IP_ADDRESS> -a redisdb --rdb /tmp/redisdump.rdb bgsave`

As `redis-cli` is not a dedicated backup tool but a client for `redis`, only a limited number of flags are available by default,
as you can see [here](pkg/source/redisdump/cli.go#L7).

#### Restic

In case you're running your backup with the `--restic`-flag, you need to provide a [valid configuration for restic](https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html).
You can either configure `restic` via `brudi`s `.yaml`-configuration, or via the [environment variables](https://restic.readthedocs.io/en/latest/040_backup.html#environment-variables) used by `restic`.

If you're already using `restic` in your environment, you should have everything set up perfectly to use `brudi` with `--restic`.

##### Forget

It's also possible to run `restic forget`-cmd after executing `restic backup` with `brudi` by using `--restic-forget`.
The `forget`-policy is defined in the configuration `.yaml` for brudi.

Example `.yaml`-configuration:

```yaml
restic:
    global:
      flags:
        # you can provide the repository also via RESTIC_REPOSITORY
        repo: "s3:s3.eu-central-1.amazonaws.com/your.s3.bucket/myResticRepo"
    backup:
      flags:
        # in case there is no hostname given, the hostname from source backup is used
        hostname: "MyHost"
      # these paths are backuped additionally to your given source backup
      paths: []
  forget:
    flags:
      keepLast: 48
      keepHourly: 24
      keepDaily: 7
      keepWeekly: 2
      keepMonthly: 6
      keepYearly: 2
    ids: []
```

#### Sensitive data: Environment variables

In case you don't want to provide data directly in the `.yaml`-file, e.g. sensitive data like passwords, you can use environment-variables.
Each key of the configuration is overwritable via environment-variables. Your variable must specify the whole path to a key, seperated by `_`.
For example, given this `.yaml`:

```yaml
mongodump:
  options:
    flags:
      username: "" # we will override this by env
      password: "" # we will override this by env
      host: <IP_ADDRESS>
      port: 27017
      gzip: true
      archive: /tmp/dump.tar.gz
```

Set your env's:

```shell
export MONGODUMP_OPTIONS_FLAGS_USERNAME="root"
export MONGODUMP_OPTIONS_FLAGS_PASSWORD="mongodbroot"
```

As soon as a variable for a key exists in your environment, the value of this environment-variable is used in favour of your `.yaml`-config.

#### Gzip support for binaries without native gzip support

The tools `mysqldump`, `pg_dump` and `redis-cli` don't natively support `gzip`. However, if the desired path for the backup file is suffixed with `.gz`,
brudi will automatically `gzip` the backup after creation and delete the uncompressed backup file. For restoration, the file will be automatically
uncompressed. Example for mysql:

```yaml
mysqldump:
  options:
    flags:
      host: <IP_ADDRESS>
      port: 3306
      password: mysqlroot
      user: root
      opt: true
      allDatabases: true
      resultFile: /tmp/test.sqldump.gz
    additionalArgs: []
```

```yaml
mysqlrestore:
  options:
    flags:
      host: <IP_ADDRESS>
      port: 3306
      password: mysqlroot
      user: root
      Database: test
    additionalArgs: []
    sourceFile: /tmp/test.sqldump.gz
```

#### Restoring from backup

##### TarRestore

```yaml
tarrestore:
  options:
    flags:
      extract: true
      gzip: true
      file: /tmp/test.tar.gz
      target: "/"
    additionalArgs: []
  hostName: autoGeneratedIfEmpty
```

Running: `brudi tarrestore -c ${HOME}/.brudi.yml`

Becomes the following command:
`tar -x -z -f /tmp/test.tar.gz -C /`

##### MongoRestore

 ```yaml
 mongorestore:
   options:
     flags:
       host: <IP_ADDRESS>
       port: 27017
       username: root
       password: mongodbroot
       gzip: true
       archive: /tmp/dump.tar.gz
     additionalArgs: []
 ```

 Running: `brudi mongorestore -c ${HOME}/.brudi.yml `

 Becomes the following command:
 `mongorestore --host=<IP_ADDRESS> --port=27017 --username=root --password=mongodbroot --gzip --archive=/tmp/dump.tar.gz`

 All available flags to be set in the `.yaml`-configuration can be found [here](pkg/source/mongorestore/cli.go#L7).

##### MySQLRestore

```yaml
mysqlrestore:
  options:
    flags:
      host: <IP_ADDRESS>
      port: 3306
      password: mysqlroot
      user: root
      Database: test
    additionalArgs: []
    sourceFile: /tmp/test.sqldump
```

Running: `brudi mysqlrestore -c ${HOME}/.brudi.yml`

Becomes the following command:
`mysql --database=test --host=<IP_ADDRESS> --password=mysqlroot --port=3306 --user=root < /tmp/test.sqldump`

All available flags to be set in the `.yaml`-configuration can be found [here](pkg/source/mysqlrestore/cli.go#L7).

##### PgRestore

Restoration for PostgreSQL databases is split into two commands, `psql` and `pgrestore`. Which one to use depends on the format of the dump created with `pg_dump`:

`psql` can be used to restore plain-text dumps, which is the default format.

`pgrestore` can be used if the `format` option of `pg_dump` was set to `tar`, `directory` or `custom`.

###### Restore using pg_restore

```yaml
pgrestore:
  options:
    flags:
      host: <IP_ADDRESS>
      port: 5432
      username: postgresuser
      password: postgresroot
      dbname: postgres
    additionalArgs: []
    sourcefile: /tmp/postgres.dump
```

Running: `brudi pgrestore -c ${HOME}/.brudi.yml`

Becomes the following command:
`pg_restore  --host=<IP_ADDRESS>  --port=5432 --username=postgresuser --db-name=postgres /tmp/postgress.dump`

This command has to be used if the `format` option was set to `tar`, `directory` or `custom` in `pg_dump`.

All available flags to be set in the `.yaml`-configuration can be found [here](pkg/source/pgrestore/cli.go#L7).

###### Restore using psql

```yaml
psql:
  options:
    flags:
      host: <IP_ADDRESS>
      port: 5432
      user: postgresuser
      password: postgresroot
      dbname: postgres
    additionalArgs: []
    sourcefile: /tmp/postgres.dump
```

Running: `brudi pgrestore -c ${HOME}/.brudi.yml`

Becomes the following command:
`psql  --host=<IP_ADDRESS>  --port=5432 --user=postgresuser --db-name=postgres < /tmp/postgress.dump`

This command has to be used if the `format` option was set to `plain` in `pg_dump`, which is the default.

All available flags to be set in the `.yaml`-configuration can be found [here](pkg/source/psql/cli.go#L7).

##### Restoring using restic

Backups can be pulled from a `restic` repository and applied to your server by using the `--restic` flag in your brudi command.
Example configuration for `mongorestore`:
```yaml
mongorestore:
  options:
    flags:
      host: <IP_ADDRESS>
      port: 27017
      username: root
      password: mongodbroot
      gzip: true
      archive: /tmp/dump.tar.gz
    additionalArgs: []
restic:
  global:
    flags:
      repo: "s3:s3.eu-central-1.amazonaws.com/your.s3.bucket/myResticRepo"
  restore:
    flags:
      target: "/"
    id: "latest"
```

This will pull the latest snapshot of `/tmp/dump.tar.gz` from the repository, which `mongorestore` then uses to restore the server.
It is also possible to specify concrete snapshot-ids instead of `latest`.

## Featurestate

### Source backup methods

- [x] `mysqldump`
- [x] `mongodump`
- [x] `tar`
- [x] `pg_dump`
- [x] `redisdump`

### Restore backup methods

- [x] `mysqlrestore`
- [x] `mongorestore`
- [x] `tarrestore`
- [x] `pgrestore`
- [ ]  `redisrestore`

### Incremental backup of the source backups

- [x] `restic`
  - [x] `commands`
    - [x] `restic backup`
    - [x] `restic forget`
    - [x] `restic restore`
  - [x] `storage`
    - [x] `s3`

repository: mittwald/brudi

gha_language: Go

repo_name: mittwald/brudi


text:
# Next JS documentation website themes template

free download source code gratis

[test drive demo →](https://beposresto.vercel.app/)

[read documentation →](https://www.hockeycomputindo.com/2021/05/create-website-using-next-js-free.html)

For build modern website documentation using nestra next js framework

------------

with light and dark mode

![free download next js website template themes gratis](https://1.bp.blogspot.com/-942LuVNEA6k/YLTP9MEZfsI/AAAAAAAAOU8/zEB6vLqXU-UzR3lHocOcSPiNMlVuJNklwCLcBGAsYHQ/s1348/free%2Bwebapp%2Bthemes%2Btemplate%2Bnext%2Bjs%2B%25281%2529.png)

Home page with light mode

![free download next js website template themes gratis](https://1.bp.blogspot.com/-pjDg1mS-cJo/YLTP9KRKtSI/AAAAAAAAOVA/BKsVEYR_-BoiE1e8UPkzDTBDIssYHzEjQCLcBGAsYHQ/s1348/free%2Bwebapp%2Bthemes%2Btemplate%2Bnext%2Bjs%2B%25282%2529.png)

Home page article on dark mode

![free download next js website template themes gratis](https://1.bp.blogspot.com/-Q_mhc4ziLXE/YLTP9nmwV3I/AAAAAAAAOVE/4hNlVNCmABk6dy-yzUxIBhyCBpKjnxP9wCLcBGAsYHQ/s1349/free%2Bwebapp%2Bthemes%2Btemplate%2Bnext%2Bjs%2B%25284%2529.png)

documentation article non image light mode

![free download next js website template themes gratis](https://1.bp.blogspot.com/-j3xA0R5ABLI/YLTP8qM2j_I/AAAAAAAAOU4/az0k8hQGzDY_rrZdG7b99VZjNwdLNypwgCLcBGAsYHQ/s1349/free%2Bwebapp%2Bthemes%2Btemplate%2Bnext%2Bjs%2B%25283%2529.png)

documentation non image dark mode

![free download next js website template themes gratis](https://1.bp.blogspot.com/-XEl0OtC8HGc/YLTP-cGdeuI/AAAAAAAAOVM/caMTZ5biTH8iuc-LHH3iwR_sTMCVa07iwCLcBGAsYHQ/s1349/free%2Bwebapp%2Bthemes%2Btemplate%2Bnext%2Bjs%2B%25286%2529.png)

documentation article with image light mode

![free download next js website template themes gratis](https://1.bp.blogspot.com/-_QhJjrcIOSs/YLTP-ChPFLI/AAAAAAAAOVI/4DdBWpLHXtc7rYVue8pIO3h4ELvcuzazACLcBGAsYHQ/s1349/free%2Bwebapp%2Bthemes%2Btemplate%2Bnext%2Bjs%2B%25285%2529.png)

documentation article with image on dark mode

------------

How to install ??

+ Need node js
+ Download Source code
+ Extract file download on project folder

Now run command using node
+ npm install
+ npm start

open web browser localhost:3000

---------------

Setting website and change article doc, just [read documentation →](https://www.hockeycomputindo.com/2021/05/create-website-using-next-js-free.html)

repository: mesinkasir/nextdocsthemes

branch_name: refs/heads/main

repo_name: mesinkasir/nextdocsthemes


text:
go-sudoku-grid-generator
========================

A simple sudoku generator written in Go

go-sudoku-grid-generator is a simple port based on the Java-based
[SodokuGridGenerator](https://github.com/valcol/SodokuGridGenerator)
by [valcol](https://github.com/valcol).
which appears to be similar but a simpler version of the C-based
[sudoku-generator](https://github.com/xkaycee/sudoku-generator)
by [xkaycee](https://github.com/xkaycee)

Bugs/Limitations
----------------

Solutions don't appear to be unique and the puzzle diffculty logic was
just copied from the project I ported from. All in all, this was just
a bit of fun and not intende to a production ready Sudoku generator.
I'll probably make updates as I learn more about Sudoku or
create an entirely new project.

License:
--------

The MIT License (MIT)

Copyright (c) 2014-2015 Doug Sparling

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

 IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

repository: dsparling/go-sudoku-grid-generator

repo_name: dsparling/go-sudoku-grid-generator


text:
# Responsi

Responsi makes **responsive and retina-ready** image management easy, lightweight and asynchronous.

## Why Responsi (or The Features)

- **Dead simple**: it's easy to use and the implementation is brutally simple
- **Imgs and bgs**: one solution for both `<img>` and css `background-image`
- **Art directing**: you can choose different files for different viewports
- **Forgiving**: Responsi will try to come up with something even if you skip some declarations
- **Lightweight**: barely 1k minified, serve it gzipped and it's like one bit and a half
- **Self contained**: no dependency whatsoever
- **Cross browser**: works everywhere, with a simple fallback even when js is disabled
- **Asynchronous**: you can load images after the `DOMContentLoaded` event, or even create responsi elements dynamically

## Assembly Instructions (or Responsi How-To)

Did I say dead simple? Yep, dead simple indeed. You can specify three images for small, medium and large screens/dpi using the `data-responsi-<size>` attribute. This is your responsi element:

```html
<span class="responsi-img" data-responsi-large="large.jpg" data-responsi-medium="medium.jpg" data-responsi-small="small.jpg" data-responsi-alt="This is the alt test"></span>
```

You can use most elements as responsi elements, but you might want to use `span`s for images and `div`s for background elements for their respective `inline` and `block` nature.

For `<img>`s you can also specify an `alt` attribute using `data-responsi-alt`.

And here you fire the function:

```javascript
document.addEventListener('DOMContentLoaded', function() {
	responsi.init();
}, false);
```

### Configuration

You can pass a configuration object as a parameter to the `responsi.init()` function. You can figure out what you need to pass to achieve your goal by looking into the source code, it's pretty well commented. More on the default choice below. Here is the default configuration:

```javascript
defaultConfig = {
	phones	: 480,
	tabs	: 768,
	laps	: 1200,
	desks	: 1900
};
```

And as an icing on the cake, you have a bunch of helper properties that might come in handy:

```javascript
responsi.viewport // contains the viewport width, shorthand for document.documentElement.clientWidth
responsi.dpi // contains the dpi, shorthand for window.devicePixelRatio (assumes 1 if devicePixelRatio is not supported)
responsi.phones // contains the breakpoint for phones
responsi.tabs // contains the breakpoint for tabs
responsi.laps // contains the breakpoint for laps
responsi.desks // contains the breakpoint for desks
```

### Fire responsi after the initial `init()`

Adding responsi elements dynamically? Great, then just call `responsi.fire()` to have them working. This allows you to fire the function without the need to specify configuration parameters again.

### Forgive Me, Responsi!

Responsi tries to be rather forgiving in the declaration. If the `data-responsi-<size>` is not available, responsi will look for other sizes starting from the largest.

The `data-responsi-alt` always tries to come up with something. If you don't set it, responsi will take the document `<title>` tag. Please, note that even though this is good for SEO, it might be pretty bad for accessibility, so use it sparingly and always declare an alt attribute when you can. If you want to set an empty alt attribute (see [decorative images](http://en.wikipedia.org/wiki/Alt_attribute#Decorative_images "Decorative images and alt attributes on Wikipedia")), set `data-responsi-alt` to an empty string.

### Fallback when JavaScript is Disabled

You can very easily specify a fallback for the crazy nerds that browse the interwebs with almighty JavaScript disabled. Here is your solution:

```html
<span class="responsi-img" data-responsi-large="large.jpg" data-responsi-medium="medium.jpg" data-responsi-small="small.jpg" data-responsi-alt="This is the alt test">
	<noscript><img src="large.jpg" alt="Some text" /></noscript>
</span>
```

Fallbacks for `background-image` can be set via css.

## Default Config (or How did you decide the breakpoints?)

A lot of thought has been put into coming up with the default config and after all this thought, I thought that no such amount of thought is really needed. Simply put, responsi is not a silver bullet and it certainly isn't a precise solution. It's more of a catch-all solution. Easy as that.

Responsi doesn't try to address every single possible combination of screen width and pixel density, because that's simply not possible, nor is desirable. Not only on the implementation side, but also on the image management size. I don't want to maintain [two-thousand-seven-hundred-fifty-two](http://www.oldielyrics.com/lyrics/rodriguez/heikkis_suburbia_bus_tour.html "2752 unscheduled stops") images in order to display only one to the user, nor do you.

The solution I adopted is a three-sizes-fit-all. Based on [this article](http://www.netvlies.nl/blog/design-interactie/retina-revolution "Retina revolution") and a fair amount of testing, I assumed that highly compressed @2x images are smaller and perform better to the eye than mildly compressed full-size images. I am aware of the overhead this causes to the GPU when displaying this stuff, but then again, this is not a perfect world and we need to work with shitloads of constrains. For example, we don't even have a native way to display @2x images to @2x screens, we still have to double the size of the image and resize it manually.

So, we have small, medium and large. That's it. You then decide the size you need in your specific cases, which may vary greatly. Some images will be full-screen, other will be half of some wrap-container and so on. Again, refer to the source code for more information on the various breakpoints.

### So, no `<picture>` and `srcset`?

I tried, gee, I tried. The spec is simply crazy. `<picture>` and `srcset` will certainly cover a lot more possibilities than responsi, but if I need the support of a NASA engineer to display an humble image to my user, something must be wrong.

Support is still highly insufficient, and the polyfill is immensely heavier than my humble responsi. Responsi addresses 90% of use cases with ease. Those who need more control, please go ahead and use other solutions.

Finally, `<picture>` will certainly become my first choice when browser support get widespread and most importantly when browsers decide to support more advanced features such as image choice based on network quality and user preferences. This isn't yet implemented anywhere and it shouldn't even come too soon, as far as I know.

## Responsi on Bower!

Responsi can also be installed via [Bower](http://http://bower.io "A package manager for the web"). Search for `responsi.js`.

**Happy Responsive Coding!**

## Changelog

- **0.2.0**: This is the first important update. It adds the possibility to fire responsi after the initial init by calling `responsi.fire()` and allows to use the various breakpoints set for responsi, so that you can do something like `if ( responsi.viewport > responsi.phones )`. So cool.
- **0.1.1**: Minor update to the Bower manifest, the readme and add an informational header to the top of the file.
- **0.1.0**: Initial release.

## MIT License

Copyright (c) 2015 Emanuele Feliziani

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

 IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

repository: GioSensation/responsi

repo_name: GioSensation/responsi


text:
---
layout: page
title: ""
meta-title: ""
bigimg:
  - "/img/P718006624.jpg" : "Alexandra Park (Toronto), photo by Nevena Novakovic (2017)"
---

Welcome to HPS/PL 120, Caltech's intro philosophy of science course for Spring 2019.

Meetings: Th 7-9:55pm.

Location: 117 Dabney.

Office Hours: Tuesdays 1:30-2:30pm.

First paper [topics](essay1.md).

Second paper [topics](essay2.md).

## Schedule

**Thur April 4: Introduction**

Readings: None

Presenters: None

**Thur April 11: No Class**

**Thur April 18: Writing Workshop**

**Thur April 25: Falsificationism**

Readings: [Popper, Logic of Scientific Discovery (Chs 1, 2, and 4)](popper1.pdf)

Presenters: Abhisnek Lingineni (Chs 1-2), Aadyot Bhatnagar (Ch 4)

**Thur May 2:**

Readings: [Strevens, Notes on BCT](http://www.nyu.edu/classes/strevens/BCT/BCT.pdf) chs. 5, 6, 7, 9; and [Gelman and Shalizi 2013](http://www.stat.columbia.edu/~gelman/research/published/philosophy.pdf)

Presenters: Nick Redd (Strevens), Felipe Gomez (Gelman)

**Thur May 9:**

Readings: [Belot, Bayesian Orgulity](https://sites.google.com/site/gordonbelot/home/papers-etc/bayesian-orgulity) (click on "preprint" to access the pdf), and [Pomatto and Sandroni](http://www.its.caltech.edu/~lpomatto/an_axiomatic_theory.pdf)

Presenters: Tanner Moore (Belot) , Sean Yu (Pomatto and Sandroni)

Notes: you will find some other responses to Belot's claim, when you click on the link to his paper, above.

**Thur May 16:**

Readings: [Lindley 1976](lindley.pdf), [Goodman 1999](http://www.perfendo.org/docs/bayesprobability/5.3_goodmanannintmed99all.pdf) (focus on part 1, part 2 is optional), [Bohannon 2015](https://io9.gizmodo.com/i-fooled-millions-into-thinking-chocolate-helps-weight-1707251800) (last item is a pop article).

Presenters: Mohith Manohara. Notes for presentation: Lindley and part 1 of Goodman is central. The rest only if helpful.

**Thur May 23:**

Readings: [Ioannidis (2005)](https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124); and [Gelman and Loken (2013)](http://www.stat.columbia.edu/~gelman/research/unpublished/p_hacking.pdf); and [Simmons (2011)](fpp.pdf).

Optional: [Nuzzo (2014)](https://www.nature.com/news/scientific-method-statistical-errors-1.14700), and [XKCD cartoon](https://xkcd.com/882/)

Presenters: Aaron Ayres (Ioannidis), Alexa Lauinger (Gelman and Loken), Zixiao Li (Simmons)

**Thur May 30:**

Readings: [Huttegger (2015)](https://faculty.sites.uci.edu/shuttegg/files/2011/03/PPRFinal.pdf), and [Steele and Stefansson (draft)](steele.pdf)

Presenters: Eitan Borgia (Huttegger), Enrico Borba (S&S)

**Thur June 6:**

Readings: [Sober 2002](https://827166e3-a-bf4ca052-s-sites.googlegroups.com/a/wisc.edu/sober/selected-papers/PS%202002%20InstrumentalismParsimonyandAkaike.pdf?attachauth=ANoY7cq2qPbIseMzWC9j5-yo8cZK0RQ9w4wApKZy5I3cn66_TwJHifr1b_tP-gJP1LFLL0gvM-ZyRjohSgQ-eP4cLsceRoj4jdknb3YWYrgZ8S3ilrWDcxED14r5Xp_maSEnwh6bQsQa2J9BTFiDqgXBSfV7zOkZFLOUhnFVkxWpl9EmM35xqu6GjhZ8BmjR5opSSRP_0IptZchENPm8KYL9DBJBl_mByRuK42bD_l9z7GolDv4NeLVhxJ2Hcxjhe-dUisThmGHc&attredirects=0), [Sober 2016](https://aeon.co/essays/are-scientific-theories-really-better-when-they-are-simpler), [Breitenbach 2015](http://angelabreitenbach.weebly.com/uploads/5/7/5/7/57579773/breitenbach-beautyproofs.pdf), [Hajek 2009](https://link.springer.com/article/10.1007%2Fs10670-009-9154-1)

Presenters: Melissa (Breitenbach), Michaelangelo (2 Sober articles), Zach (Hajek)