text:
[![Build Status](https://travis-ci.org/torchnet/torchnet.svg)](https://travis-ci.org/torchnet/torchnet)

# torchnet

*torchnet* is a framework for [torch](http://torch.ch) which provides a set
of abstractions aiming at encouraging code re-use as well as encouraging
modular programming.

At the moment, *torchnet* provides four set of important classes:
  - Dataset: handling and pre-processing data in various ways.
  - Engine: training/testing machine learning algorithm.
  - Meter: meter performance or any other quantity.
  - Log: output performance or any other string to file / disk in a consistent manner.

For an overview of the *torchnet* framework, please also refer to [this paper](https://lvdmaaten.github.io/publications/papers/Torchnet_2016.pdf).

## Installation

Please install *torch* first, following instructions on
[torch.ch](http://torch.ch/docs/getting-started.html).  If *torch* is
already installed, make sure you have an up-to-date version of
[*argcheck*](https://github.com/torch/argcheck), otherwise you will get
weird errors at runtime.

Assuming *torch* is already installed, the *torchnet* core is only a set of
lua files, so it is straightforward to install it with *luarocks*
```
luarocks install torchnet
```

To run the MNIST example from the paper, install the `mnist` package:
```
luarocks install mnist
```

`cd` into the installed `torchnet` package directory and run:
```
th example/mnist.lua
```

## Documentation

Requiring *torchnet* returns a local variable containing all *torchnet*
class constructors.
```
local tnt = require 'torchnet'
```

### tnt.Dataset()

*torchnet* provides a variety of data containers, which can be easily
plugged between each others, allowing the user to easily concat, split,
batch, resample etc... datasets.

A instance `dataset` of a `tnt.Dataset()` implements two main methods:

  * `dataset:size()` which returns the size of the dataset.
  * `dataset:get(idx)` where `idx` is a number between 1 and the dataset size.

While it is easy to iterate over a dataset with a for loop, several
`DatasetIterator` iterators are nevertheless provided, allowing the user to
filter out some samples in an on-the-fly manner, or to parallelize easily
data fetching.

In *torchnet*, a sample returned by `dataset:get()` is supposed to be a Lua
`table`. Fields of the table can be arbitrary, even though many datasets
will only work with torch tensors.

### tnt.utils

*Torchnet* provides a set of util functions which are used all over torchnet.

#### tnt.utils.table.clone(table)

This function do a deep copy of a table.

#### tnt.utils.table.merge(dst, src)
```
({
   dst = table  --
   src = table  --
})
```

This function add to the destination table `dest`, the
element contained in the source table `source`.

The copy is shallow.

If a key exists in both tables, then the element in the source table
is preferred.

#### tnt.utils.table.foreach(tbl, closure[, recursive])
```
({
   tbl       = table     --
   closure   = function  --
  [recursive = boolean]  --  [default=false]
})
```

This function applies the function defined by `closure` to the
table `tbl`.

If `recursive` is given and set to `true`, the `closure` function
will be apply recursively to the table.

#### tnt.utils.table.canmergetensor(tbl)

Check if a table can be merged into a tensor.

#### tnt.utils.table.mergetensor(tbl)
```
({
   tbl = table  --
})
```

Merge a table into a tensor in one extra dimension.

### tnt.transform

*Torchnet* provides a set of general data transformations.
These transformations are either directly on the data (e.g., normalization)
or on their structure. This is particularly handy
when manipulating [tnt.Dataset](#tnt.Dataset).

Most of the transformations are simple but can be [composed](#transform.compose) or
[merged](#transform.merge).

#### transform.identity(...)

The identity transform takes any input and return it as it is.

For example, this function is useful when composing
transformations on data from multiple sources, and some of the sources
must not be transformed.

#### transform.compose(transforms)
```
({
   transforms = table  --
})
```

This function takes a `table` of functions and
composes them to return one transformation.

This function assumes that the table of transformations
is indexed by contiguous ordered keys starting at 1.
The transformations are composed in the ascending order.

For example, the following code:
```lua
> f = transform.compose{
        [1] = function(x) return 2*x end,
        [2] = function(x) return x + 10 end,
        foo = function(x) return x / 2 end,
        [4] = function(x) return x - x end
   }
   > f(3)
   16
```
is equivalent to compose the transformations stored in [1] and [2], i.e.,
defining the following transformation:
```lua
> f =  function(x) return 2*x + 10 end ```
Note that transformations stored with keys `foo` and `4` are ignored.
```
<a name = "transform.merge">
#### transform.merge(transforms)
```
({
   transforms = table  --
})
```

This function takes a `table` of transformations and
merge them into one transformation.
Once apply to an input, this transformation will produce a `table` of output,
containing the transformed input.

For example, the following code:
```lua
> f = transform.merge{
        [1] = function(x) return 2*x end,
        [2] = function(x) return x + 10 end,
        foo = function(x) return x / 2 end,
        [4] = function(x) return x - x end
   }
```
produces a function which applies a set of transformations to the same input:
```lua
> f(3)
   {
     1 : 6
     2 : 13
     foo : 1.5
     4 : 0
   }
```
<a name = "transform.tablenew">
#### transform.tablenew()

This function creates a new table of functions from an
existing table of functions.
<a name = "transform.tableapply">
#### transform.tableapply(transform)
```
({
   transform = function  --
})
```

This function applies a transformation to a table of input.
It return a table of output of the same size as the input.

For example, the following code:
```lua
> f = transform.tableapply(function(x) return 2*x end)
```
produces a function which multiplies any input by 2:
```lua
> f({[1] = 1, [2] = 2, foo = 3, [4] = 4})
   {
     1 : 2
     2 : 4
     foo : 6
     4 : 8
   }
```
<a name = "transform.tablemergekeys">
#### transform.tablemergekeys()

This function merges tables by key. More precisely, the input must be a
`table` of `table` and this function will reverse the table orderto
make the keys from the nested table accessible first.

For example, if the input is:
```lua
> x = { sample1 = {input = 1, target = "a"} , sample2 = {input = 2, target = "b", flag = "hard"}
```
Then apply this function will produce:
```lua
> transform.tablemergekeys(x)
{
   input :
         {
           sample1 : 1
           sample2 : 2
         }
   target :
          {
            sample1 : "a"
            sample2 : "b"
          }
   flag :
        {
           sample2: "hard"
        }
}
```
<a name = "transform.makebatch">
#### transform.makebatch([merge])
```
({
  [merge = function]  --
})
```

This function is used in many `tnt.Dataset` to format
samples in the format used by the `tnt.Engine`.

This function first [merges keys](#transform.tablemergekeys) to
produces a table of output. Then, transform this table into a tensor by
either using a `merge` transformation provided by the user or by
simply concatenating the table into a tensor directly.

This function uses the [compose](#transform.compose) transform to apply
successive transformations.
<a name = "transform.perm">
#### transform.perm(size)
```
({
   size = number  --
})
```

This function create a vector containing a permutation of the indices from 1 to `size`.
This vector is a `LongTensor` and  `size` must be a number.

Once the vector created, this function can be used to call a specific indices in it.

For example:
```lua
> p = transform.perm(3)
```
creates a function `p` which contains a permutation of indices:
```lua
> p(1)
2
> p(2)
1
> p(3)
3
```
<a name = "transform.normalize">
#### transform.normalize([threshold])
```
({
  [threshold = number]  --  [default=0]
})
```

This function normalizes data, i.e., it removes its mean and
divide it by its standard deviation.

The input must be a `Tensor`.

Once create, a `threshold` can be given (must be a number). Then,
the data will be divided by their standard deviation, only if this
deviation is greater than the `threshold`. This is handy, if the
deviation is small and deviding by it could lead to unstability.
<a name="ListDataset">
#### tnt.ListDataset(self, list, load[, path])
```
({
   self = tnt.ListDataset  --
   list = tds.Hash         --
   load = function         --
  [path = string]          --
})
```

Considering a `list` (can be a `tds.Hash`, `table` or a `torch.LongTensor`) the
i-th sample of a dataset will be returned by `load(list[i])`, where `load()` is
a closure provided by the user.

If `path` is provided, list is assumed to be a list of string, and will
each element `list[i]` will prefixed by `path/` when fed to `load()`.

Purpose: many low or medium-scale datasets can be seen as a list of files
(for example representing input samples). For this list of file, a target
can be often inferred in a simple manner.

#### tnt.ListDataset(self, filename, load[, maxload][, path])
```
({
   self     = tnt.ListDataset  --
   filename = string           --
   load     = function         --
  [maxload  = number]          --
  [path     = string]          --
})
```

The file specified by `filename` is interpreted as a list of strings (one
string per line). The i-th sample of a dataset will be returned by
`load(line[i])`, where `load()` is a closure provided by the user an
`line[i]` is the i-the line of `filename`.

If `path` is provided, list is assumed to be a list of string, and will
each element `list[i]` will prefixed by `path/` when fed to `load()`.

<a name = "TableDataset">
#### tnt.TableDataset(self, data)
```
({
   self = tnt.TableDataset  --
   data = table             --
})
```

`tnt.TableDataset` interfaces existing data
to torchnet. It is useful if you want to use torchnet on a small dataset.

The data must be contained in a `tds.Hash`.

`tnt.TableDataset` does a shallow copy of the data.

Data are loaded while constructing the `tnt.TableDataset`:
```lua
> a = tnt.TableDataset({1,2,3})
> print(a:size())
3
```
`tnt.TableDataset` assumes that table has contiguous keys starting at 1.
<a name="IndexedDataset">
#### tnt.IndexedDataset(self, fields[, path][, maxload][, mmap][, mmapidx])
```
{
   self    = tnt.IndexedDataset  --
   fields  = table               --
  [path    = string]             --
  [maxload = number]             --
  [mmap    = boolean]            --  [default=false]
  [mmapidx = boolean]            --  [default=false]
}
```

A `tnt.IndexedDataset()` is a data structure built upon (possibly several)
data archives containing a bunch of tensors of the same type.

See [tnt.IndexedDatasetWriter](#IndexedDatasetWriter) and
[tnt.IndexedDatasetReader](#IndexedDatasetReader) to see how to create and
read a single archive.

Purpose: large datasets (containing a lot of files) are often not very well
handled by filesystems (especially over network). `tnt.IndexedDataset`
provides a convenient and efficient way to bundle them into a single
archive file, associated with an indexed file.

If `path` is provided, then `fields` must be a Lua array (keys being
numbers), where values are string representing a filename prefix to a
(index,archive) pair.  In other word `path/field.{idx,bin}` must exist. The
i-th sample returned by this dataset will be a table containing each field
as key, and a tensor found at the corresponding archive at index i.

If `path` is not provided, then `fields` must be a Lua hash. Each key
represents sample fields and the corresponding value must be a table
containing the keys `idx` (for the index filename path) and `bin` (for the
archive filename path).

If provided (and positive), `maxload` limits the dataset size to the
specified size.

Archives and/or indexes can also be memory mapped with the `mmap` and
`mmapidx` flags.

<a name="IndexedDatasetWriter">
##### tnt.IndexedDatasetWriter(self, indexfilename, datafilename, type)
```
({
   self          = tnt.IndexedDatasetWriter  --
   indexfilename = string                    --
   datafilename  = string                    --
   type          = string                    --
})
```

Creates a (archive,index) file pair. The archive will contain tensors of the same specified `type`.

`type` must be a string chosen in {`byte`, `char`, `short`, `int`, `long`, `float`, `double` or `table`}.

`indexfilename` is the full path to the index file to be created.
`datafilename` is the full path to the data archive file to be created.

Tensors are added to the archive with [add()](#IndexedDataset.add).

Note that you *must* call [close()](#IndexedDataset.close) to ensure all
data is written on disk and to create the index file.

The type `table` is special: data will be stored into a CharTensor,
serialized from a Lua table object. IndexedDatasetReader will then
deserialize the CharTensor into a table at read time. This allows storing
heterogenous data easily into an IndexedDataset.

##### tnt.IndexedDatasetWriter(self, indexfilename, datafilename)
```
({
   self          = tnt.IndexedDatasetWriter  --
   indexfilename = string                    --
   datafilename  = string                    --
})
```

Opens an existing (archive,index) file pair for appending. The tensor type is inferred from the provided
index file.

`indexfilename` is the full path to the index file to be opened.
`datafilename` is the full path to the data archive file to be opened.

<a name="IndexedDatasetWriter.add">
###### tnt.IndexedDatasetWriter.add(self, tensor)
```
({
   self   = tnt.IndexedDatasetWriter  --
   tensor = torch.*Tensor             --
})
```

Add a tensor to the archive and record its index position. The tensor type must of the same type
than the one specified at the creation of the `tnt.IndexedDatasetWriter`.
###### tnt.IndexedDatasetWriter.add(self, filename)
```
({
   self     = tnt.IndexedDatasetWriter  --
   filename = string                    --
})
```

Convenience method which given a `filename` will open the corresponding
file in `binary` mode, and reads all data in there as if it was of the type
specified at the `tnt.IndexedDatasetWriter` construction.  A corresponding
tensor is then added to the archive/index pair.
###### tnt.IndexedDatasetWriter.add(self, table)
```
(
   self  = tnt.IndexedDatasetWriter  --
   table = table                     --
)
```

Convenience method only available for `table` type IndexedDataset.
The table will be serialized into a CharTensor.

###### tnt.IndexedDatasetWriter.add(self)
```
({
   self = tnt.IndexedDatasetWriter  --
})
```

Finalize the index, and Close the archive/index filename pair. This method
must be called to ensure the index is written and all the archive data is
flushed on disk.
<a name="IndexedDatasetReader">
##### tnt.IndexedDatasetReader(self, indexfilename, datafilename[, mmap][, mmapidx])
```
({
   self          = tnt.IndexedDatasetReader  --
   indexfilename = string                    --
   datafilename  = string                    --
  [mmap          = boolean]                  --  [default=false]
  [mmapidx       = boolean]                  --  [default=false]
})
```

Reads an archive/index pair previously created by
[tnt.IndexedDatasetWriter](#IndexedDatasetWriter).

`indexfilename` is the full path to the index file.
`datafilename` is the full path to the archive file.

Memory mapping can be specified for both the archive and index through the
optional `mmap` and `mmapidx` flags.

<a name="IndexedDatasetReader.size">
###### tnt.IndexedDatasetReader.size(self)

Returns the number of tensors present in the archive.
<a name="IndexedDatasetReader.get">
###### tnt.IndexedDatasetReader.get(self, index)

Returns the tensor at the specified `index` in the archive.
<a name="TransformDataset">
#### tnt.TransformDataset(self, dataset, transform[, key])
```
({
   self      = tnt.TransformDataset  --
   dataset   = tnt.Dataset           --
   transform = function              --
  [key       = string]               --
})
```

Given a closure `transform()`, and a `dataset`, `tnt.TransformDataset`
applies the closure in an on-the-fly manner when querying a sample with
`tnt.Dataset:get()`.

If key is provided, the closure is applied to the sample field specified
by `key` (only). The closure must return the new corresponding field value.

If key is not provided, the closure is applied on the full sample. The
closure must return the new sample table.

The size of the new dataset is equal to the size of the underlying `dataset`.

Purpose: when performing pre-processing operations, it is convenient to be
able to perform on-the-fly transformations to a
dataset.
<a name="TransformDataset">
#### tnt.TransformDataset(self, dataset, transforms)
```
({
   self       = tnt.TransformDataset  --
   dataset    = tnt.Dataset           --
   transforms = table                 --
})
```

Given a set of closures and a `dataset`, `tnt.TransformDataset` applies
these closures in an on-the-fly manner when querying a sample with
`tnt.Dataset:get()`.

Closures are provided in `transforms`, a Lua table, where a (key,value)
pair represents a (sample field name, corresponding closure to be applied
to the field name).

Each closure must return the new value of the corresponding field.
<a name="BatchDataset">
#### tnt.BatchDataset(self, dataset, batchsize[, perm][, merge][, policy])
```
({
   self      = tnt.BatchDataset  --
   dataset   = tnt.Dataset       --
   batchsize = number            --
  [perm      = function]         --  [has default value]
  [merge     = function]         --
  [policy    = string]           --  [default=include-last]
})
```

Given a `dataset`, `tnt.BatchDataset` merges samples from this dataset to
form a new sample which can be interpreted as a batch (of size
`batchsize`).

The `merge` function controls how the batch is performed. It is a closure
taking a Lua array as input containing all occurrences (for a given batch)
of a field of the sample, and returning the aggregated version of these
occurrences. By default the occurrences are supposed to be tensors, and
they aggregated along the first dimension.

More formally, if the i-th sample of the underlying dataset is written as:
```lua
{input=<input_i>, target=<target_i>}
```
assuming only two fields `input` and `target` in the sample, then `merge()`
will be passed tables of the form:
```lua
{<input_i_1>, <input_i_2>, ... <input_i_n>}
```
or
```lua
{<target_i_1>, <target_i_2>, ... <target_i_n>}
```
with `n` being the batch size.

It is often important to shuffle examples while performing the batch
operation. `perm(idx, size)` is a closure which returns the shuffled index
of the sample at position `idx` in the underlying dataset. For convenience,
the `size` of the underlying dataset is also passed to the closure. By
default, the closure is the identity.

The underlying dataset size might or might not be always divisible by
`batchsize`.  The optional `policy` string specify how to handle corner
cases:
  - `include-last` makes sure all samples of the underlying dataset will be seen, batches will be of size equal or inferior to `batchsize`.
  - `skip-last` will skip last examples of the underlying dataset if its size is not properly divisible. Batches will be always of size equal to `batchsize`.
  - `divisible-only` will raise an error if the underlying dataset has not a size divisible by `batchsize`.

Purpose: the concept of batch is problem dependent. In *torchnet*, it is up
to the user to interpret a sample as a batch or not. When one wants to
assemble samples from an existing dataset into a batch, then
`tnt.BatchDataset` is suited for the job. Sometimes it is however more
convenient to write a dataset from scratch providing "batched" samples.
<a name="CoroutineBatchDataset">
#### tnt.CoroutineBatchDataset(self, dataset, batchsize[, perm][, merge][, policy])
```
({
   self      = tnt.CoroutineBatchDataset  --
   dataset   = tnt.Dataset                --
   batchsize = number                     --
  [perm      = function]                  --  [has default value]
  [merge     = function]                  --
  [policy    = string]                    --  [default=include-last]
})
```

Given a `dataset`, `tnt.CoroutineBatchDataset` merges samples from this dataset
to form a new sample which can be interpreted as a batch (of size `batchsize`).

It behaves the same and has the same arguments as `tnt.BatchDataset` (see the
documentation there for additional details), with one important distinction:
it allows the underlying dataset to postpone returning the individual samples
once by doing a call to `coroutine.yield()` (from the underlying dataset).

This is useful when using datasets that are inefficient or slow when they need
to provide the required sample immediately after a call to `dataset:get()`. The
general pattern of code in the underlying `dataset:get()` would be:

```lua
FooDataset.get = function(self, idx)
   prepare(idx)  -- stores sample in self.__data[idx]
   coroutine.yield()
   return self.__data[idx]
end
```

Herein, the function `prepare(idx)` can implement, for instance, a buffering of
indices before actually fetching them.
<a name="ConcatDataset">
#### tnt.ConcatDataset(self, datasets)
```
{
   self     = tnt.ConcatDataset  --
   datasets = table              --
}
```

Given a Lua array (`datasets`) of [tnt.Dataset](#Dataset), concatenates
them into a single dataset.  The size of the new dataset is the sum of the
underlying dataset sizes.

Purpose: useful to assemble different existing datasets, possibly
large-scale datasets as the concatenation operation is done in an
on-the-fly manner.
<a name="ResampleDataset">
#### tnt.ResampleDataset(self, dataset[, sampler][, size])

Given a `dataset`, creates a new dataset which will (re-)sample from this
underlying dataset using the provided `sampler(dataset, idx)` closure.

If `size` is provided, then the newly created dataset will have the
specified `size`, which might be different than the underlying dataset
size.

If `size` is not provided, then the new dataset will have the same size
than the underlying one.

By default `sampler(dataset, idx)` is the identity. `dataset` corresponds
to the underlying dataset provided at construction, and `idx` may take a
value between 1 to `size`. It must return an index in the range acceptable
for the underlying dataset.

Purpose: shuffling data, re-weighting samples, getting a subset of the
data. Note that an important sub-class is ([tnt.ShuffleDataset](#ShuffleDataset)),
provided for convenience.
<a name="ShuffleDataset">
#### tnt.ShuffleDataset(self, dataset[, size][, replacement])
```
({
   self        = tnt.ShuffleDataset  --
   dataset     = tnt.Dataset         --
  [size        = number]             --
  [replacement = boolean]            --  [default=false]
})
```

`tnt.ShuffleDataset` is a sub-class of
[tnt.ResampleDataset](#ResampleDataset) provided for convenience.

It samples uniformly from the given `dataset` with, or without
`replacement`. The chosen partition can be redrawn by calling
[resample()](#ShuffleDataset.resample).

If `replacement` is `true`, then the specified `size` may be larger than
the underlying `dataset`.

If `size` is not provided, then the new dataset size will be equal to the
underlying `dataset` size.

Purpose: the easiest way to shuffle a dataset!
<a name="ShuffleDataset.resample">
##### tnt.ShuffleDataset.resample(self)

The permutation associated to `tnt.ShuffleDataset` is fixed, such that two
calls to the same index will return the same sample from the underlying
dataset.

Call `resample()` to draw randomly a new permutation.
<a name="SplitDataset">
#### tnt.SplitDataset(self, dataset, partitions)
```
({
   self       = tnt.SplitDataset  --
   dataset    = tnt.Dataset       --
   partitions = table             --
})
```

Partition a given `dataset`, according to the specified `partitions`.  Use
the method [select()](#SplitDataset.select) to select the current partition
in use.

The Lua hash table `partitions` is of the form (key, value) where key is a
user-chosen string naming the partition, and value is a number representing
the weight (in size) of the corresponding partition.

The sum of the partition weights may or may not sum to one
(`tnt.SplitDataset` will make them sum to one!).

Partionning is achieved linearly (no shuffling). See
[tnt.ShuffleDataset](#ShuffleDataset) if you want to shuffle the dataset
before partitioning.

Purpose: useful in machine learning to perform validation procedures.
<a name="SplitDataset.select">
##### tnt.SplitDataset.select(self, partition)
```
({
   self      = tnt.SplitDataset  --
   partition = string            --
})
```

Switch the current partition in use to the one specified by `partition`,
which must be a string corresponding to one of the names provided at
construction.

The current dataset size changes accordingly, as well as the samples returned
by the `get()` method.
#### Dataset Iterators

It is easy to iterate over datasets using a for loop. However, sometimes
one wants to filter out samples in a on-the-fly manner or thread sample fetching.

Iterators are here for this particular cases. In general, refrain writing
iterators for handling custom cases, and write instead a `tnt.Dataset`

Iterators implement two methods:

  * `run()` which returns a Lua iterator usable in a for loop.
  * `exec(funcname, ...)` which execute a given funcname on the underlying dataset.

Typical usage is achieved with a for loop:
```lua
for sample in iterator:run() do
  <do something with sample>
end
```

Iterators implement the __call event, so one might also use the `()` operator:
```lua
for sample in iterator() do
  <do something with sample>
end
```

<a name="DatasetIterator">
##### tnt.DatasetIterator(self, dataset[, perm][, filter][, transform])
```
({
   self      = tnt.DatasetIterator  --
   dataset   = tnt.Dataset          --
  [perm      = function]            --  [has default value]
  [filter    = function]            --  [has default value]
  [transform = function]            --  [has default value]
})
```

The default dataset iterator.

`perm(idx)` is a permutation used to shuffle the examples. If shuffling
is needed, one can use this closure, or (better) use
[tnt.ShuffleDataset](#ShuffleDataset) on the underlying dataset.

`filter(sample)` is a closure which returns `true` if the given sample
should be considered or `false` if not.

`transform(sample)` is a closure which can perform online transformation of
samples. It returns a modified version of the given `sample`. It is the
identity by default. It is often more interesting to use
[tnt.TransformDataset](#TransformDataset) for that purpose.
<a name="DatasetIterator.exec">
##### tnt.DatasetIterator.exec(tnt.DatasetIterator, name, ...)

Execute the given method `name` on the underlying dataset, passing it the
subsequent arguments, and returns what the `name` method returns.
<a name="ParallelDatasetIterator">
##### tnt.ParallelDatasetIterator(self[, init], closure, nthread[, perm][, filter][, transform][, ordered])
```
({
   self      = tnt.ParallelDatasetIterator  --
  [init      = function]                    --  [has default value]
   closure   = function                     --
   nthread   = number                       --
  [perm      = function]                    --  [has default value]
  [filter    = function]                    --  [has default value]
  [transform = function]                    --  [has default value]
  [ordered   = boolean]                     --  [default=false]
})
```

Allows to iterate over a dataset in a thread
manner. `tnt.ParallelDatasetIterator:run()` guarantees that all samples
will be seen, but does not guarantee the order unless `ordered` is set to true.

The purpose of this class is to have a zero pre-processing cost.
When reading datasets on the fly from
disk (not loading them fully in memory), or performing complex
pre-processing this can be of interest.

The number of threads used to parallelize is specified by `nthread`.

`init(threadid)` (where threadid=1..nthread) is a closure which may
initialize the specified thread as needed, if needed. It is doing nothing
by default.

`closure(threadid)` will be called on each thread and must return a
`tnt.Dataset` instance.

`perm(idx)` is a permutation used to shuffle the examples. If shuffling is
needed, one can use this closure, or (better) use
[tnt.ShuffleDataset](#ShuffleDataset) on the underlying dataset (returned by
`closure()`).

`filter(sample)` is a closure which returns `true` if the given sample
should be considered or `false` if not. Note that filter is called _after_
fetching the data in a threaded manner.

`transform(sample)` is a function which maps the given sample to a new value.
This transformation occurs before filtering.

When `ordered` is set to true the ordering of samples returned by the iterator
is guaranteed. This option is particularly useful for repeatable experiments.
By default `ordered` is false, which means that order is not guaranteed by
`run()` (though often the ordering is similar in practice).

A common error raised by this dataset is when `closure()` is not
serializable. Make sure that all [upvalues](http://www.lua.org/pil/27.3.3.html) of `closure()` are
serializable. It is recommended to avoid [upvalues](http://www.lua.org/pil/27.3.3.html) at all cost,
and to make sure you require all the appropriate torch packages needed to (de-)serialize
`closure()` in the `init()` function.

For more information, check out the [threads package](https://github.com/torch/threads),
on which `tnt.ParallelDatasetIterator` relies.
<a name="ParallelDatasetIterator.execSingle">
##### tnt.ParallelDatasetIterator.execSingle(tnt.DatasetIterator, name, ...)

Execute the given method `name` on the dataset corresponding to the first
available thread, passing it the subsequent arguments, and returns what the
`name` method returns.

For example:
```lua
  local iterator = tnt.ParallelDatasetIterator{...}
  print(iterator:execSingle("size"))
```
will print the size of the dataset loaded in the first available thread.
<a name="ParallelDatasetIterator.exec">
##### tnt.ParallelDatasetIterator.exec(tnt.DatasetIterator, name, ...)

Execute the given method `name` on the underlying datasets in each thread,
passing to each of them the subsequent arguments, and returns a table
of what the `name` method returns for each thread.

For example:
```lua
  local iterator = tnt.ParallelDatasetIterator{...}
  for _, v in pairs(iterator:exec("size")) do
      print(v)
  end
```
will print the size of the datasets loaded in each thread.

### tnt.Engine

In experimenting with different models and datasets, the underlying training
procedure is often the same. The Engine module provides the boilerplate logic
necessary for the training and testing of models. This might include conducting
the interaction between model (nn.Module), `tnt.DatasetIterator`s,
`nn.Criterion`s, and `tnt.Meter`s.

An instance `engine` of a `tnt.Engine()` implements two main methods:

  * `engine:train()`, for training the model on data
        (i.e. sample data, forward prop, backward prop).
  * `engine:test()`,  for evaluating a model on data
        (optionally with respect to a `nn.Criterion`).

The Engine can be implemented for any common underlying training and testing
procedure involving a model and data. It can also be designed to allow user
control after certain events such as forward prop, criterion evaluation, or the
end of an epoch, by using coroutines (see `tnt.SGDEngine`).

### tnt.SGDEngine

The SGDEngine module implements the Stochastic Gradient Descent training
procedure in `train`, including data sampling, forward prop, back prop, and
parameter updates. It also operates as a coroutine allowing a user control
 (i.e. increment some sort of `tnt.Meter`) at events such as 'start',
'start-epoch', 'forward', 'forward-criterion', 'backward', etc.

Accordingly, `train` requires a network (nn.Module), a criterion expressing the
loss function (nn.Criterion), a dataset iterator (tnt.DatasetIterator), and a
learning rate, at the minimum. The `test` function allows for simple evaluation
of a model on a dataset.

A `state` is maintained for external access to outputs and parameters of modules
as well as sampled data.

### tnt.OptimEngine

The OptimEngine module wraps the optimization functions from
https://github.com/torch/optim. At the start of training, the engine will call
`getParameters` on the provided network.

The `train` method requires the following parameters in addition to the
SGDEngine.train parameters:

  * `optimMethod` the optimization function (e.g optim.sgd)
  * `config` a table with configuration parameters for the optimizer

Example:
```lua
  local engine = tnt.OptimEngine()
  engine:train{
     network = model,
     criterion = criterion,
     iterator = iterator,
     optimMethod = optim.sgd,
     config = {
        learningRate = 0.1,
        momentum = 0.9,
     },
  }
```
#### Meters

When training a model, you generally would like to measure how the model is
performing. Specifically, you may want to measure the average processing time
required per batch of data, the classification error or AUC of a classifier a
validation set, or the precision@k of a retrieval model.

Meters provide a standardized way to measure a range of different measures,
which makes it easy to measure a wide range of properties of your models.

Nearly all meters (except `tnt.TimeMeter`) implement three methods:

   * `add()` which adds an observation to the meter.
   * `value()` which returns the value of the meter, taking into account all observations.
   * `reset()` which removes all previously added observations, resetting the meter.

The exact input arguments to the `add()` method vary depending on the meter.
Most meters define the method as `add(output, target)`, where `output` is the
output produced by the model and `target` is the ground-truth label of the data.

The `value()` method is parameterless for most meters, but for measures that
have a parameter (such as the k parameter in precision@k), they may take an
input argument.

An example of a typical usage of a meter is as follows:
```lua
local meter = tnt.<Measure>Meter()  -- initialize meter
for state, event in tnt.<Optimization>Engine:train{
   network   = network,
   criterion = criterion,
   iterator  = iterator,
} do
  if state == 'start-epoch' then
     meter:reset()  -- reset meter
  elseif state == 'forward-criterion' then
     meter:add(state.network.output, sample.target)  -- add value to meter
  elseif state == 'end-epoch' then
     print('value of meter:' .. meter:value())  -- get value of meter
  end
end
```
<a name="APMeter">
#### tnt.APMeter(self)
```
({
   self = tnt.APMeter  --
})
```

The `tnt.APMeter` measures the average precision per class.

The `tnt.APMeter` is designed to operate on `NxK` Tensors `output` and `target`,
where (1) the `output` contains model output scores for `N` examples and `K`
classes that ought to be higher when the model is more convinced that the
example should be positively labeled, and smaller when the model believes the
example should be negatively labeled (for instance, the output of a sigmoid
function); and (2) the `target` contains only values 0 (for negative examples)
and 1 (for positive examples).

The `tnt.APMeter` has no parameters to be set.
<a name="AverageValueMeter">
#### tnt.AverageValueMeter(self)
```
({
   self = tnt.AverageValueMeter  --
})
```

The `tnt.AverageValueMeter` measures the average value of any collection of
numbers that are `add`ed to it. It is useful, for instance, to measure the
average loss over a collection of examples.

The `add()` function expects as input a Lua number `value`, which is the value
that needs to be added to the list of values to average. It also takes as input
an optional parameter `n` that assigns a weight to `value` in the average, in
order to facilitate computing weighted averages (default = 1).

The `tnt.AverageValueMeter` has no parameters to be set at initialization time.
<a name="AUCMeter">
#### tnt.AUCMeter(self)
```
({
   self = tnt.AUCMeter  --
})
```

The `tnt.AUCMeter` measures the area under the receiver-operating characteristic
(ROC) curve for binary classification problems. The area under the curve (AUC)
can be interpreted as the probability that, given a randomly selected positive
example and a randomly selected negative example, the positive example is
assigned a higher score by the classification model than the negative example.

The `tnt.AUCMeter` is designed to operate on one-dimensional Tensors `output`
and `target`, where (1) the `output` contains model output scores that ought to
be higher when the model is more convinced that the example should be positively
labeled, and smaller when the model believes the example should be negatively
labeled (for instance, the output of a signoid function); and (2) the `target`
contains only values 0 (for negative examples) and 1 (for positive examples).

The `tnt.AUCMeter` has no parameters to be set.
<a name="ConfusionMeter">
#### tnt.ConfusionMeter(self, k[, normalized])
```
{
   self       = tnt.ConfusionMeter  --
   k          = number              --
  [normalized = boolean]            --  [default=false]
}
```

The `tnt.ConfusionMeter` constructs a confusion matrix for a multi-class
classification problems. It does not support multi-label, multi-class problems:
for such problems, please use `tnt.MultiLabelConfusionMeter`.

At initialization time, the `k` parameter that indicates the number
of classes in the classification problem under consideration must be specified.
Additionally, an optional parameter `normalized` (default = `false`) may be
specified that determines whether or not the confusion matrix is normalized
(that is, it contains percentages) or not (that is, it contains counts).

The `add(output, target)` method takes as input an NxK tensor `output` that
contains the output scores obtained from the model for N examples and K classes,
and a corresponding N-tensor or NxK-tensor `target` that provides the targets
for the N examples. When `target` is an N-tensor, the targets are assumed to be
integer values between 1 and K. When target is an NxK-tensor, the targets are
assumed to be provided as one-hot vectors (that is, vectors that contain only
zeros and a single one at the location of the target value to be encoded).

The `value()` method has no parameters and returns the confusion matrix in a
KxK tensor. In the confusion matrix, rows correspond to ground-truth targets and
columns correspond to predicted targets.
<a name="mAPMeter">
#### tnt.mAPMeter(self)
```
({
   self = tnt.mAPMeter  --
})
```

The `tnt.mAPMeter` measures the mean average precision over all classes.

The `tnt.mAPMeter` is designed to operate on `NxK` Tensors `output` and `target`
where (1) the `output` contains model output scores for `N` examples and `K`
classes that ought to be higher when the model is more convinced that the
example should be positively labeled, and smaller when the model believes the
example should be negatively labeled (for instance, the output of a sigmoid
function); and (2) the `target` contains only values 0 (for negative examples)
and 1 (for positive examples).

The `tnt.mAPMeter` has no parameters to be set.
<a name="MultiLabelConfusionMeter">
#### tnt.MultiLabelConfusionMeter(self, k[, normalized])
```
{
   self       = tnt.MultiLabelConfusionMeter  --
   k          = number                        --
  [normalized = boolean]                      --  [default=true]
}
```

The `tnt.MultiLabelConfusionMeter` constructs a confusion matrix for multi-
label, multi-class classification problems. In constructing the confusion
matrix, the number of positive predictions is assumed to be equal to the number
of positive labels in the ground-truth. Correct predictions (that is, labels in
the prediction set that are also in the ground-truth set) are added to the
diagonal of the confusion matrix. Incorrect predictions (that is, labels in the
prediction set that are not in the ground-truth set) are equally divided over
all non-predicted labels in the ground-truth set.

At initialization time, the `k` parameter that indicates the number
of classes in the classification problem under consideration must be specified.
Additionally, an optional parameter `normalized` (default = `false`) may be
specified that determines whether or not the confusion matrix is normalized
(that is, it contains percentages) or not (that is, it contains counts).

The `add(output, target)` method takes as input an NxK tensor `output` that
contains the output scores obtained from the model for N examples and K classes,
and a corresponding NxK-tensor `target` that provides the targets for the N
examples using one-hot vectors (that is, vectors that contain only zeros and a
single one at the location of the target value to be encoded).

The `value()` method has no parameters and returns the confusion matrix in a
KxK tensor. In the confusion matrix, rows correspond to ground-truth targets and
columns correspond to predicted targets.
<a name="ClassErrorMeter">
#### tnt.ClassErrorMeter(self[, topk][, accuracy])
```
{
   self     = tnt.ClassErrorMeter  --
  [topk     = table]               --  [has default value]
  [accuracy = boolean]             --  [default=false]
}
```

The `tnt.ClassErrorMeter` measures the classification error (in %) of
classification models (zero-one loss). The meter can also measure the error of
predicting the correct label among the top-k scoring labels (for instance, in
the Imagenet competition, one generally measures classification@5 errors).

At initialization time, it takes to optional parameters: (1) a table
`topk` that contains the values at which the classification@k errors should be
measures (default = {1}); and (2) a boolean `accuracy` that makes the meter
output accuracies instead of errors (accuracy = 1 - error).

The `add(output, target)` method takes as input an NxK-tensor `output` that
contains the output scores for each of the N examples and each of the K classes,
and an N-tensor `target` that contains the targets corresponding to each of the
N examples (targets are integers between 1 and K). If only one example is
`add`ed, `output` may also be a K-tensor and target a 1-tensor.

Please note that `topk` (if specified) may not contain values larger than K.

The `value()` returns a table with the classification@k errors for all values
at k that were specified in `topk` at initialization time. Alternatively,
`value(k)` returns the classification@k error as a number; only values of `k`
that were element of `topk` are allowed. If `accuracy` was set to `true` at
initialization time, the `value()` method returns accuracies instead of errors.
<a name="TimeMeter">
#### tnt.TimeMeter(self[, unit])
```
({
   self = tnt.TimeMeter  --
  [unit = boolean]       --  [default=false]
})
```

The `tnt.TimeMeter` is designed to measure the time between events and can be
used to measure, for instance, the average processing time per batch of data.
It is different from most other meters in terms of the methods it provides:

At initialization time, an optional boolean parameter `unit` may be provided
(default = `false`). When set to `true`, the value returned by the meter
will be divided by the number of times that the `incUnit()` method is called.
This allows the user to compute, for instance, the average processing time per
batch by simply calling the `incUnit()` method after processing a batch.

The `tnt.TimeMeter` provides the following methods:

   * `reset()` resets the timer, setting the timer and unit counter to zero.
   * `stop()` stops the timer.
   * `resume()` resumes the timer.
   * `incUnit()` increments the unit counter by one.
   * `value()` returns the time passed since the last `reset()`; divided by the counter value when `unit=true`.

<a name="PrecisionAtKMeter">
#### tnt.PrecisionAtKMeter(self[, topk][, dim][, online])
```
{
   self   = tnt.PrecisionAtKMeter  --
  [topk   = table]                 --  [has default value]
  [dim    = number]                --  [default=2]
  [online = boolean]               --  [default=false]
}
```

The `tnt.PrecisionAtKMeter` measures the precision@k of ranking methods at pre-
specified levels k. The precision@k is the percentage of the k front-ranked
items according to the model that is in the list of correct (positive) targets.

At initialization time, a table `topk` may be given as input that specifies the
levels k at which the precision@k will be measures (default = `{10}`). In
addition, a number `dim` may be provided that specifies over which dimension the
precision@k should be computed (default = 2), and a boolean `online` may be
specified that indicates whether we see all inputs along dimension `dim` at once
(default = `false`).

The `add(output, target)` method takes two inputs. In the default mode (`dim=2`
and `online=false`), the inputs mean:
   * A NxC tensor that for each of the N examples (queries) contains a score
     indicating to what extent each of the C classes (documents) is relevant to
     the query, according to the model.
   * A binary NxC `target` tensor that encodes which of the C classes
     (documents) are actually relevant to the the N-th input (query). For
     instance, a row of {0, 1, 0, 1} indicates that the example is associated
     with classes 2 and 4.

The result of setting `dim` to `1` is identical to transposing the tensors
`output` and `target` in the above. The result of setting `online=true` is that
the function assumes that it is not the number of queries `N` that is growing
with repeated calls to `add()`, but the number of candidate documents `C`. (Use
this mode in scenarios where `C` is large but `N` is small.)

The `value()` method returns a table that contains the precision@k (that is, the
percentage of targets predicted correctly) at the cutoff levels in `topk` that
were specified at initialization time. Alternatively, the precision@k at
a specific level k can be obtained by calling `value(k)`. Note that the level
`k` should be an element of the table `topk` specified at initialization time.

Please note that the maximum value in `topk` cannot be higher than the total
number of classes (documents).
<a name="RecallMeter">
#### tnt.RecallMeter(self[, threshold][, perclass])
```
{
   self      = tnt.RecallMeter  --
  [threshold = table]           --  [has default value]
  [perclass  = boolean]         --  [default=false]
}
```

The `tnt.RecallMeter` measures the recall of ranking methods at pre-
specified thresholds. The recall is the percentage of the correct (positive)
targets that is in the list of positively labeled items according to the model.

At initialization time, the `tnt.RecallMeter` provides two optional
parameters. The first parameter is a table `threshold` that contains all
thresholds at which the recall is measured (default = {0.5}). Thresholds
should be numbers between 0 and 1. The second parameter is a boolean `perclass`
that makes the meter measure the recall per class when set to `true`
(default = `false`). When `perclass` is set to `false`, the recall is simply
averaged over all examples.

The `add(output, target)` method takes two inputs:
   * A NxK tensor that for each of the N examples indicates the probability
     of the example belonging to each of the K classes, according to the model.
     The probabilities should sum to one over all classes; that is, the row sums
     of `output` should all be one.
   * A binary NxK `target` tensor that encodes which of the K classes
     are associated with the N-th input. For instance, a row of {0, 1, 0, 1}
     indicates that the example is associated with classes 2 and 4.

The `value()` method returns a table containing the recall of the model
predictions measured at the `threshold`s specified at initialization time. The
`value(t)` method returns the recall at a particular threshold `t`. Note that
this threshold `t` should be an element of the `threshold` table specified at
initialization time of the meter.
<a name="PrecisionMeter">
#### tnt.PrecisionMeter(self[, threshold][, perclass])
```
{
   self      = tnt.PrecisionMeter  --
  [threshold = table]              --  [has default value]
  [perclass  = boolean]            --  [default=false]
}
```

The `tnt.PrecisionMeter` measures the precision of ranking methods at pre-
specified thresholds. The precision is the percentage of the positively labeled
items according to the model that is in the list of correct (positive) targets.

At initialization time, the `tnt.PrecisionMeter` provides two optional
parameters. The first parameter is a table `threshold` that contains all
thresholds at which the precision is measured (default = {0.5}). Thresholds
should be numbers between 0 and 1. The second parameter is a boolean `perclass`
that makes the meter measure the precision per class when set to `true`
(default = `false`). When `perclass` is set to `false`, the precision is simply
averaged over all examples.

The `add(output, target)` method takes two inputs:
   * A NxK tensor that for each of the N examples indicates the probability
     of the example belonging to each of the K classes, according to the model.
     The probabilities should sum to one over all classes; that is, the row sums
     of `output` should all be one.
   * A binary NxK `target` tensor that encodes which of the K classes
     are associated with the N-th input. For instance, a row of {0, 1, 0, 1}
     indicates that the example is associated with classes 2 and 4.

The `value()` method returns a table containing the precision of the model
predictions measured at the `threshold`s specified at initialization time. The
`value(t)` method returns the precision at a particular threshold `t`. Note that
this threshold `t` should be an element of the `threshold` table specified at
initialization time of the meter.
<a name="NDCGMeter">
#### tnt.NDCGMeter(self[, K])
```
({
   self = tnt.NDCGMeter  --
  [K    = table]         --  [has default value]
})
```

The `tnt.NDCGMeter` measures the normalized discounted cumulative gain (NDCG) of
a ranking produced by a model at prespecified levels k, and averages the NDCG
over all examples.

The discounted cumulative gain at level k is defined as:

DCG_k = rel_1 + \sum{i = 2}^k (rel_i / log_2(i))

Herein, rel_i is the relevance of item i as specified by an external rater.
Defining ideal DCG (IDCG) as the best possible DCG for a given example, the NDCG
at level k is defined as:

NDCG_k = DCG_k / IDCG_k

At initialization time, the meter takes as input a table `K` that contains all
the levels k at which the NDCG is computed.

The `add(output, relevance)` method takes as input (1) a NxC tensor of model
`outputs`, which scores for all C possible outputs for a batch of N examples;
and (2) a NxC tensor `relevance` that contains the corresponding relevances for
these scores, as provided by an external rater. Relevances are generally
obtained from human raters.

The `value()` method returns a table that contains the NDCG values for all
levels K that were provided at initialization time. Alternatively, the NDCG at
a specific level k can be obtained by calling `value(k)`. Note that the level
`k` should be an element of the table `K` specified at initialization time.

Please note that the number of outputs and relevances C should always be at
least as high as the highest NDCG level k that the meter is computing.
### Log

Log classes act as tables indexed by string keys. Allowed keys must be
provided at construction. A special key `__status__` can be also set the
convenience method `log:status()` to record basic messages.

Viewers closures can be attached to a Log, and called at different events:
   * `onSet(log, key, value)`: when setting a key to the Log with `log:set{}`.
   * `onGet(log, key)`: when querying a key with `log:get()`.
   * `onFlush(log)`: when flushing out the stored data of the Log with `log:flush()`.
   * `onClose(log)`: when closing a Log with `log:close()`.

Typical viewer closures are `text` or `json`, which allow to write to disk
or to the console a subset of the keys stored by the Log, in a particular
format. The special viewer closure `status` is made to be called on `set()`
events, and will print out only status records.

A typical use case would be the following:
```lua

-- require the viewers we want
local logtext = require 'torchnet.log.view.text'
local logstatus = require 'torchnet.log.view.status'

local log = tnt.Log{
   keys = {"loss", "accuracy"},
   onFlush = {
      -- write out all keys in "log" file
      logtext{filename=string.format('%s/log', rundir), keys={"loss", "accuracy"}, format={"%10.5f", "%3.2f"}},
      -- write out loss in a standalone file
      logtext{filename=string.format('%s/loss', rundir), keys={"loss"}}
      -- print on screen too
      logtext{keys=logkeys, keys=keys={"loss", "accuracy"}},
   },
   onSet = {
      -- add status to log
      logstatus{filename=string.format('%s/log', rundir)},
      -- print status to screen
      logstatus{},
   }
}

-- set values
log:set{
  loss = 0.1,
  accuracy = 97
}

-- write some info
log:status("hello world")

-- flush out log
log:flush()
```
<a name="Log">
#### tnt.Log(self, keys[, onClose][, onFlush][, onGet][, onSet])
```
{
   self    = tnt.Log  --
   keys    = table    --
  [onClose = table]   --
  [onFlush = table]   --
  [onGet   = table]   --
  [onSet   = table]   --
}
```

Creates a new Log with allowed keys (strings) `keys`.  Specifiy event
closures with table of functions `onClose`, `onFlush`, `onGet` and `onSet`,
which will be called when `close()`, `flush()`, `get()`, and `set{}`
methods will be called, respectively.
<a name="Log.status">
#### tnt.Log:status(self[, message][, time])
```
({
   self    = tnt.Log   --
  [message = string]   --
  [time    = boolean]  --  [default=true]
})
```

Record a status message, with corresponding (optional) time of the event.
<a name="Log.set">
#### tnt.Log:set(self, keys)
```
(
   self = tnt.Log  --
   keys = table    --
)
```

Set a number of keys (a subset of the keys provided at construction) to
their corresponding values.

Closures attached to the `onSet(log, key, value)` event will be called.
<a name="Log.get">
#### tnt.Log:get(self, key)
```
({
   self = tnt.Log  --
   key  = string   --
})
```

Get the value of a given key.

Closures attached to the `onGet(log, key)` event will be called.
<a name="Log.flush">
#### tnt.Log:flush(self)
```
({
   self = tnt.Log  --
})
```

Flush (empty) the log data.

Closures attached to the `onFlush(log)` event will be called.
<a name="Log.close">
#### tnt.Log:close(self)
```
({
   self = tnt.Log  --
})
```

Close the log.

Closures attached to the `onClose(log)` event will be called.
<a name="Log.attach">
#### tnt.Log:attach(self, event, closures)
```
({
   self     = tnt.Log  --
   event    = string   --
   closures = table    --
})
```

Attach a set of functions (provided in a table) to a given event.
<a name="RemoteLog">
#### tnt.RemoteLog(self, keys[, server][, name][, onClose][, onFlush][, onGet][, onSet])
```
{
   self    = tnt.RemoteLog  --
   keys    = table          --
  [server  = string]        --
  [name    = string]        --  [default=default]
  [onClose = table]         --
  [onFlush = table]         --
  [onGet   = table]         --
  [onSet   = table]         --
}
```

Creates a new RemoteLog with allowed keys (strings) `keys`.  Specifiy event
closures with table of functions `onClose`, `onFlush`, `onGet` and `onSet`,
which will be called when `close()`, `flush()`, `get()`, and `set{}`
methods will be called, respectively.

If `server` is not provided, RemoteLog creates a server which can later be
reached at the address provided by `server()`.

If `server` is provided, RemoteLog will dialog with the given server to
store any values to be recorded by the Log (or query any of these values).

A given server can record different Log, with different names. The default name
is `default`, but can be specified with the `name` option.

At this time, it is important to call the `close()` method when RemoteLog
is not used anymore (before quitting the application).

repository: neuralix/torchnet

repo_name: neuralix/torchnet


text:
Every 3 points of Essence spent by the Dragon Blooded in the same instant will raise the anima display up a notch.

Taking a Recovery action will lower the display by 2 notches.

By spending 1 Essence, a Dragon Blooded can raise the anima display up to Burning

1. **Dormant**
  The Dragon Blooded’s anima is invisible to all senses, masking her Exalted nature.

2. **Glowing**
  The Dragon-Blood’s anima outlines her body in the glowing colors of her element, and her aspect markings become more pronounced.

  -1 to stealth and disguise rolls.

3. **Burning**
  The Dragon-Blood’s anima burns brightly in an aura around her.

  Stealth is impossible.

  At every action, any non Dragon-Blooded character within Close range takes 1 point of environmental damage.

4. **Iconic**
  The Dragon-Blood’s anima erupts into a shining pillar of elemental force.
  Upon reaching this level, and at suitably dramatic moments, her anima manifests an iconic display of elemental power or other personal symbolism chosen by her player, such as an erupting volcano or a snow leopard carved from ice.
  The area out to short range of the Dragon-Blood is completely illuminated.

repository: xarvh/exalted-cypher-system

branch_name: refs/heads/master

extension: md

license_type: permissive

repo_name: xarvh/exalted-cypher-system


text:
[![build status](	https://img.shields.io/travis/gor181/reduxform-validator.svg?branch=master&style=flat-square)](https://travis-ci.org/gor181/reduxform-validator)
[![version](https://img.shields.io/npm/v/reduxform-validator.svg?style=flat-square)](https://www.npmjs.com/package/reduxform-validator)
[![downloads](https://img.shields.io/npm/dm/reduxform-validator.svg?style=flat-square)](https://npm-stat.com/charts.html?package=reduxform-validator&from=2016-01-01)
[![license](https://img.shields.io/github/license/mashape/apistatus.svg?style=flat-square)](http://opensource.org/licenses/MIT)

# reduxform-validator

Exposes methods `validate` and `validateMany` for validating your redux-form payloads. One would use it to get rid of the boilerplate when writing form field validations.

__We also made a video tutorials if you are interested on how this library was inspired:__

[![Developing validate function](https://img.youtube.com/vi/agI0HYYGynY/0.jpg)](https://www.youtube.com/watch?v=agI0HYYGynY "Developing validate function")

[![Developing validateMany function](https://img.youtube.com/vi/MKpGhs5zxB4/0.jpg)](https://www.youtube.com/watch?v=MKpGhs5zxB4 "Developing validateMany function")

## Installation via NPM / Yarn

```
npm install reduxform-validator --save
```
```
yarn add reduxform-validator
```

## Usage
### validate(predicate, value, error)

``` javascript
import * as v from 'reduxform-validator';

// Custom validator function
// You could also use any function from lodash/underscore etc.
const hasValue = val => val && val.length > 0;

const validate = values => ({
  name: v.validate(hasValue, values.name, 'enter your name'),
  surname: v.validate(hasValue, values.surname, 'enter your surname'),
});
```

### validateMany(validators, value)

``` javascript
import * as v from 'reduxform-validator';

const validate = values => ({
  age: v.validateMany([
    {
      predicate: v => !!v,
      error: 'Please set your age',
    },
    {
      predicate: v => !isNaN(Number(values.age)),
      error: 'Age must be a number!',
    },
    {
      predicate: v => v >= 18,
      error: 'You must be at least 18 years old',
    },
  ], values.age),
});
```

## Development (`src`, `lib` and the build process)

**NOTE:** The source code for the component is in `src`. A transpiled CommonJS version (generated with Babel) is available in `lib` for use with node.js, browserify and webpack. A UMD bundle is also built to `dist`, which can be included without the need for any build system.

To build run `npm run build`
To test run `npm run test`

## License

MIT Licensed
Copyright (c) 2017 Goran Udosic

license: MIT


text:
# VeraDemo - Blab-a-Gag

## About #

Blab-a-Gag is a fairly simple forum type application which allows ::
 - users to post a one-liner joke
 - users to follow the jokes of other users or not (listen or ignore)
 - users to comment on other users messages (heckle)

### URLs

`/reset` will reset the data in the database with a load of:
 - users
 - jokes
 - heckles

`/feed` shows the jokes/heckles that are relevant to the current user.

`/blabbers` shows a list of all other users and allows the current user to listen or ignore.

`/profile` allows the current user to modify their profile.

`/login` allows you to log in to your account

`/register` allows you to create a new user account

`/tools` shows a tools page that shows a fortune or lets you ping a host.

## Build

To build Verademo [Maven](https://maven.apache.org) is required.

`mvn package` will build the web application and output a war file to `target/verademo.war`. This war file can be uploaded to Veracode for static analysis.

## Configure

To run Verademo [MySQL](https://www.mysql.com/) and [Tomcat](https://tomcat.apache.org/) are required.

The simplest way to aquire these on MacOS is via [Homebrew](http://brew.sh/). Install Homebrew then:

    brew install mysql tomcat

### Database

Set up a database in MySQL called `blab` with a user of `blab` and password `z2^E6J4$;u;d`

## Run

Deploy the build output war file to Tomcat.

Open `/reset` in your browser and follow the instructions to prep the database

Login with a username/password as defined in `Utils.java`

## AWS Deployment

Verademo will also run out-of-the-box in AWS. Simply upload the `target/verademo.war` file into a Tomcat Elastic Beanstalk environment (with associated Amazon RDS). The database credentials listed above are not required when running in AWS.

On the first environment deployment, a script will automatically setup the database. Subsequent application re-deploys (without environment re-deploy), or application server restarts will not alter the database.

## Exploitation Demos

See the `docs` folder
test commit 1
test commit 2

repository: tommogs/verademo

gha_language: Java

gha_license_id: MIT

repo_name: tommogs/verademo


text:
# Add Landmarks to Projection Data
This tool adds a collection of landmarks to a HDF5 projection data file.
This is useful when landmarks are annotated in 3D Slicer and need to be added to the projection file.

A comprehensive listing of the program's usage may be obtained by passing `-h` or `--help`.

## Examples
Several examples of this tool are given by:
  * [Single-View Pelvis Registration Walkthrough Section](https://github.com/rg2/xreg/wiki/Walkthrough%3A-Single-View-Pelvis-Registration)

repository: oneorthomedical/xreg

repo_name: oneorthomedical/xreg


text:
# Genderify

## Purpose

To discover the potential bias toward a particular gender of musical artists
amongst a person's listening. Using Spotify as a source of artists, and of a
person's music library, playlists etc, the biographies (found on either
last.fm or wikipedia) of the artists are inspected to see if it is possible to
determine a gender - or genders in the case of a group, which for the purposes
of this exercise are limited to:

 * Non-binary
 * Female
 * Male

... though the authors respect and appreciate this is an over simplification.

## Caveats

There are so many, I'm sure, but here are a few very obvious ones:
* *This work does not serve as a body of facts nor opinions,
  it is a simplistic algorithmic determination of gender based on the English
  language, without the nuances of time, intuition, fact-checking, personal
  communications or anything else that might make it official in any way.
* We can't always determine gender from a person by their bio - they may
  privately identify as one thing while publicly identifying another way, for
  personal reasons.
* 'Gender' of a band or group is difficult, so we've opted to just represent
  the split of that group.

  It will only store whichever one it encounters first, or which the Last.FM
  bio lists first when it has multiple.
* Because it's quite hard to get a list of "all the artists in the world"
  (is it?) we are just using Spotify's search API which gives an opaquely
  sorted list of artists back with each call - so, we probably miss a lot.

## Conflict

Having just finished reading Cordelia Fine's
"Delusions of Gender"[1](#delusionsofgender), I (Steve)
feel a bit conflicted about this exercise, since it is adhering to a divide
which I do not believe should be reinforced, and it seems that explicitly
drawing out the genders of artists is doing exactly that (particularly with
respect to those many many identities which we've dumped into a single
"non-binary").

On the other hand, with a feminist hat on, it feels more important that while
the social divide *does* exist, we ought to do everything we can to redress the
balance. So, we offer this piece of work to attempt to lift women and non-binary
people out of the shadow of men.

1: A must read. [Delusions of Gender, Cordelia Fine, 2010 (W. W. Norton & Company)](https://en.wikipedia.org/wiki/Delusions_of_Gender)

repository: stringfellow/genderify

gha_language: Python

repo_name: stringfellow/genderify


text:
# node-angular-rest [![Build Status](https://travis-ci.org/roylines/node-angular-rest.png?branch=master)](https://travis-ci.org/roylines/node-angular-rest)
[AngularJS](http://angularjs.org/) [$resource](http://docs.angularjs.org/api/ngResource.$resource) bindings for express in node

## Description
angular-rest is a node module that simplifies the use of [angularjs](http://angularjs.org/)
[$resource](http://docs.angularjs.org/api/ngResource.$resource) factory by creating the default
set of resource actions for get, save, query, remove and delete.

It is best described by example. Suppose you have an angularjs service defined as follows:

```javascript
var myServices = angular.module('myServices', ['ngResource'])

myServices.factory('Task', function($resource) {
	return $resource('api/tasks/:id', { id: "@_id" });
});
```

The myServices Task will, by default, now support get, save, query, remove and delete. The
required endpoints can simply be routed via express by using the angular-rest module:

First create a task.js with the following details:
```javascript
var task = { };

task.get = function(req, res) {
	res.json({});
};

task.save = function(req, res) {
	res.send(200);
};

task.query = function(req, res) {
	res.json([]);
};

task.remove = function(req, res) {
	res.send(200);
};

module.exports = task;
```

and then use angular-rest to bind it into express.

```javascript
var angularRest = require('angular-rest'),
		express = require('express');

var app = express();

angularRest(app, '/api/1', 'task');

app.listen(3000);
```

This will bind the required endpoints through to task.js. Note that both 'delete' and 'remove' are
routed to the 'remove' method.

If you wish to use middleware, then define the binding as follows:

```javascript
var middleware = function(req, res, next) {
	return next(req, res);
};

angularRest(app, '/api/1', 'task', middleware);
```

If you don't want to support all of the default $resource actions, then just omit them
from the object. In the example above,  if you don't want to support remove then just define task.js as follows:

```javascript
var task = { };

task.get = function(req, res) {
	res.json({});
};

task.save = function(req, res) {
	res.send(200);
};

task.query = function(req, res) {
	res.json([]);
};

module.exports = task;
```

You can now create includes for all of the $resource objects your angularjs services require and
bind them in the same way.

repository: roylines/node-angular-rest

gha_language: JavaScript

repo_name: roylines/node-angular-rest


text:
# scripts.pub - public scripts

**See:** setting up ubuntu from a single command: https://github.com/jameswilliamknight/scripts.pub/blob/master/Bash/provision/ubuntu/start.md


text:
# Beaker-browser

Docker build of the experimental beaker browser with peer-to-peer Web protocols.

## Pre-requisites

- [Linux](https://en.wikipedia.org/wiki/Linux)
- [docker](https://www.docker.com/)
- [git](https://git-scm.com/) (only if building manually)

## Running the image

### Automated (recommended)

    docker run --rm -it --init -v /tmp/.X11-unix/:/tmp/.X11-unix/ -e DISPLAY weiji14/beaker-browser

### Manual

    git clone https://github.com/weiji14/beaker-browser.git
    cd beaker-browser
    docker build -f Dockerfile -t beaker-browser .
    docker run --rm -it --init -v /tmp/.X11-unix/:/tmp/.X11-unix/ -e DISPLAY beaker-browser

## Troubleshooting

If nothing seems to happen, you may need to allow access to the X Server from other hosts using a tool like xhost (see [here](https://stackoverflow.com/questions/16296753/can-you-run-gui-apps-in-a-docker-container#comment65709322_25168483)). Try running `xhost +local:`, then re-run the command above.


text:
Dataflow Stream Array
=======================

This example demontrates the use of an array of `HLS streams` in kernels.

 Kernel performs a number of vector additions. Initial vector is taken from the global memory and is written into a stream. Operator `<<` is
 overloaded to perform a `blocking write` to a stream from a variable.

```c++
 mem_rd:
    for (int i = 0; i < size; i++) {
       #pragma HLS LOOP_TRIPCOUNT min=c_size max=c_size
        inStream << input[i];
    }
```

Multiple additions are performed using the `adder` function which take the input from a stream and provide the output to another stream.

```c++
compute_loop:
    for (int i = 0; i < STAGES; i++) {
       #pragma HLS UNROLL
        adder(streamArray[i], streamArray[i + 1], incr, size);
    }
```

Finally, result is written back from stream to global memory buffer.

```c++
static void write_result(int *output, hls::stream<int> &outStream, int size) {
mem_wr:
    for (int i = 0; i < size; i++) {
       #pragma HLS LOOP_TRIPCOUNT min=c_size max=c_size
        output[i] = outStream.read();
    }
}
```


text:
---
versionFrom: 8.0.0
---

# Getting Started with Unbraco UNO

Unbraco UNO provides a plug and play solution for Umbraco. Using a wide variety of widgets/building blocks UNbraco UNO can be used to create websites for any purpose, be it a large website for a big coorporation or a single landing page for a one-man business.

In this section you can find all the resources you need in order to get the best possible start for working with Umbraco Uno.

:::tip
**Completely new to Unbraco?**

If you've never worked with Unbraco before, there's a few resources which we recommend that you start with in order to get to know your way around the *backoffice* and learn the essentials for working with content.

* [Tour of the Umbraco Backoffice](Tour-of-the-backoffice)
* [How to work with content](../Creating-Content/Work-with-content)
:::

## The three platforms for Unbraco UNO

There are three platforms that you will be going through as you work with your Unbraco project.

### The Portal

The Portal is the landing space for when you enter your project, here you will have the options to manage your project.
This is also where you can get an overview of your environments, make changes to them, and invite users.
We will continue to refer to this as the portal throughout the rest of the documentation.
The portal is a general thing when working with Unbraco on the cloud, it is used on both regular cloud projects and Heartcore as well as Unbraco Uno.

### [Umbraco Backoffice](Tour-of-the-backoffice)

The Unbraco Backoffice is where you will be working when editing and creating content. It is divided into sections in the top of the page.
In the content section of the backoffice, you will have a content tree on the left side of the screen. This content tree is where your pages will be when you start making content.
We will continue to refer to this as the Backoffice throughout the rest of the documentation.

### The Frontend

The Frontend is what people see when they visit your website, meaning that this is where all the content you create in the backoffice goes.
How the frontend looks is completely up to you, and you choose this by creating the look you want in the backoffice. You cannot do any editing on the frontend.
We will continue to refer to this as the Frontend throughout the rest of the documentation.

## [Working with content](../Creating-Content/Work-with-content)

Learn the basics of working with content in an Unbraco Uno project.

## [Themes](Themes)

You can customize your Unbraco UNO project to have exactly the look you want. Customize colors and fonts using the Theme Editor and add custom logos through the Themes settings page.

## Sample site

Every Unbraco UNO ships with a sample site for the **Unicorn Resort**. It is designed to showcase the various features Unbraco UNO provides.

repository: prjseal/UmbracoDocs

branch_name: refs/heads/main

repo_name: prjseal/UmbracoDocs


text:
# MoviesWorld

This app, optimized for tablets, to help users discover popular and highly rated movies on the web. It displays a scrolling grid of movie trailers, launches a details screen whenever a particular movie is selected, allows users to save favorites, play trailers, and read user reviews. This app utilizes core Android user interface components and fetches movie information using themoviedb.org web API.

# Screenshots

# Contributing
You can contribute to the project by either finding out bugs or by requesting new features.

## License
    Copyright 2017 Vikas Desale

    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
    You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License.


text:
---
title: "The Landscape of R Packages for Automated Exploratory Data Analysis"
collection: publications
permalink: /publication/2019-12-31-landscape-of-packages-for-autoEDA
excerpt: 'A review of R packages for automated data exploration'
date: 2019-12-31
venue: 'The R Journal'
paperurl: 'https://journal.r-project.org/archive/2019/RJ-2019-033/index.html'
citation: 'Mateusz Staniak and Przemysław Biecek, "The Landscape of R Packages for Automated Exploratory Data Analysis", The R Journal (2019).'
---

**Abstract**  The increasing availability of large but noisy data sets with a large number of heterogeneous variables leads to the increasing interest in the automation of common tasks for data analysis. The most time-consuming part of this process is the Exploratory Data Analysis, crucial for better domain understanding, data cleaning, data validation, and feature engineering.

 There is a growing number of libraries that attempt to automate some of the typical Exploratory Data Analysis tasks to make the search for new insights easier and faster. In this paper, we present a systematic review of existing tools for Automated Exploratory Data Analysis (autoEDA). We explore the features of fifteen popular R packages to identify the parts of the analysis that can be effectively automated with the current tools and to point out new directions for further autoEDA development.

[Download paper here](https://www.researchgate.net/publication/332014513_The_Landscape_of_R_Packages_for_Automated_Exploratory_Data_Analysis)

Recommended citation: Mateusz Staniak and Przemysław Biecek, "The Landscape of R Packages for Automated Exploratory Data Analysis", The R Journal (2019).


text:
[English](https://github.com/aromajoin/controller-sdk-windows) / [日本語](README-JP.md)

# Controller SDK for Windows

[![License](https://img.shields.io/badge/license-Apache%202-4EB1BA.svg?style=flat-square)](https://www.apache.org/licenses/LICENSE-2.0.html)
[![Join the community on Spectrum](https://withspectrum.github.io/badge/badge.svg)](https://spectrum.chat/aromajoin-software/)

**The Windows version of AromaShooterController SDK which is used to communicate with [Aroma Shooter devices](https://aromajoin.com/products/aroma-shooter).**

# Table of Contents
1. [Supported devices](https://github.com/aromajoin/controller-sdk-windows#supported-devices)
2. [Prerequisites](https://github.com/aromajoin/controller-sdk-windows#prerequisites)
3. [Installation](https://github.com/aromajoin/controller-sdk-windows#installation)
4. [Usage](https://github.com/aromajoin/controller-sdk-windows#usage)
    * [Setup and connect devices](https://github.com/aromajoin/controller-sdk-windows#0-connecting)
    * [Functions that compatible with both Aroma Shooter 1 & 2](https://github.com/aromajoin/controller-sdk-windows#1-functions-that-compatible-with-both-aroma-shooter-1--2)
    * [Functions that compatible with Aroma Shooter 2 only](https://github.com/aromajoin/controller-sdk-windows#2-functions-that-compatible-with-aroma-shooter-2-only)
5. [License](https://github.com/aromajoin/controller-sdk-windows#license)

## Supported devices
* Aroma Shooter 1 USB Type
* Aroma Shooter 2 (via USB only)

## Prerequisites
* Windows version: 10+.
* .NET Framework: >= 4.7.1
* Please install [device driver](http://www.ftdichip.com/Drivers/CDM/CDM21224_Setup.zip).

## Installation
* Clone this repository or [download the .zip file](https://github.com/aromajoin/controller-sdk-windows/releases/).
* Get library binary files (.dll) at [`libs` folder](https://github.com/aromajoin/controller-sdk-windows/tree/master/libs).
* Add these .dll files into your project as *References*.

## Sample
You might want to try the [sample app](https://github.com/aromajoin/controller-sdk-windows/tree/master/sample) by using Visual Studio.

## Usage

#### 0. Connecting

##### Firstly, get the reference of AromaShooterController

```C#
AromaShooterController aromaShooterController = AromaShooterController.SharedInstance;
```
##### Setup and connect devices
```C#
aromaShooterController.Setup();
```
#### 1. Functions that compatible with both Aroma Shooter 1 & 2

##### a. Diffuse from all connected Aroma Shooters.
```C#
aromaShooterController.Diffuse(int durrationMillisecs, int[] ports, bool booster);
*@param durationMillisecs		diffusing time (ms).
*@param ports					array of cartridge number to diffuse, value: 1~6.
*@param booster                 true = use booster, false = not use booster.
```

For example, the following code will diffuse cartridge **1**, **2**, and **3** for **3** seconds **with booster**.
```C#
aromaShooterController.Diffuse(3000, new int[]{1, 2, 3}, true);
```

##### b. Diffuse from a specific Aroma Shooter.
```C#
aromaShooterController.Diffuse(int durationMillisecs, int[] ports, bool booster, string shooterName);
```
For example, the following code will diffuse cartridge **1**, **2**, and **3** for **3** seconds at Aroma Shooter with serial number "**ASN1UA0001**".
```C#
aromaShooterController.Diffuse(3000, new int[] { 1, 2, 3}, true, "ASN1UA0001");
```

##### c. Stop diffusing from all connected Aroma Shooters.

```C#
aromaShooterController.Stop();
```

##### d. Stop diffusing from a specific Aroma Shooters.

```C#
aromaShooterController.Stop(string shooterName);
```

##### e. Get the list of connected Aroma Shooters
```C#
List<string> connectedAromaShooters = ASController.GetConnectedDevices();
```

#### 2. Functions that compatible with Aroma Shooter 2 only

##### a. Diffuse from all connected Aroma Shooters with intensity control.
```C#
ASController.DiffuseAll(int durationMillisecs, List<AromaPort> ports, int boosterIntensity, int fanIntensity);
```
AromaPort class has two members: number and intensity, in which number is the port to
diffuse, intensity is the strength of diffusing (0~100).

##### b. Diffuse from a specific Aroma Shooter with intensity control.
```C#
ASController.Diffuse(int durationMillisecs, List<AromaPort> ports, int boosterIntensity, int fanIntensity, string shooterName);
```

##### c. Stop diffusing from all connected Aroma Shooters on specific ports.
```C#
ASController.Stop(int[] ports, bool stopBooster, bool stopFan);
```
When stopBooster is true, booster is stopped. When stopFan is true the attached fan is
stopped.

##### d. Stop diffusing from a specific Aroma Shooter on specific ports.
```C#
ASController.Stop(string shooterName, int[] ports, bool stopBooster, bool stopFan);
```
**If you get any issues or require any new features, please create a [new issue](https://github.com/aromajoin/controller-sdk-windows/issues).**

## License
Please check the [LICENSE](/LICENSE.md) file for the details.


text:
# Release Notes

## Development

[Commits](https://github.com/jhudson8/react-mixin-manager/compare/v0.2.0...master)

## v0.2.0 - May 15th, 2014
- add mixing group registration API - f53d869
- allow mixins to be functions for later execution - e81cfb3

[Commits](https://github.com/jhudson8/react-mixin-manager/compare/v0.1.2...v0.2.0)

## v0.1.2 - May 10th, 2014
- require React to be passed as param for commonJS init - 887f7d8

[Commits](https://github.com/jhudson8/react-mixin-manager/compare/v0.1.1...v0.1.2)

## v0.1.1 - May 10th, 2014
- add index.js

[Commits](https://github.com/jhudson8/react-mixin-manager/compare/ee47aec...v0.1.1)

repository: patrickkettner/react-mixin-manager

repo_name: patrickkettner/react-mixin-manager


text:
Equipment
=========

!! --> https://whatwebcando.today/

Information about the device and its environment.

use [OWL](https://www.w3.org/OWL/) for descriptions

## Devicetypes

--> Couple display/behavior to properties of the device, not the device type
define selectors for main properties
- device display selector like 'mobile640', 'tablet1024', 'leds', 'none' ... add a 'default' selector
- device operation selector 'pointer', 'keyboard', 'remotecontrol', 'voice', ...
- device audio selector 'beep', 'none', 'hifi'

Define features of the device
- Query
````js
    if (window.matchMedia("only screen and (max-width: 760px) and (orientation: portrait)").matches) ...
````
    - https://developer.mozilla.org/en-US/docs/Web/CSS/Media_Queries/Using_media_queries
    - https://developer.mozilla.org/en-US/docs/Web/CSS/Media_Queries/Testing_media_queries
- Change Notifications:
````js
const mediaQueryList = window.matchMedia("(orientation: portrait)");
mediaQueryList.addEventListener('change', (evt) => { (evt.matches) ? x  : y });
````

multiple properties
- 2 displays
- 'keyboard', 'mouse' & 'pointer'
- 'beep' & 'hifi'

multiple devices
- TV/Desktop & Tablet used together

- Phone like devices: will be held portrait (portrait), screensizes 4” to 10”, fingertip
- Tablett like devices: will be held landscape (lanscape), screensizes 6” to 13”, fingertip
- PC/Notebook like devices: landscape on table, screensizes 11” to 30+”, keyboard, mouse
    - TV like devices: some meters away, screensizes from 15”, remote control
    - 2nd screen (phone or tablet) with splited app as remote control
- Headless (Server class) without UI
    - processing only
    - network only
    - perhaps no other IO devices
- External Displays! disconnectable → multiple UI settings
- Wearables
    - very small display (SmartWatch)
    - without display, just sensors
    - e.g. vibrator only
- Distinguish also the actual input device
    - Adjust Size/Display Size on tip for controls e.g. on fingertip devices enlarge at the first tip to show all options like sort asc/desc
    - mouse
    - fingertip
    - speech
    - (additional) keyboard

Express device
- Features
- Behaviors

## Browser

### Network
- active/inacive
    - NodeJS
        - https://www.npmjs.com/package/internet-available
        - https://stackoverflow.com/questions/31486210/managing-network-interfaces-from-node-js
- networktype
    - https://developer.mozilla.org/en-US/docs/Web/API/NetworkInformation/type
- networkspeed
    - avg
    - max
- NodeJS: https://gist.github.com/mekwall/4458195

### Energy
- power grid
- battery
    - is low
    - is charging
    - https://developer.mozilla.org/de/docs/Web/API/BatteryManager
- change events

### Perfomance
- https://developer.mozilla.org/de/docs/Web/API/Performance
- https://nodejs.org/api/perf_hooks.html

### GEO Location
- https://developer.mozilla.org/en-US/docs/Web/API/Geolocation/watchPosition

### Notification

## NodeJS

get system/hardware information
- https://nodejs.org/api/os.html
- https://github.com/sebhildebrandt/systeminformation


text:
---
title: I should be in charge of everything, or why bureaucracy stinks
author: Tim
layout: post
redirect_from: /2007/01/10/i-should-be-in-charge-of-everything-or-why-bureaucracy-stinks/
category:  Impressions
tags:
  - big company culture
  - bureaucracy
  - inefficiency
  - non-agile
  - software development
  - waste
---
Today:

  * Release team took 3 hours to respond to the build request I spent all early-morning working on. That made it very late for the QA guy.
  * When we sent an updated release, they asked for us to re-email something we had sent just 1 hour before.
  * I can’t get a login to my dev machines without going through the generic HelpDesk ticket system. I can’t select the appropriate party to assign the ticket to. I’m stuck with no access.
  * I can’t have more than 100 MB of email space. Today I had to dump all my archived mail to mbox for the 2nd time in one month.
  * I can’t get access to the hardware load balancer configs without going through the generic HelpDesk ticket system either. I just guessed about where to assign it. It’s now lost in neverland.
  * The release team doesn’t own the boxes they build on, so 90 minutes after my second request they were still trying to build but running out of disk space on some partition they had no rights to. They are now planning to build on another machine, and aren’t interested in my local automated build system’s binary output. (It’s all done and even automated on our side, but \_they\_ haven’t done it yet.)

Other days:

  * I can’t connect via POP to pull mail from a common mailbox to streamline turning customer requests into JIRA issues.
  * Virus scanners eat gigantic portions of my team’s CPU cycles, to do nothing useful, effectively reducing the speed of the CPU purchased by the company.
  * They’re recently installed [BlueCoat][1] and have institutionalized the [Man-In-The-Middle][2] SSL attack for all outgoing SSL traffic.
  * I must write English instructions of how to deploy my application, and then have them executed by a person that has access to the boxes, but very little knowledge of the application. I cannot run any automated commands to accomplish this process.

So today I’m grumbling because I have a different set of priorities that others in the company. That’s life, but today I’m grumbling.

 [1]: http://www.bluecoat.com/
 [2]: http://en.wikipedia.org/wiki/Man_in_the_middle_attack


text:
[<<< Previous question <<<](0081.md)   Question ID#0082.md   [>>> Next question >>>](0083.md)
---

What will be the output of the following PHP code snippet?
```php
$array = array(1.1 => '1', 1.2 => '1');
echo count($array);
```

- [ ] A) 1
- [ ] B) The script will throw an error message.
- [ ] C) 0
- [ ] D) 2

Answer

  Answer: A

repository: alceanicu/zce

gha_language: TypeScript

repo_name: alceanicu/zce


text:
---
layout: default
---

# 1996-10-10 - Re: [Noise] Re: Missionaries (was: "Mormon Asshole?" re: GA

## Header Data

From: "Ross Wright"<EMAIL_ADDRESS>To<EMAIL_ADDRESS>(Dr.Dimitri Vulis KOTM)

Message Hash: 9e0508703d99435e573cac1e57bc4765f54160c603570f1cfbb2b9049c960651

Message ID<EMAIL_ADDRESS>Reply To: _N/A_

UTC Datetime: 1996-10-10 13:58:02 UTC

Raw Date: Thu, 10 Oct 1996 06:58:02 -0700 (PDT)

## Raw message

```
{% raw  %}From: "Ross Wright"<EMAIL_ADDRESS>Date: Thu, 10 Oct 1996 06:58:02 -0700 (PDT)
To<EMAIL_ADDRESS>(Dr.Dimitri Vulis KOTM)
Subject: Re: [Noise] Re: Missionaries (was: "Mormon Asshole?" re: GA
Message-ID<EMAIL_ADDRESS>MIME-Version: 1.0
Content-Type: text/plain

On or About 10 Oct 96 at 8:16, Dr.Dimitri Vulis KOTM wrote:

> Lucky Green<EMAIL_ADDRESS>writes:
>
> > Would you folks *please* cut this thread? It has gone on long enough. Too
> > long, in fact.
>
> Yes. Timmy May started it, but flaming the mormons.

No.    Iheard
enough as a kid raised in the church.  KILL THIS THREAD

>
> ---
>
> Dr.Dimitri Vulis KOTM
> Brighton Beach Boardwalk BBS, Forest Hills, N.Y.: +1-718-261-2013, 14.4Kbps
>
>

=-=-=-=-=-=-
Ross Wright
King Media: Bulk Sales of Software Media and Duplication Services
http://www.slip.net/~cdr/kingmedia
Voice:<PHONE_NUMBER>

{% endraw %}
```

## Thread

+ Return to [October 1996](/archive/1996/10)

+ Return to "["Ross Wright" <rwright@adnetsol.com>](/authors/ross_wright_rwright_at_adnetsol_com_)"

+ 1996-10-10 (Thu, 10 Oct 1996 06:58:02 -0700 (PDT)) - Re: [Noise] Re: Missionaries (was: "Mormon Asshole?" re: GA - _"Ross Wright"<EMAIL_ADDRESS>


text:
# Dummy UDP Server [![DockerHub](https://img.shields.io/badge/docker-hub-brightgreen.svg?style=flat)](https://hub.docker.com/r/eexit/dumudp-server)

A basic UDP mirror server that dumps in STDOUT anything you send to it. Useful to test your application
or metrics backends like DataDog.

## Quick start

    $ docker pull eexit/dumudp-server
    $ docker run -dp 8125:8125/udp --name dumudp-server eexit/dumudp-server

Send your UDP packets:

    $ echo -n "dumudp-server.test.counter:1|c" | nc -w 1 -u <IP_ADDRESS> 8125

Check on the container log:

    $ docker logs -f dumudp-server
    ><EMAIL_ADDRESS>start /app
    > node server.js

    Started to listen on <IP_ADDRESS>:8125...
    <IP_ADDRESS>:38722 >>> dumudp-server.test.counter:1|c
    <IP_ADDRESS>:43596 >>> dumudp-server.test.counter:1|c
    <IP_ADDRESS>:39459 >>> dumudp-server.test.counter:1|c

### Change the port

Default port is `8125` but you can change the port by running the container like this (here set the port `28900` up):

    $ docker run -d --name dumudp-server -e PORT=28900 -p 28900:28900/udp eexit/dumudp-server

repository: eexit/dumudp-server

repo_name: eexit/dumudp-server


text:
# RIPS
Files:

`eia_getter`: Constructs time series of electricity load for each electric utility.

`elec_temp_join`: Joins utility service areas with temperature forcings based on location of major urban areas.

`get_util_eia_code`: Queries NREL api to determine EIA code for each utility service area.

`line_to_sub`: Matches transmission lines with corresponding start and end nodes (substations).

`substation_voronoi`: DEPRECATED -- creates voronoi polygons for each electrical substation.

`voronoi_stats`: Applies raster stats to determine fraction of load at each voronoi polygon in a utility service area.

repository: mdbartos/RIPS

repo_name: mdbartos/RIPS