text:
# TGEN

Fake ***T***ime series data ***GEN***erator tool written in Go.

Linux 64 bit binary is [here](tgen.linux.amd64.gz).

Windows 64 bit binary is [here](tgen.windows.amd64.exe.gz).

Or you can clone and use `go build` with your desired architecture. TGEN was
built with `go version go1.13.3 linux/amd64`.

## Background

I worked on a project where we needed to design a large time series data
warehouse whose input systems were unrelieable and many edge cases identified.
I even put together an article on the cases if you're interested
[here](https://www.linkedin.com/pulse/what-gotchas-edge-cases-when-processing-raw-time-series-dragan-rajak/).

The requirements for a fake timeseries generator were:

 * Generate data as a realtime stream or a batch input file

 * Generate plain text CSV or JSON in any format

 * Generate gigabytes of data for stress and volume testing

 * Generate millions of unique identities (see
   [here](https://www.linkedin.com/pulse/what-time-series-data-do-we-really-collect-dragan-rajak/)
   for background on terminology)

 * Generate consistent, repeatable data to facilitate testing

 * Simulate missing samples and missing data (two different concepts outlined
   below)

 * Generate seasonality, spikes, outliers and errors

I couldn't find what I wanted so as anyone would do I just put together
something quick and took it as an opportunity to learn a bit of Go.

This tool implements all of the above requirements and uses the
[go-fake-ts](https://github.com/powerpu/go-fake-ts) library which generates the
underlying numbers.

## Concepts

*You can have a read through the [go-fake-ts
library](https://github.com/powerpu/go-fake-ts) readme or the [Golang
documentation](https://godoc.org/github.com/powerpu/go-fake-ts) documentation
for some underlying concepts of the libary. They're summarised here for
convenience.*

The package breaks down time series data into two major parts, samples and the
actual data.

There is a distinctinction between "bad samples" and "bad data".

An example of the distinction would be in an example of collecting CPU and
memory data from a server.

In a "valid sample" the collection server would connect and retrieve this
information from the destination server in a particular way with no issues.

In a "bad sample" scenario, the destination server would be running fine but
due to a network outage a sample would not be collected.

In a "bad data" scenario, the destination server would be running and the
network would be up but due to a rogue process the CPU value would not be able
to be collected while memory could. The end result is that the CPU data is
"bad".

## Basics

There are 4 types of objects one can create in the config. They are:

 * **Random** - A true/false pattern that changes randomly (e.g. 5% true)

 * **Pattern** - A true/false pattern that repeats (e.g. 2 true, 1 false)

 * **Time** - Time one would plot on the X-axis of a time series graph (e.g.
   steady 30 minute intervals)

 * **Data** - Actual numbers one would plot as a feature/metric in a timeseries
   graph (e.g. 0 to 100, going up)

The general usage would be to instantiate define the types in the config file,
design the relevant output data template and run the tool to generate the data.
Refer to "Workflow" below for more info.

Ultimately the goal is to generate time series data that may look as follows:

```
  Timestamp, CPU, Memory
<PHONE_NUMBER>,23.5,97.2
<PHONE_NUMBER>,26.5,84.9
  ...etc...
```

One would use the true/false value of a Pattern or Random to determine what to
print.

An example simulation scenario could be generating data where every 24th hourly
sample fails because you want to simulate a scheduled job taking up too many
resources and thus not retrieving data.

For this you would use a 23 good samples and 1 bad samples pattern with time
going up by 1 hour intervals.

## Workflow

The fundamental workflow of how you would use the tool is as follows:

 1. Identify the sorts of *numerical* use cases you would like to generate.
    Examples include "I need numbers between 0 and 100 that trend upwards" or
    "I need numbers to follow seasonality every 500 points" or "I need numbers
    between 50 and 100 that trend upwars but spike to 110 every 100 samples for
    3 samples".

 1. Identify the sorts of *times* you would like to generate. Examples include
    "Perfect hourly, 1:00:00, 2:00:00 etc..." or "Half hourly but with a
    potential delay of up to 10 seconds, 1:00:03, 2:00:09 etc...".

 1. Identify the sorts of *missing sample patterns* you would like to see.
    Examples include "No missing samples", "Regularly, 12 valid samples
    followed by 3 bad ones", "Randomly, about 15% need to be bad samples".

 1. Use the `./tgen playarea` command to start a webserver and connect to it to
    visualise the different configuration parameters for your fake data.

 1. Put the parameters the playarea command generates into the config file (you
    can get defaults if you run `./tgen default` to start you off).

 1. Familarise yourself a little with Golang
    [templates](https://golang.org/pkg/text/template/) and use them to generate
    data generation templates that fit your needs be they CSV, JSON or
    something else entirely.

 1. Use the tool to generate the fake data using `./tgen generate`.

## Example Scenario

Let's say we want to create the following scenario:

 * Collect data from 3 rain sensors

 * Collection frequency is 1 hour

 * Sensor 1 is always perfect

 * Sensor 2 is uncontactable (i.e. bad) 15% of the time

 * Sensor 3 is always contactable but due to a bug cannot provide every 8th
   sample

What you would do in this scenario is create the following in your config:

 * A `random` that will give us `true` 85% of the time (100% good - 15% bad = 85% good). This will be used for sensor 2.

 * A `pattern` that will give us `true` 7 times and `false` 1 time. This will be used for Sensor 3.

 * A `time` that will increment perfectly by 1 hour. This will be used for all Sensors.

 * 3 different types of `data` to represent each sensors conditions/values.

The config may look like this:

```
RANDOM
ID,SEED,GOOD_PCT,DESCRIPTION
r1,1,0.85,80% good
PATTERN
ID,PATTERN_GOOD,PATTERN_BAD,DESCRIPTION
p1,7,1,7 good 1 bad
TIMES
ID,INCREMENT,VARIANCE,DIRECTION,DESCRIPTION
t1,3600000,0,0,1 hour time no variance
DATA
ID,STRETCH_START,STRETCH_END,SLOPE,BUMP,FROM,TO,LIMIT_UPPER,LIMIT_LOWER,PERMA_BUMP_AT,PERMA_BUMP_BY,PERMA_BUMP_SMOOTHER,USE_RANDOM,RANDOM_SEED,RANDOM_BIAS,GENERATE_SPIKES,SPIKE_SUSTAIN,SPIKE_EVERY,SPIKE_TO,SPIKE_WOBBLE,SPIKE_WOBBLE_FACTOR,SPIKE_SMOOTHER,USE_SEASONALITY,SEASONALITY_WAVE1,SEASONALITY_WAVE2,SEASONALITY_WAVE3,SEASONALITY_WAVE4,SEASONALITY_WAVE5,DESCRIPTION
d1,1,1,0,0,50,100,false,false,0,0,0,true,1,0.5,false,5,100,100,false,200,20,false,300,1,1,1,1,Random example
d2,1,1,0,0,10,100,false,false,43,100,1,true,1,0.5,false,5,100,100,false,200,20,false,300,1,1,1,1,Random example with a permanent bump
d3,1,1,0,0,30,100,false,false,0,0,0,false,1,0.5,false,5,100,100,false,200,20,true,300,1,1,1,1,Seasonality example
```

Parameters are explained in the "Help for 'Generate'" below and in the
[NewData function Godoc](https://godoc.org/github.com/powerpu/go-fake-ts#NewData)
documentation of the [go-fake-ts](https://github.com/powerpu/go-fake-ts) library.

You would also normally use `./tgen playarea` to help you generate the desired
config for `DATA` since it can accept many parameters.

Let's say you want to generate a simple CSV that may look like this:

```
SensorName,Timestamp,  mmPerHour
Sensor-1,<PHONE_NUMBER>, 23.5
Sensor-2,<PHONE_NUMBER>, 6.5
Sensor-3,<PHONE_NUMBER>, 0.1
Sensor-1,<PHONE_NUMBER>, 23.5
Sensor-2,<PHONE_NUMBER>, 6.5
Sensor-3,<PHONE_NUMBER>, <missing>
Sensor-1,<PHONE_NUMBER>, 23.5
Sensor-3,<PHONE_NUMBER>, 0.1
...etc...
```

Notice how `Sensor-3` has "bad data" at `1580003600` while `Sensor-2` has a
"bad sample" `1580007200` (i.e. it's actually missing).

Your Golang template for this example would look like this:

```
Sensor-1,{{ $.t1.Val.Unix }},{{ $.d1.Val }}
{{- if $.r1.Val }}
Sensor-2,{{ $.t1.Val.Unix }},{{ $.d2.Val }}
{{- end }}
Sensor-3,{{ $.t1.Val.Unix }},{{ if $.p1.Val }}{{ $.d3.Val }}{{ else }}<missing>{{ end }}
```

You would then run `./tgen generate -c example.conf -t example.template` and
would see the following:

```
Sensor-1,1582258623,-74.37104824258068
Sensor-3,1582258623,66.3612572924182
Sensor-1,1582262223,-74.41366470389394
Sensor-2,1582262223,-54.57002078285556
Sensor-3,1582262223,67.72191749239796
Sensor-1,1582265823,-74.40755940656285
Sensor-2,1582265823,-54.56554356481276
Sensor-3,1582265823,69.08138376940538
Sensor-1,1582269423,-74.39862143076873
Sensor-2,1582269423,-54.558989049230405
Sensor-3,1582269423,70.4390598166005
Sensor-1,1582273023,-74.45354889738809
Sensor-2,1582273023,-54.59926919141793
Sensor-3,1582273023,71.79435011239748
Sensor-1,1582276623,-74.15663251792097
Sensor-2,1582276623,-54.381530513142046
Sensor-3,1582276623,73.14666018167978
Sensor-1,1582280223,-73.97096610806699
Sensor-2,1582280223,-54.245375145915794
Sensor-3,1582280223,<missing>
Sensor-1,1582283823,-73.71534051521435
Sensor-2,1582283823,-54.05791637782386
Sensor-3,1582283823,75.83996853654665
Sensor-1,1582287423,-73.6529642565065
Sensor-2,1582287423,-54.012173788104754
Sensor-3,1582287423,77.1797854480721
Sensor-1,1582291023,-73.6533284540953
Sensor-2,1582291023,-54.01244086633656
Sensor-3,1582291023,78.51425990315435
```

This generates 10 samples by default, which is 30 lines since the template
represents one sample (actually 29 in our example since one sample is missing).

There are many other options you can pass. Refer to "Help for 'Generate'" below
for more parameters you could pass.

For example if you want to generate a continuous stream of numbers with a 1
second delay you would run the following:

```
./tgen generate -c example.conf -t example.template -s -r 1000
```

## Embedded Documentation

The tool also has documentation embedded in itself and is shown below.

### No arguments
```
[tgen]$ ./tgen

(T)ime series data (Gen)erator is a flexible fake time series data
generator built with love by me, myself and I in Go. Complete documentation is
available at https://github.com/powerpu/tgen

Usage:
  tgen [flags]
  tgen [command]

Available Commands:
  default     Prints out the default internal config and template
  generate    Generates fake time series data
  help        Help about any command
  playarea    Starts a webserver playarea to quickly visualise generated data

Flags:
  -h, --help   help for tgen

Use "tgen [command] --help" for more information about a command.
```

### Help for "default"

```
[tgen]$ ./tgen help default
Prints out the default internal config and template you can use as a starting
point for developing your own fake data.

Usage:
  tgen default [flags]

Flags:
  -h, --help   help for default
```

### The "default" templates

```
[tgen]$ ./tgen default

Printing the default config:
===========================

RANDOM
ID,SEED,GOOD_PCT,DESCRIPTION
r0,1,0,All bad
r1,1,0.1,10% good
r2,1,0.2,20% good
r3,1,0.3,30% good
r4,1,0.4,40% good
r5,1,0.5,50% good
r6,1,0.6,60% good
r7,1,0.7,70% good
r8,1,0.8,80% good
r9,1,0.9,90% good
r10,1,1,All good
PATTERN
ID,PATTERN_GOOD,PATTERN_BAD,DESCRIPTION
p0,0,1,All bad
p1,1,1,1 good 1 bad
p2,2,1,2 good 1 bad
p3,3,1,3 good 1 bad
p4,3,2,3 good 2 bad
p5,5,2,5 good 2 bad
p6,24,24,24 good 24 bad
p7,144,144,144 good 144 bad
p8,7,1,7 good 1 bad
p9,14,2,14 good 2 bad
p10,1,0,All good
TIMES
ID,INCREMENT,VARIANCE,DIRECTION,DESCRIPTION
t0,300000,0,0,5 minute time no variance
t1,300000,5000,0,5 minute time 5s variance
t2,300000,30000,0,5 minute time 30s variance
t3,300000,100000,0,5 minute time 1m variance
t3p,300000,0,0,5 minute time no variance
t3e,300000,100000,0,5 minute time 1m either way variance
t3a,300000,100000,1,5 minute time 1m always positive variance
t3b,300000,100000,-1,5 minute time 1m always negative variance
t4,3600000,0,0,1 hour time no variance
t5,3600000,60000,0,1 hour time 1m variance
t6,3600000,300000,0,1 hour time 5m variance
t7,3600000,600000,0,1 hour time 10m variance
t8,86400000,0,0,1 day time no variance
t9,86400000,3600000,0,1 day time 1h variance
t10,86400000,18000000,0,1 day time 5h variance
t11,86400000,36000000,0,1 day time 10h variance
DATA
ID,STRETCH_START,STRETCH_END,SLOPE,BUMP,FROM,TO,LIMIT_UPPER,LIMIT_LOWER,PERMA_BUMP_AT,PERMA_BUMP_BY,PERMA_BUMP_SMOOTHER,USE_RANDOM,RANDOM_SEED,RANDOM_BIAS,GENERATE_SPIKES,SPIKE_SUSTAIN,SPIKE_EVERY,SPIKE_TO,SPIKE_WOBBLE,SPIKE_WOBBLE_FACTOR,SPIKE_SMOOTHER,USE_SEASONALITY,SEASONALITY_WAVE1,SEASONALITY_WAVE2,SEASONALITY_WAVE3,SEASONALITY_WAVE4,SEASONALITY_WAVE5,DESCRIPTION
d0,1,1,0,0,-100,100,false,false,0,0,0,true,1,0.5,false,5,100,100,false,200,20,false,300,1,1,1,1,Random example
d1,1,1,0,0,-100,100,false,false,43,100,1,true,1,0.5,false,5,100,100,false,200,20,false,300,1,1,1,1,Random example with a permanent bump
d2,1,1,0,0,-100,100,false,false,0,0,0,false,1,0.5,false,5,100,100,false,200,20,true,300,1,1,1,1,Seasonality example
d3,1,1,0,0,-100,100,false,false,0,0,0,true,1,0.5,true,5,200,100,false,200,20,false,300,1,1,1,1,Random example with a spike
d4,1,1,0,100,-100,100,false,false,0,0,0,true,1,0.5,true,5,200,-100,false,200,20,false,300,1,1,1,1,Random example with a negative spike
d5,1,100,0,0,-100,100,false,false,0,0,0,false,1,0.5,false,5,100,100,false,200,20,true,300,1,1,1,1,Seasonality example with a stretch
d6,100,1,0,0,-100,100,false,false,0,0,0,false,1,0.5,false,5,100,100,false,200,20,true,300,1,1,1,1,Seasonality example with a squish

Printing the default template:
===========================

example,uuid={{ uuid }} [r5 random]({{ if $.r5.Val }}TRUE{{ else }}FALSE{{ end }}) [p2 Pattern]({{ $.p2.Val }}) [d0 Value]({{$.d0.Val}}) [d1 Value]({{$.d1.Val}}) [d2 Value]({{$.d2.Val}}) [d3 Value]({{$.d3.Val}}) [t3b Time Epoch]({{$.t3b.Val.Unix}})  [t3b Time]({{$.t3b.Val}})
```

### Help for "Generate"

```
[tgen]$ ./tgen help generate
Generates fake time series data based on configuration and templates

Usage:
  tgen generate [flags]

Flags:
  -c, --config string           Which config file to use?

                                You can generate a sample config and templates using the 'config' command.

                                The config file is a simple CSV file in the following format:

                                RANDOM
                                [RANDOM HEADER]
                                {RANDOM ENTRIES}
                                PATTERN
                                [PATTERN HEADER]
                                {PATTERN ENTRIES}
                                TIMES
                                [TIMES HEADER]
                                {TIMES ENTRIES}
                                DATA
                                [DATA HEADER]
                                {DATA ENTRIES}

                                Where:

                                 'RANDOM' indicates a the literal word RANDOM on its own line indicating that
                                         the lines that follow are loading random entries.

                                 'PATTERN' indicates a the literal word TIMES on its own line indicating that
                                         the lines that follow are loading pattern entries.

                                 'TIMES' indicates a the literal word TIMES on its own line indicating that the
                                         lines that follow are loading time entries.

                                 'DATA'  indicates a the literal word DATA on its own line indicating that the
                                         lines that follow are loading data entries.

                                RANDOM
                                ======

                                An example RANDOM entry may look like the following:

                                 r3,1,0.3,30% good

                                For 'RANDOM' the header/columns are as follows:

                                 ID
                                    An id for this entry so that you can refer to it in the template like this
                                    '$.r0.Val'

                                 SEED
                                    A seed to use for any random numbers used internally

                                 GOOD_PCT
                                    A percentage of samples that will return 'true'

                                 DESCRIPTION
                                    Optional description to help the user

                                PATTERN
                                =======

                                An example PATTERN entry may look like the following:

                                 p2,2,1,2 good 1 bad

                                For 'PATTERN' the header/columns are as follows:

                                 ID
                                    An id for this entry so that you can refer to it in the template like this
                                    '$.p0.Val'

                                 PATTERN_GOOD
                                    Starting at 0, how many good samples to generate

                                 PATTERN_BAD
                                    After the good samples, how many "bad" sampled to generate.  For example if
                                    you want every third sample to be "bad" you would set PATTERN_GOOD to 2 and
                                    PATTERN_BAD to 1. You can then use it in the template like this

                                          '{{- if $.p0.Val -}}'

                                    to determine whether this is a 'true' value.

                                 DESCRIPTION
                                    Optional description to help the user

                                TIMES
                                =====

                                An example TIME entry may look like the following:

                                 t3e,300000,100000,0,5 minute time 1m either way variance

                                For 'TIMES' the header/columns are as follows:

                                 ID
                                    An id for this entry so that you can refer to it in the template like this
                                    '$.t3e.Val'

                                 INCREMENT
                                    Increment (in milliseconds) for the next entry

                                 VARIANCE
                                	When the next value is calculated, you can use a variance to add/subtract
                                	up to the amount of milliseconds to that time. If you want perfect times
                                	set this to 0.

                                 DIRECTION
                                	When the next value is calculated, you can use a value less than 0 to
                                	indicate the variance will always subtract from the intended time, a value
                                	of 0 to either subtract or add at a 50% chance or a value above 0 to always
                                	add to the intended time.

                                	In other words if time is 12:00:00PM, variance is 5 seconds and direction
                                	-1 this could yield 11:59:57, but never more than 12:00:00PM. Similarly, if
                                	direction is 1 it could yield 12:00:03 but never less than 12:00:00PM. And
                                	finally, if direction is 0 then either case is equally probable.

                                 DESCRIPTION
                                    Optional description to help the user

                                DATA
                                ====

                                An example DATA entry may look like the following:

                                 d2,1,1,0,0,-100,100,false,false,43,100,1,true,1,0.5,false,5,100,100,false,200,20,false,300,1,1,1,1,0 to 100 soft lmit upward slope likely to breach max

                                For 'DATA' the header/columns are as follows:

                                 ID
                                    An id for this entry so that you can refer to it in the template like this
                                    '$.d2.Val'

                                 STRETCH_START
                                 STRETCH_END
                                	When a default graph is generated this value "stretches" or "squishes" the
                                	data up and down. For example if the generated data has a minimum of 35 and
                                	a maximum of 60, "stretching" it will increase the difference between the
                                	minumum and maximum while "squishing" will decrease it.

                                	A value above 1 will stretch, a value below 1 but more than 0 will "squish"
                                	while a value of 1 will be "normal".

                                	STRETCH_START and STRETCH_END values indicates the start and stop values.
                                	This is useful if you want to "amplify over time" or "dampen over time".

                                	For example if STRETCH_START is 100, STRETCH_END is 0 and we want 100
                                	samples then we will gradually reduce the stretch from 100 to 0 over 100
                                	linear steps (i.e. the stretch value at the 50th sample will be 50).

                                 SLOPE
                                	This will set the slope of the generated data. 0 means whatever the
                                	underlying data is. Positive values means data will trend upwards, negative
                                	values mean data will trend downward.

                                 BUMP
                                	Tweaking this number will "bump" the random value up (or down if negative).
                                	For example if a maximum is 60, a positive "bump" will increase this value,
                                	a negative "bump" will decrease it. This is useful if you want to ensure
                                	values breach min/max in certain cases.

                                 FROM
                                    Generate data "from" this number. E.g. CPU values may be from 0 to 100.

                                 TO
                                    Generate data "to" this number. E.g. CPU values may be from 0 to 100.

                                 LIMIT_UPPER
                                    If numbers go above "to", when this is "TRUE" values will be set to the
                                    "to" value. E.g. a CPU cannot go above 100%.

                                 LIMIT_LOWER
                                    If numbers go below "from", when this is "TRUE" values will be set to the
                                    "from" value. E.g. a CPU cannot go below 0%.

                                 PERMA_BUMP_AT
                                	Generate data "from" this number. E.g. CPU values may be from 0 to 100. Use
                                	0 to disable permanent bump.

                                 PERMA_BUMP_BY
                                	What value should we bump to, expressed as a percentage of "to". Can be
                                	negative too.

                                 PERMA_BUMP_SMOOTHER
                                	When smoother is 1 we will go straight from current value to bump value.
                                	When smoother is above 1 when we will take this number of samples to reach
                                	the bump value (i.e. gradually but quickly rise to a spike). Use 0 to
                                	disable permanent bump.

                                 USE_RANDOM
                                    Whether to generate random numbers. When "TRUE" numbers will be generated
                                    based on the below parameters.

                                 RANDOM_SEED
                                    A seed to use for any random numbers used internally

                                 RANDOM_BIAS
                                    A parameter between 0 and 1. Use it to control the range and slope of your
                                    data. Generally lower biases make the slope negative but which values
                                	entirely depend on the underlying dataset. As always, have a play to
                                    see the effects.

                                 GENERATE_SPIKES
                                    Whether to generate spikes. When "TRUE" spikes will be generated based on
                                    the below parameters.

                                 SPIKE_SUSTAIN
                                    When a spike is reached, for how many samples should we sustain it.

                                 SPIKE_EVERY
                                    Starting at 0, every n samples will reach the desired spike value.

                                 SPIKE_TO
                                    What value should we spike to, expressed as a percentage of "to".

                                 SPIKE_WOBBLE
                                    When we're spiking and sustaining it, do we use a flat value at the top or
                                    do we "bounce off the top" a little for a little variation? When "TRUE" it
                                	means that when sustaining we'll add a little variance. We respect and will
                                	never break the SPIKE_TO value though!

                                 SPIKE_WOBBLE_FACTOR
                                    A magic number to tweak the "wobbliness". Have a play with this value.
                                	Generally a higher value will mean smoother values.

                                 SPIKE_SMOOTHER
                                    When smoother is 0 we will go straight from current value to spike value.
                                    When smoother is above 0 when we will take this number of samples to reach
                                    the spike value (i.e. gradually but quickly rise to a spike)

                                 USE_SEASONALITY
                                    Whether to generate seasonality using SIN. When "TRUE" waves will be
                                    generated using the WAVE parameters below.

                                 SEASONALITY_WAVE[1-5]
                                    Indicates number of points where one SIN cycle will be complete.  Each wave
                                    is summed to generate interference.

                                 DESCRIPTION
                                    Optional, but highly desirable, description to help the user.

                                You can use the 'playarea' command to play with the values and generate the
                                corresponding DATA entries.

                                Any row starting with a '#' will be treated as a comment.
  -f, --fromTime string         Date and/or time to use as a starting point for *all* time generation.
                                Uses Go date format, see https://golang.org/pkg/time/#Parse. (default "now()")
  -d, --fromTimeFormat string   Date format to use for the 'from' parameter.
                                The default format is the output of the 'date' command. (default "Mon Jan 2 15:04:05 MST 2006")
  -z, --fromTimeZone string     Timezone to use.
                                See https://en.wikipedia.org/wiki/List_of_tz_database_time_zones.
  -h, --help                    help for generate
  -j, --jitter int              What "jitter" to add to the rate in milliseconds.
                                At random a number between 0 and value submitted will be added or subtracted at
                                random for a variance in rate of output. For example, if rate is 100 and jitter
                                is 20 actual rate will be between 80 and 120.
  -o, --offset int              Generates data from a particular starting point. Used for continuing runs.
  -p, --out string              Print to file. Use '-' for STDOUT. (default "-")
  -r, --rate int                What delay in milliseconds to use between each sample.
                                This is useful if you need to run this continuously but slowly as opposed to
                                one large dump.
  -n, --samples int             Number of samples to generate. (default 10)
  -m, --stats int               Print the stats of the data in
                                the input config not the numbers. This is useful if you want to get a feel
                                for what the data will look like.
  -s, --stream                  Continuously generate data.
  -t, --template string         Template to use.
                                Uses the Go template language (See https://golang.org/pkg/text/template/)
```

### Help for "Playarea"

```
[tgen]$ ./tgen help playarea
Starts a webserver which you can use to tweak parameters and see the effect
they have on the random data that will be generated by the 'generate' command

Usage:
  tgen playarea [flags]

Flags:
  -h, --help       help for playarea
  -p, --port int   Port to use for playarea server. (default 8080)
```

Once you open your browser you will be greeted with a page that looks like this:

![Playarea screencap](playarea-screenshot.png)

A word of warning though, the page has been awarded "best webpage design" in
the 1997 international web design awards.

repository: powerpu/tgen

repo_name: powerpu/tgen


text:
# Example DSF project

## code
This code example is an blank DSF file from: https://github.com/daad-adventure-writer/DRC

## visual code settings.json

    {
        "editor.renderWhitespace": "boundary",
        "[sce]": {
            "files.encoding": "cp437"
        },
        "[dsf]": {
            "files.encoding": "iso88591"
        },
        "files.associations": {
            "*.dsf": "dsf",
            "*.sce": "sce"
        },
        "workbench.colorTheme": "Outrun Electric",
    }

Do you want learn more? https://medium.com/@uto_dev/a-daad-tutorial-for-beginners-1-b2568ec4df05

license: Unlicense

branch_name: refs/heads/master

extension: md

license_type: permissive


text:
# barbelhisto
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Build Status](https://app.travis-ci.com/projectbarbel/barbelhisto-core.svg?branch=master)](https://app.travis-ci.com/projectbarbel/barbelhisto-core)
[![Maven Central](https://img.shields.io/maven-central/v/org.projectbarbel/barbelhisto.svg)](https://search.maven.org/search?q=barbelhisto)

A lightweiht easy to use Java library to store the history of changes of domain objects in bi-temporal format.

`BarbelHisto` tracks two time dimensions for you: **record time** is when a change to a domain object was recorded in the system (created and inactivated) and **effective time** is when this change is opposed to become effective or valid from a business viewpoint.

The library implements Martin Fowlers Temporal Pattern that can be found here: https://martinfowler.com/eaaDev/timeNarrative.html

# Features

- bi-temporal (auditing-proof) data storage based on blazing fast [cqengine](https://github.com/npgall/cqengine) collections
- small easy-to-use API (as simple as bi-temporal data storage can get)
- backed by transactional, high performance collections
- works out of the box with default implementation and configuration
- manage different time zones of client and server
- persistence support for heap, off-heap, disk storage and custom data stores
- provides basic functionality, extendible for individual needs
- remember every change to the data, nothing is ever going to be deleted
- build-in thread safety by minimizing mutability and object locking
- pretty print function to learn about bitemporal data storage

# Get a version of BarbelHisto
Get a version on [Maven Central](https://search.maven.org/search?q=barbelhisto).

Download the [actual snapshot releases](https://github.com/projectbarbel/barbelhisto-core/releases).

Or clone this git repository to try some of the features of `BarbelHisto`.

# Getting started with POJOs
Get started turorial on [http://www.projectbarbel.org/docs/getstarted](http://www.projectbarbel.org/docs/getstarted).
# Memory analysis
Memory analysis results. One hour run, 5000 Objects saved every 10 seconds. `OnHeapPersistence`.
<br>Result details [here](https://github.com/projectbarbel/barbelhisto-core/blob/master/performance/BarbelCoreSaveMemoryTest.txt).
![Memory](https://user-images.githubusercontent.com/876604/53019849-fb084b00-3455-11e9-8625-6c2bf1be1576.png)

license: MIT

repository: projectbarbel/barbelhisto-core

gha_language: Java

gha_license_id: MIT

repo_name: projectbarbel/barbelhisto-core


text:
# Deploying XMiDT

## Docker/awscli

In order to deploy into Docker, make sure [Docker is installed](https://docs.docker.com/install/). Additionally for goaws, make sure that awscli is intalled.

#### Deploy
_**Note**_: While Tr1d1um is not part of XMiDT(it is WebPA), it is recommended to be
brought up for current ease of use. Future releases will deprecate Tr1d1um.

1. Clone this repository

2. Run `deploy/docker-compose/deploy.sh`

    This will build `simulator` and `goaws` locally. It will then run `docker-compose up` which uses images of `talaria`, `scytale`, `petasos`, `caduceus`, and `tr1d1um` from dockerhub.

    To pull specific versions of the images, just set the `<SERVICE>_VERSION` env variables when running the shell script.

    ```
    TALARIA_VERSION=x.x.x deploy/docker-compose/deploy.sh
    ```

    If you only want to bring up, for example, the scytale and talaria, run:
    ```bash
    deploy/docker-compose/deploy.sh scytale talaria
    ```
    _**Note**_: Bringing up a subset of services can cause problems.

    This can be done with any combination of services.

3. To bring the containers down:
   ```bash
   docker-compose -f docker-compose.yml down
   ```

### INFO
The docker-compose file provides 1 full datacenter with one talaria in a "backup"
datacenter. Since this is pure docker, not swarm or kubernetes, it's easiest to
deal with just one datacenter. Since all ports are exposed, the names might seem a little weird.

#### Connection
##### Inside Docker
If the Parodus instance is inside of docker, life is easy! Just connect to the cluster with `petasos:6400`.

##### Outside Docker
if the Parodus instance is outside of docker and the ports are exposed correctly, life
will be hard since you will need to handle the redirect.
You can initially connect to 'localhost:6400' but on the redirect change `talaria-1:6210` to `localhost:6210`

or you can just connect to a talaria `localhost:6200`

Once connected you should see it connected via [metrics](http://localhost:9090/graph?g0.range_input=1h&g0.expr=xmidt_talaria_device_count&g0.tab=0)

### Interact with the machines

Checkout that petasos is working:
```
curl -i -H "X-Webpa-Device-Name: mac:112233445566" "http://localhost:6400"
```

Should give you the following:
```
HTTP/1.1 307 Temporary Redirect
Content-Type: text/html; charset=utf-8
Location: http://talaria-0:6200
X-Petasos-Build: development
X-Petasos-Flavor: development
X-Petasos-Region: local
X-Petasos-Server: localhost
X-Petasos-Start-Time: 04 Jun 19 02:12 UTC
Date: Tue, 04 Jun 2019 02:16:58 GMT
Content-Length: 57

<a href="http://talaria-0:6200">Temporary Redirect</a>.
```

Checkout that tr1d1um is able to talk with scytale & talaria:
```
curl -i -H "Authorization: Basic dXNlcjpwYXNz" "http://localhost:6100/api/v2/device/mac:112233445577/config?names=Foo"
```

Should give you:
```
HTTP/1.1 404 Not Found
X-Scytale-Build: development
X-Scytale-Flavor: development
X-Scytale-Region: local
X-Scytale-Server: localhost
X-Scytale-Start-Time: 04 Jun 19 02:12 UTC
X-Talaria-Build: development
X-Talaria-Flavor: development
X-Talaria-Region: local
X-Talaria-Server: localhost
X-Talaria-Start-Time: 04 Jun 19 02:12 UTC
X-Tr1d1um-Build: development
X-Tr1d1um-Flavor: development
X-Tr1d1um-Region: local
X-Tr1d1um-Server: localhost
X-Tr1d1um-Start-Time: 04 Jun 19 02:11 UTC
X-Webpa-Transaction-Id: LQxoB5sUSGWPNgAzxRIXLA
X-Xmidt-Message-Error: The device does not exist
X-Xmidt-Span: "http://petasos:6400/api/v2/device/send","2019-06-04T02:27:26Z","2.185274ms"
Date: Tue, 04 Jun 2019 02:27:26 GMT
Content-Length: 87
Content-Type: text/plain; charset=utf-8

{"code": 404, "message": "Could not process device request: The device does not exist"}
```

Check out that your simulator is connected:

```
curl -i -H "Authorization: Basic dXNlcjpwYXNz" "http://localhost:6100/api/v2/device/mac:112233445566/stat"
```

Should give you something similar to:

```
HTTP/1.1 200 OK
Content-Type: application/json
X-Scytale-Build: development
X-Scytale-Flavor: development
X-Scytale-Region: local
X-Scytale-Server: localhost
X-Scytale-Start-Time: 10 Jun 19 06:36 UTC
X-Talaria-Build: development
X-Talaria-Flavor: development
X-Talaria-Region: local
X-Talaria-Server: localhost
X-Talaria-Start-Time: 10 Jun 19 06:36 UTC
X-Tr1d1um-Build: development
X-Tr1d1um-Flavor: development
X-Tr1d1um-Region: local
X-Tr1d1um-Server: localhost
X-Tr1d1um-Start-Time: 10 Jun 19 06:36 UTC
X-Webpa-Transaction-Id: CIyqnI23RjWhyC_vNO7hbA
X-Xmidt-Span: "http://petasos:6400/api/v2/device/mac:112233445566/stat","2019-06-10T06:38:13Z","2.947332ms"
Date: Mon, 10 Jun 2019 06:38:13 GMT
Content-Length: 231

{"id": "mac:112233445566", "pending": 0, "statistics": {"bytesSent": 0, "messagesSent": 0, "bytesReceived": 0, "messagesReceived": 0, "duplications": 0, "connectedAt": "2019-06-10T06:37:02.915435853Z", "upTime": "1m10.110197482s"}}
```

Read a single parameter:

```
curl -i -H "Authorization: Basic dXNlcjpwYXNz" "http://localhost:6100/api/v2/device/mac:112233445566/config?names=Device.DeviceInfo.X_CISCO_COM_BootloaderVersion" -H "X-Xmidt-Partner-ID: comcast,nbc,sky"
```

Results in:
```
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
X-Scytale-Build: development
X-Scytale-Flavor: development
X-Scytale-Region: local
X-Scytale-Server: localhost
X-Scytale-Start-Time: 10 Jun 19 06:36 UTC
X-Talaria-Build: development
X-Talaria-Flavor: development
X-Talaria-Region: local
X-Talaria-Server: localhost
X-Talaria-Start-Time: 10 Jun 19 06:36 UTC
X-Tr1d1um-Build: development
X-Tr1d1um-Flavor: development
X-Tr1d1um-Region: local
X-Tr1d1um-Server: localhost
X-Tr1d1um-Start-Time: 10 Jun 19 06:36 UTC
X-Webpa-Transaction-Id: cDYIIKLgoDtrt3XrVfUKkg
X-Xmidt-Span: "http://petasos:6400/api/v2/device/send","2019-06-10T06:45:04Z","15.869854ms"
Date: Mon, 10 Jun 2019 06:45:04 GMT
Content-Length: 163

{"parameters":[{"name":"Device.DeviceInfo.X_CISCO_COM_BootloaderVersion","value":"<IP_ADDRESS>","dataType":0,"parameterCount":1,"message":"Success"}],"statusCode":200}
```

repository: xmidt-org/xmidt

gha_language: Shell

repo_name: xmidt-org/xmidt


text:
---
layout: post
title:      "Binding and this"
date:       2020-11-24 12:16:45 -0500
permalink:  binding_and_this
---

One tricky concept when learning programming is the concept of "this". It can sometimes be difficult to determine the context of "this" and what it is referring to within functions and within our applications. Examining where the function is called can be very helpful in determining "this". Here are the basics:

* When JavaScript function is called, it has its own associated object of an execution context that can be accessed by "this"
* When "this" is used inside a function, it is representing the execution context
* "This" can either be implicit, meaning the code does not specifically say what "this" is, or it can be explicit, meaning the code specifically tells the function what context "this" should refer to

### Implicit Binding
Implicit binding happens when dot notation is used to call a function. Whatever is to the left of the dot when it is called is the  provided context of the function. Let's break down the code below:

```

let hailey = {
name:  "Hailey,
sayName: function (){
console.log(`My Name is ${this.name}`)
    }
}

hailey.sayName()

*Output*
My Name is Hailey

```

In this example, I am creating an object called 'hailey', which contains a name and a function called sayName. Inside the sayName function, I am making use of "this". After I establish my object, I then call the sayName function on the object 'hailey'. Since 'hailey' is to the left of the dot, it implicitly becomes the execution context and is referred to when "this" is used in the sayName function.

As a general rule, if this is used implicitly, the context can be determined by looking at what is to the left of the dot where the function that contains this was invoked.

### Explicit Binding
In some cases, we may want to explicitly tell a function what context we want it called on. The first option is to use .call() or .apply() on a function to tell it exactly what we want passed in. For example:

```

let hailey = {
name:  "Hailey

}

function sayName(){
console.log(`My Name is ${this.name}`)
    }

sayName.call(hailey)

*Output*
My Name is Hailey

```

Both .call() and .apply() take a the first argument passed and make it the execution context, or the "this" inside the function that is being called.

If we want a more permanent explicit solution, we can use .bind(). Bind creates a copy of the original function with the execution context locked in, so whenever the copy is called, "this" is pre-defined. For example:

```

let hailey = {
name:  "Hailey

}

function sayName(){
console.log(`My Name is ${this.name}`)
    }

let haileyGreeting = sayName.bind(hailey)

haileyGreeting()

*Output*
My Name is Hailey

```

### "this" and ES6
Something to be mindful of when using "this" is what notation the function is being written in. in ES6, arrow functions allow us to simplify some code. But it is important to remember that arrow functions are automatically bound to it’s parent's context and do not have their own "this" context, as do traditional functions. When used inside an arrow function, "this" refers to the enclosing scope.  Example below:

```

let greeting = {

type: "Hello",
names: [
"Hailey",
"Lisa",
"Rich",
"Greg"
]}

let sayHello = function () {
this.names.forEach(name => console.log (`${this.type} ${name}`))
}

sayHello.call(greeting)

*Output*
Hello Hailey
Hello Lisa
Hello Rich
Hello Greg

```

Here we can see that our forEach function is using an arrow function within that refers to "this". Rather than "this" referring to the scope of its own function, the forEach, we can see that it is referring to the "this" defined in it's enclosing scope that we called the function on - the greeting object.

Hopefully this post was helpful in determining the scope of "this" in code. In summary, it is best to look at where the function is being called to determine if it is implicit binding, explicit binding, or used in an arrow function.

extension: markdown


text:
---
title: Mollitia - API - Module - Bulkhead
---
# Bulkhead

<pg-bulkhead></pg-bulkhead>

## Usage

The `Bulkhead` module allows you to **limit** concurrent executions of your circuit.

``` javascript
// Imports needed components
const { Circuit, Bulkhead, BulkheadOverloadError, BulkheadQueueWaitError } = require('mollitia');
// Creates a circuit
const circuit = new Circuit({
  options: {
    modules: [
      // Creates a bulkhead module
      new Bulkhead({
        concurrentSize: 2, // Allows 2 concurrent requests, if oversizing, goes in a queue.
        queueSize: 2, // Allows 2 requests to be in queue, if oversizing, it will be rejected with a BulkheadOverloadError.
        maxQueueWait: 30000 // After 30 seconds waiting, a queued request will be rejected with a BulkheadQueueWaitError.
      })
    ]
  }
});

// Let's say this function is running multiple times in a relatively short amount of time
circuit.fn(myFunction).execute()
  .then(() => {
    // The succeed normally (directly, or has been in a queue)
  })
  .catch((err) => {
    if (err instanceof BulkheadOverloadError) {
      // When the function has been called, the concurrent methods are at maximum, and the queue is full.
    } else if (err instanceof BulkheadQueueWaitError) {
      // The function has been waiting too long in queue (more than 30 seconds).
    }
    // It failed normally (directly, or has been in a queue)
  });
```

## Options

| Name             | Description                                                        | Default |
|:-----------------|:-------------------------------------------------------------------|:--------|
| `concurrentSize` | The number of concurrent requests that can be running in parallel. | `10`    |
| `queueSize`      | The number of requests that can be queued.                         | `10`    |
| `maxQueueWait`   | The amount of time before a queued request is rejected.            | `60000` |

## Events

| Name       | Description                          | Params                         |
|:-----------|:-------------------------------------|:-------------------------------|
| `execute`  | Called when the module is executed.  | `Mollitia.Circuit` **circuit** |

## Methods

| Name       | Description                          | Returns                         |
|:-----------|:-------------------------------------|:-------------------------------|
| `getExecParams`  | Returns the circuit function parameters.  | `any[]` **params** |

repository: vspirkin/mollitia

repo_name: vspirkin/mollitia


text:
# bomweather

[![PyPI version](https://badge.fury.io/py/bomweather.svg)](https://badge.fury.io/py/bomweather) [![Build Status](https://travis-ci.org/aguinane/bomweather.svg?branch=master)](https://travis-ci.org/aguinane/bomweather) [![Coverage Status](https://coveralls.io/repos/github/aguinane/bomweather/badge.svg)](https://coveralls.io/github/aguinane/bomweather)

Load weather data from the Australian Bureau of Meteorology (BOM) website.

# Find closest observation site to a location

```python
> from bomweather import closest_obs_station
> station = closest_obs_station(lat=-27.470125, lon=153.021072)
> print(station['site_name'], station['product'], station['wmo'])
BRISBANE IDQ60801 94576
```

# Get latest observation data

```python
> from bomweather import ObservationSite
> obs = ObservationSite(wmo=94576, product='IDQ60801')
> last_obs = obs.last_observation()
Observation(wmo=94576, name='Brisbane', history_product='IDQ60801', local_dt=datetime.datetime(2019, 1, 31, 6, 30), utc_dt=datetime.datetime(2019, 1, 30, 20, 30, tzinfo=<UTC>), lat=-27.5, lon=153.0, apparent_t=28.4, cloud=None, cloud_base_m=None, cloud_oktas=None, cloud_type=None, delta_t=2.2, gust_kmh=7, gust_kt=4, air_temp=24.5, dewpt=21.2, press=1014.3, press_msl=1014.3, press_qnh=1014.3, rain_trace=0.0, rel_hum=82, vis_km=None, weather=None, wind_dir='S', wind_spd_kmh=2, wind_spd_kt=1)
```

# Find closest forecast product to a location

```python
> from bomweather import closest_forecast_location
> station = closest_forecast_location(lat=-27.470125, lon=153.021072)
> print(station['site_name'], station['product'])
oxley IDQ10095
```

# Get forecast data

```python
> from bomweather import Forecast
> f = Forecast(product='IDQ10095')
> print(f)
<Forecast IDQ10095 Brisbane - Issued 2019-02-02T16:37:01+10:00>
> print(f.forecasts[-1])
ForecastPeriod(start=datetime.datetime(2019, 2, 8, 14, 0, tzinfo=<UTC>), end=datetime.datetime(2019, 2, 9, 14, 0, tzinfo=<UTC>), forecast_icon='17', forecast_text='Partly cloudy. Medium (40%) chance of showers, most likely later in the day. Light winds becoming northeasterly 15 to 25 km/h during the day.', temp_max='33', temp_min='22', precis='Possible shower.', precis_prob='40%', precis_range='0 to 4 mm')
```

repository: aguinane/bomweather

gha_language: Python

repo_name: aguinane/bomweather


text:
# DriveDrop
Create an embeddable form that can be used to allow anyone to upload files to your Google Drive

You can create a copy of DriveDrop by clicking below
## [- Click Here To Clone DriveDrop -](https://script.google.com/d/1mA2_OMh2Fdt9qTg2MYr5NIy_QtYbE_wAtecHt8NgBvtWC5lVCBT-sSha/edit?newcopy=true)

## Instructions
1. Clone the code
2. Select 'Publish → Deploy as web app'
3. Set 'Execute the app as: Me'
4. Set 'Who has access to the app: Anyone, even anonymous'
5. Click 'Update'

From here you will get a prompt that says
```
Authorization required

DriveDrop by mholgatem (Master) needs your permission to access your data on Google.
```
1. Click 'Review Permissions'
2. Select the account that you want files to upload to
3. You will get a scary looking warning because you have not submitted this app to be verified by Google
4. Click advanced

5. Click 'Go to DriveDrop by mholgatem (Master) (unsafe)'
6. This will show you the consent screen, review the permissions, then click 'Allow'

Congratulations! Your form is now live! Anyone with the link can now upload files to your Google drive!

If you want to embed your new form, you need to make a few changes to the code:
1. Open server.gs
2. Find the Config section for the picker
```
  PICKER :
  {
    embedded : false, //set to true if you want to embed in an Iframe
    URL      : "https://script.google.com", // default URL
    embedURL : "http://your.website.com" // URL to use if embedded
  }
```
3. set embedded to 'true'
4. set embedURL to the URL of your site

repository: mholgatem/drivedrop

repo_name: mholgatem/drivedrop


text:
# Currier

[![Build Status](https://travis-ci.org/tigerpixel/Currier.svg?branch=master)](https://travis-ci.org/tigerpixel/Currier)
[![Version](https://img.shields.io/cocoapods/v/Currier.svg?style=flat)](http://cocoapods.org/pods/Currier)
[![Platform](https://img.shields.io/cocoapods/p/Currier.svg?style=flat)](http://cocoapods.org/pods/Currier)
[![Carthage compatible](https://img.shields.io/badge/Carthage-compatible-4BC51D.svg?style=flat)](https://github.com/Carthage/Carthage)
[![License](https://img.shields.io/cocoapods/l/Currier.svg?style=flat)](http://cocoapods.org/pods/Currier)

Currier transforms any Swift function into a curried version of that function. This is achieved by wrapping it in a simple 'curry' call.

Given the following function:

```swift
func myMultiply(first: Float, second: Int) -> Double {
    return Double(first) * Double(second)
}
```

Currier will produce a curried version:

```swift
let curriedMultiply: (Float) -> (Int) -> Double = curry(myMultiply) // Currier used here!

let multiplyByTwo: (Int) -> Double = curriedMultiply(2.0) // Partial application

let result: Double = multiplyByTwo(10)

assert(20 == result) // Assertion true

```

The previous example can be condensed down to the following:

```swift
let sameResult = curry(myMultiply)(2.0)(10)
```

The unit tests provide examples of calling the curry function using different numbers of parameters. The version of the function which takes two parameters is explicitly typed and commented.

## Requirements

There are no external requirements for this project, just Swift.

- iOS 8.0+ / macOS 10.9+ / tvOS 9.0+ / watchOS 2.0+
- Xcode 11+
- Swift 5.1+

For older versions of Swift and Xcode, please see prior releases.

## Installation

### Swift Package Manager

The [Swift Package Manager](https://swift.org/package-manager) is the official tool for managing the distribution of Swift code. It is currently only available for all Apple platforms. It can also be used with Linux but this project does not fully support that at this point.

If you use it to manage your dependencies, simply add Currier to the dependencies value of your Package.swift file.

```swift
dependencies: [
.package(url: "https://github.com/tigerpixel/Currier.git", from: "1.3.0")
]
```

### Cocoapods

Currier is available through [CocoaPods](https://cocoapods.org/pods/Currier). To install it, simply add the following line to your Podfile:

```ruby
pod "Currier"
```

### Carthage

If you use [Carthage](https://github.com/Carthage/Carthage) to manage your dependencies, simply add the following line to your Cartfile:

```ogdl
github "tigerpixel/Currier"
```

If you use Carthage to build your dependencies, make sure you have added `Currier.framework` to the "_Linked Frameworks and Libraries_" section of your target, and have included them in your Carthage framework copying build phase.

### Git Submodule

1. Add the Currier repository as a [submodule](https://git-scm.com/book/en/v2/Git-Tools-Submodules) of your application’s repository.
1. Run `git submodule update --init --recursive` from within the Currier folder.
1. Drag and drop `Currier.xcodeproj` into your application’s Xcode project or workspace.
1. On the “General” tab of your application target’s settings, add `Currier.framework`. to the “Embedded Binaries” section.
1. If your application target does not contain Swift code at all, you should also set the `EMBEDDED_CONTENT_CONTAINS_SWIFT` build setting to “Yes”.

## MIT License

Currier is available under the MIT license. Details can be found within the LICENSE file.

repository: tigerpixel/Currier

gha_language: Swift

repo_name: tigerpixel/Currier


text:
# Sr. Software Engineer - Services

at FanTank

Los Angeles, CA. - USA

We're looking for an experienced platform engineer who is passionate about designing, creating and delivering highly scalable distributed services. You will be part of a small team responsible for building and maintaining systems, which touch nearly every aspect of FanTank's user functionality and internal infrastructure.

You will play a key role in both design and implementation of services to drive new product features, and improve performance and scalability of the site. We put a strong emphasis on individual ownership and value engineers who take pride in working over the full lifecycle of a project.

**What You'll Do:**

- Design and develop high-performance distributed services using the latest tools & frameworks
- Research, prototype and build solutions to unique & challenging problems
- Contribute to Open Source software projects ( [https://github.com/tumblr/colossus](https://github.com/tumblr/colossus))
- Continually look for opportunities to generalize, simplify and reuse code
- Work with product, marketing and other engineering teams to build a better experience for our users
- Collaborate with other team members on design, architecture and code reviews
- Provide expertise and advice for teams across the company
- Contribute to refining and improving our processes around service development and deployment as FanTank becomes more service oriented

**What We're Looking For:**

- 4-5 years of relevant industry experience
- A solid background in Computer Science fundamentals
- Experience successfully managing large amounts of data and/or high throughput on distributed systems
- Professional experience with low-level optimizing, tuning and debugging for performance

- Significant professional development experience in Scala, or a desire and appropriate background to learn it
- Java/C++/PHP/Go experience is a bonus as we often work on other areas of FanTank's stack

**Tools We Like:**

- Scala
- Colossus, Akka, Slick, Play, SBT
- MySQL, Memcached, Redis, Kafka
- Puppet, Docker, GitHub, Jira

repository: Hakaze/crowdfund-example-app

repo_name: Hakaze/crowdfund-example-app


text:
# Importing documents

Importing an existing document to GitBook is easy. Upload it when creating a book using the `Import` tab.

### Accepted formats

| Type | Extension |
| ---- | --------- |
| Microsoft Word Document | `.docx` |
| DocBook v5.x | `.xml` |
| HTML file | `.html` |

To use this feature as its most, we recommend using:
* Microsoft Word 2007+ documents
* DocBook v5.x documents

If you have existing DocBook documents in version 4, consider [converting it automatically](http://doccookbook.sourceforge.net/html/en/dbc.structure.db4-to-db5.html) to version 5 for optimized compatibility.

### Conversion

The import feature uses the [gitbook-convert](https://github.com/GitbookIO/gitbook-convert) CLI utility. This module is in charge of every aspect of the conversion of the original document to markdown files, along with the creation of the `SUMMARY.md`.

[gitbook-convert](https://github.com/GitbookIO/gitbook-convert) divide a big document into chapters and sub-chapters, based on the document's structure. Thus, each first-level header of the original document will be converted to a chapter. If a chapter contains second-level headers, a folder instead of a new file is created for this chapter. This folder will contain each sub-chapters.

In a chapter's folder, a `README.md` file containing the chapter preface is created. The whole content before the first second-level header is considered to be a chapter preface. The same rule applies for the whole content before the first first-level header which is used as the book's preface.

For `.docx` documents, [gitbook-convert](https://github.com/GitbookIO/gitbook-convert) will also export all included images into the `assets/` folder.

If you ever need more flexibility, consider using [gitbook-convert](https://github.com/GitbookIO/gitbook-convert) locally, create a repository and import it using Git or GitHub.

### Troubleshooting

We're always happy to help out with your books or any other questions you might have. You can ask a question or signal an issue on the following form at [github.com/GitbookIO/gitbook-convert](https://github.com/GitbookIO/gitbook-convert/issues).

repository: SummerRC/documentation

repo_name: SummerRC/documentation


text:
# Thumbnailer sample

A .NET core application showing how to use the TakeSnapshot feature to get images from a video. This is not optimized at all, since you must play the whole video in order to get all the frames.

Please note : you can't use this method to get all frames from the video, as the `TimeChanged` event can be triggered very infrequently.
It's more a way to extract some pictures from the video. If you really need each frame, you should try `SetVideoFormatCallbacks`, as done in Vlc.DotNet.Wpf ( /!\ Expect performance impact )

branch_name: refs/heads/develop


text:
---
layout: post
title: "Circle CI + jekyll in github"
description: "Use Circle CI to deploy jekyll build result to another repo in github"
tags: [web, note]
---
## Purpose
The reason why we need to cope with this is that github doesn't support all jekyll plugins. Therefore when you upload jekyll projects with something like [Jekyll assets](https://github.com/jekyll/jekyll-assets) tags, github would show build fail, and won't create any page. To handle this problem, github recommends users to build static pages on other machines, and push them to the repository. The build script in [chalk](https://github.com/nielsenramon/chalk) pushes static files in gh-page branch, and ask users to choose the shown branch in repository settings. Basically this is a good idea, but due to the bug in github (users cannot choose master or gh-page in settings when build fail), it cannot work anyway.

## Solution
To deal with it, the only solution is to create another repository to hold the jekyll source, and use deploy script to push the built files to username.github.io.  The service I have chosen is Circle CI, but the basic concept and steps should be the same for others:

1. Create a new key for the repository you're going to build on, and link it to the CI service you use. You could choose Circle CI from the list of services already integrated with github in repository settings.

2. Create a new key for the repository you're going to deploy to, or directly use the user key to link to the CI project we have set in [step 1](#1). Notice you cannot use the same key created in [step 1](#1) since github forbids it. To add new keys in Circle CI please follow this [link](https://circleci.com/gh/zenixls2/zenixls2/edit#ssh).

3. Create a circle.yml in your repository root. A simple example may look like:

   ```yaml
   machine:
     ruby:
       version: 2.4.0

   dependencies:
     pre:
       - ./bin/prebuild_environment_script.sh

   test:
     post:
       - ./bin/build_and_deploy_script.sh
       - any thing else
   ```

4. In your `./bin/build_and_deploy_script.sh`, you have to call `jekyll build` to create static htmls in `_site` directly, and push this folder to the target repository. Notice in [step 2](#2) we already imported custom keys, so we could ignore the passphase or any ssh configuration things in containers when using git push. One sample script may look like:

   ```bash
   #!/bin/bash
   bower install
   bundle exec jekyll build

   cd _site
   git init
   git remote add origin git@github.com:username/username.github.io.git

   git config user.email "user_email_address"
   git config user.name "username"
   git add -fA
   git commit --allow-empty -m "$(git log -1 --pretty=%B) [ci skip]"
   git push -f -q origin master

   cd ..
   rm -rf _site
   exit 0
   ```

5. Almost done. Just push your changes to your jekyll source repository and wait for the auto-build invokation in CI.


text:
paradigms
=========

This directory contains the data and code for calculating *average conditional entropy* as described in:

* Ackerman, Farrell, James P. Blevins, & Robert Malouf. 2009. "Parts and wholes: Patterns of relatedness in complex morphological systems and why they matter." In *Analogy in Grammar: Form and Acquisition*, ed. by James P. Blevins & Juliette Blevins, 54–82. Oxford: Oxford University Press.

* Ackerman, Farrell and Robert Malouf. 2013. "Morphological organization: The Low Conditional Entropy Conjecture." *Language* 89:429-464.

* Ackerman, Farrell and Robert Malouf. In press. "Implicative relations in word-based morphological systems." In *Cambridge Handbook of Morphology*, ed. by Gregory Stump and Raphael Finkel. Cambridge: Cambridge University Press.

See [ModernGreek.ipynb] and [Saami.ipynb] for examples of it in action.

### Data sources:

amele.txt
:	Amele possessor suffixes (Roberts 1987; Hein & Müller 2009)

arapesh.txt
:	Arapesh number suffixes (Dobrin 1999; Hein & Müller 2009)

burmeso.txt
:	Burmeso object agreement prefixes (Donohue 2001; Baerman et al. 2010)

finnish.txt
:	Finnish case and number suffixes (Ackerman et al. 2009)

fur.txt
:	Fur verbal tense and agreement markers (tones and suffixes) (Jakobi 1990; Stump & Finkel 2007)

kwerba.txt
:	Kwerba subject agreement prefixes (De Vries & De Vries 1997; Stump & Finkel 2007)

mazatec.txt
:	Chiquihuitlán Mazatec (Jamieson 1982; Capen 1996; Baerman & Corbett 2010)

mgreek.txt
:	Modern Greek nominal inflection (Ralli 1994; Ralli 2002)

ngiti.txt
:	Ngiti verbal inflection (stem-initial vowel and tone) (Kutsch Lojenga 1994; Stump & Finkel 2007)

nuer.txt
:	Nuer case suffixes (Frank 1999; Baerman et al. 2010)

russian.txt
:	Russian case suffixes (Baerman et al. 2010)

saami.txt
:   Pite Saami nouns (stem grade and suffix) (Wilbur 2014)

repository: rmalouf/morphology

repo_name: rmalouf/morphology


text:
# Change Log

All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.

## [2.5.2](https://github.com/nuxt/friendly-errors-webpack-plugin/compare/v2.5.1...v2.5.2) (2021-10-14)

## [2.5.1](https://github.com/nuxt/friendly-errors-webpack-plugin/compare/v2.5.0...v2.5.1) (2021-04-09)

# [2.5.0](https://github.com/nuxt/friendly-errors-webpack-plugin/compare/v2.0.0-beta.0...v2.5.0) (2019-05-20)

# [2.4.0](https://github.com/nuxt/friendly-errors-webpack-plugin/compare/v2.3.2...v2.4.0) (2018-12-07)

### Features

* display children errors and warnings ([812ffe4](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/812ffe4))

## [2.3.2](https://github.com/nuxt/friendly-errors-webpack-plugin/compare/v2.3.1...v2.3.2) (2018-11-23)

## [2.3.1](https://github.com/nuxt/friendly-errors-webpack-plugin/compare/v2.3.0...v2.3.1) (2018-11-23)

### Bug Fixes

* empty message in module not found error ([ac2c233](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/ac2c233))

# [2.3.0](https://github.com/nuxt/friendly-errors-webpack-plugin/compare/v2.2.0...v2.3.0) (2018-11-03)

# [2.2.0](https://github.com/nuxt/friendly-errors-webpack-plugin/compare/v2.1.0...v2.2.0) (2018-11-03)

### Bug Fixes

* default value for options.logLevel ([1ac5b70](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/1ac5b70))
* ignore WAIT info with consola ([93964fa](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/93964fa))
* logLevel can be lower case value ([8db42c9](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/8db42c9))
* logLevel can be lower case value ([27d166b](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/27d166b))
* message in windows ([ed4c84b](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/ed4c84b))
* remove duplicate SyntaxError ([8953760](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/8953760))

### Features

* add consola reporter ([60e447c](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/60e447c))
* add reporter options ([7547af3](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/7547af3))
* add reporters ([ccddab5](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/ccddab5))
* refactor reporter to standard logger ([00c979d](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/00c979d))
* skip branch build in pr ([25ca868](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/25ca868))
* **consola:** use tagged reporter ([e414613](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/e414613))
* **sandbox:** test with consola.wrapConsole ([8d98ae0](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/8d98ae0))

# [2.1.0](https://github.com/nuxt/friendly-errors-webpack-plugin/compare/v2.0.2...v2.1.0) (2018-10-26)

### Features

* sync with friendly-errors-webpack-plugin ([75c5c3b](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/75c5c3b))

## [2.0.2](https://github.com/nuxt/friendly-errors-webpack-plugin/compare/v2.0.1...v2.0.2) (2018-03-20)

### Features

* clear console also according to log level ([655203d](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/655203d))

## [2.0.1](https://github.com/nuxt/friendly-errors-webpack-plugin/compare/v2.0.0...v2.0.1) (2018-03-20)

### Bug Fixes

* npm badge ([9bb4d3a](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/9bb4d3a))

### Features

* add log level ([ddcf45a](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/ddcf45a))

# [2.0.0](https://github.com/nuxt/friendly-errors-webpack-plugin/compare/v1.6.1...v2.0.0) (2018-03-16)

### Bug Fixes

* comma-dangle error ([2360434](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/2360434))
* lint script ([c3e88c9](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/c3e88c9))
* move comma-dangle into rules ([9cd26c5](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/9cd26c5))
* test failures ([34ac16b](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/34ac16b))
* windows eslint ([01f8059](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/01f8059))

### Features

* add release script ([bcd208f](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/bcd208f))
* upgrade to webpack4 ([458efe9](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/458efe9))
* upgrade to webpack4 ([8927021](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/8927021))

# [2.4.0](https://github.com/nuxt/friendly-errors-webpack-plugin/compare/v2.3.2...v2.4.0) (2018-12-07)

### Features

* display children errors and warnings ([812ffe4](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/812ffe4))

## [2.3.2](https://github.com/nuxt/friendly-errors-webpack-plugin/compare/v2.3.1...v2.3.2) (2018-11-23)

## [2.3.1](https://github.com/nuxt/friendly-errors-webpack-plugin/compare/v2.3.0...v2.3.1) (2018-11-23)

### Bug Fixes

* empty message in module not found error ([ac2c233](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/ac2c233))

# [2.3.0](https://github.com/nuxt/friendly-errors-webpack-plugin/compare/v2.2.0...v2.3.0) (2018-11-03)

### Bug Fixes

* default value for options.logLevel ([1ac5b70](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/1ac5b70))
* ignore WAIT info with consola ([93964fa](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/93964fa))

### Features

* **consola:** use tagged reporter ([e414613](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/e414613))
* **sandbox:** test with consola.wrapConsole ([8d98ae0](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/8d98ae0))

# [2.2.0](https://github.com/nuxt/friendly-errors-webpack-plugin/compare/v2.1.0...v2.2.0) (2018-11-03)

### Bug Fixes

* logLevel can be lower case value ([8db42c9](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/8db42c9))
* logLevel can be lower case value ([27d166b](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/27d166b))
* message in windows ([ed4c84b](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/ed4c84b))
* remove duplicate SyntaxError ([8953760](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/8953760))

### Features

* add consola reporter ([60e447c](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/60e447c))
* add reporter options ([7547af3](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/7547af3))
* add reporters ([ccddab5](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/ccddab5))
* refactor reporter to standard logger ([00c979d](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/00c979d))
* skip branch build in pr ([25ca868](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/25ca868))

# [2.1.0](https://github.com/nuxt/friendly-errors-webpack-plugin/compare/v2.0.2...v2.1.0) (2018-10-26)

### Features

* sync with friendly-errors-webpack-plugin ([75c5c3b](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/75c5c3b))

# 1.7.0 (2018-04-05)

## [2.0.2](https://github.com/nuxt/friendly-errors-webpack-plugin/compare/v2.0.1...v2.0.2) (2018-03-20)

### Features

* clear console also according to log level ([655203d](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/655203d))

## [2.0.1](https://github.com/nuxt/friendly-errors-webpack-plugin/compare/v2.0.0...v2.0.1) (2018-03-20)

### Bug Fixes

* npm badge ([9bb4d3a](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/9bb4d3a))

### Features

* add log level ([ddcf45a](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/ddcf45a))

# [2.0.0](https://github.com/nuxt/friendly-errors-webpack-plugin/compare/v1.6.1...v2.0.0) (2018-03-16)

### Bug Fixes

* comma-dangle error ([2360434](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/2360434))
* lint script ([c3e88c9](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/c3e88c9))
* move comma-dangle into rules ([9cd26c5](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/9cd26c5))
* only transforms tests with babel. prevents [#35](https://github.com/nuxt/friendly-errors-webpack-plugin/issues/35) ([#41](https://github.com/nuxt/friendly-errors-webpack-plugin/issues/41)) ([c8c4f3e](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/c8c4f3e))
* test failures ([34ac16b](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/34ac16b))
* windows eslint ([01f8059](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/01f8059))

### Features

* add release script ([bcd208f](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/bcd208f))
* upgrade to webpack4 ([458efe9](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/458efe9))
* upgrade to webpack4 ([8927021](https://github.com/nuxt/friendly-errors-webpack-plugin/commit/8927021))

repository: nuxt/friendly-errors-webpack-plugin

gha_language: JavaScript

repo_name: nuxt/friendly-errors-webpack-plugin


text:
---
layout: post
title:  "Nachos, but make them roast potato"
date:   2021-08-02 18:30:00
author: oli
image: /images/blog/roast-potato-nachos-1.jpg
tags: ["airfryer", "roast","potatoes", "nachos","2021"]
---

Our household loves roast potatoes done in the airfrier.  War lass loves nachos.  I am less keen on them.  Luckily it turns out that you can marry the two things and everybody loves them!

## You will need

* Enough potatoes for 2
* A couple of small onions if you want to do roasted onion topping
* A teaspoon of vegetable oil
* Whatever you like to top nachoes with.  In my case, peppers (grilled), avacado, natural yoghurt, cheese and lots of sliced spring onions

## Do

* Chop to potatoes so they're no bigger than the first joint of your thumb
* Half the onions, peel and cut though the middle so they're still anchors together but can open up
* Toss it all in a bowl with the oil
* Air fry at 200oC for about 20 minutes or until the potatoes are crisp (I had to take the onion out part way though so it didn't burn)
* Pile onto a plate and top as normal

## Results

Can't be eaten with your fingers, requires a fork.  Very satisfying.

![GET IN MY FACE](/images/blog/roast-potato-nachos-1.jpg)
![GET IN MY FACE FASTER](/images/blog/roast-potato-nachos-2.jpg)

repository: coldclimate/omnomfrickinnom

gha_language: HTML

repo_name: coldclimate/omnomfrickinnom


text:
# ChangeControl
ChangeControl is a tool for managing automated changes in node applications and was inspired by [liquibase](http://www.liquibase.org). You might use it to bootstrap your application with reference data, or perform a migration prior to deploying a new release. Without tools like ChangeControl these types of change require human interaction and accumulate to the point where performing a release involves black magic. Creating a new environment requires deep magic from the dawn of time!

## Dependencies
ChangeControl requires a [Redis](http://www.redis.com) instance to track which changes have been executed. We appreciate this will be a major inconvenience to others, however our project uses Redis, and we wanted something up quickly. Sorry.

## Concepts
At the finest granularity is a Change, which is little more than a JavaScript function and an id.

    Change = A JavaScript function + an Id

You can execute changes, pretend to execute them, or record that you've executed them, even if you haven't. Changes are clumped into ChangeSets. We tend to create a single ChangeSet per release.

    ChangeSet = A list of Changes + an Id

Just like Changes, you can execute ChangeSets, pretend to execute them or just record that you have. Finally you have the ChangeLog. Every time you execute a Change a record is written to the ChangeLog.

    ChangeLog = A list of the changes that have been executed

Once a Change has been recorded in the ChangeLog it typically won't be executed again. Furthermore if someone modifies the Change after it has been executed ChangeControl will complain loudly and fail. It would obviously be bad for two processes to execute a ChangeSet at the same time, so the ChangeLog is locked during processing. Once the ChangeSet has been fully applied, the lock is released. This leads to problems when if one of your Changes dies unexpectedly or is killed, since the ChangeLog will still be locked. If this happens you need to manually 'unlock' the ChangeLog. You can also 'dump' the ChangeLog or 'clear' it.

## Usage

1. Create a ChangeSet
    ```js
    // changes/release-1.0.js
    var ChangeSet = require('changecontrol').ChangeSet;

    exports.init = function(redis, changeLog) {

        var changeSet = ChangeSet.create('release-1.0', changeLog);

        changeSet.add('init:foo:bar', function(next) {
            redis.set('foo:bar', 'a', next);
        });

        changeSet.add('init:pirates', function(next) {
            var multi = redis.multi();
            multi.mset(
                'Blackbeard', 'Queen Anne\'s Revenge',
                'Long John Silver', 'Hispaniola'
            );
            multi.exec(next);
        });

        return changeSet;
    };
    ```
1. Initialise the ChangeLog
    ```js
    var changeLog = require('changecontrol').ChangeLog.create('my-awesome-project', redis);
    ```
1. Initialise the ChangeSet
    ```js
    var changeSet = require('changes/release-1.0').init(changeLog, redis);
    ```
1. Execute the ChangeSet
    ```js
    changeSet.execute(function(err) {
        console.log("Pieces of Eight");
    })
    ```

In practice you'll (hopefully) want to execute all the ChangeSets found in the 'changes' directory automatically when your application starts. You'll probabably also want a script for testing the changes locally and for unlocking the ChangeLog when something unexpected happens. You'll find a starter for ten in the examples folder.

### Targetting specific changes
You can add a parameter to changeSet.execute to be more specific about the change(s) you want to execute, e.g.
```js
changeSet.execute('init:pirates:*', function(err) {
    console.log("Pieces of Eight");
})
```

Would execute 'init:pirates:captains' and 'init:pirates:ships', but not 'init:ninja:masters'. The sync, pretend and clear operations also expect a similar first parameter.

### Run Always
Sometimes you'll want to run a Change everytime your application starts (e.g. backing up log files). You can do this by specifying frequency = 'always' when you define your Change, e.g.

```js
changeSet.add('backup:ships:log', function(next) {
    // Some Code
}, { frequency: 'always'});
```


text:
# Autoregressive Model

An [autoregressive (AR) process](https://en.wikipedia.org/wiki/Autoregressive_model) is a stochastic process with an autoregressive structure, i.e., past realizations influence its future realizations.

This model-zoo example illustrates how to use Flux's recurrent layers to model an AR process.

The example contains the following files:
+ [utils.jl](utils.jl):
    + `generate_process`: generates an AR process
    + `batch_timeseries`: transforms a vector into the proper format for recurrent layers in Flux and allows to batch the time series as required.

+ [model.jl](model.jl): creates and trains the recurrent model to predict the generated AR process.

## Example loss

Running the model with the hyperparameters currently given in the example, we obtain the following train and test losses. We see that the model begins to overfit after around 30 epochs.

![loss](loss.png)

repository: c-p-murphy/model-zoo

gha_language: Julia

gha_license_id: NOASSERTION

repo_name: c-p-murphy/model-zoo


text:
# SAF-T AO

[![GitHub All Releases](https://img.shields.io/github/downloads/assoft-portugal/SAF-T-AO/total)](https://github.com/assoft-portugal/SAF-T-AO/releases)
[![GitHub issues](https://img.shields.io/github/issues-raw/assoft-portugal/SAF-T-AO)](https://github.com/assoft-portugal/SAF-T-AO/issues)
[![GitHub tag (latest SemVer)](https://img.shields.io/github/v/tag/assoft-portugal/SAF-T-AO)](https://github.com/assoft-portugal/SAF-T-AO/releases)
[![License](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/assoft-portugal/SAF-T-AO/blob/master/LICENSE)
[![Discussion](https://img.shields.io/badge/discussion-yammer-blue.svg)](https://www.yammer.com/assoft-members/#/threads/inGroup?type=in_group&feedId=17835940&view=all)

Official XSD from the Government of Angola for use in SAF-T AO.

> The Standard Audit File for Tax Purposes (SAF-T) is a standardized XML file used for exporting the accounting information of a company to the tax authorities. The file contains accounting data that can be exported from an original accounting system for a specific time period.
>
> The SAF-T file is based on a directive by the Organization for Economic Co-operation and Development (OECD).
>
> [Oracle, JD Edwards EnterpriseOne Applications OECD Standard Audit File for Tax Purposes (SAF-T) Localizations Implementation Guide](https://docs.oracle.com/cd/E16582_01/doc.91/e97460/ch_eu_saft_xml.htm#EOAST109).

## VAT implementation in Angola

The VAT legislation will take effect starting in October 2019. This deadline applies to software billing solutions, SAF-T file, and invoicing rules:

- Software solutions must follow the requirements of the new legal system in order to process sales documents, invoices, and other financial documents in accordance with the legal framework. This rule applies to all the subjects of the VAT scheme.
- Software creators must apply to a software certification program by the government which is mandatory.
- Starting from 1 January 2020 it will be mandatory to produce a monthly SAF-T file and send it to the government tax authorities.
- From 1 January 2020, all taxpayers with an annual turnover of more than AKZ 50,000,000 will be required to produce the above referred SAF-T files and use certified software.

## XML Validation

### Unix

- [xmllint](http://xmlsoft.org/xmllint.html)

### Windows

- [XmlPad](https://xmlpad-mobile.com/) with a graphical user interface
- [Test-XML](https://www.powershellgallery.com/packages/Test-XML/1.0) with [PowerShell](https://docs.microsoft.com/en-us/powershell/)

### Installing xmllint

```bash
sudo apt install libxml2-utils
```

### XML validation against XSD

```bash
xmllint -schema schema.xsd file.xml --noout
```

repository: bacelar-eticadata/SAF-T-AO

repo_name: bacelar-eticadata/SAF-T-AO


text:
![SwiftTween Logo](https://github.com/heart/SwiftTween/blob/master/images/logo.png?raw=true)

# SwiftTween

[![CI Status](http://img.shields.io/travis/heart/SwiftTween.svg?style=flat)](https://travis-ci.org/heart/SwiftTween)
[![Version](https://img.shields.io/cocoapods/v/SwiftTween.svg?style=flat)](http://cocoapods.org/pods/SwiftTween)
[![License](https://img.shields.io/cocoapods/l/SwiftTween.svg?style=flat)](http://cocoapods.org/pods/SwiftTween)
[![Platform](https://img.shields.io/cocoapods/p/SwiftTween.svg?style=flat)](http://cocoapods.org/pods/SwiftTween)

SwiftTween is a lightweight value tweener easy to use.

## Example

To run the example project, clone the repo, and run `pod install` from the Example directory first.

### the Example project
![SwiftTween Example app](https://github.com/heart/SwiftTween/blob/master/images/example_app.png?raw=true)

## Requirements
Swift 4.0

## Installation

SwiftTween is available through [CocoaPods](http://cocoapods.org). To install
it, simply add the following line to your Podfile:

```ruby
pod 'SwiftTween'
```

## Example
```swift
let view = UIView()
view.layer.backgroundColor = UIColor.red.cgColor
view.frame = CGRect(x: 0, y: 0, width: 100, height: 100)

let t = SwiftTween()

t.onUpdate = {
    (value:Double,timePassed:Double,finish:Bool)->Void in

    let f = view.frame

    view.frame = CGRect(x:CGFloat(value-20) ,
                            y:CGFloat(200-value),
                            width:f.size.width,
                            height:f.size.height)

    if finish {
    	print("Tween Finished")
    }else{
    	print("Time Passed > \(timePassed)")
    }

}

t.tween(startValue:0, endValue:200 , timeDuration:1 , ease:Ease.Linear )
```

## You can get the tween value at any spot of the timline
```swift
let duration = 20
let currentTime = 10
let beginningValue = 0
let finalValue = 100

let value_At_Half_Of_Timeline = EasingFunction.getValue(
				ease:Ease.Linear,
				currentTime: currentTime,
				beginningValue: beginningValue,
				finalValue: finalValue ,
				totalDuration:duration )

print( value_At_Half_Of_Timeline ) //print 50 because it's a Linear easing
```

## You also can get the ratio of value at any spot of the timline
```swift
let timelineRatio = 0.5

let value_Ratio_At_Half_Of_Timeline = EasingFunction.getRatio(ease:Ease.Linear,  timeRatio:timelineRatio )

print( value_At_Half_Of_Timeline ) //print 0.5 because it's a Linear easing
```

## Easing Type
```swift
Ease.Linear
Ease.InQuad
Ease.OutQuad
Ease.InOutQuad
Ease.InCubic
Ease.OutCubic
Ease.InOutCubic
Ease.InQuart
Ease.OutQuart
Ease.InOutQuart
Ease.InQuint
Ease.OutQuint
Ease.InOutQuint
Ease.InSine
Ease.OutSine
Ease.InOutSine
Ease.InExpo
Ease.OutExpo
Ease.InOutExpo
Ease.InCirc
Ease.OutCirc
Ease.InOutCirc
Ease.InElastic
Ease.OutElastic
Ease.InOutElastic
Ease.InBack
Ease.OutBack
Ease.InOutBack
Ease.InBounce
Ease.OutBounce
Ease.InOutBounce
```

## Author

i am Heart<EMAIL_ADDRESS>
## License

SwiftTween is available under the MIT license. See the LICENSE file for more info.

repository: heart/SwiftTween

repo_name: heart/SwiftTween


text:
---
title: 'How to setup WordPress locally: Installing MySQL'
excerpt: The third post in the series Testing WordPress Locally deals with installing MySQL on windows.
category: "programming"
tags:
    - tutorial
    - wordpress
series: 'Setting up WordPress locally'
---

## Installing MySQL on Windows

-   Go to the MySQL Website and download the installer file from the windows section.
-   Run the installer.
-   Choose the Developer Default and click on Execute.
-   On the MySQL Server Configuration Dialog, enter the password for the root/administrator access.

## Testing MySQL on Windows

-   Run the command prompt as **Administrator**.
-   Navigate to the bin directory in the MySQL server installation directory.
-   Type _mysql -u root -p_.
-   At the prompt, enter the password.
-   MySQL starts, and you should see the MySQL prompt.
-   Type _quit_ to exit.

## Activating MySQL for PHP

-   Open the _php.ini_ file for editing
-   Remove the semicolon from the line _;extension_dir ="ext"_
    -   If the line does not exist, add the line.
-   Change ext to point to the ext folder in the PHP install directory.
-   Remove the semicolon from the line _;extension=php_mysqli.dll_
    -   If the line does not exist, add the line.
-   Restart the APACHE server.


text:
This extension makes it easy to `DROP ROLE` in PostgreSQL.

It provides a function that helps you to revoke all privileges of a role in
a database.

Example
-------

You want to get rid of user `dummy` in the database.

If you want to keep the objects owned by the role, change their ownership to
a different role:

    REASSIGN OWNED BY dummy TO otheruser;

Then drop everything still owned by the role.  This is required even if you
ran `REASSIGN OWNED`, as some objects (for example user mappings) are not
affected.

    DROP OWNED BY dummy;

Then use the extension to get rid of all the permissions.  When using `psql`,
you can use the convenient `\gexec` to execute the function results as an
SQL script:

    CREATE EXTENSION drop_role_helper SCHEMA public;
    SELECT * FROM public.drop_role_helper('dummy') \gexec

Repeat the above statements in all databases in the PostgreSQL cluster.
Then you can drop the role:

    DROP ROLE dummy;

Installation
------------

To install the software, make sure that the correct `pg_config` is on the
`PATH`, change into the directory with the software and run

    sudo make install

Then install the extension in the databases where you need it:

    CREATE EXTENSION drop_role_helper;

If you don't have permission to install software on the database server,
you can simply run the SQL script that creates the function.  In that case
you don't need to call `CREATE EXTENSION`.

Usage
-----

The extension creates a single table function `drop_role_helper` that takes
a role name as argument and returns a table of SQL statements.

You need no special permissions to run the function, but to execute the
SQL statements returned by the function, you need to be a superuser or the
owner of all the affected objects.  If the role to be dropped has default
privileges, you must also be a member of the role for which the default
privileges are defined.

Support
-------

Please report problems by creating a [Github issue][1].
Commercial support is available from [Cybertec][2].

  [1]: https://github.com/cybertec-postgresql/drop_role_helper/issues
  [2]: https://www.cybertec-postgresql.com/

license: PostgreSQL


text:
# Final-Project
### Isabel Orticelle Final Project

I made a book depository so that an average user can add, delete, and edit books to this database for various purposes such as to keep track of various author's book or to keep track of the books he or she has read. The Author table is a one to many relationship in which one author can have many books.

### Database Design
Author Table: *one to many relationship*

Author ID |  Author Name | About
------------|---------------|-------------
1 | Stephen King |Stephen Edwin King is an American author of horror, supernatural fiction, suspense, science fiction, and fantasy
2 | JK Rowling |JK Rowling is a British novelist, philanthropist, film producer, television producer and screenwriter who is best known for writing the Harry Potter fantasy series
3 | F. Scott Fitzgerald | Francis Scott Key Fitzgerald was an American writer, whose works illustrate the Jazz Age. While he achieved limited success in his lifetime, he is now widely regarded as one of the greatest American writers of the 20th century.

Book Table

Book ID | Title(Name) | Year | Author | Summary
---------|-------|-------|------|------------
1 | It | 1986 | Stephen King | Seven young outcasts in Derry, Maine, are about to face their worst nightmare -- an ancient, shape-shifting evil that emerges from the sewer every 27 years to prey on the town's children. Banding together over the course of one horrifying summer, the friends must overcome their own personal fears to battle the murderous, bloodthirsty clown known as Pennywise.
2 | Harry Potter and the Sorcerer's Stone | 1997 | JK Rowling | A boy learns on his eleventh birthday that he is the orphaned son of two powerful wizards and possesses unique magical powers of his own. He is summoned from his life as an unwanted child to become a student at Hogwarts, an English boarding school for wizards. There, he meets several friends who become his closest allies and help him discover the truth about his parents' mysterious deaths
3 | The Great Gatsby| 1925 | F Scott Fitzgerald | The Great Gatsby is the story of eccentric millionaire Jay Gatsby as told by Nick Carraway, a Midwesterner who lives on Long Island but works in Manhattan. Gatsby's enormous mansion is adjacent to Carraway's modest home, and Carraway becomes curious about his neighbor after being invited to one of his famous parties

### Instructions

Run the Depository

- **Windows Instructions**

Go to [my GitHub](https://github.com/isabel-orticelle/Final-Project) and clone the repo so that you can store it locally.

Open Powershell

Type the following command to open the project directory:

    cd Final-Project

Type the following command to create Virtualenv so that we can setup a virtual environment:

    pip install virtualenv

Create the venv folder:

    virtualenv venv

Activate the virtual environment:

    venv/scripts/activate

Install the requirements for the project:

    pip install -r requirements.txt

 Initialize the database:

    python manage.py deploy

 Run the server:

    python manage.py runserver -d

 View the website in your web browser by typing your IP address into the address bar.

    http://<IP_ADDRESS>:5000/

- **Mac Instructions**

Go to [my GitHub](https://github.com/isabel-orticelle/Final-Project)  and clone the repo so that you can store it locally.

Open the Command Line

Type the following command to open the project directory:

    cd Final-Project

Type the following command to create Virtualenv so that we can setup a virtual environment:

    sudo pip install virtualenv

Create the venv folder:

    virtualenv venv

Activate the virtual environment:

    venv/bin/activate

Install the requirements for the project:

    pip install -r requirements.txt

 Initialize the database:

    python manage.py deploy

 Run the server:

    python manage.py runserver -d

 View the website in your web browser by typing your IP address into the address bar.

    http://<IP_ADDRESS>:5000/

repository: isabel-orticelle/Final-Project

repo_name: isabel-orticelle/Final-Project


text:
sinatra-web-app
===============

Structure for a modular Sinatra web app (Ruby 2.3.1 + Rake + Bundler + Sinatra 1.4.7).

## Install Dependencies

Install dependencies with `bundle install --without production` (or development in a production environment).

## Run Tests

Run tests with `bundle exec rake`.

## Package

Create a package with `bundle exec rake package`.

## Start Server

Start server with `bundle exec rackup`. The app will deploy to http://localhost:9292.

repository: kytkemo/sinatra-web-app

repo_name: kytkemo/sinatra-web-app


text:
# Organizer (//org)

A multi-purpose inventory management solution. Similar to GearCollector; uses packets.

For the purpose of this addon, a `bag` is: "Safe", "Storage", "Locker", "Satchel", "Sack", "Case", "Wardrobe", "Safe 2".

For commands that use a filename, if one is not specified, it defaults to Name_JOB.lua, e.g., Rooks_PLD.lua
For commands that specify a bag, if one is not specified, it defaults to all, and will cycle through all of them.

The addon command is `org`, so `org freeze` will freeze, etc.

This utility is still in development and there are at least a couple of known issues (it does not always move out gear that is currently equipped, argument parsing could be better). It is designed to work simplest as a snapshotting utility (freeze and organize without arugments), but it should work no matter what you want to do with it.

### Settings

#### auto_heal
Setting this feature to anything other than false will cause Organizer to use /heal after getting/storing gear.

#### bag_priority
The order that bags will be looked in for requested gear.

#### dump_bags
The order that bags will be filled with unspecified gear from your inventory.

#### item_delay
A delay, in seconds, between item storage/retrieval. Defaults to 0 (no delay)

### Commands
Commands below are written with their arguments indicated using square brackets, but you should not use square brackets when entering the commands in game. Default options are italicized.

#### Freeze

```
freeze [bag] [filename]
```

Freezes the current contents of a `bag` or **all bags** to the specified `filename` or **Name_ShortJob.lua** in the respective data directory/directories. This effectively takes a snapshot of your inventory for that job. So using `//org freeze` as a Dancer named Pablo would result in freezing all of your bags in files named Pablo_DNC.lua.

#### Get

```
get [bag] [filename]
```

Thaws the frozen state specified by `filename` or **Name_ShortJob.lua** and `bag` or **all bags** and makes one attempt to move towards that state.

#### Tidy

```
tidy [bag] [filename]
```

Thaws a frozen state specified by `filename` or **Name_ShortJob.lua** and `bag` or **all bags** and makes one attempt to purge anything currently in inventory that shouldn't be into dump bags.

#### Organize

```
organize [bag] [filename]
```

Thaws a frozen state specified by `filename` or **Name_ShortJob.lua** and `bag` or **all bags** and executes repeated Get and Tidy commands until a steady state is reached (aka. you have your gear). With no arguments, it will attempt to restore the entire thawed snapshot.

### Gearswap integration
Additionally, Organizer integrates with GearSwap. In your lua, just add this:

```
include('organizer-lib')
```

And then in your Mog House, after changing jobs:

```
//gs org
```

And it will fill your inventory with the items from your sets, and put everything else away (it does a very good job, even when there are space concerns, but it's not perfect. Make sure to do a "//gs validate" after!)

Additionally, if you have extra items you want to bring along, simply define a table named `organizer_items` like so:

```
organizer_items = {
    echos="Echo Drops",
    shihei="Shihei",
    orb="Macrocosmic Orb"
}
```

repository: Windower/Lua

branch_name: refs/heads/dev

gha_language: Lua

repo_name: Windower/Lua


text:
---
layout: page
title: Tech Journal
permalink: /techjournal/
---

This will be a series of posts on basics of Machine Learning, Software engineering, Data engineering and Computer science. Whatever tickles my fancy :)

# Machine Learning and Deep learning

1. [Loss Functions](./_posts/2023-02-14-cost-functions.markdown)
2. [Optimization Algorithms](./_posts/2023-02-15-Optimization-Algos.markdown)

# Programming

1. Async Programming (coming soon)
2. [Decorator pattern python](./_posts/2023-04-09-Decorator-pattern.markdown)

# Devops

1. Basics of CI/CD (coming soon with an example)

# Engineering tricks and tips

1. [Private Package](./_posts/2023-02-18-Private-Python-Packages.markdown)
2. [Terminal Setup](./_posts/2023-01-28-Terminal-prep.markdown)
3. [Environment Setup](./_posts/2021-02-08-Environment.markdown)

gha_language: SCSS


text:
# About

The conditional ternary operator is used to write a condensed expression that returns one of two alternate values based on some condition.
It is often referred to as the "ternary operator".
The name stems from the fact that the operator has three operands: `predicate ? consequent-expression : alternative-expression`
It can be used as a replacement for short if-else statements.

Similar to `if` statements, JavaScript will perform implicit type conversion to evaluate the condition.
If the condition is truthy, the operand on the left-hand side of the colon will be returned.
Otherwise the result of the ternary expression is the operand on the right-hand side of the colon.

```javascript
const year = 2020;

year > 2000 ? 'in the past years' : 'a long time ago';
// => 'in the past years'
```

repository: fabianmarz/javascript

branch_name: refs/heads/main

repo_name: fabianmarz/javascript


text:
GitHub for Jive - Developer Notes
=================================

This service is for anything that is coming from Github or anything coming from the
browser that will go to or query Github via a proxied api call.

gitHubController.js
-------------------
GET /github/user/repos
GET /github/place/issues
POST /github/place/newIssue
POST /github/place/changeIssueState
POST /github/place/changeIssueLabels
GET /github/place/comments
POST /github/place/newComment

gitHubOAuthController.js
------------------------
GET /github/oauth/callback
GET /github/oauth/authorize