text:
---
title: "Fluctuation and Dispersion analyses"
author: "Fred Hasselman & Maarten Wijnants"
date: "1/14/2018"
output:
  bookdown::html_document2:
    variant: markdown+hard_line_breaks
    fig_caption: yes
    highlight: pygments
    keep_md: yes
    self_contained: yes
    number_sections: yes
    theme: spacelab
    toc: yes
    toc_float: true
    collapsed: false
    smooth_scroll: true
    code_folding: show
    bibliography: [refs.bib, packages.bib]
    biblio-style: apalike
    csl: apa.csl
    includes:
        before_body: assignmentstyle.html
    pandoc_args: ["--number-offset","4"]
editor_options:
  chunk_output_type: console

---

# **Fluctuation and Disperion analyses** {-}

### Three plus two time series {-}

We are going to analyse the same 3 time series as during the last session, now using fluctuation and dispersion analyses.

**Importing data**

Two ways:

A. By downloading:

  1. Follow the link, e.g. for [`series.xlsx`](https://github.com/complexity-methods/CSA-assignments/blob/master/assignment_data/BasicTSA_arma/series.xlsx).
  2. On the Github page, find a button marked **Download** (or **Raw** for text files).
  3. Download the file
  4. Load it into `R` using the code below

```r
library(rio)
series <- rio::import("series.xlsx")
```

B. By directly importing the file in R from Github:

   1. Copy the `url` associated with the **Download**  button [on Github](https://github.com/complexity-methods/CSA-assignments/blob/master/assignment_data/BasicTSA_arma/series.csv) (right-click).
   2. The copied path should contain the word 'raw' somewhere in the url.
   3. Call `rio::import(url)`

```r
library(rio)
series <- rio::import("https://github.com/complexity-methods/CSA-assignments/raw/master/assignment_data/BasicTSA_arma/series.xlsx")
```

Remember this is what the time series and retun plots looked like.

![](ASSIGNMENTS_scaling_files/figure-html/unnamed-chunk-3-1.png)<!-- -->

Now we'll add two additional time series using the function `noise_powerlaw()`, run the code below:

```r
library(casnet)
series$TS_4 <- noise_powerlaw(alpha = -1, N = 1024, standardise = TRUE)
series$TS_5 <- noise_powerlaw(alpha = -2, N = 1024, standardise = TRUE)
```

![](ASSIGNMENTS_scaling_files/figure-html/unnamed-chunk-5-1.png)<!-- -->

###  Standardised Dispersion Analysis {.tabset .tabset-fade .tabset-pills}

#### Questions {-}

* Common data preparation before running fluctuation analyses like `SDA`:
    + Normalize the time series (using `sd` based on `N`, not `N-1`, e.g. by using `ts_standardise()`, or use the arguments of the function `fd_sda()`)
    + Check whether time series length is a power of `2`. If you use `nextn(length(TS),factors=2)`, the number you get is the length your time series should be to create equal bin sizes. You can pad the series with 0s before analysis, look at package `invctr` at the infix function `%+]%`.
* Perform `sda` on the 5 time series, use `minData = 6` to exclude the larger bin sizes from estimating  the scaling relation.
* Compare what you find for `fd_sda()` to the other techniques (`fd_RR`,`SampEn`, `acf`, `pacf`)

#### Answers {-}

```r
library(casnet)

# All N are already a power of 2
N1 <- nextn(length(series$TS_1), factors = 2)
N2 <- nextn(length(series$TS_2), factors = 2)
N3 <- nextn(length(series$TS_3), factors = 2)
N4 <- nextn(length(series$TS_4), factors = 2)
N5 <- nextn(length(series$TS_5), factors = 2)

#Suppose the time series Y had length 10, you could have used:
library(invctr)
Y <- 1:10
Y %+]% 6
##  [1]  1  2  3  4  5  6  7  8  9 10  0  0  0  0  0  0

# The default of fd_sda() is to standardise in the mean and sd
fdSDA1 <- fd_sda(series$TS_1, minData = 6, doPlot = TRUE, tsName = "TS1")
##
##
## fd_sda:	Sample rate was set to 1.
```

![](ASSIGNMENTS_scaling_files/figure-html/unnamed-chunk-6-1.png)<!-- -->

```
##
## ~~~o~~o~~casnet~~o~~o~~~
##
##  Standardised Dispersion Analysis
##
##  Full range (n = 10)
## Slope = -0.64 | FD = 1.64
##
##  Fit range (n = 8)
## Slope = -0.56 | FD = 1.56
##
## ~~~o~~o~~casnet~~o~~o~~~
fdSDA2 <- fd_sda(series$TS_2, minData = 6, doPlot = TRUE, tsName = "TS2")
##
##
## fd_sda:	Sample rate was set to 1.
```

![](ASSIGNMENTS_scaling_files/figure-html/unnamed-chunk-6-2.png)<!-- -->

```
##
## ~~~o~~o~~casnet~~o~~o~~~
##
##  Standardised Dispersion Analysis
##
##  Full range (n = 10)
## Slope = -0.58 | FD = 1.58
##
##  Fit range (n = 8)
## Slope = -0.39 | FD = 1.39
##
## ~~~o~~o~~casnet~~o~~o~~~
fdSDA3 <- fd_sda(series$TS_3, minData = 6, doPlot = TRUE, tsName = "TS3")
##
##
## fd_sda:	Sample rate was set to 1.
```

![](ASSIGNMENTS_scaling_files/figure-html/unnamed-chunk-6-3.png)<!-- -->

```
##
## ~~~o~~o~~casnet~~o~~o~~~
##
##  Standardised Dispersion Analysis
##
##  Full range (n = 10)
## Slope = -0.64 | FD = 1.64
##
##  Fit range (n = 8)
## Slope = -0.59 | FD = 1.59
##
## ~~~o~~o~~casnet~~o~~o~~~
fdSDA4 <- fd_sda(series$TS_4, minData = 6, doPlot = TRUE, tsName = "TS4")
##
##
## fd_sda:	Sample rate was set to 1.
```

![](ASSIGNMENTS_scaling_files/figure-html/unnamed-chunk-6-4.png)<!-- -->

```
##
## ~~~o~~o~~casnet~~o~~o~~~
##
##  Standardised Dispersion Analysis
##
##  Full range (n = 10)
## Slope = -0.75 | FD = 1.75
##
##  Fit range (n = 8)
## Slope = -0.18 | FD = 1.18
##
## ~~~o~~o~~casnet~~o~~o~~~
fdSDA5 <- fd_sda(series$TS_5, minData = 6, doPlot = TRUE, tsName = "TS5")
##
##
## fd_sda:	Sample rate was set to 1.
```

![](ASSIGNMENTS_scaling_files/figure-html/unnamed-chunk-6-5.png)<!-- -->

```
##
## ~~~o~~o~~casnet~~o~~o~~~
##
##  Standardised Dispersion Analysis
##
##  Full range (n = 10)
## Slope = -0.05 | FD = 1.05
##
##  Fit range (n = 8)
## Slope = -0.03 | FD = 1.03
##
## ~~~o~~o~~casnet~~o~~o~~~

# If you ask to return the powerlaw (returnPLAW = TRUE) the function plotFD_loglog() can make the plot as well
fdS5 <- fd_sda(series$TS_5, returnPLAW = TRUE, minData = 6, doPlot = TRUE, tsName = "TS5")
##
##
## fd_sda:	Sample rate was set to 1.
```

![](ASSIGNMENTS_scaling_files/figure-html/unnamed-chunk-6-6.png)<!-- -->

```
##
## ~~~o~~o~~casnet~~o~~o~~~
##
##  Standardised Dispersion Analysis
##
##  Full range (n = 10)
## Slope = -0.05 | FD = 1.05
##
##  Fit range (n = 8)
## Slope = -0.03 | FD = 1.03
##
## ~~~o~~o~~casnet~~o~~o~~~
plotFD_loglog(fdS5, title = "SDA", subtitle = "TS5")
```

![](ASSIGNMENTS_scaling_files/figure-html/unnamed-chunk-6-7.png)<!-- -->

###  Spectral Slope {.tabset .tabset-fade .tabset-pills}

#### Questions {-}

* Common data preparation before running spectral analyses:
    + Normalize the time series (using `sd` based on `N`, not `N-1`, e.g. by using `ts_standardise()`, or use the arguments of the function `fd_psd()`)
    + Check whether time series length is a power of `2`. If you use `nextn(length(TS),factors=2)`, the number you get is the length your time series should be to create equal bin sizes. You can pad the series with 0s before analysis, look at package `invctr` at the infix function `%+]%`.
* Perform `fd_psd()` on the 3 time series.
* Compare to what you find for `fd_psd` to the other measures (`fd_RR`,`SampEn`, `acf`, `pacf`)

#### Answers {-}

```r
library(casnet)

# All N are already a power of 2
N1 <- nextn(length(series$TS_1), factors = 2)
N2 <- nextn(length(series$TS_2), factors = 2)
N3 <- nextn(length(series$TS_3), factors = 2)
N4 <- nextn(length(series$TS_4), factors = 2)
N5 <- nextn(length(series$TS_5), factors = 2)

#Suppose the time series Y had length 10, you could have used:
library(invctr)
Y <- 1:10
Y %+]% 6
##  [1]  1  2  3  4  5  6  7  8  9 10  0  0  0  0  0  0

# The default of fd_psd() is to standardises on the mean and sd, and detrended
fdPSD1 <- fd_psd(series$TS_1, doPlot = TRUE, tsName = "TS1")
##
##
## fd_psd:	Sample rate was set to 1.
```

![](ASSIGNMENTS_scaling_files/figure-html/unnamed-chunk-7-1.png)<!-- -->

```
##
## ~~~o~~o~~casnet~~o~~o~~~
##
##  Power Spectral Density Slope
##
##  All frequencies (n = 512)
## Slope = 0.18 | FD = 1.57
##
##  Hurvich-Deo (n = 36)
## Slope = 0.29 | FD = 1.61
##
## ~~~o~~o~~casnet~~o~~o~~~
fdPSD2 <- fd_psd(series$TS_2, doPlot = TRUE, tsName = "TS2")
##
##
## fd_psd:	Sample rate was set to 1.
```

![](ASSIGNMENTS_scaling_files/figure-html/unnamed-chunk-7-2.png)<!-- -->

```
##
## ~~~o~~o~~casnet~~o~~o~~~
##
##  Power Spectral Density Slope
##
##  All frequencies (n = 512)
## Slope = -0.27 | FD = 1.4
##
##  Hurvich-Deo (n = 79)
## Slope = -0.98 | FD = 1.2
##
## ~~~o~~o~~casnet~~o~~o~~~
fdPSD3 <- fd_psd(series$TS_3, doPlot = TRUE, tsName = "TS3")
##
##
## fd_psd:	Sample rate was set to 1.
```

![](ASSIGNMENTS_scaling_files/figure-html/unnamed-chunk-7-3.png)<!-- -->

```
##
## ~~~o~~o~~casnet~~o~~o~~~
##
##  Power Spectral Density Slope
##
##  All frequencies (n = 512)
## Slope = 0.12 | FD = 1.54
##
##  Hurvich-Deo (n = 32)
## Slope = 0.02 | FD = 1.51
##
## ~~~o~~o~~casnet~~o~~o~~~
fdPSD4 <- fd_psd(series$TS_4, doPlot = TRUE, tsName = "TS4")
##
##
## fd_psd:	Sample rate was set to 1.
```

![](ASSIGNMENTS_scaling_files/figure-html/unnamed-chunk-7-4.png)<!-- -->

```
##
## ~~~o~~o~~casnet~~o~~o~~~
##
##  Power Spectral Density Slope
##
##  All frequencies (n = 512)
## Slope = -0.84 | FD = 1.23
##
##  Hurvich-Deo (n = 28)
## Slope = 0.19 | FD = 1.57
##
## ~~~o~~o~~casnet~~o~~o~~~
fdPSD5 <- fd_psd(series$TS_5, doPlot = TRUE, tsName = "TS5")
##
##
## fd_psd:	Sample rate was set to 1.
```

![](ASSIGNMENTS_scaling_files/figure-html/unnamed-chunk-7-5.png)<!-- -->

```
##
## ~~~o~~o~~casnet~~o~~o~~~
##
##  Power Spectral Density Slope
##
##  All frequencies (n = 512)
## Slope = -1.9 | FD = 1.1
##
##  Hurvich-Deo (n = 51)
## Slope = -1.76 | FD = 1.11
##
## ~~~o~~o~~casnet~~o~~o~~~
```

###  Detrended Fluctuation Analysis {.tabset .tabset-fade .tabset-pills}

#### Questions {-}

* Data preparation before running `DFA` analyses are not really necessary, except perhaps:
    + Check whether time series length is a power of `2`. If you use `nextn(length(TS),factors=2)`, the number you get is the length your time series should be to create equal bin sizes. You can pad the series with 0s before analysis, look at package `invctr` at the infix function `%+]%`.
* Perform `fd_dfa()` on the 5 time series.
* Compare to what you find for `fd_dfa` to the other measures (`fd_RR`,`SampEn`, `acf`, `pacf`)

#### Answers {-}

```r
library(casnet)

# All N are already a power of 2
N1 <- nextn(length(series$TS_1), factors = 2)
N2 <- nextn(length(series$TS_2), factors = 2)
N3 <- nextn(length(series$TS_3), factors = 2)
N4 <- nextn(length(series$TS_4), factors = 2)
N5 <- nextn(length(series$TS_5), factors = 2)

#Suppose the time series Y had length 10, you could have used:
library(invctr)
Y <- 1:10
Y %+]% 6
##  [1]  1  2  3  4  5  6  7  8  9 10  0  0  0  0  0  0

# The default of fd_psd() is to standardises on the mean and sd, and detrended
fdDFA1 <- fd_dfa(series$TS_1, doPlot = TRUE, tsName = "TS1")
##
##
## fd_dfa:	Sample rate was set to 1.
```

![](ASSIGNMENTS_scaling_files/figure-html/unnamed-chunk-8-1.png)<!-- -->

```
##
## ~~~o~~o~~casnet~~o~~o~~~
##
##  Detrended FLuctuation Analysis
##
##  Full range (n = 8)
## Slope = 0.44 | FD = 1.55
##
##  Exclude large bin sizes (n = 7)
## Slope = 0.45 | FD = 1.54
##
## ~~~o~~o~~casnet~~o~~o~~~
fdDFA2 <- fd_dfa(series$TS_2, doPlot = TRUE, tsName = "TS2")
##
##
## fd_dfa:	Sample rate was set to 1.
```

![](ASSIGNMENTS_scaling_files/figure-html/unnamed-chunk-8-2.png)<!-- -->

```
##
## ~~~o~~o~~casnet~~o~~o~~~
##
##  Detrended FLuctuation Analysis
##
##  Full range (n = 8)
## Slope = 1.1 | FD = 1.16
##
##  Exclude large bin sizes (n = 7)
## Slope = 1.23 | FD = 1.12
##
## ~~~o~~o~~casnet~~o~~o~~~
fdDFA3 <- fd_dfa(series$TS_3, doPlot = TRUE, tsName = "TS3")
##
##
## fd_dfa:	Sample rate was set to 1.
```

![](ASSIGNMENTS_scaling_files/figure-html/unnamed-chunk-8-3.png)<!-- -->

```
##
## ~~~o~~o~~casnet~~o~~o~~~
##
##  Detrended FLuctuation Analysis
##
##  Full range (n = 8)
## Slope = 0.47 | FD = 1.52
##
##  Exclude large bin sizes (n = 7)
## Slope = 0.47 | FD = 1.53
##
## ~~~o~~o~~casnet~~o~~o~~~
fdDFA4 <- fd_dfa(series$TS_4, doPlot = TRUE, tsName = "TS4")
##
##
## fd_dfa:	Sample rate was set to 1.
```

![](ASSIGNMENTS_scaling_files/figure-html/unnamed-chunk-8-4.png)<!-- -->

```
##
## ~~~o~~o~~casnet~~o~~o~~~
##
##  Detrended FLuctuation Analysis
##
##  Full range (n = 8)
## Slope = 0.95 | FD = 1.22
##
##  Exclude large bin sizes (n = 7)
## Slope = 0.98 | FD = 1.21
##
## ~~~o~~o~~casnet~~o~~o~~~
fdDFA5 <- fd_dfa(series$TS_5, doPlot = TRUE, tsName = "TS5")
##
##
## fd_dfa:	Sample rate was set to 1.
```

![](ASSIGNMENTS_scaling_files/figure-html/unnamed-chunk-8-5.png)<!-- -->

```
##
## ~~~o~~o~~casnet~~o~~o~~~
##
##  Detrended FLuctuation Analysis
##
##  Full range (n = 8)
## Slope = 1.35 | FD = 1.1
##
##  Exclude large bin sizes (n = 7)
## Slope = 1.4 | FD = 1.09
##
## ~~~o~~o~~casnet~~o~~o~~~
```

### Compare results {.tabset .tabset-fade .tabset-pills}

#### Questions {-}

* Create a table that compares the Fractal Dimension estimates based on the limited fit range for all time series (columns) with for each analysis (rows)
* Create the same for the full fitrange.
* What can you conclude about these series based on these results?

#### Answers {-}

```r
# Fitrange FD
knitr::kable(data.frame(
           TS1 = c(fdSDA1$fitRange$FD,fdPSD1$fitRange$FD,fdDFA1$fitRange$FD),
           TS2 = c(fdSDA2$fitRange$FD,fdPSD2$fitRange$FD,fdDFA2$fitRange$FD),
           TS3 = c(fdSDA3$fitRange$FD,fdPSD3$fitRange$FD,fdDFA3$fitRange$FD),
           TS4 = c(fdSDA4$fitRange$FD,fdPSD4$fitRange$FD,fdDFA4$fitRange$FD),
           TS5 = c(fdSDA5$fitRange$FD,fdPSD5$fitRange$FD,fdDFA5$fitRange$FD),
           row.names = c("SDA","PSD","DFA")
           ),digits = 2,caption = "FD based on limited fit range")
```

Table: (\#tab:unnamed-chunk-9)FD based on limited fit range

        TS1    TS2    TS3    TS4    TS5
----  -----  -----  -----  -----  -----
SDA    1.56   1.39   1.59   1.18   1.03
PSD    1.61   1.20   1.51   1.57   1.11
DFA    1.54   1.12   1.53   1.21   1.09

</br>

```r
# Full range FD
knitr::kable(data.frame(
           TS1 = c(fdSDA1$fullRange$FD,fdPSD1$fullRange$FD,fdDFA1$fullRange$FD),
           TS2 = c(fdSDA2$fullRange$FD,fdPSD2$fullRange$FD,fdDFA2$fullRange$FD),
           TS3 = c(fdSDA3$fullRange$FD,fdPSD3$fullRange$FD,fdDFA3$fullRange$FD),
           TS4 = c(fdSDA4$fullRange$FD,fdPSD4$fullRange$FD,fdDFA4$fullRange$FD),
           TS5 = c(fdSDA5$fullRange$FD,fdPSD5$fullRange$FD,fdDFA5$fullRange$FD),
           row.names = c("SDA","PSD","DFA")
           ),digits = 2,caption = "FD based on full fit range")
```

Table: (\#tab:unnamed-chunk-10)FD based on full fit range

        TS1    TS2    TS3    TS4    TS5
----  -----  -----  -----  -----  -----
SDA    1.64   1.58   1.64   1.75   1.05
PSD    1.57   1.40   1.54   1.23   1.10
DFA    1.55   1.16   1.52   1.22   1.10

## **Surrogate testing: Beyond the straw-man null-hypothesis**

Look at the explanation of surrogate analysis on the [TiSEAN website](https://www.pks.mpg.de/~tisean/Tisean_3.0.1/)
(Here is [direct link](https://www.pks.mpg.de/~tisean/Tisean_3.0.1/docs/surropaper/node5.html) to the relevant sections)

### Constrained realisations of complex signals {.tabset .tabset-fade .tabset-pills}

We'll look mostly at three different kinds of surrogates:

* *Randomly shuffled*: $H_0:$ The time series data are independent random numbers drawn from some probability distribution.
* *Phase Randomised*: $H_0:$ The time series data were generated by a stationary linear stochastic process
* *Amplitude Adjusted ,Phase Randomised*: $H_0:$ The time series data were generated by a rescaled linear stochastic process.

#### Questions {-}

* Figure out how many surrogates you minimally need for a 2-sided test.
* Package `nonlinearTseries` has a function `FFTsurrogate()` and package `fractal` has `surrogate()`. Look them up and try to create a surrogate test for time series `TS1`, `TS3`, and `TS4` of the previous assignments.
    + If you use `fractal::surrogate()`, choose either `aaft` (amplitude adjusted Fourier transform) or `phase` (phase randomisation)
    + `nonlinearTseries::FFTsurrogates` will calculate phase randomised surrogates.
* In package `casnet` there's a function that will calculate and display a point-probability based on a distribution of surrogate measures and one observed measure: `plotSUR_hist`

#### Answers {-}

Create a number of surrogates, calulate the slopes and plot the results.

* First let's get the values we found earlier (we'll use the FD from `sda`, then we do the same for `psd`,but you can take any of the measures as well)

```r
TS1_FDsda <- fd_sda(series$TS_1)$fitRange$FD
```

```
##
##
## fd_sda:	Sample rate was set to 1.
##
##
## ~~~o~~o~~casnet~~o~~o~~~
##
##  Standardised Dispersion Analysis
##
##  Full range (n = 10)
## Slope = -0.64 | FD = 1.64
##
##  Fit range (n = 9)
## Slope = -0.61 | FD = 1.61
##
## ~~~o~~o~~casnet~~o~~o~~~
```

```r
TS3_FDsda <- fd_sda(series$TS_3)$fitRange$FD
```

```
##
##
## fd_sda:	Sample rate was set to 1.
##
##
## ~~~o~~o~~casnet~~o~~o~~~
##
##  Standardised Dispersion Analysis
##
##  Full range (n = 10)
## Slope = -0.64 | FD = 1.64
##
##  Fit range (n = 9)
## Slope = -0.58 | FD = 1.58
##
## ~~~o~~o~~casnet~~o~~o~~~
```

```r
TS5_FDsda <- fd_sda(series$TS_5)$fitRange$FD
```

```
##
##
## fd_sda:	Sample rate was set to 1.
##
##
## ~~~o~~o~~casnet~~o~~o~~~
##
##  Standardised Dispersion Analysis
##
##  Full range (n = 10)
## Slope = -0.05 | FD = 1.05
##
##  Fit range (n = 9)
## Slope = -0.04 | FD = 1.04
##
## ~~~o~~o~~casnet~~o~~o~~~
```

* To create the surrogates using `fractal::surrogate()`
    + You'll need to repeatedly call the function and store the result
    + To get into a dataframe we have to use `unclass()` otherwise `R` doesn't recognise the object
    + Function `t()` transposes the data from 39 rows to 39 columns
* Once you have the surrogates in a datafram, repeatedly calculate the measure you want to compare, in this case `fd_sda()`.

```r
# NOW CREATE SURROGATES
library(plyr)
library(tidyverse)
library(fractal)

# For a two-sided test at alpha = .05 we need N=39
Nsurrogates <- 39

TS1surrogates <- as.data.frame(t(ldply(1:Nsurrogates, function(s) unclass(surrogate(x=series$TS_1,method="aaft")))))
colnames(TS1surrogates) <- paste0("S",1:NCOL(TS1surrogates))

# Add the observed data
TS1surrogates$Obs <- series$TS_1
plotTS_multi(TS1surrogates)
```

![](ASSIGNMENTS_scaling_files/figure-html/unnamed-chunk-12-1.png)<!-- -->

```r
library(fractal)
# Now we calculate FD for each series
TS1surrogates_FD <- plyr::laply(1:Nsurrogates,function(s) fd_sda(y=TS1surrogates[,s], silent = TRUE)$fitRange$FD)

TS3surrogates <- t(plyr::ldply(1:Nsurrogates, function(s) unclass(surrogate(x=series$TS_3 ,method="aaft"))))
colnames(TS3surrogates) <- paste0("aaftSurr",1:NCOL(TS3surrogates))
TS3surrogates_FD <- plyr::laply(1:Nsurrogates,function(s) fd_sda(y=TS3surrogates[,s], silent = TRUE)$fitRange$FD)

TS5surrogates <- t(plyr::ldply(1:Nsurrogates, function(s) unclass(surrogate(x=series$TS_5 ,method="aaft"))))
colnames(TS5surrogates) <- paste0("aaftSurr",1:NCOL(TS5surrogates))
TS5surrogates_FD <- plyr::laply(1:Nsurrogates,function(s) fd_sda(y=TS5surrogates[,s], silent = TRUE)$fitRange$FD)
```

* Collect the results in a dataframe, so we can compare.

```r
x <- data.frame(Source = c("TS1", "TS3", "TS5"),
                FDsda = c(TS1_FDsda, TS3_FDsda, TS5_FDsda),
                FDsdaAAFT.median= c(median(TS1surrogates_FD),median(TS3surrogates_FD),median(TS5surrogates_FD)))
```

* Call function `plotSUR_hist()` to see the results of the test.

```r
plotSUR_hist(surrogateValues = TS1surrogates_FD, observedValue = TS1_FDsda, sides = "two.sided", doPlot = TRUE, measureName = "sda FD TS1")
```

![](ASSIGNMENTS_scaling_files/figure-html/unnamed-chunk-15-1.png)<!-- -->

```r
plotSUR_hist(surrogateValues = TS3surrogates_FD, observedValue = TS3_FDsda, sides = "two.sided", doPlot = TRUE, measureName = "sda FD TS3")
```

![](ASSIGNMENTS_scaling_files/figure-html/unnamed-chunk-15-2.png)<!-- -->

```r
plotSUR_hist(surrogateValues = TS5surrogates_FD, observedValue = TS5_FDsda, sides = "two.sided", doPlot = TRUE, measureName = "sda FD TS5")
```

![](ASSIGNMENTS_scaling_files/figure-html/unnamed-chunk-15-3.png)<!-- -->

 * Let's print the results into a table

```r
x$FDsda <- c(TS1_FDsda,TS3_FDsda,TS5_FDsda)
x$FDsdaAAFT.median <- c(median(TS1surrogates_FD),median(TS3surrogates_FD),median(TS5surrogates_FD))
knitr::kable(x, digits = 2, booktabs=TRUE,formt="html")
```

Source    FDsda   FDsdaAAFT.median
-------  ------  -----------------
TS1        1.61               1.59
TS3        1.58               1.55
TS5        1.04               1.07

**FD: SPECTRAL SLOPE**

```r
# TS1,TS3,TS5
TS1_FDpsd <- fd_psd(series$TS_1)$fitRange$FD
```

```
##
##
## fd_psd:	Sample rate was set to 1.
##
##
## ~~~o~~o~~casnet~~o~~o~~~
##
##  Power Spectral Density Slope
##
##  All frequencies (n = 512)
## Slope = 0.18 | FD = 1.57
##
##  Hurvich-Deo (n = 36)
## Slope = 0.29 | FD = 1.61
##
## ~~~o~~o~~casnet~~o~~o~~~
```

```r
TS3_FDpsd <- fd_psd(series$TS_3)$fitRange$FD
```

```
##
##
## fd_psd:	Sample rate was set to 1.
##
##
## ~~~o~~o~~casnet~~o~~o~~~
##
##  Power Spectral Density Slope
##
##  All frequencies (n = 512)
## Slope = 0.12 | FD = 1.54
##
##  Hurvich-Deo (n = 32)
## Slope = 0.02 | FD = 1.51
##
## ~~~o~~o~~casnet~~o~~o~~~
```

```r
TS5_FDpsd <- fd_psd(series$TS_5)$fitRange$FD
```

```
##
##
## fd_psd:	Sample rate was set to 1.
##
##
## ~~~o~~o~~casnet~~o~~o~~~
##
##  Power Spectral Density Slope
##
##  All frequencies (n = 512)
## Slope = -1.91 | FD = 1.1
##
##  Hurvich-Deo (n = 73)
## Slope = -1.92 | FD = 1.1
##
## ~~~o~~o~~casnet~~o~~o~~~
```

* To create the surrogates using `fractal::surrogate()`
    + You'll need to repeatedly call the function and store the result
    + To get into a dataframe we have to use `unclass()` otherwise `R` doesn't recognise the object
    + Function `t()` transposes the data from 39 rows to 39 columns
* Once you have the surrogates in a dataframe, repeatedly calculate the measure you want to compare, in this case `fd_sda()`.

```r
# NOW CREATE SURROGATES
library(dplyr)

# For a two-sided test at alpha = .05 we need N=39
Nsurrogates <- 39

TS1surrogates <- t(ldply(1:Nsurrogates, function(s) unclass(fractal::surrogate(x=series$TS_1 ,method="aaft"))))
colnames(TS1surrogates) <- paste0("aaftSurr",1:NCOL(TS1surrogates))
TS1surrogates_FD <- laply(1:Nsurrogates,function(s){fd_psd(y=TS1surrogates[,s], silent = TRUE)$fitRange$FD})

TS3surrogates <- t(ldply(1:Nsurrogates, function(s) unclass(surrogate(x=series$TS_3 ,method="aaft"))))
colnames(TS3surrogates) <- paste0("aaftSurr",1:NCOL(TS3surrogates))
TS3surrogates_FD <- laply(1:Nsurrogates,function(s) fd_psd(y=TS3surrogates[,s], silent = TRUE)$fitRange$FD)

TS5surrogates <- t(ldply(1:Nsurrogates, function(s) unclass(surrogate(x=series$TS_5 ,method="aaft"))))
colnames(TS5surrogates) <- paste0("aaftSurr",1:NCOL(TS5surrogates))
TS5surrogates_FD <- laply(1:Nsurrogates,function(s) fd_psd(y=TS5surrogates[,s], silent = TRUE)$fitRange$FD)
```

* Call function `plotSUR_hist()` to get the results of the test.

```r
plotSUR_hist(surrogateValues = TS1surrogates_FD, observedValue = TS1_FDpsd, sides = "two.sided", doPlot = TRUE, measureName = "psd FD TS1")
```

![](ASSIGNMENTS_scaling_files/figure-html/unnamed-chunk-19-1.png)<!-- -->

```r
plotSUR_hist(surrogateValues = TS3surrogates_FD, observedValue = TS3_FDpsd,sides = "two.sided", doPlot = TRUE, measureName = "psd FD TS3")
```

![](ASSIGNMENTS_scaling_files/figure-html/unnamed-chunk-19-2.png)<!-- -->

```r
plotSUR_hist(surrogateValues = TS5surrogates_FD, observedValue = TS5_FDpsd,sides = "two.sided", doPlot = TRUE, measureName = "psd FD TS5")
```

![](ASSIGNMENTS_scaling_files/figure-html/unnamed-chunk-19-3.png)<!-- -->

 * Let's print the results into a table

```r
xx <-x
xx$FDpsd <- c(TS1_FDpsd,TS3_FDpsd,TS5_FDpsd)
xx$FDpsdAAFT.median <- c(median(TS1surrogates_FD),median(TS3surrogates_FD),median(TS5surrogates_FD))
knitr::kable(xx, digits = 2, booktabs=TRUE,formt="html")
```

Source    FDsda   FDsdaAAFT.median   FDpsd   FDpsdAAFT.median
-------  ------  -----------------  ------  -----------------
TS1        1.61               1.59    1.61               1.59
TS3        1.58               1.55    1.51               1.55
TS5        1.04               1.07    1.10               1.13

repository: complexity-methods/CSA-assignments

repo_name: complexity-methods/CSA-assignments


text:
# Sinopia Sample Data from verso
This directory and sub-directories contain JSON Profiles and the language RDF XML
from the Library of Congress [Verso][VERSO] middleware source code repository
that is used in the Sinopia Profile Editor project.

## Log

### 2019-01-22

Small manual updates for JSON schema v0.0.2 validity:

- adjusted defaultURI and defaultLiteral when they weren't inside defaults object
    - the only slightly questionable one was when a non-empty defaultLiteral was provided without a defaultURI -- see ec91e62 (Prints and Photographs profile)
- changed contact to author (894d6f5 and others)
- removed adherence and source attributes from Admin Metadata - too soon! (ecae94d)

### 2018-12-18

Small manual updates for JSON schema v0.0.1 validity:

- fixed bad type value in propertyTemplate Cartographic (8405430)
- fixed bad type value in propertyTemplate in Moving Image BluRay DVD a889153
- fixed Edition - author, not contact (03c5b14)
- got rid of empty (invalid) dataTypeURI strings by removing the valueDataType for those (nothing else is of consequence) (8a7ee4b)

### 2018-12-17
Update to ecb3c1e from https://github.com/lcnetdev/verso

### 2018-10-31
Add README.md

### 2018-10-30
Initial import of the latest profiles in [Verso][VERSO]. Here is the listing of
profiles from the SHA commit 67786e709e2aa207f6605e6a484b69d01acb9a60.

branch_name: refs/heads/master

extension: md

gha_language: HTML

license_type: permissive


text:
yarn package -c.mac.identity=null

  Electron React Boilerplate uses Electron, React, React Router, Webpack and React Fast Refresh.

[![Build Status][github-actions-status]][github-actions-url]
[![Dependency Status][david-image]][david-url]
[![DevDependency Status][david-dev-image]][david-dev-url]
[![Github Tag][github-tag-image]][github-tag-url]

[![OpenCollective](https://opencollective.com/electron-react-boilerplate/backers/badge.svg)](#backers)
[![OpenCollective](https://opencollective.com/electron-react-boilerplate/sponsors/badge.svg)](#sponsors)
[![Good first issues open][good-first-issue-image]][good-first-issue-url]
[![StackOverflow][stackoverflow-img]][stackoverflow-url]

## Install

- **If you have installation or compilation issues with this project, please see [our debugging guide](https://github.com/electron-react-boilerplate/electron-react-boilerplate/issues/400)**

First, clone the repo via git and install dependencies:

```bash
git clone --depth 1 --single-branch https://github.com/electron-react-boilerplate/electron-react-boilerplate.git your-project-name
cd your-project-name
yarn
```

## Starting Development

Start the app in the `dev` environment:

```bash
yarn start
```

## Packaging for Production

To package apps for the local platform:

```bash
yarn package
```

## Docs

See our [docs and guides here](https://electron-react-boilerplate.js.org/docs/installation)

## Donations

**Donations will ensure the following:**

- 🔨 Long term maintenance of the project
- 🛣 Progress on the [roadmap](https://electron-react-boilerplate.js.org/docs/roadmap)
- 🐛 Quick responses to bug reports and help requests

## Backers

Support us with a monthly donation and help us continue our activities. [[Become a backer](https://opencollective.com/electron-react-boilerplate#backer)]

## Sponsors

## Maintainers

## License

MIT © [Electron React Boilerplate](https://github.com/electron-react-boilerplate)

[github-actions-status]: https://github.com/electron-react-boilerplate/electron-react-boilerplate/workflows/Test/badge.svg
[github-actions-url]: https://github.com/electron-react-boilerplate/electron-react-boilerplate/actions
[github-tag-image]: https://img.shields.io/github/tag/electron-react-boilerplate/electron-react-boilerplate.svg?label=version
[github-tag-url]: https://github.com/electron-react-boilerplate/electron-react-boilerplate/releases/latest
[stackoverflow-img]: https://img.shields.io/badge/stackoverflow-electron_react_boilerplate-blue.svg
[stackoverflow-url]: https://stackoverflow.com/questions/tagged/electron-react-boilerplate
[david-image]: https://img.shields.io/david/electron-react-boilerplate/electron-react-boilerplate.svg
[david-url]: https://david-dm.org/electron-react-boilerplate/electron-react-boilerplate
[david-dev-image]: https://img.shields.io/david/dev/electron-react-boilerplate/electron-react-boilerplate.svg?label=devDependencies
[david-dev-url]: https://david-dm.org/electron-react-boilerplate/electron-react-boilerplate?type=dev
[good-first-issue-image]: https://img.shields.io/github/issues/electron-react-boilerplate/electron-react-boilerplate/good%20first%20issue.svg?label=good%20first%20issues
[good-first-issue-url]: https://github.com/electron-react-boilerplate/electron-react-boilerplate/issues?q=is%3Aopen+is%3Aissue+label%3A"good+first+issue"

license: MIT

branch_name: refs/heads/main


text:
---
layout: post
title:  "gem5 joins Season of Docs!"
author: Bobby R. Bruce
date:   03-05-2020
categories: project
---

![](/assets/img/season_of_docs.png)

We are pleased to announce that the gem5 project will be participating in this
year's [Season of Docs](https://developers.google.com/season-of-docs). Season
of Docs is a program, run by Google, with the goal of fostering collaboration
between open source projects and technical writers.

In this blog post we shall outline who we are and what kind of projects we are
looking for technical writers to undertake. During the exploration phase (May
11th -- June 8th), we shall begin collaborating with technical writers to
flesh-out project ideas, with the goal of having a technical writer join us
later in the year to undertake their proposed project.

More general information on Season of Docs can be found at
<https://developers.google.com/season-of-docs>.

# Who are we?

gem5 is an open source computer architecture simulator. The tool is used by a
number of organizations from university research labs to large
industrial players such as Google and ARM. While we have been successful in
developing a well engineered and popular product, documentation has been left
to software engineers with little technical writing experience. We therefore
wish to mentor a technical writer to improve, and expand our documentation.

The gem5 project is highly collaborative, and as such we accept contributions
from many individuals across the world. As a technical writer, you would be
joining this community, which we hope you'll find a rewarding experience.

# Interested?

For those interested in applying, we'd advise first looking through our
[current documentation](http://www.gem5.org/documentation/), in particular our
[Learning gem5](http://www.gem5.org/documentation/learning_gem5/introduction/)
tutorial, which should give the reader a step-by-step guide on how to setup and
run gem5. We would then like to hear suggestions from technical writers on how
best our documentation may be improved.

We have noted below some projects ideas. These are by no means final. We
expect an applicant to use these as starting points in order to find
interesting work within our project. As we want to utilize technical writing
expertise as fully as possible, we encourage technical writers to
speak up and reach out if if they have any suggestions. It will be our role, as
mentors, to give technical writers the context needed to refine these ideas
into something of use to the gem5 community.

We do not expect technical writers to fully understand the ins and outs of the
gem5 project. In fact, we want technical writers to use their naiveté to
their advantage. Our documentation has been written by those with familiarity
to the project. We thereby believe a "fresh pair of eyes" would be of benefit
in many areas. A new technical writer within the gem5 team should be able to
identify "pain points" in our documentation and help us improve them.

## Project Idea: Audit of current documentation

Our documentation has been developed over a long period of time, by different
people, with different goals. We therefore do not have a good understanding
of what areas of gem5 are documented sufficiently, and which are not. An
audit of our current documentation, and what needs further action, would be of
great benefit to the community. We can then generate a plan on how to improve
our documentation moving forward.

## Project Idea: Beginner's Guide Documentation

At present our [Learning gem5](
http://www.gem5.org/documentation/learning_gem5/introduction/) tutorial
documentation is our "entry point" for beginners to use gem5. We are unsure if
this documentation is sufficient, whether it should be expanded upon, or
if it should be replaced with something more concise. In this task we are
looking to have documentation which will improve how we introduce people to the
gem5 project.

Tangentially to this, our project regularly has new contributors, many
of whom need taught how to follow our contribution procedures. Typically we
direct wannabe contributors to our [contribution guidelines](
http://www.gem5.org/contributing) and our [CONTRIBUTION.md](
https://gem5.googlesource.com/public/gem5/+/refs/heads/stable/CONTRIBUTING.md)
document. As with our Learning gem5 documentation, we would like this
investigated and improved upon if need be.

## Project Idea: Improve Source-code Documentation

We currently document our codebase through [Doxygen](http://www.doxygen.nl/).
Like other areas of our documentation, [our Doxygen documentation](
doxygen.gem5.org) has been developed by many people, over
many years, thereby leaving much of it written in an inconsistent manner.
Furthermore, many areas of the code-base are entirely undocumented. This
project would in-part be an audit of what we have, then development of a plan
for improvement. We would also not be opposed to a technical writer beginning
work on this plan.

The source for gem5 can be found at
<https://gem5.googlesource.com/public/gem5/>.

## Project Idea: Improve the documentation website

At present, [our documentation](http://www.gem5.org/documentation/) is created
in a Markdown/Jekyll website. We have came across some limitations with this.
In particular, is not easily navigable. Those technical writers
with some web-dev/Jekyll experience would be of great use in coming up with an
alternative way to present our documentation --- new layouts, new styles, etc.

The source for our website can be found at
<https://gem5.googlesource.com/public/gem5-website/>.

# Mentors

A technical writers working within gem5 will be mentored by
[Dr. Bobby R. Bruce](https://www.bobbybruce.net) and [Prof. Jason Lowe-Power](
https://faculty.engineering.ucdavis.edu/lowepower/), both of who are situated
in the University of California, Davis. Please take note of the California
time-zone (assume standard 9AM to 5PM work hours) and how this would work for
you. We are willing accept applicants from anywhere in the world. Work does
not need to occur during California work hours, but virtual meetings will be
regular.

If interested in working on gem5, please contact Dr. Bruce at "bbruce -at-
ucdavis.edu" during the Season of Docs exploration phase (May 11th -- June
8th). Dr. Bruce can help you flesh out your ideas prior to submitting an
official application. More information on the Season of Docs application
process can be found on the [Season of Docs website](
https://developers.google.com/season-of-docs).


text:
# Provider MVC

Flutter Provider MVC Implementation.

## Getting Started

This project is the base of the Flutter package [provider_mvc](https://pub.dev/packages/provider_mvc)

For more information see this article
[Flutter Provider MVC<EMAIL_ADDRESS>
## Examples

Let's see some examples of MVC app.

### Counter App

As a [default example](https://github.com/aymentoumi/ProviderMVC/tree/master/example), we developed our version of the famous flutter counter app.

![image](https://raw.githubusercontent.com/aymentoumi/ProviderMVC/master/example/image/counter.gif)

#### Step 1

**Model** class definition.
Because our app is too basic we can even skip model declaration but for pedagogical reasons we will create one.
The model can be an easy PODO (Plain Old Dart Object).

```
class Value {
    int value = 0;
}
```

#### Step 2

**Controller** class definition:

```
class CounterController extends Controller {
  final Value _count = Value();

  int get count => _count.value;

  void reset() {
    _count.value = 0;
    update();
  }

  void increment() {
    _count.value++;
    update();
  }
}
```

#### Step 3

**View** class definition.

```
class CounterView extends View<CounterController> {
  CounterView() : super(CounterController());

  @override
  Widget builder(
          BuildContext context, CounterController controller, Widget widget) =>
      Column(
        mainAxisAlignment: MainAxisAlignment.center,
        children: <Widget>[
          Text(
            'You have pushed the button this many times:',
          ),
          Text(
            '${controller.count}',
            style: Theme.of(context).textTheme.headline4,
          ),
          RaisedButton(
            child: Text("Reset"),
            onPressed: controller.reset,
          )
        ],
      );
}
```

#### Step 4

The app class.

```
class CounterApp extends StatelessWidget {
  final _counter = CounterView();

  @override
  Widget build(BuildContext context) => Scaffold(
        appBar: AppBar(
          title: Text('Flutter Demo Home Page'),
        ),
        body: Center(
          child: _counter,
        ),
        floatingActionButton: FloatingActionButton(
          onPressed: () => _counter.controller.increment(),
          tooltip: 'Increment',
          child: Icon(Icons.add),
        ),
      );
}
```

#### Step 5

The main file.

```
final CounterApp counterApp = CounterApp();

void main() => runApp(
      MaterialApp(
          debugShowCheckedModeBanner: false,
          theme: ThemeData(
            primarySwatch: Colors.blue,
          ),
          home: counterApp),
    );
```

### Other examples

[**BG Color App**](https://github.com/aymentoumi/ProviderMVC/tree/master/Samples/bg_color)

![image](https://raw.githubusercontent.com/aymentoumi/ProviderMVC/master/Samples/bg_color/image/bg_color.gif)

[**Bulbs App**](https://github.com/aymentoumi/ProviderMVC/tree/master/Samples/bulbs)

![image](https://raw.githubusercontent.com/aymentoumi/ProviderMVC/master/Samples/bulbs/image/bulbs.gif)

[**Movie List App**](https://github.com/aymentoumi/ProviderMVC/tree/master/Samples/movie_list)

![image](https://raw.githubusercontent.com/aymentoumi/ProviderMVC/master/Samples/movie_list/image/movie_list.gif)

[**Github Portfolio App**](https://github.com/aymentoumi/ProviderMVC/tree/master/Samples/portfolio)

![image](https://raw.githubusercontent.com/aymentoumi/ProviderMVC/master/Samples/portfolio/image/portfolio.gif)

[**Project Management Board App**](https://github.com/aymentoumi/ProviderMVC/tree/master/Samples/project_board)

![image](https://raw.githubusercontent.com/aymentoumi/ProviderMVC/master/Samples/project_board/image/project_board.gif)

[**Community Feed App**](https://github.com/aymentoumi/ProviderMVC/tree/master/Samples/community_feed)

![image](https://raw.githubusercontent.com/aymentoumi/ProviderMVC/master/Samples/community_feed/image/community_feed.gif)

repository: baydim/ProviderMVC

repo_name: baydim/ProviderMVC


text:
## Web Component

### Features

Before use features you need set some values.

For Browserstack you need set USER_NAME and API_KEY in config.json file and run Grunt task update-keys or set USER_NAME and API_KEY in files:
* .gemini.yml
* browserstack.json

And for JFROG you need set API_KEY in Gruntfile.js and REGISTRY_URL in .bowerrc.

All features you can use via Grunt task:
* wct-test-local - run unit test via Web Component Tester(WCT) local;
* wct-test-remote - run unit test via Web Component Tester(WCT) remote on Browserstack Cloud service;
* gemini-local-update - updated reference screenshots via Gemini local(Use PhantomJS);
* gemini-local-test - run UI test via Gemini local(Use PhantomJS);
* gemini-remote-update - updated reference screenshots via Gemini remote on Browserstack Cloud service;
* gemini-remote-test - run UI test via Gemini remote on Browserstack Cloud service;
* performance-test - run performance test remote on Browserstack Cloud service;
* update-keys - update Browserstack credential from config.json file;
* private-deploy - deploy component to JFROG;
* lint - JSHint code check and create report;
* lint-console - JSHint code check to console;
* jscs-check - JSCS code check.

gha_language: JavaScript


text:
CAD to USD exchange rate prediction
================
Fanli Zhou
2020/6/20

-   [Summary](#summary)
-   [Methods](#methods)
    -   [Data](#data)
    -   [Analysis Methods](#analysis-methods)
-   [Results and Discussion](#results-and-discussion)
    -   [Data Visualization](#data-visualization)
    -   [Feature Engineering and Model
        Training](#feature-engineering-and-model-training)
    -   [Models Evaluation](#models-evaluation)
    -   [Conclusions](#conclusions)
-   [Limitations](#limitations)
-   [References](#references)

Summary
=======

In this project, I attempted to predict CAD to USD exchange rates in 30
days given previous exchange rates. This project is mainly for me to
practice analyzing time series data and applying machine learning
methods on those data. Through this project, I realized how difficult it
is to make prediction in practice and feature engineering has great
potentials in improving predictions. If time allows, I would like to try
more machine learning methods and try more feature engineering methods.

Methods
=======

Data
----

The CAD to USD exchange rates data used in this project are obtained
using the [Alpha Vantage
API](https://www.alphavantage.co/documentation/). The data include daily
CAD to USD exchange rates from 2018-01-01 to 2019-12-31. Only daily
close data are used for this project. Data were imputated with last
observation carried over imputation to fill any gaps.

Analysis Methods
----------------

ARIMA models, classic supervised learning models, and LSTM models were
used to predict CAD to USD exchange rates in 30 days given previous
exchange rates. The Python programming language (Van Rossum and Drake
2009), the R programming language (R Core Team 2019) and the following
Python/R packages were used: pandas (McKinney 2010), Numpy (Oliphant
2006), Scipy (McKinney 2010), Scikit-learn (Pedregosa et al. 2011),
Matplotlib (Hunter 2007), Statsmodels (Seabold and Perktold 2010), Keras
(Chollet and others 2015), Tensorflow (Abadi et al. 2016), pmdarima
(Smith and others, n.d.), tqdm (Costa-Luis 2019), LightGBM (Ke et al.
2017), requests (Chandra and Varanasi 2015), knitr (Xie 2014), tidyverse
(Wickham 2017), rjson (Couture-Beil 2018). The code used to perform the
analysis and create this report can be found here:
https://github.com/flizhou/CAD_USD_exchange_rate_predictor.

Results and Discussion
======================

The detailed analysis is in [`eda.ipynb`](../scripts/eda.ipynb).
Cumstomized functions are in [`functions.py`](../scripts/functions.py)

### Data Visualization

The time series is first split into train (670 observations), valid (30
observations) and test (30 observations) datasets.

![](../results/data_visual.png)

### Feature Engineering and Model Training

To better analyse the time series, I included lagged features, which
include data in previous dates, in the analysis. To evaluate models, I
calculate the `Mean Absolute Percentage Error` (`MAPE`), the
`Root Mean Squared Error` (`RMSE`), and the `Min-Max Error` between
predictions and observations.

1.  `ARIMA` Forecast

I tried `ARIMA` by hand, `auto-ARIMA` and `auto-SARIMA` to find the best
`ARIMA` model. The best model is a `ARIMA(2, 0, 0) x (0, 1, 1, 7)`
model.

![](../results/final_arima_2_0_0_x_1_1_1_7.png)

Figure 2. `ARIMA(2, 0, 0) x (0, 1, 1, 7)` Forcast.

1.  Classic Supervised Learning Forecast

The time series is not stationary. A stationary time series should have
constant mean and variance. So I removed the trend with differencing to
stabilize the mean and used log or Box-Cox transformation to stabilize
the variance. I tried `Random Forests`, `k-nearest neighbors` (`KNN`),
`Gradient Boosting Regressor`, `Light GBM Regressor` models.
Cross-validation was used to optimize `lag` and model hyperparameters.
The best model is a `Random Forests` model with Box-Cox transformation
of the data. The model achieved best performace with 12 lag features and
the hyperparameter, `n_estimators`, set to 100.

![](../results/final_random_forests_box-cox_transformation.png)

Figure 3. `Random Forests` (Box-Cox transformation) Forcast.

1.  `LSTM` Recurrent Neural Network Forecast

To train `LSTM` models, data were first transformed with `MinMAxScaler`
and ross-validation was used to optimize `lag`. The final `LSTM` uses
seven lags.

![](../results/final_lstm.png)

Figure 4. `LSTM` Forcast.

### Models Evaluation

Among the three final models, the `ARIMA(2, 0, 0) x (0, 1, 1, 7)` model
achieved the best scores.

|                                               |   MAPE |   RMSE | Min.Max.Error |
|:----------------------------------------------|-------:|-------:|--------------:|
| ARIMA(2, 0, 0) x (0, 1, 1, 7)                 | 0.0067 | 0.0061 |        0.0067 |
| Final Random Forests (Box-Cox transformation) | 0.0111 | 0.0097 |        0.0111 |
| Final LSTM                                    | 0.0170 | 0.0145 |        0.0170 |

Conclusions
-----------

In this analysis, I attempted to predict CAD to USD exchange rate in 30
days and my final best model is an `ARIMA(2, 0, 0) x (0, 1, 1, 7)`
model.

Limitations
===========

Models were only tested on the given test dataset and may not give good
predictions on new data. Besides, those models may not work well if new
external factors begin to affect the exchange rate. In this projectI
only tried lag features for feature engineering. There are great
potentials in feature engineering. If time allows, I could try other
feature engineering options such as rolling windows, time stamps and
external features.

References
==========

Abadi, Martı́n, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis,
Jeffrey Dean, Matthieu Devin, et al. 2016. “Tensorflow: A System for
Large-Scale Machine Learning.” In *12th
![\\{](https://latex.codecogs.com/png.latex?%5C%7B "\{")Usenix![\\}](https://latex.codecogs.com/png.latex?%5C%7D "\}")
Symposium on Operating Systems Design and Implementation
(![\\{](https://latex.codecogs.com/png.latex?%5C%7B "\{")Osdi![\\}](https://latex.codecogs.com/png.latex?%5C%7D "\}")
16)*, 265–83.

Chandra, Rakesh Vidya, and Bala Subrahmanyam Varanasi. 2015. *Python
Requests Essentials*. Packt Publishing Ltd.

Chollet, Francois, and others. 2015. “Keras.” GitHub. 2015.
<https://github.com/fchollet/keras>.

Costa-Luis, Casper O. da. 2019. “‘Tqdm‘: A Fast, Extensible Progress
Meter for Python and Cli.” *Journal of Open Source Software* 4 (37):
1277. <https://doi.org/10.21105/joss.01277>.

Couture-Beil, Alex. 2018. *Rjson: JSON for R*.
<https://CRAN.R-project.org/package=rjson>.

Hunter, J. D. 2007. “Matplotlib: A 2D Graphics Environment.” *Computing
in Science & Engineering* 9 (3): 90–95.
<https://doi.org/10.1109/MCSE.2007.55>.

Ke, Guolin, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma,
Qiwei Ye, and Tie-Yan Liu. 2017. “LightGBM: A Highly Efficient Gradient
Boosting Decision Tree.” In *NIPS*.

McKinney, Wes. 2010. “Data Structures for Statistical Computing in
Python.” In *Proceedings of the 9th Python in Science Conference*,
edited by Stéfan van der Walt and Jarrod Millman, 51–56.

Oliphant, Travis E. 2006. *A Guide to Numpy*. Vol. 1. Trelgol Publishing
USA.

Pedregosa, Fabian, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel,
Bertrand Thirion, Olivier Grisel, Mathieu Blondel, et al. 2011.
“Scikit-Learn: Machine Learning in Python.” *Journal of Machine Learning
Research* 12 (Oct): 2825–30.

R Core Team. 2019. *R: A Language and Environment for Statistical
Computing*. Vienna, Austria: R Foundation for Statistical Computing.
<https://www.R-project.org/>.

Seabold, Skipper, and Josef Perktold. 2010. “Statsmodels: Econometric
and Statistical Modeling with Python.” In *9th Python in Science
Conference*.

Smith, Taylor G., and others. n.d. “pmdarima: ARIMA Estimators for
Python.” <http://www.alkaline-ml.com/pmdarima>.

Van Rossum, Guido, and Fred L. Drake. 2009. *Python 3 Reference Manual*.
Scotts Valley, CA: CreateSpace.

Wickham, Hadley. 2017. *Tidyverse: Easily Install and Load the
’Tidyverse’*. <https://CRAN.R-project.org/package=tidyverse>.

Xie, Yihui. 2014. “Knitr: A Comprehensive Tool for Reproducible Research
in R.” In *Implementing Reproducible Computational Research*, edited by
Victoria Stodden, Friedrich Leisch, and Roger D. Peng. Chapman;
Hall/CRC. <http://www.crcpress.com/product/isbn/9781466561595>.


text:
# Code of Conduct

Above all, SQLAlchemy places great emphasis on polite, thoughtful, and
constructive communication between users and developers.
Please see our current Code of Conduct at
[Code of Conduct](https://www.sqlalchemy.org/codeofconduct.html).

repository: sqlalchemy/sqlalchemy

gha_language: Python

gha_license_id: MIT

repo_name: sqlalchemy/sqlalchemy


text:
---
layout: default
---

# 1997-09-08 - Miscellaneous Debris

## Header Data

From: Damaged Justice<EMAIL_ADDRESS>To<EMAIL_ADDRESS>Message Hash: 97ae822bb19ec7d31acc502dd0f0c3106e378a904ae0dc1e683a936a71a501c5

Message ID<EMAIL_ADDRESS>Reply To: _N/A_

UTC Datetime: 1997-09-08 09:07:40 UTC

Raw Date: Mon, 8 Sep 1997 17:07:40 +0800

## Raw message

```
{% raw  %}From: Damaged Justice<EMAIL_ADDRESS>Date: Mon, 8 Sep 1997 17:07:40 +0800
To<EMAIL_ADDRESS>Subject: Miscellaneous Debris
Message-ID<EMAIL_ADDRESS>MIME-Version: 1.0
Content-Type: text

Our old pinko buddy Vigdor Schriebman is the latest to attack Old Media's
definitions of "journalist" at

http://www.MediaINFO.com/ephome/news/newshtm/stories/090597n3.htm

Another self-contradicting bunch of viewpoints on net.telephony at

http://<IP_ADDRESS>/wire/news/1997/09/0905telco.html

New technology being put to old tricks as students plaigarize at

http://cnn.com/TECH/9709/07/illicit.papers/index.html

"The Internet Ushers In New Age Of Personal Diplomacy" at

http://www.infotech.co.nz/current/ntchil.html

          "Individuals around the globe can reach a consensus and make
          contact with others despite the differing political views of
          their governments. They can become "personal ambassadors" for
          their countries."

And the Infowarspooks are back for another round of "the Internet is so
important the government needs to do SOMETHING" at

http://www.nando.net/newsroom/ntn/info/090697/info7_3765_noframes.html

Enjoy! All of the above links courtesy of http://www.newslinx.com/

--
Tell your friends 'n neighbors you read this on the evil pornographic Internet
"Where one burns books, one will also burn people eventually." -Heinrich Heine
People and books aren't for burning. No more Alexandrias, Auschwitzs or Wacos.

{% endraw %}
```

## Thread

+ Return to [September 1997](/archive/1997/09)

+ Return to "[Damaged Justice <frogfarm@yakko.cs.wmich.edu>](/authors/damaged_justice_frogfarm_at_yakko_cs_wmich_edu_)"

+ 1997-09-08 (Mon, 8 Sep 1997 17:07:40 +0800) - Miscellaneous Debris - _Damaged Justice<EMAIL_ADDRESS>


text:
# Library

## Set up (Windows)

* Install [XAMPP](https://www.apachefriends.org/es/index.html)
* Install [Composer](https://getcomposer.org/download/)
* You will need npm (Node Package Manager) to install some node_modules.
So install [Nodejs](https://nodejs.org/es/) default installation includes npm).
* Clone this repo in your htdocs (Example: C:\xamp\htdocs\).
* **cd** to the library folder
* Run **composer install**.
* Run **npm install**.
* Run **npm run dev** to compile assets
* Copy the content of **.env.example** to a new file called **.env**
* Run **php artisan key:generate**
* Create a new database using phpMyAdmin and call it **library**.
* Edit the **.env** file to match your Database configuration such a`DB_HOST`, `DB_PORT`, `DB_DATABASE`, `DB_USERNAME`, and `DB_PASSWORD`.
* Run **php artisan migrate**
* Edit the file C:\xampp\apache\conf\extra\httpd-vhosts.conf and add the next tags at the end (Change the directory path to your own).
```HTML
<VirtualHost *:80>
    ServerAdmin mylibrary.dev
    DocumentRoot "C:/xampp/htdocs/library/public"
</VirtualHost>

<VirtualHost *:80>
    ServerAdmin localhost
    DocumentRoot "C:/xampp/htdocs/"
</VirtualHost>
```
* Edit C:\Windows\System32\drivers\etc\hosts *as administrator* and add the next lines at the end
```html
<IP_ADDRESS> localhost
<IP_ADDRESS> mylibrary.dev
```
* restart your apache server on XAMPP control.
* Now go to http://mylibrary.dev on your web browser.

### Notes
* Sometimes Chrome forces https connection. If so, your not going to be able to open the app, so try to use a different browser insted (like firefox). Or [create a Self Certificate and enable SSL](https://stackoverflow.com/questions/42951159/why-the-connection-appear-to-be-not-secure-on-my-php-webapp-depolyed-on-my-loc)
* You can run **php artisan serve** as well to see the web app on `http://localhost:8000`.

## Set up (Linux)
* Set a LAMP environment in your system. You can search for docs of how to do it in your own distro.
* Create a `/etc/apache2/sites-available/library.conf` file.
* Have Apache2 service and MySql or MariaDB service running.
* Install [Composer](https://getcomposer.org/download/)
* You will need npm (Node Package Manager) to install some node_modules.
So install [Nodejs](https://nodejs.org/es/) (default installation includes npm).
* **cd** to the library folder
* Run **composer install**.
* Run **npm install**.
* Copy the content of **.env.example** to a new file called **.env**
* Run **php artisan key:generate**
* Run **npm run dev** to compile assets.
* Create a new database called **library**.
* Edit the **.env** file to match your Database configuration such a`DB_HOST`, `DB_PORT`, `DB_DATABASE`, `DB_USERNAME`, and `DB_PASSWORD`.
* Run **php artisan migrate**
* Edit the `/etc/apache2/apache2.conf` file.
Add a Directory field like:
```html
<Directory /path/to/the/repository>
	Options Indexes FollowSymLinks
	AllowOverride All
	Require all granted
</Directory>
```
* Create a config file to the `/etc/apache2/sites-available/` directory.
Call it **library.conf** and add your configuration like:
```html
<VirtualHost *:80>.
    ServerName library.dev
    ServerAdmin webmaster@localhost
    # Your Custom folder
    DocumentRoot /path/to/the/repository/public
    <Directory /path/to/the/repository/public>
        Options Indexes FollowSymLinks
        AllowOverride All
        Require all granted
    </Directory>
    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log common
</VirtualHost>
```
* Change your apache2 site with **sudo a2dissite 000-default.conf**
* Enable the new configuration with **sudo a2ensite library.conf**
* Reload your service with **sudo service apache reload** or an equivalent for your distro.
* Now you can go to your browser on http://localhost or you can run **php artisan serve** on the repository directory to serve the web app on localhost:8000

### Notes
* Sometimes Chrome forces https connection. If so, your not going to be able to open the app, so try to use a different browser insted (like firefox). Or [create a Self Certificate and enable SSL](https://stackoverflow.com/questions/42951159/why-the-connection-appear-to-be-not-secure-on-my-php-webapp-depolyed-on-my-loc)
* You can run **php artisan serve** as well to see the web app on `http://localhost:8000`.
* There is a file called **migrate.sql** with the queries to create the database if you prefer instead of using migration.
* There is a file called **populateDB.sql** with queries to add categories and books to start with.

repository: OsbaldoGMederos/library

repo_name: OsbaldoGMederos/library


text:
zmattools
=========

Tools to generate and manipulate z-matrices

Badges
======
[![Build/Test Status](https://travis-ci.org/SGenheden/zmattools.svg?branch=master)](https://travis-ci.org/SGenheden/zmattools)

Description
===========
zmatools is a package for generating and manipulate the z-matrix of a small molecules

Currently it has two tools:

1. Generating the z-matrix from a structure file
2. Producing a trajectory when each dihedral is sampled in a specific range, assuming the rest of the atoms are fixed.

Installation
============
To install this tool, clone or download the git repository. Change to the downloaded directory and install the software with

```
python setup.py install
```

Examples
========

To find the available commands

```
zmat -h
```

To find the help of individual commands

```
zmat generate -h
```

To generate a z-matrix to be used with Gaussian

```
zmat generate model0.mol2 --gaussian
```

To obtain verbose information on the z-matrix generation

```
zmat generate model0.mol2 --verbosity 2
```

To produce a dihedral trajectory in the range -180 to 180 in 15 degrees increment

```
zmat traj model0.mol2
```

the trajectories will be in xyz-formats in ``model0_dihed_traj4.xyz``,
``model0_dihed_traj5.xyz``etc.

To produce a dihedral trajectory in the range -180 to 180 in 30 degrees increment

```
zmat traj model0.mol2 -d 30
```

repository: SGenheden/zmattools

repo_name: SGenheden/zmattools


text:
# Ramaricium flavomarginatum (Burt) Ginns, 1979 SPECIES

#### Status
ACCEPTED

#### According to
The Catalogue of Life, 3rd January 2011

#### Published in
Bot. Notiser 132(1): 97 (1979)

#### Original name
Coniophora flavomarginata Burt, 1926

### Remarks
null

repository: mdoering/backbone

repo_name: mdoering/backbone


text:
# 0.3.0 / 2016-05-04

Add support for upload and multi upload fields.

# 0.2.3 / 2016-04-07

Default check_box element values to false.

# 0.2.2 / 2016-02-23

Remove local type coercion using dry-data. We rely on a `Dry::Validation::Schema` to do this now. The form definition API has not yet changed, though. We still require field types to be specified, but there is no longer any restriction over what is entered. We'll remove this in a future release, once we can infer types from the schema.

# 0.2.1 / 2016-02-23

Fix issue where form could not be built with input data with native data types (it presuming input would be HTML form-style input everywhere).

Add default (empty) input data argument for `Form#build`. This allows you simply to call `MyForm.build` for creating a "new" form (i.e. one without any existing input data).

# 0.2.0 / 2016-02-22

Require a dry-validation schema for each form. The schema should completely represent the form's expected data structure.

Update the API to better support the two main use cases for a form. First, to prepare a form with sane, initial input from your database and send it to a view for rendering:

```ruby
my_form = MyForm.new(my_schema)
my_form.build(input) # returns a `Formalist::Form::Result`
```

Then, to receive the data from the posted form, coerce it and validate it (and potentially re-display the form with errors):

```ruby
my_form = MyForm.new(my_schema)
my_form.receive(input).validate # returns a `Formalist::Form::ValidatedResult`
```

The main differences are as such:

* `#build` expects already-clean (e.g. properly structured and typed) data
* `#receive` expects the kind of data that your form will submit, and will then coerce it according to the validation schema, and then send the sanitised output to `#build`
* Calling `#validate` on a result object will validate the input data and include any error messages in its AST

# 0.1.0 / 2016-01-18

Initial release.

repository: andrewcroome/formalist

gha_language: Ruby

repo_name: andrewcroome/formalist


text:
This application is a portfolio website that allows the owner of the website to upload and manage collections of images. An artist friend wanted a website that he could use to share his paintings with friends and family.

Technical Elements:
* React & Redux UI
* Node.js server
* Cloudinary for image hosting
* Digitalocean for site hosting

![Images from website](screenshot.png)

### Frontend

The React frontend communicates with the node.js API, which sends metadata about the images, including links to the cloudinary images. The admin features are only available via route. By navigating to baseurl/login, the user enters credentials and recieves a jwt, which is sent back with all post/save requests.

### Server

The node server is a simple REST api. I initially considered graphQL, but decided that the data structure was a classic case for REST architecture. There are CRUD routes for albums, images, and information. Albums and images are linked by a join table which also contains sorting order for displaying images within an album.

### Deployment

The site was deployed on a digitalocean droplet, where I installed git and pm2. I have been SSHing into the environment to pull down latest, which is kind of a pain. Next time, I will definitely try and integrate with Travis or Jenkins to make it less painful.

The site is hosted through NGINX reverse proxy, which serves up the static files and routes API requests to the node server. I set up environment variables while in the shell.

I used letsencrypt for a SSL cert and goDaddy for domain name.

Getting everything right on Nginx and SSL was not fun, but it seems to be working.

Check out the live site [here.](https://stephenrawls.com)

Run `node build.js` for build analyzer.

repository: nathandpeterson/portfolio-manager-react

repo_name: nathandpeterson/portfolio-manager-react


text:
---
id: selectors
title: Selectors
---

The [WebDriver Protocol](https://w3c.github.io/webdriver/) provides several selector strategies to query an element. WebdriverIO simplifies them to keep selecting elements simple. Please note that even though the command to query elements is called `$` and `$$`, they have nothing to do with jQuery or the [Sizzle Selector Engine](https://github.com/jquery/sizzle).

While there are so many different selectors available, only a few of them provide a resilient way to find the right element. For example, given the following button:

```html
<button
  id="main"
  class="btn btn-large"
  name="submission"
  role="button"
  data-testid="submit"
>
  Submit
</button>
```

We __do__ and __do not__ recommend the following selectors:

| Selector                                      | Recommended  | Notes                                                       |
| --------------------------------------------- | ------------ | ----------------------------------------------------------- |
| `$('button')`                                 | 🚨 Never      | Worst - too generic, no context.                            |
| `$('.btn.btn-large')`                         | 🚨 Never      | Bad. Coupled to styling. Highly subject to change.          |
| `$('#main')`                                  | ⚠ Sparingly | Better. But still coupled to styling or JS event listeners. |
| `$(() => document.queryElement('button'))` | ⚠ Sparingly | Effective querying, complex to write.                       |
| `$('button[name="submission"]')`              | ⚠ Sparingly | Coupled to the `name` attribute which has HTML semantics.   |
| `$('button[data-testid="submit"]')`           | ✅ Good       | Requires additional attribute, not connected to a11y.       |
| `$('aria/Submit')` or `$('button=Submit')`    | ✅ Always     | Best. Resembles how the user interacts with the page.       |

## CSS Query Selector

If not indicated otherwise, WebdriverIO will query elements using the [CSS selector](https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Selectors) pattern, e.g.:

```js reference useHTTPS
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/example.js#L7-L8
```

## Link Text

To get an anchor element with a specific text in it, query the text starting with an equals (`=`) sign.

For example:

```html reference
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/example.html#L3
```

You can query this element by calling:

```js reference useHTTPS
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/example.js#L16-L18
```

## Partial Link Text

To find a anchor element whose visible text partially matches your search value, query it by using `*=` in front of the query string (e.g. `*=driver`).

You can query the element from the example above by also calling:

```js reference useHTTPS
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/example.js#L24-L26
```

__Note:__ You can't mix multiple selector strategies in one selector. Use multiple chained element queries to reach the same goal, e.g.:

```js
const elem = await $('header h1*=Welcome') // doesn't work!!!
// use instead
const elem = await $('header').$('*=driver')
```

## Element with certain text

The same technique can be applied to elements as well.

For example, here's a query for a level 1 heading with the text "Welcome to my Page":

```html reference
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/example.html#L2
```

You can query this element by calling:

```js reference useHTTPS
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/example.js#L35-L36
```

Or using query partial text:

```js reference useHTTPS
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/example.js#L42-L43
```

The same works for `id` and `class` names:

```html reference
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/example.html#L4
```

You can query this element by calling:

```js reference useHTTPS
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/example.js#L45-L55
```

__Note:__ You can't mix multiple selector strategies in one selector. Use multiple chained element queries to reach the same goal, e.g.:

```js
const elem = await $('header h1*=Welcome') // doesn't work!!!
// use instead
const elem = await $('header').$('h1*=Welcome')
```

## Tag Name

To query an element with a specific tag name, use `<tag>` or `<tag />`.

```html reference
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/example.html#L5
```

You can query this element by calling:

```js reference useHTTPS
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/example.js#L61-L62
```

## Name Attribute

For querying elements with a specific name attribute you can either use a normal CSS3 selector or the provided name strategy from the [JSONWireProtocol](https://github.com/SeleniumHQ/selenium/wiki/JsonWireProtocol) by passing something like [name="some-name"] as selector parameter:

```html reference
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/example.html#L6
```

```js reference useHTTPS
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/example.js#L68-L69
```

__Note:__ This selector strategy it deprecated and only works in old browser that are run by the JSONWireProtocol protocol or by using Appium.

## xPath

It is also possible to query elements via a specific [xPath](https://developer.mozilla.org/en-US/docs/Web/XPath).

An xPath selector has a format like `//body/div[6]/div[1]/span[1]`.

```html reference
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/xpath.html
```

You can query the second paragraph by calling:

```js reference useHTTPS
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/example.js#L75-L76
```

You can use xPath to also traverse up and down the DOM tree:

```js reference useHTTPS
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/example.js#L78-L79
```

## Accessibility Name Selector

Query elements by their accessible name. The accessible name is what is announced by a screen reader when that element receives focus. The value of the accessible name can be both visual content or hidden text alternatives.

:::info

You can read more about this selector in our [release blog post](/blog/2022/09/05/accessibility-selector)

:::

### Fetch by `aria-label`

```html reference
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/aria.html#L1
```

```js reference useHTTPS
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/example.js#L86-L87
```

### Fetch by `aria-labelledby`

```html reference
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/aria.html#L2-L3
```

```js reference useHTTPS
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/example.js#L93-L94
```

### Fetch by content

```html reference
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/aria.html#L4
```

```js reference useHTTPS
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/example.js#L100-L101
```

### Fetch by title

```html reference
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/aria.html#L5
```

```js reference useHTTPS
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/example.js#L107-L108
```

### Fetch by `alt` property

```html reference
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/aria.html#L6
```

```js reference useHTTPS
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/example.js#L114-L115
```

## ARIA - Role Attribute

For querying elements based on [ARIA roles](https://www.w3.org/TR/html-aria/#docconformance), you can directly specify role of the element like `[role=button]` as selector parameter:

```html reference
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/aria.html#L13
```

```js reference useHTTPS
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/example.js#L131-L132
```

## ID Attribute

Locator strategy "id" is not supported in WebDriver protocol, one should use either CSS or xPath selector strategies instead to find elements using ID.

However some drivers (e.g. [Appium You.i Engine Driver](https://github.com/YOU-i-Labs/appium-youiengine-driver#selector-strategies)) might still [support](https://github.com/YOU-i-Labs/appium-youiengine-driver#selector-strategies) this selector.

Current supported selector syntaxes for ID are:

```js
//css locator
const button = await $('#someid')
//xpath locator
const button = await $('//*[@id="someid"]')
//id strategy
// Note: works only in Appium or similar frameworks which supports locator strategy "ID"
const button = await $('id=resource-id/iosname')
```

## JS Function

You can also use JavaScript functions to fetch elements using web native APIs. Of course, you can only do this inside a web context (e.g., `browser`, or web context in mobile).

Given the following HTML structure:

```html reference
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/js.html
```

You can query the sibling element of `#elem` as follows:

```js reference useHTTPS
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/example.js#L139-L143
```

## Deep Selectors

Many frontend applications heavily rely on elements with [shadow DOM](https://developer.mozilla.org/en-US/docs/Web/Web_Components/Using_shadow_DOM). It is technically impossible to query elements within the shadow DOM without workarounds. The [`shadow$`](https://webdriver.io/docs/api/element/shadow$) and [`shadow$$`](https://webdriver.io/docs/api/element/shadow$$) have been such workarounds that had their [limitations](https://github.com/Georgegriff/query-selector-shadow-dom#how-is-this-different-to-shadow). With the deep selector you can now query all elements within any shadow DOM using the common query command.

Given we have an application with the following structure:

![Chrome Example](https://github.com/Georgegriff/query-selector-shadow-dom/raw/main/Chrome-example.png "Chrome Example")

With this selector you can query the `<button />` element that is nested within another shadow DOM, e.g.:

```js reference useHTTPS
https://github.com/webdriverio/example-recipes/blob/e8b147e88e7a38351b0918b4f7efbd9ae292201d/selectors/example.js#L147-L149
```

## Mobile Selectors

For hybrid mobile testing, it's important that the automation server is in the correct *context* before executing commands. For automating gestures, the driver ideally should be set to native context. But to select elements from the DOM, the driver will need to be set to the platform's webview context. Only *then* can the methods mentioned above can be used.

For native mobile testing, there is no switching between contexts, as you have to use mobile strategies and use the underlying device automation technology directly. This is especially useful when a test needs some fine-grained control over finding elements.

### Android UiAutomator

Android’s UI Automator framework provides a number of ways to find elements. You can use the [UI Automator API](https://developer.android.com/tools/testing-support-library/index.html#uia-apis), in particular the [UiSelector class](https://developer.android.com/reference/androidx/test/uiautomator/UiSelector) to locate elements. In Appium you send the Java code, as a string, to the server, which executes it in the application’s environment, returning the element or elements.

```js
const selector = 'new UiSelector().text("Cancel").className("android.widget.Button")'
const button = await $(`android=${selector}`)
await button.click()
```

### Android DataMatcher and ViewMatcher (Espresso only)

Android's DataMatcher strategy provides a way to find elements by [Data Matcher](https://developer.android.com/reference/android/support/test/espresso/DataInteraction)

```js
const menuItem = await $({
  "name": "hasEntry",
  "args": ["title", "ViewTitle"]
})
await menuItem.click()
```

And similarly [View Matcher](https://developer.android.com/reference/android/support/test/espresso/ViewInteraction)

```js
const menuItem = await $({
  "name": "hasEntry",
  "args": ["title", "ViewTitle"],
  "class": "androidx.test.espresso.matcher.ViewMatchers"
})
await menuItem.click()
```

### Android View Tag (Espresso only)

The view tag strategy provides a convenient way to find elements by their [tag](https://developer.android.com/reference/android/support/test/espresso/matcher/ViewMatchers.html#withTagValue%28org.hamcrest.Matcher%3Cjava.lang.Object%3E%29).

```js
const elem = await $('-android viewtag:tag_identifier')
await elem.click()
```

### iOS UIAutomation

When automating an iOS application, Apple’s [UI Automation framework](https://developer.apple.com/library/prerelease/tvos/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/UIAutomation.html) can be used to find elements.

This JavaScript [API](https://developer.apple.com/library/ios/documentation/DeveloperTools/Reference/UIAutomationRef/index.html#//apple_ref/doc/uid/TP40009771) has methods to access to the view and everything on it.

```js
const selector = 'UIATarget.localTarget().frontMostApp().mainWindow().buttons()[0]'
const button = await $(`ios=${selector}`)
await button.click()
```

You can also use predicate searching within iOS UI Automation in Appium to refine element selection even further. See [here](https://github.com/appium/appium/blob/master/docs/en/writing-running-appium/ios/ios-predicate.md) for details.

### iOS XCUITest predicate strings and class chains

With iOS 10 and above (using the `XCUITest` driver), you can use [predicate strings](https://github.com/facebook/WebDriverAgent/wiki/Predicate-Queries-Construction-Rules):

```js
const selector = `type == 'XCUIElementTypeSwitch' && name CONTAINS 'Allow'`
const switch = await $(`-ios predicate string:${selector}`)
await switch.click()
```

And [class chains](https://github.com/facebook/WebDriverAgent/wiki/Class-Chain-Queries-Construction-Rules):

```js
const selector = '**/XCUIElementTypeCell[`name BEGINSWITH "D"`]/**/XCUIElementTypeButton'
const button = await $(`-ios class chain:${selector}`)
await button.click()
```

### Accessibility ID

The `accessibility id` locator strategy is designed to read a unique identifier for a UI element. This has the benefit of not changing during localization or any other process that might change text. In addition, it can be an aid in creating cross-platform tests, if elements that are functionally the same have the same accessibility id.

- For iOS this is the `accessibility identifier` laid out by Apple [here](https://developer.apple.com/library/prerelease/ios/documentation/UIKit/Reference/UIAccessibilityIdentification_Protocol/index.html).
- For Android the `accessibility id` maps to the `content-description` for the element, as described [here](https://developer.android.com/training/accessibility/accessible-app.html).

For both platforms, getting an element (or multiple elements) by their `accessibility id` is usually the best method. It is also the preferred way over the deprecated `name` strategy.

```js
const elem = await $('~my_accessibility_identifier')
await elem.click()
```

### Class Name

The `class name` strategy is a `string` representing a UI element on the current view.

- For iOS it is the full name of a [UIAutomation class](https://developer.apple.com/library/prerelease/tvos/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/UIAutomation.html), and will begin with `UIA-`, such as `UIATextField` for a text field. A full reference can be found [here](https://developer.apple.com/library/ios/navigation/#section=Frameworks&topic=UIAutomation).
- For Android it is the fully qualified name of a [UI Automator](https://developer.android.com/tools/testing-support-library/index.html#UIAutomator) [class](https://developer.android.com/reference/android/widget/package-summary.html), such `android.widget.EditText` for a text field. A full reference can be found [here](https://developer.android.com/reference/android/widget/package-summary.html).
- For Youi.tv it is the full name of a Youi.tv class, and will being with `CYI-`, such as `CYIPushButtonView` for a push button element. A full reference can be found at [You.i Engine Driver's GitHub page](https://github.com/YOU-i-Labs/appium-youiengine-driver)

```js
// iOS example
await $('UIATextField').click()
// Android example
await $('android.widget.DatePicker').click()
// Youi.tv example
await $('CYIPushButtonView').click()
```

## Chain Selectors

If you want to be more specific in your query, you can chain selectors until you've found the right element. If you call `element` before your actual command, WebdriverIO starts the query from that element.

For example, if you have a DOM structure like:

```html
<div class="row">
  <div class="entry">
    <label>Product A</label>
    <button>Add to cart</button>
    <button>More Information</button>
  </div>
  <div class="entry">
    <label>Product B</label>
    <button>Add to cart</button>
    <button>More Information</button>
  </div>
  <div class="entry">
    <label>Product C</label>
    <button>Add to cart</button>
    <button>More Information</button>
  </div>
</div>
```

And you want to add product B to the cart, it would be difficult to do that just by using the CSS selector.

With selector chaining, it's way easier. Simply narrow down the desired element step by step:

```js
await $('.row .entry:nth-child(2)').$('button*=Add').click()
```

### Appium Image Selector

Using the  `-image` locator strategy, it is possible to send an Appium an image file representing an element you want to access.

Supported file formats `jpg,png,gif,bmp,svg`

Full reference can be found [here](https://github.com/appium/appium/blob/master/docs/en/advanced-concepts/image-elements.md)

```js
const elem = await $('./file/path/of/image/test.jpg')
await elem.click()
```

**Note**: The way how Appium works with this selector is that it will internally make a (app)screenshot and use the provided image selector to verify if the element can be found in that (app)screenshot.

Be aware of the fact that Appium might resize the taken (app)screenshot to make it match the CSS-size of your (app)screen (this will happen on iPhones but also on Mac machines with a Retina display because the DPR is bigger than 1). This will result in not finding a match because the provided image selector might have been taken from the original screenshot. You can fix this by updating the Appium Server settings, see the [Appium docs](https://github.com/appium/appium/blob/master/docs/en/advanced-concepts/image-elements.md#related-settings) for the settings and [this comment](https://github.com/webdriverio/webdriverio/issues/6097#issuecomment-726675579) on a detailed explanation.

## React Selectors

WebdriverIO provides a way to select React components based on the component name. To do this, you have a choice of two commands: `react$` and `react$$`.

These commands allow you to select components off the [React VirtualDOM](https://reactjs.org/docs/faq-internals.html) and return either a single WebdriverIO Element or an array of elements (depending on which function is used).

**Note**: The commands `react$` and `react$$` are similar in functionality, except that `react$$` will return *all* matching instances as an array of WebdriverIO elements, and `react$` will return the first found instance.

#### Basic example

```jsx
// index.jsx
import React from 'react'
import ReactDOM from 'react-dom'

function MyComponent() {
    return (
        <div>
            MyComponent
        </div>
    )
}

function App() {
    return (<MyComponent />)
}

ReactDOM.render(<App />, document.querySelector('#root'))
```

In the above code there is a simple `MyComponent` instance inside the application, which React is rendering inside a HTML element with `id="root"`.

With the `browser.react$` command, you can select an instance of `MyComponent`:

```js
const myCmp = await browser.react$('MyComponent')
```

Now that you have the WebdriverIO element stored in `myCmp` variable, you can execute element commands against it.

#### Filtering components

The library that WebdriverIO uses internally allows to filter your selection by props and/or state of the component. To do so, you need to pass a second argument for props and/or a third argument for state to the browser command.

```jsx
// index.jsx
import React from 'react'
import ReactDOM from 'react-dom'

function MyComponent(props) {
    return (
        <div>
            Hello { props.name || 'World' }!
        </div>
    )
}

function App() {
    return (
        <div>
            <MyComponent name="WebdriverIO" />
            <MyComponent />
        </div>
    )
}

ReactDOM.render(<App />, document.querySelector('#root'))
```

If you want to select the instance of `MyComponent` that has a prop `name` as `WebdriverIO`, you can execute the command like so:

```js
const myCmp = await browser.react$('MyComponent', {
    props: { name: 'WebdriverIO' }
})
```

If you wanted to filter our selection by state, the `browser` command would looks something like so:

```js
const myCmp = await browser.react$('MyComponent', {
    state: { myState: 'some value' }
})
```

#### Dealing with `React.Fragment`

When using the `react$` command to select React [fragments](https://reactjs.org/docs/fragments.html), WebdriverIO will return the first child of that component as the component's node. If you use `react$$`, you will receive an array containing all the HTML nodes inside the fragments that match the selector.

```jsx
// index.jsx
import React from 'react'
import ReactDOM from 'react-dom'

function MyComponent() {
    return (
        <React.Fragment>
            <div>
                MyComponent
            </div>
            <div>
                MyComponent
            </div>
        </React.Fragment>
    )
}

function App() {
    return (<MyComponent />)
}

ReactDOM.render(<App />, document.querySelector('#root'))
```

Given the above example, this is how the commands would work:

```js
await browser.react$('MyComponent') // returns the WebdriverIO Element for the first <div />
await browser.react$$('MyComponent') // returns the WebdriverIO Elements for the array [<div />, <div />]
```

**Note:** If you have multiple instances of `MyComponent` and you use `react$$` to select these fragment components, you will be returned an one-dimensional array of all the nodes. In other words, if you have 3 `<MyComponent />` instances, you will be returned an array with six WebdriverIO elements.

## Custom Selector Strategies

If your app requires a specific way to fetch elements you can define yourself a custom selector strategy that you can use with `custom$` and `custom$$`. For that register your strategy once in the beginning of the test:

```js
browser.addLocatorStrategy('myCustomStrategy', (selector, root) => {
    /**
     * scope should be document if called on browser object
     * and `root` if called on an element object
     */
    const scope = root ? root : document
    return scope.querySelectorAll(selector)
})
```

Given the following HTML snippet:

```html
<div class="foobar" id="first">
    <div class="foobar" id="second">
        barfoo
    </div>
</div>
```

Then use it by calling:

```js
const elem = await browser.custom$('myCustomStrategy', '.foobar')
console.log(await elem.getAttribute('id')) // returns "first"
const nestedElem = await elem.custom$('myCustomStrategy', '.foobar')
console.log(await elem.getAttribute('id')) // returns "second"
```

**Note:** this only works in an web environment in which the [`execute`](/docs/api/browser/execute) command can be run.

repository: webdriverio/webdriverio

gha_language: TypeScript

repo_name: webdriverio/webdriverio


text:
# #495 Negative Feedback Amplifier

Examining the design and performance of an ac-coupled negative feedback amplifier constructed with BJTs.

![Build](./assets/NegativeFeedbackAmplifier_build.jpg?raw=true)

## Notes

Negative feedback is by definition the concept of using a fraction of the output to dampen the input.
When designed appropriately, this can trade some gain for higher linearity and stability,
and reduced sensitivity to component parameter variations.

## Circuit

The circuit design used here is based on a fairly common "Negative Feedback Amplifier Kit" (easily found on Aliexpress or eBay).
It essentially a two-stage common-emitter amplifier with a feedback loop from the output of the final stage to the emitter of the first stage.
NB: the circuit is exactly as covered in 10-5 Two-Stage Feedback of
[Electronic Principles by Albert Paul Malvino](https://www.goodreads.com/book/show/942642.Electronic_Principles).

The feedback loop creates an AC voltage divider between the output and ground:

    ve = RF1/(RF1 + RF2) . vo

Thus the negative feedback: the first stage output is reduced in proportion to an increase in the second stage output

![Schematic](./assets/NegativeFeedbackAmplifier_schematic.jpg?raw=true)

See [LEAP#389 TwoStageCommonEmitterAmplifier](../TwoStageCommonEmitterAmplifier), for a similar two-stage CE amplifier but without feedback.

## Breadboard Construction

![Breadboard](./assets/NegativeFeedbackAmplifier_bb.jpg?raw=true)

![NegativeFeedbackAmplifier_bb_build](./assets/NegativeFeedbackAmplifier_bb_build.jpg?raw=true)

Scope trace with a 1kHz 200mV peak-peak input and finely adjusted bias and feedback:

CH1 (Yellow): input
CH2 (Blue): output

![scope_bb](./assets/scope_bb.gif?raw=true)

## Protoboard Construction

![protoboard_layout_and_board](./assets/protoboard_layout_and_board.jpg?raw=true)

Under test:

![under_test](./assets/under_test.jpg?raw=true)

Scope trace with a 1kHz 200mV peak-peak input and finely adjusted bias and feedback:

CH1 (Yellow): input
CH2 (Blue): output

![scope_out](./assets/scope_out.gif?raw=true)

## Credits and References

* [Negative-feedback amplifier](https://en.wikipedia.org/wiki/Negative-feedback_amplifier) - wikipedia
* [Chapter 4 - Bipolar Junction Transistors - Feedback](https://www.allaboutcircuits.com/textbook/semiconductors/chpt-4/feedback/)
* [10.1 Cascade of two single transistor stages](https://wiki.analog.com/university/courses/electronics/text/chapter-10) - Analog wiki
* [Design of 2 stage BJT CE amplifier](http://ampdesigns.tripod.com/2_Stage_BJT_amplifier.html)
* [Electronic Principles by Albert Paul Malvino](https://www.goodreads.com/book/show/942642.Electronic_Principles)
* [Multistage Transistor Amplifiers](http://www.youtube.com/watch?v=FbdZ46VdTjE)
* [LEAP#389 TwoStageCommonEmitterAmplifier](../TwoStageCommonEmitterAmplifier)

repository: tardate/LittleArduinoProjects

repo_name: tardate/LittleArduinoProjects


text:
---
layout: post
status: publish
published: true
title: How to install java and set JAVA_HOME correctly in Ubuntu
date: 2008-03-18 13:46:00.000000000 +01:00
categories:
- hacks
tags: []
---

Install Java 6 like this: (Note: the package is in *multiverse*)

    sudo apt-get install sun-java6-jdk

Edit `/etc/jvm`, add this line to the top of the list: `/usr/lib/jvm/java-6-sun`

Thanks to the above step, the `java` executable will find the *right* version of java. You can confirm this with `java -version`. However, setting `JAVA_HOME` is a completely different matter.

These commands will detect, set and verify `JAVA_HOME`:

    . /usr/share/java-common/java-common.sh
    eval $(jvm_config)
    export JAVA_HOME
    echo $JAVA_HOME

repository: janosgyerik/homepage

branch_name: refs/heads/gh-pages

extension: markdown

repo_name: janosgyerik/homepage


text:
For the next major version of Styled UI, the HelpBox component has been rebuilt to use Styled System's theme and variant APIs.

You can opt in to the new API now by importing `{ HelpBox } from '@faithlife/styled-ui/v6'`. When v6 is released, the `/v6` entrypoint will continue to be supported with a deprecation warning until v7 is released.

## Colors and Their Meaning

```react
showSource: true
---
<Stack spacing={3}>
	<HelpBox variant="primary" handleClose={() => true}>
		<HelpBox.Body>This is a helpful alert.</HelpBox.Body>
		<HelpBox.Footer><Button variant="primary" size="small">CTA</Button></HelpBox.Footer>
	</HelpBox>
	<HelpBox variant="danger" handleClose={() => true}>
		<HelpBox.Body>This is an error alert.</HelpBox.Body>
		<HelpBox.Footer><Button variant="primary" size="small">CTA</Button></HelpBox.Footer>
	</HelpBox>
	<HelpBox variant="success" handleClose={() => true}>
		<HelpBox.Body>This is a successful alert.</HelpBox.Body>
		<HelpBox.Footer><Button variant="primary" size="small">CTA</Button></HelpBox.Footer>
	</HelpBox>
	<HelpBox variant="warning" handleClose={() => true}>
		<HelpBox.Body>This is a cautious alert.</HelpBox.Body>
		<HelpBox.Footer><Button variant="primary" size="small">CTA</Button></HelpBox.Footer>
	</HelpBox>
	<HelpBox variant="minor" handleClose={() => true}>
		<HelpBox.Body>This is a minor alert.</HelpBox.Body>
		<HelpBox.Footer><Button variant="primary" size="small">CTA</Button></HelpBox.Footer>
	</HelpBox>
</Stack>
```

## Variations

```react
showSource: true
---
<Stack spacing={3}>
	<HelpBox showLightBulb handleClose={() => true}>This is an alert with a light bulb.</HelpBox>
	<HelpBox hideIcon handleClose={() => true}>This alert has its icon hidden.</HelpBox>
	<HelpBox showRightIcon>This alert is showing its icon on both sides.</HelpBox>
	<HelpBox stacked handleClose={() => true}>
		<HelpBox.Body>This alert's contents are stacked.</HelpBox.Body>
		<HelpBox.Footer><Button variant="primary" size="small">CTA</Button></HelpBox.Footer>
	</HelpBox>
	<HelpBox>
		<HelpBox.Body>This alert doesn't handle closing.</HelpBox.Body>
		<HelpBox.Footer><Button variant="primary" size="small">CTA</Button></HelpBox.Footer>
	</HelpBox>
</Stack>
```

## Large Alerts

```react
showSource: true
---
<Stack spacing={3}>
	<HelpBox large handleClose={() => true}>This is a large alert.</HelpBox>
	<HelpBox showLightBulb large handleClose={() => true}>This is a large alert with a light bulb.</HelpBox>
</Stack>
```

repository: zachsnoek/styled-ui

repo_name: zachsnoek/styled-ui