text:
ASAP Operators          {#mainpage}
==============

This document describes the ASAP library of data analytics operators and the Swan parallel programming language on which it is built. The library defines data types, classes and methods that are helpful in data analytics. The code is written using the Swan language to indicate opportunities for parallelisation.

# Swan: An Extension of the Cilk Parallel Programming Language

Swan is an extension to the Cilk parallel programming language,
which was originally designed at the Supercomputing Technologies group at MIT,
and is currently supported by Intel
under the name [Cilk Plus](https://www.cilkplus.org).
Swan extends Cilk by adding dataflow dependences to express more complex
parallel patterns than Cilk. One of these is pipeline parallelism.
Moreover, Swan adds annotations to parallel for loops that help to
increase performance.

## The Cilk Parallel Programming Language

The following is a brief overview of the key aspects of Cilk.
Full details are provided
[online](https://www.cilkplus.org/cilk-documentation-full).

### Spawn and Sync

Parallelism is expressed by indicating that two (or more) pieces of code
may execute in parallel. Typically, this implies that these pieces of code
do not write to variables or memory locations that the other reads or writes to.
These pieces of code may be any legal C/C++ code. In practice, Cilk requires
that at least one piece of code is extracted in a function or isolated
in a C++ lambda expression (an anonymous function).

Parallelism is introduced by adding the `cilk_spawn` keyword to the function
call statement. We say that the function is _spawned_ rather than _called_.
The spawned function may execute in parallel with the remainder
of the calling function. This is called the _continuation_ of the calling
function. The parallelism exists until
a `cilk_sync` statement is encountered, or until the end of the calling
function, whichever is encountered first.

The Cilk runtime is allowed to execute the spawned function with the
continuation of the calling function, but is not obliged to do so.
In fact, the Cilk runtime only executes in parallel as many spawns
as is required to keep all CPU cores busy. Beyond this, it executes
the spawned statements in a sequential manner, as this is much more efficient.
Adding spawn statements to a program, thus, has little overhead in case
they are not selected for parallel execution by the runtime.

The following is an example of Quicksort expressed in Cilk ([source: https://github.com/tjd/sqlite_cpp/blob/master/cilk_test/qsort/qsort.cilk](https://github.com/tjd/sqlite_cpp/blob/master/cilk_test/qsort/qsort.cilk)):

```
void qsort(int *begin, int *end) {
    if (begin != end) {
        --end;  // Exclude last element (pivot) from partition
        int * middle = std::partition(begin, end,
                          std::bind2nd(std::less<int>(), *end));
        std::swap(*end, *middle);    // move pivot to middle
        cilk_spawn qsort(begin, middle);
        qsort(++middle, ++end); // Exclude pivot and restore end
        cilk_sync;
    }
}
```

Quicksort first partitions the range of values to sort using a pivot.
It then recursively sorts the range of values less than the pivot
and the range of values larger than the pivot. As these subranges
are independent (non-overlapping), they can be sorted in parallel.
This is indicated by labelling the first recursive call with `cilk_spawn`.

Note that one could also add the `cilk_spawn` keyword to the second
recursive call. This is however redundant as it was already apparent that
this call may execute in parallel with the first recursive call.

### Parallel for loops
The spawn/sync mechanism is very versatile. It can be used to express
parallel for loops, a common idiom, as well. However, this is somewhat
tedious. The Cilk compiler allows programmers to annotate parallel for
loops using the `cilk_for` keyword, e.g.:

```
cilk_for(int i=0; i < n; ++i) {
     a[i] = ...;
}
```

The compiler outlines the body of a `cilk_for` loop in a distinct function
and generates code to call the loop body in parallel, using the `cilk_spawn`
statement.

Every iteration of the loop should modify distinct memory locations.
There are moreover restrictions on the structure of the loop iteration.
In essence, the number of iterations of the loop must be known at execution
time just before starting the loop. This implies that the loop should not
have `break` statements and that the loop iteration variable (`i`) is
modified only by the loop increment statement (`++i`, the third part in the
for loop syntax).

### Generalized Reductions
Cilk provides definitions for generalized reductions
that are associative but not necessarily commutative.
As the reduction operation need not be commutative, many operations
such as list prepend/append and hash-map insert can now be expressed
as reduction operations. In these cases it is guaranteed that the reduction
variable contains the same value as computed by the serial elision
of the Cilk program.

Cilk defines reductions with three components:
a data type, an associative operation and an identity value for that operation.
These components are defined in a monoid class that serves as
the basis for a _reducer_ class definition.

The following is the definition of a Cilk reducer
for a hash-map data type:
```
template<class map_type>
class map_reducer {
   struct Monoid : cilk::monoid_base<map_type> (-*\label{lst:red:map:monoid}*-){
      static void reduce(map_type * left, map_type * right) {(-*\label{lst:red:map:reduce}*-)
         for(typename map_type::const_iterator
             I=right->cbegin(), E=right->cend(); I != E; ++I)
            (*left)[I->first] += I->second;
         right->clear();
      }
      static void identity(map_type * p) const {(-*\label{lst:red:map:identity}*-)
         new (p) map_type();
      }
   };
   cilk::reducer<Monoid> imp_;(-*\label{lst:red:map:imp}*-)

public:
   map_reducer() : imp_() { }
   typename map_type::value_type & operator[](
          const typename map_type::key_type & key) {
      return imp_.view()[key];
   }
   typename map_type::const_iterator cbegin() {
      return imp_.view().cbegin();
   }
   typename map_type::const_iterator cend() {
      return imp_.view().cend();
   }
   void swap(map_type & other) {
      return imp_.view().swap(other);
   }
   map_type & get_value() {
      return imp_.view();
   }
};
```

It is assumed that the template parameter
`map_type` defines a hash-map type that
is compatible to the C++ standard's `std::map`.
The definition consists of a Monoid class,
which defines the base type
(through the `monoid_base` template parameter),
the identity
value (through an initialization function) and the reduction function.
It is assumed that hash-maps are reduced
by taking the join of all keys and that the values for common keys
are further reduced using an `operator +=`.
This behavior is specified in the `reduce` function.

The runtime system dynamically creates copies of the reduction variable,
and reduces those copies as needed. These copies are called _views_.
A view is created for a worker thread when it first accesses
the reduction variable. The view is initialized with the identity element.
The worker retains the view when spawning a task. When
an idle worker steals a continuation from another worker's deque, it does
not receive a view for that reduction variable. The view is created only on the
first access to the reduction variable.
When a worker completes a spawned task leaving its spawn deque
empty, or when a worker executes a `cilk_sync` statement,
the view is reduced with that of a sibling task.

The example above defines a `map_reducer` class.
The member value `imp_` is declared as an instance of the
`reducer` class, specialized by the
`Monoid` definition.
The object `imp_` manages the creation, lookup and destruction of views.
The `map_reducer` class further provides access to the underlying
view through the `operator []` in order to add items to the
hash-map.

The `map_reducer` class may be used in parallel code as follows:
```
map_reducer<std::map<std::string,size_t>> map;
cilk_for(std::vector<std::string>::const_iterator
         I=vec.cbegin(); I != vec.cend(); ++I) {
   map[*I]++;
}
```

The `cilk_for` construct
creates parallelism. Each concurrently executing loop iteration
references the same instance of the `map_reducer` class,
but the `cilk::reducer` object `imp_` serves up
different views in concurrently executing iterations.
All views are reduced prior to
completion of the `cilk_for` loop.

Note that the reduction operation should ideally execute in
constant-time, otherwise the execution time of the program will depend
on the number of reduction operations performed.
The number of reduction operations is, in any case,
proportional to the number of steal operations.

### Array Notation

Cilk Plus supports an array notation that facilitates
auto-vectorization, i.e., the use of SIMD vector instructions
to accelerate processing. The array notation allows
for 3 fields in an array section expression: `a[i:l:s]`, where
`i` is the start index of the array section,
`l` is the length and
`s` is the stride. Each element of the array notation is optional,
but at least one colon must be present.
Default values are
0 for `i`,
the length of the array for `l`, provided it is known at compile-time,
and 1 for `s`.
E.g., `a[:]` indicates the full array if its size is statically known,
while `a[:10:2]` indicates the elements at indices 0, 2, 4, 6, 8.

Expressions may be built up using array notations, e.g.,
the statement
`c[:] = a[:]+2*b[:];` is equivalent to
```
for(int i=0; i < n; ++i)
    c[i] = a[i] + 2*b[i];
```
assuming each array was declared with length `n`.

One can also map functions over all elements of an array section.
E.g., `a[:] = pow(b[:])` applies the function `pow` to each
element of array `b`
and stores the result in the corresponding element of array `a`.
Reductions are specified using built-in functions that may be applied
to arbitrary array sections. E.g.,
`__sec_reduce_add(a[::2])` returns the sum of the array elements
at even positions of `a`.

The key advantage of the array notation is that it enables the compiler
to auto-vectorize the code. Vectorization can be important towards performance
as map-reduce programs often exhibit a data streaming pattern.

## Swan's Extensions to Cilk

### Dataflow dependences

Dependencies are tracked at the object level.
An object must be declared as a `versioned` object
in order to enable dependency tracking.
Versioned objects
support automatic tracking of dependencies as well as creating
new versions of the object in order to increase task-level parallelism
(a.k.a. renaming).

Dependency tracking is enabled on tasks that
take particular types as arguments:
the `indep`, `outdep` and `inoutdep` types.
These types are little more than a wrapper around
a versioned object that extends its type with the memory access mode
of the task: input, ouput or input/output (in/out).
The language allows only to pass versioned objects to such
arguments.

When spawning a task, the scheduler analyzes the signature of the spawned
procedure for arguments with a memory access mode. If none of the arguments
describe a memory access mode, then the spawn statement is an
_unconditional spawn_ and it has the same semantics as a
Cilk spawn.
Otherwise, the spawn statement
is a _conditional spawn_.
The memory accesses of the task are tracked and,
depending on runtime conditions, the task either executes
immediately or it is queued up in a set of pending tasks.

The `sync` statement in our language has the same semantics
as the Cilk sync statement: it postpones the execution of a procedure
until all child tasks have finished execution.

We consider only situations where dependencies are tracked between the
children of a single parent procedure. Each dynamic procedure instance
may have a task graph that restricts the execution order of its children.
This restriction ensures that
all parallel executions compute the same value as the sequential
elision of the program.
Note that the sequential elision of the program always respects the dependencies
in the program: by deducing dependencies from input/output properties,
there can never be backward dependencies in the sequential elision.
Furthermore, by having multiple independent task graphs in a program,
we can mitigate the performance impact of building the task graph in
serial fashion.

Our model allows arbitrarily mixing fork/join style and task graph execution.
The only problematic issue to allow this is that we must take care
when nesting task graphs, in particular when passing versioned objects
across multiple dependent spawns. To make this work correctly, we must
use distinct metadata for every dependent spawn
to track its dependencies separately.

The following is an example of square matrix multiplication expressed
in Swan using runtime tracking and enforcement of task dependencies.
Here, the matrix multiplication is performed _by blocks_, i.e., matrices
are partitioned in sub-blocks and parallelism between operations
on sub-blocks is made explicit using data-flow annotations.

```
typedef float (*block_t)[16]; // 16x16 tile
typedef swan::versioned<float[16][16]> vers_block_t;
typedef swan::indep<float[16][16]> in_block_t;
typedef swan::inoutdep<float[16][16]> inout_block_t;

void mul_add(in_block_t A, in_block_t B, inout_block_t C) {
    block_t a = (block_t)A; // Recover pointers
    block_t b = (block_t)B; // to the raw data
    block_t c = (block_t)C; // from the versioned objects
    // ... serial implementation on a 16x16 tile ...
}

void matmul(vers_block_t * A, vers_block_t * B,
             vers_block_t * C, unsigned n) {
    for( unsigned i=0; i < n; ++i ) {
        for( unsigned j=0; j < n; ++j ) {
            for( unsigned k=0; k < n; ++k ) {
                cilk_spawn mul_add( (in_block_t)A[i*n+j],
                                 (in_block_t)B[j*n+k],
                                 (inout_block_t)C[i*n+k] );
            }
        }
    }
    cilk_sync;
}
```

### Performance hints for parallel for loops

Swan adds two annotations to `cilk_for` loops. These annotations
inform the runtime how to schedule tasks on CPU cores most efficiently.

The _NUMA_ annotation facilitates performance tuning for systems with
a Non-Uniform Memory Architecture (NUMA), e.g., multi-socket machines.
This annotation indicates that the iterations of the loop should be
scheduled on distinct NUMA domains (sockets). It is the programmer's
responsibility to ensure that there are no more loop iterations than
NUMA domains.

Example usage:
```
    int chunk = (len + num_numa_domains - 1) / num_numa_domains;
#pragma cilk numa(strict)
    cilk_for(int d=0; d < num_numa_domains; ++d)
    	cilk_for(int i=d*chunk; i < std::min((d+1)*chunk,len); ++i)
	    a[i] = ...;
```

In the example above, the outer loop (with loop iteration variable `d`)
is annotated as a NUMA loop. Each iteration of this loop will be executed
on a distinct NUMA domain. The iterations of the inner loop
(using loop iteration variable `i`) are spread over the CPU cores of
one the NUMA domain of the corresponding `d` value.

Swan furthermore accelerates execution of fine-grain parallel loops through
an alternative runtime with much lower overhead compared to the Cilk runtime.
Fine-grain loops should currently not be nested inside
other parallel constructs.
They can be invoked as follows:

```
#pragma cilk finegrain
    cilk_for(int i=0; i < n; ++i) {
	 a[i] = ...;
    }
```

# The ASAP operator library for text analytics

The operator library provides basic data structures and algorithms
and is primarily focused on text analytics. These operators should be
considered examples of how to use the underlying data structures,
how to optimize memory usage and
how to use the Swan language to implement analytics.
The following discussion describes the main organization of the library
and links to per-class and per-method documentation for details on
the API and arguments.

## Data types

### Vector types

The basic data collection of homogeneous elements
is a vector. It is indexed by an integer type
starting with index zero and running up to the length of the vector.
There are two vector types:
dense vectors (\ref asap::dense_vector) provide a storage location for
every element in the range 0...length;
sparse vectors (\ref asap::sparse_vector) provide storage locations
only for those elements set explicitly. Missing values are treated as zero.

Vectors may have ownership over their storage, in which case the memory
policy template type is \ref asap::mm_ownership_policy .
When vectors have ownership over the storage, then they allocate and
deallocate the storage themselves.
Alternatively, vectors may not have ownership
(memory policy \ref asap::mm_no_ownership_policy). In this case, the
storage is managed externally, e.g., by a vector set. It is much more
efficient to manage the storage of large data sets centrally, in one go,
as this reduces memory allocation overhead and improves memory layout.

Various vector operations are accelerated using vector instructions
(SIMD - Single Instruction Multiple Data).

### Extended vector types

Sparse and dense vectors can be extended to store additional information.
This can be handy to cache information, or to store information associated
to vectors more compactly.
The type \ref asap::vector_with_add_counter extends
a \ref asap::dense_vector or \ref asap::sparse_vector type with a counter.
The \ref asap::vector_with_sqnorm_cache caches the square of the norm
of the vector (the Euclidean distance between the vector and itself).
The cached value is not automatically updated when the vector is changed, but
needs to be explicitly recalculated.

### Vector set type

A vector set is a list of vectors.
As with vectors, we distinguish
dense vector sets (\ref asap::dense_vector_set) and
sparse vector sets (\ref asap::sparse_vector_set).

### Word banks

Text analytics often require to store a large set of individual
text fragments. These are accelerated using three data types that
use different memory allocation policies.

The \ref asap::word_bank_malloc class stores a large number of text fragments
and makes individual memory allocations and deallocations for every text
fragment in the set.

The \ref asap::word_bank_pre_alloc class references text fragments
within a pre-allocated chunk of text, i.e., it can reference words
in a text document that is read into memory as single string.
E.g., mapping the file into memory using the mmap system call is very
efficient in terms of I/O. The \ref asap::word_bank_pre_alloc
class stores pointers into the mmap'ed memory region.

The \ref asap::word_bank_managed uses region-based memory management.
Regions are large chunks of pre-allocated storage where bump-pointer
allocation is used to efficiently add strings to the word bank.
It is typically much more
memory-efficient compared to \ref asap::word_bank_pre_alloc .

### Word containers

Where a word bank simply enumerates a set of words or text fragments,
a word container
provides additional functionality by means of an index into the words,
or by associating values to the words.

The class \ref asap::word_list simply allows to construct a list
or enumeration of words. While there may be repetition of words in the
list, the word bank can ensure each word is stored only once.
The \ref asap::word_list dictates the sequence in which words occur.

A \ref asap::word_map associates additional information to each unique word.
This information could be, e.g., the frequency of the word in a file.

An \ref asap::kv_list is a key-value abstraction over words.
Like the \ref asap::word_map it associates values to words. However,
where the \ref asap::word_map may use a map (e.g., std::map) or
the optimized \ref asap::hash_table to record the associated values, the
\ref asap::kv_list records the associated values as a list of key-value pairs.

All word containers inherit from the class \ref asap::word_container.

Similarly to words, the classes
\ref asap::ngram_map and \ref asap::ngram_kv_list provide associate storage
for n-grams. They inherit from the base class \ref asap::ngram_container.

### Data set type

Data sets are described by the \ref asap::data_set type.
In essence, a \ref asap::data_set is a combination of a vector set
(either sparse or dense), and two elements of type \ref asap::word_container
that describe the labels for rows and columns on the data set.
The data set also has a label to identify it or describe its contents.

### Auxiliary

The \ref asap::hash_table class is designed as an efficient associative
data structure, using a low-overhead hash table.
It is much faster than std::unordered_map.

## Input/output formats

### WEKA file format

We provide routines to read and write data sets in the
(WEKA file format)[http://www.cs.waikato.ac.nz/ml/weka/arff.html].
See methods \ref asap::arff_read and \ref asap::arff_write .

## Analytics Operators

### Term frequency

The ASAP operators library
provides the method \ref asap::word_catalog
for computing the term frequency of a document.
It takes a filename as argument and stores
the words and their frequency using one of the \ref asap::word_container types.
The method internally parallelizes scanning over the file contents.

The method \ref asap::ngram_catalog similarly calculates the frequency
of occurence of n-grams. The number of terms in the 'n'-gram is specified
in the \ref asap::ngram_container type that is used with this method.

Both implementations internally use parallel execution. They use the
word bank types and word/ngram container types for efficient memory
management.

### TF-IDF

The Term Frequency-Inverse Document Frequency (TF-IDF) operator
(\ref asap::tfidf)
takes as input a list of \ref asap::word_catalog objects, each one
corresponding to a distinct file. It also requires a pre-computed
\ref asap::word_catalog object that lists the number of files each word
occurs in. The method performs a processing step on these
objects to calculate the TF-IDF score. It produces a data set using sparse
vectors that compactly stores the TF-IDF scores.
Example usage of the \ref asap::tfidf, in conjunction with
\ref asap::word_catalog is provided in \ref tfidf_mix.cpp.

The \ref asap::tfidf_by_words method is similar to \ref asap:tfidf.
The distinction between these methods is that \ref asap::tfidf produces
one vector of TF-IDF scores per input file, whereas \ref asap::tfidf_by_words
produces one vector per word.
The difference in processing time is minimal. It is much faster to
select the method that generates data in the appropriate format compared
to transposing the data set.

The program \ref tfidf_mix.cpp implements multiple variants, using different
data structures in different steps.
The best options to use are '-c HLHHH' when the output should not be sorted
by words, and '-c HLHLI -s' when the words should be sorted alphabetically in the output.

### K-Means clustering

The K-means clustering operator is implemented through
the \ref asap::kmeans_operator class that encapsulates the state and methods
related to K-means clustering.
It stores the data in a
\ref asap::kmeans_data_set, a special type of \ref asap::data_set,
that stores additional data relevant to K-Means clustering, such as the
cluster centers and the Sum of Squared Errors (SSE) score for the clusering.
Typcially, the input points are sparse vectors while cluster centers are
dense vectors. The operator further normalizes the coordinates
in order to improve convergence.

The K-Means clustering operator uses the
extended vector types to store additional information on the cluster
centers. In particular, it uses
\ref asap::vector_with_add_counter
to count the number of points mapped to a cluster and it uses
\ref asap::vector_with_sqnorm_cache
to speedup the calculation of the Euclidean distance between a point
and a cluster center.
The latter optimization allows to compute the Euclidean distance
between a dense vector and a sparse vector by considering that
each coordinate of the sparse vector corresponds to a
deviation to the norm of the dense vector. The time complexity of this
operation is proportional to the number of non-zeroes in the sparse vector,
which is typically much less than the length of the vector.


text:
# Data Structures and Algorithms in JavaScript

This repository covers the implementation of the most important algorithms and data structures.

This goes along with [these posts series](https://adrianmejia.com/tags/tutorial-algorithms/) that explain each implementation in details.

[![Interactive Data Structures](https://user-images.githubusercontent.com/418605/46118890-ba721180-c1d6-11e8-82bc-6a671428b422.png)](https://embed.kumu.io/85f1a4de5fb8430a10a1bf9c5118e015)

## Data Structures
We are covering the following data structures.

### Linear Data Structures
1. **Arrays**: Built-in in most languages so not implemented here. [Code](https://github.com/amejiarosario/algorithms.js/blob/master/src/data-structures/arrays/array.js) | [Details](https://adrianmejia.com/blog/2018/04/28/data-structures-time-complexity-for-beginners-arrays-hashmaps-linked-lists-stacks-queues-tutorial/#Array).
2. **Linked Lists**: each data node has a link to the next (and previous). [Code](https://github.com/amejiarosario/algorithms.js/tree/master/src/data-structures/linked-lists) | [Details](https://adrianmejia.com/blog/2018/04/28/data-structures-time-complexity-for-beginners-arrays-hashmaps-linked-lists-stacks-queues-tutorial/#Linked-Lists).
3. **Queue**: data flows in a "first-in, first-out" (FIFO) manner. [Code](https://github.com/amejiarosario/algorithms.js/tree/master/src/data-structures/queues) | [Details](https://adrianmejia.com/blog/2018/04/28/data-structures-time-complexity-for-beginners-arrays-hashmaps-linked-lists-stacks-queues-tutorial/#Queues).
4. **Stacks**:  data flows in a "last-in, first-out" (LIFO) manner. [Code](https://adrianmejia.com/blog/2018/04/28/data-structures-time-complexity-for-beginners-arrays-hashmaps-linked-lists-stacks-queues-tutorial/#Stacks) | [Details](https://github.com/amejiarosario/algorithms.js/tree/master/src/data-structures/stacks).

### Non-Linear Data Structures
1. **Trees**: data nodes has zero or more adjacent nodes a.k.a. children. Each node can only have one parent node otherwise is a graph not a tree. [Code](https://github.com/amejiarosario/algorithms.js/tree/master/src/data-structures/trees) | [Details](https://adrianmejia.com/blog/2018/06/11/data-structures-for-beginners-trees-binary-search-tree-tutorial/)
	1. **Binary Trees**: same as tree but only can have two children at most. [Details](https://adrianmejia.com/blog/2018/06/11/data-structures-for-beginners-trees-binary-search-tree-tutorial/#Binary-Trees)
	1. **Binary Search Trees** (BST): same as binary tree, but the nodes value keep this order `left < parent < rigth`. [Code](https://github.com/amejiarosario/algorithms.js/blob/master/src/data-structures/trees/binary-search-tree.js) | [Details](https://adrianmejia.com/blog/2018/06/11/data-structures-for-beginners-trees-binary-search-tree-tutorial/#Binary-Search-Tree-BST)
	2. **AVL Trees**: Self-balanced BST to maximize look up time. [Code](https://github.com/amejiarosario/algorithms.js/blob/master/src/data-structures/trees/avl-tree.js) | [Details](https://adrianmejia.com/blog/2018/07/16/self-balanced-binary-search-trees-with-avl-tree-data-structure-for-beginners/)
	3. **Red-Black Trees**:  Self-balanced BST more loose than AVL to maximize insertion speed. [Code](https://github.com/amejiarosario/algorithms.js/blob/master/src/data-structures/trees/red-black-tree.js) | Details
2. **Maps**: key-value store.
	1. **Hash Maps**: implements map using a hash function. [Code](https://github.com/amejiarosario/algorithms.js/blob/master/src/data-structures/hash-maps/hashmap.js) | [Details](https://adrianmejia.com/blog/2018/04/28/data-structures-time-complexity-for-beginners-arrays-hashmaps-linked-lists-stacks-queues-tutorial/#HashMaps)
	2. **Tree Maps**: implement map using a self-balanced BST. WIP
3. **Graphs**: data *nodes* that can have a connection or *edge* to zero or more adjacent nodes. Unlike trees, nodes can have multiple parents, loops. [Code](https://github.com/amejiarosario/algorithms.js/blob/master/src/data-structures/graphs/graph.js) | [Details](https://adrianmejia.com/blog/2018/05/14/data-structures-for-beginners-graphs-time-complexity-tutorial/)

## Algorithms
1. Searching algorithms (WIP)
2. Sorting algorithms (WIP)

# Notes
Some notes while working on this project

## Tests
Running one test without changing file
```sh
jest -t '#findNodeAndParent'
```

Running one test changing code
```js
it.only('should return with an element and its parent', () => {
// ...
});
```

##  English Words

Getting some (200k+) English words are useful for testing and benchmarking.

```sh
cat /usr/share/dict/words > benchmarks/dict.txt
```

## ESLint

 Disabling ESLints
```js
somthing(t) =>  1  // eslint-disable-line no-unused-vars
// eslint-disable-next-line no-use-before-define
const  thing  =  new Thing();

/*eslint-disable */
//suppress all warnings between comments
alert('foo');
/*eslint-enable */

/* eslint-disable no-alert, no-console */
alert('foo');
console.log('bar');
/* eslint-enable no-alert */
```

license: MIT

branch_name: refs/heads/master

extension: md

gha_license_id: MIT

license_type: permissive


text:
# JSON Web Key Set

## What is JSON Web Key Set (JWKS)?

The JSON Web Key Set (JWKS) endpoint is a read-only endpoint that returns
the authorization server's public key set in the JWKS format.
This contains the signing key(s) that the Relying Party (RP) uses to validate signatures from the server.
For more information on this endpoint, see the [OpenID Connect Discovery specification](https://openid.net/specs/openid-connect-discovery-1_0.html).

---

## Usage of JWKS

The main benefit of allowing JWKS endpoint configuration is its ability to handle key rotation by external identity providers.
Configuring this endpoint would enable you to programmatically discover JSON web keys and allow the third party
identity providers to publish new keys without having the overhead of notifying each and every client application.
This allows smooth key rollover and integration.

---

## How does it work?

The following sequence diagram illustrates the scenario where a JWT obtained
from a third party IdP is validated using the JWKS-based JWT Validator.

![JWKS validation flow]({{base_path}}/assets/img/concepts/jwks-validation-flow.png)

The steps of the above diagram are explained below:

**Step 1:**

-   User requests a JWT assertion from the identity provider.

-   A valid JWT is returned with the response.

**Step 2:**

-   The user initiates a token request to WSO2 Identity Server’s token endpoint
    using JWT grant type with the obtained JWT assertion.

-   Access token issuer handles all the requests sent to the token
    endpoint.

**Step 3:**

-   Access token issuer invokes the JWT grant handler to validate the
    provided JWT assertion.

-   Deploys and configures the JWT client-handler artifacts.

**Step 4:**

-   JWT grant handler invokes the JWKS based JWT validator to validate
    the JWT signature using IdP’s jwks endpoint.

**Step 5:**

-   JWKS based JWT validator validates the JWT using IdP’s JWKS.

**Step 6:**

-   Upon JWT grant validation, access token issuer issues a new access
    token to the user.

Here, the retrieved JWKS is cached against the jwks\_uri. When
validating a JWT, we use the 'kid' header parameter which is an
indicator for the key used to sign the JWT at the IdP, and compare it
against the JWKS 'kid' properties. If a matched 'kid' is found among the
key set, the JWT will be validated using the corresponding key. In case
of key rollover at the IdP, the JWT is signed using the new keys and
hence the matching key is not found in the cached JWKS. When a matching
key is not found, the validator retrieves the latest JWKS from the IdP
JWKS endpoint and obtain the matching key for signature validation.

---

## Sample JSON Web Key Set

The default JWKS of WSO2 Identity Server is as follows.

```
{
   "keys":[
      {
         "kty":"RSA",
         "e":"AQAB",
         "use":"sig",
         "kid":"ZjRmYTMwNTJjOWU5MmIzMjgzNDI3Y2IyMmIyY2EzMjdhZjViMjc0Zg",
         "alg":"RS256",
         "n":"j3mYMT8N2SR8cpimdMOTpk_M8fOxPF1BHQAiCtld4nbksILgJKsA34GSP5Oh4gLW21VCEPPzdGLnqfwM6ZoG_X0rcK5--VbqH_vH4Cclba6fqlLxCvTiRbPJ58Pe7-biCeQ368dG2aeBPV3EhO8br3Z_LcQXASmhSWps8J3GOSx_49xzkHh59J2gJHhnvjxcszZAF35SLAb6F-2rJQrOJs6u7WfJv4NQxSyhcgcr4_-77JzNFEVUj4TPSBy2WGAgK5ttP5-kG3-rKs0lQjTo9h_hK89KjbbRvoqZdpxnwQYxFDOk0CxijZVO5Cs3cabeUHZeXehHSgXj6W-VGMiDgw"
      },
      {
         "kty":"RSA",
         "e":"AQAB",
         "use":"sig",
         "kid":"ZjRmYTMwNTJjOWU5MmIzMjgzNDI3Y2IyMmIyY2EzMjdhZjViMjc0Zg_RS256",
         "alg":"RS256",
         "n":"j3mYMT8N2SR8cpimdMOTpk_M8fOxPF1BHQAiCtld4nbksILgJKsA34GSP5Oh4gLW21VCEPPzdGLnqfwM6ZoG_X0rcK5--VbqH_vH4Cclba6fqlLxCvTiRbPJ58Pe7-biCeQ368dG2aeBPV3EhO8br3Z_LcQXASmhSWps8J3GOSx_49xzkHh59J2gJHhnvjxcszZAF35SLAb6F-2rJQrOJs6u7WfJv4NQxSyhcgcr4_-77JzNFEVUj4TPSBy2WGAgK5ttP5-kG3-rKs0lQjTo9h_hK89KjbbRvoqZdpxnwQYxFDOk0CxijZVO5Cs3cabeUHZeXehHSgXj6W-VGMiDgw"
      }
   ]
}
```

| Property Value                 | Description         |
| --------------------- | ------------- |
| kty | The public key type.|
| e           | The exponent value of the public key.|
| use         | Implies how the key is being used. The value sig represents signature.|
| kid         | The thumbprint of the certificate. This value is used to identify the key that needs to be used to verify the signature.  |
| alg         | The algorithm used to secure the JSON Web Signature.  |
| n           | The modulus value of the public key.  |

gha_language: CSS


text:
# Android-Arduino-Automotive

# Repository Titles

> How to Install Android Studio on Ubuntu 18.04

> Arduino project

# How to Install Android Studio on Ubuntu 18.04

![Android Studio](https://www.mindinventory.com/blog/wp-content/uploads/2020/03/Android-studio-36-1520x500.png)

Android Studio is a full-featured cross-platform IDE that helps you build applications on every type of Android device. It is based on JetBrains’ IntelliJ IDEA and includes everything you need for Android development.

Android Studio build system is powered by Gradle allowing you to create multiple build variants for different devices from a single project.

This tutorial explains how to install Android Studio on Ubuntu 18.04. The same instructions apply for Ubuntu 16.04 and any Ubuntu-based distribution, including Kubuntu, Linux Mint, and Elementary OS.

***Prerequisites***

You’ll need to be logged in as a user with sudo access to be able to install packages on your Ubuntu system.

***Installing Java OpenJDK***

Android Studio requires OpenJDK version 8 or above to be installed to your system.
We’ll install OpenJDK 8. The installation is pretty simple, start by updating the package index:

***Installing Java OpenJDK***

Android Studio requires OpenJDK version 8 or above to be installed to your system.
We’ll install OpenJDK 8. The installation is pretty simple, start by updating the package index:

```javascript
// Run

sudo apt update

```
Install the OpenJDK 8 package by typing:

```javascript
// Run

sudo apt install openjdk-8-jdk

```
Verify the installation by typing the following command which will print the Java version:

```javascript
// Run

java -version

```

The output should look something like this:

```javascript
// The output

openjdk version "1.8.0_191"
OpenJDK Runtime Environment (build 1.8.0_191-8u191-b12-2ubuntu<IP_ADDRESS>-b12)
OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)

```
***Installing Android Studio***

At the time of writing this article, the latest stable version of Android Studio is version <IP_ADDRESS>. The easiest way is to install Android Studio on Ubuntu 18.04 is by using the snappy packaging system.

To download and install the Android Studio snap package, open your terminal using the Ctrl+Alt+T keyboard shortcut and type:

```javascript
// run

sudo snap install android-studio --classic

```
Once the installation is complete, you will see the following output:

```javascript
// The output

android-studio <IP_ADDRESS> from Snapcrafters installed
```
That’s it. Android Studio has been installed on your Ubuntu desktop.

# Arduino project

![Arduino cover](https://scontent.ftun1-1.fna.fbcdn.net/v/t1.0-9/13645146_1429686307057684_8500685619593774449_n.png?_nc_cat=106&_nc_sid=dd9801&_nc_ohc=2S3wf_2HAFMAX84Stun&_nc_ht=scontent.ftun1-1.fna&oh=c67fbb17886d7f4eb1a21e474266c822&oe=5EBF053D)

***Arduino IDE***

The Arduino Integrated Development Environment is a cross-platform application that is written in functions from C and C++. It is used to write and upload programs to Arduino compatible boards, but also, with the help of 3rd party cores, other vendor development boards.

***Which language is used in Arduino IDE?***

The "Arduino language" is simply C++ (and therefore also C). All the standard system librarys are written in C/C++ and the IDE uses the gcc g++ compiler.

![Arduino](https://i.pinimg.com/564x/c6/b4/72/c6b4725ca5fcde572245ced9ade303be.jpg)

# FAQ

- **How do I do *specifically* so and so?**
    - No problem! Just do this.

# Support

Reach out to me at one of the following places!

- Linkedin at `Kalifiabillal`
- Twitter at `@Kalifiabillal`

# License

- **[MIT license](http://opensource.org/licenses/mit-license.php)**
- Copyright 2015 © Kalifiabillal.


text:
# mt-webserver
Multi-threading Web Server for node.js

While node.js is very successful to implement webservers to handle quite simple calls efficiently, programming database applications with complex transactions spanning multiple tables becomes quite difficult to write.

The multi-threading web server allows to program applications in the classical object-oriented/procedural style.

Problems with node:

-   complex sequential procedurs require extensive additional programming effort to not block the main thread
-   intuitive procedural OO programming is not possible - the beauty of the easy to learn and to use JavaScript programming language is lost
-   a single error in the main thread can halt or impair the whole application
-   the main thread becomes easily a bootleneck for the whole system
-   the programmer must make sure that the garbage collection really can clean up or the node's process RAM usage increases over time

Solution:

+   starting a new thread for reasonable complex request handlers
+   other requests are handled asap with standard node techniques
+   the main thread works as HTTP router distributing requests to HTTP handler threads
+   a thread pool helps to minimize start-up time
+   optional caching the HTTP response into RAM or to file system fastens subsequent calls
+   a shared ACID key/value store makeing data exchange between threads easy

Drawbacks:

*   Starting a new thread is expensive, so it should not be used for every minor HTTP request
*   Most existing libraries are designed with callbacks and heavily used libs should have some simple wrapper to hide their asynchronous behavior for devlopers
    not comfortable with async and promises, so they can stick to strict procedural programming

## Roadmap
0.0 - Repository setup, using tiny-worker (because the original code is older than the worker_threads built into node.js)

0.1 - Using node's worker_threads and messages to route HTTP requests and responses between main thread and worker threads

0.2 - Route HTTP requests through network API. Each worker-thread binds to a separate port and HTTP requests are forwarded by the main thread to this ports, similar to tjanczuk/iisnode. Advantage: each worker-thread can implement a regular web server using the http API.

0.3 - Basic web server functionality in main thread to serve simple requests without starting a worker-thread, e.g. deliver file content and cached HTTP responses

0.4 - Thread pool and intelligent routing HTTP requests to existing threads, e.g. depending on session cookie values so the same thread handles requests in a session. Throwing aways worker-threads from time to time frees memory allocation that could not be cleaned up by the JavaScript garbage collector.

0.5 - Shared ACID key/value store supporting transactions

...

1.0 Fully-fledged webserver ready for productive use

(the order of the roadmap items might change, depending on the demand)

repository: Naradana/mt-webserver

branch_name: refs/heads/main

repo_name: Naradana/mt-webserver


text:
# LookUp
###  This is my contribution to the topic **Data Visualisation**, second project in the class *Steel Ant - Eingabe Ausgabe* taught by Fabian Morón Zirfas, University of applied science - Potsdam (Germany).

Microsite: http://martakarta.github.io/LookUp

### The ISS
Goal of the project was to create a Data Visualisation based on the geographical position of the International Spase Station, available on https://spotthestation.nasa.gov/sightings.

### Concept
The inspiration for this visualisation was the thought, that the ISS is actually visible naked eyes from the earth. For doing that is thou necessary to know when to look up. The json data in this repository delivers the all the visible moments of the ISS in December 2015 from the main cities of Germany.

*Red = moments of visibility; White = German cities*

The sourse code filters this information and shows which day and at what time the ISS is visible in the chosen german city.

### Berlin
As an example the visualisation of the moments of visibility in Berlin. The axis x rappresent the december days and the axis y the hours in a day.

repository: martakarta/LookUp

gha_language: JavaScript

repo_name: martakarta/LookUp


text:
# Day 10

## Tasks: Underfitting and Overfitting & Random Forests

1. [Read this tutorial](https://www.kaggle.com/dansbecker/underfitting-and-overfitting) (from Lesson 5 of the Intro to ML course)
2. [Complete this exercise](https://www.kaggle.com/kernels/fork/1259126) (from Lesson 5 of the Intro to ML course)
3. [Read this tutorial](https://www.kaggle.com/dansbecker/random-forests) (from Lesson 6 of the Intro to ML course)
4. [Complete this exercise](https://www.kaggle.com/kernels/fork/1259186) (from Lesson 6 of the Intro to ML course)


text:
# spike-recorder

[![Actions Status][actions-badge]][actions-link]
[![Documentation Status][rtd-badge]][rtd-link]
[![Code style: black][black-badge]][black-link]

[![PyPI version][pypi-version]][pypi-link]
[![Conda-Forge][conda-badge]][conda-link]
[![PyPI platforms][pypi-platforms]][pypi-link]

[![Gitter][gitter-badge]][gitter-link]

This package implements a Python interface for the
[Backyard Brains Spike Recorder](https://backyardbrains.com/products/spikerecorder), a neural recording application. It
is based off a [fork](https://github.com/davidt0x/Spike-Recorder) of the original C++ code, found
[here](https://github.com/BackyardBrains/Spike-Recorder). In addition, it contains two psychological experiment
applications written in Python that control and record events via the SpikeRecorder.

## Installation

You can install this library from [PyPI](https://pypi.org/project/spike-recorder/) with pip:

```bash
python -m pip install spike-recorder
```

## Usage

To run the SpikeRecorder application simply invoke it on the command line

```bash
spike-recorder
```

If you want to launch the SpikeRecorder application alongside either of the experiments
below then invoke them with the `--spike-reord` option.

## Iowa Gambling Task

![Iowa Task Screenshot](docs/images/iowa_task_screenshot.png?raw=true "Iowa Task Screenshow")

To launch the Iowa Gambling Task Experiment, run:

```bash
iowa
```

There are some available options and arguments:

```
usage: iowa [-h] [--spike-record] [--total-deck-pulls TOTAL_DECK_PULLS] [--seed SEED]

optional arguments:
  -h, --help            show this help message and exit
  --spike-record        Launch Backyard Brains Spike Recorder in background. Default is do not run.
  --total-deck-pulls TOTAL_DECK_PULLS
                        The total number of deck pulls in the experiment. Default is 100.
  --seed SEED           Seed for the random number generators that control deck randomness.
                        Default is 0 so behaviour between runs will be random but fixed

```

## Libet Experiment

![Libet Experiment Screenshot](docs/images/libet_task_screenshot.png?raw=true "Iowa Task Screenshow")

To launch the Libet Task Experiment, simply run:

```bash
libet
```

If you wish to adjust the speed of the clock or the number of trials in either phase, see the available options:

```
usage: libet [-h] [--spike-record] [--num-trials-paradigm1 NUM_TRIALS_PARADIGM1] [--num-trials-paradigm2 NUM_TRIALS_PARADIGM2] [--clock-hz-paradigm1 CLOCK_HZ_PARADIGM1] [--clock-hz-paradigm2 CLOCK_HZ_PARADIGM2]

optional arguments:
  -h, --help            show this help message and exit
  --spike-record        Launch Backyard Brains Spike Recorder in background.
                        Default is do not run.
  --num-trials-paradigm1 NUM_TRIALS_PARADIGM1
                        The number of trials to conduct for paradigm one. Default is 20.
  --num-trials-paradigm2 NUM_TRIALS_PARADIGM2
                        The number of trials to conduct for paradigm two,
                        in which time of urge is asked. Default is 20.
  --clock-hz-paradigm1 CLOCK_HZ_PARADIGM1
                        The number of full rotations the clock makes per second in paradigm one.
                        Default is 1 but can be set lower than 1.
  --clock-hz-paradigm2 CLOCK_HZ_PARADIGM2
                        The number of full rotations the clock makes per second in paradigm two.
                        Default is 1 but can be set lower than 1.

```

[actions-badge]:            https://github.com/davidt0x/py-spike-recorder/workflows/CI/badge.svg
[actions-link]:             https://github.com/davidt0x/py-spike-recorderactions
[black-badge]:              https://img.shields.io/badge/code%20style-black-000000.svg
[black-link]:               https://github.com/psf/black
[conda-badge]:              https://img.shields.io/conda/vn/conda-forge/spike-recorder
[conda-link]:               https://github.com/conda-forge/spike-recorder-feedstock
[gitter-badge]:             https://badges.gitter.im/PrincetonUniversity/py-spike-recorder.svg
[gitter-link]:              https://gitter.im/PrincetonUniversity/py-spike-recorder?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge
[pypi-link]:                https://pypi.org/project/spike-recorder/
[pypi-platforms]:           https://img.shields.io/pypi/pyversions/spike-recorder
[pypi-version]:             https://badge.fury.io/py/spike-recorder.svg
[rtd-badge]:                https://readthedocs.org/projects/spike-recorder/badge/?version=latest
[rtd-link]:                 https://spike-recorder.readthedocs.io/en/latest/?badge=latest
[sk-badge]:                 https://scikit-hep.org/assets/images/Scikit--HEP-Project-blue.svg

gha_language: Python


text:
![image](docgen/images/GridDyn_FullColor.png "GridDyn")
============
[![Build Status](https://travis-ci.org/LLNL/GridDyn.svg?branch=master)](https://travis-ci.org/LLNL/GridDyn)
[![Build status](https://ci.appveyor.com/api/projects/status/e3rygs874w04a25n?svg=true)](https://ci.appveyor.com/project/griddyn/griddyn)
[![Gitter chat](https://badges.gitter.im/LLNL/GridDyn.png)](https://gitter.im/LLNL/GridDyn)

GridDyn is a power system simulator developed at Lawrence Livermore National Laboratory.
The name is a concatenation of Grid Dynamics, and as such usually pronounced as "Grid Dine".
It was created to meet a research need for exploring coupling between transmission, distribution, and communications system simulations.
While good open source tools existed on the distribution side, the open source tools on the transmission side were limited in usability
either in the language or platform or simulation capability, and commercial tools while quite capable simply did not allow the access
to the internals required to conduct the research. Thus the decision was made to design a platform that met the needs of the research project.
Building off of prior efforts in grid simulation, GridDyn was designed to meet the current and future research needs of the various grid related
research and computational efforts. It is written in C++ making use of recent improvements in the C++ standards. It is intended to be cross platform with
regard to operating system and machine scale. The design goals were for the software to be easy to couple with other simulation,
and be easy to modify and extend. It is very much still in development and as such, the interfaces and code is likely to change,
in some cases significantly as more experience and testing is done. It is our expectation that the performance, reliability,
capabilities, and flexibility will continue to improve as projects making use of the code continue and new ones develop.
We expect there are still many issues so any bug reports or fixes are welcome.
And hopefully even in its current state and as the software improves the broader power systems research community will find it useful.

Documentation
----------------

[**Users Guide**](docs/manuals/GridDynUserManual.pdf)

** API Guide ** --coming soon

Also available are a series of presentations
- [Intro](docs/presentations/Griddyn_intro.pptx)
- [Execution Flow](docs/presentations/GridDyn_execution_flow.pptx)
- [Objects](docs/presentations/GridDyn_objects.pptx)
- [Object Construction and properties](docs/presentations/GridDyn_object_construction_and_properties.pptx)
- [States and offsets](docs/presentations/stateData_solverModes_solverOffsets.pptx)
- [Validation and Performance](docs/presentations/GridDyn_validation_and_performance.pptx)

Installation
------------------------
[Installation Guide](installation.md).

Quick Start
------------------------
[quick start guide](quickStart.md).

Get Involved!
------------------------

GridDyn is an open source project. Questions, discussion, and
contributions are welcome. Contributions can be anything from new
packages to bugfixes, or even new core features. We are actively working on improving it and
making it better, as well as development related to specific projects.

### Contributions

We are still working out the details of accepting contributions
For the moment you can submit a
[pull request](https://help.github.com/articles/using-pull-requests/).
and we can work with you to make sure the licensing is order, which basically involves making sure your contributions are released back to the repo under a BSD license like the rest of the code.

Before you send a PR, your code should pass all the non-experimental test cases in testSystem, testLibrary, and testComponents

If the code is a new feature or new model, it should have additional test cases explicitly testing it

A style check is periodically run on the code to ensure consistent indentation and spacing. While some style guidance has been followed it is not rigorously enforced yet.
A more formal style guide will likely evolve in the near future.

The current git master branch is considered experimental so there is no stable branch to maintain as everything is in development. It is anticipated a more formal branching structure will be defined once the code base undergoes more testing and validation, and other features are added.

Authors
----------------
GridDyn was originally written by Philip Top<EMAIL_ADDRESS>A number of other people have contributed, see the Users Guide for more details

Release
----------------
GridDyn is released under a BSD license. For more details see the
[LICENSE](LICENSE) file.

``LLNL-CODE-681053``

repository: nightlark/GridDyn

gha_license_id: NOASSERTION

repo_name: nightlark/GridDyn


text:
---
title: "Using with gatsby-starter-blog and markdown files"
date: "2019-03-10T23:46:37.121Z"
---

### Install packages

Firstly, add the necessary packages to your gatsby project

```
npm i --save gatsby-plugin-cloudinary-image-gallery
```

If you plan on using the component inside your markdown files / posts add the `rehype-react` and `gatsby-component-remark`

```
npm i --save gatsby-plugin-cloudinary-image-gallery
```

### Setup gatsby-plugin-cloudinary-image-gallery

### Create a StaticQuery component

Then create a component that wraps your PhotoGrid component in a Static Query, I called it gallery.js

```javascript
import React from "react"
import PropTypes from "prop-types"
import { StaticQuery, graphql } from "gatsby"

import ImageGallery from "gatsby-plugin-cloudinary-image-gallery"

const Gallery = ({ folder, columns, orientation }) => (
  <StaticQuery
    query={imageGalleryQuery}
    render={data => (
      <ImageGallery
        folder={folder}
        columns={columns}
        data={data}
        orientation={orientation}
      />
    )}
  />
)

const imageGalleryQuery = graphql`
  query galleryQuery {
    cloudinaryImage: allCloudinaryImage {
      edges {
        node {
          id
          folder
          thumb
          imgUrl
          width
          height
          orientation
        }
      }
    }
    site {
      siteMetadata {
        author
        social {
          twitter
        }
      }
    }
  }
`

Gallery.propTypes = {
  folder: PropTypes.string.isRequired,
}

export default Gallery
```

You can now use the component in your other components.

## Markdown usage

### Update /templates/blog-post.js

1. Register your component using rehype-react (see [https://using-remark.gatsbyjs.org/custom-components/](https://using-remark.gatsbyjs.org/custom-components/))

```
import rehypeReact from "rehype-react"
import Gallery from "../components/gallery"

const renderAst = new rehypeReact({
  createElement: React.createElement,
  components: { "photo-gallery": Gallery },
}).Compiler
```

2. Replace

```
<div dangerouslySetInnerHTML={{ __html: post.html }} />
```

with

```
{ renderAst(post.htmlAst) }
```

3. Change `html` to `htmlAst` in your pageQuery

### Setup gatsby-component-remark

Update your gatsby-config with

```
{
  resolve: `gatsby-transformer-remark`,
  options: {
    plugins: [
      ...
      `gatsby-remark-component`,
    ],
  },
}
```

### Add the component to your markdown file:

```
<photo-gallery folder="gatsby"></photo-gallery>
```


text:
Role Name
=========

Role for deployment of the latest artifact from Nexus repository to QA and CI environment

Requirements
------------

Any pre-requisites that may not be covered by Ansible itself or the role should be mentioned here. For instance, if the role uses the EC2 module, it may be a good idea to mention in this section that the boto package is required.

Role Variables
--------------

Before use this role you should define variables in defaults/main.yml:

NEXUS_SERVER_IP
NEXUS_SERVER_PORT               Default: 8081
MAVEN_REPOSTITORY               Default: maven-repository
MAVEN_GROUPID                   Default: java
MAVEN_ARTIFACTID                Default: spring-boot-smoke-test-web-ui
MAVEN_EXTENTION                 Default: jar

JAVAAPP_GROUP                   Default: appmgr
JAVAAPP_USER                    Default: jvmapps

SYSTEMD_SERVICE_NAME            Default: javapp
JAVAAPP_SERVICE_PORT            Default: 8080

Example Playbook
----------------

Including an example of how to use your role (for instance, with variables passed in as parameters) is always nice for users too:

    - hosts: servers
      roles:
         - { role: rolename }

License
-------

BSD

Author Information
------------------

Evgeniy Naryshkin<EMAIL_ADDRESS>

repository: allainmoyo/GraduationWork

repo_name: allainmoyo/GraduationWork


text:
<!-- SPDX-FileCopyrightText: no
     SPDX-License-Identifier: CC0-1.0
-->

# Contributing to QuatBot

Welcome to QuatBot! We're happy you would like to add something to it.
Pull Requests (PRs) are always welcome: you don't need to ask if you
are allowed to fix something. It may be worthwhile to open a PR early
(before it's done or even before any work is done) to discuss a
new feature.

## Coding Style

QuatBot uses automated formatting tools. Feel free to write your code
any way you like, but please apply the tools before pushing to a PR:

- clang-format13 and the associated `.clang-format` file for C++
- gersemi for CMake

repository: adriaandegroot/QuatBot

branch_name: refs/heads/quatbot

repo_name: adriaandegroot/QuatBot


text:
** Create a GPO, in the common tab, choose item level targeting and create a group of users.

**What is loopback processing?
** Loopback processing => This policy directs the system to apply the set of GPOs for the computer to any user who logs on to a computer affected by this policy.
   - [Loopback processing of Group Policy](https://support.microsoft.com/en-us/help/231287/loopback-processing-of-group-policy)
   - [User Group Policy loopback processing mode](https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-2000-server/cc978513(v=technet.10))

1. Link the required user policy to computer OU
2. Decide the computer policy object to use
3. Configure GPO loopback processing
The setting is located on Computer Configuration > Policies > Administrative Templates > System > Group Policy > Configure user Group Policy loopback processing mode.
Double click the setting. Set it as Enabled then select the mode from the dropdown menu.

As mentioned in the opening, there are two modes for loopback processing:
Replace: When selected, user policies linked to computer OU will override the other user policies that linked to the user OU.
Merge: When selected, user policies linked to computer OU will be applied along with the other user policies that linked to the user OU. If any conflicting setting between policies, GPO will process them normally based on the link order.
Based on the requirement in this scenario, the best suitable mode is Replace because "Dev User Policy" must be applied instead of the other policies that applied normally via the user OU.
When loopback processing has been enabled, those user policies should be replaced by the "Dev User Policy" that is linked to the computer OU. Like a normal GPO, loopback processing should be applied once the policy refreshed, or we can force it by using command `gpupdate /force`

**What is the difference between Enterprise Admins and Domain Admins?**

Enterprise Admins (only appears in the forest root domain)

Description:
Members of this group have full control of all domains in the forest. By default, this group is a member of the Administrators group on all domain controllers in the forest. By default, the Administrator account is a member of this group. Because this group has full control of the forest, add users with caution.

Default user rights:

Access this computer from the network; Adjust memory quotas for a process; Back up files and directories; Bypass traverse checking; Change the system time; Create a pagefile; Debug programs; Enable computer and user accounts to be trusted for delegation; Force shutdown from a remote system; Increase scheduling priority; Load and unload device drivers; Allow log on locally; Manage auditing and security log; Modify firmware environment values; Profile single process; Profile system performance; Remove computer from docking station; Restore files and directories; Shut down the system; Take ownership of files or other objects.

Enterprise Administration:

Administering the AD Schema (Schema Admins is technically the only thing required…)
Creating Certificate Authority (Root and Issuing)
Managing Certificate Templates (Default or otherwise)
DHCP Authorization
Forest trust relationships
Forest Preparation and Functional Level management
Global Sites and Services Management and administration (for all domains)
Creation of Sites & Site-Links
Creation of IP Subnets
Terminal Services Licensing
Creation and Destruction of Domains
FSMO Role Seizure (Domain Naming, Schema)
[Schema only needs schema admins…]
Global Domain Controller Replication Management
Global Domain Management
Global Group Policy Management
Global Administrative Control for All Domain users and computers
Take ownership of all forest and domain resources

Domain Admins

Description:
Members of this group have full control of the domain. By default, this group is a member of the Administrators group on all domain controllers, all domain workstations, and all domain member servers at the time they are joined to the domain. By default, the Administrator account is a member of this group. Because the group has full control in the domain, add users with caution.

Default user rights:
Access this computer from the network; Adjust memory quotas for a process; Back up files and directories; Bypass traverse checking; Change the system time; Create a pagefile; Debug programs; Enable computer and user accounts to be trusted for delegation; Force a shutdown from a remote system; Increase scheduling priority; Load and unload device drivers; Allow log on locally; Manage auditing and security log; Modify firmware environment values; Profile single process; Profile system performance; Remove computer from docking station; Restore files and directories; Shut down the system; Take ownership of files or other objects.

Domain Administration

\*\*CAUTION\*\* => By default, Domain Admins in the Root Domain can make themselves Enterprise Admins
Domain / DC Group Policy Management
Domain user and computer administration
Delegation of rights within Domain
FSMO Role Seizure (RID, PDC, Infrastructure)
Domain Controller Installation (DCPROMO)
Domain Controller Recovery (DRM)
Domain Controller Replication Management
Sites and Services Management for Domain level Controllers (Replication & Global Catalog)
Enterprise Domain Services (SCOM, SCCM) (System Container Modification)
Creation of Organizational Units and other AD objects in Domain
Domain Preparation and Function Level Management
Creation of domain level DFS Namespaces

**What is LDAP? Why it is used.**
LDAP is the Lightweight Directory Access Protocol. Its an active directory protocal ,Basically, it's a protocol used to access data from a database. LDAP is an internet standard protocol that runs over TCP/IP.

**What is Active Directory? Why it used.**
Active Directory is a Directory Service created by Microsoft. It is included with most Windows Server operating systems. Active Directory is primarily used to store directory objects like users and groups and computers printers. Using Active Directory brings a number of advantages to your network:
Centralized user account management
Centralized policy management (group policy)
Better security management

**What is an Active Directory?**
It is a database (Active Directory is a special-purpose database)

**What is an object**
An object is an instance of storage of a class, user is an object, computer account is also an object etc.

**What is an attribute?**
Property of an Object called an attribute, think of user object, and think of all available attributes associating with user object, name, last name, logon name etc.

**What is the name of Active Directory Database, when you install Domain Controller, by default directory database gets created?**
Active Directory database is. DIT database, it is NTDS.dit

**NTDS.DIT Database is partitioned database, what are the partitions on it?**
1. Domain
2. Configuration
3. Schema
4. Application (if created)

**In Which partition Exchange Server information is kept (tricky question)**
All of the partitions.
1. Domain
2. Configuration
3. Schema

**Considering single Forest with 12 domains, how many FSMO role in total exist?**
Total FSMO roles in this scenario = 38
12 X 3 = 36 (PDC, RID, Infrastructure) Master
2 For each Forest (Domain Naming Master and Schema Master per Forest)
Total = 38

**What is FSMO?**
Flexible Single Master Operations

**What is the least important FSMO role, considering day to day operations which one of FSMO role absence would be least significant impact and Why?**
Perhaps Schema Master if you assume you don't extend your schema every day or RID master

**What is the most important FSMO role, considering day to day operations which one of FSMO role absence would be most significant impact and why?**
PDC Emulator (explain why)

**What is the difference in between seizing and moving FSMO roles?**
Peaceful operation versus forcing it. Seizing is the operation when FSMO role owner is no longer accessible and after seizing FSMO roles, you cannot bring the previous FSMO role owner back to network.

**What is KCC and what does it do?**
Knowledge Consistence Checker, it builds replication connections in between domain controllers

**What is LSAS**
Lsass.exe (Local Security Authority Subsystem Service) is the process which, on an Active Directory domain controller, is responsible for providing Active Directory database lookups, authentication,
and replication

**What kind of replication Active Directory uses? is it pull or push?**
Active Directory uses pull replication. In pull replication, a destination replica requests information from a source replica. The request specifies the information that the destination needs, based on its knowledge of changes already received from the source and from all other domain controllers in the domain. When the destination receives information from the source, it applies that information, bringing itself more up-to-date. The destination's next request to the source excludes the information that has already been received and applied.

The alternative is push replication. In push replication, a source sends information to a destination unsolicited, in an attempt to bring the destination more up-to-date. Push replication is problematical because it is difficult for the source to know what information the destination needs. Perhaps the destination has received the same information from another source. If a source sends information to a destination, there is no guarantee that the destination is going to apply it; if the source assumes otherwise, the system is unreliable.

**Define DSRM Mode?**
Directory Services Restore Mode (DSRM) is a special boot mode for repairing or recovering Active Directory. It is used to log on to the computer when Active Directory has failed or needs to be restored.
To manually boot in Directory Services Restore Mode, press the `F8` key repeatedly. Do this immediately after BIOS POST screen, before the Windows logo appears. (Timing can be tricky; if the Windows logo appears you waited too long.) A text menu menu will appear. Use the up/down arrow keys to select Directory Services Restore Mode or DS Restore Mode. Then press the Enter key.

**Where is the AD database held? What other folders are related to AD?**
The AD data base is stored in c:\windows\ntds\NTDS.DIT.

**Have you ever Installed AD?**
To Install Microsoft Active Directory:
Ensure that you log on to the computer with an administrator account to perform installation.
Click electing Start > Administration Tools > Server manager > Configure your Server.
In the Welcome page, click Next.
In the Operating system compatibility panel, click Next.
On the Domain Controller Type panel, select Domain controller for a new domain and click Next.
On the Create New Domain panel, select Domain in a new forest and click Next.
On the New Domain Name panel, enter the DNS suffix for your new Active Directory. This name will be used during Tivoli Provisioning Manager installation, so make a note of it. Click Next.
On the NetBIOS Domain Name panel, enter the NetBIOS name of the domain. The first part of the DNS name is usually sufficient. Click Next.
On the Database and Logs panel, select the desired folders for the Database and Logs. C:\Windows\NTDS is the default. Click Next.
On the Shared System Volume panel, enter a valid directory for the system volume. C:\Windows\Sysvol is the default. Click Next to continue.
If you configured DNS successfully, the Permissions setting panel is displayed. Select Permissions compatible only with Windows 2000 or Windows Server 2003. Click Next.
On the Directory Services Restore Mode Administrator Password panel, enter a valid password to be used when running the Directory Services in Restore Mode. Click Next
Verify the settings and Click Next to begin the Active Directory configuration. The server will be rebooted as part of the process.

**What is the use of SYSVOL folder**
All active directory data base security related information store in SYSVOL folder and it's only created on NTFS partition. SYSVOL is a folder that exists on all domain controllers. It is the repository for all of the active directory files. It stores all the important elements of the Active Directory group policy. The File Replication Service or FRS allows the replication of the SYSVOL folder among domain controllers. Logon scripts and policies are delivered to each domain user via SYSVOL. SYSVOL stores all of the security related information of the AD.

**What is global catalog**
The Global Catalog is a database that contains all of the information pertaining to objects within all domains in the Active Directory environment

**What is Active Directory Schema?**
Schema is an active directory component describes all the attributes and objects that the directory service uses to store data.

**What is PDC emulator and how would one know whether PDC emulator is working or not?**
PDC Emulators: There is one PDC emulator per domain, and when there is a failed authentication attempt, it is forwarded to PDC emulator. It acts as a "tie-breaker" and it controls the time sync across the domain. These are the parameters through which we can know whether PDC emulator is working or not.
• Time is not syncing
• User's accounts are not locked out
• Windows NT BDCs are not getting updates
• If pre-windows 2000 computers are unable to change their passwords.

**What is RID Master?**
RID master stands for Relative Identifier for assigning unique IDs to the object created in AD.

**What is Infrastructure Master?**
Infrastructure Master is accountable for updating information about the user and group and global catalogue.

**What are the five Flexable Single Master Operation Roles (FSMO)**
The 5 FSMO server roles:
Schema Master Forest Level One per forest
Domain Naming Master Forest Level One per forest
PDC Emulator Domain Level One per domain
RID Master Domain Level One per domain
Infrastructure Master Domain Level One per domain
Schema Master and Domain Naming Master are forest wide role and only available one on each Forest, Other roles are Domain wide and one for each Domain AD replication is multi master replication and change can be done in any Domain Controller and will get replicated to others Domain Controllers, except above file roles, this will be flexible single master operations (FSMO), these changes only be done on dedicated Domain Controller so it's single master replication.

**Which FSMO role is the most important? And why?**
Interesting question which role is most important out of 5 FSMO roles or if one role fails that will impact the end-user immediately Most amateur administrators pick the Schema master role, not sure why maybe they though Schema is very critical to run the Active Directory . Correct answer is PDC, now the next question why? Will explain role by role what happens when a FSMO role holder fails to find the answer.

> Schema Master – Schema Master needed to update the Schema, we don't update the schema daily right, when will update the Schema? While the time of operating system migration, installing new Exchange version and any other application which requires extending the schema So if are Schema Master Server is not available, we can't able to update the schema and no way this will going to affect the Active Directory operation and the end-user
> Schema Master needs to be online and ready to make a schema change, we can plan and have more time to bring back the Schema Master Server
>
> Domain Naming Master – Domain Naming Master required to creating a new Domain and creating an application partition, Like Schema Master we don't create Domain and application partition frequently. So if are Domain Naming Master Server is not available, we can't able to create a new Domain and application partition, it may not affect the user, user event didn't aware Domain Naming Master Server is down
>
> Infrastructure Master – Infrastructure Master updates the cross domain updates, what really updates between Domains? Whenever user login to Domain the TGT has been created with the list of access user got through group membership (user group membership details) it also contain the user membership details from trusted domain, Infrastructure Master keep this information up-to-date, it update reference information every 2 days by comparing its data with the Global Catalog (that's why we don't keep Infrastructure Master and GC in same server) In a single Domain and single Forest environment there is no impact if the Infrastructure Master server is down. In a Multi Domain and Forest environment, there will be impact and we have enough time to fix the issue before it affect the end-user.
>
> RID Master– Every DC is initially issued 500 RID's from RID Master Server. RID's are used to create a new object on Active Directory, all new objects are created with Security ID (SID) and RID is the last part of a SID. The RID uniquely identifies a security principal relative to the local or domain security authority that issued the SID When it gets down to 250 (50%) it requests a second pool of RID's from the RID master. If RID Master Server is not available the RID pools unable to be issued to DC's and DC's are only able to create a new object depends on the available RID's, every DC has anywhere between 250 and 750 RIDs available, so no immediate impact
>
> PDC – PDC required for Time sync, user login, password changes and Trust, now you know why the PDC is important FSMO role holder to get back online, PDC role will impact the end-user immediately and we need to recover ASAP The PDC emulator Primary Domain Controller for backwards compatibility and it's responsible for time synchronizing within a domain, also the password master. Any password change is replicated to the PDC emulator ASAP. If a logon request fails due to a bad password the logon request is passed to the PDC emulator to check the password before rejecting the login request.

**Where is the AD database held and how would you create a backup of the database?**
The database is stored within the windows NTDS directory. You could create a backup of the database by creating a backup of the System State data using the default NTBACKUP tool provided by windows or by Symantec's Netbackup. The System State Backup will create a backup of the local registry, the Boot files, the COM+, the NTDS.DIT file as well as the SYSVOL folder.

**What are the components of AD?**
• Logical Structure: Trees, Forest, Domains and OU
• Physical Structures: Domain controller and Sites

**Briefly explain how Active Directory authentication works**
When a user logs into the network, the user provides a username and password. The computer sends this username and password to the KDC which contains the master list of unique long term keys for each user. The KDC creates a session key and a ticket granting ticket. This data is sent to the user's computer. The user's computer runs the data through a one-way hashing function that converts the data into the user's master key, which in turn enables the computer to communicate with the KDC, to access the resources of the domain.
or:
When a user enters a user name and password, the computer sends the user name to the Key Distribution Centre (KDC). The KDC contains a master database of unique long term keys for every principal in its realm. The KDC looks up the user's master key (KA), which is based on the user's password. The KDC then creates two items: a session key (SA) to share with the user and a Ticket-Granting Ticket (TGT). The TGT includes a second copy of the SA, the user name, and an expiration time. The KDC encrypts this ticket by using its own master key (KKDC), which only the KDC knows. The client computer receives the information from the KDC and runs the user's password through a one-way hashing function, which converts the password into the user's KA. The client computer now has a session key and a TGT so that it can securely communicate with the KDC. The client is now authenticated to the domain and is ready to access other resources in the domain by using the Kerberos protocol.

**Mention what is the difference between domain admin groups and enterprise admins group in AD?**
Enterprise Admin Group
• Members of this group have complete control of all domains in the forest.
• By default, this group belongs to the administrators group on all domain controllers in the forest.
• As such this group has full control of the forest, add users with caution.
Domain Admin Group
• Members of this group have complete control of the domain
• By default, this group is a member of the administrators group on all domain controllers, workstations and member servers at the time they are linked to the domain.
• As such the group has full control in the domain, add users with caution.

**What is Kerberos?**
Kerberos is an authentication protocol for network. It is built to offer strong authentication for server/client applications by using secret-key cryptography.

**What is FSMO?**
Flexible single master operation is a specialized domain controller (DC) set of tasks, used where standard data transfer and update methods are inadequate. AD normally relies on multiple peer DCs, each with a copy of the AD database, being synchronized by multi-master replication.

**What is Active Directory Partitions?**
Active Directory partition is how and where the AD information logically stored.

**What are all the Active Directory Partitions?**
• Schema
• Configuration
• Domain
• Application partition

**What is KCC?**
KCC (knowledge consistency checker) is used to generate replication topology for inter site replication and for intra-site replication. Within a site replication traffic is done via remote procedure calls over ip, while between sites it is done through either RPC or SMTP.

**Explain what intrasite and intersite replication is and how KCC facilitates replication**
The replication of DC's inside a single site is called intrasite replication whilst the replication of DC's on different sites is called Intersite replication. Intrasite replication occurs frequently while Intersite replication occurs mainly to ensure network bandwidth.
KCC is an acronym for the Knowledge Consistency Checker. The KCC is a process that runs on all of the Domain Controllers. The KCC allows for the replication topology of site replication within sites and between sites. Between sites, replication is done through SMTP or RPC whilst Intersite replication is done using procedure calls over IP.

**Why do we need Netlogon?**
Maintains a secure channel between this computer and the domain controller for authenticating users and services. If this service is stopped, the computer may not authenticate users and services, and the domain controller cannot register DNS records.

**Explain about Trust in AD ?**
To allow users in one domain to access resources in another, Active Directory uses trusts. Trusts inside a forest are automatically created when domains are created. The forest sets the default boundaries of trust, not the domain, and implicit, transitive trust is automatic for all domains within a forest. As well as two-way transitive trust, AD trusts can be a shortcut (joins two domains in different trees, transitive, one- or two-way), forest (transitive, one- or two-way), realm (transitive or nontransitive, one- or two-way), or external (nontransitive, one- or two-way) in order to connect to other forests or non-AD domains.

**Different modes of AD restore ?**
A nonauthoritative restore is the default method for restoring Active Directory. To perform a nonauthoritative restore, you must be able to start the domain controller in Directory Services Restore Mode. After you restore the domain controller from backup, replication partners use the standard replication protocols to update Active Directory and associated information on the restored domain controller.
An authoritative restore brings a domain or a container back to the state it was in at the time of backup and overwrites all changes made since the backup. If you do not want to replicate the changes that have been made subsequent to the last backup operation, you must perform an authoritative restore. In this one needs to stop the inbound replication first before performing the An authoritative restore.

**What is Global Catalog and its function?**
The global catalog is a distributed data repository that contains a searchable, partial representation of every object in every domain in a multidomain Active Directory Domain Services (AD DS) forest. The global catalog is stored on domain controllers that have been designated as global catalog servers and is distributed through multimaster replication. Searches that are directed to the global catalog are faster because they do not involve referrals to different domain controllers.
The global catalog provides the ability to locate objects from any domain without having to know the domain name. A global catalog server is a domain controller that, in addition to its full, writable domain directory partition replica, also stores a partial, read-only replica of all other domain directory partitions in the forest.
Forest-wide searches. The global catalog provides a resource for searching an AD DS forest. Forest-wide searches are identified by the LDAP port that they use. If the search query uses port 3268, the query is sent to a global catalog server.User logon. In a forest that has more than one domain, two conditions require the global catalog during user authentication: Universal Group Membership Caching: In a forest that has more than one domain, in sites that have domain users but no global catalog server, Universal Group Membership Caching can be used to enable caching of logon credentials so that the global catalog does not have to be contacted for subsequent user logons. This feature eliminates the need to retrieve universal group memberships across a WAN link from a global catalog server in a different site.
• In a domain that operates at the Windows 2000 native domain functional level or higher, domain controllers must request universal group membership enumeration from a global catalog server.
• When a user principal name (UPN) is used at logon and the forest has more than one domain, a global catalog server is required to resolve the name.
Exchange Address Book lookups. Servers running Microsoft Exchange Server rely on access to the global catalog for address information. Users use global catalog servers to access the global address list (GAL).

**What is RODC? Why do we configure RODC?**
Read only domain controller (RODC) is a feature of Windows Server 2008 Operating System. RODC is a read only copy of Active Directory database and it can be deployed in a remote branch office where physical security cannot be guaranteed. RODC provides more improved security and faster log on time for the branch office.
46. What is role seizure? Who do we perform role seizure?
Role seizure is the action of assigning an operations master role to a new domain controller without the support of the existing role holder (generally because it is offline due to a hardware failure). During role seizure, a new domain controller assumes the operations master role without communicating with the existing role holder. Role seizure can be done using repadmin.exe and Ntdsutil.exe commands.

**What do you understand by Garbage Collection? Explain.**
Garbage collection is a process of Active Directory. This process starts by removing the remains of previously deleted objects from the database. These objects are known as tombstones.  And the process starts a defragmentation thread to claim additional free space. The garbage collection process is running on all the domain controllers in an interval of 12 hours.

### GPO / OU

**What is the difference between local, global and universal groups**
Domain local groups assign access permissions to global domain groups for local domain resources. Global groups provide access to resources in other trusted domains. Universal groups grant access to resoures in all trusted domains.
Domain Local Group: Use this scope to grant permissions to domain resources that are located in the same domain in which you created the domain local group. Domain local groups can exist in all mixed, native and interim functional level of domains and forests. Domain local group memberships are not limited as you can add members as user accounts, universal and global groups from any domain. Just to remember, nesting cannot be done in domain local group. A domain local group will not be a member of another Domain Local or any other groups in the same domain.
Global Group: Users with similar function can be grouped under global scope and can be given permission to access a resource (like a printer or shared folder and files) available in local or another domain in same forest. To say in simple words, Global groups can be use to grant permissions to gain access to resources which are located in any domain but in a single forest as their memberships are limited. User accounts and global groups can be added only from the domain in which global group is created. Nesting is possible in Global groups within other groups as you can add a global group into another global group from any domain. Finally to provide permission to domain specific resources (like printers and published folder), they can be members of a Domain Local group. Global groups exist in all mixed, native and interim functional level of domains and forests.
Universal Group Scope: These groups are precisely used for email distribution and can be granted access to resources in all trusted domain as these groups can only be used as a security principal (security group type) in a windows 2000 native or windows server 2003 domain functional level domain. Universal group memberships are not limited like global groups. All domain user accounts and groups can be a member of universal group. Universal groups can be nested under a global or Domain Local group in any domain.

**What is group nesting.**
Adding one group as a member of another group is called 'group nesting'. This will help for easy administration and reduced replication traffic

**What is Domain controller?**
A domain controller (DC) is a server that handles all the security requests from other computers and servers within the Windows Server domain

**What is domain?**
A domain is a set of network resources (applications, printers, and so forth) for a group of users. The user needs only to log in to the domain to gain access to the resources, which may be located on a number of different servers in the network.

**What is Forest?**
A collection of one or more Active Directory domains that share a common schema, configuration, and global catalog.

**What is global catalog.**
The Active Directory Global Catalog is the central storage of information about objects in an Active Directory forest. A Global Catalog is created automatically on the first domain controller in the first domain in the forest. The Domain Controller which is hosting the Global Catalog is known as a Global catalog server.

**What is tree.**
An Active Directory tree is a collection of Active Directory domains that begins at a single root and branches out into peripheral, child domains. Domains in an Active Directory tree share the same namespace. An Active Directory forest is a collection of Active Directory trees, similar to a real world forest. Catalog Server.

**What is site.**
A Site object in Active Directory represents a geographic location that hosts networks.

**What Is Group Policy.**
Group Policy is a feature of the Microsoft Windows NT family of operating systems that control the working environment of user accounts and computer accounts. Group Policy provides the centralized management and configuration of operating systems, applications, and users' settings in an Active Directory environment.

**What is the order in which GPOs are applied? LSDOU**
Local Group Policy object
Site
Domain and
Organizational Units.

**What is the difference between software publishing and assigning.**
Assign Users : The software application is advertised when the user logs on. It is installed when the user clicks on the software application icon via the start menu, or accesses a file that has been associated with the software application.
Assign Computers :The software application is advertised and installed when it is safe to do so, such as when the computer is next restarted.
Publish to users : The software application does not appear on the start menu or desktop. This means the user may not know that the software is available. The software application is made available via the Add/Remove Programs option in control panel, or by clicking on a file that has been associated with the application. Published applications do not reinstall themselves in the event of accidental deletion, and it is not possible to publish to computers.

**Can I deploy non-MSI software with GPO.**
Yes, create the fiile in .zap extension.

**Name a few benefits of using GPMC.**
Easy administration of all GPOs across the entire Active Directory Forest
View of all GPOs in one single list
Backup and restore of GPOs Migration of GPOs across different domains and forest.

**How frequently is the client policy refreshed ?**
90 minutes give or take.

**Where are group policies stored ?**
C:\Windows\System32\GroupPolicy.

**Group policy backup**
To backup a single GPO, right-click the GPO, and then click Back Up.
To backup all GPOs in the domain, right-click Group Policy Objects and click Back Up All.

**What are three primary functions of Organizational Units?**
Organize Data, Deploy GPO, Delegate permissions (We use AD Groups to assign permissions to not get confused and be precise with your answer)

**If you have one OU and you have mixed server in it (Windows 2008 and Windows 2012 servers) How do, I apply GPO to only Windows 2012 Servers. Assuming I cannot create any new OU.**
Use WMI filtering and only target specific Windows Server version

**What are lingering objects?**
Lingering objects can exists if a domain controller does not replicate for an interval of time that is longer than the tombstone lifetime (TSL).

**What is TOMBSTONE lifetime?**
Tombstone lifetime in an Active Directory determines how long a deleted object is retained in Active Directory. The deleted objects in Active Directory is stored in a special object referred as TOMBSTONE. Usually, windows will use a 60- day tombstone lifetime if time is not set in the forest configuration.

**What is OU ?**
Organization Unit is a container object in which you can keep objects such as user accounts, groups, computer, printer . applications and other (OU). In organization unit you can assign specific permission to the user's. organization unit can also be used to create departmental limitation.

### AD Tools

**What tool would you use to edit AD?**
`Adsiedit.msc` is a low level editing tool for Active Directory. `Adsiedit.msc` is a Microsoft Management Console snap-in with a graphical user interface that allows administrators to accomplish simple tasks like adding, editing and deleting objects with a directory service. The `Adsiedit.msc` uses Application Programming Interfaces to access the Active Directory. Since `Adsiedit.msc` is a Microsoft Management Console snap-in, it requires access MMC and a connection to an Active Directory environment to function correctly.

**If you have lost one of the domain controllers how do you clean up Active Directory database for the lost DC?**
Meta Data Cleanup with Ntdsutil.exe is a command-line tool

**What is REPLMON?**
This GUI tool enables administrators to view the low-level status of Active Directory replication, force synchronization between domain controllers, view the topology in a graphical format, and monitor the status and performance of domain controller replication.

**What is NETDOM ?**
NETDOM is a command-line tool that allows management of Windows domains and trust relationships. It is used for batch management of trusts, joining computers to domains, verifying trusts, and secure channels.

**What tool can interact with (. DIT) database (the most common)**
`adsiedit.msc`

**Tell me few uses of NTDSUTIL commands?**
We can use ntdsutil commands to perform database maintenance of AD DS, manage and control single master operations, Active Directory Backup restoration and remove metadata left behind by domain controllers that were removed from the network without being properly uninstalled.

**How would you manage trust relationships from the command prompt?**
Netdom.exe is another program within Active Directory that allows administrators to manage the Active Directory. Netdom.exe is a command line application that allows administrators to manage trust relationship within Active Directory from the command prompt. Netdom.exe allows for batch management of trusts. It allows administrators to join computers to domains. The application also allows administrators to verify trusts and secure Active Directory channels.


text:
# Specter Init
---
Specter Init is used to create the package.json file.

To do this, you simply type:

```$ specter init``` (do not put the dollar sign.)

It will prompt you with different questions:

```Author```: this is the project author.

```Project Name```: this is the name of your project.

```description```: this is where you desribe what your project is about.

After, it will show you the generated code and ask if you are happy with your code,

If you are you type ```y or Y```

repository: iplo/Specter

repo_name: iplo/Specter