text:
# Spiking Network Simulations #

_Sections_
* **[Code Summary](#simulation-scripts)**
* **[Running Simulations](#use)**
* **[Troubleshooting](#setup)**

## Developer Notes ##
* 2021-04-14: Forked repository.
* *This code has been adapted from the original source code.*
  *  **Original code** is provided by **[Larry Shupe](https://github.com/lshupe/spikenet)**.
  * **Associated original manuscript** has been published on **[eNeuro](https://www.eneuro.org/content/8/2/ENEURO.0333-20.2021)**.
* *This README was adapted from the original `spikenet_readme.txt`, with some modifications.*

## Contents ##

### Simulation Scripts ###

These MATLAB `script` files that run simulations with different pre-configured parameters.

* `spikenet50.m`  -- Run the base simulation without conditioning.
* `spikenet_spike_trig.m`  -- Configured for spike triggered conditioning
* `spikenet_paired_pulse.m`  -- Configured for paired pulse conditioning
* `spikenet_cycle_trig.m`  -- Configured for cycle triggered conditioning
* `spikenet_tetanic.m`  -- Configured for tetanic conditioning
* `spikenet_emg_trig`  -- Configured for EMG triggered conditioning
* `spikenet_gamma_trig`  -- Configured for gamma filtered LFP triggered conditioning
* `spikenet_FICspike_trig.m`  -- Configured for spike triggered conditioning with Fixed Intracolumn Connections (FIC).

### Helpers ###

* `spikenet50mex.c`  -- The `mex` function that runs the network in blocks of 10 seconds
* `spikenet50mex.mexw64`  -- Precompiled MATLAB `mex` function, which can be used to re-compile `spikenet50mex.c` in case you are running a different OS or version of MATLAB.
* `spikenet_wfig.m`  -- Creates a weight figure from a simulation’s saved `param.mat` file.

## Use ##

Once the simulation files have been installed, open MATLAB and change the current
directory to the folder containing these files (or add the files to the default MATLAB `path` as described **[above](#setting-the-matlab-path)**).

### Running a simulation without conditioning ###

1. Run `spikenet_no_cond.m`.
   * This script will take up to an hour to run, but will update a graph
     for every 10 seconds of simulated time.  This graph is used to monitor the progress of the simulation.  It displays average activity of excitatory and output units for the 10 second time blocks for the current training period, averages of filtered local field potentials (LFP), and the current distribution of trainable connection strengths.
   * These averages reset at the beginning of each of the four training
     periods:
     * No Conditioning,
     * Preconditioning testing,
     * Conditioning, and
     * Post-Conditioning testing
   * Connections between units are modified by the Spike-Timing Dependent
     Plasticity (STDP) rule during the Preconditioning and Conditioning periods, but
     connection strengths are held static during the testing periods to build histograms and conduct tests.

### Data structure and location ###

#### Data Location ####

> `C:/data/directory`

#### Figure Name Schema ####

All files ending in `.fig` are MATLAB `figure` files that can be opened directly in the MATLAB base workspace to reproduce the vectorized figures (and associated data).

*Note*: in each of the following filenames, (`<n>` is the the simulation trial number for each 10 second block).

* `averages_cond_<n>.fig` – The  monitoring averages saved during the conditioning period.
* `averages_nocond_<n>.fig` – The monitoring averages saved during the no-conditioning period.
* `averages_pre_<n>.fig` – The monitoring averages saved during the pre-conditioning period.
* `averages_post_<n>.fig` – The monitoring averages saved during the post-conditioning period.
* `Cond_Trig_PSTH-<n>.fig` – Spike triggered averages and histograms aligned with conditioning triggers.
* `emg_spike_response.fig` – Stimulus triggered averages of the simulated EMG.
* `lfp_spike_response.fig` – Spike-triggered LFP averages.
* `lfp_stim_response.fig` – Stimulus triggered LFP averages (used for A->B evoked potential changes).
* `mean_Evoked_Potentials.fig` – Tracks MPI from the lfp_stim_response.fig when multiple repetitions through the training sequence are used (when p.ntrials_repeat > 1)
  output_spikes.mat – Contains spike times (in 0.1 ms time steps) for unit firings during the simulation.
* `param.mat` – Contains a copy of the simulations parameters and resulting connection strengths.
* `progress.txt` – A copy of the console output during the simulation.
* `PSTH.fig` – Shows correlated activity between unit populations
* `Raster-<n>.fig` -- Contains a dot raster for 2 seconds of activity and power spectrums for LFP.
* `weight_distribution.fig` – Distribution of weights between unit columns.
  * This is useful for checking the efficacy of any arbitrary conditioning method of interest in greater detail.
* `weight_matrix.fig` – Displays final connection strengths.
  * Use `spikenet_wfig.m` on the `param.mat` file to plot a formatted graph with labels.

#### Other Saved Outputs ####

These files are primarily used for validation and checking consistency/reproducibility of the simulations that were conducted.

* `spiknet.m` – Contains a copy of the `function` that runs this simulation.
  * This is useful to validate/verify that the correct parameters were implemented for a given simulation iteration.
* `spikenet50mex.c` – contains a copy of the `mex` function.
* `spikenet50mex.mexw64` – Contains a copy of the compiled `mex` function.

## Experiments ##

### Conditioning Methods ###

*Contents*

* **[No Conditioning](#no-conditioning)**
* **[Spike-triggered Stimulation](#spike-triggered-stimulation)**
* **[Paired-Pulse Stimulation](#paired-pulse-stimulation)**
* **[Cycle-triggered Stimulation](#cycle-triggered-stimulation)**
* **[Tetanic Conditioning](#tetanic-conditioning)**
* **[EMG-triggered Stimulation](#emg-triggered-stimulation)**
* **[Gamma-triggered Stimulation](#gamma-triggered-stimulation)**
* **[Spike-triggered Stimulation (Fixed Inter-columnar Connections; FIC)](#spike-triggered-fic-stimulation)**

#### No Conditioning ####

The base (non-conditioning) simulation provides information on network progression in the absence of any conditioning stimulus intervention.  This can be used to see how much random variation there is by running the network for different periods of time or by using different initial conditions.  The base simulation file `spikenet_no_cond.m` contains a standard set of parameters. **These parameters are used by the other simulations.**

To trouble-shoot parameter configurations, start here!

[*back to list of stimulation methods*](#conditioning-methods)

#### Spike-triggered Stimulation ####

This conditioning method delivers a stimulus to units in **Column B** each time that the first excitatory unit in **Column A** (`Ae1`) fires. After a given stimulus, any subsequent stimuli are "blanked" (do not occur) for the duration of the "blanking period" parameter. This prevents stimulus-locked feedback loops from occurring, which could be deleterious *in vivo*. Since the timing of the stimuli depend on detected activity in the network, this is referred to as a “closed loop” stimulation method.

##### Notable parameters #####
```matlab
% % % Key to running this simulation! % % %
p.conditioning_method = 1;  	% Selects spike triggered stimulation

% % Also important % %
p.stim_delay_ms = 10;    	% Millisecond delay between spike on Ae1 and
				% 	stimulus on Column B.
p.stim_pulse_train = 1;  	% Can be 2 or 3 for stimulus trains of 2 or 3
				% 	pulses per train.
p.stim_uV = 2000;        	% Size of conditioning stimulus
   				%	(0 can be used as a sham stimulus)
p.stim_refractory_ms = 10;  	% Refractory period on delivered stimulation.
```

[*back to list of stimulation methods*](#conditioning-methods)

#### Paired-Pulse Stimulation ####

This conditioning method delivers a stimulus to units in **Column A** followed by a delayed stimulus that is delivered to units of **Column B**.  These stimuli are not tied to any activity in the network. This is therefore an “open loop” stimulation method (with respect to the "natural" activity of the network).

##### Notable parameters #####

```matlab
% % % Key to running this simulation! % % %
p.conditioning_method = 2;  		% Selects paired pulse stimulation

% % Also important % %
p.conditioning_secs = 7; 		% 1..10 to change number of paired pulses in each 10 second time block.
p.stim_delay_ms = 10;    		% Delay between the paired pulses
					%	Units: milliseconds
p.stim_pulse_train = 1;  		% Number of stimulus pulses per train.
					%	Although the blanking period "debounces" additional stimulus trains
                                	%	increasing this count greater than one will still deliver multiple
                                	%	stimuli each time a stimulus is "triggered."
p.stim_uV = 2000;        		% Size of conditioning stimulus (0 can be used as a sham stimulus)
					%	Units: micro-volts
p.pair_uV = 2000;        		% Size of the paired pulse simulation (the second pulse)
					%	Units: micro-volts
```

#### Cycle-triggered Stimulation ####

A "closed loop" conditioning method that detects oscillations in the spike rate of **Column B** in order to trigger a stimulus train targeting units in **Column A**.

##### Notable parameters #####

```matlab
% % % Key to running this simulation! % % %
p.conditioning_method = 3;  	% Selects cycle triggered stimulation

% % Also important % %
p.bias_modulation_rate_B = 20; 	% Cycles per second modulation rate on Column B. 0 = no modulation.
				% 	Units: Hz
p.bias_modulation_amount = .2; 	% Size of modulation. 0.2 = +/- 20% of normal rate.
p.stim_pulse_train = 1;        	% Number of stimulus pulses per train.
				%	Although the blanking period "debounces" additional stimulus trains
                                %	increasing this count greater than one will still deliver multiple
                                %	stimuli each time a stimulus is "triggered."
p.stim_uV = 2000;              	% Size of conditioning stimulus (0 can be used as a sham stimulus)
				%	Units: micro-volts
```

[*back to list of stimulation methods*](#conditioning-methods)

#### Tetanic Conditioning ####

An open loop conditioning method that applies random stimulation on the units of a specific column.  This is an important control for other stimulation methods, because it is completely "open loop" with respect to both network activity and any pairing of **Column A** with **Column B** (unlike occurs in **[Paired-Pulse Stimulation](#paired-pulse-stimulation)**)

##### Notable parameters #####

```matlab
% % % Key to running this simulation! % % %
p.conditioning_method = 5; 	% Tetanic stimulation:
				%   4 => Stimulation targets Column A units
				%   5 => Stimulation targets Column B units
				%   6 => Stimulation targets Column C units

% % Also important % %
p.tetanic_freq = 10;       	% Stimulation intensity (rate parameter).
				% 	This sets the intensity of the exponentially distributed
				%	inter-stimulus intervals in the tetanic conditioning regime.
				%		Units: stimuli/sec
p.stim_refractory_ms = 10; 	% Refractory period.
				%	Sets the minimum number of milliseconds between tetanic stimuli.
				%	This effectively reduces the average rate of stimulation,
				%	since it prevents realization of a true exponential distribution.
				%	It is included to reproduce the tetanic stimulation refractory period
                                %	from in vivo protocols, which is included as a
                                %	precautionary safety measure.
				%		Units: milliseconds
p.stim_pulse_train = 1;    	% Number of stimulus pulses per train.
				%	Although the blanking period "debounces" additional stimulus trains
                                %	increasing this count greater than one will still deliver multiple
                                %	stimuli each time a stimulus is "triggered."
p.stim_uV = 2000;          	% Size of conditioning stimulus (0 can be used as a sham stimulus)
				%	Units: micro-volts
p.bias_modulation_amount = 0; 	% Normally 0.  Can put a 0.2 to 0.4 step bias spike increase on Column B.
				%	Units: arbitrary (probability)
```

[*back to list of stimulation methods*](#conditioning-methods)

#### EMG-triggered Stimulation ####

A closed loop conditioning method that applies stimuli to units in **Column B** when the output of the transfer function relating activity of units in **Column A** to the generation of electromyographic signals in some target muscle(s) crosses a specified threshold.

##### Notable parameters #####

```matlab
p.conditioning_method = 7;  	% Select EMG triggered stimulation
p.lfp_detect_level = 1000;  	% EMG detection level
				%	Units: arbitrary (see transfer function for EMG output)
p.stim_delay_ms = 0;        	% Delay between EMG level detection and stimulation of Column B.
				%	(Note: The transfer function generating synthetic EMG signals
				%				intrinsically delays the output EMG by 10-ms due to the
				%				default synaptic delay parameters between Ae and Ao units)
				%	Units: milliseconds
p.stim_pulse_train = 1;  	% Number of stimulus pulses per train.
                            	%	Although the blanking period "debounces" additional stimulus trains
                            	%	increasing this count greater than one will still deliver multiple
                            	%	stimuli each time a stimulus is "triggered."
p.stim_uV = 2000;        	% Size of conditioning stimulus (0 can be used as a sham stimulus)
				%	Units: micro-volts
```

[*back to list of stimulation methods*](#conditioning-methods)

#### Gamma-triggered Stimulation ####

A "closed loop" conditioning method that applies stimuli to units in **Column B** when the band-passed local field potential (LFP; generated via a transfer function relating the activity of units in **Column A** to a field potential source capable of generating a time-varying electrical field) crosses an arbitrary threshold.

##### Notable parameters #####

```matlab
% % % Key to running this simulation! % % %
p.conditioning_method = 8;  	% Select gamma triggered stimulation

% % Also important % %
p.stim_delay_ms = 0;          	% Millisecond delay between gamma threshold-level detection and
				%	stimulation delivery to Column B.
				%		(use value 8+ for rising level detection)
				%			Units: milliseconds
p.lfp_detect_level = -20000;  	% Filtered LFPA detection level (20000 for rising level, -20000 for falling)
p.gamma_band = [50 80];       	% Bandwidth that selects for `gamma` from the broadband local field
				%	potential (LFP) signal. (As an example, [50 80] specifies cutoff
				%	frequencies of 50Hz and 80Hz for the bandpass filter).
				%			Units: Hz
p.stim_pulse_train = 1;  	% Number of stimulus pulses per train.
                            	%	Although the blanking period "debounces" additional stimulus trains
                            	%	increasing this count greater than one will still deliver multiple
                            	%	stimuli each time a stimulus is "triggered."
p.stim_uV = 2000;        	% Size of conditioning stimulus (0 can be used as a sham stimulus)
				%	Units: micro-volts
```

[*back to list of stimulation methods*](#conditioning-methods)

#### Spike-triggered FIC Stimulation ####

Similar to normal spike-triggered conditioning ("closed loop"), except that fixed intercolumnar connections (FIC) are used in order to simulate the role of the correlated bias inputs to provide correlated activity within each column.

```matlab
% % % Key to running this simulation! % % %
p.conditioning_type = 1; 			% Note: this is the same as the default "no conditioning" condition, but
						%	uses altered parameters.

% % Also important % %
p.bias_correlated_fraction = 0.0
p.initmin = 200;  					% Lower bound on initial PSP amplitude for any connections
p.initmax = 300; 					% Upper bound on initial PSP amplitude for any connections
p.epsp2excit_incol = [p.initmin p.initmax 1 0];     	% EPSP for connections to excitatory units within a column
p.epsp2inhib_incol = [p.initmin p.initmax 1 0];     	% EPSP for connections to inhibitory units within a column
p.ipsp2excit_incol = [-p.initmax -p.initmin 1 0];   	% IPSP for connections to excitatory units within a column
p.ipsp2inhib_incol = [-p.initmax -p.initmin 1 0];   	% IPSP for connections to inhibitory units within a column
p.stim_delay_ms = 10;    				% Delay between spike on Ae1 and stimulus on Column B.
							%	Units: milliseconds
p.stim_pulse_train = 1;  	% Number of stimulus pulses per train.
                        	%	Although the blanking period "debounces" additional stimulus trains
				%	increasing this count greater than one will still deliver multiple
                            	%	stimuli each time a stimulus is "triggered."
p.stim_uV = 2000;        	% Size of conditioning stimulus (0 can be used as a sham stimulus)
				%	Units: micro-volts
p.stim_refractory_ms = 10;  	% Refractory period on delivered stimulation.
				%	Units: milliseconds
```

[*back to list of stimulation methods*](#conditioning-methods)

### General Comments ###

Most parameters are set **within the first 200 lines** of the **[main simulation script](#no-conditioning)**.

#### Other Parameters ####

The following is a list of parameters that have not yet been described. Since they have been modified in previous simulations, it is likely that they will be altered again in future simulations.

The following parameters are used to control the length of each training phase. Larger values for `p.ntrials_off` will allow the network to "settle" for longer during the pre-conditioning phase.

**Note about initial phase (conditioning off) duration.**

Larger values of `p.ntrials_off` allow better convergence to "natural" synaptic connection values given the network topology. However, convergence comes at the expense of longer simulation times!

```matlab
p.ntrials_off = 50;     % Number of initial trials with conditioning off (STDP training on).
p.ntrials_on = 50;      % Number of following trials with conditioning on (STDP training on).
p.ntrials_test = 50;    % Number of trials for summary figures (both conditioning and STDP training off)
p.ntrials_repeat = 1;   % Number of repetitions through the off/test/on/test cycle.
```

The firing threshold for units can be increased to lower overall firing rates, but must be balanced against the external bias input rates.  The correlated external biases are only correlated within each individual column.

Increasing the correlated fraction will tend to increase firing rates and also improve conditioning effects that depend on intercolumn correlated activity such as spike-triggered stimulation.

Output units only receive uncorrelated bias spikes, the rate of which can be adjusted independently.

```matlab
p.threshold = 5000;   				% Threshold for all cortical units.
						%	Units: micro-volts
p.bias_value = 350;   				% Strenth of bias potientials.
						%	Units: micro-volts
p.bias_rate = 1800;   				% Bias of each column unit.
						%	Consider smaller values if biases are modulated.
						%	Units: spikes/second
p.bias_correlated_fraction = .30; 		% Fraction of correlated bias spikes
p.correlated_bias_std_ms = 3;      		% Standard deviation of normally distributed (correlated) spiking bias.
						%	Units: milliseconds
p.out_bias_rate = 2000;            		% Uncorrelated bias spikes for output units.
						%	Units: spikes/second
```

The training rate and weakening factors can be altered to change the how fast connection strengths change.  Decreasing the training_rate usually means the network will need to train longer, but if the value is too high then there will be more variability in weights and it may seem like the network doesn’t settle down.  Increasing the weakening factor controls the effectiveness of the
weakening side (negative post-presynaptic spike time differences) of the STDP curve relative to the strengthening side (positive post-presynaptic spike time differences).

Decreasing the weakening factor can increase the overall activity of the network, but at a certain point all weights will tend to increase to the maximum allowed value.  Typically we want to have the weakening factor large enough to prevent that but not so large that connections in the network cannot be conditioned.  This will also be affected by the shape of the curve which is
controlled by fast and slow decay constants for both halves of the STDP function.

```matlab
p.training_rate = 100;    		% Train rate factor for spike-timing-dependent plasticity.
					% 	This factor accomodates both the strengthing (positive) and
					%	weakening (negative) sides of the SDTP rule,
					%	so it is always specified as a positive value.
p.train_weakening = 0.55; 		% Relative amplitude for the weakening side of the SDTP curve.
```

The network topology can be changed by altering the column connection definitions.

Initial weights are from a uniform distribution between a minimum and maximum value.

The connection probability for excitatory units is usually 1/6, but can be increased to provide increased network activity.  Inhibitory connections are only within column and have a larger connection probability to help balance the number of inhibitory vs excitatory connections in the network.

The random seed allows the use of the same set of pseudo-random connections for many different simulations.  The initial network configuration *can have a significant effect on the outcome*, **so being able to reuse a network configuration is important**. In practice, simulations should be run **multiple times with *different* random seeds**, then average together the results from those random seeds. However, for reproducibility, it is important to keep track of the random seeds that were used!

```matlab
% Initial post-spike potential (PSP) ranges in uV.
p.initmin = 100;		% Initial minimum PSP value (micro-volts)
p.initmax = 300; 		% Initial maximum PSP value (micro-volts)

% All array parameters follow the format:
%	[microvolts_min microvolts_max probability_of_connection enable_training]

% EPSPs for connections to excitatory units within a column
p.epsp2excit_incol = [p.initmin p.initmax 1/6 1];

% EPSPs for connections to inhibitory units within a column
p.epsp2inhib_incol = [p.initmin p.initmax 1/6 1];

 % EPSPs for connections to excitatory units in adjacent columns
p.epsp2excit_outcol = [p.initmin p.initmax 1/6 1];

% EPSPs for connections to inhibitory units in adjacent columns
p.epsp2inhib_outcol = [p.initmin p.initmax 1/6 1];

% IPSPs for connections to excitatory units within a column
p.ipsp2excit_incol = [-p.initmax -p.initmin 1/3 1];

% IPSPs for connections to inhibitory units within a column
p.ipsp2inhib_incol = [-p.initmax -p.initmin 1/3 1];

% EPSPs for connections to output units.
p.epsp2output_incol = [350 350 1/3 0];
% Note: These values are **not** usually trained.
%	Negative connection chance is interpreted by the simulations as:
%	 > "Connect to the first p * N units."
%	Therefore, it is a way to set a FIXED set of connections, rather than chance connections.

p.random_seed = 1; 	% Fixed random number seed.
			%	This allows us to use the same network
			%	topology (connections) for different conditioning methods.
			%
			%	The random seed should be altered at different stages of the simulations,
			%	depending on which factors are important to test and where the variability arises.
```

## Setup

### Software Requirements ###

* Windows 10 (64-bit)

  > The source code for the integrate-and-fire (IF) neural network model is written in Matlab and C for the 64-bit version running under Windows 10.

* Matlab (R2017a+)

  > The C code is written as a Matlab `mex` function, which has been pre-compiled for current (R2017a+) 64-bit versions of Matlab for PC.

 ### Troubleshooting ###

#### Parameter configuration ####

The base simulation file `spikenet_no_cond.m` contains a standard set of parameters. **These parameters are used by the other simulations.** To trouble-shoot parameter configurations, start here!

#### Re-compiling `mex` function ####

For older versions of Matlab, or for other operating systems such as Mac OSX, it is necessary to re-compile the `mex` file, which allows the code to run efficiently.

> This can be accomplished by using the `mex` command from within the MATLAB Command Window interface, specifying the correct C compiler.

*Note:* the system must have a valid compiler configured, which is then set up in MATLAB separately from the `mex` command. For more information, please see the MATLAB `mex` documentation, located **[here](https://www.mathworks.com/help/matlab/ref/mex.html)**.

#### Setting the MATLAB `path` ####

> To install the source code, copy all files to a folder on the MATLAB `path`.
>
> * This is usually the MATLAB folder within the user’s Documents folder (e.g. `C:/<user>/Documents/Matlab/` on a PC).

#### About MATLAB `script` and `function` files ####

(**For beginners**): MATLAB `.m` files that are not part of an object-oriented structure are either `script` files or `function` files.

* Unlike MATLAB a `function`, a MATLAB `script` does not require any input arguments and therefore can be treated like an executable program in MATLAB.

* To run a MATLAB `script`, type the name (without the file extension) of the `script` into the `Command Window` directly in the MATLAB editor, and press enter.

* All of the `.m` files in **this project** are MATLAB `function` files. A MATLAB `function` starts with the syntax `function <filename>(<arguments>)`.

  ```matlab
  % Example: if the file is `spikenet50.m`
  % function spikenet50()
  ```

  A MATLAB `function` that does not take any arguments can be called in the same way as a MATLAB `script`, and the only difference is that all of the `function` variables will only exist in the `function` workspace, as opposed to the base MATLAB workspace as they would if the `function` were instead a `script`. These functions can be invoked in the same way as a MATLAB `script`:

  ```matlab
  >> spikenet_no_cond;
  ```

  Alternatively, the `script` can be run by opening it in the MATLAB editor and clicking the green `Run` button at the top of the `Editor` tab.

  *Note:* using MATLAB `function` files instead of `script` files makes it less-likely that you will use incorrect parameters from one simulation while running a different simulation and is therefore better practice, although the use of `script` files can make debugging and development go faster since the `script` is effectively always running in "debug" mode with variables appearing in the Editor's Base workspace.


text:
# Extracting Phabricator Passphrase and importing into 1Password

For whatever reason, you may find yourself needing to extract the whole Phabricator Passphrase database and importing it into your 1Password.

For this, you will need to have:
1. SSH access to the server instance where your Phabricator is hosted
2. Access rights to be able to view Phabricator configuration files
3. With the above two, you can view the login information to the mysql database
4. mrc-converter-suite

## Proceed with caution, caveat emptor

Though the following steps are safe if you follow it exactly, you can do ***very bad things*** if you don't know what you are doing while logged into your Phabricator server instance or the mysql database.

I will take no responsibility if you wipe out your Phabricator server, mysql database, or you let your plain text password mysql dump file fall into the wrong hands while following these instructions.

## mrc-converter-suite

You'll need to grab this 1Password community tool from:

https://discussions.agilebits.com/discussion/30286/mrcs-convert-to-1password-utility-mrc-converter-suite

This tool is required because it allows custom fields to be added to your 1Password entry that you are importing.

Unzip the zipfile into the same directory as this Git checkout so that `mrc-converter-suite` is a directory under the Git repo.

You will need Perl available to use this tool, which I assume you do.

## Finding Phabrictor's settings file

The first thing is to find out your Phabricator's MySQL login information. You can easily do this by looking in the `local.json` configuration file for your Phabricator. This is usually found in the directory `/phabricator/conf/local/local.json`.

The four fields in `local.json` you are interested in are:
```
    "mysql.host" - eg. mysql.abc.com
    "mysql.port" - eg. 3306
    "mysql.pass" - eg. my_p@ssw0rd
    "mysql.user" - eg. phabricator
```

## Extracting the databases from MySQL

Once you have the above information, you can use `mysqldump` to grab the two databases that you will need. The two databases you are interested in are:
1. phabricator_passphrase
2. phabricator_user

The user database is used to reference the full name of the Passphrase entry's Author, which we'll add to the 1Password entry so that we'll know who created the entry.

The commands for `mysqldump` are as follows, using the example values given above:
```
    mysqldump --user=phabricator --password --host mysql.abc.com --port 3306 phabricator_passphrase > phabricator_passphrase.sql
    mysqldump --user=phabricator --password --host mysql.abc.com --port 3306 phabricator_user > phabricator_user.sql
```

Be very careful with securing the file `phabricator_passphrase.sql`, as all the passwords in the file are not encrypted and anyone who gets a copy of the file will be able to read it like an open book.

Transfer these two `.sql` files to the same directory as this Git checkout. I used scp.

## Converting Passphrase to 1Password using passphrase_to_1password.py

Finally, after all the above, you will be ready to use this Python script to do the conversion and output a `.1pif` file that can be imported into 1Password.

The most basic call you will need to do is:

```
    python3 passphrase_to_1password.py -p phabricator_passphrase.sql -u phabricator_user.sql
```

This will output two files named:
1. `output_<date and timestamp>.csv`
2. `output_<date and timestamp>.1pif`

The `.csv` file is an intermediate file that is fed to mrc-converter-suite and can be deleted.

The `.1pif` file is the one you need to import into 1Password.

If you are having problems, a couple useful options available to help are:
```
  -s, --save_intermediate_file
                        Flag to saves the intermediate data as json files (for
                        debug)
  -d {DEBUG,INFO,ERROR}, --debug_level {DEBUG,INFO,ERROR}
                        Select the debug level (default level INFO)
```

The `-s` flag will save the itermediate json files for inspection.

The `-d` option allows you to select `DEBUG` logging level for more information about the fields found for each table.

## What gets imported into 1Password

The following fields are imported into 1Password:
1. Title - The title will be the combined Passphrase index number, with `K` prepended to it, together with the `name` of the Passphrase entry.
2. Username
3. Password - This will depend on the `CredentialType` of the Passphrase entry, in my instance, there were five different types. More below.
4. Author - The name of the author of the Passphrase entry.
5. Created - This is the timestamp when the Passphrase entry was created.
6. Modified - This is the last time anyone interacted with the Passphrase entry (including viewing the secret)
7. Notes - This is the Description field in the Passphrase entry.
8. Creditial-type - This is the Passphrase entry's `CredentialType`, I only included the 5 types I found from my `mysqldump`. They are:
    1. note
    2. ssh-generated-key
    3. ssh-key-text
    4. token
    5. password

If the Credential-type is not `password`, then the imported 1Password entry's `password` field will be blank. Instead, an original field with the same name as the Credential-type will contain the secret.

## Handling escape backslashes, double and single quotes

I have tried my best to correctly handle the escape backslashes, double and single quotes. I think I got it right, but if your import turns out looking wrong, let me know.

license: MIT

branch_name: refs/heads/master

extension: md

license_type: permissive


text:
# markdown-it-synapse-lambda
Implement markdown-it-synapse-server as lambda

To deploy:
```bash
cdk deploy --profile <deployer-profile> -c CERT_ARN=<arn_domain_cert> -c DOMAIN_NAME=<custom_domain_name>
```

Then update the DNS record in Route53 to an A-record that's an alias to the API gateway created.

repository: xschildw/markdown-it-synapse-lambda

branch_name: refs/heads/develop

gha_language: Python

gha_license_id: MIT

repo_name: xschildw/markdown-it-synapse-lambda


text:
# ObjectsJson
[![Coverage](https://sonarcloud.io/api/project_badges/measure?project=Shryne_ObjectsJson&metric=coverage)](https://sonarcloud.io/dashboard?id=Shryne_ObjectsJson) [![Bugs](https://sonarcloud.io/api/project_badges/measure?project=Shryne_ObjectsJson&metric=bugs)](https://sonarcloud.io/dashboard?id=Shryne_ObjectsJson) [![Code Smells](https://sonarcloud.io/api/project_badges/measure?project=Shryne_ObjectsJson&metric=code_smells)](https://sonarcloud.io/dashboard?id=Shryne_ObjectsJson) [![Duplicated Lines (%)](https://sonarcloud.io/api/project_badges/measure?project=Shryne_ObjectsJson&metric=duplicated_lines_density)](https://sonarcloud.io/dashboard?id=Shryne_ObjectsJson)
This library offers a more object oriented way to parse json data.

# Characteristics
  * Declaritive design
  * Everything is immutable
  * Everything uses interfaces
  * Easy expandable
  * No unnecessary exception throwing
  * **No object mapping**
  * Supports conversion from and to a json file or anything else
  * Own json sources or targets can be defined
  * Uses mutation testing

# Examples
## Creating json
A json object can be created like this:
```kotlin
val json = JsonObject(
  "firstKey" to JsonString("firstValue"),
  "secondKey" to JsonInt(323),
  "thirdKey" to JsonObject(
    "fourthKey" to JsonArray(
      "a" to JsonBoolean("true"),
      "b" to JsonInt("342")
    )
  )
)
```
To export this data:
```kotlin
json.exportTo(
  FileTarget("path")
)
```
Result:
```json
{
  "firstKey": "firstValue",
  "secondKey": 323,
  "thirdKey": {
    "fourthKey": [
      1, 2, 3, 4, 5
    ]
  }
}
```

## Importing a json
To import json data, one has to specify the source of it:
```kotlin
JsonObject(
  FileSource("path")
)
```
Other sources are also possible. The JsonObject can be constructed
by using a string, a file or other json objects. It just needs a class
that implements the Source interface.

repository: Shryne/ObjectsJson

gha_language: Kotlin

repo_name: Shryne/ObjectsJson


text:
# Demo

Install dependencies (make sure you're in the demo folder)

```bash
$ cd demo
```
```bash
$ npm i
```

Create a .env file in the root of the demo folder with your credentials:

**./demo/.env**
```html
GOOGLE_API_KEY="your-API-key-here"
GOOGLE_ENGINE="your-engine-key-here"
```

Then run:
```bash
$ npm run serve
```

repository: StudioSpindle/google-srp

gha_language: JavaScript

repo_name: StudioSpindle/google-srp


text:
---
date: 2017-08-15 10:00:00+00:00
slug: tifu-by-deleting-my-work
title: '#TIFU by deleting my work'
pageDesc: 'I lost two days’ work 😱 (and restored it from Chrome’s cache 😅)'
---

Ever get that feeling you’ve lost something?

![me after deleting two days’ work](/images/blog/scared-spongebob.gif)

I’ve long been a paying user of CodePen. It’s nice for sharing snippets, quick mock-ups, or messing around with code. [CodePen Projects](https://codepen.io/pro/projects) is a newish addition. It provides a more complete development environment.

A mini-project came along for a client that gave me the perfect excuse to try CodePen Projects.

Development was smooth. Right up until my main JavaScript file disappeared. I can’t say for sure why. I was dragging files around. Renaming stuff. The UI started to feel a little clunky. I had several instances of the project open. All I know is my file had vanished. A bug? Most likely my own stupidity. No way of knowing for sure. All I know is there was a distinct lack of two day’s worth of JavaScript.

It was good code too, I’d even commented it.

The slow realisation that I may have to rewrite everything from memory was agonising. Refusing that fate, I thought long and hard…

🤔

Chrome’s cache!

I navigated to `chrome://cache/` (in Chrome, obviously). A quick search found the JavaScript file I’d lost. I then converted the hex dump to binary ([this Stack Overflow answer](https://stackoverflow.com/a/31616371) provided a solution). These hex dumps look like they can contain gzipped data, so an extra step may be necessary to decode. I didn’t have too.

I’m very lucky!

I’m also — for the time being — returning to my offline editor which can backup and commit to source control. I do love me some CodePen, it's not to blame here, but for larger projects I'm not comfortable without backups.

I was thinking of writing a script to make pulling files from the browser cache easier. But ensuring I'll never need to again is time better spent.

Be careful out there!

repository: tied/DevArtifacts

repo_name: tied/DevArtifacts


text:
# Time_series_analysis
LSTM , RNN for stochastically  order data (time series)

### Example
#### prediction plot
![img1](https://github.com/vishalbpatil1/Time_series_analysis/blob/main/prediction%2Bactual%20plot.png)

#### sample plot of multisteps prediction
![img2](https://github.com/vishalbpatil1/Time_series_analysis/blob/main/predictionsteps%204.png)

#### sample of prediction table
![img3](https://github.com/vishalbpatil1/Time_series_analysis/blob/main/prediction%20table.png)

branch_name: refs/heads/main


text:
## General Electricity sector Decarbonization Model (GEDM)
GEDM is a Python package developed to estimate the capacity and generation mixes of energy technologies in large-scale electricity systems, with a focus on renewables. It consists of an electricity dispatch model and a capacity expansion model which are formulated as mathematical programming problems. A framework is developed to control the program flow, access an external solver, and convert input and output data. This tool offers flexibility in the structure of the electricity market considered, time-slices, time periods and time horizon configuration, and all assumptions can be modified.

### Main features
A model instance consists of a collection of CSV files that define the market structure, commodities/fuels, technologies/processes, resource potentials and assumptions and so on. This model framework imports these files, builds optimization problems, solves the problems, constructs result tables and exports to CSV files.

Key features of this framework:
- Flexibilities on structuring the model
- Integration of power system flexibility measures, e.g. storage system operation, interconnection and reserve services.
- Tests on system reliability with extreme supply and demand cases
- Dedicated input/output modules that convert the data into more readable table format
- Integration of VRE potential and hourly generation data based on an existing open access dataset provided by the author
- Freely available under the MIT license

This framework offers flexibility on configuring the following:
- market structure (market -> countries/districts -> zones)
- time-slices
- time period steps
- time horizon and base year
- most assumptions: techno-economic, fuel price, carbon cost, emissions limits, development limits etc.

---

### Requirements
GEDM has been tested on Windows. Running GEDM requires the following environment:
- the [Python3](https://www.python.org/downloads/release/python-368/) programming language (version 3.6 or greater)
- Python modules and their dependencies: [Numpy](https://github.com/numpy/numpy), [Pyomo](https://github.com/Pyomo/pyomo)
- installing a mathematical problem solver: GEDM has been tested with [GAMS](https://www.gams.com/) and [GLPK](https://www.gnu.org/software/glpk/) (It is possible to use other solvers in this framework if Pyomo supports the interface.)
- installing [Python API for GAMS](https://www.gams.com/latest/docs/API_PY_TUTORIAL.html) when using GAMS as default solver (GAMS software required)
- download [GLPK](https://www.gnu.org/software/glpk/) solver and set sytem environment variable "Path" to the folder when using GLPK as default solver

### Getting started
Using this tool by the following steps:
1. install the above requirements
2. download or clone the GEDM repository
3. check the solver option file "/Input/1_model_config/01_SolverConfig.csv", create a file folder for Pyomo
4. construct a case by editing the input files (you can start with the sample case without changing anything)
5. run the model `$ Python scripts/GEDM.py`
6. check the output files

Users are expected to have experiences on Python. The main reason is when infeasible solutions returned, the users may want to have control over the program flow and formulations to find the causes, other than changing the settings and assumption. In addition, Pyomo and solvers may occasionally have softeware compatible issues. It would require some Python experience to change the codes and solve it.

---

### Main components
The overall framework and program flow is shown below. The electricity dispatch (ED) model (md_main.py, md_ED_\*.py) deals with the generation dispatch decisions for all generators, taking into account operation constraints of conventional thermal plants and storage systems, variability of renewable generation, and inter-zone transmission. The ED model solves each daily dispatch problem individually. Essential outputs include CO2 emissions, fuel consumption, electricity flows, and cost etc.

The capacity expansion (CE) model (md_CE_\*.py) decides the optimal new installation investment in each time period step. The results will be fed to the ED model for dispatch optimization. A critical feature is the mechanism of identifying extreme cases for testing system reliability. Before the program builds the CE problem, this framework selects the most extreme supply and demand hour in each calendar month, i.e. the hour with the highest residual demand. Therefore, this model optimizes the problem with configured time-slices plus these 12 cases.

An input data processing module (impt_\*.py) is developed to convert the inputs according to the market structure, temporal resolution, time zone etc. Base year, time horizon, time periods and time-slices are configurable. The original system demand and renewable generation have to be given in hourly resolution to allow such conversion. Another data processing module (\*_output.py) is constructed to extract model solutions and convert the results into readable tables in CSV files.

![Program Flow](/img/program_flow.jpg)

### Formulation of the core models
The ED and CE models are the core components formulated into linear programming problems.
Details about the model formulation can be seen in [this document](/doc/Formulation_GEDM.pdf).

---
### Input files and settings

Most settings and assumptions are separated from the codes. Users can freely design theirs models. The tables below summarize the purpose of the inputs files used to construct the models.

**Model configuration** ([document](/doc/Doc_Input_1_model_config.md))

folder path: /Input/1_model_config/
| File | Description |
| ------ | ------ |
| 01_SolverConfig.csv | Pyomo and solver options |
| 02_ModelConfig.csv | scenario and miscellaneous model assumptions |
| 03_YearStep.csv | time period steps |
| 04_TimeSlice_ED.csv | time-slice of ED model |
| 05_TimeSlice_CE.csv | time-slice of CE model |
| 11_MarketStruct.csv | define market structure (market > country/district > zone) |
| 21_CountryAssumpGroup.csv | assign assumption groups to countries/districts |
| 22_Zone_TimeZone.csv | set the time zone for every zone |

**Commodity/fuel assumptions** ([document](/doc/Doc_Input_1_model_config.md))

folder path: /Input/2_commodity/
| File | Description |
| ------ | ------ |
| 01_MainCommodity.csv | define the main types of commodities/fuels |
| 02_CostGroup.csv | set the cost group of the fuels in each country/district |
| 03_CommodityCost.csv | set the cost projection of the fuel groups |

**Process/technology assumptions** ([document](/doc/Doc_Input_2_commodity.md))

folder path: /Input/3_process/
| File | Description |
| ------ | ------ |
| 01_MainProcess.csv | define main default processes/technologies, and their fuel type, operation mode and reserve provision |
| 02_Tech_coal.csv | set technical assumptions of coal-fired power generation technologies |
| 02_Tech_gas.csv | set technical assumptions of gas-fired power generation technologies |
| 02_Tech_other.csv | set technical assumptions of other power generation technologies |
| 02_Tech_renewable.csv | set technical assumptions of renewable power generation technologies |
| 03_Cost_coal.csv | set cost assumptions of coal-fired power generation technologies |
| 03_Cost_gas.csv | set cost assumptions of gas-fired power generation technologies |
| 03_Cost_other.csv | set cost assumptions of other power generation technologies |
| 03_Cost_renewable.csv | set cost assumptions of renewable power generation technologies |
| 04_Transmission.csv | set technical and cost assumptions of transmission links |

**Policy assumptions** ([document](/doc/Doc_Input_4_policy.md))

folder path: /Input/4_policy/
| File | Description |
| ------ | ------ |
| 01_RE_Target.csv | set renewables development target (for future model extension, currently not used) |
| 02_CarbonCost.csv | set carbon cost projections |
| 03_AncillaryService.csv | set requirements of reserve services (percentage to power demand) |
| 04_FixedNewBuild_dispatch.csv | set fixed new installation capacity of dispatchable technologies  |
| 05_CNS_EmissionLimits.csv | set the CO2 emissions limits in the CNS scenario  |
| /RE_MCP/\*.csv | set fixed renewable installation in every zones, applied in MCP scenario only |

**Existing capacity and calibration** ([document](/doc/Doc_Input_5_calibration.md))

folder path: /Input/5_calibration/
| File | Description |
| ------ | ------ |
| 01_BaseYear_Generation.csv | generation and energy flow in base year (most parameters are not used now) |
| /Zone_BaseYear_CF/\*.csv | capacity factor of dispatchable processes in every zones in base year |
| /Zone_Exist_Process/\*.csv | set the capacity and lifetime of existing processes for terrestrial zones |
| /Zone_Exist_Process_offshore/\*.csv | set the capacity and lifetime of existing processes for offshore zones |

**Transmission** ([document](/doc/Doc_Input_6_transmission.md))

folder path: /Input/6_transmission/
| File | Description |
| ------ | ------ |
| 01_Zone_conn_base.csv | set transmission links: zone connection, length, base year capacity |
| 02_Zone_conn_base_offshore.csv | set offshore links: zone connection, length, base year capacity |

**Electricity demand profile** ([document](/Doc_Input_7_demand.md))

folder path: /Input/7_demand/
| File | Description |
| ------ | ------ |
| /Zone_288/\*.csv | zonal demand profile projections in default 288 time-slices |
| /Zone_8760/\*.csv | zonal demand profile projections in hourly resolution |

**Renewables** ([document](/doc/Doc_Input_8_VRE.md))

folder path: /Input/8_VRE/
| File | Description |
| ------ | ------ |
| 01_RE_Min_Add.csv | set minimum capacity addition of renewables by time periods |
| 02_RE_Max_Add.csv | set maximum capacity addition of renewables by time periods |
| 03_RE_Hydro_Max_Cap.csv | set maximum installation capacity of hydropower |
| 04_RE_Biomass_Max_Pot.csv | set maximum supply of total biofuel (only for power plants) supply by time periods |
| /CF288/\*.csv | capacity factors of renewables in default 288 time-slices |
| /CF8760/\*.csv | capacity factors of renewables in hourly resolution |
| /DevLimit/\*.csv | maximum installation capacity of renewables by tranches |
| /SolarAreaLimit/\*.csv | maximum available areas for solar technologies deployment |

---
### Output files

The data processing module extracts solutions returned from the solver, summarizes the results, and exports as readable tables in CSV files. Every model instance has coorresponding output files. There are also files to summarize the results through the time horizon.

([document](/doc/Doc_Output.md))

/Output/(Country Code)/
| File | Description |
| ------ | ------ |
| 0_(Country_Code)_ProcessGeneration.csv | generation from every processes and transmission links through the time horizon |
| 0_(Country_Code)_Summary.csv | summary information on energy, capacity, fuel, emissions and cost through the time horizon |
| 0_(Country_Code)_ZoneBalance.csv | energy supply, demand, spill and transmission in every zones through the time horizon |
| 0_(Country_Code)_ZoneCapacity.csv | capacity by technologies and transmission links in every zones through the time horizon |
| 0_(Country_Code)_ZoneInfo.csv | summary information on fuel, emissions and costs in every zones through the time horizon |
| (Year)_ED_ProcessOutput.csv | (ED model) annual generation from every processes and transmission links |
| (Year)_ED_ZoneBalance.csv | (ED model) annual energy supply, demand, spill and transmission in every zones |
| (Year)_ED_ZoneInfo.csv | (ED model) annual summary information on fuel consumption, emissions and costs in every zones |
| (Year)_CE_ProcessOutput.csv | (CE model) annual generation from every processes and transmission links |
| (Year)_CE_ZoneBalance.csv | (CE model) annual energy supply, demand, spill and transmission in every zones |

---
### License
This package is licensed under the [MIT License](/LICENSE).

repository: chengtachu/GEDM

repo_name: chengtachu/GEDM


text:
---
author: phwl
comments: true
date: 2007-08-19 02:38:33+00:00
slug: forster
title: Forster 2007
classes: wide
categories:
- fishing
tags:
- australia
header:
  teaser: /assets/images/2007/08/p1010269.jpg
---

Forster is about a four hour drive from Sydney. Went on a very brief overnight camping/fishing trip with my good friends from Sydney University, Warren and Chris.

{% include figure image_path="/assets/images/2007/08/p1010269.jpg" caption="" %}

Didn't arrive until late in the afternoon but the conditions were almost perfect. We had packed Chris' car to the limit with gear.

{% include figure image_path="/assets/images/2007/08/p1010304.jpg" caption="" %}

Went straight to our spot.

{% include figure image_path="/assets/images/2007/08/p1010234.jpg" caption="" %}

{% include figure image_path="/assets/images/2007/08/p1010238.jpg" caption="" %}

It wasn't long before Warren caught the first tailor (note that hat, you have to be very self-confident to be seen in public in a hat like that).

{% include figure image_path="/assets/images/2007/08/p1010239.jpg" caption="" %}

Others soon followed.

{% include figure image_path="/assets/images/2007/08/p1010249.jpg" caption="" %}

{% include figure image_path="/assets/images/2007/08/p1010246.jpg" caption="" %}

[After cleaning our fish in the dark (speed was more important than accuracy in this case), it was over to the National Park where we could cook and pitch our tent in absolute darkness. It was a beautiful moonless night, the type that makes you feel very small and insignificant in the grand scheme of things. We could see so many stars that some regions looked like clusters of clouds.](/assets/images/2007/08/p1010322.jpg)

{% include figure image_path="/assets/images/2007/08/p1010255.jpg" caption="" %}

{% include figure image_path="/assets/images/2007/08/p1010265.jpg" caption="" %}

The next morning was cold but there was a perfect mirror on the water beside which we camped.

{% include figure image_path="/assets/images/2007/08/p1010271.jpg" caption="" %}

{% include figure image_path="/assets/images/2007/08/p1010278.jpg" caption="" %}

Made breakfast.

{% include figure image_path="/assets/images/2007/08/p1010272.jpg" caption="" %}

{% include figure image_path="/assets/images/2007/08/p1010275.jpg" caption="" %}

Got some cunji for bait.

{% include figure image_path="/assets/images/2007/08/p1010294.jpg" caption="" %}

{% include figure image_path="/assets/images/2007/08/p1010297.jpg" caption="" %}

and fished at the breakwall.

{% include figure image_path="/assets/images/2007/08/p1010298.jpg" caption="" %}

In the evening, we fished the rocks again. This time I used cunji and pilchard pieces. Got a blackfish (also called luderick) on my first cast.

{% include figure image_path="/assets/images/2007/08/p1010305.jpg" caption="" %}

Here is Chris with a tailor.

{% include figure image_path="/assets/images/2007/08/p1010306.jpg" caption="" %}

{% include figure image_path="/assets/images/2007/08/p1010307.jpg" caption="" %}

We also caught some bream

{% include figure image_path="/assets/images/2007/08/p1010315.jpg" caption="" %}

and here is the largest tailor we got, caught on a small spinning rod with 3kg line. The tailor jumped about 4 times and took about 10 min to land.

{% include figure image_path="/assets/images/2007/08/p1010322.jpg" caption="" %}

Ended up with a nice mixed bag, we released some of the fish but also kept some for the table.

{% include figure image_path="/assets/images/2007/08/p1010330.jpg" caption="" %}

{% include figure image_path="/assets/images/2007/08/p1010331.jpg" caption="" %}

Overall, had a great fishing trip with good friends. Life doesn't get any better.

{% include figure image_path="/assets/images/2007/08/p1010335.jpg" caption="" %}


text:
# helm-encryption

*__Calling on contributors with Python skills to help take this project further__*

## Overview
This project aims to address some limitations of packaged charts in the [helm-secrets](https://github.com/jkroepke/helm-secrets) plugin in a helm-native way. This project does not aim to replace helm-secrets, in fact, helm-secrets is a perfectly good fit for most projects. helm-encryption works by targeting keys within an existing values file and encryption them using AES encryption. Importantly, the keys are encrypted with Helm itself, thereby not requiring this utility to decrypt the secrets within. That was the aim with this tool; i.e. to remove the dependence on external tools and plugins to decrypt secrets in Helm charts when doing an install or upgrade.

## Rationale

When attempting to use helm-secrets for my project, my team ran into a few fundamental issues with the plugin:

1. Helm secrets operates on the entire values file. Secrets are extracted into their own values file and that file is then encrypted. This in principle works just fine, however, for a large values file, moving certain keys out into a separate file suffers an inherent maintainability loss. For example, if you have a large nested yaml structure with one or two secrets, moving these into their own file will either repeat that structure (sans all the other keys) or simply move the keys to be encrypted into a long list. This maintainability hit was not an option for my project.
2. Hem secrets requires a wrapper to the Helm tool. While in principle this is fine, this does always require the presence of the helm-secrets plugin
3. Because of point 1 above, helm-secrets requires at least 2 values files. Again, in principle (for most projects) this is fine. However, if you are packaging your charts and shipping them off to a container registry or chart museum, you will run into issues when trying to install these charts. Helm (and by proxy, helm-secrets) does not support multiple values files inside a packaged chart. This therefore requires you to unpack the packaged chart first and then specify the values file you want to decrypt.
4. Lastly, helm-secrets requires an external decryption routine to decrypt secrets - thus, editing secrets first requires you to decrypt the full list of secrets, then edit the secret and then re-encrypt the secret. This seemed unnecessary for our project.

## Requirements
- Helm 3.0 or higher
- Python 3.8 or higher

## Usage

This utility makes some assumptions:
1. The default values file `values.yaml` is always encrypted, i.e. your Helm chart has only 1 values file. You can change this behaviour by using the `--values` parameter
2. Keys prefixed with `encrypted` will be encrypted (or decrypted). You can change this behaviour by specifying an alternative prefix of your choice by using the `--keyprefix` parameter.
3. There is an assumption that *.dec.yaml and *.enc.yaml are added to `.gitignore` to prevent accidental checkin.

### Encrypt

Encrypt an existing value.yaml file. Encrypted all keys starting with the word `encrypted` unless an alternative prefix is specified.
The encryption key, by default, must be base64 encoded.

`./helm-encyption.py --encrypt --path <path to helm chart> --key <aesKey>`

**Example:**

```
databaseCredentials:
  encyptedUsername: mysql
  encryptedPassword: superSecret123@
 ```

`./helm-encyption.py --encrypt --path /repo/my-helm-chart --key QiZFKUhATWNRZlRqV25acjR1N3gheiVDKkYtSmFOZFI=`

Decrypted values files will be saved as `values.yaml.enc` at the path specified with `--path`

_**NOTE:** This tool will not re-encrypt previously encrypted values provided the same (correct) key is used. It is therefore possible to append a new key to be encrypted without decrypting the file first_

### Decrypt

Decrypt an existing value.yaml file. Encrypted keys starting with the word `encrypted` will be decrypted unless an alternative prefix is specified. The decryption key, by default, must be base64 encoded.

`./helm-encyption.py --decrypt --path <path to helm chart> --key <aesKey>`

Decrypted values files will be saved as `values.yaml.dec` at the path specified with `--path`

**Example:**

`./helm-encyption.py --decrypt --path /repo/my-helm-chart --key QiZFKUhATWNRZlRqV25acjR1N3gheiVDKkYtSmFOZFI=`

### Overwrite

The encryption operation generates a new file called `values.enc.yaml`. This is done so that the file can be reviewed. The existing values file may be replaced with the newly encrypted file manually or by using this overwrite command.

`./helm-encyption.py --overwrite --path <path to helm chart>`

#### Clean

A convenience function for removing all `.dec.yaml` and `.enc.yaml` files from the specified path.

`./helm-encyption.py --clean --path <path to helm chart>`

## Usage in your Helm chart

Once you have an encrypted values file, the recommended way to pass the decryption key into your chart is using a `--set` statement.

Example:

`helm upgrade my-chart/ --set aesKey=QiZFKUhATWNRZlRqV25acjR1N3gheiVDKkYtSmFOZFI=`

In your template files, the value can be decrypted as follows:

`superSecret: {{ .Values.databaseCredentials.encryptedPassword | decryptAES (.Values.aesKey | b64dec)}}`

## Project Components

- `helm-encryption.py` - the main entrypoint
- `/encrypt` - a simple Helm 3.0 chart that performs the encryption and decryption.

## Copyright and license
© 2020-2021 Christopher Parker (chris.parker.za)

Licensed under the [Apache License, Version 2.0](https://commons.apache.org/proper/commons-bsf/license.html)

repository: chris-parker-za/helm-encryption

repo_name: chris-parker-za/helm-encryption


text:
# Vuesence Book

Minimalistic Vue.js based documentation system component

![Node.js CI](https://github.com/altrusl/vuesence-book/workflows/Node.js%20CI/badge.svg)
[![Known Vulnerabilities](https://snyk.io/test/github/altrusl/vuesence-book/badge.svg)](https://snyk.io/test/github/altrusl/vuesence-book)
[![Dependabot Status](https://api.dependabot.com/badges/status?host=github&repo=altrusl/vuesence-book)](https://dependabot.com)

![VB-preview](https://altrusl.github.io/vuesence-book/images/vb-preview.gif)

# How to use

**Vuesence Book** can be used as a Vue component in a Vue.js project or stand-alone in a browser

## Node.js usage example

```bash
npm install @vuesence/book --save
```

```html
<template>
  <div id="app" class="app">
    <vuesence-book
      header-title="My Book"
      :use-router="false"
    />
  </div>
</template>

<script>
import VuesenceBook from "@vuesence/book";

export default {
  name: "App",
  components: {
    VuesenceBook
  }
};
</script>

<style>
    @import './css/default.css';
    /* @import './css/vuepress-style.css'; */
    /* @import './css/google-style.css'; */
</style>
```
## Browser usage example

The **Vuesence Book** can be run as a stand-alone web component in the browser. `Vue` and `@vuesence/book` packages should be loaded as scripts. Vue-router must be disabled.

```html
<!DOCTYPE html>
<html lang="en">

	<head>
		<title>Vuesence Book demo</title>

		<script src="https://unpkg.com/vue"></script>
		<script src="https://unpkg.com/@vuesence/book"></script>

        <link rel="stylesheet"<EMAIL_ADDRESS>
		<!-- You can plug in any custom CSS here to style the Vuesence Book-->
		<!-- <link rel="stylesheet" href="css/vuepress-style.css"> -->
		<!-- <link rel="stylesheet" href="css/google-style.css"> -->
	</head>

	<body>
		<div id="app" class="app">
			<vuesence-book
				header-title="My Book"
				:use-router="false"
			/>
		</div>
	</body>

	<script>
		new Vue({ el: '#app' })
	</script>

</html>
```
# Features

- Simple, responsive, lightweight, fast
- Easily and fully customizable with CSS
- Very suitable for embedding help/documentation content into the website
- Easy to use content management
- Focus on performance - articles can be preloaded
- Unlimited nesting levels for TOC tree menu
- In general no dependencies (optional `marked` or other MD library for markdown support and `vue-router` for routing)

# Documentation

Detailed documentation can be found here - https://altrusl.github.io/vuesence-book/

## Content management

Unlike many other documentation systems **Vuesence Book** is not a *static site generator*. You can change the content without rebuilding the entire website.

`vbcfg.json` configuration file defines the content tree (TOC in the left column). It also contains links to articles in HTML or Markdown formats or the content of the article itself in an HTML format

```
└── vb
    ├── pages
    │   ├── intro.md
    │   ├── desc.md
    │   └── rich-content.html
    ├── vbcfg.json
    └── index.html
```

Self-explaining example of the `vbcfg.json` and articles of the documentation can be found here -
https://github.com/altrusl/vuesence-book/tree/master/docs

## Playground

Try it on codesandbox.io (Node.js version)

> ! The version on `codesandbox.io` might be slightly out of date

![sbio](https://altrusl.github.io/vuesence-book/images/csbio.jpg)

## Check out my other Vue.js components

- Sliding header - Vue.js component representing sliding header (or two different headers)
- Cloud Sync Button - a button with cloud synchronization animation
- Modal Window - simple lightweight Modal Window Vue.js component

You like Vuesence Book? Star it and retweet it!

-------

## Troubleshooting

Any bugs, issues, feature and pull requests are welcome

Please use GitHub's issue reporter or send me an <a<EMAIL_ADDRESS>

## Contribution

Contribution is always welcome and recommended. Here is how:

- Fork the repository
- Clone to your machine
- Make your changes
- Create a pull request

## Areas for improvements / involvement
- More convenient fenced code block highlighting
- Full-text search
- CSS layout improvement (responsiveness)
- Themes
- Top nav bar
- Footer for stand-alone version
- CEO issue

## Change Log

- 0.4.2 small fixes on menu
- 0.4.50 release

## License

**@vuesence/book** package is freely distributable under the terms of the [MIT license](LICENSE).


text:
# Smart Discovery Protocol (SDP)

Smart Discovery Protocol is a service discovery protocol which carries the information of the service, the device and the owner.
The information is represented and queried in ontologies like OWL and SPARQL.

The protocol can be considered as an addon to the Neighbor Discovery Protocol of IPv6  standard. Two new messages are defined,
Smart Solicitation and Smart Advertisement. These messages are ICMPv6 messages and they carry semantic information in their
payloads.

The type-length-value (TLV) options of the ICMPv6 standard are used to carry the simple URL representations of the services
in legacy standards. For the time being only TLV for SLP is added.

This is the proof of concept implementation of the protocol in RUBY. Not in production quality.
There is no event machine integration or any other performance improvement.
RAW sockets are used therefore require root permission.

For SLP integration, i needed to use parsing and matching methods for SLP URLs.
In order to show that existing APIs can be used, I have placed jSLP jar (RC3) (not uploaded) and call it as a separate process.
If you download jSLP and put it in the bin directory it works. But it was just for demonstration.

## Installation

Although I have implemented SDP  as a gem I have never tried to use it as a gem.

What I do is:

1. Download the source, git clone ...
2. use ruby >1.9.3, (advised to use RVM) other libraries including jruby does not have FULL support for ICMPv6
3. bundle install, for dependencies.
		Among the dependencies 'racket' plugin is directed to the one that I have changed.
		You may need to create its gem file by yourself. To do that also download it (my version) run rake build.
		Then gem install LOCAL_PATH/racket/racket...gem
4.  Well thats it you have installed it

## Usage

As I said I have never tried it as a gem. Instead I have created thor scripts.
Under the bin directory, you can find explorer.rb.
If you run 'ruby bin/explorer.rb' it explains you the usage. I hope you can understand whats going on from there.

## Contributing

So what is the point of making this code open?

repository: yunus/SDP

repo_name: yunus/SDP


text:
### Examples Q&A:
Q: How to run the examples?
A: Please include all files in the input directory to generate both the ERT_generator and energy calculator.
```
cd examples/hierarchy_v0.2
accelergy *.yaml components/*.yaml -o ../output --enable_flattened_arch 1
```

Q: How do see the interactions between Accelergy and the plug-ins?
A: Turn on the ```-v``` flag will allow the requests and returned energy to be printed

Q: How do interpret the data in ```reference```?
A: All data in ```reference``` files are generated using the default dummy plug-in, which returns 1 for all non-idle actions
   and returns 0 for all idle actions, regardless of the component class and attributes.


text:
In this step, we'll preform the following steps to get Conjur up & running.

1. Create a namespace for Conjur
2. Setting up Helm
3. Create a Conjur Account
4. Setting up FQDN

## Namespace

To create a namespace for Conjur, execute:
```
kubectl create namespace conjur-server
```{{execute HOST1}}

## Helm

Let's install Conjur using Helm.

First, we need to add `cyberark` repo to Helm
```
helm repo add cyberark https://cyberark.github.io/helm-charts
```{{execute HOST1}}

The response should look like this:
```
"cyberark" has been added to your repositories
```
Try the above command again if it shows something else

Then we'll update the repo
```
helm repo update
```{{execute HOST1}}

```
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "cyberark" chart repository
Update Complete. ⎈ Happy Helming!⎈
```
Try the above command again if it shows something else

Next, we need to add a role binding

```
kubectl apply -f conjur/role-binding.yaml
```{{execute HOST1}}
```
rolebinding.rbac.authorization.k8s.io/conjur-server-authn-role-binding created
```

Lastly, we can install Conjur by the following command.
Please note that it'll take a short while.

```
helm install conjur-cluster cyberark/conjur-oss \
     --set ssl.hostname=conjur.demo.com,dataKey="$(docker run --rm cyberark/conjur data-key generate)",authenticators="authn-k8s/dev" \
     --set postgres.persistentVolume.create=false \
     --set service.external.enabled=false \
     --namespace conjur-server
```{{execute HOST1}}

The system should return a long message showing how to proceed.

## Conjur Account

We'll need to get the pod name of Conjur.
```
 export POD_NAME=$(kubectl get pods --namespace conjur-server \
   -l "app=conjur-oss,release=conjur-cluster" \
   -o jsonpath="{.items[0].metadata.name}")
```{{execute HOST1}}

if the following error occurs, please wait for 10 seconds and try the above command again
```
error: unable to upgrade connection: container not found ("conjur-oss")
```

...in order to create an account, let's call it `default`

```
kubectl exec --namespace conjur-server \
    $POD_NAME \
  --container=conjur-oss \
  -- conjurctl account create "default" | tee admin.out
```{{execute HOST1}}

If `error: unable to upgrade connection: container not found ("conjur-oss")` is returned, don't worry!
It just means we're faster than the computer! 🤣
Wait for a moment and try again.

It should generate unique public key & API key for admin, something simiiar to the one below.
```
Created new account 'default'
Token-Signing Public Key: -----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA0oDXDvz2QmdQYEJloDCD
k5Gd5ipLVsNf41V6Gya28fG4WcPVGhmBUlDFv8uXICBYC+8JBurcY+WF7bMWY/Oz
1xBnvLyrVvHd629bgQboMODlNZbdQppeV+m2ahBAMTahMnzYE+0YhE9ElIUoywyk
GSDnkq8z/bKMqWNSgmFMrqeR9v7bTZBHNvHGQnbFxD5Y3mE3nNErmQctl11KfyC/
cRIKGJCMRFaxQBvy8UppYKCcArrlry8qRM1VbB4v22BIpSoQI4vZZhObxmWTsOPV
hZeh/T8h8gvW8djShqiW+1I7BVS5gYI9nnMTdk/aVE91LGiCOwO2s0gpTRuv17fO
hwIDAQAB
-----END PUBLIC KEY-----
API key for admin: 54myrp3dgvwsc1x3mgpn1v8hksn33qxcnm3tt7tecryncqg2th4cdq
```

Please note that we've stored the public key and admin API key in `admin.out` file for demo.
For your own environment, please keep them safe & secure.

## FQDN

Let's give our setup a Fully qualified domain name (FQDN) - `conjur.demo.com`
Typically we will configure it in the DNS system.
This time, for demo purpose, we'll simply update the `/etc/hosts` file

```
export CONJUR_URL=$(kubectl describe svc conjur-cluster-conjur-oss -n conjur-server |grep Endpoints | awk '{print $2}')
export SERVICE_IP=$(echo $CONJUR_URL | awk  -F ':' '{print $1}')
echo $SERVICE_IP
```{{execute HOST1}}

An IP address should be displayed.   If not, that means the service is starting.
please wait for a moment and repeat the above step again.

Once the service IP is ready, let's add it to the `/etc/hosts` file

```
echo "$SERVICE_IP conjur.demo.com" >> /etc/hosts
```{{execute HOST1}}

Great! Conjur is now up & running.
Let's setup the Conjur client

repository: cyberark/katacoda-scenarios

gha_language: Shell

repo_name: cyberark/katacoda-scenarios


text:
# Polygon

## Company blurb

Polygon provides the core components and tools to join the new, borderless economy and society. We envision an open, borderless world. A world in which people and machines collaborate and exchange value globally and freely, without gatekeepers or intermediaries. A world in which communities thrive, unconstrained by artificial borders and archaic regulations.

## Company size

5

## Remote status

We don’t believe in traditional companies, hierarchy and management. Anyone is welcome to contribute code, ideas or anything else that can help make our vision a reality!

## Region

**Worldwide**

## Company technologies

JavaScript, HTML5, CSS3, Postgres, MySQL, Mongodb, ERC20, ERC 721

## Office locations

Mumbai, Maharashtra, India

## How to apply

https://polygon.technology/careers/#all-roles

repository: remoteintech/remote-jobs

repo_name: remoteintech/remote-jobs


text:
# Aruba Cloud Driver for Docker Machine

## Table of Contents
* [Overview](#overview)
* [Requirements](#requirements)
* [Installation](#installation)
  * [From a Release](#from-a-release)
  * [From Source](#from-source)
* [Usage](#usage)
  * [Available Options](#available-options)
  * [Examples](#examples)
* [License](#license)

## Overview

The Aruba Cloud Driver is a plugin for Docker Machine which allows you to automate the provisioning of Docker hosts on Aruba Cloud Servers. The plugin is based on the [Go Aruba Cloud SDK](https://github.com/Arubacloud/goarubacloud) and [Cloud API](http://kb.cloud.it/en/api.aspx).

To acquire Aruba Cloud Cloud API credentials visit https://www.cloud.it.

## Requirements

  * [Docker Machine](https://docs.docker.com/machine/install-machine/) 0.9.0 or a newer version

Windows and Mac OS X users may install [Docker Toolbox](https://www.docker.com/products/docker-toolbox) package that includes the latest version of the Docker Machine.

## Installation

### From a Release

The latest version of the `docker-machine-driver-arubacloud` binary is available on the [GithHub Releases](https://github.com/Arubacloud/docker-machine-driver-arubacloud/releases) page.
Download the `tar` archive and extract it into a directory residing in your PATH. Select the binary that corresponds to your OS and according to the file name prefix:

* Linux: docker-machine-driver-arubacloud-linux
* Windows: docker-machine-driver-arubacloud-windows

To extract and install the binary, Linux and Mac users can use the Terminal and the following commands:

```bash
sudo tar -C /usr/local/bin -xvzf docker-machine-driver-arubacloud*.tar.gz
```

If required, modify the permissions to make the plugin executable:

```bash
sudo chmod +x /usr/local/bin/docker-machine-driver-arubacloud
```

Windows users may run the above commands without `sudo` in Docker Quickstart Terminal that is installed with [Docker Toolbox](https://www.docker.com/products/docker-toolbox).

Otherwise Windows users can extract the zip manually and add the plugin folder to the PATH environment variable.

### From Source

Make sure you have installed [Go](http://www.golang.org) and configured [GOPATH](http://golang.org/doc/code.html#GOPATH) properly.

To download the repository and build the driver run the following:

```bash
go get -d -u github.com/Arubacloud/docker-machine-driver-arubacloud
cd $GOPATH/src/github.com/Arubacloud/docker-machine-driver-arubacloud
make build
```

To use the driver run:

```bash
make install
```

This command will install the driver into `/usr/local/bin`.

Otherwise, set your PATH environment variable correctly. For example:

```bash
export PATH=$GOPATH/src/github.com/Arubacloud/docker-machine-driver-arubacloud/bin:$PATH
```

If you are running Windows, you may also need to install GNU Make, Bash shell and a few other Bash utilities available with [Cygwin](https://www.cygwin.com).

## Usage

You may want to refer to the Docker Machine [official documentation](https://docs.docker.com/machine/) before using the driver.

Verify that Docker Machine can see the Aruba Cloud driver:

```bash
docker-machine create -d arubacloud --help
```

### Available Options

  * `--ac_username`:  Aruba Cloud username.
  * `--ac_password`: Aruba Cloud password.
  * `--ac_admin_password`: Virtual machine admin password.
  * `--ac_endpoint`: Aruba Cloud Data Center (dc1,dc2,dc3,etc.).
  * `--ac_template`: Virtual machine template.
  * `--ac_size`: Size of the virtual machine.
  * `--ac_action`: Type of action.
  * `--ac_ip`: Specify an IP already purchased on Aruba Cloud (Valid only if ac_action="NewPro").
  * `--ac_ssh_key`: Path of private key file. (Both private and public key must be in the same folder).

|          CLI Option             |Default Value 	| Environment Variable           | Required |
| --------------------------------|--------------------| ------------------------------ | -------- |
| `--ac_username`	          |			     | `AC_USERNAME`            		| yes      |
| `--ac_password`       	   |		    | `AC_PASSWORD`         		 	| yes      |
| `--ac_admin_password`        	   |			| `AC_ADMIN_PASSWORD`            | yes      |
| `--ac_endpoint`                  |`dc1`	| `AC_ENDPOINT`               	| yes      |
| `--ac_template`         	|`ubuntu1604_x64_1_0`	   | `AC_TEMPLATE`         		 	| yes      |
| `--ac_size`    		|`Large`   		   | `AC_SIZE`       				| yes      |
| `--ac_action`    		|`NewSmart`   		   | `AC_ACTION`       				| yes      |
| `--ac_ip`    		|   		   | `AC_IP`       				| no      |

Valid values for `--ac_size` are `Small`, `Medium`, `Large`, `Extra Large`.

####Size mapping for Pro machines

|          Size             |CPU 	| Ram (GB)          | Disk Size (GB) |
| --------------------------------|--------------------| ------------------------------ | -------- |
| `Small`	          |1			     | 1           		| 20      |
| `Medium`       	   |1		    | 2         		 	| 40      |
| `Large`        	   |2			| 4            | 80      |
| `Extra Large`                  |4	| 8              	| 160      |

Valid values for `--ac_action` are `NewSmart`, `NewPro`, `Attach`.

Available parameters for `--ac_endpoint` are shown in the next table.

| Parameter |                 Data Center Location                 |
|-----------|------------------------------------------------------|
| `dc1`     | Italy 1                                              |
| `dc2`     | Italy 2                                              |
| `dc3`     | Czech republic 									   |
| `dc4`     | France                                               |
| `dc5`     | Deutschland                             			   |
| `dc6`     | United Kingdom                            		|
| `dc7`     | Italy 3                             		|
| `dc8`     | Poland                           		|

Supported values for `--ac_template` are listed below.

|              Parameter                |					OS						|
|---------------------------------------|-------------------------------------------|
| `debian8_x64_1_0`                     | `Debian 8 64bit`							|
| `ubuntu1604_x64_1_0`                	| `Ubuntu Server 16.04 LTS 64bit`			|
| `centos7_x64_1_0`                     | `CentOS 7.x 64bit`						|

### Examples

#### Create using defaults:

```
docker-machine create --driver arubacloud \
 --ac_username		              "ARU-XXXX" \
 --ac_password			          "xxxxxxx" \
 --ac_admin_password		      "yyyyyyyy" \
MyDockerHostName
```

#### Create a Smart machine specifying template, endpoint and size:

```
docker-machine create --driver arubacloud \
 --ac_username		              "ARU-XXXX" \
 --ac_password			          "xxxxxxx" \
 --ac_endpoint			          "dc1" \
 --ac_template	                  "ubuntu1404_x64_1_0" \
 --ac_size				          "Large" \
 --ac_admin_password		      "yyyyyyyy" \
 --ac_action		      		  "NewSmart" \
MyDockerHostName
```
#### Create a Pro machine specifying template, endpoint and size:

If you don't specify an IP Address a new one will be automatically purchased.

```
docker-machine create --driver arubacloud \
 --ac_username		              "ARU-XXXX" \
 --ac_password			          "xxxxxxx" \
 --ac_endpoint			          "dc1" \
 --ac_template	                  "ubuntu1404_x64_1_0" \
 --ac_size				          "Large" \
 --ac_admin_password		      "yyyyyyyy" \
 --ac_action		      		  "NewPro" \
MyDockerHostName
```

#### Create a Pro machine specifying template, endpoint, size and IPAddress:

```
docker-machine create --driver arubacloud \
 --ac_username		              "ARU-XXXX" \
 --ac_password			          "xxxxxxx" \
 --ac_endpoint			          "dc1" \
 --ac_template	                  "ubuntu1404_x64_1_0" \
 --ac_size				          "Large" \
 --ac_admin_password		      "yyyyyyyy" \
 --ac_action		      		  "NewPro" \
 --ac_ip		      			  "xx.xx.xx.xx" \
MyDockerHostName
```
#### Attach to an existing machine:

Note:
- In order to attach to an existing machine, this machine must have been created with an SSH key.
- You must put private and public key in the same folder on the machine where docker-machine commands will be launched, and pass this folder as an argument (Eg. --ac_ssh_key="private_key_path").

Dashboard url is different depending on the selected endpoint:

|[DC1](https://admin.dc1.computing.cloud.it/Login.aspx)	    |[DC2](https://admin.dc2.computing.cloud.it/Login.aspx)	    |[DC3](https://admin.dc3.computing.cloud.it/Login.aspx)	    |[DC4](https://admin.dc4.computing.cloud.it/Login.aspx)	    |[DC5](https://admin.dc5.computing.cloud.it/Login.aspx)	    |[DC6](https://admin.dc6.computing.cloud.it/Login.aspx)	    |
|-----------------------------------------------------------|-----------------------------------------------------------|-----------------------------------------------------------|-----------------------------------------------------------|-----------------------------------------------------------|-----------------------------------------------------------|

![alt tag](img/cloudpanel.PNG)

####View instance list

```
docker-machine ls
NAME    			ACTIVE   	DRIVER       STATE   	URL   				SWARM   	DOCKER    ERRORS
MyDockerHostName   	*        	arubacloud   Running    tcp://<IP_ADDRESS>             	v1.10.0
default   			-       	arubacloud   Running    tcp://<IP_ADDRESS>             	v1.10.0

```

#### Get the IP address

```
docker-machine ip MyDockerHostName
<IP_ADDRESS>

```

#### Verify Docker Engine

Verify Docker Engine is installed correctly by running docker commands.

Start with something basic like docker run hello-world, or for a more interesting test, run a Dockerized webserver on your new remote machine.

In this example, the -p option is used to expose port 80 from the nginx container and make it accessible on port 8000 of the host.

```
$ docker run -d -p 8000:80 --name webserver kitematic/hello-world-nginx
 Unable to find image 'kitematic/hello-world-nginx:latest' locally
 latest: Pulling from kitematic/hello-world-nginx
 a285d7f063ea: Pull complete
 2d7baf27389b: Pull complete
 ...
 Digest: sha256:ec0ca6dcb034916784c988b4f2432716e2e92b995ac606e080c7a54b52b87066
 Status: Downloaded newer image for kitematic/hello-world-nginx:latest
 942dfb4a0eaae75bf26c9785ade4ff47ceb2ec2a152be82b9d7960e8b5777e65

```

In a web browser, go to http://<host_ip>:8000 to bring up the webserver home page. You got the <host_ip> from the output of the docker-machine ip <machine> command you ran in a previous step. Use the port you exposed in the docker run command.

![alt tag](img/nginx-webserver.png)

#### Stop and remove an instance

```
docker-machine stop MyDockerHostName
docker-machine rm MyDockerHostName
About to remove MyDockerHostName
WARNING: This action will delete both local reference and remote instance.
Are you sure? (y/n): y
Successfully removed MyDockerHostName

```

## License

This code is released under the Apache 2.0 License.

Copyright (c) 2017 Aruba Cloud


text:
# mylist

MEANjs application to keep track of my myanimelist anime/manga lists, provide statistics on those lists, and various other bits & bobs.

In my opinion, would provide little value to others, with the exception of some interesting code snippets.

 change history

cero-23 (2.3.1)
- removed replace attribute from mal-search directive.
  - unconfirmed to be the culprit for the long load times.
  - angular 2 doesn't use replace so it's probably a "better practice"

cero-23 (2.3.0)
- added float-label css, starting to apply it throughout the application
- added loading spinner to malSearch directive.
  - involes creating new styling size for loading-spinner, 'control'
- changed malSearch to use ng-change and ng-model-options instead of $watch and $timeout
- perform updateOnMal when addOnMal fails
  - typically fails as already on list as a 'plan to watch'
- added mal icon to lists when item has mal id.
- added ng-min to update manga popup for linked tasks.
- made footer hr not touch the sides.

cero-22 (2.2.0)
- added support for adding/updating/removing anime and manga on MyAnimeList
- added version as a variable sourced from the package.json
  - will allow for 'single location of truth'

cero-13 (2.1.3)
- babel-polyfill re-added to server.js, yes, it was required.
- on create anime/manga return to list page.
- added mal links to manga pages.

cero-10 (2.1.0)
- reworked manga pages to be structured the same as anime.
- added mal search to manga pages

cero-00 (2.0.0)
- integrated mal search feature using popura submodule.

bankai-02 (1.28.2)
- fixed spinner css to appear above all content.
- fixed unsorted linked task dropdown.
- changed linked manga final ch/vol input fields to text, as when number arrows are always visible.

bankai-00 (1.28.0)
- added episode rating popup to anime linked task.
- made final episode input appear in create anime.
- added loading spinner to edit anime.
- shrunk linked task dialogs to md.

arrancar-24 (1.27.24)
- turn off autocompletes on forms
- altered page min-height to have footer permanently off screen without scrolling.

arrancar-23 (1.27.23)
- fix create anime 'in season?' bug.

arrancar-22 (1.27.22)
- fixed mangaitems maximum chapters / volumes bug.
- fix create anime image bug.

arrancar-21 (1.27.21)
- fixed bug of videoless anime appearing in watch list.
  - Added a check after watch-list query:
    > if the anime title, in folder name syntax, is NOT in the list of directories
      then the item should be spliced.

arrancar-20 (1.27.20)
- created rewatch/reread tracking features.
  - Added revisits property to animeitems.meta (mangaitems will require if I want to track re-reads).
  - Added item revisit function to item service.
  - Call item revisit function went updating an animeitem (mangaitem will require this if I want to track re-reads).
  - Added revisits to item history modal dialog.

- Fixed task update manga bug of being able to go beyond the final chapter. (set ng-max);

arrancar-18 (1.27.18)

repository: bakuzan/mylist

repo_name: bakuzan/mylist


text:
---
hrs_structure:
  division: '2'
  volume: '11'
  title: '26'
  chapter: '487'
  section: 487-13
type: hrs_section
tags:
  - Business
menu:
  hrs:
    identifier: HRS_0487-0013
    parent: HRS0487
    name: Chapter 487 Consumer Protection
weight: 61060
title: Consumer Protection
full_title: Chapter 487 Consumer Protection
---
**§487-13 Unlicensed acts; penalties.** (a) Any person who furnishes commodities or services for which a license, registration, or certificate is required from the department of commerce and consumer affairs or any board or commission thereunder without having such license, registration, or certificate is engaged in an unlawful act or practice and shall be subject to the penalty provided in subsection (b).

(b) Any person who engages in an unlawful act or practice as provided in subsection (a) shall be fined by a sum not less than $500 nor more than $2,500 for each unlawful act or practice, which sum shall be collected in a civil suit brought by the office of consumer protection or the department of commerce and consumer affairs.

(c) Any contract for the furnishing of commodities or services by an unlicensed, unregistered, or uncertificated person shall be void and shall prevent such person from recovering the contract price or the reasonable value thereof. [L 1974, c 120, §1; am L 1977, c 18, §1; am L 1982, c 204, §4; am L 1988, c 101, §1; am L 2008, c 19, §67]

repository: SamMade/BlueAndKrabby

repo_name: SamMade/BlueAndKrabby


text:
Take hold of the stalk and thrash it on the floor. *But, don't stop there!*

- Pick up the grain, and grind out a fine powder.
- Mix with baking soda, baking powder, and add a little water.
- Bake at 350° for 8 – 10 minutes.

And, by golly you have a biscuit.

Guys, don't think you have been simply given a plant. You have been given an opportunity.

*Now, go make a biscuit with your day!*

repository: mwender/workdayinspire

repo_name: mwender/workdayinspire


text:
---
title: Goal Setting for Developers
---

I'll be writing a lot about my personal process for goal setting. Here is an index of those posts and the logical order to read through the exercises.

## Table of Contents

### Introduction

  * Are you achieving your most important goals?
  * How to win at goal setting with the 80/20 rule

### Pick

How do you decide which goals to pursue? Here's how you brainstorm and refine a list of your most impactful goals.

  * Your life in balance: How to choose your focus?
  * Daring to dream about your ideal life
  * The haystack: How to brainstorm potential goals
  * How to uncover your hidden goals
  * What's the one thing that will make everything easier?

### Plan

Once you know what you want to achieve, what is the best way to clarify that goal into an actionable plan?

  * Goldilocks Goals: How to know if it's just right?
  * How will you know you're successful?
  * Do you know why you want this?
  * Why are you going to fail?
  * Are you looking at the whole picture?
  * How can you make success inevitable?

### Perform

Don't just let your plan collect dust! Here's how to make sure you follow through on your new plans.

  * If you want to succeed, you need to schedule success
  * Are you on track? How to create a success dashboard
  * How to love failure: A lesson in reframing

repository: andrewshell/blog-asdo-gatsby

repo_name: andrewshell/blog-asdo-gatsby


text:
sa_falco
========

[![Build Status](https://travis-ci.com/softasap/sa_falco.svg?branch=master)](https://travis-ci.com/softasap/sa_falco)
[![Build Status](https://github.com/softasap/sa_borg_server/workflows/CI/badge.svg?event=push)](https://github.com/softasap/sa_borg_server/actions?query=workflow%3ACI)
![Version on Galaxy](https://img.shields.io/badge/dynamic/json?style=flat&label=galaxy&prefix=v&url=https://galaxy.ansible.com/api/v1/roles/softasap/sa-docker/&query=latest_version.version)

Example of usage:

Simple

```YAML

     - {
         role: "sa_falco"
       }

```

Advanced

```YAML

roles:
     - {
         role: "sa_falco"
       }

```

Usage with ansible galaxy workflow
----------------------------------

If you installed the `sa_falco` role using the command

`
   ansible-galaxy install softasap.sa_falco
`

the role will be available in the folder `softasap.sa_falco`
Please adjust the path accordingly.

```YAML

     - {
         role: "softasap.sa_falco"
       }

```

##Checkers:

https://github.com/adrienverge/yamllint
yamllint -c yamllint.yml

ansible-lint

Copyright and license
---------------------

Code is dual licensed under the [BSD 3 clause] (https://opensource.org/licenses/BSD-3-Clause) and the [MIT License] (http://opensource.org/licenses/MIT). Choose the one that suits you best.

Reach us:

Subscribe for roles updates at [FB] (https://www.facebook.com/SoftAsap/)

Join gitter discussion channel at [Gitter](https://gitter.im/softasap)

Discover other roles at  http://www.softasap.com/roles/registry_generated.html

visit our blog at http://www.softasap.com/blog/archive.html

repository: softasap/sa-A-sample

repo_name: softasap/sa-A-sample


text:
# GCN for Node Classification

This is an example of using the Graph Convolutional network (GCN) algorithm [1] for semi-supervised node classification
in a homogeneous network.

## Requirements
All examples use Python 3.6 and the StellarGraph library. To install the StellarGraph library
follow the instructions at: https://github.com/stellargraph/stellargraph

Additional requirements are Pandas, Numpy and Scikit-Learn which are installed as depdendencies
of the StellarGraph library.

## CORA dataset

Currently the examples in this directory are tested on the CORA dataset. The GCN model assumes that node
features are available for all nodes in the graph.

The dataset can be downloaded from https://linqs-data.soe.ucsc.edu/public/lbc/cora.tgz

The following is the description of the dataset:
> The Cora dataset consists of 2708 scientific publications classified into one of seven classes.
> The citation network consists of 5429 links. Each publication in the dataset is described by a
> 0/1-valued word vector indicating the absence/presence of the corresponding word from the dictionary.
> The dictionary consists of 1433 unique words. The README file in the dataset provides more details.

Download and unzip the [cora.tgz](https://linqs-data.soe.ucsc.edu/public/lbc/cora.tgz) file to a location on your computer and pass this location
as a command line argument to this script.

## Running the script

The example script can be run on supplying the location of the downloaded CORA dataset
with the following command:
```
python gcn-cora-example.py -l <path_to_cora_dataset>
```

Additional arguments can be specified that change the GCN model architecture and training parameters, a
description of these arguments is displayed using the help option to the script:
```
python gcn-cora-example.py --help
```

## References

[1]	Semi-Supervised Classification with Graph Convolutional Networks. T. Kipf, M. Welling.
ICLR 2017. arXiv:1609.02907 ([link](https://arxiv.org/abs/1609.02907))

repository: JunhaoWang/stellargraph

repo_name: JunhaoWang/stellargraph


text:
# Change Log

All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.

## [0.5.3](https://github.com/tenseijs/tensei/compare/v0.5.2...v0.5.3) (2020-12-26)

### Bug Fixes

* **cms:** fix color issues in cms components ([e14cd5b](https://github.com/tenseijs/tensei/commit/e14cd5bc3487fdceca06da70a5506d52f5711ed1))

# [0.5.0](https://github.com/tenseijs/tensei/compare/v0.4.4...v0.5.0) (2020-12-17)

### Bug Fixes

* **cms:** fix border radius on cms components ([9fe4aaa](https://github.com/tenseijs/tensei/commit/9fe4aaa7c684b98923d158cea314b5000ebd5c0c))
* **core:** fix all package licesnes ([92ddc6a](https://github.com/tenseijs/tensei/commit/92ddc6a7ef0fa2e1397336147ec674974d89c1a8))

### Features

* add new components for cms ([db26903](https://github.com/tenseijs/tensei/commit/db26903d950acb69658e71f6dbd0c5d2f7264854))

## [0.4.4](https://github.com/tenseijs/tensei/compare/v0.4.3...v0.4.4) (2020-12-07)

### Features

* **cms:** add cms dashboard ([8584bc1](https://github.com/tenseijs/tensei/commit/8584bc137f0cd6e69e807baf59689d37c371ab10))

repository: sambasallah/tensei

repo_name: sambasallah/tensei


text:
# lazy-json [![Build Status](https://img.shields.io/circleci/project/egoist/lazy-json/master.svg?style=flat-square)](https://circleci.com/gh/egoist/lazy-json/tree/master)

lazy-json makes it happen to require JSON file with comments in it.

It's awesome for `lazy-json` to work with Atom plugin [`language-json-comments`](https://atom.io/packages/language-json-comments)!

## Install

```bash
npm install --save lazy-json
```

## Example

```javascript
// test.lazy.json
{
  // here is my comments
  "github": "oriented programming"
  /* things all done
  ** go home
  */
}

// usage
import { requireJSON } from 'lazy-json'
const obj = requireJSON('./test.lazy')
console.log(obj.github)

// it's not ended with `.json`
const obj = requireJSON('./.babelrc', true)
```

## Related

- [strip-json-comments](https://github.com/sindresorhus/strip-json-comments) by [@sindresorhus](https://github.com/sindresorhus)

## License

MIT © [EGOIST](https://github.com/egoist)

repository: egoist/lazy-json

repo_name: egoist/lazy-json


text:
# PDF-To-Image
Convert PDF files to PNG Files.

### About
A simple app built with React.js and electron.

### Instructions
1. Open PDF-To-Image.
2. Drag a pdf file over the dropzon until it turns green.
3. Drop the pdf file and a .png file of the same name will be created in the same folder as the dropped .pdf file.

### Bugs
Currently the windows version is experienceing issues with GraphicsMagick for Node.
One can drag .pdf's on to convert them and it will try to but you will get a png with a file size of 0.
for now the windows version is more of a UI showcase then anything else.

repository: afridley/PDF-To-Image

repo_name: afridley/PDF-To-Image


text:
# Hypercord

Hypercord is a cool new store-driven Discord client modification.

## [Hypercord Server](https://discord.gg/qP3UvsAf2G)

---

## [Install Guide](https://github.com/wispykai/hypercord/wiki/Install-Guide)

## [FAQ](https://github.com/wispykai/hypercord/wiki/FAQ)

repository: wispykai/hypercord

repo_name: wispykai/hypercord


text:
dbaccess
===

To build an assembly, run:

    mvn clean assembly:assembly

This creates a jar at:

    target/dbaccess-0.1.0-SNAPSHOT.jar

which can be placed in:

    $BROOKLYN_HOME/lib/dropins

### Opening in an IDE

To open this project in an IDE, you will need maven support enabled
(e.g. with the relevant plugin).  You should then be able to develop
it and run it as usual.  For more information on IDE support, visit:

    https://brooklyn.incubator.apache.org/v/latest/dev/env/ide/

### Customizing the Assembly

The artifacts (directory and tar.gz by default) which get built into
`target/` can be changed.  Simply edit the relevant files under
`src/main/assembly`.

### More About Apache Brooklyn

Apache Brooklyn is a code library and framework for managing applications in a
cloud-first dev-ops-y way.  It has been used to create this sample project
which shows how to define an application and entities for Brooklyn.

This project can be extended for more complex topologies and more
interesting applications, and to develop the policies to scale or tune the
deployment depending on what the application needs.

For more information consider:

* Visiting the Apache Brooklyn home page at https://brooklyn.incubator.apache.org
* Finding us on IRC #brooklyncentral or email (click "Community" at the site above)
* Forking the project at  http://github.com/apache/incubator-brooklyn/

A sample Brooklyn project should specify its license.

repository: ivanayov/dbaccess-entity

repo_name: ivanayov/dbaccess-entity


text:
![img](https://raw.githubusercontent.com/rogerknl/fontFaceRecon_frontEnd/develop/public/logo.png)

> AutoFontStyleGuideFontFace is an application which provides you with all fonts of a specific url.

## Screenshots

![img](https://raw.githubusercontent.com/rogerknl/fontFaceRecon_frontEnd/develop/public/screen.png)

## Try it

This application has been already deployed, you can try it [here](http://font.knl.cat).

## Getting started

1. Clone the repo

```
git clone https://github.com/rogerknl/fontFaceRecon_backEnd.git
cd fontFaceRecon_backEnd
```

2. Install dependencies

```
npm install
```

3. Start development server

```
npm run dev
```

##### *** This app requires its frontEnd you can find it [here](https://github.com/rogerknl/fontFaceRecon_frontEnd) ***

## Built with

- [KOA ](https://koajs.com/) - Framework for node js
- [Redis](https://redis.io/) - ElasticCache
- [Puppeteer](https://www.npmjs.com/package/puppeteer) - Library which provides a high-level API to control Chromium over the [DevTools Protocol](https://chromedevtools.github.io/devtools-protocol/)

## Contributing

Improvements are welcome

Fork the repo and do your thing. Push to your fork and submit a pull request.

## Author

Roger Canela - [Github](https://github.com/rogerknl) - [LinkedIn](https://www.linkedin.com/in/roger-canela-2a085826/)

## License

This project is licensed under the MIT License.


text:
# [Array.diff](https://www.codewars.com/kata/array-dot-diff "https://www.codewars.com/kata/523f5d21c841566fde000009")

Your goal in this kata is to implement a difference function, which subtracts one list from another
and returns the result. It should remove all values from list `a`, which are present in list `b`
keeping their order.

```
Kata.arrayDiff(new int[] {1, 2}, new int[] {1}) => new int[] {2}
```

If a value is present in `b`, all of its occurrences must be removed from the other:

```
Kata.arrayDiff(new int[] {1, 2, 2, 2, 3}, new int[] {2}) => new int[] {1, 3}
```

repository: ParanoidUser/codewars-handbook

gha_language: Java

repo_name: ParanoidUser/codewars-handbook