text:
# Sentiment Analysis with Neural Networks
---
> Create a Neural Network deep learning model to classify the sentiment of StockTwits messages.

## Overview
---
In the **Sentiment Analysis with Neural Networks** project a deep learning model is built to classify the sentiment of messages from [StockTwits](https://stocktwits.com/), a social network for investors and traders. The model will be able to predict if any particular message is positive or negative. From this, a signal of the public sentiment for various ticker symbols will be generated. The training and testing data is packed in [twits.rar](https://github.com/kollepas/ai_for_trading/blob/master/project_06_-_sentiment_analysis_with_neural_networks/twits.rar) file.

## Installation
---
- Clone/Download the folder.
- Unrar the _twits.rar_ file using 'Extract Here'.
- The project is in the [Jupyter Notebook file](https://github.com/kollepas/ai_for_trading/blob/master/project_06_-_sentiment_analysis_with_neural_networks/project_06_-_sentiment_analysis_with_neural_networks.ipynb).
- Unlike most other projects, this one has no _requirements.txt_ file attached to it. Beside the basic 'standard' Python packages, `nltk` and `torch` must be installed.

## How to use?
---
Open the [Jupyter Notebook file](https://github.com/kollepas/ai_for_trading/blob/master/project_06_-_sentiment_analysis_with_neural_networks/project_06_-_sentiment_analysis_with_neural_networks.ipynb) and run through the code blocks. The notebook itself comes with detailed explanation about each step.

## License
---
This project is licensed under the MIT License - see the [LICENSE](https://github.com/kollepas/ai_for_trading/blob/master/LICENSE) file for details.


text:
# Logging and Visualizations

### EFK for Aggregated Logs
The OpenShift platform provides an aggregated logging stack that is automatically configured to centralize and store logs from application pods. These logs are only retained for a short period of time, currently about 14 days, but can be used to help identify issues with application pods.

Kibana is the primary interface for viewing and querying logs.

#### Access the archive link from a pod
The shortcut towards accessing the Kibana is from the `Logs` tab of a running pod. Kibana can also be accessed directly at its [url](https://kibana-openshift-logging.apps.silver.devops.gov.bc.ca/).

- Select the running `rocketchat-[username]` pod and select the Logs tab

![](./images/10_logging_01.png)

- Click on the "Show in Kibana" link to go to Kibana
  - Kibana login is setup with SSO, you will see the same login page as of OpenShift console
  - Kibana inherits the same RBAC as OpenShift, you will only have access to the same namespace/project set as from OpenShift cluster

- Once you have logged in, for the first time you be asked to setup a search index. Follow the 2 steps from Kibana with the following value:
  - Index pattern: `app*`
  - Timestamp field name: `@timestamp`

![](./images/10_logging_setup_01.png)

![](./images/10_logging_setup_02.png)

- Review the logging interface and the query that has been automatically populated (there are more examples to explore at the end of this section)

![](./images/10_logging_02.png)

- Modify the query and time picker to select the entire namespace within the last few hours

![](./images/10_logging_03.png)

- Review how Kibana surfaces key information about the log sources in the left panel

![](./images/10_logging_04.png)

- Create a simple visualization of the information surfaced by Kibana

![](./images/10_logging_viz_01.png)

#### Some useful queries you can try on:
- Get logs for the whole namespace:
  ```sql
    kubernetes.namespace_name:"<namespace_name>"
  ```
- Use application labels to query logs from the same deployment:
  ```sql
    kubernetes.namespace_name:"<namespace_name>" AND kubernetes.flat_labels:"deploymentconfig=<dc_name>"
  ```
- Get error logs only:
  ```sql
    kubernetes.namespace_name:"<namespace_name>" AND level:error
  ```

Next page - [Metrics](./13_metrics.md)

repository: BCDevOps/devops-platform-workshops

branch_name: refs/heads/master

extension: md

gha_language: JavaScript

license_type: permissive

repo_name: BCDevOps/devops-platform-workshops


text:
# 5 It was DNS

There's an old IT proverb:

It's not DNS
There's no way it's DNS
It was DNS

It's a useful to remember how much DNS does for us. It is the backbone of nearly every single system on a network. Sure, from time to time you might be doing stuff over straight IP addresses, but chances are you are highly reliant on DNS to make sure things are communicating properly. Active Directory makes this twice as important, because a large portions of it require properly configured DNS to function.

branch_name: refs/heads/main


text:
## Messaging Plugin for Xamarin and Windows

The Messaging plugin makes it possible to make a phone call, send a sms or send an e-mail using the default messaging applications on the different mobile platforms.

## Setup

* Available on NuGet: https://www.nuget.org/packages/Xam.Plugins.Messaging [![NuGet](https://img.shields.io/nuget/v/Plugin.Permissions.svg?label=NuGet)](https://www.nuget.org/packages/Xam.Plugins.Messaging/)
* Install into your PCL or .NET Standard project and Client projects.

**Platform Support

|Platform|Supported|Version|
| ---- | :----: | :----: |
|Xamarin.iOS Unified|Yes|iOS 7+|
|Xamarin.Android|Yes|API 14+|
|Windows 10 UWP|Yes|10+|
|Tizen|Yes|4.0+|

### Release Notes

Change log history [available here](ChangeLog.md)

### API Usage

Full details on the API [are available here](Details.md)

#### Android Nougat

If your application targets Android N (API 24) or newer, you must use version 4.0.0+ that has support for using a [File Provider](https://developer.android.com/reference/android/support/v4/content/FileProvider.html) for adding atttachments due to the file system permission changes introduced with Android N.

You also need to add a few additional configuration files to adhere to the new strict mode:

* Add the following to your `AndroidManifest.xml` inside the `<application>` tags (**YOUR_APP_PACKAGE_NAME** must be set to your app package name):

```xml
<provider android:name="android.support.v4.content.FileProvider"
          android:authorities="YOUR_APP_PACKAGE_NAME.fileprovider"
          android:exported="false"
          android:grantUriPermissions="true">
   <meta-data android:name="android.support.FILE_PROVIDER_PATHS" android:resource="@xml/file_paths"></meta-data>
</provider>
```

* Add a new folder called `xml` into your *Resources* folder and add a new XML file called `file_paths.xml`.  Add the following code:

```xml
<?xml version="1.0" encoding="utf-8"?>
<paths xmlns:android="http://schemas.android.com/apk/res/android">
   <external-path name="external_files" path="." />
</paths>
```
Note : Make sure the file `file_paths.xml`is AndroidResource.
### Contributors

* [cjlotz](https://github.com/cjlotz)
* [jamesmontemagno](https://github.com/jamesmontemagno)

### License

[The MIT License (MIT)](LICENSE.md)

license: MIT

gha_language: C#

gha_license_id: MIT


text:
A large part of the library consists of a tree of mixins, with one mixin for each Qt class.
These mixins also inherit from each other and are applied to the Qt classes.
That way each class gets all helper methods from all sub-mixins.

Example: The class `TreeView` inherits from original Qt Class `QTreeView` and gets helper methods
from `TreeViewMxin`, `AbstractItemViewMixin`, `AbstractScrollAreaMixin`,
`FrameMixin`, `WidgetMixin` and `ObjectMixin`.

To illustrate this, we will use some of our included models:

!!! Example "Class hierarchy example"

    === "Subclass tree"

        ```py
        from prettyqt import itemmodels, widgets

        app = widgets.app()
        widget = widgets.TreeView()
        model = itemmodels.SubClassTreeModel(core.AbstractItemModelMixin)
        widget.set_model(model)
        widget.show()
        ```

          ![Image title](images/abstractitemmodelmixin_subclasses.png)

AbstractitemModelMixin subclasses

    === "Parentclass tree"

        ```py
        from prettyqt import itemmodels, widgets

        app = widgets.app()
        widget = widgets.TreeView()
        model = itemmodels.ParentClassTreeModel(widgets.TreeWidget)
        widget.set_model(model)
        widget.show()
        ```

          ![Image title](images/treewidget_parentclasses.png)

TreeWidget parent classes

    === "MRO tree"

        ```py
        from prettyqt import itemmodels, widgets

        app = widgets.app()
        widget = widgets.TreeView()
        model = itemmodels.ParentClassTreeModel(widgets.TreeWidget, mro=True)
        widget.set_model(model)
        widget.show()
        ```

          ![Image title](images/treewidget_mro.png)

TreeWidget MRO

gha_language: Python


text:
# gt4opencv
Glamorous Toolkit for the OpenCV

## OpenCV and TensorFlow

### Mac

OpenCV version 3:
```
brew install opencv@3
```

Tensorflow version 1.14.0:
```
brew install tensorflow
```

## How to load

You can load the whole code in a [Glamorous Toolkit](https://gtoolkit.com/install/) image using the following snippet:

```
Metacello new
    baseline: 'GToolkit4OpenCVWithoutGT';
    repository: 'github://feenkcom/gt4opencv/src';
    load
```


text:
Elebeta Text Tables (ETT)
================================================================================

1. Introduction
--------------------------------------------------------------------------------

ETT extends Comma-Separated Values (CSV, [RFC 4180](http://tools.ietf.org/html/rfc4180)): data is stored in ~~comma-seperated~~ something-separated values and also follows the same escaping rules from RFC 4180. However, ETT allows for arbitrary newline sequences, separator characters and escaping characters, that are specified in the file itself. Additionally ETT supports schema references and enforces column naming and column typing.

2. Structure
--------------------------------------------------------------------------------

An ETT file has two sections: a header section and a data section, both of them CSV compatible. The first part of the header (where all CSV parsing settings for the rest of the document are defined) is not actually in CSV, but should still be readable by most programs.

##2.1 Header

The header is composed of CSV parsing settings, a schema reference and column definitions (column types and column names).

###2.1.1 CSV parsing settings

All CSV parsing settings follow the format

	<keyword>-<value><newline sequence>

where value is a non-quoted (or escaped) string.

####<IP_ADDRESS> Newline configuration (LF, CR+LF or other)

The newline sequence used throughout the file will appear after the NEWLINE- keyword; it can be any 1 or 2 length sequence of Unicode characters, however common LF or CR+LF sequence are recommended on most cases.

	NEWLINE-<newline sequence>

_From now on, the selected newline sequence will be referred to as {nl}._

####<IP_ADDRESS> Encoding

The encoding used on the file: either UTF-8 or ISO (local code page).

	CODING-UTF-8{nl}
	CODING-ISO{nl}

####<IP_ADDRESS> Separator configuration

The value separator used in the document; it can be any Unicode character.

	SEPARATOR-<separator>{nl}

_From now on, the selected separator will be referred to as {sp}._

####<IP_ADDRESS> Escape configuration

The character used to escape itself, separators and newline sequences in values; it can be any Unicode character.

	ESCAPE-<escape character>{nl}

_From now on, the selected escape character will be referred to as {esc}._

###2.1.2 Optionally blank schema name

ETT supports embedding a reference to a schema, so that applications can more easily identify, validate and process these files automatically. A common usage for this it to have it set as the fully qualified Haxe class name used for the records.

	CLASS-<optionally blank schema name>{nl}

###2.1.2 Column types

	<1st column type>{sp}<2nd column>{sp}>...<nth column>{nl}

Type definitions are always trimmed for whitespace.

The types are:

####<IP_ADDRESS> Basic types:

	Bool           a true or false enumeration
	Int            a 32-bit integer
	Float          a 64-bit floating point number
	String         locally encoded text
	Date           local date
	Timestamp      milliseconds from Unix epoch in milliseconds
	HaxeSerial     Haxe serialized data

None of these are, by themselves, nullable.

####<IP_ADDRESS> Nullables:

Any type may be wrapped by Null<> for it to be interpreted as nullable.

	Null<Bool>
	Null<Int>
	...
	Null<HaxeSerial>

####<IP_ADDRESS> Geometry:

	Geometry<Point>        a point (an object with x,y coordinates)
	Geometry<LineString>   a sequence of points

####<IP_ADDRESS> Other types that may be added in the future:

	Base16
	Base64
	SI<[meters],[kilograms],[seconds]>

###2.1.3 Column names

	<1st column name>{sp}<2nd column>{sp}>...<nth column>{nl}

Column names are always trimmed for whitespace.

**ATTENTION**: column names must be unique.

##2.2 Data

Table records in CSV format

3 Data encoding and other format considerations
--------------------------------------------------------------------------------

##3.1 Escaping

Default RFC 4180 escaping rules apply.

##3.2 Null encoding

A blank field (empty string) is always interpreted as a null. On trimmed fields, a resulting empty string will also be treated as a null.

##3.3 Boolean encoding

	true
	false

Always trimmed for whitespace.

##3.4 Integer encoding

For now, any Haxe supported integer text encoding:

	128
	0x80

Always trimmed for whitespace.

**ATTENTION**: all integer should fit in the standard Haxe 32-bit integer.

##3.5 Floating point number encoding

For now, any Haxe supported integer text encoding:

	1.1
	1.
	.1
	11e-1

Always trimmed for whitespace.

##3.6 String encoding

...

May be wrapped by Trim<> for whitespace trimming.

##3.7 Date and timestamp encoding

For date encoding, follow the standard Date.hx rules.

For timestamp encoding, this should be any floating point UTC timestamp, with **milliseconds** since Unix epoch.

Always trimmed for whitespace.

##3.8 Haxe serialization format

Follows the [Haxe Serialization Format](http://haxe.org/manual/serialization/format).

Always trimmed for whitespace.

##3.9 Geometry

Both Point and LineString consist on the internal representation of the WKT (well known text) for these types of geospacial objects. Coordinates are assumed to be in the WGS 84 reference coordinate system and in decimal degrees.

###3.9.1 Point

	<x coordinate> <y coordinate>

###3.9.2 LineString

	<point 0>, <point 1>, ..., <point n>

##3.10 End of file

Properly formated files should end with a newline. However, readers should be able to parse files without the ending newline.

4 Examples
--------------------------------------------------------------------------------

RodoTollSim Node File (schema Elebeta:RodoTollSim:Node), making heavy use of the auto trimming of non String fields:

	NEWLINE-
	CODING-ISO
	SEPARATOR-|
	ESCAPE-"
	CLASS-example.Node
	Int|      Geometry<Point>
	 id|          coordinates
	  1|-40.298736 -20.310141
	  2|-40.291990 -20.302984
	  3|-40.311600 -20.310500

Alternative (bare bones) RodoTollSim Node File (schema Elebeta:RodoTollSim:Node):

	NEWLINE-
	CODING-ISO
	SEPARATOR-|
	ESCAPE-"
	CLASS-example.Node
	Int|Geometry<Point>
	id|coordinates
	1|-40.298736 -20.310141
	2|-40.291990 -20.302984
	3|-40.311600 -20.310500

RodoTollSim Link File (schema Elebeta:RodoTollSim:Link) making heavy use of the auto trimming of non String fields:

	NEWLINE-
	CODING-ISO
	SEPARATOR-|
	ESCAPE-"
	CLASS-example.Link
	Int|        Int|         Int|   Float|   Int|Float
	 id|startNodeId|finishNodeId|distance|typeId| toll
	  1|          1|           2|      69|    50|    0
	  2|          2|           1|      81|    30|  3.0
	  3|          2|           3|     159|    70|    0
	  4|          3|           2|     158|    80|    0

Alternative (bare bones) RodoTollSim Link File (schema Elebeta:RodoTollSim:Link):

	NEWLINE-
	CODING-ISO
	SEPARATOR-|
	ESCAPE-"
	CLASS-example.Link
	Int|Int|Int|Float|Int|Float
	id|startNodeId|finishNodeId|distance|typeId|toll
	1|1|2|69|50|0
	2|2|1|81|30|3.0
	3|2|3|159|70|0
	4|3|2|158|80|0

path: /doc/ett/Elebeta Text Tables (ETT).md

filename: Elebeta Text Tables (ETT).md


text:
# Changelog

## {{Next Version}}

### Misc

- NOTE: Arch instructions now say to install from source. Arch tests removed temporarily (#916)
- NOTE: there is an issue when repo directory (or a parent dir) contains a space (#135)
- Improve error messaging when we cannot find git repo (#874)
- Temporarily disable apk builds on alpine (#881)
- Have `hide -v` show output from gnupg
- Documentation updates and fixes

## 0.5.0

### Features

- Adds `SECRETS_GPG_ARMOR` env variable to use `gpg --armor`
  when encrypting files, so secret files are stored
  in text format rather than binary (#631)
- Allow gnupg permission warnings in `tell`, `hide`, `reveal`, and `removeperson` (#811)
- `git secret init` now sets `.gitsecret/keys` permission to 0700 (#811)
- Improve verbose and non-verbose output

### Bugfixes

- Fix adding newlines to `.gitignore` entries (#643)
- Fix `cat` and `reveal` on named files while in repo subdir (#710)
- Fix `clean`, `hide`, `reveal` so they only remove marked secret files (#833)
- Fix for `removeperson` if same email is present multiple times (#638)
- Correct error message about files missing from .gitignore

### Misc

- Rename `killperson` command to `removeperson` (#684)
- Improve error messaging decrypting nonexistent files (#706)
- Improve, expand, correct, and update docs (#699)
- Update docs for use with CI/CD server (#675)
- Upgrade bats-core to v1.6.0 (#755)
- Test, and build RPMS, with Rocky and Alma Linux instead of CentOS (#765)
- Automate testing code on windows using WSL (#846)
- Automate testing code on FreeBSD (#455)
- Improve testing of .gitignore contents (#792)
- Automate running verbose tests with SECRETS_TEST_VERBOSE=1 (#794)
- Improve documentation about installing on Windows (#843)

## 0.4.0

### Bugfixes

- Escape filenames with special characters before adding to `.gitignore`
- Better error handling around telling an email twice (#634)
- Fix for `-P` (#647)

### Misc

- Removed `test-kitchen`
- Moved from `travis` to GitHub Actions
- Changed almost all infrastructure code
- Moved away from Bintray to Artifactory
- Changes how GitHub Pages work
- Add security disclaimer for git-secret-killperson
- Improve documentation about releases
- Man page improvements

## Version 0.3.3

### Bugfixes

- In 'tell', warn about disabled, revoked, expired, or invalid keys (#552, #508, #317, #290, #283, #238)
- Error if 'tell' is used on an email address with multiple keys (#552)
- Don't let 'reveal' clobber secret files (#579)
- Updated test key fixture that had expired (#607)

### Misc

- Improve docs about using gpg with git-secret (#577)
- Text improvements and More about security in git-secret.7 man page (#603)
- Reflect changes in ruby bundler during build process
- Upgrade build process to ansible 2.9
- Use shellcheck 0.7.1 with CI, not 'latest' (#609)
- Improve output of `git-secret add`

## Version 0.3.2

### Bugfixes

- Fix mention of version in git-secret add man page (#544)

### Misc

- Update developer docs, especially regarding mac, docker, and test-kitchen (#195)
- Update man pages to mention version documented (#420)

## Version 0.3.1

### Misc

- Update man pages

## Version 0.3.0

### Features

- Support SECRETS_PINENTRY env var for gnupg --pinentry-mode parameter (#221)
- Show output from gnupg if 'hide' fails (#516, #202, #317)
- Add support for Busybox (#478)

### Bugfixes

- Use OSX's mktemp on OSX, even if there's another version in PATH. (#485)
- Make rsync a build requirement on debian (#500)
- Use gnupg1, not gnupg2, when tests specify gnupg1 (#241)
- Note dependencies gawk, bash, and coreutils in linux packages (#493)
- Handle case of key having no email and a comment (#527)
- Avoid blank lines from output of 'clean -v'

### Misc

- Improve messaging and logic around deleting tmp files.
- Add note about secrets and old keys (#499)
- Transition build process from python 2 to python 3 (#487)
- Upgrade build process from ansible 2.5 to ansible 2.8
- Fix build process when installing gnupg2 source deps on Ubuntu
- Close file descriptor 3 when running gnupg subprocesses (#521)
- Small optimization in 'hide'
- Improve code comments
- Update docs to note that git-secret repos modified by git-secret 0.2.3 and
  later are not backward compatible with pre-0.2.3 versions of git-secret. (#536)

## Version 0.2.6

### Features

- git-secret is now available in Fedora, link added to README.md. (#315)
- Support automated testing on windows with Travis CI (#372)
- Support SECRETS_VERBOSE env var to enable verbosity (#323)
- Use gpg without --quiet when decrypting in verbose mode (#394)
- Add -v options to 'tell' and 'reveal', showing gpg output (#320, #395)
- Change 'init' to never ignore .secret files (#362)
- 'add' appends filepaths to .gitignore by default (#225)
- Automate the GitHub release (#411)

### Bugfixes

- Fix 'hide -m' when used as first hide operation (#466)
- Fix code to respect $TMPDIR when generating tmp files (#451)
- Be more careful when deleting test files (#360)
- Use separate directory when testing, instead of using $BATS_TMPDIR directly (#407)
- Fix 'whoknows -l' and related tests on FreeBSD (#454)
- Fix git-secret init when used on busybox (#475)
- Update git-secret.io, fix utils/gh-branch.sh to use 'git all --add' (#344)
- Fix link to homebrew's git-secret in README.md (#310)
- Remove diagnostic output from test results (#324)
- Remove un-needed redirection in 'reveal' (#325)
- Fix link to current contributors in CONTRIBUTING.md (#331)
- Fix tests when running from git hooks (#334)
- Fix typo, remove temp directory in utils/tests.sh (#347)
- Spelling fixes
- Fix re: SECRETS_DIR in 'init' and SECRETS_EXTENSION in test_reveal.bats (#364)
- git-secret will fail if you pass params or filenames that are not understood (#390)
- Use SECRETS_GPG_COMMAND env var in gpg version check (#389)
- Add header to git-secret.7 man page, for debian and doc improvement (#386)
- Respect DESTDIR when installing as per GNU/debian/etc recommendations (#424)
- Use git check-ignore to test for files ignored by git

### Misc

- Improve docs about hide -m option (#467)
- Document SECRETS_VERBOSE and improve env var docs (#396)
- Setting SECRETS_TEST_VERBOSE env var shows debug info during tests (EXPERIMENTAL)
- Add documentation about how to write tests.
- Suppress 'cleaning up temp files' messages unless in a verbose mode.
- Improve git-secret user messaging.
- Update CHANGELOG.md to mention fix for #281 in v0.2.5 (#311)
- Add text explaining git-secret Style Guide and Development Philosophy
- Use Shellcheck on tests/ files, changes for Shellcheck in tests/ (#368)
- Use Shellcheck on MacOS/osx travis tests (#403)
- Show commands run by Makefile as per debian upstream recommendations (#386)
- Upgrade bats-core to v1.1.0, import bats-core into vendor/bats-core (#377)
- Use gawk to parse emails from gpg output
- Optimize code that parses keyrings
- Remove unused code

## Version 0.2.5

### Features

- Add support for FreeBSD (#244)
- Add -l option to whoknows, which shows key expiration dates (#283)
- Add -P option (preserve permissions) to reveal and hide (#172)
- Add -F option (force, changing some errors to warnings) to hide and reveal (#253)
- Allow user to specify name of secret dir at runtime using SECRETS_DIR env var, and test (#247, #250)

### Bugfixes

- Fix issues with spaces in paths and filenames (#226, #135)
- Fix issue when 'hide' used in subdir of repo (#230)
- Fix issues in 'changes' with trailing newlines (#291)
- Fix 'hide' to only count actually hidden files as hidden (#280)
- Fixed bugs and improved error messages (#174)
- Issue error message when unable to hide a secret (#202, #238)
- Accept gpg key with no name, only an email (#227)
- Require keys to be specified by email, as documented (#267)
- Disallow 'git secret tell' or 'killperson' with emails that are not in keyring (also #267)

### Misc

- Added notes about packages and for package maintainers (#281)
- Improve documentation regarding operation with different versions of GPG (#274, #182)
- Documentation improvements, error message and text improvements, and typo fixes (#254)
- git-secret RFC#001 added, documenting a path towards independence from gpg binary formats (#208)
- Add tests for expired gpg keys, and gpg keys with only emails (#276)

## Version 0.2.4

### Features

- Added `git secret cat` feature (#141)

### Bugfixes

- `git secret hide` and `git secret changes` check for files more carefully (#153, #154)

### Misc

- Documentation and error message improvements (#126, #136, #144, #150)
- Build and CI fixes (#152, #179, #186, #188, #189)
- Migrate to `bats-core` bash testing framework

## Version 0.2.3

### Features

- Added `-m` option to `hide` command, files will only be hidden when modifications are detected (#92)
- Changed how path mappings file works: colon delimited FSDB in `.gitsecret/paths/mapping.cfg', so git-secret
  can store checksums of hidden files. Note this means git-secret repos modified by git-secret 0.2.3
  or later are not backward compatible with pre-0.2.3 versions of git-secret. (#92)
- `git secret init` now adds `random_seed` to `.gitignore` (#93)

### Bugfixes

- Dropped `git check-ignore`, using `git add --dry-run` instead to check for ignored files (#105,#38)
- Fixed `gnupg` >= 2.1 CI tests (#6)

### Misc

- Now users can run local CI tests using test-kitchen (#6)
- Migrated travis ci tests to test-kitchen for Linux platforms.
- Added more `gpg` version to test matrix (#99)
- Added CentOS to test matrix (#38,#91)
- All tested Linux platforms now use latest release of `shellcheck`
- Added Alpine to test matrix, and apk is now built. (#75)

## Version 0.2.2

### Features

- Change how the `usage` command works (#48)
- Now `git-secret` works from any place inside `git-tree` (#56)
- Added `-d` option to the `hide` command: it deletes unencrypted files (#62)
- Added new command `changes` to see the diff between the secret files (#64)
- Now it is possible to provide multiple emails to the `killperson` command (#73)
- Now it is possible to provide multiple emails to the `tell` command (#72)

### Bugfixes

- Fixed bug when `_user_required` was not working after re-importing keys (#74)
- Refactored `hide` and `clean` commands to be shorter

### Misc

- Now every doc in this project refer to `git-secret.io` instead of old `gh-pages` website (#71)
- Now installation section is removed from main `man` file (#70)
- Now "See also" sections in the `man` pages are clickable (#69)
- Added "Manual" section to the manuals (#61)
- Added `CentOS` container for `ci` testing (#38)
- Tests are refactored. Added `clean` command tests, removed a lot of hard-coded things, moved tests execution from `./temp` folder to `/tmp`, added a lot of new check in old tests, and some new test cases (#52)
- `shellcheck` is now supported with `make lint`

## Version 0.2.1

### Misc

- Added `CONTRIBUTING.md` and `LICENSE.md`.
- New brand logo in the `README.md`.
- Added autodeploy to `bintray` in `.travis.yml`.
- Now everything is tested inside the `docker`-containers and `OSX` (MacOS) images on `travis`.
- Added `.ci/` folder for continuous integration, refactored `utils/` folder.
- Everything is `shellcheck`ed (except `tests/`).

## Version 0.2.0

### Features

- Added `changes` command to see the difference between current version of the hidden files and the committed one
- Added `-f` option to the `reveal` command to remove prompts

### Bugfixes

- Some bugs are fixed

### Misc

- New installation instructions
- Changed the way files were decrypted, now it is a separate function

## Version 0.1.2

### Features

- Added `-i` option to the `git-secret-add` command, which auto adds unignored files to the `.gitignore`

### Misc

- `.github` templates added
- Documentation improved with `Configuration` section
- `Makefile` improvements with `.PHONY` and `install` target
- Added extra tests: for custom filenames and new features

## Version 0.1.1

### Features

- Added `--dry-run` option to the `git secret` command, which prevents any actions.

### Misc

- Removed animation from docs, now using `asciinema.org`
- `install_full_fixture()` returns a fingerprint
- `uninstall_full_fixture()` receives two args
- Fixed bug when tests were failing with `gpg2`
- New travis strategy: testing both `gpg` and `gpg2`

## Version 0.1.0

### Features

- Implementation of git secret add
- Implementation of git secret clean, with -v option
- Implementation of git secret hide, with -c 'clean' and -v option
- Implementation of git secret init
- Implementation of git secret killperson
- Implementation of git secret list
- Implementation of git secret remove, with -c option
- Implementation of git secret reveal, with -d homedir and -p passphrase options
- Implementation of git secret tell, with -m email and -d homedir options
- Implementation of git secret usage

repository: sobolevn/git-secret

gha_language: Shell

repo_name: sobolevn/git-secret


text:
---
layout: post
title:  "GSoC Week 3"
date:   2018-06-03
excerpt: "The third week has now ended. Let’s talk about some important points."
tag:
- GSoC
comments: true
---

# Google Summer of Coding Week 3

Hi, all. 😀

The third week has now ended. Let’s talk about some important points.

I couldn’t code for two-three consecutive days this week as I had an exam. But still, I was able to do what I had planned, to some extent.

## PaddlePaddle Docker appliance on the CWRU HPC
This week, I gave a try for Docker appliance following the steps Prof. Peter did before. I had an idea that the image itself has no problem. And what we need to do is to modify $HOME path inside the image. It turned out that my idea is right. As a result, we can finally run DeepSpeech2 on CWRU HPC! Here's what I've done:

**1.Login**

{% highlight shell %}
CynthiaHsus-MacBook-Pro:~ suwi$ ssh<EMAIL_ADDRESS>Last login: Mon May 28 05:12:11 2018 from <IP_ADDRESS>
sxx186@redhen1:~$ ssh<EMAIL_ADDRESS>Last login: Mon May 28 05:12:13 2018 from redhen1.cosi.cwru.edu
[sxx186@hpclogin ~]$ srun -p gpu -C gpuk40 --gres=gpu:1 --pty bash
srun: job 7813756 queued and waiting for resources
srun: job 7813756 has been allocated resources
[sxx186@gpu028t ~]$ module load singularity/2.5.1
{% endhighlight %}

**2.Download the Docker image and rename it as you like**

{% highlight shell %}
[sxx186@gpu028t ~]$ singularity pull docker://paddlepaddle/deep_speech:latest-gpu
WARNING: pull for Docker Hub is not guaranteed to produce the
WARNING: same image on repeated pull. Use Singularity Registry
WARNING: (shub://) to pull exactly equivalent images.
Docker image path: index.docker.io/paddlepaddle/deep_speech:latest-gpu
Cache folder set to /home/sxx186/.singularity/docker
[17/17] |===================================| 100.0%
Importing: base Singularity environment
Exploding layer: sha256:d5c6f90da05dc7e77d2e5fef63c341ab05ba2a03396ab5ae8f18814a7bbf5265.tar.gz
Exploding layer: sha256:bbbe761fcb565a007b458e09e08ecb88947f647f57be819a492a6b23694cefd8.tar.gz
Exploding layer: sha256:7afa5ede606fb1845e42f0d4816d2a7593a2b666ff9ca4722dcd2cff8a541acf.tar.gz
Exploding layer: sha256:f6b7253b56f434d6e4d97d259cb1007481cf670df2e04229e83cf37db33d96eb.tar.gz
Exploding layer: sha256:2b8db33536d447786cf57cca870fdd7d50d55fd67060c06252edc1c8db456a8f.tar.gz
Exploding layer: sha256:e0f506dd4a700abd7006016a537351f3e6e429a346bd8b457d91b80894f13884.tar.gz
Exploding layer: sha256:21ae5b69f65daa289d53eca8f567ec983773aa10c79290326de8e525d1721ec2.tar.gz
Exploding layer: sha256:87bc59b3df1f85363c35d004ffdda7b4d9c9f0138159ec4a7fc11336c609c255.tar.gz
Exploding layer: sha256:8c5da5017d0a1fb008aa5b1b4ff7b8d4579ad01963c08855e1e6cb44274275e4.tar.gz
Exploding layer: sha256:624201d912d1758f0b40bfe94507b2c58032a1032034cdb3a2d6ab7b411bcf11.tar.gz
Exploding layer: sha256:65f018df44bdeee1c0b9cf20877862a6729b325bf3efd0d1dc8ba9c68b621239.tar.gz
Exploding layer: sha256:9c7a53ae7ba0b8e176b44dc8b8ac8ee2867bd6703a0e7c7300ab325f0d955a8a.tar.gz
Exploding layer: sha256:8eb318e8051e88cca6a629972690ad2b1e0a43b8673d51ec8ea0b45b28f16293.tar.gz
Exploding layer: sha256:9b014bae4feef5c6253d4b2cf0d76817649a605dbbb23a6c36e143aa5a537dda.tar.gz
Exploding layer: sha256:132470f5050df9b5570e5e6c24906a9d23b44f9bfb57faf02c9f3887f34bd5b9.tar.gz
Exploding layer: sha256:fd77b9e6c0999686b6b57f51f72240669dcff93728ff2de7c17f448ee650a301.tar.gz
Exploding layer: sha256:1257e2a32bf9be4faafba2b0275486f51c3878bab8d0fb9d3099677f39cb4520.tar.gz
Exploding layer: sha256:11f2a6377069ab1f79d93ca05a42f15e0569d3582e6383f6aabb83efac99187b.tar.gz
WARNING: Building container as an unprivileged user. If you run this container as root
WARNING: it may be missing some functionality.
Building Singularity image...
Singularity container built: ./deep_speech-latest-gpu.simg
Cleaning up...
Done. Container is at: ./deep_speech-latest-gpu.simg
[sxx186@gpu028t ~]$ git clone https://github.com/PaddlePaddle/DeepSpeech.git
Cloning into 'DeepSpeech'...
remote: Counting objects: 2420, done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 2420 (delta 4), reused 6 (delta 3), pack-reused 2408
Receiving objects: 100% (2420/2420), 888.18 KiB | 0 bytes/s, done.
Resolving deltas: 100% (1563/1563), done.

mv deep_speech-suwi.simg DeepSpeech/
{% endhighlight %}

**3.Get into the image and set $HOME**

{% highlight shell %}
cd /mnt/rds/redhen/gallina/Singularity/DeepSpeech2/DeepSpeech/
singularity shell -e -H /mnt/rds/redhen/gallina/Singularity/DeepSpeech2/DeepSpeech/ deep_speech-suwi.simg
Singularity: Invoking an interactive shell within container...

unset HOME
export HOME="/mnt/rds/redhen/gallina/Singularity/DeepSpeech2/DeepSpeech/"
{% endhighlight %}

**4.Run the code(run_data.sh as an example, and you can see Tiny data preparation done.)**

{% highlight shell %}
cd examples/aishell/
sh run_data.sh
{% endhighlight %}

## Conclusion
And thus, the third week ends. 12 – 3 = 9 weeks of awesomeness still left. 😉I will be back with 4th week’s blog soon. Thank you for reading. Bye.


text:
[![MIT license](https://img.shields.io/badge/License-MIT-blue.svg)](https://github.com/proplants/plantbook/blob/main/LICENSE)
[![codecov](https://codecov.io/gh/proplants/plantbook/branch/main/graph/badge.svg)](https://codecov.io/gh/proplants/plantbook)
[![lint workflow](https://github.com/proplants/plantbook/actions/workflows/lint.yml/badge.svg)](https://github.com/proplants/plantbook/actions?query=workflow%3Agolangci-lint)

# Plantbook, plants social network.

![](assets/README_content/github_plants.png)

## For generate plantbook-server

nedd to be installed [go-swagger#install](https://goswagger.io/install.html)

```bash
# after change spec run command below...
swagger generate server -t internal/api -A plantbook -f ./api/swagger/swagger.yaml --main-package=../../../cmd/plantbook-server
# for help type: swagger generate server --help
-t internal/api # path for put generated files
-A plantbook # application name
-f ./api/swagger/swagger.yaml # path to swagger speca
--main-package=../../../cmd/plantbook-server # path for put main file location
```

## Contents

- [Plantbook, plants social network.](#plantbook-plants-social-network)
  - [For generate plantbook-server](#for-generate-plantbook-server)
  - [Contents](#contents)
  - [1. Introduction](#1-introduction)
  - [2. Components](#2-components)
  - [3. Main classes](#3-main-classes)

## 1. Introduction

Plantbook is an online service that intended for the plant owners and gardeners who look for a tool that would allow
them to take care about their plants better. Every plant can optionally have a name and eventually a blog about
its life, that can contain photos, some sad or happy stories.

## 2. Components

![](assets/README_content/components.png)

## 3. Main classes

![](assets/README_content/classes.png)

repository: proplants/plantbook

gha_language: Go

repo_name: proplants/plantbook


text:
# Zed-Attack-Proxy-Resources
This contains the best resources from online for learning Zed Attack Proxy (ZAP) from OWASP. Zed Attack Proxy is a penetration testing tool with
loads of functionality such as scanning, fuzzing, ajax spiders etc. It supports both automated and manual penetration testing. Here are the
links for learning ZAP.
## Zed-Attack-Proxy Official Guide
https://github.com/zaproxy/zaproxy/releases/download/2.4.0/ZAPGettingStartedGuide-2.4.pdf
https://github.com/zaproxy/zap-core-help/wiki
## Zed-Attack-Proxy Videos
https://vimeo.com/32387480
https://www.youtube.com/watch?v=pYFtLA2yTR8
https://www.youtube.com/playlist?list=PLEBitBW-Hlsv8cEIUntAO8st2UGhmrjUB
## Zed-Attack-Proxy Slides
https://www.slideshare.net/adigupt09/the-owasp-zap-rn-d-presentation
https://www.slideshare.net/psiinon/automating-owasp-zap-devcseccon-talk
https://www.slideshare.net/fadi/owasp-zap-61005118
## Zed-Attack-Proxy SQL Injection
https://www.pgs-soft.com/finding-security-gaps-in-your-application-with-owasp-zap-tool/
https://www.youtube.com/watch?v=dqKGGCVFTvI
## Zed-Attack-Proxy XSS
https://tododev.wordpress.com/2013/12/27/detecting-and-fixing-xss-using-owasp-tools/
https://www.youtube.com/watch?v=rmbi-VbIK6I

repository: KKGanguly/Zed-Attack-Proxy-Resources

repo_name: KKGanguly/Zed-Attack-Proxy-Resources


text:
# aptsearcher

The system simplifies the search of an apartment.

## architecture draft

![aptsearcher architecture diagram](./aptsearcher.png)

Periodically parser components parse results of apartment listing on the [craigslist](https://craigslist.org) and possibly other sources. Parsed results passed to the [record processor](#record-processor).

Record processor firs of all save every record to persistent storage. Saved items required for future model training.

Second, the record processor sends a request to the price predictor and compares the predicted price with a stated price.
If a stated price is much less than the predicted price system should inform users about a good deal.

## components

### craiglist parser

This parser parses Craiglist site for new apartment listings and grabs details of each new listing.

Possible implementation of parser: [apthunt](https://github.com/vistrcm/apthunt).

### Other parsers

Optional parser of additional resources: local websites, or others.

All parsers should implement the same interface.

### Record processor

The main logic applied to every record from any parser.

1) Save the record to the datastore,
2) use record data to predict the price of the listed apartment,
3) if the predicted price is higher than the stated price, send a notification.

Notification can be send via telegram channel.

### Data Storage

Please think carefully about what type of data store to use. It may be object storage like [AWS S3](https://aws.amazon.com/s3/), more complex key-value store like [Google Cloud Datastore](https://cloud.google.com/datastore/). Find a good solution designed for "mostly write" workloads. Do not introduce unnecessary complexity. Plus, think about the export of the whole database to the table-like format, for example, CSV.

Maybe multiple storages for different types of data: tabular, text, images.

### Price predictor

The price predictor is using a model created by (#trainer) to predict the price of the apartment based on input parameters.

Most probably straightforward web/gRPC application taking JSON data as input and return predicted price or bucket of predicted prices.

### Trainer

Trains a machine-learning model to predict apartment prices based on a dataset from the [Data Storage](#data-storage).

You can use any possible way to build and train the model. A couple of suggestions provided below.

You can use methods described in:

* [Fast.AI course, Lesson 4: NLP; Tabular data; Collaborative filtering; Embedding](https://course.fast.ai/videos/?lesson=4) tabular data example is a good fit for this task.
* [Machine Learning with TensorFlow on Google Cloud Platform Specialization](https://www.coursera.org/specializations/machine-learning-tensorflow-gcp) provides many insights on how to build a similar model with Tensorflow.

To provide results soon, you can start with a single model working on tabular data: predict the price based on numerical and categorical information.

The second step may be to train 3 different models:

* tabular - same as above,
* textual - model predicting the price based on the text description of the property,
* visual - model predicting the price based on photos of the property.

You can calculate the weighted average of model results and use it as a final prediction.

As the third step, you can create a single model considering all three types of data: tabular, text, and photos.

## code repository

Create a GitHub repository (public or private) that contains source code and installation instructions or extend existing [apthunt](https://github.com/vistrcm/apthunt) repository.

repository: vistrcm/projideas

repo_name: vistrcm/projideas


text:
---
layout: page
title: aspect
description: naturally recursive, minimal mind-mapping utility
img: /assets/img/2018-07-01-project.png
---

[source code](https://github.com/wilimitis/aspect)

aspect is a minimalistic application designed to enable the user to store notes in recursive tree structure. Such a structure, I believe, models the most natural way in which the mind access information, for example

Root
- Programming
  - JavaScript
    - p5.js
- Exercise
  - Yoga
    - Vinyasa
      - Chaturanga Dandasana
        - ...

aspect was designed for both Android and iOS using React Native and Redux.

extension: markdown


text:
+++
draft = false
date = "2018-06-14T11:25:00+02:00"
title = "Subquadratic Encodings for Point Configurations"
abstract = ""
abstract_short = ""
event = "SoCG 2018"
event_url = "https://www.renyi.hu/conferences/socg18"
location = "Budapest, Hungary"

selected = false
math = true

url_pdf = ""
url_slides = "https://ipfs.xn--mxac.cc/ipfs/QmckEbpCyq5wgkY8KSGpUs9VAJ3vJVm7zHbMn9VUaUE6yv"
url_video = ""

# Optional featured image (relative to `static/img/` folder).
[header]
image = "headers/bubbles-wide.jpg"
caption = ""

+++

<!--Embed your slides or video here using-->
<!--[shortcodes](https://gcushen.github.io/hugo-academic-demo/post/writing-markdown-latex/).-->
<!--Further details can easily be added using *Markdown* and $\rm \LaTeX$ math-->
<!--code. -->

repository: aureooms-research/homepage

gha_language: HTML

repo_name: aureooms-research/homepage


text:
type-prog
=========

This is my attempt to grok type-level programming in Scala.  This source is discussed in the following blog posts

1. [Type programming: Shifting from values to types](http://proseand.co.nz/2014/02/17/type-programming-shifting-from-values-to-types/): [Tag 1.0](https://github.com/joescii/type-prog/tree/1.0)
2. [Type programming: Recursive types](http://proseand.co.nz/2014/03/10/type-programming-recursive-types/): [Tag 2.0](https://github.com/joescii/type-prog/tree/2.0)
3. [Type programming: Constraining values](http://proseand.co.nz/2014/03/17/type-programming-constraining-values): [Tag 3.0](https://github.com/joescii/type-prog/tree/3.0)
4. [Type programming: Constraining values with equivalent types](http://proseand.co.nz/2014/03/24/type-programming-constraining-values-with-equivalent-types): [Tag 4.0](https://github.com/joescii/type-prog/tree/4.0)

repository: joescii/type-prog

repo_name: joescii/type-prog


text:
# Jsonv

A library that provides the following:

* A simple and extensible DSL to create JSON validators based on the Json4s AST and on scalaz `Validation`
* A Json4s serializer that can serialize and de-serialize scalaz `Validation` instances

## Quick example

From tests:

```
val json = parse("""{ "foo": 1, "bar": { "baz": true, "qux": [ 1, 2, 3 ] }, "quz": "blag", "dub": 1.2 }""")

val validator = valid[JObject] <~ props(
  _("foo")[JInt],
  _("bar")[JObject] <~ props(
    _("baz")[JBool],
    _("qux")[JArray]
  ),
  _("quz")[JString],
  _("dub")[JDouble]
)

validator(json) should equal(Success(json))
```

For examples of usage please refer to tests.

## TODO

Things that I would like to add:

* More validation predicates
* Support for building the base validator from a case class (if possible)
* More documentation

repository: Astrac/jsonv

repo_name: Astrac/jsonv


text:
# MarketStore
[![CircleCI](https://circleci.com/gh/alpacahq/marketstore.svg?style=shield)](https://circleci.com/gh/alpacahq/marketstore) [![GoDoc](http://img.shields.io/badge/godoc-reference-blue.svg)](https://godoc.org/github.com/alpacahq/marketstore) [![chatroom icon](https://patrolavia.github.io/telegram-badge/chat.png)](https://t.me/joinchat/HKxN3BGm6mE5YBt79CMM3Q)
[![codecov](https://codecov.io/gh/alpacahq/marketstore/branch/master/graph/badge.svg)](https://codecov.io/gh/alpacahq/marketstore)

Read this in [日本語(Japanese)](README.ja.md)

## Introduction
MarketStore is a database server optimized for financial time-series data.
You can think of it as an extensible DataFrame service that is accessible from anywhere in your system, at higher scalability.

It is designed from the ground up to address scalability issues around handling large amounts of financial market data used in algorithmic trading backtesting, charting, and analyzing price history with data spanning many years, and granularity down to tick-level for the all US equities or the exploding cryptocurrencies space. If you are struggling with managing lots of HDF5 files, this is perfect solution to your problem.

The batteries are included with the basic install - you can start pulling crypto price data from [GDAX](https://docs.gdax.com/#get-historic-rates) and writing it to the db with a simple [plugin](#plugins) configuration.

MarketStore enables you to query DataFrame content over the network at as low latency as your local HDF5 files from disk, and appending new data to the end is two orders of magnitude faster than DataFrame would be. This is because the storage format is optimized for the type of data and use cases as well as for modern filesystem/hardware
characteristics.

MarketStore is production ready! At [Alpaca](https://alpaca.markets) it has been used in production for years in serious business. If you encounter a bug or are interested in getting involved, please see the [contribution section](#development) for more details.

## Install

### Docker
If you want to get started right away, you can bootstrap a marketstore db instance using our latest [docker image](https://hub.docker.com/r/alpacamarkets/marketstore/tags/). The image comes pre-loaded with the default mkts.yml file and declares the VOLUME `/data`, as its root directory. To run the container with the defaults:
```sh
docker run -i -p 5993:5993 alpacamarkets/marketstore:latest
```

If you want to run a custom `mkts.yml` you can create a new container
and load your mkts.yml file into it:
```sh
docker create --name mktsdb -p 5993:5993 alpacamarkets/marketstore:latest
docker cp mkts.yml mktsdb:/etc/mkts.yml
docker start -i mktsdb
```

You can also [bind mount](https://docs.docker.com/storage/bind-mounts/)
the container to a local host config file: a custom `mkts.yml`:
```sh
docker run -v /full/path/to/mkts.yml:/etc/mkts.yml -i -p 5993:5993 alpacamarkets/marketstore:latest
```
This allows you to test out the image [included
plugins](https://github.com/alpacahq/marketstore/tree/master/plugins#included)
with ease if you prefer to skip the copying step suggested above.

By default the container will not persist any written data to your
container's host storage. To accomplish this, bind the `data` directory to
a local location:
```sh
docker run -v "/path/to/store/data:/data" -i -p 5993:5993 alpacamarkets/marketstore:latest
```
Once data is written to the server you should see a file tree layout
like the following that will persist across container runs:
```sh
>>> tree /<path_to_data>/marketstore
/<path_to_data>/marketstore
├── category_name
├── WALFile.1590868038674814776.walfile
├── SYMBOL_1
├── SYMBOL_2
├── SYMBOL_3
```

If you have built the
[cmd](https://github.com/alpacahq/marketstore/tree/master/cmd) package
locally, you can open a session with your running docker instance using:
```sh
marketstore connect --url localhost:5993
```

### Source
MarketStore is implemented in Go, so you can install it by `go install`.
```sh
go install github.com/alpacahq/marketstore/v4@latest
# export GOROOT=$HOME/go
# export PATH=$PATH:$GOROOT/bin
marketstore --version
```

You can build it from source easily too.
You need Go 1.11+ as it uses `go mod` to manage dependencies.
``` sh
go get -u github.com/alpacahq/marketstore
```
then compile and install the project binaries using
``` sh
make install
```
Optionally, you can install the project's included plugins using
``` sh
make plugins
```

### Homebrew on macOS

You can also install marketstore using the [Homebrew](https://brew.sh)
package manager for macOS.

```sh
$ brew tap zjhmale/marketstore
$ brew install --HEAD marketstore
```

To upgrade marketstore in the future, use `upgrade` instead of `install`.

Then you are equipped with marketstore service plist also

```sh
$ brew services start marketstore
$ brew services stop marketstore
```

## Usage
You can list available commands by running
```
marketstore
```
or
```
$GOPATH/bin/marketstore
```
depending on your GOPATH.

You can create a new configuration file named `mkts.yml`, populated with defaults by running:
```
$GOPATH/bin/marketstore init
```
and then start the marketstore server with:
```
$GOPATH/bin/marketstore start
```

The output will look something like:
```
example@alpaca:~/go/bin/src/github.com/alpacahq/marketstore$ marketstore
I0619 16:29:30.102101    7835 log.go:14] Disabling "enable_last_known" feature until it is fixed...
I0619 16:29:30.102980    7835 log.go:14] Initializing MarketStore...
I0619 16:29:30.103092    7835 log.go:14] WAL Setup: initCatalog true, initWALCache true, backgroundSync true, WALBypass false:
I0619 16:29:30.103179    7835 log.go:14] Root Directory: /example/go/bin/src/github.com/alpacahq/marketstore/project/data/mktsdb
I0619 16:29:30.144461    7835 log.go:14] My WALFILE: WALFile.1529450970104303654.walfile
I0619 16:29:30.144486    7835 log.go:14] Found a WALFILE: WALFile.1529450306968096708.walfile, entering replay...
I0619 16:29:30.244778    7835 log.go:14] Beginning WAL Replay
I0619 16:29:30.244861    7835 log.go:14] Partial Read
I0619 16:29:30.244882    7835 log.go:14] Entering replay of TGData
I0619 16:29:30.244903    7835 log.go:14] Replay of WAL file /example/go/bin/src/github.com/alpacahq/marketstore/project/data/mktsdb/WALFile.1529450306968096708.walfile finished
I0619 16:29:30.289401    7835 log.go:14] Finished replay of TGData
I0619 16:29:30.340760    7835 log.go:14] Launching rpc data server...
I0619 16:29:30.340792    7835 log.go:14] Initializing websocket...
I0619 16:29:30.340814    7835 plugins.go:14] InitializeTriggers
I0619 16:29:30.340824    7835 plugins.go:42] InitializeBgWorkers
```

## Configuration
In order to run MarketStore, a YAML config file is needed. A default file (mkts.yml) can be created using `marketstore init`. The path to this file is passed in to the `start` command with the `--config` flag, or by default it finds a file named mkts.yml in the directory it is running from.

### Options
Var | Type | Description
--- | --- | ---
root_directory | string | Allows the user to specify the directory in which the MarketStore database resides
listen_port | int | Port that MarketStore will serve through for JSON-RPC API
grpc_listen_port | int | Port that MarketStore will serve through for GRPC API
timezone | string | System timezone by name of TZ database (e.g. America/New_York)
log_level | string  | Allows the user to specify the log level (info | warning | error)
stop_grace_period | int | Sets the amount of time MarketStore will wait to shutdown after a SIGINT signal is received
wal_rotate_interval | int | Frequency (in minutes) at which the WAL file will be trimmed after being flushed to disk
stale_threshold | int | Threshold (in days) by which MarketStore will declare a symbol stale
disable_variable_compression | bool | disables the default compression of variable data
triggers | slice | List of trigger plugins
bgworkers | slice | List of background worker plugins

### Default mkts.yml
```yml
root_directory: data
listen_port: 5993
grpc_listen_port: 5995
log_level: info
stop_grace_period: 0
wal_rotate_interval: 5
stale_threshold: 5
```

## Clients
After starting up a MarketStore instance on your machine, you're all set to be able to read and write tick data.

### Python
[pymarketstore](https://github.com/alpacahq/pymarketstore) is the standard
python client. Make sure that in another terminal, you have marketstore running
* query data
```python
import pymarketstore as pymkts
param = pymkts.Params('BTC', '1Min', 'OHLCV', limit=10)
cli = pymkts.Client()
reply = cli.query(param)
reply.first().df()
```
shows
```python
Out[5]:
                               Open      High       Low     Close     Volume
Epoch
2018-01-17 17:19:00+00:00  10400.00  10400.25  10315.00  10337.25   7.772154
2018-01-17 17:20:00+00:00  10328.22  10359.00  10328.22  10337.00  14.206040
2018-01-17 17:21:00+00:00  10337.01  10337.01  10180.01  10192.15   7.906481
2018-01-17 17:22:00+00:00  10199.99  10200.00  10129.88  10160.08  28.119562
2018-01-17 17:23:00+00:00  10140.01  10161.00  10115.00  10115.01  11.283704
2018-01-17 17:24:00+00:00  10115.00  10194.99  10102.35  10194.99  10.617131
2018-01-17 17:25:00+00:00  10194.99  10240.00  10194.98  10220.00   8.586766
2018-01-17 17:26:00+00:00  10210.02  10210.02  10101.00  10138.00   6.616969
2018-01-17 17:27:00+00:00  10137.99  10138.00  10108.76  10124.94   9.962978
2018-01-17 17:28:00+00:00  10124.95  10142.39  10124.94  10142.39   2.262249
```
* write data
```python
import numpy as np
import pandas as pd
data = np.array([(pd.Timestamp('2017-01-01 00:00').value / 10**9, 10.0)], dtype=[('Epoch', 'i8'), ('Ask', 'f4')])
cli.write(data, 'TEST/1Min/Tick')
# Out[10]: {'responses': None}

cli.query(pymkts.Params('TEST', '1Min', 'Tick')).first().df()
```
shows
```python
                            Ask
Epoch
2017-01-01 00:00:00+00:00  10.0

```

* Variable length records

Marketstore is a database that achieves high performance by limiting the number of records in a timeframe to one.
The supported timeframes are from `1D` (1 day) to `1Sec` (1 second), and basically,
the longer the timeframe is, the faster you can read and write data.

However, it also supports data that does not arrive at a specific interval or more frequently than every second,
such as board data and TICK data. Such kind of data is called variable-length records in marketstore.

You can use the variable-length records feature by specifying `isvariablelength=True`
when you write with pymarketstore.

```python
import numpy as np, pandas as pd, pymarketstore as pymkts

symbol, timeframe, attribute_group = "TEST", "1Sec", "Tick"
data_type = [('Epoch', 'i8'), ('Bid', 'f4'), ('Ask', 'f4'), ('Nanoseconds', 'i4')]
tbk = "{}/{}/{}".format(symbol, timeframe, attribute_group)
client = pymkts.Client()

# --- write variable-length records (=multiple records in a single timeframe (1Sec))
data = np.array([
    (pd.Timestamp('2021-01-01 00:00:00').value / 10 ** 9, 10.0, 20.0, 1000000),
    (pd.Timestamp('2021-01-01 00:00:00').value / 10 ** 9, 30.0, 40.0, 2000000),
    (pd.Timestamp('2021-01-01 00:00:00').value / 10 ** 9, 50.0, 60.0, 3000000),
], dtype=data_type)
client.write(data, tbk, isvariablelength=True)

# --- query variable-length records
params = pymkts.Params(symbol, timeframe, attribute_group)
print(client.query(params=params).first().df())

# --- tearDown
client.destroy(tbk)
```
shows
```
                            Bid Ask Nanoseconds
Epoch
2021-01-01 00:00:00+00:00 10.0 20.0 1000000
2021-01-01 00:00:00+00:00 30.0 40.0 2000000
2021-01-01 00:00:00+00:00 50.0 60.0 3000000
```

Because the data type of `Epoch` column is always set as 'i8'(=int64) and
 it's not sufficient for describing a date with sub-second accuracy,
the sub-second information is stored in another column (=`Nanoseconds` column, the data type is 'i4') in marketstore.

### Command-line
Connect to a marketstore instance with
```
// For a local db-
marketstore connect --dir <path>
// For a server-
marketstore connect --url <address>
```
and run commands through the sql session.

## Plugins
Go plugin architecture works best with Go1.10+ on linux. For more on plugins, see the [plugins package](./plugins/) Some featured plugins are covered here -

### Streaming
You can receive realtime bars updates through the WebSocket streaming feature. The
db server accepts a WebSocket connection on `/ws`, and we have built a plugin that
pushes the data.  Take a look at [the package](./contrib/stream/)
for more details.

### GDAX Data Feeder
The batteries are included, so you can start pulling crypto price data from [GDAX](https://docs.gdax.com/#get-historic-rates)
right after you install MarketStore. Then you can query DataFrame content
over the network at as low latency as your local HDF5 files from disk, and
appending new data to the end is two orders of magnitude faster than
DataFrame would be.  This is because the storage format is optimized for
the type of data and use cases as well as for modern filesystem/hardware
characteristics.

You can start pulling data from GDAX if you configure the data poller.
For more information, see [the package](./contrib/gdaxfeeder/)

### On-Disk Aggregation
This plugin allows you to only worry about writing tick/minute level data. This plugin handles time-based aggregation
on disk. For more, see [the package](./contrib/ondiskagg/)

## Replication
You can replicate data from a master marketstore instance to other marketstore instances.
In `mkts.yml` config file, please set the config as the following:

- master instance
```
replication:
  # when enabled=true, this instance works as master instance and accept connections from replicas
  enabled: true
  # when tls_enabled=true, transport security between master and replica is enabled.
  # tls_enabled: true # both master and replica should have tls_enabled=true to enable TLS
  # public/private key pair from a pair of files. The files must contain PEM encoded data.
  # The cert file may contain intermediate certificates following the leaf certificate to form a certificate chain.
  # cert_file: "/Users/dakimura/projects/misks/tmpcert/server.crt" # both master and replica should have this config to enable TLS
  # key_file: "/Users/dakimura/projects/misks/tmpcert/server.key"
  # port to be used for the replication protocol
  listen_port: 5996
```

- replica instance(s)
```
replication:
  # when master_host is set, this instance works as a replica instance
  master_host: "<IP_ADDRESS>:5995"
  # when tls_enabled=true on master server, GRPC communication between master and replica is encrypted by SSL.
  # tls_enabled: true
  # cert_file: "/Users/dakimura/projects/misks/tmpcert/server.crt" # both master and replica should have this config to enable TLS

```

### limitations
- Currently, the replication connection is initialized only at a startup of a marketstore replica instance.
Please be sure to start the master instance first when you want to replicate data.

- Currently, only `write` API is supported. `delete` API result won't be reflected to replica instances.

- When replication is enabled on a replica instance, the instance is set to read-only mode and  write API call(s) to the instance will fail.

## Development
If you are interested in improving MarketStore, you are more than welcome! Just file issues or requests in GitHub or contact<EMAIL_ADDRESS>Before opening a PR please be sure tests pass-

``` sh
make unittest
```

### Plugins Development
We know the needs and requirements in this space are diverse.  MarketStore
provides strong core functionality with flexible plug-in architecture.
If you want to build your own, look around [plugins](./plugins/)

repository: alpacahq/marketstore

repo_name: alpacahq/marketstore


text:
# Tutorial 3 - Metaverse Identifiable Token (MIT)

In this tutorial you will learn how to

* What an MIT is
* how to customize and issue an MIT from your avatar
* Transfer MIT's programatically
* Verify a hash stored in the MIT's content
* Integrate MIT's into your app

## Introduction
What are MIT's

MIT stands for Metaverse Identifiable Token. MIT is Metaverse's non-fungible token standard. You can think of them as single Issue Assets with a unique identifying symbol. MIT's can also store a content string.

What are they used for
MIT's can be used to:
* represent real world assets
* represent virtual assets
* store data on the Metaverse Blockchain such as cryptographic hashes

MIT's can be transferred between avatars. Only Avatars can issue MIT's

## Hands on Tutorial

First lets create an html front end

```

<!DOCTYPE html>
<html lang="en" dir="ltr">
  <head>
    <meta charset="utf-8">
    <title></title>
  </head>
  <body>

    <h3> Issue MIT </h3>

    <input placeholder = "MIT Symbol" ></input><br>
    <input placeholder = "Content" ></input><br>
    <select>Avatar</select><br>
    <label placeholder = "recipient Address" ></label>

    <h3> Display MITs</h3>

    <th>MST</th>
    <th>Issuer</th>
    <th>Symbol</th>
    <th>Content</th>

    <h3> Transfer MIT </h3>

    <select>MIT</select><br>
    <input>AVATAR</input><br>
    <button>Send</button><br>

    <h3> Verify File </h3>

    <label>upload</label><br>
    <select>MIT</select><br>

    <button>Verify</button>

  </body>
</html>

```

## NodeJS

Create a function to issue an MIT

```
let address = wallet.getAddress(1)
let address2 = wallet.getAddress(0)
console.log(address)
console.log(address2)
let recipient_address =  wallet.getAddress(1)
let change_address = wallet.getAddress(1)

let height = await blockchain.height()
let txs = await blockchain.addresses.txs([address])
let utxos = await Metaverse.output.calculateUtxo(txs.transactions, [address]) //Get all utxo
let result = await Metaverse.output.findUtxo(utxos, target, height) //Collect utxo for given target
let tx = await Metaverse.transaction_builder.registerMIT(result.utxo, recipient_address, description, symbol, content, change_address, result.change)
tx = await wallet.sign(tx)
tx = await tx.encode()
tx = await blockchain.transaction.broadcast(tx.toString('hex'))
console.log(tx)
```

Create a function to Transfer an MIT

```
let wallet = await Metaverse.wallet.fromMnemonic(mnemonic, 'testnet')
let address = wallet.getAddress(0)
let height = await blockchain.height()
let txs = await blockchain.addresses.txs(address)
let utxos Metaverse.output.calculateUtxo(txs.transactions, [address])) //Get all utxo
let results = await Promise.all([
                  Metaverse.output.findUtxo(utxos, {}, height),
                  Metaverse.output.filter(utxos, {
                      type: "mit"
                  })
              ])
let tx = await Metaverse.transaction_builder.transferMIT(results[0].utxo.concat(results[1]), "nova", recipient_address, recipient_avatar, "MIT_SUPERNOVA", change_address, results[0].change))
tx = await wallet.sign(tx))
tx = await tx.encode())
tx = await blockchain.transaction.broadcast(tx.toString('hex')))
console.log(tx)

```

Create a function to verify a MIT content hash
```
```

Integrate into your Dapp

To interact with metaversejs in your webapp, you need to reference metaversejs in your HTML.

```
<script type="text/javascript" src="/dist/metaverse.min.js"></script>
```

Also reference your tut3.js file.

```
<script type="text/javascript" src="tut3.js"></script>
```

Verify that you have connected metaverse to the webapp by opening the browser console and typing "Metaverse". You should see the Metaverse object come up and look something like

''' '''

Next connect elements to the js functions and youre done!

## Whats next?

You have completed the Metaverse Tutorial series. For further information checkout the [examples](https://github.com/antron3000/MetaverseTutorials/tree/master/examples) section or read the official [Metaverse Documentation](https://docs.mvs.org).

Join the Metaverse developer community today and build amazing dapps.

repository: msgpo/MetaverseTutorials

repo_name: msgpo/MetaverseTutorials


text:
---
layout: default
---

# 1995-07-19 - Re: Anti-Electronic Racketeering Act of 1995 (fwd)

## Header Data

From: Ray Arachelian<EMAIL_ADDRESS>To: Bill Stewart<EMAIL_ADDRESS>Message Hash: da8d5c9d8eb5d3e175bec8751757368d84914959852bbcee265624c4a8a97ae6

Message ID<EMAIL_ADDRESS>Reply To<EMAIL_ADDRESS>UTC Datetime: 1995-07-19 21:51:48 UTC

Raw Date: Wed, 19 Jul 95 14:51:48 PDT

## Raw message

```
{% raw  %}From: Ray Arachelian<EMAIL_ADDRESS>Date: Wed, 19 Jul 95 14:51:48 PDT
To: Bill Stewart<EMAIL_ADDRESS>Subject: Re: Anti-Electronic Racketeering Act of 1995 (fwd)
In-Reply-To<EMAIL_ADDRESS>Message-ID<EMAIL_ADDRESS>MIME-Version: 1.0
Content-Type: text/plain

On Thu, 13 Jul 1995, Bill Stewart wrote:

> >Eh, what do "virus hackers" have to do with encryption, why is it these
> >morons justify the destruction of encryption by mentioning hackers and
> >viruses?
>
> You're parsing the title wrong.  It's an act to support racketeering
> through opposition to electronic communications.  What viruses have to
> do with encryption is that encryption makes it easier to prevent viruses,
> and Senator Grassley wants to stop that.  And the term "strong" was used in its
> correct engineering meaning, as in "It's a vessel of fertilizer which is
> very strong and promotes growth".

Erm, not quite.  Stealth viruses supposedly use "encryption" to hide
themselves, but then, I shouldn't mention this, might give El Federale a
bit more fuel to burn us with.  (But even these beasts can be caught
easily if you know how... i.e. create a large executable that does
nothing but quit to the operating system.  Run it every day and compare
it every day.  The day it changes is the day a virus infected it.)

Still, you could write beneficial viruses, or virus like programs that
are beneficial in nature in some way. KOH for instance?

However, none of the above has any iota of anything to do with linking
the four horsemen of LEA's to crypto in any real-life-already-proven
situation in any significant numbers.  Banning crypto for EVERYONE in
order to catch maybe, what, two zit-bearing kids hoarding beaver shots
downloaded from alt.bin.erotica.pix a year is a tremendous loss of
everyone's privacy.

> Hey, Julf, we've got your number!  And we're making sure nobody's got any
> encryption to prevent fraud with.

Hell, at this point, my guess is that the mafia(s) doesn't use crypto, or
that if it does, it can be caught via other means.  A strong, well
developed crypto system in use by the mafia would more than likely never
happen... not until mobsters get into computers.  Ditto for terrorists.

If they did use crypto, I suspect they wouldn't get caught.  (For the
paranoid, assuming they used crypto, and they didn't get caught, then the
FBI or other TLA is doing the same as the gov't in Farenheight 451...
pick someone else, and jail them.  Otherwise, how do you explain all the
jailbird mobsters?)

  Stop attacking easy targets.  Do your jobs.  Confront the
real criminals.  What's the matter?  Is it easier to go after crypto
geeks than it is to arrest drug dealers who shoot back?

=================================================================93=======
 + ^ + |  Ray Arachelian | Amerika: The land of the Freeh. | \-_    _-/  |
  \|/<EMAIL_ADDRESS>Where day by day, yet another   |  \  --  /   |
<--+-->|                 | Constitutional right vanishes.  |6 _\-  -/_  6|
  /|\  |    Just Say     |                                 |----\  /---- |
 + v + | "No" to the NSA!| Jail the censor, not the author!|     \/      |
=======/---------------------------------------------------------VI------/
      /  I watched and weeped as the Exon bill passed, knowing that yet /
     / another freedom vanished before my eyes.  How soon before we see/
    /a full scale dictatorship in the name of decency? While the rest /
   /of_the_world_fights_FOR_freedom,_our_gov'ment_fights_our_freedom_/

{% endraw %}
```

## Thread

+ Return to [July 1995](/archive/1995/07)

+ Return to "[Ray Arachelian <sunder@escape.com>](/authors/ray_arachelian_sunder_at_escape_com_)"
+ Return to<EMAIL_ADDRESS>(Bill Stewart)](/authors/stewarts_at_ix_netcom_com_bill_stewart_)"

+ 1995-07-14 (Thu, 13 Jul 95 23:33:00 PDT) - [Re: Anti-Electronic Racketeering Act of 1995 (fwd)](/archive/1995/07/9a2d6eed21b887e112c3943e88d84b24bc5b0bcb46419dc45af9e5d7d7cf9e61) -<EMAIL_ADDRESS>(Bill Stewart)_
  + 1995-07-19 (Wed, 19 Jul 95 14:51:48 PDT) - Re: Anti-Electronic Racketeering Act of 1995 (fwd) - _Ray Arachelian<EMAIL_ADDRESS>


text:
---
schema: pacelf
title: Regulation of the immune response in lymphatic filariasis and onchocerciasis
organization: King, C.L., Nutman, T.B.
notes: The persistence of microfilariae in the blood or skin accompanied by a prominent eosinophilia and elevated serum IgE levels are common features of human infection with filarial parasites. In this review Christopher King and Thomas Nutman discuss recent findings on the role of interleukin 4 (IL-4), IL-5 and gamma-interferon (IFN-γ) in the induction of these immediate hypersensitivity responses. They discuss the role of hypersensitivity in immunity, the development of immune tolerance to filarial antigens and suggest that could explain the impaired immune response of some individuals to filarial infections and the persistence of the microfilaremic state.
access: Restricted

resources:
- name: Regulation of the immune response in lymphatic filariasis and onchocerciasis
  url: '/docs/MULTICOUNTRY_GlobalKing__Nutman_1991_Immunoparasitology_today_1-s2.0-S0167569905800167-main.txt'
  format: Electronic
  access: Restricted
  pages: A54-A8

category: Scientific Papers
access: Restricted
journal: Immunology Today
publisher: N/A
language: English
hardcopy_location: N/A
work_location: Multicountry Global
year: 1991
decade: 1990
PacELF_ID: 754
---

repository: DanielBaird/jkan

repo_name: DanielBaird/jkan


text:
# About
This repository contains an ANTLR4 grammar for the logic input languages provided by the TPTP [1].

The grammar was successfully tested on all valid syntactical problems of the TPTP.

# Contents
* tptp_v7_0_0_0.g4

    The actual grammar.

* test

    Contains code to test the grammar on a set of TPTP problems.

* test_cases

    Contains additional test cases for (unofficial) non-classical logics.

[1] http://www.tptp.org/


text:
# Action SuperCross Python Library

There's no description, no documentation, no installation procedure, no usage examples, no comments in code and no tests. Deal with it.

## Known limitations

Due to a recent bug in construct library replays are always saved in Across 1.2 format, even if they are explicitly set to be Across 1.0. Construct 2.8.8 should fix this problem.

repository: abulaatikko/across

repo_name: abulaatikko/across


text:
# Pretty Much Everything
Telling the story of his work and how it all came to be pieces of his book, from Field Notes to #thicklines

@draplin on Twitter and Instagram

Buy the book. https://www.amazon.ca/gp/aw/d/1419720171/ref=mp_s_a_1_1?ie=UTF8&qid=1492958704&sr=8-1&pi=AC_SX236_SY340_FMwebp_QL65&keywords=pretty+much+everything&dpPl=1&dpID=61%2BweZRVDUL&ref=plSrch

1.

2. Contract Negotiations

“I was never concerned with how much I was gonna make.”

3. 1,500 Books

“I made a point of meeting every single person who worked on the book and thanking them.”

“Pack in as much as possible.  I **got** to be in there all day.”

4.  We were making progress and we took a big combover step back. But Leigh knows what to do. She uses creativity and wit to fight back.” Leigh’s project can be found at @_notes_to_self

- made postcards in response to Donald Trump, ie. Aaron’s fav, “Wait until our periods sync up”

5. DDC Shop Tour

“There’s no boss here.”

“I was told it was gonna suck.”

“I don’t have anymore than anyone here. I work hard.”

“Someone told me ‘you show a lot of secrets’, like it was wrong to show all this stuff.”

6. Accessible By Design

“The book had to be affordable.”

7. Some Numbers

“From the third printing, we got up to 30,000.”

Book now I into it's fourth edition. Over 31000 copies sold.

8. An Update

“Obama admin. used intelligence over force.” Designed Obama stimulus package logo.

Finds himself wondering how to be  creative now that Trump is in.

Made posters for Bernie.

9. Closing

“What’s cooler, the universe having an edge, or not?”

“You could go forever in any direction. That is real, right now. Get cosmic. If I’m having a hard day making logos I just think, the universe is eternal in every direction.”

“Denounce Don Trump whenever you can.”

Twitter/Insta: @draplin


text:
# Conditional Random fields

1. define many feature functions : how likely is current word an adjective ?

F : (sentence, word pos, current word label, prev word label) -> score

2. assign weight to each feature functions

3. transform scores into probabilities by exp and normalize

4. train CRF using gradient descent

5. predict label using dynamic programming algo

http://blog.echen.me/2012/01/03/introduction-to-conditional-random-fields/

```
Conditional random fields can be viewed as simplified versions of higher-order, conditional Boltzmann machines in which the hidden units have been eliminated. This makes the learning problem convex, but removes the ability to learn new features (Hinton, Boltzmann machines)
```

# Sutton and McCallum

http://homepages.inf.ed.ac.uk/csutton/publications/crftut-fnt.pdf

# applications

## named entity recognition

```
In HMM, your tag of the current word is dependent only on the tag of the previous word.
And the current word is dependent on the current tag only.

No such constraints for CRF, which makes it more flexible and powerful.
```
https://spoonshot.com/blog/2018/03/10/name-entity-recognition-using-crf/

# packages

pycrf

sklearn-crfsuite

# TODO

Xiao Li SigIR 2009 Extracting Structured Information from User Queries with Semi-Supervised Conditional Random Fields https://www.microsoft.com/en-us/research/publication/extracting-structured-information-from-user-queries-with-semi-supervised-conditional-random-fields/

repository: sanjosh/csdocs

repo_name: sanjosh/csdocs


text:
# Sources

One of the developer tools most powerful features is the sources tab. This section is essentially an integrated development environment which allows you to build, test, and debug web apps.

#### Navigate within the sources tab with the following shortcuts

| key-stroke | Action |
| :---: | --- |
| `ctrl` + `F` | Search text within the current file |
| `ctrl` + `shift` + `F` | Search text accross all files\* |

\* Available in all tabs \(network, sources, elements, etc\)

repository: Rgghgh/chrome-dev-tools

repo_name: Rgghgh/chrome-dev-tools


text:
---
title: OpenVPN on Docker Swarm
---
# OpenVPN

Sometimes you need an OpenVPN tunnel between your docker hosts and some other environment. I needed this to provide connectivity between swarm-deployed services like Home Assistant, and my IOT devices within my home LAN.

OpenVPN is one application which doesn't really work in a swarm-type deployment, since each host will typically require a unique certificate/key to connect to the VPN anyway.

In my case, I needed each docker node to connect via [OpenVPN](http://www.openvpn.org) back to a [pfsense](http://www.pfsense.org) instance, but there were a few gotchas related to OpenVPN at CentOS Atomic which I needed to address first.

## SELinux for OpenVPN

Yes, SELinux. Install a custom policy permitting a docker container to create tun interfaces, like this:

````bash
cat << EOF > docker-openvpn.te
module docker-openvpn 1.0;

require {
 type svirt_lxc_net_t;
 class tun_socket create;
}

#============= svirt_lxc_net_t ==============
allow svirt_lxc_net_t self:tun_socket create;

EOF

checkmodule -M -m -o docker-openvpn.mod docker-openvpn.te
semodule_package -o docker-openvpn.pp -m docker-openvpn.mod
semodule -i docker-openvpn.pp
````

## Insert the tun module

Even with the SELinux policy above, I still need to insert the "tun" module into the running kernel at the host-level, before a docker container can use it to create a tun interface.

Run the following to auto-insert the tun module on boot:

````bash
cat << EOF >> /etc/rc.d/rc.local
# Insert the "tun" module so that the vpn-client container can access /dev/net/tun
/sbin/modprobe tun
EOF
chmod 755 /etc/rc.d/rc.local
````

## Connect the VPN

Finally, for each node, I exported client credentials, and SCP'd them over to the docker node, into /root/my-vpn-configs-here/. I also had to use the NET_ADMIN cap-add parameter, as illustrated below:

````bash
docker run -d --name vpn-client \
  --restart=always --cap-add=NET_ADMIN --net=host \
  --device /dev/net/tun \
  -v /root/my-vpn-configs-here:/vpn:z \
  ekristen/openvpn-client --config /vpn/my-host-config.ovpn
````

Now every time my node boots, it establishes a VPN tunnel back to my pfsense host and (_by using custom configuration directives in OpenVPN_) is assigned a static VPN IP.

repository: geek-cookbook/geek-cookbook

repo_name: geek-cookbook/geek-cookbook


text:
kepler-makelc
=============

These programs are intended for folks who care about Eclipsing Binaries observed by _Kepler_. They may be applicable to a wider set of situations, but that is the original use.

### makelc.py

Deletes NaNs, puts SAP flux data from multiple quarters onto the same median level, and nicely lines up gaps between quarters. Does NOT use any interpolation.

Additionally runs kepcotrend to get a detrended flux, and performs the operations listed above on that, too. So you can use whichever you prefer.

The full light curve (all quarters, and both SAP and detrended (CBV) data) is written to a large text file. Both flux and magnitude units are calculated, and both have error bars.

Works for ANY valid KIC target your heart desires (no need to already have the data)!

### ELClcprep.py

Breaks light curves into 'chunks' for use with Jerry Orosz's modeling program ELC. You need to specify some details about your star in the code before running this program.

### lc_functions.py

Some handy functions that are used in both programs above.

### various .txt and .png files

Examples outputs from the above.

repository: mrawls/kepler-makelc

repo_name: mrawls/kepler-makelc


text:
# GANerator

### Try out, analyze and generate your generators in PyTorch!

An easily usable, performant and extensible cross-platform interface for all
your GAN needs.

Training [Generative Adversarial Networks](https://arxiv.org/abs/1406.2661)
(GANs) is a tiresome and hard task. This is primarily because GAN training has
not been well understood yet even after a lot of research went into creating,
understanding and implementing new methods for regularizing and improving GAN
training (such as [virtual batch
normalization](https://arxiv.org/abs/1606.03498), [spectral
normalization](https://arxiv.org/abs/1802.05957) or
[boundary equilibria](https://arxiv.org/abs/1703.10717)).

In an effort to improve and accelerate research in GAN training, this
repository collects state of the art approaches and best practices in a modular
fashion to enable users to quickly switch between and try out different methods
to find out what works best for their current situation. For that, I provide a
Jupyter Notebook. A pure Python version can be generated from the notebook
where parameters are supplied through command line arguments making automated
experiments much easier.

This framework resulted in an analysis on combinations of normalization
functions for GANs. In one day, 72 GANs in total were trained on 36 combinations
of normalization functions (a small and a large GAN for each combination) and
the [Fréchet Inception Distance](https://arxiv.org/abs/1706.08500) of their
generations was compared. The report is available
[here](gan-normalization-comparison.pdf).

## Current Features

Only late versions of Python 3 are supported.

Supported framework:
- PyTorch

Other:
- True and tested normalizations for GANs (including a [virtual batch
  normalization](https://arxiv.org/abs/1606.03498) layer)
- Experimental setup for testing and saving many different parameters
- Many [GAN hacks](https://github.com/soumith/ganhacks)
- Generate Python source code from the notebook
- Cross platform

This repository offers working multiprocessing on multiple GPUs and CPUs, a
collection of smart parameters that are documented and easily usable. In the
end, you have an understandable environment for GAN training working out of the
box.

## Setup

You will need [NumPy](https://www.numpy.org/),
[Matplotlib](https://matplotlib.org/) and [PyTorch](https://pytorch.org/)
(including Torchvision). If you want to use the [Jupyter](https://jupyter.org/)
notebook, install that as well (otherwise generate the source code as described
below).

To save animations of your GAN training, you may want either
[FFmpeg](https://ffmpeg.org/) (mp4) or [Pillow](https://python-pillow.org/)
(gif).

Using [Anaconda](https://www.anaconda.com/), everything except for PyTorch and
FFmpeg can be installed in one line:
```zsh
conda install numpy matplotlib jupyter pillow
```

I do not offer installation instructions or a setup file due to the differences
between CPU and GPU packages and CUDA versions for PyTorch.
Please look that stuff up yourself in the following links.

Installation instructions for:
- [NumPy](https://www.scipy.org/install.html)
- [Matplotlib](https://matplotlib.org/users/installing.html)
- [PyTorch](https://pytorch.org/get-started/locally/)
- [Jupyter](https://jupyter.org/install)
- [Pillow](https://pillow.readthedocs.io/en/stable/installation.html)

## Generating source code and running experiments

You can generate source code from the Jupyter notebook. Any parameter will be
passable as command line arguments, where the default values are the ones
currently in the notebook.

For the parameters, there are some rules you need to follow to generate source
code. These are listed in `src/ipynb_to_py.py`.

To generate, simply execute the following:
```zsh
./src/ipynb_to_py.py
```

If you then want to start the experiments, modify and execute
`src/run_experiments.py`. Edit the `test_params` dictionary in that file and
enter the following in your command line of choice:
```zsh
./src/run_experiments.py --debug
```

That only started a dry run. To start the tests for real, omit the `--debug`
argument and execute again to see your computer go to work.

A `--distributed` flag is planned that will start each test on a separately
created cloud machine. That, however, is still a TODO and will only support
one cloud service provider out of the box.

## Experimental dataset

The system was set up using
[CelebA](http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html), but for the
experiments, [FFHQ](https://github.com/NVlabs/ffhq-dataset) will be used.

### Coding style

At the moment, in the notebook, I do not adhere to the PEP 8 line size
guideline. I feel that notebooks are in most cases supposed to be in fullscreen
and readability would suffer, especially concerning documentation. Sorry! If
there are enough complaints or I change my mind, I will modify the code
accordingly.

Sorry for not enough documentation, that will change with time.

## TODOs

Methods to implement (always growing):
- feature matching
- minibatch discrimination
- historical averaging
- bounded equilibrium GANs
- some missing stuff from [ganhacks](https://github.com/soumith/ganhacks)

Evaluation methods:
- semi-supervised learning

Other:
- more documentation of methods
- automatic experiments for parameter combinations
- convolutional layer visualization
- distributed computing version
- grid search supporting cloud computing providers
- more datasets
- data other than images (sound, 3D objects, ...)
- arbitrary data (anything else)
- [CelebA-HQ](https://arxiv.org/abs/1710.10196) generation
- perhaps more frameworks
- Julia version

## Papers

This is a collection of the papers used for this project.

- [Generative Adversarial Nets](https://arxiv.org/abs/1406.2661)
- [Unsupervised Representation Learning with Deep Convolutional Generative
  Adversarial Networks](https://arxiv.org/abs/1511.06434)
- [Improved Techniques for Training GANs](https://arxiv.org/abs/1606.03498)
- [BEGAN: Boundary Equilibrium Generative Adversarial
  Networks](https://arxiv.org/abs/1703.10717)
- [Spectral Normalization for Generative Adversarial
  Networks](https://arxiv.org/abs/1802.05957)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain
  Image-to-Image Translation](https://arxiv.org/abs/1711.09020)
- [Progressive Growing of GANs for Improved Quality, Stability, and
  Variation](https://arxiv.org/abs/1710.10196)
- [Conditional Generative Adversarial Networks](https://arxiv.org/abs/1411.1784)
- [Large Scale GAN Training For High Fidelity Natural Image
  Synthesis](https://arxiv.org/abs/1809.11096)

## Other resources

- [PyTorch DCGAN
  Tutorial](https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html)
  (which a lot is based on)
- [ganhacks](https://github.com/soumith/ganhacks)

repository: janEbert/GANerator

repo_name: janEbert/GANerator


text:
# Webapp helpers

Common JS and CSS for web applications.

## Features

- Load Google fonts asynchronously (+ Material design icons).
- Try to define user language and populate the associated input.
- Back to top link.
- Smooth scroll when user click internal links.
- Hide URL hash when user click internal links.
- Save input values to a data attribute for live reset.
- Make form fields persistent (require Basil.js).
- Watch the reset button to clear cookies / local storage.
- Add a .disabled class to unsupported field types.
- Add a simple parallax function.
- Load the Google Adsense library then add a class allowing to unhide ad containers.
- Show a lot of debug information in console including some  plugins (Garlic, Parsley, Mixitup, i18next, jQuery UI sortable).

## Install

Download with Bower.

```
bower install chrisbo246/webapp-helpers
```

Load **webapp-helpers.js** with a script tag in your main .html.

```
<script src="/bower_components/webapp-helpers/dist/scripts/webapp-helpers.js"></script>
```

Edit your main.scss and import **webapp-helpers.scss**.

```
@import "app/styles/webapp-helpers";
```

## Usage

You can eventually customize the default settings to change some selectors or disable some functionalities.

<!--

## Development

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.

### Prerequisites

Install [Node](https://nodejs.org/en/download/) on your local machine then download dependencies.

```
npm install -g gulp-cli bower yo generator-webapp
```

### Installing

Install NPM and Bower packages.

```
npm install
bower install
```

### Testing

```
gulp serve
```

[http://localhost:9000/](http://localhost:9000/)

## Deployment

```
gulp build
```

Then upload the **dist** directory content to your web server.

## Contributing

Please read [CONTRIBUTING.md](CONTRIBUTING.md) for details on our code of conduct, and the process for submitting pull requests to us.

-->

## Bugs

Please use the [GitHub issue tracker](https://github.com/chrisbo246/webapp-helpers/issues) for all bugs and feature requests. Before creating a new issue, do a quick search to see if the problem has been reported already.

## Author

[chrisbo246](https://github.com/chrisbo246)

See also the list of [contributors](https://github.com/chrisbo246/webapp-helpers/contributors) who participated in this project.

## License

This project is licensed under the MIT License. See the [opensource.org](https://opensource.org/licenses/MIT) website for details.


text:
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) [![Build status](https://ci.appveyor.com/api/projects/status/9lfkr80jahecns88?svg=true)](https://ci.appveyor.com/project/nenitiko/vueclispaextensions) [![NuGet Badge](https://buildstats.info/nuget/numind.aspnetcore.spaservices.extensions)](https://www.nuget.org/packages/Numind.AspNetCore.SpaServices.Extensions)
[![numind-aspnetcore MyGet Build Status](https://www.myget.org/BuildSource/Badge/numind-aspnetcore?identifier=0820550d-1cad-49bb-8036-b26826b88476)](https://www.myget.org/)

# A Vue-Cli support for AspNetCore SPAs

Allows use of vue-cli-service in ASPNetCore Spa Applications by supporting a development experience similar to both UseReactDevelopmentServer & UseAngularCliServer.
See: https://www.nuget.org/packages/Microsoft.AspNetCore.SpaServices.Extensions/
This library supports both NPM and Yarn package managers.

## Usage

The ASPNET Core framework has support for a new setup when creating SPA pplications, using the default templates generated code which uses the new Microsoft.AspNetCore.SpaServices.Extensions package. Both React and Angular development setups are supported, the code in Startup.cs looks somewhat like this:

```csharp
        public void Configure(IApplicationBuilder app, IHostingEnvironment env)
        {
            ...
            app.UseSpa(spa =>
            {
                spa.Options.SourcePath = "ClientApp";

                if (env.IsDevelopment())
                {
                    // Angular
                    spa.UseAngularCliServer(npmScript: "start");
                    ...
                    // react
                    spa.UseReactDevelopmentServer(npmScript: "start");
                }
            });
        }
```

This package adds a new extension method `UseVueCliDevelopmentServer` and support for Yarn Package Manager by way of a new parameter which default to NPM.

```csharp
        public void Configure(IApplicationBuilder app, IHostingEnvironment env)
        {
            ...
            app.UseSpa(spa =>
            {
                spa.Options.SourcePath = "ClientApp";

                if (env.IsDevelopment())
                {
                    spa.UseVueCliDevelopmentServer(npmScript: "start", packageManager: PackageManager.Yarn);
                }
            });
        }
```

UseVueCliDevelopmentServer will call the run command (ex: `yarn run start`) of the specified package manager in `packageManager:`  argument.

Your `package.json` script section should look like the following in order for the vue-cli development server to launch:

```javascript
  "scripts": {
    "start": "vue-cli-service serve",
    ...
  }
```

## License
https://www.apache.org/licenses/LICENSE-2.0

repository: numindsoftware/VueCliSpaExtensions

repo_name: numindsoftware/VueCliSpaExtensions


text:
# Contributing

When contributing to this repository, please first discuss the change you wish to make via an issue with the owners of this repository before making a change.

Please note we have a code of conduct, please follow it in all your interactions with the project.

We love your input! We want to make contributing to this project as easy and transparent as possible, whether it's:

- Reporting a bug
- Discussing the current state of the code
- Submitting a fix
- Proposing new features
- Becoming a maintainer

## We Develop with Github
We use github to host code, to track issues and feature requests, as well as accept pull requests. Pull requests are the best way to propose changes to the codebase (we use [Github Flow](https://guides.github.com/introduction/flow/index.html)). We actively welcome your pull requests.

## Pull Request Process

1. Open a new issue if one doesn't already exist for the pull request.
2. Check that a pull request doesn't already exist for the issue you are referencing.
3. Wait to be assigned the issue before creating a pull request.

## Report bugs using Github's [issues](https://github.com/Aveek-Saha/MusicPlayer/issues)
We use GitHub issues to track public bugs. Report a bug by [opening a new issue](); it's that easy!

Use the Bug report template when opening an issue for a bug report.

repository: Aveek-Saha/js-data-structs

repo_name: Aveek-Saha/js-data-structs


text:
# Michelle Keegan launches new party collection with Very for Christmas 2019

Published at: **2019-11-06T17:52:36+00:00**

Author: **Melisha Kaur, https://www.mirror.co.uk/authors/melisha-kaur/**

Original: [mirror](https://www.mirror.co.uk/3am/style/celebrity-fashion/michelle-keegan-very-dresses-christmas-20826336)

She is loved by the nation not only for her acting skills, but for her immaculate beauty and style.
And Michelle Keegan is looking stunning as ever as she stars in her latest collection for online retailer Very, which recently launched on very.co.uk in time for the Christmas party season.
The 32-year-old British actress, who best known for starring in Coronation Street, poses in a range of styles from the partywear collection - including a glitzy cobalt midi dress (£35).
There are 25 items to choose from in Michelle's new Very clothing range, with prices starting from just £25.
The brunette bombshell looks undeniably glamorous in a black midi-dress with a plunging neckline (£35).
Adding some colour to the range, the actress can also be seen flaunting her enviable figure in a bright green mini-dress with puffed sleeves and sweetheart neckline (£65).
Speaking of the new line, Michelle said: “This collection is all about being bolder with your style and standing out from the crowd!
"I wanted to incorporate statement sequin looks alongside more pared back tailored pieces to make sure that there is something for everyone to look sensational this party season!”
As well as stunning partywear pieces, the new Michelle Keegan for Very range includes cosy winter items like a Premium Faux Fur Coat in navy (£75) and a Long Sleeve Denim Bodycon Dress in black (£45).
The British star recently revealed that the beauty trick she uses to achieve her flawless appearance is LVL lash lifts.
Looking for more party outfits? Here's our roundup of the best little black dresses for every occasion under £60.

license: Unlicense

repository: dsadlollioygjmzvcxv/tdc-exported

repo_name: dsadlollioygjmzvcxv/tdc-exported


text:
How to use this Vagrantfile
---

Just execute:

```
    vagrant up
```

It may take some time depending on your internet connection.

You can modify the domain and regex for the domain used in the Vagrantfile:

```ruby
    domain: "runbot.vauxoo.com",
    regex_domain: "~^(.*)\.runbot\.vauxoo\.colima$"
```

It'll be replaced in the nginx configuration file. By default creates a VM with 2GB of ram, this also can be changed in:

```ruby
    vb.memory = "2048"
```

The provisioning is made by the ansible files located in the ansible folder, they are separated
by application (instance, docker, postgres and nginx) so it is easier to debug.