text:
# __Python__ Science _"Pipeline"/"Flow"_

This document is created to link my personal files, and locate faster some resources, concepts, codes that I found inside the books in previous lectures. It tends to guide my AI team easy and faster to resources very spread on Internet.

The order here is important, is the way in which I see the process:

#### About _Science Pipelines_: [Python Data Sci Essentials, Chapter 3](./SCIENCE/2015_Boschetti_and_Massaron-Py_Data_Sci_Essentials.pdf)

#### About _Machine Learning_: [Practical Machine Learning](./04_ML/2018_Practical_ML_with_Py.pdf)

Contain the basics on ML. The second part is a broad description of ML pipeline. And the last part is well detailed (by chapter) exposition of Real World Case Studies.

## General Pipeline
- Collecting Data
- Data Munging
- Data Wrangling
- Data Analysis/Exploring
    1. Feature Extraction
    2. Dimensionality Reduction
    3. Outliers Treatment
    4. Testing & Validation
    5. Hyper-param optimization
    6. Feature Selection
    2. Data Plotting/Visualization
- Building Models
- Model Evaluation
- [__Deploying Models into Apps__](./04_ML/2015_Sebastian_Raschka-Py_ML.pdf)
- Monitoring Models (Feedback)

## Baselines Methods / Important Knowledge
* About Machine Learning
    1. _Supervised_ __Classification__
    2. _Semi-Supervised_ [__Reinforcement Learning__](./05_Deep_Learning/2018_Reinforcement_Learning.pdf)
    3. _Unsupervised_ __Clustering__, __Anomaly detection__
* About Software Architecture
    1. Microservices
* Any other?

## Applications - Products
- Text Classifier
- __Recommendation Systems__
    1. [Content & Collaborative Filtering](./04_ML/2016_ML_for_the_web/ML_4the_Web.pdf)
    2. [Afinity Analysis: Apriori algthm](./04_ML/2015_Learning_Data_Mining/2015_Learning_Data_Mining_with_Py.pdf)
    3. [The million Song Dataset Example](./04_ML/2018_Practical_ML_with_Py.pdf)

# Technologies

### Big Data Management System
* Google BigQuery

### Text Wrangling
* __Ppal__:  Google Cloud Natural Language: Text Analysis Service + Google Deep Learning Models
* __Under review__: Google Cloud DataProc (Hadoop & Apache Spark jobs)
* __Under review__: Firebase ML Toolkit
* __Alt to study__: Spacy

### Data Analysis
* __Ppal__: Google Cloud DataLab
* __Alt to study__: Sklearn
* __Under review__: Firebase ML Toolkit

### Visualization
* __to study__: Google Cloud DataLab
* __to study__: Google Visualization API
    1. [Charts](https://developers.google.com/chart/interactive/docs/more_charts.html)
* __Alt to study__: Altair [_notebook collection_](../pyVideosData/06_Visualization/SCI_PLOT_Altair_2018_PyCon-Jake_VanderPlas-Exploratory_Data_Visualization_with_Vega_Vega-Lite_and_Altair/notebooks)

### Building Deep Learning Model
* __Ppal__: [TensorFlow](./04_ML/2017_Hands_on_ML/2017_Hands_On_ML_with_Sklearn_and_Tf.pdf)
* __Model Visualization__: Tensorboard
* __Alt to study__: Keras or PyTorch

### Deployment Models
* __Ppal__: [TensorFlow-server
* __Utility Apps__: Docker, Kubernetes

[__Deploying Models into Apps__](./04_ML/2015_Sebastian_Raschka-Py_ML.pdf)

### Web UI:
* Angular

## Pipeline Introspection

- Models Deployment:
    1. [Model Deployment as a Service](./04_ML/2018_Practical_ML_with_Py.pdf)
- Monitoring
    1. [Strategies to managing model robustness](./04_ML/2016_Advanced_ML.pdf)

## Standards & Protocols

#### DataTypes to watch
- HDFS Hadoop Distributed File System
- CSV
- [HDF5](./01_Numpy/2019_Numerical_Python_2ndE.pdf)
- pandas.DataFrame

## Learning Resources

#### Links
- [Google DeepMind Team](https://deepmind.com/)
- [Kaggle Blog](http://blog.kaggle.com) especialized on ML topics
- About [Netflix Recommendation System Algorithm](http://techblog.netflix.com/2012/04/netflix-recommendations-beyond-5-stars.html)
- [DataBricks Blog](https://databricks.com/blog)
- [Numfocus](https://numfocus.org)

#### Interesting Articles
- [How to organize a Data Science Team](https://multithreaded.stitchfix.com/blog/2016/03/16/engineers-shouldnt-write-etl/)
- [Techs for small DataScience Team](https://towardsdatascience.com/everything-a-data-scientist-should-know-about-data-management-6877788c6a42)

#### Basic Reading
- About [Google Cloud Services](https://en.wikipedia.org/wiki/Google_Cloud_Platform)
- [Google Movil App Platform Firebase](https://en.wikipedia.org/wiki/Firebase)

### Future Knowledge to Investigate
- Hierarchical Temporal Memory (HTM)
- Cascade Forests
- Capsule Networks (CapsNets),...

#### Events

- TensorFlow Dev SUMMIT
- Google Cloud Next
- PyData
- Europython
- SciPy
- PyCon

#### Users to follow on Twitter


text:
Peerplays-UI
============

### Getting started

Peerplays-UI depends on Node.js. While it should work using versions as old as 0.12.x, it is recommended to use v5.x.

On Ubuntu and OSX, the easiest way to install Node is to use the [Node Version Manager](https://github.com/creationix/nvm).
For Windows users there is [NVM-Windows](https://github.com/coreybutler/nvm-windows).

To install NVM for Linux/OSX, simply copy paste the following in a terminal:
```
curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.30.2/install.sh | bash
nvm install v5
nvm use v5
```

Once you have Node installed, you can clone the repo:
```
git clone https://github.com/PBSA/peerplays-core-gui
cd peerplays-core-gui
```

Before launching the GUI you will need to install the npm packages for each subdirectory:
```
cd dl; npm install
cd ../web; npm install
```

## Running the dev server

Once all the packages have been installed you can start the development server by going to the ./web folder and running:
```
npm start
```

Once the compilation is done the GUI will be available in your browser at: localhost:8082. Hot Reloading is enabled so the browser will live update as you edit the source files.

## Production
If you'd like to host your own wallet somewhere, you should create a production build and host it using NGINX or Apache. In order to create a prod bundle, simply run the following command:
```
npm run build
```
This will create a bundle in the /dist folder that can be hosted with the web server of your choice.

license: MIT

repository: TheTaconator/peerplays-core-gui

branch_name: refs/heads/master

extension: md

license_type: permissive

repo_name: TheTaconator/peerplays-core-gui


text:
---
title: Shift in Alcohol Consumption, charting a new future for drinking.
date: "2021-02-06T22:40:32.169Z"
template: "post"
draft: false
slug: "shift-in-alcohol-consumption-charting-a-new-future"
category: "Data"
tags:
  - "Python"
  - "hierarchical clustering"
  - "treemap"
description: "THE world drank the equivalent of 4.7 litres of pure alcohol per person in 2010. That’s the result of skimming into a dataset published by World Health Organisation, Global Information System on Alcohol and Health (GISAH). I also found that drinkings might have gone out of fashion with a steep decline in consumption compared with 5 years ago."
socialImage: "/media/cluster.jpg"
---

THE world drank the equivalent of 4.7 litres of pure alcohol per person in 2010. That’s the result of skimming into a dataset published by World Health Organisation, Global Information System on Alcohol and Health (GISAH).

I also found that drinkings might have gone out of fashion with a steep decline in consumption compared with 5 years ago (Economist reported 6.1 litres of pure alcohol per person in 2005) (https://www.economist.com/graphic-detail/2011/02/14/drinking-habits)

This makes sense. In almost every developed country, teenagers are starting with alcohol intake later than they used to. Part of the reason is the fact that they socialise a lot with each other online, rather than in person. Better parenting may contribute too. With given historical patterns, this later onset of drinking means today’s teenagers will be a generation of light drinkers when they grow up. Millennials, now in their 20s and 30s, are also drinking less alcohol. For them, moderation is part of a healthier lifestyle—the pursuit of which is starting to look like a mainstream trend, rather than a fad.

We now have a realistic hope for a world with much less alcohol.

Looking closer into countries around the world

The biggest boozers are mostly found in Europe while alcohol was least consumed in Africa. Interestingly, the Czechs & French are the most bibulous, getting through 11.8 litres each, nearly half of a litre more than the Germans in second place.

![treemap-drinking.jpg](/media/treemap-drinking.jpg)

## Alcohol Consumption and Life Expectancy

In Africa, there is a negative correlation between Alcohol Consumption and Life Expectancy. The more litres of pure alcohol per person that country consumed, the lower its people’s life timespan.

![drinking-by-continent.jpg](/media/drinking-by-continent.jpg)

## Drinkings Preferences Around the World

Inspired Hannahyan, I also employed hierarchical clustering methodology (hclust) and produced 5 clusters of drinking preferences. I came the same conclusion that countries that are geographically close has similar drinking habit, like we can see Russia, Belarus, Slovakia in one cluster, Switzerland and Luxembourg in one cluster, Belgium, Netherlands, Austria in one cluster.

![cluster.jpg](/media/cluster.jpg)

You can find my notebook code here (https://github.com/Rachelios/A-cup-of-tea-and-a-good-book/tree/master/Alcohol%20Consumption)

*Originally published by [Rachel Le](http://heyiamrachel.com/) on [Medium](https://lenguyenbichngoc95.medium.com/shift-in-alcohol-consumption-ba14a24a10cf).*

repository: Rachelios/rachel-blog

repo_name: Rachelios/rachel-blog


text:
# ExtraCreditsSpring2019GameJam

This repository is result Extra Credits Game Jam #3.
This was completed solo and was unable to submit on time missing the deadline by **5 minutes**. It was personally chosen not to submit to the jam, due to game crashing errors.

The unfinished is the version done by the deadline.  The other copy is the update as how the game has progressed.  The game heavily emphasizes on precalculated **linear interpolation** and **bezier curve** calculations for core mechanics of the player and its patterned movement in its world.

>>Here is the link to the [game page](https://kale-eos.itch.io/the-bezier-effect).
Further work will be returned to at a later (currently unknown) time to fully flesh out the game in it's entirety.

```
This was started on Unity 2018.3.11 and continued on Unity 2019.1.2.  Audio was developed through FL Studio 10.
```

gha_language: C#


text:
Pinger monitors network connectivity via ping monitoring.  It will periodically ping a network address and collect
information on the ping latency.  It tracks the average ping latency as well failure of the network link as
indicated by lack of ping responses.

Pinger is a .Net C# Windows Forms application.

Build it with Visual Studio.


text:
---
title: US Government and Civics
date: '2020-10-24'
tags:
- civics
- khan academy
- learning
- us government
absolute-note-number:
zettel: 21a
---

## Foundations of American Democracy

### The Social Contract

> Do we even need government? Why do we need government?

#### Without it

- unlimited rights
- no laws
  - freeset possible speech, religion, property, movement (go anywhere)
  - tax/take from others
  - just settle on others' land
  - punish/imprison others

#### Problems

Someone else can infringe on your rights. You can infringe on theirs.

> As long as men live without a common power to keep them all in awe, they are in the condition known as 'war'; and it is a war of every man against every man.

- Thomas Hobbes, *Leviathan*, 1861

... in war the two chief virtues are force and fraud

... no such thing as ownership. no distinction between mine and thine.

... every man has a right to everything - even to someone else's body

This was written around the time after British civil war when there was a lot of chaos.

Hobbes was an advocate of a very strong central government.

**Social Contract**. willingness to give up some rights in order to keep others.

"should be contented with as much liberty against other men as he would allow other men against himself"

#### Still a big question

What form should the government take?

## Democratic Ideas of US Government

Notion of natural rights.

Life, liberty, and property - John Locke

what about others?

- physical harm on others
- imprison others
- Tax others

Let's give these rights to the government in order to protect the others.

Ok, so what type should it be?

Throughout history... monarchy.

Or a small group - oligarchy.

Other extreme where the people are sovereign - popular sovereignty (democracy)

Founding fathers were concerned with pure democracy because factions could form and then a majority faction could strip natural rights from others.

Thus...

### Limited Government

(this can exist in other governments besides democracy)

- constitution

- checks and balances

- bill of rights

### Notion of Republic

founding fathers liked to call US a republic.

to some - any gov not a monarchy

to others - democracy, but still protect rights of minorities

founding fathers - something to prevent passions of pure democracy. Representatives. logistically more practical.

#### Federalist Papers

Alexander Hamilton has idea to publish a series of papers.

Writes most. Recruits James Madison and John Jay.

Federalist Papers #10 (most famous) - James Madison discusses how to avoid factions taking over the government and doing things that are not in the interest of the people.

Small quote:

> The two great points of difference between a democracy and a republic are: first, the delegation of the government, in the latter, to a small number of citizens elected by the rest; secondly, the greater number of citizens and a greater sphere of country, over which the latter may be extended.

## Democratic Ideals in the Declaration of Independence

US Constitution replaces Articles of Confederation

life, liberty, and the pursuit of happiness - natural rights

"Among these" -- implies there may be more?

... it is the right of the people to alter or to abolish it, and to institude new government

repository: michaelyuen/personal-website

branch_name: refs/heads/main

gha_language: HTML

repo_name: michaelyuen/personal-website


text:
# react-perf
This is a collection of articles that describe performance issues with React and how to tackle them

<EMAIL_ADDRESS>https://github.com/reactjs/redux/issues/1303
https://github.com/reactjs/react-redux/blob/master/docs/api.md
https://marmelab.com/blog/2017/02/06/react-is-slow-react-is-fast.html
http://benchling.engineering/deep-dive-react-perf-debugging/
https://engineering.musefind.com/how-to-benchmark-react-components-the-quick-and-dirty-guide-f595baf1014c#.pkbvqb54o
https://auth0.com/blog/optimizing-react/#
https://onsen.io/blog/react-state-management-redux-store/
<EMAIL_ADDRESS>https://medium.com/javascript-scene/10-tips-for-better-redux-architecture-69250425af44#.q4nyr8st5
<EMAIL_ADDRESS>http://blog.rangle.io/react-and-redux-performance-with-reselect/
https://tylermcginnis.com/building-user-interfaces-with-pure-functions-and-function-composition-in-react-js/?utm_source=reactnl&utm_medium=email
https://medium.com/@dan_abramov/mixins-are-dead-long-live-higher-order-components-94a0d2f9e750
http://benchling.engineering/deep-dive-react-perf-debugging/
https://github.com/reactjs/reselect#q-how-do-i-test-a-selector
https://hackernoon.com/react-stateless-functional-components-nine-wins-you-might-have-overlooked-997b0d933dbc

repository: ionyRojas/react-perf

repo_name: ionyRojas/react-perf


text:
---
layout: docs
title: Alt Instances
description: Generating instances of your flux universe
permalink: /docs/altInstances/
---

# Creating instances of your Flux

If using a singleton is scary to you, or if you believe dependency injection is the way to go then this is the feature for you. You don't have to settle for traditional flux and its singleton stores. You can create separate instances of the Alt universe and then inject these into your view via dependency injection or if you're using React, contexts. With this approach you get isomorphism for free, without having to worry about flushing your stores, since each request will generate its own instance.

Using instances of alt is fairly straightforward.

```js
class MyAlt extends Alt {
  constructor(config = {}) {
    super(config);

    this.addActions('myActions', ActionCreators);
    this.addStore('storeName', Store);
  }
}

var flux = new MyAlt();
```

# AltClass

## AltClass#constructor

> constructor(config: object) : Alt

The alt constructor takes an optional configuration object. This is where you can configure your alt instance.

```js
var flux = new Alt({
  dispatcher: new MyDispatcher()
});
```
### Config Object

The following properties can be defined on the config object:

#### dispatcher

By default alt uses Facebook's Flux [dispatcher](https://github.com/facebook/flux/blob/master/src/Dispatcher.js), but you can provide your own dispatcher implementation for alt to use. Your dispatcher must have a similar interface to the Facebook dispatcher including, `waitFor`, `register`, and `dispatch`.

One easy way to provide your own dispatcher is to extend the Facebook dispatcher. The following example shows a dispatcher that extends Facebook's dispatcher, but modifies it such that all dispatched payloads are logged to the console and `register` has a custom implementation.

```js
class MyDispatcher extends Dispatcher {
  constructor() {
    super();
  }

  dispatch(payload) {
    console.log(payload);
    super.dispatch(payload);
  }

  register(callback) {
    // custom register implementation
  }
}
```

#### serialize

This controls how store data is serialized in snapshots. By default alt uses `JSON.stringify`, but you can provide your own function to serialize data.

#### deserialize

This controls how store data is deserialized from snapshot/bootstrap data. By default alt uses `JSON.parse`, but you can provide your own function to deserialize data.

## AltClass#addActions

> addActions(actionsName: string, actions: [ActionsClass](createActions.md)): undefined

Creates the actions for this instance.

## AltClass#addStore

> addStore(name: string, store: [StoreModel](createStore.md)): undefined

Creates the store for this instance.

## AltClass#getActions

> getActions(): [Actions](actions.md)

Retrieves the created actions. This becomes useful when you need to bind the actions in the store.

```js
class MyStore {
  constructor() {
    this.bindActions(this.alt.getActions('myActions'));
  }
}
```

## AltClass#getStore

> getStore(): [AltStore](stores.md)

Retrieves the store instance that was created.

```js
var state = alt.getStore('myStore').getState();
```

repository: kledk/alt

repo_name: kledk/alt


text:
---
layout: demopagenew
title: Labs, Tests, and Vitals - Clinical NLP Demos & Notebooks
seotitle: 'Clinical NLP: Labs, Tests, and Vitals - John Snow Labs'
subtitle: Run 300+ live demos and notebooks
full_width: true
permalink: /labs_tests_and_vitals
key: demo
nav_key: demo
article_header:
  type: demo
license: false
mode: immersivebg
show_edit_on_github: false
show_date: false
data:
  sections:
    - secheader: yes
      secheader:
        - subtitle: Labs, Tests, and Vitals - Live Demos & Notebooks
          activemenu: labs_tests_and_vitals
      source: yes
      source:
        - title: Detect demographics and vital signs using rules
          id: detect_demographics_and_vital_signs_using_rules
          image:
              src: /assets/images/Detect_demographics_and_vital_signs_using_rules.svg
          excerpt: Automatically detect demographic information as well as vital signs using our out-of-the-box Spark NLP Contextual Rules. Custom rules are very easy to define and run on your own data.
          actions:
          - text: Live Demo
            type: normal
            url: https://demo.johnsnowlabs.com/healthcare/CONTEXTUAL_PARSER/
          - text: Colab
            type: blue_btn
            url: https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/streamlit_notebooks/healthcare/CONTEXTUAL_PARSER.ipynb
        - title: Detect lab results
          id: detect_lab_results
          image:
              src: /assets/images/Detect_lab_results.svg
          excerpt: Automatically identify Lab test names and Lab results from clinical documents using our pretrained Spark NLP model.
          actions:
          - text: Live Demo
            type: normal
            url: https://demo.johnsnowlabs.com/healthcare/NER_LAB/
          - text: Colab
            type: blue_btn
            url: https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/streamlit_notebooks/healthcare/NER_LAB.ipynb
        - title: Detect clinical events
          id: detect_clinical_events
          image:
              src: /assets/images/Detect_clinical_events.svg
          excerpt: Automatically identify a variety of clinical events such as Problems, Tests, Treatments, Admissions or Discharges, in clinical documents using two of our pretrained Spark NLP models.
          actions:
          - text: Live Demo
            type: normal
            url: https://demo.johnsnowlabs.com/healthcare/NER_EVENTS_CLINICAL
          - text: Colab
            type: blue_btn
            url: https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/streamlit_notebooks/healthcare/NER_EVENTS_CLINICAL.ipynb
        - title: Identify gender using context and medical records
          id: identify_gender_using_context_and_medical_records
          image:
              src: /assets/images/Detect_demographic_information.svg
          excerpt: Identify gender of a person by analyzing signs and symptoms using pretrained Spark NLP Classification model.
          actions:
          - text: Live Demo
            type: normal
            url: https://demo.johnsnowlabs.com/healthcare/CLASSIFICATION_GENDER/
          - text: Colab
            type: blue_btn
            url: https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/streamlit_notebooks/healthcare/CLASSIFICATION_GENDER.ipynb
        - title: Extract neurologic deficits related to NIH Stroke Scale (NIHSS)
          id: extract_neurologic_deficits_relatedNIH_stroke_scale
          image:
              src: /assets/images/Extract_neurologic_deficits_related_NIH_Stroke_Scale.svg
          excerpt: This demo shows how neurologic deficits can be extracted in accordance with their NIH Stroke Scale using a Spark NLP Healthcare NER model.
          actions:
          - text: Live Demo
            type: normal
            url: https://demo.johnsnowlabs.com/healthcare/NER_NIHSS/
          - text: Colab
            type: blue_btn
            url: https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/streamlit_notebooks/healthcare/NER_NIHSS.ipynb
        - title: Identify relations between scale items and measurements according to NIHSS
          id: identify_relations_between_scale_items_their_measurements_according
          image:
              src: /assets/images/Identify_relations_between_scale_items_measurements_according.svg
          excerpt: This demo shows how relations between scale items and their measurements can be identified according to NIHSS guidelines using a Spark NLP Healthcare RE model.
          actions:
          - text: Live Demo
            type: normal
            url: https://demo.johnsnowlabs.com/healthcare/RE_NIHSS/
          - text: Colab
            type: blue_btn
            url: https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/streamlit_notebooks/healthcare/RE_NIHSS.ipynb
---

repository: JohnSnowLabs/spark-nlp

gha_language: Scala

repo_name: JohnSnowLabs/spark-nlp


text:
# ChampEdxFile

A Python 2.7 module to read and manipulate .edx Radiation Patterns output by TICRA's CHAMP horn design software.

Currently this can load simple far field horn radiation patterns, select patterns by frequency and plot
individual radiation patterns by component, cut and frequency.

THIS CODE HAS ONLY BEEN MINIMALLY TESTED

Depends on:
 - numpy
 - lxml.etree
 - io.StringIO
 - matplotlib.pyplot
 - NumpyUtility (see my other repository)

repository: PaulKGrimes/ChampEdxFile

repo_name: PaulKGrimes/ChampEdxFile


text:
## Preface
This provider doesn't cover all the features of routeros. It's just a proof-of-concept.

## Build
```bash
make
```

## Run
- run VM with routeros
    ```bash
    vagrant up --provider=virtualbox
    ```

- make basic config file
    ```bash
    mkdir -p /tmp/terraform/terraform.d/linux_amd64/plugins/

    cat > /tmp/terraform/main.tf << EOF
    provider "ros" {
      address = "<IP_ADDRESS>:8728"
      username = "vagrant"
      password = "vagrant"
    }

    resource "ros_interface_bridge" "bridge0" {
        name = "bridge0"
        fast_forward = true
        disabled = false
    }

    resource "ros_dhcp_server" "dhcp0" {
        name = "dhcp0"
        interface = "${ros_interface_bridge.bridge0.name}"
    }

    resource "ros_dhcp_server_lease" "sample-host" {
        address = "<IP_ADDRESS>"
        server = "${ros_dhcp_server.dhcp0.name}"
        mac = "00:11:22:33:44:55"
        disabled = false
        client_id = "sample-host"
    }
    EOF
    ```
- copy `terraform-provider-routeros` binary in terraform's plugin directory
    ```bash
    make install
    ```

- init and apply terraform config
    ```
    cd /tmp/terraform/
    terraform init
    terraform apply
    ```

## Testing
### Acceptance
It means you have to up vagrant box with routeros.
```
vagrant up --provider=virtualbox
make test
```

## License
MIT


text:
# StegoTool
![StegoTool](assets/stegotool-banner-small.jpg)

## What is Steganography?
Steganography is the technique of hiding secret data within an ordinary, non-secret, file or message in order to avoid detection; the secret data is then extracted at its destination. The use of steganography can be combined with encryption as an extra step for hiding or protecting data.

Steganography can be used to conceal almost any type of digital content, including text, image, video or audio content; the data to be hidden can be hidden inside almost any other type of digital content. The content to be concealed through steganography -- called hidden text -- is often encrypted before being incorporated into the innocuous-seeming cover text file or data stream. If not encrypted, the hidden text is commonly processed in some way in order to increase the difficulty of detecting the secret content.

## What is StegoTool?
StegoTool is a tool that can be used to hide or retrieve a file using the principles of Steganography.

![StegoTool Decode Mode](assets/stegotool-decodemode.PNG)

![StegoTool Encode Mode](assets/stegotool-encodemode.PNG)

# Documentation
Visit our [Documentation Page](https://iamkhattar.github.io/stego-tool/) to view detailed documentation for StegoTool.

# Contributors

# License

[MIT](https://opensource.org/licenses/MIT)

repository: Saeedado/stego-tool

repo_name: Saeedado/stego-tool


text:
# Element API for Craft CMS

This plugin makes it easy to create a JSON API for your entries (and other element types) in [Craft CMS](http://buildwithcraft.com).

It’s powered by Phil Sturgeon’s excellent [Fractal](http://fractal.thephpleague.com/) package.

## Requirements

This plugin requires Craft CMS 4.0 or later.

## Installation

You can install this plugin from the Plugin Store or with Composer.

#### From the Plugin Store

Go to the Plugin Store in your project’s Control Panel and search for “Element API”. Then click on the “Install” button in its modal window.

#### With Composer

Open your terminal and run the following commands:

```bash
# go to the project directory
cd /path/to/my-project.test

# tell Composer to load the plugin
composer require craftcms/element-api

# tell Craft to install the plugin
./craft plugin/install element-api
```

## Setup

To define your API endpoints, create a new `element-api.php` file within your `config/` folder. This file should return an array with an `endpoints` key, which defines your site’s API endpoints.

Within the `endpoints` array, keys are URL patterns, and values are functions that define the endpoint configurations.

```php
<?php

use craft\elements\Entry;
use craft\helpers\UrlHelper;

return [
    'endpoints' => [
        'news.json' => function() {
            return [
                'elementType' => Entry::class,
                'criteria' => ['section' => 'news'],
                'transformer' => function(Entry $entry) {
                    return [
                        'title' => $entry->title,
                        'url' => $entry->url,
                        'jsonUrl' => UrlHelper::url("news/{$entry->id}.json"),
                        'summary' => $entry->summary,
                    ];
                },
            ];
        },
        'news/<entryId:\d+>.json' => function($entryId) {
            return [
                'elementType' => Entry::class,
                'criteria' => ['id' => $entryId],
                'one' => true,
                'transformer' => function(Entry $entry) {
                    return [
                        'title' => $entry->title,
                        'url' => $entry->url,
                        'summary' => $entry->summary,
                        'body' => $entry->body,
                    ];
                },
            ];
        },
    ]
];
```

### Endpoint Configuration Settings

Endpoint configuration arrays can contain the following settings:

#### `class`

The class name of the Fractal resource that should be used to serve the request. If this isn’t set, it will default to `craft\elementapi\resources\ElementResource`. (All of the following configuration settings are specific to that default class.)

#### `elementType` _(Required)_

The class name of the element type that the API should be associated with. Craft’s built-in element type classes are:

- `craft\elements\Asset`
- `craft\elements\Category`
- `craft\elements\Entry`
- `craft\elements\GlobalSet`
- `craft\elements\MatrixBlock`
- `craft\elements\Tag`
- `craft\elements\User`

```php
'elementType' => craft\elements\Entry::class,
````

#### `criteria`

An array of parameters that should be set on the [Element Query](https://docs.craftcms.com/v3/element-queries.html) that will be fetching the elements.

```php
'criteria' => [
    'section' => 'news',
    'type' => 'article',
],
```

#### `contentType`

The content type the endpoint responses should have.

```php
'contentType' => 'application/foo+json',
```

By default, the content type will be:

- `application/javascript` for endpoints that define a JSONP [callback](#callback)
- `application/feed+json` for endpoints where the [serializer](#serializer) is set to `jsonFeed`
- `application/json` for everything else

#### `transformer`

The [transformer](http://fractal.thephpleague.com/transformers/) that should be used to define the data that should be returned for each element. If you don’t set this, the default transformer will be used, which includes all of the element’s direct attribute values, but no custom field values.

```php
// Can be set to a function
'transformer' => function(craft\elements\Entry $entry) {
    return [
        'title' => $entry->title,
        'id' => $entry->id,
        'url' => $entry->url,
    ];
},

// Or a string/array that defines a Transformer class configuration
'transformer' => 'MyTransformerClassName',

// Or a Transformer class instance
'transformer' => new MyTransformerClassName(),
```

Your custom transformer class would look something like this:

```php
<?php

use craft\elements\Entry;
use League\Fractal\TransformerAbstract;

class MyTransformerClassName extends TransformerAbstract
{
    public function transform(Entry $entry)
    {
        return [
            // ...
        ];
    }
}
```

#### `one`

Whether only the first matching element should be returned. This is set to `false` by default, meaning that _all_ matching elements will be returned.

```php
'one' => true,
```

#### `paginate`

Whether the results should be paginated. This is set to `true` by default, meaning that only a subset of the matched elements will be included in each response, accompanied by additional metadata that describes pagination information.

```php
'paginate' => false,
```

#### `elementsPerPage`

The max number of elements that should be included in each page, if pagination is enabled. By default this is set to 100.

```php
'elementsPerPage' => 10,
```

#### `pageParam`

The query string param name that should be used to identify which page is being requested. By default this is set to `'page'`.

```php
'pageParam' => 'pg',
```

Note that it cannot be set to `'p'` because that’s the parameter Craft uses to check the requested path.

#### `resourceKey`

The key that the elements should be nested under in the response data. By default this will be `'data'`.

```php
'resourceKey' => 'entries',
```

#### `meta`

Any custom meta values that should be included in the response data.

```php
'meta' => [
    'description' => 'Recent news from Happy Lager',
],
```

#### `serializer`

The [serializer](http://fractal.thephpleague.com/serializers/) that should be used to format the returned data.

Possible values are:

- `'array'` _(default)_ – formats data using the [ArraySerializer](http://fractal.thephpleague.com/serializers/#arrayserializer).
- `'dataArray'` – formats data using the [DataArraySerializer](http://fractal.thephpleague.com/serializers/#dataarrayserializer).
- `'jsonApi'` – formats data using the [JsonApiSerializer](http://fractal.thephpleague.com/serializers/#jsonapiserializer).
- `'jsonFeed'` – formats data based on [JSON Feed V1.1](https://www.jsonfeed.org/version/1.1/) (see the [JSON Feed](#json-feed) example below).
- A custom serializer instance.

#### `includes`

The [include names](http://fractal.thephpleague.com/transformers/#including-data) that should be included for the current request, if any.

```php
'includes' => (array)Craft::$app->request->getQueryParam('include'),
```

Note that this setting requires a custom transformer class that’s prepped to handle includes:

```php
class MyTransformerClassName extends TransformerAbstract
{
    protected $availableIncludes = ['author'];

    public function includeAuthor(Entry $entry)
    {
        return $this->item($entry->author, function(User $author) {
            return [
                'id' => $author->id,
                'name' => $author->name,
            ];
        });
    }

    // ...
}
```

#### `excludes`

The [include names](http://fractal.thephpleague.com/transformers/#including-data) that should be excluded for the current request, which would otherwise have been included (e.g. if they were listed as a default include), if any.

```php
'excludes' => 'author',
```

Like [`includes`](#includes), this setting requires a custom transformer class.

#### `callback`

If this is set, a JSONP response will be returned with an `application/javascript` content type, using this setting value as the callback function.

For example, if you set this to:

```php
'callback' => 'foo',
```

Then the response will look like:

```javascript
foo({ /* ... */ });
```

Note that if you set this, the `jsonOptions` and `pretty` settings will be ignored.

#### `jsonOptions`

The value of the `$options` argument that will be passed to [`json_encode()`](http://php.net/manual/en/function.json-encode.php) when preparing the response. By default `JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE` will be passed.

```php
'jsonOptions' => JSON_UNESCAPED_UNICODE,
```

#### `pretty`

Shortcut for adding `JSON_PRETTY_PRINT` to `jsonOptions`.

```php
'pretty' => true,
```

#### `cache`

Whether the output should be cached, and for how long.

Possible values are:

- `true` _(default)_ – results are cached for the duration specified by the `cacheDuration` Craft config setting, or until a relevant element is saved or deleted.
- `false` – results are never cached.
- an integer – results are cached for the given number of seconds.
- a [interval spec](http://www.php.net/manual/en/dateinterval.construct.php) string – results are cached for the duration specified.

Note that the `onBeforeSendData` event does not get triggered when the cache is warm.

```php
'cache' => 'PT1M', // one minute
```

#### `cacheKey`

The key that responses should be cached with, if something custom is needed.

### Dynamic URL Patterns

URL patterns can contain dynamic subpatterns in the format of `<subpatternName:regex>`, where `subpatternName` is the name of the subpattern, and `regex` is a valid regular expression. For example, the URL pattern “`news/<entryId:\d+>.json`” will match URLs like `news/100.json`. You can also use the tokens `{handle}` and `{slug}` within your regular expression, which will be replaced with the appropriate regex patterns for matching handles and  element slugs.

Any subpattern matches in the URL pattern will be mapped to the endpoint config function’s arguments. For example, if a URL pattern contains an `entryId` subpattern, then you can add an `$entryId` argument to your endpoint config function, and whatever matches the URL subpattern will be passed to `$entryId`.

```php
'news/<entryId:\d+>.json' => function($entryId) {
    return [
        'elementType' => craft\elements\Entry::class,
        'criteria' => ['id' => $entryId],
        'one' => true,
    ];
},
```

### Setting Default Configuration Settings

You can specify default values for your endpoint configuration settings by adding a `defaults` key alongside your `endpoints` key (**not** within it).

```php
use craft\elements\Entry;

return [
    'defaults' => [
        'elementType' => Entry::class,
        'elementsPerPage' => 10,
        'pageParam' => 'pg',
        'transformer' => function(Entry $entry) {
            return [
                'title' => $entry->title,
                'id' => $entry->id,
                'url' => $entry->url,
            ];
        },
    ],

    'endpoints' => [
        'news.json' => function() {
            return [
                'criteria' => ['section' => 'news'],
            ]
        },
        'news/<entryId:\d+>.json' => function($entryId) {
            return [
                'criteria' => ['id' => $entryId],
                'one' => true,
            ];
        },
    ]
];
```

## Examples

Here are a few endpoint examples, and what their response would look like.

### Paginated Entry Index Endpoint

```php
'ingredients.json' => function() {
    return [
        'criteria' => ['section' => 'ingredients'],
        'elementsPerPage' => 10,
        'transformer' => function(craft\elements\Entry $entry) {
            return [
                'title' => $entry->title,
                'url' => $entry->url,
                'jsonUrl' => UrlHelper::url("ingredients/{$entry->slug}.json"),
            ];
        },
        'pretty' => true,
    ];
},
```

```json5
{
    "data": [
        {
            "title": "Gin",
            "url": "/ingredients/gin",
            "jsonUrl": "/ingredients/gin.json"
        },
        {
            "title": "Tonic Water",
            "url": "/ingredients/tonic-water",
            "jsonUrl": "/ingredients/tonic-water.json"
        },
        // ...
    ],
    "meta": {
        "pagination": {
            "total": 66,
            "count": 10,
            "per_page": 10,
            "current_page": 1,
            "total_pages": 7,
            "links": {
                "next": "/ingredients.json?p=2"
            }
        }
    }
}
```

### Single Entry Endpoint

```php
'ingredients/<slug:{slug}>.json' => function($slug) {
    return [
        'criteria' => [
            'section' => 'ingredients',
            'slug' => $slug
        ],
        'one' => true,
        'transformer' => function(craft\elements\Entry $entry) {
            // Create an array of all the photo URLs
            $photos = [];
            foreach ($entry->photos->all() as $photo) {
                $photos[] = $photo->url;
            }

            return [
                'title' => $entry->title,
                'url' => $entry->url,
                'description' => (string)$entry->description,
                'photos' => $photos
            ];
        },
        'pretty' => true,
    ];
},
```

```json5
{
    "title": "Gin",
    "url": "/ingredients/gin",
    "description": "<p>Gin is a spirit which derives its predominant flavour from juniper berries.</p>",
    "photos": [
        "/images/drinks/GinAndTonic1.jpg"
    ]
}
```

### JSON Feed

Here’s how to set up a [JSON Feed](https://jsonfeed.org/) ([Version 1.1](https://www.jsonfeed.org/version/1.1/)) for your site with Element API.

Note that `photos`, `body`, `summary`, and `tags` are imaginary custom fields.

```php
'feed.json' => function() {
    return [
        'serializer' => 'jsonFeed',
        'elementType' => craft\elements\Entry::class,
        'criteria' => ['section' => 'news'],
        'transformer' => function(craft\elements\Entry $entry) {
            $image = $entry->photos->one();

            return [
                'id' => (string)$entry->id,
                'url' => $entry->url,
                'title' => $entry->title,
                'content_html' => (string)$entry->body,
                'summary' => $entry->summary,
                'image' => $image ? $image->url : null,
                'date_published' => $entry->postDate->format(\DateTime::ATOM),
                'date_modified' => $entry->dateUpdated->format(\DateTime::ATOM),
                'authors' => [
                    ['name' => $entry->author->name],
                ],
                'language' => $entry->getSite()->language,
                'tags' => array_map('strval', $entry->tags->all()),
            ];
        },
        'meta' => [
            'description' => 'Recent news from Happy Lager',
        ],
        'pretty' => true,
    ];
},
```

```json5
{
    "version": "https://jsonfeed.org/version/1",
    "title": "Happy Lager",
    "home_page_url": "http://domain.com/",
    "feed_url": "http://domain.com/feed.json",
    "description": "Craft demo site",
    "items": [
        {
            "id": "24",
            "url": "http://domain.com/news/the-future-of-augmented-reality",
            "title": "The Future of Augmented Reality",
            "content_html": "<p>Nam libero tempore, cum soluta nobis est eligendi ...</p>",
            "date_published": "2016-05-07T00:00:00+00:00",
            "date_modified": "2016-06-03T17:43:36+00:00",
            "author": {
                "name": "Liz Murphy"
            },
            "tags": [
                "augmented reality",
                "futurism"
            ]
        },
        {
            "id": "4",
            "url": "http://domain.com/news/barrel-aged-digital-natives",
            "title": "Barrel Aged Digital Natives",
            "content_html": "<p>Nam libero tempore, cum soluta nobis est eligendi ...</p>",,
            "date_published": "2016-05-06T00:00:00+00:00",
            "date_modified": "2017-05-18T13:20:27+00:00",
            "author": {
                "name": "Liz Murphy"
            },
            "tags": [
                "barrel-aged"
            ]
        },
        // ...
    ]
}
```

repository: craftcms/element-api

gha_language: PHP

gha_license_id: MIT

repo_name: craftcms/element-api


text:
# Rust-oracle
[![Test](https://img.shields.io/github/actions/workflow/status/kubo/rust-oracle/run-tests.yml?branch=master)](https://github.com/kubo/rust-oracle/actions/workflows/run-tests.yml)
[![Crates.io](https://img.shields.io/crates/v/oracle.svg)](https://crates.io/crates/oracle)
[![Docs](https://docs.rs/oracle/badge.svg)](https://docs.rs/oracle)
[![Docs (in development)](https://img.shields.io/badge/docs-in_development-486D9F)](https://www.jiubao.org/rust-oracle/oracle/)

This is an [Oracle database][] driver for [Rust][] based on [ODPI-C][].

## Change Log

See [ChangeLog.md](https://github.com/kubo/rust-oracle/blob/master/ChangeLog.md).

## Build-time Requirements

* C compiler. See `Compile-time Requirements` in [this document](https://github.com/alexcrichton/cc-rs#compile-time-requirements).

## Run-time Requirements

* Oracle client 11.2 or later. See [ODPI-C installation document][].

## Supported Rust Versions

The oracle crate supports **at least** 6 rust minor versions including the stable
release at the time when the crate was released. The MSRV (minimum supported
rust version) may be changed when a patch version is incremented though it will
not be changed frequently. The current MSRV is 1.54.0.

## Usage

Put this in your `Cargo.toml`:

```text
[dependencies]
oracle = "0.5"
```
## Optional Features

The following features can be enabled from Cargo.toml:

Feature	| Description | available version
---|---|---
`chrono` | Implements [`ToSql`] and [`FromSql`] for [chrono] data types. | any
`stmt_without_lifetime` | Removes `conn` lifetime from [`Statement`]. This is available to avoid lifetime conflicts. | since 0.5.6
`aq_unstable` | Enables [Oracle Advanced Queuing support][aq]. This is unstable. It may be changed incompatibly by minor version upgrades. | since 0.5.5

## Examples

Executes select statements and get rows:

```rust,no_run
use oracle::{Connection, Error};

// Connect to a database.
let conn = Connection::connect("scott", "tiger", "//localhost/XE")?;

let sql = "select ename, sal, comm from emp where deptno = :1";

// Select a table with a bind variable.
println!("---------------|---------------|---------------|");
let rows = conn.query(sql, &[&30])?;
for row_result in rows {
    let row = row_result?;
    // get a column value by position (0-based)
    let ename: String = row.get(0)?;
    // get a column by name (case-insensitive)
    let sal: i32 = row.get("sal")?;
    // Use `Option<...>` to get a nullable column.
    // Otherwise, `Err(Error::NullValue)` is returned
    // for null values.
    let comm: Option<i32> = row.get(2)?;

    println!(" {:14}| {:>10}    | {:>10}    |",
             ename,
             sal,
             comm.map_or("".to_string(), |v| v.to_string()));
}

// Another way to fetch rows.
// The rows iterator returns Result<(String, i32, Option<i32>)>.
println!("---------------|---------------|---------------|");
let rows = conn.query_as::<(String, i32, Option<i32>)>(sql, &[&10])?;
for row_result in rows {
    let (ename, sal, comm) = row_result?;
    println!(" {:14}| {:>10}    | {:>10}    |",
             ename,
             sal,
             comm.map_or("".to_string(), |v| v.to_string()));
}

# Ok::<(), oracle::Error>(())
```

Executes select statements and get the first rows:

```rust,no_run
use oracle::Connection;

// Connect to a database.
let conn = Connection::connect("scott", "tiger", "//localhost/XE")?;

let sql = "select ename, sal, comm from emp where empno = :1";

// Print the first row.
let row = conn.query_row(sql, &[&7369])?;
let ename: String = row.get("empno")?;
let sal: i32 = row.get("sal")?;
let comm: Option<i32> = row.get("comm")?;
println!("---------------|---------------|---------------|");
println!(" {:14}| {:>10}    | {:>10}    |",
         ename,
         sal,
         comm.map_or("".to_string(), |v| v.to_string()));
// When no rows are found, conn.query_row() returns `Err(Error::NoDataFound)`.

// Get the first row as a tupple
let row = conn.query_row_as::<(String, i32, Option<i32>)>(sql, &[&7566])?;
println!("---------------|---------------|---------------|");
println!(" {:14}| {:>10}    | {:>10}    |",
         row.0,
         row.1,
         row.2.map_or("".to_string(), |v| v.to_string()));

# Ok::<(), oracle::Error>(())
```

Executes non-select statements:

```rust,no_run
use oracle::Connection;

// Connect to a database.
let conn = Connection::connect("scott", "tiger", "//localhost/XE")?;

conn.execute("create table person (id number(38), name varchar2(40))", &[])?;

// Execute a statement with positional parameters.
conn.execute("insert into person values (:1, :2)",
             &[&1, // first parameter
               &"John" // second parameter
              ])?;

// Execute a statement with named parameters.
conn.execute_named("insert into person values (:id, :name)",
                   &[("id", &2), // 'id' parameter
                     ("name", &"Smith"), // 'name' parameter
                    ])?;

// Commit the transaction.
conn.commit()?;

// Delete rows
conn.execute("delete from person", &[])?;

// Rollback the transaction.
conn.rollback()?;

# Ok::<(), oracle::Error>(())
```

Prints column information:

```rust,no_run
use oracle::Connection;

// Connect to a database.
let conn = Connection::connect("scott", "tiger", "//localhost/XE")?;

let sql = "select ename, sal, comm from emp where 1 = 2";
let rows = conn.query(sql, &[])?;

// Print column names
for info in rows.column_info() {
    print!(" {:14}|", info.name());
}
println!("");

// Print column types
for info in rows.column_info() {
    print!(" {:14}|", info.oracle_type().to_string());
}
println!("");

# Ok::<(), oracle::Error>(())
```

Prepared statement:

```rust,no_run
use oracle::Connection;

let conn = Connection::connect("scott", "tiger", "//localhost/XE")?;

// Create a prepared statement
let mut stmt = conn.statement("insert into person values (:1, :2)").build()?;
// Insert one row
stmt.execute(&[&1, &"John"])?;
// Insert another row
stmt.execute(&[&2, &"Smith"])?;

# Ok::<(), oracle::Error>(())
```

This is more efficient than two `conn.execute()`.
An SQL statement is executed in the DBMS as follows:

* step 1. Parse the SQL statement and create an execution plan.
* step 2. Execute the plan with bind parameters.

When a prepared statement is used, step 1 is called only once.

## NLS_LANG parameter

[NLS_LANG][] consists of three components: language, territory and
charset. However the charset component is ignored and UTF-8(AL32UTF8) is used
as charset because rust characters are UTF-8.

The territory component specifies numeric format, date format and so on.
However it affects only conversion in Oracle. See the following example:

```rust,no_run
use oracle::Connection;

// The territory is France.
std::env::set_var("NLS_LANG", "french_france.AL32UTF8");
let conn = Connection::connect("scott", "tiger", "")?;

// 10.1 is converted to a string in Oracle and fetched as a string.
let result = conn.query_row_as::<String>("select to_char(10.1) from dual", &[])?;
assert_eq!(result, "10,1"); // The decimal mark depends on the territory.

// 10.1 is fetched as a number and converted to a string in rust-oracle
let result = conn.query_row_as::<String>("select 10.1 from dual", &[])?;
assert_eq!(result, "10.1"); // The decimal mark is always period(.).

# Ok::<(), oracle::Error>(())
```

Note that NLS_LANG must be set before first rust-oracle function execution if
required.

## TODO

* [BFILEs (External LOBs)](https://www.oracle.com/pls/topic/lookup?ctx=dblatest&id=GUID-5834BC49-4053-40FF-BE39-B14342B1201E) (Note: Reading contents of BFILEs as `Vec<u8>` is supported.)
* Scrollable cursors
* Better Oracle object type support
* XML data type
* [JSON data type](https://oracle-base.com/articles/21c/json-data-type-21c)

## Related Projects

Other crates for connecting to Oracle:
* [Sibyl]: an OCI-based interface supporting both blocking (threads) and nonblocking (async) API

Oracle-related crates:
* [r2d2-oracle]: Oracle support for the [r2d2] connection pool
* [bb8-oracle]: [bb8] connection pool support for oracle
* [include-oracle-sql]: an extension of [include-sql] using [Sibyl] for database access

## License

Rust-oracle and ODPI-C bundled in rust-oracle are under the terms of:

1. [the Universal Permissive License v 1.0 or at your option, any later version](http://oss.oracle.com/licenses/upl); and/or
2. [the Apache License v 2.0](http://www.apache.org/licenses/LICENSE-2.0).

[Rust]: https://www.rust-lang.org/
[ODPI-C]: https://oracle.github.io/odpi/
[ODPI-C installation document]: https://oracle.github.io/odpi/doc/installation.html
[Oracle database]: https://www.oracle.com/database/index.html
[NLS_LANG]: https://www.oracle.com/pls/topic/lookup?ctx=dblatest&id=GUID-86A29834-AE29-4BA5-8A78-E19C168B690A
[`FromSql`]: https://docs.rs/oracle/latest/oracle/sql_type/trait.FromSql.html
[`Connection`]: https://docs.rs/oracle/latest/oracle/struct.Connection.html
[`Statement`]: https://docs.rs/oracle/latest/oracle/struct.Statement.html
[`ToSql`]: https://docs.rs/oracle/latest/oracle/sql_type/trait.ToSql.html
[aq]: https://docs.rs/oracle/latest/oracle/aq/index.html
[bb8]: https://crates.io/crates/bb8
[bb8-oracle]: https://crates.io/crates/bb8-oracle
[chrono]: https://docs.rs/chrono/0.4/chrono/
[include-sql]: https://crates.io/crates/include-sql
[include-oracle-sql]: https://crates.io/crates/include-oracle-sql
[r2d2]: https://crates.io/crates/r2d2
[r2d2-oracle]: https://crates.io/crates/r2d2-oracle
[Sibyl]: https://crates.io/crates/sibyl

repository: kubo/rust-oracle

gha_language: Rust

repo_name: kubo/rust-oracle


text:
# Performant Python Guide

## Profiling
**Tools**:
- [cProfile](https://docs.python.org/2/library/profile.html)
    + comes by default with the standard Python interpreter (cPython) since version 2.5
    + exclusively measures CPU time
    + pays no attention to memory consumption and other memory related stats
    + main APIs:
        * run(command, filename=None, sort=-1)
        * exec(command, \_\_main\_\_.\_\_dict\_\_, \_\_main\_\_.\_\_dict\_\_)
        * runctx(command, globals, locals, filename=None)

    ```python
    # *** PROFILER RESULTS ***
    # fib_ver1 (XXXX\tmp\profile_examples\main.py:5)
    # function called 109 times
    #
    #          326 function calls (6 primitive calls) in 0.000 seconds
    #
    #    Ordered by: cumulative time, internal time, call count
    #
    #    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    #     109/1    0.000    0.000    0.000    0.000 main.py:5(fib_ver1)
    #     108/2    0.000    0.000    0.000    0.000 profilehooks.py:235(new_fn)
    #     108/2    0.000    0.000    0.000    0.000 profilehooks.py:329(__call__)
    #         1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
    #         0    0.000             0.000          profile:0(profiler)
    ```

    | column name | meaning |
    | ----------- | ------- |
    | ncalls | reports the number of calls to the function. If there are two numbers in this column, it means there was recursion. The second one is the number of primitive calls, and the first one is the total number of calls. This number can be helpful to identify the possible bugs (unexpected high numbers) or possible inline expansion points.) |
    | tottime | the total time spent inside the function (excluding the time spent doing subcalls to other functions). This particular information can help the developer find long running loops that could be optimized. |
    | percall | the quotient of tottime divided by ncalls. |
    | cumtime | the cumulative time spent inside the function including the time spent in subfunctions (this includes recursive calls as well). This number could help identify higher level errors, such as those in the selection of the algorithm. |
    | percall | the quotient of cumtime divided by primitive calls. |
    | filename:lineno(function) |  the file name, line number, and function name of the analyzed function. |

- [line_profiler](https://github.com/rkern/line_profiler)
    + pip install line_profiler
    + The profiler was designed to be used as a decorator
    + execute the profiler: kernprof -l script_to_profile.py

    ![lp](pics/line_profiler.png)

- [RunSnakeRun](http://www.vrplumber.com/programming/runsnakerun/)
    + a small GUI utility that allows you to view (Python) cProfile or Profile profiler dumps in a sortable GUI view.
    + installation:

        ```sh
        apt-get install python-profiler python-wxgtk2.8 python-setuptools
        pip install  SquareMap RunSnakeRun
        ```

    + run:

        ```sh
        python -m cProfile -o xxx.prof xxx.py \<paras\>
        ```

    ![png](pics/rsr.png)

## Tips and tricks of Python
- Memoization / lookup tables
- Usage of default arguments

    ```python
    import math
    #original function
    def degree_sin(deg):
        return math.sin(deg * math.pi / 180.0)
    #optimized function, the factor variable is calculated during function creation time,
    #and so is the lookup of the math.sin method.
    def degree_sin(deg, factor=math.pi/180.0, sin=math.sin):
        return sin(deg * factor)
    ```

- List comprehension and generators
    + dis module to show the bytecode
    + when generating a list, the for loop should not be your weapon of choice.
    + when generating big lists, consider using generators.

    ![png](pics/perf1.png)

- ctypes
    + allows the developer to reach under the hood of Python and tap into the power of the C language.
- String concatenation
    + In Python, strings are immutable, which means that once you create one you can't really change its value.
    + variable interpolation
    + join
- Membership testing
    + set and dict
- collections
- funcion calls can be expensive
- When possible, sort by the key, instead of assign a cmp
- 1 is better than True
- Multiple assignments are slow
- Chained comparisons are good
- Using namedtuples instead of regular (small) objects

## Multithreading & multiprocessint
- Multithreading
    + benefits
        * Responsiveness
        * Faster execution
        * Lower resource consumption
        * Simplified sharing and communication
        * Parallelization
    + drawbacks
        * Thread synchronization
        * Crash due to problematic thread
        * Deadlocks
    + GIL
        * The official version of Python (CPython) has a limitation called GIL. This GIL prevents multiple native threads from running Python's bytecode at once, which effectively trumps parallelism.
        * A GIL is necessary because the memory management in CPython is not thread safe.
    +  modules in python
        * The thread module: This provides some limited ability to work with threads. It's simple to use, and for small tasks, it adds little overhead.
        * The threading module: This is newer and included in Python since version 2.4. It provides a more powerful and higher level support for threads.
        * Interthread communication using events

        | use threads | do not use threads |
        | ----------- | ------------------ |
        | For heavy IO-bound scripts| To optimize scripts that are heavily CPU bound |
        | When parallelism can be replaced by concurrency | For programs that must take advantage of multicore systems |
        | For GUI development | |

- Multiprocessing

    | Adventages | Disadventages |
    | ---------- | ------------- |
    | Takes advantage of multicore systems | Larger memory footprint |
    | Separate memory space removes race conditions from the equation | Harder to share mutable data between processes |
    | Child processes are easily interruptible (killable) | Interprocess communication (IPC) is harder than with threads |
    | Avoids the GIL limitation (although only in the case of CPython) | |

    + Pool class, which represents a pool of worker processes that facilitate different ways to execute a set of tasks in subprocesses, main apis:
        * apply
        * apply_async
        * map

    + Interprocess communication
        * The Queue class provides a thread-safe and process-safe first-in-first-out (FIFO) mechanism to exchange data.
        * Pipes provide a bidirectional channel of communication between two processes. The Pipe() function returns a pair of connection objects, each representing one side of the pipe. Each connection object has both a send() and a recv() method.
        * Events. Event objects can't be passed into worker functions. If you try to do that, a runtime error will be issued, saying that semaphore objects can only be shared between processes through inheritance.

## PyPy & Cython
- PyPy, which consists of a version of Python and is implemented in RPython. It has a JIT compiler in charge of optimizing the code during execution time.
- Cython, which is basically a transpiler of (static typing) Python code into C code.
    ![png](pics/cython.png)

    | pypy use cases | cython use cases |
    | -------------- | ---------------- |
    | You're not dealing with an execute once script. | You're familiar with C code. |
    | Full support of third-party libraries is not required. | Losing Python's readability is not a problem. |
    | You need your code to be compatible with CPython. | Full support of the Python language is needed. |

## Lightning Fast Number Crunching
- [Numba](http://numba.pydata.org)
    +  allows you to write high-performance functions in pure Python by generating optimized machine code.
    + decorator style jit
    + main features:
        * On-the-fly code generation
        * Native code generation for both CPU and GPU hardware
        * Integration with Python's scientific software, thanks to the Numpy dependency

- [Parakeet](http://www.parakeetpython.com)
    + a runtime compiler for scientific computing in Python which uses type inference, data parallel array operators, and a lot of black magic to make your code run faster.
    + decorator style jit
    + very specific because it only supports a very narrow subset of the resulting combination of Python and NumPy
    +  the performance gain can be amazing, Parakeet only supports a very limited version of Python, so it is not really meant to be a general purpose optimizer (quite the opposite actually).

- [pandas](http://pandas.pydata.org)
    + provides a set of high-performance data structures and analysis tools.
    + on top of numpy

**References:**
- Mastering Python High Performance

repository: grehujt/SmallPythonProjects

repo_name: grehujt/SmallPythonProjects


text:
---
title: Protecting Yourself and Your Funds
description: Getting Started
tags:
  - Safety & Security
  - Phishing
priority: 100
date_published: '2015-07-15'
date_modified: '2021-10-26'
---

![Cryptocurrency Security Guide](../assets/staying-safe/protecting-yourself-and-your-funds/cryptocurrenty-security-guide.jpg)

(Source: [@Jennicide](https://twitter.com/Jennicide))

## 1. Get yourself a hardware wallet.

One of the safest and easiest ways to store your ETH, Tokens, ETC, BTC, and many other coins is via a **Ledger Nano S** or **Trezor**. Both are hardware wallets. Both work with MyCrypto.com and both cost less than $100.

- [Buy a Ledger](https://www.ledgerwallet.com/r/1985?path=/products/)
- [Buy a Trezor](https://shop.trezor.io/?offer_id=10&aff_id=1735)

If you don't want one of these nifty devices, use [cold storage](/how-to/offline/using-mycrypto-for-cold-storage) for a majority of your savings. Please. Pretty please.

## 2. Bookmark your crypto sites.

Use those bookmarks and only those. Don't type in the addresses by hand.

## 3. Install EAL or MetaMask.

Install the [EAL Chrome Extension](https://chrome.google.com/webstore/detail/etheraddresslookup/pdknmigbbbhmllnmgdfalmedcmcefdfn) or the [MetaMask Chrome Extension](https://chrome.google.com/webstore/detail/metamask/nkbihfbeogaeaoehlefnkodbefgpgknn) to warn you if you go to a crypto-phishing link.

## 4. [Use MyCrypto locally and offline](/how-to/offline/how-to-run-mycrypto-offline-and-locally).

<Alert label="The desktop application is not actively maintained.">

The MyCrypto desktop application is part of an older version of MyCrypto that is not actively maintained. You can find the latest version of MyCrypto on [mycrypto.com](https://mycrypto.com/).

</Alert>

## 5. Do not trust random URLs.

Do not trust communications, addresses, or URLs sent via private message. Always verify information with a secondary source.

- Don’t click any link related to anything crypto, money, banking, or services like Dropbox / Google Drive / Gmail in any email ever.
- And if the scammy clickbait was simply too irresistible for you, don’t enter any information on the page.
- Never enter your private keys, passwords, or sensitive data on a website that you were sent via message.

## 6. Turn on 2FA for everything.

- Go do it. Right now. Quit your excuses. Choose Google Authenticator over Authy. Don't use your phone number. Then, make sure your phone number is NOT tied to your Google account (look in privacy settings). Turns out, you and your BFF Mr. Hacker can "recover" access to your account via that number, completely destroying the point of 2FA.
- PS: MyCrypto is client-side, meaning 2FA won't do anything in our case. 2FA is for ensuring the security of your password on a server.
- PSS: Don't forget to cold store your backup words for these 2FA things. It's a huge pain when your phone goes for a swim and your entire life is 2FA'd.

## 7. For token sales:

Do not trust any address except the one posted on the official site.

- Bookmark the URL before the sale. Get the address of the URL from your bookmark at time of purchase. Do not trust any other source (especially a bot on Slack). PS: When are token sales going to start using ENS names?

## 8. Double-check the URL and triple-check GitHub URLs

- Check it. Then, check it again right before entering any information. This is especially important for any sites that require usernames, passwords, email addresses, private keys, or any other personal information. SSL certs do not mean a site is trustworthy, just that they bought an SSL cert. Not sure about the correct URL? Cross-reference Reddit, Twitter, GitHub, Slack and wherever else the project hangs out.
- GitHub URLs are much easier to fake and much easier to miss. Instead of downloading from that unverified URL on Reddit, seek out the URL on your own. Following the developers of these repos on Twitter, friending them on Reddit (lol ... but seriously it's nice because their name will be orange), or starring said repos on GitHub helps.

## 9. Always verify that the site you landed on is legit.

- Especially if you are about to enter your private key or download an application. What is legit? A service that people have used for a decent period of time with good results. If the URL has been registered in the last week or the site "just launched," err on the side of caution and avoid it for a while.

## 10. Google the service name + "scam" or "reviews."

- Scam sites rarely last long. Value real comments by real people over a random blog. Value a collection of information over a single source. Understand that legit services will likely have a mix of positive and negative reviews over a long period of time. Scam sites typically have either no one talking about them at all, a lot of yelling about how people got robbed, or the most perfect reviews ever. The latter one is just as much of a red flag as the first one.

## 11. Don't run remote-access software (e.g., TeamViewer).

- Don't ever ... but especially not on a computer with keys on them. The number of security holes in these programs is atrocious. It would be a shame if you enabled 2FA on everything in your life but then let a single string of characters give someone access to your entire computer and every account.

## 12.

- Brain wallets are wallets where the key is derived from a word or phrase you _choose_. Human brains don't have the ability to create high-entropy seeds. Using a phrase that you make up, even if it seems "rare" or "random" is not as secure as using MyCrypto's randomness, and these phrases can be brute-forced by the millions. [Read more](https://arstechnica.com/security/2013/10/how-the-bible-and-youtube-are-fueling-the-next-frontier-of-password-cracking/). [And more](https://arstechnica.com/security/2016/02/password-cracking-attacks-on-bitcoin-wallets-net-103000/).

## 13. Install a good adblocker.

Install an adblocker that actually turns off Google and Bing ads.

- We recommend going with uBlock Origin. If you are already using Adblock Plus, it does not hide Google Ads from you. Go into your Adblock Plus settings and uncheck the box that says “Allow some non-intrusive advertising.”

## 14. Don’t click on advertisements.

- With or without an adblocker, you should never, ever click on advertisements.

## 15. Clean out your history.

If you have accidentally visited or typed a malicious site, clean out your recent history and autocomplete.

- This will prevent you from typing kra… and having it autocomplete to the malicious krakken.com.

## 16. No one is giving you free or discounted ETH.

- Even for completing a survey. ;)

## 17. Don't trust Slack DMs.

The guys who just finished their token sale don't want to sell you tokens via Slack DM.

- Neither does that smokin' hot 125px x 125px avatar.

## 18. Don't unlock your account to check your balance.

 Check your balance via [Etherscan](https://etherscan.io/), [Ethplorer](https://ethplorer.io/), or [Etherchain](https://www.etherchain.org/).

## 19.

- Think for a moment. Don't assume—ask. Don't blindly follow—question. If something doesn't seem right, if you feel like the luckiest person on Earth, or if you find yourself pondering, "I wonder why I haven't seen this on Reddit yet," there is likely a reason.

repository: MyCryptoHQ/knowledge-base-content

gha_language: TypeScript

repo_name: MyCryptoHQ/knowledge-base-content


text:
# contributing

Everyone is encouraged to help improve this project.

Please submit pull requests against the **master branch**.

Our [Concourse](http://concourse.ci) CI system is available at [ci.lattice.cf](https://ci.lattice.cf).

Here are some ways *you* can contribute:

* by using alpha, beta, and prerelease versions
* by reporting bugs
* by suggesting new features
* by writing or editing documentation
* by writing specifications
* by writing code (**no patch is too small**: fix typos, add comments, clean up inconsistent whitespace)
* by refactoring code
* by closing [issues](https://github.com/cloudfoundry-incubator/lattice/issues)
* by reviewing patches

Also see the [Development Readme](development/README.md)

## Submitting an Issue
We use the [GitHub issue tracker](https://github.com/cloudfoundry-incubator/lattice/issues) to track bugs and features.
Before submitting a bug report or feature request, check to make sure it hasn't already been submitted.
You can indicate support for an existing issue by voting it up.
When submitting a bug report, please include a [Gist](http://gist.github.com/) that includes a stack trace and any
details that may be necessary to reproduce the bug including the Lattice version.

## Submitting a Pull Request

1. Propose a change by opening an issue.
2. Fork the project.
3. Create a topic branch.
4. Implement your feature or bug fix.
5. Commit and push your changes.
6. Submit a pull request.

## Choosing Stories From our [Tracker](https://www.pivotaltracker.com/n/projects/1183596)

[Pivotal Tracker](http://www.pivotaltracker.com/) is the way that Cloud Foundry projects organize and prioritize work and releases. With Tracker, work is organized into stories that are actionable and have been prioritized.  The team typically works on stories found in the Current and Backlog columns.

Stories not (yet) prioritized are kept in the Icebox. The Icebox is a grab-bag of feature requests, GitHub Issues, or partially-developed ideas. Stories in the Icebox may never be prioritized, and may not always be maintained in the same disciplined manner as the Backlog.

Periodically, the Lattice team goes through through both the Backlog and the Icebox, and tags stories using the 'community' label.   These are stories that are particularly well-suited to community contribution, and make excellent candidates for people to work on.

repository: damon-pivotal/lattice-release

repo_name: damon-pivotal/lattice-release


text:
Universal Approximation with Certified Networks
==============================================================================================================================================================================================

This repository contains the associated code for the paper
[Universal Approximation with Certified Networks](https://openreview.net/forum?id=B1gX8kBtPr)
which has been accepted to appear at ICLR 2020.

Background
----------

Many systems have been proposed to certify that neural networks are robust to adversarial attack, for example, the SRILab's [ERAN](https://github.com/eth-sri/eran).
In response to the observation that many networks are emperically robust to adversarial attack yet not provable with these techniques,
provably robust training has been developed, an early example being SRILab's [DiffAI](https://github.com/eth-sri/diffai).

While training neural networks to be certifiably robust with efficient techniques is a continually improving front, there still appears to be a significant gap
between what these training systems have achieved versus standard robust training. In our paper [Universal Approximation with Certified Networks](https://openreview.net/forum?id=B1gX8kBtPr),
we strengthening the [universal approximation theorem](https://en.wikipedia.org/wiki/Universal_approximation_theorem)
by showing that ReLU-networks which are interval-certifiable are also universal approximators.
We targeted interval certification as it is the simplest and most efficient certification method.
Its simplicity implies that our result extends to more precise commonly used domains such as that proposed by [Singh et al. 2019](https://files.sri.inf.ethz.ch/website/papers/DeepPoly.pdf).

Theorem
--------

Our theorem says that for all d > 0,
and for every continuous function, f,
from a compact subset, C, of a real d-dimensional space to the real line,
there is a ReLU network, n,
such that we can use interval analysis to prove that every box B which is a subset of C,
n(B) is a subset of f(B)±d.

Code for Construction
---------------------

![Levels](https://github.com/eth-sri/UniversalCertificationTheory/blob/master/media/overview.png?raw=true)

Our paper proves the above theorem by constructing such a network n, using the fact that f(B) exists.
A visualization of the components of the resulting network is shown above.
In this repository, we provide code in both Haskell and Python for constructing this network given an input function f and the code to compute f(B) exactly.

This repository contains two files:

* Python:  PythonConstruction.py
* Haskell: HaskellConstruction.hs

Python Usage
------------

Tested With:
* Python 3.6.4
* numpy 1.14.1

The class ```Interval``` provides intervals together with the functionality to
execute the arithmetik operations found in ReLU networks. To initialize an
object of  class ```func_I``` one provides the concrete function
```func``` mapping points in [0,1]2 to ℝ, a function ```Delta_k_func```
generating the sets Δk, the minimum ```minimum``` and the maximum
```maximum``` of ```func``` on [0,1]2. Furthermore, to simplify the
implementation we restirct ourselfs here to Lipschitz continuous function. The
Lipschitz constant ```lip_const``` of ```func``` on [0,1]2 is the last
argument.

This is implemented and tested for the functions
```sqrt_func```(**x**):= (1 + x0 + x1)1/2 and
```square_func```(**x**):= x02 +
x12.

Haskell Usage
-------------

Tested With:
* GHC 8.0.1

This module exposes a function "buildProvableNet f l u epsilon delta m"
which takes a function f represented as closed box to min and max in that region,
a lower bound vector l of the region, an upper bound vector r of the region,
an epsilon and a delta to use, and an integer constant m such that if |x - y| <= 1/m, then |f(y)−f(x)| <= delta.

The function "test" demonstrates its usage.  This can be run by executing the following command:

```
ghc HaskellConstruction.hs -e "test"
```

Citing This Work
----------------

```
@inproceedings{
    baader2020universal,
    title={Universal Approximation with Certified Networks},
    author={Maximilian Baader and Matthew Mirman and Martin Vechev},
    booktitle={International Conference on Learning Representations},
    year={2020},
    url={https://openreview.net/forum?id=B1gX8kBtPr}
}
```

Contributors
------------

* [Maximilian Baader](https://www.sri.inf.ethz.ch/people/max) -<EMAIL_ADDRESS>* [Matthew Mirman](https://www.mirman.com) -<EMAIL_ADDRESS>* [Martin Vechev](https://www.sri.inf.ethz.ch/vechev.php) -<EMAIL_ADDRESS>

License and Copyright
---------------------

* Copyright (c) 2020 [Secure, Reliable, and Intelligent Systems Lab (SRI), ETH Zurich](https://www.sri.inf.ethz.ch/)
* Licensed under the [MIT License](https://opensource.org/licenses/MIT)

repository: eth-sri/UniversalCertificationTheory

repo_name: eth-sri/UniversalCertificationTheory


text:
# @dojo/widgets/split-pane widget

Dojo's `SplitPane` component divides a container into two resizable panes that can oriented horizontally or vertically. It expects two child nodes.

## Features

- Can be nested to create more than two panes
- Panes can be in a horizontal or vertical orientation
- Will collapse to a stacked layout at a set width

## Themeing

The following CSS classes are used to style the `SplitPane` widget and should be provided by custom themes:

- `collapsed`: Applied to the root node when the width is below `collapseWidth`
- `column`: Applied to the root node if `direction` is set to `Direction.column`
- `divider`: Applied to the divider node
- `leading`: Applied to the first pane in the DOM
- `root`: Applied to the top-level node
- `row`: Applied to the root node if `direction` is set to `Direction.row`
- `trailing`: Applied to the last pane in the DOM

## Example Usage

*Basic Row Example*
```typescript
import SplitPane, { Direction } from '@dojo/widgets/split-pane';
import { w } from '@dojo/framework/core/vdom';

w(SplitPane, {
	key: 'splitpane1',
	direction: Direction.row,
	onResize: (size: number) => {
		this.setState({ rowSize: size });
	},
	size: this.state.rowSize
}, [
	v('div', [ 'top' ]),
	v('div', [ 'bottom' ])
]);
```

*Column example that collapses at 600px*
```typescript
import SplitPane, { Direction } from '@dojo/widgets/split-pane';
import { w } from '@dojo/framework/core/vdom';

w(SplitPane, {
	key: 'splitpane2',
	collapseWidth: 600,
	direction: Direction.column,
	onResize: (size: number) => {
		this.setState({ columnSize: size });
	},
	size: this.state.columnSize
}, [
	v('div', [ 'left' ]),
	v('div', [ 'right' ])
]);
```

*Nested SplitPanes*
```typescript
import SplitPane, { Direction } from '@dojo/widgets/split-pane';
import { w } from '@dojo/framework/core/vdom';

w(SplitPane, {
	key: 'nested',
	direction: Direction.column,
	onResize: (size: number) => {
		this.setState({ nestedSizeA: size });
	},
	size: this.state.nestedSizeA
}, [
	v('div'),
	w(SplitPane, {
		key: 'inner',
		direction: Direction.row,
		onResize: (size: number) => {
			this.setState({ nestedSizeB: size });
		},
		size: this.state.nestedSizeB
	})
])
```

repository: imbkj/widgets

repo_name: imbkj/widgets


text:
# esp32-ble-wifi-conf
Using BLE to set WiFi configuration on ESP32 boards

Sometimes we just don't want to hardcode our SSID and password into hardware.

This is one of the solution.
Using BLE (Bluetooth Low Energy) capability of ESP32, we can set WiFi SSID and password after flashing the hardware.

This code uses Preferences.h, where we can input up to two SSID at one time.
And will choose between SSID depending on which one has a greater availibility.

This code itself is very simple and uses simple XOR encryption to transmit and receive data.

Usage example:
Simplest way to use this code.

```cpp
#include "src/ESP32_BLE_WiFi_Conf.h";

//global variable
Esp32BLEWifiConf bleconf;

void setup()
{
  bleconf.begin();
}

void loop() {
  bleconf.process();
}
```

Other example:

We are getting command from BLE;
and acknowledge by sending a string back.

```cpp
#include "src/ESP32_BLE_WiFi_Conf.h";

//global variable
Esp32BLEWifiConf bleconf;

void setup()
{
  bleconf.begin();
}

void loop() {
  bleconf.process();
  if(bleconf.isBLECommand()) {
     std::string command = bleconf.getBLECommand();
     // Do something with command
     Serial.print(command.c_str());
     Serial.print('\n');
     bleconf.notifyBLE("OK");
  }
}
```

Encryption provided is minimal.
Simple strings XOR with number 1 to obfuscate data transmission.

NOTE: Please alter the code to avoid any security issue.

```cpp
std::string XOR(std::string wifiCredential)
{
    // encode the data
    int keyIndex = 1;
    std::string encodedCredential;

    for (int index = 0; index < wifiCredential.length(); index++)
    {
        encodedCredential += (char)wifiCredential[index] ^ keyIndex;
    }

    return encodedCredential;
};
```

Provide yourself a decoding algorithm on your software.

Some rudimentary software is provided at https://esp32pwa.web.app to check for features.

Happy coding!

Cheers.


text:
# svelte-basic-usage

Simple demo of using `svelte-jsoneditor` in a Svelte application.

Install dependencies once:

```
npm install
```

Start development server:

```
npm run dev
```

license: ISC

repository: leafOfTree/svelte-jsoneditor

gha_license_id: NOASSERTION

repo_name: leafOfTree/svelte-jsoneditor


text:
> The official Splunk documentation for this page is [Install on Windows](https://docs.splunk.com/Observability/gdi/opentelemetry/install-windows.html). For instructions on how to contribute to the docs, see [CONTRIBUTING.md](../CONTRIBUTING#documentation.md).

# Windows Manual

The following deployment options are supported:

- [Collector MSI](#collector-msi-installation)
- [Fluentd MSI](#fluentd-msi-installation) for log collection
- [Chocolatey](#chocolatey-installation)
- [Docker](#docker)

## Getting Started

All installation methods offer [default
configurations](https://github.com/signalfx/splunk-otel-collector/blob/main/cmd/otelcol/config/collector)
which can be configured via environment variables. How these variables are
configured depends on the installation method leveraged.

### Collector MSI Installation

A Windows MSI package (64-bit only) is available to download at
[https://github.com/signalfx/splunk-otel-collector/releases
](https://github.com/signalfx/splunk-otel-collector/releases).

The Collector will be installed to
`\Program Files\Splunk\OpenTelemetry Collector`, and the
`splunk-otel-collector` service will be created but not started.

A default config file will be copied to
`\ProgramData\Splunk\OpenTelemetry Collector\agent_config.yaml` if it does not
already exist.  This file is required to start the `splunk-otel-collector`
service.

#### GUI

Double-click on the downloaded package and follow the wizard.

#### PowerShell

In a PowerShell terminal:

```sh
Start-Process -Wait msiexec "/i PATH_TO_MSI /qn"
```

Replace `PATH_TO_MSI` with the *full* path to the downloaded package, e.g.
`C:\your\download\folder\splunk-otel-collector-0.4.0-amd64.msi`.

#### Configuration

Before starting the `splunk-otel-collector` service, the following variables
in the default config file need to be replaced by the appropriate values for
your environment:

- `${SPLUNK_ACCESS_TOKEN}`: The Splunk access token to authenticate requests
- `${SPLUNK_API_URL}`: The Splunk API URL, e.g. `https://api.us0.signalfx.com`
- `${SPLUNK_HEC_TOKEN}`: The Splunk HEC authentication token
- `${SPLUNK_HEC_URL}`: The Splunk HEC endpoint URL, e.g. `https://ingest.us0.signalfx.com/v1/log`
- `${SPLUNK_INGEST_URL}`: The Splunk ingest URL, e.g. `https://ingest.us0.signalfx.com`
- `${SPLUNK_TRACE_URL}`: The Splunk trace endpoint URL, e.g. `https://ingest.us0.signalfx.com/v2/trace`
- `${SPLUNK_BUNDLE_DIR}`: The location of your Smart Agent bundle for monitor functionality, e.g. `C:\Program Files\Splunk\OpenTelemetry Collector\agent-bundle`

After updating all variables in the config file, start the
`splunk-otel-collector` service by rebooting the system or running the
following command in a PowerShell terminal:

```sh
Start-Service splunk-otel-collector
```

To modify the default path to the configuration file for the
`splunk-otel-collector` service, run `regdit` and modify the `SPLUNK_CONFIG`
value in the
`HKLM:\SYSTEM\CurrentControlSet\Control\Session Manager\Environment`
registry key, or run the following PowerShell command (replace `PATH` with the
full path to the new configuration file):

```powershell
Set-ItemProperty -path "HKLM:\SYSTEM\CurrentControlSet\Control\Session Manager\Environment" -name "SPLUNK_CONFIG" -value "PATH"
```

To add or remove command line options for the `splunk-otel-collector` service,
run `regedit` and modify the `ImagePath` value in the
`HKLM:\SYSTEM\CurrentControlSet\Services\splunk-otel-collector` registry key,
or run the following PowerShell command (replace `OPTIONS` with the desired
command line options):

```powershell
Set-ItemProperty -path "HKLM:\SYSTEM\CurrentControlSet\Services\splunk-otel-collector" -name "ImagePath" -value "C:\Program Files\Splunk\OpenTelemetry Collector\otelcol.exe OPTIONS"
```

For example, to change the default exposed metrics address of the Collector to
`<IP_ADDRESS>:9090`, run the following PowerShell command:

```powershell
Set-ItemProperty -path "HKLM:\SYSTEM\CurrentControlSet\Services\splunk-otel-collector" -name "ImagePath" -value "C:\Program Files\Splunk\OpenTelemetry Collector\otelcol.exe --metrics-addr <IP_ADDRESS>:9090"
```

To see all available command line options, run the following PowerShell
command:

```powershell
& 'C:\Program Files\Splunk\OpenTelemetry Collector\otelcol.exe' --help
```

After modifying the configuration file or registry key, apply the changes by
restarting the system or running the following PowerShell commands:

```powershell
Stop-Service splunk-otel-collector
Start-Service splunk-otel-collector
```

#### Service Logging

The Collector logs and errors can be viewed in the Windows Event Viewer when run as a service. The service logs are
displayed in `Event Viewer` > `Windows Logs` > `Application` and are visible by source "collector".

The Collector will have a log level of info by default.  To change this in versions 0.36.0 and later
you can add a "logs" entry in the [service telemetry definition](https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/troubleshooting.md#logs) in the currently used config before restarting the service:

```yaml
service:
  telemetry:
    logs:
      level: debug
```

For older versions of the Collector you can alter the service `ImagePath` before restarting:

```sh
Set-ItemProperty -path "HKLM:\SYSTEM\CurrentControlSet\Services\splunk-otel-collector" -name "ImagePath" -value "C:\Program Files\Splunk\OpenTelemetry Collector\otelcol.exe --log-level debug"
Restart-Service splunk-otel-collector

# Reverting after observing logs:
Set-ItemProperty -path "HKLM:\SYSTEM\CurrentControlSet\Services\splunk-otel-collector" -name "ImagePath" -value "C:\Program Files\Splunk\OpenTelemetry Collector\otelcol.exe"
Restart-Service splunk-otel-collector
```

### Fluentd MSI Installation

If log collection is required, perform the following steps to install Fluentd
and forward collected log events to the Collector (requires `Administrator`
privileges):

1. Install, configure, and start the Collector as described in the previous
   section.  The Collector's default configuration file
   (`\ProgramData\Splunk\OpenTelemetry Collector\agent_config.yaml`) listens
   for log events on `<IP_ADDRESS>:8006` and sends them to the Splunk
   Observability Cloud.
1. Check [https://docs.fluentd.org/installation/install-by-msi#td-agent-v4](
   https://docs.fluentd.org/installation/install-by-msi#td-agent-v4) to install
   the Fluentd MSI.  Requires version 4.0 or newer.
1. Configure Fluentd to collect log events and forward them to the Collector:
   - Option 1: Update the default config file provided by the Fluentd MSI at
     `\opt\td-agent\etc\td-agent\td-agent.conf` to collect the desired log
     events and [forward](https://docs.fluentd.org/output/forward) them to
     `<IP_ADDRESS>:8006`.
   - Option 2: The installed Collector package provides a custom Fluentd config
     file
     (`\Program Files\Splunk\OpenTelemetry Collector\fluentd\td-agent.conf`) to
     collect log events from the Windows Event Log
     (`\Program Files\Splunk\OpenTelemetry Collector\fluentd\conf.d\eventlog.conf`)
     and forwards them to `<IP_ADDRESS>:8006`.  To utilize these files, backup any
     files as necessary in the `\opt\td-agent\etc\td-agent\` directory, and
     copy the contents from the
     `\Program Files\Splunk\OpenTelemetry Collector\fluentd\` directory to the
     `\opt\td-agent\etc\td-agent\` directory.
1. Apply the changes by restarting the system or by running the following
   Powershell commands to restart the Fluentd service:
   ```sh
   Stop-Service fluentdwinsvc
   Start-Service fluentdwinsvc
   ```
   **Note**: The `fluentdwinsvc` service must be restarted in order for any
   changes made to the Fluentd config files to take effect.
1. The Fluentd service logs and errors can be viewed in
   `\opt\td-agent\td-agent.log`.
1. See [https://docs.fluentd.org/configuration](
   https://docs.fluentd.org/configuration) for general Fluentd configuration
   details.

### Chocolatey Installation

#### Package Parameters
The following package parameters are available:

- `/SPLUNK_ACCESS_TOKEN`: The Splunk access token (org token) used to send metric data to Splunk Observability Suite.
- `/SPLUNK_REALM`: The parameter will be saved to the `\ProgramData\Splunk\OpenTelemetry Collector\SPLUNK_REALM` file. If not specified default is ("us0").
- `/SPLUNK_INGEST_URL:`: URL of the Splunk ingest  (e.g. `https://ingest.SPLUNK_REALM.signalfx.com`). Default value is `https://ingest.us0.signalfx.com`.
- `/SPLUNK_API_URL`: URL of the API endpoint (e.g. `https://api.SPLUNK_REALM.signalfx.com`). Default value is `https://api.us0.signalfx.com`.
- `/SPLUNK_HEC_TOKEN`: Splunk HEC is HTTP Event Collecter token which will collect the metrics and logs of host system in to splunk. Default value is same as `SPLUNK_ACCESS_TOKEN`
- `/SPLUNK_HEC_URL`: URL of Splunk HEC (e.g. `https://ingest.$SPLUNK_REALM.signalfx.com/v1/log`). Default value is `https://ingest.us0.signalfx.com/v1/log`
- `/SPLUNK_TRACE_URL`: Trace url is end point where apllication traces will be collected. URL of Splunk TRACE (e.g. `https://ingest.$SPLUNK_REALM.signalfx.com/v2/trace`). Default value is `https://ingest.us0.signalfx.com/v2/trace`
- `/SPLUNK_BUNDLE_DIR`: The path to the Agent bundle directory. The default path is provided by the Collector package. If the specified path is changed from the default value, the path should be an existing directory on the node. The SPLUNK_BUNDLE_DIR environment variable will be set to this value for the Collector service.
- `/MODE`: The mode option is used for setting config_path to `\ProgramData\Splunk\OpenTelemetry Collector\agent_config.yaml` or `\ProgramData\Splunk\OpenTelemetry Collector\gateway_config.yaml` in OpenTelemetry Collector. Possible values are `agent` and `gateway`. Default value is `agent`.
- `/WITH_FLUENTD`: Whether to install and configure Fluentd to forward log events to the collector. Possible values are `true` and `false`. Default value is `true`. The Fluentd MSI package will be downloaded from `https://packages.treasuredata.com`.

To pass parameters, use `--params "''"` :
```sh
choco install splunk-otel-collector --params="'/SPLUNK_ACCESS_TOKEN:YOUR_SPLUNK_ACCESS_TOKEN /SPLUNK_REALM:YOUR_SPLUNK_REALM'".
```

If the parameter is specified, the keys/values will be created/updated to the system environment registry - `HKLM:\SYSTEM\CurrentControlSet\Control\Session Manager\Environment`.

If the parameter is not specified, the values will be fetch from the system environment registry, and if the system environment registry does not have a key/value in that case Default values will be used.

### Docker

Deploy the latest Docker image:

```bash
$ docker run --rm -e SPLUNK_ACCESS_TOKEN=12345 -e SPLUNK_REALM=us0  `
	-p 13133:13133 -p 14250:14250 -p 14268:14268 -p 4317:4317 -p 6060:6060  `
	-p 8888:8888 -p 9080:9080 -p 9411:9411 -p 9943:9943 `
	--name=otelcol quay.io/signalfx/splunk-otel-collector-windows:latest
```
### Custom Configuration

If using a custom configuration file, you will need to mount the directory containing the file and either use the `SPLUNK_CONFIG=<path>` environment variable or the `--config=<path>` command line argument (replace `<path>` with the path to the custom file within the container).

Example with `SPLUNK_CONFIG`:

```bash
$ docker run --rm -e SPLUNK_ACCESS_TOKEN=12345 -e SPLUNK_REALM=us0 `
	-e SPLUNK_CONFIG=c:\splunk_config\gateway_config.yaml -p 13133:13133  `
	-p 14250:14250 -p 14268:14268 -p 4317:4317 -p 6060:6060 -p 8888:8888 -p 9080:9080 `
	-p 9411:9411 -p 9943:9943 -v ${PWD}\splunk_config:c:\splunk_config:RO `
	--name otelcol quay.io/signalfx/splunk-otel-collector-windows:latest
```

Example with `--config`:

```bash
$ docker run --rm -e SPLUNK_ACCESS_TOKEN=12345 -e SPLUNK_REALM=us0 `
    -p 13133:13133 -p 14250:14250 -p 14268:14268 -p 4317:4317 -p 6060:6060 `
    -p 8888:8888 -p 9080:9080 -p 9411:9411 -p 9943:9943 `
    -v ${PWD}\splunk_config:c:\splunk_config:RO `
    --name otelcol quay.io/signalfx/splunk-otel-collector-windows:latest `
    --config c:\splunk_config\gateway_config.yaml
```

> For mounting configuration files on a windows container, we have to specify a directory name in which the configuration file is present. because just like Linux containers we can not mount files to containers.

repository: uralian/splunk-otel-collector

repo_name: uralian/splunk-otel-collector


text:
### [CVE-2017-7975](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-7975)
![](https://img.shields.io/static/v1?label=Product&message=n%2Fa&color=blue)
![](https://img.shields.io/static/v1?label=Version&message=n%2Fa&color=blue)
![](https://img.shields.io/static/v1?label=Vulnerability&message=n%2Fa&color=brighgreen)

### Description

Artifex jbig2dec 0.13, as used in Ghostscript, allows out-of-bounds writes because of an integer overflow in the jbig2_build_huffman_table function in jbig2_huffman.c during operations on a crafted JBIG2 file, leading to a denial of service (application crash) or possibly execution of arbitrary code.

### POC

#### Reference
- https://bugs.ghostscript.com/show_bug.cgi?id=697693

#### Github
No PoCs found on GitHub currently.

repository: trickest/cve

repo_name: trickest/cve


text:
## Overview
This example shows how http routes can use headers for matching incoming requests.

## Prerequisites
[Walkthrough: App Mesh with EKS](../eks/)

## Setup

1. Clone this repository and navigate to the walkthrough/howto-k8s-cloudmap folder, all commands will be ran from this location
2. **Your** account id:
    ```
    export AWS_ACCOUNT_ID=<your_account_id>
    ```
3. **Region** e.g. us-west-2
    ```
    export AWS_DEFAULT_REGION=us-west-2
    ```
4. **ENVOY_IMAGE** environment variable is set to App Mesh Envoy, see https://docs.aws.amazon.com/app-mesh/latest/userguide/envoy.html
    ```
    export ENVOY_IMAGE=...
    ```
5. Deploy
    ```.
    ./deploy.sh
    ```

repository: dozen/aws-app-mesh-examples

repo_name: dozen/aws-app-mesh-examples