text:
---
layout: engineering-education
status: publish
published: true
url: /regularization-to-prevent-overfitting/
title: Regularization to Prevent Overfitting
description: This article will provide an overview on how to prevent overfitting of data using Regularization. The goal is to understand what overfitting is, and how to prevent them using regularization techniques.
author: collins-ayuya
date: 2021-02-08T00:00:00-17:00
topics: []
excerpt_separator: <!--more-->
images:

  - url: /engineering-education/regularization-to-prevent-overfitting/hero.jpg
    alt: Regularization hero image
---
Overfitting impacts the accuracy of Machine Learning models. The model attempts to capture the data points that do not represent the accurate properties of data. These data points may be considered as noise. To avoid the occurrence of overfitting, we may use a method called regularization.
<!--more-->
In this article, we will mathematically explore two types of regularization.

### Contents
1. Regularization and Overfitting
2. Lasso Regression
3. Ridge Regression

### Prerequisites
As a prerequisite, a basic understanding of Machine Learning is required. To learn more about Machine Learning, read this [article](/engineering-education/supervised-learning-algorithms/). The equations covered in this article are very simple and easy to follow.

### Regularization and overfitting
Overfitting is an occurrence that impacts the performance of a model negatively.

It occurs when a function fits a limited set of data points too closely. Data often has some elements of random noise within it. For example, the training data may contain data points that do not accurately represent the properties of the data. These points are considered as noise.

When a function fits a set of such datapoints too closely, the model learns from noise in the data. This impacts the ability of the model to make reliable predictions on test data. The model may also be unable to fit additional data. This may lead to sub-par performance on test data.

The image below shows the phenomena of overfitting, underfitting, and the correct fit.

![overfitting](/engineering-education/regularization-to-prevent-overfitting/overfitting.png)

*Overfitting, correct fit, and underfitting.*

[Source](https://analyticsindiamag.com/regularization-in-machine-learning-a-detailed-guide/)

For a better understanding of overfitting, bias and variance, check out my previous [article](/engineering-education/ensemble-bias-var/).

Overfitting occurs because a model fails to generalize the data that contains a lot of irrelevant data points. As mentioned in the previous paragraph, data points that do not reflect the properties of the data are considered to be irrelevant. An example is noise. Regularization is the answer to overfitting.

It is a technique that improves model accuracy as well as prevents the loss of important data due to underfitting. When a model fails to grasp an underlying data trend, it is considered to be underfitting. The model does not fit enough points to produce accurate predictions. This means that it is likely to miss out on important data points that may have a telling impact on model accuracy. Hence we say important data may be lost as a result of underfitting.

Regularization is a technique that adds information to a model to prevent the occurrence of overfitting. It is a type of regression that minimizes the coefficient estimates to zero to reduce the capacity (size) of a model. In this context, the reduction of the capacity of a model involves the removal of extra weights.

Regularization removes extra weights from the selected features and redistributes the weights evenly. This means that regularization discourages the learning of a model of both high complexity and flexibility. A highly flexible model is one that possesses the freedom to fit as many data points as possible.

Furthermore, in this context, we may judge the complexity of a predictive model by the number of features it possesses. A model with a lot of features to learn from is at a greater risk of overfitting. By discouraging the learning of (or use of) highly complex and flexible models, the risk of overfitting is lowered.

Let’s use a linear regression equation to explain regularization further.

$$ Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + … + \beta_p X_p $$

$Y$ represents the value that is to be predicted. $\beta_i$ stands for the regressor coefficient estimates for the corresponding predictor $X_i$. And, $X_i$ represents the weights or magnitudes assigned to various predictors (independent variables). Here, `i` represents any value greater than or equal to `0`, and less than `p`.

A loss function is involved in the fitting process. It is computed as the difference between the actual and predicted output from a model. A loss function provides a means of assessing how well an algorithm models given data. It is used to minimize the error, in turn optimizing the weights. In this context, the loss function is referred to as the residual sum of squares (RSS).

Below is the equation for the loss function.

$$ RSS = \sum\limits_{i=1}^n( y_i – \beta_0 - \sum\limits_{j=1}^p {\beta_j x_ij )^2} $$

Based on the training data, the loss function will adjust the coefficients. If the presence of noise or outliers is found in the training data, the approximated coefficients will not generalize well to the unseen data. Regularization comes into play and shrinks the learned estimates towards zero.

In other words, it tunes the loss function by adding a penalty term, that prevents excessive fluctuation of the coefficients. Thereby, reducing the chances of overfitting.

### Lasso regression
Lasso regression is a regularization technique used to reduce model complexity. It is also known as L1 regularization. Lasso stands for Least Absolute Shrinkage and Selector Operator.

Let’s look at the equation below:

$$ \sum\limits_{i=1}^n( y_i – \beta_0 - \sum\limits_{j=1}^p {\beta_j x_ij )^2} + \lambda \sum\limits_{j=1}^p \mid \beta_j \mid = RSS + \lambda \sum\limits_{j=1}^p \mid \beta_j \mid $$

We note that it has a slight variation to the previously discussed loss function, with the introduction of a penalty term. To penalize highly fluctuating coefficients, lasso uses absolute values of the regression coefficients $ (\mid \beta\mid) $.

Lasso minimizes the regression coefficients to regularize the model parameters. Sometimes, Lasso can reduce regression coefficients to zero, which is particularly important when it comes to feature selection.

[Feature selection](https://machinelearningmastery.com/an-introduction-to-feature-selection/) refers to the process of choosing relevant variables and predictors to construct a model. Here, the feature selection process is attributed to the ability of lasso to reduce some regression coefficients to zero. It occurs after the regression coefficients are shrunk.

The predictors whose coefficients are reduced to zero will not be included in the final model. These are the predictors considered to have less importance. This is how some features are eliminated. However, every non-zero regression coefficient is selected for use in the model. This greatly assists in minimizing prediction errors.

Lasso also helps improve the prediction accuracy of the models. The shrinking coefficients minimizes the bias and improves the variance of models. A context that favors the use of lasso is, when we have a small dataset with more number of features.

The tuning parameter $\lambda$ controls the shrinkage. From the equation, when $\lambda$ is zero, the equation is reduced to the linear regression loss function equation. The greater the value of $\lambda$, the greater the reduction of the coefficients will be towards zero.

### Ridge regression
Ridge regression refers to a type of linear regression where in order to get better predictions in the long term, we introduce a small amount of bias. It is also known as L2 regularization. In ridge regression, we have the same loss function with a slight alteration in the penalty term, as shown below:

$$ \sum\limits_{i=1}^n( y_i – \beta_0 - \sum\limits_{j=1}^p {\beta_j x_ij )^2} + \lambda \sum\limits_{j=1}^p \beta_j^2 = RSS + \lambda \sum\limits_{j=1}^p \beta_j^2 $$

As we can see, the main difference between the above equation and the general equation for the loss function is that $ \lambda \sum\limits_{j=1}^p \beta_j^2 $ contains the squared value of the regression coefficients. The tuning parameter is $\lambda$. Higher values of the coefficients represent a model with greater flexibility. To penalize the flexibility of our model, we use a tuning parameter that decides the extent of the penalty. To minimize the function, these coefficients should be small. L2 regularization ensures the coefficients do not rise too high.

When the value of $\lambda$ tends to zero, the L2 regularization equation becomes the loss function of the linear regression model. The penalty term will have no effect. This means that if the value of $\lambda$ is minimum, the model can be called a simple linear regression model.

Conversely, when the $\lambda$ value tends to infinity, the effect of the shrinkage penalty increases. As a result, the coefficient estimates of the ridge regression will approach zero. This underlines the importance of choosing a good value for $\lambda$.

Linear or polynomial regression will likely prove unsuccessful if there is high [collinearity](https://machinelearningmind.com/2019/10/19/multicollinearity-how-to-fix-it/) between the independent variables. Ridge regression is a potential solution to handle multicollinearity.

Collinearity is a condition in which there are two features in data that are heavily correlated to each other. Ridge regression adds an amount of bias to the regression estimates to reduce errors. We shall explore this in a different article.

The predictors of least importance refer to predictors that bear no telling influence on the predictive power of the model. They may also be predictors that do not accurately describe the properties of data, such as noise.

Ridge regression has one clear disadvantage: model interpretability. When shrinking the coefficients of the predictors of least importance, it will reduce them to be close to zero.

Here’s the problem.

As a result, one may end up including all the coefficients in the final model. The final model ends up containing the predictors one may prefer to eliminate. As such, no feature selection is done. Whereas lasso regression manages to force some coefficient estimates to zero when the $\lambda$ is large enough. We can say lasso performs better feature selection.

Furthermore, the resulting models can be considered sparser when compared to the ones resulting from the L2 technique. This is because some of the estimate coefficients can be truly zero. The predictors that are of little to no importance are eliminated thus reducing the number of predictors.

### Wrapping up
Regularization is a formidable technique to prevent overfitting. There are several regularization techniques. We have looked at two of the most popular ones. Through this article, we have understood how the two techniques help in preventing overfitting and how they are derived from the linear regression loss function equation.

Until next time, good luck!

### References and Further Reading
1. [The Elements of Statistical Learning](https://www.springer.com/gp/book/9780387848570)

2. [Regularization in Machine Learning](https://towardsdatascience.com/regularization-in-machine-learning-76441ddcf99a)

3. [Regularization in Machine Learning](https://www.geeksforgeeks.org/regularization-in-machine-learning/)

4. [Regularization In Machine Learning – A Detailed Guide](https://analyticsindiamag.com/regularization-in-machine-learning-a-detailed-guide/)

5. [Regularization in Machine Learning](https://www.javatpoint.com/regularization-in-machine-learning)

6. [What is Regularization in Machine Learning?](https://codeburst.io/what-is-regularization-in-machine-learning-aed5a1c36590)

7. [Regularization Machine Learning](https://www.educba.com/regularization-machine-learning/)

8. [Regularization](https://datascience.eu/machine-learning/regularization-in-machine-learning/)

---

Peer Review Contributions by: [Srishilesh P S](/engineering-education/authors/srishilesh-p-s/)

<!-- MathJax script -->
<script type="text/javascript" async
    src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
    MathJax.Hub.Config({
    tex2jax: {
      inlineMath: [['$','$'], ['\\(','\\)']],
      displayMath: [['$$','$$']],
      processEscapes: true,
      processEnvironments: true,
      skipTags: ['script', 'noscript', 'style', 'textarea', 'pre'],
      TeX: { equationNumbers: { autoNumber: "AMS" },
           extensions: ["AMSmath.js", "AMSsymbols.js"] }
    }
    });
    MathJax.Hub.Queue(function() {
      // Fix <code> tags after MathJax finishes running. This is a
      // hack to overcome a shortcoming of Markdown. Discussion at
      // https://github.com/mojombo/jekyll/issues/199
      var all = MathJax.Hub.getAllJax(), i;
      for(i = 0; i < all.length; i += 1) {
          all[i].SourceElement().parentNode.className += ' has-jax';
      }
    });
    MathJax.Hub.Config({
    // Autonumbering by mathjax
    TeX: { equationNumbers: { autoNumber: "AMS" } }
    });
  </script>

repository: mbuguac/engineering-education

repo_name: mbuguac/engineering-education


text:
+++
# Custom widget.
# An example of using the custom widget to create your own homepage section.
# To create more sections, duplicate this file and edit the values below as desired.
widget = "custom"
active = true
date = 2016-04-20T00:00:00

# Note: a full width section format can be enabled by commenting out the `title` and `subtitle` with a `#`.
title = "My other 'me's"
subtitle = ""

# Order that this section will appear in.
weight = 60

+++

I have a lot of random code online [on GitHub](https://github.com/amcgail),
    I sometimes [tweet](https://twitter.com/somekindofalec),
    I publish analyses on [Rpubs](http://rpubs.com/amcgail/),
    I have some content on [YouTube](http://youtube.com/amcgail/).

license: MIT

repository: amcgail/academic-website

path: /content/home/more about me.md

branch_name: refs/heads/master

extension: md

filename: more about me.md

license_type: permissive

repo_name: amcgail/academic-website


text:
# ParticleLoop
Simple JavaScript website with generated animation of particles on continuous random loop.

It's purpose could be to be displayed in the background of website or just generating some nice-looking giphs.

As mine target I will expand this project in the future and use it on my website.

Test it yourself here https://krzysztoft415.github.io/P.ParticleLoop/

Project was inspired by youtuber The Coding Train and his many great videos.

# How it works
Code is entirely written in JavaScript language.
Firstly it is creating canvas and then array of particles, that afterwards are displayed on the canvas.
Each particle has own unique path, color shift, and change of diameter, all determined by random 2d perlin noise generated each time website is loaded.


text:
# Youtube Music DL

![Continuous Integration](https://github.com/dougppaz/youtube-music-dl/workflows/Continuous%20Integration/badge.svg) ![Continuous Deploy](https://github.com/dougppaz/youtube-music-dl/workflows/Continuous%20Deploy/badge.svg)

Youtube Music Downloader. Chrome Extension to download music file with tags and cover from [Youtube Music](https://music.youtube.com).

# Usage

## Download and Install Chrome Extension

1. Download latest release [here](https://github.com/dougppaz/youtube-music-dl/releases/latest/download/youtube-music-dl.zip).
1. Extract files to directory of your taste.
1. Visit `chrome://extensions` (via omnibox or menu -> Tools -> Extensions).
1. Enable Developer mode by ticking the checkbox in the upper-right corner.
1. Click on the "Load unpacked extension..." button.
1. Select the directory containing extracted files.

## Use Youtube Music DL Extension

### Play a music in [Youtube Music](https://music.youtube.com)

![Play a music in Youtube Music](docs/images/use_ce_1.png)

### Click on Download button

![Click on Download button](docs/images/use_ce_2_1.png)

#### or open Youtube Music DL Extension popup, to choose a format

![Open Youtube Music DL Extension popup, to choose a format](docs/images/use_ce_2_2.png)

### Done

![Done](docs/images/use_ce_3.png)

## Thanks to

- Icons made by [Pixel perfect](https://www.flaticon.com/authors/pixel-perfect) from [www.flaticon.com](https://www.flaticon.com/)

repository: OmgImAlexis/youtube-music-dl

branch_name: refs/heads/main

gha_license_id: MIT

repo_name: OmgImAlexis/youtube-music-dl


text:
# TTAImagePickerController

English | [中文](https://github.com/TMTBO/TTAImagePickerController/blob/master/README_zh.md)

**A Lightweight image selection framework**

[![CI Status](http://img.shields.io/travis/TMTBO/TTAImagePickerController.svg?style=flat)](https://travis-ci.org/TMTBO/TTAImagePickerController)
[![Version](https://img.shields.io/cocoapods/v/TTAImagePickerController.svg?style=flat)](http://cocoapods.org/pods/TTAImagePickerController)
[![License](https://img.shields.io/cocoapods/l/TTAImagePickerController.svg?style=flat)](http://cocoapods.org/pods/TTAImagePickerController)
[![Platform](https://img.shields.io/cocoapods/p/TTAImagePickerController.svg?style=flat)](http://cocoapods.org/pods/TTAImagePickerController)

## Feature

* A Lightweight image selection framework, Low memory consumption
* Support Device orientation and iPad
* Almost identical to the `UIImagePickerController` interface, easy to get started
* Convenient preview function
* A lot of small details

## ScreenShot

![ScreenShot](https://github.com/TMTBO/TTAImagePickerController/blob/master/TTAImagePicker_all.png)
![ScreenShot1](https://github.com/TMTBO/TTAImagePickerController/blob/master/TTAImagePicker_update1.png)

## Example

To run the example project, clone the repo, and run `pod install` from the `Example` directory first.

## Requirements

* iOS > 8.0
* swift >= 5.0

## Installation

* `TTAImagePickerController` is available through [CocoaPods](http://cocoapods.org). To install
it, simply add the following line to your Podfile:`pod "TTAImagePickerController" `

*  Manual import：
	Drag and drop All files in the `TTAImagePickerController` folder to project, import the main file：`import TTAImagePickerController`

## How to use

**1. Create**

```
// Create the image picker with the assets that you had selected which will show as selected in the picker
let imagePicker = TTAImagePickerController(selectedAsset: selectedAssets)
// Set pickerDelegate
imagePicker.pickerDelegate = self
// Set allow take picture in the picker, default: true
imagePicker.allowTakePicture = allowTakePickerSwitch.isOn
// Set allow user delete images in the picker, default: false
imagePicker.allowDeleteImage = allowDeleteImageSwitch.isOn
// Set support large titles for iOS 11
imagePicker.supportLargeTitles = showLargeTitles.isOn
// Set the max pick number, default is 9
imagePicker.maxPickerNum = Int(maxImageCountTextField.text ?? "9") ?? 9

// You can custom the picker apperance
// imagePicker.selectItemTintColor = .red
// imagePicker.barTintColor = .orange
// imagePicker.tintColor = .cyan

present(imagePicker, animated: true, completion: nil)
```

**2. Confirm and implement delegate**

```
// implement the delegate method and when finished picking, you will get the images and assets that you have selected
func imagePickerController(_ picker: TTAImagePickerControllerCompatiable, didFinishPicking images: [UIImage], assets: [TTAAsset]) {
	print("got the images")
	selectedImages = images
	selectedAssets = assets
	imagesCollectionView.reloadData()
}
```

**Extra function you can get**

```
// On the other hand, you can preview the images directly and deselected some of them
// What you need to do:
// Create a instance of `TTAPreviewViewController` (dependency the delegate in step 2)
extension ViewController: UICollectionViewDelegate {
    func collectionView(_ collectionView: UICollectionView, didSelectItemAt indexPath: IndexPath) {
        collectionView.deselectItem(at: indexPath, animated: true)
        let previewVc = TTAPreviewViewController(selected: selectedAssets, index: indexPath.item, delegate: self)
        present(previewVc, animated: true, completion: nil)
    }
}
```

## Recnet Add

* 20190328

> added: add swift5.0 support

* 20170917
> Fix reselected and delete image error
> Add iOS 11 support
> Add iPhone support

* 20170820
> Add Camera Support
> Support Image deleteation in picker

* 20170806
> Gif Support
> Video Support

## What to do next
1. Your advice is welcome

## Author

TobyoTenma<EMAIL_ADDRESS>
## License

TTAImagePickerController is available under the MIT license. See the LICENSE file for more info.

repository: TMTBO/TTAImagePickerController

repo_name: TMTBO/TTAImagePickerController


text:
# Changelog

## [Unreleased]
### Added

### Changed

### Deprecated

### Removed

### Fixed

### Security
## [0.4.0]
### Added
- Added IntelliJ 2021.1 support
### Changed

### Deprecated

### Removed

### Fixed

### Security
## [0.3.0]
### Added
- Added IntelliJ 2020.3 support

### Changed

### Deprecated

### Removed

### Fixed

### Security
## [0.2.0]
### Added

### Changed
- Changed test file and ignored file colors from project tree view

### Deprecated

### Removed
- Removed test file highlighting from editor tabs

### Fixed

### Security
## [0.1.1]
### Fixed
- Fixed default string color to match original theme string color

### Changed
- More balanced color for added lines in diff/merge view

## [0.1.0]
### Added
- First version of the theme, with basic support of languages
  - Java
  - Clojure
  - Kotlin
  - TypeScript
  - JavaScript
  - CoffeeScript
  - HTML/XML
  - CSS/LESS/SASS
  - JSON
  - YAML
  - Dockerfile


text:
# nomad-compose
Use docker-compose yml files as input for Hashicorp's nomad

This script converts docker-compose yml files to nomad job templates

Usage: ./compose2nomad compose.yml [servicename]

This script resolves `extends` stanzas. You can add specific labels the compose file
to influence some nomad-specific options, such as count, constraints, etc.
The implementation depends on the setting. Examples:

```yaml
    nomad.count: 1
    nomad.datacenters:
      - mydc
    nomad.constraints._cluster._type: constraint
    nomad.constraints._cluster.attribute: '${meta.role}'
    nomad.constraints._cluster.value: myrole
    nomad.constraints._distinct._type: constraint
    nomad.constraints._distinct.distinct_hosts: true
```

TODO:

* currently only a single task is generated (while both compose and nomad
  support multiple tasks in a single file)
* add more nomad-specific configuration options

repository: BartWillems/nomad-compose

repo_name: BartWillems/nomad-compose


text:
Compiled example
----------------
![Example](normal-distribution-cumulative-density-function.png)

Requirements
------------

LaTeX and `gnuplot`

Credits
-------

This image is based on the one of [Claudio Fiandrino](http://tex.stackexchange.com/a/60956/5645).

repository: MartinThoma/LaTeX-examples

gha_language: TeX

repo_name: MartinThoma/LaTeX-examples


text:
[![NPM version](https://badge.fury.io/js/cdk-gitlab.svg)](https://badge.fury.io/js/cdk-gitlab)
[![PyPI version](https://badge.fury.io/py/cdk-gitlab.svg)](https://badge.fury.io/py/cdk-gitlab)
![Release](https://github.com/pahud/cdk-gitlab/workflows/Release/badge.svg)

# cdk-gitlab

High level CDK construct to provision GitLab integrations with AWS

# Sample

```ts
import { Provider, FargateJobExecutor, FargateRunner, JobExecutorImage } from 'cdk-gitlab';

const provider = new Provider(stack, 'GitlabProvider', { vpc });

// create a Amazon EKS cluster
provider.createFargateEksCluster(stack, 'GitlabEksCluster', {
  clusterOptions: {
    vpc,
    version: eks.KubernetesVersion.V1_19,
  },
});

// create a default fargate runner with its job executor
provider.createFargateRunner();

// alternatively, create the runner and the executor indivicually.
// first, create the executor
const executor = new FargateJobExecutor(stack, 'JobExecutor', {
  image: JobExecutorImage.DEBIAN,
});

// second, create the runner with the task definition of the executor
new FargateRunner(stack, 'FargateRunner', {
  vpc,
  executor,
});

// TBD - create Amazon EC2 runner for the GitLab
provider.createEc2Runner(...);

});
```

# Fargate Runner with Amazon ECS

On deployment with `createFargateRunner()`, the **Fargate Runner** will be provisioned in Amazon ECS with AWS Fargate and [Amazon ECS Capacity Providers](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/cluster-capacity-providers.html). By default, the `FARGATE` and `FARGATE_SPOT` capacity providers are available for the Amazon ECS cluster and the runner and job executor will run on `FARGATE_SPOT`. You can specify your custom `clusterDefaultCapacityProviderStrategy` and `serviceDefaultCapacityProviderStrategy` properties from the `FargateRunner` construct for different capacity provider strategies.

# Deploy

```sh
cdk deploy -c GITLAB_REGISTRATION_TOKEN=<TOKEN>
```

repository: danielbayerlein/cdk-gitlab

repo_name: danielbayerlein/cdk-gitlab


text:
# LoRaWAN Gateway Scripts

## LoRa-GW-Installer.sh ##

This script is used to install all necessary software packages on a Raspberry PI for building up a LoRaWAN Gateway in conjunction with a connected LoRa Concentrator (iC980-SPI, RHF0M301-SPI, RAK831-SPI or any other by manual setup). If there was any wrong input you can re run the script to update any settings.

Run the script in command line:
```
      sudo bash  LoRa-GW-Installer.sh
```

The script creates a source folder “lora” inside the directory where the installer script is executed. Inside of that “lora” folder the official github projects “packet_forwarder” and “lora_gateway” are cloned and build before a configuration menu appears. Below an example setup:
 ```
------ Configuration ------

Choose LoRa Concentrator from list:

<1> iC980A-SPI
<2> RHF0M301-SPI
<3> RAK831-SPI
<4> Another one. Manual setup required.

Please enter number: 2
 ```
-> The script now creates a start script (./lora/start_lora_gateway.sh) that is used to reset the LoRa concentrator at startup before running the packet-forwarder. Here depending on the used LoRa Concentrator a different reset pin has to be defined. It’s the pin at the RPI where the reset pin of the LoRa Concentrator is connected to.
Manual setup (option 4) would lead to the following configuration options:
 ```
------ Manual setup ------

Name of LoRa concentrator:
My-LoRa-Concentrator

GPIO pin of Raspberry PI where concentrator reset pin is connected to:
16

------ Setting up for My-LoRa-Concentrator concentrator ------
 ```
The configuration then asks for the region of the Gateway. Currently only EU868 and US915 regions are supported:
 ```
------ Setting up for RHF0M301-SPI concentrator ------

Choose frequency band / region of Gateway:
<1> US915
<2> EU868
Please enter number: 1

------ Setting up US915 region ------
```

After choosing the frequency band the corresponding global conf with the frequency patterns is copied from ./lora/packet_forwarder/lora_pkt_fwd/cfg/ to ./lora/packet_forwarder/lora_pkt_fwd/global_conf.json

Then enter the connection details to connect the Gateway to the LoRaWAN Network server:
```
IP/URL of LoRaWAN-Server:
my-lorawan-server.com
LoRaWAN-Server UP Port:
1680
LoRaWAN-Server DOWN Port:
1680

Creating lorawan-gateway.service file in /etc/systemd/system/
Created symlink /etc/systemd/system/multi-user.target.wants/lorawan-gateway.service → /etc/systemd/system/lorawan-gateway.service.

=========================================================================

 SETUP DONE! CONGRATULATIONS! :)
 To start the LoRa Gateway please run the following command in a terminal:

  > gw_start

 or use the created service to run it in the background:

  > sudo service lorawan-gateway start
  > sudo service lorawan-gateway stop
  > sudo service lorawan-gateway status

 Configuration files are located in:
  /usr/bin/start_lora_gateway.sh [SERVICE]
  /home/pi/lora/start_lora_gateway.sh [TERMINAL APP]
  /etc/systemd/system/lorawan-gateway.service
  /home/pi/lora/packet_forwarder/lora_pkt_fwd/global_conf.json
  /home/pi/lora/packet_forwarder/lora_pkt_fwd/local_conf.json

=========================================================================
```

## LoRa-GW-Channel-Setup.sh ##

This script is used to change that the channel configuration the LoRa Gateway is using for listening to LoRaWAN devices.

### Prerequisites

Make sure to follow these steps:

1. Place this script inside the /lora/packet_forwarder/lora_pkt_fwd/ folder where global_conf.json file is located.
2. Configure a **service** for the packet forwarder with the **restart** option. If you used the `LoRa-GW-Installer.sh` script, this is already done.

Run it specifying the [CHANNEL_CONF] as well as the [BAND_REGION] parameter.

### Usage

```
            sudo bash LoRa-GW-Channel-Setup.sh [CHANNEL_CONF] [BAND_REGION]
```

Below you can find a table to identify the correct [CHANNEL_CONF] parameter used to setup a specific channel group
(__setup_freq_0__ and __setup_freq_1__ are the frequency parameters that are written in the global_conf.json file to setup the corresponding channel group.):

US915 BAND:

| CHANNEL_CONF | setup_freq_0 | setup_freq_1  | Channel_1 | Channel_2 | Channel_3 | Channel_4 | Channel_5 | Channel_6 | Channel_7 | Channel_8 |
| ------------- | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- |
| 0 | 902700000 | 903400000 | 902300000 | 902500000 | 902700000 | 902900000 | 903100000 | 903300000 | 903500000 | 903700000 |
| 1 | 904300000 | 905000000 | 903900000 | 904100000 | 904300000 | 904500000 | 904700000 | 904900000 | 905100000 | 905300000 |
| 2 | 905900000 | 906600000 | 905500000 | 905700000 | 905900000 | 906100000 | 906300000 | 906500000 | 906700000 | 906900000 |
| 3 | 907500000 | 908200000 | 907100000 | 907300000 | 907500000 | 907700000 | 907900000 | 908100000 | 908300000 | 908500000 |
| 4 | 909100000 | 909800000 | 908700000 | 908900000 | 909100000 | 909300000 | 909500000 | 909700000 | 909900000 | 910100000 |
| 5 | 910700000 | 911400000 | 910300000 | 910500000 | 910700000 | 910900000 | 911100000 | 911300000 | 911500000 | 911700000 |
| 6 | 912300000 | 913000000 | 911900000 | 912100000 | 912300000 | 912500000 | 912700000 | 912900000 | 913100000 | 913300000 |
| 7 | 913900000 | 914600000 | 913500000 | 913700000 | 913900000 | 914100000 | 914300000 | 914500000 | 914700000 | 914900000 |

EU868 BAND:

| CHANNEL_CONF | setup_freq_0 | setup_freq_1  | Channel_1 | Channel_2 | Channel_3 | Channel_4 | Channel_5 | Channel_6 | Channel_7 | Channel_8 |
| ------------- | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- |
| 0 | 867500000 | 868500000 | 867100000 | 867300000 | 867500000 | 867700000 | 867900000 | 868100000 | 868300000 | 868500000 |
| 1 | 869100000 | 870100000 | 868700000 | 868900000 | 869100000 | 869300000 | 869500000 | 869700000 | 869900000 | 870100000 |

### Examples

```
          sudo bash LoRa-GW-Channel-Setup.sh 0 US915
          sudo bash LoRa-GW-Channel-Setup.sh 1 EU868
```

## Continuous-Channel-Switch.sh ##

This script is used to modify the channel setup periodically for scanning purposes.

### Prerequisites

Make sure to follow these steps:

1. Place this script inside the /lora/packet_forwarder/lora_pkt_fwd/ folder where global_conf.json file is located.
2. Configure a **service** for the packet forwarder with the **restart** option. If you used the `LoRa-GW-Installer.sh` script, this is already done.

### Usage:
```
      sudo bash Continuous-Channel-Switch.sh [OPTIONS]

[Options]:
	-t/--time_interval
		Time interval between each channel switch. [NUM][s/m/h/d] (seconds, minutes, hours or days)
        	-t=1m (for one minute interval)
                -t=3h (for three hour interval)
                --time_interval=7d (for seven days interval)

	 -b/--band
		LoRaWAN region / frequency band that the gateway is operating in.
                -b=US915 (US915 region, 902-928MHz)
                -band=EU868 (EU868 region, 863-870MHz)

	 -c/--channel_conf
		List of channel configurations. '0': channels 0-7, '1': channels 8-15, ...
                -c=0,1,5,3
                --channel_conf=0,1,2,3,4,5,6,7

	 -s/--gateway_service
		Service that runs packet_forwarder and needs to be restarted to effect the changes of new channel setup, default service name is 'lorawan-gateway' if no parameter is set.
                -s=my_own_gateway_service
                --gateway_service=my_own_gateway_service
```
### Examples

This example command goes though every set of channel switching every 3 hours:

```
        sudo bash Continous-Channel-Switch.sh -t=3h -c=0,1,2,3,4,5,6,7 -b=US915
```
More examples:

```
      sudo bash Continuous-Channel-Switch.sh -t=1d -c=0,1,2,3 -b=US915
      sudo bash Continuous-Channel-Switch.sh -t=5h -c=0,1 -s=my-own-gw-service -b=EU868
```

repository: IOActive/laf

gha_language: Python

repo_name: IOActive/laf


text:
# The Recipe of Python Version Management (pyenv) v20160422

This repository contains Dockerfile of [pyenv](https://github.com/yyuu/pyenv) for [Docker](https://www.docker.com/)'s automated build published to the public [Docker Hub Registry](https://hub.docker.com/).

## Base Docker Image

* [repositoryjp/centos:6.7](https://hub.docker.com/r/repositoryjp/centos/)

## How to install Python

If you want to use Python, run below commands to install Python that you want.

[1] Confirm installable versions by pyenv.

	pyenv install --list

[2] Install Python from pyenv.

	# Example:
	#
	# Install Python 2.7.11
	#
	pyenv install 2.7.11

## Installation

1. Install [Docker](https://www.docker.com/).

2. Download automated build from public [Docker Hub Registry](https://hub.docker.com/): `docker pull repositoryjp/pyenv-v20160422:cento-6.7`

## Usage

    docker run -i -t -d -P repositoryjp/pyenv-v20160422:cento-6.7

## License

See the file [LICENSE](../../../../../LICENSE).

## Author

[Shinya Kinoshita](http://www.shinyakinoshita.com) ( [REPOSITORY](http://www.repositories.jp) )

repository: repository-jp/docker-cookbook

gha_language: Nginx

repo_name: repository-jp/docker-cookbook


text:
---
title:      Exercise 4.3-8
clrs:       [4, 3]
date:       2012-08-28 20:35
---

>Using the master method in Section 4.5, you can show that the solution to the recurrence $$T(n) = 4T(n/2) + n^2$$ is $$T(n) = \Theta(n^2)$$. Show that a substitution proof with the assumption $$T(n) \le cn^2$$ fails. Then show how to subtract off a lower-order term to make a substitution proof work.

Let us assume $$T(n) \le cn^2$$ for all $$n \ge n_0$$, where $$c$$ and $$n_0$$ are positive constants.

$$\begin {align}
T(n) & \le 4c\left( \frac n 2\right)^2 + n^2 \\
     & = cn^2 + n^2
\end {align}$$

With this we cannot prove our assumption in it's exact form.

Now, let us assume $$T(n) \le cn^2 - n$$ for all $$n \ge n_0$$, where $$c$$ and $$n_0$$ are positive constants.

$$\begin {align}
T(n) & \le 4\left(c\left( \frac n 2\right)^2 - \frac n 2\right) + n^2 \\
     & = 4\left(c\frac {n^2} 4 - \frac n 2\right) + n^2 \\
     & = cn^2 - 2n + n^2 \\
     & = cn^2 - n - n + n^2 \\
     & = cn^2 - n - n(1 - n)
\end {align}$$

Uh oh! From this we cannot say the last step will be less than or equal to $$cn^2 - n$$ when $$n$$ is sufficiently large.

So, what does that mean?
This means we are trying to prove a wrong assumption. And if you use Master Theorem, you will see the asymptotic bound of the given recurrence is $$\Theta(n^2 \lg n)$$, not $$\Theta(n^2)$$ as given in the problem statement.

So, let us assume $$T(n) \le cn^2 \lg n - n$$ for all $$n \ge n_0$$, where $$c$$ and $$n_0$$ are positive constants.

$$\begin {align}
T(n) & \le 4\left(c\left( \frac n 2\right)^2 \lg \left( \frac n 2\right) - \frac n 2\right) + n^2 \\
     & = 4\left(c\frac {n^2} 4 \lg \left( \frac n 2\right) - \frac n 2\right) + n^2 \\
     & = cn^2\lg \left( \frac n 2\right) - 2n + n^2 \\
     & = cn^2\lg n - cn^2 - 2n + n^2 \\
     & = cn^2\lg n - n - (cn^2 - n^2 + n) \\
     & \le cn^2\lg n - n
\end {align}$$

The last step holds as long as $$c \ge 1$$.
Hence, our assumption is correct.

PS: As usual we have just proved $$O$$-bound. But to show $$\Theta$$-bound, we also need to show $$\Omega$$-bound, which can be done by adding the lower order term instead of subtracting.

repository: AMKohn/CLRS

repo_name: AMKohn/CLRS


text:
# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

## [0.6.0] - 2021-02-20

### Changed

- **Breaking** - Change handles to localized strings
- Pluralize image alt text collection

## [0.5.2] - 2021-01-31

### Added

- Sort order on products for default sorting

### Changed

- **Breaking** - Change protobuf service namespace to prevent message clashes when using multiple services
- **Breaking** - Isolate search sort key enum

## [0.5.1] - 2021-01-30

### Changed

- **Breaking** - Aligned protobuf product service name

## [0.5.0] - 2021-01-30

### Added

- Persist and delete product commands
- In-memory backend persist and delete command handlers

### Changed

- **Breaking** - Migrated to .NET 5
- **Breaking** - Updated refactored and renamed service definition
- **Breaking** - Refactored health check query
- Use Protobuf generated models and services directly instead of mapping and re-implementing services, reduce code required by a lot

### Removed

- **Breaking** - Local entity interfaces, models and mappers, no longer needed
- **Breaking** - ProductService and HealthService

## [0.4.2] - 2020-12-28

### Added

- Image focal center

## [0.4.1] - 2020-12-03

### Changed

- In-memory backend product provider made overridable on startup
- In-memory query handlers now support product collection being null

## [0.4.0] - 2020-08-13

### Changed

- Search term changed to nullable in service definition
- **Breaking** - Refactored search response to provide edge-style results, with each result carrying a cursor

## [0.3.0] - 2020-08-11

### Added

- Image model and proto mapper

### Changed

- **Breaking** - Updated refactored service definition
- **Breaking** - Removed deprecated queries, query handlers and service methods
- **Breaking** - Extending search query, adding cursor-based pagination
- **Breaking** - Changed health-check service name

## [0.2.0] - 2020-07-21

### Added

- Service endpoint, queryies and query handlers for fetching multiple products by category ids

### Changed

- Service endpoint for fetching multiple products by ids or handles pluralized
- Queries and query handlers for fetching multiple products by ids or handles pluralized

## [0.1.2] - 2020-07-16

### Added

- Service endpoint for fetching multiple products by ids or handles
- Queries and query handlers for fetching multiple products by ids or handles

## [0.1.1] - 2020-07-15

### Changed

- Fixed typo in product model price attribute

## [0.1.0] - 2020-07-08

### Added

- CHANGELOG file
- README file describing project
- Azure Pipelines based CI/CD setup
- Product v1 gRPC server implementation and mappers
- Health v1 gRPC server implementation and mappers
- Products models
- Sample application with mock data
- Queries and query handlers for fetching data and running health-checks
- Product service using CQRS for data retrival
- Health service using CQRS for status checks
- In-memory backend providing default query handlers

[unreleased]: https://github.com/SorenA/lightops-commerce-services-product/compare/0.6.0...develop
[0.6.0]: https://github.com/SorenA/lightops-commerce-services-product/tree/0.6.0
[0.5.2]: https://github.com/SorenA/lightops-commerce-services-product/tree/0.5.2
[0.5.1]: https://github.com/SorenA/lightops-commerce-services-product/tree/0.5.1
[0.5.0]: https://github.com/SorenA/lightops-commerce-services-product/tree/0.5.0
[0.4.2]: https://github.com/SorenA/lightops-commerce-services-product/tree/0.4.2
[0.4.1]: https://github.com/SorenA/lightops-commerce-services-product/tree/0.4.1
[0.4.0]: https://github.com/SorenA/lightops-commerce-services-product/tree/0.4.0
[0.3.0]: https://github.com/SorenA/lightops-commerce-services-product/tree/0.3.0
[0.2.0]: https://github.com/SorenA/lightops-commerce-services-product/tree/0.2.0
[0.1.2]: https://github.com/SorenA/lightops-commerce-services-product/tree/0.1.2
[0.1.1]: https://github.com/SorenA/lightops-commerce-services-product/tree/0.1.1
[0.1.0]: https://github.com/SorenA/lightops-commerce-services-product/tree/0.1.0


text:
---
layout: post
title: Complete Business Analysis Course Curriculum
subtitle: A Comprehensive Curriculum for Beginners Business Analyst
tags: [system analysis, business analysis, business analyst]
image: /assets/img/diary.png
bigimg: /assets/img/back.gif
thumbnail-img: /assets/img/diary.png
share-img: /assets/img/diary.png
cover-img: /assets/img/cover.jpg
---

Are you interested in business analysis but don’t know where to start? This complete curriculum covers the essential skills and knowledge required for business analysis and can be completed independently. Whether you are a fresh graduate or have basic IT skills and want to switch to an business analyst career and succeed in your professional development, this high-quality training provides an opportunity to land your first business analyst job.

**Module 1: Introduction to Business Analysis**

In this module, we will introduce you to the role of business analysts, their responsibilities, and the business environment. Business Analyst demand, salary and career roadmap.

- 1.1 Overview of Business Analysis

  - Clarify the importance of business analysis in organizations.
  - Explain the role and responsibilities of business analysts.
  - Discuss the skills and qualities required for successful business analysts.
  - Provide real-world examples and case studies to illustrate the value of business analysis.

- 1.2 Business Analysis Frameworks

  - Provide a comprehensive introduction to popular business analysis frameworks such as BABOK, Agile, and Waterfall.
  - Explain the key principles, processes, and techniques associated with each framework.
  - Discuss the benefits and limitations of different frameworks.
  - Highlight the importance of adapting and tailoring frameworks based on project requirements and organizational context.

Click on any of the links below to access the information directly:

- [x] [**How to Make a Successful Career Transition From Software Testing to Business Analysis**](https://rafayethossain.github.io/2019-01-15-How-to-Tranisition-to-Business-Analysis-from-Software-Testing/)
- [x] [**An Introduction to Business Analysis and the Business Analyst**](https://rafayethossain.github.io/2019-01-22-Introduction-to-Business-Analysis/)

**Module 2: Requirements Gathering and Analysis**
Requirements gathering is a critical part of any IT project, and in this module, you will learn various techniques for eliciting and documenting requirements. You will understand how to prioritize and validate requirements to ensure completeness, consistency, and feasibility. Moreover, you will learn how to create requirement traceability matrices and documentation for future use.

- 2.1 Types of Requirements

  - Explain different types of requirements, including business, functional, non -functional, and technical requirements.
  - Illustrate the relationship between different types of requirements and their impact on project success.
  - Provide examples and case studies to help participants understand and differentiate between various types of requirements.

- 2.2 Eliciting and Documenting Requirements

  - Introduce a variety of techniques for eliciting requirements, such as interviews, surveys, observation, and prototyping.
  - Explain the importance of clear and concise requirement documentation.
  - Provide guidance on creating effective use cases, user stories, and business process models.
  - Demonstrate the use of process modeling techniques like BPMN and UML.

- 2.3 Requirements Analysis and Prioritization

  - Discuss methods for analyzing and prioritizing requirements, including MoSCoW analysis and Kano analysis.
  - Provide practical examples and exercises to help participants practice requirements analysis techniques.
  - Emphasize the importance of involving stakeholders in the analysis and prioritization process.

- 2.4 Requirements Validation and Verification

  - Explain the process of validating and verifying requirements to ensure completeness, consistency, and feasibility.
  - Discuss techniques for conducting process walkthroughs and stakeholder reviews.
  - Introduce the concept of requirement traceability matrices and their role in managing requirements throughout the project lifecycle.

- 2.5 Collaboration with Development Teams

  - Emphasize the importance of collaboration between business analysts and development teams.
  - Discuss strategies for effective collaboration during software design and architecture activities.
  - Highlight the role of business analysts in defining software system boundaries and interfaces.

Click on any of the links below to access the information directly:

- [x] [**Everything You Need to Kow About Software Requirements**](https://rafayethossain.github.io/2019-03-03-What-is-Software-Requirements/)
- [x] [**Requirements Elicitation Techniques You Must Learn**](https://rafayethossain.github.io/2019-03-30-Requirement-Elicitation-Complete-Guidelines/)
- [x] [**How a Pro BA Analyze Software Requirements**](https://rafayethossain.github.io/2019-04-04-Requirement-Analysis-Guidelines/)
- [x] [**BPMN Basics for Business Analyst**](https://rafayethossain.github.io/2019-04-20-BPMN-Basic-Guidelines-with-Example/)
- [x] [**Requirements Specification Techniques**](https://rafayethossain.github.io/2019-05-01-Requirement-Specification-Techniques/)
- [x] [**Preparing a Software Requirement Specification (SRS) Document That Works**](https://rafayethossain.github.io/2019-05-07-How-to-Write-SRS-Document/)
- [x] [**Software Requirements Approval to Kick off Development**](https://rafayethossain.github.io/2019-06-06-Requirement-Approval-Process/)

**Module 3: Project Management for Business Analysts**
In this module, we will cover project management concepts and methodologies such as Waterfall, Agile, and Scrum. You will learn how to prepare a business case, create a project charter, and manage project scope. Moreover, you will learn how to perform stakeholder analysis, communication planning, risk management, and contingency planning.

- 3.1 Introduction to Project Management Concepts, Project Lifecycles and Methodologies

  - Provide an overview of project management concepts and methodologies commonly used, including Waterfall, Agile, and Scrum.
  - Explain the role of business analysts in each phase of the project lifecycle.
  - Discuss the advantages and disadvantages of different methodologies and their impact on business analysis activities.
  - Emphasize the need for flexibility and adaptability in selecting and applying methodologies.
  - Discuss the importance of project planning, tracking, and control.

- 3.2 Business Case & Project Charter

  - Explain the process of preparing a business case and creating a project charter.
  - Provide guidance on identifying project objectives, benefits, risks, and constraints.
  - Discuss the role of business analysts in defining project scope and objectives.

- 3.3 Stakeholder Analysis and Communication Planning

  - Discuss techniques for stakeholder analysis and identification.
  - Explain the importance of effective communication planning in project success.
  - Provide guidance on creating stakeholder communication plans.
  - Discuss effective communication and collaboration techniques for business analysts.
  - Provide practical tips for engaging stakeholders and building productive relationships.
  - Highlight the importance of active listening, empathy, and assertiveness in stakeholder interactions.
  - Introduce tools and techniques for facilitating effective communication, such as meetings, presentations, and documentation.

- 3.4 Risk Management and Contingency Planning

  - Introduce the concepts of risk management and contingency planning.
  - Discuss techniques for identifying, assessing, and mitigating project risks.
  - Explain the role of business analysts in risk management activities.

- 3.5 Feasibility Studies and Cost -Benefit Analysis

  - Explain the process of conducting feasibility studies and cost -benefit analysis.
  - Provide guidance on evaluating potential solutions and making informed recommendations.
  - Highlight the role of business analysts in assessing the feasibility and viability of project options.

- 3.6 Collaboration with Technical Teams

  - Emphasize the importance of collaboration between business analysts and technical teams.
  - Discuss strategies for effective collaboration during solution evaluation and selection.
  - Highlight the role of business analysts in presenting recommendations and business cases to technical stakeholders.

Click on any of the links below to access the information directly:

- [x] [**Project Initiation Activities as a Business Analyst**](https://rafayethossain.github.io/2019-02-07-Project-Initiation-Business-Analysis-Activities/)
- [x] [**Project Inititation Activities**](https://rafayethossain.github.io/2019-02-25-How-to-Prepare-Business-Case-Business-Analyst/)
- [x] [**How to Prepare a Business Case for Project**](https://rafayethossain.github.io/2019-02-25-How-to-Prepare-Business-Case-Business-Analyst/)
- [x] [**How to Manage Your Project Stakeholder Using RACI Matrix**](https://rafayethossain.github.io/2019-02-27-Stakeholder-Management-Business-Analyst/)

- [x] [**Change Control Process in SDLC**](https://rafayethossain.github.io/2019-07-07-Change-Control-Process-in-SDLC/)

**Module 4: Agile and Lean Methodologies for Business Analysis**
In this module, we will introduce you to Agile and Lean methodologies and frameworks such as Scrum, Kanban, and Lean Six Sigma. You will learn how to collaborate with cross-functional teams, perform iterative and incremental development, and apply Agile requirements gathering and analysis. Moreover, you will learn how to apply Lean Six Sigma principles for continuous improvement.

- 4.1 Overview of Agile and Lean Methodologies

  - Provide a comprehensive introduction to Agile and Lean methodologies and frameworks, including Scrum, Kanban, and Lean Six Sigma.
  - Explain the key principles, values, and practices associated with Agile and Lean.
  - Discuss the benefits and challenges of implementing Agile and Lean methodologies.

- 4.2 Agile Requirements Gathering and Analysis

  - Explain how requirements gathering and analysis activities are performed in Agile environments
  - Discuss the role of business analysts in Agile teams and their interaction with cross-functional stakeholders.
  - Provide examples of Agile artifacts and techniques, such as user stories, product backlogs, and sprint planning.
  - Highlight the iterative and incremental nature of Agile requirements gathering and analysis.

  - 4.3 Collaboration and Team Dynamics in Agile Environments

  - Discuss strategies for effective collaboration and team dynamics in Agile environments.
  - Highlight the role of business analysts in facilitating cross-functional collaboration.
  - Provide guidance on resolving conflicts and managing stakeholder expectations in Agile teams.

- 4.4 Continuous Improvement and Lean Six Sigma Principles

  - Explain the concept of continuous improvement and its relevance to Agile and Lean methodologies.
  - Introduce Lean Six Sigma principles and their application in process improvement.
  - Discuss techniques for identifying and eliminating waste in business processes.

Click on any of the links below to access the information directly:

- [x] [**Agile Project Managment with Srum Roles and Responsibilites**](https://rafayethossain.github.io/2022-10-10-Agile-Scrum-in-a-Nutshell/)

**Module 5: Communication and Collaboration Skills**
In this module, we will cover the importance of effective communication and collaboration in business analysis. You will learn various communication and collaboration tools and techniques used in business analysis. You will also learn how to develop persuasive arguments and recommendations using evidence and logic. Moreover, you will learn how to present complex information effectively.

- 5.1 Importance of Effective Communication and Collaboration

  - Discuss the critical role of effective communication and collaboration in business analysis.
  - Highlight the impact of poor communication on project success.
  - Provide examples and case studies to illustrate the importance of effective communication and collaboration.

- 5.2 Communication and Collaboration Tools and Techniques

  - Introduce a range of communication and collaboration tools and techniques used in business analysis.
  - Provide practical guidance on selecting and using appropriate tools for different communication purposes
  - Discuss the benefits and limitations of different communication channels, such as meetings, emails, and collaboration platforms.

- 5.3 Effective Written and Verbal Communication Techniques

  - Explain techniques for effective written and verbal communication in business analysis.
  - Discuss the importance of active listening, empathy, and assertiveness in effective communication.
  - Provide tips for improving written communication skills, including clarity, conciseness, and grammar.

- 5.4 Presentation Skills for Business Analysts

  - Provide guidance on presenting complex information effectively.
  - Introduce tools and techniques for creating engaging and informative presentations.
  - Discuss best practices for delivering presentations and handling questions from stakeholders.

- 5.5 Developing Persuasive Arguments and Recommendations
  - Explain the importance of developing persuasive arguments and recommendations based on evidence and logic.
  - Discuss techniques for gathering and analyzing evidence to support recommendations.
  - Provide guidance on structuring arguments and communicating recommendations effectively.

**Module 6: Business Analysis Tools and Techniques**
In this module, we will introduce you to popular business analysis tools such as Jira, Trello, and Asana. You will learn how to use Business Process Management (BPM) tools such as Signavio or Bizagi. You will also learn how to prototype and wireframe using tools such as Figma, Adobe XD, or Sketch. Moreover, you will learn how to perform UML modeling and diagramming using tools such as Visio or Lucidchart.

- 6.1 Overview of Popular Business Analysis Tools

  - Introduce popular business analysis tools such as Jira, Trello, and Asana.
  - Discuss the benefits and limitations of different tools for different analysis activities.
  - Provide practical examples and use cases to illustrate the application of business analysis tools.

- 6.2 Business Process Management (BPM) Tools

  - Introduce Business Process Management (BPM) tools such as Signavio or Bizagi.
  - Explain the role of BPM tools in process modeling, analysis, and improvement.
  - Provide hands -on exercises and tutorials for using BPM tools effectively.

- 6.3 Prototyping and Wireframing Tools

  - Introduce prototyping and wireframing tools such as Figma, Adobe XD, or Sketch.
  - Explain the benefits of prototyping and wireframing in requirements validation and stakeholder communication.
  - Provide practical examples and tutorials for creating prototypes and wireframes using different tools.

- 6.4 UML Modeling and Diagramming Tools

  - Introduce UML modeling and diagramming tools such as Visio or Lucidchart.
  - Explain the role of UML diagrams in requirements specification and software design.
  - Provide hands-on exercises and tutorials for creating UML diagrams using different tools.

- 6.5 Best Practices for Selecting and Using Business Analysis Tools

  - Discuss best practices for selecting and evaluating business analysis tools.
  - Provide guidance on tool adoption, integration, and maintenance.
  - Highlight considerations for tool licensing, scalability, and security.

**Module 7: Data Analysis and Visualization**
In this module, we will cover the fundamentals of data analysis and modeling. You will learn how to collect, clean, and process data using tools such as Excel, Python, or R. You will also learn how to visualize data and design dashboards using tools such as Tableau or Power BI. Moreover, you will learn about data warehousing and business intelligence.

- 7.1 Data Gathering and Analysis Techniques

  - Introduce techniques for gathering and analyzing data in business analysis.
  - Discuss methods for data collection, data cleaning, and data validation.
  - Provide guidance on selecting appropriate data analysis techniques for different scenarios.

- 7.2 Data Modeling and Data Flow Diagrams

  - Explain the concepts of data modeling and data flow diagrams.
  - Provide practical examples and exercises for creating data models and diagrams.
  - Discuss the role of business analysts in data modeling and analysis activities.

- 7.3 Assessing Solution Feasibility and Cost-Benefit Analysis

  - Discuss techniques for assessing solution feasibility and conducting cost-benefit analysis.
  - Highlight the role of business analysts in evaluating the economic viability of proposed solutions.
  - Provide guidance on considering factors such as project costs, benefits, and risks in the analysis.

- 7.4 Data Visualization and Dashboard Design

  - Introduce data visualization techniques and principles.
  - Discuss the role of data visualization in conveying complex information effectively.
  - Provide hands-on exercises and tutorials for creating data visualizations and designing dashboards.

- 7.5 Introduction to Data Warehousing and Business Intelligence

  - Explain the concepts of data warehousing and business intelligence.
  - Discuss the role of business analysts in data warehouse design and implementation.
  - Provide examples and case studies of business intelligence applications in business analysis.

**Module 8: Testing for Business Analysts**
In this module, we will cover the introduction to quality assurance, creating test plans, and manual and automated testing.

- 8.1 Introduction to Quality Assurance

  - Explain the concept of quality assurance and its importance in software development.
  - Discuss the role of business analysts in ensuring software quality through quality assurance practices.
  - Provide an overview of software testing methodologies and techniques.

- 8.2 Creating Test Plans

  - Explain the process of creating test plans for software projects.
  - Discuss the components of a test plan and their importance.
  - Provide guidance on identifying test objectives, test strategies, and test cases.

- 8.3 Manual and Automated Testing

  - Introduce the concepts of manual and automated testing.
  - Discuss the advantages and limitations of each testing approach.
  - Provide examples and case studies to illustrate the application of manual and automated testing.

- 8.4 Collaboration with QA Teams

  - Discuss the importance of collaboration between business analysts and quality assurance (QA) teams
  - Highlight the role of business analysts in providing requirements input for testing activities.
  - Provide guidance on effective collaboration and communication with QA teams.

- 8.5 User Acceptance Testing (UAT)

  - Explain the purpose and process of user acceptance testing (UAT).
  - Discuss the role of business analysts in facilitating UAT activities.
  - Provide guidance on creating UAT plans, test cases, and conducting UAT sessions.

**Module 9: Change Management**

- 9.1 Change Management Fundamentals

  - Introduce the concepts of change management and its importance in IT projects.
  - Discuss the role of business analysts in change management activities.
  - Provide an overview of change management frameworks and techniques.

- 9.2 Change Impact Assessment and Stakeholder Analysis

  - Explain the process of assessing change impact and conducting stakeholder analysis.
  - Discuss techniques for identifying and managing resistance to change.
  - Provide guidance on creating change management plans and communication strategies.

- 9.3 Training and Adoption Strategies

  - Discuss strategies for training and adoption of new systems or processes.
  - Highlight the role of business analysts in developing training materials and conducting training sessions.
  - Provide guidance on monitoring and evaluating the effectiveness of training programs.

- 9.4 Transition Planning and Support

  - Explain the importance of transition planning and support in project implementation.
  - Discuss techniques for managing the transition from the current state to the future state.
  - Provide guidance on creating transition plans, conducting readiness assessments, and managing risks during implementation.

- 9.5 Post-Implementation Review and Lessons Learned

  - Explain the importance of post-implementation review and lessons learned.
  - Discuss techniques for conducting post-implementation reviews and capturing lessons learned.
  - Provide guidance on identifying areas for improvement and implementing corrective actions.

**Module 10: Professional Development**
Our commitment to your success doesn't stop with the end of the course. We understand that navigating the job market can be challenging. In this, we will guide you to land your first BA role

- 10.1 Preparing for Job Interviews

  - Provide guidance on resume preparation and tailoring it to the business analyst role.
  - Discuss common interview questions and techniques for providing effective responses.
  - Offer tips for showcasing relevant skills, experiences, and accomplishments during interviews.

- 10.2 Building a Professional Network

  - Highlight the importance of building a professional network in the field of business analysis.
  - Provide strategies for networking, including attending industry events, joining professional associations, and utilizing online platforms.
  - Offer guidance on building and maintaining professional relationships.

- 10.3 Navigating the Job Market in Business Analysis

  - Discuss current trends and job market dynamics in business analysis.
  - Provide insights into job search strategies, including leveraging job boards, social media, and professional networks.
  - Offer guidance on identifying and evaluating job opportunities.

- 10.4 Creating a Job Search Strategy

  - Guide students in developing a personalized job search strategy based on their career goals and aspirations.
  - Provide resources and techniques for identifying suitable job openings.
  - Offer guidance on tailoring application materials and leveraging personal connections during the job search process.

- 10.5 Resume Preparation

  - Offer detailed guidance on creating an effective and targeted resume for business analyst positions.
  - Provide tips for highlighting relevant skills, experiences, and achievements.
  - Offer insights into resume formatting, structure, and key sections.

- 10.6 Job Interview Preparation

  - Offer comprehensive strategies for preparing for job interviews, including research, self-assessment, and mock interviews.
  - Provide guidance on presenting oneself professionally and confidently during interviews.
  - Offer techniques for answering behavioral and technical interview questions.

    10.7 Behavioral Interviewing Techniques

  - Explain the concept of behavioral interviewing and its significance in the hiring process.
  - Offer guidance on effectively answering behavioral questions, providing specific examples from past experiences.
  - Provide tips for demonstrating key competencies and skills sought by employers.

- 10.8 Handling Technical Interview Questions

  - Discuss common technical interview questions in the field of business analysis.
  - Provide guidance on answering technical questions effectively, showcasing technical knowledge and problem-solving abilities.
  - Offer tips for addressing unfamiliar or challenging technical questions.

By the end of the course, you will have the skills and tools to become a successful business analyst. Don't miss out on this opportunity to advance your career. Contact me now to enroll in this course and take the first step towards becoming an Business Analyst.

_Please note that this is an extended outline based on the given course. You may adapt and modify it further based on your specific needs, target audience, and course duration_

**Course Materials and Resources:**

- Slide decks and lecture notes for each module
- Recommended readings, articles, and online resources
- Templates and tools for business analysis artifacts (e.g., requirements templates, process modeling software)
- Real-world case studies and scenarios
- Access to relevant software and tools during the course

**Assessment and Grading:**

- Regular quizzes and assignments to assess understanding of key concepts
- Evaluation of practical exercises, such as requirement documentation and process models
- Analysis and presentation of real-world case studies
- Final exam covering the course material

**Prerequisites:**
There are no specific prerequisites for this course. However, basic knowledge of business processes and familiarity with software applications is beneficial.

# About Content Creator:

Hi, This is **Rafayet Hossain**

A Seasoned Business Systems Analyst, Project Manager, and SQA Engineer with experience in driving digital changes within organizations. I specialize in understanding business needs and developing software solutions to improve processes and drive growth. I am skilled in managing projects, analyzing data, and ensuring quality in the final product. I am passionate about using my expertise to help organizations reach their goals and succeed. Let's work together to improve your business and drive success.

So, if you're ready to take your career to the next level, don't wait - contact me today to learn more about my course and how it can help you achieve your goals!

👉 Contact Information :

[![Linkedin](https://img.shields.io/badge/-LinkedIn-blue?style=flat&logo=Linkedin&logoColor=white)](https://www.linkedin.com/in/rafayethossain/)
<EMAIL_ADDRESS>
---

---

{: .box-success}
Enjoy !!!
**See Yaaa, Next.**

![Diary](/assets/img/diary.png "Diary")


text:
# Game Engine
Groovy/Java Game Engine

## Warnings

This engine is under hard development. We do not recommand its uses for production.

Last stable version: [v0.1](https://github.com/scorsi/gameengine/tree/v0.1)

Developpement version: [dev branch](https://github.com/scorsi/gameengine/tree/dev)

## Useful links

- See [com/scorsi/example](https://github.com/scorsi/gameengine/tree/master/src/main/groovy/com/scorsi/example) package for implementation example
- [Wiki pages](https://github.com/scorsi/gameengine/wiki)
- [Releases notes](https://github.com/scorsi/gameengine/releases)


text:
# digimon-checksum-fixer

A tool to fix checksum on Digimon All-Star Rumble (PS3) save-games.

### Credits

This tool is based on the [Security Archive](https://community.wemod.com/t/philymasters-security-archive/3923) notes by Philymaster


text:
# DEPRECATED
# tahapi

Tahapi is an IONIC android app which has written in 2 weeks with IONIC(1.x) with zero IONIC knowledge. Android platform files has been removed for reducing the file size. One must add android platform to project before testing.

```
//adding android platform cli code
$ ionic cordova add android
```
Crosswalk bugs can be seen because of the latest updates on IONIC CLI and version changes. This project actually added to the github as a code reference. A working version of project resides on google play.

App's main purpose is listing images and videos which acquired from an web api as JSON. Users can comment and rate these pictures and videos. Also users can upload images or videos from  to server from their device galleries or taking them on fly. Big videos are compressed with Cordova-Video-Plugin on the background.

Users can log in the app with google or facebook.

Api keys must inserted at right locations for this purpose.

```
//Facebook
$cordovaOauth.facebook("[FACEBOOK_API_KEY]", ["email", "user_website", "user_location", "user_relationships"]).then(function(result) {...

//Google
 $cordovaOauth.google("[GOOGLE_API_KEY]", ["email"]).then(function(result) {...
```
Application gets its data from an web api with $http.get . But in this version of project api links left blank.

```
$http.get('[WEB-API-SERVER]', {params: para})...
```

Project is developed in very limited time with zero knowledge of library.

Never tested on IOS

repository: dahico-dk/tahapi

repo_name: dahico-dk/tahapi


text:
# T1 and T2 Tracks

## Table Of Contents

- [Introduction](#introduction)
- [For Participants](#for_participants)
  - [Getting Started](#getting_started)
  - [Starting Your Development](#starting_your_development)
  - [Developing Your Dockerfile](#developing_your_dockerfile)
  - [Developing Your Algorithm](#developing_your_algorithm)
  - [Measuring your Algorithm](#measuring_your_algorithm)
  - [How To Get Help](#how_to_get_help)
  - [Leaderboard](#leaderboard)
- [For Evaluators](#for_organizers)

## Introduction

The T1 and T2 tracks evaluate algorithms on standardized Azure CPU servers.

**Track 1**: In-memory indices with [FAISS](https://github.com/facebookresearch/faiss) as the baseline. Search would use Azure [Standard_F32s_v2 VMs](https://docs.microsoft.com/en-us/azure/virtual-machines/fsv2-series) with 32 vCPUs and 64GB RAM.

**Track 2:** Out-of-core indices with [DiskANN](https://github.com/Microsoft/diskann) as the baseline. In addition to the limited DRAM in T1, index can use an SSD for search. Search would use Azure [Standard_L8s_v2 VMs](https://docs.microsoft.com/en-us/azure/virtual-machines/lsv2-series) with 8 vCPUS, 64GB RAM and a local SSD Index constrained to 1TB.

Index construction for both tracks would use Azure [Standard_F64s_v2 VM](https://docs.microsoft.com/en-us/azure/virtual-machines/fsv2-series) with 64vCPUs, 128GB RAM and an additional [4TB of Premium SSD](https://docs.microsoft.com/en-us/azure/virtual-machines/disks-types) to be used for storing the data, index and other intermediate data. There is a **time limit for 4 days per dataset** for index build.

Queries will be supplied in one shot and the algorithm can execute the queries in any order.

We will release plots for recall vs QPS separately for tracks T1 and T2.
Additionally, we will release leaderboards for T1 and T2. The  metric for the leaderboard in each track will be the sum of improvements in recall over the baseline at the target QPS over all datasets. **The target recall
for T1 is 10000 QPS and for T2 is 1500 QPS.**

Participants must submit their algorithm via a pull request and (optionally) index file(s) upload (one per participating dataset).

## For_Participants

### Requirements

You will need the following installed on your machine:
* Python ( we tested with Anaconda using an environment created for Python version 3.8 ) and Docker.
* Note that we tested everything on Ubuntu Linux 18.04 but other environments should be possible.

### Getting_Started

This section will present a small tutorial about how to use this framework and several of the key scripts you will use throughout the development of your algorithm and eventual submission.

First, clone this repository and cd into the project directory:
```
git clone <REPO_URL>
cd <REPO_URL>
```
Install the python package requirements:
```
pip install -r requirements_py38.txt
```
Create a small, sample dataset.  For example, to create a dataset with 10000 20-dimensional random floating point vectors, run:
```
python create_dataset.py --dataset random-xs
```
To create a smaller slice of the competition datasets (e.g. 10M slice of deep-1B), run:
```
python create_dataset.py --dataset deep-10M
```
To see a complete list of datasets, run the following:
```
python create_dataset.py --help
```

For T2, set up the local SSD on Azure Ls8v2 machine by running the following under sudo.
```
parted /dev/nvme0n1 mklabel gpt mkpart primary 0% 100%;
mkfs -t ext4 /dev/nvme0n1;
mkdir /nvme;
mount /dev/nvme0n1 /nvme/;
echo "/dev/nvme0n1 /nvme  ext4   defaults    0   0" >> /etc/fstab;
```
You might also want to create a symbolic link to a folder under `/nvme`
```
sudo mkdir /nvme/data
sudo chmod 777 /nvme/data
ln -s /nvme/data data
```

Build the docker container for the T1 or T2 baselines:
```
#for T1
python install.py --algorithm faissconda
#for T2
python install.py --algorithm diskann
```
Run a benchmark evaluation using the algorithm's definition file:
```
python run.py --algorithm faiss_t1 --dataset random-xs
python run.py --algorithm diskann-t2 --dataset random-xs
```

For the competition dataset (e.g. deep-1B), running the following command downloads a prebuilt index and runs the queries locally.
```
python run.py --algorithm faiss_t1 --dataset deep-1B
python run.py --algorithm diskann-t2 --dataset deep-1B
```

Now plot QPS vs recall:
```
python plot.py --algorithm faiss_t1 --dataset random-xs
python plot.py --algorithm diskann-t2 --dataset random-xs
```
This will place a plot into the *results/* directory.

### Starting_Your_Development

First, please create a short name for your team without spaces or special characters.  Henceforth in these instructions, this will be referenced as [your_team_name].

Create a custom branch off main in this repository:
```
# For t1
git checkout -b t1/[your_team_name]
# For t2
git checkout -b t2/[your_team_name]
```

### Developing_Your_Dockerfile

This framework evaluates algorithms in Docker containers by default.  Your algorithm's Dockerfile should live in *install/Docker.[your_team_name]*.  Your Docker file should contain everything needed to install and run your algorithm on a system with the same hardware.

Please consult the Dockerfiles [here](../install/Dockerfile.faissconda) and [here](../install/Dockerfile.diskann) for examples.

To build your Docker container, run:
```
python install.py --install [your_team_name]
```

### Developing_Your_Algorithm

Develop and add your algorithm's python class to the [benchmark/algorithms](../benchmark/algorithms) directory.
* You will need to subclass from the [BaseANN class](../benchmark/algorithms/base.py) and implement the functions of that parent class.
* You should consult the examples already in the directory.
* If it is difficult to write a Python wrapper, please consult [HttpANN](../benchmark/algorithms/httpann_example.py) for a RESTful API.

When you are ready to test on the competition datasets, use the create_dataset.py script as follows:
```
python create_dataset.py --dataset [sift-1B|bigann-1B|text2image-1B|msturing-1B|msspacev-1B|ssnpp-1B]
```
To benchmark your algorithm, first create an algorithm configuration yaml in your teams directory called *algos.yaml.*  This file contains the index build parameters and query parameters that will get passed to your algorithm at run-time.  Please look at [algos.yaml](../algos.yaml).

If your machine is capable of both building and searching an index, you can benchmark your algorithm using the run.py script.
```
python run.py --algorithm diskann-t2 --dataset deep-1B
```
This will write the results to the toplevel [results](../results) directory.

To build the index and upload it to Azure cloud storage without querying it:
```
python run.py --algorithm diskann-t2 --dataset deep-1B --upload-index --blob-prefix <your Azure blob container path> --sas-string <your sas string to authenticate to blob>
```
To download the index from cloud storage and query it on another machine:
```
python run.py --algorithm diskann-t2 --dataset deep-1B --download-index --blob-prefix <your Azure blob container path> --sas-string <your sas string to authenticate to blob>
```

### Measuring_Your_Algorithm

Now you can analyze the results using plot.py. Sudo might be required here. To avoid sudo, run `sudo chmod -R 777 results/` before invoking these scripts.
```
python plot.py --algorithm [your_team_name] --dataset deep-1B
```
This will place a plot of the algorithms performance into the toplevel [results](../results) directory.

The plot.py script supports other benchmarks.  To see a complete list, run:
```
python plot.py --help
```

You can plot additional metrics, e.g. mean SSD IOs vs recall/AP for T2, using:
```
python plot.py --dataset deep-1B -x k-nn -y mean_ssd_ios
```

Here are all the FAISS baseline recall@10 (AP for SSNPP) vs throughput plots for T1:
* [msturing-1B](results/T1/msturing-1B.png)
* [bigann-1B](results/T1/bigann-1B.png)
* [text2image-1B](results/T1/text2image-1B.png)
* [deep-1B](results/T1/deep-1B.png)
* [msspacev-1B](results/T1/msspacev-1B.png)
* [ssnpp-1B](results/T1/ssnpp-1B.png)

Here are all the DiskANN baseline recall@10 (AP for SSNPP) vs throughput plots for T2:
* [msturing-1B](results/T2/msturing-1B.png)
* [bigann-1B](results/T2/bigann-1B.png)
* [text2image-1B](results/T2/text2image-1B.png)
* [deep-1B](results/T2/deep-1B.png)
* [msspacev-1B](results/T2/msspacev-1B.png)
* [ssnpp-1B](results/T2/ssnpp-1B.png)

Here are all the DiskANN baseline recall@10 (AP for SSNPP) vs mean SSD IOs plots for T2:
* [msturing-1B](results/T2/msturing-1B-IO.png)
* [bigann-1B](results/T2/bigann-1B-IO.png)
* [text2image-1B](results/T2/text2image-1B-IO.png)
* [deep-1B](results/T2/deep-1B-IO.png)
* [msspacev-1B](results/T2/msspacev-1B-IO.png)
* [ssnpp-1B](results/T2/ssnpp-1B-IO.png)

To get a table overview over the best recall/ap achieved over a certain QPS threshold, run
```
python3 data_export.py --output res.csv
python3 eval/show_operating_points.py --algorithm $ALGO --threshold $THRESHOLD res.csv
```

For the track1 baseline, the output, this led to

```
                         recall/ap
algorithm dataset
faiss-t1  bigann-1B       0.634510
          deep-1B         0.650280
          msspacev-1B     0.728861
          msturing-1B     0.703611
          ssnpp-1B        0.753780
          text2image-1B   0.069275
```

For the track2 baseline, this led to

```
                         recall/ap
algorithm   dataset
diskann-t2  bigann-1B       0.94913
            deep-1B         0.93706
            msspacev-1B     0.90095
            msturing-1B     0.93564
            ssnpp-1B        0.16274
            text2image-1B   0.48854
```

### Submitting_Your_Algorithm

A submission is composed of a pull request to this repo with the following.
* Your algorithm's python class, inheriting from `BaseANN`, placed in the [benchmark/algorithms/](../benchmark/algorithms) directory.
* A Dockerfile in `install/` describing how to retrieve, compile and set up requirements for your algorithm.
* For each dataset you are participating in, add to [algos.yaml](../algos.yaml)
  * 1 index build configuration
  * 10 search configuration
* Add an entry to [CI test list](../.github/workflows/benchmarks.yml) for the random-xs dataset, and for the random-range-xs dataset if your algorithm supports range search. We can start working with larger datasets once these tests pass.
* An URL to download any prebuilt indices placed in `algos.yaml`. **This is optional, but strongly encourages.** This would help us evaluate faster, although we would build your index to verify the time limit. Please see `faiss_t1.py` and `diskann-t2.py` for examples. If you are unable to host the index on your own Azure blob storage, please let us know and we can arrange to have it copied to organizer's account.

We will run early PRs on organizer's machines to the extent possible and provide any feedback necessary.

### How_To_Get_Help

There are several ways to get help as you develop your algorithm using this framework:
* You can submit an issue at this github repository.
* Send an email to the competition's T1/T2 organizer<EMAIL_ADDRESS>* Send en email to the competition's googlegroup<EMAIL_ADDRESS>
### Leaderboard

This leaderboard is based on the standard recall@10 vs throughput benchmark that has become a standard benchmark when evaluating and comparing approximate nearest neighbor algorithms. We have run FAISS and DiskANN as baselines for T1 and T2 respectively, and for each dataset chosen the best recall amongst configurations providing at least **10K QPS for T1 and 1500 QPS for T2**.  The recall of the baselines at this QPS threshold is listed [above](#measuring_your_algorithm).

Algorithms will be ranked on how much their recall surpasses the baselines at these QPS thresholds.   We will add up the recall improvements of each algorithm on all data sets it competes on. Participants are required to commit to at least 3 datasets, and ideally more. Algorithms that work on more datasets are at an advantage as they can benefit from additional scores. Any recall regression compared to the baseline on the datasets committed to will be subtracted from the final score.

## For_Evaluators

repository: SuperSupeng/big-ann-benchmarks

repo_name: SuperSupeng/big-ann-benchmarks


text:
# Home Recipes app

This client runs on the following platforms:
* Android
* iOS
* Web app
* MacOS app

### Note:
To create the apk to run on an Andriod phone, you need to run the following command:
    ```
    flutter build apk -debug
    ```

### Steps taken to connect client to server

1. Install dart

2. Activate protoc plugin
    ```
    pub global activate protoc_plugin
    ```

3. Add pub-cache to PATH
    ```
    PATH="$PATH":"$HOME/.pub-cache/bin"
    ```

4. Generate dart code from protos
    ```
    protoc ./defs/recipes-service.proto  --dart_out=grpc:client/home_recipes_app/lib/generated
    ```

5. Add dependencies to pubspec.yaml
    ```
    grpc: ^2.1.3
    protobuf: ^1.0.1
    uuid: ^2.0.4
    ```

6. Implement client logic

repository: kaushikchaubal/home-recipes

gha_language: Dart

repo_name: kaushikchaubal/home-recipes


text:
# Contribution Guidelines

## Prerequisites

By contributing, you assert that:

* The contribution is your own original work.
* You have the right to assign the copyright for the work (it is not owned by your employer, or you have been given copyright assignment in writing).
* You [license] the contribution under the terms applied to the rest of the code.
* You agree to follow the [code of conduct] from the Cake project.

## Contributing process

This repository uses [GitFlow] with default configuration.
Development is happening on `develop` branch.

To contribute:

* Fork this repository.
* Create a feature branch from `develop`.
* Implement your changes.
* Push your feature branch.
* Create a pull request.

## Build

To build this package we are using Cake.

On Windows PowerShell run:

```powershell
./build
```

On OSX/Linux run:

```bash
./build.sh
```

## Release

See [Cake.Recipe documentation] how to create a new release of this addin.

[license]: LICENSE
[code of conduct]: https://github.com/cake-build/cake/blob/develop/CODEOFCONDUCT.md
[GitFlow]: (http://nvie.com/posts/a-successful-git-branching-model/)
[Cake.Recipe documentation]: https://cake-contrib.github.io/Cake.Recipe/docs/usage/creating-release

branch_name: refs/heads/develop

gha_language: XSLT


text:
---
layout: post
title: Model Primitive Hierarchical Lifelong Reinforcement Learning
comments: True
excerpt:
tags: ['2019', 'AAMAS 2019', 'Catastrophic Forgetting', 'Continual Learning', 'Hierarchical Reinforcement Learning', 'Lifelong Learning', 'Reinforcement Learning', AAMAS, AI, CL, HRL, RL]
---

## Introduction

* The paper presents a framework that uses diverse suboptimal world models that can be used to break complex policies into simpler and modular sub-policies.

* Given a task, both the sub-policies and the controller are simultaneously learned in a bottom-up manner.

* The framework is called as Model Primitive Hierarchical Reinforcement Learning (MPHRL).

* [Link to the paper](https://arxiv.org/abs/1903.01567)

## Idea

* Instead of learning a single transition model of the environment (aka *world model*) that can model the transitions very well, it is sufficient to learn several (say *k*) suboptimal models (aka *model primitives*).

* Each *model primitive* will be good in only a small part of the state space (aka *region of specialization*).

* These *model primitives* can then be used to train a gating mechanism for selecting sub-policies to solve a given task.

* Since these *model primitives* are sub-optimal, they are not directly used with model-based RL but are used to obtain useful functional decompositions and sub-policies are trained with model-free approaches.

## Single Task Learning

* A gating controller is trained to choose the sub-policy whose *model primitive* makes the best prediction.

* This requires modeling *p(Mk \| st, at, st+1)* where *p(Mk)* denotes the probability of selecting the *kth model primitive*. This is hard to compute as the system does not have access to *st+1*  and *at* at time *t* before it has choosen the sub-policy.

* Properly marginalizing *st+1* and *at* would require expensive MC sampling. Hence an approximation is used and the gating controller is modeled as a categorical distribution - to produce *p(Mk \| st)*. This is trained via a conditional cross entropy loss where the ground truth distribution is obtained from transitions sampled in a rollout.

* The paper notes that technique is biased but reports that it still works for the downstream tasks.

* The gating controller composes the sub-policies as a mixture of Gaussians.

* For learning, PPO algorithm is used with each *model primitives* gradient weighted by the probability from the gating controller.

## Lifelong Learning

* Different tasks could share common subtasks but may require a different composition of subtasks. Hence, the learned sub-policies are transferred across tasks but not the gating controller or the baseline estimator (from PPO).

## Experiments

* Domains:

    * Mujoco ant navigating different mazes.

    * Stacker arm picking up and placing different boxes.

* Implementation Details:

    * Gaussian subpolicies

    * PPO as the baseline

    * Model primitives are hand-crafted using the true next state provided by the environment simulator.

* Single Task

    * Only maze task is considered with the start position (of the ant) and the goal position is fixed.

    * Observation includes distance from the goal.

    * Forcing the agent to decompose the problem, when a more direct solution may be available, causes the sample complexity to increase on one task.

* Lifelong Learning

    * Maze

        * 10 random Mujoco ant mazes used as the task distribution.

        * MPHRL takes almost twice the number of steps (as compared to PPO baseline) to solve the first task but this cost gets amortized over the distribution and the model takes half the number of steps as compared to the baseline (summed over the 10 tasks).

    * Pick and Place

        * 8 Pick and Place tasks are created with max 3 goal locations.

        * Observation includes the position of the goal.

* Ablations

    * Overlapping *model primitives* can degrade the performance (to some extent). Similarly, the performance suffers when redundant primitives are introduced indicating that the gating mechanism is not very robust.

    * Sub-policies could quickly adapt to the previous tasks (on which they were trained initially) despite being finetuned on subsequent tasks.

    * The order of tasks (in the 10-Mazz task) does not degrage the performance.

    * Transfering the gating controller leads to negative transfer.

* Notes

    * I think the biggest strength of the work is that accurate dynamics model are not needed (which are hard to train anyways!) through the experimental results are not conclusive given the limited number of domains on which the approach is tested.

url: https://raw.githubusercontent.com/shagunsodhani/papers-I-read/8301c9e41f2faa4dd72dfdb94d518afa3f8f510b/site/_posts/2019-03-12-Model Primitive Hierarchical Lifelong Reinforcement Learning.md

repository: shagunsodhani/papers-I-read

gha_language: CSS

repo_name: shagunsodhani/papers-I-read


text:
CrayCrafting
===============

Crafting at its craziest! Each recipe is given a new output!

Works in a few ways.

New recipes per world or dimention.

New recipes based off a timer (Ultimate crazy!)

Please make an issue if you have suggestions or bug reports.

Include a log/stacktrace and screenshot where possible!

Download
--------

We have downloads on [curseforge](http://minecraft.curseforge.com/mc-mods/223249-craycrafting) and on [our website](http://doubledoordev.net/) (more up to date).

repository: takochako/CrayCrafting

repo_name: takochako/CrayCrafting