text:
---
author: guidedmissile
comments: true
date: 2015-06-10 04:48:57+00:00
layout: post
link: https://inlovewithcode.wordpress.com/2015/06/10/how-to-if-else-if-in-awk/
slug: how-to-if-else-if-in-awk
title: How to 'if else if' in awk?
wordpress_id: 350
categories:
- Linux Scripting
- Unix
tags:
- Bash
- Shell Scripting
---

Have you ever face an issue where you have made multiple files from one big file based on tag/column value present in a line? Then this post is for you. At the end of the post, i.e. in next 5 mins you will be able to answer below questions.

#### How to split a file based on condition?
How to segregate data in file?
How to use if else if condition in awk?
How to pass variable to awk?

Problem statement: I have a list of servers in inventory.csv. Now I want to segregate unix team's data into multiple files based on team name.
I know that I have teams - teamA, teamB, teamC

    # defining myPathiable
    folder_location="/tmp/sam"

    # using 'grep -iw' we search for Unix team data
    # using '-v' option passing variable inside awk
    grep -iw unix inventory.csv | awk -v myPath="${folder_location}" -F: '{
    # using condition if & else if condition seggregating data
    # below conditon for teamA
    if ($8 == "teamA") print > "myPath/teamA.csv"
    # below conditon for teamB
    else if ($8 == "teamB") print > "myPath/teamB.csv"
    # below conditon for teamC
    else if ($8 == "teamC") print > "myPath/teamB.csv"
    # optimism # below conditon if there is a new team which is not teamA or teamB or teamC
    else print > "myPath/others.csv"
    }'

Re-writing above code

    #!\bin\bash
    folder_location="/tmp/sam"
    grep -iw unix inventory.csv | awk -v myPath="${folder_location}" -F: '{
    if ($8 == "teamA") print > "myPath/teamA.csv"
    else if ($8 == "teamB") print > "myPath/teamB.csv"
    else if ($8 == "teamC") print > "myPath/teamB.csv"
    else print > "myPath/others.csv"
     }'

Assignment: In programming, parsing a file twice is a sin. In the above code we are using 'grep' to parse & find the unix data. If you have understood awk, you should be able to make above code more beautiful just with -awk. Good Luck.


text:
# Image Gallery 02: Using Event Delegation

More improved on performance than 'image-gallery-01', because it does not create 1000s of add event listeners,

## Brief

You have been provided with some HTML, CSS and image assets and a few lines of JavaScript code; you need to write the necessary JavaScript to turn this into a working program.

## Credits

- _MDN_ - Image Gallery

license: MIT

repository: hyfydistro/frontend-cool-features

branch_name: refs/heads/master

extension: md

license_type: permissive

repo_name: hyfydistro/frontend-cool-features


text:
[Lire en français](BlazorDatagrid_RELEASE_NOTE.md)
# Release notes
**5.0.1**
update dependency to BlazorNInput

**5.0.0**

_New_
- Update to .Net6
- You don't have to specified the colection used in each column and cells anymore

```razor
<DataGridColumn ColumnName="TemperatureC" DisplayColumnName="TemperatureC" Filter="true"></DataGridColumn>

<Cell Context="cellcontext">
    @cellcontext.Summary
</Cell>
```

instead of

```razor
<DataGridColumn Items="@forecasts" ColumnName="TemperatureC" DisplayColumnName="TemperatureC" Filter="true"></DataGridColumn>

<Cell Items="@forecasts" Context="cellcontext">
    @cellcontext.Summary
</Cell>
```

**4.1.1**

_Fixes_
- Fixed a problem with a dependancy
- Fixed a problem with rendering cell content
- Using the double curly brace between `<Cell></Cell>` tags prevents C # methods from working. This feature is therefore withdrawn. You have to go through the context instead. The use of braces in the Content parameter is not impacted.
-

```razor
<Cell Items="@forecasts" Context="cellcontext">
    @cellcontext.Summary
</Cell>
```
instead of
```razor
<Cell Items="@forecasts" Context="cellcontext">
    {{Summary}}
</Cell>
```

**4.1.0**

_New_
- When the datagrid is not editable, it is possible to go through the context to retrieve the data when the content is placed between the tags `<Cell></Cell>`
```razor
<Cell Items="@forecasts" Context="cellcontext">
    @cellcontext.Summary
</Cell>
```
**4.0.0**

_New_
- Update to .Net5
- Variables in double braces are now interpreted when content is passed between tags `<Cell></Cell>`

_Fix_
- Fixed crash when the Parameter Content of the Cell component is null.

**3.0.0**
- Structural change requiring to review the implementation of the datagrid in the code.
- Adding a Cell component to the datagrid to manage cells. This component is placed in the GridRow part and becomes mandatory.

* Added new parameters for the Cell component
    * Items which is the same parameter as for the datagrid
    * Content which contains cell content
    * ValidationPattern which allows you to add a validation pattern
    * LabelError which contains the error message in case of validation failure

**2.0.2**
> - If the input list is null, the result is a table with 0 element.

**2.0.1**
> - fix a bug on the display of number of result when an item is add in the start list.

**2.0.0**
> - Add parameter format to display date to a specfific format.
> - Add parameter Editable allow the datagrid to be editable.
> - Add parameter ReadOnly who set a column ReadOnly when the datagrid is editable
> - Add an error label when you enter an invalide format in the editable datagrid.
> - Add a keyword labelError in the translation dictionnary

**1.11.0**
> - Update to .Net Core 3.1
> - Updating pagination calculation

**1.10.2**
> - Add loading message in the dictionnary

**1.10.1**
> - Update column sorting

**1.10.0**
> - Improved column filtering.
> - Adding a load message during a long filter

**1.9.0**
> - The PageSize parameter now allows you to set the initial number of items displayed. If it is not present, the initial number is the first value of the selector.

**1.8.0**
> - Compatibility with .NET Core 3.0 Release Candidate 1

**1.7.0**
> - Compatibility with Blazor preview 9

**1.6.0**
> - updating filter style
> - Adding the parameter **DropdownFilter** to replace the filter input text by a list witch contains all different values of the column
> - adding a delay before filtering

**1.5.0**
> - Adding a selector for the number of display per page
> - Adding the parameter **ShowPageSelector** to display or not the selector
> - New look for the pagination
> - Clicking the next or previous pagination segment button places the current page on the first or last page of the new segment, respectively
> - Update the translation to incorporate the selector
> - Adding the parameter **PageSelector** to personalise the selector

**1.4.0**
> - Added the translation of the Datagrid by passing a translation dictionnary in parameter.
>   - New parameter : Translation

**1.3.0**
> - Add the following parameters:
>   - TableClass: add the specified classes to the tag ```<table>```
>   - TheadClass: add the specified classes to the tag ```<thead>```
>   - TbodyClass: add the specified classes to the tag ```<tbody>```
>   - TableStyle: add the specified styles to the tag ```<table>```
>   - TheadStyle: add the specified styles to the tag ```<thead>```
>   - TbodyStyle: add the specified styles to the tag ```<tbody>```

**1.2.0**
> - Add compatibility with Blazor preview 8

**1.1.1**
> - The result number is now the total number filtered instead of the number of result in the current page.

**1.1.0**
> - Add the possibility to show the total number of items on top of the datagrid.
> - The number of displayed page is update when a filter is applied.
> - Instead of using DisplayColumnName, it's possible to pass the header's text between the column tag:  ```<DataGridColumn>header text</DataGridColumn>```

**1.0.0**
> - Initial release

gha_language: C#

gha_license_id: MIT


text:
### [CVE-2017-9480](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-9480)
![](https://img.shields.io/static/v1?label=Product&message=n%2Fa&color=blue)
![](https://img.shields.io/static/v1?label=Version&message=n%2Fa&color=blue)
![](https://img.shields.io/static/v1?label=Vulnerability&message=n%2Fa&color=brighgreen)

### Description

The Comcast firmware on Cisco DPC3939 (firmware version dpc3939-P20-18-v303r20421746-170221a-CMCST) devices allows local users (e.g., users who have command access as a consequence of CVE-2017-9479 exploitation) to read arbitrary files via UPnP access to /var/IGD/.

### POC

#### Reference
- https://github.com/BastilleResearch/CableTap/blob/master/doc/advisories/bastille-23.upnp-directory-write.txt

#### Github
No PoCs found on GitHub currently.

repository: trickest/cve

branch_name: refs/heads/main

gha_language: HTML

repo_name: trickest/cve


text:
# oda-finalproject

> Nuxt.js project

Authors: Sierra Lorenzini, Rikki Kendall, Charlotte Roh

Notes (Sierra):
In general I am happy with how this website turned out. I learned a lot about Vue, Nuxt, and Firebase and feel
more comfortable now developing a more complex website. I worked pretty hard to implement user authorization and authentication (users and admins can donate, admins can add projects to projects page) and the dual language functionality through a Vuex store. Nuxt took a lot of trial and error to figure out but in the end it was nice to generate a static website espeically considering this page is meant to be more informational than a dynamic application.

One thing that still isn't working (and I have no idea why) is the image carousel on the Why page. It is there, but only shows up when you click on the big white area. This is particilarly strange considering the Why
used my exact code just modified for different data in the Why page and it works perfectly there. I tried posting on Piazza and doing a lot of research but couldn't figure out how to fix it.

Notes (Rikki):
I had an incredible experience with this website development. The organization that the group had planning out each section and having a great time with it really. On a personal level, I got to sum up all of the things we learned throughout the semester and put them all together. I learned an incredible amount about Nuxtjs and how it makes it easier to employ Vue and integrate many features into the website. The bilingual aspect of the site was also another great feature of our website that was interesting to implement. From the site header and footer, the About page, Why page, and every other feature of the website we got to collaborate on, I definitely learned an incredible amount and really hope that this website does eventually get utilized for the ODA.

## Build Setup

``` bash
# install dependencies
$ npm install # Or yarn install

# serve with hot reload at localhost:3000
$ npm run dev

# build for production and launch server
$ npm run build
$ npm start

# generate static project
$ npm run generate
```

For detailed explanation on how things work, checkout the [Nuxt.js docs](https://github.com/nuxt/nuxt.js).

gha_language: JavaScript


text:
# Realtime Pub/Sub scenario

This page explains what happens when clients exchange data with Kuzzle, using the [Publish/Subscribe pattern](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern).

Kuzzle is able to manage two different types of data:
* persistent data => using the "_create_", "_createOrUpdate_", or "_delete_" actions.
* volatile/realtime data => using the "_publish_" action.

Kuzzle handles data differently, depending if it's persistent or not.

This page describes the process for **non persistent** data, as well as real-time notification about persisted data.

If you wish to learn about how Kuzzle stores persistent data, please read [Writing persistent data to Kuzzle](write.md)

Remember the [Architecture overview](../architecture.md) and focus on the components involved by reading actions:
![pubsub_overview](../images/kuzzle_pubsub_scenario_overview.png)

## 1st step : subscription

The following diagram shows how two different clients, a Websocket and a MQ one, subscribe to data.

![pubsub_scenario_details1](../images/kuzzle_pubsub_scenario_details1.png)

\#1. The client application opens a Websocket or a MQ connection and emits a "subscribe" event with some filters.
(see [API Documentation](http://kuzzleio.github.io/kuzzle-api-documentation/#on))

For instance, to be notified about all contents posted to the collection "users", containing a field "hobby" equals to "computer":
```json
{
  "requestId": "ed4faaff-253a-464f-a6b3-387af9d8483d",
  "index": "mainindex",
  "collection": "users",
  "action": "on",
  "body": {
    "term": {"hobby": "computer" }
  },
  "state": "all"
}
```

(see [Filters](../filters.md) for more details about filters)

\#1b. The client then listens to the ```<requestId>``` event on the socket.
Kuzzle will get back to him with a corresponding Room ID using this event.

Sample Javascript code, using Websocket:

```javascript
  this.socket.once("ed4faaff-253a-464f-a6b3-387af9d8483d", function(response) {
    callback(response);
  });
```

\#2. The ```Router Controller``` interprets the input request and transfer the subscription message to the ```Funnel Controller```.

Sample message:
```json
{
  "index": "mainindex",
  "collection": "users",
  "controller": "subscribe",
  "action": "on",
  "filter": {
    "term": {"hobby": "computer" }
  }
}
```

\#3. The ```Funnel Controller``` validates the message and transfer it to the ```Subscribe Controller```.

\#4. The ```Subscribe Controller``` calls the ```HotelClerk``` internal component to create the subscription.

\#5. The ```HotelClerk``` calls the ```DSL``` component to get a formated filter related to the subscription (see [DSL Readme for more details](../../lib/api/dsl/README.md)).

\#6. The ```HotelClerk``` creates a channel related to the filters and give it back to the ```Subscribe Controller```.

\#7. The channel is sent back to the Websocket (or MQ) Router through the internal components.

\#8. The Websocket (or MQ) Router emits a ```<requestId>``` event to the client, containing the subscribed channel ID.

Sample response content:

```json
{
  "status": 200,
  "error": null,
  "index": "mainindex",
  "collection": "users",
  "controller": "subscribe",
  "action": "on",
  "state": "all",
  "requestId": "ed4faaff-253a-464f-a6b3-387af9d8483d",
  "result": {
    "roomId": "78c5b0ba-fead-4535-945c-8d64a7927459",
    "channel": "c5cd8bdc-06a4-4d6e-bae3-61f1a8ac2982"
  }
}
```

\#8. The client now listens to this ```channel``` events to be notified about new messages corresponding to his subscription filters.

## 2nd step : notify about real-time actions
The following diagram shows how Kuzzle handles a new message and how subscribed clients are notified:

![pubsub_scenario_details2](../images/kuzzle_pubsub_scenario_details2.png)

\#1. A new content is published to the ```Notifier``` component.

The "_publish_" method can be triggered:
* either direclty by the ```Write Controller``` for non persistent data (using the [publish](http://kuzzleio.github.io/kuzzle-api-documentation/#publish) action).
* or by the ```Plugins Manager``` when a 'data:create' event is triggered, to notify users in real-time before the data are sent to the storage Engine.

\#2. The ```Notifier``` calls the ```DSL``` component to test registered filters that match the content, and get related rooms.

\#3. The ```Notifier``` uses the ```Notification Cache``` engine to store the mapping content/rooms into cache.

\#4. The ```Notifier``` calls the ```HotelClerk``` to get the channels related to the rooms.

\#5. The ```Notifier``` broadcasts the message to each related channel to the Websocket (or MQ) router.

\#6. Finally, the router sends the message to the clients who subscribed to it.

## 3rd step : notify about persisted data

![pubsub_scenario_details2](../images/kuzzle_pubsub_scenario_details3.png)

\#1. The ```Notifier``` component is notified about a new action by the ```Internal Broker``` (see [step \#11. in write scenario](write.md)).

\#2. The ```Notifier``` calls the ```Notification Cache``` to get the rooms related to the content.

\#3. The ```Notifier``` calls the ```HotelClerk``` to get the channels related to the rooms.

\#4. The ```Notifier``` broadcasts the message to each related channel to the Websocket (or MQ) router.

\#5. Finally, the router sends the message to the clients who subscribed to it.

## Related pages

* [Architecture overview](../architecture.md)
* [API Documentation](http://kuzzleio.github.io/kuzzle-api-documentation)

repository: MudaliarArun/kuzzle

repo_name: MudaliarArun/kuzzle


text:
# Agent Menu

Agents are Empire's implants that are used to interact and assign tasks and collect information. A list of active agents is displayed when entering this page and highlghts active agents green and stale agents red. From this menu, you can kill, remove, rename, and clear an agent using their respective commands.

![](https://user-images.githubusercontent.com/20302208/100279870-2ddd4e00-2f1c-11eb-9431-c1ba796af721.jpg)

## ****

repository: BC-SECURITY/Empire

gha_language: PowerShell

repo_name: BC-SECURITY/Empire


text:
```toml
[advisory]
id = "RUSTSEC-2021-0119"
package = "nix"
date = "2021-09-27"
url = "https://github.com/nix-rust/nix/issues/1541"
categories = ["memory-corruption"]
keywords = ["nss"]
aliases = ["CVE-2021-45707", "GHSA-76w9-p8mg-j927", "GHSA-wgrg-5h56-jg27"]

[versions]
patched = ["^0.20.2", "^0.21.2", "^0.22.2", ">= 0.23.0",]
unaffected = ["< 0.16.0"]

[affected]
os = ["linux", "freebsd", "android", "netbsd", "dragonfly", "openbsd", "fuchsia"]
functions = { "nix::unistd::getgrouplist" = [">= 0.16.0"] }
```

# Out-of-bounds write in nix::unistd::getgrouplist

On certain platforms, if a user has more than 16 groups, the
`nix::unistd::getgrouplist` function will call the libc `getgrouplist`
function with a length parameter greater than the size of the buffer it
provides, resulting in an out-of-bounds write and memory corruption.

The libc `getgrouplist` function takes an in/out parameter `ngroups`
specifying the size of the group buffer. When the buffer is too small to
hold all of the requested user's group memberships, some libc
implementations, including glibc and Solaris libc, will modify `ngroups`
to indicate the actual number of groups for the user, in addition to
returning an error. The version of `nix::unistd::getgrouplist` in nix
0.16.0 and up will resize the buffer to twice its size, but will not
read or modify the `ngroups` variable. Thus, if the user has more than
twice as many groups as the initial buffer size of 8, the next call to
`getgrouplist` will then write past the end of the buffer.

The issue would require editing /etc/groups to exploit, which is usually
only editable by the root user.

repository: faern/advisory-db

gha_license_id: NOASSERTION

repo_name: faern/advisory-db


text:
dotfiles
========

These go in your home directory (~/). The background colour in .Xresources is for urxvt. Compositing is required for alpha transparency in both urxvt and conky. mpc is required for the mpd portion of the conky output. These files were used to create [this setup](http://imgur.com/a/5XDwf#0).

repository: Anomitee/dotfiles

repo_name: Anomitee/dotfiles


text:
# My Personal "HelloWorld" VHDL FPGA Learning Playground

## Getting Started

 To build this project you will need Altera Quartus II, Altera USB Blaster and A-C4E6 Cyclone IVe FPGA development board. Also an Eclipse IDE (CDT) with Veditor plugin will come handy if you want to change VHDL projects source code.
 To start, you can just:

```bash
git clone https://github.com/startupit69/HelloLCD1602.git
cd HelloLCD1602/quartus
make clean
make
make program
```

The result on your A-C4E6 board should be something like this:

[![A-C4E6](https://github.com/startupit69/HelloLCD1602/blob/master/ilustration.jpg)](https://github.com/startupit69/HelloLCD1602/blob/master/ilustration.jpg)

## Copyright and license

Code and documentation copyright 2016 Ivan Tanaskovic. Code released under [the MIT license](https://github.com/startupit69/HelloLCD1602/blob/master/LICENSE)


text:
# Vagrant-Windows
[![Build Status](https://travis-ci.org/WinRb/vagrant-windows.png)](https://travis-ci.org/WinRb/vagrant-windows)
[![Code Climate](https://codeclimate.com/github/WinRb/vagrant-windows.png)](https://codeclimate.com/github/WinRb/vagrant-windows)
[![Gem Version](https://badge.fury.io/rb/vagrant-windows.png)](http://badge.fury.io/rb/vagrant-windows)

### This plugin is deprecated, please use [Vagrant 1.6+](http://www.vagrantup.com/downloads.html) which natively supports Windows guests over WinRM.

This [Vagrant](http://www.vagrantup.com/) plugin allows you to standup Windows guests using [WinRM](http://msdn.microsoft.com/en-us/library/aa384426\(v=vs.85\).aspx) instead of SSH on older versions of Vagrant.

## Getting Started
1. Install the vagrant-windows plugin.
2. Create and configure a Windows guest base box.
3. Create a Vagrantfile.
4. vagrant up

## Installation

Vagrant 1.6 will natively support Windows guests, this plugin should not be used with Vagrant 1.6 or later.

- For Vagrant 1.1 -> 1.5 execute `vagrant plugin install vagrant-windows`.
- For Vagrant 1.0 execute `vagrant plugin install vagrant-windows --plugin-version 0.1.2`.

## Creating a Base Box

Supported Guest Operating Systems:
- Windows 7
- Windows 8
- Windows Server 2008
- Windows Server 2008 R2
- Windows Server 2012

Windows Server 2003 and Windows XP are not supported by the maintainers of this project. Any issues regarding any unsupported guest OS will be closed. If you still insist on using XP or Server 2003, [this](http://stackoverflow.com/a/18593425/18475) may help.

You'll need to create a new Vagrant base box. Create a new Windows VM in VirtualBox, configure some Windows settings (see below) then follow the [Vagrant packaging instructions](http://docs.vagrantup.com/v2/cli/package.html).

  - Create a vagrant user, for things to work out of the box username and password should both be "vagrant".
  - Turn off UAC (Msconfig)
  - Disable complex passwords
  - [Disable Shutdown Tracker](http://www.jppinto.com/2010/01/how-to-disable-the-shutdown-event-tracker-in-server-20032008/) on Windows 2008/2012 Servers (except Core).
  - [Disable "Server Manager" Starting at login](http://www.elmajdal.net/win2k8/How_to_Turn_Off_The_Automatic_Display_of_Server_Manager_At_logon.aspx) on Windows 2008/2012 Servers (except Core).
  - Enable and configure WinRM (see below)

### WinRM Configuration

These commands assume you're running from a regular command window and not PowerShell.
```
   winrm quickconfig -q
   winrm set winrm/config/winrs @{MaxMemoryPerShellMB="512"}
   winrm set winrm/config @{MaxTimeoutms="1800000"}
   winrm set winrm/config/service @{AllowUnencrypted="true"}
   winrm set winrm/config/service/auth @{Basic="true"}
   sc config WinRM start= auto
```

__NOTE:__ For the WinRM service to work in Windows 7, the network profile has to be changed from Public to Work. To ensure that networks added later (i.e. through the Vagrantfile) also are made non-public, add the following to  your Vagrantfile:

```ruby
# Ensure that all networks are set to private
config.windows.set_work_network = true
```

#### Additional WinRM 1.1 Configuration

These additional configuration steps are specific to Windows Server 2008 (WinRM 1.1). For Windows Server 2008 R2, Windows 7 and later versions of Windows you can ignore this section.

1. Ensure the Windows PowerShell feature is installed
2. [Change the default WinRM port](http://technet.microsoft.com/en-us/library/ff520073\(v=ws.10\).aspx) (see below) or [upgrade to WinRM 2.0](http://www.microsoft.com/en-us/download/details.aspx?id=20430).

```
netsh firewall add portopening TCP 5985 "Port 5985"
winrm set winrm/config/listener?Address=*+Transport=HTTP @{Port="5985"}
```

### Required Windows Services

If you like to turn off optional Windows services you'll need to ensure you leave these services enabled for vagrant-windows to continue to work:

  - Base Filtering Engine
  - Remote Procedure Call (RPC)
  - DCOM Server Process Launcher
  - RPC Endpoint Mapper
  - Windows Firewall
  - Windows Remote Management (WS-Management)

## The Vagrant File

Add the following to your Vagrantfile

```ruby
config.vm.guest = :windows
config.vm.network :forwarded_port, guest: 5985, host: 5985, id: "winrm", auto_correct: true
```

Example:
```ruby
Vagrant.configure("2") do |config|

  # Configure base box parameters
  config.vm.box = "vagrant-windows2008r2"
  config.vm.box_url = "./vagrant-windows2008r2.box"
  config.vm.guest = :windows

  # Port forward WinRM and RDP
  config.vm.network :forwarded_port, guest: 3389, host: 3389
  config.vm.network :forwarded_port, guest: 5985, host: 5985, id: "winrm", auto_correct: true

end
````

Available Config Parameters:

* ```config.windows.halt_timeout``` - How long Vagrant should wait for the guest to shutdown before forcing exit, defaults to 30 seconds
* ```config.windows.halt_check_interval``` - How often Vagrant should check if the system has shutdown, defaults to 1 second
* ```config.windows.set_work_network``` - Force network adapters to "Work Network". Useful for Win7 guests using private networking.
* ```config.winrm.username``` - The Windows guest admin user name, defaults to vagrant.
* ```config.winrm.password``` - The above's password, defaults to vagrant.
* ```config.winrm.host``` - The IP of the guest, but because we use NAT with port forwarding this defaults to localhost.
* ```config.winrm.guest_port``` - The guest's WinRM port, defaults to 5985.
* ```config.winrm.port``` - The WinRM port on the host, defaults to 5985. You might need to change this if your hosts is also Windows.
* ```config.winrm.max_tries``` - The number of retries to connect to WinRM, defaults to 20.
* ```config.winrm.timeout``` - The max number of seconds to wait for a WinRM response, defaults to 1800 seconds.

Note - You need to ensure you specify a config.windows and a config.winrm in your Vagrantfile. Currently there's a problem where Vagrant will not load the plugin config even with defaults if at least one of its values doesn't exist in the Vagrantfile.

------------------------------------------------------------

## What Works?

- vagrant up|halt|reload|provision
- Chef Vagrant Provisioner
- Puppet Vagrant Provisioner
- Shell Vagrant provisioner. Batch files or PowerShell (ps1) scripts are supported as well as inline scripts.

## Troubleshooting

#### When I run the winrm command I get: "Error: Invalid use of command line. Type "winrm -?" for help."
- You're running the winrm command from powershell. You need to put ```@{MaxMemoryPerShellMB="512"}``` etc in single quotes:
```
   winrm set winrm/config/winrs '@{MaxMemoryPerShellMB="512"}'
```

#### I get a 401 auth error from WinRM
- Ensure you've followed the WinRM configuration instructions above.
- Ensure you can manually login using the specified config.winrm.username you've specified in your Vagrantfile.
- Ensure your password hasn't expired.
- Ensure your password doesn't need to be changed because of policy.

#### I get a non-401 error from WinRM waiting for the VM to boot
- Ensure you've properly setup port forwarding of WinRM
- Make sure your VM can boot manually through VBox.

#### SQL Server cookbook fails to install through Vagrant
- Ensure UAC is turned off
- Ensure your vagrant user is an admin on the guest
- The SQL Server installer uses a lot of resources, ensure WinRM Quota Management is properly configured to give it enough resources.
- See [COOK-1172](http://tickets.opscode.com/browse/COOK-1172) and http://stackoverflow.com/a/15235996/82906 for more information.

If all else fails try running [vagrant with debug logging](http://docs.vagrantup.com/v2/other/debugging.html), perhaps that will give
you enough insight to fix the problem or file an issue.

## Contributing

1. Fork it.
2. Create a branch (git checkout -b my_feature_branch)
3.  Push to the branch (git push origin my_feature_branch)
5. Create a pull requst from your branch into master (Please be sure to provide enough detail for us to cipher what this change is doing)

### Development

Clone this repository and use [Bundler](http://gembundler.com) to get the dependencies:

```
$ bundle install
```

Once you have the dependencies, you can run the tests with `rake`:

```
$ bundle exec rake spec
```

If those pass, you're ready to start developing the plugin. You can test
the plugin without installing it into your Vagrant environment by just
creating a [Vagrantfile](http://docs.vagrantup.com/v2/plugins/packaging.html)
in the top level of this directory (it is gitignored) that uses it, and
use bundler to execute Vagrant:

```
$ bundle exec vagrant up
```

### Installing Vagrant-Windows From Source

If you want to globally install your locally built plugin from source, use the following method (this would be for 1.2.0):

```
 bundle install
 bundle exec rake build
 vagrant plugin install pkg/vagrant-windows-1.2.0.gem
```

Keep in mind you should have Ruby 1.9.3 and Ruby DevKit installed. Check out the following gist that can get you what you need (from blank system to fully ready): [Install Vagrant Windows Plugin From Source Gist](https://gist.github.com/ferventcoder/6251225).

## References and Shout Outs

- Chris McClimans - Vagrant Branch (https://github.com/hh/vagrant/blob/feature/winrm/)
- Dan Wanek - WinRM GEM (https://github.com/zenchild/WinRM)
  - +1 For being super responsive to pull requests.
- Mike Griffen - Added first vagrant-windows unit tests and updated readme
- Geronimo Orozco - Shell provisioner support
- David Cournapeau - Added config.windows.set_work_network option
- keiths-osc - Fixed Vagrant 1.2 shared folder action
- Rob Reynolds - Updated readme installation and box configuration notes
- stonith - Updated readme winrm config notes
- wenns - Updated readme to advise against forwarding RDP
- Joe Fitzgerald - Added VMWare support and improved retry logic.


text:
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) > [@rocket.chat/fuselage-hooks](./fuselage-hooks.md) > [useStableArray](./fuselage-hooks.usestablearray.md)

## useStableArray variable

Hook to create an array with stable identity if its elements are equal.

<b>Signature:</b>

```typescript
useStableArray: <T extends unknown[]>(array: T, compare?: <T_1>(a: T_1, b: T_1) => boolean) => T
```

repository: robogenixaiorg/RoboChatComponents

branch_name: refs/heads/develop

gha_language: CSS

repo_name: robogenixaiorg/RoboChatComponents


text:
# Amplify UI React Core

`@aws-amplify/ui-react-core` is a React platform agnostic utility library for Amplify UI internal usage in the `@aws-amplify/ui-react` and `@aws-amplify/ui-react-native` packages.

repository: aws-amplify/amplify-ui

gha_language: TypeScript

repo_name: aws-amplify/amplify-ui


text:
---
title: "Share service volumes in composition steps"
description: "How to share data in compositions"
group: example-catalog
sub_group: ci-examples
redirect_from:
  - /docs/yaml-examples/examples/shared-volumes-of-service-from-composition-step-for-other-yml-steps/
  - /docs/shared-volumes-of-service-from-composition-step-for-other-yml-steps/
toc: true
---
Using this repository, we'll help you get up to speed with basic functionality such as building Docker images and using the shared volumes.

This project uses Node Js to build an application which will eventually become a distributable Docker image.
To share volumes of service in composition step for other yml steps you can use the variable {% raw %}```${{CF_VOLUME_NAME}}```{% endraw %}. It will refer to the volume that was generated for the specific flow. Can be used in conjunction with a composition to provide access to your cloned repository.

>Read more about caching build dependencies our [blog](https://codefresh.io/blog/caching-build-dependencies-codefresh-volumes/){:target="_blank"}.

## Looking around
In the root of this repository you'll find a file named `codefresh.yml`, this is our build descriptor that describes the different steps that comprise our process. Let's quickly review the contents of this file:

  `codefresh.yml`
{% highlight yaml %}
step_file_generation:
  type: composition
  composition:
    version: '2'
    services:
      service1:
        volumes:
          - {% raw %}${{CF_VOLUME_NAME}}{% endraw %}:/codefresh/volume
        image: {% raw %}${{build_step}}{% endraw %}
        command: bash -c "echo hello > /codefresh/volume/myfile.txt"
  composition_candidates:
    test:
      image: {% raw %}${{build_step}}{% endraw %}
      command: echo hello
{% endhighlight %}

>Example
 Just head over to the example [**repository**](https://github.com/codefreshdemo/cf-example-shared-volumes-composition-step){:target="_blank"} in GitHub, and follow the instructions there.

The way the volume is shared between builds is that upon build completion we create an image of the volume state to be used in the next builds. If you run 2 builds in parallel from the same pipeline and at the same time, each will use the same last volume image, but it’ll run separately on both. The volume image you’ll get upon completion is the state of the build that finished last.

## Related articles
[CI pipeline examples]({{site.baseurl}}/docs/example-catalog/ci-examples/)

gha_language: SCSS


text:
# Changelog

## [1.4.2]

### Added

- Utility functions in `pydgn.evaluation.util` to retrieve configuration files in a model selection folder and filter them for post-hoc analyses.
- Updated tutorial in documentation accordingly

## [1.4.1] Support for Python 3.8-3.11

### Changed

- Commented the single temporal graph learning example to remove the dependency from torch-geometric package. The code to support temporal learning stays.
- Removed the configuration sample for the temporal setting
- PyG requirements is <= 2.3.0, so we can remove all the other dependencies
- Modified the `setup/create_environment.sh` file to use Python 3.9 and PyG 2.3.0

## [1.3.1] Weighted Loss implementation

### Added

- You can specify weights for loss in `AdditiveLoss` by passing a dictionary of (loss name, loss weight) entries as an argument.
  See the documentation of `AdditiveLoss` for more info or the example in `examples/MODEL_CONFIGS/config_SupToyDGN.yml`.

### Fixed

- Better handling of `len()` in `TUDatasetInterface`
- Fixed minor bug when shuffle was set to false that triggered an assertion in training engine

### Changed

- Package requirements now specify an upper bound on some packages, including Pytorch and PyG to ensure compatibility. Better be safe than sorry :)

## [1.3.0] Support for Pytorch 1.13.0, CUDA 11.6, CUDA 11.7, PyG 2.1.0, Ray 2.1.0, support + minor fixes

### Changed

- Updates to tests to make the fake datasets compatible with PyG 2.1.0

### Fixed

- IDLE ray workers not deallocating GPUs

- Now we sort the data list returned by training engine as if samples were not shuffled.

  Meaning the returned data list is consistent with the original ordering of the dataset.

#### Comments

We tried to provide support for creating an environment with PyG 2.2.0, but importing the library seems to cause
`segmentation fault` in certain cases. Therefore, we will wait until the issue is fixed and then update the script.

## [1.2.6] Minor changes

### Added

- You can now specify a specific subset of gpus to use in the configuration file.

  Just add the optional field `gpus_subset: 1,2,3` if you want to only use GPUs with index 1,2 and 3.

## [1.2.5] Reverting to previous Ray version

### Changed

- Ray 2.0.0 seems to have a problem with killing `IDLE` processes and releasing their resources, i.e. OOM on GPU.
  We are reverting to a version that we were using before and did not have this problem.

## [1.2.4] Minor fixes + tests for main functionalities

### Fixed

- Minor check in splitter
- Minor fix in link prediction splitter, one evaluation link was being left out
- Minor fix in early stopper, `epoch_results` dict was overwritten after applying early stopping. Does not affect performances since the field is re-initialized the subsequent epoch by the training engine.
- Removed setting random seed for map-style dataset. It was not useful (see Torch doc on reproducibility) and could cause transforms based on random sampling (e.g. negative sampling) to behave always in the same way

### Changed

- Changed semantics of gradient clipper, as there are not many alternatives out there

## [1.2.3] Added support for single graph tasks

At the moment, the entire graph must fit in CPU/GPU memory. `DataProvider` extensions to partition the graph using PyG should not be difficult.

### Added

- New splitter, `SingleGraphSplitter`, which randomly splits nodes in a single graph (with optional stratification)
- New provider, `SingleGraphDataProvider`, which adds mask fields to the single DataBatch object (representing the graph)

### Changed

- renamed method `get_graph_targets` of `Splitter` to `get_targets`, and modified it to make it more general

## [1.2.2] Telegram Bot Support!

### Added

- Telegram bot support. Just specify a telegram configuration in a YAML file and let the framework do the rest! Just remember not to push your telegram config file in your repo!

## [1.2.1] Splitter Fix after 1.2.0 changes, deprecating 1.2.0

### Fixed

- the changed introduced in splitter causes seed to be resetted when splits were loaded during each experiment. Now it has been fixed by setting the seed only when split() is called.
- minor in num_features of OGBGDatasetInterface

## [1.2.0] Simplified Metric usage

### Changed

- Simplified metrics to either take the mean score over batches or to compute epoch-wise scores (default behavior).
  In the former case, the result may be affected by batch size, especially in cases like micro-AP and similar scores.
  Use it only in case it is too expensive (RAM/GPU memory) to compute the scores in a single shot.

### Fixed

- Bug in splitter, the seed was not set properly and different executions led to different results. This is not a problem whenever the splits are publicly released after the experiments (which is always the case).
- Minor in data loader workers for iterable datasets

## [1.1.0] Temporal PyDGN for single graph sequences

### Added

- Temporal learning routines (with documentation), works with single graphs sequences
- Template to show how we can use PyDGN on a cluster (see `cluster_slurm_example.sh`) - launch using `sbatch cluster_slurm_example.sh`. **Disclaimer**: you must have experience with slurm, the script is not working out of the box and settings must be adjusted to your system.

### Fixed

- removed method from `OGBGDatasetInterface` that broke the data split generation phase.
- added `**kwargs` to all datasets

### Changed

- Extended behavior of ``TrainingEngine`` to allow for null target values and some temporal bookkeeping (allows a lot of code reuse).
- Now ``batch_loss`` and ``batch_score`` in the ``State`` object are initialized to ``None`` before training/evaluation of a new batch starts. This could have been a problem in the temporal setting, where we want to accumulate results for different snapshots.

## [1.0.9] Iterable Dataset implementation for large datasets stored on disk in chunks of files

We provide an implementation of iterable-style datasets, where the dataset usually doesn't fit into main memory and
it is stored into different files on disk. If you don't overwrite the ``__iter__`` function, we assume to perform data splitting at
file level, rather than sample level. Each file can in fact contain a list of ``Data`` objects, which will be streamed
sequentially. Variations are possible, depending on your application, but you can use this new dataset class as a good starting point.
If you do, be careful to test it together with the iterable versions of the data provider, engine, and engine callback.

### Added

- Implemented an Iterable Dataset inspired by the [WebDataset](https://github.com/webdataset/webdataset) interface
- Similarly, added ``DataProvider``, ``Engine`` and ``EngineCallback`` classes for the Iterable-style datasets.

### Changed

- Now we can pass additional arguments at runtime to the dataset

## [1.0.8] Minor changes

### Changed

- (Needs simple test) Setting ``CUDA_VISIBLE_DEVICES`` variable before cuda is initialized, so that in ``--debug`` mode we can use the GPU with the least amount of used memory.
- Commented a couple of lines which forces OMP_NUM_THREADS to 1 and Pytorch threads to 1 as well. It seems we don't need them anymore.

## [1.0.7] Minor fix

### Fixed

- to comply with `TUDataset`, we do not override the method `__len__` anymore

## [1.0.5] Minor fix

### Fixed

- `load_dataset` does not assume anymore that there exists a `processed` data folder, but it is backward-compatible with previous versions.
- fixed an indexing bug on target data for node classification experiments (caused program to crash)
- Metric: renamed `_handle_reduction` to `_expand_reduction` and created a new helper routine `_update_num_samples` to allow a user to decide how to compute and average scores.

## [1.0.4] Minor fix

### Fixed

- use of fractions of GPUs for a single task
- changed signature in forward to allow a dictionary (for MultiScore) or a value (for basic metrics)
- added squeeze in MulticlassAccuracy when target tensor has shape (?, 1)

## [1.0.3] Minor fix

### Fixed

- Same bug as before but for `pydgn-dataset` =).

## [1.0.2] Minor fix

### Fixed

- Bug that prevented locating classes via dotted paths in external projects.

## [1.0.0] PyDGN with Documentation

### Added

- A documentation (it was about time!!!)
- Possibility of specifying inner and outer validation ratio
- We can now use a specific data loader and specify its arguments in the configuration file
- We can now force a metric to compute node-based or graph-based metrics, rather than looking at the ground truth's shape.
- Possibility of evaluating on validation (and test) every `n` epochs
- Use entrypoints to simplify usage of the library
- All arguments must be now specified in the config file. There is a template one can use in the doc.

### Changed (IMPORTANT!)

- Removed any backward compatibilities with very old versions (<=0.6.2)
- Substituted Loss and Score classes with a single Metric class, to avoid code redundancy
- Pre-computed random outer validation splits (extracted from 10% of outer training set) for data splits from "A Fair Comparison on Graph Neural Networks for Graph Classification". Note that this does not impact the test splits.

## [0.7.3]

### Changed

- The setup installation files now work with Pytorch 1.10 and Pytorch Geometric 2.0.3, and the library assumes Python >= 3.8

## [0.7.2]

### Fixed
- Fixed minor bug in experiment. The function create_unsupervised_model looked for supervised_config, rather than unsupervised_config, when looking for the readout
- Feature request: loss, score, and additiveloss now take a parameter `use_nodes_batch_size` to force computation w.r.t. input nodes rather than target dimension (the default)

## [0.7.1]

### Added

- Minor refactoring of the engines to avoid redundant flow of information

### Fixed
- Fixed a bug in EventHandler. If one extends EventHandler with new events, which are triggered by a training engine, make sure that callbacks that implement the EventHandler interface do not break when the new events are triggered.
- Refactored Profiler to abstract from the EventHandler. This created problems when a callback implmenented an interface that extends EventHandler. If the callback does not implement a particular method, nothing happens and the dispatcher moves on.

## [0.7.0] - PyDGN temporal (with [Alessio Gravina](http://pages.di.unipi.it/gravina/) based on [Pytorch Geometric Temporal](https://pytorch-geometric-temporal.readthedocs.io/en/latest/modules/root.html)) + minor fixes

### Added

- PyDGN Temporal: Support for `single graph sequence` tasks, the most common use case at the moment
  (tested for supervised experiments only)

### Fixed

- Minor in `cgmm_incremental` experiment
- loss/score now considers the case of reduction=mean (default) and sum when computing the epoch's loss/score

## [0.6.1] - Stop and Resume from different devices

### Fixed

- When using checkpoints, we can now switch devices without getting a deserialization error

## [0.6.0] - Evaluator's Refactoring

### Added

- Evaluator now stores and displays values for the loss used. Also, validation of final runs is kept.

### Modified

- Heavy refactoring of the evaluator

### Fixed

- Removed old code in `TUDatasetInterface`
- Epochs' log starts from 1 :D

## [0.5.1] - Code for E-CGMM + Minor fixes

### Added

- Code for E-CGMM ("Modeling Edge Features with Deep Bayesian Graph Networks", IJCNN 2021)
- ConstantEdgeIfEmpty transform (used by e.g., E-CGMM)

### Fixed

- Changed name from `transforms` to `transform` for data preprocessing config files (backward compatible)
- Minor fix when handling edge data with incremental models like E-CGMM
- Fix in graph readout: forgot to pass arguments to super when inheriting from `GraphReadout`

## [0.5.0] - Pytorch 1.8.1, random search, and many fixes

### Added

- Data splits from "A fair comparison of graph neural networks for graph classification", ICLR 2021.

- Replaced strings with macros, to improve maintainability

- Support to replicability with seeds. Debug CPU mode and parallel CPU mode can reproduce the same results. With CUDA
  things change due to DataLoader implementation. Even by running the whole exp again, some runs in GPU differ slightly.

- Added current_set to Score and MultiScore, to keep track of whether the set under consideration is TRAINING,
  VALIDATION or TEST

- Random Search support: specify a `num_samples` in the config file with the number of random trials, replace `grid`
  with `random`, and specify a sampling method for each hyper-parameter. We provide different sampling methods:
    - choice --> pick at random from a list of arguments
    - uniform --> pick uniformly from min and max arguments
    - normal --> sample from normal distribution with mean and std
    - randint --> pick at random from min and max
    - loguniform --> pick following the recprocal distribution from log_min, log_max, with a specified base

- Implemented a 2-way training scheme for CGMM and variants, which allows to first compute and store the graph
  embeddings, and then load them from disk to solve classification tasks. Very fast and lightweight, allowing to easily
  try 1K configuration for each outer fold.

- Early stopping can now work with loss functions rather than scores (but a score must be provided nonetheless)

### Changed

- Minor improvements in result files

- Debug mode now prints output to the console

- Refactored engine for Link Prediction, by subclassing the TrainingEngine class

- Added chance to mini-batch edges (but not nodes) in single graph link prediction, to reduce the computational burden

- Compute statistics in iocgmm.py: removed 1 - from bottom computation, because it assigned 1 to nodes with degree 0

### Fixed:

- ProgressManager can load the elapsed time of finished experiments, in case you stop and resume the entire process

- Fix for semi-supervised graph regression in `training/util.py` (added an `.unsqueeze(0)` on the `y` variable for graph
  prediction tasks)

- Last config of model selection was not checked in debug mode

- Moving model to device inside experiments, before the model is passed to the optimizer. This solves optimizer
  initialization problems e.g., with Adagrad

- Minor fix in AdditiveLoss

- Minor fix in Progress Manager due to Ray upgrade in PyDGN 0.4.0

- Refactored both CGMM and its incremental training strategy

- Improved evaluator code to define just once remote ray functions

- Jupyter notebook for backward compatibility with our ICLR 2020 data splits

- Backward compatibility in Splitter that handles missing validation fields in old splits.

- Using data root provided through cli rather than the value stored in dataset_kwargs.pt by data preprocessing
  operations. This is because data location may have changed.

- Removed load_splitter from utils, which assumed a certain shape of the splits filename. Now we pass the filepath to
  the data provider.

- minor fix in AdditiveLoss and MultiScore

- MultiScore now does not make any assumption on the underlying scores. Plus, it is easier to maintain.

- CGMM (and variants) mini batch computation (with full batch training) now produces the same loss/scores as full batch
  computation

- minor fix in evaluator.py

- memory leak when not releasing output embeddings from gpu in `engine.py`

- releasing score output from gpu in `score.py`

## [0.4.0] - 2020-10-13

#### Ray support to distributed computation!

### Added:

- [Ray](https://github.com/ray-project/ray) transparently replaces multiprocessing. It will help with future extensions
  to multi-GPU computing

- Support for parallel executions on potentially different GPUs
  (with Ray we can now allocate a predefined portion of a GPU for a task)

- Dataset and Splitter for Open Graph Benchmark graph classification datasets

### Changed:

- Modified dataset/utils.py removing the need for Path objects

- Refactored evaluation: risk assessment and model selection logic is now greatly simplified. Code is more robust and
  maintainable.

- Print indented config

- Moved s2c to evaluation/utils.py

- Improved LaunchExperiment

- Config files should now specify the complete path to a specific experiment class

- Renamed files and folders to follow Python convention

### Fixed:

- bug fix when extending the list of final run jobs. We need to add to waiting variable the lastly scheduled jobs only.

- bug fix in evaluator when using ray

- Engine now saves (EngineCallback) and restores `stop_training` in checkpoint

## [0.3.2] - 2020-10-4

### Added

- Implemented a Multi Loss to combine different loss functions while plotting the individual components as well

- Added standard deviation when multiple final runs are used

### Changed

- Removed redundant files and refactored a bit to simplify folder structures

- Removed utils folder, moved the few methods in the other folders where they were needed

- Skipping those final runs that are already completed!

- For each config in k-fold model selection, skipping the experiment of a specific configuration that has produced a
  result on a specific fold

- K-Fold Assesser and K-Fold Selector can handle hold-out strategies as well. This simplifies maintenance

- The wrapper can now be customized by passing a specific engine_callback EventHandler class in the config file

- No need to specify `--debug` when using a GPU. Experiments that need to run in GPU will automatically trigger
  sequential execution

- Model selection is now skipped when a single configuration is given

- Added the type of experiment to the experiments folder name

- `OMP_NUM_THREADS=1` is now dynamically set inside `Launch_Experiment.py` rather by modifying the `.bashrc` file

- Simplified installation. README updated

- Final runs' outputs are now stored in different folders

- Improved splitter to allow for a simple train val test split with no shuffling. Useful to reuse holdout splits that
  have been already provided.

- Improved provider to use outer validation splits in run_test, if provided by the splitter (backward compatibility)

- Made Scheduler an abstract class. EpochScheduler now uses the epoch to call the step() method

### Fixed

- Removed ProgressManager refresh timer that caused non-termination of the program

- Continual experiments: `intermediate_results.csv` and `training_results.csv` are now deleted when restarting/resuming
  an experiment.

- Minor fix in engine about the `batch_input` field of a `State` variable

- Error when dumping a configuration file that contains a scheduler

- Error when validation was provided but early stopper was not. removed if that prevented validation and test scores
  from being computed

## [0.2.0] - 2020-09-10

A series of improvements and bug fixes. We can now run link prediction experiments on a single graph. Data splits
generated are incompatible with those of version 0.1.0.

### Added

- Link prediction data splitter (for single graph)
- Link prediction data provider (for single graph)

### Changed

- Improvements on progress bar manager
- Minor improvements on plotter

### Fixed

- Nothing relevant

### Additional Notes

The library now creates new processes using the `spawn` method. Spawning rather than forking prevents Pytorch from
complaining (see https://github.com/pytorch/pytorch/wiki/Autograd-and-Fork
and https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods). Also, there is a warning on a
leaked semaphore which we will ignore for now. Finally, `spawn` will be useful to implement CUDA
multiprocessing https://pytorch.org/docs/stable/notes/multiprocessing.html#multiprocessing-cuda-note. However, the
Pytorch DataLoader in a child process breaks if `num_workers > 0`. Waiting for Pytorch to address this issue.

## [0.1.0] - 2020-07-14

We use PyDGN on a daily basis for our internal projects. In this first release there are some major additions to
previous and unreleased version that greatly improve the user experience.

### Added

- Progress bars show the status of the experiments (with average completion time) for outer and inner cross validation (
  CV).
  ![](https://github.com/diningphil/PyDGN/blob/main/images/progress.png)
- Tensorboard visualization is activated by default when using a Plotter.
- A new profiler keeps track of the time spent on each event (see Event engine).
  ![](https://github.com/diningphil/PyDGN/blob/main/images/profiler.png)
- All experiments can be interrupted at any time and resumed gracefully (the engine looks for the last checkpoint).

### Changed

- Removed all models that are not necessary to try and test the library.

### Fixed

- Various multiprocessing issues caused by using the `fork` method with Pytorch.

### Additional Notes

The library now creates new processes using the `spawn` method. Spawning rather than forking prevents Pytorch from
complaining (see https://github.com/pytorch/pytorch/wiki/Autograd-and-Fork
and https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods). Also, there is a warning on a
leaked semaphore which we will ignore for now. Finally, `spawn` will be useful to implement CUDA
multiprocessing https://pytorch.org/docs/stable/notes/multiprocessing.html#multiprocessing-cuda-note. However, the
Pytorch DataLoader in a child process breaks if `num_workers > 0`. Waiting for Pytorch to address this issue.

repository: diningphil/PyDGN

gha_language: Python

repo_name: diningphil/PyDGN


text:
# FIXME

* BoundingBoxFilter should reset the box if the cursor is up

# NEW STUFF

* RotateEvent
* TiltEvent
* Cursor-to-mouse converter (with setMinCursorCount(), setMaxCursotCount(), reference to Window, etc.)
* Tap gesture with min/max tap count (including delay to make sure there is no more)
* Fire gesture events only for wanted touch targets
* "DoubleTapAndSlideToZoom" gesture
* Dispatchers should also translate cursors
* UserDispatcher
* Can dispatch cursors to the REST_TOUCH_TARGET instead of the SCREEN_TOUCH_TARGET
* Make input cursors already on a touch target => remove CursorProcessor interface & repackage everything
* Swing-based cursor-to-target dispatcher
* JavaFX-based cursor-to-target dispatcher
* Generic cursor-to-target dispatcher
* Gesture lock and priority
* Better packaging: Swing, basic gestures, Windows, MacBook trackpad, JavaFX, etc.
* Windows 7/8 touch events
* JavaFX touch events
* MacBook trackpad / magic mouse events
* Mouse/keyboard simulated touch events
* Handle unintended finger up
* Keyboard modifiers in Cursor
* Screen resolution changes at runtime

# THINK ABOUT IT

* SimpleProfile: setSource(), addFilters(), addGestures(), addInertia(), addGestureListenersPerRegion()
* CursorEvent vs. CursorProcessor vs. CursorCountChange
* Cursors inertia vs. mean cursor inertia vs. gesture inertia
* 2 paradigms: up/down changes vs. current state, CursorUpdateEvent#getAdded/Removed/Updated/Stationary/CurrentCursors()
* Block state ON/OFF/BYPASS
* Decouple Surface / Screen / Canvas
* Application-specific: change color while drawing
* Application-specific: auto-pan while drawing near the screen edge (different than drag)
* Android support?
* Plugin for NASA World Wind?
* TUIO input for JavaFX: TUIO events to JavaFX touch events
* Multi-user gesture
* Multi-touch-target gesture
* DSL


text:
# ReactJSGallery
#### ReactJSGallery: A simple gallery app that compares ones growth in their life-time.

### How to compile and test it locally
1. Clone the project: `https://github.com/PromasterGuru/ReactJSGallery.git`
2. cd to project directory: `cd ReactJSGallery`
3. Install dependencies: `npm install`
4. Open node server: `npm run server`

__[Live Demo](https://reactjs-gallery.herokuapp.com/)__

repository: PromasterGuru/ReactJSGallery

repo_name: PromasterGuru/ReactJSGallery


text:
## Writeup Template

### You can use this file as a template for your writeup if you want to submit it as a markdown file, but feel free to use some other method and submit a pdf if you prefer.

---

**Advanced Lane Finding Project**

The goals / steps of this project are the following:

* Compute the camera calibration matrix and distortion coefficients given a set of chessboard images.
* Apply a distortion correction to raw images.
* Use color transforms, gradients, etc., to create a thresholded binary image.
* Apply a perspective transform to rectify binary image ("birds-eye view").
* Detect lane pixels and fit to find the lane boundary.
* Determine the curvature of the lane and vehicle position with respect to center.
* Warp the detected lane boundaries back onto the original image.
* Output visual display of the lane boundaries and numerical estimation of lane curvature and vehicle position.

[//]: # (Image References)

[image1]: ./camera_cal/calibration1.jpg "Distorted"
[image2]: ./output_images/perspectiveTransformOutput_un_calibration1.jpg "Undistorted"
[image3]: ./output_images/binarythreshold.jpg "Binary Threshold"
[image4b]: ./output_images/perspectiveTransformOutput_calibration3out.jpg "Another Warpeed Image"
[image4]: ./output_images/warpedImage.jpg "Warpeed Image"
[image5]: ./output_images/fitPolynomial.jpg "Fit Polynomial"
[image6]: ./output_images/pipeline_007test2.jpg "Lanes Transformed Back on the image"
[video1]: ./output_images/project_video_output.mp4 "Video"

## [Rubric](https://review.udacity.com/#!/rubrics/571/view) Points

### Here I will consider the rubric points individually and describe how I addressed each point in my implementation.

---

### Writeup / README

#### 1. Provide a Writeup / README that includes all the rubric points and how you addressed each one.  You can submit your writeup as markdown or pdf.  [Here](https://github.com/udacity/CarND-Advanced-Lane-Lines/blob/master/writeup_template.md) is a template writeup for this project you can use as a guide and a starting point.

You're reading it!

It is best to explain the final pipeline and explain it step by step :

The following image explains very much what I am doing to each individual image, I am starting with finding an undistortedImage and then doing a binary threshod, after which I am unwarping the image, after which I did lane pixel detection using section wise pixel detection.
```
 def __pipeline__(self,img):
        img = self.__ms_undistortImage__(img)
        binary_threshold_image = self.__hls_select__(img)
        warpedImage = self.__warpImage__(binary_threshold_image)
        out_img = self.__fit_polynomial__(warpedImage)
        self.__measure_curvature_real__()
        # approximation of curvature of center of the lane using the curvature of left lane boundary and right lane boundary
        self.curvature = np.int_((self.left_curverad+self.right_curverad)/2)
        dewarpedImage = self.__dewarpImage__(out_img)
        imageWithLane = cv2.addWeighted(np.int_(img),1,np.int_(dewarpedImage),0.4,0)
        cv2.putText(imageWithLane,'Curvature = ' + str(self.curvature) + ' m' ,(150,100),cv2.FONT_HERSHEY_SIMPLEX,1,(255,0,0))
        if(self.distanceFromCenter < 0):
            cv2.putText(imageWithLane,'Car is driving ' + str(np.abs(self.distanceFromCenter)) + ' m' + ' left of lane center',(150,150),cv2.FONT_HERSHEY_SIMPLEX,1,(255,0,0))
        else:
            cv2.putText(imageWithLane,'Car is driving ' + str(np.abs(self.distanceFromCenter)) + ' m' + ' right of lane center',(150,150),cv2.FONT_HERSHEY_SIMPLEX,1,(255,0,0))
        return imageWithLane
```

### Pipeline (single images)

#### Camera Calibration

#### 1. Briefly state how you computed the camera matrix and distortion coefficients. Provide an example of a distortion corrected calibration image.

![Distorted Image][image1]
I am showing this sectino with respect to the calibration1.jpg image given as an example insidet the camaera_cal folder with the project.

The methods __undistort__ is responsible for calculating the cameraMatrix and distortionParameters and I am using cv2.calibrateCamera function to feed in object points and final expected points, for which I am using cv2.findChessboardCorners function.
After running the __undistortImage__ on the image calibration1.jpg I am getting final result as the next image
![UnDistorted Image][image2]

#### 2. Describe how (and identify where in your code) you used color transforms, gradients or other methods to create a thresholded binary image.  Provide an example of a binary image result.

```
	hls = cv2.cvtColor(img, cv2.COLOR_BGR2HLS)
	#Apply a threshold to the S channel
	L = hls[:,:,1]
	S = hls[:,:,2]
	binary_output = np.zeros_like(S)
	#third (L > 60) is helping in removing shadows
	#L > 200 is helping in detecting white lines
	binary_output[((S > thresh[0]) & (S <= thresh[1]) & (L > 60)) | (L > 200)] = 255
```
![alt text][image3]

#### 3. Describe how (and identify where in your code) you performed a perspective transform and provide an example of a transformed image.

The perspective transform matrix is calculated using the function

```python
	def __calculatePerspectiveTranformMatrixFromRoadSample__(self):
        if self.verbosity == 2:
            print('inside __perspectiveTranformOnRoad__ function')

        fname = self.testImageLocation + "straight_lines1.jpg"
        img = cv2.imread(fname)
        undistortedImage = self.__undistortImage__(img)
        offset = 300
        img_size = (img.shape[1], img.shape[0])

        src = np.array([[205,720],[596,450],[685,450],[1100,720]], np.double).reshape(4,1,2)

        dst = np.float32([
                [offset, img_size[1]],
                [offset, 0],
                [img_size[0]-offset, 0],
                [img_size[0]-offset, img_size[1]]
                ])
        self.__updateCameraMatrix__(np.float32(src),np.float32(dst))
        warpedImage = self.__warpImage__(undistortedImage)
        cv2.imwrite(self.outputImageLocation + 'afterPerspectiveTransform_' + fname.split('/')[-1],warpedImage)
```

![alt text][image4]

Another image showing the image warping is
![alt text][image4b]

#### 4. Describe how (and identify where in your code) you identified lane-line pixels and fit their positions with a polynomial?

Then I used density based approach to find the max density portion in the frame portions to locate the pixels corresponding to the left and right lane, this is done in the function __find_lane_pixels__, after that I have used __fit_polynomial__ function to fit a 2nd order polynomial on left and right lanes, This is followed by converting the result from pixel space to meter space and then to calculate the final curvature of the lane I am simply averaging the results of the left and the right lanes.

* I am avoiding any fancy averaging for now because this code was looking sufficient enought to generate good lane lines,
* have placed one check though that If the lane line is not detected then I am choosing the previous lane line as the current, the same idea can be extended to using the average of last few lane lines to calculate the present lane curvatures.

![alt text][image5]

#### 5. Describe how (and identify where in your code) you calculated the radius of curvature of the lane and the position of the vehicle with respect to center.

The measure curvature real is calculating the curvature in metres from the pixel space using
```
self.left_curverad = ((1 + (2*left_fit_cr[0]*y_eval*my + left_fit_cr[1])**2)**1.5) / np.absolute(2*left_fit_cr[0])
self.right_curverad = ((1 + (2*right_fit_cr[0]*y_eval*my + right_fit_cr[1])**2)**1.5) / np.absolute(2*right_fit_cr[0])
```

#### 6. Provide an example image of your result plotted back down onto the road such that the lane area is identified clearly.

The final implementation can be seen in the next image. The lane lines are transformed back to the image space and then plotted on top of original image to show the final result, this is done using function __dewarpImage__ and then merging the result with the original image and adding the text on top of it.
![alt text][image6]

---

### Pipeline (video)

#### 1. Provide a link to your final video output.  Your pipeline should perform reasonably well on the entire project video (wobbly lines are ok but no catastrophic failures that would cause the car to drive off the road!).

Here's a [link to my video result][video1]

---

### Discussion

#### 1. Briefly discuss any problems / issues you faced in your implementation of this project.  Where will your pipeline likely fail?  What could you do to make it more robust?
Well I will say that this code is very far from going into a real car, specially any change in the lighting condition is breaking it, may be an adaptive thresholding technique should be used to extend it to a more ggeneral scenario(even to pass the challenge video case)

repository: msinvent/CarND-Advanced-Lane-Lines

repo_name: msinvent/CarND-Advanced-Lane-Lines