text:
# Visual Regression Test

This helps identify visual UI differences between previous and current pages. Test can run on both headless and non-headless modes (browserstack).

## Configuration

To be able to run visual regression with any web application, some configuration is required:

### Environment Variables

Create `.env` file under the root of web application project and add below variables

| Name                       | Value            | Description                                                                                |
| -------------------------- | ---------------- | --------------------------------------------------------------------------------------------- |
| `BROWSERSTACK_USERNAME`    | `username`       | Browserstack Username used to launch device. Contact QA for relavant Browserstack Credentials |
| `BROWSERSTACK_KEY`         | `somekey`        | BrowserStack key for visual and screenshot tests                                              |
| `BUILD_KEY`                | `some value`     | Key is required to create new directory on S3 bucket for screenshots                          |
| `STORYBOOK_URL`            | `base url`       | This is base url of web application                                                           |
| `VISUAL_REGRESSION_LOG`    | `1,2 or 3`       | Info = 1, Debug = 2, Full = 2                                                                 |
| `VISUAL_REGRESSION_CONFIG` | `path of config` | Path of visual regression config. Default value: `visual-regression/config/index.js`          |

### AWS Authentication on CLI

- CLI needs AWS authentication to upload / download objects to / from S3. If you don't want to use S3 then AWS authentication is not required.

## Headless Tests

Headless tests run faster than non-headless because they don't require any device or browser to launch.

Navigate to directory (../projects/applications/"project"), then run following command:

```
// execute headless test
npm run visual-regression -- --headless
// or
npm run headless -- --name "name of test"

// execute headless test and download and upload baseline from s3
npm run headless -- --name "name of test" --downloads3 --uploads3 --uploads3onmismatch

// execute headless test and download and upload baseline from directory on s3
npm run headless -- --name "name of test" --downloads3 --uploads3 --uploads3onmismatch --buildkey "Build name"
```

### Arguments

Below are the supporting arguments for Headless mode:

| Argument               | Value               | Description                                                                                |
| ---------------------- | ------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| `--headless`           | n/a                 | Runs test in headless mode                                                                                |
| `--name`               | `some name of test` | Runs only provided tests (single or multiple). Check visual regression config file for name of test (e.g. label)                  |
| `--update`             | n/a                 | Updates all baseline screenshot                                                                                |
| `BUILD_KEY`            | `some value`        | Key is required to create new directory on S3 bucket for screenshots                                                              |
| `--vrconfig`           | `path of config`    | Loads visual regression config. Default value: `visual-regression/config/index.js`                                                |
| `--downloads3`         | n/a                 | Downloads baseline screenshot from s3 before execution                                                                            |
| `--uploads3`           | n/a                 | Uploads baseline screenshot to s3 after execution. It works only if screenshot doesn't exist on s3 or `--update` argument is used |
| `--uploads3onmismatch` | n/a                 | Uploads baseline screenshot to s3 only if mismatch occurs                                                                         |

### Testing Multiple Pages

You can test multiple page URLs by passing a string of comma separated IDs, for example

```
npm run visual-regression -- --name "test1, test2" --headless
```

### Change View Port Settings

Change page view port. For example, to use page view port width 1600, add/modify below code inside `bsOptions.vars` of visual regression config. Default viewport width is 800

```
headless: {
  viewportWidth: 1600
}
```

Puppeteer emulators can also be used. For example, to use `iPhone 6` device settings, add / modify below code inside `bsOptions.vars` of visual regression config. List of puppeteer devices can be found here https://github.com/GoogleChrome/puppeteer/blob/master/DeviceDescriptors.js

```
headless: {
  emulateDevice: 'iPhone 6'
}
```

View port can also be for each test separately and it will override default viewport settings.

```
{
  label: 'test1',
  headless: {
    viewportWidth: 1600
    // OR
    emulateDevice: 'iPhone 6'
  }
},
```

Note: `emulateDevice` will override `viewportWidth` and viewport settings within test will always take precedence on default

## Browser Tests

To run tests on device and browser in browserstack, simply add `--env` in command line argument and then paste label of browser that is mentioned in config.

Navigate to directory (../projects/applications/projectname), then run following command to run all tests against all browsers:

```
npm run visual-regression
```

### Arguments

Below are the supporting arguments for Headless mode:

| Argument               | Value               | Description                                                                                |
| ---------------------- | ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
| `--name`               | `one/more test`     | Runs only provided tests (single or multiple). Check visual regression config file for name of test (e.g. label). Default: Run all tests       |
| `--env`                | `one/more browsers` | Runs test in mentioned device / browsers only. See visual regression config of web project to see label of browsers. Default: Run all browsers |
| `--update`             | n/a                 | Updates all baseline screenshot                                                                                |
| `--local`              | n/a                 | Force browserstack local to launch. If browser includes `browserstack.local=true` then there is no need to put this argument in commandline.   |
| `--vrconfig`           | `path of config`    | Loads visual regression config. Default value: `visual-regression/config/index.js`                                                             |
| `--downloads3`         | n/a                 | Downloads baseline screenshot from s3 before execution                                                                                |
| `--uploads3`           | n/a                 | Uploads baseline screenshot to s3 after execution. It works only if screenshot doesn't exist on s3 or `--update` argument is used              |
| `--uploads3onmismatch` | n/a                 | Uploads baseline screenshot to s3 only if mismatch occurs                                                                                |
| `--dontflag`           | n/a                 | Flag all the tests as passed but still save diff files                                                                                |

### Testing Specific Pages Or Browsers

You can test multiple page URLs by passing a string of comma separated IDs, for example

```
npm run visual-regression -- --name "test1, test2"
```

You can test specific page URL against multiple browsers by passing a string of comma separated browserIDs, for example

```
npm run visual-regression -- --name "test1" --env "iphone7plus_safari, galaxys6_chrome"
```

## Notes

- First time, it will update reference directory with baseline screenshots (named as "headless.png") and second time it will compare with it.

- All screenshots can be downloaded / uploaded to S3 bucket, simply add `--downloads3`, `--uploads3` or `--uploads3onmismatch` arguments in commandline. It needs AWS authentication on CLI. Need to contact with DevOps.

## Things to remember

1. After cloning fresh repository, add `.env` file and environment variables `STORYBOOK_URL`, `BROWSERSTACK_USERNAME`, `BROWSERSTACK_KEY`
2. If there is no name or environment provided in command then script will run for all tests or environments within config.
3. Command should run within directory where visual regression config exists (e.g. service-now)
4. You need AWS access on CLI to be able to download / upload image to s3.

license: ISC

repository: amjadalibb/visual-regression

repo_name: amjadalibb/visual-regression


text:
# Lazada

A Ruby API for http://www.lazada.com.my

## Installation

Add this line to your application's Gemfile:

```ruby
gem 'lazada'
```

And then execute:

    $ bundle

Or install it yourself as:

    $ gem install lazada

## Lazada::Client

The client is the base for all communication with the API.

```ruby
client = Lazada::Client.new(api_key, user_id)
```

Default timezone is 'UTC'. Can be specified otherwise. Refer to lib/lazada/client.

## Lazada::Client methods
### Lazada::Client#get_products
Get all or a range of products.
```ruby
client.get_products(filter)
```

Possible filter values are all, live, inactive, deleted, image-missing, pending, rejected, sold-out.

### Lazada::Client#create_product
Create a new product.
```ruby
client.create_product({ 'SellerSku' => 'lz001A', ... })
```

### Lazada::Client#update_product
Update the attributes of one or more existing products.
```ruby
client.update_product({ 'SellerSku' => 'lz001A', 'Price' => 12  })
```

### Lazada::Client#remove_product
Removes one or more products
```ruby
client.remove_product(seller_sku)
```

### Lazada::Client#get_categories
Get the list of all product categories.
```ruby
client.get_categories
```

### Lazada::Client#get_category_attributes
Returns a list of attributes with options for a given category.
```ruby
cleint.get_category_attributes(primary_category_id)
```

### Lazada::Client#get_orders
Get the customer details for a range of orders.
```ruby
client.get_orders({ status: status })
```
Possible status values are pending, canceled, ready_to_ship, delivered, returned, shipped and failed.

### Lazada::Client#get_order
Get the order items for a single order.
```ruby
client.get_order(order_id)
```

### Lazada::Client#get_order_items
Returns the items for one order.
```ruby
client.get_order_items(order_id)
```

### Lazada::Client#set_images
Set the Images for a Product, by associating one or more URLs with it.
```ruby
client.set_images({ 'ProductImage' => { 'SellerSku' => 'lz001A', 'images' => { ... } } })
```

### Lazada::Client#feed_status
Returns detailed information about a specified feed.
```ruby
client.feed_status(feed_id)
```

### Lazada::Client#feed_list
Returns all feeds created in the last 30 days.
```ruby
client.feed_list
```

## Development

After checking out the repo, run `bin/setup` to install dependencies. You can also run `bin/console` for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).

## Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/yoolk/lazada. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.

## License

The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).

license: MIT

repository: cySheng/lazada

branch_name: refs/heads/master

extension: md

gha_language: Ruby

gha_license_id: MIT

license_type: permissive

repo_name: cySheng/lazada


text:
# Carry_Capacity_Fixed
The behavior of carry capacity for pawns is 35 kg multiplied by a body type multiplier. This mod aims to change that. Pawns will now be able to hold more in inventory, as well as increase caravan capacity. This mod effects animals and colonists.
Suggested to use with "Pick Up And Haul".

[Original Inspiration](https://ludeon.com/forums/index.php?topic=47837.msg451757#msg451757)
[RimWorld 1.0 download](https://github.com/DrZhivago1/carry-capacity-fix/releases/tag/v2.0)

### Important Info
- Save-friendly (add or remove)
- No known incompatibilities

### Mod Support
[Alpha Animals](https://steamcommunity.com/sharedfiles/filedetails/?id=1541721856)
[Boats](https://steamcommunity.com/sharedfiles/filedetails/?id=1931133197)


text:
---
sidebar_position: 2
---

# Config File

The config file - which unless specified in the cli should live in `./owl.config.json` - is used to describe how Owl should run your app and your tests. Below you can find all the options that can be specified.

### Options

| Name                   | Required | Default   | Description                                                                                |
| ---------------------- | -------- | --------- | -------------------------------------------------------------------------------------------- |
| **general**            |          |           |                                                                                |
| `debug`                | false    | `false`   | Prevents the CLI/library from printing any logs/output.                                      |
| `report`               | false    | `true`    | Generate an HTML report, displaying the baseline, latest & diff images.                      |
| **ios config**         |          |           |                                                                                |
| `ios.workspace`        | true     |           | Path to the `.xcworkspace` file of your react-native project                                 |
| `ios.scheme`           | true     |           | The name of the scheme you would like to use for building the app                            |
| `ios.configuration`    | true     | `Debug`   | The build configuration that should be used.                                                 |
| `ios.buildCommand`     | false    |           | Overrides the `xcodebuild` command making the above options obselete                         |
| `ios.binaryPath`       | false    |           | The path to the binary, if you are using a custom build command                              |
| `ios.quiet`            | false    |           | Passes the quiet flag to `xcode builds`                                                      |
| **android config**     |          |           |                                                                                |
| `android.buildCommand` | false    |           | Overrides the `assembleDebug` gradle command. Should build the apk                           |
| `android.buildType`    | false    | `Release` | Can be one of `debug` or `release`. Used to call either `assembleDebug` or `assembleRelease` |
| `android.binaryPath`   | false    |           | The path to the binary, if you are using a custom build command                              |
| `android.packageName`  |          |           | The package name/unique identifier of the app                                                |
| `android.quiet`        | false    |           | Passes the quiet flag to `gradlew`                                                           |

### Example

```json title="owl.config.json"
{
  "ios": {
    "workspace": "ios/OwlDemoApp.xcworkspace",
    "scheme": "OwlDemoApp",
    "device": "iPhone 13 Pro"
  },
  "android": {
    "packageName": "com.owldemoapp"
  }
}
```

repository: masiddee/react-native-owl

branch_name: refs/heads/main

repo_name: masiddee/react-native-owl


text:
# microBrew-server
> Homebrew done the smart way

Brewing beer at home is a really cool hobby. Sometimes, it can be a bit tedious, but nothing can compete with the taste of the beer that you brewed yourself.

To help myself with this task, I'm building a **microBrew** suite that will help me to automate some of the tasks, and hopefully make the whole process a bit easier.

This project containe the source code for the server used for temperature monitoring and heating element control. It communicates with **microBrew-controller** using very simple text based protocol.

## Getting Started

### Installing Dependencies
Before you can get started with development, you need to make sure that all prerequisites are installed on your system. Project uses **InfluxDB** for recording temperatures, and **CouchDB** for storing brew batch metadata and recipes in the future. Those can be either installed on your system or run inside Docker container using provided `docker-compose.yml` file.

Next, create Python virtual environment that will be used for development, and install required dependencies:

```bash
# Create virtual environment
virtualenv venv

# Activate virtual environment
venv/Scripts/activate.sh # linux
# or
venv\Scripts\activate.bat # windows

# Finally install dependencies
pip install requirements.txt
```

### Initial Development Setup

Running `docker-compose up` for the first time will fetch pull all required images from docker hub and create containers for InfluxDB and CouchDB. However, initial configuration needs to be performed to set up initial accounts that will be used by the application.

#### CouchxDB Setup

Open following [link](http://localhost:5984/_utils/#createAdmin/) in a browser and configure admin account.

For development just use `root` as username and password.

#### InfluxDB Setup

InfluxDB doesn't require authentication by default, let's leave it this way for development.

#### Chronograf Setup

Chronograf is a dashboard that can be used for viewing records in `InfluxDB` database.

Open following [link](http://localhost:8888/) in a browser. In setup wizard update connection url to `http://timeseries-db:8086`.

#### docker-compose variables

Server reads its configuration from environment variables. All of the variables listed below, need to be set up on your dev machine before running the server. Take a look at `docker-compose.yml` to see how those are configured in docker environment.

| Variable Name  | Description            |
| -------------- | ---------------------- |
| TS_DB_HOST     | InfluxDB hostname      |
| TS_DB_PORT     | InfluxDB port          |
| TS_DB_USERNAME | InfluxDB username      |
| TS_DB_PASSWORD | InfluxDB password      |
| TS_DB_DATABASE | IndluxDB database name |
| BR_DB_HOST     | CouchDB hostname       |
| BR_DB_PORT     | CouchDB port           |
| BR_DB_USERNAME | CouchDB username       |
| BR_DB_PASSWORD | CouchDB password       |
| BR_DB_DATABASE | CouchDB database name  |

gha_language: Python


text:
# Doing a pull request

Once we checked that our first 0.0.1 version is deployed into Staging, let’s change our application in a different git branch and create a pull request to merge into master.

## Creating the GitHub Pull Request

Let’s get into the local git repo:

```bash
cd ~/jx-gke-lab/jx-go-http
```

Then, create a new branch for the pull request and check it out from your Cloud Shell:

```bash
git checkout -b my-pr
```

Open the Cloud Shell editor and update the main.go file:

```bash
cloudshell edit main.go
```

The Cloud Shell editor should open the file, that you can now edit graphically.  Update line 10 so it the variable title has a new value. The new line should like like this:

        title := "Jenkins X golang http example by <your name>"

When done, press Ctrl + S (or Command + S if you're using a Mac).
Now, stage the change for commit and commit the change to GitHub:

```bash
git add main.go
git commit -m "New welcome message"
```

Finally, push the changes to GitHub:

```bash
git push --set-upstream origin my-pr
```

Now we are ready to create the Pull Request (PR). So, let’s use the GitHub web console to create the pull request with the changes you pushed from your “my-pr” branch:
* Go to https://github.com/<your_github_username>/jx-go-http
* Check that your changes are waiting for creating the PR and select “Compare & pull request” (DON’T FORGET TO SIGN IN INTO GITHUB!)
* Then edit your PR message (e.g. “New welcome message”)
* Check your changes

Here you have some screenshots to see the process:

![Compare and PR in GirHub](https://raw.githubusercontent.com/dcanadillas/jenkins-x-lab/master/images/GitHubPR.png)

![Create a GitHub PR](https://raw.githubusercontent.com/dcanadillas/jenkins-x-lab/master/images/GHCreatePR.png)

> **Note:** Instead of using GitHub web console you can also create the pull request directly from Jenkins X with the following command
>
> `$> jx create pr -t “New welcome message” `
>
> Open then the link that is the output of the command and you go directly to the pull request created.

## Preview the application and approve the Pull Request

Once the Pull Request is created on GitHub we sould use the Prow commands to approve the pull request via ChatOps, so then the merge and pipelines are triggered automatically once one of the approvers in the “OWNERS” file of the application repo select `/approve` or `/lgtm` (Looks Good to Me).

The following actions happened automatically when we created the PR:
* An approval notifier is saying that “This PR is NOT APPROVED” and suggest some ChatOps commands that can be used from Prow

![Approval Notifier in GitHub](https://raw.githubusercontent.com/dcanadillas/jenkins-x-lab/master/images/GHPRApprovalNotifier.png)

* Then, a size label is attached to the change proposed in the PR (size/XS) in this case

![Automated Jenkins X label](https://raw.githubusercontent.com/dcanadillas/jenkins-x-lab/master/images/GHLabel.png)

* After a minute more or less a Preview environment is created and deployed

![Automated Jenkins X label](https://raw.githubusercontent.com/dcanadillas/jenkins-x-lab/master/images/GHPreviewPR.png)

Check the progress of the PR build:

```bash
jx get activity -f dcanadillas/jx-go-http/PR-1
```

Output (do not copy)

```
STEP                                                 STARTED AGO DURATION STATUS
dcanadillas/jx-go-http/PR-1 #1                            15m29s      57s Succeeded
  from build pack                                         15m29s      57s Succeeded
    Credential Initializer 96dnz                          15m29s       0s Succeeded
    Working Dir Initializer R2ngt                         15m28s       0s Succeeded
    Place Tools                                           15m27s       0s Succeeded
    Build Container Build                                 15m25s      17s Succeeded
    Build Make Linux                                      15m26s      10s Succeeded
    Git Merge                                             15m26s       4s Succeeded
    Git Source Dcanadillas Jx Go Http Pr 1 Ser V6bpx      15m26s       1s Succeeded https://github.com/dcanadillas/jx-go-http
    Postbuild Post Build                                  15m25s      18s Succeeded
    Promote Jx Preview                                    15m25s      53s Succeeded
    Promote Make Preview                                  15m25s      32s Succeeded
  Preview                                                 14m33s           https://github.com/dcanadillas/jx-go-http/pull/1
    Preview Application                                   14m33s           http://jx-go-http.jx-dcanadillas-jx-go-http-pr-<IP_ADDRESS>.245.nip.io
```

You can check the preview environment by:

```bash
jx get previews
```

Output (do not copy)

```
PULL REQUEST                                     NAMESPACE                      APPLICATION
https://github.com/dcanadillas/jx-go-http/pull/1 jx-dcanadillas-jx-go-http-pr-1 http://jx-go-http.jx-dcanadillas-jx-go-http-pr-<IP_ADDRESS>.245.nip.io
```

So, you can check the application preview environment by going to the previous URL in the output of ‘jx get previews’. There you should see your changes in Development to be merged into Master and then to be able to deploy a new version into Staging:

![Preview of Staging](images/GHPreviewPR.png)

In this lab we didn’t configure a different GitHub user to act as “a bot” so you cannot approve your own PR with the `/approve` command of Prow. So, in this case we will close the PR manually to trigger the promotion pipeline into Staging:

![Merging manually the PR](images/GHMergePR.png)

> **Note:**
>
> During the installation process you could have configured a different GitHub user for the “Pipeline Server”. This user then act as a “bot” to generate the comments automatically in the Pull Request to work with Prow.
>
> Then, you could approve a pull request just by commenting “/approve” in the PR messages if you are one of the approvers defined in the repo `OWNERS` file.
> ![Approve Prow command in GitHub](https://raw.githubusercontent.com/dcanadillas/jenkins-x-lab/master/images/ApproveProw.png)

This should trigger the promotion pipeline into Staging. Let’s check the execution:

```bash
jx get activity -f jx-go-http -w
```

Output (do not copy)

```
dcanadillas/jx-go-http/master #2                           1m25s          Running Version: 0.0.2
  from build pack                                          1m25s          Running
    Credential Initializer Slgsg                           1m25s       0s Succeeded
    Working Dir Initializer B878d                          1m24s       0s Succeeded
    Place Tools                                            1m23s       0s Succeeded
    Build Container Build                                  1m22s      17s Succeeded
    Build Make Build                                       1m22s       9s Succeeded
    Build Post Build                                       1m21s      17s Succeeded
    Git Merge                                              1m22s       2s Succeeded
    Git Source Dcanadillas Jx Go Http Master R Tlsmg       1m22s       1s Succeeded https://github.com/dcanadillas/jx-go-http
    Promote Changelog                                      1m21s      23s Succeeded
    Promote Helm Release                                   1m21s      27s Succeeded
    Promote Jx Promote                                     1m21s          Running
    Setup Jx Git Credentials                               1m22s       2s Succeeded
  Promote: staging                                           45s      44s Succeeded
    PullRequest                                              45s      44s Succeeded  PullRequest: https://github.com/dcanadillas-kube/environment-jenkins-x-lab-staging/pull/2 Merge SHA: bfff9a3be0eef8d845256979d9aa89702e31afb4
    Update                                                    1s       0s Succeeded
    Promoted                                                  1s       0s Succeeded  Application is at: http://jx-go-http.jx-staging.<IP_ADDRESS>.nip.io
```

Once we check in the previous output that the Pull Request #2 is succeeded, we can see our new version deployed:

```bash
jx get version
```

Output (do not copy)

```
APPLICATION STAGING PODS URL
jx-go-http  0.0.2   1/1  http://jx-go-http.jx-staging.<IP_ADDRESS>.nip.io
```

Let’s check the application deployed into staging by:

```bash
jx open jx-go-http -e staging
```

If we open the output url shown from the previous output in a browser tab the new version 0.0.2 of the application with the changes is shown.

Now you can go to the [next section](./JXPromotion.md):

<!-- [![Setup your environment](https://gstatic.com/cloudssh/images/open-btn.png)](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/dcanadillas/jenkins-x-lab&tutorial=labs/JXPromotion.md) -->

repository: dcanadillas/jenkins-x-workshop

gha_language: Shell

repo_name: dcanadillas/jenkins-x-workshop


text:
[![License](http://img.shields.io/:license-apache-blue.svg?style=flat-square)](http://www.apache.org/licenses/LICENSE-2.0.html)
[![Build Status](https://travis-ci.org/indigo-dc/ansible-role-disvis-powerfit.svg?branch=master)](https://travis-ci.org/indigo-dc/ansible-role-disvis-powerfit)

Role and Dockerfile for disvis and powerfit application
=======================================================

Roles and Dockerfiles to install the disvis or powerfit application:

* https://github.com/haddocking/disvis.git
* https://github.com/haddocking/powerfit.git

The application can be chosen from a role variable.

Introduction
------------

The repository contains ansible-roles that are published in
ansible galaxy: https://galaxy.ansible.com/indigo-dc/disvis-powerfit/

The directories docker-disvis and docker-powerfit are linked to
dockerhub with automatic build of those 2 images. These images can run
disvis or powerfit either in CPUs, or in GPUs, the NVIDIA driver version
is show below in the the ansible-role default variable
*nvidia_driver_version*

Requirements
------------

The role LIP-Computing.ansible-role-nvidia should be run before in order
to install the nvidia driver and is hosted in the repository:

* https://github.com/LIP-Computing/ansible-role-nvidia

Check the README to install and run the role

Role Variables
--------------

The variables that can be passed to this role and a brief description
about them are as follows.

1. Haddocking application: disvis or powerfit
   * haddock_app: Default = disvis
2. URL with opencl headers
   * opencl_url: Default = http://www.lip.pt/~david/cl_include.tgz

Install the Playbook
--------------------

To install the role:

```
$ ansible-galaxy install indigo-dc.disvis-powerfit
```

Run the playbook
----------------

An example of playbook for the disvis:

```
---
- hosts: localhost
  roles:
    - { role: indigo-dc.disvis-powerfit haddock_app: disvis }
```

Or execute:

```
$ ansible-playbook /etc/ansible/roles/indigo-dc.disvis-powerfit/tests/disvis.yml
```

An example of playbook for the powerfit:

```
---
- hosts: localhost
  roles:
    - { role: indigo-dc.disvis-powerfit, haddock_app: powerfit }
```

Or execute:

```
$ ansible-playbook /etc/ansible/roles/indigo-dc.disvis-powerfit/tests/powerfit.yml
```

Run the disvis application
--------------------------

The example runs disvis on the CPU with 2 threads:

```
$ cd /home
$ mkdir out
$ export LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu:/usr/local/lib64
$ export EX_DIR=/usr/local/disvis/test-cases/PRE5-PUP2-complex
$ disvis ${EX_DIR}/O14250.pdb ${EX_DIR}/Q9UT97.pdb ${EX_DIR}/restraints.dat -p 2 -a 20.0 -vs 2 -d /home/out
```

To run in the GPU:

```
$ cd /home
$ mkdir out
$ export LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu:/usr/local/lib64
$ export EX_DIR=/usr/local/disvis/test-cases/PRE5-PUP2-complex
$ disvis ${EX_DIR}/O14250.pdb ${EX_DIR}/Q9UT97.pdb ${EX_DIR}/restraints.dat -g -a 20.0 -vs 2 -d /home/out
```

Run the powerfit application
----------------------------

The example runs powerfit on the CPU with 2 threads:

```
$ cd /home
$ ./run-powerfitCPU.sh
```

To run in the GPU:

```
$ cd /home
$ ./run-powerfitGPU.sh
```

License
-------

Apache v2

Author Information
------------------

Mario David<EMAIL_ADDRESS>
LIP Lisbon: http://www.lip.pt

Indigo DataCloud: https://www.indigo-datacloud.eu/

Acknowledgments
---------------

* https://github.com/haddocking/disvis.git
* https://github.com/haddocking/powerfit.git

repository: indigo-dc/ansible-role-disvis-powerfit

repo_name: indigo-dc/ansible-role-disvis-powerfit


text:
---
layout: default
---

# 1997-05-06 - Custom programming, Web site design & development, and more.

## Header Data

From: "DataET Research Data Engineering Technologies"<EMAIL_ADDRESS>To<EMAIL_ADDRESS>Message Hash: 0fb4b22bd75314b236c802774716055f6ab75c1e2352602f59509d89e7c29287

Message ID<EMAIL_ADDRESS>Reply To: _N/A_

UTC Datetime: 1997-05-06 22:53:48 UTC

Raw Date: Wed, 7 May 1997 06:53:48 +0800

## Raw message

```
{% raw  %}From: "DataET Research Data Engineering Technologies"<EMAIL_ADDRESS>Date: Wed, 7 May 1997 06:53:48 +0800
To<EMAIL_ADDRESS>Subject: Custom programming, Web site design & development, and more.
Message-ID<EMAIL_ADDRESS>MIME-Version: 1.0
Content-Type: text/plain

Visit DataET Research, an excellent source for high-quality, but extensively
inexpensive, software development and marketing solutions.

SERVICES:

o Custom Software Development Service
o Custom Database Application Development Service
o Help File / System Development Service
o Installation Program Development Service
o Shareware Distribution & Marketing Service
o Trial Usage Tracking System Development Service
o Web Site Design & Development Service
o Press Release Distribution Service

If you are interested in having a spectacular Web site or customized software
package developed for your organization, VISIT: http://www.dataet.com.

Free cost and time estimates are given.

PRODUCTS:

o VSA2048 Crytography (Encryption) Module
o VSA2048 QuickCrypt (Encryption)

If you are interested in trying out the latest encryption software developed by
DataET Research, VISIT: http://www.dataet.com.

Shareware trial edition versions are available.

DataET Research
Data Engineering Technologies
VISIT: http://www.dataet.com

---------------------------------------------------------
Get Your *Web-Based* Free Email at http://www.hotmail.com
---------------------------------------------------------

{% endraw %}
```

## Thread

+ Return to [May 1997](/archive/1997/05)

+ Return to "["DataET Research Data Engineering Technologies" <dataet@hotmail.com>](/authors/dataet_research_data_engineering_technologies_dataet_at_hotmail_com_)"
+ Return to<EMAIL_ADDRESS>(Dr.Dimitri Vulis KOTM)](/authors/dlv_at_bwalk_dm_com_dr_dimitri_vulis_kotm_)"
+ Return to<EMAIL_ADDRESS>(Igor Chudov @ home)](/authors/ichudov_at_algebra_com_igor_chudov__at__home_)"
+ Return to "[Ray Arachelian <sunder@brainlink.com>](/authors/ray_arachelian_sunder_at_brainlink_com_)"

+ 1997-05-06 (Wed, 7 May 1997 06:53:48 +0800) - Custom programming, Web site design & development, and more. - _"DataET Research Data Engineering Technologies"<EMAIL_ADDRESS>  + 1997-05-07 (Wed, 7 May 1997 23:53:45 +0800) - [Re: Custom programming, Web site design & development, and more.](/archive/1997/05/5a8049733148e831039814d5c3730d9c1ec7e405148f97e58e031db2b9054388) - _Ray Arachelian<EMAIL_ADDRESS>    + 1997-05-08 (Thu, 8 May 1997 09:14:11 +0800) - [Re: Custom programming, Web site design & development, and more.](/archive/1997/05/4c34aff61a8425f8573b2220f46d998480ca03087b1a00c5737dd4cbf72c62af) -<EMAIL_ADDRESS>(Igor Chudov @ home)_
      + 1997-05-08 (Thu, 8 May 1997 11:26:24 +0800) - [Re: Custom programming, Web site design & development, and more.](/archive/1997/05/2e75c3cdff2f7d3d71791ce4cc0a6006e00ccae55ddedc7aafddd73fceeeb0d8) -<EMAIL_ADDRESS>(Dr.Dimitri Vulis KOTM)_
      + 1997-05-08 (Fri, 9 May 1997 00:15:30 +0800) - [Re: Custom programming, Web site design & development, and more.](/archive/1997/05/dc1a27178fb5045861db49a52423f9b193b76efc6e0f9badbb08686c75f1fcf4) - _Ray Arachelian<EMAIL_ADDRESS>


text:
---
title: "Validating Strings in Visual Basic | Microsoft Docs"
ms.custom: ""
ms.date: "2015-07-20"
ms.prod: .net
ms.reviewer: ""
ms.suite: ""
ms.technology:
  - "devlang-visual-basic"
ms.tgt_pltfrm: ""
ms.topic: "article"
dev_langs:
  - "VB"
helpviewer_keywords:
  - "Visual Basic, strings"
  - "Visual Basic code, strings"
  - "strings [Visual Basic], validating"
  - "validation, strings"
ms.assetid: 9a1acf9b-7e2c-40cb-bba4-656ee2bc2ed0
caps.latest.revision: 15
author: dotnet-bot
ms.author: dotnetcontent

translation.priority.ht:
  - "cs-cz"
  - "de-de"
  - "es-es"
  - "fr-fr"
  - "it-it"
  - "ja-jp"
  - "ko-kr"
  - "pl-pl"
  - "pt-br"
  - "ru-ru"
  - "tr-tr"
  - "zh-cn"
  - "zh-tw"
---
# Validating Strings in Visual Basic
This section discusses how to validate strings in [!INCLUDE[vbprvb](../../../../csharp/programming-guide/concepts/linq/includes/vbprvb_md.md)].

## In This Section
 [How to: Validate File Names and Paths in Visual Basic](../../../../visual-basic/programming-guide/language-features/strings/how-to-validate-file-names-and-paths.md)
 How to determine whether a string represents a file name or path.

 [How to: Validate Strings That Represent Dates or Times](../../../../visual-basic/programming-guide/language-features/strings/how-to-validate-strings-that-represent-dates-or-times.md)
 How to determine whether a string represents a valid date.

 [Using Regular Expressions with the MaskedTextBox Control in Visual Basic](../../../../visual-basic/programming-guide/language-features/strings/using-regular-expressions-with-the-maskedtextbox-control.md)
 Demonstrates how to convert simple regular expressions to work with the <xref:System.Windows.Forms.MaskedTextBox> control.

 [Walkthrough: Validating That Passwords Are Complex](../../../../visual-basic/programming-guide/language-features/strings/walkthrough-validating-that-passwords-are-complex.md)
 How to determine whether a string has the characteristics of a strong password.

## See Also
 [Strings](../../../../visual-basic/programming-guide/language-features/strings/index.md)
 [MaskedTextBox Control](../../../../framework/winforms/controls/maskedtextbox-control-windows-forms.md)


text:
---
layout: post
title:  "Spark源码学习笔记（十三）"
date:   2020-06-30 23:14:13 +0800
tags:
- Spark
color: rgb(207, 207, 207)
cover: '/blog/Halloween/2020_05_14_05.webp'
subtitle: 'OutputCommitCoordinator'
---

> OutputCommitCoordinator主要用于判断task是否有权限提交输出到HDFS，canCommit用于判断task是否有提交权限，使用endpointRef发送消息给endpoint
>
```scala
def canCommit(
      stage: Int,
      stageAttempt: Int,
      partition: Int,
      attemptNumber: Int): Boolean = {
    val msg = AskPermissionToCommitOutput(stage, stageAttempt, partition, attemptNumber)
    coordinatorRef match {
      case Some(endpointRef) =>
        ThreadUtils.awaitResult(endpointRef.ask[Boolean](msg),
          RpcUtils.askRpcTimeout(conf).duration)
      case None =>
        logError(
          "canCommit called after coordinator was stopped (is SparkEnv shutdown in progress)?")
        false
    }
  }
```

> endpoint接收到消息后判断能否提交，如果task被标记为失败，则没有权限，提交过或者stage已经结束也会导致失败

```scala
private[scheduler] def handleAskPermissionToCommit(
      stage: Int,
      stageAttempt: Int,
      partition: Int,
      attemptNumber: Int): Boolean = synchronized {
    stageStates.get(stage) match {
      case Some(state) if attemptFailed(state, stageAttempt, partition, attemptNumber) =>
        logInfo(s"Commit denied for stage=$stage.$stageAttempt, partition=$partition: " +
          s"task attempt $attemptNumber already marked as failed.")
        false
      case Some(state) =>
        val existing = state.authorizedCommitters(partition)
        if (existing == null) {
          logDebug(s"Commit allowed for stage=$stage.$stageAttempt, partition=$partition, " +
            s"task attempt $attemptNumber")
          state.authorizedCommitters(partition) = TaskIdentifier(stageAttempt, attemptNumber)
          true
        } else {
          logDebug(s"Commit denied for stage=$stage.$stageAttempt, partition=$partition: " +
            s"already committed by $existing")
          false
        }
      case None =>
        logDebug(s"Commit denied for stage=$stage.$stageAttempt, partition=$partition: " +
          "stage already marked as completed.")
        false
    }
  }
```
> 其他的就是stage开始和结束会调用该类中的stageStart/stageEnd方法，添加/移除相应的stageId及相关信息
>
> 当task完成时DAGScheduler会调用taskCompleted方法，如果状态是失败，stage会做记录

```scala
private[scheduler] def taskCompleted(
      stage: Int,
      stageAttempt: Int,
      partition: Int,
      attemptNumber: Int,
      reason: TaskEndReason): Unit = synchronized {
    val stageState = stageStates.getOrElse(stage, {
      logDebug(s"Ignoring task completion for completed stage")
      return
    })
    reason match {
      case Success =>
      // The task output has been committed successfully
      case _: TaskCommitDenied =>
        logInfo(s"Task was denied committing, stage: $stage.$stageAttempt, " +
          s"partition: $partition, attempt: $attemptNumber")
      case _ =>
        // Mark the attempt as failed to blacklist from future commit protocol
        val taskId = TaskIdentifier(stageAttempt, attemptNumber)
        stageState.failures.getOrElseUpdate(partition, mutable.Set()) += taskId
        if (stageState.authorizedCommitters(partition) == taskId) {
          logDebug(s"Authorized committer (attemptNumber=$attemptNumber, stage=$stage, " +
            s"partition=$partition) failed; clearing lock")
          stageState.authorizedCommitters(partition) = null
        }
    }
  }
```
> 这块的东西比较简单，没有涉及到其他的东西，主要就是判断task是否有提交输出权限到hdfs

gha_language: HTML


text:
---
layout: inner
title: find empty json objects in postgres
tags: ["postgres","json"]
---
Say you have a content column of type json, which holds arrays, and you want to delete the empty ones:

<pre>
select * from events where content::text = '[]'::text;
</pre>


text:
---
name: Bug report
about: Create a report to help us improve
title: "[Bug]"
labels: bug
assignees: ''

---

#make sure you run the program thru the command line
https://github.com/zaze06/TicTacTo/wiki/How-to-start-thru-command-line

#You game disent start?
follow this https://github.com/zaze06/TicTacTo/wiki/My-game-isent-starting if it still doesn't start then you can report the issue

**Describe the bug**
A clear and concise description of what the bug is.

**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

**Expected behavior**
A clear and concise description of what you expected to happen.

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Desktop (please complete the following information):**
 - OS: [e.g. iOS]
 - Java version
 - Version [e.g. 22]

**Crach File**
Please use https://gist.github.com

**Additional context**
Add any other context about the problem here.

gha_language: Java


text:
---
layout: docs-cat
description: 'Use cases and examples for WoTT Agent and Dashboard'
title: 'Use Cases and Examples for WoTT After Setup'
full_description: Some use cases and examples for the WoTT Agent and Dashboard that you can use for inspiration as well as to help you get to grips with our services and secure your IoT devices.
title_content: 'Use Cases and Examples'
redirect_from:
    - /documentation/setup
---

## How to use WoTT

Now that you have the WoTT Agent installed on your device(s), you're ready to use our certificates and credential manager for your projects. Recall that these two services have different purposes regarding your projects and what you hope to achieve.

If you have active projects that utilize security and wish to experiment with WoTT's capabilities by yourself, here are some pointers to keep in mind:

* The WoTT directory on your device is only accessible by `sudo` group users
* The file path for WoTT information you call is in the form of `/opt/wott`
* WoTT CA issues certificates are x509 and are downloaded as a `.crt` file into this path `/opt/wott/certs`
* The certificates refresh every 7 days - you will need to account for this in your projects
* Credentials are downloaded as JSON files into this path `/opt/wott/credentials`
* WoTT fetches information via daemon every 15 minutes

If you need a refresher on credentials, refer back to the [Getting Started guide]({{site.url}}/documentation/getting-started). Otherwise, you are now all set up to use WoTT in your own applications. If you need any assistance, we have a [support thread](https://www.raspberrypi.org/forums/viewtopic.php?f=41&t=243538) where you can interact directly with our developers and contributors alike.

Let's move forward and together make edge devices more secure.

## Some use cases to inspire your projects

If you do not have any specific projects in mind; or are downloading WoTT to test our services out, then you can refer to some of our pre-written examples below to give you an idea of scenarios where you can use our Agent.

Many of these examples include using WoTT to secure connections with popular cloud platforms like Google Cloud or Adafruit IO with more use cases constantly being developed and added. If you have a platform in mind that we do not have a guide for, please suggest it on the support thread- or request to add it to our repository yourself; we're open-source after all and welcome contributors.

Our use cases can be divided into certificate managed or credential managed examples as both are required for different functionalities. We'll separate them below for clarity. As a general rule, certificate managed examples tend to be a bit trickier. Once again, please go to our support thread if you need any help.

### Certificate managed examples

* [Google Core IoT]({{site.url}}/blog/tutorials/2019/06/14/google-core-iot)
* [A Simple WebApp]({{site.url}}/blog/tutorials/2019/06/16/simple-webapp)
* [Configuring Nginx with Client Certificate Authentication (mTLS)]({{site.url}}/blog/tutorials/2019/07/15/mtls-with-nginx)
* [Securing Access to Mosquitto Broker]({{site.url}}/blog/tutorials/2019/07/15/mosquitto-mqtt)
* [Connecting Edge IoT Devices to a Kubernetes Cluster]({{site.url}}/blog/tutorials/2019/07/18/edge-to-kubernetes)
* [Securing Access to Mosquitto Broker from Paho Client]({{site.url}}/blog/tutorials/2019/07/19/paho-mqtt)

### Credential managed examples

* [Simple WebApp with Basic HTTP Auth access]({{site.url}}/blog/tutorials/2019/06/18/simple-webapp-auth)
* [Using WoTT credentials to manage Adafruit IO access through an MQTT Client]({{site.url}}/blog/tutorials/2019/06/27/adafruit-io)
* [Managing access to Screenly OSE via HTTP authentication with WoTT authentication]({{site.url}}/blog/tutorials/2019/06/25/screenly-ose)

## Final notes and thanks

Firstly from all of us, thank you for using WoTT.

While IoT has been a major revolution of the modern day lifestyle, from our homes to supermarkets; it poses a very real and exploitable security threat- but it doesn't need to. Through these examples, we hope you can see how easy it is to use WoTT to secure your projects so that you can be confident going forward as developers in the global phenomenom that is IoT.

At WoTT, we are a community of security-minded individuals and we value your contributions as the developers that use our product. We therefore encourage you to interact with us on our support thread and Github so that we can build a more secure IoT future together.

repository: Menmarin/wott-io

repo_name: Menmarin/wott-io


text:
# Privacy Policy
###### version 1.0 of This Agreement was created on June 11, 2021.

ScoreBoard is an application for counting goals and a timer in sports games and outputting results to third-party streamer programs (OBS, WireCast, etc.).

## What information do we collect?

ScoreBoard does not collect information.
For the ScoreBoard to work properly, it prompts the user for access to a user-selected directory to save the results.

We do not use or have access to your personal data, including all data related to your files, contacts, programs and library on your device.

## Contact Us

If you have any questions or concerns about our Privacy Policy, please [contact us](https://kopsap4ik.github.io/ScoreBoard/support.html)

repository: ropelletier/ScoreBoard

repo_name: ropelletier/ScoreBoard


text:
---
title: Weeknotes 7
date: 2021-05-24T23:00:00.000Z
weeknote:
  season: "1"
  episode: "7"
social_description: Workshopping and starting to dig into maps
social_image: content/manyhats.jpeg
---
## Wearing multiple hats

This week I found myself doing something I said I wanted to avoid, which was 'wearing a different hat'.

{% imgr { imagePath: 'content/manyhats.jpeg', lazyload: true, width: '271', height: '186', alt: 'A man in a suit wearing multiple top hats' } %}

I'm trying to guard against becoming a multidisciplinary individual within an already busy multidisciplinary team because in my experience it nearly always sets an unsustainable expectation.

Don't get me wrong. I really enjoy it, who doesn't like variety and continually learning or honing different skills? But there is a reason we split design work into different disciplines.

At the start it can be exciting and fun but soon the regular context switching and multiple lines of communication will mean that you, your work and your team will suffer as you slowly but surely burn out.

In a previous role somebody described me using the words of Bilbo Baggins, you're 'Like butter spread over too much bread'.

However putting all that aside. I love being using my experience and this time it was 'needs must', so I picked up my service design hat and attempted to brush some of the rust off it.

The reason I needed to step in was due to our current service designer leaving for their next contract.

I took it as a compliment when they nominated me to pick up the design and facilitation of a remote kick-off workshop between Policy colleagues, my team and planning experts from various of Local Authorities.

Things were in flux so I couldn't do anything too specific, so what did I do?

## A "Project point of departure"

Working at pace with input from various people who all want to get something out of it can be daunting especially if you're new like I am.

So I fell back on a simple but effective format for a 'kick off' workshop, it's probably known by multiple names but the one which has stuck in my head is a 'Project point of departure' exercise.

### What is a project point of departure?

[A Project Point of Departure](https://toolbox.hyperisland.com/project-point-of-departure) (PPOD) is a way for individuals and teams to define the structure, direction and first steps of a project. The individual or team works through a set of questions and documents the answers in a shareable document.

{% imgr { imagePath: 'content/mural-pathfinder-kickoff.png', lazyload: true, width: '767', height: '523', alt: 'A screenshot of Mural board with lots of post-it notes', classes: 'content-image content-image--borders content-image--left' } %}

This can either be a document that develops with the project, or it can be left as just a clear and concise record of the starting-point.

Mine was a variation on the following standard questions:

1. What is the overall purpose of the project? (express this in one sentence)
2. What specific outcomes should be achieved by the end of the project? (aim for 2-4 bullets)
3. Who are you doing the project for? And what value does it provide to those people? (aim for 3 bullets or less)
4. Who is involved and what are they responsible for?
5. What needs to happen by when? And how much money do you have? (broken down into bullet points, on a broad level)
6. How will the team work together, how will you communicate, divide tasks, collaborate, approach decision making, etc. (try to define about 5 guidelines with short descriptions for each)
7. What does success look like? What does failure look like? (aim for 4-5 bullet point for each one)
8. What projects are connected to this one? Are there any other documents or data sources that we need to take into account? (list the connections with hyperlinks to key documents)

In my part of the MHCLG we have access to an online collaboration tool called '[Mural](https://www.mural.co/)', I used that as the place to capture contributions from the local authorities which obviously were based all across the country.

After some hiccups with people getting access to the board. It worked quite well. I used the lovely timer facility to keep things on track which helped a lot when trying balance between giving people thinking time as well as contribution time.

## Other happenings

* I have been asked to sit on a panel reviewing applications for a design brief our team has put on the [Government digital market place](https://www.digitalmarketplace.service.gov.uk/).
* I have started thinking about how I might influence the design and build of digital maps.

gha_language: SCSS


text:
# rancher

Installs [Rancher](http://rancher.com/rancher-io/)

# attributes

documentation for attributes is inline at: [attributes/default.rb](attributes/default.rb)

# recipes

## server.rb

Installs Rancher labs server.

Run this on your `primary` node.

## agent.rb

Installs Rancher labs agent.

Run this on all nodes.

# Usage

## Kitchen

This is mostly used for cookbook development and testing:

```
$ kitchen converge ubuntu
$ curl localhost:8080
```

## Chef-Provisioning

Note: If you have chefdk 0.6.0 you may need to install a newer `chef-provisioning` gem to overcome the bug described [here](https://github.com/chef/chef-provisioning/pull/337).  To do this run `chef gem install chef-provisioning`.

### Vagrant

This will create a two node rancher cluster utilizing the vagrant module for chef provisioning:

```
$ rake berks
$ rake server agent
$ curl localhost:8080
```

to access the VMs you need to be in the `provision/vms` directory:

```
$ cd provision/vms
$ vagrant ssh agent
vagrant@agent:~$ sudo docker ps
CONTAINER ID        IMAGE                  COMMAND             CREATED             STATUS              PORTS               NAMES
40ce0296df02        rancher/agent:v0.7.9   "/run.sh run"       4 minutes ago       Up 4 minutes                            rancher-agent

```

to cleanup the VMs run:

```
$ rake destroy_machines
```

# Development

## Publish Cookbook to Supermarket

```
$ rake publish
```

repository: dullyouth/cookbook-rancher

repo_name: dullyouth/cookbook-rancher


text:
# City of Boston Test Infrastructure Demo

This project was started using [Gatsby Docs Starter](https://github.com/ericwindmill/gatsby-starter-docs)

The Docker configuration was borrowed from [Gatsby-Docker](https://github.com/gatsbyjs/gatsby-docker) and extended for educational purposes.

## bin directory

To run these commands from the project directory, prefix them with `./bin/` like `./bin/foo` to run the `foo` command.

**`docker-build`**

This builds the Gatsby site and then builds the Docker container.

It includes three build arguments in the command: `RELEASE`, `APP_NAME`, `BRANCH_NAME`, and `INSTANCE_NAME`. These are included in the container's environment as variables and also written to a text file named `RELEASE.txt`.

By default `RELEASE` is the git hash, `APP_NAME` is name of the current directory, `BRANCH_NAME` and `INSTANCE_NAME` are both the name of the current branch.

**`docker-run`**

This runs the Docker container.

You can also run commands within the Docker container by adding them at the end of the command. For example `./bin/docker-run bash` will launch the container and put you into the bash shell. You can use `ctrl-d` to exit.

repository: CityOfBoston/testdemo

repo_name: CityOfBoston/testdemo


text:
# GRStatusBar

Safari-like status bar for OS X apps.

This adds a `statusBar` property to NSWindow, the easiest way to show a status message is by just setting the `text` property on your window's `statusBar`.

[See demo app to learn more](https://github.com/insidegui/GRStatusBarDemo).

[Read the docs](http://cocoadocs.org/docsets/GRStatusBar).

![screenshot](https://raw.githubusercontent.com/insidegui/GRStatusBar/master/screenshot.png)

## Installing

### IMPORTANT: If you're using Swift 3

Checkout the branch `swift3` if you're using Xcode 8 and Swift 3 on your project, this branch will be merged into master when Xcode 8 comes out of beta.

### Using CocoaPods:

Add a `Podfile` that contains at least the following information to the root of your app project, then do `pod install`.
If you're unfamiliar with CocoaPods, read [using CocoaPods](http://guides.cocoapods.org/using/using-cocoapods.html).

```ruby
platform :osx, '10.11'      # only 10.11 and up supported
pod 'GRStatusBar'
use_frameworks!
```

### Manually

Just download the code and add `GRStatusBar.framework` to the `Embedded Binaries` section of your app's target config.

## ❤

[![Flattr this git repo](http://api.flattr.com/button/flattr-badge-large.png)](https://flattr.com/submit/auto?user_id=insidegui&url=https://github.com/insidegui/GRStatusBar.git)

repository: antmd/GRStatusBar

repo_name: antmd/GRStatusBar


text:
### [CVE-2007-4498](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2007-4498)
![](https://img.shields.io/static/v1?label=Product&message=n%2Fa&color=blue)
![](https://img.shields.io/static/v1?label=Version&message=n%2Fa&color=blue)
![](https://img.shields.io/static/v1?label=Vulnerability&message=n%2Fa&color=brighgreen)

### Description

The Grandstream SIP Phone GXV-3000 with firmware <IP_ADDRESS>, Loader <IP_ADDRESS>, and Boot <IP_ADDRESS> allows remote attackers to force silent call completion, eavesdrop on the phone's local environment, and cause a denial of service (blocked call reception) via a certain SIP INVITE message followed by a certain "SIP/2.0 183 Session Progress" message.

### POC

#### Reference
- http://securityreason.com/securityalert/3059

#### Github
No PoCs found on GitHub currently.

repository: trickest/cve

repo_name: trickest/cve


text:
# Notes and misc
Adopting a "write deletable" code means code will start to seem less precious.
Might also guard against over-engineering, or the urge to write a
"once-and-for-all" solution.
It acknowledges that information and/or business demands change often, and
sometimes unpredictably.
Like with composition over inheritance, the benefits of coding this way maybe
affect 20% of the code but 80% of

Making code easy to read & reason about should be a primary goal when writing
code. Removing unnecessary/old code reduces the cognitive load a developer has
to digest when encountering a new (or forgotten) body of code.

## further reading:
    * [Life-changing magic of deleting code](https://medium.engineering/the-life-changing-magic-of-deleting-code-be5cb3ffb391)
    Take time after a release or completing a project to remove unused code and
    clean up. Use the `// TODO: ` comment and tag someone's name there. Remove
    it after the release.
    * [Deleting Code](https://nedbatchelder.com/text/deleting-code.html)
    Dos and Don'ts on deleting code.
    dont commit commented out code; or at least leave a comment as to WHY the code is commented out (instead of being removed)
    leave a comment block in place of where a chunk of code was removed.
    Explain in a comment why it was removed and include a place to find it.
    * [Write code that is easy to delete, not to
    extend](http://programmingisterrible.com/post/139222674273/write-code-that-is-easy-to-delete-not-easy-to)
    Koan-like, several ideas that are good springboards toward fleshing out
    and applying to one's own daily work.
    * [On the Criteria to be used for Decomposing Systems into
    Modules](https://www.cs.umd.edu/class/spring2003/cmsc838p/Design/criteria.pdf)
    OG blueprint for modular system. Instead of dividing program into modules along lines of
    process flow (ie Process A ==> Process B ==> Process C = Module A ==> B ==>
    C) consider how the processes share/exchange information. Focus on
    decomposing along the
    input/output of a module, hiding the internal functionality.

## Toward a definition of "deletable code"
what makes code deletable?
1. minimal interface
    * fewer arguments of a defined type (probably?)
    * clearly defined & predictable return
2. minimal dependencies/dependents?
3. modular/contained/portable (see above)

* whats the relationship between practicing code reuse and writing
  deletable code?

repository: jafow/deleteable-code-talk

repo_name: jafow/deleteable-code-talk


text:
# Spryker Demoshop
[![License](https://img.shields.io/github/license/spryker/demoshop.svg)](https://github.com/spryker/demoshop/)

In order to install Spryker Demoshop on your machine, you can follow the instructions described in the link below:

* [Installation - spryker.github.io/getting-started/installation/guide/](https://spryker.github.io/getting-started/installation/guide/)

If you encounter any issues during or after installation, you can first check our Troubleshooting article:

* [Troubleshooting - spryker.github.io/getting-started/installation/troubleshooting/](https://spryker.github.io/getting-started/installation/troubleshooting/)

repository: dantleech/demoshop

gha_language: PHP

repo_name: dantleech/demoshop


text:
# Getting started in command line: ogr2ogr

### 1. start the OsGeo4W command shell, where you can run ogr2ogr commands:
<img src= ./osgeo4wshell_where2.jpg width="300" height="80" />

### 2. . type ogr2ogr -help (hit enter); this shows you all the possible options to use with ogr2ogr.
    Usage: ogr2ogr [--help-general] [-skipfailures] [-append] [-update]
               [-select field_list] [-where restricted_where|@filename]
               [-progress] [-sql <sql statement>|@filename] [-dialect dialect]
               [-preserve_fid] [-fid FID] [-limit nb_features]
               [-spat xmin ymin xmax ymax] [-spat_srs srs_def] [-geomfield field]
               [-a_srs srs_def] [-t_srs srs_def] [-s_srs srs_def]
               [-f format_name] [-overwrite] [[-dsco NAME=VALUE] ...]
               dst_datasource_name src_datasource_name
               [-lco NAME=VALUE] [-nln name]
               [-nlt type|PROMOTE_TO_MULTI|CONVERT_TO_LINEAR|CONVERT_TO_CURVE]
               [-dim XY|XYZ|XYM|XYZM|layer_dim] [layer [layer ...]]

### 3. try running an ogr2ogr transformation: SDE feature class to KML
    ogr2ogr -a_srs epsg:3005 -f "KML" T:\aTest01.kml OCI:BCGWusername/BCGWpassword@IDWPROD1:noTable -sql "select * from WHSE_FOREST_VEGETATION.RSLT_OPENING_SVW  where OPENING_ID = '-228240000'" -progress

*Copy the above line, substitute your own BCGW login info, and paste into the OSGeo4W window and hit enter. Several warnings will pop up, but an output KML file will successfully write to the T: .*

Here's how the command above works:

`ogr2ogr` calls the executable file, E:\sw_nt\QGIS_3.8\bin\ogr2ogr.exe

`-a_srs epsg:3005` assigns an output spatial reference using epsg code 3005 (i.e. BC Albers)

`-f "KML" ` specifies the output format is "KML" (this is one of tow KML drivers in ogr)

`T:\aTest01.kml` this is the output folder and filename; if the folder does not exist, you must create it first

`OCI:BCGWusername/BCGWpassword@IDWPROD1:noTable` this specifies the source file, using the Oracle Spatial (OCI = Oracle Call Interface) driver. This must include a BCGW username and BCGW password, and reference the Oracle instance (@IDWPROD1) found in the tnsnames.ora file in E: . The *noTable* is a placeholder table name that prevents the OCI driver from searching through the entire BCGW for the table you want (you specify that in the SQL statement).

`-sql` this indictaes the results are to be filtered via an Oracle SQL statement

`"select * from WHSE_FOREST_VEGETATION.RSLT_OPENING_SVW  where OPENING_ID = '-228240000'"` this is an SQL statement that fetches a single row in the RESULTS Openings dataset

`-progress` This shows a progress indicator as below:

    0...10...20...30...40...50...60...70...80...90...100 - done.

### 4. below are several examples (using the same RESULTS Openings dataset and query) of transforming data from the BCGW. You'll notice that different transformations produce differnt warnings. It may seem like all the warnings are a problem, but most only affect things likw field name length or layer names.

#### Create a Shapefile (readable in ESRI and QGIS):
    ogr2ogr -a_srs epsg:3005 -f "ESRI Shapefile" T:\aShapefile01.shp OCI:BCGWusername/BCGWpassword@IDWPROD1:noTable -sql "select * from WHSE_FOREST_VEGETATION.RSLT_OPENING_SVW  where OPENING_ID = '-228240000'" -progress

#### Create a GeoPackage (readable in ESRI and QGIS):
    ogr2ogr -a_srs epsg:3005 -f "GPKG" T:\aGeoPackage.gpkg OCI:BCGWusername/BCGWpassword@IDWPROD1:noTable -sql "select * from WHSE_FOREST_VEGETATION.RSLT_OPENING_SVW  where OPENING_ID = '-228240000'" -progress

#### Create a GeoJSON (readable in QGIS):
<<<<<<< HEAD
    ogr2ogr -a_srs epsg:3005 -f "GeoJSON" T:\aJSON.json OCI:BCGWusername/BCGWpassword@IDWPROD1:noTable -sql "select * from WHSE_FOREST_VEGETATION.RSLT_OPENING_SVW  where OPENING_ID = '-228240000'" -progress
=======
    ogr2ogr -a_srs epsg:3005 -f "GeoJSON" T:\aJSON.json OCI:BCGWusername/BCGWpassword@IDWPROD1:noTable -sql "select * from WHSE_FOREST_VEGETATION.RSLT_OPENING_SVW  where OPENING_ID = '-228240000'" -progress
>>>>>>> 914a8a9b9673b25c5a9409bff5fa61639b9103da

repository: bcgov/gis-pantry

gha_language: Jupyter Notebook

repo_name: bcgov/gis-pantry


text:
---
title:  Needs More?
---

## 1. Get from capability registry

You can get more from official capability registry by using KubeVela [plugin](../../kubectlplugin)。

### List

By default, the commands will list capabilities from [repo](https://registry.kubevela.net) maintained by KubeVela.

```shell
$ kubectl vela trait --discover
Showing traits from registry: https://registry.kubevela.net
NAME           	REGISTRY	  DEFINITION                    		APPLIES-TO
service-account	default  	                              		    [webservice worker]
env            	default 		                                    [webservice worker]
flagger-rollout	default       canaries.flagger.app          		[webservice]
init-container 	default 		                                    [webservice worker]
keda-scaler    	default       scaledobjects.keda.sh         		[deployments.apps]
metrics        	default       metricstraits.standard.oam.dev		[webservice backend task]
node-affinity  	default		                              		    [webservice worker]
route          	default       routes.standard.oam.dev       		[webservice]
virtualgroup   	default		                              		    [webservice worker]
```

Note that the `--discover` flag means show all traits not in your cluster.

### Install

Then you can install a trait like:

```shell
$ kubectl vela trait get init-container
Installing component capability init-container
Successfully install trait: init-container
```

## 2. Designed by yourself

* Read [how to edit definitions](../../platform-engineers/cue/definition-edit) to build your own capability from existing ones.
* [Build your own capability from scratch](../../platform-engineers/cue/advanced)
  and learn more features about how to [define custom traits](../../platform-engineers/traits/customize-trait).


text:
# Speaker Profiling

This Repository contains the code for estimating the Age, Height and Gender of a speaker with their speech signal. The repository experiments with both TIMIT and NISP Dataset.

![model architeture](assets/wav2vecframework.PNG)

[DEMO Colab Notebook](https://colab.research.google.com/drive/1WDBtlhg87BiPlg-IrIiFxyY5eaOVqkob?usp=sharing)

## Installation

Use the package manager [pip](https://pip.pypa.io/en/stable/) to install the required packages for preparing the dataset, training and testing the model.

```bash
pip install -r requirements.txt
```

## Usage

### Download the dataset
```bash
# Timit Dataset
wget https://data.deepai.org/timit.zip
unzip timit.zip -d 'path to timit data folder'

# NISP Dataset
git clone https://github.com/iiscleap/NISP-Dataset.git
```

### Prepare the dataset for training and testing
```bash
# TIMIT Dataset
python TIMIT/prepare_timit_data.py --path='path to timit data folder'

# NISP Dataset
python NISP/prepare_nisp_data.pt --nisp_repo_path='path to nisp data repo folder'
```

### Update Config and Logger
Update the config.py file to update the batch_size, gpus, lr, etc and change the preferred logger in train_.py files

### Training(Dev Model, to make sure everything is set as expected for training)
```bash
# TIMIT Dataset
python train_timit.py --dev=True --data_path='path to final data folder'

# NISP Dataset
python train_nisp.py --dev=True --data_path='path to final data folder'
```

### Training(also check for other arguments in the train_....py file)
```bash
# TIMIT Dataset
python train_timit.py --data_path='path to final data folder'

# NISP Dataset
python train_nisp.py --data_path='path to final data folder'
```

### Test the Model
```bash
# TIMIT Dataset
python test_timit.py --data_path='path to final data folder' --model_checkpoint='path to saved model checkpoint'

# NISP Dataset
python test_nisp.py --data_path='path to final data folder' --model_checkpoint='path to saved model checkpoint'
```

## Results

[Wandb Runs](https://wandb.ai/shangeth/SpeakerProfiling?workspace=user-shangeth)
### TIMIT Baseline
| Model                                  	| Height RMSE 	|        	| Height MAE 	|        	| Age RMSE 	|        	| Age MAE 	|        	| Gender Acc 	|
|----------------------------------------	|-------------	|--------	|------------	|--------	|----------	|--------	|---------	|--------	|------------	|
|                                        	| Male        	| Female 	| Male       	| Female 	| Male     	| Female 	| Male    	| Female 	|            	|
| MFCC_LSTM-Attn                         	| 7.5         	| 6.6    	| 5.5        	| 5.2    	| 7.7      	| 8.4    	| 5.6     	| 5.9    	| 0.975      	|
| MelSpec_LSTM-Attn                      	| 7.7         	| 8.1    	| 5.8        	| 6.5    	| 7.7      	| 8.7    	| 5.5     	| 6.1    	| 0.669      	|
| MFCC_CNN-LSTM-Attn                     	| 7.5         	| 6.8    	| 5.7        	| 5.3    	| 8.2      	| 8.7    	| 5.4     	| 6.1    	| 0.989      	|
| MelSpec_CNN-LSTM-Attn                  	| 7.5         	| 7.4    	| 5.8        	| 5.8    	| 8.2      	| 8.4    	| 5.8     	| 5.9    	| 0.96       	|
| wav2vec(no-finetune)-LSTM-Attn         	| 7.4         	| 6.4    	| 5.5        	| 5.1    	| 7.2      	| 8.2    	| 5.0     	| 5.7    	| 0.994      	|
| wav2vec(finetune 56)-LSTM-Attn         	| 7.5         	| 6.2    	| 5.5        	| 4.9    	| 7.5      	| 7.9    	| 5.5     	| 5.7    	| 0.994      	|
| wav2vec(finetune 6)-LSTM-Attn          	| 7.6         	| 6.7    	| 5.6        	| 5.3    	| 7.0      	| 8.2    	| 4.9     	| 5.6    	| 0.993      	|
| wav2vec(finetune 56)-LSTM-Attn(Only H) 	| 7.4         	| 6.2    	| 5.6        	| 4.9    	|          	|        	|         	|        	|            	|
| multi-scale-cnn(Only H) 	| 7.5         	| 6.1    	| 5.9        	| 4.7    	|          	|        	|         	|        	|            	|

### TIMIT Previous Results
|        Model        	|  Height 	|  RMSE  	|  Height 	|   MAE  	|  Age 	|  RMSE  	|  Age 	|   MAE  	| Gender Acc 	|
|:-------------------:	|:-------:	|:------:	|:-------:	|:------:	|:----:	|:------:	|:----:	|:------:	|:----------:	|
|                     	|   Male  	| Female 	|   Male  	| Female 	| Male 	| Female 	| Male 	| Female 	|            	|
|       [1] 2019      	|   6.85  	|  6.29  	|    -    	|    -   	|  7.6 	|  8.63  	|   -  	|    -   	|            	|
|  [2] 2016 (fusion)  	|   **6.7**   	|   6.1  	|  **5.0**   	|   5.0  	|  7.8 	|   8.9  	|  5.5 	|   6.5  	|            	|
| [2] 2016 (baseline) 	|   7.0   	|   6.5  	|   5.3   	|   5.2  	|  8.1 	|   9.1  	|  5.7 	|   6.2  	|            	|
|       [3] 2020      	|    -    	|    -   	|    -    	|    -   	| 7.24 	|  8.12  	| 5.12 	|  **5.29**  	|    **0.996**   	|
|       [4] 2009      	|   6.8   	|   6.3  	|   5.3   	|   5.1  	|   -  	|    -   	|   -  	|    -   	|            	|

### NISP
|    Model   	| Height 	|  RMSE  	| Height 	|   MAE  	|  Age 	|  RMSE  	|  Age 	|   MAE  	| Gender Acc 	|
|:----------:	|:------:	|:------:	|:------:	|:------:	|:----:	|:------:	|:----:	|:------:	|:----------:	|
|            	|  Male  	| Female 	|  Male  	| Female 	| Male 	| Female 	| Male 	| Female 	|            	|
|   [5] TMP  	| 6.17   	| 6.93   	| 5.22   	| 5.30   	| 5.60 	| 5.57   	| 4.40 	| 4.42   	|            	|
| [5] Comb-3 	| 6.13   	| 6.70   	| 5.16   	| 5.30   	| 5.63 	| 4.99   	| 3.80 	| 3.76   	|            	|
| Our Method 	| 6.49   	| 6.37   	| 5.32   	| 5.12   	| 5.48 	| 5.71   	| 3.70 	| 4.22   	| 0.984      	|

## Contributing
Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

Please make sure to update tests as appropriate.

## License
[MIT](https://choosealicense.com/licenses/mit/)

## Reference
- [1] S. B. Kalluri, D. Vijayasenan and S. Ganapathy, "A Deep Neural Network Based End to End Model for Joint Height and Age Estimation from Short Duration Speech," ICASSP 2019 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, United Kingdom, 2019, pp. 6580-6584, doi: 10.1109/ICASSP.2019.8683397.
- [ 2 ]  Rita Singh, Bhiksha Raj, and James Baker, “Short-term analysis for estimating physical parameters of speakers,” in Proc. of IWBF. IEEE, 2016, pp. 1–6
- [ 3 ] Joint gender and age estimation based on speech signals using x-vectors and transfer learning ICASSP 2021.
- [ 4 ] Mporas, I., Ganchev, T. Estimation of unknown speaker’s height from speech. Int J Speech Technol 12, 149–160 (2009). https://doi.org/10.1007/s10772-010-9064-2


text:
[![CI Status](https://github.com/AY2122S1-CS2103-F09-1/tp/workflows/Java%20CI/badge.svg)](https://github.com/AY2122S1-CS2103-F09-1/tp/actions)

![Ui](docs/images/Ui.png)

* This is **a project for NUS School of Computing students** from CS2103 module (AY2122S1-CS2103-F09-1).

* This desktop application is used for **NUS Teaching Assistant** to keep track the modules they teach and the status of students enrolled in the specific modules.
* It is named `Teaching Assistant’s Buddy (TAB)` (`TAB` for short).
* For the detailed documentation of this project, see the **[Teaching Assistant’s Buddy Product Website](https://ay2122s1-cs2103-f09-1.github.io/tp/)**.

## Features

**:information_source: Notes about the command format:**

* Lowercase letter(s) followed by a slash are prefixes that prefix a parameter, to be supplied by the user.

  e.g. in `add module m/<module name>`, `m/` is a prefix that prefixes the `<module name>` parameter.

* Words in `<angle brackets>` are the parameters prefixed by prefixes, to be supplied by the user.

  e.g. in `add module m/<module name>`, `<module name>` is a parameter which can be used as `add module m/CS2105`.

* Parameters must be prefixed with their specific prefixes.

  e.g. the command `add module m/CS2105` will add the module CS2105 to TAB, where the commands `add module i/CS2105` and `add module CS2105` will not work.

* Parameters must only contain permitted characters, as listed in the [Parameter summary](#parameter-summary-with-prefix-included) below.

* Optional parameters are denoted in this user guide using square brackets '[]', at least one optional parameter must be present when executing a command.

  e.g. in `edit task m/<module name> ti/<task id> [a/<edited task name>] [d/<edited task deadline>]`, `a/<edited task name>` and `d/<edited task deadline>` are the optional parameters

* If duplicate parameters are provided, only the latest parameters will be processed.

* Students in different modules with the same student id are treated as different students.

* Tasks can be added to modules without students, but will only become visible after students have been added to the module.

### Add a new module: `add module`

Add a new module to TAB.

Module name must be an NUS module code, which consists of 2-4 capital letters, followed by 4 numbers, followed by 0-4 capital letters.

Format: `add module m/<module name>`

Examples:
* `add module m/CS2103`:
  * Add module CS2103 to TAB
* `add module m/CS2100`:
  * Add module CS2100 to TAB

![AddModule](images/AddModule.png)

### Edit a module's name: `edit module`

Edit the name of the module.

Module name must be an NUS module code, which consists of 2-4 capital letters, followed by 4 numbers, followed by 0-4 capital letters.
The new module name must not be identical to the old module name.

Format: `edit module m/<old module name> mn/<new module name>`

Examples:
* `edit module m/CS2103 mn/CS2105`:
  * Edit the module name of module "CS2103" to "CS2105"

![EditModule](images/EditModule.png)

### Delete a module from TAB: `delete module`

Delete the specified module from TAB.
Module name must be an NUS module code, which consists of 2-4 capital letters, followed by 4 numbers, followed by 0-4 capital letters.

Format: `delete module m/<module name>`

Examples:
* `delete module m/CS2103`:
  * Delete module CS2103 from TAB
* `delete module m/CS2100`:
  * Delete module CS2100 from TAB

![DeleteModule](images/DeleteModule.png)

### Add a student to a specific module: `add student`

Add a new student to the specified module by providing the module name, student id, student name, student tele handle, and the student email.

Students in different modules with the same student id are treated as different students.

Also automatically copies over the task list from the module to the student.

Student id must be a valid NUS student id, which starts with the capital letter 'A', followed by 7 numbers and ends with a capital letter.

Student name should only contain alphanumeric characters and spaces, and it should not be blank.

Student tele handle must start with @, and followed by 5-32 characters containing only alphanumeric characters and the underscore '_'.

Student email must follow the format specified in the [Parameter summary](#parameter-summary-with-prefix-included) section below.

Student id, student tele handle, and student email must be unique for each student in the same module.

Duplicate student names are allowed to cater for students who may have the same name.

Format: `add student m/<module name> i/<student id> n/<student name> t/<student tele handle> e/<student email>`

Examples:
* `add student m/CS2103 i/A0123456A n/John Doe t/@johndoe <EMAIL_ADDRESS>  * Add student with id A0123456A to module CS2103
* `add student m/CS2100 i/A0123457A n/Amy t/@amylee <EMAIL_ADDRESS>  * Add student with id A0123457A to module CS2100

![AddStudent](images/AddStudent.png)

### Edit a student's information: `edit student`

Edit a student's information.

Students in different modules with the same student id are treated as different students.

The identity parameter (student id) must be provided.

At least one optional parameter (name/tele handle/email) must be provided, and only provided parameters will be changed.

If the optional parameters contain the same information as is already present, the edit command will still be executed successfully.

Student name should only contain alphanumeric characters and spaces, and it should not be blank.

Student tele handle must start with @, and followed by 5-32 characters containing only alphanumeric characters and the underscore '_'.

Student email must follow the format specified in the [Parameter summary](#parameter-summary-with-prefix-included) section below.

Format: `edit student m/<module name> i/<student id> [n/<edited student name>] [t/<edited student tele handle>] [e/<edited student email>]`

Examples:
* `edit student m/CS2103 i/A0123456A n/John Doe`:
  * Edit the name of the student with id A0123456A in the module CS2103 to John Doe
* `edit student m/CS2100 i/A0123457A t/@amylee <EMAIL_ADDRESS>  * Edit the tele handle and email of the student with id A0123457A in the module CS2100 to @amylee and<EMAIL_ADDRESS>respectively

![EditStudent](images/EditStudent.png)

### Delete a student from a specific module: `delete student`

Delete the specified student from the specified module by providing the module name and the student id.

Format: `delete student m/<module name> i/<student id>`

Examples:
* `delete student m/CS2103 i/A1234567A`:
  * Delete student with id A1234567A from module CS2103’s list of students
* `delete student m/CS2100 i/A7654321A`:
  * Delete student with id A7654321A from module CS2100’s list of students

![DeleteStudent](images/DeleteStudent.png)

### Add a new task to a module: `add task`

Add a new task to the specified module by providing the module name, task id, task name, and the task deadline.

The new task will be added to all students in the module, as well as for future students who are not yet added.
Tasks can be added to modules without students, but will only become visible after student(s) have been added to the module.

Task id must begin with a capital letter 'T', followed by a positive integer.

Task name must only contain alphanumeric characters and spaces, and it must not be blank.

Task deadline can contain either the date and time, or only the date, or only the time.

Date must be in the form "YYYY-MM-DD".

Time must be in the form "hh:mm" in 24-hrs format, and will appear in TAB in 12-hrs format.

Format: `add task m/<module name> ti/<task id> a/<task name> d/<task deadline>`

Examples:
* `add task m/CS2103 ti/T1 a/assignment1 d/2021-10-12`:
  * Add the task assignment1 with deadline 'Tue, Oct 12 2021' to module CS2103
* `add task m/CS2100 ti/T1 a/lab1 d/2021-10-13 16:00 `:
  * Add the task lab1 with deadline 'Wed, Oct 13 2021 04:00 pm' to module CS2100

![AddTask](images/AddTask.png)

### Edit a task's information: `edit task`

Edit a task's information.

The task's information will be edited for all students in the module, as well as for future students who are not yet added.
The identity parameters (module name/task id) must be provided.

At least one optional parameter (name/deadline) must be provided, and only provided parameters will be changed.

If the optional parameters contain the same information as is already present, the edit command will still be executed successfully.

Task name must only contain alphanumeric characters and spaces, and it must not be blank.

Task deadline can contain either the date and time, or only the date, or only the time.

Date must be in the form "YYYY-MM-DD".

Time must be in the form "hh:mm" in 24-hrs format, and will appear in TAB in 12-hrs format.

Format: `edit task m/<module name> ti/<task id> [a/<edited task name>] [d/<edited task deadline>]`

Examples:
* `edit task m/CS2103 ti/T10 a/Final exam d/2021-11-23`:
  * Edit the name and deadline of the task with task id 'T10' to 'Final exam' and 'Tue, Nov 23 2021' respectively
* `edit task m/CS2103 ti/T10 d/2021-11-23 17:00`:
  * Edit only the deadline of the task with task id 'T10' to 'Tue, Nov 23 2021 5:00 pm'

![EditTask](images/EditTask.png)

### Delete a task from a module: `delete task`

Delete the specified task from the specified module by providing the module name and the task id.

Also delete the task from all the students in that module.

Format: `delete task m/<module name> ti/<task id>`

Examples:
* `delete task m/CS2103 ti/T1`:
  * Delete the task with task id T1 from CS2103
* `delete task m/CS2105 ti/T2`:
  * Delete the task with task id T2 from CS2105

![DeleteTask](images/DeleteTask.png)

### Mark a task as done: `mark done`

Mark the specified task from the specified module under the specified student as done.

This means that this student has completed this task of this module.

Format: `mark done m/<module name> i/<student id> ti/<task id>`

Examples:
* `mark done m/CS2103 i/A1234567A ti/T1`:
  * Mark the task with id T1 under student with id A1234567A of module CS2103 as done.
  * This means that student A1234567A has completed task T1 of module CS2103.

![MarkDone](images/MarkDone.png)

### Mark a task as undone: `mark undone`

Mark the specified task from the specified module under the specified student as undone.

This means that this student has not completed this task of this module, possibly to correct a mistake by the TA.

Format: `mark undone m/<module name> i/<student id> ti/<task id>`

Examples:
* `mark undone m/CS2103 i/A1234567A ti/T1`:
  * Mark the task with id T1 under student with id A1234567A of module CS2103 as undone.
  * Student A1234567A has not completed task T1 of module CS2103 yet.
  * But for some reason (e.g. human error) the task is marked as done.

![MarkUndone](images/MarkUndone.png)

### Find a student: `find`

Find the student in the module specified along with the student's information.

Format: `find m/<module name> i/<student id>`

Examples:
* `find m/CS2103 i/A0123456A`:
  * Find and show student with id A0123456A from module CS2103

![Find](images/Find.png)

### Go to homepage: `home`

Return to the homepage which shows the complete list of modules and students along with their tasks.

Format: `home`

![Home](images/Home.png)

### Clear all information: `clear`

Clear all information stored in the TAB application.

Format: `clear`

:warning: **Caution:** Clearing is irreversible, please use with caution.

![Clear](images/Clear.png)

### Exit the application: `exit`

Exit the TAB application.

Format: `exit`

### Saving the data

TAB data are saved in the hard disk automatically after any command that changes the data. There is no need to save manually.

### Editing the data file (JSON)

TAB data are saved as a JSON file `[JAR file location]/data/assistantbuddy.json`. Advanced users are welcome to update data directly by editing that data file.

:warning: **Caution:**
If your changes to the data file makes its format invalid, TAB will discard all data and start with an empty data file at the next run.

## FAQ

**Q**: How do I assign tasks to students?

**A**: Adding the student to the module will automatically add the tasks in the module to the student.

**Q**: How do I see the full TAB after executing the `find` command?

**A**: Use the `home` command to return to the homepage showing the full TAB.

--------------------------------------------------------------------------------------------------------------------

## Glossary

**Alphanumeric**: Any characters from [0-9], [a-z], or [A-Z].

**TAB**: Teaching Assistant's Buddy.

**Tele handle**: A telegram user handle that must start with @, and followed by 5-32 characters containing only alphanumeric characters and the underscore '_'.

--------------------------------------------------------------------------------------------------------------------
## Command summary

**Optional parameters** are surrounded by square brackets '[]' in Format, and may be omitted from Examples to show they are optional.

Action | Format, Examples
--------|------------------
**Add Module** | `add module m/<module name>`
 e.g., `add module m/CS2103`
**Add Student** | `add student m/<module name> i/<student id> n/<student name> t/<student tele handle> e/<student email>`
 e.g., `add student m/CS2103 i/A0123456A n/John Doe t/@johndoe <EMAIL_ADDRESS>**Add Task** | `add task m/<module name> ti/<task id> a/<task name> d/<task deadline>` <br> e.g., `add task m/CS2103 ti/T1 a/assignment1 d/2021-10-12 12:00`
**Clear** | `clear`
 e.g., `clear`
**Delete Module** | `delete module m/<module name>`
 e.g., `delete module m/CS2103`
**Delete Student** | `delete student m/<module name> i/<student id>`
 e.g., `delete student m/CS2103 i/A1234567A`
**Delete Task** | `delete task m/<module name> ti/<task id>`
 e.g., `delete task m/CS2103 ti/T1`
**Edit Module** | `edit module m/<old module name> mn/<new module name>`
 e.g., `edit module m/CS2103 mn/CS2105`
**Edit Student** | `edit student m/<module name> i/<student id> [n/<edited student name>] [t/<edited student tele handle>] [e/<edited student email>]`
 e.g., `edit student m/CS2103 i/A0123457A t/@amylee <EMAIL_ADDRESS>**Edit Task** | `edit task m/<module name> ti/<task id> [a/<edited task name>] [d/<edited task deadline>]` <br> e.g., `edit task m/CS2103 ti/T10 a/Final exam d/2021-11-23`
**Exit** | `exit`
 e.g., `exit`
**Find** | `find m/<module name> i/<student id>`
 e.g., `find m/CS2103 i/A0123456A`
**Home** | `home`
 e.g., `home`
**Mark Done** | `mark done m/<module name> i/<student id> ti/<task id>`
 e.g., `mark done m/CS2103 i/A1234567A ti/T1`
**Mark Undone** | `mark undone m/<module name> i/<student id> ti/<task id>`
 e.g., `mark undone m/CS2103 i/A1234567A ti/T1`

--------------------------------------------------------------------------------------------------------------------

## Parameter summary with prefix included

Parameter with prefix | Permitted characters
`m/<module name>`
 `m/<old module name>`
 `mn/<new module name>` | Module name must be an NUS module code, which consists of 2-4 capital letters, followed by 4 numbers, followed by 0-4 capital letters.
`i/<student id>` | Student id must be a valid NUS student id, which starts with the capital letter 'A', followed by 7 numbers and ends with a capital letter.
`n/<student name>`
 `n/<edited student name>` | Student name should only contain alphanumeric characters and spaces, and it should not be blank.
`t/<student tele handle>`
 `t/<edited student tele handle>` | Student tele handle must start with @, and followed by 5-32 characters containing only alphanumeric characters and the underscore '_'.
`e/<student email>`
 `e/<student email>` | Student email must follow the format specified in the [Email format](#email-format)) section below.
`ti/<task id>` | Task id must begin with a capital letter 'T', followed by a positive integer.
`a/<task name>`
 `a/<edited task name>` | Task name must only contain alphanumeric characters and spaces, and it must not be blank.
`d/<task deadline>`
 `d/<edited task deadline>` | Task deadline can contain either the date and time, or only the date, or only the time.
 Date must be in the form "YYYY-MM-DD".
 Time must be in the form "hh:mm" in 24-hrs format, and will appear in TAB in 12-hrs format.

### Email format
Emails should be of the format local-part@domain and adhere to the following constraints:

1. The local-part should only contain alphanumeric characters and these special characters, excluding the parentheses, ( +_.- ).
2. The local-part may not start or end with any special characters.
3. This is followed by a '@' and then a domain name.
4. The domain name is made up of domain labels separated by periods.
5. The domain name must:

   5.1. End with a domain label at least 2 characters long.

   5.2. Have each domain label start and end with alphanumeric characters.

   5.3. Have each domain label consist of alphanumeric characters, separated only by hyphens, if any.

gha_license_id: NOASSERTION


text:
---
layout: post
comments: true
title: Serverless Meeting minutes generator on GCP with Vertex AI and Cloud Functions
excerpt: Learn how to create a Serverless meeting minutes generator on GCP with Vertex AI (Chirp and PaLM) Cloud Functions and PubSub
tags: [ai,gcp,genai]
toc: true
img_excerpt:
---

![GCP Serverless Meeting minutes generator architecture]({{ "/assets/2023/08/20230807-serverless-meeting-minutes-architecture-gcp.svg" | absolute_url }})

GCP is a powerful platform for building all sort of applications. It hosts a variety of services which are scalable, reliable, cost-effective, easy to use, and can be easily integrated together.

In a previous article, we saw [how to leverage GCP's Vertex AI to develop an automated meeting minutes generator]({{ "/2023/08/04/meeting_minutes_gcp/" }}). In this article, we will re-architecture that application to make it more scalable and capable of processing audio recordings asynchronously. We will use Cloud Storage to host the recordings, and use Cloud Functions and PubSub to trigger the processing as new recordings are uploaded. For the speach to text and summary generation, we will use Chirp and PaLM from Vertex AI.

## Infrastructure setup

The above diagram illustrates a high level architecture for our serverless meeting minutes generator application.

Audio recording files are uploaded to a Cloud Storage bucket. Every time, a new recording is uploaded an new entry is appended to PubSub queue with information about the file. This triggers a Cloud Function that will process the recoding, generate the meeting minutes and then save them to a Cloud Storage bucket.

When the generation of minutes fail within the Cloud Function, the original Cloud Storage event will be forwarded to a [Dead-Letter Queue (DLQ)](https://cloud.google.com/pubsub/docs/handling-failures), and then sent back to the main queue for reprocessing.

We will use Cloud Deployment Manager to provision all the needed services. Let's configure the different resources as follows:
```yaml
resources:

# PubSub queue for notifications when audio files are uploaded
- name: recordings-upload-topic
  type: pubsub.v1.topic
  properties:
    name: recordings-upload-topic

# Cloud Storage bucket where audio files will be uploaded
- name: recordings-bucket
  type: storage.v1.bucket
  properties:
    name: recordings-bucket
    location: us-central1
    storageClass: STANDARD
    notificationConfig:
      topic: recordings-upload-topic
      eventTypes:
        - OBJECT_FINALIZE
```

Save the content of the previous snippet into a `resources.yaml` file then use `gcloud` to provision them as follows:

```shell
gcloud deployment-manager deployments create logs-deployment --config resources.yaml
```

> Note: For more information on configuring Pub/Sub notifications to send information about changes to objects in a bucket, check the official documentation on [pubsub notifications](https://cloud.google.com/storage/docs/pubsub-notifications).

## Minutes generation
For generating the meeting minutes, we will use the same code from previous article on [how to leverage GCP's Vertex AI to develop an automated meeting minutes generator]({{ "/2023/08/04/meeting_minutes_gcp/" }}) with one small change as illustrated by the following `diff` patch:

```diff
    request = cloud_speech.RecognizeRequest(
        recognizer=f"projects/{project_id}/locations/us-central1/recognizers/_",
        config=chirp_config,
-       content=audio_bytes,
+       uri=gcs_uri,
    )
```

Instead of passing the audio in the body of the request sent to Chirp, we will pass a URI to where the audio file is stored in a Cloud Storage bucket. For examples on how to Chrip API with GS refer to this [snippet](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/main/speech/snippets/transcribe_gcs_v2.py).

With this change, our `transcribe_audio` function becomes:

```python
def transcribe_audio(gcs_uri):
    """Transcribes audio from a Google Cloud Storage URI"""
    request = cloud_speech.RecognizeRequest(
        recognizer=f"projects/{project_id}/locations/us-central1/recognizers/_",
        config=chirp_config,
        uri=gcs_uri,
    )
    response = chirp_client.recognize(request=request)
    transcript = None
    for result in response.results:
        transcript = result.alternatives[0].transcript

    return transcript
```

## Deploying to Cloud Function
After creating the logic to generate meeting minutes from recordings, we can now expose this functionality in a Cloud Function.

In a `main.py` file, add the following snippet of a [Cloud Function V2](https://codelabs.developers.google.com/codelabs/cloud-starting-cloudfunctions-v2) that accepts a [CloudEvent](https://github.com/cloudevents/sdk-python) as input. It also register the `cloudevent_handler` method with the Functions Framework so that it will be invoked with proper input.

Upon receiving the event, the `cloudevent_handler` method will be called to extract the path to the audio recordings file and uses `meeting_minutes` and `transcribe_audio` to generate the meeting minutes, then upload them to Google Storage.

```python
import os
from cloudevents.http import CloudEvent
import functions_framework
from lib import meeting_minutes, transcribe_audio

PROJECT_ID = os.getenv("GOOGLE_CLOUD_PROJECT")

def upload_to_gcs(bucket_name, file_name, file_content):
    client = storage.Client()
    bucket = client.bucket(bucket_name)
    blob = bucket.blob(file_name)

def cloudevent_handler(cloud_event: CloudEvent) -> None:
    print(f"Received event with ID: {cloud_event['id']} and data {cloud_event.data}")
    # Get the bucket and file name from the event.
    data = cloud_event.data["message"]["data"]
    gs_uri = f"gs://{data["bucket"]}/{data["name"]}"
    # Generate meeting minutes
    transcript = transcribe_audio(gs_uri)
    minutes = meeting_minutes(transcript)
    # Upload
    upload_to_gcs(data["bucket"], data["name"] + ".txt", minutes)

if __name__ == "__main__":
  # Register the function with the Functions Framework.
  functions_framework.cloud_event(cloudevent_handler)
```

> Note: we could also have used the<EMAIL_ADDRESS>decorator to register our handler with the Functions Framework

In the same directory as the previous `main.py`, define a `requirements.txt` file to declare all of our dependencies:

```
cloudevents
functions_framework=3.*
google-cloud-aiplatform==1.29.0
google-cloud-speech=2.21.0
google-cloud-storage
```

Finally, we can deploy our Cloud Function using the `gcloud` CLI from the same directory containing the source code as follows

```shell
gcloud functions deploy minutes-function \
  --gen2 \
  --runtime python39 \
  --entry-point cloudevent_handler \
  --source . \
  --region $REGION \
  --trigger-topic $TOPIC
```

Alternatively, we could have also deployed the function to Cloud Run. All we needed to do is to define a Dockerfile to manually install the dependencies and package the source code. Then, build and deploy the container as with any typical Cloud Run application.

The following `Dockerfile` is an example of how we would define the Container image. For more details refer to official example on how deploying a CloudEvent Function to Cloud Run with the Functions Framework - [link](https://github.com/GoogleCloudPlatform/functions-framework-python/tree/main/examples/cloud_run_cloud_events).

```Dockerfile
FROM python:3.9-slim

ENV PYTHONUNBUFFERED TRUE

WORKDIR /app
COPY . .

RUN pip install -r requirements.txt

CMD ["functions-framework", "--target=cloudevent_handler", "--signature-type=cloudevent"]
```

> Note how on container startup we invoke the Functions Framework in a `CMD` step and specify the `cloudevent_handler` as the entry point function.

## That's all folks
In this article we saw how easy it is to use Google Cloud to build innovative and scalable applications. We used the following services from GCP:
- Cloud Storage to store audio files and
- PubSub to react to events such us when a new audio file is uploaded to Cloud Storage.
- Cloud Functions used to process meeting recording files and generating the minutes.
- Two foundation models from Vertex AI: Chirp for speech to text and PaLM for text generation.

I hope you enjoyed this article, feel free to leave a comment or reach out on twitter [@bachiirc](https://twitter.com/bachiirc).

gha_language: CSS