text:
(autodelta)=
# AutoDelta Mechanism

Inspired by [Huggingface transformers AutoClasses](https://huggingface.co/docs/transformers/v4.16.2/en/model_doc/auto#transformers.AutoModel) , we provide an AutoDelta features for the users to

1. Easily to experiment with different delta models
2. Fast deploy from configuration file, especially from the repos in [DeltaCenter](https://examplelink).

## Easily load from dict, so that subject to change the type of delta models.

```python
from opendelta import AutoDeltaConfig, AutoDeltaModel
from transformers import T5ForConditionalGeneration

backbone_model = T5ForConditionalGeneration.from_pretrained("t5-base")
```

We can load a config from a dict
```python
config_dict = {
    "delta_type":"lora",
    "modified_modules":[
        "SelfAttention.q",
        "SelfAttention.v",
        "SelfAttention.o"
    ],
    "lora_r":4}
delta_config = AutoDeltaConfig.from_dict(config_dict)
```

Then use the config to add a delta model to the backbone model
```python
delta_model = AutoDeltaModel.from_config(delta_config, backbone_model=backbone_model)

# now visualize the modified backbone_model
from bigmodelvis import Visualization
Visualizaiton(backbone_model).structure_graph()
```

````{collapse} <span style="color:rgb(141, 99, 224);font-weight:bold;font-style:italic">Click to view output</span>
```{figure} ../imgs/t5lora.png
---
width: 600px
name: t5lora
---
```
````

## Fast deploy from a finetuned delta checkpoints from DeltaCenter

```python
# use tranformers as usual.
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
t5 = AutoModelForSeq2SeqLM.from_pretrained("t5-large")
t5_tokenizer = AutoTokenizer.from_pretrained("t5-large")
# A running example
inputs_ids = t5_tokenizer.encode("Is Harry Poter wrtten by JKrowling", return_tensors="pt")
t5_tokenizer.decode(t5.generate(inputs_ids)[0])
# >>> '<pad><extra_id_0>? Is it Harry Potter?</s>'
```

Load delta model from delta center:
```python
# use existing delta models
from opendelta import AutoDeltaModel, AutoDeltaConfig
# use existing delta models from DeltaCenter
delta = AutoDeltaModel.from_finetuned("thunlp/Spelling_Correction_T5_LRAdapter_demo", backbone_model=t5)
# freeze the whole backbone model except the delta models.
delta.freeze_module()
# visualize the change
delta.log()

t5_tokenizer.decode(t5.generate(inputs_ids)[0])
# >>> <pad> Is Harry Potter written by JK Rowling?</s>
```

**Hash check**

Since the delta model only works together with the backbone model.
we will automatically check whether you load the delta model the same way it is trained.

We calculate the trained model's [md5](http://some_link) and save it to the config. When finishing loading the delta model, we will re-calculate the md5 to see whether it changes.

 Note that performance is guaranteed by passing the hash check, but there are cases where the hash check is not passed but performance is still normal for various reasons. We are checking the reasons for this. Please consider this feature as a supplement.

Pass `check_hash=False` to disable the hash checking.

repository: thunlp/OpenDelta

repo_name: thunlp/OpenDelta


text:
# Creating a build configuration
This tutorial shows how to create a build configuration that runs npm to install dependencies.

## Module structure

```
.
├── package.json
├── .will.yml
```

## Creating our first build configuration

### Preparing the package.json file:

Create a `package.json` file in the root directory, with content:
``` json
{
  "name": "second",
  "dependencies": {
    "express": ""
  }
}
```

### Preparing the will file:

In the previous tutorials, we created a basic configuration with the `about` field.
Our current goal is to extend it with a build configuration.

#### First we need to learn about two new components of the will config:

`step` - describes the build steps: a step can be a shell command or predefined operation, like file(s) reflecting( copying ).

`build` - describes the build configuration: a [build](../Build.md) configuration is a sequence of build steps.

#### Add a build step:

This step executes the shell command: `npm install`.

```yaml
step :

  npm.install :
    currentPath : '.'
    shell : npm install
```

`npm.install` - name of the step.

`currentPath` - path to the directory from where the command will be executed, current working directory in this case.

`shell` - command to execute.

#### Add a build configuration:

```yaml
build :

  debug :
    criterion :
      default : 1
    steps :
      - npm.install
```

`debug` - name of the configuration.

`criterion` - defines criteria which help to select the necessary configuration, sage of criterions will be described later.

`steps` - list of steps to execute in the current build configuration.

## Testing the build configuration:

The final version of the `.will.yml` file would be:

  Click to expand!

```yaml

about :

  name : second
  description : "Second module"
  version : 0.0.1

step :

  npm.install :
    currentPath : '.'
    shell : npm install

build :

  debug:
    criterion :
      default : 1
    steps :
      - npm.install
```

#### To launch the current ( default ) build configuration, run:

> Command should be executed from the root directory of the module.

```
will .build
```

Dependecies defined in the `package.json` will be installed:

```
  Building debug
    > npm install
  added 48 packages from 36 contributors and audited 121 packages in 2.302s

  Built debug in 2.875s
```
#
[Back to content](../README.md)

license: MIT

repository: Wandalen/willbe

branch_name: refs/heads/master

extension: md

gha_language: JavaScript

gha_license_id: MIT

license_type: permissive

repo_name: Wandalen/willbe


text:
---
layout: post
author: csiu
title: "Blog setup"
date: 2015-09-13
categories: update
tags: setup
header-img: img/home-bg.jpg
header-img-source: Ryan McGuire
---
For my first entry, it's only fitting to disclose how this blog was setup.
Previously, I was aware that this can be done using [GitHub Pages](https://pages.github.com), but it was only this weekend that I decided to sit down and do this.

The blog setup consists of 2 core steps: Setup and Customization.
Obviously, Setup is necessary whereas Customization is not
due to the list of default themes provided by GitHub.

# Setup
In Setup, you will need a GitHub account
to create a new GitHub repository named `username.github.io`
where `username` refers to your Github username.

Your site should then be found at **http://username.github.io**

To make the site less blank, you can then
[launch the automatic page generator](https://help.github.com/articles/creating-pages-with-the-automatic-generator/)
(found under Settings > GitHub Pages: Launch automatic page generator).

# Customize
In my cause, I didn't like the selection in the automatic page generator.
I wanted something different and so I turned to [Jekyll](https://jekyllrb.com),
a simple, blog-aware static site generator
[supported by GitHub Pages](https://help.github.com/articles/using-jekyll-with-pages).

Initially, I needed to download jekyll:

~ $ gem install jekyll

To test jekyll, I followed [jekyll's quick-start guide](http://jekyllrb.com/docs/quickstart/) and did:

~ $ jekyll new myblog
~ $ cd myblog
~/myblog $ jekyll serve

Going to **http://localhost:4000** would then present a preview of the new site.

## A different theme
To customize the theme, I picked [clean blog](http://startbootstrap.com/template-overviews/clean-blog/) from [jekyllthemes.org](http://jekyllthemes.org)

## A different header
To customize the header image, I used an image from [Ryan McGuire](http://www.gratisography.com/)'s collection.
In an [article by Casey Ark](http://www.entrepreneur.com/article/238646),
other amazing free stock photo websites can be found.

repository: csiu/blog

repo_name: csiu/blog


text:
# Code conventions

## TypeScript configs

Each package has its own `.tsconfig.json` file that configures TypeScript and extends the base configuration in `tsconfig-package-base.json`. These files should be kept as similar as possible.

## TSDoc

We document our code with [TSDoc](https://tsdoc.org/) to avoid duplicating our type definitions in our documentation. This is enforced via ESLint with the [eslint-plugin-tsdoc](https://www.npmjs.com/package/eslint-plugin-tsdoc) plugin.

## Conventional Commits

We use the [Conventional Commits Specification](https://www.conventionalcommits.org/en/v1.0.0/) to write commit messages. This is important as it will be used to determine the version number during the CI workflow and to generate automated CHANGELOG.md files in each package. This is enforced with [commitlint](https://commitlint.js.org/#/) to ensure that every commit follows our format.

Keep in mind that your commit messages will become part of a change log, so you should write them to be as human-readable and user-friendly as possible.

Each commit has a **type** and an optional **scope**.

**Type** must be one of the following:

- `build`: Changes that affect the build system or external dependencies
- `chore`: Other changes that don't modify source or test files
- `ci`: Changes to our CI configuration files and scripts
- `docs`: Documentation-only changes
- `feat`: A new feature
- `fix`: A bug fix
- `perf`: A code change that improves performance
- `refactor`: A code change that neither fixes a bug nor adds a feature
- `style`: Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc.)
- `test`: Adding missing tests or correcting existing tests

**Scope**, if included, must be a directory name within `packages/`, such as `web-cra`.

*Examples of valid commit format:*
```
fix: remove a circular dependency
```
- the type for this commit is `fix`
- there is not a scope for this commit (scopes are optional)
- a `fix` type of commit results in a patch version bump when `yarn bump-versions` is run later

```
feat(web-cra): add Profile page
```
- the type for this commit is `feat`
- the scope for this commit is `web`, which corresponds to `packages/web` (scopes are optional)
- a `feat` type of commit results in a minor version bump when `yarn bump-versions` is run later

```
feat(web-cra)!: lowercase email addresses before sending to API
```
- the type for this commit is `feat`
- the scope for this commit is `web-cra`, which corresponds to `packages/web-cra` (scopes are optional)
- a `!` denotes a breaking change, which results in a major version bump when `yarn bump-versions` is run later

## Prettier

We are using [prettier](https://prettier.io/) with a pre-commit hook to format code. We've configured prettier to work with [eslint](https://eslint.org/) and [stylelint](https://stylelint.io/), and the two tools are performing different functions. Prettier is enforcing a consistent code style so we don't have to think about things like line breaks and semicolons. ESlint is protecting us from code smells like unused imports. We are using the widely vetted rules in the [Airbnb style guide](https://github.com/airbnb/javascript) as our base linting configuration.

There are several config files in the root of the monorepo that are used to set things up:
- .eslintrc.js => configures `eslint`
- .prettierrc.js => configures `prettier`
- .stylelintrc => configures `stylelint`

Note that each package can also include its own .eslintrc.js to extend the root configuration. The root eslint configuration assumes the use of TypeScript.

### Set up VS Code with Prettier

If you are using VSCode, it's a good idea to add the [Prettier - code formatter extension](https://marketplace.visualstudio.com/items?itemName=esbenp.prettier-vscode). Check out the installation instructions for details on how to set up Prettier as the default code formatter for VS Code.

## Pre-commit hook

We are using [husky](https://github.com/typicode/husky) to provide a pre-commit hook that runs [lint-staged](https://github.com/okonet/lint-staged) to clean up our code. This may take a few seconds; this is because we are using lerna to run the `precommit` script in every package per [this example](https://github.com/sudo-suhas/lint-staged-multi-pkg). This setup may need to be adjusted later. There is a `.lintstagedrc.json` in each package that configures the `lint-staged` command; typically, it runs `eslint --fix` to fix TypeScript files and `stylelint --fix` to fix CSS files. This happens automatically before each commit. If there is a linting error that can't be fixed automatically, it will be shown in the console when you `git commit`.

repository: alexnitta/woodshed

branch_name: refs/heads/main

repo_name: alexnitta/woodshed


text:
---
title: "Study on non-lysosomal endocytosis"
authors:
- China Pharmaceutical University
- Mar.2018-Jun.2018
date: "2019-05-15T00:00:00Z"
doi: "https://www.thno.org/v08p3474"

# Schedule page publish date (NOT publication's date).
publishDate: "2017-01-01T00:00:00Z"

# Publication type.
# Legend: 0 = Uncategorized; 1 = Conference paper; 2 = Journal article;
# 3 = Preprint / Working Paper; 4 = Report; 5 = Book; 6 = Book section;
# 7 = Thesis; 8 = Patent
publication_types: ["2"]

# Publication name and optional abbreviated publication name.
publication: ""
publication_short: ""

abstract: Potent Protein Therapeutics via a Non-lysosomal Route.

# Summary. An optional shortened abstract.
summary: Found my programming skills surprising useful in variable control & outcome prediction, and got started with some really cool instrument workstations.

tags:
- Drug Delivery
featured: false

# links:
# - name: ""
#   url: ""
url_pdf:
url_code: ''
url_dataset: ''
url_poster: ''
url_project:
url_slides: ''
url_source: ''
url_video: ''

# Featured image
# To use, add an image named `featured.jpg/png` to your page's folder.
image:
  #caption: 'Image credit: [**Unsplash**](https://unsplash.com/photos/jdD8gXaTZsc)'
  focal_point: ""
  preview_only: false

# Associated Projects (optional).
#   Associate this publication with one or more of your projects.
#   Simply enter your project's folder or file name without extension.
#   E.g. `internal-project` references `content/project/internal-project/index.md`.
#   Otherwise, set `projects: []`.
projects: []

# Slides (optional).
#   Associate this publication with Markdown slides.
#   Simply enter your slide deck's filename without extension.
#   E.g. `slides: "example"` references `content/slides/example/index.md`.
#   Otherwise, set `slides: ""`.
slides: example
---
### For this project, I helped to reveal the influence of nanoparticle properties on their endocytosis.

Specifically, we confirmed **rod-shaped** active pure drug nanoparticles enable **caveolae-mediated endocytosis**, which potentialy benefited the present miRNA delivery system, including prolonged blood circulation, protection of miRNA against decomposition by enzymatic and acidic conditions, and combination use of drugs and miRNA.

This experience was incredibly rewarding, as it provided me methodology and theoretical basis to address problems on my next project(see **Baicalein-mediated Delivery of p53 in Therapeutics of Pulmonary Hypertension**) and helped me gain the confidence to work independently. Surprisingly, I found my **programming skills** surprising useful in variable control & outcome prediction.Also, I noticed that while numerous promising biomacromolecule drugs were discovered and utilized, their applications were severely limited by their unfavorable pharmacodynamic properties and toxicities, leading to the necessity of suitable delivery systems.


text:
# GetSovereigntyCampaigns200Ok

## Properties
Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
**AttackersScore** | **float32** | Score for all attacking parties, only present in Defense Events.  | [optional] [default to null]
**CampaignId** | **int32** | Unique ID for this campaign. | [default to null]
**ConstellationId** | **int32** | The constellation in which the campaign will take place.  | [default to null]
**DefenderId** | **int32** | Defending alliance, only present in Defense Events  | [optional] [default to null]
**DefenderScore** | **float32** | Score for the defending alliance, only present in Defense Events.  | [optional] [default to null]
**EventType** | **string** | Type of event this campaign is for. tcu_defense, ihub_defense and station_defense are referred to as \"Defense Events\", station_freeport as \"Freeport Events\".  | [default to null]
**Participants** | [**[]GetSovereigntyCampaignsParticipant**](get_sovereignty_campaigns_participant.md) | Alliance participating and their respective scores, only present in Freeport Events.  | [optional] [default to null]
**SolarSystemId** | **int32** | The solar system the structure is located in.  | [default to null]
**StartTime** | [**time.Time**](time.Time.md) | Time the event is scheduled to start.  | [default to null]
**StructureId** | **int64** | The structure item ID that is related to this campaign.  | [default to null]

[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)

repository: antihax/goesi

gha_language: Go

repo_name: antihax/goesi


text:
kor-input-json
===

[![Build Status](https://travis-ci.org/ksss/kor-input-json.svg)](https://travis-ci.org/ksss/kor-input-json)

JSON input plugin for [kor](https://github.com/ksss/kor).

# Usage

```
$ cat table.json
{"foo": 100, "bar": 200}
{"bar": 500, "baz": 600}

$ kor json csv < table.json
foo,bar,baz
100,200,
,500,600

$cat table-single.json
[{"foo":100,"bar":200},{"bar":500,"baz":600}]

$ kor json --single markdown < table-single.json
| foo | bar | baz |
| --- | --- | --- |
| 100 | 200 |  |
|  | 500 | 600 |
```

## Installation

Add this line to your application's Gemfile:

```ruby
gem 'kor-input-json'
```

And then execute:

    $ bundle

Or install it yourself as:

    $ gem install kor-input-json

## Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/ksss/kor-input-json. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](contributor-covenant.org) code of conduct.

## License

The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).

## Refs

- https://github.com/ksss/kor

repository: ksss/kor-input-json

repo_name: ksss/kor-input-json


text:
[![Waffle.io - Columns and their card count](https://badge.waffle.io/sammyCatlady42/Kalaha.svg?columns=all)](https://waffle.io/sammyCatlady42/Kalaha)

[![Build Status](https://travis-ci.org/sammyCatlady42/Kalaha.svg?branch=master)](https://travis-ci.org/SammyCatlady42/htwg.se.Kalaha)

[![Coverage Status](https://coveralls.io/repos/github/sammyCatlady42/Kalaha/badge.svg?branch=master)](https://coveralls.io/github/sammyCatlady42/Kalaha?branch=master)
# Kalaha
=====================================================
## A game from two students for the lecture SoftwareEngineering

- at the University of Applied Science HTWG Konstanz.

The project has
* a folder structure prepared for a MVC-style application
* *ScalaTest* and as dependency aswell as dependencies to other libraries in the build.sbt (commented out at start).
* *scalastyle-sbt-plugin* and *sbt-scoverage* sbt plugins
-It requires Java 11 on your local platform.

repository: Creliv/Kalaha

gha_language: Scala

gha_license_id: NOASSERTION

repo_name: Creliv/Kalaha


text:
---
layout: post
title:  "Google Bard prompt engineer job - Chief Creativity Connoisseur"
author: Pramod
categories: [ ChatGPT, Midjourney ]
image: assets/images/ai-series.png
beforetoc: "Google Bard AI prompt engineering jobs and openings for everyone"
featured: true
toc: true
---

## Job Title
Google Bard AI Prompt Engineer (aka Chief Creativity Connoisseur)
**Location:** Any where in the world.

## Job Description

Do you have a passion for creativity, language, and artificial intelligence?  Do you want to help shape the future of conversational AI? Then we want you to join
the TipSeason Bard team as our newest Prompt Engineer!

As a Prompt Engineer, you'll be responsible for crafting the perfect prompts that will inspire Google Bard to write
poetry, tell jokes, and have conversations that are both informative and entertaining. You'll work closely with our AI
engineers to develop new algorithms that will help Bard generate even more creative responses, and you'll be the driving
force behind our efforts to make Bard the most engaging chatbot in the world.

But that's not all! As the Chief Creativity Connoisseur, you'll be tasked with keeping up with the latest trends in pop
culture, literature, and the arts, and finding new ways to incorporate them into Bard's conversations. You'll need to be
quick-witted, resourceful, and able to come up with new ideas on the fly. And of course, you'll need to be comfortable
working in a fast-paced, dynamic environment where the only constant is change.

## Qualifications

A degree in English, Creative Writing, or a related field
Excellent writing skills and a deep understanding of language and grammar
Familiarity with machine learning, natural language processing, and conversational AI
A passion for pop culture, literature, and the arts
A sense of humor and an appreciation for the absurd
The ability to work independently and collaboratively in a team environment
Experience with Python, TensorFlow, or other programming languages is a plus

## Benefits

* Competitive salary and stock options
* Medical, dental, and vision insurance
* 401(k) plan with company match
* On-site gym, yoga classes, and free healthy snacks
* Unlimited vacation policy (with manager approval)
* Opportunities for career growth and advancement
* Access to cutting-edge AI technology and research

So if you're ready to put your creativity to work and help shape the future of conversational AI, we want to hear from
you! Apply today to join the Google Bard team and become a part of something truly innovative and exciting.

------
Alright! Hope you had a fun time reading this blog post :-)
Before you get too excited and start submitting your resume, we must clarify that this job posting for the Google Bard
AI Prompt Engineer is purely fictitious and intended for entertainment purposes only. While we have no doubt that there
are real jobs out there that require similar skills and qualifications, this particular job posting is not one of them.
So, if you were ready to apply, we suggest you take a break, relax and enjoy some witty conversations with ChatGPT or
BardAI of your liking.
Meanwhile here is a amazing article (this time for real) about [Google bard AI vs ChatGPT]()

Make your closed ones smile by sharing this blog post :-)

Subscribe to more interesting AI and tech tutorials straight into your mail.


text:
# Laravel 8 Starter Package featuring GraphQL, Cloudinary API, and ESLint pre-installed

## Who is this for?
This repository serves as a complete starting point for anyone attempting to use Laravel along with GraphQL in order to support static website libraries, such as [GatsbyJS](https://gatsbyjs.com). In such a use case where this backend would only serve as your API, the entire `resources/js` folder can be ignored, and for such use cases, I haven't modified it too much.

If you're considering using it as the only _source of truth_ for your entire project, then you might also find it useful.

## Contents

This project consists of the following packages I additionaly set-up:
- [GraphQL](https://lighthouse-php.com/) (with MySQL, a complete switch to [MongoDB](https://github.com/jenssegers/laravel-mongodb) is being discussed in my head)
- ESLint ([huge thanks to Wesbos for his template](https://github.com/wesbos/eslint-config-wesbos))
- [Cloudinary API](https://github.com/cloudinary-labs/cloudinary-laravel)

### Post-Installation Set-up, AKA How to make above mentioned packages work
* `php artisan vendor:publish --tag=lighthouse-schema`
* `npx install-peerdeps --dev eslint-config-wesbos`
* `php artisan vendor:publish --provider="CloudinaryLabs\CloudinaryLaravel\CloudinaryServiceProvider" --tag="cloudinary-laravel-migration"`
* `php artisan vendor:publish --provider="CloudinaryLabs\CloudinaryLaravel\CloudinaryServiceProvider" --tag="cloudinary-laravel-config"`

Or follow the guides for each package, I might have pasted unnecessary lines 😊

If you're going to use Cloudinary API, don't forget to set the `CLOUDINARY_URL` key in your `.env` file.

Note that the `graphql/schema.graphql` file is empty. Please create any models you might want to use in your project by yourself and define the necessary Types, Fields, or Directives.

## License

The Laravel framework is open-sourced software licensed under the [MIT license](https://opensource.org/licenses/MIT).


text:
# AI-Voice-Assistant
Interactive voice assistant that can take limited voice commands and perform actions automatically.

# Functionality

* It is an interactive utility which can take voice commands and even has functionality to train your model to recognize commands from specific users only (voice recognition).
* Once voice recognition is done, it has functionality to recognize specific key words in sentences and perform automatic actions accordingly.
* It can play music from your local directory, it can play music from web.
* You can command it to tell weather, search something from wikipedia and speak the output (limited sentences).
* It has a functionality to detect your expression, age using trained Neural Networks and play songs accordingly.

# Packages Used

* pyttsx3
* datetime
* speech_recognition
* sounddevice
* scipy
* Numpy
* librosa
* Pandas
* sklearn
* keras
* wikipedia
* webbrowser
* multiprocessing
* playsound
* selenium
* mutagen
* time
* random
* os

# Steps Involved

* The code makes use of **pyttsx3** for text to speech functionality.
* Method **speak()** takes in text and converts that text to speech.
* Method **greet()** captures the current time and greets accordingly.
* **takeCommand()** uses speech_recognition library to recognize what user is saying.
* **take_voice_sample()** this method is used to record the input voice of the user and store it as a .wav file. This method is used only for the purpose of training voice recognition model.
* Method **parser()** is used to parse the .wav voice samples to tabular data which can be used to train voice recognition model. **data_processing_for_model()** method returns the final processed data.
* **train_voice_model()** creates and saves the model as .h5 and returns the Neural Network model for training voice samples.
* Then we have **authenticate()** method, which is optional. It is simply used to authenticate the user with their voice from the model trained in steps above.
* **detect_expression_and_age()** method takes input from default webcam and detects face, determines expression and age and displays the text accordingly.
* I have a separate repository which explains how to train a model for human facial expression detection (Please refer it, if required).
* Finally, the driver code uses the above methods depending on the keywords in the user's sentence.

repository: Adios-rgb/AI-Voice-Assistant

repo_name: Adios-rgb/AI-Voice-Assistant


text:
---
date: 2019-01-29 13:12:00 +0000
# Update page modified by changing the date.
# In Vim use `:r! date -u +"%F %T %z"`
---

### Introduction

Bitcoin mining is currently too expensive for an individual to join. However, the field of blockchain software is still in its infancy. Without a question, hardware will be used to accelerate computation around blockchain developments and cryptocurrencies.

We, and open to YOU, are making this project to be a group editable resource to break thru the noise of the web, and paid reviews, or guides, to be an open community process for jumpstarting the world of Blockchain Hardware from the individual up. We are Open Source, Blockchain Hardware and Software contributors. We live and work in Shenzhen, Hong Kong, Beijing, Taiwan, Berlin, San Francisco, and more. We are coming together to bring software developers closer to hardware realities.

Please add your knowledge to these guides, and see you at a future event in Shenzhen :)

{% include footer_meta.md mod_date=page.date %}


text:
# CoordinatorStorytelling-UI

Deployed Site: https://coordinatedstorytelling.netlify.com/

## Local Hosting

* `less-watch-compiler less css index.less` for less compiler
* Open in local hosting/live server and edit

## GitHub Workflow

* Fork a copy of this repository
* Clone __your__ forked version of the repository: git clone {github-clone-link}
* Create a new branch: git checkout -b {branch-name}
* Add and Commit your changes regularly
    * git add .
    * git commit -m {message of commit}
* Push commits: git push origin {branch-name} __Do not git push master__

gha_language: CSS


text:
# Skupper

[![skupper](https://circleci.com/gh/skupperproject/skupper.svg?style=shield)](https://app.circleci.com/pipelines/github/skupperproject/skupper)

Skupper enables cloud communication by enabling you to create a Virtual Application Network.

This application layer network decouples addressing from the underlying network infrastructure.
This enables secure communication without a VPN.

You can use Skupper to create a network from namespaces in one or more Kubernetes clusters as described in the [Getting Started](https://skupper.io/start/index.html).
This guide describes a simple network, however there are no restrictions on the topology created which can include redundant paths.

Connecting one Skupper site to another site enables communication both ways.
Communication can occur using any path available on the network, that is, direct connections are not required to enable communication.

Skupper supports [anycast](https://en.wikipedia.org/wiki/Anycast) and [multicast](https://en.wikipedia.org/wiki/Multicast) communication using the application layer network (VAN), allowing you to configure your topology to match business requirements.

Skupper does not require any special privileges, that is, you do not require the `cluster-admin` role to create networks.

# Useful Links
Using Skupper

* [Getting Started](https://skupper.io/start/index.html)
* [Examples](https://skupper.io/examples/index.html)
* [Documentation](https://skupper.io/docs/index.html)
* [Skupper Docker](https://github.com/skupperproject/skupper-docker) allows you run Skupper on your laptop

Developing Skupper

* [Community](https://skupper.io/community/index.html)
* [Site controller](cmd/site-controller/README.md)
* [CLI](cmd/skupper/README.md) (This replaces the [Skupper CLI repo](https://github.com/skupperproject/skupper-cli))
* [Console (Preview)](https://github.com/skupperproject/skupper-console)

# Licensing
Skupper uses the [Skupper Router](https://github.com/skupperproject/skupper-router) project and is released under the same [Apache License 2.0](https://github.com/skupperproject/skupper/blob/main/LICENSE).

repository: skupperproject/skupper

repo_name: skupperproject/skupper


text:
# Bright Nucleus Country Codes Database

[![Latest Stable Version](https://poser.pugx.org/brightnucleus/country-codes/v/stable)](https://packagist.org/packages/brightnucleus/country-codes)
[![Total Downloads](https://poser.pugx.org/brightnucleus/country-codes/downloads)](https://packagist.org/packages/brightnucleus/country-codes)
[![Latest Unstable Version](https://poser.pugx.org/brightnucleus/country-codes/v/unstable)](https://packagist.org/packages/brightnucleus/country-codes)
[![License](https://poser.pugx.org/brightnucleus/country-codes/license)](https://packagist.org/packages/brightnucleus/country-codes)

This is a Composer plugin that provides an automated version of the free MaxMind GeoLite CSV Country Codes database.

The main advantage is that the downloaded database will be checked for updates on each `composer install` and `composer update`.

## Table Of Contents

* [Attribution](#attribution)
* [Installation](#installation)
* [Basic Usage](#basic-usage)
* [Contributing](#contributing)
* [License](#license)

## Attribution

This product includes GeoLite Legacy data created by MaxMind, available from
http://www.maxmind.com.

## Installation

The only thing you need to do to make this work is adding this package as a dependency to your project:

```BASH
composer require brightnucleus/country-codes
```

## Basic Usage

On each `composer install` or `composer update`, a check will be made to see whether there's a new version of the database available. If there is, that new version is downloaded.

Usage is pretty straight-forward. Just use one of the two provided static methods:

```PHP
<?php

use BrightNucleus\CountryCodes\Country;

// Get the name from an ISO 3166 country code.
$name = Country::getNameFromCode( 'US' ); // Returns 'United States'.

// Get the ISO 3166 country code from a country name.
$code = Country::getCodeFromName( 'United States' ); // Returns 'US'.
```

## Contributing

All feedback / bug reports / pull requests are welcome.

## License

This code is released under the MIT license.

For the full copyright and license information, please view the [`LICENSE`](LICENSE) file distributed with this source code.

repository: donmhico/country-codes

repo_name: donmhico/country-codes


text:
[![Lifecycle: stable](https://img.shields.io/badge/lifecycle-stable-green.svg)](https://www.tidyverse.org/lifecycle/#stable)
[![CRAN Status](https://www.r-pkg.org/badges/version/causaloptim)](https://cran.r-project.org/package=causaloptim)
[![Travis build status](https://travis-ci.org/sachsmc/causaloptim.svg?branch=master)](https://travis-ci.org/sachsmc/causaloptim)

# causaloptim: An Interface to Specify Causal Graphs and Compute Bounds on Causal Effects

When causal quantities are not identifiable from the observed data, it still may be possible to bound these quantities using the observed data. We outline a class of problems for which the derivation of tight bounds is always a linear programming problem and can therefore, at least theoretically, be solved using a symbolic linear optimizer. We provide a user friendly graphical interface for setting up such problems via DAGs, which only allow for problems within this class to be depicted. The user can then define linear constraints to further refine their assumptions to meet their specific problem, and then specify a causal query using a text interface. The program converts this user defined DAG, query, and constraints, and returns tight bounds. The bounds can be converted to R functions to evaluate them for specific datasets, and to latex code for publication.

## Development status

This package is in stable development. The interface is unlikely to have major changes at this time. New features may be added over time.

## Installation

```{r}
remotes::install_github("sachsmc/causaloptim")
```

Or use the web application: https://sachsmc.shinyapps.io/causaloptimweb/

## Usage

Launch the shiny app to get started, results are saved in the `results` object:

```{r}
results <- specify_graph()
```

## References

M.C. Sachs, E.E. Gabriel, and A. Sjölander, "Symbolic Computation of Tight Causal Bounds", 2020. Preprint available at https://sachsmc.github.io/causaloptim/articles/CausalBoundsMethods.pdf .

A. Balke and J. Pearl, "Counterfactual Probabilities: Computational Methods,Bounds, and Applications"  UCLA Cognitive Systems Laboratory, Technical Report (R-213-B).
In R. Lopez de Mantaras and D. Poole (Eds.), Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI-94), Morgan Kaufmann, San Mateo, CA, 46-54, July 29-31, 1994.
https://ftp.cs.ucla.edu/pub/stat_ser/R213-B.pdf .

repository: jbdatascience/causaloptim

repo_name: jbdatascience/causaloptim


text:
[![GitHub Release](https://img.shields.io/github/v/release/hivemq/mqtt-cli?style=for-the-badge)](https://github.com/hivemq/mqtt-cli/releases)
[![GitHub Workflow Status (branch)](https://img.shields.io/github/actions/workflow/status/hivemq/mqtt-cli/.github/workflows/check.yml?branch=master&style=for-the-badge)](https://github.com/hivemq/mqtt-cli/actions/workflows/check.yml?query=branch%3Amaster)
[![CLI Downloads](https://img.shields.io/github/downloads/hivemq/mqtt-cli/total?style=for-the-badge)](https://github.com/hivemq/mqtt-cli/releases)
[![CLI License](https://img.shields.io/github/license/hivemq/mqtt-cli?style=for-the-badge)](https://github.com/hivemq/mqtt-cli/blob/develop/LICENSE)

mqtt-cli is a feature-rich MQTT Command Line Interface.

[Getting Started](#getting-started) •
[Publish](#publish) •
[Subscribe](#subscribe) •
[Shell](#shell) •
[Test](#test) •
[How to Contribute](#how-to-contribute)

## Getting Started

- [Installation instructions](https://hivemq.github.io/mqtt-cli/docs/installation/)
- [Building from source](https://hivemq.github.io/mqtt-cli/docs/installation/#building-from-source)
- [Full documentation](https://hivemq.github.io/mqtt-cli)

## Publish

- Publish a message
- Quick start: `mqtt pub -t your-topic -m "your message" -h your-mqtt-broker.com`
- [Further documentation](https://hivemq.github.io/mqtt-cli/docs/publish/)

## Subscribe

- Subscribe to topics and receive output directly on the console
- Quick start: `mqtt sub -t your-topic -h your-mqtt-broker.com`
- [Further documentation](https://hivemq.github.io/mqtt-cli/docs/subscribe/)

## Shell

- Enter the mqtt-cli shell mode to access more MQTT functionality
- Quick start: `mqtt sh`
- [Further documentation](https://hivemq.github.io/mqtt-cli/docs/shell/)

## Test

- Run tests against a broker to find out its features and limitations
- Quick start: `mqtt test -h your-mqtt-broker.com`
- [Further documentation](https://hivemq.github.io/mqtt-cli/docs/test/)

## How to Contribute
- If you want to request a feature or report a bug, please [create a GitHub Issue using one of the provided templates](https://github.com/hivemq/mqtt-cli/issues/new/choose)
- If you want to make a contribution to the project, please have a look at the [contribution guide](CONTRIBUTING.md)

repository: hivemq/mqtt-cli

gha_language: Java

repo_name: hivemq/mqtt-cli


text:
//[sdk](../../../index.md)/[com.robotemi.sdk.listeners](../index.md)/[OnRobotReadyListener](index.md)

# OnRobotReadyListener

[androidJvm]\
interface [OnRobotReadyListener](index.md)

## Functions

| Name | Summary |
|---|---|
| [onRobotReady](on-robot-ready.md) | [androidJvm]
abstract fun [onRobotReady](on-robot-ready.md)(isReady: [Boolean](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/-boolean/index.html))
Called when connection with robot was established. |

repository: WiseCare-Dev/sdk

gha_language: Kotlin

repo_name: WiseCare-Dev/sdk


text:
*************************************
# Release Notes for OpenMDAO-XDSM 1.0

May 16, 2023

## Backwards Incompatible API Changes:

- None

## Backwards Incompatible NON-API Changes:

- The omxdsm package was restructured, and some functions were moved into new files.

## New Features:

- Short class names (class names without module path), by passing class_names=short

## Bug Fixes:
- Fixed duplicate AutoIndepVarComp, when newer OpenMDAO versions were used.

*************************************
# Release Notes for OpenMDAO-XDSM 0.8

May 9, 2023

## Backwards Incompatible API Changes:

- None

## Backwards Incompatible NON-API Changes:

- None

## New Features:

- Compatibility with OpenMDAO 3.26 and beyond.

## Bug Fixes:
- Sellar example updates. In the current version include_indepvarcomps=False, class_names=False
    options need to be used to create PDFs. To be fixed in a future release.

*************************************
# Release Notes for OpenMDAO-XDSM 0.7

January 27, 2023

## Backwards Incompatible API Changes:

- None

## Backwards Incompatible NON-API Changes:

- None

## New Features:

- Compatibility with OpenMDAO 3.21 and beyond.

## Bug Fixes:
- None

*************************************
# Release Notes for OpenMDAO-XDSM 0.6

July 31, 2020

## Backwards Incompatible API Changes:

- None

## Backwards Incompatible NON-API Changes:

- None

## New Features:

- Automatic IndepVarComps (new in OpenMDAO 3.2 - POEM_015) are supported.

## Bug Fixes:
- Models with Automatic IndepVarComps failed, which is now fixed.

*************************************
# Release Notes for OpenMDAO-XDSM 0.5

April 25, 2020

## Backwards Incompatible API Changes:

- None

## Backwards Incompatible NON-API Changes:

- The xdsmjs source code was removed.
- The xdsmjs Python package (https://pypi.org/project/xdsmjs/) was added to the dependencies.

## New Features:

- None

## Bug Fixes:
- None

*************************************
# Release Notes for OpenMDAO-XDSM 0.4

March 27, 2020

## Backwards Incompatible API Changes:

- None

## Backwards Incompatible NON-API Changes:

- pyxdsm tests are not skipped, and pyxdsm is always tries to be imported. This can only break your code, if pyxdsm
is not installed.

## New Features:

- None

## Bug Fixes:
- None

*************************************
# Release Notes for OpenMDAO-XDSM 0.3

March 23, 2020

## Backwards Incompatible API Changes:

- None

## Backwards Incompatible NON-API Changes:

- pyxdsm is now on pypi, added as a dependency (instead of being optional), since now it is easy to install it.

## New Features:

- Updated docs.

## Bug Fixes:
- None

*************************************
# Release Notes for OpenMDAO-XDSM 0.2

March 14, 2020

## Backwards Incompatible API Changes:

- None

## Backwards Incompatible NON-API Changes:

- Dependencies of the `six` package were removed. That means, that OpenMDAO-XDSM will be compatible only with
    Python 3 (OpenMDAO 3 also dropped the Python 2 compatibility)

## New Features:

- New option `include_inepvarcomps`. If it is false, the `IndepVarComp` instances of the model do not show up as
    subsystems on the diagram, instead they are converted to a design variable or external input, which are connected
    to the target system(s) of the `IndepVarComp`. This simplifies the XDSM, without the loss of much information.

## Bug Fixes:
- None

repository: onodip/OpenMDAO-XDSM

gha_language: HTML

repo_name: onodip/OpenMDAO-XDSM


text:
{#mainpage}

# Introduction

R2C (which stands for *your Renderer to Clarisse*) is a helper library designed to simplify the work of 3rd party developers that need to tightly integrate an external rendering engine into Clarisse. R2C provides a few specialized helper classes that are largely inspired from the scene and render delegate design pattern introduced by Pixar in USD Imaging Library (Hydra). It delivers a simplified API hiding the complexity of  scene item and their life-cycle/resource data management as well as render output buffers and interactive rendering.

> **Note:** R2C has been entirely built using the standard SDK shipped with Clarisse. It was exclusively written outside Clarisse's development trunk and there was no secret API involved :-)
> In other words **any** 3rd party developers experienced with Clarisse API would have been able to write R2C!

R2C is really reducing the learning curve of Clarisse API to 3rd party developers that have no or very little experience in Clarisse. Using R2C they can quickly focus on translating Clarisse's scenes and geometry structures into their renderers instead of spending time learning the intricacies of Clarisse API.

> **Important:** The R2C library is still at its early stage. It comes with a few limitations that can be easily lifted which are listed in the limitations section.

# Package Description

R2C package is an external module that is not (yet) part of the standard SDK of Clarisse. It comes with a fully doxygen style documented source code as well as cmake files to build the library. In the package, you'll  also find a working example of a non-feature complete integration of Redshift renderer in Clarisse.

We strongly recommend you to use this example as reference when integrating your renderer into Clarisse. It comes with a full documentation and can even be compiled and run provided you have access to a license of Redshift and its Redshift SDK.

R2C library source code can be found in `r2c` directory.

# Using the R2C API

As mentioned in the introduction, R2C API is largely inspired by the scene and render delegates design pattern introduced by Pixar in their USD Imaging Library.

The way R2C API works is that you have to provide an implementation of your renderer by deriving from the abstract *R2cRenderDelegate* class. The class has to be associated to an instance of a *R2cSceneDelegate*. The *R2cSceneDelegate* defines and manages internally a full Clarisse render scene. It also drives the render delegate according to the updates of the scene.

## Clarisse Object Model Framework

If you don't have any experience of Clarisse API, we highly advise you to read the following articles:
- [**Introduction to Clarisse Object Model**](https://www.clarissewiki.com/4.0/sdk/of_intro.html)
- [**Introduction to Clarisse Modules**](https://www.clarissewiki.com/4.0/sdk/declaring_modules.html)
- [**Introduction to Clarisse Interface Definition**](https://www.clarissewiki.com/4.0/sdk/cid_intro.html)

These articles describe fundamental concepts of Clarisse you need to understand before going any further.

## R2cSceneDelegate the Render Scene Hub

The scene delegate manages a render scene defined by a camera, a render settings (which must inherit from the *OfClass* *Renderer*), a set of lights and a set of geometries inputs which can be easily set through its class members. The *R2cSceneDelegate* acts a render scene hub by notifying the associated render delegate what happens in the scene. In fact, each time an input or its dependencies are modified by the user in Clarisse, the render delegate gets a notification so that it can update its own internal scene representation accordingly.

### Types of scene updates

There are a 3 distinct types of scene updates the scene delegate propagates to the render delegate:
- geometry modifications
- instancer modifications
- lights modifications

For each of these updates, items can be either be inserted, removed or modified in such way that:
- when an item is inserted into the scene, the scene delegate calls *R2cRenderDelegate::insert*.
- when an item is removed from the scene, it calls *R2cRenderDelegate::remove*
- when an item is modified it calls *R2cRenderDelegate::dirty*

Since it is very likely that the render delegate needs to do specific operations for geometries, instancers and lights, the render delegate interface specializes these methods for each type of items:
```cpp
class R2C_EXPORT R2cRenderDelegate : public CoreBaseObject {
/*...*/
    virtual void insert_geometry(R2cItemDescriptor item) = 0;
    virtual void remove_geometry(R2cItemDescriptor item) = 0;
    virtual void dirty_geometry(R2cItemDescriptor item, const int& dirtiness) = 0;

    virtual void insert_light(R2cItemDescriptor item) = 0;
    virtual void remove_light(R2cItemDescriptor item) = 0;
    virtual void dirty_light(R2cItemDescriptor item, const int& dirtiness) = 0;

    virtual void insert_instancer(R2cItemDescriptor item) = 0;
    virtual void remove_instancer(R2cItemDescriptor item) = 0;
    virtual void dirty_instancer(R2cItemDescriptor item, const int& dirtiness) = 0;
/*...*/
```
### Getting item information from the Scene Delegate

The scene delegate can always be accessed by the render delegate. As a matter of fact they always work in tandem.  Actually when creating a scene delegate, you must associate it with a render delegate using *R2cSceneDelegate::set_render_delegate* in order to be notified. Since they have such a close relationship, they both store back pointers to each other respectively.
> **Note:** It is possible to set a render delegate after the scene delegate has been populated.  In that case, the render delegate still gets properly synchronized. In the same way, it is possible to remove a render delegate from a scene delegate by passing a *nullptr* to *R2cSceneDelegate::set_render_delegate*. When doing so, the scene delegate automatically calls *R2cRenderDelegate::clear*.

#### Item descriptors, IDs and good practice

Each time the scene delegate notifies the render delegate from a scene update, it passes a *R2cItemDescriptor*. describing a unique item of the scene.  Depending on the context, this id can identify a geometry, an instancer, a light, a material etc. The *R2cItemDescriptor* class actually abstracts an *OfObject* which can be easily retrieved using *R2cItemDescriptor::get_item*. The reason why R2C API introduces this abstraction is to simplify the work of the renderer plugin writers.

> However, we do not recommend you to store *R2cItemDescriptor*. **Their life cycle should be considered valid only within the call of a method in the render delegate.**

The good practice is to store *R2cItemId* which are ids identifying a unique item of the scene. They can be easily obtained using *R2cItemDescriptor::get_id*. These ids can be used in hash maps etc...

> To get an item descriptor back from an id you just have to use *R2cSceneDelegate::get_render_item*.

Indeed, unlike *OfObject*, using *R2cSceneDelegate* you can retrieve an new *R2cItemDescriptor* and check the validity of the underlying item since it is managed by the scene delegate. That way it is possible to use *R2cItemDescriptor::is_destroyed* to detect if the item has been destroyed. Even when it is destroyed, you can still get its full name, type etc. However, *R2cItemDescriptor::get_item* will return as expected a *nullptr*. For more information, please refer to the documentation.

> **Important:** When *R2cRenderDelegate::remove* methods are called, it is possible that *R2cItemDescriptor* passed as argument may have been already destroyed! Make sure to check *R2cItemDescriptor::is_destroyed*

The good practice is to store/use ids in dedicated lists in the implementation of inserted and removed items. The idea is to make sure the operation is performed as quickly as possible to keep the application interactive. The actual operation of insertion/conversion/removal should be only done when *R2cRenderDelegate::render* is actually called. Obviously, in some cases it can make sense to free memory as soon as possible when getting notified that an item allocating a large chunk of memory is removed from the scene.

## R2cRenderBuffer

The *R2cRenderBuffer* is an abstract class that manages Clarisse render buffers. It is used to pass the actual rendered images and AOVs to Clarisse. The interface is pretty straight forward and we invite you to refer to the documentation for more information.

The R2C library currently provide only one implementation *ClarisseLayerRenderBuffer* which is specialized to fill the render buffer of a ImageLayer.

## Exposing Material and Texture Network

Shader integration in Clarisse can be done in two ways, either by inheriting from the class ModuleMaterial for materials, or by inheriting from the class ModuleTextureOperator for textures. By doing so, obviously it will be possible to assign your materials with the material linker or by dragging the material on a scene object in the image view, but also with the shading layer. All the shading assignment is managed automatically by the Scene Delegate. A shader network can be displayed automatically in the Material Editor.

There are a few things to know when implementing the integration of shaders. For all the texturable attributes of the shaders (i.e : attributes allowing to plug a texture as input), you must have to specify the classes of textures supported by the renderer, so it is not possible to connect anything else inside. You can do it by using the method OfAttr::set_texture_filters().

A lot of DCC allow to create shading networks with multiple outputs per node. In order to display and manage such graph, we have introduced a new outputs system in the Clarisse object framework. You can register such outputs at the level of the class or the object. It will be displayed in the material editor, and it will be possible to plug it in a texturable attribute. There is also an API to get the output bound to an attribute, this make it possible to reproduce the connection that have be created in Clarisse in the DCC.

Everything mentioned in this section have been implemented in the Redshift example. You can take a look at the file redshift_utils.cc in the redshift example library.

## Examples

The R2C library provides three working examples of integrating a renderer into Clarisse.

### Kubix

Found in the `module.kubix` directory, Kubix serves as a simple example of what it would take to implement a custom renderer from scratch into Clarisse.

![Spherix image](kubix_render.png)

It only supports basic features, but covers most aspects of a renderer, including:
- *Geometries* rendered as bounding boxes in Kubix
- *Lights* used to uniformly illuminate the scene in Kubix
- *Materials* used to uniformly color the Kubix objects
- *Textures* used to assign a uniform color to Kubix materials

The implementation of the *KubixRenderDelegate* can be found in `kubix_render_delegate.h` and `kubix_render_delegate.cc`. Most of the machinery of using R2C can be found in those files.

Despite its simplicity, the Kubix renderer should be seen as a steping stone towards understanding how one would implement a custom renderer into Clarisse, demonstrating the guidelines and general principles of R2C. For more thorough examples, please refer to [`Spherix`](#spherix) and [`Redshift`](#redshift).

### Spherix

Found in the `module.spherix` directory, Spherix serves as a simple example of what it would take to integrate an existing external renderer into Clarisse by simulating one.

![Spherix image](spherix_render.png)

It only supports basic features, but covers most aspects of a renderer, including:
- *Geometries* rendered as spheres in Spherix
- *Lights* used to uniformly illuminate the scene in Spherix
- *Materials* used to uniformly color the Spherix objects
- *Textures* used to assign a uniform color to Spherix materials

The implementation of the *SpherixRenderDelegate* can be found in `spherix_render_delegate.h` and `spherix_render_delegate.cc`. Most of the machinery of using R2C can be found in those files.

One interesting topic addressed by the example is the dynamic creation and registration of Clarisse classes describing External shaders used as  materials, lights and texture. The method *register_shader* can be found in `spherix_register_shaders.h`.
It also show how the external shaders will be automaticaly synchronized when changing a Clarisse attribute.

### Redshift

Found in the `module.redshift` directory, this examples demonstrates how to integrate a production renderer.

The Redshift integration consists of a *LayerRedshift* which serves as a render scene hub that defines 5 attributes/inputs:
- A *Camera* where users set the render viewpoint.
- A *Renderer* where users set render settings by creating an item of class *RendererRedshift*
- A group of *Geometries* where users can specify a group of geometries (*SceneObjects*) defining what to render
- A group of *Lights* where users can specify a group of lights (*Light*) defining which lights are in the render scene
- A *Shading Layer* which can define dynamically material association.

The implementation of the Redshift layer can be found in `layer_redshift.cc`. The layer actually owns the instances of the *R2cSceneDelegate* (which is set to the inputs of the layer) and the *RedshiftRenderDelegate* which implement the render delegate.

The implementation of the *RedshiftRenderDelegate* can be found in `redshift_render_delegate.h` and `redshift_render_delegate.cc`

One interesting topic addressed by the example is the dynamic creation and registration of Clarisse classes describing Redshift materials. The method *register_shader* can be found in `redshift_utils.cc`. This file also shows how to convert Clarisse native geometries and instancers. Have a look at both *RedshiftUtils::CreatePolymesh* and *RedshiftUtils::CreateInstancer*.

# Redshift in Clarisse video
You can find a video snapshot of the integration running in Clarisse [**here**](https://youtu.be/9Yg7vC3Xwq4)

# Limitations

Currently, the R2C library is not feature complete. However, it already provides an easier API to renderer plugin writers that should cover a lot of their needs. Also the library can be easily extended by either parties. Our idea was to offer a first working version of the API and then improve it according to your feedback.

## Redshift example

The Redshift integration example is a prototype to serve as an example for renderer plugin writers. It can be easily improved to support more feature since many important ones are missing right now. To list major missing ones:

- no curve/hair geometry support
- no motion blur support but can easily be implemented
- no AOVs support
- no progress reporter implemented
- minimal *Camera* support (only *CameraPerspective* works)
- minimal Redshift render settings support
- Light support is a proof of concept but it can be easily finalized
- Some implicit geometries are not supported such as *GeometrySphere* or *GeometryCylinder*. This could be a good use case of adding dynamically specific Redshift attributes to these classes offering users tessellation level attributes.

  ![Tessellation Attributes](redshift_tessellation_attributes.png)

- Shading group visibility is not supported since Redshift can't set the shading group visibility without rebuilding the entire geometry. Maybe an alternative approach would be to apply in such shading group a fully transparent material but what about performance issues?
- no displacement support
- no *GeometryBundle* support (could be implemented like instancers)
- no volume support
- No support of multiple UV maps (pretty straight forward to implement)

# Future work

- Implement the non-flattened version of Instancers (pretty straight forward)
- Multithreaded version of the *RedshiftRenderDelegate::sync* to create geometries and synchronize index in parallel to speed up sync.
- Add progress bar in *RedshiftRenderDelegate::sync* to notify the application of what's currently happening

Copyright (c) 2020 Isotropix SAS. All rights reserved.


text:
% VCFROC(1) vcfroc (vcflib) | vcfroc (VCF statistics)
% Erik Garrison and vcflib contributors

# NAME

**vcfroc**

# SYNOPSIS

**vcfroc** [options] [<vcf file>]

# DESCRIPTION

Generates a pseudo-ROC curve using sensitivity and specificity estimated against a putative truth set. Thresholding is provided by successive QUAL cutoffs.

# OPTIONS

```

options:
    -t, --truth-vcf FILE      use this VCF as ground truth for ROC generation
    -w, --window-size N       compare records up to this many bp away (default 30)
    -c, --complex             directly compare complex alleles, don't parse into primitives
    -r, --reference FILE      FASTA reference file

```

# EXIT VALUES

**0**
: Success

**not 0**
: Failure

# SEE ALSO

[vcflib](./vcflib.md)(1)

# OTHER

## Source code

[vcfroc.cpp](https://github.com/vcflib/vcflib/blob/master/src/vcfroc.cpp)

# LICENSE

Copyright 2011-2023 (C) Erik Garrison and vcflib contributors. MIT licensed.

<!--
  Created with ./scripts/bin2md.rb scripts/bin2md-template.erb
-->

repository: vcflib/vcflib

repo_name: vcflib/vcflib


text:
---
title: Heaven and Hell- Week 2- Heaven- Part 2
date: 2015-11-08 00:00:00 Z
link: https://s3.amazonaws.com/podcast.lifestonechurch.net/2015-11-08.mp3
podcastLength: '00:38:02'
size: 36529859
image: heavenAndHell.jpg
speaker: Pastor Ben Helton
description: Heaven and Hell
vimeoId: 145354856
verse:
---

## Sermon Notes

> "Joy is the business of Heaven." - C. S. Lewis

1 Corinthians 2:9

### What we will be like

1 John 3:2

1. We will no longer *sin*{: class="sermon-answer"}.

    Philippians 1:6

1. We will not be subject to the same laws of *physics*{: class="sermon-answer"}.

    John 20:19, 26

1. We will still *eat*{: class="sermon-answer"} and *touch*{: class="sermon-answer"}.

    John 20:27

    Revelation 22:2

1. We will never be *sick*{: class="sermon-answer"} again.

    Revelation 22:3, 21:4

    Isaiah 35:5-6

1. We will *recognize*{: class="sermon-answer"} each other.

    Matthew 17:3

1. We will have *deeper*{: class="sermon-answer"} relationships.

    Matthew 22:30

### What heaven will be like

Revelation 21

1. We will live in a new *Jerusalem*{: class="sermon-answer"}.

    Revelation 21

1. We will be *with*{: class="sermon-answer"} Christ.

    Revelation 22:4, 14

### How to get to Heaven

Revelation 22:14-15, 22:17

Romans 6:23

repository: myLordJ/lifestonechurch

branch_name: refs/heads/gh-pages

repo_name: myLordJ/lifestonechurch


text:
---
tags:
  - projects
  - data-science
  - python
  - classification
author: Jocelyn Ong
layout: posts
author_profile: true
---
Predicting whether a loan application to Lending Club will be approved

{% include toc %}

# Introduction

In this post, we'll be looking at data obtained from [Lending Club](https://www.lendingclub.com/info/download-data.action){:target="_blank"}. [Lending Club is an online credit marketplace, and it facilitates peer-to-peer loans and financing options. The loans are still considered issued by a bank and are subject to the same consumer protection, fair lending, and disclosure requirements as any other bank loan.](https://www.lendingclub.com/public/about-us.action){:target="_blank"}

# TL;DR

We look at common details between loans rejected and loans approved to see if we can figure out what are the factors affecting loan approval and whether we can predict if a future loan would be approved.

Based on our regression analysis, we find that the purpose of the loan and the employment length of the applicant have the greatest effect on whether a loan is approved. The loan amount actually has the smallest impact. So it doesn't mean that just because you're borrowing a small amount, you're more likely to have your loan approved.

We also performed a regression analysis with purpose dummy variables, and found that stating "Car" can be quite detrimental to your loan approval.

# About the data

There are 2 different data sets available from Lending Club. For approved loans, the data set contains a lot more data, including repayment and loan default details. For rejected loans, we have information on the purpose of the loan, the state and zip code, and the applicant's debt to income ratio etc.

For each data set, Lending Club also separated them by dates, so our first step was to combine all the different dates into one big data set.

## Classes

Our classes fall under the column "policy code". There are 3 different classes.

- 0: rejected
- 1: approved
- 2: rejected because there were no suitable products available

Technically, if we wanted to see whether a loan application would be approved or rejected, we could combine 0 and 2. I left them as they were to see how our regression analysis would be affected.

## Features

For the current stage of the project, we will only consider whether loan applications are approved or rejected. I do believe we have enough data to look into predicting loan defaults later on, but that's for another time.

After looking through the columns on each dataset, we matched the most likely/ most useful column from the approved loans to the rejected loans, then combined the 2 into a data set on loan applications.

Our features are:

- loan amount
- debt to income ratio
- employment length
- zip code and state
  - there should be some collinearity here but removing zip code did not show any major change in the importance of state as a feature
- purpose

# Modeling

In my first round of modeling, I used numbers to represent our categorical features. This was just so I could easily take a look at the coefficients later on to determine which of the features were important to our model.

We used a logistic regression model with 3-fold cross validation - which took a little while to run because we had quite a number of instances. Looking at our coefficients:

[![feature_importance]({{ site.url }}{{ site.baseurl }}/images/loan_approval/feat_imp.png)]({{ site.url }}{{ site.baseurl }}/images/loan_approval/feat_imp.png)

Each of the classes has their own regression coefficients and they tell us which features are important to determining whether or not an instance belongs to a certain class and how each feature affects the likelihood that it belongs to a certain class.

For example, if we look at class 0 (rejection of loan application), we see that purpose has a coefficient of 0.22 and employment length has a coefficient of -0.29. What do these numbers mean? As purpose increases by 1, there is an additional 0.22 chance that the instance will be of class 0 than of another class. (Although of course increase purpose by 1 doesn't make any sense because each number represents a category.) Looking employment length makes more sense, because we encoded the number of years of employment to a corresponding number. So a coefficient of -0.29 means that when the number of years of employment increases by 1, the instance has 0.29 less chance of being classified as class 0 (i.e. it is less likely to be rejected).

## Dummy variables

So the coefficients for purpose don't make any sense now. To make sense of it, we created a model using dummy variables.

[![feature_importance_dummies]({{ site.url }}{{ site.baseurl }}/images/loan_approval/feat_imp_dummies.png)]({{ site.url }}{{ site.baseurl }}/images/loan_approval/feat_imp_dummies.png)

With dummies, the coefficients are how the chances of being classified as a certain class change between each dummy variable. For example, if the purpose of the loan was for business, there is 0.04 less chance of getting the loan rejected than if the purpose of the loan was other (base case).

# Round up

Based on our models, we can see that Lending Club is more likely to approve a loan when employment lengths are greater. We can also look at how different purposes affect the likelihood of getting a loan rejected.

Thank you for reading this post! [You can view my notebook here](https://github.com/jocelyn-ong/data-science-projects/blob/master/others/lending_club/predicting_application_approvals.ipynb){:target="_blank"}. (I wasn't able to upload all the data though, because the file sizes are too big.)

gha_language: SCSS


text:
# Automated alerts - Plaid + Open Banking + Python

This python script is designed to show you how to use the Plaid to query your transaction data.

It sends me an email each day outlining total number of transactions and details those transactions which are a credit to one of my accounts.

Why did I build this? Read [this](https://medium.com/@ravlondon/automate-your-bank-with-python-e9a9135e7923)

You need to sign-up to [Plaid](https://www.plaid.com) first which provides free developer access to your bank accounts.

repository: monkeydust/openbanking

gha_language: Python

repo_name: monkeydust/openbanking


text:
---
title: "Review on DEF CON@COMROKFLT 2018"
date: "2018-06-19T09:00:00.009Z"
category: "general"
---
So I went to the Def Con at Command ROK Fleet in Busan, jointly held with the CNFK (Commander, US Naval Forces Korea) and won the first place.

Well seriously I'm not good AT ALL, but the people who came were worse...

You know, if I go and test myself out in public whitehat contest, I would get dropped out in preliminaries. But this is inside the army.. so whatever.

The game was set up on a virtual environment.

Participants connected to virtual computers connected within the same subnet loaded upon AWS via [mRemoteNG](https://mremoteng.org/).

The whole game was set up by the company called [Sentek Global](https://en.wikipedia.org/wiki/Sentek_Global) that used [Haiku Cyber Range](https://www.haikurange.com/) to make a virtual hacking (or more precisely, 'already hacked') environment. The subnet included about 8 compromised end user devices, and they displayed certain symptoms. And we had to find those symptoms and the cause, and suggest countermeasures to be taken against them. That was how it was scored.

Seriously, if any of you penultimate students studying Computer Science came to this contest, you would have won it.

Anyways it was a good experience at least. I also met US POs and had a chance to talk with them.

branch_name: refs/heads/dev


text:
# Architecture

Imagine we have give following input: `-(1 + 3) * 5`.
The calculator outputs: `-20`. But how does it know this?

We interact with our calculator through a REPL,
a read, evaluate, print loop:

```mermaid
    flowchart LR
       A[READ user input as a string] --> B[EVALUATE input to a number] --> C[PRINT result as a string] --> A
```

Reading input and writing output are done by Java.
It gets interesting in the *evaluate* step:
we need to understand and process the input string.
This is done by combining a parser, a lexer and an evaluator:

```mermaid
    flowchart LR
x>grammar] -.-> b[Lexer]
a>input string] -.-> b
b --tokens--> c[Parser]
--parse tree--> d[Evaluator]
```

Based on our [grammar](src/main/antlr4/nl/arothuis/antlr4calculator/core/parser/Calculator.g4),
a *lexer* and *parser* are generated by ANTLR.
The lexer breaks down the given input into tokens,
which are the smallest units of meaning in a programming language.
A parser is a program that takes these tokens and builds a parse tree,
which represents the structure of the program according to the grammar of the programming language.

The code that isn't generated by ANTLR consists of two examples:
a visitor-based one and a listener-based one. For each, we've
made an *evaluator*, which evaluates the nodes of a parse tree
into a number, and a *calculator*, which is the name for our
interpreter: it consists of the entire process of lexing, parsing
and evaluating. Our *REPL* is used
to connect user input (`Reader`) with
one of our `Calculator` classes and an output (`Writer`).
We have two Main-classes to demonstrate the composition of either approach.

All together, the structure could be summarized in the following diagram:
```mermaid
classDiagram

class Reader { <<interface>> }
class Calculator { <<interface>> }
class Writer { <<interface>> }

class ListenerMain
class VisitorMain

ListenerMain ..> Repl
VisitorMain ..> Repl

class Repl
class VisitorBasedCalculator
class ListenerBasedCalculator

Repl --> Reader
Repl --> Calculator
Repl --> Writer

Calculator <|.. VisitorBasedCalculator
Calculator <|.. ListenerBasedCalculator

VisitorBasedCalculator ..> CalculationVisitor
VisitorBasedCalculator ..> CalculatorParser
VisitorBasedCalculator ..> CalculatorLexer

ListenerBasedCalculator ..> CalculationListener
ListenerBasedCalculator ..> CalculatorParser
ListenerBasedCalculator ..> CalculatorLexer

CalculationListener --|> CalculatorBaseListener
CalculationVisitor --|> CalculatorBaseVisitor~Double~

```

## What does a parse tree look like?

The input `-(1 + 3) * 5` is translated into
the following parse tree, based on our grammar:

```mermaid
flowchart TD

a[start] --- b[expr: MultiplicationOrDivision]
b --- c1[expr: Negation]
b --- c2[STAR: *]
b --- c3[expr: Number]
c3 --- d3[NUMBER: 5]
c1 --- d1[MINUS: -]
c1 --- d2[expr: Parentheses]
d2 --- e1["LPAREN: ("]
d2 --- e2[expr: AdditionOrSubtraction]
d2 --- e3["RPAREN: )"]
e2 --- f1[expr: Number]
e2 --- f2[PLUS: +]
e2 --- f3[expr: Number]
f1 --- g1[NUMBER: 1]
f3 --- g2[NUMBER: 3]
```

In order to calculate the result, we need to traverse the tree.
The traversal starts at the root node of the parse tree and visits each node.
At each node, we apply the appropriate operation based on the node's type
(either an expression or a terminal). The result of the evaluation is
collected as the traversal progresses,
and is returned when the traversal is complete.

There are two approaches to traversing the parse tree:
the *listener approach* and the *visitor approach*.

## Listener versus Visitor
In the *listener approach*
we use ANTLR's TreeWalker to walk over the nodes in the parse tree,
which we give our calculator in its walk-method.
Our ListenerBasedCalculator *listens* to events during this traversal.
We use a stack to keep track of the number we've seen last.

The *visitor approach* does not use a TreeWalker to traverse
the parse tree. Our calculator navigates through the tree itself,
by visiting the parse tree.

### How does the Visitor work?
The Visitor visits each node in the given parse tree.
This principle is based on the
[visitor pattern](https://refactoring.guru/design-patterns/visitor).

If the node is an expression, it tries to evaluate the expression
until terminals (in this grammar: *numbers*) are found.
For non-terminals (in this grammar: *operations*),
How the operations should be applied
is defined in the visitor's methods. If a node is visited that is
a number, it is converted to Double and returned.

In our grammar, numbers are our only terminals. They are
the only tokens that need to be converted into a Java datatype
(Double, for convenience's sake). Operations, if any, are then applied
to the returned values.

The order of operations is specified in the grammar.
This is a feature of ANTLR4 through the order
of possible matches for a certain production rule.
Labels have been added to the grammar to make it easier
to identify a rule and certain tokens by name, considerably
simplifying the work the Visitor has to do.

### How does the Listener work?
The Listener listens to `enter` and `exit` events
for each rule and token matched.
These events are emitted when traversing the parse tree.
This mechanism is comparable to the
[Observer pattern](https://refactoring.guru/design-patterns/observer)

An `enter` event occurs before traversing a node's children,
an `exit` event occurs after traversing a node's children.
Before dealing with the result of a non-terminal, its children
should be traversed first.

In this example, a [stack-based](https://en.wikipedia.org/wiki/Stack_(abstract_data_type))
listener is implemented.
The stack records the last number seen according to Last In First Out.
For each number traversed a Double is pushed
to the stack.
For each operation performed, the right number is popped off,
the left number is popped off and the result of the operation
is pushed back on the stack.


text:
# https://buoyant.io/2016/11/18/a-service-mesh-for-kubernetes-part-v-dogfood-environments-ingress-and-edge-routing/

```sh
kubectl apply -f linkerd-ingress.yml
kubectl apply -f node-name-test.yml
kubectl logs node-name-test
kubectl apply -f hello-world-legacy.yml
kubectl apply -f world-v2.yml
kubectl apply -f api-legacy.yml

kubectl get svc
export INGRESS_LB=<IP_ADDRESS>
# l5d          NodePort    <IP_ADDRESS>   <none>        80:32156/TCP,4141:32292/TCP,9990:30555/TCP   4m
curl -s -H "Host: www.hello.world" <IP_ADDRESS>:80
curl -s -H "Host: api.hello.world" $INGRESS_LB

curl -H "Host: www.hello.world" -H "l5d-dtab: /host/world => /srv/world-v2;" $INGRESS_LB

kubectl apply -f nginx.yml
kubectl get svc
# nginx        NodePort    <IP_ADDRESS>    <none>        80:31082/TCP
curl -s <IP_ADDRESS>
export INGRESS_LB=<IP_ADDRESS>
curl -s -H "Host: www.hello.world" $INGRESS_LB
curl -s -H "Host: api.hello.world" $INGRESS_LB
curl -H "Host: www.hello.world" -H "l5d-dtab: /host/world => /srv/world-v2;" $INGRESS_LB

curl -H "Host: www.hello.world" --cookie "special_employee_cookie=dogfood" $INGRESS_LB
curl -H "Host: www.hello.world" -H "l5d-dtab: /host/world => /srv/world-v2;" $INGRESS_LB
```

In previous installments of this series, we’ve shown you how you can use linkerd to capture top-line service metrics, transparently add TLS across service calls, and perform blue-green deploys. These posts showed how using linkerd as a service mesh in environments like Kubernetes adds a layer of resilience and performance to internal, service-to-service calls. In this post, we’ll extend this model to ingress routing.

Although the examples in this post are Kubernetes-specific, we won’t use the built-in Ingress Resource that Kubernetes provides (for this, see Sarah’s post).  In the examples below, we’ll be reaching far beyond what they provide.

STEP 1: DEPLOY THE LINKERD SERVICE MESH

Starting with our basic linkerd service mesh Kubernetes config from the previous articles, we’ll make two changes to support ingress: we’ll modify the linkerd config to add an additional logical router, and we’ll tweak the VIP in the Kubernetes Service object around linkerd. (The full config is here: linkerd-ingress.yml.)

Here’s the new ingress logical router on linkerd instances that will handle ingress traffic and route it to the corresponding services:

routers:
- protocol: http
  label: ingress
  dtab: |
    /srv                    => /#/io.l5d.k8s/default/http ;
    /domain/world/hello/www => /srv/hello ;
    /domain/world/hello/api => /srv/api ;
    /host                   => /$/io.buoyant.http.domainToPathPfx/domain ;
    /svc                    => /host ;
  interpreter:
    kind: default
    transformers:
    - kind: io.l5d.k8s.daemonset
      namespace: default
      port: incoming
      service: l5d
  servers:
  - port: 4142
    ip: <IP_ADDRESS>

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
routers:
- protocol: http
  label: ingress
  dtab: |
    /srv                    => /#/io.l5d.k8s/default/http ;
    /domain/world/hello/www => /srv/hello ;
    /domain/world/hello/api => /srv/api ;
    /host                   => /$/io.buoyant.http.domainToPathPfx/domain ;
    /svc                    => /host ;
  interpreter:
    kind: default
    transformers:
    - kind: io.l5d.k8s.daemonset
      namespace: default
      port: incoming
      service: l5d
  servers:
  - port: 4142
    ip: <IP_ADDRESS>

In this config, we’re using linkerd’s routing syntax, dtabs, to route requests from domain to service—in this case from “api.hello.world” to the api service, and from “www.hello.world” to the world service. For simplicity’s sake, we’ve added one rule per domain, but this mapping can easily be generified for more complex setups. (If you’re a linkerd config aficionado, we’re accomplishing this behavior by combining linkerd’s default header token identifier to route on the Host header, the domainToPathPfx namer to turn dotted hostnames into hierarchical paths, and the io.l5d.k8s.daemonset transformer to send requests to the corresponding host-local linkerd.)

We’ve added this ingress router to every linkerd instance—in true service mesh fashion, we’ll fully distribute ingress traffic across these instances so that no instance is a single point of failure.

We also need modify our k8s Service object to replace the outgoing VIP with aningress VIP on port 80. This will allow us to send ingress traffic directly to the linkerd service mesh—mainly for debugging purposes, since the this traffic will not be sanitized before hitting linkerd. (In the next step, we’ll fix this.)

The Kubernetes change looks like this:

---
apiVersion: v1
kind: Service
metadata:
  name: l5d
spec:
  selector:
    app: l5d
  type: LoadBalancer
  ports:
  - name: ingress
    port: 80
    targetPort: 4142
  - name: incoming
    port: 4141
  - name: admin
    port: 9990

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
---
apiVersion: v1
kind: Service
metadata:
  name: l5d
spec:
  selector:
    app: l5d
  type: LoadBalancer
  ports:
  - name: ingress
    port: 80
    targetPort: 4142
  - name: incoming
    port: 4141
  - name: admin
    port: 9990

All of the above can be accomplished in one fell swoop by running this command to apply the full linkerd service mesh plus ingress Kubernetes config:

$ kubectl apply -f https://raw.githubusercontent.com/linkerd/linkerd-examples/master/k8s-daemonset/k8s/linkerd-ingress.yml

1
2
$ kubectl apply -f https://raw.githubusercontent.com/linkerd/linkerd-examples/master/k8s-daemonset/k8s/linkerd-ingress.yml

STEP 2: DEPLOY THE SERVICES

For services in this example, we’ll use the same hello and world configs from the previous blog posts, and we’ll add two new services: an api service, which calls both hello and world, and a new version of the world service, world-v2, which will return the word “earth” rather than “world”—our growth hacker team has assured us their A/B tests show this change will increase engagement tenfold.

The following commands will deploy the three hello world services to the default namespace. These apps rely on the nodeName supplied by the Kubernetes downward API to find Linkerd. To check if your cluster supports nodeName, you can run this test job:

kubectl apply -f https://raw.githubusercontent.com/linkerd/linkerd-examples/master/k8s-daemonset/k8s/node-name-test.yml
1
kubectl apply -f https://raw.githubusercontent.com/linkerd/linkerd-examples/master/k8s-daemonset/k8s/node-name-test.yml
And then looks at its logs:

kubectl logs node-name-test
1
kubectl logs node-name-test
If you see an ip, great! Go ahead and deploy the hello world app using:

$ kubectl apply -f https://raw.githubusercontent.com/linkerd/linkerd-examples/master/k8s-daemonset/k8s/hello-world.yml
$ kubectl apply -f https://raw.githubusercontent.com/linkerd/linkerd-examples/master/k8s-daemonset/k8s/api.yml
$ kubectl apply -f https://raw.githubusercontent.com/linkerd/linkerd-examples/master/k8s-daemonset/k8s/world-v2.yml
1
2
3
$ kubectl apply -f https://raw.githubusercontent.com/linkerd/linkerd-examples/master/k8s-daemonset/k8s/hello-world.yml
$ kubectl apply -f https://raw.githubusercontent.com/linkerd/linkerd-examples/master/k8s-daemonset/k8s/api.yml
$ kubectl apply -f https://raw.githubusercontent.com/linkerd/linkerd-examples/master/k8s-daemonset/k8s/world-v2.yml

If instead you see a “server can’t find …” error, deploy the hello-world legacy version that relies on hostIP instead of nodeName:

$ kubectl apply -f https://raw.githubusercontent.com/linkerd/linkerd-examples/master/k8s-daemonset/k8s/hello-world-legacy.yml
$ kubectl apply -f https://raw.githubusercontent.com/linkerd/linkerd-examples/master/k8s-daemonset/k8s/api-legacy.yml
$ kubectl apply -f https://raw.githubusercontent.com/linkerd/linkerd-examples/master/k8s-daemonset/k8s/world-v2.yml
1
2
3
$ kubectl apply -f https://raw.githubusercontent.com/linkerd/linkerd-examples/master/k8s-daemonset/k8s/hello-world-legacy.yml
$ kubectl apply -f https://raw.githubusercontent.com/linkerd/linkerd-examples/master/k8s-daemonset/k8s/api-legacy.yml
$ kubectl apply -f https://raw.githubusercontent.com/linkerd/linkerd-examples/master/k8s-daemonset/k8s/world-v2.yml

At this point we should be able to test the setup by sending traffic through the ingress Kubernetes VIP. In the absence of futzing with DNS, we’ll set a Host header manually on the request:

$ INGRESS_LB=$(kubectl get svc l5d -o jsonpath="{.status.loadBalancer.ingress[0].*}")
$ curl -s -H "Host: www.hello.world" $INGRESS_LB
Hello (<IP_ADDRESS>) world (<IP_ADDRESS>)!!
$ curl -s -H "Host: api.hello.world" $INGRESS_LB
{"api_result":"api (<IP_ADDRESS>) Hello (<IP_ADDRESS>) world (<IP_ADDRESS>)!!"}

1
2
3
4
5
6
$ INGRESS_LB=$(kubectl get svc l5d -o jsonpath="{.status.loadBalancer.ingress[0].*}")
$ curl -s -H "Host: www.hello.world" $INGRESS_LB
Hello (<IP_ADDRESS>) world (<IP_ADDRESS>)!!
$ curl -s -H "Host: api.hello.world" $INGRESS_LB
{"api_result":"api (<IP_ADDRESS>) Hello (<IP_ADDRESS>) world (<IP_ADDRESS>)!!"}

Or if external load balancer support is unavailable for the cluster, use hostIP:

$ INGRESS_LB=$(kubectl get po -l app=l5d -o jsonpath="{.items[0].status.hostIP}"):$(kubectl get svc l5d -o 'jsonpath={.spec.ports[0].nodePort}')
1
$ INGRESS_LB=$(kubectl get po -l app=l5d -o jsonpath="{.items[0].status.hostIP}"):$(kubectl get svc l5d -o 'jsonpath={.spec.ports[0].nodePort}')
Success! We’ve set up linkerd as our ingress controller, and we’ve used it to route requests received on different domains to different services. And as you can see, production traffic is hitting the world-v1 service—we aren’t ready to bring world-v2out just yet.

STEP 3: A LAYER OF NGINX

At this point we have functioning ingress. However, we’re not ready for production just yet. For one thing, our ingress router doesn’t strip headers from requests, which means that external requests may include headers that we do not want to accept. For instance, linkerd allows setting the l5d-dtab header to apply routing rules per-request. This is a useful feature for ad-hoc staging of new services, but it’s probably not appropriate calls from the outside world!

For example, we can use the l5d-dtab header to override the routing logic to use world-v2 rather than the production world-v1 service the outside world:

$ curl -H "Host: www.hello.world" -H "l5d-dtab: /host/world => /srv/world-v2;" $INGRESS_LB
Hello (<IP_ADDRESS>) earth (<IP_ADDRESS>)!!

1
2
3
$ curl -H "Host: www.hello.world" -H "l5d-dtab: /host/world => /srv/world-v2;" $INGRESS_LB
Hello (<IP_ADDRESS>) earth (<IP_ADDRESS>)!!

Note the earth in the response, denoting the result of the world-v2 service. That’s cool, but definitely not the kind of power we want to give just anyone!

We can address this (and other issues, such as serving static files) by adding nginx to the mix. If we configure nginx to strip incoming headers before proxying requests to the linkerd ingress route, we’ll get the best of both worlds: an ingress layer that is capable of safely handling external traffic, and linkerd doing dynamic, service-based routing.

Let’s add nginx to the cluster. We’ll configure it using this nginx.conf. We’ll use the proxy_pass directive under our virtual servers www.hello.world and api.hello.world to send requests to the linkerd instances, and, for maximum fanciness, we’ll strip linkerd’s context headers using the more_clear_input_headers directive (with wildcard matching) provided by the Headers More module.

(Alternatively, we could avoid third-party nginx modules by using nginx’sproxy_set_header directive to clear headers. We’d need separate entries for each l5d-ctx- header as well as the l5d-dtab and l5d-sample headers.)

Note that as of linkerd 0.9.0, we can clear incoming l5d-* headers by setting clearContext: true on the ingress router server. However, nginx has many features we can make use of (as you’ll see presently), so it is still valuable to use nginx in conjunction with linkerd.

For those of you following along at home, we’ve published an nginx Docker image with the Headers More module installed (Dockerfile here) as buoyantio/nginx:1.11.5. We can deploy this image with our config above using this Kubernetes config:

$ kubectl apply -f https://raw.githubusercontent.com/linkerd/linkerd-examples/master/k8s-daemonset/k8s/nginx.yml

1
2
$ kubectl apply -f https://raw.githubusercontent.com/linkerd/linkerd-examples/master/k8s-daemonset/k8s/nginx.yml

After waiting a bit for the external IP to appear, we can test that nginx is up by hitting the simple test endpoint in the nginx.conf:

$ INGRESS_LB=$(kubectl get svc nginx -o jsonpath="{.status.loadBalancer.ingress[0].*}")
$ curl $INGRESS_LB
200 OK

1
2
3
4
$ INGRESS_LB=$(kubectl get svc nginx -o jsonpath="{.status.loadBalancer.ingress[0].*}")
$ curl $INGRESS_LB
200 OK

Or if external load balancer support is unavailable for the cluster, use hostIP:

$ INGRESS_LB=$(kubectl get po -l app=nginx -o jsonpath="{.items[0].status.hostIP}"):$(kubectl get svc nginx -o 'jsonpath={.spec.ports[0].nodePort}')
1
$ INGRESS_LB=$(kubectl get po -l app=nginx -o jsonpath="{.items[0].status.hostIP}"):$(kubectl get svc nginx -o 'jsonpath={.spec.ports[0].nodePort}')
We should be able to now send traffic to our services through nginx:

$ curl -s -H "Host: www.hello.world" $INGRESS_LB
Hello (<IP_ADDRESS>) world (<IP_ADDRESS>)!!
$ curl -s -H "Host: api.hello.world" $INGRESS_LB
{"api_result":"api (<IP_ADDRESS>) Hello (<IP_ADDRESS>) world (<IP_ADDRESS>)!!"}

1
2
3
4
5
$ curl -s -H "Host: www.hello.world" $INGRESS_LB
Hello (<IP_ADDRESS>) world (<IP_ADDRESS>)!!
$ curl -s -H "Host: api.hello.world" $INGRESS_LB
{"api_result":"api (<IP_ADDRESS>) Hello (<IP_ADDRESS>) world (<IP_ADDRESS>)!!"}

Finally, let’s try our header trick and attempt to communicate directly with the world-v2 service:

$ curl -H "Host: www.hello.world" -H "l5d-dtab: /host/world => /srv/world-v2;" $INGRESS_LB
Hello (<IP_ADDRESS>) world (<IP_ADDRESS>)!!

1
2
3
$ curl -H "Host: www.hello.world" -H "l5d-dtab: /host/world => /srv/world-v2;" $INGRESS_LB
Hello (<IP_ADDRESS>) world (<IP_ADDRESS>)!!

Great! No more earth. Nginx is sanitizing external traffic.

STEP 4: TIME FOR SOME DELICIOUS DOGFOOD!

Ok, we’re ready for the good part: let’s set up a dogfood environment that uses theworld-v2 service, but only for some traffic!

For simplicity, we’ll target traffic that sets a particular cookie,special_employee_cookie. In practice, you probably want something more sophisticated than this—authenticate it, require that it come from the corp network IP range, etc.

With nginx and linkerd installed, accomplishing this is quite simple. We’ll use nginx to check for the presence of that cookie, and set a dtab override header for linkerd to adjust its routing. The relevant nginx config looks like this:

if ($cookie_special_employee_cookie ~* "dogfood") {
  set $xheader "/host/world => /srv/world-v2;";
}

proxy_set_header 'l5d-dtab' $xheader;

1
2
3
4
5
6
if ($cookie_special_employee_cookie ~* "dogfood") {
  set $xheader "/host/world => /srv/world-v2;";
}

proxy_set_header 'l5d-dtab' $xheader;

If you’ve been following the steps above, the deployed nginx already contains this configuration. We can test it like so:

$ curl -H "Host: www.hello.world" --cookie "special_employee_cookie=dogfood" $INGRESS_LB
Hello (<IP_ADDRESS>) earth (<IP_ADDRESS>)!!

1
2
3
$ curl -H "Host: www.hello.world" --cookie "special_employee_cookie=dogfood" $INGRESS_LB
Hello (<IP_ADDRESS>) earth (<IP_ADDRESS>)!!

The system works! When this cookie is set, you’ll be in dogfood mode. Without it, you’ll be in regular, production traffic mode. Most importantly, dogfood mode can involve new versions of services that appear anywhere in the service stack, even many layers deep—as long as service code forwards linkerd context headers, the linkerd service mesh will take care of the rest.

Conclusion

In this post, we saw how to use linkerd to provide powerful and flexible ingress to a Kubernetes cluster. We’ve demonstrated how to deploy a nominally production-ready setup that uses linkerd for service routing. And we’ve demonstrated how to use some of the advanced routing features of linkerd to decouple the traffic-serving topology from the deployment topology, allowing for the creation of dogfood environments without separate clusters or deploy-time complications.
Note: there are a myriad of ways to deploy Kubernetes and different environments support different features. Learn more about deployment differences here.

For more about running linkerd in Kubernetes, or if you have any issues configuring ingress in your setup, feel free to stop by our linkerd community Slack, ask a question on Discourse, or contact us directly!

repository: reymont/ReadAndLearn

gha_language: Jupyter Notebook

repo_name: reymont/ReadAndLearn