text:
---
author: admin
comments: true
date: 2015-01-12 20:37:49+00:00
excerpt: '[:en]A formal model of space market equilibrium dynamics and cyclicality,
  from "Commercial Real Estate Analysis and Investments" by Geltner and Miller, written
  in R as a Shiny app.[:]'
layout: post
link: http://kaftanowicz.com/visualization-space-market-model/
slug: visualization-space-market-model
title: (English) Visualization of space market model
wordpress_id: 174
---

[:en]I'm currently reading _Commercial Real Estate Analysis and Investments_ by Geltner and Miller. I'll write a full review later; right now I'll focus on section 6.2, titled _Formal model of space market equilibrium dynamics and cyclicality_.

A reader gets warned (by a footnote and an asterisk before section number) that this part is more advanced and difficult. This post is my attempt at making it easier by using more diagrams, more visualization and less scary looking Greek letters (I'll switch to suggestively named variables instead). I hope that my different approach will make this topic easier to understand for at least some people.

<!-- more -->We are trying to make a simple mathematical model of some space market, for example the market for office space in some city. In year 1 there is the stock of office space ready to be leased for a particular rent. There is also some need for office space, represented by the number of office workers. This need caused demand. Some space gets occupied, the rest stays vacant. Let's assume that no construction occurs in the first year.

What happens in following years?
(I) Developers take a look at rents that were a few years ago. (How many years exactly is dictated by so-called construction lag). If rents were higher than some trigger rent, developers build more office space.
(II) This new construction is added to stock ready to be leased.
(III) The amount of occupied space changes, satisfying the demand from previous year.
(IV) Vacancy rate changes.
(V) Rent changes, too - if vacancy rate is higher than some natural vacancy rate prevailing in this particular market, rents will fall, otherwise they will rise.
(VI) Need increases as the city and the number of its office workers grows.
(VII) Demand changes to reflect increased need and change in rent.
These steps are then repeated specified number of times.

If we can write it, maybe we can draw it, too?

[![Space Market Flowchart](http://kaftanowicz.com/wp-content/uploads/2015/01/Space-Market-Flowchart.png)](http://kaftanowicz.com/wp-content/uploads/2015/01/Space-Market-Flowchart.png)

Now it looks much more clearly. We see the main cycle, representing the demand side of the market: demand -> occupied space -> vacancy rate -> rent -> demand, with input from need influencing demand. We also see additional cycle of supply side: rent -> construction -> stock -> vacancy rate. They are joined by vacancy rate, influenced both by occupied space and stock, and rent, which influences both demand and construction activity.

We can write it out as equations, too:

`if (Rent[t-construction.lag] > trigger.rent){
  Construction[t] = supply.sensitivity*(Rent[t-construction.lag]-trigger.rent)
}else{
  Construction[t] = 0} # (I)
Stock[t] = Stock[t-1] + Construction[t] # (II)
OccupiedSpace[t] = min(Demand[t-1], Stock[t]) # (III)
VacancyRate[t] = (Stock[t] - OccupiedSpace[t])/Stock[t] # (IV)
Rent[t] = Rent[t-1]*(1-rent.sensitivity*(VacancyRate[t] - NaturalVacancyRate)/NaturalVacancyRate) # (V)
Need[t] = Need[t-1]*(1+need.growth.rate) # (VI)
Demand[t] = demand.intercept -demand.sensitivity*Rent[t] +demand.per.need*Need[t] # (VII)
`

Something[t] means the value of something in year t. Numerals are there to help relate equations to sentences from a few paragraphs before.

Those equations tell us more than a diagram, but are harder to visualize. It would be great to see how market dynamics changes when we change some parameters, would't it?

[Here's an app](https://kaftanowicz.shinyapps.io/SpaceMarketModel/) doing exactly this that I've written in R, using the Shiny framework. We can play with parameters and observe their influence on model. [Here is the code](https://github.com/kaftanowicz/spacemarketmodel).

This model is very simple and based on strong assumptions - and by "strong", I mean "questionable" - such as:
- neither developers nor space users nor landlords can predict changes in rent and only react to its current or past values,
- occupied space changes every year so that all demand from previous year gets satisfied,
- no office space available on the market ever gets removed from the market; the stock can only stay the same or increase thanks to construction works.
It is, hovewer, interesting model and as book's authors note, it is useful for getting the general idea of real estate market dynamics.

Initial values are those that the authors have chosen, with few improvements (supply and demand sensitivity should be 0.3*10^6, not 0.3 - otherwise we won't get that nice graphs with fluctuating construction activity, rent and vacancy rates; also, trigger rent was not specified in the book). However, by tweaking some of the parameters, we can model even more interesting market behaviour. Try it yourself![:]

extension: markdown


text:
# VFL Logistic Regression with HE Example

This example show you how to create an end-to-end VFL Logistic Regression application with 2 clients and 1 server, using Homomorphic Encryption to protect data passing to FlServer on BigDL PPML.
The targets and outputs of each client's linear will be protected by Homomorphic Encryption, clients will encrypt targets and outputs to cipher text, then send the cipher texts to FL server. FLServer will compute loss and grad using cipher texts, return the cipher-text results back to clients. After receive the cipher texts, Clients can use their private secrets to decrypt the returned cipher texts to loss and grad. The plain-text loss and grad will be given back linear to compute linear's grad, the rest work is just normal LR training.

### Data
We use [Census Income Dataset](https://archive.ics.uci.edu/ml/datasets/Census+Income) data in this example.

To simulate the scenario of two parties, we use select different features of Census data.

The original data has 15 columns. In preprocessing, some new feature are created from the combinations of some existed columns.

* data of client 1: `age`, `education`, `occupation`, cross columns: `edu_occ`, `age_edu_occ`
* data of client 2: `relationship`, `workclass`, `marital_status`

### Download BigDL assembly

Download BigDL assembly all in one jar from [BigDL-Release](https://bigdl.readthedocs.io/en/latest/doc/release.html), file name is bigdl-assembly-spark_[version]-jar-with-all-dependencies.jar

### Generate secret
`GenerateCkksSecret` will generate two secret file in given folder, one is `all_secret`, another is `compute_secret`.
`all_secret`: contains all secret for both encryption and computing.
`compute_secret`: contains only computing secret for compute the cipher text, used on FLServer.
```bash
java -cp bigdl-assembly-[version]-jar-with-all-dependencies.jar com.intel.analytics.bigdl.ppml.fl.example.ckks.GenerateCkksSecret [folder path]
```
This secret generation is not for production use, secret should be protected by Key Management Service.

### Start FLServer
Before starting server, modify the config file, `ppml-conf.yaml`, this application has 2 clients globally, and set the absolute path to ckks secret. So use following config:
```yaml
clientNum: 2
ckksSercetPath: /[absolute folder path]/compute_secret
```
Then start FLServer at server machine
```bash
java -cp bigdl-assembly-[version]-jar-with-all-dependencies.jar com.intel.analytics.bigdl.ppml.fl.FLServer
```

## Start Local Trainers
Start the local Logistic Regression trainers at 2 training machines
```bash
java -cp bigdl-assembly-[version]-jar-with-all-dependencies.jar com.intel.analytics.bigdl.ppml.fl.example.ckks.VflLogisticRegressionCkks
    -d [path to adult dataset]
    -i 1
    -s [path to all_secret]
# change -i 1 to -i 2 at client-2
```

The example will train the data and evaluate the training result.

repository: analytics-zoo/BigDL

branch_name: refs/heads/main

extension: md

gha_language: Jupyter Notebook

license_type: permissive

repo_name: analytics-zoo/BigDL


text:
# Herald: Intrusion Detection System using IR and ML

Herald - _noun_; a harbinger of events or things.

## Overview

Herald is an intrusion detection system using infrared radiation and anomaly detection using machine learning and Bolt IoT wifi module. The IoT device makes use of IR sensor and cloud technology for alerting the stakeholders of the detected anomaly.

## Things used in this project

![Hardware](.github/hardware.jpg)
Most of the hardware components can be found in the Bolt IoT starter kit and digital components are relatively inexpensive.

### Hardware Components

- Bolt IoT Wifi Module - 1 unit
- Male-Male jumper wires - 3 units
- Mini Breadboard - 1 unit
- FC-51 or TCRT5000 Infrared Radiation Sensor - 1 unit
- Powerbank capable of providing at least 5V - 1 unit
- USB cable to power the module - 1 unit
- Bolt IoT Starter box (for containing) - 1 unit

### Software/digital requirements

- Python
- Cloud VPS
- SMTP Credentials
- Bolt Cloud Pro Account (for better API rate limit, not required for testing)

## Story

### Introduction

An intrusion detection system monitors an area for malicious activity or privacy violations. These are reported to the person(s) whom it concerns via an alert system. It can protect your home/office by detection intrusion.

Our system utilizes machine learning to adjust the alert parameters and emails the concerned entities via Simple Mail Transfer Protocol.

### FC-51 or TCRT5000. Which one to use?

![TCRT5000 and FC-51](.github/sensors.png)
Both IR sensors are capable to provide valid results for this experiment however, for anomaly detection, the TCRT5000 should work better than FC-51. The reason being that TCRT5000 can provide both analog and digital output depending upon the proximity of the object and the FC-51 sensor can only provide either HIGH or LOW output depending upon the detection and doesn't consider the proximity of the object.

While creating this project, I tried to purchase the TCRT5000 sensor but due to COVID-19 lockdown and restrictions on online shopping, I was unable to purchase it. So, I had to settle with an FC-51 sensor. Ideally, I should have used a TCRT5000.

I have written this and the code with compatibility of both sensors and either can be used without changing the code and connections.

### Procedure

#### Setting up the hardware

Step 1: Connect the sensor to breadboard. This is optional. In case you want to connect the sensor directly to the Bolt IoT Wifi Module, you may do so.

![breadboard and sensor](.github/sensor_breadboard.jpg)

Step 2: Connect the male-male jumper wires to the corresponding pins on the breadboard.

![jumper with breadboard and sensor](.github/with_jumper.jpg)

Step 3: Connect the other end of jumper wires to the Bolt module. Connection should match as:

```
Sensor to Bolt Module connections:
VCC <-> 5V
GND <-> GND
OUT <-> A0
```

![bolt](.github/connected_bolt.jpg)

Step 4: Power on the bolt module via a power bank or direct connection of at least 5V.

Step 5: Verify connectivity on bolt module and connections. The IR sensor should show an LED for power and another LED may light up in case it detects some object in proximity. You may adjust the sensitivity by adjusting the potentiometer.

![](.github/compl.jpg)

#### Loading the software

The code is written in the Python language and is coded to be suitable with both types of hardware described above. You will need to install Python which can be done easily (just google it if you don’t have it) and then install the Bolt IoT library via pip.

For deployment purpose, you will need a VPS for hosting the script and SMTP credentials for emailing. You may create a VPS via [DigitalOcean](https://m.do.co/c/c50ea001b767) or AWS and use Gmail credentials for sending emails (you should create a new Gmail account for this instead of using a personal account). However, if you are just trying the project, you may skip these and run on your PC.

Step 1: Log in to your server

Step 2: Assuming you already have python3 installed, run this command to install the Bolt IoT library

```bash
$ pip3 install boltiot
```

Step 3: Clone the git repository containing the code and navigate to it

```bash
$ git clone https://github.com/muhammadmuzzammil1998/herald; cd herald
```

![git clone](.github/git_clone.png)

#### Configuring the system

After cloning the repository, you have to edit the configuration file `config.py` as per your own configurations and requirements. The contents of config.py are used in the main code and have description along the line as well.

You might want to adjust the api_limit_margin and api_limit_per_minute as per your bolt account or use the pro account.

```bash
$ nano config.py # To edit the configurations
```

```python
# API and Device configurations #

api_key = ""              #Bolt user API key
device_id = ""            #Bolt device ID
cooldown = 10             #Cooldown after an unsuccessful api call to device in seconds
frame_size = 5            #Frame size for computing z-score
factor = 2                #Factor for computing z-score
api_limit_per_minute = 20 #Ratelimit for the API requests per minute
api_limit_margin = 1      #API calls to leave as margin
interval = 60 / (api_limit_per_minute - api_limit_margin) #Optional edit: Interval for checking status

# Email configurations #

smtp_server = ":587"  #SMTP server with port in format smtp.gmail.com:587
sender_name = "Herald - the intrusion detector" #Sender's name to display
sender_email = ""     #Username for SMTP server
sender_password = ""  #Password for SMTP server
recipients = ""       #Single recipient in string or a list of recipients in array form
```

- api_key: You can obtain it from cloud.boltiot.com
- device_id: Same as cloud.boltiot.com
- cooldown: Time to pause after an unsuccessful api call to device
- frame_size: Used to calculate the boundary for data
- factor: Used for anomaly detection
- api_limit_per_minute: Ratelimit for the API requests per minute
- api_limit_margin: API calls to leave for other stuff
- interval: Automatically calculated as per above two parameters
- smtp_server: Address of your SMTP server with port. We are using TLS encryption with 587 port. Gmail uses smtp.gmail.com:587.
- sender_name: Name to display on email alert
- sender_password: Password for your gmail account
- recipients: Email addresses of the stakeholders who should be informed. Can be string or list of strings.

#### Starting Herald

![final](.github/final_build.jpg)

Ensure that you have correctly connected the pins as VCC to 5V, GND to GND, and OUT to A0.

Type the following command to start the program. Make sure your Bolt module is connected to the internet and cloud, as indicated by the blue and green LEDs on wifi module and LED on the sensor module.

```bash
$ python3 main.python3
```

![execution](.github/exec.png)

As you can see from the above output, an alert is sent when bounds changed. The bounds are calculated using Machine Learning so that minor object detection, such as curtain flowing in front, can be handled without triggering a false alert.

Bounds are calculated using z-score analysis for anomaly detection as:

![zscore](.github/z.png)

### Code explanation

#### z_score

z_score function takes in the dataset, frame size, and the factor to adjust the boundary of the given curve. It returns the range in which the next data point should fall.

#### sensor_data

sensor_data takes in the pin to read from and returns the value read when successful. In case there is an error, the function also deals with it.

#### verify_dataset

To ensure that we have sufficient data to generate boundaries for anomaly detection.

#### Why read analog instead of digital read?

The reason here for opting for analog is that the, as discussed earlier, I wished to build this with an advanced sensor but could not get my hands on it. Therefore, to be compatible with both sensors, I chose to read analog input.

## Schematics

![schematics](.github/diagram.png)

## Conclusion

To conclude, we built an intrusion detection system using Bolt IoT which was able to sense the presence when passing through. I installed the device in my home and the device was powered by a power bank. It worked and I received emails when passing through the door.

license: MIT


text:
## Jupyter Notebook Broker App Example

This example shows how to deploy a per-user Jupyter Notebook.

## Setup

1. Clone the source repo and change to the examples directory:

```bash
gcloud source repos clone kube-pod-broker --project=cloud-ce-pso-shared-code
```

```bash
cd kube-pod-broker/examples
```

## Option 1 - Deployment with GitOps

1. Run script to create and configure gitops Cloud Source Repository for the app:

```bash
cd jupyter-notebook
```

```bash
./jupyter-notebook-gitops-init.sh
```

> Follow the prompts to create the repository and trigger the cloud build.

2. Wait for Cloud Build to complete.

3. Navigate to the broker web interface and launch the app.

4. Make any changes to the app and push them to the `gitops` remote to deploy them to the cluster.

5. If you want to deploy to clusters in other regions, edit the `cloudbuild.yaml` file.

## Option 2 - Deployment with kubectl

1. Deploy the manifests and BrokerAppConfig custom resource:

```bash
kubectl apply -k jupyter-notebook/
```

2. Navigate to the broker web interface and launch the app.

repository: GoogleCloudPlatform/selkies

branch_name: refs/heads/master

gha_language: Go

repo_name: GoogleCloudPlatform/selkies


text:
# Ways to consume text with Tensorflow Decision Forest models

This is a documentation Markdown page. For more information, see the Markdown Reference
(go/documentation-reference) and the documentation Style Guide (go/documentation-style).

[TOC]

#### Background

Decision forest learning algorithms work differently than gradient descent based
models like neural networks or linear predictors. These differences manifest
themselves across a variety of modeling decisions, but are especially pronounced
when a dataset contains variable length categorical features, like tokenized
text features, which tend to require architecture-specific feature engineering.
This guide outlines the tradeoffs between different feature engineering
strategies for text features in decision forest models.

In the following sections, we will refer to a dataset with these features, and
assume we are predicting whether a user is going to purchase a specific product:

   Feature

   User ID

   Prior Purchases

   Product Title

   Product Description

   Example data

   1234

   [“TV”, “Vacuum”]

   “WiFi Router”

   “This router is …”

\
In this example, “Prior Purchases” is a sparse text feature (or a set of
categorical items). “Product Title” is as well, but is not tokenized in this
example. “Product Description” is a natural language feature, which has
different properties than the other features, because we expect the vocabulary
to be large (or unbounded), for word-order to matter, and to have other semantic
and lexical properties inherent to the language. The strategies we describe
below are appropriate for all of these features, but will have different
tradeoffs for each one.

#### Quick Reference

The best solution, if training and inference cost is not a concern, is to use
both categorical-sets and pre-trained embeddings for each text feature, since
they have complementary strengths and weaknesses. We recommend this unless one
of the constraints mentioned below are present.

   Inference Speed

   Training Speed

   Ability to memorize token <> label relationships

   Generalization

   Multiple Categoricals

   Fastest (++)

   Fastest (++)

   Limited

   Limited (+)

   multi-hot

   Fast (+)

   Fast (assuming relatively small vocab size) (++)

   Good

   Limited (+)

   Categorical-sets

   Fastest (+++)

   Slower (+)

   Best

   Limited (++)

   embedding

   Slowest (assuming non-trivial encoder ops, like matrix multiplications) (+ to +++)

   Fastest (assuming vocab size >> embedding dimension) (+++)

   Bad

   Good (++ to +++)

#### N-Grams

N-grams (e.g. {"the", "cat", "is", "blue"} -> {"<start> the", "the cat", "cat
is", "is blue", "blue <end>"}) can be beneficial in many cases, and capture
local lexical information. They are supported in all of the methods below, but
come at the cost of a dramatically larger vocabulary size, which can make them
impractical due to training cost.

### Discouraged Strategies

#### One-hot / Multi-hot encoding / Bag of Words

One-hot encoding is a classic strategy for densifying sparse text. Here we
assume an extension where a sparse text feature is represented by either a
multi-hot (1s for all contained tokens) or count-based vectorization (the count
for each token in the vocabulary).

For example, if the vocabulary is 4 items, and indexed like [“TV”, “Vacuum”,
“Wifi”, “Router”], the feature “Prior Purchases” would be a dense vector <1,
1, 0, 0>. If counts were taken into account and the feature was [“TV”, “TV”,
“Vacuum”], it would be <2, 1, 0, 0>.

**Pros**

*   Since decision forest splits are learned on individual features, this is
    less expensive at training time than categorical sets.
*   Unlike the former, does not need any clipping or padding, and tokens have
    the same semantics across examples (i.e. “TV” will be constant across splits
    regardless of position).

**Cons**

*   This strategy often leads to highly unbalanced and sparse splits, which can
    make DF learning algorithms either slower to converge or subpar. This is
    because:
    *   More splits are needed to learn the same information
    *   Highly sparse trees generalize worse than balanced trees, usually
        resulting in a less accurate model.
*   Does not take into account positional information. This may hurt performance
    for natural language features.
*   Learning numerical splits on categorical data is sub-optimal; there are
    optimizations for finding categorical splits that are not leveraged here.
*   The training computational complexity scales linearly with the number of
    vocabulary items (which will each be consumed as a numerical feature). In
    practice, unless the dataset is very small (in which case large vocabularies
    may encourage overfitting), this makes vocabularies of > 5k items very slow
    to train.
*   The training memory consumption will be 1 byte (for one-hot) or 4 bytes (for
    counts) per vocabulary item per example, since at indexing time, the data
    will be stored as a dense version of the sparse data. This can grow
    prohibitively large for larger vocabularies and datasets.

#### Multiple Categorical Features with a fixed length

Since categorical features can be efficiently learned by decision forest
algorithms, one natural way to consume sparse tokens is to pad / clip such that
there are a fixed number of input tokens per example, and each token position is
a separate and independent feature. In the example above, if “Prior Purchases”
has at most 5 tokens, we can create features f1...f5 representing tokens 1-5,
and discard any tokens > 5, and add missing values for examples where there are
< 5.

**Pros**

*   This is efficient to train.
*   This may not hurt model quality if there is a low variance in the number of
    tokens per example, and the tokens are independent.
*   This may capture additional semantics (like purchase order in the example)
    more naturally than other methods.

**Cons**

*   Adds semantics onto “missing” padded tokens that will serve as noise to the
    model. This will be especially pronounced if there is a large variance in
    the number of tokens per example, which may happen for example with the
    “Product Description” feature.
*   The learned trees will be highly sensitive to ordering, i.e. if the feature
    is [“A”, “B”] the prediction will be different than the prediction for [“B”,
    “A”], and if the latter was never seen in the data, the model will be unable
    to generalize from the former. In general, this will require much more data
    to learn position invariance.
*   By default, each token will be represented by a feature with a different
    vocabulary. Even if you force the implementation to consider the same set of
    vocabulary items per feature, f1=”TV” will be a different vocabulary item
    than f2=”TV.” This means the algorithm will be less efficient in learning
    the relationship between the token “TV” and the label -- it will have to
    learn it separately for each token position.

### Better Strategies

#### Categorical Sets

Categorical Sets (https://arxiv.org/pdf/2009.09991.pdf) are TF-DFs default
feature representation for sparse text. A categorical set is effectively a bag
of words, ignoring duplicates and ordering. For example, the feature “The TV is
the best” would be represented by the categorical set {“best”, “is”, “the”,
“TV}.

The native categorical set splitter, according to benchmarks on a variety of
tasks (see paper), usually outperforms multi-hot and fixed-length categorical
features. In theory, both categorical set splits and boolean splits on one-hot
encoded features can learn the same information. However, take the following
example, where the tree is trying to learn the following function:

```
if description contains “high” AND “speed” AND “gaming”:
  return True
```

In this case, the native categorical set splitter will learn 1 split, where
{“high”, “speed”, “gaming”} => True.

A one hot representation would require 3 splits, on “high”, “split”, and
“gaming,” and would need to find reasonable leaf nodes for all possible
disjunctions of those categories (i.e. “high” and not “speed”). In practice,
one-hot encoding leads to highly unbalanced trees that cannot be optimized
efficiently by the best performing decision forest learning algorithms.

Pros

*   Best at learning bag-of-words information for decision forests.
*   Highly efficient to serve (can be served with QuickScorer, which can serve
    large trees in up to sub-microsecond-per-example time). The serving time
    complexity is linear in the number of items in each example’s categorical
    set, which in practice is much smaller than the vocabulary size.
*   Optimizes a single vocabulary per feature, so semantics are shared.

Cons

*   The cost of training a categorical set split scales with num\_examples \*
    vocab size, so similar to the one-hot algorithm, the trainable vocabulary
    size can be fairly small (N thousand) in practical settings. Note that this
    training speed can be improved by adjusting the sampling fraction of the
    greedy algorithm, but it may achieve sub-optimal quality.

#### Embeddings

Neural Networks have displayed state of the art performance on a variety of NLP
tasks, and pre-trained embeddings consumed as numerical features empirically
also work well with decision forest algorithms, despite the features being used
very differently internally. Note that here we refer to “embedding” as any
neural network encoding, e.g. the output of transformer / convolutional /
recurrent layers.

Using pre-trained embeddings works well with neural networks in part because the
initialization of a vector space where similar tokens or sentences are close in
euclidean space has shown to transfer well across NLP tasks, and the gradients
from that initialization are smaller and faster to converge than a fully random
initialization. However, decision trees use embeddings as individual numeric
features, and learn axis-aligned partitions of those individual features[^1].
This means it is near impossible to utilize the same semantic information -- a
dot product or a matrix multiplication, for example, cannot be represented with
a set of axis-aligned splits. Furthermore, unlike neural networks, which can
update the embeddings through gradient descent during training, the default
decision forest learning algorithms are non-differentiable, meaning that the
embeddings must stay frozen. Note that there is work
(https://arxiv.org/pdf/2007.14761.pdf, for example) on differentiable decision
forests. However, perhaps in part because in practice not all the bits of
information in an embedding are actually utilized, even by neural networks, this
strategy still works well with decision forests.

**Pros:**

*   Can deal with much larger vocabulary sizes -- since an embedding is
    effectively a densification into a small number of embedding dimensions, it
    is unlikely that the number of input features to the decision forest
    increases dramatically.
*   Can generalize better, in theory, since similar embeddings can share sets of
    partitions. Note that a big caveat here is that, as mentioned above, basis
    transformations or rotations in vector space can have two similar embeddings
    be completely different in the axis-aligned partitioned space for decision
    forests.
*   Can naturally encapsulate recurrence / word order, for example if the
    encoder contains convolutions, attention, or an RNN.
*   Can leverage information from another dataset (pre-training for transfer
    learning).

**Cons**

*   Not good at memorizing information -- the splits can cause fuzzy
    classifications or high sensitivity to phrasing (i.e. “the router is great”
    vs “a great router”) will produce different embeddings, which may be close
    in euclidean space but not necessarily have similar numerical features.
*   Slowest to serve, because the full encoder forward pass needs to be done at
    inference time. The actual latency is highly dependent on the architecture
    that produced the embeddings; i.e., a transformer encoder will typically be
    much slower than a raw embedding table lookup with mean-pooling.

## Notes

[^1]: Enabling oblique splits can allow learning non-axis aligned information,
    but it will still be on a dimension-by-dimension basis.


text:
This is a simple Prolog interpreter that is compatible with Unity3D,
allowing Prolog code to be mixed with the standard Unity-supported
languages.  It is open source (MIT License), but is easiest to use by
grabbing the DLL and placing it in the Assets folder of your Unity
project.  You also need to copy the files in the Editor subdirectory
to the Editor directory of your Unity project; these will ensure that
your .prolog files get copied over to your standalone builds so that
they can be loaded.

See the Documentation folder for more information.

To compile the code on your own, you either need to copy the Prolog
directory into your Unity project, or build the DLL on your own using
Visual Studio.  The latter will require that you put a copy of the
UnityEngine.dll file from the Unity3D distribution in the top-level
directory of the Visual Studio project.  The Unity DLL is not
distributed with the source for copyright reasons.

Please direct any questions to Ian Horswill<EMAIL_ADDRESS>

repository: monadicus/UnityProlog

repo_name: monadicus/UnityProlog


text:
# Metise

[Arxiv](https://arxiv.org/abs/2106.10542) | [Colab](https://colab.research.google.com/drive/1eIQfcljLeq5gWOipm9fyuVoVvX0zenLZ)

Tensorflow implementation to learn a decompression mapping on images compressed using colour density.

For example:

![Figure 1](https://imgur.com/Jpwz78i.png)

If the differences look subtle, that's due to the fact that at full-scale, the compressed images (with a compression factor of 2.667) look more discrete, and pushing the factor even higher distorts it beyond visually lossless return.  It may, however, be possible with training on larger datasets for a longer period of time.

## Running the Code

The Colab repository for the paper contains all the code, with some description as well.  We recommend running it there.

## Acknowledgements

Code and network architecture heavily borrowed from the [pix2pix](https://github.com/phillipi/pix2pix) framework.

repository: Jozdien/Metise

repo_name: Jozdien/Metise


text:
# cognitive-services-speech-sdk-rs SpeechSDK

---
[![License](https://img.shields.io/badge/License-Apache-blue.svg)](LICENSE-APACHE)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE-MIT)
[![Crates.io][crates-badge]][crates-url]
[![rustdoc][rustdoc-badge]][rustdoc-url]
[![Build Status](https://www.travis-ci.com/jabber-tools/cognitive-services-speech-sdk-rs.svg?branch=main)](https://www.travis-ci.com/github/jabber-tools/cognitive-services-speech-sdk-rs)

[crates-badge]: https://img.shields.io/crates/v/cognitive-services-speech-sdk-rs.svg
[crates-url]: https://crates.io/crates/cognitive-services-speech-sdk-rs
[rustdoc-badge]: https://img.shields.io/badge/rustdoc-0.1.2-green.svg
[rustdoc-url]: https://jabber-tools.github.io/cognitive_services_speech_sdk_rs/doc/0.1.2/cognitive_services_speech_sdk_rs/index.html

## Repository location

Speech SDK files can be found in separate repository [cognitive-services-speech-sdk-rs-files](https://github.com/jabber-tools/cognitive-services-speech-sdk-rs-files).


background: 1992-1999 Maryland State Department of Education, Division of Rehabilitation Services providing counseling and training to individuals with disabilities.   2010- State Director, Women for Ehrlich  2011-2013- District Director, Senator J.B. Jennings  2011-2015- Principal, CTJ Consulting  2015-2018- Department of Budget and Management Administrator (Training Specialist) 2018- present Maryland Transportation Authority, Administrator (Learning and Development Specialist)

education: Bachelor of Arts, Towson University (International Studies)

full-name: Tammy Larkin

jurisdiction: Baltimore County

last-name: Larkin

party: Republican

question: Do you support the findings of the Commission on Innovation and Excellence in Education? Are you committed to funding associated reforms, and if so, how?

answer: I support excellence in our schools.  I agree with some of the findings of the Kirwan Commission, but disagree with others.  I think there are multiple ways to improve education in Maryland.  Each year the school system requests and receives additional funding.  I think we need to evaluate the rate of return on the dollars and make sure the dollars are going into the classroom for students and teachers.  I also believe we need to make sure students who are on the college prep path have the classes needed to be successful in college and make sure schools are offering the right amount of job training and other vocational programs for students who do not plan to pursue college.

shortname: Kirwan

question: Is Maryland’s transportation spending appropriately balanced between roads and transit? Does the state have the resources to meet its transportation needs? With the cancellation of the Red Line and the advent of BaltimoreLink, is the Baltimore region adequately served by transit?

answer: I drive the Baltimore Beltway and I-95 daily and I believe we need to look at all matters of transportation in the region.  Again, I go back to are state taxpayers getting a worthy rate of return on the public transportation needs in the region, particularly light rail.   The current Administration in Annapolis has worked to improve bus routes in the region and this is a step in the right direction.

shortname: Transportation

question: Do you support the legalization of recreational marijuana?

answer: No, I do not support legalized marijuana.

shortname: Marijuana

question: At a time when the federal government’s commitment to Chesapeake Bay restoration is questionable, what new steps should Maryland take to protect this resource?

answer: The Chesapeake Bay is one of the state's greatest natural resources and draws people from all over the region to our state to enjoy. Many in our State rely on the Bay for their livelihood, while others simply enjoy its recreational opportunities.   I am committed to making sure the Bay is safe for all Marylanders and others to enjoy.  The state has made significant progress over the last twenty years cleaning up the Bay and I support efforts to maintain the health of the Bay.

shortname: Chesapeake Bay

question: What steps should Maryland take to ensure the broadest possible access to affordable health care?

answer: Health care is too costly. I'm focused on lowering the price of insurance premiums for hard working families.  The costs associated with Obamacare are not sustainable to Maryland taxpayers and my priority is lowering the cost for Marylanders so they can afford the care they need.

shortname: Health Care

question: What role should the state play in helping Baltimore address violent crime?

answer: The leaders of the City of Baltimore must get tough on criminals.  It is the murder capital of the country.  That is not a statistic anyone wants Maryland to be known for.  All levels of law enforcement must work together to make the streets of Baltimore safer.  We cannot ignore the impact that the opioid epidemic is having in our city and its correlation to higher crime.  Instead of talking about stricter gun laws, let's try focusing on enforcing some of the ones we have on the books.

shortname: Crime

question: How would you characterize Maryland’s business climate? What can the state do to foster the creation of more family-supporting jobs?

answer: The business climate is improving but there is still much to be done in order to make the business climate stronger in the state.  For too long, leaders in Annapolis have made it difficult with burdensome overregulation and increased taxes/fees.  Small businesses are the economic engine in my district and the state.  We need to remove barriers to help businesses grow and thrive.  The legislature continues to be of the mindset that big businesses drive the economy,  but it really is small businesses. My priority is repealing unnecessary taxes/fees and regulations so small business owners can thrive, not just survive.

shortname: Business Climate

question: Do you support the creation of a non-partisan, independent body to draw legislative and congressional district maps after each census?

answer: Yes

shortname: Redistricting

question: Does the Law Enforcement Officers Bill of Rights adequately balance protections for police and the public? Should it be changed, and if so, how?

answer: Our men and women who work everyday to keep Marylanders safe and arrest criminals need to know we support them.  I believe the Law Enforcement Bill of Rights is a step in the right direction.  It does not protect law enforcement officers who commit crimes from prosecution, like the ones people read about in Baltimore City.

shortname: LEOBR

question: What strategy would you adopt to address the opioid addiction and overdose crisis?

answer: First get tougher on the drug dealers and producers.  Increase the penalty for jail and fines for these individuals who are dealing death in our communities.  This crisis knows no geographic or socioeconomic bounds.  We also must teach children earlier about the harmful repercussions with opioid use.

shortname: Opioids

question: What if anything should the state do to address income inequality?

answer: The state should not be telling its citizens where they need to live or work.  Individuals in difficult circumstances or low-income areas can be and are successful. We must celebrate, advocate, and highlight these success stories to continue to inspire others.

shortname: Income inequality

question: Do the state’s Public Information Act and open meetings laws adequately ensure Marylanders’ ability to exercise oversight of the government?

answer: If the open meeting laws are properly followed then yes I believe they are appropriate.  When I'm in the legislature I will push for online viewing of committee hearings and floor debate.

shortname: Transparency

race: state-house

residence: White Marsh

social-fb: TammyLarkinfordelegate

title: Tammy Larkin

gha_language: JavaScript


text:
---
layout: default
title: Publishable
parent: Component Annotations
nav_order: 1
---
# Publishable

The easiest way to configure an entity resource as publishable, use the following annotation and trait:

```php
use Silverback\ApiComponentsBundle\Annotation as Silverback;
use Silverback\ApiComponentsBundle\Entity\Utility\PublishableTrait;

#[Silverback\Publishable]
class Foo
{
    use PublishableTrait;
```

The default property name are:
- `publishedAt` for the published time
- `publishedResource` is the association to published resource.
- `draftResource` is the reverse association to the draft resource

To customize these properties, you can update the annotation and your class:

```php
use Silverback\ApiComponentsBundle\Annotation as Silverback;

#[Silverback\Publishable(fieldName: 'publicationDate', associationName: 'originalResource', reverseAssociationName: 'newResource')]
class Foo
{
    // If not set, the Doctrine mapping is automatically configured with type="date" nullable
    private $publicationDate;

    // If not set, the Doctrine mapping is automatically configured with OneToOne self-referenced association nullable
    private $originalResource;

    // If not set, the Doctrine mapping is automatically configured with OneToOne self-referenced reverse association
    private $newResource;

    public function getPublicationDate(): ?\DateTimeInterface
    {
        return $this->publicationDate;
    }

    public function setPublicationDate(?\DateTimeInterface $publicationDate): self
    {
        $this->publicationDate = $publicationDate;

        return $this;
    }

    public function isPublished() : bool
    {
        return null !== $this->publicationDate && new \DateTimeImmutable() >= $this->publicationDate;
    }

    public function getOriginalResource(): ?self
    {
        return $this->originalResource;
    }

    public function setOriginalResource(?self $originalResource): self
    {
        $this->originalResource = $originalResource;

        return $this;
    }

    public function getNewResource(): ?self
    {
        return $this->newResource;
    }

    public function setNewResource(?self $newResource): self
    {
        $this->newResource = $newResource;

        return $this;
    }
```

Configure the security with expression language, for users who have access to publishable resources:

```yaml
# config/packages/silverback_api_components.yaml:
silverback_api_components:
    # ...
    publishable:
        permission: "is_granted('ROLE_ADMIN')" # default value
```

By default, default validators are applied on save, even for draft objects. If you want to apply validators only for
published resource, configure it as following:

```php
use Silverback\ApiComponentsBundle\Annotation as Silverback;
use Silverback\ApiComponentsBundle\Entity\Utility\PublishableTrait;

#[Silverback\Publishable]
class Foo
{
    use PublishableTrait;

    /**
     * This constraint will be applied on draft and published resources.
     */
    #[Assert\NotBlank]
    public string $name = '';

    /**
     * This constraint will be applied on published resources only.
     */
    #[Assert\NotBlank(groups: ['Foo:published'])]
    public string $description = '';
}
```

You can define a custom validation group for published resources:

```php
use Silverback\ApiComponentsBundle\Annotation as Silverback;
use Silverback\ApiComponentsBundle\Entity\Utility\PublishableTrait;

#[Silverback\Publishable(validationGroups: ['custom_validation_group'])]
class Foo
{
    use PublishableTrait;

    /**
     * This constraint will be applied on draft and published resources.
     */
    #[Assert\NotBlank]
    public string $name = '';

    /**
     * This constraint will be applied on published resources only.
     * The "Foo:published" validation does not exist anymore.
     */
    #[Assert\NotBlank(groups: ['custom_validation_group'])]
    public string $description = '';
}
```

gha_language: PHP

gha_license_id: MIT


text:
# Rakuten Toolbar Automated

## Warning: This script is for educational purposes only. It will not be updated and you are using it at your own risk of Rakuten disabling your account. I take no responsibilities for any loss of property.

### Packages needed
Selenium `pip install -U selenium`

### SETUP
1. Line 11: Fill in Chrome profile path on line 11: `options.add_argument("user-data-dir=PATH")`.
You can find this by typing `chrome://version/` in your browser.

2. Line 14: Fill in the path to your chrome driver. `(executable_path='DRIVER_PATH', chrome_options=options)`

3. Line 23 & 31: Remove the quotation marks and edit your username `(r"USER")` and password `("PASS")` into it for first run. This is not needed for subsequent runs.

4. Line 34: Fill in the dictionary `dsearch = []` with at least 32 search terms in English/Japanese. Search terms can be repeated. Separate with commas and parse it as a string, for example: `[u"月",u"火"]`.

repository: Isopach/Rakuten-Toolbar-Automated

repo_name: Isopach/Rakuten-Toolbar-Automated


text:
---
templateKey: blog-post
title: 'A Taste of Thailand '
date: 2018-12-12T21:00:00.000Z
description: A food diary (in progress)
tags:
  - Travel
  - SEA
  - Thailand
  - Flavours
---
Dear diary,

Here are all the wonderful flavours I've experienced in Thailand. Will update as we explore more!

The basics. A spoon and fork are the main eating utensils.
The spoon is the main eating vessel and is accompanied by the fork which acts as the helper. Soups get a traditional soup spoon and sometimes a regular spoon too if served with rice. Grilled meats are eaten off the skewers. And when we're not sure, we can take a note from the locals.

Street meat. A delicious option any time of day, especially for breakfast. These smoky skewers kept my taste buds longing for more! Just enough crisp and moisture from a low and slow grill. Each vendor has different options (chicken, pork, sausage, ? balls, pork belly) so we just had to find the one we wanted.
Food cart in Sathon district, Bangkok.

Mystery soup. Found at a busy street cart -popular among the motor taxi drivers, we had to try this place.  Accompanied with a typical tray for custom seasoning including dry red chilli, light brown sugar, vinegar with chili, soy sauce.
Street cart in Sathon district, Bangkok.

Green Thai curry. A rich coconut milk based curry. With strong notes of basil and green Thai chillies,  subtle notes of keffir lime leaves and coconut. Perfectly cooked eggplants and pork.
Le Rêve Garden&Eatery, Bangkok

Tom Kha Gai. Light and tangy/lime-y coconut soup with a notes of ginger, lemon grass and slight heat.
Le Rêve Garden&Eatery, Bangkok.

Fried rice with pork. “Pretty standard fried rice” -Andrew
Le Rêve Garden&Eatery, Bangkok.

Butterfly pea drinks. A purple concoction made from butterfly pea tea (caffeine free herbal tea made from a flower). Cool and refreshing. Subtle taste (describe).
Le Rêve Garden&Eatery, Bangkok.

Butterfly pea with coconut milk dessert. Cool, lightly sweetened coconut milk with bits of ice for crunch and slightly-harder-than-gelatinous butterfly pea chunks.
Le Rêve Garden&Eatery, Bangkok.

Bubble tea. Taro flavoured milk tea with honey pearls.
Street cart in Sathon district, Bangkok.

Double deep fried chicken wings. Moist yet perfectly crispy. Served with hot chili and sweet chili sauce.
Asiatique food booth (near Farris wheel) serving various fried foods, Bangkok.

Shrimp and chicken sauté with fried egg on rice. A well balanced sauté sauce with heat from chilies, sweetness and umami flavour from fish or soy sauce. Rice topped with a perfectly done crispy-bottom runny-top egg. Served with a side of chilli vinegar.
Food court under BZ27, Sathon district, Bangkok.

Taiwanese milk tea.
Street cart, Sathon district, Bangkok.

Red thai curry. Served slightly dry (less saucy) with chicken and peas on rice. This is one dish made me say "peas, please".
(Restaurant name), Sathon district, Bangkok.

Deep fried pork belly. Rich, crunchy with good crispy fat to meat ratio.
(Restaurant name), Sathon district, Bangkok.

Coconut milk drink. Refreshing coconut milk with ice and coconut cream balances lightness and richness. The coconut flavour was prominent, similar to the level you'd get from freshly shaved coconut meat.
Street cart near Bangkok train station.

Basil pork sauté. A light sweet sauce with garlic, big chunks of red chilli and a strong Thai basil flavour. Served on rice with a perfectly cooked crispy runny egg and star-shaped cucumber.
12Go Café.

Fried rice. Subtle flavours and green onions throughout the fried rice. Flavour was not smoky as usual.
12Go Café.

Croissants. Rich buttery pastry that makes for a prefect travel snack.
Family mart, Bangkok.

Fried chicken. Seasoned with kimin, salt, white pepper. A local speciality.
Haadyai Station, Hat Yai.


text:
---
title:  "Weather Report"
date:   2018-10-10 23:39:00
categories: text
---

<img src="{{site.baseurl}}/assets/WeatherReports/WeatherReportConsolidated.png">

This is an **analysis and visualization** of ***weather*** of 550+ cities across the world. The main objective was to showcase the relationship of various factors like temperature, humidity, cloudiness and wind speed with respect to the distance from the equator.

For this analysis, approx. 600 coordinate pairs of latitude and longitude were created randomly and then [citypy](https://pypi.org/project/citipy/) was used to generate city names for respective coordinates. After selecting unique non-repeat observations, a weather check was performed on each of the cities using a series of successive API calls from OpenWeatherMap. Then all different point of interests were plotted against latitude using [Matplotlib](https://matplotlib.org/) and [Seaborn](https://seaborn.pydata.org/) libraries.

In conclusion, an equatorial climate is a type of tropical climate in which there is no dry season. It is usually found at latitudes within five degrees of the equator and tropical rainforest is the natural vegetation.


text:
---
title: "Taxons"
description: "Here we describe the Seller API Taxon resource."
date: 2020-11-12T13:26:54+01:00
lastmod: 2020-11-12T13:26:54+01:00
draft: false
images: []
menu:
  docs:
    parent: "sellerapi"
    identifier: "taxons-sellerapi"
weight: 985
toc: true
---

Taxons in Marketplacer represent the categories for [adverts](/docs/sellerapi/adverts) in a heirarchical fashion. For instance, on Bike Exchange the "Hybrid Bikes" taxon is one of the children of the "Bikes" taxon.

## Listing taxons

This endpoint returns all taxons.

### Request

    GET /api/v2/taxons

### Response

  Status: 200 OK

```json
{
  "data": [
    {
      "id": 1,
      "type": "taxons",
      "attributes": {
        "name": "Hybrid Bikes",
        "tree_name": "Bikes - Hybrid Bikes",
        "slug": "hybrid-bikes"
      }
    }
  ]
}
```

## Search for taxons

This endpoint allows you to search for a taxon by a word in its name or by a substring of its name.

### Request

If you want to search for the "Hybrid Bikes" taxon, make a query like this:

    GET /api/v2/taxons/search?name=Hybrid%20Bikes

If you want to search for all taxons that include the word "Hybrid":

    GET /api/v2/taxons/search?name=Hybrid

If you want to search by a substring of a word, add the `autocomplete` parameter, like:

    GET /api/v2/taxons/search?name=Bik&autocomplete=1

If you want to get the option value information for an individual taxon, use the taxon id or slug:

    GET /api/v2/taxons/4

or

    GET /api/v2/taxons/bikes

### Response

The response will include other taxons that may match your query.

  Status: 200 OK

```json
{
  "data": [
    {
      "id": 1,
      "type": "taxons",
      "attributes": {
        "name": "Hybrid Bikes",
        "tree_name": "Bikes - Hybrid Bikes",
        "slug": "hybrid-bikes"
      }
    }
  ]
}
```

repository: DamilolaAdegunwa/mapi

repo_name: DamilolaAdegunwa/mapi


text:
AXIA-LUFS
=========
An EBU R128 loudness meter for Axia Livewire streams with a short-term (3 sec.) and/or momentary (100ms) loudness readouts. 100ms Resolution.

### What is Axia Livewire?
Axia Livewire is an IP networked audio protocol used in broadcast. Among other things, it allows for 32767 channels of raw multicast PCM audio distributed over a standard IP network. Axia nodes are connected to the network and provide audio I/O.
### What does a Livewire audio stream consist of?
Raw PCM (big endian, 24-bit signed, 48000 kHz, stereo interleaved) delivered in the form of multicast IP/UDP/RTP packets.
### How does this work?
PCM from each packet is buffered and handed to libebur128 for loudness measurement.
### To Do
Buffer input / confirm packet sequence. (UDP does not guarantee packets arrive in order; xNodes buffer 15-100ms the recieve side.)
### Setup and Use
* Get `libebur128`, and then compile axialufs.c.
* Update index.js to point to the axialufs binary & supply an Axia multicast address.
* Lauch the node server: `node index.js`
* Visit `localhost:3000` in your browser.

repository: hopla/AXIA-LUFS

repo_name: hopla/AXIA-LUFS


text:
---
layout: post
categories: recipes
tags: [recipes, sweets, peanut butter cookies, cookies]
excerpt:
title: "3 Minute Peanut Butter Cookies"
image:
  feature: 31-1.jpg
redirect_from:
  - recipes/3-minute-peanut-butter-cookies.html
---

These are the cookies you want to be making if it's 8pm on a Sunday night and you get the sudden cravings for peanut butter cookies.

I originally got this recipe off of a Foodnetwork gluten free recipe, but I in no way chose to make these for health related reasons.  I chose this recipe because it had 5 ingredients (3-4 of which you can find in probably 90% of households) and you an make it in one bowl.  From start to finish - baking time included, these cookies take a total of about 15 minutes to make.  There's no butter.  You don't have to cream anything.  I made mine with a single fork.  It was just one of those recipes where I  knew I had all the ingredients, where I knew I didn't have to clean twenty different items afterward, and where I knew that in 15 minutes (prep and bake time included) I would be eating some yummy peanut butter cookies.

The only downside to these cookies if there are any would be that it is a bit crumbly because of the lack of flour and butter and also that it can be a little too peanut butter-y - meaning at times, eating the cookies will sort of be like eating a spoonful of peanut butter - again not necessarily a bad thing, but it can stick to your teeth.  And yes, did I mention this recipe is extremely peanut butter heavy?

Tips: I heard someone had added a teaspoon of baking soda to the recipe and that made the cookie a bit more crispy and less crumbly.  I also recommend adding 1/2-3/4 cup sugar instead of the full cup as it would be way too sweet.  I sprinkled on a little coarse salt on top of the cookies before I baked them because I think that really brings out the peanut butter flavor. Also, because it is the fall and I had a bag of dried cranberries, I decided to mix those in as well.  The flavor was delicious, but the oven has the tendency to dry out the cranberries, so what I would recommend is to rehydrate the cranberries either in some water or orange juice before putting them into the batter.

Everything in one bowl: peanut butter eggs, sugar, vanilla, salt

Bowl of smooth peanut butter dough

Spoon them into balls and press down with a fork

10 minutes in a 350 oven and they're ready to serve

Ingredients

(makes 24 cookies)

1 cup natural peanut butter

1/2 - 3/4 cup sugar

1 large egg, lightly beaten

1 teaspoon vanilla extract

Coarse sea salt

Optional: rehydrated cranberries

Directions
1. Preheat the oven to 350 degrees Fahrenheit.  In a bowl, mix the peanut butter, sugar, egg and vanilla until well combined.
2. Spoon 1 tablespoon of mixture about 1 inch apart onto ungreased baking sheets. Flatten the dough with the tines of a fork, making a crosshatch pattern on the cookies.
3. Sprinkle coarse salt on top of the cookies. Bake for 10 minutes rotating the pans halfway.

repository: Ibexian/gatsby-advanced-starter

repo_name: Ibexian/gatsby-advanced-starter


text:
# SaasChecklist
Want to make a SAAS?   Getting Started
1. Designing the app
1. Authentication
1. Domains
1. Hosting
1. Programming language(s)
1. Staff?
1. Security
1. Cost to do business
1. Marketing
1. Monitoring
    1. Uptime
    1. [Google Webmaster Tools / Search Console](https://www.google.com/webmasters/tools/home?hl=en)
    1. [Chrome User Experience Report](https://developers.google.com/web/tools/chrome-user-experience-report/)
    1. [Pagespeed Insights](https://developers.google.com/speed/pagespeed/insights/)
    1. [Lighthouse](https://developers.google.com/web/tools/lighthouse/)

repository: StoneCypher/SaasChecklist

repo_name: StoneCypher/SaasChecklist


text:
+++
# Experience widget.
widget = "experience"  # See https://sourcethemes.com/academic/docs/page-builder/
headless = true  # This file represents a page section.
active = true  # Activate this widget? true/false
weight = 40  # Order that this section will appear.

title = "Experience"
subtitle = ""

# Date format for experience
#   Refer to https://sourcethemes.com/academic/docs/customization/#date-format
date_format = "Jan 2006"

# Experiences.
#   Add/remove as many `[[experience]]` blocks below as you like.
#   Required fields are `title`, `company`, and `date_start`.
#   Leave `date_end` empty if it's your current employer.
#   Begin/end multi-line descriptions with 3 quotes `"""`.
[[experience]]
  title = "Google Summer of Code 2019"
  company = "Participant in Red Hen Lab"
  company_url = "https://summerofcode.withgoogle.com"
  location = "Remote"
  date_start = "2019-05-09"
  date_end = "2019-08-15"
  description = """
  Design and develop an online deep learning course for humanists

  - More than 31,000 students registered, 5,999 of whom submitted proposal, and only 1,276 students were selected (4.1%, 16.9%).
  - Mentor: Francis Steen, Mark Turner and Rajesh Kasturirangan
  - Repo: https://github.com/yogayu/deepLearningCourse
  """

[[experience]]
  title = "Baidu Inc."
  company = "Research and Development Engineer Intern"
  company_url = "https://en.wikipedia.org/wiki/Baidu"
  location = "Beijing, China"
  date_start = "2017-04-30"
  date_end = "2016-12-01"
  description = """
  - Implemented and fixed iOS client video module related functions: video release page, live display page, video player and data log, etc
  - Optimized and developed automatic page refresh strategy and page preload list data strategy to effectively improve user experience
  """

[[experience]]
  title = "Douban Inc."
  company = "iOS Software Engineer Intern"
  company_url = "https://www.douban.com/"
  location = "Beijing, China"
  date_start = "2016-07-01"
  date_end = "2016-11-30"
  description = """
  - [Participated in the douban summer camp](http://data2art.com/douban-intern.html) and shared the experience from entry to the first App on the shelves
  - Investigated iOS hotfix tools such as JSPatch, waxPatch and Rollout, and created JSPatch hotfix Demo
  - Developed record page related requirements: customized list display, favorites, album details, comments, etc
  - Optimized and fixed the iPad version App: redesigned the overall UI framework, adapt 10+ page layout and interaction
  """

#[[experience]]
#  title = "Pulse App"
#  company = "iOS Independent Developer"
#  company_url = "http://pulse.data2art.com/"
#  location = ""
#  date_start = "2016-05-01"
#  date_end = "2016-11-01"
#  description = """
#Pulse is a clean and beautiful Metronome. Independently complete the whole process from idea, design, coding, propaganda to the shelf.
#  """
+++

repository: Yogayu/youxinyu

gha_language: HTML

repo_name: Yogayu/youxinyu


text:
# Custom CMS for ketzu.net

This is a hobby Django project to realise a custom CMS for my webpage [ketzu.net](https://ketzu.net).

## Overview

This project is split into 6 django Apps to represent various different but independent parts:

 * photography: Photo galleries and a gallery overview.
 * science: Simple list of published papers with file storage for the pdfs.
 * medium: A blog like list of medium articles, although it could link to any external source.
 * videography: A list of embedded youtube videos with a title and description.
 * projects: A list of projects with optional github and project links.
 * web: A front page with a personal picture, description and favorites of the previous 5 apps.

The project uses a sqlite database, as the limitation of single writer is easily solved on a personal website.
This has additional advantages for deployment.

## Deployment

For ease of testing the project, the project contains a dockerfile to build a container image hosting the site.
The project is built into a docker container on [dockerhub](https://hub.docker.com/repository/docker/ketzu/knet) on push and can be pulled from there as ketzu/knet.
The container will run migrations on before starting gunicorn.

Media files will need to be served through a different container.

repository: ketzu/knet-django

gha_language: Python

repo_name: ketzu/knet-django


text:
# Linux Security and System Hardening

This document aims to define basic and useful practices to harden your linux setup. All examples will be focused on debian-based systems, but they also will be extendable to other linux flavours.

Tip: most of that tools should be run as root, to be aware of all system files

## Chkrootkit

A linux rootkit scanner

### Install

```
apt-get install chkrootkit
```

### Usage

```
chkrootkit
```

### Results

All results will be displayed in screen

## Rkhuner

Tool for scanning backdoors, rootkits and local exploits

### Install

```
apt-get install rkhunter
```

### Config

Add package manager in rkhunter configuration to prevent false positives.

Edit /etc/rkhunter.conf and add the following property (for debian, use the properly package manager for your distro)

```
PKGMGR=DPKG
```

### Usage

```
rkhunter -c
```

### Results

All results will be displayed in screen. For further details, a log is available in /var/log/rkhunter.log

## ClamAV

Antivirus engine

### Install

```
apt-get install clamav
```

### Usage

To update signatures

```
freshclam
```

To run clamav

```
clamscan -r -i DIRECTORY
```

## Lynis

Lynis is a system hardening auditing tool. It automates the audit of a given system and reports which kind of threats can be found in the system.

### Install

The recommended way to install it is just clone the latest version of its git repository

```
cd /usr/local
git clone https://github.com/CISOfy/lynis
```

### Usage

Just call "audit system" function from your lynis base folder

```
lynis audit system
```

To run lynis without pauses

```
lynis audit system --quick
```

### Results

The results are displayed on screen, during the system scan.
Additional details can be found, after audit system function pass, in /var/log/lynis.log
You can also find all audit discovers in a report file located in /var/log/lynis-report.dat

## Suricata

IDS/IPS software

### Install

```
sudo add-apt-repository ppa:oisf/suricata-stable
sudo apt-get update
sudo apt-get install suricata
```

### Configuration

All configuration will lie in /etc/suricata/suricata.yaml

Tip: at least, check that current interface is the desired one (default is eth0)

### Usage

With this installation, suricata will run in background. It can be found as a service

```
service suricata status
```

### Troubleshooting

Somethimes, trying to restart suricata service, pid file appears to be stalled. Just remove it

```
sudo rm /var/run/suricata.pid
```

And restart again

## Fail2ban

Focused on attacks detected, it bans the ips that perform them. Logs based

### Install

```
apt-get install fail2ban
```

### Configure

- File location: /etc/fail2ban/fail2ban.conf (or fail2ban.local for overrides)
- Logs are located in /var/log/fail2ban.log
- /etc/fail2ban/jail.local will be used to configure white list, ban times, retry amounts and which services to log (by default, only ssh)

## SELinux

TODO

repository: IGPla/docurepo

repo_name: IGPla/docurepo


text:
![Alt text](/week-3/Devtools/imgs/1.png)
![Alt text](/week-3/Devtools/imgs/2.png)
![Alt text](/week-3/Devtools/imgs/3.png)
![Alt text](/week-3/Devtools/imgs/4.png)
![Alt text](/week-3/Devtools/imgs/5.png)
![Alt text](/week-3/Devtools/imgs/6.png)
![Alt text](/week-3/Devtools/imgs/7.png)
![Alt text](/week-3/Devtools/imgs/8.png)
![Alt text](/week-3/Devtools/imgs/9.png)

**How can you use Chrome's DevTools inspector to help you format or position elements?**
1) I can change the size of my text
2) Position of divs ( which also teaches me more about positioning becouse I see changes in real time)
3) I can play with display ( use display inline, inline-block or block)
4) I can set margins, paddings, borders
5) I can float and clear elements
7) I can set some fix values
8) I can use Chrome DevTools as a wireframing

**How can you resize elements on the DOM using CSS?**
1) I can resize elements using fixed ( px ) or relative values ( %, em)
2) I can go to inspect.element window and play with element style setting some properties

**What are the differences between Absolute, Fixed, Static, and Relative positioning? Which did you find easiest to use? Which was most difficult?**
1) Absolute ( relative to parent )
2) Fixed ( relative to window )
3) Static ( element doesn't has any position)
4) Relative ( positioning from the point where element would normaly be )
Absolute was the hardest becouse you have to keep thinking about 2 elements instead of 1.

**What are the differences between Margin, Border, and Padding?**
1) Border - (without padding) edges of the content
2) Margin - space between border and others elements
3) Padding - space betweent content and border

**What was your impression of this challenge overall? (love, hate, and why?)**
I am generally in love in DevTools becouse they safe much time. I don;t go back and for to implement changes and se how they look on the page. I change things on site, then copy them and paste into my code.

gha_language: Ruby


text:
---
layout: default
---

# 1994-05-12 - Re: PGP 2.5 available from Electronic Frontier Foundation ftp site

## Header Data

From: Tommy the Tourist<EMAIL_ADDRESS>To<EMAIL_ADDRESS>Message Hash: 6808bc65768cde1660da8b4f1974f98a2589195b39cffbeb8e442df70ac78d0d

Message ID<EMAIL_ADDRESS>Reply To: _N/A_

UTC Datetime: 1994-05-12 15:26:34 UTC

Raw Date: Thu, 12 May 94 08:26:34 PDT

## Raw message

```
{% raw  %}From: Tommy the Tourist<EMAIL_ADDRESS>Date: Thu, 12 May 94 08:26:34 PDT
To<EMAIL_ADDRESS>Subject: Re: PGP 2.5 available from Electronic Frontier Foundation ftp site
Message-ID<EMAIL_ADDRESS>MIME-Version: 1.0
Content-Type: text/plain

-----BEGIN PGP SIGNED MESSAGE-----

    From: Stanton McCandlish<EMAIL_ADDRESS>    Date: Wed, 11 May 1994 16:46:49 -0400 (EDT)

    Despite the patent & licensing issues being resolved, PGP is still
    not legally exportable from the United States (except to Canada),
    due to ITAR export restrictions which categorize cryptographic
    materials as weapons of war.  Thus, EFF can only make PGP and
    other crypto tools and source code available to US and Canadian
    nationals currently residing in the US or Canada and connecting to
    EFF's site from a US or Canadian site.

I was under the impression that NAFTA is the reason that Canada is
included.  Am I misinformed?  If NAFTA is the reason, isn't Mexico
equivalent to Canada in this context?  Could someone clarify this
aspect of the export situation, or perhaps point me at a document that
explains the situation?

			Zeke

-----BEGIN PGP SIGNATURE-----
Version: 2.3a

iQCVAgUBLdJIuBVg/9j67wWxAQEebAP/flhqUugfCUJ9at1nI8kCbkXiF10NYfcE
s+1+ZFNnvz16gwI/O7nEfrIHKQl6mqmqT8T4e2JCsMiw7uM7L3vYIKHJvRek45gk
/6JoUE7sjVb8nyvyct9sKeExAGqKFLxAAsOZfYno88qOMAE4nc3QRxMoqLb3XDbI
EbxPLyo8T/s=
=RqOv
-----END PGP SIGNATURE-----

------------
To respond to the sender of this message, send mail to
<EMAIL_ADDRESS>starting your message with
the following 7 lines:
::
Response-Key: ideaclipper

====Encrypted-Sender-Begin====
MI@```$YS^P;+]AB?X9TW6\8WR:>P&2'9,7.YM5[D--('+[6(0O]013@CTWLQ
FKWZ$M2G9>G1>/=5O1[0U.E)J&63=DHF($P$KX#I8T]5&9!GFQ]@`
====Encrypted-Sender-End====

{% endraw %}
```

## Thread

+ Return to [May 1994](/archive/1994/05)

+ Return to "[Phil Karn <karn@qualcomm.com>](/authors/phil_karn_karn_at_qualcomm_com_)"
+ Return to "[SINCLAIR  DOUGLAS N <sinclai@ecf.toronto.edu>](/authors/sinclair_douglas_n_sinclai_at_ecf_toronto_edu_)"
+ Return to "[Tommy the Tourist <nobody@soda.berkeley.edu>](/authors/tommy_the_tourist_nobody_at_soda_berkeley_edu_)"

+ 1994-05-12 (Thu, 12 May 94 08:26:34 PDT) - Re: PGP 2.5 available from Electronic Frontier Foundation ftp site - _Tommy the Tourist<EMAIL_ADDRESS>  + 1994-05-12 (Thu, 12 May 94 08:33:30 PDT) - [Re: PGP 2.5 available from Electronic Frontier Foundation ftp site](/archive/1994/05/c65206ce4e3c2fffb086f2ed73e63d94668c6dde8a3aad296c9d0ef71aad7aa9) - _SINCLAIR  DOUGLAS N<EMAIL_ADDRESS>  + 1994-05-12 (Thu, 12 May 94 10:02:17 PDT) - [Re: PGP 2.5 available from Electronic Frontier Foundation ftp site](/archive/1994/05/998deb7c21d724e33e1e328d19c1f9b6397752dd51d74edc20a5d12fc7b598b0) - _Phil Karn<EMAIL_ADDRESS>


text:
# Latest Ionic
### based on the latest Cordova with the latest Android and the latest Node.js
----
### Pull from Docker Hub
```
docker pull rahulcv/ionic:latest
```

### Build from GitHub
```
docker build -t rahulcv/ionic github.com/rahulcv/docker-ionic
```

### Run image
```
docker run -it rahulcv/ionic bash
```

### Use as base image
```Dockerfile
FROM rahulcv/ionic:latest
```

----

repository: RahulCV/docker-ionic

repo_name: RahulCV/docker-ionic


text:
# Change Log
All notable changes to this project will be documented in this file.

## v0.4.0-beta.1 - 2017-03-15
### Changed
- Depend on Angular 4.0.0

## v0.2.0-beta.4 - 2017-03-15
### Fixed
- Resolved depend on Angular 2.0.0 (closes [#3](https://github.com/nglibs/i18n-router/issues/3))
- Limited to Angular 2.x.x (use v0.4.0 for Angular 4.x.x)
- Workaround for AoT compilation

### Changed
- Updated deps
- Some refactoring

## v0.2.0-beta.3 - 2017-02-23
### Fixed
- Forced to use TypeScript 2.1.x

### Changed
- Updated README.md
- Updated deps

## v0.2.0-beta.2 - 2017-02-20
### Added
- Added 'I18NRouterHttpLoader`

## v0.2.0-beta.1 - 2017-02-17
### Added
- Added unit tests

### Changed
- Some refactoring

## v0.2.0-alpha.4 - 2017-02-13
### Fixed
- Resolved `Argument of type 'string[]' is not assignable to parameter of type 'string'.)` issue
- Resolved providing entryComponents issue (provides dynamically now)

## v0.2.0-alpha.3 - 2017-02-13
### Fixed
- Resolved AoT compilation issue

## v0.2.0-alpha.2 - 2017-02-11
### Changed
- Updated README.md
- Some refactoring

## v0.2.0-alpha.1 - 2017-02-11
- Pre-release


text:
# How To Run a C# script

To run it from a console:

- Install the `dotnet-script` tool: `dotnet tool install -g dotnet-script`
- Build the solution with `PulumiAzure` from the parent folder
- Execute `dotnet script main.csx`

```
    └─ core.ResourceGroup     rg          created
    └─ storage.Account        sa          created
```


text:
#Pinclone, a Pinterest clone built with Flask

*disclaimer: This project is for purpose of my personal studies.
Feel free to use any of the code, but it is not guaranteed to work correctly and I cannot promise any support.*

##Tutorial
This project builds on the excellent Youtube tutorial "[PyCharm Web Magic: Building a Pinterest Clone](http://www.youtube.com/watch?v=2geC50roans)" by the JetBrains team. I highly recommend following that, even if you do not use PyCharm. My code will most likely differ significantly from the tutorial, since I tend to explore different alleys while studying.

##Live example
I plan to host this project for all to see. This readme will be updated with a link to the website.

repository: mkscala/flask-pinclone

repo_name: mkscala/flask-pinclone


text:
---
layout: relation
title: 'dep:comp'
shortdef: 'unspecified dependency'
udver: '2'
---

The `dep:comp` relation is used in **UD_French-GSD** and in **UD_French-Spoken** for passive constructions (with the lemma *se* for the dependent).

~~~ sdparse
Max s' intéressa à elle \n Max took interest in her
dep:comp(intéressa,s')
~~~

It is also used in construction *plus de*, *beaucoup de*:

~~~ sdparse
le concept demande beaucoup de moyens \n the concept requires a lot of means
dep:comp(beaucoup,moyens)
~~~
<!-- Interlanguage links updated Út 9. května 2023, 20:04:11 CEST -->

repository: UniversalDependencies/docs

branch_name: refs/heads/pages-source

repo_name: UniversalDependencies/docs


text:
# Cluster Management

This doc is in progress.

## Upgrading a cluster

The `cluster/kube-push.sh` script will do a rudimentary update; it is a 1.0 roadmap item to have a robust live cluster update system.

## Updgrading to a different API version

There is a sequence of steps to upgrade to a new API version.

1. Turn on the new version.
2. Upgrade the cluster's storage to use the new version.
3. Upgrade all config files. Identify users of the old api version endpoints.
3. Turn off the old version.

### Turn on or off an API version for your cluster

TODO: There's an apiserver flag for this.

### Switching your cluster's storage API version

TODO: This functionality hasn't been written yet.

### Switching your config files to a new API version

You can use the kube-version-change utility to convert config files between different API versions.

```
$ hack/build-go.sh cmd/kube-version-change
$ _output/go/bin/kube-version-change -i myPod.v1beta1.yaml -o myPod.v1beta3.yaml
```

repository: CADBOT/kubernetes

repo_name: CADBOT/kubernetes


text:
---
layout: til
title: "Titanian Montes and Middle-Earth"
date: 2015-04-09
modified: 2015-04-09
author: OctoMiao
comments: true
categories: misc
filter: misc
summary: Titanian Montes Are Named After Mountains From Middle-Earth Created By J.R.R. Tolkien
---

For instance, Doom Mons is a mountain range located on the southern hemisphere of Titan.

Checkout [list of geological features on Titan](https://en.wikipedia.org/wiki/List_of_geological_features_on_Titan#Montes) for more names.


text:
\*\*

Performing the following tasks will help you become more familiar with the various tools for manipulating object files.

### A
How many object files are contained in the versions of `libc.a` and `libm.a` on your system?

### B
Does `gcc -Og` produce different executable code than `-Og -g`?

### C.
What shared libraries does the GCC driver on your system use?

## Answer

### A
By invoking `ar -t` it outputs the list of object files contained in the static library.

On my system, `libc.a` includes 1676 object files, and `libm.a` includes 745 object files.

### B
No, they have the same executable code. The only difference between them is the production of `gcc -Og -g` contains debug sections.

### C
By invoking `ldd` we can see the libraries the executable file depends on.

On my system, the GCC use `linux-vdso.so.1`, `libm.so.6`, `libc.so.6`, and`ld-linux-x86-64.so.2` for interpreter.


text:
---
author: reneVolution
comments: false
date: 2013-07-13 08:57:22+00:00
layout: post
slug: ffmpeg-2-0
title: FFmpeg 2.0 is released
wordpress_id: 365
categories:
- FFmpeg
tags:
- codecs
- encoding
- FFmpeg
- filter
- video
---

[![FFmpeg Logo](/images/FFmpeg-Logo.svg_.png)]({% post_url 2013-07-13-ffmpeg-2-0 %})

Wow, what a big jump from FFmpeg 1.2 => 2.0. But there is a whole bunch of new impressive features e.g FTP Protocol support or filter graph description readable from file just to name two. You can find all updates at [http://www.ffmpeg.org](http://www.ffmpeg.org). I'm really looking forward to test the new features and share my observations here.


text:
---
title: "Deploying image classification (mxnet) using Mxnet Model Server (mms)"
excerpt: "How to deploying image classification (mxnet) using Mxnet Model Server (mms)."
categories:
  - Machine Learning
tags:
  - tutorial
  - mms
published: true
comments: true
toc: true
support: true
order: 9
author: vugia.truong
---

# Summary

In this post, i will summarize steps required when deploying a simple image classification (mxnet) using Mxnet Model Server (mms).

# Detail

Like other deeplearning framework, mxnet also provides tons a pretrained model and tools cover nearly all of machine learning task like image classification, object detection, segmentation, ... These pretrained models and tools are inside the [gluon-cv](https://gluon-cv.mxnet.io/) package.

For similarity, i'll use the pretrained Resnet50 model. For other model please see [model_zoo](https://gluon-cv.mxnet.io/model_zoo)

* [source code : https://github.com/gachiemchiep/mms_example](https://github.com/gachiemchiep/mms_example).

## Run the image classification service

* Download source code

```bash
$ git clone https://github.com/gachiemchiep/mms_example
```

* Re-create anaconda environment and activate it

```bash
$ cd mmx_example
# This will create environment called DL with python=3.7
$ conda env create -f environment.yml
# Activate anaconda
$ conda activate DL
```

* Create mms model archive file

```bash
$ mkdir mms_example/mms_example/services
$ cd mms_example/mms_example
$ model-archiver --model-name mxnet_resnet50 --model-path mxnet_resnet50 --handler img_classifier_service:handle --export-path ../model-archives/ --force
# A new file called mxnet_resnet50.mar should be created inside model-archives directory
```

* Start mms server

```bash
$ cd mms_example/mms_example
$ mxnet-model-server --start
```

* Test

```bash
$ cd mms_example/mms_example/resources
$ curl -X POST http://<IP_ADDRESS>:8080/predictions/mxnet_resnet50 -T chair.jpg
```

* Output should be as follow

```bash
[
  {
    "rocking chair": "0.4921491"
  },
  {
    "folding chair": "0.08000506"
  },
  {
    "wool": "0.027606748"
  },
  {
    "cowboy hat": "0.024664406"
  },
  {
    "velvet": "0.020736441"
  }
]
```

* Shutdown the model server

```bash
mxnet-model-server --stop
```

## Source code explain

At this state, our source will be as follow

```bash
mms_example/
├── environment.yml
├── LICENSE
├── mms_example
│   ├── common
│   ├── config.properties
│   ├── __init__.py
│   ├── logs                                        : log directory
│   │   ├── access_log.log
│   │   ├── mms_log.log
│   │   ├── mms_metrics.log
│   │   ├── model_log.log
│   │   └── model_metrics.log
│   ├── model-archives
│   │   └── mxnet_resnet50.mar                      : model archieve file which we created
│   ├── resources
│   ├── services
│   │   ├── __init__.py
│   │   └── mxnet_resnet50
│   │       ├── img_classifier_service.py           : service file
│   │       └── __init__.py
│   └── tests
└── README.md
```

The most important file is [img_classifier_service.py](https://github.com/gachiemchiep/mms_example/blob/master/mms_example/services/mxnet_resnet50/img_classifier_service.py), this include all of our logic from loading model, reading uploader data, inference, ...

* Load model

```python
class ResNet50Classifier():
    def __init__(self):
        self.ctx = None
        self.net = None
        self.initialized = False

    def initialize(self, context):
        """
        Load the model and mapping file to perform infernece.
        :param context: model relevant worker information
        :return:
        """

        properties = context.system_properties
        gpu_id = properties.get("gpu_id")
        self.ctx = mx.cpu() if gpu_id is None else mx.gpu(gpu_id)
        self.net = gluoncv.model_zoo.get_model("ResNet50_v1d", pretrained=True, ctx=self.ctx)
        self.initialized = True
```

* Read uploaded data and convert to mxnet's NDArray

```python
    def preprocess(self, data):
        """
        Scales, crops, and normalizes a PIL image for a mxnet model,
        :param data:
        :return: ndarray
        """

        img_arrs = []
        for idx in range(len(data)):
            img = data[idx].get("data")
            if img is None:
                img = data[idx].get("body")

            if img is None:
                img = data[idx].get("data")

            if img is None or len(img) == 0:
                self.error = "Empty image input"
                return None

            img_arr = mx.image.imdecode(img)
            img_arrs.append(img_arr)

        img_arrs = gluoncv.data.transforms.presets.imagenet.transform_eval(img_arrs)
        return img_arrs
```

* Do the inferencing

```python
    def inference(self, img, topk=5):
            """
            :param img:
            :param topk:
            :return:
            """
            pred = self.net(img.as_in_context(self.ctx))
            # map predicted values to probability by softmax
            probs = mx.nd.softmax(pred)[0].asnumpy()
            # find the 5 class indices with the highest score
            inds = mx.nd.topk(pred, k=topk)[0].astype('int').asnumpy().tolist()

            rets = []
            for i in range(topk):
                ret = dict()
                ret[self.net.classes[inds[i]]] = str(probs[inds[i]])
                rets.append(ret)

            return [rets]

```

* The final logic

```python
_service = ResNet50Classifier()

def handle(data, context):
    if not _service.initialized:
        _service.initialize(context)

    if data is None:
        return None

    data = _service.preprocess(data)
    data = _service.inference(data)
    data = _service.postprocess(data)

    return data

```

In the last section, we used the following command to create model-archive

```bash
$ model-archiver --model-name mxnet_resnet50 --model-path mxnet_resnet50 --handler img_classifier_service:handle --export-path ../model-archives/ --force
```

As we can see, the model-archive will use **handle** method as logic. This method is defined inside the   **img_classifier_service.py** file.

## Benchmark result

My computer has the following specs :

```bash
cpu:
                    AMD Ryzen 5 2600 Six-Core Processor
graphics card:
                    nVidia GP106 [GeForce GTX 1060 6GB]
memory:
                    16GB
```

My computer only have GTX-1060 GPU card, the benchmarking result is as follow :

```bash
# benchmarking command
$ ab -k -l -n 10000 -c 100 -T "image/jpeg" -p kitten.jpg http://<IP_ADDRESS>:8080/predictions/mxnet_resnet50
# This will send 10000 request with 100 concurrency. So please be patient. i would take around 5 minutes

# GPU usage: only 673 MB
(DL) gachiemchiep@gachiemchiep:services$ nvidia-smi
Thu Jun 13 20:55:58 2019
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.67       Driver Version: 418.67       CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 106...  On   | 00000000:1C:00.0  On |                  N/A |
| 40%   59C    P2    46W / 120W |   1702MiB /  6075MiB |     45%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0      1933      G   /usr/lib/xorg/Xorg                           699MiB |
|    0      3685      G   ...quest-channel-token=3700236306674113463   214MiB |
|    0      7697      G   ...p/pycharm-educational/12/jre64/bin/java     2MiB |
|    0      8227      C   ...chiep/opt/miniconda2/envs/DL/bin/python   673MiB |
|    0     22775      G   ...-token=CCBE267BA54B1DB7F35623396BB493CD    43MiB |

# Benchmark result
Benchmarking <IP_ADDRESS> (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests

Server Software:
Server Hostname:        <IP_ADDRESS>
Server Port:            8080

Document Path:          /predictions/mxnet_resnet50
Document Length:        Variable

Concurrency Level:      100
Time taken for tests:   242.090 seconds
Complete requests:      10000
Failed requests:        0
Keep-Alive requests:    10000
Total transferred:      4320000 bytes
Total body sent:<PHONE_NUMBER>
HTML transferred:       1970000 bytes
Requests per second:    41.31 [#/sec] (mean)
Time per request:       2420.901 [ms] (mean)
Time per request:       24.209 [ms] (mean, across all concurrent requests)
Transfer rate:          17.43 [Kbytes/sec] received
                        4483.74 kb/s sent
                        4501.16 kb/s total

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.6      0       6
Processing:    53 2409 154.0   2417    2679
Waiting:       53 2409 154.0   2417    2679
Total:         58 2409 153.6   2417    2679

Percentage of the requests served within a certain time (ms)
  50%   2417
  66%   2445
  75%   2462
  80%   2477
  90%   2519
  95%   2572
  98%   2618
  99%   2636
 100%   2679 (longest request)

```

* Note : by default configuration, mms only queue 100 jobs. So if you need more concurrency, increase the value of  job_queue_size inside [config.properties](https://github.com/gachiemchiep/mms_example/blob/master/mms_example/config.properties). With a value of job_queue_size=1000 , i can increase the -c to 200

```bash
# benchmarking command
$ ab -k -l -n 10000 -c 200 -T "image/jpeg" -p kitten.jpg http://<IP_ADDRESS>:8080/predictions/mxnet_resnet50

# Benchmark result
Benchmarking <IP_ADDRESS> (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests

Server Software:
Server Hostname:        <IP_ADDRESS>
Server Port:            8080

Document Path:          /predictions/mxnet_resnet50
Document Length:        Variable

Concurrency Level:      200
Time taken for tests:   241.262 seconds
Complete requests:      10000
Failed requests:        0
Keep-Alive requests:    10000
Total transferred:      4320000 bytes
Total body sent:<PHONE_NUMBER>
HTML transferred:       1970000 bytes
Requests per second:    41.45 [#/sec] (mean)
Time per request:       4825.240 [ms] (mean)
Time per request:       24.126 [ms] (mean, across all concurrent requests)
Transfer rate:          17.49 [Kbytes/sec] received
                        4499.13 kb/s sent
                        4516.61 kb/s total

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    7  83.2      0    1023
Processing:  2037 4771 198.9   4794    6032
Waiting:     2037 4771 198.9   4794    6031
Total:       2043 4778 230.1   4794    7054

Percentage of the requests served within a certain time (ms)
  50%   4794
  66%   4830
  75%   4851
  80%   4864
  90%   4903
  95%   4936
  98%   4965
  99%   5006
 100%   7054 (longest request)

```

Quite a good benchmark resulti guess.

End


text:
---
{
  "title": "Unsupervised Bilingual Word Embedding Agreement for Unsupervised Neural Machine Translation",
  "date": "2019-07-28",
  "authors": [
    "Haipeng Sun",
    "Rui Wang",
    "Kehai Chen",
    "M. Utiyama",
    "Eiichiro Sumita",
    "T. Zhao"
  ],
  "abstract": "Unsupervised bilingual word embedding (UBWE), together with other technologies such as back-translation and denoising, has helped unsupervised neural machine translation (UNMT) achieve remarkable results in several language pairs. In previous methods, UBWE is first trained using nonparallel monolingual corpora and then this pre-trained UBWE is used to initialize the word embedding in the encoder and decoder of UNMT. That is, the training of UBWE and UNMT are separate. In this paper, we first empirically investigate the relationship between UBWE and UNMT. The empirical findings show that the performance of UNMT is significantly affected by the performance of UBWE. Thus, we propose two methods that train UNMT with UBWE agreement. Empirical results on several language pairs show that the proposed methods significantly outperform conventional UNMT.",
  "links": [
    {
      "resource": "PDF",
      "icon": "pdf",
      "url": "https://pdfs.semanticscholar.org/58af/f08ffa4db37f0a965b9f0e33c7c24cf34c16.pdf"
    },
    {
      "resource": "Semantic Scholar",
      "icon": "semanticscholar",
      "url": "https://www.semanticscholar.org/paper/58aff08ffa4db37f0a965b9f0e33c7c24cf34c16"
    }
  ],
  "supervision": [],
  "tasks": [],
  "methods": [],
  "thumbnail": "unsupervised-bilingual-word-embedding-agreement-for-unsupervised-neural-machine-translation-thumb.jpg",
  "card": "unsupervised-bilingual-word-embedding-agreement-for-unsupervised-neural-machine-translation-card.jpg",
  "s2_paper_id": "58aff08ffa4db37f0a965b9f0e33c7c24cf34c16"
}
---

repository: virtualgraham/unsupervisedpapers

repo_name: virtualgraham/unsupervisedpapers