text:
# Message Encryption with Dataflow PubSub Stream Processing

Dataflow Streaming pipeline where each pubsub messages payload data is encrypted or digitally singed.

This technique can be used where a common "message bus" shared PubSub topic is used by N message producers and where
the consumer needs to either verify the integrity of the message that was submitted or unwrap the contents of the message
given an encryption key owned by the producer.

In other words,

- ProjectA runs the Dataflow Pipeline DF1 and owns a PubSub Topic T1
- ProjectB owns a service account with permissions to submit messages to Topic T1
- ProjectB owns KMS key K1 and uses it to _encrypt_ a message.
- ProjectA has permissions on ProjectB's KMS K1 to _decrypt_
- ProjectB can now wrap a PubSub message with K1 and place it in T1
- ProjectA Dataflow Pipline will read each message and use K1 to decrypt it in the pipeline

In this why, the common shared topic can contain messages from a variety of sources where each message producer retains the
ownership of the KMS key used to decrypt the message.   Users can extend the pattern above for any number of message producers with
their own KMS Keys.  (eg. 10 different producers all sumbit messages to the shared topic).

This techinque is rather convoluted use of dataflow and pubsub in that normally you would setup different DF pipelines or Topics.

This sample extends the 4th technique detailed here:

- [Message Payload Encryption in Google Cloud Pub/Sub](https://github.com/salrashid123/gcp_pubsub_message_encryption)

The specific code sample here is the basic DataFlow WordCount.  A single producer emits encypted messages to a common pubsub topic.

Each message is decrypted by Dataflow worker and a count of words per window is displayed by a subscriber pubsub topic.

that is

- ProjectA runs Streaming pipeline that reads from Topic T1, decrypts a message then writes the message to Topic T2
- ProjectB uses KMS to encrypt a message and places it on Topic T1
- PubsubSubscriber reads messages emitted on T1 and displays the WordCount

There are several optimizations applied to the dataflow and pubsub message decryption:  instead of using `KMS` to encrypt and
decrypt _each_ pubsub message, an expiring AES Symmetric key is generated at the message producer side and that is what
gets encrypted by a `KMS` key.  The encrypted key is only valid for say 100seconds and is placed within the attributes of the pubsubmessage
itself.  This is basically `envelope encryption`.

One problem with PubSub and envelope encryption is that the message data in the attribute set is lost unless the payload has the message
and originally intended attribute set as well

## Setup

The setup steps creates two projects, several pubsub topics and KMS keys.  While you can run this whole sample in one GCP project,
I've setup two to demonstrate tenancy and separation of access.

To run these commands, you need to either be a project owner or have the ability to create GCP projects

### Command Reference

*  `publisher.py`  Publish a signed or encrypted message to the common message bus topic.
   - `--mode`: either `encrypt` or `sign`
   - `--service_account`: path to the service account json file to encrypt/sign and place on the common pubsub topic.
   - `--pubsub_project_id`: projectID for the shared pubsub topic
   - `--pubsub_topic`: shared topic name
   - `--kms_project_id`: projectID that hosts the KMS keyring and keyid for encryption/sign
   - `--kms_location`: location of the keyring
   - `--kms_key_ring_id`: KMS key ring name
   - `--kms_key_id`: the key to use for encryption/sign

*  `pubsub_bus.py`  Dataflow Pipeline Runner that reads the common pubsub topic, decrypts messages and outputs those to another topic.
   - `--mode`: either `decrypt` or `verify`
   - `--output_topic`: topic to write pubsub messages after wordcount DF pipeline
   - `--input_subscription`: Subscription to read messages from te shared topic
   - `--setup_file`:  path to setup.py to enable local modules. Set it to `pwd/setup.py` as shown below
   - `--extra_package`: extra packages.  Set it to (`pwd/gcp_encryption/dist/gcp_encryption-0.0.1.tar.gz`)
   - `--project`: projectID for the dataflow pipeline
   - `--temp_location`: GCS bucket for DF
   - `--staging_location`: staging GCS bucket for DF

*  `df_subscriber.py`  Pubsub message subscriber that reads the *output* messages from the DF pipeline
   - `--service_account`: Service Account that has access to read an output subscription _from_ DF
   - `--pubsub_project_id`: projectID for the pubsub topic that DF writes _to_
   - `--pubsub_subscription`: Subscription to read messages from te output topic

*  `subscriber.py`  Pubsub message subscriber that **directly** reads from a shared pubsub topic (similar to Dataflow Pipeline); (optional) this is used for debugging only.  TO use this script. the `service_account` must have a subscription to the common message bus _and_ have access to the KMS keys that was used to encrypt the payload.
   - `--mode`: either `decrypt` or `verify`
   - `--service_account`: path to the service account json file to encrypt/sign and place on the common pubsub topic.
   - `--pubsub_project_id`: projectID for the shared pubsub topic
   - `--pubsub_subscription`:  Subscription to read messages from te shared topic

### Setup Environment variables

Since we will create several projects, in one shell, setup some variables
```

export NONCE=$(date '+%Y-%m-%d')-$USER
export df_PROJECT=df-project-$NONCE
export REGION=us-central1
export tenant_1=tenant-1-$NONCE
export tenant_2=tenant-2-$NONCE
```

### Create DataFlow Project

Now create a project that will host the dataflow and pubsub common topic.

Remember to associate a billing account with this project.

```
gcloud projects create $df_PROJECT --name "Dataflow Pubsub Bus" --enable-cloud-apis
gcloud services enable dataflow.googleapis.com cloudkms.googleapis.com --project $df_PROJECT
gcloud config set project df-project-$NONCE

gcloud auth application-default login
```

- Create a service account that will subscribe to events emitted by the DF pipeline

```
cd dataflow_src/
mkdir certs
gcloud iam service-accounts create df-project --display-name="Dataflow Project Service Account" --project $df_PROJECT
gcloud iam service-accounts keys  create certs/svc-provider.json<EMAIL_ADDRESS>--project $df_PROJECT
```

You should see two service accounts if its a newly created project:
    ```
    $ gcloud iam service-accounts list --project $df_PROJECT
    NAME                                    EMAIL
    Dataflow Project Service Account<EMAIL_ADDRESS>    Compute Engine default service account<EMAIL_ADDRESS>    ```

- Create the pubsub Topic and allow DF to subscribe:

```
gcloud pubsub topics create common-topic --project $df_PROJECT
gcloud pubsub subscriptions create message-bus --topic=common-topic --project $df_PROJECT
```

![images/service_accounts_df.png](images/service_accounts_df.png)

- Use Google Cloud console or gcloud  to set IAM permission on the DF Project service account to subscribe

```bash
export df_PROJECT_NUMBER=`gcloud projects describe  $df_PROJECT --format="value(projectNumber)"`
export<EMAIL_ADDRESS>
echo "{
  \"bindings\": [
    {
      \"role\": \"roles/pubsub.subscriber\",
      \"members\": [
        \"serviceAccount:$df_COMPUTE_SVC_ACCOUNT\",
      ]
    }
  ],
}" >policy.txt

gcloud  --project  $df_PROJECT pubsub subscriptions set-iam-policy message-bus policy.txt
rm policy.txt
```

  ![images/df_subscriber.png](images/df_subscriber.png)

- Create a temp GCS bucket for use by DataFlow:

```
$ gsutil mb gs://$df_PROJECT-dftemp
```

- Add IAM permissions for you to setup the DF pipeline:

```
gsutil iam ch user:$(gcloud config get-value core/account):objectCreator,objectViewer gs://$df_PROJECT-dftemp
```

- Create output topic/subscriptions for DF

```
gcloud pubsub topics create df_out --project $df_PROJECT
gcloud pubsub subscriptions create df_subscribe --topic=df_out --project $df_PROJECT
```

- Use cloud console to add IAM permissions to the topic and subscription

1)  goto  topic 'df_out'  add the  "Compute Engine default service account" as a pubslisher

```bash
echo "{
  \"bindings\": [
    {
      \"role\": \"roles/pubsub.publisher\",
      \"members\": [
        \"serviceAccount:$df_COMPUTE_SVC_ACCOUNT\",
      ]
    }
  ],
}" >policy.txt

gcloud  --project  $df_PROJECT pubsub topics set-iam-policy df_out policy.txt
rm policy.txt
```

 ![images/df_out_publish.png](images/df_out_publish.png)

2) goto subscription for 'df_out' --> 'df_subscribe' add the  "Dataflow Project Service Account " as a subscirber

```bash
echo "{
  \"bindings\": [
    {
      \"role\": \"roles/pubsub.subscriber\",
      \"members\": [
<EMAIL_ADDRESS>      ]
    }
  ],
}" >policy.txt

gcloud  --project  $df_PROJECT pubsub subscriptions set-iam-policy df_subscribe policy.txt
rm policy.txt
```

 ![images/df_out_subscribe.png](images/df_out_subscribe.png)

### Run DF pipeline:

Now that we've setup the first part, lets see if we can run the dataflow pipeline as is:

- Setup a virtualenv and package the custom libraries:

```bash
gcloud auth application-default login
cd dataflow_src/
virtualenv env --python=/usr/bin/python3.7
source env/bin/activate
cd gcp_encryption/
python setup.py sdist
cd ../

# todo upgrade https://github.com/googleapis/python-kms/blob/release-v2.0.0/UPGRADING.md

pip install apache-beam[gcp] google-cloud-kms==1.4.0 lorem cryptography expiringdict tink
```

- Now run the pipeline with `DataFlowRunner`  (you can also just use `DirectRunner` for local testing if your user credentials
has access to the topics and subscriptions defined above)

```bash
python pubsub_bus.py \
    --mode=decrypt \
    --region=$REGION \
    --runner DirectRunner \
    --setup_file `pwd`/setup.py  \
    --extra_package=`pwd`/gcp_encryption/dist/gcp_encryption-0.0.1.tar.gz  \
    --max_num_workers=1 \
    --project $df_PROJECT \
    --temp_location gs://$df_PROJECT-dftemp/temp \
    --staging_location gs://$df_PROJECT-dftemp/stage  \
    --output_topic projects/$df_PROJECT/topics/df_out  \
    --input_subscription projects/$df_PROJECT/subscriptions/message-bus
```

  ![images/df_job.png](images/df_job.png)

### Create subscriber for DF pipeline Output

In a new window, reset the environment variables
```bash
source env/bin/activate
export NONCE=$(date '+%Y-%m-%d')-$USER
export df_PROJECT=df-project-$NONCE
export tenant_1=tenant-1-$NONCE
export tenant_2=tenant-2-$NONCE
```

Now run the subscriber

```bash
python df_subscriber.py --service_account certs/svc-provider.json \
  --pubsub_project_id $df_PROJECT \
  --pubsub_subscription df_subscribe
```

At the end of this step, you should have a pubsub pipeline running and a subscriber listening for the output of the pipeline.

### Create Message Producer Project and KMS permissions

The following step will setup a pubsub message producer to send encrypted messages into the common, shared topic

In a new window, reset the environment variables
```bash
source env/bin/activate
export NONCE=$(date '+%Y-%m-%d')-$USER
export df_PROJECT=df-project-$NONCE
export tenant_1=tenant-1-$NONCE
export tenant_2=tenant-2-$NONCE
```

- Create the tenant/producer project

```bash
gcloud projects create $tenant_1 --name "Tenant-1" --enable-cloud-apis
gcloud services enable cloudkms.googleapis.com pubsub.googleapis.com --project $tenant_1
```

Associate a billing account with `$tenant_1`

- Create a service account on that project

```
gcloud iam service-accounts create tenant-1-svc --display-name="Tenant-1 Service Account" --project $tenant_1
gcloud iam service-accounts keys  create certs/svc-tenant-1.json<EMAIL_ADDRESS>--project $tenant_1
```

- Create a KMS keyring and Key
```
gcloud kms keyrings create tenant1-keyring  --location $REGION --project $tenant_1
gcloud kms keys create key1 --location=$REGION --purpose=encryption --keyring=tenant1-keyring --project $tenant_1
```

Your tenant project should now show a single service account
```
  $ gcloud iam service-accounts list --project $tenant_1
  NAME                      EMAIL
  Tenant-1 Service Account<EMAIL_ADDRESS>```

- Set KMS permissions on the key

The following commands will be different for you:

First set an IAM policy on the key such that the Service Account that runs Dataflow can use that key to Decrypt:

Note, your service account will be different

```bash
# first make sure you have this set in env
echo $df_COMPUTE_SVC_ACCOUNT

gcloud kms keys add-iam-policy-binding key1 \
      --keyring tenant1-keyring \
      --location $REGION \
      --member=serviceAccount:$df_COMPUTE_SVC_ACCOUNT \
      --role='roles/cloudkms.cryptoKeyDecrypter' --project $tenant_1
```

Second, set KMS permissions so the service account that places a message on the common topic has the rights to use
that key to encrypt:

Note, your service account will be different
```bash
gcloud kms keys add-iam-policy-binding key1 \
      --keyring tenant1-keyring \
      --location $REGION \
<EMAIL_ADDRESS>\
      --role='roles/cloudkms.cryptoKeyEncrypter' --project $tenant_1
```

  ![images/tenant_kms_permissions.png](images/tenant_kms_permissions.png)

### Add tenant to DF Pubsub Topic

Switch to the main Dataflow Pipeline project

Use Cloud console to add the tenant/producers's service account rights to place messages on the shared Topic:

```bash
echo "{
  \"bindings\": [
    {
      \"role\": \"roles/pubsub.publisher\",
      \"members\": [
<EMAIL_ADDRESS>      ]
    }
  ],
}" >policy.txt

gcloud  --project  $df_PROJECT pubsub topics set-iam-policy common-topic policy.txt
rm policy.txt
```

```
add<EMAIL_ADDRESS>to topic
           projects/df-project-2018-11-04/topics/common-topic
    as publisher
```

  ![images/df_topic_publish.png](images/df_topic_publish.png)

### RUN Publisher for tenant1

Now try to run the publisher script:

```bash

```

What that will do is create an AES key, wrap it with KMS, then place the message on the shared topcic:

```bash
python publisher.py  --mode encrypt --service_account 'certs/svc-tenant-1.json' --pubsub_project_id $df_PROJECT \
   --pubsub_topic common-topic --kms_project_id $tenant_1  \
   --kms_location us-central1 --kms_key_ring_id tenant1-keyring --kms_key_id key1

2018-11-04 09:56:43,242 - root - INFO - >>>>>>>>>>> Start Encryption with locally generated key.  <<<<<<<<<<<
2018-11-04 09:56:43,242 - root - INFO - Rotating symmetric key
2018-11-04 09:56:43,242 - root - INFO - Starting KMS encryption API call
2018-11-04 09:56:43,261 - googleapiclient.discovery - INFO - URL being requested: POST https://cloudkms.googleapis.com/v1/projects/tenant-1-2018-11-04/locations/us-central1/keyRings/tenant1-keyring/cryptoKeys/key1:encrypt?alt=json
2018-11-04 09:56:43,803 - root - INFO - End KMS encryption API call
2018-11-04 09:56:43,812 - root - INFO - Start PubSub Publish
2018-11-04 09:56:43,821 - root - INFO - Published Message: izqIBBc0sKIPi3a2rhwin+grNWYr46rAjbsxduS12nQfaBI8E7112/WSCjw8+daJe+33XFQ5YHHtX/+mYL+UojhAYQxHK31On44eGTpyia0awS1zDF6PB5geju9wVQD4S7HwpAhENTS8ivGwiFAgeDuDQZm3PQQliNv/p5m9bWPeKfmbE8oFou8L1juZqncQxNdtuB+i7J/h/GuYfUFVVh43LANzGiXq9UkbdaBnGto+IX4oxt7NbsyACEOcWcXTI40peQ2DXIl8LoPij1/xtHuBRu2qiRsSN+wzwmfDqX7ox16OAKBxphjT94B2zWhe+FXK7fPxe9HW9mEKpY4Ltz+5QtgHBs/fZfNSvc99Tqo=
```

---

The dataflow pipeline should be still running from the previous step:

The DF pipeline here will be reading in a message, decrypting it and placing the wordcounts on another output topic

```
$ python pubsub_bus.py --runner DataFlowRunner --setup_file `pwd`/setup.py   --requirements_file requirements.txt --extra_package=`pwd`/gcp_encryption/dist/gcp_encryption-0.0.1.tar.gz  --max_num_workers=1 --project $df_PROJECT --temp_location gs://$df_PROJECT-dftemp/temp --staging_location gs://$df_PROJECT-dftemp/stage  --output_topic projects/$df_PROJECT/topics/df_out  --input_subscription projects/$df_PROJECT/subscriptions/message-bus --region $REGION --mode decrypt

Starting
...
gs://df-project-2018-11-04-dftemp/stage/beamapp-srashid-1104140856-238869.1541340536.238981/apache_beam-2.8.0-cp27-cp27mu-manylinux1_x86_64.whl
INFO:root:Create job: <Job
 createTime: u'2018-11-04T14:09:04.660304Z'
 currentStateTime: u'1970-01-01T00:00:00Z'
 id: u'2018-11-04_06_09_03-6245556987231969631'
 location: u'us-central1'
 name: u'beamapp-srashid-1104140856-238869'
 projectId: u'df-project-2018-11-04'
 stageStates: []
 steps: []
 tempFiles: []
 type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:root:Created job with id: [2018-11-04_06_09_03-6245556987231969631]
INFO:root:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-04_06_09_03-6245556987231969631?project=df-project-2018-11-04
INFO:root:Job 2018-11-04_06_09_03-6245556987231969631 is in state JOB_STATE_PENDING
INFO:root:2018-11-04T14:09:06.635Z: JOB_MESSAGE_DETAILED: Checking permissions granted to controller Service Account.
INFO:root:2018-11-04T14:09:07.440Z: JOB_MESSAGE_BASIC: Worker configuration: n1-standard-4 in us-central1-a.
INFO:root:2018-11-04T14:09:07.761Z: JOB_MESSAGE_DETAILED: Expanding CollectionToSingleton operations into optimizable parts.
INFO:root:2018-11-04T14:09:07.767Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:root:2018-11-04T14:09:07.774Z: JOB_MESSAGE_DETAILED: Expanding SplittableProcessKeyed operations into optimizable parts.
INFO:root:2018-11-04T14:09:07.776Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into streaming Read/Write steps
INFO:root:2018-11-04T14:09:07.780Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:root:2018-11-04T14:09:07.792Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:root:2018-11-04T14:09:07.794Z: JOB_MESSAGE_DETAILED: Fusing consumer ReadFromPubSub/Map(_from_proto_str) into ReadFromPubSub/Read
INFO:root:2018-11-04T14:09:07.796Z: JOB_MESSAGE_DETAILED: Fusing consumer split into decode
INFO:root:2018-11-04T14:09:07.798Z: JOB_MESSAGE_DETAILED: Fusing consumer group/MergeBuckets into group/ReadStream
INFO:root:2018-11-04T14:09:07.801Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteToPubSub/Write/NativeWrite into encode
INFO:root:2018-11-04T14:09:07.803Z: JOB_MESSAGE_DETAILED: Fusing consumer encode into format
INFO:root:2018-11-04T14:09:07.805Z: JOB_MESSAGE_DETAILED: Fusing consumer format into count
INFO:root:2018-11-04T14:09:07.807Z: JOB_MESSAGE_DETAILED: Fusing consumer group/WriteStream into WindowInto(WindowIntoFn)
INFO:root:2018-11-04T14:09:07.808Z: JOB_MESSAGE_DETAILED: Fusing consumer count into group/MergeBuckets
INFO:root:2018-11-04T14:09:07.810Z: JOB_MESSAGE_DETAILED: Fusing consumer WindowInto(WindowIntoFn) into pair_with_one
INFO:root:2018-11-04T14:09:07.812Z: JOB_MESSAGE_DETAILED: Fusing consumer decode into ReadFromPubSub/Map(_from_proto_str)
INFO:root:2018-11-04T14:09:07.814Z: JOB_MESSAGE_DETAILED: Fusing consumer pair_with_one into split
INFO:root:2018-11-04T14:09:07.818Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:root:2018-11-04T14:09:07.841Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:root:2018-11-04T14:09:07.892Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:root:2018-11-04T14:09:08.051Z: JOB_MESSAGE_DEBUG: Executing wait step start2
INFO:root:2018-11-04T14:09:08.062Z: JOB_MESSAGE_DEBUG: Starting worker pool setup.
INFO:root:2018-11-04T14:09:08.066Z: JOB_MESSAGE_BASIC: Starting 1 workers...
INFO:root:Job 2018-11-04_06_09_03-6245556987231969631 is in state JOB_STATE_RUNNING
INFO:root:2018-11-04T14:09:10.927Z: JOB_MESSAGE_BASIC: Executing operation ReadFromPubSub/Read+ReadFromPubSub/Map(_from_proto_str)+decode+split+pair_with_one+WindowInto(WindowIntoFn)+group/WriteStream
INFO:root:2018-11-04T14:09:10.927Z: JOB_MESSAGE_BASIC: Executing operation group/ReadStream+group/MergeBuckets+count+format+encode+WriteToPubSub/Write/NativeWrite
INFO:root:2018-11-04T14:09:52.021Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:root:2018-11-04T14:09:57.246Z: JOB_MESSAGE_DEBUG: Executing input step topology_init_attach_disk_input_step
```

The pubsub subscriber listening to the output of the Dataflow pipeline will show the decrypted lorem ipsum messages and word counts:

```
$ python df_subscriber.py --service_account 'certs/svc-provider.json' --pubsub_project_id $df_PROJECT --pubsub_subscription df_subscribe
2018-11-04 10:18:27,122 INFO >>>>>>>>>>> Start <<<<<<<<<<<
2018-11-04 10:18:27,365 INFO Listening for messages on projects/df-project-2018-11-04/subscriptions/df_subscribe
2018-11-04 10:18:51,289 INFO magnam: 7
2018-11-04 10:18:51,371 INFO quiquia: 7
2018-11-04 10:18:51,374 INFO velit: 8
2018-11-04 10:18:51,374 INFO etincidunt: 5
2018-11-04 10:18:51,376 INFO Dolore: 1
2018-11-04 10:18:51,378 INFO tempora: 9
2018-11-04 10:18:51,451 INFO Magnam: 2
2018-11-04 10:18:51,574 INFO consectetur: 4
2018-11-04 10:18:51,577 INFO amet: 5
2018-11-04 10:18:51,580 INFO adipisci: 3
2018-11-04 10:18:51,582 INFO porro: 9
2018-11-04 10:18:51,583 INFO Etincidunt: 2
2018-11-04 10:18:51,584 INFO Modi: 1
2018-11-04 10:18:51,585 INFO dolor: 1
2018-11-04 10:18:51,586 INFO quaerat: 3
2018-11-04 10:18:51,588 INFO modi: 5
2018-11-04 10:18:51,589 INFO Quiquia: 2
2018-11-04 10:18:51,590 INFO Non: 1
2018-11-04 10:18:51,591 INFO Velit: 1
2018-11-04 10:18:51,592 INFO neque: 7
2018-11-04 10:18:51,593 INFO Eius: 4
2018-11-04 10:18:51,594 INFO Adipisci: 1
2018-11-04 10:18:51,595 INFO labore: 12
2018-11-04 10:18:51,596 INFO numquam: 4
2018-11-04 10:18:51,598 INFO Dolorem: 1
2018-11-04 10:18:51,599 INFO Aliquam: 2
2018-11-04 10:18:51,677 INFO Ipsum: 1
2018-11-04 10:18:51,679 INFO quisquam: 6
2018-11-04 10:18:51,680 INFO ipsum: 5
2018-11-04 10:18:51,681 INFO ut: 9
2018-11-04 10:18:51,683 INFO Neque: 2
2018-11-04 10:18:51,684 INFO Ut: 2
2018-11-04 10:18:51,685 INFO sit: 9
2018-11-04 10:18:51,687 INFO Sed: 2
2018-11-04 10:18:51,688 INFO voluptatem: 6
2018-11-04 10:18:51,689 INFO Quisquam: 2
2018-11-04 10:18:51,689 INFO Labore: 1
2018-11-04 10:18:51,690 INFO aliquam: 2
2018-11-04 10:18:51,692 INFO sed: 5
2018-11-04 10:18:51,693 INFO Voluptatem: 1
2018-11-04 10:18:51,694 INFO Dolor: 3
2018-11-04 10:18:51,695 INFO Est: 2
2018-11-04 10:18:51,696 INFO eius: 5
2018-11-04 10:18:51,833 INFO non: 7
2018-11-04 10:18:51,834 INFO est: 7
2018-11-04 10:18:51,837 INFO Numquam: 1
2018-11-04 10:18:51,837 INFO dolore: 2
2018-11-04 10:18:51,839 INFO dolorem: 8
```

## Verify message integrity (sign/verify)

Each message producer can also just submit the message without encryption with an wrapped `HMAC` integrity signature.

The DataFlow pipeline will read the unencrypted message as well as the encrypted HMAC singature and then use KMS to
decrypt the signature key.  With the paintext HMAC key, the pipeline can verify the message integrity.

To use this, simply alter `publisher.py` and pass in ```--mode=sign```.

On the DF Pipeline side, set the `--mode` argument to `verify`

## Non-Repudiation
The HMAC based signing and AES encryption provides only integrity and confidentiality…they do not provide non-repudiation (i.e. assertion that the message originator is authentic. You will need access to a Public-Private keypair to verify and sign the pubsub message similar to the integrity check above.

There are two ways to achieve this:

1. Use Cloud KMS APIs to sign each message and provide metadata attribute link to the KMS key that signed it.
*  https://cloud.google.com/kms/docs/create-validate-signatures then on the Dataflow side, use a cached copy of the public key reference to verify https://cloud.google.com/kms/docs/retrieve-public-key

2. Use a service_account’s public/private key reference to achieve a similar result to above:

*  Use the private key for a given service_account to sign a message. Attach metadata reference to the URL of the public portion of that service key as a message attachement. On the DataFlow side, download the public key and use that to verify the signature. This technique is described here for purely pubsub: [salrashid123/gcp_pubsub_message_encryption](https://github.com/salrashid123/gcp_pubsub_message_encryption/tree/master/2_svc)

## Ephemeral session encryption keys

The AES encryption and HMAC keys are wrapped with KMS but to avoid repeated roundtrips, these keys are held in cache at _both_
the client and at Dataflow workier.  Each key can expire at different intervals:  if the DF worker does not recognize a session
key in cache, it makes a KMS api call to decrypt it.  THe DF service then saves both the wrapped key and decrypted version of it
in an expiring dictionary:

```
from expiringdict import ExpiringDict
self.cache = ExpiringDict(max_len=100, max_age_seconds=200)
```
### PubSub Message modification

*WARNING*:

Inorder to embed encrypted or signed payload, the specific technique used here involves modification of the normal PubSub.

THat is, normally a pubsub message contains several user-defined and automatic fields:

- [pubsub.proto](https://github.com/googleapis/googleapis/blob/master/google/pubsub/v1/pubsub.proto#L340)
```text
message PubsubMessage {
  // The message data field. If this field is empty, the message must contain
  // at least one attribute.
  bytes data = 1;

  // Optional attributes for this message.
  map<string, string> attributes = 2;

  // ID of this message, assigned by the server when the message is published.
  // Guaranteed to be unique within the topic. This value may be read by a
  // subscriber that receives a `PubsubMessage` via a `Pull` call or a push
  // delivery. It must not be populated by the publisher in a `Publish` call.
  string message_id = 3;

  // The time at which the message was published, populated by the server when
  // it receives the `Publish` call. It must not be populated by the
  // publisher in a `Publish` call.
  google.protobuf.Timestamp publish_time = 4;
}
```

This technique alters the message in fundamental ways.

- For encryption, the actual pubsub message is in the format:

```python
publisher.publish(topic_name, data=encrypted_message.encode('utf-8'), kms_key=name, dek_wrapped=dek_encrypted)
```

or
```json
  data:  encrypted text of the original message
  kms_keyname: fully qualified path to the kms key that encrypted this message
  dek_wrapped: wrapped AES encryption key for this specific message
```

- For sign, the actual pubsub message is in the format:

```python
publisher.publish(topic_name, data=json.dumps(cleartext_message), kms_key=name, sign_key_wrapped=sign_key_wrapped, signature=msg_hash)
```

or
```json
  data:  plaintext of the original message
  kms_key: fully qualified path to the kms key that encrypted this message
  sign_key_wrapped: wrapped HMAC signing key for this specific message
  signature: signature of the original message
```

Note, in both cases, the PubSUb attributes are overrided.  Inorder to embed attributes, they will need to get placed as JSON within a message
eg.

```text
    cleartext_message = {
            "data" : "foo bar".encode(),
            "attributes" : {
                "epoch_time":  int(time.time()),
                "a": "aaa",
                "c": "ccc",
                "b": "bbb"
            }
    }
```

## Conclusion

THis is just a POC i wrote up with just a bit of practical use:  Sure you can encrypt a message and place
and encrypted message into pubsub, but to achieve tenancy, it'd be eaiser ot just setup different topics  per tenant
alltogether.

However, if you do need a shared pubsub topic and you need to ensure message secrecy or integrity per producer, you can
explore variations of this technique.


text:
---
layout: post
title: Nano01(自動運転)-【C++】-Lesson12-Vectors and Iterators容器与迭代器
date: 2019-01-01 07:00:12
categories: C++
tags: C++
---
* content
{:toc}

# 1. Vectors Introduction

C++中有很多很好用的容器类，比如vector:

1. The vector has size 0 when we instantiate it.
2. We use 'resize' to change the size of the vector.

```cpp
// constructing vectors
#include <iostream>
#include <vector>  //Need to include the vector library!

int main ()
{
  //creating a vector of integers
  std::vector<int> vectorInts;
  std::cout<<"vectorInts has "<<vectorInts.size()<<" elements\n";

  //Changing the size of vectorInts to 6
  vectorInts.resize(6);
  std::cout<<"\n\nvectorInts now has "<<vectorInts.size()<<" elements\n";

  return 0;
}
```

```
vectorInts has 0 elements
vectorInts now has 6 elements
```

# 4. Vectors and Iterators

vector比array更通用，可以在运行时修改其长度，也可以插入元素到vector。

```cpp
// constructing vectors
// constructing vectors
#include <iostream>
#include <vector>

int main ()
{
  //creating a vector of integers
  std::vector<int> vectorInts;
  //creating an iterator for the vector
  std::vector<int>::iterator it;

  std::cout<<"vectorInts has "<<vectorInts.size()<<" elements\n";

  std::cout<<"\n\nAdding four elements to the vector\n";
  //assigning the value 3 to 4 elements of the vector
  vectorInts.assign(4,3);
  std::cout<<"vectorInts has "<<vectorInts.size()<<" elements\n";

  //printing the contents of vectorInts
  std::cout<<"VectorInts has these elements:\n";
  for (it = vectorInts.begin(); it != vectorInts.end(); ++it)
    std::cout<<*it<<" ";

  return 0;
}

```

```
vectorInts has 0 elements

Adding four elements to the vector
vectorInts has 4 elements
VectorInts has these elements:
3 3 3 3
```

# 5. Vectors and Its Prog Ans

```cpp
// constructing vectors
// constructing vectors
#include <iostream>
#include <vector>

using namespace std;

void printVector(vector<float> vIn);

 void printVector(vector<float> vIn)
 {//printing the contents of vIns
    //TODO: Complete the function
    vector<float>::iterator it;
    for (it = vIn.begin(); it != vIn.end(); ++it)
        cout<<*it<<" ";
 }

```

```cpp
// Goal: pracitce constructing vectors
// and their iterators
// Call the vector vFloat
// Call the iterator it

//TODO: Add the necessary libraries
#include "main.hpp"

int main ()
{
  //TODO: create a vector of floats
   vector<float> vFloat;

  std::cout<<"vFloat has "<<vFloat.size()<<" elements\n";

  //TODO: add elements to the library

  std::cout<<"\n\nAdding 10 elements to the vector\n";

  //TODO: assign the value 8.8 to 10 elements of the vector
   vFloat.assign(10, 8.8);
   std::cout<<"vFloat has "<<vFloat.size()<<" elements\n";

  //TODO: Complete the Print function in main.hpp
  // Call the function here to print out each element of vFloat
  printVector(vFloat);

  return 0;
}
```

输出：

```cpp
vFloat has 0 elements

Adding 10 elements to the vector
vFloat has 10 elements
8.8 8.8 8.8 8.8 8.8 8.8 8.8 8.8 8.8 8.8
```

# 7. assign Elements

参考下面的代码，通过assign赋值的时候，会把之前的vector覆盖：

```cpp
// constructing vectors
// constructing vectors
#include <iostream>
#include <vector>

int main ()
{
  //creating a vector of integers
  std::vector<int> vectorInts;
  //creating an iterator for the vector
  std::vector<int>::iterator it;

  std::cout<<"vectorInts has "<<vectorInts.size()<<" elements\n";

  std::cout<<"Adding 23 to the vector\n";
  vectorInts.assign(1,23);
  for (it = vectorInts.begin(); it != vectorInts.end(); ++it)
    std::cout<<*it<<" ";

  std::cout<<"\n\nAdding four elements to the vector\n";
  //assigning the value 3 to 4 elements of the vector
  vectorInts.assign(4,3);
  std::cout<<"vectorInts has "<<vectorInts.size()<<" elements\n";

  //printing the contents of vectorInts
  std::cout<<"VectorInts has these elements:\n";
  for (it = vectorInts.begin(); it != vectorInts.end(); ++it)
    std::cout<<*it<<" ";

  return 0;
}

```

输出：

```cpp
vectorInts has 0 elements
Adding 23 to the vector
23

Adding four elements to the vector
vectorInts has 4 elements
VectorInts has these elements:
3 3 3 3
```

# 8. push_back Elements

有时我们想全覆盖的形式，那可以用上面的assign，但是有时想从末尾追加的话：

```cpp
vectorInts.push_back(24);
vectorInts.push_back(25);
vectorInts.push_back(26);
vectorInts.push_back(27);
```

修改上面的示例代码：

```cpp
// constructing vectors
// constructing vectors
#include <iostream>
#include <vector>

int main ()
{
  //creating a vector of integers
  std::vector<int> vectorInts;
  //creating an iterator for the vector
  std::vector<int>::iterator it;

  std::cout<<"vectorInts has "<<vectorInts.size()<<" elements\n";

  std::cout<<"Adding 23 to the vector\n";
  vectorInts.assign(1,23);
  for (it = vectorInts.begin(); it != vectorInts.end(); ++it)
    std::cout<<*it<<" ";

  std::cout<<"\n\nAdding four elements to the vector\n";
  //assigning the value 3 to 4 elements of the vector
  vectorInts.push_back(24);
  vectorInts.push_back(25);
  vectorInts.push_back(26);
  vectorInts.push_back(27);
  std::cout<<"vectorInts has "<<vectorInts.size()<<" elements\n";

  //printing the contents of vectorInts
  std::cout<<"VectorInts has these elements:\n";
  for (it = vectorInts.begin(); it != vectorInts.end(); ++it)
    std::cout<<*it<<" ";

  return 0;
}
```

```cpp
vectorInts has 0 elements
Adding 23 to the vector
23

Adding four elements to the vector
vectorInts has 5 elements
VectorInts has these elements:
23 24 25 26 27
```

# 9. insert elements

通过insert，可以在任意我们想插入的地方添加新的value，比如：

```cpp
it  = vectorInts.begin() + 1;
vectorInts.insert(it, -1); // 在第一个元素后添加-1
```

比如：

```cpp
// constructing vectors
// constructing vectors
#include <iostream>
#include <vector>

int main ()
{
  //creating a vector of integers
  std::vector<int> vectorInts;
  //creating an iterator for the vector
  std::vector<int>::iterator it;

  vectorInts.push_back(0);
  vectorInts.push_back(1);
  vectorInts.push_back(2);
  vectorInts.push_back(3);
  vectorInts.push_back(4);
  vectorInts.push_back(5);
  vectorInts.push_back(6);
  vectorInts.push_back(7);

  std::cout<<"vectorInts has "<<vectorInts.size()<<" elements\n";

  //printing the contents of vectorInts
  std::cout<<"VectorInts has these elements:\n";
  for (it = vectorInts.begin(); it != vectorInts.end(); ++it)
    std::cout<<*it<<" ";

  //insert an element after the first element
  it  = vectorInts.begin() + 1;
  vectorInts.insert(it, -1);
  std::cout<<"\n\nAfter the insert\n";
for (it = vectorInts.begin(); it != vectorInts.end(); ++it)
    std::cout<<*it<<" ";

  //insert an element after the third element
  it  = vectorInts.begin();
  vectorInts.insert(it + 3, -2);
  std::cout<<"\n\nAfter the insert\n";
for (it = vectorInts.begin(); it != vectorInts.end(); ++it)
    std::cout<<*it<<" ";

  //insert an element after the third element
  it  = vectorInts.begin();
  vectorInts.insert(it + 5, -3);
  std::cout<<"\n\nAfter the insert\n";
for (it = vectorInts.begin(); it != vectorInts.end(); ++it)
    std::cout<<*it<<" ";

  return 0;
}
```

```cpp
vectorInts has 8 elements
VectorInts has these elements:
0 1 2 3 4 5 6 7

After the insert
0 -1 1 2 3 4 5 6 7

After the insert
0 -1 1 -2 2 3 4 5 6 7

After the insert
0 -1 1 -2 2 -3 3 4 5 6 7
```

# 10. emplace Elements

C++11中，针对顺序容器(如vector、deque、list)，新标准引入了三个新成员：` emplace_front、emplace和emplace_back`，这些操作构造而不是拷贝元素。这些操作分别对应`push_front、insert和push_back`，允许我们将元素放置在容器头部、一个指定位置之前或容器尾部。

当调用push或insert成员函数时，我们将元素类型的对象传递给它们，这些对象被拷贝到容器中。而当我们调用一个emplace成员函数时，则是将参数传递给元素类型的构造函数。

The difference makes emplace more efficient than insert in special cases.

- `insert:` copies objects into the vector.
- `emplace:` construct them inside of the vector.

如下代码：

```cpp
struct Foo
{
  Foo(int n, double x);
};

std::vector<Foo> v;
v.emplace(someIterator, 42, 3.1416);
v.insert(someIterator, Foo(42, 3.1416));
```

# 11. Add Elements Prog. Quiz

In this programming quiz, use these member functions to complete the functions in the header file.

- assign
- push_back
- insert
- emplace

```cpp
// constructing vectors
// constructing vectors
#include <iostream>
#include <vector>

using namespace std;

  void printVector(vector<int> vIn);
  void assignFunction(vector<int> vInts, int in);
  void pushBackFunction(vector<int> vInts, int in);
  void emplaceFunction(vector<int> vInts, int loc, int in);

 void printVector(vector<int> vIn)
 {//printing the contents of vIns
  //TODO: Finish the function
  vector<int>::iterator it;
  for (it=vIn.begin();it!=vIn.end();++it){
      cout << *it;
  }
 }

  void assignFunction(vector<int> vInts, int in)
  {
      cout<<"\nassigning "<<in<<" and printing the vector\n";
      //TODO: Finish the function
      int iSize = vInts.size();
      vInts.assign(iSize,in);

      printVector(vInts);
  }
  void pushBackFunction(vector<int> vInts, int in)
  {
      cout<<"\npush back "<<in<<" and printing the vector\n";
      //TODO: Finish the function
      vInts.push_back(in);
      printVector(vInts);
  }
  void emplaceFunction(vector<int> vInts,  int loc, int in)
  {
      vector<int>::iterator it;
      cout<<"\nemplacing "<<in<<" and printing the vector\n";
      //TODO: Finish the function
      it = vInts.begin() + loc;
      vInts.emplace(it,in);
      printVector(vInts);
  }
```

```cpp
// Goal: pracitce constructing vectors
// and their iterators
// Call the vector vFloat
// Call the iterator it

#include "main.hpp"

int main ()
{
  vector<int> vInts;
  vInts.assign(10, 5);
  printVector(vInts);
  assignFunction(vInts, 1);
  pushBackFunction(vInts, 2);
  emplaceFunction(vInts,1, 3);

  return 0;
}
```

输出：

```cpp
5 5 5 5 5 5 5 5 5 5
assigning 1 and printing the vector
1 1 1 1 1 1 1 1 1 1
push back 2 and printing the vector
5 5 5 5 5 5 5 5 5 5 2
emplacing 3 and printing the vector
5 3 5 5 5 5 5 5 5 5 5
```

上面的每一次输出，均不是基于前一次的结果，因为传递进入的值本身没有被改变。

# 13. clear Vector

We can remove all elements from the vector at the same time using the method ::clear.

```cpp
// constructing vectors
// constructing vectors
#include <iostream>
#include <vector>

int main ()
{
  //creating a vector of integers
  std::vector<int> vectorInts;
  //creating an iterator for the vector
  std::vector<int>::iterator it;

  vectorInts.push_back(0);
  vectorInts.push_back(1);
  vectorInts.push_back(2);
  vectorInts.push_back(3);
  vectorInts.push_back(4);
  vectorInts.push_back(5);
  vectorInts.push_back(6);
  vectorInts.push_back(7);

  std::cout<<"vectorInts has "<<vectorInts.size()<<" elements\n";

  //printing the contents of vectorInts
  std::cout<<"VectorInts has these elements:\n";
  for (it = vectorInts.begin(); it != vectorInts.end(); ++it)
    std::cout<<*it<<" ";

  //clear the vector
  vectorInts.clear();
  std::cout<<"\nAfter clear, the vector has these elements:\n";
  for (it = vectorInts.begin(); it != vectorInts.end(); ++it)
    std::cout<<*it<<" ";

  return 0;
}
```

```cpp
vectorInts has 8 elements
VectorInts has these elements:
0 1 2 3 4 5 6 7
After clear, the vector has these elements:
```

# 14. erase Vector

1. delete a single element from a vector For example:

```cpp
//erase the 5th element in the vector
vectorInts.erase(vectorInts.begin()+4);
```

2. delete a range of elements form a vector For example:

```cpp
//erase a range of elements in the vector
vectorInts.erase(vectorInts.begin()+1, vectorInts.begin()+3);
```

示例代码如下：

```cpp
// constructing vectors
// constructing vectors
#include <iostream>
#include <vector>

int main ()
{
  //creating a vector of integers
  std::vector<int> vectorInts;
  //creating an iterator for the vector
  std::vector<int>::iterator it;

  vectorInts.push_back(0);
  vectorInts.push_back(1);
  vectorInts.push_back(2);
  vectorInts.push_back(3);
  vectorInts.push_back(4);
  vectorInts.push_back(5);
  vectorInts.push_back(6);
  vectorInts.push_back(7);

  std::cout<<"vectorInts has "<<vectorInts.size()<<" elements\n";

  //printing the contents of vectorInts
  std::cout<<"VectorInts has these elements:\n";
  for (it = vectorInts.begin(); it != vectorInts.end(); ++it)
    std::cout<<*it<<" ";

  //clear the vector
  it = vectorInts.begin();
  vectorInts.erase(it + 4);
  std::cout<<"\nAfter clear, the vector has these elements:\n";
  for (it = vectorInts.begin(); it != vectorInts.end(); ++it)
    std::cout<<*it<<" ";

  return 0;
}
```

输出(删除了第四个之后的一个元素，即第五个元素)：

```
vectorInts has 8 elements
VectorInts has these elements:
0 1 2 3 4 5 6 7
After clear, the vector has these elements:
0 1 2 3 5 6 7
```

# 15. pop_back

We can remove the last element of a vector using ::pop_back.

```cpp
//pop the last element off the vector
vectorInts.pop_back();
```

pop_back **DOES NOT return a value!**，没有返回值。

```cpp
// constructing vectors
// constructing vectors
#include <iostream>
#include <vector>

int main ()
{
  //creating a vector of integers
  std::vector<int> vectorInts;
  //creating an iterator for the vector
  std::vector<int>::iterator it;

  vectorInts.push_back(0);
  vectorInts.push_back(1);
  vectorInts.push_back(2);
  vectorInts.push_back(3);
  vectorInts.push_back(4);
  vectorInts.push_back(5);
  vectorInts.push_back(6);
  vectorInts.push_back(7);

  std::cout<<"vectorInts has "<<vectorInts.size()<<" elements\n";

  //printing the contents of vectorInts
  std::cout<<"VectorInts has these elements:\n";
  for (it = vectorInts.begin(); it != vectorInts.end(); ++it)
    std::cout<<*it<<" ";

  //pop the last element off the vector
  vectorInts.pop_back();
  std::cout<<"\n\nAfter pop_back(), the vector has these elements:\n";
  for (it = vectorInts.begin(); it != vectorInts.end(); ++it)
    std::cout<<*it<<" ";

  //pop_back does not return the element that was removed.
  //For example, the line of code below will return an error.
  int a;
  //a = vectorInts.pop_back(); //this line does not compile
  return 0;
}
```

```cpp
vectorInts has 8 elements
VectorInts has these elements:
0 1 2 3 4 5 6 7

After pop_back(), the vector has these elements:
0 1 2 3 4 5 6
```

license: MIT

branch_name: refs/heads/master

extension: md

license_type: permissive


text:
# SR6

[BANE](./BANE) is able to output compressed background and rms images using the `--compress` option. If you have a compressed file and want to expand it to have the same number of pixels as your original image then you need to use SR6.

If you have an image that you, for some reason, want to compress using a super-lossy algorithm known as decimation, then SR6 is what you want.

Usage is:

```console
usage: SR6 [-h] [-o OutputFile] [-f factor] [-x] [-m MaskFile] [--debug] [--version] [--cite] [infile]

optional arguments:
  -h, --help     show this help message and exit

Shrinking and expanding files:
  infile         input filename
  -o OutputFile  output filename
  -f factor      reduction factor. Default is 4x psf.
  -x             Operation is expand instead of compress.
  -m MaskFile    File to use for masking pixels.

Other options:
  --debug        Debug output
  --version      show program's version number and exit
```

In order to be able to expand a file, the file needs to have some special keywords in the fits header. These are inserted automatically by BANE, but you could probably fidget them for yourself if you had the need.

You should be able to shrink any file that you choose.

repository: PaulHancock/Aegean

branch_name: refs/heads/main

gha_language: Python

gha_license_id: NOASSERTION

repo_name: PaulHancock/Aegean


text:
## Install

Copy `tan` directory to your Java project.

### Requirements

| Platform | Version
| :------  | :------
| Java     | 7+
| Android  | 7.0+

## Example

```java
import tan.Tanserver;
import java.io.IOException;

/* Initialize connection information.  */
Tanserver tan = new Tanserver("tanserver.org", 2579);

/* Get JSON string from the server.  */
tan.getJSON("api", "{}", new Tanserver.Callback() {

    @Override
    public void onSuccess(String jsonString) {
        System.out.println(jsonString);
    }

    @Override
    public void onFailure(IOException err) {
        /* An error has occurred, notify the user of network failure.  */
        System.out.println("ERROR! " + err.getMessage());
    }
});
```

### getJSON()

| Declaration        | `void` getJSON(String userApi, String jsonString, Callback callback)
| :------            | :------
| Description        | Get JSON string from the server.
| Param `userApi`    | API provided by the server.
| Param `jsonString` | The JSON string that will be sent to the server.
| Param `callback`   | Needs to implement `void onSuccess(String jsonString)` and `void onFailure(IOException err)`. When the request is successful, `onSuccess()` will be called, otherwise `onFailure()` will be called.
| Note               | Android must add `<uses-permission android:name="android.permission.INTERNET"/>` in `AndroidManifest.xml`

### Common reasons why onFailure() is called

1. The client is offline or the server is not started.

2. SSL handshake failed.

3. API does not exist.

4. `jsonString` is an invalid JSON string.

5. `jsonString` is too large and exceeds `client_max_body_size`.

repository: tansrv/tanserver-connectors

repo_name: tansrv/tanserver-connectors


text:
  COVID-19 Outbreak Tracker

Woohoo!! 🎉🎉 Finally its shipped in the real-world. 📦

Well, Upon learning & practicing **ReactJS & Gatsby** I have came up with an idea to build a real-world application & my minds popped out with this awesome idea! ⚡ Been building this since last 6-7 days. ⏱ Aaaand finally its here. **MY FIRST EVER OPENSOURCE CONTRIBUTION!** Self learning really paid off! 💯

I've learnt so many things in the way of making this application 📝 and ways to do the same work in different ways. Specially from [Ben Awad](https://www.youtube.com/channel/UC-8QAzbLcRglXeN_MY9blyw) 🥇 is truly an inspiration to me. Apart from that I wanna thank to these mentors as well for being there to teach people in a effective way 💯

1. 🔥 [Traversy Media](https://www.youtube.com/channel/UC29ju8bIPH5as8OGnQzwJyA)
2. 🔥 [Scott from LevelUpTuts](https://www.youtube.com/channel/UCyU5wkjgQYGRB0hIHMwm2Sg)
3. 🔥 [Dev' ed](https://www.youtube.com/channel/UClb90NQQcskPUGDIXsQEz5Q)
4. 🔥 [Ihatetomatoes](https://www.youtube.com/user/ihatetomatoesblog)

----------------------------------------------------------

✅ This is a simple, beautiful ⚛ **React & Gatsby** application that fetches public APIs & shows global coronavirus outbreak data. Public data from _[http://worldometers.info](http://worldometers.info)._

## 🚀 Quick start

1.  **Clone this Repo**

    Clone this repo by typing following command

    ```shell
    git clone https://github.com/cyrus43/covid-19kenya
    ```

2.  **Start developing.**

    Navigate into your new site’s directory and start it up.

    ```shell
    npm i
    cd covid-19kenya/
    gatsby develop
    ```
    or
    ```shell
    npm i
    cd covid-19kenya/
    npm start
    ```

3.  **Open the source code and start editing!**

    Your site is now running at `http://localhost:8000`!

-----

## ⚡ Core features

1.  **Application config in one place**

2.  **100% SEO optimized**

3. **100% PWA**

4. **Data passing with ```props```**

5. **Built with ```sass```**

6. **Modular file structure** and more..

----

## 🚀 Contribution

✅ _If you wanna contribute to #OpenSource community, please create a *PR*._

  ### ***Contributors***
  - 💻 Author: [Cyrus mbithi](https://twitter.com/cyrusmbithi92)

gha_language: JavaScript

gha_license_id: MIT


text:
> [Go to French version](/fr/Règlements/Décrets,%20ordonnances%20et%20règlements%20statutaires/2015/86.md)

# First Nations Elections Regulations

**SOR/2015-86**

Enabling authorities:
- [FIRST NATIONS ELECTIONS ACT](/en/Acts/Statutes%20of%20Canada/2014/c.%205.md)

Date registered: 2 April 2015

----------

His Excellency the Governor General in Council, on the recommendation of the Minister of Indian Affairs and Northern Development, pursuant to section 41 of the [First Nations Elections Act](/en/Acts/Statutes%20of%20Canada/2014/c.%205.md)[a], makes the annexed First Nations Elections Regulations.

a: S.C. 2014, c. 5

## Interpretation

**Definitions**

**1**

- **(1)** The following definitions apply in these Regulations.

***Act*** means the [First Nations Elections Act](/en/Acts/Statutes%20of%20Canada/2014/c.%205.md). (*Loi*)

***quorum***, in relation to the council of a First Nation, means a majority of the members of the council, or five members if the council consists of nine or more members. (*quorum*)

***Register number*** means the number assigned to a person registered under section 5 of the [Indian Act](/en/Acts/Revised%20Statutes%20of%20Canada/I/I-5.md). (*numéro de registre*)

**[Indian Act](/en/Acts/Revised%20Statutes%20of%20Canada/I/I-5.md)**

- **(2)** Unless the context otherwise requires, other words and expressions used in these Regulations have the same meaning as in the [Indian Act](/en/Acts/Revised%20Statutes%20of%20Canada/I/I-5.md).

## Electoral Officer and Deputy Electoral Officers

**Appointment of electoral officer**

**2**

- **(1)** The council of the First Nation must, by resolution, appoint an electoral officer or, if it is not possible for the council to form a quorum, the Minister must appoint an electoral officer, who
	- **(a)** has not been found guilty of an offence under the Act within the last two years before the appointment; and
	- **(b)** is certified in accordance with subsection (2).

**Certification**

- **(2)** A person is certified if they successfully complete a training program that is approved by the Minister on the responsibilities of the electoral officer under the Act and these Regulations.

**Revocation of certification**

- **(3)** The certification is revoked if the electoral officer is found guilty of an offence under the Act.

**Appointment of deputy electoral officer**

- **(4)** The electoral officer may appoint one or more deputy electoral officers.

## Voters List

**Provision of information**

**3**

- **(1)** At least 65 days before the day on which an election is to be held
	- **(a)** the First Nation must provide the electoral officer with the information set out in subsection (2), if the First Nation holding the election has assumed control of its own membership under section 10 of the [Indian Act](/en/Acts/Revised%20Statutes%20of%20Canada/I/I-5.md); and
	- **(b)** the Registrar must provide the electoral officer with the information set out in subsection (2), if the Band List of the First Nation holding the election is maintained in the Department under section 11 of the [Indian Act](/en/Acts/Revised%20Statutes%20of%20Canada/I/I-5.md).

**Compilation of list**

- **(2)** The electoral officer must compile a voters list that contains the following information:
	- **(a)** the names of all electors, in alphabetical order; and
	- **(b)** each elector’s band membership or Register number or, if the elector does not have a band membership or Register number, their date of birth.

**Revision of list**

- **(3)** The electoral officer must revise the voters list if it is demonstrated that
	- **(a)** an elector’s name has been omitted from the list;
	- **(b)** an elector’s name is incorrectly set out in the list; or
	- **(c)** the name of a person not entitled to vote is included in the list.

**Demonstration of omission and inclusion**

- **(4)** For the purposes of subsection (3),
	- **(a)** a person may demonstrate that an elector’s name has been omitted from, or incorrectly set out in, the voters list by presenting to the electoral officer written evidence from the Registrar or from the First Nation that the elector is in the Band List and will be at least 18 years of age on the day of the election; and
	- **(b)** a person may demonstrate that the name of a person not entitled to vote has been included in the voters list by presenting to the electoral officer written evidence that that person is not in the Band List or will not be a least 18 years of age on the day of the election.

**Addresses**

**4**

- **(1)** At least 65 days before the day on which an election is to be held, the First Nation must provide the electoral officer with a list setting out the last known postal address and email address of each elector who does not reside on the reserve.

**Provision of electors’ names and addresses**

- **(2)** On the request of a candidate for election as chief or councillor, the electoral officer must provide the candidate with a list of the names of electors and the address of any elector who has consented to have their address released to the candidates.

## Nomination Meeting

**Notice of nomination meeting**

**5**

- **(1)** At least 25 days before the day on which a nomination meeting is to be held, the electoral officer must
	- **(a)** post a notice of the nomination meeting and a list of the names of electors in one conspicuous place on the reserve; and
	- **(b)** send by mail and email a notice of the nomination meeting, a voter declaration form and a form on which the elector may request a mail-in ballot to the addresses provided under subsection 4(1).

**Content of notice**

- **(2)** A notice of a nomination meeting must contain the following information:
	- **(a)** the date, time, duration and location of the nomination meeting;
	- **(b)** the number of positions to be filled;
	- **(c)** a description of the manner in which an elector can nominate a candidate or second the nomination of a candidate;
	- **(d)** a statement that an elector must not nominate more than one candidate for each position to be filled in accordance with subsection 9(4) of the Act;
	- **(e)** the date on which the election is to be held and the location and hours of operation of each polling station;
	- **(f)** the date on which any advance poll will be held and the location and hours of operation of each advance polling station;
	- **(g)** the electoral officer’s name, phone number, fax number, postal address and email address;
	- **(h)** a statement that the elector may permit the electoral officer to release their address to the candidates;
	- **(i)** a statement that, if the elector wants to receive a mail-in ballot, they must make a written request to the electoral officer;
	- **(j)** if the council of the First Nation has, by resolution, imposed a candidacy fee under section 8, a statement that sets out the fee to be paid by each candidate; and
	- **(k)** a statement that a nominee who wishes to become a candidate must remit to the electoral officer or deputy electoral officer, by 6 p.m. on the third day following the day on which the nomination meeting closes,
		- **(i)** a signed declaration accepting the nomination and attesting to their eligibility to be a candidate under the Act, and
		- **(ii)** any applicable candidacy fee, in the form of cash, certified cheque, money order or electronic transfer payable to the electoral officer.

**Record of names**

- **(3)** The electoral officer must record the names of electors to whom a notice of the nomination meeting was sent or delivered, the postal address and email address of those electors and the date on which the notice was sent or delivered.

**Voter declaration form**

- **(4)** A voter declaration form must be signed by the elector and attest to the following information:
	- **(a)** the elector’s name;
	- **(b)** the name of the elector’s band and the elector’s Register or band membership number; and
	- **(c)** the elector’s date of birth.

**Witness**

- **(5)** A voter declaration form must contain the name, address, telephone number and signature of a witness who is at least 18 years of age and who attests to the fact that the person completing and signing the voter declaration form is the person whose name is set out in the form.

**Witness**

- **(6)** The voter declaration form of the elector who enlisted the assistance of another person under subsection 17(2) must be signed by a witness that attests to the fact that the elector is the person whose name is set out in the form and that the ballot was marked in the manner directed by the elector.

**Nomination of candidates**

**6**

- **(1)** An elector may nominate a candidate and second the nomination of a candidate,
	- **(a)** by delivering or sending by mail, email or fax to the electoral officer a nomination and a voter declaration form; or
	- **(b)** by orally nominating the candidate or seconding the nomination of the candidate at the nomination meeting.

**Witness**

- **(2)** The witness to the elector’s signature appearing on the voter declaration form is not to be considered a seconder to the nomination.

**Mailed nominations**

- **(3)** Mailed nominations that are not received by the electoral officer before the beginning of the nomination meeting are void.

**Nomination meeting**

**7**

- **(1)** A nomination meeting for an election must be held at least 35 days before the date of the election.

**Reading nominations**

- **(2)** At the beginning of the nomination meeting, the electoral officer must read aloud the nominations that have been received.

**Two nominations**

- **(3)** If the same person receives two written nominations for the same position, the second nomination is considered to second the first nomination.

**Duration**

- **(4)** A nomination meeting must remain open for at least three hours.

## Candidates

**Candidacy fee**

**8** The council of a First Nation may by resolution impose a candidacy fee of up to $250 on every candidate for election as chief or councillor.

**Declaration and fee**

**9**

- **(1)** To become a candidate, a nominee must remit to the electoral officer or deputy electoral officer, by 6 p.m. on the third day following the day on which the nomination meeting closes,
	- **(a)** a signed declaration accepting the nomination for no more than one of the positions for which they have been nominated and attesting to their eligibility to be a candidate under the Act; and
	- **(b)** if the council of a First Nation has passed a resolution referred to in section 8, the candidacy fee, in the form of cash, certified cheque, money order or electronic transfer payable to the electoral officer.

**In trust**

- **(2)** The electoral officer must hold the candidacy fees in trust.

**Withdrawal of candidacy**

**10**

- **(1)** A candidate may withdraw their candidacy at any time prior to the close of the polls by submitting to the electoral officer a written declaration of withdrawal, signed by the candidate in the presence of the electoral officer, a justice of the peace, a notary public or a commissioner for oaths.

**Death of candidate**

- **(2)** A candidate who dies before the close of the polls is considered to have withdrawn their candidacy.

**Candidacy fee**

- **(3)** A candidate who withdraws forfeits the candidacy fee and is considered to have received no votes.

## Elections

**Close of nomination meeting**

**11** As soon as feasible after the deadline set out in subsection 9(1) expires, the electoral officer must
- **(a)** if there is only one candidate for chief, declare that person to be elected by acclamation;
- **(b)** if the number of candidates for councillor does not exceed the number of positions to be filled, declare those persons to be elected by acclamation;
- **(c)** if there are more candidates than the number of positions to be filled, announce that an election will be held on the date set out in the notice referred to in paragraph 5(2)(e); and
- **(d)** if, after candidates have been declared elected in accordance with paragraph (a) or (b), the number of positions filled is less than the number of positions that are required to be filled for the council of the First Nation to have quorum, post and send a notice of another nomination meeting in the manner described in subsection 5(1).

**Notice of acclamations**

**12**

- **(1)** If, after candidates have been elected by acclamation, the number of positions filled is greater than or equal to the number of positions that are required to be filled for the council of the First Nation to have quorum, the electoral officer must post in at least one conspicuous place on the reserve, and mail to every elector who does not reside on the reserve for whom an address was provided under subsection 4(1), a notice that sets out the names of the persons who have been elected by acclamation and states that an election will not be held.

**Refund of candidacy fee**

- **(2)** Candidates who are elected to the position of chief or the position of councillor by acclamation are, for the purposes of section 11 of the Act, considered to have received more than five percent of the total votes cast and are entitled to be refunded the candidacy fee.

**Return of candidacy fee**

- **(3)** The electoral officer must return the candidacy fee to candidates elected by acclamation as soon as feasible.

**Content of ballots**

**13**

- **(1)** As soon as feasible after the deadline set out in subsection 9(1) expires, the electoral officer must prepare ballots setting out
	- **(a)** the names of the candidates for chief in alphabetical order by surname; and
	- **(b)** the names of the candidates for councillor in alphabetical order by surname.

**Additional distinguishing information**

- **(2)** If two or more candidates have the same name, the electoral officer must add to the ballots any additional information that is necessary to distinguish between those candidates.

**Notice of election**

**14** No later than 30 days before the day on which the election is to be held, the electoral officer must post in at least one conspicuous place on the reserve a notice that sets out
- **(a)** the date on which the election is to be held and the location and hours of operation of each polling station;
- **(b)** the date on which any advance poll is to be held and the location and hours of operation of each advance polling station;
- **(c)** the date on which and the time and place at which the counting of the votes is to take place;
- **(d)** the number of positions to be filled;
- **(e)** a statement that, if the elector wants to receive a mail-in ballot, they must make a written request to the electoral officer and provide the electoral officer with proof of identity; and
- **(f)** the electoral officer’s name, phone number, fax number, postal address and email address.

**Mail-in ballot**

**15** An elector who wants to receive a mail-in ballot must make a written request to the electoral officer that includes a copy of their proof of identity.

**Mail-in ballot package**

**16**

- **(1)** No later than 30 days before the day on which the election is to be held, the electoral officer must mail to every elector who has made a written request a mail-in ballot package consisting of
	- **(a)** a ballot, initialed on the back by the electoral officer or deputy electoral officer;
	- **(b)** an outer return envelope that is pre-addressed to the electoral officer and, if the elector’s address is in Canada, is postage-paid;
	- **(c)** an inner envelope marked “Ballot” for insertion of the completed ballot;
	- **(d)** a voter declaration form;
	- **(e)** instructions regarding voting by mail-in ballot;
	- **(f)** the notice set out in section 14;
	- **(g)** a statement that the elector may vote in person at a polling station on the day of the election, or at an advance polling station if applicable, in lieu of voting by mail-in ballot, if
		- **(i)** they return the unused mail-in ballot to the electoral officer or deputy electoral officer, or
		- **(ii)** they provide the electoral officer or deputy electoral officer with a sworn affidavit stating that they have lost their mail-in ballot; and
	- **(h)** a list of the names of any candidates who were elected by acclamation.

**Six or more days before election**

- **(2)** If an elector makes a written request for a mail-in ballot six or more days before the day on which the election is to be held, the electoral officer must mail, or deliver at an agreed time and place, a mail-in ballot package to the elector as soon as feasible after receipt of the request.

**Voters list**

- **(3)** The electoral officer must indicate on the voters list, next to the name of each elector to whom a mail-in ballot package was mailed or delivered, that a package has been provided to that elector and keep a record of the date on which, and the address to which, each package was mailed or delivered.

**Mail-in ballot**

**17**

- **(1)** An elector may vote by mail-in ballot by
	- **(a)** marking the ballot with a cross, check mark or other mark that clearly indicates the elector’s choice, but does not identify the elector, next to the name of the candidates for whom they intend to vote;
	- **(b)** folding the ballot in a manner that conceals the candidates’ names and any marks on the ballot without hiding the initials on the back;
	- **(c)** placing the ballot in the inner envelope and sealing that envelope;
	- **(d)** completing and signing the voter declaration form;
	- **(e)** placing the inner envelope and the completed voter declaration form in the outer envelope; and
	- **(f)** delivering or mailing the mail-in ballot package to the electoral officer or deputy electoral officer before the time at which the polls close.

**Assistance of another person**

- **(2)** If an elector is unable to vote in the manner set out in subsection (1), the elector may enlist the assistance of another person.

**Voided mail-in ballot**

- **(3)** A mail-in ballot is void if the mail-in ballot package is not received by the electoral officer or deputy electoral officer before the time at which the polls close.

**Safekeeping of mail-in ballot**

- **(4)** The electoral officer or deputy electoral officer must ensure the safekeeping of the mail-in ballot packages until they are opened in accordance with section 22.

**Advance poll**

**18**

- **(1)** The electoral officer may establish an advance polling station at any location that the electoral officer considers suitable and hold an advance poll for the period beginning on the tenth day and ending on the fifth day before the day on which the election is to be held.

**Procedures**

- **(2)** The procedures set out in sections 20 and 21 apply to advance polling stations.

**On close of advance polling station**

- **(3)** As soon as the advance polling station closes, the electoral officer must seal the ballot box in a manner that prevents it from being opened without breaking the seal, place their initials on the seal, invite two witnesses to initial the seal, and ensure the safekeeping of the ballot box until the counting of the votes following the close of the polling stations on the day of the election.

**Polling stations**

**19** The electoral officer must establish at least one polling station on the reserve on the day of the election unless the electoral officer is unable to do so, in which case the electoral officer must establish a polling station as close to the reserve as possible.

**Polling station materials**

**20**

- **(1)** The electoral officer must, before the poll is open, ensure that each polling station is equipped with ballot boxes, ballots, materials for marking the ballots and any other necessary materials for the conduct of the vote.

**Compartments**

- **(2)** The electoral officer or deputy electoral officer must provide a compartment at each polling station where the electors can mark their ballots without being observed by any other person.

**Security**

- **(3)** The electoral officer or deputy electoral officer may designate a person to maintain order at a polling station.

**Hours**

- **(4)** Polling stations must be open from 9:00 a.m. to 8:00 p.m. on the day of the election.

**Candidate’s representatives**

- **(5)** A candidate is entitled to two representatives in a polling station.

**Sealed ballot box**

- **(6)** The electoral officer or deputy electoral officer must, before the polling station is opened, open the ballot box, call all persons present to witness that it is empty, seal the box in a manner that prevents it from being opened without breaking the seal and place it in a location that is visible to the voters.

**Seal**

- **(7)** The seal of a ballot box must not be broken and the ballot box must not be opened during the time that the polling station is open.

**Repeat of procedure**

- **(8)** If another ballot box is required during the time that the polling station is open, the electoral officer or deputy electoral officer must repeat the procedure set out in subsection (6).

**Ballot**

**21**

- **(1)** Subject to subsection (3), the electoral officer or deputy electoral officer must provide a ballot on which their initials have been placed to any person who has not voted at an advance poll, who attends at a polling station and whose name is set out in the voters list.

**Marked voters list**

- **(2)** The electoral officer or deputy electoral officer must place a mark on the voters list next to the name of each elector who was given a ballot.

**Mail-in ballot**

- **(3)** An elector who received a mail-in ballot package under section 16 may obtain a ballot and vote in person at a polling station if the elector
	- **(a)** returns the unused mail-in ballot to the electoral officer or deputy electoral officer; or
	- **(b)** provides the electoral officer or deputy electoral officer with a sworn affidavit stating that they have lost their mail-in ballot.

**Procedure**

- **(4)** After receiving a ballot, an elector must
	- **(a)** immediately proceed to the compartment provided for marking ballots;
	- **(b)** mark the ballot with a cross, check mark or other mark that clearly indicates the elector’s choice, but does not identify the elector, next to the name of the candidates for whom they intend to vote;
	- **(c)** fold the ballot in a manner that conceals the candidates’ names and any marks on the ballot without hiding the initials on the back; and
	- **(d)** give the ballot to the electoral officer or deputy electoral officer.

**Ballot box**

- **(5)** The electoral officer or deputy electoral officer must, without unfolding the ballot, verify the initials placed on it and return the ballot to the elector to deposit in the ballot box or, at the elector’s request, deposit it in the ballot box.

**Privacy**

- **(6)** Subject to subsection (7), while an elector is in the compartment provided for marking ballots, no other person is allowed to be in the compartment or be in a position to see the manner in which the elector marks their ballot.

**Assistance**

- **(7)** At the request of any elector who is unable to vote in the manner set out in subsection (4), the electoral officer or deputy electoral deputy must, in the presence of a witness that the elector has selected, assist that elector by marking their ballot in the manner directed by the elector and return the ballot to the elector to deposit in the ballot box or, at the elector’s request, deposit it in the ballot box.

**Note on voters list**

- **(8)** The electoral officer or deputy electoral officer must make a note on the voters list next to the elector’s name to indicate that the ballot was marked by the electoral officer or deputy electoral officer at the elector’s request.

**Spoiled ballot**

- **(9)** An elector who has inadvertently made their ballot unusable may return it to the electoral officer or deputy electoral officer and is, on one occasion, entitled to obtain another ballot, and the electoral officer or deputy electoral officer must write the word “cancelled” on the spoiled ballot and preserve it.

**Forfeiture**

- **(10)** Any person who has received a ballot and who declines to vote or who leaves the polling station without voting forfeits their right to vote at the election.

**Voters list**

- **(11)** In the case of a person who forfeits their right to vote, the electoral officer or deputy electoral officer must make a note on the voters list next to that person’s name to indicate that the person received a ballot and declined to vote and, if possible, write the word “declined” on the back of the ballot and preserve it.

**Entitlement to vote before closure**

- **(12)** Every elector who is inside the polling station at the time fixed for closing the poll is entitled to vote before the poll is to be closed.

**Rejection or acceptance of mail-in ballot**

**22** At the time and on the date set for the counting of the votes in the notice referred to in section 14, the electoral officer or deputy electoral officer must, in the presence of everyone present, open each envelope containing a mail-in ballot that was received before the close of the polls and, without unfolding the ballot,
- **(a)** reject the ballot if
	- **(i)** it is not accompanied by a voter declaration form or the voter declaration form is not signed or witnessed,
	- **(ii)** the name of the elector set out in the voter declaration form is not on the voters list, or
	- **(iii)** the voters list shows that the elector has already voted; or
- **(b)** place a mark on the voters list next to the elector’s name that is set out in the voter declaration form and deposit the ballot in a ballot box.

**Ballots**

**23** After all mail-in ballots have been deposited in a ballot box, the electoral officer or deputy electoral officer must, in the presence of everyone present, open all ballot boxes and
- **(a)** examine the ballots and reject any ballots
	- **(i)** that do not contain the electoral officer’s or deputy electoral officer’s initials, or
	- **(ii)** on which any marks appear that would identify the voter;
- **(b)** declare any portion of a ballot on which votes have been given for more candidates than are to be elected to an office as void with respect to the candidates for that office;
- **(c)** take note of any objection made by any candidate or their representative to any ballot found in the ballot box and decide any question arising out of the objection;
- **(d)** number any objection and place a corresponding number on the back of the ballot with their initials and the word “allowed” or “disallowed”, as the case may be;
- **(e)** from the ballots that are not rejected and from the portions of ballots that are not void, count the votes given for each candidate who has not withdrawn before the close of the polls; and
- **(f)** prepare and sign a statement of the number of votes for each candidate, the number of ballots that are rejected and the number of ballots that have a portion declared void.

**Declaration**

**24**

- **(1)** Subject to subsection (2), after the completion of the counting of the votes, the electoral officer must, in the presence of everyone present, declare to be elected the candidates having the highest number of votes.

**Five or fewer votes**

- **(2)** If the difference between the number of votes of a candidate with the highest number of votes — who would otherwise be declared elected — and another candidate for the same position is five or fewer, the electoral officer must establish a date, time and place for a recount of the votes cast for those candidates and announce that date, time and place in the presence of everyone present.

**Time of recount**

- **(3)** A recount must commence within 24 hours after the announcement by the electoral officer that a recount is necessary.

**Handling of ballots**

- **(4)** If the recount is not to be conducted immediately after the counting of the votes, the electoral officer must
	- **(a)** deposit all ballots in envelopes and seal them in a manner that prevents them from being opened without breaking the seal;
	- **(b)** place their initials on the seal and have any two people present do the same;
	- **(c)** deposit the sealed envelopes into a ballot box and seal that box in a manner that prevents it from being opened without breaking the seal; and
	- **(d)** ensure the safekeeping of the sealed ballot box until the time established for a recount.

**Recount**

- **(5)** On the date and at the time and place established for a recount, the electoral officer must open the sealed ballot box and the sealed envelopes in the presence of everyone present and conduct a recount.

**Declaration**

- **(6)** After the completion of the recount, the electoral officer must, in the presence of everyone present, declare to be elected the candidates having the highest number of votes and, in the case where a draw has been held in accordance with section 24 of the Act, any candidate who has won the draw.

**Statement**

- **(7)** Within four days after completion of the counting of the votes or, if there has been a recount, within four days after the recount, the electoral officer must
	- **(a)** sign and post, in a conspicuous place on the reserve, a statement indicating the number of votes cast for each candidate and the names of the elected candidates; and
	- **(b)** send a copy of the statement to the Department.

## Disposal of Ballots and Election Documents

**Retention of documents**

**25**

- **(1)** The electoral officer must deposit all ballots in envelopes, seal them and ensure their safekeeping along with other election documents, for a period of 120 days following the election.

**Destruction of documents**

- **(2)** At the end of the period set out in subsection (1), the electoral officer must destroy the ballots and election documents, unless they are served, in accordance with section 34 of the Act, with an application to contest the election.

## Candidacy Fee

**Return of candidacy fee**

**26** Within 30 days of the day on which the electoral officer declared the results of the election, the electoral officer must
- **(a)** refund the candidacy fee to every candidate who received more than five percent of the total votes cast; and
- **(b)** remit to the First Nation the candidacy fees of all candidates who received five percent or less of the total votes cast.

## Coming into Force

**
> S.C. 2014, c. 5 or on registration.
**

[*]**27** These Regulations come into force on the day on which the [First Nations Elections Act](/en/Acts/Statutes%20of%20Canada/2014/c.%205.md), chapter 5 of the Statutes of Canada, 2014 comes into force but if they are registered after that day, they come into force on the day on which they are registered.

*: [Note: Regulations in force April 2, 2015, *see* SI/2015-27.]

repository: TravisMadill/CanadianLaws

path: /en/Regulations/Statutory Orders and Regulations/2015/86.md

repo_name: TravisMadill/CanadianLaws


text:
# [siesta-action](https://github.com/marketplace/actions/siesta-action) [![Visitors(Today/Total)](https://hits.seeyoufarm.com/api/count/incr/badge.svg?url=https%3A%2F%2Fgithub.com%2FShantanuDash%2Fsiesta-action&count_bg=%2379C83D&title_bg=%23555555&icon=&icon_color=%23E7E7E7&title=Visitors(Today/Total)&edge_flat=false)](https://hits.seeyoufarm.com)
_Do consider starring this repo/action if you like it. Also let me know about any improvements or suggestions._

I use the [latest stable release ](https://gitlab.com/siesta-project/siesta/-/tree/rel-4.1) of  [Siesta](https://gitlab.com/siesta-project/siesta) and using github action perform small DFT calculations.

### Usage:
Keep your directory structure similar to the following.

**Note** : The only neccessary files are the:
  - input-directory with the input files
  - .github/workflows/ directory with the Your-worflow-file.yml

```
GitHub Repository
├── input-directory
│   ├──
│   ├── input-file.fdf
│   └── psf files
└── .github
    └── workflows
        └── Your-worflow-file.yml
```

The various input parameters are as follows:

```yaml
file-name:
  description: 'Name of the input file(with extention)'
  default: 'ip.fdf'
  required: true

dir-name:
  description: 'Name of the directory(folder) input files'
  default: 'ip'
  required: true

out-name:
  description: 'Name of the output file'
  default: 'ip.out'
  required: true
 ```
 #### A few important points before we use the action
 - action/checkout@v2 should be used before this action to clone the input files
 - actions/upload-artifact@v2 should be used after this action to save all the output

Here's a sample workflow which can be used directly:
```yaml
name: Manually triggered workflow To Run Siesta
on:
  workflow_dispatch:
    inputs:
      file-name:
        description: 'Name of the input file(with extention)'
        default: 'ip.fdf'
        required: true

      dir-name:
        description: 'Name of the directory(folder) input files'
        default: 'ip'
        required: true

      out-name:
        description: 'Name of the output file'
        default: 'ip.out'
        required: true
jobs:
  DFT-calculations-Siesta:
    runs-on: ubuntu-latest
    steps:
      - name: checkout
        uses: actions/checkout@v2

      - name: siesta-action
        uses<EMAIL_ADDRESS>        with:
            file-name: ${{ github.event.inputs.file-name }}
            dir-name: ${{ github.event.inputs.dir-name }}
            out-name: ${{ github.event.inputs.out-name }}

      - name: Upload Results
        uses: actions/upload-artifact@v2
        with:
            name: DFT-Output
            path: ${{ github.event.inputs.dir-name }}/

```

After you have added the input files and the workflow file, you can trigger/run the workflow as shown below:
1. Click on the actions tab. ![Step-1](assets/Image%201.png)
2. Click on the name of the workflow. ![Step-2](assets/Image%202.png)
3. Click on `Run workflow`  ![Step-3](assets/Image%203.png)
4. Change the input values according to your needs and then click on the green button `Run workflow` ![Step-4](assets/Image%204.png)
5. Click on the workflow name ![Step-5](assets/Image%205.png)
6. Click on the job name ![Step-6](assets/Image%206.png)
7. If the workflow runs successfully you should see something like this, otherwise you can click on the steps and see where it went wrong and start the workflow again. ![Step-7](assets/Image%207.png)
8. After the workflow has run sucessfully go back to the previous page( you might have to refresh). ![Step-8](assets/Image%208.png)

#### You can view an example repo [here](https://GitHub.com/ShantanuDash/temp)
[![ReadMe Card](https://github-readme-stats.vercel.app/api/pin/?username=ShantanuDash&repo=temp&show_owner=True)](https://GitHub.com/ShantanuDash/temp)

repository: fabionakagomi/siesta-action

repo_name: fabionakagomi/siesta-action


text:
## Steps to launch training

### Tencent GN10X-8xV100 (single node)
Launch configuration and system-specific hyperparameters for the GN10X-8xV100
single node submission are in the `config_GN10X.sh` script.

Steps required to launch single node training:

```
cd ../implementations/pytorch
docker build --pull -t mlperf-nvidia:object_detection .
CONT=mlperf-nvidia:object_detection DATADIR=<path/to/data/dir> LOGDIR=<path/to/output/dir> DGXSYSTEM=GN10X ./run_with_docker.sh


text:
### Number Plate Recognition

#### A yolov4 tiny model trained using Darknet and converted to pytorch to improve efficency

### Environment

Developed in an Arch environment but should work on Windows and Mac as well

### Screenshots

### Installation

#### Pytorch

#### Install Dependencies

Common
```bash
conda install numpy ninja pyyaml mkl mkl-include setuptools cmake cffi typing_extensions future six requests dataclasses
```

On Linux
```bash
# Add LAPACK support for the GPU if needed
conda install -c pytorch magma-cuda102  # or [ magma-cuda101 | magma-cuda100 | magma-cuda92 ] depending on your cuda version
```

On MacOS
```bash
# Add these packages if torch.distributed is needed
conda install pkg-config libuv
```

On Windows
```bash
# Add these packages if torch.distributed is needed.
# Distributed package support on Windows is a prototype feature and is subject to changes.
conda install -c conda-forge libuv=1.39
```

#### Get the PyTorch Source
```bash
git clone --recursive https://github.com/pytorch/pytorch
cd pytorch
# if you are updating an existing checkout
git submodule sync
git submodule update --init --recursive
```

#### Install PyTorch
On Linux
```bash
export CMAKE_PREFIX_PATH=${CONDA_PREFIX:-"$(dirname $(which conda))/../"}
python setup.py install
```
On macOS
```bash
export CMAKE_PREFIX_PATH=${CONDA_PREFIX:-"$(dirname $(which conda))/../"}
MACOSX_DEPLOYMENT_TARGET=10.9 CC=clang CXX=clang++ python setup.py install
```
[**More installation instructions**](https://github.com/pytorch/pytorch#from-source)

#### OpenCV
On Linux

install deoendencies

```bash
sudo apt-get update
sudo apt-get install build-essential
sudo apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
```

optional

```bash
sudo apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-devlibtiff-dev libjasper-dev libdc1394-22-dev
```

Build

```bash
mkdir ~/src
cd ~/src
git clone https://github.com/opencv/opencv.git
cd opencv
mkdir build && cd build

cmake -D CMAKE_BUILD_TYPE=RELEASE \
      -D CMAKE_INSTALL_PREFIX=/usr/local \
      -D INSTALL_PYTHON_EXAMPLES=ON \
      -D INSTALL_C_EXAMPLES=ON ..
make -j$(nproc)
sudo make install
```

#### Openalpr

Install openALPR for Ubuntu (Much easier and had a higher success rate)

```bash
# Install prerequisites
sudo apt-get install libopencv-dev libtesseract-dev git cmake build-essential libleptonica-dev
sudo apt-get install liblog4cplus-dev libcurl3-dev

# If using the daemon, install beanstalkd
sudo apt-get install beanstalkd

# Clone the latest code from GitHub
git clone https://github.com/openalpr/openalpr.git

# Setup the build directory
cd openalpr/src
mkdir build
cd build

# setup the compile environment
RUN cmake -D WITH_BINDING_JAVA=OFF \
  -D WITH_DAEMON=OFF \
  -D WITH_GPU_DETECTOR=ON \
  -D CMAKE_INSTALL_PREFIX:PATH=/usr \
  -D CMAKE_INSTALL_SYSCONFDIR:PATH=/etc ..

# compile the library
make -j4

# Install the binaries/libraries to your local system (prefix is /usr)
sudo make install

# Test the library
wget http://plates.openalpr.com/h786poj.jpg -O lp.jpg
alpr lp.jpg
```

Once you've completed the steps, go to the git repo you've cloned to your machine during the above installation and install the python bindings. runtime_data folder is also available in this repo, which will be helpful for next step.

```bash
cd openalpr/src/bindings/python/
sudo python3 setup.py install
```

#### Usage

```bash

~/openalpr/src/build
alpr test.png

python main.py -cfgfile <path to cfg file>\
               -weightfile <path to weight file>\
               -namesfile <path to classes.names file>\
               -output <name for the output video>\
               -source <path to video if you want to use video else index for webcams>
```

#### Up vote

If you found this useful please upvote this repository

#### Opimizations

Optimize openalpr for some perfomance gains

Modify file /etc/openalpr/openalpr.conf
```python
; This configuration file overrides the default values specified
; in /usr/share/openalpr/config/openalpr.defaults.conf
hardware_acceleration = 1
gpu_id = 0
gpu_batch_size = 10
```
Modify file /usr/share/openalpr/openalpr.defaults.conf
```python
max_detection_input_width = 800
max_detection_input_height = 600

detector = lbpgpu

skip_detection = 1

max_plate_angle_degrees = 30
```
change the resolution in cfg/yolov4.cfg

```bash

[net]
# Testing
#batch=1
#subdivisions=1
# Training
batch=64
subdivisions=16
width=608      #change the height and width lower resolutions
height=608     #improve perfomance and higher resolutions
channels=3     #improve accuracy
momentum=0.9
decay=0.0005
angle=0
saturation = 1.5
exposure = 1.5
hue=.1

```

#### Perfomance

- GPU Tesla T4 (without OCR)

| Res     | FPS (avg) |
| ------- | --------- |
| 800x800 |     56    |
| 640x640 |     68    |
| 608x608 |     72    |
| 480x480 |     91    |
| 416x416 |     98    |
| 320x320 |    106    |

I couldn't get Openalpr to build on colab. With ocr FPS will tank ~10fps

Added numba to optimize the inference loop a little bit its not much but its something
there was a bit of a perfomance boost for me  while running in an Intel Pentium N3710
especially in nms from 278avg to 177avg.

If you are much more familiar with numba you can help by optimizing it much more
if possible i dont own a GPU currently so cant test anything out.

#### Todo

1. Save cropped plate images

#### Credits

[**Darknet**](https://github.com/AlexeyAB/darknet)
YOLOv4 - Neural Networks for Object Detection (Windows and Linux version of Darknet )

[**pytorch-YOLOv4**](https://github.com/Tianxiaomo/pytorch-YOLOv4)
PyTorch ,ONNX and TensorRT implementation of YOLOv4

[**Pytorch**](https://pytorch.org/)
Tensors and Dynamic neural networks in Python with strong GPU acceleration

[**OpenCV**](https://opencv.org/)
Open Source Computer Vision Library

[**Openalpr**](https://github.com/openalpr/openalpr)
Automatic License Plate Recognition library

repository: rustam-forks/License-Plate-Recognition

repo_name: rustam-forks/License-Plate-Recognition


text:
# @year/1990

A year 1990

## Install

~~~
npm install @year/1990
~~~

## Usage

~~~js
const day = require('@year/1990/01/01')
~~~

Returns a [Date](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date) object.

## Related

* [@year/1989](https://github.com/antonmedv/year/tree/master/packages/1989)
* [@year/1991](https://github.com/antonmedv/year/tree/master/packages/1991)

## License

MIT

repository: antonmedv/year

repo_name: antonmedv/year


text:
<img src="https://static.openpay.com.au/brand/logo/openpay_logo_transparent.svg" width="170" alt="Openpay Logo">

Introduction
---------

### Objectives and Target Audience

This guide describes the process of installing and configuring the Openpay Widget / Extension to Magento 2.X. You should have an existing Magento installed.

Widgets explain how Openpay is a flexible, interest-free payment method. It showcases the minimum and maximum order value, allowing the user to choose an option to pay weekly or fortnightly, over available months.

### What is Openpay Widget?
Openpay Widgets is a Magento Extension with a predefined set of configuration options.

### Placement of the Widgets

Openpay Widget plugin is a set of widgets, when installed and activated will show the widgets on the following pages:

- Info Belt – Top section of the HomePage
- Product Catalog / Listing Page
- Product Detail Page
- Cart Page
- Checkout Page
- Landing Page

Compatibility
---------
- PHP 5.6 or later
- Magento 2.x to 2.4.X

Requirements
---------
- At least 5.6 or the later version of PHP
- cURL extension for PHP
- JSON extension for PHP
- Multibyte String extension for PHP
- Magento should be pre-installed on your web server. The extension has been tested on Magento version 2.x to 2.4.X
- SSL: A valid security certificate is required to work over a secure channel (HTTPS) from the Magento Admin Panel or while submitting the form data from the storefront. Self-signed SSL certificates are not supported
- Curl (version 7.20.0 - 7.4.0)

[Magento 2.3 technology stack requirements](https://devdocs.magento.com/guides/v2.3/install-gde/system-requirements.html)

[Magento 2.4 system requirements](https://devdocs.magento.com/guides/v2.4/install-gde/system-requirements.html)

Installation
---------

### Composer

Use [composer](https://getcomposer.org) to install the Openpay payment module. Follow the [installation instructions](https://getcomposer.org/doc/00-intro.md) if you do not already have
composer installed.

Once composer is installed, execute the following command in your project root to install this library:

```sh
composer require opy/widgets
```

Using server SSH Access, please go to project root folder and run the following commands to ensure the configuration tasks are run, database schema is updated and Magento's cache is cleared.

- php bin/magento setup:upgrade

- php bin/magento setup:static-content:deploy -f

- php bin/magento cache:flush

Or, you can do it manually from the admin by following the path: `System` > `Tools` > `Cache Management` and then clicking on the `Flush Magento Cache` button.

#### Important note

Please make sure you have already installed the Openpay payment plugin to make the widgets work as expected.

Admin Setup
---------

In the backend, expand `Openpay  Widgets -> General Configuration -> Enable Widgets -> Yes -> Save Config`

To set up `Openpay Widgets`, fill out the fields as per your Openpay account configuration. (This information will be provided by your Openpay Ecommerce Manager or Account Manager).

###### Command to run the Payment Plugin:

```sh
composer require opy/module-payment
```

Check with Openpay / Ecommerce Manager for your Plan Tiers configuration.


text:
---
layout: page
navtitle: Readium Candidate Statements
title: Readium Candidate Statements
permalink: /membership/elections/
---
This page has links to all the board candidate's statements from 2014 to 2021.
<hr>

 - [2014](./2014/)
 - [2015](./2015/)
 - [2016](./2016/)
 - [2017](./2017/)
 - [2018](./2018/)
 - [2020](./2020/)
 - [2021](./2021/)
 - [2023](./2023/)

gha_language: HTML


text:
IFabrikator CMS Core
========================

Welcome to the IFabrikator CMS Core - a basic concept project for
building CMS applications based on Symfony 2 Standard Edition

1) Basic concepts
----------------------------------

Any internet based application or a site can be looked at as a hierarchical list of
web pages (sections and subsections). A web page is a very simple structure consisting of:

  * Page title

  * Meta-keywords and meta-description tags

  * Breadcrumbs

  * Menus

  * One or more content areas

The appearance  of a web page is determined by an html-template and css files.

Thus a web page at data level can be easily presented by a number of database
table fields:

    CREATE TABLE `Page` (
        `id` int(11) NOT NULL AUTO_INCREMENT,
        `parent_id` int(11) DEFAULT NULL,
        `site_id` int(11) DEFAULT NULL,
        `title` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
        `language` varchar(16) COLLATE utf8_unicode_ci NOT NULL,
        `slug` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
        `is_active` tinyint(1) NOT NULL,
        `is_logon_required` tinyint(1) NOT NULL,
        `url` varchar(1024) COLLATE utf8_unicode_ci NOT NULL,
        `redirect` varchar(1024) COLLATE utf8_unicode_ci NOT NULL,
        `meta_description` longtext COLLATE utf8_unicode_ci NOT NULL,
        `meta_keywords` longtext COLLATE utf8_unicode_ci NOT NULL,
        `template` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
        PRIMARY KEY (`id`),
        KEY `IDX_B438191E727ACA70` (`parent_id`),
        KEY `IDX_B438191EF6BD1646` (`site_id`),
        CONSTRAINT `FK_B438191EF6BD1646` FOREIGN KEY (`site_id`) REFERENCES `Site` (`id`),
        CONSTRAINT `FK_B438191E727ACA70` FOREIGN KEY (`parent_id`) REFERENCES `Page` (`id`)
    ) ENGINE=InnoDB AUTO_INCREMENT=5 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci

### Dynamic routing

The classic MVC concept implies a configuration fixed route. When the request hits the route it is dispatched
to the corresponding and again firmly fixed controller method.

When building a CMS this inflexible relation between route and controller method is a problem.
The actual web resource is structured by a site administrator with the help of
an administrative interface. So dynamic routing becomes a must.

Now what if we need several different types of content being placed on the same web page:
i.e. a text and a feedback form? By the way, we would prefer the types of content and their
order to be again decided by site administrator. Later we could change our mind and think of
the feedback form for example to be placed on a different page. How do we achieve that without
making changes to the controller source code?

### ApplicationController

IFabrikator provides a one and only entry point for all requests - ApplicationController.
This is a standard Symfony 2 controller class which dispatches all the requests to our
application (site) except other standard controllers, routes to which are placed above
the ApplicationController route in routing.yml.

### "Controls" and the "forward" method

A "control" is a functional unit placed on a web page. It can be referred to as a mini web
application. For instance a control can display a news list or an authorization form.
In the administrative interface we can see controls that display or edit the site's
page tree, user permissions and so on.

A control is relatively independent from a web page. In theory there should be no
problem to move a control to another place of a web page or to a different web page.
It can be deactivated or removed from page. A web page can contain one or more controls or
even contain none. In the world of Java this kind of mini web applications is called
portlets.

Thus from the IFabrikator CMS's point of view there are pages (Page entities in program code and DB),
which represent a site's structure, and controls placed on these pages (Control entities). A web resource
functionality is determined by presence of controls of different types attached to pages.

ApplicationController calls other controller classes (Control entities) using standard "forward"
method.

2) Installation
-------------------------------------

### Use Composer (*recommended*)

As Symfony uses [Composer][2] to manage its dependencies, the recommended way
to create a new project is to use it.

If you don't have Composer yet, download it following the instructions on
http://getcomposer.org/ or just run the following command:

    curl -s http://getcomposer.org/installer | php

Then, use the `create-project` command to generate a new Symfony application:

    php composer.phar create-project symfony/framework-standard-edition path/to/install

Composer will install Symfony and all its dependencies under the
`path/to/install` directory.

### Download an Archive File

To quickly test Symfony, you can also download an [archive][3] of the Standard
Edition and unpack it somewhere under your web server root directory.

If you downloaded an archive "without vendors", you also need to install all
the necessary dependencies. Download composer (see above) and run the
following command:

    php composer.phar install

### Create database

After finishing it you will have to create a database and put its connection parameters into parameters.yml
(must be first created from parameters.yml.dist)

Then create database tables:

    php app/console doctrine:schema:update --force

### Install fixtures

And load the project fixtures:

    php app/console doctrine:fixtures:load

### Configure your web-server

Create a virtual host in your web-server config with the name of symbare.ifedor.loc (comes with fixtures). You
are free to change it afterwards.

[1]:  http://symfony.com/doc/2.3/book/installation.html
[2]:  http://getcomposer.org/
[3]:  http://symfony.com/download

repository: fedorov-iv/symbare

repo_name: fedorov-iv/symbare


text:
# Sericeomyces sericifer (Locq.) Døssing, 1991 SPECIES

#### Status
SYNONYM

#### According to
The Catalogue of Life, 3rd January 2011

#### Published in
in Knudsen & Hansen, Nordic Jl Bot. 11(4): 481 (1991)

#### Original name
Pseudobaeospora sericifera Locq., 1952

### Remarks
null

repository: mdoering/backbone

path: /life/Fungi/Basidiomycota/Agaricomycetes/Agaricales/Agaricaceae/Leucoagaricus/Leucoagaricus sericifer/ Syn. Sericeomyces sericifer/README.md

repo_name: mdoering/backbone