text:
# Chapter 2 - Lexical Scope
There are two models of how scope work. Most common is lexical scope other is dynamic scope.

## Lex-Time
The lexing process examines a string of source code characters and assigns semantic meaning to the tokens as a result of some stateful parsing.

Lexical scope is based on where variables and blocks of scope are authored. Set in stone by the time the lexer processes your code. Think of scopes as bubbles.

Scope bubbles are defined by where the blocks of scope are written. Each function creates a new bubble of scope.

### Look ups
The structure and relative placement of these scope bubbles fully explains to the Engine all the places it needs to look to find an identifier.

Scope look-up stops once it finds the first match. The same identifier name can be specified at multiple layers of nested scope, which is called "shadowing" (the inner identifier "shadows" the outer identifier).

Global variables are also automatically properties of the global object (window in browsers, etc.) No matter where a function is invoked from, or even how it is invoked, its lexical scope is only defined by where the function was declared.

## Cheating Lexical
If lexical scope is defined only by where a function is declared, which is entirely an author-time decision, how could there possibly be a way to "modify" (aka, cheat) lexical scope at run-time? Cheating lexical scope leads to poorer performance.

While the eval(..) function can modify existing lexical scope if it takes a string of code with one or more declarations in it, the with statement actually creates a whole new lexical scope out of thin air, from the object you pass to it.

### Performance
Both eval(..) and with cheat the otherwise author-time defined lexical scope by modifying or creating new lexical scope at runtime.

The JavaScript Engine has a number of performance optimizations that it performs during the compilation phase.

No matter how smart the Engine may be about trying to limit the side-effects of these pessimistic assumptions, there's no getting around the fact that without the optimizations, code runs slower.

## Review
Lexical scope means that scope is defined by author-time decisions of where functions are declared. The lexing phase of compilation is essentially able to know where and how all identifiers are declared, and thus predict how they will be looked-up during execution.


text:
---
title: AWS ECS Integration Details
keywords:
tags: [integrations]
sidebar: doc_sidebar
permalink: integrations_aws_ecs.html
summary: Send AWS ECS data to Wavefront using cAdvisor or AWS Fargate.
---
[Amazon Elastic Container Service (ECS)](https://aws.amazon.com/ecs/) is Amazon's Docker container orchestration system. From the Amazon ECS website:

```quote
Amazon Elastic Container Service (ECS) is a highly scalable, high-performance container orchestration service that supports Docker containers and allows you to easily run and scale containerized applications on AWS. Amazon ECS eliminates the need for you to install and operate your own container orchestration software, manage and scale a cluster of virtual machines, or schedule containers on those virtual machines.
```

Wavefront supports an Amazon Web Services [built-in integration](amazon_ecs.html) that allows Wavefront to collect useful high-level metrics about ECS using the AWS CloudWatch API, but that's only part of your setup.

## Overview

The integration basics are covered in our [AWS ECS Integration](amazon_ecs.html) page.

This page provides detailed steps on how to install and configure the Wavefront ECS integration either by creating a cAdvisor task definition or by creating an AWS Fargate task definition. After you complete these steps, the integration provides:

- Monitoring of important CloudWatch metrics related to Amazon ECS.
- Monitoring of detailed metrics about individual containers, services, and clusters running in your AWS ECS environment.

### Prerequisites

- Access to Amazon Web Services.
- Access to a [Wavefront proxy](proxies_installing.html) - Preferably running in AWS or a place accessible to your ECS instances.
- Wavefront AWS integration - Parts of the ECS integration use CloudWatch metrics, which can be acquired by configuring the Wavefront AWS integration.

### Configure the AWS Integration

Set up the [AWS integration](integrations_aws_metrics.html). This allows Wavefront to collect useful high-level metrics about ECS using the AWS CloudWatch API.

**Note:** To ensure that dashboards display correctly, use only the default EC2 instance name for the ECS cluster.

## Create AWS ECS Fargate Task Definition for Wavefront

You can set up Wavefront to collect the data from AWS Fargate containers. These steps create an ECS task definition that ensures the Wavefront Collector automatically runs on each Fargate instance in your ECS cluster.

After you've performed the setup, you can view and examine the data in our AWS Fargate dashboard in your Wavefront instance. See the screenshot at the bottom of this page.

1. Within AWS Services, navigate to **ECS**.
1. Click **Task Definitions**, then **Create new Task Definition**.
  ![create task def](images/create_new_task_definition.png)
1. Select the Fargate launch type and click **Next Step**.
![fargate launch type](images/aws_fargate.png)

1. Scroll to the bottom of the new Task Definition form and click **Configure via JSON**.
  1. Delete the content and paste the following snippet into the JSON form field:
  ```
  {
        "family": "wavefront-task-def",
        "networkMode": "awsvpc",
        "containerDefinitions": [
            {
                "name": "tomcat-example",
                "image": "tomcat:latest",
                "essential": true,
				"portMappings": [
					{
					  "hostPort": 8080,
					  "protocol": "tcp",
					  "containerPort": 8080
					}
				]
            },
			{
                "name": "fargate-metrics-collector",
                "image": "wavefronthq/wavefront-fargate-collector:latest",
                "essential": true,
                "environment": [],
				"portMappings": [
					{
					  "hostPort": 8000,
					  "protocol": "tcp",
					  "containerPort": 8000
					}
				],
				"command": [
					"-port=8000",
					"-storage_driver=wavefront",
					"-storage_driver_options=storage_driver_wf_proxy_host=YOUR_PROXY_ADDRESS storage_driver_wf_metric_prefix=aws.fargate.ecs."
				]
			}
        ],
        "requiresCompatibilities": [
            "FARGATE"
        ],
        "cpu": "256",
        "memory": "512"
  }
   ```
  2. In the JSON form, set the `storage_driver_wf_proxy_host` property to the proxy address and port of your Wavefront instance. Use the format `<wavefront_proxy_IP>:<port>` and click **Save**.
1. Click **Create** at the bottom of the Task Definition form.
2. After the task is created, click **View Task Definition**, select **Actions > Run Task** and specify the task information. Most fields are self-evident, here are a few.
  * **Cluster**--Select the cluster on which your task has to run.
  * **Number of tasks**--Select a number (minimum 1).
  * **Task Group** (Optional)--Task group name for identifying a set of related tasks.
  * **Launch Type**--Fargate

   ![run task](images/aws_fargate_run_task.png)
9. Click **Run Task**.

## Create AWS ECS EC2 Task Definition for Wavefront

Wavefront maintains an image of cAdvisor that includes a Wavefront storage driver. These steps create an ECS task definition that ensures the Wavefront cAdvisor container automatically runs on each EC2 instance in your ECS cluster.

After you've performed the setup, you can view and examine the data in our AWS EC2 dashboard in your Wavefront instance. The screenshots at the bottom of this page show the AWS EC2 dashboard and the AWS Fargate dashboard.

1. Within AWS Services, navigate to **ECS**.
1. Click **Task Definitions**, then **Create new Task Definition**.
  ![create task def](images/create_new_task_definition.png)
1. Select the launch type that you want your task to be compatible with and click **Next Step**.

   ![select launch type](images/select_launch_type.png)
1. Scroll to the bottom of the new Task Definition form and click the **Configure via JSON** button.
   1. Delete the content and paste the [JSON example](https://raw.githubusercontent.com/wavefrontHQ/integrations/master/aws-ecs/example-task-definition.json) into the JSON form field.
   1. In the JSON form, set the `-storage_driver_wf_proxy_host` property  to `<wavefront_proxy_ip_address>:<port>` and click **Save**.
1. Click the **Create** button at the bottom of the Task Definition form.
1. Select **Actions > Run Task** and specify the task information:
   ![actions menu](images/actions_run_task.png)
   1. In the **Cluster** dropdown, select the cluster on which your task has to run.
   2. Enter the number of tasks (minimum 1) of same definition you want to run.
   3. (Optional) Enter the Task Group name to identify a set of related tasks.
1. In the **Placement Templates** dropdown select **One Task Per Host**. This ensures that each EC2 instance in your ECS cluster has a Wavefront cAdvisor task.

   ![actions menu](images/one_task_per_host.png)
1. Click **Run Task**.

## View ECS Container Metrics

You can view the ECS Container metrics on the dashboards we set up for the two different options.

Here's a screenshot of the AWS ECS dashboard.
![db aws ecs](images/db_aws_ecs.png)

Here's a screenshot of the AWS ECS Fargate dashboard.

![ecs fargate](images/aws_ecs_fargate.png)

repository: srujann/docs

branch_name: refs/heads/master

extension: md

license_type: permissive

repo_name: srujann/docs


text:
[![DOI](https://zenodo.org/badge/176246204.svg)](https://zenodo.org/badge/latestdoi/176246204)

# trait-geo-diverse-dl

This repository develops the application of deep learning to species distribution
modelling (DL-SDM). The repository consists of three sub-projects:

- [pilot](pilot) - contains the same occurrences and GIS data as used in this
  [application](https://github.com/naturalis/trait-geo-diverse-ungulates) of a MaxEnt SDM to determine ecological niches of the world's ungulates.
  The purpose of this subproject is to assess the performance of DL-SDM in a
  like for like comparison with MaxEnt modelling.
- [data_extended](data_extended) - incorporates additional occurrence data. The
  purpose of this project is to develop insight into the number of occurrences
  that needs to be collected for credible modelling.
- [data_GIS_extended](data_GIS_extended) - also incorporates additional GIS
  data.

![](images/presentation_image_resize.png)

license: MIT

repository: Redkenn/trait-geo-diverse-dl

repo_name: Redkenn/trait-geo-diverse-dl


text:
---
layout: default
---

# 1994-12-16 - Many-Hop Remailer Ping Messages

## Header Data

From: "L. McCarthy"<EMAIL_ADDRESS>To<EMAIL_ADDRESS>Message Hash: e5e2d81c5cb08c245b9a884de6eb7b25fe101b25d4cbe9d10299b8240d145389

Message ID<EMAIL_ADDRESS>Reply To: _N/A_

UTC Datetime: 1994-12-16 00:31:52 UTC

Raw Date: Thu, 15 Dec 94 16:31:52 PST

## Raw message

```
{% raw  %}From: "L. McCarthy"<EMAIL_ADDRESS>Date: Thu, 15 Dec 94 16:31:52 PST
To<EMAIL_ADDRESS>Subject: Many-Hop Remailer Ping Messages
Message-ID<EMAIL_ADDRESS>MIME-Version: 1.0
Content-Type: text/plain

-----BEGIN PGP SIGNED MESSAGE-----

- -----BEGIN PGP SIGNED MESSAGE-----

Some agent periodically sends out several hundred line remailer ping messages
(unencrypted) with many hops planned; a sample is included below. It would be
nice if the parties responsible would remove my address (and, for example,
Eric Frias' address and the leri address) since I'm not operating a remailer
at this address any longer. Adding in the newer remailers might be a wise move
if this is supposed to produce results representative of the entire remailer
bramble. In the immortal words of Kibo, "PLEASE UPDATE YOUR COSMIC PARAMETERS."
Thanks.

- - -L. Futplex McCarthy

Forwarded message:
> To<EMAIL_ADDRESS>>
> ::
> Anon-To<EMAIL_ADDRESS>>
> ::
> Anon-To<EMAIL_ADDRESS>>
> ::
> Anon-To<EMAIL_ADDRESS>[many lines omitted...]

- -----BEGIN PGP SIGNATURE-----
Version: 2.6.1

iQCVAwUBLvDfr2f7YYibNzjpAQEIygP/cQrEeELKjsqFjzEj7vD7R0/NfZQ52H8G
WhHF+11aQfYN+XJYKv/Ah8/ifyxHNhwjaidk3GS3IEkVganOUv7Ay3ax4HsZiR/J
vkSKL0tW7YB2ZM4faL+Z4y71P6190VzERu4Wsuku89KLxEhGvSLGSZQOQamK3LgG
938B9PolQmg=
=gyVz
- -----END PGP SIGNATURE-----
- ---
[This message has been signed by an auto-signing service.  A valid signature
means only that it has been received at the address corresponding to the
signature and forwarded.]

-----BEGIN PGP SIGNATURE-----
Version: 2.6.2
Comment: Gratis auto-signing service

iQBFAwUBLvDg/ioZzwIn1bdtAQG6mgF+IxsKDp5IhQnSGEivFBuXBhX7lS44Unvs
DKzub0n5Zsu/D1cSAAKGoE8zYUXQq4mi
=U3vB
-----END PGP SIGNATURE-----

{% endraw %}
```

## Thread

+ Return to [December 1994](/archive/1994/12)

+ Return to "["L. McCarthy" <lmccarth@ducie.cs.umass.edu>](/authors/l_mccarthy_lmccarth_at_ducie_cs_umass_edu_)"

+ 1994-12-16 (Thu, 15 Dec 94 16:31:52 PST) - Many-Hop Remailer Ping Messages - _"L. McCarthy"<EMAIL_ADDRESS>

gha_license_id: MIT


text:
---
title: 'Css Paint'
tags: ['PaintWorklet', 'Houdini']
---

## Support

[caniuse](https://caniuse.com/?search=PaintWorklet)

[Is Houdini ready yet](https://ishoudinireadyyet.com/)

## CSS Paint API 설명

[mozilla PaintWorklet](https://developer.mozilla.org/en-US/docs/Web/API/PaintWorklet)

[paintapi](https://wit.nts-corp.com/2018/05/02/5214)

-   [구글 문서](https://developers.google.com/web/updates/2018/01/paintapi) 번역

[CSS Paint API로 그래픽 그리기](https://tympanus.net/codrops/2021/06/18/drawing-graphics-with-the-css-paint-api/)

[mozila CSS Painting API](https://developer.mozilla.org/en-US/docs/Web/API/CSS_Painting_API)

## 글 들

[Houdini 가 생긴 이유?](https://www.smashingmagazine.com/2016/03/houdini-maybe-the-most-exciting-development-in-css-youve-never-heard-of/)

## awesome

[awesome-css-houdini](https://github.com/CSSHoudini/awesome-css-houdini)

## polyfill

[link](https://github.com/GoogleChromeLabs/css-paint-polyfill)

## example

[tooltip](https://github.com/liyangguang/css-houdini)

[CodePen Home Houdini Circles PaintWorklet](https://codepen.io/bramus/pen/PoGbbzL)

[css houdini](https://css-houdini.rocks/)

gha_language: JavaScript


text:
# Causal Inference for Social Discrimination Reasoning

________________________________________________________________

			Python Version Dependency
________________________________________________________________

	1-  Python version 3.6.3
	2-  Anaconda Spyder version 3.2.4

________________________________________________________________

			Package Dependencies
________________________________________________________________

### We need to install the following packages

	1-  pandas
	2-  ExtraTreesClassifier from sklearn.ensemble
	3-  LogisticRegression from sklearn.linear_model
	4-  numpy
	5-  time
	6-  threading
	7-  math
	8-  matplotlib.pyplot
	9-  tree from sklearn
	10- StringIO from sklearn.externals.six
	11- pydotplus (sometimes graphviz has to be installed seperately)

_________________________________________________________________

			Input File and Basic Execution
_________________________________________________________________

	1-  Input file needs to be in a csv format
	2-  Input file should contain two binary attributes "class" & "sensitive" with the following values
	    class:
	    	positive class (e.g., yes) --> 1
	    	negative class (e.g., no) --> 0
	    sensitive:
	    	treatment group (e.g., male) --> 1
	    	control group (e.g., female) --> 0
	3-  Keep the input file in the same directory as the python files or else you would have to set an
		absolute/relative path in the instructions.py file
	4-  Open instruction.py
	5-  Add file name as a parameter to the call of "CDDTPSA" function at the bottom of the code
	6-  Code has the following set of files
		instructions.py
		Propensity_infogain.py
		Propensity_chisquare.py
		NormalizeAndSplit
		KNN.py
		Nuteralize_Instances.py
		Create_Bins.py
		Probability_Graph.py
		RDC_RDNC_Graph.py
		Group_Comp_Graph.py
		Regression_Tree.py
	7-  Output would contain 6 graphs and 2 trees in PDF format in the same folder else you would have to set
	an absolute/relative path in the last four files


text:
# Patches for pfetch on SerenityOS

## `0001-Use-bin-bash-as-the-shebang.patch`

Use /bin/bash as the shebang

/bin/sh is a symlink to Shell, which is not sh-compatible.

repository: SerenityOS/serenity

repo_name: SerenityOS/serenity


text:
# todos

# Todos List App With PHP , Composer And Ajax

Todos is a simple todo list app made with OOP PHP, MySQL , jQuery and Ajax.

## Installation:

- Import sql file (wedevs_todo.sql) to your database.
- Database is in (database) folder.
- Define Your project url without last slash (/) sing in custom.js file which is located (assets/js) folder.
- Change database info in (app/classes/Todo.php) in constructor function.

```php
// Change your db name with your Database name.
$con= mysqli_connect('localhost', 'username', 'pasword', 'database_name');
```

## Features:

- Add Tasks.
- Edit Tasks.
- Delete Single Tasks.
- Delete Bulk Tasks.
- Mark Tasks as Finished.
- Filter Completed Tasks.
- Filter Active Tasks.
- Filter Active and Completed Tasks.
- ...

### Enjoy!


text:
---
layout: post
title: Study Interaction Case and Graphics Update!
---
Zdravo, svima! :) (Hello, Everyone in Bosnian)

Last week, I mentioned that I was responsible for creating a case study. This study was to analyze how elementary students interacted with
specific words given to them. What I accomplished this week was detailing the study and progressing more in my graphics. To sum up, the study
is compromised of drawing the shapes for common words, in different manners. For example, the word loop might have 3 or 4 different
associated drawings. The student then will pick which shape they believe "loop" best represents. In addition, they will be asked to complete
a paper matching exercise and wil be shown a demonstration. This demonstration will consist of us (the researchers) explaining to the students
what we want them to understand from the word.

Moreover, to get a better idea of how to design our blocks better, we will conduct a color association evaluation. This will also help us figure
out why that color would match that particular word.

Now to talk about graphics. What I learned is that choosing to use the HTML5 canvas tag is not a deal breaker. Because of this, I opted
not to use it. I also learned about how I will mesh the blocks that I already created a file for with the new graphics file. I learned more about
the power of using the <div> tag and some other useful HTML knowledge that will make the creation of graphics easier.

My goal for this week is to write a detailed description of how our research team will conduct this study. And also write out why
we are doing a particular exercise. In other words, what exactly are we trying to find through doing this exercise with the students. What
are the "right" questions to ask them to get the best reponses for them. I chose a "mixed-medium", if you will, of conducting this study
because I predict students will provide us with different responses through different interactions.


text:
# isugifNF/blast

```
----------------------------------------------------
                                    \\---------//
      ___  ___        _   ___  ___    \\-----//
       |  (___  |  | / _   |   |_       \-//
      _|_  ___) |__| \_/  _|_  |        // \
                                      //-----\\
                                    //---------\\
      isugifNF/blast  v1.0.0
    ----------------------------------------------------
```

[Genome Informatics Facility](https://gif.biotech.iastate.edu/) | [![Nextflow](https://img.shields.io/badge/nextflow-%E2%89%A519.10.0-brightgreen.svg)](https://www.nextflow.io/)

---

### Introduction

**isugifNF/blast** is a [nextflow pipeline](https://www.nextflow.io/) for [BLAST](https://blast.ncbi.nlm.nih.gov/Blast.cgi)-ing a fasta sequence file against a genome.

The workflow processes many sequences from a large fasta file `--query`, splits them into smaller fasta files, and parallel [BLAST](https://blast.ncbi.nlm.nih.gov/Blast.cgi) them against a given genome `--genome`. The pipeline can be run on a local laptop `-profile local` or on an HPCC cluster `-profile ceres` or `-profile condo`.

### Installation and running on Ceres HPCC

Nextflow is already installed on Ceres HPCC. Therefore, running **isugifNF/blast** involves (1) allocating a debug node `salloc -N 1 -p debug -t 01:00:00`, (2) loading nextflow `module load nextflow`, and (3) running the pipeline `nextflow run isugifNF/blast`. The `--help` flag prints out the usage statement.

```
salloc -N 1 -p debug -t 01:00:00
module load nextflow
nextflow run isugifNF/blast --help
```

<details><summary>see usage statement</summary>

```
Usage:
      The typical command for running the pipeline is as follows:
      nextflow run parallelBLAST.nf --query QUERY.fasta --genome GENOME.fasta -profile local
      nextflow run parallelBLAST.nf --query QUERY.fasta --dbDir "blastDatabaseDirectory" --dbName "blastPrefixName" -profile local

      Mandatory arguments:
       --query                        Query fasta file of sequences you wish to BLAST
       --genome                       Genome from which BLAST databases will be generated
       or
       --query                        Query fasta file of sequences you wish to BLAST
       --dbDir                        BLAST database directory (full path required)
       --dbName                       Prefix name of the BLAST database
       -profile                       Configuration profile to use. Can use multiple (comma separated)
                                      Available: test, condo, ceres, local, nova

       Optional arguments:
       --outdir                       Output directory to place final BLAST output
       --outfmt                       Output format ['6 qseqid sseqid pident length mismatch gapopen qstart qend sstart send evalue bitscore qlen slen frames salltitles qcovs']
       --options                      Additional options for BLAST command [-evalue 1e-3]
       --outfileName                  Prefix name for BLAST output [blastout]
       --threads                      Number of CPUs to use during blast job [16]
       --chunkSize                    Number of fasta records to use when splitting the query fasta file
       --app                          BLAST program to use [blastn;blastp,tblastx,blastx]
       --help                         This usage statement.
```

</details>

### Dependencies if running locally

Nextflow is written in groovy which requires java version 1.8 or greater (check version using `java -version`). But otherwise can be installed if you have a working linux command-line.

```
java -version
curl -s https://get.nextflow.io | bash

# Check to see if nextflow is created
ls -ltr nextflow
#> total 32
#> -rwx--x--x  1 username  staff    15K Aug 12 12:47 nextflow
```

The pipeline **isugifNF/blast** can be run using singularity/docker containers or with a local [install of BLAST](https://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/LATEST/). For a local install, select the appropriate installer if you are on windows (`*.exe`), linux (`*linux.tar.gz`), or MacOS (`*.dmg`). There are a few different ways to install BLAST locally, including brew, ports, and conda.

```
# test run using locally installed blast
nextflow run isugifNF/blast -profile test

# test run using containers (docker/singularity)
nextflow run isugifNF/blast -profile test,docker
nextflow run isugifNF/blast -profile test,singularity

# test run on Ceres HPC
nextflow run isugifNF/blast -profile test,ceres
```

Docker/singularity runs will take a few minutes since it needs to download the ncbi/blast container. Subsequent runs will be faster.

### Credits

These scripts were originally written for use on Ceres and Condo HPCC by Andrew Severin ([@isugif](https://github.com/isugif)), Siva Chudalayandi ([@Sivanandan](https://github.com/Sivanandan)), and Jennifer Chang ([@j23414](https://github.com/j23414))

repository: isugifNF/blast

repo_name: isugifNF/blast


text:
# WSUS-Maintenance
WSUS configure and maintain documentation/solution

The script is an adaptation of the original script by Mike Galvin.
It has been enhanced and further developed.

# Features and Requirements
* Auto updating with the Github repository (https://github.com/SANCLA/WSUS-Maintenance)
* The script will run the WSUS server cleanup process, which will delete obsolete updates, as well as declining expired and superseded updates.
* The script can optionally create a log file and e-mail the log file to an address of your choice.
* The script can be run locally on a WSUS server, or on a remote sever.
* The script requires that the WSUS management tools be installed.
* The script has been tested on Windows 10 and Windows Server 2016.

# How to install
1. Create a new folder
   C:\SANCLA-scripts\WSUS-Maintenance
2. Copy the contents of this repository to the newly created folder
3. Create a new task in Windows Task Planner to run daily or weekly:
   * Run as SYSTEM user, run with highest privileges
   * Stop the task if it rungs longer than: 8 hours
   * Program/script: powershell.exe
   * Add arguments: -ExecutionPolicy Bypass -Command "& 'C:\SANCLA-scripts\WSUS-Maintenance\Wsus-Maintenance.ps1' -Server [your-wsus-server] -Port 8530 -Update yes"
   * Replace -Server [your-wsus-server]  with your WSUS server, eg wsus01.domain.local
   * Replace -Port 8530 with the used WSUS port. Default is 8530. Check IIS binding for the correct port...

# Credits
* https://gallery.technet.microsoft.com/scriptcenter/WSUS-Maintenance-w-logging-d507a15a

# Documentation

## Generating A Password File
The password used for SMTP server authentication must be in an encrypted text file. To generate the password file, run the following command in PowerShell, on the computer that is going to run the script and logged in with the user that will be running the script. When you run the command you will be prompted for a username and password. Enter the username and password you want to use to authenticate to your SMTP server.
Please note: This is only required if you need to authenticate to the SMTP server when send the log via e-mail.

1. $creds = Get-Credential
2. $creds.Password | ConvertFrom-SecureString | Set-Content c:\scripts\ps-script-pwd.txt

After running the commands, you will have a text file containing the encrypted password.
When configuring the -Pwd switch enter the path and file name of this file.

## Configuration
The table below shows all the command line options available with descriptions and example configurations.

| Command Line Switch | Mandatory | Description | Example |
| --- | --- | --- | --- |
| -Server | Yes | The WSUS server that should be cleaned. | wsus01.domain.local |
| -Port | Yes | The port that the WSUS service is running on. | 8530 |
| -Update | Yes | If the script will automatically update. Default will update unless included. | no |
| -SendTo	| No | The email address to send the log file to. |<EMAIL_ADDRESS>|
| -From | No* | The email address that the log file should be sent from. |<EMAIL_ADDRESS>|

\* This switch isn’t mandatory but is required if you wish to email the log file.

repository: SANCLA/WSUS-Maintenance

repo_name: SANCLA/WSUS-Maintenance


text:
# Steal This Software!

This document lists features and techniques in Petmail that could (should!)
be leveraged independently in other projects. I hope that Petmail, regardless
of its success or failure as an actual product, can have a positive influence
other projects. So please consider stealing these tricks in your own work.

## "nodedir" for Development and Deployment

Like we did in Tahoe-LAFS, each Petmail node stores all of its state in a
single "base directory" or "node directory" ("nodedir"), created by the
"create-node" command. The default location of the nodedir is under the
user's home directory, in a subdirectory named after the project itself
(~/.petmail, or perhaps "~/Library/Application Support/Petmail" on OS-X). But
it is trivial to put it elsewhere (using arguments in commands like "petmail
create-node NODEDIR" and "petmail start NODEDIR", and options in commands
like "petmail --basedir=NODEDIR open"), which makes it easy to spin up
multiple nodes on a single machine, for testing a "local testgrid".

In addition, Petmail puts a short script (also named "petmail") in the node
directory, which simply adds a "--basedir=" option and delegates to the
original "petmail" executable. It is easier to run "./NODEDIR/petmail
COMMAND" than to run "bin/petmail --basedir=NODEDIR COMMAND".

Since the script is the only executable in the nodedir (and it contains no
subdirectories), this also enables tab-completion when using multiple nodes
in a single parent directory. E.g. if the testgrid contains nodes "n1", "n2",
"n3", etc, then you can type "./n1 TAB TAB open RETURN" and will execute
"./n1/petmail open".

### Keeping All State in SQLite

Glyph recommended this to me many years ago: any machine-generated state
should be stored in a single-file database (i.e. SQLite), so it can be
manipulated atomically by the runtime code. I took this further and put all
configuration in the database too, so that the entire state of the node can
be captured by a single software version (git hash) plus the database
contents.

The downside is that you can't really use external tools to modify the node's
configuration (like how you can edit tahoe.cfg as a text file): basically
you, the program author, must provide code (subcommands of your main tool) to
configure everything. That's a drag. But the upside is that there's no
confusion about where the state or configuration lives, and there's no
hard-to-manage race between the user modifying their config and the program
modifying its state (both are atomic DB transactions).

For example, in Tahoe we'd kind of like to store the list of known servers in
a text file (one per line), because that'd be easy for a human to edit. But
the default mode is that the introducer client manages that list, so this
text file should be machine-modified. It only makes sense to edit this list
yourself if you've previously configured your node to *not* use the
introducer. So we'd wind up with one piece of code (the
connection-establishing client) that always reads from the list-of-servers
file, a separate piece of code (the introducer client) which maybe rewrites
this file, potential data-losing races if the user opens the file in an
editor while the introducer client is modifying it, concerns about if/how to
preserve comments and extra information during rewrites, and no good way for
the connection code to automatically learn that the server-list has changed.

This database could be encrypted to protect node state against passive
attackers who can read disk (perhaps from a backup) but not RAM. Petmail
doesn't do this (yet), but Pond does. The key would have to be provided when
first launching the node. Encrypting the whole database would be easiest
(http://sqlcipher.net/) but would interfere with CLI frontends retrieving the
access token (see below). This could be mitigated by encrypting only specific
rows or tables (basically everything but the access tokens), or by putting
access tokens in a separate place.

## Secure All-HTTP Frontend, Access Tokens, Event Channels

All Petmail frontends (both the CLI and the browser-based GUI) work through
HTTP APIs. These APIs all use HTTP POST requests with a JSON request body and
return a JSON response body. All of these APIS require an access token,
delivered as a property of the JSON request body (not as a URL query
argument, since URLs tend to leak through malware-URL-scanning features and
Referrer headers). The node (usually) only listens on the loopback interface
(<IP_ADDRESS>) so the "connection" doesn't traverse a real network and doesn't
require encryption to be safe. No cookies are used, ever, so there is no
ambient authority, which removes vulnerability to confused deputy and CSRF
attacks.

The frontend must somehow get an access token. CLI applications do this by
reading the token directly out of the database (if you can read the nodedir,
you can see the node's entire state anyways, so this does not really change
the privilege model).

The web frontend is launched with a CLI command ("petmail open"), which
unfortunately can only use HTTP GET (not POST). To avoid revealing the secret
access token in the URL, "petmail open" creates a single-use "opener token",
writes it into the database, then opens the browser to a redirection page,
adding the opener token in a queryarg. If the opener token is good, the node
interpolates the access token into the redirection page, which self-submits a
POST (with the access token as a hidden input field) to retrieve
"control.html", the single-page frontend app. The access token is also
interpolated into this control page, where it is retrieved by Javascript code
and submitted along with each API request. This way, the only secret that
appears in a URL should be expired by the time any external servers might
learn about it.

Most API calls trigger an asynchronous action on the node. They return
immediately, with a minimal "arguments were valid" success code. Most of the
information delivered to the frontend arrives through a single EventSource
channel (aka "SSE" Server-Sent Events). One API call creates the channel,
which returns an "event token" that is used to form the EventSource URL.
Other API calls subscribe and unsubscribe this channel to various message
types. This is used to notify the frontend about the consequences of its
requests (e.g. "invitation started"), as well as events triggered by other
nodes ("message received").

# Hash-Based Dependency Management

It is difficult to install (or worse, develop) software without making
oneself vulnerable to various attackers. These attackers can deliver "evil"
variants of the code you intended to run. The top-level installation can be
protected (i.e. it will successfully meet a policy of "I want to run the
software that everybody else is running", or "I want to run the software that
author X wants me to run") by using a cryptographic hash or public key as a
starting point. However most non-trivial software depends upon additional
libraries that come from other places. To avoid being infected by evil
variants of the dependencies, the build or install process needs to apply
cryptographic checks to them too.

**[note: petmail's "safe_develop" command is out of order, so this is not currently true]**

Petmail attempts to use a model in which the Petmail author locks down the
exact contents of the dependencies. Downstream users are responsible for
getting the "right" copy of Petmail, then Petmail will ensure that it gets
the "right" copy of everything else (excluding Python and the rest of the OS,
both of which are equally vulnerable avenues of attack). It does this (albeit
not entirely successfully, yet) by using Peep to download specific claimed
versions of its dependencies, then checking hashes of these tarballs before
unpacking and installing them.

Petmail is intended to be self-contained (using Python and the OS as its
TCB), and run from a virtualenv that is only populated by Petmail's setup
code.

repository: warner/petmail

gha_language: Python

repo_name: warner/petmail


text:
# RTC

Real-time clock allows you to get time from server. You can preselect any timezone on UI to get time on hardware in required locale.

**Sketch:** [RTC](https://github.com/blynkkk/blynk-library/blob/master/examples/Widgets/RTC/RTC.ino)

gha_license_id: NOASSERTION


text:
# Javadoc Tools for Visual Studio Code

This extension allows user to generate javadoc comments for all methods within a class.
Below commands are available for use -
* **Javadoc Tools: Export Javadoc** - This command allows you to export your Javadoc as well!
  > Below properties can be set to customize this command
  > * **javadoc-tools.generateJavadoc.workspaceSourceFolder** - Sets the default source folder which is read when Generating the Javadoc. Default value is the \"src\" folder in Workspace Root
  > * **javadoc-tools.generateJavadoc.targetFolder** - Sets the target folder where the Javadoc will be generated. Default path will be ${WorkspaceRoot}\\javadoc
  > * **javadoc-tools.generateJavadoc.runMode** - Set value to run in corresponding mode. Default value is \"-public\". Possible values are ["-package","-private","-protected","-public"]

* **Javadoc Tools: Generate Comments for Select methods** - This command allows user to choose the methods for which javadoc comments need to be created
* **Javadoc Tools: Generate Javadoc Comments for Open File** - Only generates Javadoc Comments for the open File in focus
* **Javadoc Tools: Generate Javadoc Comments for Workspace** - Generates Javadoc for all classes within the workspace.
    The files will be opened in the editor and the javadoc comments will be added. This command will not autosave the modified Files.
    >Warning: Running this command on a large workspace may cause performance degradation AND will require you to wait longer while it processes the files(which is worse)
* **Generate Javadoc Comments** - New option added to the context menu for Java Clases. Generates Javadoc Comments for the selected class.

## Features

### Export Javadoc for your workspace
![Export javadoc](/img/export_javadoc.jpg)

### Generate Comments for Select methods
![Generate Comment for Select Method](/img/select_method.gif)

### Generate Javadoc Comments from Context Menu
![Generate Javadoc](/img/genFromContext.png)

### Generate Javadoc Comments for Open File/Workspace
![Generate Comments For Open File](/img/CmdPallete.png)

## Issues
For any problems with the extension please raise an issue on the github page - [https://github.com/madhavd1/vscode-javadoc-tools](https://github.com/madhavd1/vscode-javadoc-tools)
## Requirements
* [Language Support for Java(TM) by Red Hat](https://marketplace.visualstudio.com/items?itemName=redhat.java)

## Most Recent Release Notes
### V1.4.0
- Added new command **Javadoc Tools: Export Javadoc** - This command allows you to export your Javadoc as well!
>Details on how to use the new command can be found above

>For complete version history, please see the changelog.


text:
---
title: The CAP Theorem and why State Machine Replication for Two Servers and One Crash
  Failure is Impossible in Partial Synchrony
date: 2023-07-09 14:00:00 -04:00
tags:
- lowerbound
- SMR
- dist101
author: Ittai Abraham
---

In 1999, Fox and Brewer published a paper on the [*CAP principle*](https://s3.amazonaws.com/systemsandpapers/papers/FOX_Brewer_99-Harvest_Yield_and_Scalable_Tolerant_Systems.pdf), where they wrote:

> **Strong CAP Principle.** Strong Consistency, High Availability, Partition-resilience: Pick at most 2.

At [PODC 2000](https://www.podc.org/podc2000/brewer.html), Brewer gave an invited talk where he popularized the [*CAP theorem*](https://sites.cs.ucsb.edu/~rich/class/cs293-cloud/papers/Brewer_podc_keynote_2000.pdf) (an unproven conjecture at the time), which was later formalized into a lower bound by [Gilbert and Lynch, 2002](https://www.comp.nus.edu.sg/~gilbert/pubs/BrewersConjecture-SigAct.pdf). In this post we will state and prove a slightly stronger result whose essence appeared as early as PODC 1984 by [DLS84 Theorem 5](https://groups.csail.mit.edu/tds/papers/Lynch/podc84-DLS.pdf):

**Theorem:** *It is impossible to implement [State Machine Replication](https://decentralizedthoughts.github.io/2019-10-15-consensus-for-state-machine-replication/) with safety and liveness given $n$ replicas for at least two clients and an adversary that can cause crash failures to $f\geq n/2$ replicas (and any number of crash failures to clients) in the partial synchrony model.*

The consensus version of this theorem appears in [DLS88 Theorem 4.3](https://groups.csail.mit.edu/tds/papers/Lynch/jacm88.pdf), the shared memory version in [Lynch96, Theorem 17.6](https://dl.acm.org/doi/book/10.5555/2821576), and the SMR version in [Gilbert and Lynch, 2002](https://www.comp.nus.edu.sg/~gilbert/pubs/BrewersConjecture-SigAct.pdf).

In hindsight, here is my interpretation of this impossibility:

* *Any system, in the face of a perceived 50-50 split, can either maintain **safety** or **liveness**, but not **both**.*

In a [previous post](https://decentralizedthoughts.github.io/2019-11-02-primary-backup-for-2-servers-and-omission-failures-is-impossible/), we showed a similar result: that $f \geq n/2$ *omission failures* is impossible to obtain both safety and liveness in the *lock step* (synchronous) model.

Before we show the (rather simple) proof, let's connect this to the CAP theorem.

### What does the CAP theorem say?

A system cannot have all three properties: **Consistency**, **Availability**, and **Partition tolerance**. We now formally explain these in the context of [State Machine Replication](https://decentralizedthoughts.github.io/2019-10-15-consensus-for-state-machine-replication/).

**Consistency**: sometimes also called **Safety** - that clients see responses that come from a common prefix of a total ordering of commands.

**Availability**: sometimes also called **Liveness** - that a non-faulty client will eventually get a response.

**Partition Tolerance**: this typically refers to the property that even if the adversary creates a partition of the system, each part will continue to provide the desired safety and/or liveness (consistency and/or availability). We will show how using crash failures and asynchrony we can essentially create such partitions.

Note that partition tolerance is different than safety and liveness, it's more of a failure model or adversary model. As [Gilbert and Lynch, 2012](https://groups.csail.mit.edu/tds/papers/Gilbert/Brewer2.pdf) conclude:

> The CAP Theorem, in this light, is simply one example of the fundamental fact that you cannot achieve both *safety* and *liveness* in an unreliable distributed system.

Another example of this tension between safety and liveness occurs in the [blockchain unsized setting](https://decentralizedthoughts.github.io/2022-03-03-blockchain-resource-pools-and-a-cap-esque-impossibility-result/).

### Fox and Brewer's categorization

Due to this impossibility, Fox and Brewer categorized all solutions into three buckets:

1. **CA**: Consistent and Available - but not partition tolerant. The trivial solution is a centralized system. As there is no resilience to partitions, it's not clear this is an interesting choice.

2. **AP**: Available even during Partitions - but not always consistent. This is the space of [weak consistency](http://www.bailis.org/blog/safety-and-liveness-eventual-consistency-is-not-safe/) and [eventual consistency](https://dl.acm.org/doi/pdf/10.1145/224056.224070) that is common for some applications (HTTP Web caching is a great example). This [approach](https://www.usenix.org/conference/osdi-04/secure-untrusted-data-repository-sundr), in the Byzantine setting, has several important [use cases](https://www.youtube.com/watch?v=UKdLJ7-0iFM) and connections to [Eclipse attacks](https://decentralizedthoughts.github.io/2022-08-14-new-DR-LB/) that will be a topic of future posts. [Bitcoin's consensus protocol](https://bitcointalk.org/index.php?topic=9399.0) is an example of AP. Ethereum's [inactivity leak mechanism](https://eth2book.info/capella/part2/incentives/inactivity/) is another example.

3. **CP**: Consistent even during Partitions - but not always available. The system is always consistent, but a partition may cause a loss of availability. The trivial solution is a centralized system. A more interesting solution is [Two Phase Commit](https://cs.brown.edu/courses/csci1380/s19/lectures/Day13_2019.pdf) which allows replication and obtains consistency at the cost of losing availability for partitions. Many distributed systems and databases use this approach, but even one failed/partitioned server can block the system. Can we get slightly better availability?

Yes, its a [trilemma](https://en.wikipedia.org/wiki/Trilemma). Yes, in [hindsight](https://twitter.com/el33th4xor/status/1191820205456023552?s=20&t=RcutJw0wQUsTmrO0OXzpXw), Brewer writes that "2 of 3" is [misleading](https://www.infoq.com/articles/cap-twelve-years-later-how-the-rules-have-changed/) because there are more nuanced properties that CAP does not capture. We highlight one such nuance and its deep connections to the impossibility proof.
### CP-MAJ-A

A powerful approach to address the CAP theorem is to obtain consistency even during partitions and in addition a *majority partition availability (MAJ-A)* property:

**Majority partition availability (MAJ-A)**: A partition that has *the majority of the replicas* can continue to have availability (in partial synchrony).

This is exactly what [Paxos](https://www.microsoft.com/en-us/research/publication/part-time-parliament/) obtains and what many modern state machine replication systems provide (Raft, etcd, etc). Recall that Paxos is:

* Always safe.
* Maintains availability, after [GST](https://decentralizedthoughts.github.io/2019-06-01-2019-5-31-models/), for the majority partition.

Requiring just the majority part (or super majority part, like $2/3$) to be available during a partition is the path that many BFT protocols use (for example, PBFT).

### A lower bound for both CAP and CP-MAJ-A

In one lower bound we prove:

* Impossible to get both safety and liveness (aka, consistency and availability) for a system with two servers that can be partitioned (via an adversary controlling one server failure and partial synchrony).

This impossibility has two implications:

1. It proves the CAP theorem - by showing a specific problematic partition into two parts of equal size. This is the essence of [Gilbert and Lynch 2002, Theorem 2](https://www.comp.nus.edu.sg/~gilbert/pubs/BrewersConjecture-SigAct.pdf).
2. It proves that the fault tolerance of CP-MAJ-A protocols like Paxos is optimal. The best one can hope for is $f<n/2$ failures, because when $f \geq n/2$ it is possible to partition the network into two parts of equal size. This is the essence of [DLS88, Theorem 4.3](https://groups.csail.mit.edu/tds/papers/Lynch/jacm88.pdf).

### The proof

This proof is word for word very similar to the proof for [lock step synchrony and omission failures](https://decentralizedthoughts.github.io/2019-11-02-primary-backup-for-2-servers-and-omission-failures-is-impossible/). The minor difference is the use of partial synchrony instead of omissions to create the partition. We provide the proof for completeness.

Assume a protocol that is [safe and live](https://decentralizedthoughts.github.io/2019-10-15-consensus-for-state-machine-replication/) for two replicas and two clients in partial synchrony and reach a contradiction. The adversary can cause one replica crash failure and any number of client crash failures.

### World A:
Client $1$ sends command $C1$, and the adversary crashes server $2$ and client $2$ (or causes a partition between $1$ and $2$). All messages arrive immediately. Since the protocol is safe and live, the system must notify client $1$ that command $C1$ is the only committed command.

### World B:
Client $2$ sends command $C2$, and the adversary crashes server $1$ and client $1$ (or causes a partition between $1$ and $2$). All messages arrive immediately. Since the protocol is safe and live, the system must notify client $2$ that command $C2$ is the only committed command.

### World C:
Client $1$ sends command $C1$, and client $2$ sends command $C2$. Using partial synchrony, the adversary delays all communication between:
1. Client $1$ and server $2$;
2. Client $2$ and server $1$;
3. Server $1$ and server $2$.

All other messages arrive immediately.

Observe that the view of server 1 in world A and world C is indistinguishable. Since in worlds A and C, client $1$ only communicates with server $1$, it also has indistinguishable views.

Similarly, the view of server 2 in world B and world C is indistinguishable. Since in worlds B and C, client $2$ only communicates with server $2$, it also has indistinguishable views.

So in world C, the two clients will see conflicting states and this is a violation of safety.

### Notes

The proof captures the essence of the inability to tell the difference between a *crash* and a *delay* in partial synchrony.

Perhaps the slight difference between this version and previous formulations is the explicit separation to servers and clients.

It's a good exercise to extend this proof to any $f \geq n/2$.

Abadi extends the CAP theorem to [PACELC](https://www.cs.umd.edu/~abadi/papers/abadi-pacelc.pdf) to highlight the importance of *latency* (a quantitative measure) not just availability (a binary measure) and the importance of latency even when there are no partitions.

### Acknowledgments

Many thanks to Kartik Nayak and Seth Gilbert for insightful comments and feedback.

Please leave comments on [Twitter](https://twitter.com/ittaia/status/1678517296157843456?s=20).

gha_language: HTML


text:
![](https://img.shields.io/badge/Microverse-blueviolet)

# Collaborative Project: The Next Web

> This is a microverse student project for working with hierarchy of elements and responsive design

![screenshot](https://github.com/Gnwin/Godwin-Eraani-Next-Web-project/blob/nextweb/assets/images/Screenshotnewtnw.png)

> This project clones the thenextweb.com article page interface design.
> The project includes the header, the main content, footer, media queries.

## Built With

- HTML
- CSS

## Live Demo

[Live Demo Link](https://raw.githack.com/Gnwin/Godwin-Eraani-Next-Web-project/nextweb/index.html)

## Getting Started

To get a local copy up and running follow these simple example steps.

### Prerequisites

> Browser
> Text editor of your choice.
> npm has to be installed in your system.
> install the following linters on your IDE;
    - Web hint.
    - Style lint.

### Setup

> `git clone https://github.com/Gnwin/Godwin-Eraani-Next-Web-project.git`

### Usage

> Clone the repo.
> Open the index.html from your browser to view.

### Run tests

Run `npx html-validator-cli --quiet --file index.html` to run the validator over the `index.html` file.

The `--quiet` will only list `errors`. If you also want to see the warnings use the `--verbose` flag instead.
As an alternative, you can also install the `html5validator` using `pip`:

```
pip install --user html5validator
html5validator index.html
```

## Authors

👤 **Godwin Nwachukwu**

- Github: [@Gnwin](https://github.com/Gnwin)
- Linkedin: [@n-gwin](https://www.linkedin.com/in/n-gwin/)

👤 **Eraani Ruiz**

- Github: [@eraanir](https://github.com/eraanir)

## 🤝 Contributing

Contributions, issues and feature requests are welcome!

Feel free to check the [issues page](https://github.com/Gnwin/Godwin-Eraani-Next-Web-project/issues).

## Show your support

Give a ⭐ if you like this project!

## 📝 License

This project is [MIT](https://www.mit.edu/~amini/LICENSE.md) licensed.

repository: Gnwin/Godwin-Eraani-Next-Web-project

repo_name: Gnwin/Godwin-Eraani-Next-Web-project


text:
+++
title = "GnuPG"
weight = 28
app_library = "default"
app_category = "Security"
app_typ = "default"
app_ns = "Bench"
app_id = "Bench.GnuPG"
app_version = "2.0.30"
app_categories = ["Security"]
app_libraries = ["default"]
app_types = ["default"]
+++

**ID:** `Bench.GnuPG`
**Version:** 2.0.30
<!--more-->

[Back to all apps](/apps/)

## Description
GnuPG is a complete and free implementation of the OpenPGP standard as defined by RFC4880 (also known as PGP).
GnuPG allows to encrypt and sign your data and communication, features a versatile key management system
as well as access modules for all kinds of public key directories.
GnuPG, also known as GPG, is a command line tool with features for easy integration with other applications.

## Source

* Library: [`default`](/app_libraries/default)
* Category: [Security](/app_categories/security)
* Order Index: 28

## Properties

* Namespace: Bench
* Name: GnuPG
* Typ: `default`
* Website: <https://gnupg.org>
* Responsibilities: [Leiningen](/apps/Bench.Leiningen), [Maven](/apps/Bench.Maven)


text:
---
layout: post
title: "Just fix it"
date: "2019-05-28"
categories: ["Web strategy"]
---

It's easy to slip into the mindset that you should test _everything_ - especially when you sell A/B tests for a living. But there are plenty of improvements you don't need to test; you should just make them. Here's a partial list, broken down by part of the site.

### Forms

You can, and should, optimize the forms on your site. But if you've got broken validation, misaligned form fields, illegible labels, fields so narrow the text is hard to read, or a hard-to-find Submit button, just fix it.

### Navigation

Navigation is optimizable too! You'll need plenty of traffic, and it's key that a significant proportion of visitors who convert actually _use_ the navigation, but it can be done.

However, if you've got 15 items in your top navigation bar and your analytics tell you that visitors are only using 3 of them, go ahead and move the rest to the footer. If your labels are confusing, and you've heard this from user testing or customer complaints, just fix them.

### Mobile experience

If your page isn't responsive, if buttons aren't reachable, or if you have paragraphs and form fields outside of the viewport, just fix it. The more mobile visitors you have, the more urgent this fix becomes.

### Page speed

Test your site out on [Google PageSpeed Insights](https://developers.google.com/speed/pagespeed/insights/).  An 80/20 approach is ideal - Google will tell you which fixes will deliver the biggest bump in load speed. So compress images, load some files asynchronously (not your A/B test snippet though!) and stop making your visitors wait.

### Metrics

If your analytics install is broken, first of all please know that _lots_ of analytics installs are broken. Almost all of them are at least a little broken. Just fix it.

* * *

In each of these cases, you _could_ set up an A/B test to measure the impact of your fixes. There are two likely outcomes there:

1. You measure lift, pat yourself on the back, and say "Yep, fixing our broken form validation helped." Sure, but your form isn't optimized; it's just no longer broken.
2. Your results are inconclusive, and you say "I guess the cluttered nav wasn't such a problem after all." You've improved your site from a usability and aesthetic perspective, but you're unable to feel _good_ about it because you didn't get a win.

Neither of these results is as satisfying as taking decisive action, fixing what's broken, and moving forward with optimization. So just fix it!

How about you? Are there items on this list that you think it's better to test first? Are there other changes that you routinely make without testing? Hit reply and let me know.

branch_name: refs/heads/main


text:
# Robolectric

This repository is the source for the docs that live at [robolectric.org](http://robolectric.org).

## Contributing

Make sure you are in the `master` branch before making changes. This is the default branch so git should put you there automatically.

Before submitting a pull request, view the docs locally to check that the content and layout are correct. The docs are built using [Jekyll](https://jekyllrb.com/).

Assuming you don't already have [Bundler](http://bundler.io/) installed, you will first need to do:

    gem install bundler

The above command may require `sudo`.

Then run:

    bundle install
    bundle exec jekyll server --incremental

This will build the docs and make them available at [localhost:4000](http://localhost:4000). Once
your pull request is accepted, the docs will be automatically built and pushed to the web by GitHub.

If you want to update Jekyll's dependencies, you can run:

    bundle update

## Javadocs

When Robolectric is released, the `source/javadocs` directory needs to be updated. We can get latest steps to generate Javadocs at [Robolectric Wiki's release part](https://github.com/robolectric/robolectric/wiki/Performing-a-Release#release).


text:
---
path: pCompFinalProposal
date: 2020-11-25T00:38:01.869Z
title: "Physical Computing: Final Proposal"
description: Final Project Proposal for Physical Computing
---
# 3D Topography

### Concept

For my final I want to explore the topography of Connecticut.

My family sold the house I grew up in this year, which was very sudden, but for the best. I had never really been that connected to my home state, but when I realized I may never go back there, I realized that I had come to really appreciate it as a place to grow up.

We ended up renting a house not too far from where I grew up to get through the pandemic, but it made me think about what it was that caused me to become so nostalgic. What kept popping into my head was that it is a beautiful place. It has lovely forests, beautiful colors, coastal towns, and even a few mountains.

Then, I stumbled across [this site](https://cteco.uconn.edu/viewers/ctelevation/), which shows the elevation in CT using colors.

When I started thinking about my pComp final, I wanted to do something that visualized something different. One of my favorite artists is Refik Anadol, who creates brilliant visuals with machine learning. Often, they almost look 3D, like they are about to pour into our world. I thought I could sort of make that happen.

So,

I am going to visualize the topography of CT using machine learning and Physical Computing.

### How it Works

I want to create a square of motors that linearly actuate pieces of wood or acrylic in and out depending on the elevation of the image loaded.

I think that using the same set of images could get repetitive, so I am going to use RunwayML to train a model that creates fake topographies of CT. I will then pass that into P5, which will simplify the image to a 5x5 grid, then based on the color in the grid. The motor will push it in or out a certain distance.

Unfortunately, linear actuator stepper motors are super expensive, so I found [this](https://www.youtube.com/watch?v=2vAoOYF3m8U&list=WL&index=9&t=5s&ab_channel=PotentPrintables) online. I am going to 3D print these, then use them to push things in and out.

My initial instinct was to use the large motor, but the more I think about it, the more I want to use the small ones. They take up less room, and I can use fewer and lighter materials to actuate.

### Structure

These are not done because I need to 3D print the piece before I can get exact measurements and see it better, but here are the approximate measurements for the large motor mod.

![Measurements](/../assets/pComp/final/measurements.png)

I think I want to put them next to one another equidistantly and drill the motor mounts into a thin sheet of wood.

![Mounting](/../assets/pComp/final/mounting.png)

Then, I want to slide that piece into a frame that I will build on the floor somehow. Like so:

![Frame and Connection](/../assets/pComp/final/frame.png)

### Progress

My plan before next class is to have the ML model DONE, the p5 sketch to simplify it done, and have 4 motors mounted and moving (not linked to sketch yet). It will be a push, but I can do it.


text:
# Letter Tools
[![Downloads](https://pepy.tech/badge/letter-tools/month)](https://pepy.tech/project/letter-tools)

- [Letter Tools](#letter-tools)
- [Intro](#intro)
- [Installing](#installing)
- [Examples](#examples)
- [Usage](#usage)

# Intro

This is a collection of tools for doing stuff with letters.

With a randint for letters and a range for letters and more.

# Installing
To install use this command:
```shell
$ pip install letter-tools
```

# Examples
First an example for randomly picking a letter.
```py
>>> from letter_tools import rand
>>> rand("a", "h")
c
>>> rand("a", "h")
g
>>> rand(1, 10)
5
>>> rand(1, 10)
7
```

Next an example for ranges.

```py
>>> from letter_tools import range
>>> for i in range("i"):
...     print(i)
...
a
b
c
d
e
f
g
h
i

>>> for i in range("d", "g"):
...     print(i)
...
d
e
f
g

>>> for i in range(1, 5):
...     print(i)
...
1
2
3
4
5

#Using step

>>> for i in range(1, 5, 3):
...     print(i)
...
1
4

>>> for i in range("d", "g", 3):
...     print(i)
...
d
g
```

Now an example for word score.

```py
# Find the score of a word a=1, b=2 and add up the numbers for all the letters

>>> from letter_tools import word_score
>>> print(word_score("hi"))
17
>>> print(word_score("letter"))
80
>>> print(word_score("hi", opt=""))
37
>>> print(word_score("letter", opt=""))
82
```

Here we will use the scrabble score, it does almost the same thing as word score but it uses scrabble scoring rules.  a: 1, b: 3, c: 3, d: 2, e: 1, f: 4, g: 2, h: 4, i: 1, j: 8, k: 5, l: 1, m: 3, n: 1, o: 1, p: 3, q: 10, r: 1, s: 1, t: 1, u: 1, v: 4, w: 4, x: 8, y: 4, z: 10
```py
>>> from letter_tools import scrabble_score
>>> print(scrabble_score("hi"))
5
>>> print(scrabble_score("letter"))
6
```

Now for the derandomize and randomize functions.

```py
>>> # For this we will use a word 'welcome' and randomize it.
>>> from letter_tools import randomize
>>> randomize("welcome")
'mlcoeew'
>>> # Now we will derandomize 'mlcoeew'
>>> from letter_tools import derandomize
>>> derandomize("mlcoeew")
['welcome']
```

Now for custom score.

```py
>>> from letter_tools import custom_score
>>> custom_score('letter', {'l': 13, 'e': 7, 't': 16, 'r': 1})
60
```

# Usage

Word score could help you with making a game to find certain scored words. Here is an example for 100 scored words. This will not eliminate invalid words.

```py
from letter_tools import word_score

while True:
    word = input("Could you give me some words that add up to 100 exactly?\na=1, b=2 and so on\n")
    if word_score(word) == 100:
        print(f"{word} is a 100 letter word correct!!!")
        break
    else:
        print(f"{word} is a {word_score(word)} not a 100 letter word wrong.")
```

We could also make a script to find 100 scored words.
For this we will use the requests library and using [Dwyl's english word set](https://github.com/dwyl/english-words) I generated a json you could use for this.

This is the code:

```py
from letter_tools import word_score
import requests

words = requests.get("https://raw.githubusercontent.com/hostedposted/letter-tools/master/words.json").json()
for i in words:
    try:
        if word_score(i) == 100:
            print(i)
    except KeyError:
        pass
```

repository: hostedposted/letter-tools

repo_name: hostedposted/letter-tools


text:
# AWS Global Accelerator Client IP Preservation for EC2 Instance
![Architecture](asset/architecture.png)

## Preparation
Rename `terraform.tfvars.template` to `terraform.tfvars`
```shell
mv terraform.tfvars.template terraform.tfvars
```
Edit `terraform.tfvars` and configure the `instance_type` and `network_interface_count` accordingly

## Stack Provisioning
```shell
terraform init
terraform plan
terraform apply
```

## Outputs
1.


text:
# The uiautomation module

:cn:[中文版介绍](https://github.com/yinkaisheng/Python-UIAutomation-for-Windows/blob/master/readme_cn.md)

Do not use 3.7.6 and 3.8.1, comtypes doesn't work in these two versions. Install an earlier version or the latest version.
https://github.com/enthought/comtypes/issues/202

This module is for [UIAutomation](https://docs.microsoft.com/en-us/windows/win32/winauto/ui-automation-specification) on Windows(Windows XP with SP3, Windows Vista, Windows 7 and Windows 8/8.1/10).
It supports UIAutomation for the applications which implmented UIAutomation Provider, such as MFC, Windows Form, WPF, Modern UI(Metro UI), Qt(Partly), Firefox(**version<=56 or >=60**), Chrome and Electron based apps(require **--force-renderer-accessibility** command line parameter).

I developed it in my spare time and for my personal use.

uiautomation is shared under the Apache Licence 2.0.
This means that the code can be freely copied and distributed, and costs nothing to use.

uiautomation1.x supports py2, py3 and doesn't depend on any third package.

uiautomation2.0+ only supports py3 and depends on comtypes and typing(Python3.5+ built-in).
uiautomation2.0+ is not backward compatible with early versions. See [API changes](https://github.com/yinkaisheng/Python-UIAutomation-for-Windows/blob/master/API%20changes.txt).

You can install uiautomation by "pip install uiautomation". After installation, a automation.py that calls uiautomation will be in 'C:\PythonXX\Scripts\'.
You use this script to traverse UI controls.

Run 'C:\PythonXX\Scripts\automation.py -h' for help.
Run demos\automation_calculator.py to see a simple demo.

On Windows 8/8.1, to automate a Metro App, the app must be in foreground. If a Metro App was switched to background, uiautomation can't fetch its controls' information.

By the way, You should run python as **administrator**. Otherwise uiautomation may fail to enumerate controls or get controls' information on Windows 7 or higher.

[Requirements:](https://docs.microsoft.com/en-us/windows/win32/api/uiautomationclient/nn-uiautomationclient-iuiautomation)

Microsoft UIAutomation Minimum supported client:
Windows 7, Windows Vista with SP2 and Platform Update for Windows Vista, Windows XP with SP3 and Platform Update for Windows Vista [desktop apps only]

Microsoft UIAutomation Minimum supported server:
Windows Server 2008 R2, Windows Server 2008 with SP2 and Platform Update for Windows Server 2008, Windows Server 2003 with SP2 and Platform Update for Windows Server 2008 [desktop apps only]

C++ dll source code: [UIAutomationClient](https://github.com/yinkaisheng/UIAutomationClient)

---

How to use uiautomation?
run '**automation.py -h**'
![help](images/uiautomation-h.png)

Understand the arguments of automation.py, and try the following examples:
**automation.py -t 0 -n**, print current active window's controls, show fullname
**automation.py -r -d 1 -t 0**, print desktop(the root of control tree) and it's children(top level windows)

![top level windows](images/automation_toplevels.png)

automation.py prints the properties of controls and the patterns they support.
You use controls and patterns to get controls' information and operate them.

A control should support some patterns or conditionally supports some patterns according to its control type.

![patterns](images/control_pattern.png)

Refer [Control Pattern Mapping for UI Automation Clients](https://docs.microsoft.com/en-us/windows/win32/winauto/uiauto-controlpatternmapping) for the full control pattern table.

uiautomation searches controls from the control tree based on the controls' properties you supply.

Suppose the control tree is

root(Name='Desktop', Depth=0)
　　window1(Depth=1)
　　　　control1-001(Depth=2)
　　　　control1-...(Depth=2)
　　　　...
　　　　control1-100(Depth=2)
　　window2(Name='window2', Depth=1)
　　　　control2-1(Depth=2)
　　　　　　control2-1-001(Depth=3)
　　　　　　control2-1-...(Depth=3)
　　　　　　...
　　　　　　control2-1-100(Depth=3)
　　　　control2-2(Depth=2)
　　　　control2-3(Depth=2)
　　　　control2-4(Name='2-4', Depth=2)
　　　　　　editcontrol(Name='myedit1', Depth=3)
　　　　　　**editcontrol(Name='myedit2', Depth=3)**

If you want to find the EditControl whose name is 'myedit2' and type 'hi',
you can write the following code:

```python
uiautomation.EditControl(searchDepth=3, Name='myedit2').SendKeys('hi')
```

But this code run slowly because there are more than 200 controls before myedit2 in the control tree.
uiautomation has to traverse more than 200 controls before finding myedit2 if search from root in 3 search depth.
The better is:

```python
window2 = uiautomation.WindowControl(searchDepth=1, Name='window2') # search 2 times
sub = window2.Control(searchDepth=1, Name='2-4')    # search 4 times
edit = sub.EditControl(searchDepth=1, Name='myedit2')   # search 2 times
edit.SendKeys('hi')
```

This code run faster than the former.
You can also combine the four lines code into one line.

```python
uiautomation.WindowControl(searchDepth=1, Name='window2').Control(searchDepth=1, Name='2-4').EditControl(searchDepth=1, Name='myedit2').SendKeys('hi')
```

Now let's take notepad.exe for an example.
Luanch notepad.exe and run automation.py -t 3, then swith to Notepad and wait for 5 seconds

automation.py will print the controls of Notepad and save them to @AutomationLog.txt:

ControlType: PaneControl    ClassName: #32769    Name: 桌面    Depth: 0    **(Desktop window, the root control)**
　　ControlType: WindowControl    ClassName: Notepad    Depth: 1    **(Top level window)**
　　　　ControlType: EditControl    ClassName: Edit    Depth: 2
　　　　　　ControlType: ScrollBarControl    ClassName:     Depth: 3
　　　　　　　　ControlType: ButtonControl    ClassName:     Depth: 4
　　　　　　　　ControlType: ButtonControl    ClassName:     Depth: 4
　　　　　　ControlType: ThumbControl    ClassName:     Depth: 3
　　　　ControlType: TitleBarControl    ClassName:     Depth: 2
　　　　　　ControlType: MenuBarControl    ClassName:     Depth: 3
　　　　　　　　ControlType: MenuItemControl    ClassName:     Depth: 4
　　　　　　ControlType: ButtonControl    ClassName:     Name: 最小化    Depth: 3    **(Minimize Button)**
　　　　　　ControlType: ButtonControl    ClassName:     Name: 最大化    Depth: 3    **(Maximize Button)**
　　　　　　ControlType: ButtonControl    ClassName:     Name: 关闭    Depth: 3    **(Close Button)**
...

Run the following code

```python
# -*- coding: utf-8 -*-
# this script only works with Win32 notepad.exe
# if you notepad.exe is the Windows Store version in Windows 11, you need to uninstall it.
import subprocess
import uiautomation as auto

def test():
    print(auto.GetRootControl())
    subprocess.Popen('notepad.exe', shell=True)
    # you should find the top level window first, then find children from the top level window
    notepadWindow = auto.WindowControl(searchDepth=1, ClassName='Notepad')
    if not notepadWindow.Exists(3, 1):
        print('Can not find Notepad window')
        exit(0)
    print(notepadWindow)
    notepadWindow.SetTopmost(True)
    # find the first EditControl in notepadWindow
    edit = notepadWindow.EditControl()
    # usually you don't need to catch exceptions
    # but if you meet a COMError exception, put it in a try block
    try:
        # use value pattern to get or set value
        edit.GetValuePattern().SetValue('Hello')# or edit.GetPattern(auto.PatternId.ValuePattern)
    except auto.comtypes.COMError as ex:
        # maybe you don't run python as administrator
        # or the control doesn't have a implementation for the pattern method(I have no solution for this)
        pass
    edit.SendKeys('{Ctrl}{End}{Enter}World')
    print('current text:', edit.GetValuePattern().Value)
    # find the first TitleBarControl in notepadWindow,
    # then find the second ButtonControl in TitleBarControl, which is the Maximize button
    notepadWindow.TitleBarControl().ButtonControl(foundIndex=2).Click()
    # find the first button in notepadWindow whose Name is '关闭', the close button
    # the relative depth from Close button to Notepad window is 2
    notepadWindow.ButtonControl(searchDepth=2, Name='关闭').Click()
    # then notepad will popup a window askes you to save or not, press hotkey alt+n not to save
    auto.SendKeys('{Alt}n')

if __name__ == '__main__':
    test()
```

auto.GetRootControl() returns the root control(the Desktop window)
auto.WindowControl(searchDepth=1, ClassName='Notepad') creates a WindowControl, the parameters specify how to search the control
the following parameters can be used
searchFromControl = None,
searchDepth = 0xFFFFFFFF,
searchInterval = SEARCH_INTERVAL,
foundIndex = 1
Name
SubName
RegexName
ClassName
AutomationId
ControlType
Depth
Compare

See Control.\_\_init\_\_ for the comments of the parameters.
See scripts in folder **demos** for more examples.

Control.Element returns the low level COM object [IUIAutomationElement](https://docs.microsoft.com/en-us/windows/win32/api/uiautomationclient/nn-uiautomationclient-iuiautomationelement),
Almost all methods and properties of Control are implemented via IUIAutomationElement COM API and Win32 API.
when calling a control's method or property that indirectly calls Control.Element and Control.Element is None,
uiautomation starts to search the control by the properties you supply.
uiautomation will raise a LookupError exception if it can't find the control in uiautomation.TIME_OUT_SECOND(default 10 seconds).
Control.Element will has a valid value if uiautomation finds the control successfully.
You can use Control.Exists(maxSearchSeconds, searchIntervalSeconds) to check whether a control Exists, this function doesn't raise any exception.
Call Control.Refind or Control.Exists to make Control.Element invalid again and uiautomation will starts a new search.

For example:
```python
#!python3
# -*- coding:utf-8 -*-
# this script only works with Win32 notepad.exe
# if you notepad.exe is the Windows Store version in Windows 11, you need to uninstall it.
import subprocess
import uiautomation as auto
auto.uiautomation.SetGlobalSearchTimeout(15)  # set new timeout 15

def main():
    subprocess.Popen('notepad.exe', shell=True)
    window = auto.WindowControl(searchDepth=1, ClassName='Notepad')
    # or use Compare for custom search
    # window = auto.WindowControl(searchDepth=1, ClassName='Notepad', Compare=lambda control,depth:control.ProcessId==100)
    edit = window.EditControl()
    # when calling SendKeys, uiautomation starts to search window and edit in 15 seconds
    # because SendKeys indirectly calls Control.Element and Control.Element is None
    # if window and edit don't exist in 15 seconds, a LookupError exception will be raised
    try:
        edit.SendKeys('first notepad')
    except LookupError as ex:
        print("The first notepad doesn't exist in 15 seconds")
        return
    # the second call to SendKeys doesn't trigger a search, the previous call makes sure that Control.Element is valid
    edit.SendKeys('{Ctrl}a{Del}')
    window.GetWindowPattern().Close()  # close the first Notepad, window and edit become invalid even though their Elements have a value

    subprocess.Popen('notepad.exe')  # run second Notepad
    window.Refind()  # need to refind window, trigger a new search
    edit.Refind()  # need to refind edit, trigger a new search
    edit.SendKeys('second notepad')
    edit.SendKeys('{Ctrl}a{Del}')
    window.GetWindowPattern().Close()  # close the second Notepad, window and edit become invalid again

    subprocess.Popen('notepad.exe')  # run third Notepad
    if window.Exists(3, 1): # trigger a new search
        if edit.Exists(3):  # trigger a new search
            edit.SendKeys('third notepad')  # edit.Exists makes sure that edit.Element has a valid value now
            edit.SendKeys('{Ctrl}a{Del}')
        window.GetWindowPattern().Close()
    else:
        print("The third notepad doesn't exist in 3 seconds")

if __name__ == '__main__':
    main()
```
---

**If automation.py can't print the controls you see.
Maybe the controls were built by DirectUI(or CustomControl), not UI Frameworks supplied by Microsoft.
In order to support UIAutomation, an UI Framework must implement [UI Automation Provider](https://docs.microsoft.com/en-us/windows/win32/winauto/uiauto-providersoverview).**

A Microsoft UI Automation provider is a software object that exposes an element of an application's UI so that accessibility client applications can retrieve information about the element and invoke its functionality. In general, each control or other distinct element in a UI has a provider.

Microsoft includes a provider for each of the standard controls that are supplied with Microsoft Win32, Windows Forms, and Windows Presentation Foundation (WPF). This means that the standard controls are automatically exposed to UI Automation clients; you do not need to implement any accessibility interfaces for the standard controls.

If your application includes any custom controls, you need to implement UI Automation providers for those controls to make them accessible to accessibility client applications. You also need to implement providers for any third party controls that do not include a provider. You implement a provider by implementing UI Automation provider interfaces and control pattern interfaces.

---

Another UI tool [Inspect.exe](https://docs.microsoft.com/en-us/windows/win32/winauto/inspect-objects) supplied by Microsoft can also be used to traverse the UI elements. It has an UI interface while my script shows UI elements in terminal.
But I found that my script is more convenient sometimes.

![Inspect](https://docs.microsoft.com/en-us/windows/desktop/WinAuto/images/inspect.png)

---

Some screenshots:

Batch rename pdf bookmark
![bookmark](images/rename_pdf_bookmark.gif)

Microsoft Word
![Word](images/word.png)

Wireshark 3.0 (Qt 5.12)
![Wireshark](images/wireshark3.0.gif)

GitHub Desktop (Electron App)
![GitHubDesktop](images/github_desktop.png)

Pretty print dir
![PrettyPrint](images/pretty_print_dir.png)

Donate：
![微信](images/yks-wx.png) ![支付宝](images/yks-zfb.png)


text:
# 1-2 Example: Modeling Procedure for Images

### 1. Data Preparation

The cifar2 dataset is a sub-set of cifar10, which only contains two classes: airplane and automobile.

Each class contains 5000 images for training and 1000 images for testing.

The goal for this task is to train a model to classify images as airplane or automobile.

The files of cifar2 are organized as below:

![](../data/cifar2.jpg)

```python
import tensorflow as tf
print("Num GPUs Available: ", len(tf.config.experimental.list_physical_devices('GPU')))

```

<!-- #region -->
There are two ways of image preparation in TensorFlow.

The first one is constructing the image data generator using ImageDataGenerator in tf.keras.

The second one is constructing data pipeline using tf.data.Dataset and several methods in tf.image

The former is simpler and is demonstrated in [this article](https://zhuanlan.zhihu.com/p/67466552) (in Chinese).

```python
train_datagen = ImageDataGenerator(
            rescale = 1./255,
            rotation_range=40,
            width_shift_range=0.2,
            height_shift_range=0.2,
            shear_range=0.2,
            zoom_range=0.2,
            horizontal_flip=True,
            fill_mode='nearest')

test_datagen = ImageDataGenerator(rescale=1./255)
```
1. rotation_range is a value in degrees (0-180), a range within which to randomly rotate pictures
2. width_shift and height_shift are ranges (as a fraction of total width or height) within which to randomly translate pictures vertically or horizontally rescale is a value by which we will multiply the data before any other processing. Our original images consist in RGB coefficients in the 0-255, but such values would be too high for our models to process (given a typical learning rate), so we target values between 0 and 1 instead by scaling with a 1/255. factor.
3. shear_range is for randomly applying shearing transformations
4. zoom_range is for randomly zooming inside pictures
5. horizontal_flip is for randomly flipping half of the images horizontally --relevant when there are no assumptions of horizontal assymetry (e.g. real-world pictures).
6. fill_mode is the strategy used for filling in newly created pixels, which can appear after a rotation or a width/height shift.

We only apply these for the training data.

For more detailed explanation, please see this [website](https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html).

The latter is the original method of TensorFlow, which is more flexible with possible better performance with proper usage.

Below is the introduction to the second method.

<!-- #endregion -->

```python
import tensorflow as tf
gpus = tf.config.experimental.list_physical_devices('GPU')
if gpus:
  try:
    # Currently, memory growth needs to be the same across GPUs
    for gpu in gpus:
      tf.config.experimental.set_memory_growth(gpu, True)
    logical_gpus = tf.config.experimental.list_logical_devices('GPU')
    print(len(gpus), "Physical GPUs,", len(logical_gpus), "Logical GPUs")
  except RuntimeError as e:
    # Memory growth must be set before GPUs have been initialized
    print(e)

### this is required, otherwise, there is an error: Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR
```

```python

from tensorflow.keras import datasets,layers,models, Sequential, backend
# import tensorflow as tf
# from tensorflow.compat.v2 import InteractiveSession
# config = tf.ConfigProto()
# config.gpu_options.allow_growth = True
# session = InteractiveSession(config=config)

# backend.clear_session()
import os
os.environ['CUDA_VISIBLE_DEVICES']='0'

# gpus= tf.config.list_physical_devices('GPU') # tf2.1版本该函数不再是experimental
# print(gpus) # 前面限定了只使用GPU1(索引是从0开始的,本机有2张RTX2080显卡)
# tf.config.experimental.set_memory_growth(gpus[0], True)

BATCH_SIZE = 100

def load_image(img_path,size = (32,32)):
    label = tf.constant(1,tf.int8) if tf.strings.regex_full_match(img_path,".*automobile.*") \
            else tf.constant(0,tf.int8)
    img = tf.io.read_file(img_path)
    img = tf.image.decode_jpeg(img) #In jpeg format
    img = tf.image.resize(img,size)/255.0
    return(img,label)

### use the file name to define the label
### tf.io.read_file returns a tensor of string.
### tf.image.resize() has multiple methods.
```

```python
#Parallel pre-processing using num_parallel_calls and caching data with prefetch function to improve the performance
ds_train = tf.data.Dataset.list_files("../data/cifar2/train/*/*.jpg") \
           .map(load_image, num_parallel_calls=tf.data.experimental.AUTOTUNE) \
           .shuffle(buffer_size = 1000).batch(BATCH_SIZE) \
           .prefetch(tf.data.experimental.AUTOTUNE)

ds_test = tf.data.Dataset.list_files("../data/cifar2/test/*/*.jpg") \
           .map(load_image, num_parallel_calls=tf.data.experimental.AUTOTUNE) \
           .batch(BATCH_SIZE) \
           .prefetch(tf.data.experimental.AUTOTUNE)

### Use the basic functions to handle the raw data.
### tf.data.Dataset.list_files() A dataset of all files matching one or more glob patterns.
### tf.data.experimental.AUTOTUNE will prompt the tf.data runtime to tune the value dynamically at runtime.
### Shuffle(): The dataset fills a buffer with buffer_size elements,\
### then randomly samples elements from this buffer, replacing the selected elements with new elements.\
### For perfect shuffling, a buffer size greater than or equal to the full size of the dataset is required.
### Creates a Dataset that prefetches elements from this dataset.
```

```python
%matplotlib inline
%config InlineBackend.figure_format = 'svg'

#Checking part of the samples
from matplotlib import pyplot as plt

plt.figure(figsize=(8,8))
for i,(img,label) in enumerate(ds_train.unbatch().take(9)):
    ax=plt.subplot(3,3,i+1)
    ax.imshow(img.numpy())
    ax.set_title("label = %d"%label)
    ax.set_xticks([])
    ax.set_yticks([])
plt.show()

### axesan axes.SubplotBase subclass of Axes (or a subclass of Axes)
### The axes of the subplot. The returned axes base class depends on the projection used.
### It is Axes if rectilinear projection are used and projections.polar.PolarAxes if polar projection are used.
### The returned axes is then a subplot subclass of the base class.
```

![](../data/1-2-图片预览.jpg)

```python
for x,y in ds_train.take(1):
    print(x.shape,y.shape)

### take: Creates a Dataset with at most count elements from this dataset.
```

```
(100, 32, 32, 3) (100,)
```

```python

```

### 2. Model Definition

Usually there are three ways of modeling using APIs of Keras: sequential modeling using `Sequential()` function, arbitrary modeling using functional API, and customized modeling by inheriting base class `Model`.

Here we use API functions for modeling.

```python
tf.keras.backend.clear_session() #Clearing the session

inputs = layers.Input(shape=(32,32,3))
x = layers.Conv2D(32,kernel_size=(3,3))(inputs)
x = layers.MaxPool2D()(x)
x = layers.Conv2D(64,kernel_size=(5,5))(x)
x = layers.MaxPool2D()(x)
x = layers.Dropout(rate=0.1)(x)
x = layers.Flatten()(x)
x = layers.Dense(32,activation='relu')(x)
outputs = layers.Dense(1,activation = 'sigmoid')(x)

model = models.Model(inputs = inputs,outputs = outputs)

model.summary()
```

```
Model: "model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
input_1 (InputLayer)         [(None, 32, 32, 3)]       0
_________________________________________________________________
conv2d (Conv2D)              (None, 30, 30, 32)        896
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 15, 15, 32)        0
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 11, 11, 64)        51264
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 5, 5, 64)          0
_________________________________________________________________
dropout (Dropout)            (None, 5, 5, 64)          0
_________________________________________________________________
flatten (Flatten)            (None, 1600)              0
_________________________________________________________________
dense (Dense)                (None, 32)                51232
_________________________________________________________________
dense_1 (Dense)              (None, 1)                 33
=================================================================
Total params: 103,425
Trainable params: 103,425
Non-trainable params: 0
_________________________________________________________________
```

```python
import tensorflow as tf

if tf.test.gpu_device_name():
    print('Default GPU Device:\
    {}'.format(tf.test.gpu_device_name()))
else:
   print("Please install GPU version of TF")

print(tf.__version__)
```

### 3. Model Training

There are three usual ways for model training: use internal function fit, use internal function train_on_batch, and customized training loop. Here we introduce the simplist way: using internal function fit.

```python
import datetime
# import os

# stamp = datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
# logdir = os.path.join('data', 'autograph', stamp)

# We recommend using pathlib under Python3
from pathlib import Path
stamp = datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
logdir = str(Path('../data/autograph/' + stamp))

print(logdir)
tensorboard_callback = tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)

model.compile(
        optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
        loss=tf.keras.losses.binary_crossentropy,
        metrics=["accuracy"]
    )

```

```python

history = model.fit(ds_train, epochs= 10, validation_data=ds_test,
                    callbacks = [tensorboard_callback], workers = 4, use_multiprocessing=False)

```

```
Train for 100 steps, validate for 20 steps
Epoch 1/10
100/100 [==============================] - 16s 156ms/step - loss: 0.4830 - accuracy: 0.7697 - val_loss: 0.3396 - val_accuracy: 0.8475
Epoch 2/10
100/100 [==============================] - 14s 142ms/step - loss: 0.3437 - accuracy: 0.8469 - val_loss: 0.2997 - val_accuracy: 0.8680
Epoch 3/10
100/100 [==============================] - 13s 131ms/step - loss: 0.2871 - accuracy: 0.8777 - val_loss: 0.2390 - val_accuracy: 0.9015
Epoch 4/10
100/100 [==============================] - 12s 117ms/step - loss: 0.2410 - accuracy: 0.9040 - val_loss: 0.2005 - val_accuracy: 0.9195
Epoch 5/10
100/100 [==============================] - 13s 130ms/step - loss: 0.1992 - accuracy: 0.9213 - val_loss: 0.1949 - val_accuracy: 0.9180
Epoch 6/10
100/100 [==============================] - 14s 136ms/step - loss: 0.1737 - accuracy: 0.9323 - val_loss: 0.1723 - val_accuracy: 0.9275
Epoch 7/10
100/100 [==============================] - 14s 139ms/step - loss: 0.1531 - accuracy: 0.9412 - val_loss: 0.1670 - val_accuracy: 0.9310
Epoch 8/10
100/100 [==============================] - 13s 134ms/step - loss: 0.1299 - accuracy: 0.9525 - val_loss: 0.1553 - val_accuracy: 0.9340
Epoch 9/10
100/100 [==============================] - 14s 137ms/step - loss: 0.1158 - accuracy: 0.9556 - val_loss: 0.1581 - val_accuracy: 0.9340
Epoch 10/10
100/100 [==============================] - 14s 142ms/step - loss: 0.1006 - accuracy: 0.9617 - val_loss: 0.1614 - val_accuracy: 0.9345
```

```python

```

### 4. Model Evaluation

```python
%load_ext tensorboard
#%tensorboard --logdir ../data/keras_model
```

```python
from tensorboard import notebook
notebook.list()
```

<!-- #raw -->

<!-- #endraw -->

```python
#Checking model in tensorboard
notebook.start("--logdir ../data/autograph")
```

```python

```

![](../data/1-2-tensorboard.jpg)

```python
import pandas as pd
dfhistory = pd.DataFrame(history.history)
dfhistory.index = range(1,len(dfhistory) + 1)
dfhistory.index.name = 'epoch'

dfhistory
```

![](../data/1-2-dfhistory.jpg)

```python
%matplotlib inline
%config InlineBackend.figure_format = 'svg'

import matplotlib.pyplot as plt

def plot_metric(history, metric):
    train_metrics = history.history[metric]
    val_metrics = history.history['val_'+metric]
    epochs = range(1, len(train_metrics) + 1)
    plt.plot(epochs, train_metrics, 'bo--')
    plt.plot(epochs, val_metrics, 'ro-')
    plt.title('Training and validation '+ metric)
    plt.xlabel("Epochs")
    plt.ylabel(metric)
    plt.legend(["train_"+metric, 'val_'+metric])
    plt.show()
```

```python
plot_metric(history,"loss")
```

![](../data/1-2-Loss曲线.jpg)

```python
plot_metric(history,"accuracy")
```

![](../data/1-2-Accuracy曲线.jpg)

```python
#Evaluating data using model.evaluate function
val_loss,val_accuracy = model.evaluate(ds_test,workers=4)
print(val_loss,val_accuracy)

```

```
0.16139143370091916 0.9345
```

### 5. Model Application

We can use model.predict(ds_test) for prediction.

We can also use model.predict_on_batch(x_test) to predict a batch of data.

```python
model.predict(ds_test)
```

```
array([[9.9996173e-01],
       [9.5104784e-01],
       [2.8648047e-04],
       ...,
       [1.1484033e-03],
       [3.5589080e-02],
       [9.8537153e-01]], dtype=float32)
```

```python
for x,y in ds_test.take(1):
    print(model.predict_on_batch(x[0:20]))
```

```
tf.Tensor(
[[3.8065155e-05]
 [8.8236779e-01]
 [9.1433197e-01]
 [9.9921846e-01]
 [6.4052093e-01]
 [4.9970779e-03]
 [2.6735585e-04]
 [9.9842811e-01]
 [7.9198682e-01]
 [7.4823302e-01]
 [8.7208226e-03]
 [9.3951421e-03]
 [9.9790359e-01]
 [9.9998581e-01]
 [2.1642199e-05]
 [1.7915063e-02]
 [2.5839690e-02]
 [9.7538447e-01]
 [9.7393811e-01]
 [9.7333014e-01]], shape=(20, 1), dtype=float32)
```

```python

```

### 6. Model Saving

We recommend model saving with the original way of TensorFlow.

```python
# Saving the weights, this way only save the tensors of the weights
model.save_weights('../data/tf_model_weights.ckpt',save_format = "tf")
```

```python
# Saving model structure and parameters to a file, so the model allows cross-platform deployment

model.save('../data/tf_model_savedmodel', save_format="tf")
print('export saved model.')

model_loaded = tf.keras.models.load_model('../data/tf_model_savedmodel')
model_loaded.evaluate(ds_test)
```

```
[0.16139124035835267, 0.9345]
```

```python

```

Please leave comments in the WeChat official account "Python与算法之美" (Elegance of Python and Algorithms) if you want to communicate with the author about the content. The author will try best to reply given the limited time available.

You are also welcomed to join the group chat with the other readers through replying **加群 (join group)** in the WeChat official account.

![image.png](../data/Python与算法之美logo.jpg)


text:
Title: IModule
Description: Modules must implement this interface.
---
Modules must implement this interface. At startup, all found and specified assemblies are scanned for classes that implement `IModule` and those that do are automatically registered as available modules.

# Members
---

## Methods

  - `IEnumerable<IDocument> Execute(IReadOnlyList<IDocument> inputs, IExecutionContext context)`

    The implementation of this method should contain the logic for the module. This method should never be called directly, instead call `IExecutionContext.Execute(...)` if you need to execute a module from within another module.


text:
## TS Smoosh

Combine type decls with related source files.

# Use

Given a JavaScript file (or list of files) like so:

```
$ node ts-smoosh/bin ./src/some-file.js
```

Will produce `.jsx` files using nearby `.d.ts` files.

# Running Tests

```
$ node test
```

Test cases live in various directories in `./tests`. Tests run by comparing the output against golden master `.tsx` files. To update the masters, run `node ts-smoosh/bin ./test/0X-dir/somefile.js`.

# Notes

Here are articles and docs I that were helpful when writing this script.

- [Using the Compiler API](https://github.com/microsoft/TypeScript/wiki/Using-the-Compiler-API)
- [Blog post about using TS's parser directly](https://medium.com/allenhwkim/how-to-parse-typescript-from-source-643387971f4e)
- [TypeScript compiler APIs revisited](https://blog.scottlogic.com/2017/05/02/typescript-compiler-api-revisited.html)
- [TypeScript API Playground on Glitch](https://typescript-api-playground.glitch.me/#example=Transformation%203)

gha_language: TypeScript


text:
## 5.0.0 - 2016-10-30

* Add support for `options.shell`
* Improve parsing of shebangs by using [`shebang-command`](https://github.com/kevva/shebang-command) module
* Refactor some code to make it more clear
* Update README caveats