text:
# ESP32 Oberon Compiler User's Guide

Three applications are supplied:

- The ESP32 Oberon compiler (named `OberonESP32`)
- The module initialization table generator (named `OIOrderESP32`)
- The smb file viewer (named `ORToolESP32`)

The sequence of use of the compiler is as follow:

1. The compiler must be called once for each module that is part of the targeted application. It will generate an assembly language file (extension `.S`) and will call the assembler to generate the object file (extension `.o`).

2. After that, the modules init table generator must be called to create the table that will be used by the application initialization code to call each module init code in sequence. This table is an assembly language file (extension `.S`). The generator will then call the assembler to generate the corresponding object file (extension `.o`).

3. The linker can then be called to generate the application. In the context of the ESP-IDF framework, a template is available to show how to integrate an Oberon program to the framework. This is using a combination of CMake and Make related configuration files and source code that will be used to produce and upload an application. (TBC)

## Compiler parameters

The ESP32 Oberon Compiler is a single executable that can be launched as follows:

```sh
$ OberonESP32 [-c][-l][-o folder][-p path][-s][-v][-x][-z] filename
```

Where the parameters are:

- **-c** : Add code to do the following additional checks at runtime:

  + Array index boundary limits
  + NIL pointers dereference
  + Division by zero
  + Pointer types validation
  + Array copy overflow
  + Illegal procedure call

- **-l** : (lowercase “L”) Source code will be output as comments in assembly language generated code.
- **-o folder** : Output folder for generated files. If not present, the current folder is used.
- **-p path** : Search path for imported modules, as a list of folder names separated with character “:”. An imported module is first seached in the current folder, then the folder of the module being compiled, then in sequence in each folder of the path.
- **-s** : Override Symbol file. The symbol file will be generated even if it already exists.
- **-v** : Output compiler version number.
- **-x** : Optimize register usage.
- **-z** : Output optimization information in assembler output.
- **filename** : Oberon source filename to be compiled.

The compiler will produce:
- an ESP32 assembly language source code (extension `.S`);
- the object code (extension `.o`);
- a symbol file (extension `.smb`) if it does not exists; or the **-s** option is present.

Note that for a CDECL module, only the symbol file (extension `.smb`) will be generated.

## Compiler modification to support ESP-IDF Framework integration

Please look in document [Changes to Oberon-07 definition](./Changes%20to%20%20Oberon-07%20definition.md) for additional functionalities in support of ESP-IDF integration.

## Runtime support files

[This section will be substantially modified to supply ESP-IDF integration guidance.]

An ESP32 Oberon program requires runtime support. The compiler is supplied with many files that you can find in the lib folder:

- An application startup function that is called by the ESP32 bootstrap once the application has been loaded in memory. It is located in `lib/init/init.S`. This file contains many include instructions to add the other pieces of code present in the same folder. These files supply support for interupts management and floating-point divide and square-root functions.

- A Linker Script file that supply memory mapping information to the linker. It is located in `lib/ld/esp32.ld`.

- Several Oberon modules that are located in the `lib/modules` folder. They are described in the following sections

## Modules Initialization Sequence

The application startup code is an assembly language function named `oberon_startup()` that must be called by an ESP-IDF C piece of code (usually the C main app function). You can find it in file `lib/init/init.S`. Once started, it calls the initialization code of each Oberon module in sequence. The order of these calls is important as some module may rely on other modules to be initialized prior to their own.

The supplied `OIOrderESP32` application is responsible of generating the module initialization table that will allow for the proper sequencing of initialisation calls made by `oberon_startup()`.

(TBC)

## Some type sizes and memory alignment

| Type     | Size    | Alignment | Low Value            | High Value          |
|----------|:-------:|:---------:|:--------------------:|:-------------------:|
| BOOLEAN  | 8 bits  | Byte      |        FALSE         |      TRUE           |
| CHAR     | 8 bits  | Byte      |          0X          |      0FFX           |
| BYTE     | 8 bits  | Byte      |           0          |      255            |
| SHORTINT | 16 bits | 2 Bytes   |           0          |     65535           |
| INTEGER  | 32 bits | 4 bytes   |     -2147483648      |<PHONE_NUMBER>        |
| LONGINT  | 64 bits | 4 Bytes   | -9223372036854775808 |<PHONE_NUMBER>854775807 |
| SET      | 32 BITS | 4 Bytes   |          {}          |   {0 .. 31}         |

## Standard Module

The ESP32 Oberon Compiler is supplied with a variaty of modules for use by applications. This section presents each of these modules and the specific aspects of their interface in relation with the targetted architecture.

### Module Kernel

Support for:

- Heap allocation through the `New` procedure.
- Garbage collection
- General timing functions
- Error trap support

This module is currently being developped. The application program that requires this module must initialize it through the `Kernel.Init` procedure.

### Module Out

This module implements part of the [Oakwood Guidelines](http://www.edm2.com/index.php/The_Oakwood_Guidelines_for_Oberon-2_Compiler_Developers) `Out` Module definition with some added/removed procedures. The procedure is using the ESP32 UART0 serial port to transmit data sent by the users's application. By default, the connexion baud rate is set to 115K in the Open procedure called by the module initialization. This module, for now, uses active loops to synchronise with the *first in first out* (Fifo) output buffer. The Oakwood's `LongReal` procedure is not available. The following procedures have been added:

- `PROCEDURE SetBaudRate*(baudRate: INTEGER);` Change baud rate according to the value passed as a parameter. The following usual baud rate are supplied as constants:

  + `Baud9600`
  + `Baud19200`
  + `Baud38400`
  + `Baud76800`
  + `Baud115K`

- `PROCEDURE Flush;` Active loop waiting for the Fifo output buffer to be empty.

- `PROCEDURE RealFix*(x: REAL; n, k: INTEGER)` Write the real x in n field positions in fixed point notation with f fraction digits.

```Pascal
MODULE Out;
  CONST  (* INTEGER constants *)
    Baud115K*,
    Baud76800*,
    Baud38400*,
    Baud19200*,
    Baud9600*;

  PROCEDURE Open*;
  PROCEDURE Flush*;
  PROCEDURE Char*(ch: CHAR);
  PROCEDURE String*(s: ARRAY OF CHAR);
  PROCEDURE Int*(x, n: INTEGER);
  PROCEDURE Hex*(x, n: INTEGER);
  PROCEDURE Real*(x: REAL; n: INTEGER);
  PROCEDURE RealFix*(x: REAL; n,k: INTEGER);
  PROCEDURE Ln*;
  PROCEDURE SetBaudRate*(baudRate: INTEGER);
END Out.
```

### Module In

This module implements part of the [Oakwood Guidelines](http://www.edm2.com/index.php/The_Oakwood_Guidelines_for_Oberon-2_Compiler_Developers) `In` Module definition with some added/removed procedures. The procedure is using the ESP32 UART0 serial port to receive data for the users's application. By default, the connexion baud rate is set to 115K in the Open procedure called by the module initialization. This module, for now, uses active loops to synchronise with the *first in first out* (Fifo) input buffer. The Oakwood's `LongReal` and `LongInt` procedures are not available. The following procedures have been added:

- `PROCEDURE SetBaudRate*(baudRate: INTEGER);` Changes baud rate according to the value passed as a parameter. The following usual baud rate are supplied as constants in the Out module:

  + `Out.Baud9600`
  + `Out.Baud19200`
  + `Out.Baud38400`
  + `Out.Baud76800`
  + `Out.Baud115K`

- `PROCEDURE SetTimeout*(seconds: INTEGER);` Specifies the timeout duration while receiving characters. Timeout is set to 30 seconds by default. It is reset after each character received. If timeout occurs, the variable `In.Done` will be `FALSE`.

- `PROCEDURE SetEcho*(echo: BOOLEAN);` If `TRUE` (default value), received characters will be echoed to the stream through the `Out.Char()` procedure.

- `PROCEDURE Flush;` Empty the input Fifo buffer.

When Echo is enabled, the backspace key on the keyboard can be used to erase characters at the end of line. A sequence of backspace-space-backspace will then be sent to the terminal through the `Out.Char` procedure.

This module requires the Out and Kernel modules. The Kernel module must have been properly initialized through the use of the `Kernel.Init` procedure through user's application.

```Pascal
MODULE In;
  VAR
    Done*: BOOLEAN;                           (* status of last operation *)
  PROCEDURE SetBaudRate*(baudRate: INTEGER);
  PROCEDURE SetTimeout*(seconds: INTEGER);
  PROCEDURE SetEcho*(echo: BOOLEAN);
  PROCEDURE Open*;
  PROCEDURE Flush*;                           (* Clear Fifo *)
  PROCEDURE Char*(VAR ch: CHAR);
  PROCEDURE String*(VAR str: ARRAY OF CHAR);
  PROCEDURE Int*(VAR i: INTEGER);
  PROCEDURE Real*(VAR x: REAL);               (* Not Yet Available *)
  PROCEDURE Name*(VAR name: ARRAY OF CHAR);   (* Not Yet Available *)
  PROCEDURE Line*(VAR line: ARRAY OF CHAR);
END In.
```

### Module Strings

The module Strings provides a set of operations on strings, i.e., on string constants and character arrays, both of which contain the character 0X as a terminator.  All positions in strings start at 0. This module implements the [Oakwood Guidelines](http://www.edm2.com/index.php/The_Oakwood_Guidelines_for_Oberon-2_Compiler_Developers) Strings Module definition with some added procedures.

```Pascal
MODULE Strings;
  PROCEDURE Length* (s: ARRAY OF CHAR): INTEGER;
  PROCEDURE Write* (src: ARRAY OF CHAR; VAR dest: ARRAY OF CHAR;
                    at: INTEGER): INTEGER;
  PROCEDURE Append* (extra: ARRAY OF CHAR; VAR dest: ARRAY OF CHAR);
  PROCEDURE WriteChar* (c: CHAR; VAR dest: ARRAY OF CHAR; at: INTEGER): INTEGER;
  PROCEDURE AppendChar* (c: CHAR; VAR dest: ARRAY OF CHAR);
  PROCEDURE WriteInt* (x, n: INTEGER; VAR dest: ARRAY OF CHAR;
                      at: INTEGER): INTEGER;
  PROCEDURE AppendInt* (x, n: INTEGER; VAR dest: ARRAY OF CHAR);
  PROCEDURE Copy* (src: ARRAY OF CHAR; VAR dest: ARRAY OF CHAR);
  PROCEDURE Insert* (source: ARRAY OF CHAR; pos: INTEGER;
                    VAR dest: ARRAY OF CHAR);
  PROCEDURE Delete* (VAR s: ARRAY OF CHAR; pos, n: INTEGER);
  PROCEDURE Replace* (source: ARRAY OF CHAR; pos: INTEGER;
                      VAR dest: ARRAY OF CHAR);
  PROCEDURE Extract* (source: ARRAY OF CHAR; pos, n: INTEGER;
                      VAR dest: ARRAY OF CHAR);
  PROCEDURE Pos* (pattern, s: ARRAY OF CHAR; pos: INTEGER): INTEGER;
  PROCEDURE Cap* (VAR s: ARRAY OF CHAR);
  PROCEDURE Match* (string, pattern: ARRAY OF CHAR): BOOLEAN;
END Strings.
```

### Module Math

This module was retrieved from the Project Oberon source code. I've added some REAL constants that would be useful for mathematical functions usage. The sqrt procedure is using an assembly language optimized function, as for REAL division generated by the compiler. Those two functions can be found in file `lib/src/math_support.S`.

```Pascal
MODULE Math;
  CONST
    E*         = 2.7182818285;
    PI*        = 3.1415926536;
    SQRT2*     = 1.4142135623;
    SQRT3*     = 1.7320508076;
    LOG10E*    = 0.4342944819;
    LN3OVER2*  = 0.5493061443;
    PIOVER2*   = 1.5707963268;
    PIOVER3*   = 1.0471975512;
    PIOVER4*   = 0.7853981634;
    PIOVER6*   = 0.5235987756;
    ONEOVERPI* = 0.3183098862;
    TWOOVERPI* = 0.6366197724;

  PROCEDURE sqrt*(x: REAL): REAL;
  PROCEDURE exp*(x: REAL): REAL;
  PROCEDURE ln*(x: REAL): REAL;
  PROCEDURE sin*(x: REAL): REAL;
  PROCEDURE cos*(x: REAL): REAL;
  PROCEDURE tan*(x: REAL): REAL;    (* -pi/2 <= x <= pi/2 *)
  PROCEDURE arcsin*(x: REAL): REAL; (*  -1.0 <= x <=  1.0 *)
  PROCEDURE arccos*(x: REAL): REAL; (*  -1.0 <= x <=  1.0 *)
  PROCEDURE arctan2*(x, y: REAL): REAL;
  PROCEDURE arctan*(x: REAL): REAL;
END Math.
```

## Compiler debugging tools

### $D+ (Debug On) and $D- (Debug OFF)

The pragmas `$D+` and `$D-` can be put anywhere in the application source code inside comments. This will trigger the compiler to output code generator module `ORG` procedure calls during the compilation phase. This trace is to help resolve issues with the compiler.

### Source code as comments in assembler output

The compiler option **-l** (lowercase “L”) will trigger the compiler to emit source code lines as comments in the generated assembly language ouput. This is helpfull to compare generated assembly language instructions with Oberon program statements.

path: /doc/Users Guide.md

filename: Users Guide.md


text:
---
title: Publish a Story
---

Publishing a story besides using the `publish` property via creation, can be done by using a GET request for each story you want to publish.

;examplearea

Example Request

<RequestExample url="https://mapi.storyblok.com/v1/spaces/606/stories/2141/publish" httpMethod="GETOAUTH"></RequestExample>

repository: vitaliytv/storyblok-docs

branch_name: refs/heads/master

extension: md

license_type: permissive

repo_name: vitaliytv/storyblok-docs


text:
---
layout: article
title: "Project GTAC: Understanding drivers for HR expenditure"
categories: articles
modified: 2018-01-26T17:01:00-01:00
tags: [2017]
comments: false
ads: false
---

### Project Information

* **Partner**: Government Technical Advisory Centre (GTAC)
* **Students**: Basisipho Jack (UP), Mncedisi Comfort Ketwa (AIMS), Xichavo Maringa (UP)
* **Project Lead**: Quentin Williams, Dhiren Seetharam

* **Project Mentors**: Antonius Mamphiswana, Thompho Rambuda
* **Year**: 2017/2018

### Project Description

Over the past eleven years, remuneration expenditure has grown at a rate higher than inflation and faster than personnel. This is of great concern to the Government Technical Advisory Centre (GTAC), an agency of the National Treasury,  as it readies itself for the upcoming salary negotiations between government and the unions.

The target audience of this project include GTAC as well as union members and leaders negotiating on behalf of employees. Baring that in mind, it is vital that the  Project Treasury team provide greater understanding of historical data and forecasting of future spending  to both parties. This allows for informed decision-making and a win-win situation (Nash-Equilibrium) in the salary negotiation game.

Project Treasury is assigned to provide data analytics, modelling, forecasting and visualisation of personnel numbers and compensation expenditure based on detailed personnel records of employees in the Education Department. With the aid of appointments, promotions, terminations, headcount data and expenditure data; the Project Treasury team aims to explain relationships in the transctional personnel records extracted. The main objectives of the project are to find the key drivers of remuneration expenditure and to forecast as to how this expenditure will behave in future.

The team’s focus is to develop and design a tool (dashboard)  which illustrates the performance of key drivers and how they drive remunerations expenditure. Additionally, this tool will allow the GTAC to use the models used by the team to analyse and visualize big administrative datasets in future.

license: MIT

gha_language: CSS


text:
# Project Hello7000 Changelog

## Version 2.0.x
### Version 2.0.1 (May 28, 2019)
* Added support for the capitals of the five United States territories.
* Updated the Disclaimer window.
* Updated Tips.

### Version 2.0 (August 13, 2018)
* Added a "Tips" window that can be launched via a button on the menu bar.
* Added support for addition and subtraction (with some restrictions).
* Added support for state capitals. (Note: Currently, only the capitals of the 50 U.S. states are supported.)
* Other updates to the database.
* Other minor changes.

## Version 1.0.x
### Version 1.0 (May 12, 2018)
* This is Version 1.0 of Project Hello7000!


text:
# Sub Applications - Mounts

If you need to have two independent FastAPI applications, with their own independent OpenAPI and their own docs UIs, you can have a main app and "mount" one (or more) sub-application(s).

## Mounting a **FastAPI** application

"Mounting" means adding a completely "independent" application in a specific path, that then takes care of handling everything under that path, with the _path operations_ declared in that sub-application.

### Top-level application

First, create the main, top-level, **FastAPI** application, and its *path operations*:

```Python hl_lines="3  6-8"
{!../../../docs_src/sub_applications/tutorial001.py!}
```

### Sub-application

Then, create your sub-application, and its *path operations*.

This sub-application is just another standard FastAPI application, but this is the one that will be "mounted":

```Python hl_lines="11  14-16"
{!../../../docs_src/sub_applications/tutorial001.py!}
```

### Mount the sub-application

In your top-level application, `app`, mount the sub-application, `subapi`.

In this case, it will be mounted at the path `/subapi`:

```Python hl_lines="11  19"
{!../../../docs_src/sub_applications/tutorial001.py!}
```

### Check the automatic API docs

Now, run `uvicorn` with the main app, if your file is `main.py`, it would be:

```console
$ uvicorn main:app --reload

<span style="color: green;">INFO</span>:     Uvicorn running on http://<IP_ADDRESS>:8000 (Press CTRL+C to quit)
```

And open the docs at <a href="http://<IP_ADDRESS>:8000/docs" class="external-link" target="_blank">http://<IP_ADDRESS>:8000/docs.

You will see the automatic API docs for the main app, including only its own _path operations_:

And then, open the docs for the sub-application, at <a href="http://<IP_ADDRESS>:8000/subapi/docs" class="external-link" target="_blank">http://<IP_ADDRESS>:8000/subapi/docs.

You will see the automatic API docs for the sub-application, including only its own _path operations_, all under the correct sub-path prefix `/subapi`:

If you try interacting with any of the two user interfaces, they will work correctly, because the browser will be able to talk to each specific app or sub-app.

### Technical Details: `root_path`

When you mount a sub-application as described above, FastAPI will take care of communicating the mount path for the sub-application using a mechanism from the ASGI specification called a `root_path`.

That way, the sub-application will know to use that path prefix for the docs UI.

And the sub-application could also have its own mounted sub-applications and everything would work correctly, because FastAPI handles all these `root_path`s automatically.

You will learn more about the `root_path` and how to use it explicitly in the section about [Behind a Proxy](./behind-a-proxy.md){.internal-link target=_blank}.

repository: samuelcolvin/fastapi

gha_language: Python

gha_license_id: MIT

repo_name: samuelcolvin/fastapi


text:
Details for running SMILESMerge's accessory scripts can be found in the file TUTORIAL.md located within the directory /SMILESMerge/tutorial/.

Accessory scripts are redistributed with permissions from AutoGrow4 version 4.0.2

repository: gkxiao/SMILESMerge

branch_name: refs/heads/main

repo_name: gkxiao/SMILESMerge


text:
# stutils
A Utils Library.

[![Build Status](https://travis-ci.org/wantee/stutils.svg)](https://travis-ci.org/wantee/stutils)
[![License](http://img.shields.io/:license-mit-blue.svg)](https://github.com/wantee/stutils/blob/master/LICENSE)

## Features
* Logging
* Configure
* Data structure, eg. Stack, Queue, Heap
* Socket wrapper
* Dict and Alphabet
* Semaphore

## Build
```shell
$ git clone https://github.com/wantee/stutils.git
$ cd stutils/src
$ make -j 4
$ make test
```

## Usage
1. Add include directory to CFLAGS.

  ```
CFLAGS += -I$(PATH_TO_STUTILS)/include
```

2. Include header(s) in your source file.

  ```
#include <stutils/st_xxx.h>
```

## Contributing

1. Fork it ( https://github.com/wantee/stutils.git )
2. Create your feature branch (`git checkout -b my-new-feature`)
3. Commit your changes (`git commit -am 'Add some feature'`)
4. Push to the branch (`git push origin my-new-feature`)
5. Create a new Pull Request

## License

**stutils** is open source software licensed under the MIT License. See the `LICENSE` file for more information.

repository: songmeixu/stutils

repo_name: songmeixu/stutils


text:
Simulation of Cycloid Drawing Machine in Processing.JS

Physical machine designed by [Joe Freedman](http://leafpdx.bigcartel.com/product/cycloid-drawing-machine) and available for [purchase at his online store](http://leafpdx.bigcartel.com/product/cycloid-drawing-machine).

Processing simulation by [Jim Bumgardner](http://krazydad.com/about.php)

![Cycloid Drawing Machine](http://i.imgur.com/q4CFLI6.png "Cycloid Drawing Machine")

![Cycloid Drawing Machine](https://media.giphy.com/media/xT9DPiyGTZ0LMrnYys/giphy.gif "Cycloid Drawing Machine")

Try the live demo here:

[wheelof.com/sketch/](http://wheelof.com/sketch/)

Read about how this was made here:

[krazydad.com/blog](http://krazydad.com/blog/2015/07/12/cycloid-drawing-machine-simulation/)

CDMS is licensed under the [MIT License](https://github.com/jbum/CDMS/blob/master/LICENSE.txt)

repository: jbum/CDMS

repo_name: jbum/CDMS


text:
# Scripts
Several small BASH scripts that I use on my personal machines.

## autorip.sh

A script for ripping CD collections automatically.

## backup.sh

A script for backing up a computer to local and remote hard drives.

## micspam.sh

A script for piping audio to applications from the command line.

## todo.sh

An extremely simple todo list managing script.

## update-discord-canary.sh

A simple script to make updating Discord Canary easier.

repository: TastyDucks/Scripts

repo_name: TastyDucks/Scripts


text:
# Open-Doors-Uni-2018

The presentation I made at Open Doors of Sofia University, Physics Department. The original title was "Scalar tensor theories - the new true theory of gravity?", but more suitable would be "Brief history of our understanding of Gravity"

## Getting Started

You should not have any trouble with running the project. In "Papers" there are the papers used, in "images" are the images for the presentation, ins "src" is the source python code for the simulations, "Notes" are my notes on the presentation.

The simulations consist in solving ODE equation for free falling bodies (zero initial velocity) for different models

- Aristotelian mechanics - presumes the velocity is proportional to the mass
- Galilean mechanics - the rate of change of velocity is proportional to constant
- Newtonian mechanics - the rate of change of velocity is proportional to forces actin on the body, in this particular case it presumes there is air resistance proportional to velocity

The simulations clearly show that light bodies fall the same hight approximately the same time, but for heavier ones the time may change - thus the primitive idea of Aristotle.

- Newtonian orbit - calculates the orbit in Newtonian gravity, and shows the existence of anomaly - oscillating perihelion of orbits.
- GR orbit - calculates the orbit predicted in General relativity, it clearly shows that despite Newtonian orbit should be closed, when general relativity effects take place, it becomes open - and thus the explanation of Mercury perihelion anomaly


text:
# Galactic Labs - Customer Group Payment Filters

This simple module adds a new section to the Customer Groups admin page, allowing you to define payment options that ***should not*** be allowed for the chosen customer group.

***Note: These settings will affect both frontend and backend orders.***

## Installation

Require the module

```bash
composer require galacticlabs/customer-group-payment-filters
```

Enable the module

```bash
php bin/magento module:enable GalacticLabs_CustomerGroupPaymentFilters
```

Run setup to install module and set up table(s)

```bash
php bin/magento setup:upgrade
```

## Usage

Once installed, you can choose which payment methods should be restricted by going to `Stores > Other Settings > Customer groups` and selecting the group you would like to place restrictions on.

The edit screen has a new section called Disallowed Payment Options. This contains a multiselect list of the payment options in the Magento system. Choose the ones you'd like to disable for the customer group, then save for changes to take effect. These options are also available when creating a new customer group.

***Note: These options will not affect anything if the chosen payment options are themselves not enabled.***

![Edit Page Screenshot](docs/edit-page-screenshot.png)

## Bugs/Feature Requests & Contribution

Please do open a pull request on GitHub should you want to contribute, or create an issue.

## Todo

I plan on cleaning this up a bit and also adding some tests to prove functional correctness. I'd also be interested in discussions on how I could have achieve the same results using alternative/better techniques. I don't like the fact i'm accessing the request object inside the repository save method. Is it possible to perform this action in the admin controller instead? Unfortunately I hit an issue when saving a new customer group in that I couldn't access the ID of the new group in the same request. I'm sure with more time I would of figured it but this was created very quickly.

* Caching should be added

## License
[MIT](https://opensource.org/licenses/MIT) - Do as you wish 👍

gha_language: PHP


text:
# UnityGB

UnityGB allows you to run a Game Boy ROM inside your Unity project.
It has been written in C# and is only using Mono allowing you to export the project to any platform supported by Unity.

UnityGB became the main subject of several articles like [CNET, Gameboy emulator on the Oculus Rift: Gameception](http://www.cnet.com/news/gameboy-emulator-on-the-oculus-rift-gameception/).

UnityGB is still in alpha version, many bugs and glitches can occur. Also, the code is far from being perfect and needs a lot of optimization. Any and all help welcome! Please feel free to contribute to this project.

Please also take some time to take a look at our other applications on our website [Takohi.com](http://www.takohi.com) and our other [Unity assets](https://www.assetstore.unity3d.com/en/?gclid=CjwKEAjwuoOpBRCSy6yQm66J1g8SJABrXW48qCgi3rfBrvrAQu55uQeS0U3YH51O-Ybf6N1ZDwJVQRoCqBrw_wcB#!/search/page=1/sortby=popularity/query=publisher:4241). :+1:

## Compatibilities

### What is supported

* Game Boy games with cartridge <= MBC5
* Input
* Sound
* Save (save files are compatible with Visual Boy Advance and other emulators)

### Planned features

* Game Boy Color compatibility
* Link Cable
* Oculus Rift

### Known issues

* Sound synchronization
* Pokemon Yellow: Pikachu voices are not played
* Some glitches with sprites order

## Demonstrations

**Hilarious Gameboy Emulator with Leap Motion in Unity**, by [Pierce Wolcott](https://piercewolcott.com/)
[![Gameboy Emulator with Leap Motion in Unity](https://i.vimeocdn.com/video/514669012_590x332.jpg)](https://vimeo.com/124805471 "Leap Boy")

**Oculus Rift Gameboy Emulator in Unity**, by [Shane O'Brien](http://www.youtube.com/watch?v=wby8pMrYYaM)
[![Oculus Rift Gameboy Emulator in Unity](http://img.youtube.com/vi/wby8pMrYYaM/0.jpg)](http://www.youtube.com/watch?v=wby8pMrYYaM "Oculus Rift Gameboy Emulator in Unity")

**UnityGB4BB10, GameBoy Emulator for BlackBerry 10**
[![UnityGB4BB10](http://www.filearchivehaven.com/wp-content/uploads/2014/08/UnityGB4BB-Games-1024x576.png)](http://www.filearchivehaven.com/2014/08/17/proud-to-announce-another-gameboy-emulator-for-blackberry-10-unitygb4bb10/ "UnityGB4BB10")

**Official Web Demo**, by [Takohi](http://www.takohi.com)
[![Official Demo](https://bitbucket.org/repo/8MjKzK/images/2954418396-unitygb_demo_screenshot_2.png)](http://www.takohi.com/data/unity/unitygb/ "Official Demo")

## Usage
**UnityGB** can work in several ways. What you have to do is to make your own implementation about how to manage inputs (controls) and outputs (video and sound).
In the Unity package and the sources, you will already find one classic scene outputing the video in a classic texture, the audio trough the [OnAudioFilterRead](http://docs.unity3d.com/Documentation/ScriptReference/MonoBehaviour.OnAudioFilterRead.html) method, and joypad input from the keyboard.

### Joypad

An example of how to implement the controls can be found in *DefaultEmulatorManager.cs*.

When you want to emulate an input from the player into the game, you just have to call the *SetInput* method from the emulator reference:
```
void SetInput(Button button, bool pressed);
```
Button is an enum with these values:

```
public enum Button {Up, Down, Left, Right, A, B, Start, Select};
```
For example, if you want to press the button start when the player is hitting the space button:

```
if(Input.GetKeyDown(KeyCode.Space))
    Emulator.SetInput(EmulatorBase.Button.Start, true);
else if(Input.GetKeyUp(KeyCode.Space))
    Emulator.SetInput(EmulatorBase.Button.Start, false);
```

### Video

An example of how to implement the video can be found in *DefaultVideoOutput.cs*.

In order to output the video from the emulator or in other words, to display the Game Boy screen, you will have to make your own class implementing the *IVideoOutput* interface.

This interface has only two methods:

```
void SetSize(int w, int h);
void SetPixels(uint[] colors);

```

The *SetSize* method will be called after the rom is loaded. It will tell you what is the size of the screen in pixels (160x144, it should never change) so you can initialize your way to display.

The method *SetPixels* will be called every time the frame has been updated. As parameter, an array of colors that corresponds to the color of each pixel. The size of the array will be *width x height*.

Here is a method to convert a uint color into a Unity color:

```
private Color UIntToColor(uint color)
{
    byte a = (byte)(color >> 24);
    byte r = (byte)(color >> 16);
    byte g = (byte)(color >> 8);
    byte b = (byte)(color >> 0);
    return new Color(r / 255f, g / 255f, b / 255f, a / 255f);
}
```

### Audio

An example of how to implement the audio can be found in *DefaultAudioOutput.cs*.

With Unity, the only way to dynamically produce audio is by using the method [OnAudioFilterRead(float[] data, int channels)](https://docs.unity3d.com/Documentation/ScriptReference/MonoBehaviour.OnAudioFilterRead.html) from a *MonoBehaviour* object.

In the package, we already provide a way to produce the audio trough the *DefaultAudioOutput* class. If you want to use it, just attach that script to a game object and add to it an *AudioSource* component.

If you still want to make your own implementation for outputing audio, you will have to make a class that implements the interface *IAudioOutput*.

This interface has three methods:
```
int GetOutputSampleRate();
int GetSamplesAvailable();
void Play(byte[] data, int offset, int count);
```

* *GetOutputSampleRare()* has to return the sample rate (44100Hz, ...).
* *GetSamplesAvailable()* returns the number of samples to return during the next step.
* *Play(byte[] data)* gives you the samples to play.

Samples are interleaved (%0 = Left, %1 = Right). This means for each sample: data[i] = left sample, data[i + 1] = right sample.

### Save

An example of how to manage save files can be found in *DefaultSaveMemory.cs*.

In order to manage save files, you will have to make your own class implementing the *ISaveMemory* interface.

This interface has only two methods:

```
void Save(string name, byte[] data);
byte[] Load(string name);
```

The *Save* method is called when the user stops the game, and the *Load* method is called when the game is loaded. The *name* parameter is the name of the game that is currently played.

Globally, when the *Save* method is called, you simply have to store somewhere the byte array given as parameter, then load and return this same byte array when the *Load* method is called for the same game.

### Make the emulator run

An example of how to use the emulator can be found in *DefaultEmulatorManager.cs*.

The code below is a simple way showing how to initialize and make run UnityGB with a *MonoBehaviour* script.

First, we create an instance of the emulator and load the ROM in the *Start* method.

```
void Start()
{
    // Load emulator
    IVideoOutput drawable = ...; // Reference to your IVideoOutput implementation.
    IAudioOutput audio = ...; // Reference to your IAudioOutput implementation.
    ISaveMemory saveMemory = ...; // Reference to your ISaveMemory implementation.
    Emulator = new Emulator(drawable, audio, saveMemory); // Instantiate emulator.

    byte[] rom = ...; // Load ROM binary.
    Emulator.LoadRom(www.bytes); // Load ROM into emulator.
}
```
Then we make the emulator running during the *Update* method of the *MonoBehaviour*. We also use this method to manage inputs from the user and send them to the emulator.

```
void Update()
{
    // Input
    if(Input.GetKeyDown(KeyCode.Space))
        Emulator.SetInput(EmulatorBase.Button.Start, true);
    else if(Input.GetKeyUp(KeyCode.Space))
        Emulator.SetInput(EmulatorBase.Button.Start, false);
    ...

    // Run
    Emulator.RunNextStep();
}
```

Don't forget to attach the previous MonoBehaviour to a game object in order to make it run.

## Reference Material

* [Official UnityGB thread on Unity forum](http://forum.unity3d.com/threads/245974-unityGB-Emulator-Game-Boy-for-Unity-RELEASED)
* http://meatfighter.com/gameboy/
* http://www.millstone.demon.co.uk/download/javaboy/


text:
---
layout: post
title: "Compile-time checking plain key path strings"
date: 2014-05-17 17:50:34 -0700
categories: code
tags: cocoa
subtitle: Safe and sane key paths.
---

[Key–value coding][kvc] — `valueForKeyPath:` and `setValue:forKeyPath:` — is very useful for converting data from one format to another, such as copying from a property list-like structure (e.g. deserialized JSON) into model objects proper.
This avoids a lot of boilerplate typically found in, for example, Java-based systems[^java-boilerplate].

On the Mac, with [Cocoa Bindings][bindings] you can throw together a simple UI in a flash, setting up key paths in Interface Builder.
On iOS you have to provide some of the glue yourself, but you typically let [key–value observing][kvo] do the hardest work.

## The problem

These "key–value" techniques have one main thing in common:
They use strings to reference code.
This makes it fragile. For example:

{% highlight objc %}
@implementation ContrivedExample
- (NSString *)title
{ return @"foo"; }

- (NSUInteger)contrivedLength
{ return [[self title] length]; }

- (NSUInteger)contrivedLengthKVC
{ return [[self valueForKey:@"title"] length]; }
@end
{% endhighlight %}

If we rename the method `title`, the compiler will immediately show that `contrivedLength` won't work without changes.
But `contrivedLengthKVC` will have no such warning, because `@"title"` remains a perfectly valid string.
We would only see an error at run-time, if and when that code was triggered.
Of course in real code, string references like this appear much further apart than in the above snippet, making stale references and even typos a real problem.

## The common solution

In short:
**The compiler does not validate key paths, because they are strings.**

One approach toward validation is avoid strings[^stringly-typed]. Of course, a string must emerge at some point, but pre-processor macros allow using the same expression (`title`) as both code and data, at compile-time.
Many have taken this route, resulting in a large variety[^safe-kvc-links] with different trade-offs. A couple that are representative:

1. With [libextobjc][libextobjc-kvc] you have to specify the target of the key path twice — `[self valueForKey:@keypath(self.title)]` above — though this is often redundant, and obscures that the resulting string will be simply "title".
2. [DMSafeKVC][dmsafekvc] is more concise at use — `[self valueForKey:K(title)]` — but require annotating declarations, and only checks that *some* class has an annotated “title” accessor.

[^safe-kvc-links]: [Uli Kusterer's approach using `@selector`](http://orangejuiceliberationfront.com/safe-key-value-coding/);
[Nicolas Bouilleaud's approach using `@selector`](https://gist.github.com/n-b/2394297);
[Kyle Van Essen's approach using declared targets](https://gist.github.com/kyleve/8213806);
[Andrew Pouliot's approach using declared targets](https://gist.github.com/darknoon/4482025);
[Martin Kiss's approach using declared targets](https://github.com/iMartinKiss/Valid-KeyPath);
and many more.

Compiler error messages are typically obscured due to macro expansion. And if you're writing library code[^library-code], often the best approach is to use the lowest common denominator — static strings.

[^library-code]: "Library code" refers to code shared between projects which might use different safe KVC techniques, or none at all.

A few months back, my colleagues at Fitbit discussed the ups and downs of various macro implementations and didn't arrive at consensus.
Additionally, retrofitting our codebase would be a mostly manual task and likely generate many conflicts (our repository has around 10 active committers).

I began to consider a different approach.

## The code less traveled

**The compiler does not validate key paths, because they are strings.**
In our phrasing of the problem, a second solution becomes apparent: make the compiler validate the strings!

In the past, touching the compiler has been considered off-the-table for various reasons. With clang, this is now more practical[^practicality], but the previous stigma still lingers.

This code is currently proof-of-concept quality.
(I'm only a C++ novice, and barely familiar with the clang codebase.)
It currently:

- Checks some hard-coded method calls that are passed string literal arguments (including macros that expand to string literals).
- Validates that there are corresponding getter methods, recursing down the key path.
- Checks for a method named `key` or `isKey`, or a collection property that might be backed with [KVC collection accessor methods][kvc-collection].
- Does no checking if the type is `id`, `NSDictionary`, a class annotated with `objc_kvc_container`, or a few more.
- Knows that scalar numbers will be returned as `NSNumber`.
- Uses the diagnostic output of the secondary clang, but the binary output of Xcode's bundled clang.

The compiler plug-in doesn't do everything that some of the macro approaches do, but has the giant advantage of requiring zero code changes and no dependencies.
It also doesn't exclude the use of macros, either.
Only one small project setting change is required:

If you'd like to try this with your own project, look at the [Clang-KeyPathValidator](https://github.com/jmah/Clang-KeyPathValidator) README file.
Then check out the [GitHub Issues](https://github.com/jmah/Clang-KeyPathValidator/issues) page and see if you can help out with something.
Or just help get the word out!

[^java-boilerplate]: This opinion is several years old, before Java reflection was commonly available or efficient.
[^stringly-typed]: The term "stringly-typed" is just great.
[^practicality]: It's unfortunately still not completely painless, because the particular versions shipped by Apple with Xcode are not made publicly available, and have plug-in support disabled.

[kvc]: https://developer.apple.com/library/mac/documentation/cocoa/Conceptual/KeyValueCoding/Articles/KeyValueCoding.html
[bindings]: https://developer.apple.com/library/mac/documentation/Cocoa/Conceptual/CocoaBindings/CocoaBindings.html
[kvo]: https://developer.apple.com/library/mac/documentation/Cocoa/Conceptual/KeyValueObserving/KeyValueObserving.html
[libextobjc-kvc]: https://github.com/jspahrsummers/libextobjc/blob/master/extobjc/EXTKeyPathCoding.h
[dmsafekvc]: https://github.com/delicious-monster/DMSafeKVC
[kvc-collection]: https://developer.apple.com/library/ios/documentation/cocoa/conceptual/KeyValueCoding/Articles/AccessorConventions.html#//apple_ref/doc/uid/20002174-178830-BAJEDEFB

branch_name: HEAD

extension: markdown


text:
---
layout: post
title: " Bomb Jack Dissected"
---

**TL;DR**: what I learned about the Bomb Jack arcade machine while writing
an emulator for it.

![Bomb Jack Logo](/images/bombjack_logo.png)

I wrote a little Bomb Jack arcade emulator last week, mainly to learn
how those early 8-bit arcade machines differed in design from 8-bit home
computers.

Playing arcade games like Bomb Jack as a kid at the summer fair in my home
town was one of those 'life-changing moments' that I only recognized much
later as such. On a typical summer day, after I had spent all my coin budget
on arcade games (usually didn't take very long) I walked home with the head
full of colors and sound effects, trying to figure out how those games
worked. And then for the rest of the year spend all my after-school time
trying to build rather poor impressions of those arcade games on my home
computer, not unlike a cargo-cult worshipper on a post-WWII pacific island trying to
build a US military radio station from wooden sticks.

At first I've been playing with the idea of writing a *Pengo* emulator, since
this burned itself even more into my teenager-brain than Bomb Jack (this is my
[cargo-cult version of
Pengo](https://floooh.github.io/tiny8bit/kc85.html?type=kc85_3&file=kc85/pengo.kcc) btw).
But the Pengo arcade hardware would require new chip emulators for sound and
video, while Bomb Jack only needs parts I already had working (Z80 as CPU
and the AY-3-8910 for sound), so I went for Bomb Jack first.

Bomb Jack was also a good opportunity to finally add NMI (non-maskable interrupt)
support to my Z80 emulator. None of the Z80 machines I emulated before
used NMIs, and it didn't make much sense to implement a feature when I have
no way to test whether it even works.

If you don't know what Bomb Jack is, it looks like this (not sure if I got the
screen aspect ratio right):

![Bomb Jack Screenshot](/images/bombjack_1.png)

...you can check out the WebAssembly version of the emulator here:

[https://floooh.github.io/tiny8bit/bombjack.html](https://floooh.github.io/tiny8bit/bombjack.html)

After the boot sequence has finished and the hiscore screen appears,
press **1** to insert coins, and than **Enter** (or any other key
except arrows and space) to start a game.

In the game, use **arrows keys** for direction, and **space** to jump. While
in the air tap **space** to reduce falling speed.

The source code is here:

[https://github.com/floooh/chips-test/blob/master/examples/sokol/bombjack.c](https://github.com/floooh/chips-test/blob/master/examples/sokol/bombjack.c)

This uses the [chips headers](https://github.com/floooh/chips) to provide
the Z80 and AY-3-8910 emulations, and the [sokol headers](https://github.com/floooh/sokol) as cross-platform wrapper (application
entry, rendering, input and audio).

## Step 1: Research

"Research" is a grand word for typing "Bombjack arcade hardware specs" into
Google ;)

Compared to popular 80's home computers (or even obscure Eastern European
computers, which often still have active communities), there's very little
information for Bomb Jack on the web.

The two useful pieces of information I found are the original [hardware
schematics](https://github.com/floooh/emu-info/blob/master/misc/bombjack-schematics.pdf),
and of course the [MAME emulator source
code](https://github.com/mamedev/mame/blob/master/src/mame/drivers/bombjack.cpp).

There's also a project which implements [Bomb Jack in an FGPA](http://papilio.cc/index.php?n=Playground.BombJack)
where the VHDL source yielded some details where the schematics miss information.

The MAME sources may be a bit hard to grok, because the arcade emulations
are usually just a bunch of macros describing how the different hardware
parts interact, but there's little actual *source code* to read.

But the macro-hardware-descriptions, and especially the comments are still
very helpful to understand how the hardware works, and in those places where
it all becomes a bit too obscure (for instance the [video decoding
part](https://github.com/mamedev/mame/blob/master/src/mame/video/bombjack.cpp))
a bit of trial and error, and running the finger through the hardware
schematics is enough to clear up the confusion.

## Hardware Overview

The most interesting feature of the Bomb Jack hardware is that it's actually
**two computers** duck-taped together, there's a **mainboard** with a Z80 CPU
and the video decoding hardware, and a separate **soundboard** with its own
Z80 CPU and 3 (yes, three!!) AY-3-8910 sound chips.

The video decoding hardware on the mainboard isn't implemented in an
integrated circuit, instead it's lots and lots of little general-purpose
chips (taking up 6 of the 10 schematics pages). For the emulator I took a
shortcut here: instead of emulating the video decoding hardware piece by
piece, I only emulated its behaviour by creating the right visual output
from the inputs, without caring too much how the hardware inbetween actually
works.

This shortcut-approach is totally fine for a dedicated arcade machine which
only ever needs to run one program. If the game looks and feels right, the
emulation is "good enough".

This 'shortcut approach' is also an important difference to most home
computer emulators: some games demand a more precise emulation than others,
for instance machines like the C64 or Amstrad CPC need a very precise emulation down
to the clock cycle for the video systems for some games and graphics demos to
work.

It also means that my existing CPU and sound-chip emulators are a
bit overkill for Bomb Jack, for instance the Z80 CPU is running with
machine-cycle granularity when a simpler and faster instruction granularity
emulation would definitely be good enough.

## The Main Board

Usually the first thing I try to find out when writing a new emulator is the
memory map (where are the ROM and RAM areas, the video memory, and
special IO addresses or ports).

The Bomb Jack mainboard only has one 'interesting' chip, the Z80 CPU running
at 4 MHz. The entire remaining space on the mainboard is dedicated to the
video decoding hardware (besides a couple of RAM and ROM chips).

The 16-bit address space looks like this:

- **0000..7FFF**: 32 KByte ROM
- **8000..8FFF**: 4 KByte general purpose RAM
- **9000..93FF**: 1 KByte video RAM
- **9400..97FF**: 1 KByte color RAM
- **9820..987F**: 96 bytes sprite RAM
- **9C00..9CFF**: 256 bytes color palette RAM
- **9E00, B000..B005, B800**: IO ports
- **C000..DFFF**: 8 KByte ROM

The IO port area looks like this, some of the ports are write-only,
some are read-only, and some have different functions when read or written:

- **9E00**:     write: the current background-image number, read: ---
- **B000**:     read: player 1 joystick state, write: NMI enable/disable mask
- **B001**:     read: player 2 joystick state, write: ---
- **B002**:     read: coins and start buttons, write: ---
- **B003**:     read: CPU watchdog, write: ???
- **B004**:     read: dip-switches 1, write: flip screen
- **B005**:     read: dip-switches 2, write: ---
- **B800**:     write: command to soundboard, read: ---

There's a few noteworthy things:

- There's a LOT of ROM (40 KBytes), and not a lot of RAM (about 7 KBytes, with
only 4 KByte 'general purpose RAM')
- There's only 2 KBytes for 'display RAM', divided into 2 chunks of 1 KByte,
this looks very small for driving a colorful 256x256 display which appears to have
per-pixel colors
- It's a memory-mapped-IO system!

Memory-mapped-IO is a bit unusual for a Z80 machine, since one of the
defining features of the Z80 is the separate 16-bit address space for device-IO,
so that no precious memory address space needs to be wasted. Memory mapped IO
is typically a feature found on computers with a 6502 CPU.

A quick look into the schematics confirms this: There's no IORQ pin anywhere
to be found on the mainboard CPU, only the MREQ pin is connected (which
is used to initiate a memory read or write):

![Bomb Jack IORQ](/images/bombjack_iorq.png)

This means I don't need to care about IO requests in the emulator's mainboard CPU
tick function at all, only about memory requests.

While snooping around the schematics, there's another interesting mainboard CPU detail:

Only the NMI pin is connected, while the INT pin is always kept high/inactive (meaning
no 'regular' maskable interrupts will happen, only non-maskable interrupts):

![Bomb Jack IORQ](/images/bombjack_nmi.png)

This is also quite unusual for a Z80 machine. All Z80-based home computers
I've seen so far had it the other way around, they only used maskable
interrupts, and never non-maskable interrupts. The maskable interrupt
handling on the Z80 is very flexible and a big improvement over the rather
primitive interrupt system of its 'illegitimate father', the Intel 8080, or
its rival, the MOS 6502. But this improved flexibility is also more complex
to implement in hardware (unless other Z80-family chips are used as interrupt
source, those have the complex interrupt 'daisy-chain' protocol already baked
in).

Ok, enough with the hardware details for now, onward to the emulator!

## The Boot Sequence

The next step after figuring out the memory configuration is to hook up an
emulated CPU to the emulated memory map, write some adhoc visualization of
the video memory content, and start ticking the CPU.

And surprisingly, this crude approach is often enough to run through the
boot sequence and get *something* on screen. For the Bomb Jack emulator I
simply took the content of the 1 KByte video memory at the range 0x9000 to
0x93FF as a 32x32 byte matrix, and when the byte is 0, render a black 8x8
pixel block, otherwise a white pixel block.

Then simply start the emulated CPU and hope for the best. And behold! Something
recognizable appears:

![Bomb Jack Boot 1](/images/bombjack_boot_1.png)
![Bomb Jack Boot 2](/images/bombjack_boot_2.png)

The left image looks a lot like the hardware testing screen during boot, and
the right image like the hiscore screen that appears when the boot
sequence is over:

![Bomb Jack Boot 3](/images/bombjack_boot_3.png)
![Bomb Jack Boot 4](/images/bombjack_boot_4.png)

...but 90 degrees rotated (which figures because arcade cabinets often had
the screen in vertical 'portrait mode' orientation).

Ok very promising start!

The next step would be figuring out how to turn
those white blocks into colored pixels... (huuuge jump now, the details
are described below in the section on video decoding).

At first everything went quite smoothly, the test screen during boot had
pixels and colors (later I noticed that the color decoding was all wrong, but
anyway):

![Bomb Jack Boot 5](/images/bombjack_boot_6.png)

But where the hiscore screen should appear I ended up with a black screen.
Hacking the background color to 'not black' revealed that the pixels are rendered,
but the color palette was set to all black. Hmm...

![Bomb Jack Boot 5](/images/bombjack_boot_5.png)

After staring at this screen for a few minutes, I remembered that some colors
on the hiscore screen were animated, and when there's animation there needs
to be some timer, and the obvious time source in this hardware config would be the
display's VSYNC signal, and the VSYNC is hooked up to the CPU's NMI pin (or
rather the VBLANK, which is the short duration between the VSYNC signal and the
cathode ray tube beam travelling back to the top-left corner).

And I didn't have implemented any of this yet...

On the next evening, after I had added a first version of the NMI handling
to the Z80 emulation and hooked this up to an adhoc vsync/vblank counter
in the mainboard's CPU tick function, a lot of things suddenly started to
happen!

First, the hiscore screen had colors, and some of them were animated:

![Bomb Jack NMI 1](/images/bombjack_nmi_1.png)

And after a few seconds, more exciting things happened! The hiscore screen
disappeared, and a weird rendition of the first map showed up. This was
clearly the arcade cabinet's attract mode, I could see the first level, and
some color-animated bombs which disappeared as an imaginary Bomb Jack jumped
around the map collecting those bombs:

![Bomb Jack NMI 2](/images/bombjack_nmi_2.png)

Colors were still totally wrong, but nevertheless: PROGRESS!

This was the right time to take care of the rest of the video decoding:

## The Video Hardware

At first glance the Bomb Jack video hardware looks very powerful for
an 8-bit machine from 1984: even though it only has a 256x256 pixel
resolution, it can display 128 (out of 4096) colors at the same time,
and render up to 24 hardware sprites (16x16 or 32x32 pixels big), all
with per-pixel colors.

8-bit home computers of the time had about the same display resolution,
but with many restrictions when it comes to colors, those restrictions can
be seen nicely when comparing the Bomb Jack versions for the ZX Spectrum and
Amstrad CPC with the Arcade version:

The [ZX Spectrum version](https://floooh.github.io/tiny8bit/zx.html?file=zx/bombjack_zx.z80&joystick=kempston&type=zx48k)
has a pretty good pixel resolution 256x192), but very
few colors, and it suffers from the Spectrum's typical 'color clash' effect
(although the developers did a pretty good job to not make this too
obvious):

![Bomb Jack ZX](/images/bombjack_zx.png)

The [Amstrad CPC version](https://floooh.github.io/tiny8bit/cpc.html?file=cpc/bomb_jack.sna&joystick=true)
is more colorful, but in order to get more colors
they had to switch to a low-resolution display mode (160x200), with the result
that Jack and the monster are a lowres pixel mess:

![Bomb Jack CPC](/images/bombjack_cpc.png)

Compare that with the Arcade version which has about the same pixel
resolution as the ZX Spectrum, but with many more colors *and*
a per-pixel color resolution:

![Bomb Jack Arcade](/images/bombjack_arcade.png)

Now the interesting part is that the arcade version doesn't have better
graphics because it runs on more powerful hardware (it has more ROM to store
more image data, but the 'computing power' is roughly the same), instead the
hardware designers could focus on building a specialized machine for one
specific type of game, they didn't have to create a flexible and general
purpose home computer.

Here's how the display hardware works (at least my high-level interpretation
of it):

## The 3 Display Layers

The final Bomb Jack video output is composed from 3 layers: background layer,
foreground layer and sprite layer.

This layer system has 2 main advantages:

- it implements a fairly clever hardware image compression to generate
a colorful "high-resolution" image from very little data
- it drastically reduces the amount of CPU work necessary to update
dynamic elements on the screen (even at 4 MHz, an 8-bit CPU by far doesn't
have enough performance to move that much stuff around on a 256x256
display at 60 Hz)

The video hardware is fairly different from what I've seen on 8-bit home
computers, but since MAME has implemented generic helper classes for this
sort of hardware I assume it was fairly common on arcade machines.

### The Background Layer

The background layer can render 1 of 5 predefined background images hardwired
into ROM. A background image is selected by writing a value
between 1 and 5 to the address 0x9E00 (value 0 seems to be special and renders
an all-black background).

The hardware actually seems to be capable of rendering 7 different
images, but the game only uses 5. I was secretly hoping to find previously
undiscovered image data in the ROMs. But alas, there aren't any (and I probably
wasn't the first one to look either)

This is what the background layer for the first map looks like without the other two layers:

![Bomb Jack Background](/images/bombjack_bg.png)

The background layer is built from **16x16** pixel tiles.

The advantage of building the background images from tiles is that identical
tiles can be reused, so that less image data needs to be stored in ROMs. Note
how the blue sky, parts of the pyramid and the sand below the pyramid use
identical tiles:

![Bomb Jack Background Tiles](/images/bombjack_bg_tiles.png)

The background layer hardware implements another trick to save some memory,
tiles can be flipped horizontally. I almost didn't implement this because
I assumed that the software doesn't use this hardware feature, until I
noticed a subtile bug in the 3rd map background:

![Bomb Jack Background 3](/images/bombjack_bg_3.png)

The same trick is used on the 5th map, it's a bit harder to notice there
though if you don't know what to look for:

![Bomb Jack Background 5](/images/bombjack_bg_5.png)

### The Foreground Layer:

On top of the background layer is a 'foreground layer', which renders any
non-movable parts of the screen that must still be updated by the CPU
(mainly text, platforms and bombs). The layout is read from RAM (the 1 KB
video RAM and 1 KB color RAM chunks).

This is what the isolated foreground layer looks like for the first map:

![Bomb Jack Foreground](/images/bombjack_fg.png)

The foreground layer is also tiled (just like the background layer), but uses
smaller 8x8 tiles:

![Bomb Jack Background Tiles](/images/bombjack_fg_tiles.png)

The main advantage of keeping background and foreground in separate layers is
that the CPU doesn't need to care about storing and restoring background
pixels when foreground elements are created or removed.

### The Sprite Layer

Finally on top of the foreground layer, the hardware sprites are rendered.
Everything that's moving around on screen is done with sprites.
The Bomb Jack hardware can render up to 24 sprites, where each sprite
can be either 16x16 or 32x32 pixels, and sprites can be positioned
with pixel resolution:

![Bomb Jack Sprites](/images/bombjack_sprites.png)

## The 8x8 Tile Decoder

At the heart of the video decoding hardware is a color palette with
128 entries and an 8x8-pixel tile decoder. The job of the tile decoder
is to generate a 7-bit color palette index for each of the 64 pixels
in a tile.

Those 8x8 tiles are the building blocks for everything on screen, the 16x16
background tiles, 8x8 foreground tiles, and the 16x16 or 32x32 hardware
sprites.

Here's a functional diagram of this 8x8 tile decoder for the foreground
layer rendering (as far as I understood it):

![Bomb Jack Tile Decoding](/images/bombjack_8x8.jpg)

A top-to-bottom explanation of the diagram:

- The decoding process starts at the top by reading a 'tile code' byte from
the video RAM (organized as a 32x32 matrix of tile codes), and a separate
byte from the color RAM (also a 32x32 matrix). Fetching the tile- and
color-codes from video RAM only happens for the foreground layer, but I added
this because it makes the 'overall picture' a bit clearer. The 8x8 tile
decoder itself only needs a tile- and color-code as input.
- The tile code is used as index to lookup 3 separate pixel bitplanes, those
pixel bitplanes are always stored in ROM (you can think of the pixel
bitplanes as font data or sprite sheets). Each of the 3 display layers has
its own tile ROMs, and those ROMs are only visible to the decoding hardware,
not to the CPU (so they don't take up valuable CPU address space).
- A single pixel bitplane consists of 8 bytes per tile, with each byte
holding 8 pixels (one bit per pixel). Since the pixel data for each tile is
built from 3 bitplanes this means an 8x8 tile requires 24 bytes of ROM data
to describe its appearance (3 bits per pixel).
- For each of the 64 pixels in a tile, a 7-bit value is constructed. The
lower 3 bits are provided by the tile bitplanes read from the tile ROMs, and
the higher 4 bits are provided by the color value byte. This basically means
that each tile can select one of 16 'slots' from the color palette, with each
slot having 8 colors. Each tile pixel can select one of the 8 colors in the
tile's palette slot.
- This 7-bit index built from bitplanes and tile-color-value is used to
lookup a 12-bit RGB color value from the color palette (4 bits per color
channel). The color palette is located in RAM and can be manipulated by the
CPU (as far as I have seen the video-, color- and palette-RAMs are
write-only, at least the CPU never does read-accesses on those areas).

This is the basic tile decoding that's used by each of the 3 display layers,
but each layer decoding differs in small details:

- The foreground layer can actually render 512 different 8x8 tiles. This
requires 9-bit tile codes, but the video RAM only provides 8 bits per tile.
The 9th bit is 'borrowed' from the 5th bit of the color value (since only 4
bits of the color value are used to build the color palette index, there
are 4 bits free for other purposes). If the 3 bits from the 8x8 tile
bitplanes are all zero, the foreground pixel is transparent, and the
background pixel 'peeks through'.
- The background layer uses 16x16 tiles, so it only needs 16x16=256 tile code
values and 256 color values to describe one background image in the
background image ROM (512 bytes per image). The twist is though that the
16x16 pixel bitplanes are layed out as four 8x8 tiles, so that the same 8x8
tile decoder hardware can be used. As mentioned above, background tiles can
be horizontally flipped, this is controlled with one of the 'spare' color
value bits: if bit 7 of the color value is set, the tile will be flipped.
- Each hardware sprite can either be 16x16 pixels or 32x32 pixels, and the
tile bitplanes are also layed out as 4 or 16 consecutive 8x8 tiles in the
sprite tile ROMs. This means a 16x16 sprite needs 96 bytes, and a 32x32 sprite
a whopping 384 bytes in the tile ROMs. As with foreground tiles, if all 3
pixel bitplane bits are zero, the sprite pixel is transparent.

To get a better idea what the tile bitplanes look like, I wrote a little
[C program](https://gist.github.com/floooh/5f44cce30912142e726bb4ff783e24b8) to convert the tile ROMs into PNG files (with the 3 bits per
pixel converted into 8 levels of grey).

This is the foreground layer tile ROM. You can see the numbers and text
font data, the platform tiles, bombs (split in half), parts of the
Bomb Jack splash screen logo, and the score multiplier numbers that
appear at the top of the screen (btw, everything is 90 degree rotated
because the whole screen is rotated too):

![Bomb Jack Foreground Tile ROM](/images/bombjack_tilerom_fg.png)

Next the background tile ROM, this doesn't look very recognizeable because
what you see is the 8x8 tile decoding of 16x16 tiles. Each 16x16 tile
is created from four neighbouring 8x8 tiles. But it's possible to recognize
the fairly regular pieces for the greek temple on map 2, the castle
on map 3 and the sky scrapers on map 4.

![ The 16x16 sprites take up the upper half,
and the 32x32 sprites the lower half.

![Bomb Jack Sprite Tile ROM](/images/bombjack_tilerom_sprites.png)

An interesting hack in the Bomb Jack splash screen is that the logo is
constructed from foreground tiles and sprites, I guess the developers were
running out of foreground tile ROM, but had some space to spare in the
sprite ROM:

![Bomb Jack Splash 1](/images/bombjack_splash_1.png)
![Bomb Jack Splash 2](/images/bombjack_splash_2.png)
![Bomb Jack Splash 3](/images/bombjack_splash_3.png)

## The Sprite Hardware

The Bomb Jack sprite hardware is very powerful compared to what home computers
provided at the time:

- up to 24 hardware sprites can be rendered, seemingly without restrictions
how many sprites can be shown on a scanline
- sprites can be 16x16 or 32x32 pixels
- each sprite can choose one of 16 '8-color-slots' in the shared color palette
- sprites have a per-pixel color resolution
- each sprite can be vertically and horizontally flipped
- each sprite can select one of 128 hardwired sprite images in ROM

The pixel- and color-decoding of the sprite system uses the same basic 8x8
tile decoding that's also used in the background- and foreground-layer.

The sprite attributes are mapped into the address range 0x9820 to 0x987F,
96 bytes, 4 bytes per sprite. As far as I've seen, this is a write-only area,
at least there are no CPU read accesses happening on this memory range.

Each sprite is described by 4 bytes:

- **Byte 0**:
    - **Bit 7**: if set, this is a 32x32 sprite, otherwise 16x16
    - **Bits 6..0**: 7 bits to define the tile code for the sprite, used
      to look up the sprite's image bitplanes in the tile ROMs
- **Byte 1**:
    - **Bit 7**: if set, the sprite is horizontally flipped
    - **Bit 6**: if set, the sprite is vertically flipped
    - **Bits 3..0**: 4 bits to provide the color value for the tile decoder
- **Byte 2**: the sprite's X position on screen
- **Byte 3**: the sprite's Y position on screen

It's unclear what the bits 4 and 5 of byte 1 do, a comment in MAME
has this to say:

```
 e        ? (set when big sprites are selected)
 f        ? (set only when the bonus (B) materializes?)
```

## The Memory Mapped IO Ports

A few notes about the IO ports of the mainboard, as mentioned above,
the IO ports look like this:

- **9E00**:     write: the current background-image number, read: ---
- **B000**:     read: player 1 joystick state, write: NMI enable/disable mask
- **B001**:     read: player 2 joystick state, write: ---
- **B002**:     read: coins and start buttons, write: ---
- **B003**:     read: CPU watchdog, write: ???
- **B004**:     read: dip-switches 1, write: flip screen
- **B005**:     read: dip-switches 2, write: ---
- **B800**:     write: command to soundboard, read: ---

The location 0x9E00 (background image selection) has already been handled above,
and the location 0xB800 (sound command) will be handled in the next section,
this leaves the locations 0xB000 to 0xB005:

Reading from location 0xB000 and 0xB001 returns the current state of the
2 joysticks. A set bit indicates that a joystick trigger is closed:

- **bit 0**: right direction
- **bit 1**: left direction
- **bit 2**: up direction
- **bit 3**: down direction
- **bit 4**: jump button pressed

The remaining 3 bits are ignored.

Reading from 0xB002 returns the state of the coin-detector and start buttons:

- **bit 0**: player 1 coin inserted
- **bit 1**: player 2 coin inserted
- **bit 2**: player 1 start button
- **bit 3**: player 2 start button

Reading from addresses 0xB004 and 0xB005 provides the state of dip-switches
which are used to customize the arcade machine's behaviour:

- **B004**:
    - **bits 0,1**: how many 'plays' are given per coin (1, 2, 3 or 5)
    - **bits 2,3**: the same for player 2
    - **bits 4,5**: how many lifes per play (3, 4, 5 or 2)
    - **bit 6**: whether the arcade machine is a 'cocktail table', or 'upright'
    - **bit 7**: whether to play sound while idle
- **B005**:
    - **bits 3,4**: difficulty 1 (bird speed)
    - **bits 5,6**: difficulty 2 (enemy number and speed)
    - **bit 7**: ratio of special coin appearance

Finally, reading from address **B003** implements a software watchdog. The
CPU needs to read from this location frequently, otherwise the arcade
machine will perform a hardware reset. If for some reason the game
crashes, the hardware will automatically reboot.

Some IO port locations are writable:

- **B000**: whether to generate an NMI during vblank or not, this only seems to be disabled during the boot sequence
- **B004**: flip the entire screen, I haven't seen this used, but I have a theory (see below)

The flip-screen functionality is a bit baffling, since when playing the game
I haven't seen this used. I have an idea what it does though, but need to do
a bit more coding to confirm this. When the arcade machine is a 'cocktail table',
the two players are sitting opposite each other. So my guess is that when the game
switches from player 1 to player 2, it will 'flip the screen'. I haven't implemented
2-player mode in the emulator yet though.

## The Sound Board

The soundboard is its own fully working computer with a Z80 CPU (running at 3
MHz), 3 sound chips (AY-38910, running at 1.5 MHz) and some RAM and ROM. The
memory map of the soundboard looks quite simple:

- **0000..2000**: 8 KBytes ROM
- **4000..4400**: 1 KByte RAM
- **6000**: sound command from the mainboard

Since there's nothing of interest in the memory map above 0x8000, the
CPU's topmost address pin isn't even connected:

![Bomb Jack Sound A15](/images/bombjack_sound_a15.png)

The special location 0x6000 is a memory mapped IO port (8-bit latch) not
backed by actual RAM. This is the same port that's mapped to address
0xB800 on the mainboard, and this is the communication link between
the mainboard and the soundboard.

The 3 sound chips are controlled through actual Z80 output instructions,
not through memory-mapped ports. The AY-3-8910 only exposes two
IO ports, the first one is used to store a register number, and the
the second port is used to write or read the content of the register
identified by the first port.

The IO map looks like this:

- **0x00**: 1st sound-chip: select register
- **0x01**: 1st sound-chip: access selected register
- **0x10**: 2nd sound-chip: select register
- **0x11**: 2nd sound-chip: access selected register
- **0x80**: 3rd sound-chip: select register
- **0x81**: 3rd sound-chip: access selected register

A few notes about the sound chip AY-3-8910:

This is a fairly standard part which also has been very popular in home
computer designs of the time (for instance the Amstrad CPC, ZX Spectrum 128,
MSX computers and more). The AY-3-8910 spawned many variations and clones,
(for instance the Yamaha YM2149, which itself was the base for a whole family
of more powerful sound chips).

The AY-3-8910 has 3 square-wave voices, a single noise generator which could
be mixed into the 3 voices, and a single envelope generator. Since there was
only one envelope generator for all three voices it wasn't actually that
useful, and most games used the CPU to modulate pitch and volume.

This means the AY-3-8910 needs more CPU intervention to produce good sound
(unlike the much more autonomous SID chip in the C64 for instance).

It's really surprising what one can do with 3 rather simple sound chips and
a dedicated CPU to drive them though. Bomb Jack's music and sound effects are
much 'richer' than what I've heard from most home computer games.

The only really interesting thing about the soundboard is how it receives its
command from the mainboard though.

## The Sound Command Latch

The 'sound latch' is a single byte storage (an 8-bit latch) shared between the mainboard and soundboard. The latch is mapped to address 0xB800 on the main
board, and to address 0x6000 on the soundboard.

The soundboard runs a very simple interrupt service routine when an NMI is
triggered by the VSYNC which reads the hardware latch, and writes it
to a normal memory location, along with setting a 'signal bit' which tells
the 'main loop' that a new sound command has been received:

```asm
        ex af,af'               ;0066
        exx                     ;0067
        ld hl,04390h            ;0068
        set 0,(hl)              ;006b
        ld a,(06000h)           ;006d
        ld (04391h),a           ;0070
        exx                     ;0073
        ex af,af'               ;0074
        retn                    ;0075
```

The way the NMI pin is activated differs slightly from the
mainboard:

On the mainboard, the NMI pin goes active for the duration of the VBLANK.

On the soundboard however, the NMI is activated when a VSYNC triggers, and
stays active not for the duration of the VBLANK, but instead until
the interrupt service routine reads from the latch at 0x6000.

When the hardware detects a read from location 0x6000 it performs two
hardwired operations:

- the content of the sound latch is cleared to 0
- the NMI pin goes inactive

This is basically a simple debouncing which prevents that the same sound
command is executed twice.

The only remaining question now was: how often does the main board write a
new command (because this dictates how I would implement the emulation of
the 2 boards).

With some printf-debugging I found out that the main board writes at most
one new sound command per 60Hz frame, this simplified the design of the
emulators 'main loop' a lot.

The problem of running 2 separate emulated computers side by side that need
to communicate with each other is that emulation of one computer is only efficient
if it can run a lot of cycles uninterrupted.

For instance the worst case would be:

- run computer 1 for one instruction
- run computer 2 for one instruction
- repeat...

My Z80 emulator is not optimized for entering and exiting the emulation for
each instruction, because in this case it needs to flush and load the CPU
state to and from memory at the start and end of each instruction. If the CPU
can crunch through many instructions uninterrupted, it can keep (most of) the
CPU state in registers and only flush state to memory at the last
instruction.

So the ideal situation is to run an emulated system for an entire host system
frame without interruption (for a 4 MHz CPU and 60Hz frames this means
about 67k ticks per frame, or somewhere between 3k and 16k Z80
instructions).

For Bomb Jack I had to make sure that the mainboard doesn't write a new command
before the sound board is able to read the last command. Before I found out that
the mainboard only writes at most one command per frame, I was considering building
a complicated command queue, which catches writes to the mainboard sound latch,
and stores a tick count and the command byte in a queue.

Then when the sound board executes its frame, it would pull a new command from
the command queue when it has reached the command's tick count.

This would work and would be 'correct', but also add quite a bit of code complexity.

In the end I decided for a much simpler solution without any queueing. Since
the mainboard only writes one command per frame, I interleave the execution
of the two computers so that each computer runs 2 'time slices' per frame:

- run mainboard for first half of frame
- run soundboard for first half of frame
- run mainboard for second half of frame
- run soundboard for second half of frame

This guarantees that the soundboard properly sees every single command written
by the mainboard while still running each emulation for thousands of ticks.

The big assumption about this is of course that the host system runs roughly at
a 60Hz framerate :)

## One Last Thing...

One last interesting trivia about the WebAssembly version of the emulator:

The compressed download size of all files when running the WebAssembly
emulator is about 113 KBytes:

- about 2.5 KBytes for the HTML, CSS and 'manually written' JS
- 26.8 KBytes for the emscripten runtime JS file
- 83.7 KBytes for the .wasm file

The WASM file contains the embedded arcade machine ROMs.

Those ROMs have an uncompressed size of 112 KBytes.

So the **entire** compressed emulator with embedded ROMs is about the same
size as the uncompressed ROMs :)

The 112 KByte ROMs compress to about 57 KBytes, this means the actual
compressed code size in the WASM blob without the ROM data
is under 30 KBytes (84 - 57).

Not too bad for an entire 8-bit system emulator I think ;)

gha_language: HTML


text:
# shinken

[![Build Status](https://travis-ci.org/OpenConceptConsulting/puppet-shinken.svg?branch=master)](https://travis-ci.org/OpenConceptConsulting/puppet-shinken)

Offers support for management of the shinken distributed monitoring solution.

Issues?
-------

Simply open a new issue on [github](https://github.com/OpenConceptConsulting/puppet-shinken/issues)!

repository: OpenConceptConsulting/puppet-shinken

repo_name: OpenConceptConsulting/puppet-shinken


text:
Audit Service
=

[![Docker Repository on Quay](https://quay.io/repository/ukhomeofficedigital/pttg-ip-audit/status "Docker Repository on Quay")](https://quay.io/repository/ukhomeofficedigital/pttg-ip-hmrc)

## Overview

This is the Audit Service. The service accepts the following requests:
* save audit data to a database
* retrieve audit data
* retrieve audit history
* archive audit data

## Find Us

* [GitHub]

## Technical Notes

The API is implemented using Spring Boot and exposes a RESTFul interface.

The endpoints are defined in `AuditResource.java`, `AuditHistoryResource.java` and `ArchiveResource.java`.

The database schema is created and maintained by the Flyway scripts found in `src/main/resources/db/migration`.

## Building

### ACP

This service is built by Gradle on [Drone] using [Drone yaml].

### EBSA

This service is built by Gradle on [Jenkins] using the [build Jenkinsfile].

## Infrastructure

### ACP

This service is packaged as a Docker image and stored on [Quay.io]

This service is deployed by [Drone] onto a Kubernetes cluster using its [Kubernetes configuration]

### EBSA

This service is packaged as a Docker image and stored on AWS ECR.

This service is deployed by [Jenkins] onto a Kubernetes cluster using the [deploy Jenkinsfile].

## Running Locally

Check out the project and run the command `./gradlew bootRun` which will install gradle locally, download all dependencies, build the project and run it.

The API should then be available on http://localhost:8083, where:
- port 8083 is defined in `application.properties` with key `server.port`
- the paths for the various endpoints are defined in the various *Resource.java classes
- the expected request body for each endpoint contains a JSON representation of it's *Request.java class

Note that this service runs locally against a HSQL in memory database.  An example of how to run against a docker based postgres database can be found in `AuditEntryJpaJsonbTest.java`.

## Dependencies

When deployed this service depends upon a database created by [pttg-postgres].

## Versioning

For the versions available, see the [tags on this repository].

## Authors

See the list of [contributors] who participated in this project.

## License

This project is licensed under the MIT License - see the [LICENCE.md]
file for details.

[contributors]:                     https://github.com/UKHomeOffice/pttg-ip-audit/graphs/contributors
[Quay.io]:                          https://quay.io/repository/ukhomeofficedigital/pttg-ip-audit
[kubernetes configuration]:         https://github.com/UKHomeOffice/kube-pttg-ip-audit
[Drone yaml]:                       .drone.yml
[tags on this repository]:          https://github.com/UKHomeOffice/pttg-ip-audit/tags
[LICENCE.md]:                       LICENCE.md
[GitHub]:                           https://github.com/orgs/UKHomeOffice/teams/pttg
[Drone]:                            https://drone.acp.homeoffice.gov.uk/UKHomeOffice/pttg-ip-audit
[Jenkins]:                          https://eue-pttg-jenkins-dtzo-kops1.service.ops.iptho.co.uk/job/build_eue_api_audit_service/
[build Jenkinsfile]:                https://bitbucket.ipttools.info/projects/EUE-API/repos/eue-api-shared-services-toolset/browse/Jenkinsfile.pttg_ip_audit
[deploy Jenkinsfile]:               https://eue-pttg-jenkins-dtzo-kops1.service.ops.iptho.co.uk/job/deploy_np_dev_push_eue_api_project_tiller/

repository: UKHomeOffice/pttg-ip-audit

gha_language: Java

repo_name: UKHomeOffice/pttg-ip-audit


text:
# Colors
> Handling colors in Python

## Colour

[colour](https://pypi.org/project/colour/) on PyPI.

> Converts and manipulates common color representation (RGB, HSL, web, …)

This isn't about printing, just switching between formats.

```python
>>> from colour import Color
>>> c = Color("blue")
>>> c
<Color blue>

>>> c.hex
'#00f'
>>> c.hsl
(0.66..., 1.0, 0.5)
>>> c.rgb
(0.0, 0.0, 1.0)
```

## Colors

How to print colored text in Python

[ansicolors](https://pypi.org/project/ansicolors/)

> Add ANSI colors and decorations to your strings.

Examples from the docs.

```python
from colors import color

print(color('my string', fg='blue'))
print(color('some text', fg='red', bg='yellow', style='underline'))
```

As strings:

```python
'\x1b[34mmy string\x1b[0m'
'\x1b[31;43;4msome text\x1b[0m'
```

Named colors

```python
from colors import red, green, blue

print(red('This is red'))
print(green('This is green'))
print(blue('This is blue'))

print(red('red on blue', bg='blue'))
print(green('green on black', bg='black', style='underline'))
```

Using a partial.

```python
from functools import partial
from colors import color

important = partial(color, fg='red', style='bold+underline'))

print(important('this is very important!'))
```

Or without a library.

Based on [SO](https://stackoverflow.com/questions/287871/how-to-print-colored-text-in-terminal-in-python)

This might be better as a dataclass or enum.

```python
class color:

    END_C = '\033[0m'

    OK_BLUE = '\033[94m'
    OK_GREEN = '\033[92m'

    WARNING = '\033[93m'
    FAIL = '\033[91m'

    BOLD = '\033[1m'
    UNDERLINE = '\033[4m'

    HEADER = '\033[95m'

print(color.OK_BLUE, 'Hello, world', color.END_C)
```

As a function:

```python
def ok_blue(value):
    print(color.OK_BLUE, value, color.END_C)
```

gha_language: Makefile


text:
# Sslurp

v0.1 by Evan Coury

## Introduction

**WARNING:** This library is currently undergoing review and should not yet be
considered stable or secure. Proceed with caution...

  Sslurp can be used as a stand-alone library or a ZF2
module.

### CLI Root CA Bundle Updater

[update-ca-bundle.php](https://github.com/EvanDotPro/Sslurp/blob/master/bin/update-ca-bundle.php)
is a handy command-line tool for fetching and building a PEM certificate bundle
from the latest trusted CAs in the [Mozilla source
tree](https://mxr.mozilla.org/mozilla/source/security/nss/lib/ckfw/builtins/certdata.txt).
As a parameter, it requires that you pass it the proper SHA1 fingerprint of the
SSL certificate on mxr.mozilla.org. You can easily find the latest fingerprint
by going to [https://evan.pro/ssl/](https://evan.pro/ssl/).

```
Sslurp Root CA Bundle Updater

Usage:
 ./update-ca-bundle.php [-o output_file]

Options
 -o	Path/filename to the file to (over)write he update root CA bundle. Default to stdout.
```

## License

Sslurp is released under the BSD license. See the included LICENSE file.

repository: pborreli/Sslurp

repo_name: pborreli/Sslurp


text:
# CacheStoreRedis

[![Maintainability](https://api.codeclimate.com/v1/badges/c6a9256278881eab8328/maintainability)](https://codeclimate.com/github/Sage/cache_store_redis/maintainability)
[![Test Coverage](https://api.codeclimate.com/v1/badges/c6a9256278881eab8328/test_coverage)](https://codeclimate.com/github/Sage/cache_store_redis/test_coverage)
[![Gem Version](https://badge.fury.io/rb/cache_store_redis.svg)](https://badge.fury.io/rb/cache_store_redis)

Welcome to CacheStore!

This is the redis implementation for cache_store.

## Installation

Add this line to your application's Gemfile:

```ruby
gem 'cache_store_redis'
```

And then execute:

```bash
$ bundle
```

Or install it yourself as:

```bash
$ gem install cache_store_redis
```

## Testing

To run the tests locally, we use Docker to provide a Ruby container.

### Setup Images:

> This builds the Ruby docker image.

```bash
./script/setup.sh
```

### Run Tests:

> This executes the test suite

```bash
./script/test.sh
```

### Cleanup

> This is used to clean down docker images created in the setup script.

```bash
./script/cleanup.sh
```

## Development

After checking out the repo, run `bin/setup` to install dependencies. You can also run `bin/console` for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).

## Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/sage/cache_store_redis. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.

## License

Cache Store is available as open source under the terms of the
[MIT licence](LICENSE).

Copyright (c) 2018 Sage Group Plc. All rights reserved.

gha_language: Ruby


text:
Vehicle shop example
- Select vehicle by year, make, model to show price
- Sidebar navigation shows all available models
- Currency selector in header sets currency unit (default: $)

Example demonstrates how to:
- use `prefixMatch` to apply prefix to existing match
- use pre-match hook to include query string value in match: `addCurrency`
- validate identifier values with check: `currCheck `
- use `a.link` to create React link component with `retain` query string option

gha_language: JavaScript


text:
# Asynchronous Tests

Sometimes you would want to validate data with information from the server, for example - username availability. In these cases, you should add an async test to your suite, reaching the server before going in and performing the validation. An async test is a [Promise](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise) object. When it *resolves*, the test passes, and when it *rejects* the test fails. An async test will not complete unless either happens.

There are two ways to perform an async test. One, is by passing a promise as your test, and the other by returning a promise from your test function.

## Passing a Promise directly

```js
test('name', 'must be unique', new Promise((resolve, reject) => {
    fetch(`/userExists?name=${name}`)
        .then(res => res.json)
        .then(data => {
            if (data.exists) {
                reject(); // rejects and marks the test as failing
            } else {
                resolve(); // completes. doesn't mark the test as failing
            }
        });
}));
```

## Returning a Promise / Async/Await

```js
test('name',  'Should be unique', async () => {
    const res = await doesUserExist(user);
    return res;
});

test('name',  'I fail', async () => Promise.reject());
```

## Rejecting with rejection message

What if your promise can reject with different messages? No problem!
You can reject the promise with your own message by passing it to the
rejection callback.

Notice that when using rejection messages we do not need to pass `statement`
argument to `test`. This means that the statement will always be inferred
from the rejection message.

In case you do pass `statement`, it will serve as a fallback message in any
case that the rejection message is not provided.

```js
test('name', new Promise((resolve, reject) => {
    fetch(`/checkUsername?name=${name}`)
        .then(res => res.json)
        .then(data => {
            if (data.status === 'fail') {
                reject(data.message); // rejects with message and marks the test as failing
            } else {
                resolve(); // completes. doesn't mark the test as failing
            }
        });
}));
```

repository: fiverr/passable

repo_name: fiverr/passable


text:
---
layout: post
title: "Multiple Hover.com Security Issues"
date: 2013-02-28T12:05:00-05:00
comments: true
categories:
 - rant
 - hover
 - xss
 - security
---

I'm a customer of [Hover](http://www.hover.com) for my domain name needs. However, that will be changing because I don't believe that they take issues seriously.

## The first security issue

I was browsing their site, looking for a new domain, and being the constant tinkerer I am, I entered a single quote into the textfield. I noticed an error, and eventually crafted this url:

{% blockquote %}
https://www.hover.com/domains/results?q=%27%3E%3Cscript%3Ealert%28%27xss%27%29%3B%3C%2Fscript%3E
{% endblockquote %}

There's nothing magical in that URL, however it demonstrated a real vulnerability in their code:

{% img /images/hover_original_xss.png %}

From that point, it could be trivial for an attacker to redirect traffic, and steal user sessions, thus being able to purchase domains with someone else's money.

I reported this issue to them, and had an update within 3 days. They had fixed it, and that URL no longer is vulnerable.

## The second security issue

In January, I was discussing Cross-Site Scripting attacks with a coworker, and was talking about the finding I had with [Hover](http://www.hover.com) and how quickly they responded. Upon further investigation, I found that they didn't really fix it, they just put a band-aid on it. I found the following URL was still vulnerable, but it was a little harder to exploit (onMouseOver):

{% blockquote %}
https://www.hover.com/domains/results?q=hi.com%27%20style%3d%27height:10000px%27%20onmouseover%3d%27alert%28%22xss%22%29
{% endblockquote %}

{% img /images/hover_second_xss.png %}

This really depressed me, since I have spent my fair share of time being a developer, and I always tried to actually fixproblems, instead of just making the symptom go away. Again, I reported the issue, hoping that I'd get another quick turnaround. I asked them to reply within 30 days to indicate their intent on coordinating efforts for remediation. I waited, but I still have no response from them, and it's been over 30 days. I still can't believe they couldn't even respond with a, "We're working on it", response.

## The billing issue

I received an email from [Hover](http://www.hover.com) about a domain name of mine expiring soon. I went into my account, and saw that the credit card was expired, so I went to update it. Unfortunately, I got this error instead:

{% blockquote %}
Declined by Fraud Service
{% endblockquote %}

{% img /images/hover_fraud.png %}

Now, I figured that it was some sort of error, and tried filling it out a couple more times, verifying my card number was correct.After-all, I was only updating the expiration date, nothing else. Unfortunately, all this resulted in were several temporary one dollar charges on my credit card.

Again, I contacted [Hover](http://www.hover.com) and was disappointed yet again. I stated that I was trying to update my credit card details, and instead they renewed my domain for another year, and said, "it failed when you placed the order, but I was able to renew it on my end". I was a bit upset because I hadn't yet decided if I was going to renew through them or not yet, due to the previous security issues. I replied back that it still wasn't helping me for my other domains that will need renewed at some point, and was given these instructions to help troubleshoot:

1. Use only the first and last name, no middle initial.
2. Change the phone number to numbers only, no hyphens.

The first one was a bit odd, since most credit card processors want your name exactlyas it shows on the card. I complied, but it did not help. The second issue I thought was absurd because if their system didn't allow hyphens, why didn't they prevent the user from entering them? This is exactly what javascript validation is for. (Note that they should also validate it on the server-side as well)

Their next steps were to have me call, and give them my credit card number over the phone, and have a billing statement ready to verify every bit of information, as if I've never used a credit card on a website before. It was a bit insulting.

## The 0-day security issue

I had recently read an [article](http://www.skullsecurity.org/blog/2010/stuffing-javascript-into-dns-names) on inserting javascript into DNS to be used for exploitation. I was tinkering with this idea on my own domain names. Unfortunately, I ended up trying to diagnose why none of the DNS records I created in [Hover's](http://www.hover.com) DNS Manager ever made it to their public DNS servers. I'm guessing they have some filtering on the back-end. Eventually I found that their DNS manager was also vulnerable to Cross-Site Scripting:

{% img /images/hover_persistent_xss.png %}

This time, it's a persistent Cross-Site Scripting vulnerability. Every time I navigate to my DNS management screen, I now get a dialog with my cookie. All a user had to do was add a TXT record with the following data:

{% blockquote %}
<img src='https://drupal.org/files/images/sup-dog-magnet-c117515921.jpeg' onload='javascript:alert(document.cookie)\;'>
{% endblockquote %}

They apparently do not even try to filter user-input at all. It's quite depressing.

## Conclusion ([tl;dr](http://www.urbandictionary.com/define.php?term=tl%3Bdr))

[Hover](http://www.hover.com) seems to suffer from the inability to filter user-input. This has become a big problem over the last few years. There are many attacks stemming from this seemingly simple attack. I did my best, and tried reporting to them, but they seem unresponsive, and even when they are, it's not always useful.

I will be moving my domains elsewhere. I haven't yet decided where, and I have a bit of time, but I just don't feel safe keeping my data stored somewhere that can't even stop basic [OWASP Top 10](https://www.owasp.org/index.php/Top_10_2013-T10) vulnerabilities on their main page.


text:
# Team-Profile-Generator
## Description
The Team Profile Generator is a node command line interface that allows a manager to input information about each team member of their team. This will create a HTML webpage that will display a summary of each team member to easily access their team's contact information.

## Installation
1. Copy the clone link of the repository from GitHub.
2. Open Bash or Terminal Window.
3. When the console opens, navigate to the directory the repository will be added to. *Remember to use the command cd to change directories.*
4. In the console, type the command  git clone  and paste the link to repository.
5. Open repository in preferred code editor.
6. Open terminal in code editor.
7. Type in terminal npm i or npm install  to install dependency packages needed.

*Ensure the following packages are installed.*
- Inquirer
- Jest

## Usage
1. The user will navigate to index.js file in terminal.
2. The user will type  node index.js  to initiate the team manager's prompt questions. The user will answer each question. Then, will be asked to add a team member.
3. Once the user has added all members desired and selected 'Done' as their option. The user's input will be added to specific sections of the HTML.
4. The HTML file will be stored in a folder. When user opens the HTML file, it will display a nicely formatted team roster with contact information based on their input.

Picture of example HTML file:

Click the image below to watch a video demonstration:

[![Video Demonstration](https://img.youtube.com/vi/AN2tbpBgTF0/0.jpg)](https://www.youtube.com/watch?v=AN2tbpBgTF0)

## Test
Click the image below to watch a video demonstration of the testing the files:
 [![Video Demonstration](https://img.youtube.com/vi/HCfJ2xHaP20/0.jpg)](https://www.youtube.com/watch?v=HCfJ2xHaP20)

## License
MIT License

Copyright (c) 2021 Amanda Rodriguez

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

 IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

repository: Usmchurst/Team-Profile-Generator

repo_name: Usmchurst/Team-Profile-Generator


text:
### Review

Today we learned about container elements, `border-radius`, fonts and icons, using classes and complex selectors, importing external stylsheets with `<link>` tags inside the `<head>` tag, and how to use JavaScript inside of HTML with the `<script>` tag. For today's assignment, we'll be

### Resources

* MDN CSS Reference
* MDN JavaScript Reference
* MDN DOM API Reference

### Wire it up!

Recreate these layouts using your own colors, lipsum, and icons. Don't look at the source for the project; we'll do that together after it's all done. While the example gets fancy at smaller resolutions, don't spend time on that. You'll need a little JavaScript to make the dynamic parts work, and these layouts should test your full command of CSS and HTML. Ignore the navigation for getting between projects in your implementation.

* [Vertical Timeline](http://tympanus.net/Blueprints/VerticalTimeline/)
* [Horizontal Menu](http://tympanus.net/Blueprints/HorizontalDropDownMenu/)
* [Nested Accordions](http://tympanus.net/Blueprints/NestedAccordion/)

Fill in the blanks with our TIY-Orlando mockup so that the `.datePosted` elements translate the provided date and time into a relative number of hours. Use [the _new_ starting point](/Assignments/2014-10-23.html) or make your code look like mine.


text:
# HKID Validator for Android

This is a Hong Kong ID (HKID) validator design for Android

### How to install
[![](https://jitpack.io/v/seventhmoon/hkid-validator.svg)](https://jitpack.io/#seventhmoon/hkid-validator)

To get a Git project into your build:

Step 1. Add the JitPack repository to your build file

Add it in your root build.gradle at the end of repositories:

	allprojects {
		repositories {
			...
			maven { url "https://jitpack.io" }
		}
	}

Step 2. Add the dependency

	dependencies {
	        implementation 'com.github.seventhmoon:hkid-validator:1.0.2'
	}

### How to use

    HkidValidator.isValid("A123456",'3') // true, valid
    HkidValidator.isValid("A123456",'2') // false, invalid

repository: seventhmoon/hkid-validator

repo_name: seventhmoon/hkid-validator


text:
# legacyDropdownAdapter

`legacyDropdownAdapter` enables us to use the simple dropdown task with projects that were built using the original dropdown task editor in the Project Builder. It is a filter function that takes a JSON task snapshot, from a Panoptes workflow, and returns a new snapshot. The new snapshot will either be a simple dropdown task snapshot, for single menu dropdown tasks, or the original task otherwise.

It can be removed/disabled once we have support for the new task in the project builder. Remove it by removing the preprocessor from the task model, and remove the registration for the `dropdown` task type in the classifier Step model.

```js
  const snapshot = workflow.tasks.T0
  /*
  newTaskSnapshot will be a simple dropdown task, if T0 is a dropdown task with a single menu.
  Otherwise, it will be the original task snapshot.
  */
  newTaskSnapshot = legacyDropdownAdapter(snapshot)
```

repository: zooniverse/front-end-monorepo

repo_name: zooniverse/front-end-monorepo