text:
# MacOS Quick Start

MacOS is not my default OS so support for it is hit-and-miss and won't be optimised.
These steps aim to replace the quick start in the `README.md` for MacOS specifics.

The following commands will install the configuration from the repository.
Tested on MacOS 12.4.

Preliminary tuning:

1. Install Firefox browser and set as default.
2. Fix keyboard configuration to be more PC:
   * System Preferences -> Keyboard -> Modifier Keys:
     * Command = Control
     * Control = Command
     * Repeat for each keyboard.
   * System Preferences -> Keyboard -> Input Sources: add and select `British - PC`.
3. Tune System Preferences:
   * General -> Appearance = Dark.
   * General -> Accent colour = green (or anything you feel).
   * General -> Show scroll bars = when scrolling.
   * General -> Click in the scroll bar to = Jump to the spot that's clicked.
   * General -> Default web browser = Firefox.
   * General -> Recent items = None
   * Dock & Menu Bar -> Automatically hide and show the Dock = true
   * Dock & Menu Bar -> Show recent applications in Dock = false
   * Mission Control -> Automatically rearrange spaces based on most recent use = false
   * Mission Control -> Displays have separate spaces = true
   * Mission Control -> Keyboard and Mouse Shortcuts = all set to do nothing (`-`).
   * Mission Control -> Hot Corners -> Top Left = Mission Control.
   * Mission Control -> Hot Corners -> All others = do nothing (`-`).
   * Siri = Disable all options.
   * Siri -> Siri Suggestions & Privacy = Disable all options.
   * Spotlight -> Search Results = Only apps, Definition and System Preferences.
   * Notifications & Focus -> Notifications = Disable most apps keeping what you want.
   * Security & Privacy -> Privacy -> Location Services -> Enable Location Services = false.
   * Security & Privacy -> Privacy -> Apple Advertising -> Personalised Ads = false.
   * Keyboards -> Shortcuts = Tune presets as desired.
   * Keyboards -> Shortcuts -> Mission Control -> Mission Control = Alt + Tab.
   * Keyboards -> Shortcuts -> App Shortcuts -> Add -> Lock Screen = Window + L.

Install [HomeBrew](https://brew.sh/) and essential dependencies:

```bash
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
eval $(/opt/homebrew/bin/brew shellenv)
brew install --cask itsycal
brew install --cask iterm2
brew install --cask visual-studio-code

brew install coreutils gnupg nvim pinentry-mac tmux
brew install --cask meld

brew tap homebrew/cask-fonts
brew install font-fira-mono-nerd-font
```

Configure iTerm2:

1. Install iTerm2 Nord theme: <https://github.com/arcticicestudio/nord-iterm2#installation>
2. iTerm2 Preferences tuning:
   * General -> Closing -> Quit when all windows are closed = true.
   * General -> Closing -> Confirm closing multiple sessions = true.
   * General -> Selection -> Applications in terminal may access clipboard = true.
   * General -> Selection -> Automatically enter copy mode = false.
   * Appearance -> Tabs -> Show activity indicator = false.
   * Appearance -> Tabs -> Show new-output indicator = false.
   * Profiles -> Default -> Text -> Font = FiraMono Nerd Font.
   * Profiles -> Default -> Text -> Use ligatures = false.
   * Profiles -> Default -> Window -> Columns = MAX (1250; write anything above it).
   * Profiles -> Default -> Terminal -> Scrollback lines = 5000.
   * Keys -> Remap Modifiers -> Remap control key to == Left Command.
   * Keys -> Remap Modifiers -> Remap left command key to == Control.
   * Keys -> Remap Modifiers -> Remap right command key to == Control.
3. Configure copy/paste keybindings to match Gnome Terminal:
   * iTerms -> Settings -> Keys -> Key Binding -> + (at the bottom):
     * Shift + Ctrl + c == Ignore.
     * Shift + Ctrl + v == Paste.
   * System Settings -> Keyboards -> Keyboard Shortcuts -> App Shortcuts -> + (at the bottom):
     * Application == iTerm.app.
     * Menu title == `Edit - Copy`.
     * Keyboard Shortcut: Shift + Ctrl + c.

Configure Bash 5.x as the default shell:

```bash
# Install latest bash from brew.
eval $(/opt/homebrew/bin/brew shellenv)
brew install bash bash-completion

# Set homebrew bash as default shell.
sudo vi /etc/shells
> # Allow use of Bash 5.x installed with homebrew.
> /opt/homebrew/bin/bash

chsh -s /opt/homebrew/bin/bash

# Ensure brew is available to bash shells from now on.
/opt/homebrew/bin/brew shellenv > ~/.system.bashrc

# Extend PATH with new needed locations.
cat << EOF >> ~/.system.bashrc
# Favour GNU version of tools over MacOS one.
export PATH="/opt/homebrew/opt/coreutils/libexec/gnubin:\$PATH"

# Add Visual Studio Code (code).
export PATH="\$PATH:/Applications/Visual Studio Code.app/Contents/Resources/app/bin"

# Enable pip install --user binaries.
export PATH="/Users/stefano.pogliani/Library/Python/3.9/bin:\$PATH"

# Enable bash completion.
[[ -r "/opt/homebrew/etc/profile.d/bash_completion.sh" ]] && . "/opt/homebrew/etc/profile.d/bash_completion.sh"
[[ -f /Library/Developer/CommandLineTools/usr/share/git-core/git-completion.bash ]] && . /Library/Developer/CommandLineTools/usr/share/git-core/git-completion.bash
EOF
```

Exit the terminal and re-open it, then proceed to dotfiles provisioning:

```bash
# Install rust and starship
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
cargo install starship

# Initialise global git config:
git config --global user.name "Stefano Pogliani"
git config --global user.email "<EMAIL>"
git config --global core.editor nvim

# Fetch this repo and install configs.
git clone --recursive https://github.com/stefano-pogliani/dotfiles.git
cd dotfiles

# Set the appropriate profile for the system.
# We can override in CLI but this helps to avoid mistakes.
echo 'work' > ~/.dot.profile
echo 'export TARGET_MACOS=yes' > ~/.dot.variables
echo 'export TMUX_CLIPBOARD=macos' >> ~/.dot.variables

# NOTE: Many plugins are installed as part of this.
make provision
# To update the local configuration on a provisioned system:
make sync

# MacOS will only load ~/.profile for interactive shells.
echo 'source ~/.bashrc' > ~/.profile

# ADDITIONAL TASKS:
#  * Generate SSH key and add to places (if needed)
#  * Generate GPG key and add to places (if needed)
```

## GPG additional changes

```bash
# Configure the pinentry program for keychain support.
echo 'pinentry-program /opt/homebrew/bin/pinentry-mac' >> ~/.gnupg/gpg-agent.conf
```

## Mac messed with the keyboard again

In case of layout issues reset keyboard assistant data and re-identify the keyboard:

1. Disconnect all devices from Mac.
2. `sudo rm /Library/Preferences/com.apple.keyboardtype.plist`
3. Reboot.
4. Reconnect the external keyboard and follow assistant instructions.

Ref: <https://www.techwalla.com/articles/how-to-reset-the-external-keyboard-on-an-apple-mac>

repository: stefano-pogliani/dotfiles

branch_name: refs/heads/main

repo_name: stefano-pogliani/dotfiles


text:
---
title: Be Mindful What you Measure
date: 2019-04-30T23:08:02.421Z
summary: >
  When you measure something, you change it.

  "In physics, the observer effect is the theory that the mere observation of a
  situation or phenomenon inevitably changes that phenomenon."

  Quantum physics are pretty weird, but it turns out that humans are weird in
  much the same way. Who knew?
tags:
  - post
---
When you measure something, you change it.

> **[Observer Effect](https://en.wikipedia.org/wiki/Observer_effect_(physics)):**
>
> "In physics, the observer effect is the theory that the mere observation of a situation or phenomenon inevitably changes that phenomenon."

Quantum physics are pretty weird, but it turns out that humans are weird in much the same way. Who knew?

 AP tests are coming up, and the ACT is still on the horizon.  SAT results come out Thursday. I work tirelessly to resist the idea that my education is about beating a test, but it's not easy to avoid.

In the pursuit of more data, we've created tests that attempt to evaluate students' abilities. These tests aren't perfect, of course — you can't effectively measure skills like interpersonal communication on a multiple choice exam — but it seems reasonable to expect that more data will result in better decision-making. Unfortunately, when we measure what students have learned, we change what they learn.

It's no secret that schools, if they want to do a "good" job (according to the measurement tools at our disposal), must teach the skills that are testable and overlook the ones that aren't. But the compatibility of a skill with a multiple-choice exam is not at all correlated with its usefulness in the real world. (Or, if you're feeling really spicy, you might even argue that a correlation exists and is downward-sloping.) As a result, our choice to collect more data has caused the quality of education to decline.

# We must be careful what we measure.

**If measurement can affect an outcome then we need to be thoughtful about what we measure and how we might change things as a result.** We all know that measuring a programmer's success in "lines of code" is a futile exercise.

“Measuring programming progress by lines of code is like measuring aircraft building progress by weight.” - Bill Gates
— Programming Wisdom (@CodeWisdom) March 3, 2017

On my best days, I write negative code. But if you want me to make that number big, it's an easy game to play:

```
if (
  true
)
{
  console.log('6 lines already! 🎉 Am I a rockstar developer?')
}
```
We encourage what we measure, so we have to be sure that we're measuring the right thing.

Fortunately, this measurement effect can be weaponized for good too. That's why [Discourse](https://www.discourse.org/) displays the amount of time a user has spent reading, not how much they have written:

> "Printing a post count number next to every user's name implies that the more you post, the better things are. [...] *Is this even true?*
>
> I find that the value of conversations has little to do with how much people are talking. I find that too much talking has a *negative* effect on conversations."
>
> \- [Jeff Atwood (Coding Horror)](https://blog.codinghorror.com/because-reading-is-fundamental-2/)

Instead, by measuring time spent reading, the system encourages people to read more. It's not complicated, but — as is often the case with simple things — it's [startlingly effective](https://blog.codinghorror.com/for-a-bit-of-colored-ribbon/).

## How can we apply this idea ourselves?

When we are developing our own products, we can — we must — think about the effects that our measurements will have, and choose to emphasize the measurements that we believe matter. If you're making a social media app, think about [the numbers](https://dev.to/grv19/number-of-followers-to-be-shown-on-a-profile-pbi) you display next to users' posts. Do they promote the kind of interactions you're looking for?

Instagram is testing hiding like count from audiences,

as stated in the app: "We want your followers to focus on what you share, not how many likes your posts get" pic.twitter.com/MN7woHowVN
— Jane Manchun Wong (@wongmjane) April 18, 2019

There are no set rules. The answers to questions like these will change depending on your circumstances and goals. But there's no doubt in my mind that simply measuring and displaying data is a great way to manipulate it.

Perhaps we can measure how we measure. That might do the trick.

license: MIT

branch_name: refs/heads/master

extension: md

license_type: permissive


text:
# arXiv IT Advisory Group

The arXiv Next Generation project is a multi-year effort to modernize the legacy software system that supports arXiv.org. arXiv-NG is a large project, with considerable risks. Our [Scientific](/about/people/scientific_ad_board) and [Member Advisory](/about/people/member_ad_board) Boards continue to provide valuable high-level guidance about the project, but we also recognize the need for critical feedback from experts and peers who are actively working in this space. In addition, we recognize that many of the technical decisions that we make throughout this project will have social and scientific impacts that may be difficult for us to anticipate on our own.

The arXiv IT Advisory Group (AG) is to review and discuss technical decisions made by the arXiv IT team, provide critical and constructive feedback and advice, and hold arXiv IT leadership accountable for development and quality goals. Establishing this group is part of our broader effort to increase transparency and communication around IT operations, and so we also hope that the outputs of this group will enhance stakeholder awareness and buy-in.

## Membership

The IT AG is composed of a handful of experts who are actively involved in developing IT infrastructure to support scholarship. Members are selected and invited by the arXiv IT Team Lead and participate on a voluntary basis.

## Current members

- Daina Bouquin, Center for Astrophysics, Harvard & Smithsonian
- Mark Doyle, American Physical Society
- Alberto Accomazzi, Astrophysics Data System
- Thorsten Schwander, SLAC National Accelerator Laboratory, Stanford University
Operations

## Meetings

We plan to convene two or three meetings per year, held electronically (e.g., over Zoom or Skype), on dates that are mutually convenient.
- In advance of each meeting we will provide a summary of IT-related activity over the intervening period, including follow-up regarding earlier recommendations of the AG.
- The meeting itself will be an opportunity for AG members to ask questions and discuss recent development activity and operational objectives. Given the small size of the group, we hope that this will be an open and informal discussion.
- Following each meeting, we will compile a brief (approximately one page) summary of what was discussed, and any specific recommendations. We will ask AG members to attest to its accuracy, and optionally provide additional comments and recommendations. We will make this document publicly available, and share it with our Advisory Boards, funders, and other stakeholders.

In order to keep our regular meetings brief and efficient, we will provide members with regular updates concerning:
- Planned and new versions of software components, including checkpoints such as alpha and beta testing.
- External collaborations.
- Updates to our architectural documentation.
- Requests for input regarding APIs, technology choices, and IT-related partnerships.

Members are encouraged to ask questions and raise topics of interest or concern throughout the year.

repository: jetatar/arxiv-docs

repo_name: jetatar/arxiv-docs


text:
# React Redux cli

CLI for common tasks when building apps w/ react and redux.

## Usage
`cd` into your app root.

Add a `rr-cli.config.js` file.

Run commands - e.g. rr add component NAME -scss

## rr-cli.config.js
```javascript
{
  "type": "next.js|rruhe",
}
```

## Next.js usage (type === 'next.js')

### Commands

* add component (rr add component NAME)
* add page (rr add page NAME)
* remove component (rr remove component NAME)
* remove page (rr remove page NAME)
* rename component (rr rename component NAME)
* rename page (rr rename page NAME)
* add redux duck (rr add duck NAME)

## React redux universal hot example usage (type === 'rruhe')

### Commands

* add component (rr add component NAME)
* add container (rr add container NAME)
* remove component (rr remove component NAME)
* remove container (rr remove container NAME)
* rename component (rr rename component NAME)
* rename container (rr rename container NAME)
* add redux duck (rr add duck NAME)

### Options
* -scss
* -asyncConnect
* -connect

repository: iamdevonbutler/react-cli

repo_name: iamdevonbutler/react-cli


text:
# RDS

## Create a PostgreSQL instance in RDS
1. Go to **RDS** under **Database** section.
2. Click on **Launch a DB Instance**.
3. Click on PostgreSQL logo, tick the _"Only enable options eligible for RDS Free Usage Tier"_ checkbox at the bottom and click Next.
7. Enter a name on _DB Instance identifier_ (we will need it later, so don’t forget it).
8. Enter a username and password and click Next (again, we will need these later).
9. Select No on **Publicly Accessible**.
10. Availability Zone: `us-east-1a`.
11. On **VPC security groups** choose _Select existing VPC security groups_ and select the security group you create when [launching the EC2 instance](/workshop/s3-web-ec2-api-rds/02-EC2-instances.md#launch-your-first-ec2-instance).
11. Pick a db name and click Launch Instance (again, we will need the database name later).
12. Click View Your DB Instances.

Now our instance is created. We configure its access, allowing every instance under the security group that was created in the previous section to connect.

## Add DB parameters on Parameters Store

As before, we will need some variables stored in the parameter store, including the database name, username, password and endpoint. These variables are referenced in [this file](/backend/conduit/settings/ec2.py), so Django can access the database.

1. Go to **RDS** under **Database** section.
2. Click on Instances.
3. See details of your db and copy the **Endpoint**. This will be the value for `DATABASE_HOST`.
4. Go to **EC2** under **Compute** section.
5. On the left menu select Parameter Store.
6. Click Create Parameter.
7. Enter  `/prod/api/DATABASE_NAME` as the name and a meaningful description like "Name of the PostgreSQL database".
8. Enter the DB name you selected before on the value attribute.
9. Click create parameter and close.
10. Now we will need to do the same thing for the username and host
  1. For the username enter `/prod/api/DATABASE_USER` as the name and your database username and as the value
  2. For the host enter `/prod/api/DATABASE_HOST` as the name and the hostname you copied earlier as the value
11. For `/prod/api/DATABASE_PASSWORD` do the same steps but select as **Type: Secure String** and as KMS Key ID the key `workshopkey`.

Now we have our database parameters set, and the password encrypted. Only our EC2 instances will be able to decrypt it.

---
**Extra mile:**

- Can you `ping` the Postgres instance?
- Try to connect to the DB through your running EC2 instance.

---

**Next:** create a [CodeDeploy project to deploy your API](/workshop/s3-web-ec2-api-rds/04-code-deploy.md).

repository: jimearruti/aws-workshop

gha_license_id: MIT

repo_name: jimearruti/aws-workshop


text:
Kankun smart plug outlet controller

Full tutorial is at http://benlo.com/esp8266/KankunSmartPlug.html

scp to the Kankun

create /www/cgi-bin

chmod 755 cgi-bin

put relay.cgi in cgi-bin

upload init.lua and httpget.lua to the esp8266

/www/index.html and /www/click.mp3 are optional

They make a pretty toggle button for browser control of the Kandun outlet.

Browse to http://<IP_ADDRESS> with any web browser.

The toggle will reflect the state of the Kandun switch as well as control it.

repository: isnadh/LuaLoader

path: /examples/Kankun WiFi Plug/README.md

repo_name: isnadh/LuaLoader


text:
# FHIR Publisher Runner

This project is a simple Maven wrapper to run the FHIR Publisher tool. It will check if a neweer snapshot is available.

To run this project on Windows:
```
mvnw exec:java
```

To run this project on OSX/Linux:
```
./mvnw exec:java
```

repository: jamesagnew/fhir-publisher-runner

repo_name: jamesagnew/fhir-publisher-runner


text:
# Molecular dynamics of water

In this example, we will simulate a molecular liquid of high scientific
interest: water. At the same time we will learn how we can reuse a potential
definition between multiple simulations, and how we can switch the potential
definition in a simulation.

You will need three input files for this simulation:

- [water.xyz]: the initial configuration;
- [water.toml]: the simulation input;
- [water-fSCP.toml]: the potentials definitions.

The simulation run in a few minutes (up to 10), and will write the trajectory of
the system.

[water.xyz]: data/water.xyz
[water.toml]: data/water.toml
[water-fSCP.toml]: data/water-fSCP.toml

# The input files commented

## `water.toml` file

The main input file is pretty similar to the previous examples, with two
novelties:

- The `guess_bonds = true` entry tell Lumol to try to guess the bonds from the
  distances in the XYZ file. This is needed because we want to simulate a
  molecule, but there is not bonding information inside the XYZ format. If we
  used a PDB file instead, this would not be needed;
- The `potentials = "water-fSCP.toml"` tell Lumol to read the potentials from
  the `water-fSCP.toml` file. Using this type of input for the potentials allow
  to share the same potential for multiple simulations, and to easily change
  the potential used in the simulation.

```toml
[input]
version = 1

[[systems]]
file = "water.xyz"
cell = 28.0
guess_bonds = true
potentials = "water-fSCP.toml"

[[simulations]]
nsteps = 5000
outputs = [
    {type = "Trajectory", file = "trajectory.xyz", frequency = 10}
]

[simulations.propagator]
type = "MolecularDynamics"
timestep = "1 fs"
```

## `water-fSCP.toml` file

`water-fSCP.toml` is a standalone potential input file. It contains the same
data as a potential definition inside a `[[systems]]` section, but without the
`system.potential` prefix on all sections names.

In this input, we start by defining which version of the input format we are
using.

```toml
# f-SPC model of water, using Ewald summation for electrostatics
[input]
version = 1
```

Then, we can define some global input data: the pair potential cutoff and the
atomic charges.

```toml
[global]
cutoff = "14 A"

[charges]
O = -0.82
H = 0.41
```

The pair potential section contains the usual pairs declarations, with a few
additional options.

```toml
[[pairs]]
atoms = ["O", "O"]
lj = {sigma = "3.16 A", epsilon = "0.155 kcal/mol"}
```

We can add a `restriction` to restrict a specific pair interaction to some kind
of pairs. Here we will only account for H-H pairs inside the same molecule.

```toml
[[pairs]]
atoms = ["H", "H"]
harmonic = {k = "79.8 kcal/mol/A^2", x0 = "1.633 A"}
restriction = "IntraMolecular"
```

We can also define a non-interacting pair interaction! This is useful when some
atoms does not interact in the model we use.

```toml
[[pairs]]
atoms = ["H", "O"]
null = {}
```

Then comes the bonds and angles definitions. These are the interactions used
between bonded particles (or angles formed by two bonds and dihedral angles
formed by three bonds).

```toml
[[bonds]]
atoms = ["O", "H"]
harmonic = {k = "1054.2 kcal/mol/A^2", x0 = "1.0 A"}

[[angles]]
atoms = ["H", "O", "H"]
harmonic = {k = "75.9 kcal/mol/rad^2", x0 = "109.5 deg"}
```

Finally we specify how to compute the electrostatic interaction, this time using
the Ewald sum. We can restrict the coulombic interactions to only apply between
particles not in the same molecule using a `restriction` here too.

```toml
[coulomb]
ewald = {cutoff = "8.5 A", kmax = 3}
restriction = "InterMolecular"
```

repository: droundy/lumol

repo_name: droundy/lumol


text:
---
title: vtctl Schema, Version, Permissions Command Reference
series: vtctl
docs_nav_title: Schema Versions & Permissions
---

The following `vtctl` commands are available for administering Schema, Versions and Permissions.

## Commands

### GetSchema

Displays the full schema for a tablet, or just the schema for the specified tables in that tablet.

#### Example

GetSchema -- [--tables=<table1>,<table2>,...] [--exclude_tables=<table1>,<table2>,...] [--include-views] <tablet alias>

#### Flags

| Name | Type | Definition |
| :-------- | :--------- | :--------- |
| exclude_tables | string | Specifies a comma-separated list of tables to exclude. Each is either an exact match, or a regular expression of the form /regexp/ |
| include-views | Boolean | Includes views in the output |
| table_names_only | Boolean | Only displays table names that match |
| tables | string | Specifies a comma-separated list of tables for which we should gather information. Each is either an exact match, or a regular expression of the form /regexp/ |

#### Arguments

* <tablet alias> – Required. A Tablet Alias uniquely identifies a vttablet. The argument value is in the format <cell name>-<uid>.

#### Errors

* The <tablet alias> argument is required for the <GetSchema> command This error occurs if the command is not called with exactly one argument.

### ReloadSchema

Reloads the schema on a remote tablet.

#### Example

ReloadSchema <tablet alias>

#### Arguments

* <tablet alias> – Required. A Tablet Alias uniquely identifies a vttablet. The argument value is in the format <cell name>-<uid>.

#### Errors

* The <tablet alias> argument is required for the <ReloadSchema> command This error occurs if the command is not called with exactly one argument.

### ReloadSchemaShard

Reloads the schema on all the tablets in a shard.

#### Example

ReloadSchemaShard -- [--concurrency=10] [--include_primary=false] <keyspace/shard>

#### Flags

| Name | Type | Definition |
| :-------- | :--------- | :--------- |
| concurrency | Int | How many tablets to reload in parallel |
| include_primary | Boolean | Include the primary tablet |

#### Arguments

* <keyspace/shard> – Required. The name of a sharded database that contains one or more tables as well as the shard associated with the command. The keyspace must be identified by a string that does not contain whitespace, while the shard is typically identified by a string in the format <range start>-<range end>.

#### Errors

* The <keyspace/shard> argument is required for the <ReloadSchemaShard> command This error occurs if the command is not called with exactly one argument.

### ReloadSchemaKeyspace

Reloads the schema on all the tablets in a keyspace.

#### Example

ReloadSchemaKeyspace -- [--concurrency=10] [--include_primary=false] <keyspace>

#### Flags

| Name | Type | Definition |
| :-------- | :--------- | :--------- |
| concurrency | Int | How many tablets to reload in parallel |
| include_primary | Boolean | Include the primary tablet(s) |

#### Arguments

* <keyspace> – Required. The name of a sharded database that contains one or more tables. Vitess distributes keyspace shards into multiple machines and provides an SQL interface to query the data. The argument value must be a string that does not contain whitespace.

#### Errors

* The <keyspace> argument is required for the <ReloadSchemaKeyspace> command This error occurs if the command is not called with exactly one argument.

### ValidateSchemaShard

Validates that the schema on the primary tablet matches all of the replicas.

#### Example

ValidateSchemaShard -- [--exclude_tables=''] [-include-views] <keyspace/shard>

#### Flags

| Name | Type | Definition |
| :-------- | :--------- | :--------- |
| exclude_tables | string | Specifies a comma-separated list of tables to exclude. Each is either an exact match, or a regular expression of the form /regexp/ |
| include-views | Boolean | Includes views in the validation |

#### Arguments

* <keyspace/shard> – Required. The name of a sharded database that contains one or more tables as well as the shard associated with the command. The keyspace must be identified by a string that does not contain whitespace, while the shard is typically identified by a string in the format <range start>-<range end>.

#### Errors

* The <keyspace/shard> argument is required for the <ValidateSchemaShard> command This error occurs if the command is not called with exactly one argument.

### ValidateSchemaKeyspace

Validates that the schema on the primary tablet for shard 0 matches the schema on all of the other tablets in the keyspace.

#### Example

ValidateSchemaKeyspace -- [--exclude_tables=''] [--include-views] <keyspace name>

#### Flags

| Name | Type | Definition |
| :-------- | :--------- | :--------- |
| exclude_tables | string | Specifies a comma-separated list of tables to exclude. Each is either an exact match, or a regular expression of the form /regexp/ |
| include-views | Boolean | Includes views in the validation |

#### Arguments

* <keyspace name> – Required. The name of a sharded database that contains one or more tables. Vitess distributes keyspace shards into multiple machines and provides an SQL interface to query the data. The argument value must be a string that does not contain whitespace.

#### Errors

* The <keyspace name> argument is required for the <ValidateSchemaKeyspace> command This error occurs if the command is not called with exactly one argument.

### ApplySchema

Applies the schema change to the specified keyspace on all shards. The recommended method of applying schema changes is via [Online DDL](../../../../user-guides/schema-changes/managed-online-schema-changes/), which ensures migrations are non-blocking and keeps replication in sync throughout the operation. --ddl_strategy is used to instruct migrations via vreplication, `gh-ost` or `pt-osc` with optional parameters. -request_context allows the user to specify a custom request context for online DDL migrations.

#### Example

ApplySchema -- [--wait_replicas_timeout=10s] [--ddl_strategy=<ddl_strategy>] [--request_context=<unique-request-context>] {--sql=<sql> || --sql-file=<filename>} <keyspace>

#### Flags

| Name | Type | Definition |
| :-------- | :--------- | :--------- |
| caller_id | string | Effective caller ID used for the operation and should map to an ACL name which grants this identity the necessary permissions to perform the operation (this is only necessary when strict table ACLs are used). |
| ddl_strategy | string | Online DDL strategy, compatible with @@ddl_strategy session variable (examples: 'gh-ost', 'pt-osc', 'gh-ost --max-load=Threads_running=100' (default "direct"). |
| request_context | string | For Only DDL, optionally supply a custom unique string used as context for the migration(s) in this command. By default a unique context is auto-generated by Vitess. |
| sql | string | A list of semicolon-delimited SQL commands. |
| sql-file | string | Identifies the file that contains the SQL commands. This file needs to exist on the server, rather than on the client. |
| wait_replicas_timeout | Duration | The amount of time to wait for replicas to receive the schema change via replication (default 10s). |

#### Arguments

* <keyspace> – Required. The name of a sharded database that contains one or more tables. Vitess distributes keyspace shards into multiple machines and provides an SQL interface to query the data. The argument value must be a string that does not contain whitespace.

#### Errors

* The <keyspace> argument is required for the command<ApplySchema> command This error occurs if the command is not called with exactly one argument.
* If using a file you may need to pass the absolute path.

### CopySchemaShard

Copies the schema from a source shard's primary (or a specific tablet) to a destination shard. The schema is applied directly on the primary of the destination shard, and it is propagated to the replicas through binlogs.

#### Example

CopySchemaShard -- [--tables=<table1>,<table2>,...] [--exclude_tables=<table1>,<table2>,...] [--include-views] [--wait_replicas_timeout=10s] {<source keyspace/shard> || <source tablet alias>} <destination keyspace/shard>

#### Flags

| Name | Type | Definition |
| :-------- | :--------- | :--------- |
| exclude_tables | string | Specifies a comma-separated list of tables to exclude. Each is either an exact match, or a regular expression of the form /regexp/ |
| include-views | Boolean | Includes views in the output |
| tables | string | Specifies a comma-separated list of tables to copy. Each is either an exact match, or a regular expression of the form /regexp/ |
| wait_replicas_timeout | Duration | The amount of time to wait for replicas to receive the schema change via replication. |

#### Arguments

* <source keyspace/shard> – Required. A keyspace or shard to be used as the source of the copy.
* <source tablet alias> – Required. A Tablet Alias uniquely identifies a vttablet. The argument value is in the format <cell name>-<uid>.
* <destination keyspace/shard> – Required. The name of a sharded database that contains one or more tables as well as the shard associated with the command. The keyspace must be identified by a string that does not contain whitespace, while the shard is typically identified by a string in the format <range start>-<range end>.

#### Errors

* The <source keyspace/shard> and <destination keyspace/shard> arguments are both required for the <CopySchemaShard> command. Instead of the <source keyspace/shard> argument, you can also specify <tablet alias> which refers to a specific tablet of the shard in the source keyspace This error occurs if the command is not called with exactly 2 arguments.

### ValidateVersionShard

Validates that the version on the primary matches all of the replicas.

#### Example

ValidateVersionShard <keyspace/shard>

#### Arguments

* <keyspace/shard> – Required. The name of a sharded database that contains one or more tables as well as the shard associated with the command. The keyspace must be identified by a string that does not contain whitespace, while the shard is typically identified by a string in the format <range start>-<range end>.

#### Errors

* The <keyspace/shard> argument is required for the <ValidateVersionShard> command This error occurs if the command is not called with exactly one argument.

### ValidateVersionKeyspace

Validates that the version on the primary of shard 0 matches all of the other tablets in the keyspace.

#### Example

ValidateVersionKeyspace <keyspace name>

#### Arguments

* <keyspace name> – Required. The name of a sharded database that contains one or more tables. Vitess distributes keyspace shards into multiple machines and provides an SQL interface to query the data. The argument value must be a string that does not contain whitespace.

#### Errors

* The <keyspace name> argument is required for the <ValidateVersionKeyspace> command This error occurs if the command is not called with exactly one argument.

### GetPermissions

Displays the permissions for a tablet.

#### Example

GetPermissions <tablet alias>

#### Arguments

* <tablet alias> – Required. A Tablet Alias uniquely identifies a vttablet. The argument value is in the format <cell name>-<uid>.

#### Errors

* The <tablet alias> argument is required for the <GetPermissions> command This error occurs if the command is not called with exactly one argument.

### ValidatePermissionsShard

Validates that the permissions on the primary tablet match all the replicas.

#### Example

ValidatePermissionsShard <keyspace/shard>

#### Arguments

* <keyspace/shard> – Required. The name of a sharded database that contains one or more tables as well as the shard associated with the command. The keyspace must be identified by a string that does not contain whitespace, while the shard is typically identified by a string in the format <range start>-<range end>.

#### Errors

* The <keyspace/shard> argument is required for the <ValidatePermissionsShard> command This error occurs if the command is not called with exactly one argument.

### ValidatePermissionsKeyspace

Validates that the permissions on the primary of shard 0 match those of all of the other tablets in the keyspace.

#### Example

ValidatePermissionsKeyspace <keyspace name>

#### Arguments

* <keyspace name> – Required. The name of a sharded database that contains one or more tables. Vitess distributes keyspace shards into multiple machines and provides an SQL interface to query the data. The argument value must be a string that does not contain whitespace.

#### Errors

* The <keyspace name> argument is required for the <ValidatePermissionsKeyspace> command This error occurs if the command is not called with exactly one argument.

### GetVSchema

Displays the VTGate routing schema.

#### Example

GetVSchema <keyspace>

#### Arguments

* <keyspace> – Required. The name of a sharded database that contains one or more tables. Vitess distributes keyspace shards into multiple machines and provides an SQL interface to query the data. The argument value must be a string that does not contain whitespace.

#### Errors

* The <keyspace> argument is required for the <GetVSchema> command This error occurs if the command is not called with exactly one argument.

### ApplyVSchema

Applies the VTGate routing schema to the provided keyspace. Shows the result after application.

#### Example

ApplyVSchema -- {--vschema=<vschema> || --vschema_file=<vschema file> || --sql=<sql> || --sql_file=<sql file>} [--cells=c1,c2,...] [--skip_rebuild] [--dry-run]<keyspace>

#### Flags

| Name | Type | Definition |
| :-------- | :--------- | :--------- |
| cells | string | If specified, limits the rebuild to the cells, after upload. Ignored if skipRebuild is set. |
| dry-run | Boolean | If set, do not save the altered vschema, simply echo to console. |
| skip_rebuild | Boolean | If set, do not rebuild the SrvSchema objects. |
| sql | add vindex | A vschema ddl SQL statement (e.g. add vindex, `alter table t add vindex hash(id)`, etc) |
| sql_file | add vindex | A vschema ddl SQL statement (e.g. add vindex, `alter table t add vindex hash(id)`, etc) |
| vschema | string | Identifies the VTGate routing schema |
| vschema_file | string | Identifies the VTGate routing schema file |

#### Arguments

* <keyspace> – Required. The name of a sharded database that contains one or more tables. Vitess distributes keyspace shards into multiple machines and provides an SQL interface to query the data. The argument value must be a string that does not contain whitespace.

#### Errors

* The <keyspace> argument is required for the <ApplyVSchema> command. This error occurs if the command is not called with exactly one argument.
* Either the <vschema> or <vschema>File flag must be specified when calling the <ApplyVSchema> command.
* If using a file you may need to pass the absolute path.

#### Warnings

* In some cases, unknown params passed to Vindexes may be accepted but reported as warnings in the command response.

### GetRoutingRules

```
GetRoutingRules
```

### ApplyRoutingRules

Applies the VSchema routing rules.

#### Example

```
ApplyRoutingRules -- {--rules=<rules> | --rules_file=<rules_file>} [--cells=c1,c2,...] [--skip_rebuild] [--dry-run]
```

#### Flags

| Name | Type | Definition |
| :-------- | :--------- | :--------- |
| cells | string | If specified, limits the rebuild to the cells, after upload. Ignored if skip_rebuild is set. |
| dry-run | Boolean | If set, do not save the altered vschema, simply echo to console. |
| skip_rebuild | Boolean | If set, do not rebuild the SrvSchema objects. |
| rules | string | Specify rules as a string. |
| rules_file | string | Specify rules in a file. |

### RebuildVSchemaGraph

Rebuilds the cell-specific SrvVSchema from the global VSchema objects in the provided cells (or all cells if none provided).

#### Example

RebuildVSchemaGraph -- [--cells=c1,c2,...]

#### Flags

| Name | Type | Definition |
| :-------- | :--------- | :--------- |
| cells | string | Specifies a comma-separated list of cells to look for tablets |

#### Errors

* <RebuildVSchemaGraph> doesn't take any arguments This error occurs if the command is not called with exactly 0 arguments.

## See Also

* [vtctl command index](../../vtctl)

repository: vitessio/website

branch_name: refs/heads/prod

gha_language: Java

repo_name: vitessio/website


text:
# Installomator

_The one installer script to rule them all._

![](https://img.shields.io/github/v/release/scriptingosx/Installomator) ![](https://img.shields.io/github/downloads/scriptingosx/Installomator/latest/total) ![](https://img.shields.io/badge/macOS-10.14%2B-success) ![](https://img.shields.io/github/license/scriptingosx/Installomator)

This script is in the "we find it useful, it is working for us" stage.

Your production and deployment environment will be different, please test thoroughly before rolling it out to your production.

I have put a lot of work into making it stable and safe, but I cannot - of course - make _any_ promises that it won't break in some not yet encountered edge case.

## Support and Contributing

Discussion, support and advice around Installomator happens in the `#installomator` channel in the [MacAdmins Slack](https:/macadmins.org). Go there for support questions.

Do not create an issue just when you have a questions, but do file an issue or pull request for bugs or wrong behavior. When in doubt, ask in the above Slack channel.

If you have added a new label, then please file a pull request. (and Thank you!)

## More reading

There are a few interesting post on Installomator on my weblog:

- [Introducing Installomator](https://scriptingosx.com/2020/05/introducing-installomator/)
- [Using Installomator with Jamf Pro](https://scriptingosx.com/2020/06/using-installomator-with-jamf-pro/) by Mischa van der Bent

## Background

As a system engineer at [an Apple Authorized Enterprise Reseller](https://prowarehouse.nl), we manage a lot of Jamf instances.

Some of these instances are tightly managed, i.e. the versions of the operating system and third party software are controlled and updates will only be pushed with the management system when the administration and security team went through an approval process and then the update is automated. This is an important and valid workflow and the right fit for many deployments.

Installomator was _not_ written for these kinds of deployment.

If you are running this kind of deployment, you want to use [AutoPkg](https://github.com/autopkg/autopkg) and you can stop reading here.

There are other kinds of deployments, though. In these deployments the management system is merely used to "get the user ready" as quickly as possible when they set up a new machine, and to offer software from a self service portal. In these deployments, system and software installations are 'latest version available' and updates are user driven (though we do want to nag them).

These deployments are

- user driven
- low control
- minimal maintenance effort

These are mostly 'user controlled' Macs and we (the admins) just want to assist the user in doing the right thing. And the right thing is (often) to install the latest versions and updates when they are available.

The Mac App Store and software pushed through the Mac App Store follow this approach. When you manage and deploy software through the App Store — whether it is on iOS or macOS — neither the MacAdmin nor the user get a choice of the application version. They will get the latest version.

In such deployments, keeping the installers hosted in your management system up to date is an extra burden. AutoPkg can, well, automate much of the download/re-package/upload/stage cycle, but it still requires oversight and maintenance. Instead of downloading, re-packaging, uploading application installers to the management system, it is often easier to run a script which downloads the latest version directly from the vendor's servers and installs it.

There are obviously a few downsides to this approach:

- when your fleet is mostly on site and many will install or update at the same time, they will reach out over the internet to the vendor's servers, possibly overwhelming your internet connection
- when you download software from the internet, it should be verified to avoid man-in-the-middle or other injection attacks
- there is no control over which version the clients get, you cannot "hold back" new versions for testing and approval workflows
- some application downloads are gated behind logins or paywalls and cannot be automated this way

Some of these disadvantages can be seen as advantages in different setups. When your fleet is mostly mobile and offsite, then downloading from vendor servers will relieve the inbound connection to your management server, or the data usage on your management system's cloud server. Software vendors are pushing for subscriptions with continuous updates and feature releases, so moving the entire team to the latest versions quickly can make those available quickly. Also being on the latest release includes all current security patches.

Because this is an attractive solution for _certain kinds_ of deployment, there are already many scripts out there that will download and install the latest version of a given software. And we have built and used quite a few in-house, as well. Most importantly, [William Smith has this script](https://gist.github.com/talkingmoose/a16ca849416ce5ce89316bacd75fc91a) which can be used to install several different Microsoft applications and bundles, because Microsoft has a nice unified URL scheme.

At some point, earlier this year, I got frustrated at the number of scripts we were maintaining (or failing to). Also, my concern that most of the scripts weren't doing _any_ verification of the download was getting unbearable. So, I set out to write the one install script to rule them all...

## Goals

My goals for Installomator are:

- work with various common archive types
- verify the downloaded archive or application
- have a simple 'interface' to the admin
- single script file so it can 'easily' be copied into a management system
- extensible without deep scripting knowledge
- work independently of a specific management system
- no dependencies that may be removed from macOS in the future or are not pre-installed

In Detail:

### Archive types

Installomator can work with the following common archive and installer types:

- pkg: as a direct download, or when nested inside a dmg or zip
- dmg: for the common 'drag app to /Applications' installation style
- zip: the application is just compressed with zip or or tbz

When the download yields a pkg file, Installomator will run `installer` to install it on the current system.

Applications in dmgs or zips will be copied to `/Applications` and their owner will be set to the current user, so the install works like a standard drag'n drop installation.

(I consider it a disgrace, that Jamf, after nearly 20 years, _still_ cannot deal with 'drag'n drop installation dmgs' natively. It's not _that_ hard.)

### Verify the download

I chose to use the macOS built-in verification for the downloads. Even though download quarantine does usually not take effect with scripted downloads and installations, we _can_ use Gatekeeper's verification with the `spctl` command.

All downloads have to be signed _and_ notarized by a valid Apple Developer ID. Signature and notarization will verify the origin and that the archive has not been tampered with in transfer. In addition, the script compares the developer team ID (the ten-digit code associated with the Apple Developer account) with the _expected_ team ID, so that software signed with a different, malicious Apple ID will not be installed.

### Simple interface

When used to install software, Installomator has a single argument: the label or name of the software to be installed.

```
./Installomator.sh firefox
```

There is a debug mode and one other setting that can be controlled with variables in the code. This simplifies the actual use of the script from within a management system.

### Extensible

As of this writing, Installomator knows how to download and install more than 50 different applications. You can add more by adding a block to the _long_ `case` statement starting on line 160. Some of them are more elaborate, but most of them just need this information:

```
googlechrome)
    name="Google Chrome"
    type="dmg"
    downloadURL="https://dl.google.com/chrome/mac/stable/GGRO/googlechrome.dmg"
    expectedTeamID="EQHXZ8M8AV"
    ;;
```

When you know how to extract these pieces of information from the application and/or download, then you can add an application to Installomator.

### Not specific to a management system

I wrote this script mainly for use with Jamf Pro, because that is what we use. For testing, you can run the script interactively from the command line. However, I have tried to keep anything that is specific to Jamf optional, or so flexible that it will work anywhere. Even if it does not work with your management system 'out of the box,' the adaptations should be straightforward.

### No dependencies

The script started out as a pure `sh` script, and when I needed arrays I 'switched' to `zsh`, because that is what [we can rely on being in macOS for the foreseeable future](https://scriptingosx.com/zsh). There are quite a few places where using python would have been easier and safer, but with the python 2 run-time being deprecated, that would have added a requirement for a Python 3 run-time to be installed. XML and JSON parsing would have been better with a tool like [scout](https://github.com/ABridoux/scout) or [jq](https://stedolan.github.io/jq/), but those would again require additional installations on the client before the script can run.

Keeping the script as a `zsh` allows you to paste it into your management system's interface (and disable the DEBUG mode) and use it without requiring any other installations.

## How to use

### Interactively in the command line

The script will require one argument.

The argument can be `version` or `longversion` which will print the script's version.

```
> ./Installomator.sh version
0.1
> ./Installomator.sh longversion
Installomater: version 0.1 (20200506)
```

Other than the version arguments, the argument can be any of the labels listed in the Labels.txt file. Each of the labels will download and install the latest version of the application, or suite of applications. Since the script will have to run the `installer` command or copy the application to the `/Applications` folder, it will have to be run as root.

```
> sudo ./Installomator.sh desktoppr
```

(Since Jamf Pro always provides the mount point, computer name, and user name as the first three arguments for policy scripts, the script will use argument `$4` when there are more than three arguments.)

### Debug mode

There is a variable named `DEBUG` which is set in line 21 of the script. When `DEBUG` is set to `1` (default) no actions that would actually modify the current system are taken. This is useful for testing most of the actions in the script, but obviously not all of them.

Also when the `DEBUG` variable is `1`, downloaded archives and extracted files will be written to the script's directory, rather than a temporary directory, which can make debugging easier.

_Always remember_ to change the `DEBUG` variable to `0` when deploying.

### Use Installomator with Jamf Pro

In Jamf Pro, create a new 'Script' and paste the contents of `Installomator.sh` into the 'Script Contents' area. Under 'Options' you can change the parameter label for argument 4 to 'Application Label.'

Remember to set `DEBUG` to `0`.

Then you can use the Installomator script in a policy and choose the application to install by setting the label for argument 4.

## What it does

When it runs with a known label, the script will perform the following:

- when the application is running, prompt the user to quit or cancel
- download the latest version from the vendor
- dmg or zip archives:
    - extract the application and copy it to /Applications
    - change the owner of the application to the current user
- pkg files:
    - when necessary, extract the pkg from the enclosing archive
    - install the pkg with the `installer` tool
- clean up the downloaded files
- notify the user

## Configuring the script

As of now there are two settings that are meant to configured when deploying the script.

### Debug mode

The first is the `DEBUG` variable. When this is set to `1` the script will _not_ perform any changes to the current system. In other words, no application will be copied to the target directory and no `installer` command be performed.

In addition, files will be downloaded and extracted to the Installomator project folder instead of a temporary directory and _not_ deleted when the script exits. Also archives will _not_ be re-downloaded when they already exist in the project folder. The repository's `.gitignore` file is set up to ignore the archive file extensions.

Debug mode is useful to test the download and verification process without having to re-download and re-install an application or package on your system.

### Blocking Process actions

The `BLOCKING_PROCESS_ACTION` variable controls the behavior of the script when it finds a blocking process running.

There are five options:

- `ignore`:       continue even when blocking processes are found
- `silent_fail`:  exit script without prompt or installation
- `prompt_user`:  show a user dialog for each blocking process found abort after three attempts to quit
- `prompt_user_then_kill`: show a user dialog for each blocking process found, attempt to quit two times, kill the process finally
- `kill`:         kill process without prompting or giving the user a chance to save

The default is `prompt_user`.

### Notification

The `NOTIFY` variable controls the notifications shown to the user. As of now, there are two options: `success` (default) and `silent`.

- `success`:   notify the user after a successful install
- `silent`:    no notifications

### Adding applications/label blocks

#### Required Variables

The script requires four pieces of information to download and install an application:

```
spotify)
    name="Spotify"
    type="dmg"
    downloadURL="https://download.scdn.co/Spotify.dmg"
    expectedTeamID="2FNC3A47ZF"
    ;;
```

The four required variables are

- `name`:
The display name of the installed application without the `.app` extensions.

- `type`:
The type of installation. Possible values:
     - `dmg`: application in disk image file (drag'n drop installation)
     - `pkg`: flat pkg download
     - `zip`: application in zip archive (`zip` or `tbz` extension)
     - `pkgInDmg`: a pkg file inside a disk image
     - `pkgInZip`: a pkg file inside a zip

- `downloadURL`:
The URL from which to download the archive.
The URL can be generated by a series of commands, for example when you need to parse an xml file for the latest URL. (See `bbedit`, `desktoppr`, or `omnigraffle` for examples.)

- `expectedTeamID`:
The 10-character Developer Team ID with which the application or pkg is signed and notarized.

  Obtain the team ID by running:

   - Applications (in dmgs or zips)
     `spctl -a -vv /Applications/BBEdit.app`

   - Installation Packages (pkg)
     `spctl -a -vv -t install ~/Downloads/desktoppr-0.2.pkg`

#### Optional Variables

Depending on the application or pkg there are a few more variables you can or need to set. Many of these are derived from the required variables, but may need to be set manually if those derived values do not work.

- `archiveName`: (optional)
  The name of the downloaded file.
  When not given the `archiveName` is set to `$name.$type`

- `appName`: (optional)
  File name of the app bundle in the dmg to verify and copy (include the `.app`).
  When not given, the `appName` is set to `$name.app`.

- `targetDir`: (optional)
  dmg or zip:
    Applications will be copied to this directory.
    Default value is '`/Applications`' for dmg and zip installations.
  pkg:
    `targetDir` is used as the install-location. Default is '`/`'.

- `blockingProcesses`: (optional)
  Array of process names that will block the installation or update.
  If no `blockingProcesses` array is given the default will be:
    `blockingProcesses=( $name )`
  When a package contains multiple applications, _all_ should be listed, e.g:
    `blockingProcesses=( "Keynote" "Pages" "Numbers" )`
  When a workflow has no blocking processes, use
    `blockingProcesses=( NONE )`

- `pkgName`: (optional, only used for `dmgInPkg` and `dmgInZip`)
  File name of the pkg file _inside_ the dmg or zip.
  When not given the pkgName is set to `$name.pkg`.

- `updateTool`, `updateToolArguments`:
  When Installomator detects an existing installation of the application,
  and the `updateTool` variable is set then
     `$updateTool $updateArguments`
  Will be run instead of of downloading and installing a complete new version.
  Use this when the `updateTool` does differential and optimized downloads.
  e.g. `msupdate` (see microsoft installations)

- `updateToolRunAsCurrentUser`:
  When this variable is set (any value), `$updateTool` will be run as the current user. Default is unset and

### Configuration from Arguments

You can provide a configuration variable, such as `DEBUG` or `NOTIFY` as an argument in the form `VAR=value`. For example:

```
./Installomator.sh desktoppr DEBUG=0 NOTIFY=silent
```

Providing variables this way will override any variables set in the script.

You can even provide _all_ the variables necessary for download and installation. Of course, without a label the argument parsing will fail, so I created a special label `valuesfromarguments` which only checks if the four required values are present:

```
./Installomator.sh name=desktoppr type=pkg downloadURL=https://github.com/scriptingosx/desktoppr/releases/download/v0.3/desktoppr-0.3.pkg expectedTeamID=JME5BW3F3R valuesfromarguments
```

The order of the variables and label is not relevant. But, when you provide more than one label, all but the _last_ label will be ignored.

Providing all the variables this way might be useful for certain downloads that have a customized URL for each vendor/customer (like customized TeamView or Watchman Monitoring) or are local downloads.

## Frequently Asked Questions

### What if the latest version of the app is already installed?

Short answer: Installomator will re-download and re-install the latest over the existing installation.

Longer answer:

Installomator will try to find a currently installed app to log the version. When Installomator finds an existing app (any version) and the `updateTool` variable is set, then Installomator will _not_ download and install, but run the `updateTool` instead.

However, there is no simple generic method to actually determine the latest version of an application or installer.

We deploy Installomator usually for user initiated installations from Self Service, so re-installs don't really 'hurt' and may be a useful troubleshooting step.

When you want to have automated installations, you can use smart groups based on the app version to limit excessive re-installations.

### Why don't you just use `autopkg install`?

Short answer: `autopkg` is not designed or suited for this kind of workflow

Long answer:

The motivation to not re-invent the wheel and use and existing tool is understandable. However, `autopkg` was not designed with this use case in mind and has a few significant downsides.

First, you would have to deploy and manage autopkg on all the clients. But to do its work, `autopkg` requires recipes. So, you have to install, and update the recipe repos on the client, as well. For security reasons, you _really_ should only run trusted recipes, so you need to install and update your personal recipe overrides as well.

The recipes you use are probably spread across multiple community provided recipe-repos, so we have `autopkg` itself, several recipe-repos, and your overrides that we need to manage, each of which may need to be updated at any time.

The community recipe-repos contain several recipes for different applications. When you add a recipe-repo for an app you want, you will also install all the other recipes from that repo.

The `autopkg install` does _not_ require root or even administrative privileges. _Any_ user (even standard users) on the system can now install any of the random recipes that came with the community repos.

To prevent users installing random apps from the community repos, you can curate your own recipe-repo from the community repos and push that to the clients. At this point, you are managing autopkg, your curated repo, your recipe overrides on the clients and handling the additional work of curating and updating your recipe-repo and the overrides.

And then consider what your CISO department (if you have one) would say about the `autopkgserver` and `autopkginstalld` daemons running on all the clients...

At this point it would be easier to use AutoPkg the way it was intended: on a single admin Mac, and let it upload the pkgs to your management system, which deploys them. Each tool is doing what it is designed for.

Please don't misunderstand this as me saying that AutoPkg is a bad or poorly designed tool. AutoPkg is amazing, powerful, and useful. The [Scripting OS X recipe-repo](https://github.com/autopkg/scriptingosx-recipes) is one of the older repos. AutoPkg is valuable tool to help admins with many apps that cannot be automated with tools like Installomator, and with deployment strategies that require more control.

But it is not suited as a client install automation tool.

### Why don't you just use brew?

Read the explanation for `autopkg`, pretty much the same applies for `brew`, i.e. while it is useful on a single Mac, it is a un-manageable mess when you think about deploying and managing on a fleet of computers.

repository: matinsa/Installomator

branch_name: refs/heads/dev

repo_name: matinsa/Installomator


text:
A ==*Project Object Model*== or *POM* is the fundamental unit of work in Maven.
## test

## [ **Namespace related attributes in XML and XML Schema (XSD)**](https://stackoverflow.com/questions/34202967/xmlns-xmlnsxsi-xsischemalocation-and-targetnamespace/34204799)

- [**`xmlns`**](http://www.w3.org/TR/xml-names/#ns-decl) is part of the [W3C Namespaces in XML Recommendation](http://www.w3.org/TR/xml-names/):

  > The prefix **xmlns** is used only to declare namespace bindings and is by definition bound to the namespace name http://www.w3.org/2000/xmlns/.

  **In your example**, it declares that `http://maven.apache.org/POM/4.0.0` is the default namespace for the elements in your Maven project.

- [**`xmlns:xsi`**](http://www.w3.org/TR/xmlschema-1/#Instance_Document_Constructions) declares a standard namespace prefix (`xsi`) for a core namespace used in XSD: `http://www.w3.org/2001/XMLSchema-instance`

  > *XML Schema: Structures* also defines several attributes for direct use in any XML documents. These attributes are in a different namespace, which has the namespace name http://www.w3.org/2001/XMLSchema-instance. For brevity, the text and examples in this specification use the prefix xsi: to stand for this latter namespace; in practice, any prefix can be used.

  **In your example**, it declares the conventional binding of the `xsi` namespace prefix to `http://www.w3.org/2001/XMLSchema-instance`, which properly sets up the use of the following attributes:

  - [**`xsi:type`**](http://www.w3.org/TR/xmlschema-1/#xsi_type) allows an XML instance to associate element type information directly rather than through an XSD. See [*How to restrict the value of an XML element using xsi:type in XSD?*](https://stackoverflow.com/questions/33808790/how-to-restrict-the-value-of-an-xml-element-using-xsitype-in-xsd)

    **In your example**, `xsi:type` is not used; included here for completeness regarding `xsi`.

  - [**`xsi:nil`**](http://www.w3.org/TR/xmlschema-1/#xsi_nil) allows an empty element to be considered to be valid when the XSD might not otherwise have allowed it.

    **In your example**, `xsi:nil` is not used; included here for completeness regarding `xsi`.

  - [**`xsi:schemaLocation`** and **`xsi:noNamespaceSchemaLocation`**](http://www.w3.org/TR/xmlschema-1/#schema-loc) provide hints to the XML processor as to how to associate an XSD with an XML document. Use `xsi:schemaLocation` when there is a namespace; use `xsi:noNamespaceSchemaLocation` when there is no namespace.

    **In your example**, there is a namespace, so you properly use `xsi:schemaLocation`, whose values are space-separated **pairs** of ***namespace\*** and ***XSD-location-URI\***. Your example uses the namespace, `http://maven.apache.org/POM/4.0.0`, and namespaces are lexical naming constructs that ***[need not be retrivable](https://stackoverflow.com/a/27614076/290085)\***. Your example also uses the XSD-location-URI, `http://maven.apache.org/xsd/maven-4.0.0.xsd`, which is retrivable as it should be.

    If your example did not use a namespace, you would use `xsi:noNamespaceSchemaLocation`, whose value is a single ***XSD-location-URI\*** that hints to the location of the intended XSD and which should be retrievable.

- [**`targetNamespace`**](https://www.w3.org/TR/2001/REC-xmlschema-1-20010502/#a-target_namespace) is an attribute on the `xs:schema` root element of an XSD which specifies the namespace of the root element of the XML document instances the XSD is intended to govern. It must match the default or explicit namespace of those XML documents' root elements.

## [What does “xmlns” in XML mean?](https://stackoverflow.com/questions/1181888/what-does-xmlns-in-xml-mean)

It means [XML namespace](http://en.wikipedia.org/wiki/XML_namespace).

Basically, every element (or attribute) in [XML](http://en.wikipedia.org/wiki/XML) belongs to a namespace, a way of "qualifying" the name of the element.

Imagine you and I both invent our own XML. You invent XML to describe people, I invent mine to describe cities. Both of us include an element called `name`. Yours refers to the person’s name, and mine to the city name—OK, it’s a little bit contrived.

```xml
<person>
    <name>Rob</name>
    <age>37</age>
    <homecity>
        <name>London</name>
        <lat>123.000</lat>
        <long>0.00</long>
    </homecity>
</person>
```

If our two XMLs were combined into a single document, how would we tell the two names apart? As you can see above, there are two `name` elements, but they both have different meanings.

The answer is that you and I would both assign a namespace to our XML, which we would make unique:

```xml
<personxml:person xmlns:personxml="http://www.your.example.com/xml/person"
                  xmlns:cityxml="http://www.my.example.com/xml/cities">
    <personxml:name>Rob</personxml:name>
    <personxml:age>37</personxml:age>
    <cityxml:homecity>
        <cityxml:name>London</cityxml:name>
        <cityxml:lat>123.000</cityxml:lat>
        <cityxml:long>0.00</cityxml:long>
    </cityxml:homecity>
</personxml:person>
```

Now we’ve fully qualified our XML, there is no ambiguity as to what each `name` element means. All of the tags that start with `personxml:` are tags belonging to your XML, all the ones that start with `cityxml:` are mine.

There are a few points to note:

- If you exclude any namespace declarations, things are considered to be in the default namespace.

- If you declare a namespace without the identifier, that is, `xmlns="http://somenamespace"`, rather than `xmlns:rob="somenamespace"`, it specifies the default namespace for the document.

- The actual namespace itself, often a [IRI](http://en.wikipedia.org/wiki/Internationalized_Resource_Identifier), is of no real consequence. It should be unique, so people tend to choose a IRI/URI that they own, but it has no greater meaning than that. Sometimes people will place the schema (definition) for the XML at the specified IRI, but that is a convention of some people only.

- The prefix is of no consequence either. The only thing that matters is what namespace the prefix is defined as. Several tags beginning with different prefixes, all of which map to the same namespace are considered to be the same.

  For instance, if the prefixes `personxml` and `mycityxml` both mapped to the same namespace (as in the snippet below), then it wouldn't matter if you prefixed a given element with `personxml` or `mycityxml`, they'd both be treated as the same thing by an XML parser. The point is that an XML parser doesn't care what you've chosen as the prefix, only the namespace it maps too. The prefix is just an *indirection* pointing to the namespace.

  ```xml
  <personxml:person
       xmlns:personxml="http://example.com/same/url"
       xmlns:mycityxml="http://example.com/same/url" />
  ```

- Attributes can be qualified but are generally not. They also do *not* inherit their namespace from the element they are on, as opposed to elements (see below).

Also, element namespaces are inherited from the parent element. In other words I could equally have written the above XML as

```xml
<person xmlns="http://www.your.example.com/xml/person">
    <name>Rob</name>
    <age>37</age>
    <homecity xmlns="http://www.my.example.com/xml/cities">
        <name>London</name>
        <lat>123.000</lat>
        <long>0.00</long>
    </homecity>
</person>
```

## 配置IDEA 和 Maven

一定要用自己的 Maven ，不要用系统自带的 Maven。

==不同版本的 IDEA 和 Maven 根本不能一起搭配工作。== 比如，

```
IntelliJ IDEA 2019.1.3 (Ultimate Edition)
Build #IU-191.7479.19, built on May 28, 2019
Licensed to 超级 程序员
You have a perpetual fallback license for this version
Subscription is active until January 4, 2021
JRE: 1.8.0_202-release-1483-b58 x86_64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
macOS 10.13.6
```

该版 IDEA 和 Maven 3.6.3 在一起根本用不了，配置完毕后，项目完全不会自动加载。

建议用 Maven 3.3.9。

### 配置 Maven

解压好的 Maven bin包就放在 `/Library` 下如，`/Library/apache-maven-3.3.9` 。

在该文件夹下自建一个`repo` 文件夹用作本地仓库。

修改 `conf`  目录下的 `settings.xml` :

```
<?xml version="1.0" encoding="UTF-8"?>

<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements.  See the NOTICE file
distributed with this work for additional information
regarding copyright ownership.  The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License.  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied.  See the License for the
specific language governing permissions and limitations
under the License.
-->

<!--
 | This is the configuration file for Maven. It can be specified at two levels:
 |
 |  1. User Level. This settings.xml file provides configuration for a single user,
 |                 and is normally provided in ${user.home}/.m2/settings.xml.
 |
 |                 NOTE: This location can be overridden with the CLI option:
 |
 |                 -s /path/to/user/settings.xml
 |
 |  2. Global Level. This settings.xml file provides configuration for all Maven
 |                 users on a machine (assuming they're all using the same Maven
 |                 installation). It's normally provided in
 |                 ${maven.home}/conf/settings.xml.
 |
 |                 NOTE: This location can be overridden with the CLI option:
 |
 |                 -gs /path/to/global/settings.xml
 |
 | The sections in this sample file are intended to give you a running start at
 | getting the most out of your Maven installation. Where appropriate, the default
 | values (values used when the setting is not specified) are provided.
 |
 |-->
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
  <!-- localRepository
   | The path to the local repository maven will use to store artifacts.
   |
   | Default: ${user.home}/.m2/repository
  <localRepository>/path/to/local/repo</localRepository>
  -->
  <localRepository>/Library/apache-maven-3.3.9/repo</localRepository>
  <!-- interactiveMode
   | This will determine whether maven prompts you when it needs input. If set to false,
   | maven will use a sensible default value, perhaps based on some other setting, for
   | the parameter in question.
   |
   | Default: true
  <interactiveMode>true</interactiveMode>
  -->

  <!-- offline
   | Determines whether maven should attempt to connect to the network when executing a build.
   | This will have an effect on artifact downloads, artifact deployment, and others.
   |
   | Default: false
  <offline>false</offline>
  -->

  <!-- pluginGroups
   | This is a list of additional group identifiers that will be searched when resolving plugins by their prefix, i.e.
   | when invoking a command line like "mvn prefix:goal". Maven will automatically add the group identifiers
   | "org.apache.maven.plugins" and "org.codehaus.mojo" if these are not already contained in the list.
   |-->
  <pluginGroups>
    <!-- pluginGroup
     | Specifies a further group identifier to use for plugin lookup.
    <pluginGroup>com.your.plugins</pluginGroup>
    -->
  </pluginGroups>

  <!-- proxies
   | This is a list of proxies which can be used on this machine to connect to the network.
   | Unless otherwise specified (by system property or command-line switch), the first proxy
   | specification in this list marked as active will be used.
   |-->
  <proxies>
    <!-- proxy
     | Specification for one proxy, to be used in connecting to the network.
     |
    <proxy>
      <id>optional</id>
      <active>true</active>
      <protocol>http</protocol>
      <username>proxyuser</username>
      <password>proxypass</password>
      <host>proxy.host.net</host>
      <port>80</port>
      <nonProxyHosts>local.net|some.host.com</nonProxyHosts>
    </proxy>
    -->
  </proxies>

  <!-- servers
   | This is a list of authentication profiles, keyed by the server-id used within the system.
   | Authentication profiles can be used whenever maven must make a connection to a remote server.
   |-->
  <servers>
    <!-- server
     | Specifies the authentication information to use when connecting to a particular server, identified by
     | a unique name within the system (referred to by the 'id' attribute below).
     |
     | NOTE: You should either specify username/password OR privateKey/passphrase, since these pairings are
     |       used together.
     |
    <server>
      <id>deploymentRepo</id>
      <username>repouser</username>
      <password>repopwd</password>
    </server>
    -->

    <!-- Another sample, using keys to authenticate.
    <server>
      <id>siteServer</id>
      <privateKey>/path/to/private/key</privateKey>
      <passphrase>optional; leave empty if not used.</passphrase>
    </server>
    -->
  </servers>

  <!-- mirrors
   | This is a list of mirrors to be used in downloading artifacts from remote repositories.
   |
   | It works like this: a POM may declare a repository to use in resolving certain artifacts.
   | However, this repository may have problems with heavy traffic at times, so people have mirrored
   | it to several places.
   |
   | That repository definition will have a unique id, so we can create a mirror reference for that
   | repository, to be used as an alternate download site. The mirror site will be the preferred
   | server for that repository.
   |-->
  <mirrors>
    <!-- mirror
     | Specifies a repository mirror site to use instead of a given repository. The repository that
     | this mirror serves has an ID that matches the mirrorOf element of this mirror. IDs are used
     | for inheritance and direct lookup purposes, and must be unique across the set of mirrors.
     |
    <mirror>
      <id>mirrorId</id>
      <mirrorOf>repositoryId</mirrorOf>
      <name>Human Readable Name for this Mirror.</name>
      <url>http://my.repository.com/repo/path</url>
    </mirror>
     -->
  <mirror>
    <id>alimaven</id>
    <name>aliyun maven</name>
    <url>http://maven.aliyun.com/nexus/content/groups/public/</url>
    <mirrorOf>central</mirrorOf>
  </mirror>
  <mirror>
    <id>uk</id>
    <mirrorOf>central</mirrorOf>
    <name>Human Readable Name for this Mirror.</name>
    <url>http://uk.maven.org/maven2/</url>
  </mirror>
  <mirror>
    <id>CN</id>
    <name>OSChina Central</name>
    <url>http://maven.oschina.net/content/groups/public/</url>
    <mirrorOf>central</mirrorOf>
  </mirror>

  <mirror>
    <id>nexus</id>
    <name>internal nexus repository</name>
    <!-- <url>http://<IP_ADDRESS>:8081/nexus/content/groups/public/</url>-->
    <url>http://repo.maven.apache.org/maven2</url>
    <mirrorOf>central</mirrorOf>
  </mirror>
  </mirrors>

  <!-- profiles
   | This is a list of profiles which can be activated in a variety of ways, and which can modify
   | the build process. Profiles provided in the settings.xml are intended to provide local machine-
   | specific paths and repository locations which allow the build to work in the local environment.
   |
   | For example, if you have an integration testing plugin - like cactus - that needs to know where
   | your Tomcat instance is installed, you can provide a variable here such that the variable is
   | dereferenced during the build process to configure the cactus plugin.
   |
   | As noted above, profiles can be activated in a variety of ways. One way - the activeProfiles
   | section of this document (settings.xml) - will be discussed later. Another way essentially
   | relies on the detection of a system property, either matching a particular value for the property,
   | or merely testing its existence. Profiles can also be activated by JDK version prefix, where a
   | value of '1.4' might activate a profile when the build is executed on a JDK version of '1.4.2_07'.
   | Finally, the list of active profiles can be specified directly from the command line.
   |
   | NOTE: For profiles defined in the settings.xml, you are restricted to specifying only artifact
   |       repositories, plugin repositories, and free-form properties to be used as configuration
   |       variables for plugins in the POM.
   |
   |-->
  <profiles>
    <!-- profile
     | Specifies a set of introductions to the build process, to be activated using one or more of the
     | mechanisms described above. For inheritance purposes, and to activate profiles via <activatedProfiles/>
     | or the command line, profiles have to have an ID that is unique.
     |
     | An encouraged best practice for profile identification is to use a consistent naming convention
     | for profiles, such as 'env-dev', 'env-test', 'env-production', 'user-jdcasey', 'user-brett', etc.
     | This will make it more intuitive to understand what the set of introduced profiles is attempting
     | to accomplish, particularly when you only have a list of profile id's for debug.
     |
     | This profile example uses the JDK version to trigger activation, and provides a JDK-specific repo.
    <profile>
      <id>jdk-1.4</id>

      <activation>
        <jdk>1.4</jdk>
      </activation>

      <repositories>
        <repository>
          <id>jdk14</id>
          <name>Repository for JDK 1.4 builds</name>
          <url>http://www.myhost.com/maven/jdk14</url>
          <layout>default</layout>
          <snapshotPolicy>always</snapshotPolicy>
        </repository>
      </repositories>
    </profile>
    -->

    <!--
     | Here is another profile, activated by the system property 'target-env' with a value of 'dev',
     | which provides a specific path to the Tomcat instance. To use this, your plugin configuration
     | might hypothetically look like:
     |
     | ...
     | <plugin>
     |   <groupId>org.myco.myplugins</groupId>
     |   <artifactId>myplugin</artifactId>
     |
     |   <configuration>
     |     <tomcatLocation>${tomcatPath}</tomcatLocation>
     |   </configuration>
     | </plugin>
     | ...
     |
     | NOTE: If you just wanted to inject this configuration whenever someone set 'target-env' to
     |       anything, you could just leave off the <value/> inside the activation-property.
     |
    <profile>
      <id>env-dev</id>

      <activation>
        <property>
          <name>target-env</name>
          <value>dev</value>
        </property>
      </activation>

      <properties>
        <tomcatPath>/path/to/tomcat/instance</tomcatPath>
      </properties>
    </profile>
    -->
  </profiles>

  <!-- activeProfiles
   | List of profiles that are active for all builds.
   |
  <activeProfiles>
    <activeProfile>alwaysActiveProfile</activeProfile>
    <activeProfile>anotherAlwaysActiveProfile</activeProfile>
  </activeProfiles>
  -->
</settings>

```

这里主要改了两个地方，本地仓库目录 `<localRepository>/Library/apache-maven-3.3.9/repo</localRepository>`和 镜像源。

Mac 下修改 `~/.bahs_profile` ，

```
##Maven
export M2_HOME=/Library/apache-maven-3.3.9
export PATH=$PATH:$M2_HOME/bin
```

完了之后  `source ~/.bahs_profile`

检查 Maven 是否 安装成功 `mvn -v`

### 配置 IDEA

偏好设置里更改 Maven 配置，以下三个选项都要改成自己的 Maven 仓库配置。

<EMAIL_ADDRESS>

## tomcat

通过配置 tomcat 运行 web 应用

<EMAIL_ADDRESS>
### Exploded deployments vs WAR packages

<EMAIL_ADDRESS>
There are two ways to deploy Java web applications.

The first way is to deploy a WAR file. A WAR file is just a ZIP archive with a directory structure that is recognised by Java application servers like Tomcat.

The second way is to deploy all the individual files that make up a web application. This is called an exploded deployment, or an ==exploded WAR==. This kind of deployment can be very useful during development, as files like HTML pages and CSS files can be edited while the application is deployed and reloaded on the fly.

### The `webapps` Directory

The `webapps` directory is ==where deployed applications reside in Tomcat.==

The `webapps` directory is the default deployment location, but this can be configured with the `appBase` attribute on the `Host` element.

If Tomcat is set to autodeploy applications ==(and it is set to do this by default)== then ==any WAR file or exploded deployment copied into the `webapps` folder will be deployed automatically while Tomcat is running.==

The autodeployment of applications can be disabled by setting the `autoDeploy` attribute on the `` element to `false`. In this case, applications will de deployed on startup.

In turn, the deployment of applications on startup can be disabled by setting the `deployOnStartup` attribute on the `` element to `false`.

If both `autoDeploy` and `deployOnStartup` are false, you can deploy applications by manually adding a `` element inside the `` element in the `conf/server.xml` file. See the section "Defining the context in the `server.xml` file" for an example.

- https://octopus.com/blog/defining-tomcat-context-paths

repository: utopianist/HelloMaven

repo_name: utopianist/HelloMaven


text:
# L. 縮寫字

本頁列出在本翻譯文件及官方手冊常見的縮寫字。

ANSI

[American National Standards Institute](https://en.wikipedia.org/wiki/American\_National\_Standards\_Institute)

API

[Application Programming Interface](https://en.wikipedia.org/wiki/API)

ASCII

[American Standard Code for Information Interchange](https://en.wikipedia.org/wiki/Ascii)

BKI

[Backend Interface](https://www.postgresql.org/docs/13/bki.html)

CA

[Certificate Authority](https://en.wikipedia.org/wiki/Certificate\_authority)

CIDR

[Classless Inter-Domain Routing](https://en.wikipedia.org/wiki/Classless\_Inter-Domain\_Routing)

CPAN

[Comprehensive Perl Archive Network](https://www.cpan.org)

CRL

[Certificate Revocation List](https://en.wikipedia.org/wiki/Certificate\_revocation\_list)

CSV

[Comma Separated Values](https://en.wikipedia.org/wiki/Comma-separated\_values)

CTE

[Common Table Expression](https://www.postgresql.org/docs/13/queries-with.html)

CVE

[Common Vulnerabilities and Exposures](https://cve.mitre.org)

DBA

[Database Administrator](https://en.wikipedia.org/wiki/Database\_administrator)

DBI

[Database Interface (Perl)](https://dbi.perl.org)

DBMS

[Database Management System](https://en.wikipedia.org/wiki/Dbms)

DDL

[Data Definition Language](https://en.wikipedia.org/wiki/Data\_Definition\_Language), SQL commands such as `CREATE TABLE`, `ALTER USER`

DML

[Data Manipulation Language](https://en.wikipedia.org/wiki/Data\_Manipulation\_Language), SQL commands such as `INSERT`, `UPDATE`, `DELETE`

DST

[Daylight Saving Time](https://en.wikipedia.org/wiki/Daylight\_saving\_time)

ECPG

[Embedded C for PostgreSQL](https://www.postgresql.org/docs/13/ecpg.html)

ESQL

[Embedded SQL](https://en.wikipedia.org/wiki/Embedded\_SQL)

FAQ

[Frequently Asked Questions](https://en.wikipedia.org/wiki/FAQ)

FSM

[Free Space Map](https://www.postgresql.org/docs/13/storage-fsm.html)

GEQO

[Genetic Query Optimizer](https://www.postgresql.org/docs/13/geqo.html)

GIN

[Generalized Inverted Index](https://www.postgresql.org/docs/13/gin.html)

GiST

[Generalized Search Tree](https://www.postgresql.org/docs/13/gist.html)

Git

[Git](https://en.wikipedia.org/wiki/Git\_\(software\))

GMT

[Greenwich Mean Time](https://en.wikipedia.org/wiki/GMT)

GSSAPI

[Generic Security Services Application Programming Interface](https://en.wikipedia.org/wiki/Generic\_Security\_Services\_Application\_Program\_Interface)

GUC

[Grand Unified Configuration](https://www.postgresql.org/docs/13/config-setting.html), the PostgreSQL subsystem that handles server configuration

HBA

[Host-Based Authentication](https://www.postgresql.org/docs/13/auth-pg-hba-conf.html)

HOT

[Heap-Only Tuples](https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/access/heap/README.HOT;hb=HEAD)

IEC

[International Electrotechnical Commission](https://en.wikipedia.org/wiki/International\_Electrotechnical\_Commission)

IEEE

[Institute of Electrical and Electronics Engineers](https://standards.ieee.org)

IPC

[Inter-Process Communication](https://en.wikipedia.org/wiki/Inter-process\_communication)

ISO

[International Organization for Standardization](https://www.iso.org/home.html)

ISSN

[International Standard Serial Number](https://en.wikipedia.org/wiki/Issn)

JDBC

[Java Database Connectivity](https://en.wikipedia.org/wiki/Java\_Database\_Connectivity)

JIT

[Just-in-Time compilation](https://en.wikipedia.org/wiki/Just-in-time\_compilation)

JSON

[JavaScript Object Notation](https://www.json.org)

LDAP

[Lightweight Directory Access Protocol](https://en.wikipedia.org/wiki/Lightweight\_Directory\_Access\_Protocol)

LSN

Log Sequence Number, see [`pg_lsn`](https://www.postgresql.org/docs/13/datatype-pg-lsn.html) and [WAL Internals](https://www.postgresql.org/docs/13/wal-internals.html).

MSVC

[Microsoft Visual C](https://en.wikipedia.org/wiki/Visual\_C++)

MVCC

[Multi-Version Concurrency Control](https://www.postgresql.org/docs/13/mvcc.html)

NLS

[National Language Support](https://en.wikipedia.org/wiki/Internationalization\_and\_localization)

ODBC

[Open Database Connectivity](https://en.wikipedia.org/wiki/Open\_Database\_Connectivity)

OID

[Object Identifier](https://www.postgresql.org/docs/13/datatype-oid.html)

OLAP

[Online Analytical Processing](https://en.wikipedia.org/wiki/Olap)

OLTP

[Online Transaction Processing](https://en.wikipedia.org/wiki/OLTP)

ORDBMS

[Object-Relational Database Management System](https://en.wikipedia.org/wiki/ORDBMS)

PAM

[Pluggable Authentication Modules](https://en.wikipedia.org/wiki/Pluggable\_Authentication\_Modules)

PGSQL

[PostgreSQL](https://www.postgresql.org/docs/13/index.html)

PGXS

[PostgreSQL Extension System](https://www.postgresql.org/docs/13/extend-pgxs.html)

PID

[Process Identifier](https://en.wikipedia.org/wiki/Process\_identifier)

PITR

[Point-In-Time Recovery](https://www.postgresql.org/docs/13/continuous-archiving.html) (Continuous Archiving)

PL

[Procedural Languages (server-side)](https://www.postgresql.org/docs/13/server-programming.html)

POSIX

[Portable Operating System Interface](https://en.wikipedia.org/wiki/POSIX)

RDBMS

[Relational Database Management System](https://en.wikipedia.org/wiki/Relational\_database\_management\_system)

RFC

[Request For Comments](https://en.wikipedia.org/wiki/Request\_for\_Comments)

SGML

[Standard Generalized Markup Language](https://en.wikipedia.org/wiki/SGML)

SPI

[Server Programming Interface](https://www.postgresql.org/docs/13/spi.html)

SP-GiST

[Space-Partitioned Generalized Search Tree](https://www.postgresql.org/docs/13/spgist.html)

SQL

[Structured Query Language](https://en.wikipedia.org/wiki/SQL)

SRF

[Set-Returning Function](https://www.postgresql.org/docs/13/xfunc-c.html#XFUNC-C-RETURN-SET)

SSH

[Secure Shell](https://en.wikipedia.org/wiki/Secure\_Shell)

SSL

[Secure Sockets Layer](https://en.wikipedia.org/wiki/Secure\_Sockets\_Layer)

SSPI

[Security Support Provider Interface](https://msdn.microsoft.com/en-us/library/aa380493\(VS.85\).aspx)

SYSV

[Unix System V](https://en.wikipedia.org/wiki/System\_V)

TCP/IP

[Transmission Control Protocol (TCP) / Internet Protocol (IP)](https://en.wikipedia.org/wiki/Transmission\_Control\_Protocol)

TID

[Tuple Identifier](https://www.postgresql.org/docs/13/datatype-oid.html)

TOAST

[The Oversized-Attribute Storage Technique](https://www.postgresql.org/docs/13/storage-toast.html)

TPC

[Transaction Processing Performance Council](http://www.tpc.org)

URL

[Uniform Resource Locator](https://en.wikipedia.org/wiki/URL)

UTC

[Coordinated Universal Time](https://en.wikipedia.org/wiki/Coordinated\_Universal\_Time)

UTF

[Unicode Transformation Format](https://www.unicode.org)

UTF8

[Eight-Bit Unicode Transformation Format](https://en.wikipedia.org/wiki/Utf8)

UUID

[Universally Unique Identifier](https://www.postgresql.org/docs/13/datatype-uuid.html)

WAL

[Write-Ahead Log](https://www.postgresql.org/docs/13/wal.html)

XID

[Transaction Identifier](https://www.postgresql.org/docs/13/datatype-oid.html)

XML

[Extensible Markup Language](https://en.wikipedia.org/wiki/XML)

repository: pgsql-tw/gitbook-docs

gha_language: HTML

repo_name: pgsql-tw/gitbook-docs


text:
# Johns Hopkins Catalyst ( Blacklight Library Catalog )

Catalyst is Johns Hopkins Libraries Catalog. It is an extention of the Blacklight project
which uses Solr as a central index. Catalyst.

This project is managed by the Library Applications, with production support (monitoring, backup, recovery, security) provided by Library Operations.

Catlayst also contains important subprojects
- horizon-holding-info-servlet : runs in jetty(8.1.4) connects to Horizon DB : looks up borrow info
- traject ( uses traject_horizon ) connects to Horizon DB and manages the Solr index

## Infrastructure Requirements

- Expternal IP (for BorrowDirect to work)

## Production

### catalyst.library.jhu.edu

## Development
config/database.php

### Docker for Development

Dockerfile, docker-compose.yml and env-example file add the ability to run a local containers. Docker use is experimental at this stage.

**Edit the env-example variables they are only examples**

Copy and edit the env-example file
```
cp env-example .env
```
Build catalyst containe (users Dockerfile)
```
docker-compose build
```
The mysql container uses a local volume to persist data
create the directory
```
mkdir .data
```

Start the mysql container and populate it with data
```
docker-compose up mysql
```
Populate the database from the development database
Get user, database passwords from catalyst config/database , and see you .env file for LOCAL details
```
docker-compose exec bash
db$  mysqldump --single-transaction --lock-tables=false --add-drop-table -h mysql.mse.jhu.edu -u <CAT_USER> -p <CAT_DB> > catalyst_db.sql
```
dump takes about a minute
the restore takes a lot longer

See the following documentation on version management and deployments:
```
db$  mysql -u <LOCAL_USER> -p <LOCAL_DB> < catalyst_db.sql
```
To bring the server up (-d is for deamon mode, run in background))
```
docker-composer up
```
Terminal into containers
```
docker-composer exec catalyst bash
docker-composer exec db bash
```
Navigate to http://localhost:3000 to access catalyst

### Tips on using Docker
Run a query inside the container
```
docker-compose exec db mysql -u catalyst -p catalyst -e  "select * from users limit 2"
```

**You will have to also edit the config/database.yml with the local database settings we can move to using the dotdenv gem at a later date**

## Components
Our implementation of Blacklight includes a lot of customization. To identify where we override or extend the Blacklight Gem, start from these folders under https://github.com/jhu-sheridan-libraries/blacklight-rails/tree/master/app
- Controllers
- Views
- Models
- Helpers
Compare the contents of these folders to the corresponding folders in the Blacklight Project Version 5.15

### Web Service for Borrower and Holdings Info from Horizon.
https://github.com/jhu-sheridan-libraries/horizon-holding-info-servlet/blob/master/README.md

### Traject
Indexes catalog records from Horizon to build the Solr index
https://github.com/jhu-sheridan-libraries/catalyst-traject

### Course Reserves
There are two parts to this feature.
1. Loader is in its own project
https://github.com/jhu-sheridan-libraries/catalyst-pull-reserves

2. Display of reserves info is controlled in this project
https://github.com/jhu-sheridan-libraries/blacklight-rails/blob/master/app/controllers/reserves_controller.rb

### Virtual Shelf Browse
This feature is a gem located at
https://github.com/jrochkind/rails_stackview

### My Account
This is a home-grown feature that includes a user login and various other functionality
- Login
https://github.com/jhu-sheridan-libraries/blacklight-rails/blob/master/app/controllers/user_sessions_controller.rb
- Other functionality
https://github.com/jhu-sheridan-libraries/blacklight-rails/blob/master/app/controllers/users_controller.rb

Note: My Account uses Horizon HIP service to handle requests, renewals, holds, etc.
https://github.com/jhu-sheridan-libraries/blacklight-rails/blob/master/lib/hip_pilot.rb

## Tips

VPN and then SSH tunnel to  a solr instance to test:
Check the tunnel worked in another window

ssh -L 8983:localhost:8080<EMAIL_ADDRESS>curl http://localhost:8983/solr/#/catalyst_dev1

# Check difference with production, which is tagged
git diff production-2017-05-05-1357

# Tests with Cucumber

Prerequisites:

Install chromedriver. On mac:

```
brew install chromedriver
```

Note: if you see an error such as the following, it means that chromedriver needs to be udpated.
```
      unknown error: call function result missing 'value'
        (Session info: headless chrome=65.0.3325.162)
        (Driver info: chromedriver=2.32.498537 (cb2f855cbc7b82e20387eaf9a43f6b99b6105061),platform=Mac OS X 10.13.3 x86_64) (Selenium::WebDriver::Error::UnknownError)
```

Try the following:

```
chromedriver-update 2.36
```

## Test Catalyst

To test with localhost, just run

```
RAILS_ENV=test bundle exec cucumber
```

To test with a single scenario, use `featrures/file_name.feature:line_number`, for example

```
RAILS_ENV=test bundle exec cucumber features/basic_search.feature:9
```

To test a non-localhost server, add a `CAPYBARA_APP_HOST` environment variable

```
RAILS_ENV=test CAPYBARA_APP_HOST=https://catalyst-test.library.jhu.edu bundle exec cucumber
```

, or put it in `.env`:

```
CAPYBARA_APP_HOST=https://catalyst-test.library.jhu.edu
```

## Test services that Catalyst depends on

By default, the cucumber tests only test Catalyst features. External services not
included in the Rails app will not be tested.
To test these services, such as horizon servlet service and hip service, use the following:

```
RAILS_ENV=test bundle exec cucumber -p servcies
```

To test a certain HIP server, try

```
HIP_HOST=hip-test.library.jhu.edu RAILS_ENV=test bundle exec cucumber -p services
```

Note: the profiles are defined in `config/cucumber.yml`

# Disable Patron Account Features

During Horizon upgrade, we need to disable patron account features, while allowing patrons to use
other catalyst features.

In `config/initializers/disable_hip.rb`, set
```
JHConfig.params[:disable_hip] = true
```
, and in the same file, update the texts in
```
JHConfig.params[:disable_hip_message]
```

## Version management documentation:
https://jhulibraries.atlassian.net/wiki/spaces/CATALYST/pages/31555838/Git+Organization+and+Deployment+Flow

## Deployment documentation:
https://github.com/jhu-sheridan-libraries/catalyst-ansible

repository: ewlarson/catalyst-blacklight

repo_name: ewlarson/catalyst-blacklight