text:
# TurbulenceConvection.jl #

|||
|---------------------:|:----------------------------------------------|
| **Documentation**    | [![dev][docs-dev-img]][docs-dev-url]          |
| **DOI**              | [![DOI][zenodo-img]][zenodo-latest-url]       |
| **Docs Build**       | [![docs build][docs-bld-img]][docs-bld-url]   |
| **GHA CI**           | [![gha ci][gha-ci-img]][gha-ci-url]           |
| **Code Coverage**    | [![codecov][codecov-img]][codecov-url]        |
| **Bors enabled**     | [![bors][bors-img]][bors-url]                 |

[zenodo-img]: https://zenodo.org/badge/DOI/10.5281/zenodo.6392396.svg
[zenodo-latest-url]: https://doi.org/10.5281/zenodo.6392396

[docs-bld-img]: https://github.com/CliMA/TurbulenceConvection.jl/actions/workflows/docs.yml/badge.svg
[docs-bld-url]: https://github.com/CliMA/TurbulenceConvection.jl/actions/workflows/docs.yml

[docs-dev-img]: https://img.shields.io/badge/docs-dev-blue.svg
[docs-dev-url]: https://CliMA.github.io/TurbulenceConvection.jl/dev/

[gha-ci-img]: https://github.com/CliMA/TurbulenceConvection.jl/actions/workflows/ci.yml/badge.svg
[gha-ci-url]: https://github.com/CliMA/TurbulenceConvection.jl/actions/workflows/ci.yml

[codecov-img]: https://codecov.io/gh/CliMA/TurbulenceConvection.jl/branch/main/graph/badge.svg
[codecov-url]: https://codecov.io/gh/CliMA/TurbulenceConvection.jl

[bors-img]: https://bors.tech/images/badge_small.svg
[bors-url]: https://app.bors.tech/repositories/35146

TurbulenceConvection (Single Column Atmospheric Model in Julia) provides a framework for testing parameterizations of clouds and turbulence.
It is particularly designed to support eddy-diffusivity mass-flux modeling frameworks.

Information about the EDMF parameterization implemented in TurbulenceConvection can be found in:

Tan, Z., Kaul, C. M., Pressel, K. G., Cohen, Y., Schneider, T., and Teixeira, J. (2018)
**An extended eddy-diffusivity mass-flux scheme for unified representation of
subgrid-scale turbulence and convection.** *Journal of Advances in Modeling Earth Systems*. [doi](https://doi.org/10.1002/2017MS001162)

Cohen, Y., Lopez-Gomez, I., Jaruga, A., He, J., Kaul, C., and Schneider, T. (2020) **Unified entrainment and detrainment closures for extended eddy-diffusivity mass-flux schemes.** *Journal of Advances in Modeling Earth Systems*, 12, e2020MS002162. [doi](https://doi.org/10.1029/2020MS002162)

Lopez-Gomez, I., Cohen, Y., He, J., Jaruga, A., Schneider, T. (2020) **A generalized mixing length closure for eddy-diﬀusivity mass-flux schemes of turbulence and convection.** *Journal of Advances in Modeling Earth Systems*, 12, e2020MS002161. [doi](https://doi.org/10.1029/2020MS002161)

He, J., Cohen, Y., Lopez-Gomez, I., Jaruga, A., Schneider, T. (2021) **An improved perturbation pressure closure for eddy-diffusivity mass-flux schemes**. [preprint](https://doi.org/10.1002/essoar.10505084.2)

The code is written in Julia, and was translated from [SCAMPy](https://github.com/CliMA/SCAMPy) for the commit 496dad0c2438235684823511cacbf5761d6a237c.

Code Contributors (alphabetical):
    Yair Cohen (Caltech),
    Jia He (Caltech),
    Anna Jaruga (Caltech),
    Colleen Kaul (PNNL) --initial/primary developer of SCAMPy,
    Charles Kawczynski (Caltech),
    Ignacio Lopez-Gomez (Caltech),
    Kyle Pressel (PNNL),

Additional Acknowledgements:
    Tapio Schneider (Caltech),
    Joao Teixeira (JPL).

# User Installation #

Installation is easy:

```julia-repl
(@v1.6) pkg> add TurbulenceConvection
```

TurbulenceConvection.jl requires Julia 1.6 or higher. If you plan to develop TurbulenceConvection.jl, you may want to clone it instead:

```
git clone https://github.com/CliMA/TurbulenceConvection.jl
cd TurbulenceConvection.jl
julia --project=integration_tests
julia> ]
pkg> instantiate
```

# Running #

```
$ cd TurbulenceConvection.jl
```

TurbulenceConvection.jl can be run in the same way that SCAMPy used to, given one of the following cases:

 - Bomex
 - life_cycle_Tan2018
 - Soares
 - Rico
 - TRMM_LBA
 - ARM_SGP
 - GATE_III
 - DYCOMS_RF01
 - DYCOMS_RF02
 - GABLS

with, for example:

```
julia --project=integration_tests driver/generate_namelist.jl Soares
julia --project=integration_tests driver/main.jl Soares
```
or by calling our integraion test driver,
```
julia --project=integration_tests integration_tests/driver.jl --case Soares
```
or, interactively, with
```julia-repl
julia --project=integration_tests
julia> case_name = "Soares" # default is "Bomex"
julia> include(joinpath("integration_tests", "driver.jl"))
```

# Automated plotting  #

Upon running a particular experiment (described above), comparison plots (against [SCAMPy](https://github.com/CliMA/SCAMPy)) are automatically generated in, for example, `Output.Bomex.01/stats/comparison/`.

# Table of prognostic and diagnostic variables

 - Prognostic [✓]
 - Diagnostic [x]
 - NA [-]

| **Variable**  | **Grid-mean** | **Environment** | **Updrafts** |
| ------------- | ------------- | --------------- | ------------ |
| `u`           | [✓]           |  [x]            |  [x]         |
| `v`           | [✓]           |  [x]            |  [x]         |
| `w`           | [-]           |  [x]            |  [✓]         |
| `θ_liq_ice`   | [✓]           |  [x]            |  [✓]         |
| `q_tot`       | [✓]           |  [x]            |  [✓]         |
| `a`           | [-]           |  [x]            |  [✓]         |
| `tke`         | [x]           |  [✓]            |  [-]         |
| `θ′θ′`        | [x]           |  [✓]            |  [-]         |
| `q_tot′q_tot′`| [x]           |  [✓]            |  [-]         |
| `θ′q_tot′`    | [x]           |  [✓]            |  [-]         |

gha_language: Julia


text:
## Polygon/Box Modeling

**The easiest way to understand box modeling is to try it**. Once you’ve done it, then we can discuss exactly what’s going on, what works well and what doesn’t.

So now that you’ve done some box modeling, let’s quickly go over a few fundamentals. A mesh comprises a set of vertices (points in space) and a set of polygons (ordered lists of vertices). The simplest mesh (aside from a “nothing” mesh) is a triangle — three vertices and one polygon containing each of them.

A mesh can contain “bare” vertices — i.e. vertices not referenced by any polygon. These vertices will not be rendered.

A polygon (or “face”) can have 3 or more edges. A face ought to be planar (i.e. all the vertices should be coplanar) but need not be. When representing a face in the editor view or rendering it, Cheetah 3D will convert it to triangles internally, and three points are *always* coplanar.

The basic operations in polygon/box modeling are:

- **Changing Mode** (vertex mode, edge mode, face mode, or object mode). When you’re not in “object mode” you are working on a specific object, and it becomes your “universe”. E.g. in vertex mode, “select all” selects all the vertices of the current object.

- **Selecting** things (based on mode) and changing selections (e.g. adding to, inverting, or subtracting from a selection)

- **Deleting** whatever you have selected. Deleting edges between adjacent faces will merge those faces (usually!).

- **Creating** new faces by selecting or creating a series of vertices. (Select the vertices in counter-clockwise order viewing from the front.)

- **Moving, rotating, and/or scaling** the current selection.

- And applying various **tools** to the current selection or the mesh.

### Polygon Tools

Cheetah 3D has an excellent set of polygon tools. The only weird thing is that the organization of the Polygon Tools menu *doesn’t make any sense at all*. The only way to deal with it is keep reading through it until you memorize it.

Most Polygon tools are modal — i.e. by selecting the tool you enter a special mode. Some have an instantaneous effect which doesn’t change the current mode (which, since it’s probably Transform or Select mode is a very good thing).

#### Create Polygon

Create Polygon lets you add polygons to the current mesh by picking or creating a series of points. (If you click on or near an existing point in the mesh, that existing point is used, otherwise a new point is created in the camera plane or the main plane depending on the tool’s settings — this is almost never where you actually want the point.)

To finish a polygon you simply pick the point you started with (after picking at least three points in total).

It’s important to know that you should pick the points in **anticlockwise** order (as seen from the front/visible surface of the polygon).

![](pastedGraphic-133.jpg)

_**Helpful hint**: switch to point mode before creating a polygon so you’ll be able to see existing points more easily._

#### Fill Hole

If you want to create a polygon to fill an “obvious” hole in your mesh you can use the Fill Hole tool and simply click on one point on the edge of the hole.

Fill Hole will produce no result (it will beep) or a very bad result if there are problems with your mesh (e.g. gaps, flipped polygons, or the “hole” would produce a very odd polygon).

#### Magnet

![](pastedGraphic-134.jpg)

*The Magnet tool lets you manipulate a mesh with “falloff”. It’s a bit like “soft selections” in other modeling programs but a lot less useful (unfortunately).*

The magnet tool allows you to manipulate a mesh as though it were made of some kind of viscous goo, but it is really hard to work with because it only works in screen space or normal to the selection (and every time you do something the normal changes) which means you tend to have to get it right the first time (also it provides no useful visual feedback). If you really need this kind of functionality for anything more than a trivial tweak, I’d suggest using another program.

#### Cover, Extrude

Cover and Extrude allow you to “pull” more geometry out of a selection.

Cover is instant. Extrude is modal. This makes Cover much more pleasant to use than Extrude. Also, Cover works on edges and Extrude doesn’t.

**One important use for Extrude**

![](pastedGraphic-135.jpg)

![](pastedGraphic-136.jpg)

![](pastedGraphic-137.jpg)

*Here I want to pull more geometry out of a polygon in a symmetric model (a mesh with a Symmetry modifier) and the polygon is on the edge of the mesh. In the **middle** image you see the result of Cover followed by scaling down the polygon. At the **bottom** I first Extrude the polygon exactly zero distance (once I had the Extrude tool selected, I set it to Quantize 0.1 and then dragged the extrude out and back to exactly zero), and then I scaled the new polygon.*

Extrude always leaves the edges of the extruded polygons intact, cover does not leave the edges of polygons at the boundary of a mesh intact. This makes extrude useful for working on polygons touching the symmetry plane when doing symmetric modeling.

#### Shell

![](pastedGraphic-138.jpg)

![](pastedGraphic-139.jpg)

The Shell tool allows you to turn single-sided surfaces into two-sided solids (as per the before and after meshes shown above). Note that Cheetah 3D v6 introduces a Shell modifier which is both non-destructive and allows you to finely control the thickness of the output mesh. In most cases you’ll want to use the modifier rather than the tool.

#### Ring Cut, Jigsaw, Scalpel

Ring Cut, Jigsaw, and Scalpel are all tools for adding new edges to a mesh.

![](pastedGraphic-140.jpg)

*A plane with one of its quads split into two triangles showing the **edge loops** as dashed lines.*

![](pastedGraphic-141.jpg)

*The same plane showing one of its rings and the two loops that it is derived from.*

In terms of mesh topology, an **edge loop** is a series of edges passing through four-way intersections of edges (it may or may not form an actual loop).

A ring is a series of edges that join adjacent loops.

When you select the ring cut tool **clicking on an edge**:

1. creates a new edge loop through the ring

2. containing the edge you clicked on,

3. and passing through the point you clicked.

![](pastedGraphic-142.jpg)

![](pastedGraphic-143.jpg)

*The jigsaw tool cut the path shown through the selected polygons.*

The Jigsaw is a like the ring cut tool, but follows a selected path of polygons rather than a ring, so you can “steer” it. The jigsaw will stop cutting at the edge of a mesh or before the last polygon on a path.

The Scalpel tool lets you cut a series edges manually. You must start the series of edges on an existing edge or point, and then click out new edges one point at a time. A new edge cannot cross another edge, so to cut an edge from one polygon to another you must go through an existing edge or point.

#### Normal Scale, Rotate, Move

The tools do the same thing as the Transform tool with Orientation set to Normal. I suggest using the Transform tool instead as these tools are no longer needed.

#### Edge Rotate

![](pastedGraphic-144.jpg)

![](pastedGraphic-145.jpg)

*The selected faces are rotated around the edge (highlighted in purple in the **bottom** image).*

The edge rotate tool allows you to rotate the current selection around any edge in the mesh.

#### Point Slide

![](pastedGraphic-146.jpg)

Point slide is a useful tool for tweaking a point by moving it along one of the edges connected to it.

#### Collapse, Weld

Collapse and weld are used for simplifying a mesh’s geometry. Weld lets you merge two points (and makes all the necessary changes to edges and polygons that use those points) while Collapse turns a specified polygon into a single point.

Weld has a very important setting — “to” which determines whether the final point is the first of the two points you weld together, or the middle (average) of them.

Collapse, similarly, has a “to” setting which determines where the vertex that replaces the thing you click on is the point you clicked or the middle of the thing you clicked.

#### Collapsing Edges

There is no convenient tool for “collapsing” edges, but welding the points at either end of an edge has the equivalent effect.

#### Optimize

![](pastedGraphic-147.jpg)

Optimize is a very useful tool for cleaning up meshes. Usually the only thing you might want to change from the default setting is to change the weld radius.

**Remove unused vertices** deletes any vertices (points) from the mesh that aren’t part of any polygon.

**Weld close vertices** merges vertices within a threshold distance — the Weld Radius — of one another (this is very helpful for optimizing meshes).

I’m not sure what **Optimize polygons** does (and the online help is no use here) but I suspect it removes zero-length edges from polygons

#### Array, Clone

![](pastedGraphic-148.jpg)

![](pastedGraphic-149.jpg)

*I’ve created an array of 1 x 10 x 1 copies of the selected face with a total offset of x = 0, y = 1, z = 0.*

Array creates an array of duplicates of your current selection based on the tool settings. (It’s a lot like the Array modifier except that it produces an immediate result and operates on a selection rather than the whole mesh.)

Clone is like Array but only produces a “single row” of copies.

#### Disconnect

![](pastedGraphic-150.jpg)

![](pastedGraphic-151.jpg)

![](pastedGraphic-152.jpg)

*Here I select the top of a sphere (converted into an editable mesh first, of course) then — **middle** — I move the selection which stretches polygons bordering the selection. Oops! Finally — **bottom** — I use the Disconnect tool first then move the selection.*

Disconnect separates the current selection from the rest of your mesh without splitting it off into a new object (e.g. imagine you want to move the current selection “away” from the rest of the mesh without distorting polygons that share edges with polygons in your selection — disconnect it first).

Note that Disconnect is a modal tool that needs to be Applied (click OK in the Tool panel) or it won’t do anything.

#### Crumple

![](pastedGraphic-153.jpg)

*A 10x10 plane converted to an editable mesh and crumpled with the default tool settings.*

Crumple randomly displaces the points in your mesh (based on the tool settings).

#### Flip Normal

![](pastedGraphic-154.jpg)

![](pastedGraphic-155.jpg)

*If you take a cube, delete three faces, then flip the remaining polygons, you have a little set to place things on.*

Flip normal reverses the winding order and thus flips the facing of selected polygons. The facing of a polygon has important effects on how a polygon is smoothed and rendered in Cheetah 3D, and in most realtime applications polygons will only be visible from the “front”.

#### Ruler

![](pastedGraphic-156.jpg)

*When in ruler mode, mousing over a point will show you its exact position (in the selected coordinate system) and clicking on multiple points will show you the precise distances between them.*

The ruler allows you to see the precise position of points in global or local space and measure the precise distance between points.

#### Triangulate

![](pastedGraphic-157.jpg)

![](pastedGraphic-158.jpg)

*A dodecahedron before and after triangulation. (Because no polygons were selected the entire mesh was triangulated.)*

The triangulate tool instantly triangulates the current selection (or the entire mesh if all or no polygons are selected).

Generally you do not want to triangulate meshes, but there may be cases (e.g. Symmetric models) where you want fine control over the way the mesh is triangulated.

#### Catmull-Clarke Subdivide, Linear Subdivide

![](pastedGraphic-159.jpg)

![](pastedGraphic-160.jpg)

*Here I have selected one face and applied the Catmull-Clarke Subdivision tool to it twice. This technique is sometimes suggested as a good way to put round holes in square faces, but there are better and more precise methods. I would suggest that unless you know exactly what you’re doing, don’t use these tools.*

These tools allow you to immediately apply one level of subdivision to selected polygons.

In general if you want to subdivide an entire mesh you should use the Subdivision modifier which offers more options, greater control, and is non-destructive. Occasionally you may want to have complete control over subdivision, in which case these tools can be useful.

#### Split

Split breaks off your current selection into a new (separate) mesh. If your original mesh was called “foo” then the selection will be moved to a new mesh (with the same coordinate system) called “foo.split”.

##### Aside: Merging Meshes

**To merge two or more meshes in C3D7** simply select them (you can select a range by click and shift-click, or toggle the selection of individual meshes using command-click) and select **Tools \> Merge**.

**To merge two or more missions in earlier versions of C3D**: pick one mesh to be the “parent” and make all the others its children, then select the parent mesh and use **Tools \> Import Children** (you can now delete the children or collapse the parent).

#### Bevel, Inner Extrude

![](pastedGraphic-161.jpg)

![](pastedGraphic-162.jpg)

![](pastedGraphic-163.jpg)

*Starting with the three edges selected on the left, I perform a linear bevel with bevelLevel 0 or a Concave bevel with bevelLevel 10.*

Bevel is used to refine corners where two polygons meet by introducing new geometry. Since almost no real world objects have perfectly sharp corners, beveling is very useful for making models a little bit more realistic. It’s also very useful for refining subdivision cages.

![](pastedGraphic-164.jpg)

![](pastedGraphic-165.jpg)

*Here we have a simple box with subdivision applied. By beveling the selected edges the curvature of the subdivided surface can be given more definition in a highly controlled way.*

![](pastedGraphic-166.jpg)

![](pastedGraphic-167.jpg)

**Inner Extrude** is an odd tool. It creates more geometry around selected edges. Among other things, inner extrude is useful for refining subdivision cages.

#### Bridge

The Bridge tool joins two parts of a mesh. You pick two vertices on two different polygon edges in the mesh (“holes”) and it joins them together. The Bridge tool works best when you use two polygons with the same number of sides, and go in the same direction (i.e. with the adjacent polygons having their normals set correctly). The two vertices you select will be joined directly, and the rest of the bridge will follow from that.

##### Aside: Troubleshooting the Bridge Tool

Sometimes the bridge tool simply refuses to work. This is usually because either one or more of the faces bordering on one of the polygons is flipped, which means that the polygon isn’t well-formed.

![](pastedGraphic-168.jpg)

In this example I’ve flipped the normals of the top portion of the mesh, which confuses the bridge tool (because it reverses the order of the polygon its joining up) and completely messes up the resulting bridge.

![](pastedGraphic-169.jpg)

You should also pick the vertexes you click on with the understanding that the first vertex will be joined to the second, and everything else will follow from that. So if you pick the mismatched vertices, the bridge will be twisted. (This “twisting” effect can actually be used to your advantage in some cases.)

![](pastedGraphic-170.jpg)

Finally in this example there’s an exaggerated hidden flaw in my geometry which means that one of the polygons isn’t the polygon I thought it was and the resulting bridge is, again, a mess.

branch_name: refs/heads/master

extension: md

gha_language: JavaScript

license_type: permissive


text:
---
description: >-
  How does DroneBridge compare to traditional WiFi connections & other similar
  projects
---

# Comparison

## Compared to traditional WiFi

1. Patched WiFi drivers inside the Kernel force the WiFi adapters to send at their power limit \(configurable\)
2. Forward Error Correction \(FEC\) of sensitive data \(video\) to account for packet loss
3. Broadcasting of data. Not connection oriented like with a WiFi AP. If your card can pick up a packet you will be able to read its data. No re-sending of packets re-authorisation on connection loss. This means no guaranteed delivery.
4. Typically lower data rates than with traditional WiFi \(18Mbit compared to <150Mbit\) give more range
5. [More details on the original website](https://befinitiv.wordpress.com/wifibroadcast-analog-like-transmission-of-live-video-data/). DroneBridge and all forks still use the same principle

## Compared to forks of befinitivs WifiBroadcast

Relevant forks are:

* svpcoms WifiBroadcast \(basic TX-RX\)
* DroneBridge \(system design with library\)
* OpenHD \(follow-up on EZ-WifiBroadcast\)
* EZ-Wifibroadcast \(not under development\)

The base for all of the current projects is befinitivs WifiBroadcast. Based on that, EZ-WifiBroadcast was built with many improvements like Kernel patches for higher power output, OSD, telemetry, RC etc.. While the original WifiBroadcast is just a data link, all future forks focus on the use case in UAV applications. Custom data links are still possible with all variants.

Because the EZ-WifiBroadcast was missing a clear project structure, was more or less a bunch of scripts and did not have any kind of proper documentation, DroneBridge was built. The first releases were based on EZ-WifiBroadcast with extra programs onboard to replace some of the functionality. In parallel an app was developed to work with the system. As of release v0.6 DroneBridge does not use any WifiBroadcast code base except for the OSD, video player and FEC encoding/decoding.

Because of some disagreements regarding the future and technical development of the project, parts of the EZ-WifiBroadcast community forked OpenHD. OpenHD is built around an automated image builder.

## What to choose?

{% hint style="warning" %}
In short: The answer will be DroneBridge for almost all cases \(You are inside the DroneBridge Wiki. What did you expect!? :D\). In the special case that you only need a basic \(continuous\) data stream from one device to another and that requires FEC, it is best to choose svpcom WifiBroadcast. It is fast to setup and will get the job done just fine.
{% endhint %}

**Why DroneBridge then?**

Compared to the other projects it was developed with a system design approach in mind from the very first moment on. There is a common software library that implements all core functions like sending & receiving data. This makes is very convenient and easy to develop the system.

DroneBridge also enables users to write their own plugins. Plugins are standalone programs that get started right after boot. They can be just as powerful and performant as the core modules. You can write your own plugin in less than 10 minutes making full use of the DroneBridge common library. DroneBridge modules and the code base are very well documented. It is free of experimental scripts and follows the directive to be a general communication link. Implementations for custom hardware options \(camera switcher, GPIO in/output etc.\) can be implemented via the plugins. This means the base image is easy to understand and does not confuse new users with lots of badly documented extras that only a few people really use.

Besides that DroneBridge also offers professional logging support, support for wired connections to all android devices \(No USB tethering, no SIM required\) and many more things. In fact it was the first project to introduce a custom app that integrated convenient setting changes, telemetry and low latency video display.

## Stay tuned for more

repository: DroneBridge/Docs

repo_name: DroneBridge/Docs


text:
# python_learningresources
Links to books, repositories, blogs and videos about useful Python resources

## Language

### Language fundamentals

- [Difference between lists and tuples](https://stackoverflow.com/questions/626759/whats-the-difference-between-lists-and-tuples)
- [Create dictionary from two lists](https://stackoverflow.com/questions/209840/convert-two-lists-into-a-dictionary)
- [Index a list with another list](https://stackoverflow.com/a/1012197/4892892)
- [Single list from a list of lists](https://stackoverflow.com/questions/952914/how-to-make-a-flat-list-out-of-list-of-lists); example and expected output:
   ```python
   nested_list = [['some'], ['items']]
   flat_list = ['some', 'items']
   ```
   - [List comprehension](https://stackoverflow.com/a/952952/4892892)
      ```python
      flat_list = [item for sublst in nested_list for item in sublst]
      ```
   - [Chaining](https://stackoverflow.com/a/953097/4892892)
      ```python
      import itertools
      flat_list = list(itertools.chain.from_iterable(nested_list))
      # equivalently: flat_list = list(itertools.chain(*nested_list))
      ```
   - [Monoid](https://stackoverflow.com/a/952946/4892892)
      ```python
      flat_list = sum(nested_list, [])
- [Slice assignment](https://stackoverflow.com/questions/32448414/what-does-colon-at-assignment-for-list-do-in-python?)

   ```python

   ```
- [Using the `else` clause in `for` and `while` loops](https://stackoverflow.com/questions/9979970/why-does-python-use-else-after-for-and-while-loops)
- [Function arguments](https://www.blog.pythonlibrary.org/2022/06/28/python-101-an-intro-to-functions)

   As of Python 3.8, [positional-only arguments](https://peps.python.org/pep-0570/) are supported. These arguments cannot by passed as keywords.
   The forward slash, /, indicates to Python that all arguments before the forward-slash are positional-only arguments. Anything following the forward slash are positional or keyword arguments up to the *. The asterisk indicates that everything following it are keyword-only arguments.
   For instance, calling the function
   ```python
   def positional(name, age, /, a, b, *, key):
       pass
   ```
   Here are some invalid calls to this function:
   ```python
   >>> positional(name='Mike')
   TypeError: positional() got some positional-only arguments passed as keyword arguments: 'name'
   >>> positional('Mike', 17, 2, b=3)
   TypeError: positional() missing 1 required keyword-only argument: 'key'
   >>> positional('Mike', 17, 2, 3, 'test')
   TypeError: positional() takes 4 positional arguments but 5 were given
   >>> positional('Mike', 17, 2, b=3, keyword='test')
   TypeError: positional() got an unexpected keyword argument 'keyword'
   ```
   Some valid function calls are
   ```python
   positional('Mike', 17, 2, 3, key='test')
   positional('Mike', 17, 2, b=3, key='test')
   ```

### File handling

- [Append text file in the middle](https://stackoverflow.com/questions/10507230/insert-line-at-middle-of-file-with-python)

### Conventions

A major design priciple behind Python was that code should be easily read.

## Packaging

The directory structure of a Python project should be created so that it is easily testable, deployable and put under continuous integration. Some links I found useful:

- https://github.com/yngvem/python-project-structure
- https://www.kennethreitz.org/essays/repository-structure-and-python
- https://hynek.me/articles/testing-packaging/

## NumPy

- [NumPy version](https://stackoverflow.com/a/1520264/4892892)
   ```python
   import numpy
   print numpy.__version__
   ```
   or from the command line:
   ```python
   python -c 'import numpy; print(numpy.__version__)'
   ```
- Print information about the CPU
   ```python
   import numpy.distutils.cpuinfo.cpu
   cpu.info()
   ```
- Figure out which BLAS and LAPACK implementations are used by your NumPy (and therefore SciPy, etc.)
   ```python
   import numpy.distutils.system_info
   get_info('blas_opt')
   get_info('lapack_opt')
   ```
   [Here](https://github.com/numpy/numpy/blob/v1.19.0/numpy/distutils/system_info.py#L401-L485) is the list of all options for `get_info`.

## Best practices

- [How to order methods in a class?](https://stackoverflow.com/questions/10289461/what-is-a-good-way-to-order-methods-in-a-python-class)

## Visualization

### Plotting with Matplotlib

Browse the extensive [gallery](https://matplotlib.org/3.1.1/gallery/index.html) for examples, read the [User's Guide](https://matplotlib.org/3.1.1/contents.html), or check the [tutorials](https://matplotlib.org/3.1.1/tutorials/index.html).
Two types of plotting:
- pyplot: high-level MATLAB-like plotting for simple interactive plotting
   - [tutorial](https://matplotlib.org/3.1.1/tutorials/introductory/pyplot.html)
   - [gallery](https://matplotlib.org/3.1.1/gallery/index.html#pyplots-examples)
   - [API](https://matplotlib.org/3.1.1/api/_as_gen/matplotlib.pyplot.html)
- object-oriented API
   - [gallery](https://matplotlib.org/3.1.1/gallery/index.html)

Links to APIs I often use:
- [Figure class](https://matplotlib.org/3.1.1/api/_as_gen/matplotlib.figure.Figure.html)
- [axes module](https://matplotlib.org/3.2.1/api/axes_api.html)

[Positioning the legend](https://matplotlib.org/3.2.2/api/_as_gen/matplotlib.axes.Axes.legend.html#matplotlib.axes.Axes.legendhttps://matplotlib.org/3.2.2/api/_as_gen/matplotlib.axes.Axes.legend.html#matplotlib.axes.Axes.legend)

### Class hierarchy

If you install Pylint, it comes with [Pyreverse](https://www.logilab.org/blogentry/6883). Type
```python
pyreverse -o <output_format> <module_name>
```
to the terminal to save the class hierarchy fetched from the module `<module_name>` in `<output_format>` (`png`, `svg`, `eps`, `pdf`, `dot`, etc.). Note that `module_name` must be importable. If `module_name` is a package that contains other packages or modules, all the contents of the package will be parsed. To select a subpackage or a module, use the slash; e.g. to process only *f2py* from *numpy*, use
```python
pyreverse -o png numpy/f2py
```
It is often useful for the end user to ignore the tests. You can ignore a directory by giving the `--ignore` flag. In the example above:
```python
pyreverse -o png --ignore tests numpy/f2py
```
For further options, check the documentation:
```python
pyreverse --help
```

## GUIs

- view *pandas* DataFrame objects

  [PandasDataFrameGUI](https://github.com/bluenote10/PandasDataFrameGUI) supports useful data analysis (filtering, sorting, plotting, etc.) and has few dependencies. A common mistake is that your code already imported *matplotlib*, which selected a backend other than `'WXAgg'`. The solution is to load *matplotlib* **before** you use your own code, with
  ```python
  import matplotlib
  matplotlib.use('WXAgg')
  ```
  as shown [here](https://github.com/bluenote10/PandasDataFrameGUI/issues/3#issuecomment-324940242).

  ## Online

  ### Binder

  - [Generate custom badges](https://mybinder.readthedocs.io/en/latest/howto/badges.html#generate-custom-launch-badges-for-your-binder-repository): the link you need to insert comes from [MyBinder](https://mybinder.org/).

license: MIT


text:
# Lawrence-Test (Backend)

> This is the Server side of a job test by lawrence to develop an SDK for an API (https://github.com/evabanegacom/Lawrence-Test.git)

## Built With

- Ruby v2.7.0
- Ruby on Rails <IP_ADDRESS>
- Rspec
- JWT
- Heroku

## Core Functionalities

- **User authentication:** allows new user to register and existing users to login.

## Getting Started

To get a local copy up and running follow these simple example steps.

- On the project GitHub page, navigate to the main page of the repository [this page](https://github.com/evabanegacom/Lawrence-Test.git).
- Under the repository name, locate and click on a green button named `Code`.
- Copy the project URL as displayed.
- If you're running Windows Operating System, open your command prompt. On Linux, Open your terminal.
- Change the current working directory to the location where you want the cloned directory to be made. Leave as it is if the current location is where you want the project to be.
- Type `git clone`, and then paste the URL you copied in Step 3.

  `$ git clone https://github.com/evabanegacom/Lawrence-Test.git` Press Enter key

- Press Enter. Your local copy will be created.

## Prerequisites

- [Git](https://gist.github.com/derhuerst/1b15ff4652a867391f03).
- Web browser (Chrome/Firefox)
- [Node](https://nodejs.org/en/)
- [NPM](https://www.npmjs.com/get-npm)
- Ruby v2.7.0
- Ruby on Rails v5
- Postgresql v >= 9.5

## Project Setup

_After cloning, the following steps setup the project_

- `cd Lawrence-Test` to change the current working directory.
- `bundle install` to install all necessary dependencies.
- `rails db:create` && `rails db:migrate` to create database and tables.
- `rails s` to start the application.
  Feel free to use POSTMAN or any other similar infrastructure to test the different endpoints.

  To run test suites included, run:

- `bundle exec rspec`

## API Documentation

> The base URL for all endpoints is `https://lawrence-test.herokuapp.com/`. Some requests requires validation while some don't. The validation is implemented using tokens generated by JWT upon registration or login. This token is then sent with the headers of the requests that requires validation, so that it can be decoded and the app can determine if the token is valid or not. If valid, the user can successfully access such end-points that requires authentication.

**Summary of Available API Endpoints**
| Endpoint | Feature | Authentication |
| ------------ | ------------| ---------- |
| POST /users | Creates a new user | False |
| POST /login | Logs in a user | False |
| GET /autologin | Checks login status of the current user | True |
| GET /users | Fetches all available users for loggedIn user | True |
| GET /all_transactions | Fetches all transactions by logged in user | True |
| POST /add_money | Creates new incoming transaction for logged in user | True |
| POST /send_money | Creates new outgoing transaction for logged in user | True |

**Sampling a request that requires authentication**

_Fetch transactions: endpoint fetches all the transactions already carried-out by the logged in user. Therefore, it requires authorization._

Endpoint: `https://lawrence-test.herokuapp.com/api/v1/all_transactions`

Request (from POSTMAN):

```js
token = 'eyJhbGciOiJIUzI1NiJ9.eyJ1c2VyX2lkIjoxfQ.JC6qKuH9SG0SIiYSfhZUFTtirxN9Q47buLk0DPFFFzE'
curl --location --request GET 'https://lawrence-test.herokuapp.com/api/v1/all_transactions' \
--header 'Authorization: Bearer <token>' \
```

Response:

```json
[
  {
        "id": 1,
        "incoming_transactions": "20.0",
        "outgoing_transactions": "0.0",
        "user_id": 1,
        "created_at": "2021-03-31T13:35:18.376Z",
        "updated_at": "2021-03-31T13:35:18.376Z"
    }
]
```

_Add Money transactions: endpoint for creating incoming transactions by the loggedIn user. Therefore, it requires authorization._

Endpoint: `https://lawrence-test.herokuapp.com/api/v1/add_money`

Request (from POSTMAN):

```Previous balance was 80.0 adding 50 made it 130.0```

```js
token = 'eyJhbGciOiJIUzI1NiJ9.eyJ1c2VyX2lkIjoxfQ.JC6qKuH9SG0SIiYSfhZUFTtirxN9Q47buLk0DPFFFzE'
curl --location --request POST 'https://lawrence-test.herokuapp.com/api/v1/add_money' \
--header 'Authorization: Bearer <token>' \

--form 'incoming_transactions="50.0"' \
--form 'outgoing_transactions="0.0"'
"any of the incoming or outgoing transactions could be blank if necessary, they are by default 0.0"
```

Response:

```json
"balance": true,
    "user": {
        "id": 1,
        "name": "precious",
        "email"<EMAIL_ADDRESS>        "global_balance": "130.0"
    },
    "transact": {
        "id": 34,
        "incoming_transactions": "50.0",
        "outgoing_transactions": "0.0",
        "user_id": 1,
        "created_at": "2021-03-31T21:48:13.752Z",
        "updated_at": "2021-03-31T21:48:13.752Z"
    }
}
```

_Send Money transactions: endpoint for creating outgoing transactions by the loggedIn user. Therefore, it requires authorization._

Endpoint: `https://lawrence-test.herokuapp.com/api/v1/send_money`

Request (from POSTMAN):

```js
token = 'eyJhbGciOiJIUzI1NiJ9.eyJ1c2VyX2lkIjoxfQ.JC6qKuH9SG0SIiYSfhZUFTtirxN9Q47buLk0DPFFFzE'
curl --location --request POST 'https://lawrence-test.herokuapp.com/api/v1/send_money' \
--header 'Authorization: Bearer <token>' \

--form 'incoming_transactions="0.0"' \
--form 'outgoing_transactions="50.0"'
"any of the incoming or outgoing transactions could be blank if necessary, they are by default 0.0"
```

Response:

```Previous balance was 130.0 removing 50 made it 80.0```

```json
"balance": true,
    "user": {
        "id": 1,
        "name": "precious",
        "email"<EMAIL_ADDRESS>        "global_balance": "80.0"
    },
    "transact": {
        "id": 35,
        "incoming_transactions": "0.0",
        "outgoing_transactions": "50.0",
        "user_id": 1,
        "created_at": "2021-03-31T21:48:13.752Z",
        "updated_at": "2021-03-31T21:48:13.752Z"
    }
}
```
**Sampling a request that DOES NOT require authentication**

_Login: endpoint fetches creates a new token and ensure that the user can access all features._

Endpoint: `https://lawrence-test.herokuapp.com/api/v1/login`

Request (From POSTMAN):

```js
curl --location --request POST 'https://lawrence-test.herokuapp.com/api/v1/login' \
--form<EMAIL_ADDRESS>\
--form 'password="password"'
```

Response:

```json
{
  "user": {
    "id": 1,
    "name": "John Doe",
    "email"<EMAIL_ADDRESS>  },
  "token": "eyJhbGciOiJIUzI1NiJ9.eyJ1c2VyX2lkIjoxfQ.JC6qKuH9SG0SIiYSfhZUFTtirxN9Q47buLk0DPFFFzE",
  "logged_In": true
}
```

_User registration: endpoint creates a new user and token to ensure that the user can access all features._

Endpoint: `https://lawrence-test.herokuapp.com/api/v1/users`

Request (From POSTMAN):

```js
curl --location --request POST 'https://lawrence-test.herokuapp.com/api/v1/users' \
--form<EMAIL_ADDRESS>\
--form 'password="password"'
--form 'password_confirmation="password"'
--form 'name="johndoe"'
--form 'global_balance="50.0"'
```

Response:

```json
{
  "user": {
    "id": 1,
    "name": "John Doe",
    "email"<EMAIL_ADDRESS>    "global_balance": "50.0"
  },
  "token": "eyJhbGciOiJIUzI1NiJ9.eyJ1c2VyX2lkIjoxfQ.JC6qKuH9SG0SIiYSfhZUFTtirxN9Q47buLk0DPFFFzE",
  "logged_In": true
}
```

## Deployment

This app was deployed to Heroku and is accessible at: `https://lawrence-test.herokuapp.com/`

## Author

👤 **Precious**

- Github: [@evabanegacom](https://github.com/evabanegacom)

## 🤝 Contributing

Contributions, issues and feature requests are welcome!

Feel free to check the [issues page](https://github.com/evabanegacom/Lawrence-Test/issues/).

## Show your support

Give a ⭐ if you like this project!

## 📝 License

This project is [MIT](/LICENSE) licensed.

repository: evabanegacom/Lawrence-Test

branch_name: refs/heads/main

repo_name: evabanegacom/Lawrence-Test


text:
---
draft: false
date: 2015-02-10T10:49:00+13:00
title: Building Apps with Riot, ES6 and Webpack
slug: riot-es6-webpack-apps
tags: [6to5, ES6, Flux, Riot, Riotjs, Webpack]
---

I arrived at [Riot](https://muut.com/riotjs/)  after reading the Muut [Frameworkless JavaScript](https://muut.com/blog/technology/frameworkless-javascript.html) blog post -- read this first!  The Muut programmers have put their money where their mouths are and built Riot, a minimalist [Reactjs-like library](https://muut.com/blog/technology/riot-2.0/) called [Riot](https://muut.com/riotjs/) for building reactive UI components.

Reading the Riot documentation the thing that struck me was how easy Riot is to understand -- there are very few new terms and concepts to learn compared to React (which, to be fair, is also straight-forward compared to e.g. Polymer and Angular).

To help me learn Riot I ported my React [Flux Backbone Todos Example](https://github.com/srackham/flux-backbone-todo) to a [Riot Todo app](https://github.com/srackham/riot-todo). This post is a compilation of notes I made documenting the experience.

<!--more-->

The [tl;dr](http://en.wikipedia.org/wiki/Wikipedia:Too_long;_didn%27t_read) answers are:

1. Riot really does live up to its minimalist motto. The (unminified) Riot version of the Todo weighs in at a tiny 32KB compared to 964KB for the React version (600KB after subtracting Backbone and JQuery which are not used in the Riot version). However you look at it that's a huge difference.

2. Riot has that rare Goldilocks "just right" feel about it and is a joy to use.

3. Riot is a relatively new project, I was unable to find statistics on how it scales performance-wise in larger projects. How it fares alongside the Rect juggernaut is an open question -- I hope it does well.

If you're not familiar with Riot take a look at the [Riot website](https://muut.com/riotjs/) -- the documentation is first class. So rather than discussing how Riot works I'll just list things I've learnt that may not be obvious reading the documentation.

## Using ES6 with Riot
The [example app](https://github.com/srackham/riot-todo) is written in ES6, I use the [6to5](http://6to5.org/) transpiler to convert it to ES5 and [Webpack](http://webpack.github.io/) to bundle the compiled code along with required libraries. This approach necessitates combining JavaScript modules -- working in ES6 is particularly nice as you get to take advantage of the new ES6 import and export statements (see [this example](https://github.com/srackham/riot-todo/blob/master/src/index.js)).

Webpack [is configured](https://github.com/srackham/riot-todo/blob/master/webpack.config.js) to use the `6to5-loader` which automatically converts ES6 source to CommonJS formatted ES5 modules that are then bundled into the single `bundle.js` distributable file.

## Why I don't use Riot .tag files
Riot tag files are HTML templates containing HTML markup along with JavaScript UI logic.
If you've looked at the [Todo app](https://github.com/srackham/riot-todo) app you are probably wondering where the tag files are -- the answer is that I no longer use them, preferring JavaScript instead. By eliminating `.tag` files I've simplified my coding, tooling and workflow. For me, tag file complexity and limitations outweigh any perceived benefits.

This is not a criticism of Riot, the neat thing about tag files is that they're entirely optional and I'm documenting why I don't use them.

When you examine the compiled JavaScript you see that Riot tag files are a thin layer of syntactic sugar.

- They add an additional conceptual layer -- new(ish) syntax and semantics have to be learned.

- They add an additional compilation step.

- The tag file compiler dictates the languages and templates you can use (CoffeeScript, ES6, Jade) contradicting the "Use your favorite tools" mantra.

- The tag constructor-body logic is out of context:

    * It is not amenable to editor/IDE linting and other tooling.
    * Referencing `this` out of context means the code is not legal JavaScript and generates errors in editors/IDEs.

- There is currently no support for module formats (CommonJS, AMD) -- tag files compile to plain JavaScript .

- Tag files require tag loaders to be used directly in build tools such as Webpack and Browserify.

- The "ES6-like" constructor methods are nice but they are not legal JavaScript and are likely to always be a source of ongoing confusion (syntax and semantics). You can use _ES6 arrow functions_ to achieve the same semantics (lexically bound `this`) with almost the same brevity, for example:

``` javascript
this.add = (e) => {
  var input = e.target[0]
  this.items.push(input.value)
  input.value = ''
};
```

Here's [an example of an ES6 JavaScript tags file](https://github.com/srackham/riot-todo/blob/0.1.1/src/tags.js) using ES6 template strings and arrow functions.

ES6 Template Strings aid the readability of tag HTML templates. Similarly, if you are using CoffeeScript you can use CoffeeScript Block Strings. [JSX](http://facebook.github.io/jsx/) is another possibility -- React's JSX transformer could be modified to emit a string literal and you could then take advantage of existing JSX tooling support.

## The fundamental difference between Riot and React
The most significant difference lies in how the UI markup templates are declared:

- In React the UI template markup is baked in your JavaScript source (using the JSX language extension).
- Riot inverts the React model by putting the markup and logic in an HTML (tag) file.

The upshot of this inversion is that the React template DSL language is JavaScript whereas Riot relies on a custom template DSL (implemented with custom tags). Here are two simplified examples that generate a list from an array of todo items: the first is React JavaScript the second is the equivalent Riot tag markup:

``` html
<ul>
  todos.map(todo =>
    <li><TodoItemComponent todo={todo} /></li>)
</ul>
```

``` html
<ul>
  <li each="{todo in todos}">
    <todo-item todo="{todo}">
  </li>
</ul>
```

The first example uses JavaScript's `map` function to generate a list of `<li>` elements; the second uses Riot's custom `each` template attribute.

Steve Luscher explains why he thinks JavaScript is better than custom template DSLs [towards the end of this video](https://www.youtube.com/watch?v=rI0GQc__0SM#t=1325) -- not only do you have to learn a custom DSL you are also constrained by the features the DSL offers. For small common use-case examples like ones above there is little to choose between the two approaches, but when it comes to larger highly dynamic UI compositions the power and flexibility of React's JavaScript approach is clearly superior.

## Tips

### Avoid self-closing XHTML style tags
Do not close tags with `/>` as it does not always immediately close the tag.  When dealing with HTML5 elements `<foo />` means `<foo>` (whereas in XHTML `<foo />` means `<foo></foo>`) i.e. HTML5 ignores the `/` character. See this [Stackoverflow post](http://stackoverflow.com/a/9845124/1136455) on the subject. See also:

  * [Closing HTML5 tags with /> can produce unexpected results](https://muut.com/riotjs/forum/#!/using:closing-html5-tags-with-).

  * [Misplaced form end-tags](https://muut.com/riotjs/forum/#!/using:misplaced-form-end-tag).

### Bind tag event handlers to this
Bind tag event handlers to `this` to ensure they are always fired with the tag context (alternatively use the `var self = this` idiom). For example:

``` javascript
this.clear = function(e) {
  dispatcher.trigger(dispatcher.CLEAR_TODOS);
}.bind(this);
```

With ES6 you can achieve the same thing with lexically bound ES6 _arrow functions_:

``` javascript
this.clear = (e) => {
  dispatcher.trigger(dispatcher.CLEAR_TODOS);
};
```

### Referencing Loop items
Use the `each="{item in items}"` construct to pass the current looped item to a child custom tag. In the following example the code in the custom `todo-item` tag can refer to the current todo item with `opts.todo`:

``` html
<ul>
  <li each="{todo in opts.store.todos}">
    <todo-item store="{parent.opts.store}" todo="{todo}">
  </li>
</ul>
```

### Namespace event names
Use the colon namespacing convention to group application event names e.g. `admin:edit`, `admin:delete`, `admin:new`.

### You can use custom tag names in CSS files
Custom tags are rendered into the DOM and so they can be used for CSS selection and DOM interrogation, here's [an example](https://github.com/srackham/riot-todo/blob/master/dist/index.css).

## Debugging
When bundling with Webpack you need to use the [devtool source-map configuration option](https://github.com/srackham/riot-todo/blob/master/webpack.config.js) to produce a [source map](http://www.html5rocks.com/en/tutorials/developertools/sourcemaps/) file for you bundled app. This allows you to debug from the ES6 source code files.

To browse and debug the source code open your browser's _Sources window_ and navigate to the folder `webpack:///.`:

- In Firefox: open the Debugger (Ctrl+Shift+S).
- In Google Chrome: open the Console (Ctrl+Shift+J) then click on the _Sources_ tab to view the Sources panel.

I'm not big on debuggers and break points -- most of the time I just put strategically placed temporary `console.log()` statements in the code.

## The Big Picture
Riot (like React) is a UI library, not a framework. This is good (I prefer a small set of focused libraries to an all encompassing framework) but non-trivial applications need a coherent high-level structure (an architecture) to ease scalability, evolution and maintainability. [Flux](https://www.npmjs.com/package/flux) is increasing the architecture of choice for React-like apps. I like Flux because it's easy to understand and because it makes intuitive sense (not out of any theoretical conviction). The [Riot Todo app](https://github.com/srackham/riot-todo) uses the [RiotControl](https://github.com/jimsparkman/RiotControl) flux-style dispatcher (slightly modified) to implement the Flux architecture.

repository: srackham/stuarts-notes

repo_name: srackham/stuarts-notes


text:
---
layout: post
title: What is Docker ?
author: Nilesh Sharma
---
Docker is an open-source project which automates the deployment of Linux applications inside software containers.

## What is Docker ?
-----
From the Docker website: Docker containers wrap up a piece of software in a complete filesystem that contains everything it needs to run: code, runtime, system tools, system libraries – anything you can install on a server. This guarantees that it will always run the same, regardless of the environment it is running in.

![docker-machine-architecture](https://www.docker.com/sites/default/files/WhatIsDocker_3_Containers_2_0.png)

## Advantages of Docker:
-----
- ***Rapid application deployment*** – containers include the minimal runtime requirements of the application, reducing their size and allowing them to be deployed quickly.

- ***Portability across machines*** – an application and all its dependencies can be bundled into a single container that is independent from the host version of Linux kernel, platform distribution, or deployment model. This container can be transfered to another machine that runs Docker, and executed there without compatibility issues.

- ***Version control and component reuse*** – you can track successive versions of a container, inspect differences, or roll-back to previous versions. Containers reuse components from the preceding layers, which makes them noticeably lightweight.

- ***Sharing*** – you can use a remote repository to share your container with others for example: DockerHub. For this purpose, and it is also possible to configure your own private repository.

- ***Lightweight footprint and minimal overhead*** – Docker images are typically very small, which facilitates rapid delivery and reduces the time to deploy new application containers.

- ***Simplified maintenance*** – Docker reduces effort and risk of problems with application dependencies.


text:
# CloudmersiveConvertApiClient::TextDocumentLine

## Properties
Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
**line_number** | **Integer** | The 1-based line index of the line | [optional]
**line_contents** | **String** | The text contents of a single line of a text file | [optional]


text:
# Creating template VMs for runners

This directory provides scripts and instructions for creating a [provisioned](https://github.com/CodethinkLabs/provisioning-omnibus-VMs)
VirtualBox CentOS 6.8 VM. The created VM is suitable for running GNU and LLVM
toolchains' Omnibus scripts (available [here](https://github.com/CodethinkLabs/omnibus-codethink-toolchain)).

# Instructions

## Create VirtualBox CentOS 6.8 VM from scratch

1. Make sure that `CentOS-6.8-x86_64-minimal.iso` is in `isos` directory.
    - It can be found [here](http://mirror.nsc.liu.se/centos-store).
2. To create a VM, execute `./create_centos6.8_vm.sh -n VM_NAME`.
    - To remove the VM, execute `./remove_vm.sh -n VM_NAME`.
3. Open VirtualBox, run the VM and proceed with CentOS 6.8 installation manually.
4. Set up networking in the VM.
    - In `ifcfg-eth0`, get rid of `HWADDR` and `UUID`.
    - Copy `ifcfg-eth0` to `ifcfg-eth5`; make sure that the content of the new
      file reflects its name.
    - In `ifcfg-eth0`, make sure there is `ONBOOT=yes` whereas in `ifcfg-eth5`
      there is `ONBOOT=no`.
    - Modify `70-persistent-net.rules` so it has one record that indicates
      `eth5`.
4. *Worth doing*: Backup vdi of bare-OS VM.
5. Provision the VM with [`provisioning-omnibus-VMs`](https://github.com/CodethinkLabs/provisioning-omnibus-VMs)
   project.
    - To make ssh connection possible, switch VM's networking to bridged
      or set up port forwarding manually.
6. *Worth doing*: Backup vdi of provisioned VM.

## Create VirtualBox CentOS 6.8 VM based on existing vdi

- Once you got vdi of the provisioned VM with networking configured, you may want to
  backup it and create new VMs based on it. To do that:

1. Make sure that `VM_NAME.vdi` is in `vdis` directory.
2. Execute `./create_centos6.8_vm.sh -n VM_NAME -r`.

## Create VM templates for build and test GitLab CI runners

1. Make sure that `VM_BUILD_NAME.vdi` and `VM_TEST_NAME.vdi`,
   provisioned for build and test runners respectively,
   are in `vdis` directory.
2. Execute `sudo ./create_centos6.8_vm.sh -n VM_BUILD_NAME -r`.
3. Execute `sudo ./create_centos6.8_vm.sh -n VM_TEST_NAME -r`.

repository: CodethinkLabs/provisioning-runners-VMs

gha_language: Shell

repo_name: CodethinkLabs/provisioning-runners-VMs


text:
**This document has been superceded by the [eLife file naming and asset IDs](https://github.com/elifesciences/XML-mapping/blob/master/elife_file_naming_2016_08_25.md) document.**

---

# eLife file naming specification

## File naming and workflow

eLife receives articles from a content processor as a zip package delivered to
an Amazon S3 bucket via FTP. Our production system recognises certain metadata
in the name of the zip file, and expects files within the zip file to be named
according to the specification laid out in this document.

Some files are processed and sent to a CDN for access from the main site.

The S3 bucket for inbound content, and the location of the CDN that we populate are specified in a settings file for our production workflow. An [example settings file is available](https://github.com/elifesciences/elife-bot/blob/exp/settings-example.py).

The relevant variables in that settings file are:

> S3_monitor_queue # location of the inbound files

> ppp_cdn_bucket  # location of the CDN

In this documentation we will indicate whether a specific filetype needs to be
placed in the CDN.

## Files and the CDN

### File Types that need to be placed on the CDN

As a rule of thumb all files associated with the article need to be placed directly on the CDN.
The only exception to this rule is in relation to the figures, which need to be processed first,
and then placed on the CDN. Movies are not served by eLife, but we have no reason not to palce them on the CDN.

| Asset Type  | Example Asset Namespaces  | Example File Name | Example Extensions | DOI? |example DOI | landing page? | example landing page |
|---|---|---|---|---|---|---|---|
| Figures |  fig1, app1-fig2, fig1-figsupp1 | elife-07178-fig1-v2.gif  |  .tiff, .gif  | always | 10.7554/eLife.07178.003 | always | http://elifesciences.org/content/4/e07178v2/figure1 |
| article pdf |  - | elife-07178-v2.pdf | .pdf | always | 10.7554/eLife.07178  | always | http://elifesciences.org/content/4/e07178v2  |
| figures pdf | figures  | elife-07178-figures-v2.pdf | .pdf  | never | - | never |  - |
| xml | - | elife-07178-v2.xml  |.xml | always |10.7554/eLife.07178  | always | http://elifesciences.org/content/4/e07178v2  |
| code | code1, fig1-code1, fig1-figsupp3-code1 | elife-00230-fig1-code2-v2.m | .py, .zip, .m | always| 10.7554/eLife.00230.027 | always | http://elifesciences.org/content/2/e00288v4/figure1/figure-supp3/source-code1 |
| data | data1, fig1-data2, fig1-figsupp3-code1-data1 | elife-00230-fig1-figsupp3-code1-data1-v2.zip | .csv, .zip  | always| 10.7554/eLife.06426.014 | always |  http://elifesciences.org/content/2/e00288v4/figure4/source-data1	|
| reporting standards |  repstand1 | elife-00230-repstand1-v2.pdf  | .pdf, .docx | always | 10.7554/eLife.04525.021 | always | http://elifesciences.org/content/2/e00288v4/reporting-standard1 |
| inline media | inf1 | elife-00230-inf2-v2.gif  | .gif, .tex, .mov | never | - | never | - |
| media | fig1-figsupp2-media3, media1, fig2-media2 | elife-00012-fig1-figsupp3-media.mov | .wav, .flac | always | 10.7554/eLife.06426.012 | always | http://elifesciences.org/content/2/e00288v4/media7 |
| supplementary file | supp2, resp-supp2, app3-supp3, dec-fig2-suppfig3-supp1  |  elife-00012-supp2-v2.pdf | .docx, .pdf | always | 10.7554/eLife.01239.011 | always | http://elifesciences.org/content/2/e00288v4/supplemental-file1	|

#### Comment on non-file parents

Any given file can have as a parent one of the following items:

- the base article
- an appendix `app`
- the decision letter `dec`
- the author response `resp`
- another file, particularly a figure or figgure suppliment

#### Comments on some specific file types

**inline media**

- Inline media does not have it's own landing page
- Inline media does need to be served from the cdn
- The tagging for inline media in the XML overlaps with the tagging for inline figures

Example article with inline media: [http://elifesciences.org/content/4/e05169](), This article is an insight, but it
has all four article tabs, and the inline media is an image in the boxed section at the top of the artilce, and the image does not have a DOI.

**media**

Video files are not hosted on our CDN, but any other media file that we publish will need to be hosted on our CDN. For simplicity
we are going to place all media files onto our CDN. We currently have no examples of media that are not video.

**reporting standards**s

[http://elifesciences.org/content/4/e04525/DC5]() is an example of an article with a reporting standard.

**images**

We receive images as .tiff files, and these are converted to gif files as part of the hte publication process. These gif files need to be sent to the CDN.

**data**

[http://elifesciences.org/content/4/e06426/article-data#fig-data-additional-files]() is an example of a published article with associated data.

**code**

[http://elifesciences.org/content/2/e00631]() is an example of an article with source code.

#### Processing file names

The elife bot has an [`ArticleInfo` Class](https://github.com/elifesciences/elife-bot/blob/exp/provider/article_structure.py) that encapsulates the information that we can infer from an artilce ile name.

#### Processing files to the CDN

Currently XML files are sent to the CDN with the [DepositXML Activity](https://github.com/elifesciences/elife-bot/blob/exp/activity/activity_DepositXML.py). we anticipate that this will be extended to send all content to the CDN.

This activity runs as part of the PPP projecss, and takes the location of the CDN from the bot `settings.py` file.

#### CDN path names

The CDN base path is set by the variable

> `ppp_cdn_bucket`  # location of the CDN

in the `settings.py` file of the [elife-bot](https://github.com/elifesciences/elife-bot).

To access a specific file, e.g. `elife-00230-inf2-v2.gif` for a specific article `e00123` in the CDN then you can infer the CDN path from the article number via

> `ppp_cdn_bucket`/00123/elife-00230-inf2-v2.gif

All versions of files for a given article are placed into the same CDN directory:

> `ppp_cdn_bucket`/00123/

So for example in this directory you may have the following

> `ppp_cdn_bucket`/00123/elife-00230-inf2-v1.gif
> `ppp_cdn_bucket`/00123/elife-00230-inf2-v2.gif
> `ppp_cdn_bucket`/00123/elife-00230-inf2-v3.gif

# File naming pattern

>
	`elife-<f-id>-<status>(-<revision><r-id>)(-<asset>)(-<sa-id>)(-<data><d-id>)(-<c-id>)(-<media><m-id>)(-<reporting standard><repstand-id>)(-<supplementary file><supp-id>)|(-v<version>)(-YYYYMMDD).<ext>`

Brackets represent optional components. A pipe represents a choice on component, which may vary depending on it's state in the publishing system.

This pattern covers both inbound zip files and the contents of those zip files. We can break it out into two branches, one for the zip files and one for the assets inside the zip files.

#### Zip File naming pattern

Inbound zip files will be of the form

>
	`elife-<f-id>-<status>(-r<revision>)(-v<version>).zip`

After processing revision numbers will be dropped and zip files will be of the form

>
	`elife-<f-id>-<status>(-v<version>)(-YYYYMMDDHHMMSS).zip`

#### Assets naming pattern

>
	`elife-<f-id>-<status>(-<asset><a-id>)(-<sub-asset><sa-id>)(-<data><d-id>)(-<code><c-id>)(-<media><m-id>)(-<reporting standard><repstand-id>)(-<supplementary file><supp-id>)|(-v<version>).<ext>`

### Components

###### `<f-id>`

This is the file id and is the numerical digit that is used to make up part of the DOI for an article (`<article-id pub-id-type="doi">`). For example an eLife article with the following DOI `/10.7554/eLife.06659` will have an f-id of `06659`.

###### `<status>`

This is either `poa` (publish on accept) or `vor` (version of record).

###### `<asset>`

This refers to an asset file related to an article, ie a figure (fig), source code (code), source data (data), media (includes videos, audio and animation) (media), supplementary file (supp), (figures) the figures pdf, reporting standards (repstand).

NOTE: Change on 27th July 2015:

Decision letter, author response and Appendix are now at the level of assets:
dec - decision letter
resp - author response
app - appendix

dec and resp do not require a number, but app does as there could be multiple appendices.
These assets could have any sub assets as below. They will never exist in the file naming heirachy with no sub assets.

###### `<a-id>`

The `a-id` is the asset id, and indicates the order of an asset in the article. For example an
article with three figures will have the following asset files:

- elife-00012-fig1.tiff
- elife-00012-fig2.tiff
- elife-00012-fig3.tiff

###### `<sub-asset>`

Some assets will have sub-assets, eg figure supplements (figsupp), source data (data), or source code (code). These are indicated by the sub-asset component.

###### `<sa-id>`

Some assets have sub-asset ids, for example, an article with three main figures, where one
figure has three figure supplements, and one figure has two figure supplements will have the following:

- elife-00012-fig1.tiff
- elife-00012-fig1-figsupp1.tiff
- elife-00012-fig1-figsupp2.tiff
- elife-00012-fig1-figsupp3.tiff
- elife-00012-fig2.tiff
- elife-00012-fig3.tiff
- elife-00012-fig3-figsupp1.tiff
- elife-00012-fig3-figsupp2.tiff
- elife-00012-app1-fig1

Some assets only have sub-asset ids, for example, an article with 3 appendices, where appendix 2 has 3 figures, and figure 2 has two figure supplements will have the following:

- elife-00012-app2-fig1.tiff
- elife-00012-app2-fig2.tiff
- elife-00012-app2-fig2-figsupp1.tiff
- elife-00012-app2-fig2-figsupp2.tiff
- elife-00012-app2-fig3.tiff

###### data

Sometimes there is source data at a parent level, but sometimes assets will have source data files associated with them. These data files are indicated by the presence of a `data` in the filename.

###### `<d-id>`

Assets can have multiple source data files associated with them, and these are distinguished with the
data id.

For example, an article with three main figures, where one figure has three figure supplements, and one figure has two figure supplements, and all have associated data, with some of them having multiple data files, could have the following:

- elife-00012-fig1.tiff
- elife-00012-fig1-data1.csv
- elife-00012-fig1-data2.csv
- elife-00012-fig1-figsupp1.tiff
- elife-00012-fig1-figsupp1-data1.csv
- elife-00012-fig1-figsupp2.tiff
- elife-00012-fig1-figsupp2-data1.csv
- elife-00012-fig1-figsupp3.tiff
- elife-00012-fig1-figsupp3-data1.mol
- elife-00012-fig2.tiff
- elife-00012-fig2-data1.txt
- elife-00012-fig3.tiff
- elife-00012-fig3-data.csv
- elife-00012-fig3-figsupp1.tiff
- elife-00012-fig3-figsupp1-data1.csv
- elife-00012-fig3-figsupp1-data2.csv
- elife-00012-fig3-figsupp1-data3.csv
- elife-00012-fig3-figsupp2.tiff
- elife-00012-fig3-figsupp2-data1.csv
- elife-00012-fig3-figsupp3.tiff
- elife-00012-fig3-figsupp3-data1.csv

###### code

Sometimes there is source code at a parent level, but sometimes assets will have source code files associated with them. These source code files are indicated by the presence of a `code` in the filename.

###### `<c-id>`

Assets can have multiple source code files associated with them, and these are distinguished with the
code id.

For example, an article with a top level source code, two main figures, where one figure has source code, and one figure has one figure supplement with source code:

- elife-00012-fig1.tiff
- elife-00012-fig1-code1.py
- elife-00012-fig1-code2.zip
- elife-00012-fig2.tiff
- elife-00012-fig2-figsupp1.tiff
- elife-00012-fig2-figsupp1-code1.zip
- elife-00012-code1.r

##### v`<version>`

When the article finally gets published, it gets assigned a version number for public consumption. These version numbers are incremented only when a new version of the article makes it to the published state on the website. The publishing system has to take on the job of checking on the current version number of any prior published versions of the article, and then appropriately incrementing the version number.

We propose that the publishing system should prepare files to send to the Drupal layer, with
the anticipated version number applied to the file names, and on a successful publishing event
that version of the article bundle is archived with the appropriate version number.

As examples, we might have the following situations:

###### example 1

Article makes it all the way from submission, through first round of content processing to being published, without a PoA version.

Bundle that appears in the publishing system looks like this:

elife-00012-vor-r1.zip contains:
- elife-00012.xml
- elife-00012.pdf
- elife-00012-figures.pdf
- elife-00012-fig1.tiff
- elife-00012-fig1-data1.csv
- elife-00012-fig1-data2.csv
- elife-00012-fig1-figsupp1.tiff
- elife-00012-fig1-figsupp1-data1.csv
- elife-00012-fig1-figsupp2.tiff
- elife-00012-fig1-figsupp2-data1.csv
- elife-00012-fig1-figsupp3.tiff
- elife-00012-fig1-figsupp3-data1.mol
- elife-00012-fig2.tiff
- elife-00012-fig2-data1.txt
- elife-00012-fig3.tiff
- elife-00012-fig3-data.csv
- elife-00012-fig3-figsupp1.tiff
- elife-00012-fig3-figsupp1-data1.csv
- elife-00012-fig3-figsupp1-data2.csv
- elife-00012-fig3-figsupp1-data3.csv
- elife-00012-fig3-figsupp2.tiff
- elife-00012-fig3-figsupp2-data1.csv
- elife-00012-fig3-figsupp3.tiff
- elife-00012-fig3-figsupp3-data1.csv

Note: -r1 could be -r2, r3, r-n. This reflects production processes and is ignored by the publishing platform (with the exception of knowing which zip file to take from the production bucket when ready for publication).

On publishing event files and internal XML links get renamed and updated to (zip retains status indicator):

elife-00012-vor-v1.zip contains:
- elife-00012-v1.xml
- elife-00012-v1.pdf
- elife-00012-figures-v1.pdf
- elife-00012-fig1-v1.tiff
- elife-00012-fig1-data1-v1.csv
- elife-00012-fig1-data2-v1.csv
- elife-00012-fig1-figsupp1-v1.tiff
- elife-00012-fig1-figsupp1-data1-v1.csv
- elife-00012-fig1-figsupp2-v1.tiff
- elife-00012-fig1-figsupp2-data1-v1.csv
- elife-00012-fig1-figsupp3-v1.tiff
- elife-00012-fig1-figsupp3-data1-v1.mol
- elife-00012-fig2-v1.tiff
- elife-00012-fig2-data1-v1.txt
- elife-00012-fig3-v1.tiff
- elife-00012-fig3-data-v1.csv
- elife-00012-fig3-figsupp1-v1.tiff
- elife-00012-fig3-figsupp1-data1-v1.csv
- elife-00012-fig3-figsupp1-data2-v1.csv
- elife-00012-fig3-figsupp1-data3-v1.csv
- elife-00012-fig3-figsupp2-v1.tiff
- elife-00012-fig3-figsupp2-data1-v1.csv
- elife-00012-fig3-figsupp3-v1.tiff
- elife-00012-fig3-figsupp3-data1-v1.csv

We have appended the version number to each file.

The following JSON is sent to the Drupal container (I've redacted specifics in the call to just highlight the components related to the article versions and identifiers)

```json
{
"force": "1",
"title": " ... ",
"impact-statement": " ... ",
"version": "1",
"doi": "10.7554/eLife.00012",
"publish": "1",
"volume": " ... ",
"article-id": "10.7554/eLife.00012",
"article-version-id": "00012.1",
"pub-date": "2014-02-28",
"path": "content/2/e00012",
"article-type": "research-article",
"status": "VOR",
"categories": {},
"keywords": {},
"contributors": [],
"referenced": {},
"related-articles": [],
"children": {},
"citations": {}
}
```

On getting a success message from the Drupal layer that the publishing event has happened then the bundle gets archived under the zip file elife-00012-vor-v1.zip.

###### example 2

Production delivers an article twice to the Drupal site, and the second is the one that gets published.
The first version

#TODO: flesh out this example a bit better.

"Live" version of bundle that enters the publishing system is

elife-00012-vor-r4.zip contains:
- elife-00012.xml
- elife-00012.pdf
- elife-00012-figures.pdf
- elife-00012-fig1.tiff
- elife-00012-fig1-data1.csv
- elife-00012-fig1-data2.csv
- elife-00012-fig1-figsupp1.tiff
- elife-00012-fig1-figsupp1-data1.csv
- elife-00012-fig1-figsupp2.tiff
- elife-00012-fig1-figsupp2-data1.csv
- elife-00012-fig1-figsupp3.tiff
- elife-00012-fig1-figsupp3-data1.mol
- elife-00012-fig2.tiff
- elife-00012-fig2-data1.txt
- elife-00012-fig3.tiff
- elife-00012-fig3-data.csv
- elife-00012-fig3-figsupp1.tiff
- elife-00012-fig3-figsupp1-data1.csv
- elife-00012-fig3-figsupp1-data2.csv
- elife-00012-fig3-figsupp1-data3.csv
- elife-00012-fig3-figsupp2.tiff
- elife-00012-fig3-figsupp2-data1.csv
- elife-00012-fig3-figsupp3.tiff
- elife-00012-fig3-figsupp3-data1.csv

On publishing event files and internal XML links get renamed and updated to:

elife-00012-vor-v1.zip contains:
- elife-00012-v1.xml
- elife-00012-v1.pdf
- elife-00012-figures-v1.pdf
- elife-00012-fig1-v1.tiff
- elife-00012-fig1-data1-v1.csv
- elife-00012-fig1-data2-v1.csv
- elife-00012-fig1-figsupp1-v1.tiff
- elife-00012-fig1-figsupp1-data1-v1.csv
- elife-00012-fig1-figsupp2-v1.tiff
- elife-00012-fig1-figsupp2-data1-v1.csv
- elife-00012-fig1-figsupp3-v1.tiff
- elife-00012-fig1-figsupp3-data1-v1.mol
- elife-00012-fig2-v1.tiff
- elife-00012-fig2-data1-v1.txt
- elife-00012-fig3-v1.tiff
- elife-00012-fig3-data-v1.csv
- elife-00012-fig3-figsupp1-v1.tiff
- elife-00012-fig3-figsupp1-data1-v1.csv
- elife-00012-fig3-figsupp1-data2-v1.csv
- elife-00012-fig3-figsupp1-data3-v1.csv
- elife-00012-fig3-figsupp2-v1.tiff
- elife-00012-fig3-figsupp2-data1-v1.csv
- elife-00012-fig3-figsupp3-v1.tiff
- elife-00012-fig3-figsupp3-data1-v1.csv

We have appended the version number to each file.

The following JSON is sent to the Drupal container (I've redacted specifics in the call to just highlight the components related to the article versions and identifiers)

```json
{
"force": "1",
"title": " ... ",
"impact-statement": " ... ",
"version": "1",
"doi": "10.7554/eLife.00012",
"publish": "1",
"volume": " ... ",
"article-id": "10.7554/eLife.00012",
"article-version-id": "00012.1",
"pub-date": "2014-02-28",
"path": "content/2/e00012",
"article-type": "research-article",
"status": "VOR",
"categories": {},
"keywords": {},
"contributors": [],
"referenced": {},
"related-articles": [],
"children": {},
"citations": {}
}
```

On getting a success message from the Drupal layer that the publishing event has happened then the bundle gets archived under the zip file elife-00012-v1.zip.

###### example 3

A first version gets successfully published. Postpublication three revisions are needed to address
a tricky author issue, and the third revision makes it to version 2 on the website.

The version that exists in the archive and on the website is the following:

elife-00012-vor-v1.zip contains
- elife-00012-v1.xml
- elife-00012-v1.pdf
- elife-00012-figures-v1.pdf
- elife-00012-fig1-v1.tiff
- elife-00012-fig1-data1-v1.csv
- elife-00012-fig1-data2-v1.csv
- elife-00012-fig1-figsupp1-v1.tiff
- elife-00012-fig1-figsupp1-data1-v1.csv
- elife-00012-fig1-figsupp2-v1.tiff
- elife-00012-fig1-figsupp2-data1-v1.csv
- elife-00012-fig1-figsupp3-v1.tiff
- elife-00012-fig1-figsupp3-data1-v1.mol
- elife-00012-fig2-v1.tiff
- elife-00012-fig2-data1-v1.txt
- elife-00012-fig3-v1.tiff
- elife-00012-fig3-data-v1.csv
- elife-00012-fig3-figsupp1-v1.tiff
- elife-00012-fig3-figsupp1-data1-v1.csv
- elife-00012-fig3-figsupp1-data2-v1.csv
- elife-00012-fig3-figsupp1-data3-v1.csv
- elife-00012-fig3-figsupp2-v1.tiff
- elife-00012-fig3-figsupp2-data1-v1.csv
- elife-00012-fig3-figsupp3-v1.tiff
- elife-00012-fig3-figsupp3-data1-v1.csv

Three revisions occur, and the new "live" bundle that comes in to the publishing system contains the following:

elife-00012-vor-r6.zip contains:
- elife-00012.xml
- elife-00012.pdf
- elife-00012-figures.pdf
- elife-00012-fig1.tiff
- elife-00012-fig1-data1.csv
- elife-00012-fig1-data2.csv
- elife-00012-fig1-figsupp1.tiff
- elife-00012-fig1-figsupp1-data1.csv
- elife-00012-fig1-figsupp2.tiff
- elife-00012-fig1-figsupp2-data1.csv
- elife-00012-fig1-figsupp3.tiff
- elife-00012-fig1-figsupp3-data1.mol
- elife-00012-fig2.tiff
- elife-00012-fig2-data1.txt
- elife-00012-fig3.tiff
- elife-00012-fig3-data.csv
- elife-00012-fig3-figsupp1.tiff
- elife-00012-fig3-figsupp1-data1.csv
- elife-00012-fig3-figsupp1-data2.csv
- elife-00012-fig3-figsupp1-data3.csv
- elife-00012-fig3-figsupp2.tiff
- elife-00012-fig3-figsupp2-data1.csv
- elife-00012-fig3-figsupp3.tiff
- elife-00012-fig3-figsupp3-data1.csv

On publishing event files and internal XML links get renamed and updated to:

elife-00012-vor-v2.zip contains:
- elife-00012-v2.xml
- elife-00012-v2.pdf
- elife-00012-figures-v2.pdf
- elife-00012-fig1-v2.tiff
- elife-00012-fig1-data1-v2.csv
- elife-00012-fig1-data2-v2.csv
- elife-00012-fig1-figsupp1-v2.tiff
- elife-00012-fig1-figsupp1-data1-v2.csv
- elife-00012-fig1-figsupp2-v2.tiff
- elife-00012-fig1-figsupp2-data1-v2.csv
- elife-00012-fig1-figsupp3-v2.tiff
- elife-00012-fig1-figsupp3-data1-v2.mol
- elife-00012-fig2-v2.tiff
- elife-00012-fig2-data1-v2.txt
- elife-00012-fig3-v2.tiff
- elife-00012-fig3-data-v2.csv
- elife-00012-fig3-figsupp1-v2.tiff
- elife-00012-fig3-figsupp1-data1-v2.csv
- elife-00012-fig3-figsupp1-data2-v2.csv
- elife-00012-fig3-figsupp1-data3-v2.csv
- elife-00012-fig3-figsupp2-v2.tiff
- elife-00012-fig3-figsupp2-data1-v2.csv
- elife-00012-fig3-figsupp3-v2.tiff
- elife-00012-fig3-figsupp3-data1-v2.csv

The following JSON is sent to the Drupal container (I've redacted specifics in the call to just highlight the components related to the article versions and identifiers)

```json
{
"force": "1",
"title": " ... ",
"impact-statement": " ... ",
"version": "2",
"doi": "10.7554/eLife.00012",
"publish": "1",
"volume": " ... ",
"article-id": "10.7554/eLife.00012",
"article-version-id": "00012.2",
"pub-date": "2014-02-28",
"path": "content/2/e00012",
"article-type": "research-article",
"status": "VOR",
"categories": {},
"keywords": {},
"contributors": [],
"referenced": {},
"related-articles": [],
"children": {},
"citations": {}
}
```

###### `<ext>``

This is the file extention, common extensions are

- zip
- pdf
- xml
- tiff
- csv
- xml
- py
- sql
- docx

### WTFAQs (Wait, These Fiddly Awkard Questions)

#### What happens when only a figure gets updated?

All files associated with an article get updated too. Although this means that we may end up
incrementing files that don't change, what is important is that on publication their file names
now refer to the globally updated version of the published article. This also means that
if we need a new file to replace an existing file, we do not run into file naming conflicts. From the point of view of the CDN we now refer to a new file with a new file name, rather than having to wait for a file replacement to propogate through the CDN, meaning these kinds of changes should happen instantaneiously.

#### How do we update a version number?

We check to see if any version are already published, and if they are, before
publishing, we prepare a new set of file names with the new version number. The
Drupal layer gets notified with a request to publish this new version, and on
reciept of success we allow the system to update it's records to indicate a
new version has been published.

#### What paths need to be retained in the XML to assets?

Some components in the article XML will point to file assets. On creating a new version of
an article thse paths in the XML will need to get updated.

#### How do we update a revision number?

It is up to the content processor to handle revisions, and to send us the correctly
named new revision. Revision numbers do not affect the publishing platform.

#### How do we bootstrap a new instance of the site that respects all previous versioned published articles?

When we archive a package after successfully publishing, then we should archive all versions.
If we bootstrap a new version of the site we need to be able to utilize this archive to
be able to create a new version of the site. Ideally when sending an instruction to the
Drupal layer the publishing API will include indicators of the version that is currently
being published, and the url path on which these versions should be published.

If we loose our archive we will be unable to accurately recreate a previous state of our published
site, and we will have to do a best effort, and publish what we have using the lowest
 version numbers available to us at that time.

 We have an open question about where the date of published versions live. We don't have an answer to that yet.

#### Why are revision numbers not exposed on the final published version?

As revision numbers are no longer tracked in the system, there would be no need to expose them
in the final pubilshed version.

#### How do we update a `run` number in the production system?

Each time we receive a package from the content processor we may end up running an article
through the production system. If any steps in the system fall over we may need to
run the package through the system again. If the Drupal layer fails to publish
we may need to run the package through the system again rather than tracking these
production runs in any file names, these runs will be tracked internally by the production system.
This information will be made available on the production dashboard. Each time a package
comes through the production system, the number of times it has to run through
the production system will be tracked per package, along with whether that package
resulted in a published version. That means that for a given article - elife-01122 we may see the information similar to the following on a production dashboard

	- elife-01122 | POA | 9 runs | version 1
	- elife-01122 | POA | 5 runs | version 2
	- elife-01122 | VOR | 1 runs | version 3
	- elife-01122 | VOR | 4 runs | version 4

#### Why don't we store run number information in the file name?

The first rule of run club, is that we don't talk about run club.

#### Wait, are we really updating all the names of all the files if only one figure gets updated?

Yes, yes, we are really doing that.

#### Do we want to keep track of all the things that happen to a file before it hits the production system, and include a record of those things somehow in the file name?

At this point no.

#### Do we increment VOR version numbers if there was a POA version number before, or do we start over again with a new version number for the article, as it's name has changed to reflect the new status?

We increment the versions from PoA thorugh VoR

#### How do we prevent an old packet of an article entering the publishing system, and creating a new published version?

In good funtimes news for the publishing platform, we have decided that this is the responsability of production. The publishing platform is going to implicity trust production, and will do as it's owers bid. We would like to enable a block in the production system on publishing any v2 or higher content, pending a production manual check, with the ability to remove this block at a later date.

#### The `"article-version-id": "00013.1",` now no longer contains any reference to "VOR", why is that?

The `"article-version-id": "00013.1",` is an id internal to the JSON that gets sent to the Drupal layer and is not part of the file naming convention.

The main requirement from the Drupal side is that articles can be sorted on the article-version-id, so the presence of a `VOR` or `POA` label in this id would require logic in the Drupal layer to
be able to correctly sort the article versions, as the app would have to know which of these
kinds of state of the came before which other kinds of state. If we drop this name in the id, then
the app does not have to carry this logic, and we make the API call to the Drupal layer
less eLife specific, meaning that in future this system could be more easily adopted
by other publishers, or used in other workflows that do not have POA or VOR in them.

#### In this document have you sneakilly concatenated the names of things like figures? Why, and do you have a full specification list?

I have, mainly as I felt that file names for sub components were becoming unweildy. Here is the start of a specification list

- figure -> fig
- figuresupplement -> figsupp
- supplementary file -> supp
- media (includes videos, audio and animation) -> media
- inline-media -> inf
- source code -> code
- source data -> data
- figures (PDF if the figures) -> figures
- reporting standards -> repstand [note: not inu se as at October, 2015]
- decision letter file, for example figure -> dec-fig
- appendix asset file, for example figure (each appendix gets a unique number X) -> appX-fig
- author response asset file, for example figure -> resp-fig

## Tying POA and VOR packets in our workflow and their naming convention.

#### example 1

poa comes in and gets published immediately

package in the production system contains the following (after being generated
	via the POA process.)

elife-00012-poa.zip contains
- elife-00012-poa.xml  (not complete article XML)
- elife-00012-poa.pdf
- elife-00012-poa-supp.zip

On publication these get renamed to:

elife-00012-poa-v1.zip contains
- elife-00012-v1.xml
- elife-00012-v1.pdf
- elife-00012-supp-v1.zip

The following JSON is sent to the Drupal container

```json
{
"force": "1",
"title": " ... ",
"impact-statement": " ... ",
"version": "1",
"doi": "10.7554/eLife.00012",
"publish": "1",
"volume": " ... ",
"article-id": "10.7554/eLife.00012",
"article-version-id": "00012.1",
"pub-date": "2014-02-28",
"path": "content/2/e00012",
"article-type": "research-article",
"status": "POA",
"categories": {},
"keywords": {},
"contributors": [],
"referenced": {},
"related-articles": [],
"children": {},
"citations": {}
}
```

An update is required on the POA articles and in the production system we receive

elife-00012-poa-r1.zip contains
- elife-00012-poa.xml  (not complete article XML)
- elife-00012-poa.pdf
- elife-00012-poa-supp.zip

On publication these get renamed to:

elife-00012-poa-v2.zip contains
- elife-00012-v2.xml  (not complete article XML)
- elife-00012-v2.pdf
- elife-00012-supp-v2.zip

The following JSON is sent to the Drupal container

```json
{
"force": "1",
"title": " ... ",
"impact-statement": " ... ",
"version": "2",
"doi": "10.7554/eLife.00012",
"publish": "1",
"volume": " ... ",
"article-id": "10.7554/eLife.00012",
"article-version-id": "00012.2",
"pub-date": "2014-03-02",
"path": "content/2/e00012",
"article-type": "research-article",
"status": "POA",
"categories": {},
"keywords": {},
"contributors": [],
"referenced": {},
"related-articles": [],
"children": {},
"citations": {}
}
```

On publishing `elife-00012-poa-v2.zip` gets archived.

Later the article goes through the full content processing step and gets a VOR status.

Package comes in to the production system as:

elife-00012-vor-r4.zip contains
- elife-00012.xml
- elife-00012.pdf
- elife-00012-figures.pdf
- elife-00012-fig1.tiff
- elife-00012-fig1-data1.csv
- elife-00012-fig1-data2.csv
- elife-00012-fig1-figsupp1.tiff
- elife-00012-fig1-figsupp1-data1.csv
- elife-00012-fig1-figsupp2.tiff
- elife-00012-fig1-figsupp2-data1.csv
- elife-00012-fig1-figsupp3.tiff
- elife-00012-fig1-figsupp3-data1.mol
- elife-00012-fig2.tiff
- elife-00012-fig2-data1.txt
- elife-00012-fig3.tiff
- elife-00012-fig3-data.csv
- elife-00012-fig3-figsupp1.tiff
- elife-00012-fig3-figsupp1-data1.csv
- elife-00012-fig3-figsupp1-data2.csv
- elife-00012-fig3-figsupp1-data3.csv
- elife-00012-fig3-figsupp2.tiff
- elife-00012-fig3-figsupp2-data1.csv
- elife-00012-fig3-figsupp3.tiff
- elife-00012-fig3-figsupp3-data1.csv

no modifications are needed and on publishing these files get renamed as:

elife-00012-vor-v3.zip contains
- elife-00012-v3.xml
- elife-00012-v3.pdf
- elife-00012-figures-v3.pdf
- elife-00012-fig1-v3.tiff
- elife-00012-fig1-data1-v3.csv
- elife-00012-fig1-data2-v3.csv
- elife-00012-fig1-figsupp1-v3.tiff
- elife-00012-fig1-figsupp1-data1-v3.csv
- elife-00012-fig1-figsupp2-v3.tiff
- elife-00012-fig1-figsupp2-data1-v3.csv
- elife-00012-fig1-figsupp3-v3.tiff
- elife-00012-fig1-figsupp3-data1-v3.mol
- elife-00012-fig2-v3.tiff
- elife-00012-fig2-data1-v3.txt
- elife-00012-fig3-v3.tiff
- elife-00012-fig3-data-v3.csv
- elife-00012-fig3-figsupp1-v3.tiff
- elife-00012-fig3-figsupp1-data1-v3.csv
- elife-00012-fig3-figsupp1-data2-v3.csv
- elife-00012-fig3-figsupp1-data3-v3.csv
- elife-00012-fig3-figsupp2-v3.tiff
- elife-00012-fig3-figsupp2-data1-v3.csv
- elife-00012-fig3-figsupp3-v3.tiff
- elife-00012-fig3-figsupp3-data1-v3.csv

The following JSON is sent to the Drupal container

```json
{
"force": "1",
"title": " ... ",
"impact-statement": " ... ",
"version": "3",
"doi": "10.7554/eLife.00012",
"publish": "1",
"volume": " ... ",
"article-id": "10.7554/eLife.00012",
"article-version-id": "00012.3",
"pub-date": "2014-03-15",
"path": "content/2/e00012",
"article-type": "research-article",
"status": "VOR",
"categories": {},
"keywords": {},
"contributors": [],
"referenced": {},
"related-articles": [],
"children": {},
"citations": {}
}
```

And on successful publising `elife-00012-vor-v3.zip` gets archived.

So the version history of the PDF of this article will look like the following, from oldest to newest, for files have have been published

- elife-00012-v1.pdf  (a POA pdf)
- elife-00012-v2.pdf  (a POA pdf)
- elife-00012-v3.pdf  (a VOR pdf)

gha_language: HTML

gha_license_id: MIT


text:
# mstplot-java

**mstplot-java** is a re-write of the mstplot tool in Java.

This project is for visualizing the minimum spanning trees generated by 'mst'.
It is a simple program to generate a PNG image of a graph.  The input data to
'mstplot' is the output data from 'mst --csv'.  A graph is a set of edges.
An edge is a pair of vertices and a weight.

## File format

Each line describes an edge of two points (x1, y1), (x2, y2) and
a weight (float).  One edge per line.

```
x1\ty1\tx2\ty2\twt\n
```

## Build

ant jar to build build the jar, mstplot to
run it on the data files, ant doc to run javadoc on
the source files, and ant clean to remove generated files.

To Do
-----
* Figure out why graph-010 has a point off the graph.
* Add "doc" target

repository: crithead/mst

repo_name: crithead/mst


text:
# Paperjs Offset
The dicussion to implement a offset function in paper.js started years ago, yet the author have not decided to put a offset feature into the library. So I implement an extension of my own.

As far as I know, the author has promised recently to implement a native offset functionality in near feature, the library will be closed once the native implement is published.

This library implement both path offset and stroke offset, you may offset a path or expand a stroke like what you did in Adobe illustrator. Offset complicate path may cause unwanted self intersections, this library already take care some cases but bugs still exists. Please let me notice the false conditions in the issue pannel so I can correct it.

## Usage
For node development, use
```sh
npm install paperjs-offset
```
And then, in you project:
```javascript
import paper from 'paper'
import { PaperOffset } from 'paperjs-offset'

// call offset
PaperOffset.offset(path, offset, options)

// call offset stroke
PaperOffset.offsetStroke(path, offset, options)
```

You may still use the old way to extend paperjs module, which is **deprecated** and will be removed in future version.
```javascript
import ExtendPaperJs from 'paperjs-offset'
// extend paper.Path, paper.CompoundPath with offset, offsetStroke method
ExtendPaperJs(paper)
```

Or for web development, include the **paperjs-offset.js** or **paperjs-offset.min.js** in demo folder.

The library now exposes a global variable **PaperOffset**, again, the extension of **paper.Path** and **paper.CompoundPath** with offset/offsetStroke functions is still available, but no longer recommended.
```javascript
let path = new paper.Path(/* params */)

PaperOffset.offset(path, 10, { join: 'round' })
PaperOffset.offsetStroke(path, 10, { cap: 'round' })

// deprecated
path.offset(10, { join: 'round' })
// deprecated
path.offsetStroke(10, { cap: 'round' })
```

Both offset/offsetStroke take the form of **f(path: Path | CompoundPath, offset: number, options?: {})**, the options have following parameters:

  **join**: the join style of offset path, you can choose **'miter'**, **'bevel'** or **'round'**, default is **'miter'**.

  **limit**: the limit for miter style (refer the miterLimit's definition in paper.js).

  **cap**: the cap style of offset (only available in offsetStroke), you can choose **'butt'** and **'round'** (**'square'** is not supported yet), default is **'butt'**

  **insert**: whether the result should be insert into the canvas, default is **true**.

## Preview
There are some cases that the library may return weird result or failed silently, please let me noticed in the project issues. And in some cases the library will yeild an ok result than a perfect one. Currently the library should give good results for closed shapes, but may fail in some open curve cases, I'm still working on it.
![Preview](/public/preview.jpg)

You can use open demo folder for simple cases demonstration.

## License
Distributed under the MIT license. See [LICENSE](https://github.com/glenzli/paperjs-offset/blob/master/LICENSE) for detail.

repository: lorenrogers/paperjs-offset

repo_name: lorenrogers/paperjs-offset


text:
# [214] Shortest Palindrome

## Description

[LeetCode Problem Description](https://leetcode.com/problems/shortest-palindrome/description/)

* algorithms
* Hard (29.27%)
* Testcase Example:  '"aacecaaa"'

```md
Given a string s, you are allowed to convert it to a palindrome by adding characters in front of it. Find and return the shortest palindrome you can find by performing this transformation.
Example 1:
Input: "aacecaaa"
Output: "aaacecaaa"
Example 2:
Input: "abcd"
Output: "dcbabcd"

```

## Solution

思路类似于 kmp 算法, 构建 lookup table 跳过已经对比过的字符

[SourceCode](./solution.js)


text:
# Task 5
1. Write the sequence of instructions for the ARM processor, which sums the 32-bit STEV1 and STEV2 variables and writes the result to the 32-bit REZ variable. Repeat the task for the case when the STEV2 variable is subtracted from the STEV1 variable. Also make variations for adding and subtracting 16-bit and 8-bit variables. STEV1 and STEV2 should have initial values 10 (10) and 5 (10) in succession, and always only reserve the space for variable REZ.

2. Write the sequence of instructionss for the ARM processor, which calculates the expression STEV1 = STEV2 + STEV3-STEV1, where STEV1, STEV2, and STEV3 are 32-bit variables with initial values (define them with pseudo-ininstructions): STEV1 = 50 (16) , STEV2 = 100 (10), STEV3 = 2F (16).

3. Write down the instructions(s) for the ARM processor that load the values in registers:

    a) Unsigned load of 32-bit value 0x12345678 into register R1.

    b) Unsigned load of  8-bit value 128 into the R2 register.

    c) Signed load of  8-bit value 128 in the R3 register.

    d) Unsigned load of 16-bit value of 0xF123 to the R4 register.

    e) Signed load of 16-bit 0xF123 value in the R5 register.

Solve all cases by indirect addressing through the base register R0.

repository: mevljas/Computer-Architecture

repo_name: mevljas/Computer-Architecture


text:
# conan-Qt
Work in progress for fully configurable Qt package for conan. Standard config is optimized for low amount of dependencies and most possible amount of optimization.

The design goal was to handle a manual copy of (nearly) all qt configuration options and provide them to the conan options interface.

The default configuration behaves as follows:
 - disable as most as possible (Qt modules and external dependencies)
 - enable as most as possible optimizations
 - prefere the qt provided libs, where needed.

The goal was to create a qt conan package which is small (especially interesting for shared linking) and ships only stuff you really need for your project. This package is actively used for linux compilation and cross-compilation for windows. Sadly it is tested only by a small set of configuration options and the compilation path for mac OS is not present.

repository: cguentherTUChemnitz/conan-Qt

repo_name: cguentherTUChemnitz/conan-Qt


text:
---
title: "The best free Google fonts for your next web project"
description: "If you're looking for free commercial fonts for a web project, Google Fonts is the go-to source. Here are some of my favorite Google Fonts that I regularly use in my work."
slug: the-best-free-google-fonts-for-your-next-web-project
date: 2020-10-19
author: bryan-dugan
cover: ./best-google-fonts.png
tags: ['typography', 'design', 'user-interface']
---

If you're looking for free commercial fonts for a web project, [Google Fonts](https://fonts.google.com/) is my go-to source. Choosing a great font helps set the overall tone for a website or application and makes it look more polished. Here are some of my favorite Google Fonts that I regularly use in my work.

## Headings

These headings, or otherwise known as display fonts, are great to use for headlines that will stand out at large sizes.

[![Playfair Display](./Playfair-Display.png)](https://fonts.google.com/specimen/Playfair+Display)

[![Exo 2](./Exo-2.png)](https://fonts.google.com/specimen/Exo+2)

[![Rubik](./Rubik.png)](https://fonts.google.com/specimen/Rubik)

## Body copy

Body copy should be legible while reading large blocks of text. Some of the fonts listed in this section could also work as a headline but may not be as effective.

[![Merriweather Sans](./Merriweather-Sans.png)](https://fonts.google.com/specimen/Merriweather+Sans)

[![Merriweather](./Merriweather.png)](https://fonts.google.com/specimen/Merriweather)

[![PT Sans](./PT-Sans.png)](https://fonts.google.com/specimen/PT+Sans)

[![PT Serif](./PT-Serif.png)](https://fonts.google.com/specimen/PT+Serif)

[![Libre Baskerville](./Libre-Baskerville.png)](https://fonts.google.com/specimen/Libre+Baskerville)

[![Lora](./Lora.png)](https://fonts.google.com/specimen/Lora)

[![Lato](./Lato.png)](https://fonts.google.com/specimen/Lato)

## Multipurpose

These fonts are usually my go-to fonts as they not only look great at multiple sizes and weights but also complement each other perfectly. When in doubt, choosing one of these fonts is always a safe bet.

[![Inter](./Inter.png)](https://fonts.google.com/specimen/Inter)

[![Montserrat](./Montserrat.png)](https://fonts.google.com/specimen/Montserrat)

[![Libre Franklin](./Libre-Franklin.png)](https://fonts.google.com/specimen/Libre+Franklin)

[![Work Sans](./Work-Sans.png)](https://fonts.google.com/specimen/Work+Sans)

[![Poppins](./Poppins.png)](https://fonts.google.com/specimen/Poppins)

## Selecting fonts yourself

If you're looking to select fonts for yourself on Google Fonts, here are some tips to quickly filter and find great neutral fonts to use for your website or web-app:

1. Under Categories, select "Sans Serif." Sans Serif fonts are most commonly used in web applications. If you're looking for something for a general marketing site, feel free to select Serif as well.
2. Under "Font properties" > "Number of styles" select "10+". A font with more styles is usually more thought out and complete. You may also pick multiple weights or styles as needed.
3. Lastly, in the sorting options select "Most popular" instead of "Trending." Trending fonts are newer fonts on the rise, whereas "Most popular" are the most used.
4. When looking at a font, be sure to read the about section of that font. It usually gives details as to what the fonts best used for and sometimes some good font pairings.

url: https://raw.githubusercontent.com/bryandugan/bryan-dugan/9029b77cd4a480a3f4290f3f961d10aa2d46a059/content/posts/The best free Google fonts for your next web project/the-best-free-google-fonts-for-your-next-web-project.md

repository: bryandugan/bryan-dugan

gha_language: Vue

repo_name: bryandugan/bryan-dugan