text:
[![Build Status](https://travis-ci.org/pharo-contributions/OSSubprocess.png?branch=master)](https://travis-ci.org/marianopeck/OSSubprocess)

# OSSubprocess

OSSubprocess is a software project that allows the user to spawn Operating System processes from within Pharo language. The main usage of forking external OS processes is to execute OS commands (.e.g `cat`, `ls`, `ps`, `cp`, etc) as well as arbitrary shell scripts (.e.g `/etc/myShellScript.sh`) from Pharo.

An important part of OSSubprocess is how to manage standard streams (`stdin`, `stdout` and `stderr`) and how to provide an API for reading and writing from them at the language level.

> It was decided together with Pharo Consortium that as a first step, we should concentrate on making it work on OSX and Unix. If the tool proves to be good and accepted, we could, at a second step, try to add Windows support.
OSSubprocess is still in an exploring phase and so it might be unstable. Use this tool with that in mind. That being said, all tests are green in the tested platforms.

## Table of Contents

  * [OSSubprocess](#ossubprocess)
    * [Table of Contents](#table-of-contents)
    * [Installation](#installation)
    * [Getting Started](#getting-started)
    * [API Reference](#api-reference)
      * [Child exit status](#child-exit-status)
        * [OSSVMProcess and it's child watcher](#ossvmprocess-and-its-child-watcher)
        * [Accessing child status and interpreting it](#accessing-child-status-and-interpreting-it)
      * [Streams management](#streams-management)
        * [Handling pipes within Pharo](#handling-pipes-within-pharo)
        * [Regular files vs pipes](#regular-files-vs-pipes)
        * [Customizing streams creation](#customizing-streams-creation)
        * [Stdin example](#stdin-example)
      * [Synchronism and  how to read streams](#synchronism-and--how-to-read-streams)
        * [Synchronism vs asynchronous runs](#synchronism-vs-asynchronous-runs)
        * [When to process streams](#when-to-process-streams)
        * [Streams processing at the end](#streams-processing-at-the-end)
          * [Semaphore-based SIGCHLD waiting](#semaphore-based-sigchld-waiting)
          * [Delay-based polling waiting](#delay-based-polling-waiting)
          * [Which waiting to use?](#which-waiting-to-use)
        * [Processing streams while running](#processing-streams-while-running)
        * [Asynchronous runs](#asynchronous-runs)
      * [Sending signals to processes](#sending-signals-to-processes)
      * [System shutdown](#system-shutdown)
      * [Environment variables](#environment-variables)
        * [Setting environment variables](#setting-environment-variables)
        * [Variables are not expanded](#variables-are-not-expanded)
        * [Accessing environment variables](#accessing-environment-variables)
        * [Inherit variables from parent](#inherit-variables-from-parent)
      * [Shell commands](#shell-commands)
      * [Setting working directory](#setting-working-directory)
    * [Running the tests](#running-the-tests)
    * [Contributing](#contributing)
    * [History](#history)
    * [Future work](#future-work)
    * [Authors](#authors)
    * [License](#license)
    * [Acknowledgments](#acknowledgments)
    * [Funding](#funding)

## Installation
**Currently, OSSubprocess only works in Pharo >= 5.0 with Spur VM**. In any case, we recommend to always grab a latest image and VM. You can do that in via command line:

```bash
wget -O- get.pharo.org/alpha+vm | bash
```

Then, from within Pharo, execute the following to install OSSubprocess:

```Smalltalk
Metacello new
 	configuration: 'OSSubprocess';
 	repository: 'github://pharo-contributions/OSSubprocess:master/repository';
	version: #stable;
	load.
```

> Important: Do not load OSProcess project in the same image of OSSubprocess because the latter won't work.

> Important2: If you are installing under Linux, then you must use a threaded heartbeat VM (not the itimer one). For Pharo 5.0 and 6.0 you can search for "cog_linux32x86_pharo.cog.spur_XXXXXXXXXXXX.tar.gz" i32 [Pharo static file server](http://files.pharo.org/vm/pharo-spur32/linux/). Since Pharo 7.0, threaded heartbeat VM has become the default installation, so you shouldn't have to explicitly download a specific VM.

Besides the above installation instructions, OSSubprocess can also be installed from the `Catalog Browser`, already present in Pharo. Just open it, search for OSSubprocess, then right click, `Install stable version`.

## Getting Started
OSSubprocess is quite easy to use but depending on the user needs, there are different parts of the API that could be used. We start with a basic example and later we show more complicated scenarios.

```Smalltalk
OSSUnixSubprocess new
	command: '/bin/ls';
	arguments: #('-la' '/Users');
	redirectStdout;
	runAndWaitOnExitDo: [ :process :outString  |
		outString inspect
	]
```

Until we add support for Windows, the entry point will always be OSSUnixSubprocess, which should work in OSX, Linux and others Unix-like. You can read it's class comments for details.

A subprocess consist of at least a command/binary/program to be executed (in this example `/bin/ls`) plus some optional array of arguments.

The `#command:` could be either the program name (.e.g `ls`) or the full path to the executable (.e.g `/bin/ls`). If the former, then the binary will be searched using `$PATH` variable and may not be found.

For the `#arguments:` array, each argument must be a different element. In other words, passing `#('-la /Users')` is not correct since those are 2 arguments and hence should be 2 elements of the array. It is also incorrect to not specify `#arguments:` and specify the command like this: `command: '/bin/ls -la /Users'`. OSSubprocess does *not* do any parsing of the command or arguments. If you want to execute a command with a full string like `/bin/ls -la /Users`, you may want to take a look to `#bashCommand:` which relies on shell to do that job.

With `#redirectStdout` we are saying that we want to create a stream and that we want to map it to `stdout` of the child process. Since they are not specified, `stderr` and `stdin` will then be inherit from the parent process (Pharo VM process). If you comment out the line of `#redirectStdout` and run the example again, you can see how the output of `/bin/ls -la /Users` is printed in the terminal (where you launched your Pharo image).

Finally, we use the `#runAndWaitOnExitDo:` which is a high level API method that runs the process, waits for it until it finishes, reads and then closes the `stdout` stream, and finally invokes the passed closure. In the closure we get as arguments the original `OSSUnixSubprocess` instance we created, and the contents of the read `stdout`. If you inspect `outString` you should see the output of `/bin/ls -la /Users` which should be exactly the same as if run from the command line.

## API Reference

### Child exit status
When you spawn a process in Unix, the new process becomes a "child" of the "parent" process that launched it. In our case, the parent process is the Pharo VM process and the child process would be the one executing the command (in above example, `/bin/ls`). It is a responsibility of the parent process to collect the exit status of the child once it finishes. If the parent does not do this, the child becomes a "zombie" process. The exit status is an integer that represents how the child finished (if successful, if error, which error, if received a signal, etc etc.). Besides avoiding zombies, the exit status is also important for the user to take actions depending on its result.

#### OSSVMProcess and it's child watcher

In OSSubprocess, we have a class `OSSVMProcess` with a singleton instance accessed via a class side method `vmProcess` which represents the operating system process in which the Pharo VM is currently running. OSSVMProcess can answer some information about the OS process running the VM, such as running PID, children, etc etc. More can be added later.

Another important task of this class is to keep track of all the launched child processes (instances of `OSSUnixSubprocess`). Whenever a process is started it's registered in OSSVMProcess and unregister in certain scenarios (see senders of ``#unregisterChildProcess:``). We keep a  list of all our children, and occasionally prune all those that have already exited.

This class takes care of running what we call the "child watcher" which is basically a way to monitor children's status and collect exit code when they finish. This also guarantees not to create zombies processes. As for the implementation details, we use a SIGCHLD handler to capture a child death. For more details, see method `#initializeChildWatcher`.

*What is important here is that whether you wait for the process to finish or not (and no matter in which way you wait), the child exit code will be collected and stored in the `exitStatus` instVar of the instance of `OSSUnixSubprocess` representing the exited process, thanks to the `OSSVMProcess` child watcher.*

#### Accessing child status and interpreting it
No matter how you waited for the child process to finish, when it exited, the instVar `exitStatus` should have been set. `exitStatus` is an integer bit field answered by the `wait()` system call that contains information about the exit status of the process. The meaning of the bit field varies according to the cause of the process exit. Besides understanding `#exitStatus`, `OSSUnixSubprocess` also understands `#exitStatusInterpreter` which answers an instance of `OSSUnixProcessExitStatus`. The task of this class is to simply decode this integer bit field and provide meaningful information. Please read its class comment for further details.

In addition to `#exitStatus` and `#exitStatusInterpreter`, `OSSUnixSubprocess` provides testing methods such us `#isSuccess`, `#isComplete`, `isRunning`, etc.

Let's see a possible usage of the exit status:

```Smalltalk
OSSUnixSubprocess new
	command: '/bin/ls';
	arguments: #('-la' '/nonexistent');
	redirectStdout;
	redirectStderr;
	runAndWaitOnExitDo: [ :process :outString :errString |
		process isSuccess
			ifTrue: [ Transcript show: 'Command exited correctly with output: ', outString. ]
			ifFalse: [
				"OSSUnixProcessExitStatus has a nice #printOn: "
				Transcript show: 'Command exit with error status: ', process exitStatusInterpreter printString; cr.
				Transcript show: 'Stderr contents: ', errString.
			]
	]
```

In this example we execute `/bin/ls` passing a none existing directory as argument.

First, note that we add also a stream for `stderr` via `#redirectStderr`. Second, note how now in the `#runAndWaitOnExitDo:` we can also have access to `errString`. If you run this example with a `Transcript` opened, you should see something like:

```
Command exit with error status: normal termination with status 1
Stderr contents: ls: /nonexistent: No such file or directory
```

The `normal termination with status 1` is the information that `OSSUnixProcessExitStatus` (accessed via `#exitStatusInterpreter`) decoded for us. "Normal termination" means it was not signaled or terminated, but rather a normal exit. The exit status bigger than zero (in this case, 1), means error. What each error means depends on the program you run.

The second line of the `Transcript` is simply the `stderr` contents.

### Streams management
There are many parts of the API and features of OSSubprocess which are related to streams. In this section we will try to explain these topics.

#### Handling pipes within Pharo
Besides regular files (as the files you are used to deal with), Unix provides **pipes**. Since Unix manages pipes  polymorphically to regular files, you can use both for mapping standard streams (`stdin`, `stdout` and `stderr`). That means that both kind of files can be used to communicate a parent and a child process.

For regular files, Pharo already provides `Stream` classes to write and read from them, such as `StandardFileStream` and subclasses. For pipes things are different because there is a write end and a read end (each having a different file descriptor at Unix level). For this purpose, we have implemented the class called `OSSPipe` which represents a OS pipe and it is a subclass of `Stream`. `OSSPipe` contains both, a `reader` and a `writer`, and it implements the `Stream` protocol by delegating messages to one or the other. For example `OSSPipe>>#nextPutAll:` is delegated to the writer while `#next` is delegated to the reader. That way, we have a Stream-like API for pipes. For more details, please read the class comment of `OSSPipe`.

The question is now what type of streams are the 'reader' and 'writer' instVars of a `OSSPipe`. As I said before, the write end and read end of a pipe have different file descriptor at Unix level, hence they are seem as two different file streams. When we create a pipe via a system call, we directly get both file descriptors. That means that in Pharo, once we created a OS pipe, we already have both, the read and the write. Ideally, we could use `StandardFileStream` for this case too. However, those file streams were *already opened* by the system call to make the pipes. To solve this issue, we created what we call `OSSAttachableFileStream` which is a special subclass of `StandardFileStream` and that can be *attached* to an already existing and opened file.

To conclude, we use a system call to create an OS pipe. At Pharo level we represent that pipe as an instance of `OSSPipe`. And a `OSSPipe` has both, a reader and a writer which are both instances of `OSSAttachableFileStream` and that have been attached to the reader and writer end of the pipe.

#### Regular files vs pipes
As we said before, both regular files or pipes can be used for mapping standard streams (`stdin`, `stdout` and `stderr`) and OSSubprocess supports both and manages them polymorphically thanks to `OSSPipe` and `OSSAttachableFileStream`.

> **Important:** For regular files, you **must** use instances of `StandardFileStream` since we have some problems when using instances of `MultiByteFileStream`. So... try not to use the `FileSystem` library for creating file streams but rather `StandardFileStream` directly.

The user can decide to use one or another. Pipes are normally faster since they run in memory. On the contrary, files may do I/0 operations even with caches (at least creating and deleting the file). With pipes you do not have to handle the deletion of the files as you do with regular files. You can read more about "regular files vs pipes" in the internet and come yourself to a conclusion.

> **Important** We have found some problems when using regular files for the `stdin`. While we do not strictly forbid that, we recommend you do so only if you know very well what you are doing. Otherwise, use blocking pipes for `stdin` (default behavior).

> **Important** Our recommendation as a rule of thumb is to always use pipes rather than regular files. That is, none-blocking pipes for `stdout` and `stderr` and blocking pipes for `stdin`.

There is only one problem with pipes that you should be aware of and it's the fact that you may get a deadlock in certain situations. See [Semaphore-based SIGCHLD waiting](#semaphore-based-sigchld-waiting) for more details.

#### Customizing streams creation
Let's consider this example:

```Smalltalk
| process |
process := OSSUnixSubprocess new
			command: '/bin/ls';
			arguments: #('-la' '/nonexistent');
			defaultWriteStreamCreationBlock: [OSSVMProcess vmProcess systemAccessor makeNonBlockingPipe];
			redirectStdout;
			redirectStderrTo: (StandardFileStream forceNewFileNamed: '/tmp/customStderr.txt');
			defaultReadStreamCreationBlock: [ OSSUnixSubprocess createTempFileToBeUsedAsReadStreamOn: '/tmp' ];
			createMissingStandardStreams: true.
Halt halt.
process runAndWait.
process isSuccess
	ifTrue: [ Transcript show: 'Command exited correctly with output: ', process stdoutStream upToEndOfFile. ]
	ifFalse: [
		"OSSUnixProcessExitStatus has a nice #printOn: "
		Transcript show: 'Command exit with error status: ', process exitStatusInterpreter printString; cr.
		Transcript show: 'Stderr contents: ', process stderrStream upToEndOfFile.
	].
process closeAndCleanStreams.
```

There are many things to explain in this example. First of all, we are not using the API `#runAndWaitOnExitDo:` and so certain things must be done manually (like retrieving the contents of the streams via `#upToEndOfFile` and closing and cleaning streams with `#closeAndCleanStreams`). Now you get a better idea of what `#runAndWaitOnExitDo:` does automatically for you. The reason we are not using `#runAndWaitOnExitDo:` and instead the low level API, is to have a `Halt halt` while running the process so that you can confirm yourself the existence of the files used for the streams, as explained next.

With the methods `#redirectStdinTo:`, `#redirectStdoutTo:` and `#redirectStdoutTo:` the user is able to set a custom stream for each standard stream. The received stream could be either a `StandardFileStream` (as is the result of `StandardFileStream forceNewFileNamed: '/tmp/customStderr.txt'`) or a `OSSPipe` (as is the result of `OSSVMProcess vmProcess systemAccessor makeNonBlockingPipe`).

If you do not want to create a custom stream (like above example of `#redirectStderrTo:`) but you still want to create a default stream that would be mapped to a standard stream, then you can use the methods `#redirectStdin:`, `#redirectStdout` and `#redirectStderr`. Those methods will create *default* streams. The *default* streams are defined by the instVars `defaultWriteStreamCreationBlock` and `defaultReadStreamCreationBlock`. And these can be customized too as shown in above example. In this example, all write streams (to be used for `stdout` and `stderr`) will be pipes (forget for the moment about the none blocking part). And all read streams (only used by `stdin`) will be temp files automatically created in `/tmp`. This feature is useful if you want to change the way streams are created without having to specially create each stream. You can see how `OSSPipeBasedUnixSubprocessTest` and `OSSFileBasedUnixSubprocessTest` use these in the method `#newCommand`.

*If you see `OSSUnixSubprocess>>#initialize` you will see that by default we create pipes for all type of streams.*

Previously, we said that if the user does not specify a stream to be mapped to one of the standard streams (by any of the means explained above), the child will inherit the one of the parent process (VM Process). With the method `#createMissingStandardStreams:` one can change that behavior, and instead, create default streams for all none already defined streams. In above example, the user defined `stdout` via `#redirectStdout` and `stderr` via `#redirectStderrTo:`. So, by doing `#createMissingStandardStreams: true`, a default stream (pipe) will be created and mapped to the missing streams (only `stdin` in this example). Of course, if you are happy with default streams creation, you can avoid having to declare each stream and simple enable all. But keep in mind the costs of managing streams that you may not use at all. For this reason, the default value of `createMissingStandardStreams` is `false`.

Finally, note that since we are not using the API `#runAndWaitOnExitDo:` in this case, we must explicitly close streams via the message `#closeAndCleanStreams`. This method will also take care of deleting all those streams which were regular files (not pipes).

#### Stdin example
All of the examples so far showed how to redirect `stdout` and `stderr`. Below is an example on how can we redirect `stdin` and write to it from Pharo:

```Smalltalk
| process |
process := OSSUnixSubprocess new
				command: '/bin/cat';
				redirectStdin;
				redirectStdout;
				run.
process stdinStream
		nextPutAll: 'we are testing!';
		nextPutAll: 'we again!';
		close.
process waitForExit.
process stdoutStream upToEndOfFile inspect.
process closeAndCleanStreams.
```

If you run above example you will see how the strings are first written to the `stdin` of the child process and then the child writes them to `stdout` (that's what the `cat` command does). We finally read them from `stdout`.

Note that the `#close` we send to the `stdinStream` is very important. This is because by default, we use blocking pipes for `stdin` and that means that when the child tries to read he will be locked until data is available. Therefore, somehow, at some point, we must tell the child that we have finished writing and that is via the `#close` message.

### Synchronism and  how to read streams

####  Synchronism vs asynchronous runs
We call synchronous runs when you create a child process and you wait for it to finish before going to the next task of your code. This is by far the most common approach since many times you need to know which was the exit status (wether it was success or not) or get some output, and depending on that, change the flow of your code.

Asynchronous runs are when you run a process and you do not care it's results for the next task of your code. In other words, your code can continue its execution no matter what happened with the spawned process.

Let's see this example:

```Smalltalk
| process |
process := OSSUnixSubprocess new
				command: '/bin/ls';
				run.
process waitForExit.
process isSuccess
	ifTrue: [ self doSomethingWithSucess ]
	ifFalse: [ self doSomethingWithFailure ].
process closeAndCleanStreams.
self myNextCodeAction.
```

`myNextCodeAction` represents the user code (user of this library) that will run after spawning the process. The important question here is when should the line `self myNextCodeAction` be executed? Does `myNextCodeAction` depend on the exit status of the forked process or it needs some of its output?

If `myNextCodeAction` should be executed after the forked process has finished, then it is synchronous. Otherwise, asynchronous.

**The way the previous and next examples are written are actually synchronous. In [Asynchronous runs](#asynchronous-runs) we see how can we make asynchronous calls.**

#### When to process streams
If we have defined that we wanted to map a standard stream, it means that at some point we would like to read/write it and **do something with it**.
There are basically two possibilities here. The most common approach is to simply wait until the process has finished, and once finished, depending on the exit status, **do something** with the stream contents. What to do exactly, depends on the needs the user has for that process.

The other possibility a user may do, is to define some code that loops while the process is still running. For every cycle of the loop, it retrieves what is available from the streams and do something with it. But in this case, **doing something** is not simply accumulating the stream contents in another stream so that it is processed at the end. By doing something we really mean that the user wants to do something with that intermediate result: print it in a `Transcript`, show a progress bar, or whatever.

#### Streams processing at the end
For the scenario in which streams processing should be done once a process has exited, we provide some facilities methods that should ease the usage. Processing the streams at the end also means that somehow we should wait for the process to finish. We provide two ways of doing this.

##### Semaphore-based SIGCHLD waiting
The first way is with the method `#waitForExit`. This method is used by the high level API methods `#runAndWait` and `#runAndWaitOnExitDo:`. The wait in this scenario does **not** use an image-based delay polling. Instead, it uses a semaphore. Just after the process starts, it waits on a semaphore of the instVar `mutexForSigchld`.  That method, will do the `#signal` in the semaphore and hence the execution of `#waitForExit` will continue.

> **IMPORTANT** In this kind of waiting there is no polling and hence we do not read from the streams periodically. Instead, we read the whole contents of each stream at the end. Basically, there is a problem in general (deadlock!) with waiting for an external process to exit before reading its output. If the external process creates a large amount of output, and if the output is a pipe, it will block on writing to the pipe until someone (our VM process) reads some data from the pipe to make room for the writing. That leads to cases where we never read the output (because the external process did not exit) and the external process never exits (because we have not read the data from the pipe to make room for the writing).

> Another point to take into account is that this kind of wait also depends on the child watcher and `SIGCHLD` capture. While we do our best to make this as reliable as possible, it might be cases where we miss some `SIGCHLD` signals. There is a workaround for that (see method `#initializeChildWatcher`), but there may still be issues. If such problem happens, the child process (at our language level) never exits from the `#waitForExit` method. And at the OS level, the child would look like a zombie because the parent did not collect the exit status.

##### Delay-based polling waiting
The other way we have for waiting a child process is by doing a delay-based in-image polling. That basically means a loop in which we wait some milliseconds, and then check the exit status of the process. For this, we provide the method `#waitForExitPollingEvery:retrievingStreams:`, and it's high level API method `#runAndWaitPollingEvery:retrievingStreams:onExitDo:`.

It is important to note that as part of the loop we send `#queryExitStatus`, which is the one that sends the `waitpid()` in Unix. That means that we are fully independent of the child watcher and the `SIGCHLD` and hence much more reliable than the previous approach.

In addition, we provide a `retrievingStreams:` boolean which if true, it reads from streams as part of the loop. This solves the problem mentioned before where there could be a deadlock if there is no room in the pipe for the writing.

Below is an example of delay polling:

```Smalltalk
OSSUnixSubprocess new
	command: '/bin/ls';
	arguments: #('-la' '/Users');
	redirectStdout;
	runAndWaitPollingEvery: (Delay forMilliseconds: 50) retrievingStreams: true onExitDo: [
		:process :outString  |
		outString inspect
	]
```

Of course, the main disadvantage of the polling is the cost of it at the language level. The waiting on a semaphore (as with the `SIGCHLD`) has no cost.

##### Which waiting to use?
If you are using pipes (which is our recommendation) and you are not sure about how much the child process could write in the streams, then we recommend the polling approach (with `retrievingStreams:` with `true`).

If you are using pipes and you know the process will not write much, then you can use both approaches. If you want to be sure 100% to never ever get a zombie or related problem, then we think the polling approach is a bit more reliable. If you can live with such a possibility, then maybe the `SIGCHLD` approach could work too.

If you are using files instead of pipes, then it would depend on how long does the process take. If it is normally short, then the polling approach would be fine. If it is a long process then you may want to do the `SIGCHLD` approach to avoid the polling costs. But you can also use the polling approach and specify a larger timeout.

*As you can see, there is no silver bullet. It's up to the user to know which model would fit better for his use-case. If you are uncertain or you are not an expert, then go with the polling approach with `retrievingStreams:` on `true`.*

#### Processing streams while running
Instead of processing the streams once the process has finished, the other alternative is to process the streams while the process runs.

To demonstrate the facilities we provide for this case, we will see an example in which we invoke the command `tail -f /var/log/system.log`. In this example we use `/var/log/system.log` because it's a file whose contents grows frequently on OSX, but the example could apply to any file. What the example does is to poll with a delay and in every cycle of the loop, read from `stdout` and append that into a `Pharo Playground`:

```Smalltalk
| streamsInfo totalStdout page playground |
totalStdout := String new writeStream.
"These first lines just open a Pharo Playground with an initial content"
(page := GTPlayPage new)
	saveContent: 'initial content'.
(playground := GTPlayground new)
	openOn: page.
[
	OSSUnixSubprocess new
	command: 'tail';
	arguments: #('-f' '/var/log/system.log' );
	redirectStdout;
	redirectStderr;
	runAndWaitPollingEvery: (Delay forMilliseconds: 500)
	doing: [ :process :outStream :errStream |
		| read |
		read := outStream upToEnd.
		"Next 2 lines is to simply update the Playground"
		page saveContent: (page content, read).
		playground update.
		totalStdout nextPutAll: read.
		errStream upToEnd.
	]
	onExitDo: [ :process :outStream :errStream  |
		process closeAndCleanStreams.
		Transcript show: 'Total stdout: ', totalStdout contents.
	]
] fork.
```

If you run above code, you will see how a Pharo Playground is opened and every half second the contents are refreshed and appended at the end. The most important method here is `runAndWaitPollingEvery:doing:onExitDo:` which is the facility we provide for these scenarios. However, there are other points to mention:

* The user must explicitly close streams. It could be inside the `onExitDo:` or elsewhere, but must be done. We cannot automatically close streams because we don't know if the user has retrieved their contents or not (we cannot know what the user will do in the `doing:` closure).
* If using pipes, the user must be sure to either retrieve steam contents as part of the `doing:` block or else make sure the process will not be writing much on them. Otherwise, you may hit the deadlock mentioned in [Semaphore-based SIGCHLD waiting](#semaphore-based-sigchld-waiting).
* Contrary to regular files, the read of a pipe **consumes** the read. That is, you cannot read more than once a particular content. So for example, say you read from `stdout` in the `doing:` closure and then in `onExitDo:` you would like to do something with the total of the `stdout` (in this example, print it in the `Transcript`). If you read from `stdout` in `onExitDo:`, in this example, you will get nothing, because it was already read and consumed. To avoid loosing what you read (in case you need it later), you must store it somewhere yourself (in this example in the temp variable `totalStdout`).
* So far in this guide we have always used `upToEndOfFile` to retrieve streams contents. Sending such message is ideal once the process has exited. But if it is running and still writing into the standard streams, then the message to send is `upToEnd` which simply reads all available data in the stream. And that does not necessary mean the end of the file or the pipe (the child process may have written even more on it).
* For this type of "Processing streams while running" we strongly recommend pipes over regular files.
* If we run above code without the `fork` it will work but the Pharo UI will not be refreshed and it won't display the new Playground until the process has stopped. This could be considered as an example of asynchronous calls as explained in [Asynchronous runs](#asynchronous-runs).

####  Asynchronous runs
The previous example of the `tail -f` was an asynchronous run. Why? Because we wanted the user code to continue running (in this case the Pharo UI showing us the Playground with the `tail` contents) regardless of what happened with the spawned process. In general, the way to launch asynchronous processes is like this:

```Smalltalk
[
OSSUnixSubprocess new
	command: 'whatever';
	arguments: #();
	...
] fork.
self continueWithOtherCode.
```

### Sending signals to processes
OSSubprocess provides a way of sending UNIX signals (`SIGKILL`, `SIGTERM`, etc.) to the spawned process. There are many scenarios in which this is useful, such us terminating or killing an existing process for whatever reason you have. Some OS commands, like `tail -f`, do not even finish unless you explicitly tell it to do so. If we consider again the `tail -f` example you may now realize that that process will run forever.

Which signals to send and when it completely up to the user. Under the protocol `OS signal sending`, the class `OSSUnixSubprocess` provides one method per signal. Examples: `#sigterm`, `#sigkill`, `#sigint`, `#sighup`, etc. Each of those methods will simply send the signal in question to the external process.

See this example:

```Smalltalk
| counter |
counter := 0.
[
	OSSUnixSubprocess new
	command: 'tail';
	arguments: #('-f' '/var/log/system.log' );
	redirectStdout;
	runAndWaitPollingEvery: (Delay forMilliseconds: 500)
	doing: [ :process :outStream  |
		counter := counter + 1.
		Transcript show: outStream upToEnd; cr.
		counter > 10 ifTrue: [ process sigterm ]
	]
	onExitDo: [ :process :outStream :errStream  |
		process closeAndCleanStreams.
		Transcript show: 'Process has exited with status: ', process exitStatusInterpreter printString; cr.
	]
] fork.
```

If you open a `Transcript` and execute above example, you will see how the changes to `/var/log/system.log` are displayed there for approx. 5 seconds and then it stops. It should have also printed `Process has exited with status: exit due to signal 15`, demonstrating that the exit is managed correctly. In addition, it shows up how the `exitStatusInterpreter` correctly interprets that the exit of the process was due to signal 15, which is `SIGTERM`.

**Again, when and which signals to send depends on the user.**

### System shutdown
What would happen if you are running an asynchronous process (with `#fork`) and Pharo quits? Pharo can quit because of a crash, because the user accidentally quits it, or any other reason.

By default, when Pharo is quitting, the `OSSVMProcess vmProcess` tells all active children to `#stopWaiting`. The `#stopWaiting` basically makes children to stop waiting for the external process and just finish. This allows the external process to finish (at the OS level), even if Pharo has quitted. In this scenario, its likely the child to become an *orphan* process since it's parent has die (Pharo VM). Orphan processes do what is called  `re-parenting` which basically means they become children of the `init` process.

Contrary to the default behavior, we also provide the method `OSSUnixSubprocess >> terminateOnShutdown` which tells the system to automatically **terminate** (via `sigterm`) the external process when Pharo is quitting. We also collect the exit status in this case.

It depends on the user which behavior he wants.

### Environment variables

#### Setting environment variables

One common requirement for child processes is to define environment variables for them, that may not even exist in the parent process. Let's see an example:

```Smalltalk
OSSUnixSubprocess new
	command: '/usr/bin/git';
	arguments: #('-C' '/Users/mariano/git/OSSubprocess' 'commit');
	environmentAt: 'GIT_EDITOR' put: '/Users/mariano/bin/mate';
	redirectStdout;
	runAndWaitOnExitDo: [ :process :outString  |]
```

In this example we are executing `/usr/bin/git` and we are telling it to commit from the directory `/Users/mariano/git/OSSubprocess`. Without any particular environment variable setting, `git` will use the default editor. In my machine, it will use `nano` editor. However, in above example, we are setting a environment variable called `GIT_EDITOR` (which is *not* set in parent process) with the value `/Users/mariano/bin/mate` (TextMate editor for OSX). When we run this example, `git` will then use TextMate for entering the commit message instead of `nano`.

#### Variables are not expanded
You should not confuse the above example with variables expansions. Consider now this example:

```Smalltalk
OSSUnixSubprocess new
	command: '/bin/echo';
	arguments: #('${HOME}');
	environmentAt: 'HOME' put: 'hello';
	redirectStdout;
	runAndWaitOnExitDo: [ :command :outString |
		outString inspect
	].
```

What do you expect to be the `outString`? Well, the answer is the literal string `${HOME}`. This is because OSSubprocess does *not* expands variables. It does *set* variables, but that's not the same as expanding them.

Variable expansion is a responsibility of the shell, not of the command. Specific commands will of course consult their environment, but only for variables that they are designed to use (e.g. `git` will use $GIT_EDITOR or fall back to $EDITOR when you have to type a commit message).

To conclude this topic, see this example:

```Smalltalk
OSSUnixSubprocess new
	command: '/bin/sh';
	arguments: #('-c' 'echo ${HOME}');
	environmentAt: 'HOME' put: 'hello';
	redirectStdout;
	runAndWaitOnExitDo: [ :command :outString |
		outString inspect.
	].
```

In this case, the `outString` is indeed `hello` and this is simply because `/bin/sh` did the variable expansion before executing the `echo`.

#### Accessing environment variables
It seems some people use OSProcess (or maybe even OSSubprocess) to access environment variables. **This is not needed and it's not the best approach**. Pharo provides an out of the box way for accessing environment variables. For example, `Smalltalk platform environment at: 'PWD'` answers `/Users/mariano/Pharo/imagenes`. So, use that API for retrieving environment variables.

Finally, not that you can "expand" yourself the value of a variable prior to spawning a process and hence avoid the need of the shell. example:

```Smalltalk
OSSUnixSubprocess new
	command: '/bin/echo';
	arguments: (Array with: (Smalltalk platform environment at: 'HOME'));
	redirectStdout;
	runAndWaitOnExitDo: [ :command :outString |
		outString inspect
	].
```

In this example, `outString` will be `/Users/mariano`.

#### Inherit variables from parent
If you don't define any variable via `#environmentAt:put:`, then by default, we will make the child process inherit all the variables from the parent process. If you did define at least one variable, then by default the child inherits no variable. However, you have the method `#addAllEnvVariablesFromParentWithoutOverride` which basically adds all the variables inherit from the parent but those that you have explicitly set (if any).

### Shell commands
Sometimes, the process you want to execute is a shell. There are a couple of reasons for doing so:

* Variables expansion.
* Pipelines.
* When there is already logic in `.bash_profile`, `.profile` or such kind of file.
* Command path resolution.
* Streams redirection.
* Available shell scripts that you want to execute.
* Convenience when you don't need to have full control.

Quite some of that list can easily be done without the need of a shell, as already explained. But some of them, still need the shell. To avoid having to invoke a shell as seen in [Variables are not expanded](#variables-are-not-expanded), we provide the methods `#shellCommand:` and `#shell:command:`. Example:

```Smalltalk
OSSUnixSubprocess new
	shellCommand: 'ps -fea | grep Pharo > /tmp/testShellCommandWithStreamRedirects.deleteme';
	redirectStdout;
	runAndWaitOnExitDo: [ :command :outString |
		outString inspect.
	].
```

Of course, `outString` is empty now because the shell has redirected it to a particular file.

`shellCommand:` will first try to use the shell defined in the OS by getting the env variable `$SHELL`. If not defined, then it will fallback to `/bin/sh`. `#shell:command:` is similar to `#shellCommand:` but allows the user to specify the full path of the shell to use (for example `/bin/zsh`).

> Note that when using the shell you must type the command exactly as the shell expects it, that is, with all the escaping and everything needed.

### Setting working directory
Another common need when spawning processes is to set a working directory (`PWD`) for them. If none is set, by default, they inherit the working directory of the parent. Imagine the Pharo image was launched from a terminal while `PWD` was `/Users/mariano`. Now imagine I want to spawn a process for a git commit for the directory `/Users/mariano/git/OSSubprocess`. For this to work, I must specify the working directory for the child. Otherwise, the `commit` will be triggered in `/Users/mariano`.

To support this, we provide the method `#workingDirectory:` as this example shows:

```Smalltalk
OSSUnixSubprocess new
	command: '/usr/bin/git';
	arguments: #('commit' '-m' 'testing');
	workingDirectory: '/Users/mariano/git/OSSubprocess';
	redirectStdout;
	redirectStderr;
	runAndWaitOnExitDo: [ :process :outString :errString |
		errString inspect.
		outString inspect.
		]
```

> The implementation of `#workingDirectory::` is quite rudimentary and bad from performance point of view (read the method comment for details). If the program you are executing allows you to specify the path, then we recommend you to do so. For example, in this case, `git` allow us to specify the path with the argument `-C`. That approach would be better, as shown in [Setting environment variables](#setting-environment-variables).

##Running the tests
You need to run the tests of the package `OSSubprocess-Tests`. The current test coverage is about 65%. You may want to take a look to our [Travis CI integration](https://travis-ci.org/marianopeck/OSSubprocess/).

## Contributing
This project is developed with [GitFileTree](https://github.com/dalehenrich/filetree), which, starting in Pharo 5.0, provides what is called `Metadata-less` FileTree. That basically means that there are certain FileTree files (`version` and `methodProperties`) which are not created. **Therefore, you cannot use regular FileTree to contribute to this project. You must use `GitFileTree`.**

The following are the steps to contribute to this project:

* Fork it using Github web interface!
* Clone it to your local machine: `git clone git@github.com:YOUR_NAME/OSSubprocess.git`
* Create your feature branch: `git checkout -b MY_NEW_FEATURE`
* Download latest Pharo 5.0 and load GitFileTree and this project:

```Smalltalk
Metacello new
 	baseline: 'FileTree';
   	repository: 'github://dalehenrich/filetree:issue_171/repository';
   	load: 'Git'.
Metacello new
	baseline: 'OSSubprocess';
 	repository: 'gitfiletree:///path/to/your/local/clone/OSSubprocess/repository';
	onConflict: [ :ex | ex allow ];
	load.
```

* You can now perform the changes you want at Pharo level and commit using the regular Monticello Browser.
* Run all OSSubprocess tests to make sure you did not break anything.
* Push to the branch. Either from MC browser of with `git push origin MY_NEW_FEATURE`
* Submit a pull request from github web interface.

## History
You can see the whole changelog of the project [Changelog](CHANGELOG.md) for details about the release history.

## Future work
Besides the [issues](https://github.com/marianopeck/OSSubprocess/issues), the following are also desired features:

* Instead of using asynchronous I/O operations with FFI blocking callouts, sse synchronous I/O operations but with threaded FFI callouts. This will only be possible once Pharo VM and FFI supports threaded callouts.
* Experiment with a VM plugin with a single `forkAndExec` kind of primitive (based on OSProcess one) and avoid using `posix_spawn()` family of functions.
* Experiment with a Windows support by calling via FFI to `CreateProcess()`. Note there is a VM plugin called [ProcessorWrapper](http://leves.web.elte.hu/ProcessWrapper/) that wraps such a function. So we can base our work from it.
* Implement a pipeline at Pharo level. That is, to be able to create multiple instances of `OSSUnixSubprocess` which we will be piped each other. We can base our work in OSProcess `ProxyPipeline`.

## Authors

* **Mariano Martinez Peck** - *Initial work* - [Mariano Martinez Peck](https://github.com/marianopeck)

See also the list of [contributors](https://github.com/marianopeck/OSSubprocess/contributors) who participated in this project.

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details

## Acknowledgments

* OSSubprocess is highly influenced by a subset of the [OSProcess](http://wiki.squeak.org/squeak/708) project. There are parts which we even copied and adapted them (OSSPipe, OSSAttachableStream, OSSUnixProcessExitStatus). Other parts, we took them as inspiration (the idea of ThisOSProcess representing the VM process, the child watcher, and many others). In addition, OSSubprocess currently uses some of the OSProcess **plugin** (not OSProcess image side), such as the SIGCHLD handler or the creation of pipes.
* Took some ideas from [Limbo](https://github.com/theseion/liblimbo)

## Funding
This project is sponsored by the [Pharo Consortium](http://consortium.pharo.org/).

repository: brackendev/OSSubprocess

repo_name: brackendev/OSSubprocess


text:
---
layout: post
title: Perl Weekly Challenge 123.
comments: true
excerpt: Ugly numbers and square points

tags:
   - pwc
   - perl
---

My solutions
([task 1](https://github.com/wlmb/perlweeklychallenge-club/blob/master/challenge-123/wlmb/perl/ch-1.pl) and
[task 2](https://github.com/wlmb/perlweeklychallenge-club/blob/master/challenge-123/wlmb/perl/ch-2.pl))
to the  [The Weekly Challenge - 123](https://perlweeklychallenge.org/blog/perl-weekly-challenge-123).

# Task 1: Ugly Numbers

    Submitted by: Mohammad S Anwar
    You are given an integer $n >= 1.

    Write a script to find the $nth element of Ugly Numbers.

    Ugly numbers are those number whose prime factors are 2, 3
    or 5. For example, the first 10 Ugly Numbers are 1, 2, 3, 4,
    5, 6, 8, 9, 10, 12.

    Example
    Input: $n = 7
    Output: 8

    Input: $n = 10
    Output: 12

Ugly numbers are of the form `U_n=2**a 3**b 5**c` with `a`,
`b`, and `c` nonnegative integers.  I build an array of known ugly numbers and for each factor
(2, 3, or 5) I keep an index to the next old ugly number to
try, choosing the smallest one that after multiplication by
the appropriate prime lies above all previous ugly
numbers. To find the n-th ugly number I have to generate all the
previous ones, which can then be recycled for the next calculation

    # Perl weekly challenge 123
    # Task 1: Ugly numbers
    #
    # See https://wlmb.github.io/2021/07/27/PWC123/#task-1-ugly-numbers

    use warnings;
    use strict;
    use v5.12;
    my @uglies=(1); #known uglies
    my @u_id=(0,0,0); # indices into uglies array corresponding to
    my @factors=(2,3,5); # prime factors of uglies

    foreach(@ARGV){
        say "Input: $_ Output: ", ugly($_);
    }

    sub ugly {
        my $n=shift @_; # desired ugly number. Assume a positive integer
        while($n>@uglies){ # generate uglies until the n-th becomes known
    	my ($next_id)=sort {next_from_id($a)<=>next_from_id($b)} 0..2;
    	my $next_val=next_from_id($next_id);
    	$u_id[$next_id]++;
    	push @uglies, $next_val if $next_val>$uglies[-1];
        }
        return $uglies[$n-1];
    }

    sub next_from_id {
        my $id=shift @_;
        return $uglies[$u_id[$id]]*$factors[$id];
    }

Example: Generate the first 20 ugly numbers

    ./ch-1.pl `perl -E 'say join " ", 1..20'`

Results:

    Input: 1 Output: 1
    Input: 2 Output: 2
    Input: 3 Output: 3
    Input: 4 Output: 4
    Input: 5 Output: 5
    Input: 6 Output: 6
    Input: 7 Output: 8
    Input: 8 Output: 9
    Input: 9 Output: 10
    Input: 10 Output: 12
    Input: 11 Output: 15
    Input: 12 Output: 16
    Input: 13 Output: 18
    Input: 14 Output: 20
    Input: 15 Output: 24
    Input: 16 Output: 25
    Input: 17 Output: 27
    Input: 18 Output: 30
    Input: 19 Output: 32
    Input: 20 Output: 36

# Task 2: Square Points

    Submitted by: Mohammad S Anwar
    You are given coordinates of four points i.e. (x1, y1), (x2,
    y2), (x3, y3) and (x4, y4).

    Write a script to find out if the given four points form a
    square.

    Example
    Input: x1 = 10, y1 = 20
           x2 = 20, y2 = 20
           x3 = 20, y3 = 10
           x4 = 10, y4 = 10
    Output: 1 as the given coordinates form a square.

    Input: x1 = 12, y1 = 24
           x2 = 16, y2 = 10
           x3 = 20, y3 = 12
           x4 = 18, y4 = 16
    Output: 0 as the given coordinates doesn't form a square.

There are several properties of squares that may be used. I
guess, the easiest to use and to generalize are that the four
sides are equal and the diagonal is sqrt(2) times larger. I
use the *Perl Data Language* `PDL` to manipulate the data.

    # Perl weekly challenge 123
    # Task 2: Square points
    #
    # See https://wlmb.github.io/2021/07/27/PWC123/#task-2-square-points
    use strict;
    use warnings;
    use v5.12;
    use PDL;
    foreach(@ARGV){
        # assume data as strings "[[x0,y0],[x1,y1],[x2,y2],[x3,y3]]"
        my $m=pdl($_); # convert to 2d array
        # Separate into four vectors, translate the origin to the first
        # and sort by size
        my $v0=$m->slice(":,(0)");
        my (undef, $s1, $s2, $d)=sort {size2($a) <=> size2($b)} map {$_-$v0} $m->dog;
        # $s1 and $s2 ought to be the sides and $d the diagonal
        # check they add up and their sizes. I use 'approx' to accommodate rounding errors.
        my $ok=approx(size2($s1+$s2-$d),0) && approx(size2($s1),size2($s2))
               && approx(size2($d),2*size2($s1));
        say "Input: $m Output: $ok"
    }
    sub size2 { #  squared size of vector
        my $v=shift @_;
        return ($v*$v)->sumover;
    }

Example:

    ./ch-2.pl "[[10,20],[20,20],[20,10],[10,10]]" \
              "[[12,24],[16,10],[20,12],[18,16]]"

Results:

    Input:
    [
     [10 20]
     [20 20]
     [20 10]
     [10 10]
    ]
     Output: 1
    Input:
    [
     [12 24]
     [16 10]
     [20 12]
     [18 16]
    ]
     Output: 0

Example with a square that is not aligned with the Cartesian
axes

    ./ch-2.pl "[[0,0],[30,40],[-40,30],[-10,70]]"

Results:

    Input:
    [
     [  0   0]
     [ 30  40]
     [-40  30]
     [-10  70]
    ]
     Output: 1

The same example but shifted away from the origin

    ./ch-2.pl "[[20,30],[50,70],[-20,60],[10,100]]"

Results:

    Input:
    [
     [ 20  30]
     [ 50  70]
     [-20  60]
     [ 10 100]
    ]
     Output: 1

What is very nice about `PDL` is that now I can apply my
program without modification to squares in three dimensional
space. First I try some squares parallel to the cartesian planes

    ./ch-2.pl "[[10,20,10],[20,20,10],[20,10,10],[10,10,10]]" \
              "[[10,10,20],[20,10,20],[20,10,10],[10,10,10]]" \
              "[[10,10,20],[10,20,20],[10,20,10],[10,10,10]]"

Results:

    Input:
    [
     [10 20 10]
     [20 20 10]
     [20 10 10]
     [10 10 10]
    ]
     Output: 1
    Input:
    [
     [10 10 20]
     [20 10 20]
     [20 10 10]
     [10 10 10]
    ]
     Output: 1
    Input:
    [
     [10 10 20]
     [10 20 20]
     [10 20 10]
     [10 10 10]
    ]
     Output: 1

Now I generate a square with some other orientation by
rotating and shifting in space a simple known square. For
example, I rotate a simple unit square around the x axis and then around the z axis
by π/4 and translate by (1,2,3).

    perl -MPDL -E '$t=atan2(1,1); $c=cos($t), $s=sin($t);'\
               -E '$Rz=pdl([[$c,$s,0],[-$s,$c,0],[0,0,1]]);'\
               -E '$Rx=pdl([[1,0,0],[0,$c,$s],[0,-$s,$c]]);'\
               -E 'say +($Rz x $Rx x  pdl([[1,0,0],[1,1,0],[0,1,0]])->transpose)->transpose +pdl[1,2,3]'

Results:

    [
     [ 1.7071068  1.2928932          3]
     [ 2.2071068  1.7928932  2.2928932]
     [       1.5        2.5  2.2928932]
    ]

I use this as input to the program.

    ./ch-2.pl "[[1,2,3],[1.7071068,1.2928932,3],[2.2071068,1.7928932,2.2928932],[1.5,2.5,2.2928932]]"

Results:

    Input:
    [
     [         1          2          3]
     [ 1.7071068  1.2928932          3]
     [ 2.2071068  1.7928932  2.2928932]
     [       1.5        2.5  2.2928932]
    ]
     Output: 1

I repeat the example above but modifying one of the inputs, so
that it differs slightly from a square.

    ./ch-2.pl "[[1,2,3.001],[1.7071068,1.2928932,3],[2.2071068,1.7928932,2.2928932],[1.5,2.5,2.2928932]]"

Results:

    Input:
    [
     [         1          2      3.001]
     [ 1.7071068  1.2928932          3]
     [ 2.2071068  1.7928932  2.2928932]
     [       1.5        2.5  2.2928932]
    ]
     Output: 0

By default, the `approx` routine I used checks for equality
within a 1e-6 error, but if desired, a more strict comparison
can be made adding a third argument.

By the way, squares may be found not only in two or three
dimensions, but also in higher dimensions. Some simple examples in
four dimensions follow

    ./ch-2.pl "[[10,20,1,2],[20,20,1,2],[20,10,1,2],[10,10,1,2]]" \
              "[[10,1,2,20],[20,1,2,20],[20,1,2,10],[10,1,2,10]]" \
              "[[1,2,10,20],[1,2,20,20],[1,2,20,10],[1,2,10,10]]"

Results:

    Input:
    [
     [10 20  1  2]
     [20 20  1  2]
     [20 10  1  2]
     [10 10  1  2]
    ]
     Output: 1
    Input:
    [
     [10  1  2 20]
     [20  1  2 20]
     [20  1  2 10]
     [10  1  2 10]
    ]
     Output: 1
    Input:
    [
     [ 1  2 10 20]
     [ 1  2 20 20]
     [ 1  2 20 10]
     [ 1  2 10 10]
    ]
     Output: 1

license: MIT

branch_name: refs/heads/master

extension: md

license_type: permissive


text:
  Model REST API Documentation

  ![Python](https://img.shields.io/badge/python-v3.8-blue.svg)

## Overview

The Model REST API is a RESTful API that provides access to various machine learning models. The REST API is written in Python and uses FastAPI, a high-performance web framework to build APIs. The API has three main components: the model loader, preprocessor and the main app, which serves the endpoints. If you want to load your own trained models, place them in the `models/allergen_single` folder for models that have only one file.

## Endpoints

The REST API has three main endpoints that can be used for prediction:

- /predict_style
- /predict_allergens
- /predict_proba

## Startup

The main app can be started in the source folder with the command:

```shell
uvicorn main:app --reload
```

Or alternatively with docker:

```docker
docker-compose up -d
```

## Technology used

IDE:
[IntelliJ IDEA](https://www.jetbrains.com/idea/)

[Python 3.7](https://www.python.org/)
[Docker](https://www.docker.com/)
[scikit-learn](https://scikit-learn.org/)

branch_name: refs/heads/main

gha_language: Python

gha_license_id: MIT


text:
Programming Zooks
=================

Debugging / Creating Zooks 101
------------------------------
1. Setup a server on your own machine so that [http://localhost](http://localhost) resolves to the Gadzooks! directory.
2. Disable Greasemonkey, Tampermonkey or whatever injector you may already be using
3. Create the following bookmarklet: `javascript:$('body').append('<script src="http://localhost/inject-debug.js"></script>')`
4. Visit the [Fallen London](http://fallenlondon.storynexus.com) website
5. Open the debug bookmarklet. This should load Gadzooks! from your local machine and not from the main site.

The above instructions loads the [inject-debug.js](inject-debug.js) instead of the regular [inject.js](inject.js) injector. This forces the Gadzooks! core to load all resources from [http://localhost](http://localhost) rather than the live site. You should now be able to edit files and test them in real time by repeating steps 4 + 5, above.

Structure
---------
In its base form a Zook is simply a HTML file which can contain CSS, JavaScript and HTML. When a Zook gets loaded it is injected as the last element in the `<body>`.

The structure of a zook requires a package definition which is specified within a `<script type="application/zook">` HTML tag container.

```json
<script type="application/zook">
{
	"forced": false, // If true the zook is always enabled and cannot be disabled
	"hide": true, // If true this zook wont be shown in the options page
	"default": false, // The default state of the zook
	"title": "Simple Gadzook package", // A short descriptive title
	"description": "This Zook does nothing at all except turn the background color red. Its designed for developers and other such deviants"
}
</script>
// Anything else such as CSS, JavaScript or HTML resources goes here in regular 'style', 'script' or inline HTML tags
```

The best source for learning how a zook works is exploring the [available zooks](zooks) specifically the [test zook](zooks/test.html).

Tips
----

Things to remember:

* The main site uses jQuery 1.3.1 so support for 'modern jQ' APIs like 'on', 'one' is buggy. You are currently limited to [whats available in jQ 1.3.1](http://api.jquery.com/category/version/1.3) until the site version of jQuery gets updated.
	* `on()` is not available to monitor all items on a page. Use `live()` instead
	* `elem.find()` is not available, use `elem.children()` to filter an items child elements

repository: hash-bang/gadzooks

gha_language: HTML

repo_name: hash-bang/gadzooks


text:
# Invisible Man - VPN Client

> Free premium PPTP VPN

[![Downloads](https://img.shields.io/github/downloads/invisiblemanvpn/InvisibleMan-VPNClient/total.svg?label=downloads%20%28total%29)](https://github.com/InvisibleManVPN/InvisibleMan-VPNClient/releases)

Invisible Man is an open source, censorship and circumvention tool to give access to the open Internet and past censors. Invisible Man is built on C# WPF.

![InvisibleManScreenShot1](https://github.com/InvisibleManVPN/InvisibleMan-VPNClient/blob/master/Images/ImgInvisibleMan1.png)
![InvisibleManScreenShot2](https://github.com/InvisibleManVPN/InvisibleMan-VPNClient/blob/master/Images/ImgInvisibleMan3.png)

## All these benefits by pushing one button

- **COMPLETE PRIVACY:** With Invisible Man, your actual IP address is hidden from the services you visit. Not even your Internet service provider can see what you do online.

- **ACCESS BLOCKED CONTENT:** Change your virtual location and faceless restrictions when it comes to accessing content online.

- **Wi‑Fi SECURITY:** Even on unsecured public Wi‑Fi, your traffic is encrypted and impossible to intercept.

## Getting started

If you just want to download the application, you can use following links:

[Download Invisible Man - x64](https://github.com/InvisibleManVPN/InvisibleMan-VPNClient/releases/download/1.0/InvisibleMan-x64.exe)

[Download Invisible Man - x86](https://github.com/InvisibleManVPN/InvisibleMan-VPNClient/releases/download/1.0/InvisibleMan-x86.exe)

But if you want to get the source of Invisible Man, just use `git clone "https://github.com/InvisibleManVPN/InvisibleMan-VPNClient.git"` and open the project in Visual Studio.

## Contacts

Web page: [https://invisiblemanvpn.github.io/](https://invisiblemanvpn.github.io/)

Email: [invisiblemanvpn<EMAIL_ADDRESS>

repository: leearic/InvisibleMan-VPNClient

repo_name: leearic/InvisibleMan-VPNClient


text:
# Vue Name Generator

Generates a random name from several list of names provided by the user

## Play the demo
To see the demo version:
* Clone the repository
* Install NPM
* Launch the command npm install
* Launch npm run dev

### Install the component
```npm install vue-name-generator --save```

#### Use vue-name-generator in your component
```
<template>
    <div>
        <input type="text" :value="nameGenerated"/>
        <vue-name-generator v-model="nameGenerated" color="grey" width="15px" height="15px" :files='["./assets/adjectives","./assets/names"]' noSpace separator="-" maxLength="50" icon="mdi-autorenew" title="My title"/>
    </div>
</template>

<script>
import vueNameGenerator from "../vue-name-generator.vue";
export default{
    data(){
        return{
            nameGenerated:"no name"
        }
    },
    components: {
        vueNameGenerator
    }
}
</script>
```

##### Component properties

* color: string - (default:grey) color of the icon used to generate the names
* width: string - (default: 15px) width of the icon
* height: string - (default:15px) height of the icon
* files: array - (default: ["./assets/adjectives","./assets/names"]) inputs to generate the name. Each entry provides the relative path to the file. The order define the order in which the parts will be used for names generation. Eg: ```["./assets/adjectives","./assets/names"]```
* maxLength: number - (default: 30) maxLength of the string generated
* noSpace: boolean - (default: true) when true, remove all the spaces from the inputs
* separator: string - (default: -) separate the entry of each of the input files
* icon: string - (default: mdi-tag-plus) define the icon used for the generator
* title: string - (default: Clic to generate a new name) define the title of the span

###### Contributing

We will consider the MRs, so feel free to send some. We will also be happy to consider your feedbacks and comments.

## Authors

* **Aymeric Hermant** - *Initial work* - [VueNameGenerator](https://github.com/ahermant/vue-name-generator/)

See also the list of [contributors](https://github.com/ahermant/vue-name-generator/graphs/contributors) who participated in this project.

## License

This project is licensed under the BSD3 License - see the [LICENSE.md](LICENSE.md) file for details

## Acknowledgments

* Inspired by [golang-petname] (https://github.com/szaharici/golang-petname/) for the idea
* Insipred by [vue-json-editor](https://github.com/dirkliu/vue-json-editor) for the component structure

repository: ahermant/vue-name-generator

gha_language: JavaScript

repo_name: ahermant/vue-name-generator


text:
People

A human **person**, a type of entity used throughout the RunOrg API
for many purposes, the most important being:

 - Message and email recipients are always people. All sending APIs
   expect either individuals or [groups](/docs/#/group.md) of people
   as their parameters.
 - Everyone who logs in, using any method, is bound to a person.
 - Access restrictions are based on the [person who performs the
   action](/docs/#/concept/as.md).

repository: RunOrg/RunOrg

repo_name: RunOrg/RunOrg


text:
# Entryway-DynamoDB

This project provides a node module that allows you to persist user data to [Amazon's DynamoDB](https://aws.amazon.com/dynamodb/) via the [Entryway](https://github.com/cdellinger/entryway) authentication module.

##Installation
###Core Module

After installing [Entryway](https://github.com/cdellinger/entryway), this module can be installed as follows:

    npm install entryway-dynamodb

##Usage

In order to use this project, you need to first [create a DynamoDB account](https://aws.amazon.com/dynamodb/getting-started/).

##Configuration
You must provide two configuration values as environmental variables in order for this project to work successfully.  These two values are as follows

	ENTRYWAY_DYNAMODB_REGION
	ENTRYWAY_DYNAMODB_ENDPOINT

#####ENTRYWAY_DYNAMODB_REGION
This is the identifier of the region that you created your DynamoDB within.

#####ENTRYWAY_DYNAMODB_ENDPOINT
This is https endpoint provided by Amazon when you created your DynamoDB instance.

###Code Examples
There is not any code that is specific to this particular module in order to support DocumentDB.  All you need to do is pass a provider object based upon this module into Entryway as shown below.

	var UserSchema = require('entryway');
	var provider = require('entryway-dynamodb');
	var user = new UserSchema(provider);
	...

Beyond that all the interaction with Entryway is exactly the same as with other Entryway providers, examples which can be seen [here](https://github.com/cdellinger/entryway).

repository: cdellinger/entryway-dynamodb

repo_name: cdellinger/entryway-dynamodb


text:
---
layout: post
title: ML Reading Ch05 - Support Vector Machines
date: 2021-07-18 21:51:00
comment_id: 175
categories: [ML]
tags: [Hands-On Machine Learning]
---

> Support Vector Machine (SVM) are particularly well suited for classification of complex small- or medium-sized datasets.

# Linear SVM Classification

- `large margin classification`: an SVM classifier as fitting the widest possible street (represented by the parallel dashed lines) between the classes
- `support vectors`: the instances located on the edge of the street

*NOTE: SVMs are sensitive to the feature scales. After feature scaling (e.g., using `Scikit-Learn’s StandardScaler`), the decision boundary in the right plot looks much better.*

## Soft Margin Classification

- `hard margin classification`: strictly impose that all instances must be off the street and on the right side

2 main issue with `hard margin classification`

- it only works if the data is linearly separable
- it is sensitive to outliers.

To avoid the issue, do `soft margin classification`. The objective is to find a good balance between keeping the street as large as possible and limiting the margin violations.

- `margin violations`: instances that end up in the middle of the street or even on the wrong side

When creating an SVM model using Scikit-Learn, we can specify a number of hyperparameters. C is one of those hyperparameters.
If your SVM model is overfitting, you can try regularizing it by reducing C.

*NOTE: Unlike Logistic Regression classifiers, SVM classifiers do not output probabilities for each class. The LinearSVC class regularizes the bias term, so you should center the training set first by subtracting its mean. This is automatic if you scale the data using the StandardScaler. Also make sure you set the loss hyperparameter to "hinge", as `it is not the default value`. Finally, for better performance, you should `set the dual hyperparameter to False`, unless there are more features than training instances.*

Instead of using the `LinearSVC` class, we could use the SVC class with a linear kernel. When creating the SVC model, we would write `SVC(kernel="linear", C=1)`. Or we could use the `SGDClassifier` class, with `SGDClassifier(loss="hinge", alpha=1/(m*C))`. This applies regular `Stochastic Gradient Descent` (see Chapter 4) to train a linear SVM classifier. It does not converge as fast as the LinearSVC class, but it can be useful to handle online classification tasks or huge datasets that do not fit in memory (out-of-core training).

# Nonlinear SVM Classification

One approach to handling nonlinear datasets is to add more features, such as `polynomial` features.

## Polynomial Kernel

`kernel trick`: makes it possible to get the same result as if you had added many polynomial features, even with very high-degree polynomials, without actually having to add them

*TIP: A common approach to finding the right hyperparameter values is to use grid search (see Chapter 2). It is often faster to first do a very coarse grid search, then a finer grid search around the best values found.*

## Similarity Features

Another technique to tackle nonlinear problems is to `add features computed using a similarity function`, which measures how much each instance resembles a `particular landmark`.

Gaussian Radial Basis Function (RBF):

$$
\phi_\gamma(x, \ell) = exp(-\gamma \|x-\ell \|^2)
$$

## Gaussian RBF Kernel

The similarity features method can be useful with any Machine Learning algorithm, but it may be computationally expensive to compute all the additional features, especially on large training sets.

gamma $\gamma$ acts like a regularization hyperparameter: if your model is overfitting, you should reduce it; if it is underfitting, you should increase it (similar to the C hyperparameter).

- Increasing gamma($\gamma$) makes the bell-shaped curve narrower. As a result, each instance’s range of influence is smaller: the decision boundary ends up being more irregular, wiggling around individual instances.
- a small gamma value makes the bell-shaped curve wider: instances have a larger range of influence, and the decision boundary ends up smoother

`String kernels` are sometimes used when `classifying text documents or DNA sequences` (e.g., using the string subsequence kernel or kernels based on `the Levenshtein distance`).

*TIP: How to decide which kernel to use? you should always try the linear kernel first (remember that LinearSVC is much faster than SVC(kernel="linear")), especially if the training set is very large or if it has plenty of features. If the training set is not too large, you should also try the Gaussian RBF kernel; it works well in most cases. Then if you have spare time and computing power, you can experiment with a few other kernels, using cross-validation and grid search. You’d want to experiment like that especially if there are kernels specialized for your training set’s data structure.*

## Computational Complexity

The `LinearSVC` class is based on the `liblinear library`, which implements an optimized algorithm for linear SVMs. not support the kernel trick, but it scales almost linearly with the number of training instances and the number of features. Its training time complexity is roughly $O(m × n)$.

The algorithm takes longer if you require very high precision. This is controlled by the `tolerance hyperparameter` $\varepsilon$ (called tol in Scikit-Learn). In most classification tasks, the default tolerance is fine.

The `SVC` class is based on the `libsvm` library, which implements an algorithm that `supports the kernel trick`. The training time complexity is usually between $O(m^2 × n)$ and $O(m^3 × n)$. This algorithm is perfect for complex small or medium-sized training sets. It scales well with the number of features, especially with sparse features (i.e., when each instance has few nonzero features).

Comparison of Scikit-Learn classes for SVM classification:

| Class  |  Time Complexity | Out-of-core support | Scaling required | Kernel trick |
|---|---|---|---|---|
| LinearSVC  |  $O(m × n)$ | No | Yes | No |
| SGDClassifier  |  $O(m × n)$ | Yes | Yes | No |
| CVC  |  $O(m ^2 × n)$ to $O(m^3 × n)$ | No | Yes | Yes |

# SVM Regression

SVM Regression tries to fit as many instances as possible on the street while limiting margin violations (i.e., instances off the street). The width of the street is controlled by a hyperparameter, $\varepsilon$.

Adding more training instances within the margin does not affect the model’s predictions; thus, the model is said to be $\varepsilon$`-insensitive`.

Can use Scikit-Learn’s `LinearSVR` class to perform linear SVM Regression.

The `LinearSVR` class `scales linearly` with the size of the training set (just like the LinearSVC class), while the `SVR` class `gets much too slow` when the training set grows large (just like the SVC class).

# Under the Hood

How SVMs make predictions and how their training algorithms work.
(I might watch deeper when I roughly understand how to use it.)

- <https://learning.oreilly.com/library/view/hands-on-machine-learning/9781492032632/ch05.html>

#### References

- <https://learning.oreilly.com/library/view/hands-on-machine-learning/9781492032632/ch05.html>>

gha_language: CSS


text:
Preferences
==========

The main settings of the Administrator and Customer portal can be configured in `Config → Main → Preferences`.

![Config menu](main_menu.png)

### Default landing page
Here you can select which page (Customer or Admin portal) will be opened when your Splynx URL is accessed:

![default page](default_page.png)

### Administration portal

![admin portal](admin_portal.png)
![admin portal](admin_portal2.png)

The following settings apply to the Administration portal only:

* **Title for admin pages** - header for the tab name in the browser (`Splynx: Preferences`, `Splynx: Dashboard`);
* **Reset password** - enables/disables the option to reset the password for the Admin portal;
* **Code validity** - how many hours (from 1 up to 72) the code for the password reset is valid;
* **Attempts limit** - how many attempts can be made to change the password during the code validity period;
* **Type of resetting password** - select a method to reset password (Email or SMS);
* **Template for email message** - select the [template](configuration/system/templates/templates.md) for the email message to reset the password;
* **Text for SMS message** - type the text for the SMS message with the code (default field value: `Password reset code: {{ code }}`);
* **Characters available when generating an email check code** - specify the list of characters available for code generation via email;
* **Email check code length** - specify the length of the code for the email;
* **Characters available when generating an SMS check code** - specify the list of characters available for code generation via SMS;
* **SMS check code length** - specify the length of the code for SMS.

### Server side data table processing

![server side](server_side.png)

* **Enable processing** - enables/disables server processing in the customer list and finance section;
* **Search delay** - search delay in milliseconds (`500` by default).

### Documentation

**Please do not change this field**. The URL is used to load built-in documentation by API.

![documentation](documentation.png)

### Auto backup settings

![backup](auto_back_up.png)

Settings for auto backup.

* **Enabled** - enables/disables auto backup;
* **Period** - period for auto backup (Month, Week, Day);
* **Password** - password for auto backup (`auto_backup_` by default);
* **Prefix** - prefix for auto backup files;
* **Categories for backup** - select what to backup (Full database, RRD files, Template, Addons and handlers, Public uploads);
* **Rotation amount** - counter of auto backups for storage.

### Mention notifications

These notifications are used to notify administrators when they are mentioned within Tasks, the CRM module, Tickets, as well as in customers' comments.

![Mention](mention_notifications.png)

* **Enable Mentions notifications** - enables/disables mentions notifications;
* **Template** - select the template for notification body.

### Mention group notifications

![Mention](mention_group.png)

* **Enable Group Mentions notifications** - enable/disable group mentions notifications;
* **Template** - template for notification body.

### Generate favicons

Here you can upload a custom image to create the favicons for your Splynx. A favicon serves as branding for your website. Its main purpose is to help visitors locate your page easier e.g. in opened tabs, bookmarks etc.

![favicons](generate_favicons.png)

![favicons](generate_favicons2.png)

![favicons](favicon_example.png)

Click on `Download icons set` to download the archive with different size files (**.png** (from *16x16* to *310x310*) / **.ico** (*16x16* and *64x64*)).

Click on `Use as the system icons` to use the set of icons in the system.

repository: splynx/docs

repo_name: splynx/docs


text:
# Gateway using Default Channel Pattern
Link: https://github.com/r2d2-ai/aiflow/microgateway/internal/pattern/DefaultChannelPattern.json
Examples:
https://github.com/r2d2-ai/aiflow/microgateway/examples/api/default-channel-pattern
https://github.com/r2d2-ai/aiflow/microgateway/examples/json/default-channel-pattern

This recipe is a gateway using the defult channel pattern which uses JWT.
This receipe uses 2 triggers.
1. REST trigger:- For calling the channel pattern
Link: https://github.com/r2d2-ai/contrib/trigger/rest

2. Channel Trigger:- For calling second microgateway action
Link: https://github.com/r2d2-ai/contrib/trigger/channel

Channel Pattern:
Channels allow one action to call another action. Channel pattern checks for JWT conditions.
If valid ... it runs corresponding channel activity and message is sent over the channel defined by its name.
Channel trigger then receives the message from channel and corresponding action (eg: log activity)  is called.

#Channel Activity
| Name   |  Type   | Description   |
|:-----------|:--------|:--------------|
| channel | string | The channel to put the value on |
| value | string | The value to put on channel |

Link: https://github.com/r2d2-ai/contrib/activity/channel

#JWT
| Name   |  Type   | Description   |
|:-----------|:--------|:--------------|
| token | string | The raw token |
| key | string | The key used to sign the token |
| signingMethod | string | The signing method used (HMAC, ECDSA, RSA, RSAPSS) |
| issuer | string | The 'iss' standard claim to match against |
| subject | string | The 'sub' standard claim to match against |
| audience | string | The 'aud' standard claim to match against |

Link: https://github.com/r2d2-ai/aiflow/microgateway/activity/jwt

## Installation
* Install [Go](https://golang.org/)

## Setup
```bash
git clone https://github.com/r2d2-ai/aiflow-microgateway
cd microgateway/examples/api/default-http-pattern
```

## Testing
Start the gateway:
```bash
go run main.go
```

### Request is successful
Run the following command:
```bash
curl --request GET http://localhost:9096/endpoint -H "Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJNYXNobGluZyIsImlhdCI6MTU0NDEzMTYxOCwiZXhwIjoxNTc1NjY3NjE4LCJhdWQiOiJ3d3cubWFzaGxpbmcuaW8iLCJzdWIiOiJ0ZW1wdXNlckBtYWlsLmNvbSJ9.wgunWSIJqieRKsmObATT2VEHMMzkKte6amuUlhc1oKs"
```

You should see:
```json
{"response":"Success!"}
```
On the server screen, you get 200 response code and log service outputs "Output: Test log message service invoked"

### JWT token is invalid
Run the following command:
```bash
curl --request GET http://localhost:9096/endpoint -H "Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJNYXNobGluZyIsImlhdCI6MTU0NDEzMTYxOCwiZXhwIjoxNTc1NjY3NjE4LCJhdWQiOiJ3d3cubWFzaGxpbmcuaW8iLCJzdWIiOiJ0ZW1wdXNlckBtYWlsLmNvbSJ9.wgunWSIJqieRKsmObATT2VEHMMzkKte6amuUlhc1oK"
```

You should see:
```json
{"errorMessage":"","validationMessage":"signature is invalid"}
```


text:
# AMA-GCN: Adaptive Multi-layer Aggregation Graph Convolutional Network for Disease Prediction

```
@inproceedings{amagcn_ijcai21,
title = {AMA-GCN: Adaptive Multi-layer Aggregation Graph Convolutional Network for Disease Prediction},
author = {Chen, Hao and Zhuang, Fuzhen and Xiao, Li and Ma, Ling and Liu, Haiyan and Zhang, Ruifang and Jiang, Huiqin and He, Qing},
booktitle = {Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI)},
pages = {2235--2241},
year = {2021}
}
```

links
- [ijcai](https://www.ijcai.org/Proceedings/2021/308)
- [arXiv](https://arxiv.org/abs/2106.08732)

gha_language: Jupyter Notebook


text:
# Change Log

This project adheres to [Semantic Versioning](http://semver.org/).

This CHANGELOG follows the format located [here](https://github.com/sensu-plugins/community/blob/master/HOW_WE_CHANGELOG.md)

## [Unreleased]

## [1.4.0] - 2020-03-10

### Fixed

- Added empty systemd unit file to auto reload the unit configuration before we restart (@ValkyrieOps) - #131

### Added

- Introduced CI/CD testing for windows for agent and ctl (@derekgroh)

### Breaking Changes

- (Windows) New installation method will install the latest version of sensu-agent and sensuctl if not specified. To keep the previously installed version, specify `version '<IP_ADDRESS>65'` for `sensu_agent` and `sensu_ctl`. (@kovukono)
- (Windows) New installation method will cause a duplicate sensuctl executable due to a difference in install location between the Chocolatey package and the previously defined extraction location for the cookbook. This can be fixed by removing the folder `c:\Program Files\Sensu\sensu-cli\bin\sensuctl`, or by using the `:cleanup_legacy_cookbook_install` action for `sensu_ctl`.

### Changed

- Moved to Chocolatey to manage sensuctl and sensu-agent on Windows. (@kovukono)
- `sensu_agent` and `sensu_ctl` for Windows now receive their version from the `version` attribute of the resource instead of `node['sensu-go']['msi_version']` and `node['sensu-go']['ctl_version']`. (@kovukono)
- Added `cleanup_legacy_cookbook_install` resource to `sencu_ctl` to remove old Windows installation from this cookbook.

## [1.3.0] - 2020-10-28

### Fixed

- Fixed outfile for sensuctl install for Windows, as well as preventing unnecessary downloads for install. Fixed configure action for Windows that was unable to parse array. (@kovukono)
- Fixed `sensu_ctl` resource to only extract downloads on new versions, not every chef run (@webframp)

### Added

- `sensu_auth_oidc` resource added (@webframp)
- `sensu_auth_ldap` resource for ldap integration. (@webframp)
- `sensu_etcd_replicator` resource for managing cluster RBAC federation (@webframp)
- `sensu_search` resource added (@webframp)
- `sensu_global_config` resource to manage web ui configuration (@webframp)
- `sensu_tessen_config` resource to manage tessen analytics preference (@webframp)
- `sensu_user` resource to manage non SSO users (@webframp)

### Changed

- Rename `:ad_servers` property of `sensu_active_diretory` resource to `:auth_servers`. For consistency with `sensu_auth_ldap` resource this property was renamed and will be removed in a future cookbook version. (@webframp)
- Refactored `sensu_ctl` to use builtin (Chef 15+) `archive_file` resource and remove 3rd party `seven_zip` cookbook dependency. (@webframp)

## [1.2.0] - 2020-10-17

### Fixed

- Fix yaml rendering for agent and backend with Chef 16.x (@webframp)

### Added

- Add `header` property support to `sensu_asset` resource. (@webframp)

### Changed

- Updated the attributes for `agent` and `ctl` (windows only) to version `6.1.0`. On linux the behavior is unchanged; installed by a single package and default to `latest` unless a version is specified. (@derekgroh)

## [1.1.0] - 2020-09-27

### Added

- The following resources, `sensu_check`, `sensu_entity`, `sensu_filter`, `sensu_handler`, `sensu_hook`, `sensu_mutator`, and `sensu_secret` now expose a `namespace` attribute for controlling where the resource is created. Default is the `default` namespace. @joe-armstrong)
- Minor README change for Sensu 6.0.0 handling of agent configs. (@kovukono)

## [1.0.0] - 2020-07-24

### Breaking Changes

- `sensu_check` now does not provide defaults for `command` and `subscriptions`, and they must be required as per the upstream specs. (@kovukono)
- declare chef supported versions to use match current chef support (15+) (@majormoses)

### Fixed

- Chef 16+ support (@kovukono)
- `sensu_entity` support for missing attributes (@kovukono)

### Added

- `sensu_active_directory` resource for active directory integration. (@kovukono)
- `sensu_secrets_provider` and `sensu_secret` resource for Vault integration, along with secret support for checks, handlers, and mutators. (@kovukono)
- `sensu_asset` now exposes a `namespace` attribute for controlling where the asset is created, by default it will use `default`. Default is the `default` namespace. @joe-armstrong)
- `sensu_asset` now supports multi-built assets using the `builds` key and accepts an array of hashes for each filter. By default no filters are passed. (@cwjohnston)
- all of the following resources: `asset`, `check`, `filter`, `handler`, and `mutator` have had new aliases added to the providers with an optional namespace to ease migration issues where a resource exists in both `sensu` and `sensu-go` the generic schema is `sensu_go_$COMPONENT`. For example `sensu_check` can be referenced also via `sensu_go_check`. (@majoemoses)

## [0.3.0] - 2020-05-13

### Fixed

- agent service will restart when its config changes (@kovukono)

### Added

- `sensu-backend` action :init adding returns to account for possible return codes.  exit code 3 is returned when already init.  This allows chef-client runs to not fail when running idempotently. (@tarcinil)
- `sensu-backend` now supports specifying an apt or yum repository for packages built from source. (@kovukono)

## [0.2.0] - 2020-01-05

### Breaking Changes

- `sensu-backend` property: `config` Readme incorrectly documented the wrong (@tmonk42)
  path. Called out as a breaking change in case anyone had begun using the incorrect setting but as this has not been released it will be versioned as a patch. (@tmonk42) - #66

### Added

- `filters` resource now supports `runtime_assets` being passed (@majormoses)
- Added support for `postgres_config` resource (@cwjohnston) - #70
- Added support for `role`, `role_binding`, `cluster_role` and `cluster_role_binding` resources (@cwjohnston) - #71
- Added init action to sensu-backend to allow functionality added in 5.16.0 (@kovukono) - #77

### Changed

- Refactored resource helpers to reduce duplication. (@cwjohnston) - #72

## [0.1.0] - 2019-09-16

### Added

- Most resources now support metadata specific properties (@webframp)
- add `sensu_hook` resource (@derekgroh)
- add `debug` option for `sensu_ctl` resource to help debug (@majormoses)
- add support for sensu-go-agent on windows platform (@derekgroh)
- fix symbols in annotations and labels (@scalp42)
- add sensu_ctl resource for windows platforms (@derekgroh) - #59

### Changed

- sensuctl cli args for asset updates now uses `--namespace`
- sensuctl cli args are escaped properly (@beeerd)
- sensuctl cli commands are marked sensitive by default (@beeerd)

### Breaking Changes

- Use stable package channels (@webframp)
- Temporarily remove Debian support until stable packages are available (@webframp)
- `sensu_organization` resource removed to match upstream (@webframp)
- Switched to beta package repository as default (@webframp)
- `sensu_environment` is now `sensu_namespace` (@webframp)
- `extended_attributes` property renamed to `annotations` as part of metadata (@webframp)
- Filter `statements` property renamed to `expressions` (@webframp)
- Entity `class` property renamed to `entity_class`

## [0.0.3] - 2018-09-12

### Added

- new `sensu_entity` resource (@mercul3s)
- new `sensu_organization` resource (@mercul3s)
- new `sensu_environment` resource (@mercul3s)

## [0.0.2] - 2018-08-29

### Added

- Adding `output_metric` settings to the `sensu_check` resource

## [0.0.1]

### Added

- new `.editorconfig` to help users who have editors that support [editor config](https://editorconfig.org/)
- new PR and issue templates (@majormoses)
- links to community slack (@majormoses)
- new `sensu_mutator` resource (@webframp)
- new `sensu_filter` resource (@webframp)
- new `sensu_handler` resource (@webframp)
- new `sensu_check` resource (@webframp)
- new `sensu_ctl` resource to install and configure (@webframp)
- Created repo with initial commit (@mbbroberg)
- Added CODEOWNERS (@majormoses)
- Added skel files from Chef Partners cookbook generator (@thomasriley)

### Changed

- Updated contributing instruction (@majormoses)
- use `@sensu/chef-cookbooks` for `CODEOWNERS` rather than individual users now that there is a team to refer to instead (@majormoses)

### Fixed

- moved `CODEOWNERS` into the correct location (@majormoses)
- updated development dependencies (@majormoses)
- using a version of `'latest'` for backend and agent providers will now upgrade to the test version

<!-- prettier-ignore -->
[Unreleased]: https://github.com/sensu/sensu-go-chef/compare/1.4.0...HEAD
[1.4.0]: https://github.com/sensu/sensu-go-chef/compare/1.3.0...1.4.0
[1.3.0]: https://github.com/sensu/sensu-go-chef/compare/1.2.0...1.3.0
[1.2.0]: https://github.com/sensu/sensu-go-chef/compare/1.1.0...1.2.0
[1.1.0]: https://github.com/sensu/sensu-go-chef/compare/1.0.0...1.1.0
[1.0.0]: https://github.com/sensu/sensu-go-chef/compare/0.3.0...1.0.0
[0.3.0]: https://github.com/sensu/sensu-go-chef/compare/0.2.0...0.3.0
[0.2.0]: https://github.com/sensu/sensu-go-chef/compare/0.1.0...0.2.0
[0.1.0]: https://github.com/sensu/sensu-go-chef/compare/0.0.3...0.1.0
[0.0.3]: https://github.com/sensu/sensu-go-chef/compare/0.0.2...0.0.3
[0.0.2]: https://github.com/sensu/sensu-go-chef/compare/0.0.1...0.0.2
[0.0.1]: https://github.com/sensu/sensu-go-chef/compare/37630d8624247f0e2dc41de8de8c2ccd29d55694...0.0.1

repository: abiheiri/sensu-go-chef

repo_name: abiheiri/sensu-go-chef


text:
---
title: Managing Tasks from Group Chat - ChatOps
date: 2020-03-05
profile: false
image:
  placement: 3
tags: ["chatops", "devops", "communication", "collaboration"]
editable: true
---

Chatops is the use of tools from within a group chat tool.

One great example of Chatops is when engineers can update important information regarding an incident to a broader audience, possibly even to affected users.

Allowing people to quickly update stakeholders by typing a few quick commands from within the same environment as the related conversations has a number of clear benefits.

Let's take a look at [Building a Serverless Status Page Solution with Azure Functions, blob storage, and Microsoft Teams](/post/building_a_serverless_status_page_solution/) to illustrate.

repository: jasonhand/academic-kickstart

gha_language: Shell

repo_name: jasonhand/academic-kickstart


text:
# Reactive Programming

In computing, reactive programming is a [[declarative-programming]] paradigm concerned with data streams and the propagation of change. With this paradigm it is possible to express static (e.g., arrays) or dynamic (e.g., event emitters) data streams with ease, and also communicate that an inferred dependency within the associated execution model exists, which facilitates the automatic propagation of the changed data flow.[citation needed]

For example, in an imperative programming setting, `a:=b+c` would mean that `a` is being assigned the result of `b+c` in the instant the expression is evaluated, and later, the values of `b` and `c` can be changed with no effect on the value of `a`. On the other hand, in reactive programming, the value of `a` is automatically updated whenever the values of `b` or `c` change, without the program having to re-execute the statement `a:=b+c` to determine the presently assigned value of `a`.

## Examples

- [Elm](https://elm-lang.org/)

[//begin]: # "Autogenerated link references for markdown compatibility"
[declarative-programming]: declarative-programming "Declarative Programming"
[//end]: # "Autogenerated link references"

repository: jevakallio/concepts

repo_name: jevakallio/concepts


text:
# Snake 2 & Snake 2 beta

## Installation
First, [download cc3250mt.dll](http://www.dlldownloader.com/cc3250mt-dll) and extract the DLL to the `build` folder, then run `build/Snake2.exe` or `build/Snake2_beta.exe`.

## Known Issues
Anti-virus software will probably prevent execution of Snake2.exe, due to its linkage to cc3250mt.dll. This is a false positive, however.
(Do your own research, if you like. The so-called “malicious code” found by the scanner is not even harmful.)

## Screenshots
![Start screen of Snake 2](assets/start.png)

![Setting the playing difficulty in Snake 2](assets/level.png)

![Help screen of Snake 2](assets/help.png)

![Playing Snake 2](assets/game.png)

repository: vbwx/win-snake

repo_name: vbwx/win-snake


text:
# gocloud-blob-bucket

This package is a thin wrapper around the [Go Cloud](https://gocloud.dev/howto/blob/) package in order to be able to specify AWS S3 credentials using string values. Those values are:

* `env:` – read credentials from AWS defined environment variables.
* `iam:` – assume AWS IAM credentials).
* `{AWS_PROFILE_NAME}`.
* `{AWS_CREDENTIALS_PATH}:{AWS_PROFILE_NAME}`.

For example:

```
s3:///bucket-name?region=us-east-1&credentials=iam:
```

Additionally, if a Go Cloud URI contains a `prefix=` query parameter this package will automatically return a `blob.PrefixedBucket`.

## See also

* https://gocloud.dev/howto/blob/
* https://github.com/aaronland/go-aws-session

repository: cmu-lib/go-iiif-vips

gha_license_id: NOASSERTION

repo_name: cmu-lib/go-iiif-vips


text:
---
layout: archive
title: "Vita"
permalink: /cv/
author_profile: true
redirect_from:
  - /resume
---
{% include base_path %}

My studies led me first into the field of transport sciences, which fascinated me with its many interdependencies and complex networks. Due to the lower proportion of computer science in my studies, I have been studying computer science at the Fernuni Hagen as a second degree course since 2016. At the same time I was given the opportunity to work in computer science at the TU Bergakademie Freiberg after my doctoral thesis.

    since 04/2023
    Research Fellow at Lund University, Robotics and Semantic Systems group

    since 01/2023
    Research Fellow at Freiberg University of Mining & Technology, Ubiquitous Computing & Smart Systems group

    01/2022 - 12/2022
     Research Group Leader in ESF project RoboBoatAssist at Freiberg University of Mining & Technology, Chair of Virtual Reality & Multimedia

    10/2020
    Defense of the doctoral thesis (mark: magna cum laude) "Artificial Neural Networks in Performance Based Airport Management"

    09/2019 - 12/2021
    Research Fellow at Freiberg University of Mining & Technology, Chair of Virtual Reality & Multimedia

    since 09/2015

    Studies of Computer Science at Fernuniversität Hagen (distance learning, part-time)

    09/2015 - 08/2019

    PhD Student at German Aerospace Center (DLR) Brunswick, Institute of Flight Guidance

    08/2015
    Diploma Thesis (mark: 1.4) "Development of a model for the creation of generic flight plans for airport clusters"

    2009 - 2015

    Studies of Traffic Engineering at TU Dresden

    2007

    Abitur at Gymnasium Dresden-Cotta

    1988

    born, son of Prof. Dr. Volker Reitmann

Scholarships & Awards
======

Both in my studies and in my postgraduate career, I was given opportunities for various stays abroad, for which I am very grateful. Through the *ERASMUS* program, I visited the Loughborough University in the United Kingdom for one semester during my studies. During my doctoral studies, I was a visiting scientist at St. Petersburg State University in Russia through the *G-RISC* program. As a PostDoc, I was a visiting scientist at Massachusetts Institute of Technology (MIT) in the USA through the MISTI program.

    09/2022 - 10/2022
      Visiting Scientist at Massachusetts Institute of Technology (MIT), USA

    08/2021
    Best Paper Award at 2nd International Conference on Cyber-Physical Systems and Control (CPS&C) 2021

    06/2019
    Best Paper Award in the Weather Track at 13th USA/Europe ATM R&D Seminar

    11/2018

    Visiting Scientist at TsAGI (Moscow, Russia), IFAR Young Researchers Conference

    07/2018
    Best Conference Paper Award at 8th International conference for Research in Air Transportation (ICRAT)

    04/2017
    Russian-German Young Scientist Award

    10/2016 & 10/2017

    Visiting Scientist at St. Petersburg State University (SPbU), Faculty of Mathematics & Mechanics, Russia

    09/2013 - 04/2014
    ERASMUS Exchange Student at Loughborough University, UK

Misc
======

I am member of the following organizations / societies:

* Chaos Computer Club e. V. (Website)
* Gesellschaft für Informatik e. V. (Website)

Apart from all this, I am an enthusiastic sportsman, video games developer and try myself in FL Studio or LMMS every now and then. I am part of Bonfire Interactive, an indie startup from Aachen and Dresden in Germany, which I founded with my friend Alex.

Bonfire was officially founded in 2018, but work on our game **Pale Blue Dot** already started in 2016. Pale Blue Dot is RTS game with low poly graphics, completelly basing on an open source pipeline (e. g. Blender, Godot, Gimp). To support the development we are looking for freelancing contributors or employees which will help to make Pale Blue Dot happen.

The game is funded by the [nordmedia – Film- und Mediengesellschaft Niedersachsen/Bremen mbH](https://www.nordmedia.de).

For more information visit the official [homepage](https://www.bonfire-Interactive.com).


text:
# inferViz

## Origins

At the rstudio::conf 2018, Di Cook gave a great presentation about inference through visualization. The concept of Few's/Berk's "interoccular traumatic impact" was not new to me, but Di's take was something completely different to me. Interestingly, I remember having done something on mturk related to her presentation; unfortunately, I was not really thinking critically about the meaning of the task at the time.

Shortly after seeing Di's talk, I decided to put something together for the sake of teaching the concept of visual inference to students.

## Installation

This is not on CRAN, but you can get it here:

```r
if(!(require(devtools))) {install.packages("devtools")}

devtools::install_github("saberry/inferviz")
```

## Use

The main function, simviz, was created with the intention of being easy for students to immediately start using.

```r
library(inferviz)

simViz(mtcars, mpg, hp)
```

It is conceptually simple -- plot the real visualization, alongside visualizations with variable values randomized. The principles oftidyeval were used, so that students can just pop in data and variable names (some students who are just beginning the long journey seem to get confused by the \$). This eases the burden on getting the function working and produces visualizations without much effort.

```r
mtcars %>%
  simViz(., mpg, hp)
```

The number of distractors can be increased or decreased from the default of 3:

```r
simViz(mtcars, mpg, hp, distractors = 5)
```

There is also an "interactive" mode for simviz; by setting the answer argument to TRUE, users can select whether they see the real visualization or not.

```r
simViz(mtcars, mpg, hp, distractors = 5, answer = TRUE)
```

```
Are you ready to see the real visual?

1: No
2: Yes
```

In addition to visualization, there is also a basic linear model simulator. It behaves very much in the same way as the visualizations, but instead produces a table of coefficients. Students come armed with the knowledge that *p*-values are the ultimate arbiter of truth in science, so this is a way to nudge them towards looking at the coefficients first. My goal is to have students look at the coefficient patterns and see if they make sense given their hypotheses.

```r
simMod(mtcars, mpg, wt, distractors = 3)
```

```
  (Intercept)         wt
1   19.166464  0.2872518
2   -5.344472 -5.3444716
3   37.285126 37.2851262
4   26.415013 -1.9657745
5   19.523715  0.1762094
```

This prompts students to think through their hypotheses and see if that expected pattern is present.

repository: saberry/inferviz

repo_name: saberry/inferviz


text:
<!-- TITLE: Tree Map -->
<!-- SUBTITLE: -->

# Tree Map

Tree maps display hierarchical (tree-structured) data as a set of nested rectangles. Each branch of
the tree is given a rectangle, which is then tiled with smaller rectangles representing sub-
branches. A leaf node's rectangle has an area proportional to a specified dimension of the data.

General:

| []()                  |                 |
|-----------------------|-----------------|
| Right click           | Context menu    |

![Tree Map](../../uploads/viewers/tree-map.png "Tree Map")

See also:

  * [Viewers](../viewers.md)
  * [Table View](../../overview/table-view.md)
  * [JS API: Tree Map](https://public.datagrok.ai/js/samples/ui/viewers/tree-map)

repository: SoftwareCountry/datagrok-ai-public

repo_name: SoftwareCountry/datagrok-ai-public