text:
# Debugging with Swarming

This document outlines how to debug a test failure on a specific builder
configuration without needing to repeatedly upload new CL revisions or do CQ dry
runs.

[TOC]

## Overview & Terms

*Swarming* is a system operated by the infra team that schedules and runs tasks
under a specific set of constraints, like "this must run on a macOS 10.13 host"
or "this must run on a host with an intel GPU". It is somewhat similar to part
of [Borg], or to [Kubernetes].

An *isolate* is an archive containing all the files needed to do a specific task
on the swarming infrastructure. It contains binaries as well as any libraries
they link against or support data. An isolate can be thought of like a tarball,
but held by the CAS server and identified by a digest of its contents. The
isolate also includes the command(s) to run, which is why the command is
specified when building the isolate, not when executing it.

Normally, when you do a CQ dry run, something like this happens:

```
  for type in builders_to_run:
    targets = compute_targets_for(type)
    isolates = use_swarming_to_build(type, targets) # uploads isolates for targets
    wait_for_swarming_to_be_done()

    for isolate in isolates:
      use_swarming_to_run(type, isolate) # downloads isolates onto the bots used
    wait_for_swarming_to_be_done()
```

When you do a CQ retry on a specific set of bots, that simply constrains
`builders_to_run` in the pseudocode above. However, if you're trying to rerun a
specific target on a specific bot, because you're trying to reproduce a failure
or debug, doing a CQ retry will still waste a lot of time - the retry will still
build and run *all* targets, even if it's only for one bot.

Fortunately, you can manually invoke some steps of this process. What you really
want to do is:

```
  isolate = use_swarming_to_build(type, target) # can't do this yet, see below
  use_swarming_to_run(type, isolate)
```

or perhaps:

```
  isolate = upload_to_cas(target_you_built_locally)
  use_swarming_to_run(type, isolate)
```

## The easy way

A lot of the steps described in this doc have been bundled up into 2
tools. Before using either of these you will need to
[authenticate](#authenticating).

### run-swarmed.py

A lot of the logic below is wrapped up in `tools/run-swarmed.py`, which you can run
like this:

```
$ tools/run-swarmed.py $outdir $target [-- --gtest_filter=...]
```

See the `--help` option of `run-swarmed.py` for more details about that script.
Many flags are converted into dimensions to pass to `mb.py`; see
[Bot selection criteria](#bot-selection-criteria) for possible values. (Most
dimensions have the same name as the flag, but `--swarming-os` is just the `os`
dimension.)

Note you might need `--swarming-os Ubuntu-14.04` if you get an error like,
`UnboundLocalError: local variable 'dbus_pid' referenced before assignment`.

Web tests can be also run on swarmed. However:
- the only supported output folders are `Debug` and `Release`
- `--no-test-flags` must be specified.

For example, you can run all Web Platform Tests inside `<some-wpt-folder>` like this:
```
$ autoninja -C out/Release blink_tests
$ tools/run-swarmed.py --no-test-flags out/Release blink_wpt_tests <some-wpt-folder>
```

### mb.py run

Similar to `tools/run-swarmed.py`, `mb.py run` bundles much of the logic into a
single command line. Unlike `tools/run-swarmed.py`, `mb.py run` allows the user
to specify extra arguments to pass to the test, but has a messier command line.

To use it, run:
```
$ tools/mb/mb.py run \
    -s --no-default-dimensions \
    -d pool $pool \
    $criteria \
    $outdir $target \
    -- $extra_args
```

## A concrete example

Here's how to run `chrome_public_test_apk` on a bot with a Nexus 5 running KitKat.

```sh
$ tools/mb/mb.py run \
    -s --no-default-dimensions \
    -d pool chromium.tests \
    -d device_os_type userdebug -d device_os KTU84P -d device_type hammerhead \
    out/Android-arm-dbg chrome_public_test_apk
```

This assumes you have an `out/Android-arm-dbg/args.gn` like

```
ffmpeg_branding = "Chrome"
is_component_build = false
is_debug = true
proprietary_codecs = true
strip_absolute_paths_from_debug_symbols = true
symbol_level = 1
system_webview_package_name = "com.google.android.webview"
target_os = "android"
use_goma = true
```

## Bot selection criteria

The examples in this doc use `$criteria`. To figure out what values to use, you
can go to an existing swarming run
([recent tasks page](https://chromium-swarm.appspot.com/tasklist)) and
look at the `Dimensions` section. Each of these becomes a `-d dimension_name
dimension_value` in your `$criteria`. Click on `bots` (or go
[here](https://chromium-swarm.appspot.com/botlist)) to be taken to a UI that
allows you to try out the criteria interactively, so that you can be sure that
there are bots matching your criteria. Sometimes the web page shows a
human-friendly name rather than the name required on the commandline. [This
file](https://cs.chromium.org/chromium/infra/luci/appengine/swarming/ui2/modules/alias.js)
contains the mapping to human-friendly names. You can test your commandline by
entering `dimension_name:dimension_value` in the interactive UI.

## Building an isolate

At the moment, you can only build an isolate locally, like so (commands you type
begin with `$`):

```
$ tools/mb/mb.py isolate //$outdir $target
```

This will produce some files in $outdir. The most pertinent two are
`$outdir/$target.isolate` and `$outdir/target.isolated`. If you've already built
$target, you can save some CPU time and run `tools/mb/mb.py` with `--no-build`:

```
$ tools/mb/mb.py isolate --no-build //$outdir $target
```

Support for building an isolate using swarming, which would allow you to build
for a platform you can't build for locally, does not yet exist.

## Authenticating

You may need to log in to `https://chromium-swarm.appspot.com` to do this

```
$ tools/luci-go/isolate login
```

Use your google.com account for this. On Windows the command would be:

```
$ tools\luci-go\isolate.exe login
```

## Uploading an isolate

You can then upload the resulting isolate to the CAS server:

```
$ tools/luci-go/isolate archive \
      -cas-instance chromium-swarm \
      -i $outdir/$target.isolate \
      -dump-json $outdir/$target.archive.json
```

The archive json looks like this:

```
{
  "unit_tests": "5bee0815d2ddd2b876b49d4cce8aaa23de8a6f9e2dbf134ec409fbdc224e8c64/398"
}
```

Do not ctrl-c it after it does this, even if it seems to be hanging for a
minute - just let it finish.

## Running an isolate

Now that the isolate is on the CAS server with digest `$digest` from the
previous step, you can run on bots of your choice:

```
$ tools/luci-go/swarming trigger \
    -server https://chromium-swarm.appspot.com \
    -dimension pool=$pool \
    $criteria \
    -digest $digest
```

There are two more things you need to fill in here. The first is the pool name;
you should pick "chromium.tests" unless you know otherwise. The pool is the
collection of hosts from which swarming will try to pick bots to run your tasks.

The second is the criteria, which is how you specify which bot(s) you want your
task scheduled on. These are specified via "dimensions", which are specified
with `-dimension key=val`. In fact, the `-dimension pool=$pool` in the
command above is selecting based on the "pool" dimension. There are a lot of
possible dimensions; one useful one is "os", like `-dimension os=Linux`. Examples of
other dimensions include:

* `-dimension os=Mac10.13.6` to select a specific OS version
* `-dimension device_type="Pixel 3"` to select a specific Android device type
* `-dimension gpu=8086:1912` to select a specific GPU

The [swarming bot list] allows you to see all the dimensions and the values they
can take on.

And you can pass shard number via env flags, e.g.

* `-env GTEST_SHARD_INDEX=0` to specify which shard to run.
* `-env GTEST_TOTAL_SHARDS=1` to specify total number of shards.

If you need to pass additional arguments to the test, simply add
`-- $extra_args` to the end of the `swarming trigger` command line - anything
after the `--` will be passed directly to the test.

When you invoke `swarming trigger`, it will emit a piece of information: a
command you can run to collect the results of that task. For example:

```
To collect results use:
  swarming collect -server https://chromium-swarm.appspot.com 4a0e739053fddd10
```

The 'collect' command given there will block until the task is complete, then
produce the task's results, or you can load that URL and watch the task's
progress.

## Other notes

If you are looking at a Swarming task page, be sure to check the bottom of the
page, which gives you commands to:

* Download the contents of the isolate the task used
* Reproduce the task's configuration locally
* Download all output results from the task locally

[borg]: https://ai.google/research/pubs/pub43438
[kubernetes]: https://kubernetes.io/
[swarming bot list]: https://chromium-swarm.appspot.com/botlist

To find out repo checkout, gn args, etc for local compile, you can use
[how to repro bot failures](../testing/how_to_repro_bot_failures.md)
as a reference.

repository: chromium/chromium

repo_name: chromium/chromium


text:
# update-service-operator

This operator is developed using the [operator SDK][operator-sdk], version 1.9.0.
Installation docs are [here][operator-sdk-installation].

## Run locally

To run locally, you must set the operand image as shown below.

```
export RELATED_IMAGE_OPERAND="quay.io/app-sre/cincinnati:2873c6b"
operator-sdk run --local
```

## Using an init container to load graph data

The UpdateService graph data is loaded from an init container. Before deploying
the update-service-operator, you will need to [build and push an init container containing the graph data](docs/graph-data-init-container.md).

## Deploy operator

```
make deploy
```

By default, operator will be deployed using the default operator image `quay.io/updateservice/update-service-operator:latest`. If you want to override the default operator image with your image, set

```
export RELATED_IMAGE_OPERATOR="your-registry/your-repo/your-update-service-opertor-image:tag"
```

## Run functional tests

```
make func-test
```

To run the functional testcases locally, you must set below environment variables as shown below along with optional `RELATED_IMAGE_OPERAND` and `RELATED_IMAGE_OPERATOR`.

```
export KUBECONFIG="path-for-kubeconfig-file"
export GRAPH_DATA_IMAGE="your-registry/your-repo/your-init-container:tag"
```

## Run unit tests

```
make unit-test
```

[operator-sdk]: https://sdk.operatorframework.io/docs/
[operator-sdk-installation]: https://v1-9-x.sdk.operatorframework.io/docs/installation/

## Generating OLM manifests

Here are the steps to generate the operator-framework manifests in the bundle format
* Set the `VERSION` value in the shell
* Set the `IMG` value pointing to the OSUS operator which should be part of the operator bundle.
* Run `make bundle`.

Example:

```sh
VERSION=4.9.0
IMG=registry.com/cincinnati-openshift-update-service-operator:v4.6.0
make bundle
```

## Documentation
* [Deploy disconnected update service](./docs/disconnected-updateservice-operator.md)
* [External registry CA injection](./docs/external-registry-ca.md)
* [Using graph data init container](./docs/graph-data-init-container.md)

repository: openshift/cincinnati-operator

branch_name: refs/heads/master

extension: md

gha_language: Go

license_type: permissive

repo_name: openshift/cincinnati-operator


text:
---
title: PlatformCF - Day One
layout: post
date: 2013-09-10
---

My last few hours have been spent outside in the beautiful fall evening
in Santa Clara.  While enjoying a steam beer just miles away from its
origins (both generally in
[concept](http://en.wikipedia.org/wiki/Steam_beer) and specifically to
the [contents of my
bottle](http://www.anchorbrewing.com/beer/anchor_steam)), I’ve shared
ideas about the history of beans in Java, the advantages of git rebase
before pushing changes, and what sorts of pets people would have(of
which wildcats, dolphins, and wallabies were all accepted answers).  I’m
at PlatformCF, the first of many Cloud Foundry conferences.

The conference started midday and, although it’s Sunday, the turnout was
fantastic.  After a tasty lunch and time to connect with others, James
Watters kicked things off with a concise and effective introduction.
Leaving the question of "what could have brought 500 developers here on
a Sunday?" open, the intro was a well-placed energetic ramp to the
first talk by Jonathan Murray.  Murray, EVP and CTO of Warner Music
Group, presented a look at how Warner Music Group scrapped their legacy
code and “rebuilt the plane in the air” (switching out an engine seems
like a breeze compared to rebuilding the whole thing).  The talk exposed
some problems with adapting legacy code through time and technological
changes and the fact that, at some point, it’s more affordable to start
from scratch.  His arguments were well reasoned and persuasive and his
message was exciting.  “Now lets jump off a bridge,” he could have said,
and a line would have formed.  And what he was suggesting is sort of
like jumping off a bridge: Start from a blank sheet? ...with all that
legacy functionality to support? ..and new features being demanded by
the business at the same time?  That being said, I’d ditch the bridge
for a set of wings any day.

The conference continued in excitement with both half-hour keynotes and
10 minute lightning talks.  Thanks to a bare-footed Mike Gehard, I’m
unbearably curious about Go(and smiling about his playful jabs at Ruby,
the JVM, Maven, and JavaScript).  Ismael Ghalimi showed off STOIC, his
Node.js app (take that Mike!) that creates apps from spreadsheets. Yeah,
you heard that right, apps from spreadsheets.  Matthew Kocher’s planned
talk ended a bit early and he shared a compelling example of disaster
recovery at Cloud Foundry involving the redeployment of *everything* to
another Amazon availability zone after Amazon’s recent outage.  Ben Hale
bravely, and quite successfully, spent his lightning talk on the Java
build-pack live coding, effectively demonstrating how quick and easy it
is to customize the deployment.  He repeatedly stressed “Just say no!”
to config files, and I’m wondering if he’ll turn out like Larry “no
modes” Tesler (anybody seen Hale’s license plate?).  These are all just
tiny slices of the day, the full experience being too much to recount!
I’m looking forward to another day combining keynotes, lightning talks,
and… an unconference? Unconference within a conference. Hmm. Sounds
evil.  Doesn’t matter, I’m sure it’ll be great!

A few references mentioned not to miss:

https://developers.helloreverb.com/swagger/ - recommended by Jonathan Murray for documenting and testing your REST API
http://www.amazon.com/Continuous-Delivery-Deployment-Automation-Addison-Wesley/dp/0321601912 - recommended by Andrew Crump
http://www.cs.ucf.edu/courses/cop4020/sum2009/CSP-hoare.pdf - mentioned by barefoot Mike and a strong influence in the creation and development of the Go programming language.

license: MIT


text:
# Things to do:

## TODO, rename the package to filterlogs

1. Modify config structure to allow grouping for different use cases or apps.
    - filterlogs should be smart enough to identity the app only just through logs.
    - there should strictly be no extra column
    - read config from multiple yaml files [DONE]

2. Support N/F and N/A
    - A field is N/F if its tag is given but not found in the line
    - A field is N/A if it is not applicable for that record
    - So, if there is N/F then there is something wrong with the config

3. Support log blocks
    - Each log block will go into a single csv record.
    - This would require specifying StartBlockPattern and EndBlockPattern

4. Easy and automated config generation using some TUI or interactive menu

5. Support switching into the loglines and the output csv or go to corresponding log block
    - It should be pretty fast to switch in and switch out
    - We can also show a height configured mini logs display at the bottom of the screen
    - To make it efficient, we should be able to catch the data fast and should be able to read the huge logs in both the directions

6. Support a live mode
    - Display csv records as logs appear in real time.
    - This is like a real time dashboard that will monitor a file and generate the dashboard

## Small features
- Read config from different yaml files and combine them. There is a lot of error handling required here
- Do not display a column if that field is empty
- Make filterlogs a library which can be imported into another projects for data parsing and keys could be in the config so that the app can use it from the map returned by filterlogs
- Show logLines and their patterns

repository: parmaanu/tocsv

branch_name: refs/heads/main

gha_license_id: MIT

repo_name: parmaanu/tocsv


text:
# ReadMe Generator

## Description
A simple Node package that can generate a README.md file for github repositories

## Installation
You will need to install the inquirer package using the comand line.  Use this for documentation on how (https://www.npmjs.com/package/inquirer)

You will also need to use the normal node install by typing "npm install" in your command line in the root directory

## Usage
Simply follow the prompts once you start the process by typing in "node index.js" into the command line while in the root directory

## Credit
https://github.com/Will2tall

## License
MIT License

Copyright (c) [2021] [William Haas]

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

 IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

## Badge
![Azure DevOps coverage (branch)](https://img.shields.io/azure-devops/coverage/swellaby/opensource/25/master?color=purple&label=JavaScript&style=plastic)


text:
---
title: Mocking DLL Imports
page_title: Mocking DLL Imports | JustMock Documentation
description: Mocking DLL Imports
previous_url: /advanced-usage-mocking-dll-imports.html
slug: justmock/advanced-usage/mocking-dll-imports
tags: mocking,dll,imports
published: True
position: 10
---

# Mocking DLL Imports

In elevated mode, you can use __Telerik® JustMock__ to mock imported functions (decorated with the `[DLLImport()]` attribute).

> This feature is available only in the commercial version of Telerik JustMock. Refer to [this]({%slug justmock/getting-started/commercial-vs-free-version%}) topic to learn more about the differences between both the commercial and free versions of Telerik JustMock.

In the next examples we will use the following sample class to test:

  #### __[C#]__

  {{region DLLImport#SUT}}
    public class Foo
    {
        [DllImport("Kernel32.dll")]
        public static extern int GetCurrentProcessId();

        [DllImport("user32.dll", CharSet = CharSet.Auto)]
        public static extern int MessageBox
           (IntPtr hWnd, String text, String caption, uint type);

        [DllImport("Kernel32.dll")]
        private static extern bool IsWow64Process();

        public string FormatCurrentProcessId()
        {
            var myPId = GetCurrentProcessId();
            string returnValue = string.Format("The current process ID is {0}", myPId);
            return returnValue;
        }

        public void Start()
        {
            var msgBoxRet = MessageBox(new IntPtr(0), "Process Starts Now!", "Message Dialog", 0);

            throw new NotImplementedException();
        }

        public string Is64BitProcessMessage()
        {
            bool is64Bit = IsWow64Process();

            if (is64Bit)
            {
                return string.Format("This is a 64 bit process!");
            }
            return string.Format("This is a 32 bit process!");
        }
    }
  {{endregion}}

  #### __[VB]__

  {{region DLLImport#SUT}}
    Public Class Foo
        <DllImport("Kernel32.dll")>
        Public Shared Function GetCurrentProcessId() As Integer
        End Function

        <DllImport("user32.dll", CharSet:=CharSet.Auto)>
        Public Shared Function MessageBox(hWnd As IntPtr, text As [String], caption As [String], type As UInteger) As Integer
        End Function

        <DllImport("Kernel32.dll")>
        Private Shared Function IsWow64Process() As Boolean
        End Function

        Public Function FormatCurrentProcessId() As String
            Dim myPId = GetCurrentProcessId()
            Dim returnValue As String = String.Format("The current process ID is {0}", myPId)
            Return returnValue
        End Function

        Public Sub Start()
            Dim msgBoxRet = MessageBox(New IntPtr(0), "Process Starts Now!", "Message Dialog", 0)

            Throw New NotImplementedException()
        End Sub

        Public Function Is64BitProcessMessage() As String
            Dim is64Bit As Boolean = IsWow64Process()

            If is64Bit Then
                Return String.Format("This is a 64 bit process!")
            End If
            Return String.Format("This is a 32 bit process!")
        End Function
    End Class
  {{endregion}}

> To mock DLL imported members and types you first need to go to elevated mode by enabling JustMock from the menu.  [How to Enable/Disable](./advanced-usage#how-to-enabledisable-telerikjustmock)

## Mocking the Standard DLL Import

> **Important**
>
>  __TelerikJustMock will not instrument assemblies, containing only DLL import functions.__

We will start by testing the `FormatCurrentProcessId` function. As you can see, this method is getting the current process ID, formatting it in a way and finally returning the formatted string. To write the necessary tests for these units should not be hard. You will need:

1. To pass certain parameters.
1. To execute the function.
1. To check if the results are the same as the expected.

The interesting part comes right with the first point.

### How can you control the current process ID?

In the past this was achieved by wrapping the static `GetCurrentProcessId` method and arranging against the wrapper. However, this can become a tiresome operation.

With JustMock you are now able to directly arrange against the imported method just as you are arranging static functions (check [Static Mocking]({%slug justmock/advanced-usage/static-mocking%}) for more details).

Next is an example, demonstrating how easy is to arrange functions, decorated with the `[DLLImport()]` attribute:

  #### __[C#]__

  {{region DLLImport#CurrentPorcessId}}
    [TestMethod]
    public void FormatCurrentProcessId_OnExecute_ShouldReturnExpected()
    {
        var expected = 3500;

        // Arrange
        Mock.Arrange(() => Foo.GetCurrentProcessId()).Returns(expected);

        // Act
        var myFoo = new Foo();
        var actual = myFoo.FormatCurrentProcessId();

        // Assert
        Assert.AreEqual(string.Format("The current process ID is {0}", expected), actual);
    }
  {{endregion}}

  #### __[VB]__

  {{region DLLImport#CurrentPorcessId}}
    <TestMethod>
    Public Sub FormatCurrentProcessId_OnExecute_ShouldReturnExpected()
        Dim expected = 3500

        ' Arrange
        Mock.Arrange(Function() Foo.GetCurrentProcessId()).Returns(expected)

        ' Act
        Dim myFoo = New Foo()
        Dim actual = myFoo.FormatCurrentProcessId()

        ' Assert
        Assert.AreEqual(String.Format("The current process ID is {0}", expected), actual)
    End Sub
  {{endregion}}

Here we setup that `Foo.GetCurrentProcessId()` should return expected value when called. Thus we override the original method behavior with a one that we specify. Then we act against our method under test. And finally we assert the expectations.

## Isolating the System Under Test

Let's assume we need to isolate certain function which is depending on an extern imported method. Such is the `Start` method from our example (`Foo`) class. In it, we aim to execute some process, but warn with message before that. To test the execution logic (which in this case is insignificant), we will need to prevent the showing of the `MessageBox`. Again, using the commercial version of JustMock you are able to arrange directly against it, like so:

  #### __[C#]__

  {{region DLLImport#MessageBox}}
    [TestMethod]
    [ExpectedException(typeof(NotImplementedException))]
    public void Start_OnExecute_ShouldThrowNotImplementedException()
    {
        // Arrange
        Mock.Arrange(() => Foo.MessageBox(Arg.IsAny<IntPtr>(), Arg.AnyString, Arg.AnyString, Arg.IsAny<uint>())).DoNothing();

        // Act
        var myFoo = new Foo();
        myFoo.Start();
    }
  {{endregion}}

  #### __[VB]__

  {{region DLLImport#MessageBox}}
    <TestMethod>
    <ExpectedException(GetType(NotImplementedException))>
    Public Sub Start_OnExecute_ShouldThrowNotImplementedException()
        ' Arrange
        Mock.Arrange(Function() Foo.MessageBox(Arg.IsAny(Of IntPtr)(), Arg.AnyString, Arg.AnyString, Arg.IsAny(Of UInteger)())).DoNothing()

        ' Act
        Dim myFoo = New Foo()
        myFoo.Start()
    End Sub
  {{endregion}}

Here we specify that when `Foo.MessageBox` is called it should do nothing. Note that we are using the same matchers as in any other JustMock arrangement. Having this, we can continue with the rest of our test.

## Mock Private DLL Import

In many cases, you will find that the imported method/member needs to be set to non-public (`private`, `protected`, `internal`, etc.). Such scenarios is applied in the `Is64BitProcessMessage()` method from our example (`Foo`) class. To test this function we will directly arrange against the imported method, but using the [Mocking LINQ Queries]({%slug justmock/advanced-usage/mocking-linq-queries%}) feature of JustMock.

Here are the complete tests:

  #### __[C#]__

  {{region DLLImport#IsWiw64ProcessFirst}}
    [TestMethod]
    public void Is64BitProcessMessage_On64BitExecute_ShouldReturnExpected()
    {
        // Arrange
        Mock.NonPublic.Arrange<bool>(typeof(Foo), "IsWow64Process").Returns(true);

        // Act
        var myFoo = new Foo();
        var actual = myFoo.Is64BitProcessMessage();

        // Assert
        Assert.AreEqual("This is a 64 bit process!", actual);
    }
  {{endregion}}

  #### __[VB]__

  {{region DLLImport#IsWiw64ProcessFirst}}
    <TestMethod>
    Public Sub Is64BitProcessMessage_On64BitExecute_ShouldReturnExpected()
        ' Arrange
        Mock.NonPublic.Arrange(Of Boolean)(GetType(Foo), "IsWow64Process").Returns(True)

        ' Act
        Dim myFoo = New Foo()
        Dim actual = myFoo.Is64BitProcessMessage()

        ' Assert
        Assert.AreEqual("This is a 64 bit process!", actual)
    End Sub
  {{endregion}}

  #### __[C#]__

  {{region DLLImport#IsWiw64ProcessSecond}}
    [TestMethod]
    public void Is64BitProcessMessage_On32BitExecute_ShouldReturnExpected()
    {
        // Arrange
        Mock.NonPublic.Arrange<bool>(typeof(Foo), "IsWow64Process").Returns(false);

        // Act
        var myFoo = new Foo();
        var actual = myFoo.Is64BitProcessMessage();

        // Assert
        Assert.AreEqual("This is a 32 bit process!", actual);
    }
  {{endregion}}

  #### __[VB]__

  {{region DLLImport#IsWiw64ProcessSecond}}
    <TestMethod>
    Public Sub Is64BitProcessMessage_On32BitExecute_ShouldReturnExpected()
        ' Arrange
        Mock.NonPublic.Arrange(Of Boolean)(GetType(Foo), "IsWow64Process").Returns(False)

        ' Act
        Dim myFoo = New Foo()
        Dim actual = myFoo.Is64BitProcessMessage()

        ' Assert
        Assert.AreEqual("This is a 32 bit process!", actual)
    End Sub
  {{endregion}}

Note that, in this case we are mocking a non-public, static, imported function. This clearly shows what can be achieved and on what cost, with JustMock.

## See Also

 * [Static Mocking]({%slug justmock/advanced-usage/static-mocking%})
 * [Mocking LINQ Queries]({%slug justmock/advanced-usage/mocking-linq-queries%})

repository: Prophasi/justmock-docs

repo_name: Prophasi/justmock-docs


text:
# Workflow of release new version of services
This doc describes the flow of release mircro-services and e-commerce.

## Tips: How to create branch
we usually got three kind of reasons to create branch
- Create branch for new version
- Create branch for a new change
- Create branch for a hot fix

### Create branch for new version
  Each time, we want to upgrade one service, we need to create a new version for this.

- Naming:

  Name the branch `dev`, and change the version of this branch to the version you need, such as `0.0.4`
- Work on this branch:

  Each version may contains many small changes, such as `add a function`, `update a function`, for each change, create a branch from this `dev` branch, and after the branch finished, then merge it to the `dev`
- Merge:

  After the version tested, then you can merge it to master

### Create branch for new change
  Usually we use this kind of changes on `dev` branch.

### Create branch for hot fix
  When the master got a error, and we need to fix it as soon as possible. We create a branch from `master`, and when we fixed, then merge it to master, and change the small version of master.

# 1. Add new branch
Before start to work on a new release, we should figure out the version of this release, and create a branch from master, and change the new brench's version.

# 2. Work on the release
Each release may got several changes, we need to create new branch from this for each change.
- Create sub branch based on this branch
- Work on this branch
- Merge the sub branch: use suqsh to merge, so each sub change only got one commit

# 3. Test the on dev
Before we merge the release to master, we must test it on `dev`.

# 4. Merge to master
After tested, we merge this to master, and deploy it to `staging`.

# 5. Test on staging
Test the on staging.

# 6. Create tag and release branch from master
After test the versin on staging, then release it.

repository: lixiawss/development-process

repo_name: lixiawss/development-process


text:
# OpenWeatherMapSpecFlow

## Running the Tests

To run the tests please open the solution in Visual Studio 2019, compile it and open the **Test Explorer**.

You also need to create a `.env` files at the project level with the following structure:

```
OWA_API_ID=YOUR_OWA_ID
```

You can use the existing `template.env` file but you must change the `YOUR_OWA_ID` to your personal OpenWeather API key.

Click **Run All Tests**. In case of running of the first time you will need to setup your SpecFlow account (see https://specflow.org/getting-started/ for details).

![Test Explorer](docs/test-explorer.png)

In case of problems you might need to set the `MSBUILDSINGLELOADCONTEXT` environment variable in your system to **1**:
```
# When using Git Bash or any Linux terminal
export MSBUILDSINGLELOADCONTEXT=1
```

You can find the reults for the specific tests from the **Tests Explorer** when you click on "Open additional output for this result":
![Tokyo Test Result](docs/tokyo-test-result.png)

The information we're looking for will be displayed at the bottom of the test run output:
```
 The hottest day and time for Tokyo is Friday (05/06/2020 06:00 UTC) with temperature of 28.4 celsius.
```

## What is good?

* Good separation of responsibilities (feature files, steps, handlers, data models)
* Reusable steps for other test scenarios

## What could be done to make this better?

* Getting the tests running in an Azure DevOps Pipeline with results posted in Slack/Teams/etc
* Unit testing the data processing logic (internal test project)
* Scenario context injection so that steps from different step definition classes can use it
* Display result time in the console for specific Time Zones rather than UTC

repository: miroslawmajka/OpenWeatherMapSpecFlow

repo_name: miroslawmajka/OpenWeatherMapSpecFlow


text:
# DecisionIntervention
Proof of concept project done for a [playful design method graduate course at the University of Illinois at Champaign-Urbana](https://ischool.illinois.edu/degrees-programs/courses/is436)

## Designer Collaborators
Karla G. Sanabria Véaz  (Graduate Student in Department of Linguistics)

![DecisionIntervention](MyRepoFiles/gameWalkthrough.gif)

## Empathy building
By presenting the user with the role of a law maker, one is presented important decisions that have an impact on people. Once the decision is made, the user is transported to the circumstances that happened due to that decisions.

## Unity Project
The project was build to webgl using WebXR specifications, so it could be used through mobilde, desktop and virtual reality. It offer users the ability to test it when it was being presented (I provided the project here, but if there is interest for a live demo, I can rebuild it again, and share it here).

## Features Developed
- A modular dialogue system was created for this game to allow to customize a customized storyline within Unity.
- Funcionality was developed to stream Videos stored in Box through the experience.
- Simple assets were created using Blender to box out the environment. By using an image of a U.S senator office, I attempted to recreate it in 3D.

repository: davtamay/DecisionIntervention

repo_name: davtamay/DecisionIntervention


text:
You might as well give up asking me now. I
>have no intention of doing anything for you. You can rot so far as I'm
>concerned.

1. Is that rot with a key of 13? (ObCrypto)

2. More importantly, has anyone seen the "clueless" mailing list
recently? This is the one where you forge subscriptions for other
people, (who need to be qualified first, refer to the name of the
mailing list) then send it obvious trolls and watch it go
super-critical. Endless hours of fun.

Chris McAuliffe<EMAIL_ADDRESS>(No, not that one.)

-----BEGIN PGP SIGNATURE-----
Version: 2.6

iQCVAwUBMXRRDIHskC9sh/+lAQHR5wP9G/uAwitozfQ2Zlc4EXRfuAMxhF14ouyn
9S/nRYPiBiGOOUJRDWMGsMNUfLg+a3pqeg6m1poI2fGomIrJDvbw8cupJq75XVUo
eosjY0vMZXSeX2Ck+3c+Use/hyDZQ2AdsMTns4KMsWF3kuHDKqrwAhMBdepkhbWh
TYPqps8v8GU=
=io4U
-----END PGP SIGNATURE-----

{% endraw %}
```

## Thread

+ Return to [April 1996](/archive/1996/04)

+ Return to "["Adam Pingitore" <Adam_Pingitore@alli.wnyric.org>](/authors/adam_pingitore_adam_pingitore_at_alli_wnyric_org_)"
+ Return to<EMAIL_ADDRESS>(Chris McAuliffe)](/authors/cmca_at_alpha_c2_org_chris_mcauliffe_)"
+ Return to "["Perry E. Metzger" <perry@piermont.com>](/authors/perry_e_metzger_perry_at_piermont_com_)"

+ 1996-04-17 (Wed, 17 Apr 1996 11:15:41 +0800) - [Re: on corporations and subpoenas](/archive/1996/04/1882e758ea0dc78c504577ce63d38b9714e58f0adf56cbf23baa92be71654a93) - _"Adam Pingitore"<EMAIL_ADDRESS>  + 1996-04-17 (Wed, 17 Apr 1996 13:50:28 +0800) - [Re: on corporations and subpoenas](/archive/1996/04/398c831a4075f0aa4e7c6b8c4ac17d641688e0be5588e0f3c515db58c5f6b752) - _"Perry E. Metzger"<EMAIL_ADDRESS>    + 1996-04-17 (Wed, 17 Apr 1996 16:50:11 +0800) - Re: on corporations and subpoenas -<EMAIL_ADDRESS>(Chris McAuliffe)_


text:
> This project has two homes.
> It is ok to work in github, still, for a better decentralized web
> please consider contributing (issues, PR, etc...) throught:
>
> https://gitlab.esy.fun/yogsototh/install-haskell

---

install-haskell
===============

A script to install Haskell and minimize cabal hell.

If you are on Linux install `libgmp-dev`, `bzip2` and `zsh`.

```
curl https://raw.githubusercontent.com/yogsototh/install-haskell/master/install-haskell.sh | sudo zsh
```

If something goes wrong:

```
export SUDO_USER=$USER; curl https://raw.githubusercontent.com/yogsototh/install-haskell/master/install-haskell.sh | sudo zsh
```

repository: yogsototh/install-haskell

repo_name: yogsototh/install-haskell


text:
# 3DB: A framework for analyzing computer vision models with simulated data

## Installation

Follow instructions on: https://github.com/3db/installers

## Complete demo

For detailed, step by step demonstration of the usage of the framework, please visit: https://github.com/3db/demo

## Documentation

You can find in-depth documentation for the package [here](LINK), including a
quickstart guide, an explanation on the internal layout of the package, as well
as guides for extending 3DB to new modalities, tasks, and output formats.

## Citation

If you find 3DB helpful, please cite it as:
```
@inproceedings{leclerc2021three,
    title={3DB: A Framework for Chungus Chungus},
    author={Guillaume Leclerc, Hadi Salman, Andrew Ilyas, Sai Vemprala, ..., Ashish Kapoor, Aleksander Madry},
    year={2021},
    booktitle={Arxiv preprint arXiv:TODO}
}
```


text:
# Come to the Duck Side

Come To The Duck Side and join the Laser-Raptor forces! Mount a defending laser-tower and fend off the ducks invading the raptor paradise! Combine your powers with one of your buddies and unleash powerful laser combinations!

Made during the [Klagenfurt Game Jam 2019](https://www.itec.aau.at/gamejam/).

### [Play on itch.io](https://kruemelkatze.itch.io/come-to-the-duck-side)

repository: Boyquotes/Come-to-the-Duck-Side

repo_name: Boyquotes/Come-to-the-Duck-Side


text:
## Pretrained Models

|                 | setting | #P    | GFLOPs | PyTorch | Gluon |
|-----------------|---------|-------|--------|---------|-------|
| ResNeSt-50-fast | 1s1x64d | 26.3M | 4.34   | 80.33   | 80.35 |
| ResNeSt-50-fast | 2s1x64d | 27.5M | 4.34   | 80.53   | 80.65 |
| ResNeSt-50-fast | 4s1x64d | 31.9M | 4.35   | 80.76   | 80.90 |
| ResNeSt-50-fast | 1s2x40d | 25.9M | 4.38   | 80.59   | 80.72 |
| ResNeSt-50-fast | 2s2x40d | 26.9M | 4.38   | 80.61   | 80.84 |
| ResNeSt-50-fast | 4s2x40d | 30.4M | 4.41   | 81.14   | 81.17 |
| ResNeSt-50-fast | 1s4x24d | 25.7M | 4.42   | 80.99   | 80.97 |

### PyTorch Models

- Load using Torch Hub

```python
import torch
# get list of models
torch.hub.list('zhanghang1989/ResNeSt', force_reload=True)

# load pretrained models, using ResNeSt-50-fast_2s1x64d as an example
net = torch.hub.load('zhanghang1989/ResNeSt', 'resnest50_fast_2s1x64d', pretrained=True)
```

- Load using python package

```python
# using ResNeSt-50 as an example
from resnest.torch import resnest50_fast_2s1x64d
net = resnest50_fast_2s1x64d(pretrained=True)
```

### Gluon Models

- Load pretrained model:

```python
# using ResNeSt-50 as an example
from resnest.gluon import resnest50_fast_2s1x64d
net = resnest50_fast_2s1x64d(pretrained=True)
```

repository: WenmuZhou/ResNeSt

repo_name: WenmuZhou/ResNeSt


text:
# KKPuzzles
----------
The purpose of this project is to provide simple Jigsaw puzzle game you could easily add to your application and enrich it with a portion of entertainment.

## Installation
-------------
KKPuzzles is available through [CocoaPods](http://cocoapods.org). To install, simply add the following line to your Podfile:

```ruby
pod 'KKPuzzles'
```

## How to use
-----------
Sample of usage is provided here: [Demo](https://github.com/kkuc/KKPuzzles/tree/master/Example)

## Author
-------
Krzysztof Kuc<EMAIL_ADDRESS>
## License
--------
KKPuzzles is available under the MIT license. See the LICENSE file for more info.

repository: kkuc/KKPuzzles

gha_language: Objective-C

repo_name: kkuc/KKPuzzles


text:
# Icon Component

This is the set of ~16 icons used internally within baseui that are safe to open source. This is not the set of uber-specific icons published internally as @uber/icons (though the API is intended to align with that to reduce confusion).

![image](https://user-images.githubusercontent.com/875591/45131331-51e8c500-b141-11e8-898d-ead83d44b345.png)

## Usage

```javascript
import * as React from 'react';
import {LeftArrow} from 'baseui/icon/left-arrow';

export default () => (
  <LeftArrow size="10px" color="#ccc" />
);
```

## Exports

* `Icon` (Base Icon component used by other exports below)

From left-to-right based on the image above:

* `DeleteAlt`
* `Search`
* `Delete`
* `Check`
* `ArrowUp`
* `ArrowDown`
* `ArrowLeft`
* `ArrowRight`
* `TriangleRight`
* `TriangleDown`
* `Upload`
* `Plus`
* `Grab`
* `Alert`
* `Menu`
* `ChevronRight`
* `ChevronLeft`
* `Spinner`
* `Overflow`
* `TriangleLeft`
* `TriangleUp`
* `Filter`
* `CheckIndeterminate`

## `Icon` API

* `size: number|string` - Optional, Defaults to `scale500` (12px)
  Size of element, will be passed to the svg width/height style. Can also be a baseui `theme.sizing` constant like `sizing500`.
* `color: string` - Optional, Defaults to `currentColor`
  Color of icon, will be used as svg fill
* `title: string` - Optional
  Allows you to override the SVG `<title>` label, which is used for accessibility
* `overrides: {Svg}`
  Override the implementation, styles, etc of the component internals.

## Dependencies

No dependencies

## Accessibility

Icons should have an `<title>` attribute in the SVG that describes the icon. If the icon is on a clickable element (button, etc) customer should add their own `aria-label` to that element.

<https://simplyaccessible.com/article/7-solutions-svgs/> (See #2, #6)

<https://www.w3.org/TR/svg-aam-1.0/>

repository: nudge/baseui

repo_name: nudge/baseui


text:
---
layout: post
title: 'Elixir, Mongo, Regex'
categories:
- Elixir
tags: []
---
I built a Phoenix app for my company that provides lots of administrative insight into
our system. It gives operations and support folks some information beyond what's provided
in our traditional UI. It also tries to diagnose common problems and provide suggestions on
how they can be fixed (or indications of what else to examine).

The majority of the data in our system is stored in Postgres. That worked really well for
me in using Phoenix and Elixir since Postgres is very well supported by the [Postgrex driver](https://hex.pm/packages/postgrex).
I haven't had any issues with it. So, many thanks to everyone that worked on that.

I had a new request to provide a page that would have to read from a Mongo database. I
hadn't explored the Mongo driver situation in Elixir before. I got it to work fairly
quickly but its a lot less smooth and seemingly less mature than Postgrex. That may just
be my lack of exposure to Mongo in general. I've always lived in the relational database
world and although I've played with Mongo I've never looked at it for anything serious
and didn't know it nearly as well as I probably should have. This post is about various
things I found out. Some of them (or maybe all of them) may be no big news for folks that
use Mongo on a regular basis. But if you are more like me and don't know that much it might
be helpful. As in the case of the Postgres driver many thanks to all the developers that
have contributed their time and effort to get the Mongo driver to its current state.

[hex.pm](https://hex.pm/) is the repo for Elixir. Its fairly awesome. One thing that's
great about this is you can easily see what is the most popular library for what you are
interested in. In the case of Mongo it looks like the clear favorite is [mongodb](https://hex.pm/packages/mongodb).
The latest version is 0.2.0 created November 11, 2016.

I created a sample project to fiddle around with :

```
mix new mongo_test
```

Then I edited the mix.exs file to add the dependencies suggested in the Mongodb doc :

```
defp deps do
  [{:mongodb, "~> 0.2.0"},
   {:poolboy, ">= 0.0.0"}]
end
```

I had a mongo instance running locally. I populated a new collection with some test data :

```
db.testData.insert({"x" : "Hello" })
db.testData.insert({"x" : "Goodbye" })
db.testData.insert({"x" : "Frank" })
db.testData.insert({"x" : "Fred" })
db.testData.insert({"x" : "Four" })
db.testData.insert({"x" : "ourF" })
db.testData.insert({"x" : "ourf" })
db.testData.insert({"x" : "Foot" })
db.testData.insert({"x" : "123324"})
db.testData.insert({"x" : "123324 ABC"})
db.testData.insert({"x" : "frank"})
```

and then I fired up iex with iex -S mix to try and connect to the database and
search this collection. Starting a process that provides the connection to Mongo
is straightforward. You can name the pid with an atom so that you can reference
that in the future instead of keeping the pid around and having to manage it.

```
iex(1)> {:ok, _} = Mongo.start_link(database: "test", name: :mongo)
iex(2)> :mongo |> Mongo.find("testData", %{}) |> Enum.to_list()
  [%{"_id" => #BSON.ObjectId<58d4b1d6a6ba445542adc9d3>, "x" => "Hello"},
   %{"_id" => #BSON.ObjectId<58d4b1d6a6ba445542adc9d4>, "x" => "Goodbye"},
   %{"_id" => #BSON.ObjectId<58d4b1d6a6ba445542adc9d5>, "x" => "Frank"},
   %{"_id" => #BSON.ObjectId<58d4b1d6a6ba445542adc9d6>, "x" => "Fred"},
   %{"_id" => #BSON.ObjectId<58d4b1d6a6ba445542adc9d7>, "x" => "Four"},
   %{"_id" => #BSON.ObjectId<58d4b1d6a6ba445542adc9d8>, "x" => "ourF"},
   %{"_id" => #BSON.ObjectId<58d4b1d6a6ba445542adc9d9>, "x" => "ourf"},
   %{"_id" => #BSON.ObjectId<58d4b1d6a6ba445542adc9da>, "x" => "Foot"},
   %{"_id" => #BSON.ObjectId<58d4b1d6a6ba445542adc9db>, "x" => "123324"},
   %{"_id" => #BSON.ObjectId<58d4b1d6a6ba445542adc9dc>, "x" => "123324 ABC"},
   %{"_id" => #BSON.ObjectId<58d4b1d6a6ba445542adc9dd>, "x" => "frank"}]
```

So that seemed pretty straightforward. I then tried to make regular expressions
work. This had me stumped for hours. There is a BSON.Regex in mongodb and I "knew"
I had to use that but if you look at the test cases for mongodb you'll see that
the two tests related to BSON.Regex just form two different ones and then inspect
them to ensure they are setup properly.

```
test "inspect BSON.Regex" do
  value = %BSON.Regex{pattern: "abc"}
  assert inspect(value) == "#BSON.Regex<\"abc\">"

  value = %BSON.Regex{pattern: "abc", options: "i"}
  assert inspect(value) == "#BSON.Regex<\"abc\", \"i\">"
end
```

so I tried to do something like this :

```
iex(2)> :mongo |> Mongo.find("testData", %{x: %BSON.Regex{pattern: "F"}}) |> Enum.to_list()
iex(3)> :mongo |> Mongo.find("testData", %{x: %BSON.Regex{pattern: "F"}}) |> Enum.to_list()
** (ArgumentError) argument error
                    :erlang.iolist_size(["", 11, ["x", 0], [["F", 0], nil, 0]])
          (mongodb) lib/bson/encoder.ex:108: BSON.Encoder.document/1
           (elixir) lib/enum.ex:1229: Enum."-map/2-lists^map/1-0-"/2
    (db_connection) lib/db_connection.ex:1001: DBConnection.encode/4
    (db_connection) lib/db_connection.ex:635: DBConnection.execute/4
          (mongodb) lib/mongo.ex:368: Mongo.raw_find/5
          (mongodb) lib/mongo/cursor.ex:34: anonymous fn/6 in Enumerable.Mongo.Cursor.start_fun/6
           (elixir) lib/stream.ex:1242: anonymous fn/5 in Stream.resource/3
```

Well, nope. I'm doing something wrong but what? The problem is that the definition of
BSON.Regex doesn't define default values for the two parts of the defstruct : pattern and
options. But, the code that uses BSON.Regex expects that both of them will be there. So,
the following works fine :

```
iex(4)> :mongo |> Mongo.find("testData", %{x: %BSON.Regex{pattern: "F", options: ""}}) |> Enum.to_list()
[%{"_id" => #BSON.ObjectId<58d4b1d6a6ba445542adc9d5>, "x" => "Frank"},
 %{"_id" => #BSON.ObjectId<58d4b1d6a6ba445542adc9d6>, "x" => "Fred"},
 %{"_id" => #BSON.ObjectId<58d4b1d6a6ba445542adc9d7>, "x" => "Four"},
 %{"_id" => #BSON.ObjectId<58d4b1d6a6ba445542adc9d8>, "x" => "ourF"},
 %{"_id" => #BSON.ObjectId<58d4b1d6a6ba445542adc9da>, "x" => "Foot"}]
 ```
You can see from the results that what this did was to match on any document in our collection that
had a "F" character in the "x" field (regardless of its position).

What are the options? What do they do? The best doc I could find was on the godoc.org site.

"RegEx represents a regular expression. The Options field may contain individual characters defining the way in which the pattern should be applied, and must be sorted. Valid options as of this writing are 'i' for case insensitive matching, 'm' for multi-line matching, 'x' for verbose mode, 'l' to make \w, \W, and similar be locale-dependent, 's' for dot-all mode (a '.' matches everything), and 'u' to make \w, \W, and similar match unicode. The value of the Options parameter is not verified before being marshaled into the BSON format."

I'm not sure that these flags are applicable. It looks like the pattern in this case is always a
regex that is compiled. I can do :

```
iex> :mongo |> Mongo.find("testData", %{x: %BSON.Regex{pattern: "Frank|Fred", options: ""}}) |> Enum.to_list()
[%{"_id" => #BSON.ObjectId<58d49868a6ba445542adc9c9>, "x" => "Frank"},
 %{"_id" => #BSON.ObjectId<58d49868a6ba445542adc9ca>, "x" => "Fred"}]
```

and we find both the row with Frank and the row with Fred. The "|" character is a logical OR. I can change this
to match case-insensitive by passing "i" as an option. This does seem to have a useful impact.

```
iex> :mongo |> Mongo.find("testData", %{x: %BSON.Regex{pattern: "Frank|Fred", options: "i"}}) |> Enum.to_list()
[%{"_id" => #BSON.ObjectId<58d49868a6ba445542adc9c9>, "x" => "Frank"},
 %{"_id" => #BSON.ObjectId<58d49868a6ba445542adc9ca>, "x" => "Fred"},
 %{"_id" => #BSON.ObjectId<58d4aa7ea6ba445542adc9d2>, "x" => "frank"}]
 ```

Passing any string will match a document if the search element occurs anywhere in the string. So,

```
iex > :mongo |> Mongo.find("testData", %{x: %BSON.Regex{pattern: "an", options: ""}}) |> Enum.to_list()
[%{"_id" => #BSON.ObjectId<58d49868a6ba445542adc9c9>, "x" => "Frank"},
 %{"_id" => #BSON.ObjectId<58d4aa7ea6ba445542adc9d2>, "x" => "frank"}]
```

If you just wanted to find the strings that start with "an" then you can use an anchoring character "^"
as in :

```
iex > :mongo |> Mongo.find("testData", %{x: %BSON.Regex{pattern: "^an", options: ""}}) |> Enum.to_list()
[]
```

You can find documents that can contain numbers in the "x" field with :

```
iex >  :mongo |> Mongo.find("testData", %{x: %BSON.Regex{pattern: "[0-9]", options: ""}}) |> Enum.to_list()
[%{"_id" => #BSON.ObjectId<58d4b1d6a6ba445542adc9db>, "x" => "123324"},
 %{"_id" => #BSON.ObjectId<58d4b1d6a6ba445542adc9dc>, "x" => "123324 ABC"}]
```

Since Mongo is using a Perl compatible regex a good resource for simple regex is the
[perldoc](http://perldoc.perl.org/perlrequick.html#Simple-word-matching).

This is about all I've learned about the Elixir Mongo driver so far. I hope to spend some more
time with it over the next few days to learn more about both it and Mongo.


text:
---
title: Announcing Istio 1.17.4
linktitle: 1.17.4
subtitle: Patch Release
description: Istio 1.17.4 patch release.
publishdate: 2023-07-14
release: 1.17.4
---

This release fixes the security vulnerabilities described in our July 14th post, [ISTIO-SECURITY-2023-002](/news/security/istio-security-2023-002).

This release note describes what’s different between Istio 1.17.3 and 1.17.4. There will be an additional security release made on or after July 25th, 2023 that will fix numerous
security defects with the highest security defect considered high severity. For more information, please see the
[announcement](https://discuss.istio.io/t/upcoming-istio-v1-18-1-v1-17-4-and-v1-16-6-security-releases/15864).

{{< relnote >}}

## Security update

- __[CVE-2023-35945](https://github.com/envoyproxy/envoy/security/advisories/GHSA-jfxv-29pc-x22r)__: (CVSS Score 7.5, High):
HTTP/2 memory leak in `nghttp2` codec.

## Changes

- **Added** rolling update max unavailable to CNI Helm chart to speed up deploys.

- **Fixed** an issue where the certificate validity was not accurate for `istioctl proxy-config secret` command.

- **Fixed** an issue where Istiod might crash when a cluster is deleted and xDS cache is disabled.
  ([Issue #45798](https://github.com/istio/istio/issues/45798))

- **Fixed** an issue where specifying multiple include conditions using `--include` in a bug report
  didn't work as expected. ([Issue #45839](https://github.com/istio/istio/issues/45839))

- **Fixed** an issue where disabling a log provider through Istio telemetry API would not work.

- **Fixed** an issue where `Telemetry` would not be fully disabled unless `match.metric=ALL_METRICS` was
  explicitly specified; matching all metrics is now correctly considered as the default.

gha_language: HTML


text:
# dotfiles
These dotfiles have been tested in Nix and OSX. If you want to try them, I would fork this repo, review, and remove any configuration you don't want.

## Usage
### Install
To install, download and run.
```
sh -c "$(wget https://raw.githubusercontent.com/gesquive/dotfiles/main/.local/bin/bootstrap_dotfiles.sh -O -)"
```

license: Unlicense

repository: gesquive/dotfiles

repo_name: gesquive/dotfiles


text:
# cebulany-manager

# Build

```

# make virtualenv
virtualenv venv -ppython3.7
source venv/bin/activate
python setup.py install
cd spa/
yarn install
yarn run build
```

to run front-end in developming mode
```
yarn run dev
```

# Init

```
alembic upgrade head
python create_mock.py > mock.csv
./uwsgi-me.sh
# go to http://localhost:5000
# login (socek, socek) and upload mock.csv in transaction view
```

# Run?

```
./uwsgi-me.sh
```

# Managing users?

```
python -m cebulany.user_cli --help
```

# Develop

## Backend server

```
source venv/bin/activate
cd cebulany/
FLASK_DEBUG=1 FLASK_APP=app.py flask run
```

## Frontend server

```
cd spa
yarn run dev
```

repository: hackerspace-silesia/cebulany-manager

gha_language: Python

repo_name: hackerspace-silesia/cebulany-manager


text:
# Conjoint morphological and syntactic (dependency) parsing for Sumerian

To a large extent, Sumerian clausal syntax is morphologically expressed at the verb, e.g., in polypersonal agreement and dimensional affixes (that indicate semantic roles other than direct arguments). This information, even though incomplete due to idiosyncrasies of the writing system and spelling conventions) is essential for parsing, because it helps to disambiguate the attachment of nominal arguments. Similarly, nominal inflection indicates the grammatical role of the inflected noun, but also that of its parent(s). For successful dependency parsing, it is thus beneficial to perform conjoint parsing for/disambiguation of inflectional morphology and syntactic dependencies.

## Data directories

- `classical/`
	CDLI-CoNLL data, one word per line, 8 tab-separated columns: ID WORD SEGM POS MORPH HEAD EDGE MISC
	- `classical/royal` (tokens: train 7.500, dev 750, test 1.000)

		royal subcorpus with syntax annotations, mirror of https://github.com/cdli-gh/mtaac_syntax_corpus/tree/master/royal/release

		train/dev/test split corresponds to [Ur III corpus](https://github.com/cdli-gh/mtaac_cdli_ur3_corpus/blob/master/ur3_corpus_data/corpus_split_20180418-225438.json)
	- `classical/parallel` (tokens: train 35.000, dev 5.500, test 5.500)

		parallel subcorpus with projected annotations, mirror of https://github.com/cdli-gh/mtaac_syntax_corpus/tree/master/parallel/consolidated

		train/dev/test split corresponds to [Ur III corpus](https://github.com/cdli-gh/mtaac_cdli_ur3_corpus/blob/master/ur3_corpus_data/corpus_split_20180418-225438.json)
	- `classical/royal-synth` (tokens: train 41.000, dev 8.250, test 1.000)

		synthetic corpus generated from royal corpus, using [Synthy.py](scripts/Synthy.py)
		- test corresponds to `classical/royal`
		- dev corresponds to `classical/royal/dev` and `classical/royal/train`
		- train is generated from `classical/royal/train` and `classical/royal/dev` using Synthy, with replacements from parallel corpus (excl. test) and royal corpus (excl. test)

		generated in `classical/` using the command: `python3 ../scripts/Synthy.py 35000 royal/train.conll royal/dev.conll -support parallel/dev.conll parallel/train.conll > royal-synth/train.conll`

		In the synthetic training corpus, 50% of all tokens (20.424/41.022) have been replaced with randomly selected tokens that are either
		- morphological complex and identical in their morphological annotation (12.057) or
		- identical in morphology and dependency, having a parent with the same head word and the first child identical in POS and dependency (8.367)

		Note that the original form was given as replacement option, but only if a different word was chosen, this was marked as replacement.

- `expanded/`

	CDLI-CoNLL data with one morpheme (slot) per line, otherwise following CDLI/MTAAC conventions
	the morphological head carries the original dependency and POS annotation

	In the expanded version, one line represents a word *or a morpheme* (or the position [slot] that a morpheme could occupy). Annotations for words (morphological heads) remain largely unchanged, except that HEAD indices are updated.

	Morphological slots use the slot identifier as value of the POS column, depend on the morphological head or its syntactic parent, and use the morphological gloss as dependency.

	This data is created from classical CDLI-CoNLL using default parameters of `Deppy.py`

## Original annotation and UD mapping

We provide a heuristic mapping to UD specifications for POS tags and EDGE/DEP annotations. Note that this is not 100% correct, but should be used to training with parsers that restrict their training data to UD corpora (e.g., SpaCy).

Those parsers will work on the classical data only, not on the expanded data. This also means that we need to perform the evaluation over classical data *twice*: Once using CDLI annotations, once using the UD mapping.

The mapping files are `pos2UD.tsv` and `deps2UD.tsv`.

## Evaluation routine

- Note that all annotated data was created semi-automatically, using different methods. They do contain errors. The idea of conjoint training over two subcorpora is that the respective errors will balanced out. This is why we need additional manual evaluation.
- As for quantitative evaluation, please adhere to the following procedure
	- For any given parser, please train and evaluate parsers in four settings:
		- train and evaluate on `classical` data sets (syntactic parsing only)
		- train and evaluate on `expanded` data sets (conjoint syntactic and morphological parsing)
		- train on `expanded` data sets, remove lines with slot information, update ID/HEAD, evaluate on `classical` data sets (to evaluate conjoint parsing against syntactic baseline)
		- train and evaluate on UD-mapped `classical`data sets (syntactic parsing only, to compare with state of the art parsers that are restricted to UD tags)
	- As for the data sets, please train and evaluate parsers in five different settings:
		- train on `parallel`, evaluate on `parallel/test` and `royal/test`
		- train on `royal`, evaluate on `royal/test` and `parallel/test`
		- train on `royal-synth`, evaluate on `royal-synth/test` and `parallel/test`: Possible that synthetic data improves parsing performance.
		- train on `royal-synth` and `parallel`, evaluate on `royal/test` and `parallel/test` separately. The difference is that this uses synthetic data to balance training corpus size between royal and parallel.
		- train on `royal` and `parallel`, evaluate on `royal/test` and `parallel/test` separately. This is to check whether synthetic data hurts or improves performance on the royal corpus, in particular.
- For every parser, we have to run up to 15 (3x5) trainings and 40 (4x10) evaluations. Start with/Prioritize training and evaluation on `expanded` and on conjoint training on `royal+parallel`.
- As for evaluation metrics, we use Labelled Attachment Score (LAS), Unlabelled Attachment Score (UAS) and Label Score (LS). The focus is on LAS. UAS and LS are only diagnostics to identify sources of errors. When reporting Labelled Attachment Score, please report it globally, for every part of speech individually and for every predicted dependency individually.

## `scripts/`

- `Synthy.py`

	takes a CDLI-CoNLL corpus and generates synthetic training data by randomly replacing words that are morphologically equivalent

	note that the syntactic structure remains untouched. Do not evaluate against data taken as input to Synthy.

	On royal corpus, about 50% of all words are being replaced.

- `Slotty.py`

	small script to augment MTAAC morphology annotations to ETCSRI-style morphology annotations: infer slot information (see below)

	tested for mtaac-morph.feats (manual MTAAC annotations), royal.feats (converted Ur-III subset from ETCSRI) and full.feats (automatic annotation of the MTAAC Ur-III corpus, trained on MTAAC and the royal subcorpus)

- `Deppy.py`

	small script that uses Slotty to expand morphological glosses into the expanded format.

	Note that it performs a heuristic perculation of morphological dependents to the syntactic head.

- `Heady.py`

	small script to reduce expanded CDLI-CoNLL files (as produced by Deppy.py) to syntax annotations only
	(this reverts Deppy.py)

## Sumerian slot grammar

Sumerian grammar is organized in slots that (can) convey morphemes that express specific aspects of grammatical function, respectively.

verbal, finite (https://cdli-gh.github.io/guides/verbal_chain_slot_system.html)

	V1=NEG|MOD1|ANT|STEM|STEM-PL|STEM-RDP
		RDP 	Reduplicated Stem
		PL 	Plural (stems with special plural forms)
	V2=MOD2|MOD3|MOD4|MOD5|MOD6|MOD7|FIN|FIN-L1|FIN-L2|FIN|FIN-L2|FIN
	V3=COOR
	V4=VEN
	V5=MID|3-NH
	V6=1-SG|2-SG|3-SG-H|1-PL|3-PL
	V7=DAT
	V8=COM
	V9=ABL|TERM
	V10=L1|LOC-OB|L1-SYN|L2|L2-SYN|L3
	V11=1-SG-A|2-SG-A|3-SG-H-A|3-SG-H-P|3-SG-H-L3|1-SG-A-OB|3-SG-NH-A|3-SG-NH-P|3-SG-NH-L3|3-PL-H-P
	V12=V|V-PF|V.PL|V.PL|COP
		V 	Verb
		PF 	Present-Future (stems with special Maru forms)
		PL 	Plural (stems with special plural forms)
	V13=PL|PLEN
	V14=1-SG-A|1-SG-S|1-SG-P|2-SG-A|2-SG-S|2-SG-P|3-SG-S|3-SG-P|3-SG-A|3-SG-S-OB|1-PL-A|1-PL-S|1-PL|2-PL-A|2-PL-S|2-PL|3-PL-S|3-PL-P|3-PL|3-PL-A
	V15=SUB

verbal, non-finite (oracc.museum.upenn.edu/etcsri/parsing, https://cdli-gh.github.io/guides/verbal_chain_slot_system.html)

	?NF 	Non-Finite
	?ABS 	Absolute marker (Non-Finite only)
	?PT 	Preterite marker (Non-Finite only)
	?F 	Future marker (Non-Finite only)

	NV1=NEG
	NV2=STEM|STEM-PF|STEM-RDP|STEP-PL
	NV3=PF
	NV4=SUB
	N2=DEM
	N3=1-SG-POSS|2-SG-POSS|3-SG-H-POSS|3-SG-NH-POSS|DEM2|1-PL-POSS|2-PL-POSS|3-PL-POSS
	N4.0=PL|ORD
	N5.0=ABS|ERG|DAT-H|DAT-NH|COM|ABL|TERM|L1|L2-H|L2-NH|L3-H|L3-NH|GEN|EQU|L4|ADV
	N4.1=PL|ORD	# of parent
	N5.1=ABS|ERG|DAT-H|DAT-NH|COM|ABL|TERM|L1|L2-H|L2-NH|L3-H|L3-NH|GEN|EQU|L4|ADV
	N4.2=PL|ORD	# of grand-parent
	N5.2=ABS|ERG|DAT-H|DAT-NH|COM|ABL|TERM|L1|L2-H|L2-NH|L3-H|L3-NH|GEN|EQU|L4|ADV
	N6=COP-1-SG|COP-2-SG|COP-3-SG|COP-1-PL|COP-2-PL|COP-3-PL|EXCEPT

Nominal (oracc.museum.upenn.edu/etcsri/parsing)

	N1=N
	N2=DEM
	N3=1-SG-POSS|2-SG-POSS|3-SG-H-POSS|3-SG-NH-POSS|DEM2|1-PL-POSS|2-PL-POSS|3-PL-POSS
	N4.0=PL|ORD
	N5.0=ABS|ERG|DAT-H|DAT-NH|COM|ABL|TERM|L1|L2-H|L2-NH|L3-H|L3-NH|GEN|EQU|L4|ADV
	N4.1=PL|ORD	# of parent
	N5.1=ABS|ERG|DAT-H|DAT-NH|COM|ABL|TERM|L1|L2-H|L2-NH|L3-H|L3-NH|GEN|EQU|L4|ADV
	N4.2=PL|ORD	# of grand-parent
	N5.2=ABS|ERG|DAT-H|DAT-NH|COM|ABL|TERM|L1|L2-H|L2-NH|L3-H|L3-NH|GEN|EQU|L4|ADV
	N6=COP-1-SG|COP-2-SG|COP-3-SG|COP-1-PL|COP-2-PL|COP-3-PL|EXCEPT

However, Sumerian morphology is partially reflected in surface forms only, and transcriptions cannot be directly decomposed into morphs because of idiosyncrasies of the writing system. We can thus not provide a reliable morphological segmentation prior to syntactic parsing.

The approach taken here is to create pseudo-tokens for *all* imaginable slots of any given verb or noun, with the word being repeated, and to mark them as being applicable or empty (_). If a full-form re-occurs, a morphological analysis from the training data can be applied (unless overridden by syntactic context). If an unseen form occurs, the syntactic context will override any clues that might be drawn from morphology.
The original POS tag is then applied to the location of the stem, all "copies" of the target word take the slot name as POS and the value as dependency label. Note that the POS serves to disambiguate morphology and syntactic relations where these take the same form (e.g., ERG for the ergative morpheme and the grammatical relation between argument and verb). The head of all slots is the stem. In a future revision, the head of N4.1, N5.1 etc. may be set to the parent, the head of N5.2 to grant-parent, if these can be reliably identified in the gold data.

original

	ba-zi	MID.V.3-SG-S

expanded

	1	ba-zi	V1	12	_
	2	ba-zi	V2	12	_
	3	ba-zi	V3	12	_
	4	ba-zi	V4	12	_
	5	ba-zi	V5	12	MID
	6	ba-zi	V6	12	_
	7	ba-zi	V7	12	_
	8	ba-zi	V8	12	_
	9	ba-zi	V9	12	_
	10	ba-zi	V10	12	_
	11	ba-zi	V11	12	_
	12	ba-zi	V	0	root
	13	ba-zi	V13	12	_
	14	ba-zi	V14	12	3-SG-S
	15	ba-zi	V15	12	_

original

	ba-an-na-zi	MID.3-SG-H.DAT.V.3-SG-S

expanded

	1	ba-an-na-zi	V1	12	_
	2	ba-an-na-zi	V2	12	_
	3	ba-an-na-zi	V3	12	_
	4	ba-an-na-zi	V4	12	_
	5	ba-an-na-zi	V5	12	MID
	6	ba-an-na-zi	V6	12	3-SG-H
	7	ba-an-na-zi	V7	12	DAT
	8	ba-an-na-zi	V8	12	_
	9	ba-an-na-zi	V9	12	_
	10	ba-an-na-zi	V10	12	_
	11	ba-an-na-zi	V11	12	_
	12	ba-an-na-zi	V	0	root
	13	ba-an-na-zi	V13	12	_
	14	ba-an-na-zi	V14	12	3-SG-S
	15	ba-an-na-zi	V15	12	_

original

	szul-e2-du3-du3-e 	PN.ERG
	szu 	N.L3-NH
	ba-ti 	MID.3-SG-H-A.V.3-SG-P

expanded

	1	szul-e2-du3-du3-e 	PN	32	ERG
	2	szul-e2-du3-du3-e 	N2	1	_
	3	szul-e2-du3-du3-e 	N3	1	_
	4	szul-e2-du3-du3-e 	N4.0	1	_
	5	szul-e2-du3-du3-e 	N5.0	1	ERG
	6	szul-e2-du3-du3-e 	N4.1	1	_
	7	szul-e2-du3-du3-e 	N5.1	1	_
	8	szul-e2-du3-du3-e 	N4.2	1	_
	9	szul-e2-du3-du3-e 	N5.2	1	_
	10	szul-e2-du3-du3-e 	N6	1	_
	11	szu 	N	32	LOC
	12	szu 	N2	11	_
	13	szu 	N3	11	_
	14	szu 	N4.0	11	_
	15	szu 	N5.0	11	L3-NH
	16	szu 	N4.1	11	_
	17	szu 	N5.1	11	_
	18	szu 	N4.2	11	_
	19	szu 	N5.2	11	_
	20	szu 	N6	11	_
	21	ba-ti 	V1	32	_
	22	ba-ti 	V2	32	_
	23	ba-ti 	V3	32	_
	24	ba-ti 	V4	32	_
	25	ba-ti 	V5	32	MID
	26	ba-ti 	V6	32	_
	27	ba-ti 	V7	32	_
	28	ba-ti 	V8	32	_
	29	ba-ti 	V9	32	_
	30	ba-ti 	V10	32	_
	31	ba-ti 	V11	32	3-SG-H-A
	32	ba-ti 	V	0	root
	33	ba-ti 	V13	32	_
	34	ba-ti 	V14	32	3-SG-P
	35	ba-ti 	V15	32	_

On this basis, then, a conventional syntactic parser can be trained, taking WORD and POS columns as input, and predicting HEAD and DEP columns. Its output is, however, to be simplified prior to evaluation in that the "dependency labels" of morphemes are aggregated into a single, .-separated gloss and indexing is to be updated. Evaluation should be conducted over morphological parsing (aggregated), unlabelled dependencies (aggregated) and labelled dependencies (aggregated) separately and compared with plain dependency parsing (without expansion, without morphological features, same parser) and lookup-based morphological glossing (https://github.com/cdli-gh/glosser).

## Known issues

* Future refinements may include using the correct attachment of nominal case (to parents, grant-parents, etc.) and more sophisticated ways for morphological segmentation. At the moment, this is done heuristically in `Deppy.py` to the morphological head, its syntactic head, etc., but we do not check whether grammatical features (of gloss annotation) and dependency of the parent node correlate. Normally, we would expect that a `DAT-H` morpheme depends on an NP with the dependency `DAT`, etc. In Sumerian, only the innermost case (can) modify the morphological head (slot `N5`), the outer cases (slot `N5.1`, `N5.2`) would depend on the parent or their parent. Note that in case of appositions, the slot `N5` may be used for the case of the parent rather than the morphological head, etc.

* Training and test data was created using an array of semiautomated methods with different typical errors (see https://github.com/cdli-gh/mtaac_syntax_corpus for sources, methods and updates). Manual revision is strongly encouraged. However, we expect any parser trained on such heterogeneous data to generalize over source-specific errors and to produce a relatively trustable result.

*

## Future work

* Parsing from text: Slot expansion requires part-of-speech tagging. To annotate unseen text, consider using the CRF POS/NER taggers from https://github.com/cdli-gh/Sumerian-Translation-Pipeline. For text from the Ur-III period, data annotated in this way can be found under https://github.com/cdli-gh/mtaac_cdli_ur3_corpus/tree/master/ur3_corpus_data/annotated/pos.

repository: cdli-gh/conjoint-parsing

repo_name: cdli-gh/conjoint-parsing


text:
---
layout: post
title:  "Coursera Deep Learning Course 1 Week 3 notes: Shallow neural networks"
date: 2017-10-10 16:20:00 +0700
tags:
  - notes
  - deep learning
no-post-nav: 0
comments: 1
---

##### **Shallow Neural Network**

###### **Neural Networks Overview**

[i]: layer.

(i): training example.

###### **Neural Networks Representation**

a[0] = X: activation units of input layer.

When we count layers in neural networks, **we don't count the input layer.**

Just a recap from Machine Learning course: the hidden layers i and the output layer i will have parameters W[i], b[i] associated with them. (Unlike the past convention, the index is increased by 1).

###### **Computing a Neural Network's Output**

Figure 1. Each node in Neural Network does the same thing as the node in Logistic Regression.

Source: Coursera Deep Learning course

Vectorization: z[l] = W[l]a[l-1] + b[l], a[l] = σ(z[l]).

###### **Vectorizing across multiple example**

[l] (i): layer l, training example i.

Figure 2. Vectorized implementation.

Source: Coursera Deep Learning course

###### **Explanation for Vectorized Implementation**

No notes.

###### **Activation functions**

Sigmoid is called as a **activation function**.

Another activation function works **mostly better** than Sigmoid function is [**Hyperbolic Tangent**](http://mathworld.wolfram.com/HyperbolicTangent.html) function $$ y = tanh(x)$$ (which is a shifted version of Sigmoid function).

It's better because it crosses (0, 0) point, and tends to centering the data, so the mean of the data is close to 0, and this actually makes the learning for the next layer a little bit easier.

One exception is that when you use binary classification problem, and the output value is between 0 and 1, then you still use the Sigmoid activation function.

**The activation function can be different for different layers**: we can you tanh function for hidden layers and Sigmoid function for output layer.

One downside of tanh(x) and Sigmoid(x) function: if x is either too large or too small, the slope (derivative) of the function at x is close to 0, and this makes Gradient Descent slow.

Another popular choice of activation function: [**Rectified Linear Unit (ReLU)**](https://en.wikipedia.org/wiki/Rectifier_(neural_networks)). The slope (derivative) of this function at x is equal to either 0 (x < 0) or 1 (x >= 0).

The fact that the slope of ReLU function equals 0 when x < 0, in practice it works just fine; because generally the z values of hidden units will be > 0.

Rules of thumps for choosing activation function:

Binary classification problem: Sigmoid function is a natural choice for output layer, for hidden layers: use ReLU function.

**Pros and Cons for activation function**

* Sigmoid: Never use this except for the output layer if you're doing binary classification problem, because we have:

* Tanh: Works mostly better than Sigmoid.

* ReLU: The most commonly used activation function, use this when you're not sure what to use.

* [**Leaky ReLU**](https://en.wikipedia.org/wiki/Rectifier_(neural_networks)#Leaky_ReLUs): Feel free to try.

###### **Why do you need non-linear activation functions**

Linear activation function/Identity activation function: $$f(x) = x$$.

If you use Linear activation function (don't use non-linear activation function), regardless of how many hidden layers you have, neural network just compute the output value as a linear function of input X. So you are not computing any interesting function even if you go deeper in the network.

There may be only 1 place that you use linear activation function: when you're doing machine learning on Regression problem (output value y is a real number).

Read more: [Activation Function](https://medium.com/towards-data-science/activation-functions-in-neural-networks-58115cda9c96).

###### **Derivatives of activation function**

$$ \sigma'(z) = \sigma(z)(1 - \sigma(z))$$

$$ tanh'(z) = (\frac{e^{z} - e^{-z}}{e^{z} + e^{-z}})'= 1 - tanh^{2}(z)$$

###### **Gradient Descent for (single layer) Neural Networks**

**Gradient Descent algorithm:**

Repeat {
  * Feedforward: Compute predict for all training examples $$\hat{y}^{(i)}, ~ i = 1..m$$.

  * Backpropagating: Calculate the derivatives $$ dw^{[l]}, ~db^{[l]}$$.

  * Perform gradient descent update:

    * $$ w^{[l]} = w^{[l]} - \alpha dw^{[l]}$$.

    * $$ b^{[l]} = b^{[l]} - \alpha db^{[l]}$$.

}

**Formulas for computing derivatives**

Figure 3. Formulas for computing derivatives.

Source: Coursera Deep Learning course

###### **Backpropagation intuition (optional)**

Figure 4. Computation graph for 2 layer Neural Networks.

Source: Coursera Deep Learning course

**Perform backpropagation (apply Chain rule)**

$$ \frac{d\it\unicode{xA3}}{da^{[2]}} = -\frac{y}{a^{[2]}} + \frac{1-y}{1-a^{[2]}}$$

$$ \frac{d\it\unicode{xA3}}{dz^{[2]}} = \frac{d\it\unicode{xA3}}{da^{[2]}} \frac{da^{[2]}}{dz^{[2]}} = (-\frac{y}{a^{[2]}} + \frac{1-y}{1-a^{[2]}})a^{[2]}(1-a^{[2]}) = a^{[2]} - y = \hat{y} - y$$

$$ \frac{d\it\unicode{xA3}}{dW^{[2]}} = \frac{d\it\unicode{xA3}}{dz^{[2]}} \frac{dz^{[2]}}{dW^{[2]}} = dz^{[2]}a^{[1]} = (a^{[2]} - y)a^{[1]}$$

$$ \frac{d\it\unicode{xA3}}{db^{[2]}} = \frac{d\it\unicode{xA3}}{dz^{[2]}} \frac{dz^{[2]}}{db^{[2]}} = dz^{[2]} = a^{[2]} - y$$

Similarly to $$ W^{[1]}, ~b^{[1]}$$.

All that is just the gradient for a single training example, we can vectorize to work with m training example:

Figure 5. Vectorization for m training examples.

Source: Coursera Deep Learning course

###### **Random Initialization**

If you initialize weights (W, b) to 0, the hidden units will calculate exact the same function (this is bad because you want different hidden units to compute different functions).

Solution: Random Initialization:

* w[l] = np.random.rand((n, m)) * **0.01**, we usually prefer to initialize random small value, and it prevents the derivatives of activation function from being close to 0.
* b[l] = np.zeros((n, 1)), it's ok for b to be zero.

gha_language: CSS


text:
---
layout: post
title: Loser Talk
date: '2008-11-04 22:25:38 -0700'
mt_id: 1390
blog_id: 1
post_id: 1390
basename: loser-talk
categories:
- politics
- current-events
- history
---

Barack Obama is president.

I listened to John McCain's concession speech with disgust. It succinctly summed up McCain: a pragmatist and compromiser to the end. A principled opponent would have conceded the battle but laid the groundwork for the larger war. He would have pledged to relight the flame of liberty during the dark times ahead.

In today's climate, such a speech is unthinkable. Unity is the buzzword of the day. Like many times during his flawed, unprincipled run at the presidency, I sought comfort in the 1964 campaign. After listening to McCain's terrible convention acceptance speech, Goldwater's was a palliative. I figured that his concession in that same year might provide the inspiring call to action that McCain's wasn't. There's some of that but I think there must be a pattern of graciousness in these matters that I was unaware of. Or maybe the losing candidates are just as sick of campaigning as we are.

I've waited 'til now to make any statement about this election because I wanted to find out more of the details of the vote—not just the total but the spread of it, what it might portend at this very early date.

I know many of you expected me to make some statement last night but I held that off. I sent the President the following wire, which I think will be available for you if you don't have it now:

"To President Lyndon Johnson in Johnson City, Texas.

Congratulations on your victory. I will help you in any way that I can toward achieving a growing and better America and a secure and dignified peace. The role of the Republican party will remain in that temper but it also remains the party of opposition when opposition is called for. There is much to be done with Vietnam, Cuba, the problem of law and order in this country, and a productive economy. Communism remains our No. 1 obstacle to peace and I know that all Americans will join with you in honest solutions to these problems."

I have no bitterness, no rancor at all. I say to the President as a fellow politician that he did a wonderful job. He put together a vote total that's larger than has ever been gained in this country.

However, it's interesting to me and very surprising to me that the latest figures that I can get do not reach the totals of the 1960 election. I am disappointed in this because I thought that the American people would have turned out in greater numbers than they seem to have done.

But he did a good job and I have to congratulate him on it.

Also I want to express my gratitude to the more than 25 million people in this country who not necessarily voted for me but they voted for a philosophy I represent, a Republican philosophy that I believe the Republican party must cling to and strengthen in the years ahead.

I want to thank all of you across this nation who turned out in those numbers to support my candidacy and that of Bill Miller and the Republican party.

I don't think that I've ever seen more dedicated people in my life, people who worked as hard or who worked as long and produced the results that they did. These people are dedicated to, as I say, the Republican philosophy.

There is a two-party system in this country and we're going to keep it. We're going to devote our days and the years ahead to strengthening the Republican party, to getting more people into it and I feel that the young people coming along will provide the army that we need.

This effort that we engaged in last January 3 turns out to be a much longer effort than we thought. It's not an effort that we can drop now nor do we have any intentions of dropping it now.

I will devote—being unemployed as of January 3 or thereabouts—I'll have a lot of time to devote to this party, to its leadership and to the strengthening of the party, and that I have every intention of doing. I want to just ask the people in this country who worked so hard in this election not to be despondent, that we have a job to do and let's get along with it, because there are many questions that have to be answered.

I'm very hopeful that the President will, now that the election is over, get along with the answers that we've sought during the campaign—the answers about Vietnam, about Cuba, about Communism—Communism's continuing growth all around the world—about the growing tendency to the control of our economy and our daily lives in this country.

As I said in my wire, anything that I can do—and I'm sure that I speak for all Americans—anything that we can do to help the President get along with the solutions to these problems, we're ready, willing and able to do.

Now with that I have nothing further to say. I will entertain a few questions—not any prolonged period at it. Mr. Wagner will recognize.

extension: markdown


text:
#mjpg-streamer
mjpg-streamer was used to stream the video feed; download mjpg-streamer [here](https://sourceforge.net/projects/mjpg-streamer/).

There were issues with running the streamer on our raspberry pi, so we used a patch to fix the issue (input_uvc_patch).

repository: matthewawhite/CISSROV

repo_name: matthewawhite/CISSROV


text:
READ THIS PLS

 PHP and HTML testing. Not important but if you want, fork this. And you allowed to download this entire repository.

 ---

 ![image](https://user-images.githubusercontent.com/82641473/123585540-58bba900-d76e-11eb-8de6-12554fa277ca.png)

 Im serious the ```.*.php``` extension is not ```Hack```. I dont really know why Github named ```*.php``` is Hack lmfao

 ---

 License:

 ![image](https://media.discordapp.net/attachments/852828353651802122/858984929609777152/unknown.png?width=1025&height=134)

MIT License. [More...](LICENSE)


text:
[Go back to home page](README.md)
## Unit 1
The start of writing trash code

In this unit, you will learn the basics of writing trash code.

### Things to do before writing trash code
[See Unit 0 - What you need to do before writing trash code](unit_0.md)

### Writing the Code
After you have done the things done above, you are ready to write Trash Code!

**Remember, the first thing writing trash code is to be complicated.**

Let's say that you are writing a program that receives the input as the radius and outputs the perimeter and area.

 **

1. First thing you have to do is to declare the base class and method of your code. Let's first rename our file by a wrong spelling name which starts with lower cases(so that we will not be able to recognize them from methods). Since the file name was changed, let's change our class name, too.
```
public class area{
    public static void main(String[] args){

    }
}
```

2. Declare the variables you need with meaningless variable names, or write meaningful variable names and assign them with the wrong values.
```
public class area{
    public static void main(String[] args){
          Double i =  888.0;
          Double e = 666.0;

          Double radius = 2 * 3.14 *
    }
}
```

3. Write equations for computing the area and perimeter and put set them to other variables.
```
public class Area{
    public static void main(String[] args){
          Double i =  888.0;
          Double e = 666.0;

          Double radius = 2 * 3.14 * i;
          Double perimeter = 3.14 * i * i;
    }
}
```
4. Print random stuff out!
```
public class Area{
    public static void main(String[] args){
          Double i =  888.0;
          Double e = 666.0;

          Double radius = 2 * 3.14 * i;
          Double perimeter = 3.14 * i * i;

          System.out.println(i);
          System.out.println(e);
          System.out.println(i);
          System.out.println(perimeter);
    }
}
```
5. Oh Yeah! What messy code!


text:
---
layout: guide
search: true
section: GraphQL Pro - OperationStore
title: Authentication
desc: OperationStore uses client secrets to authenticate sync requests.
index: 4
---

When you [add a client]({{ site.base_url }}/operation_store/client_workflow#add-a-client), you also associate a _secret_ with that client. You can use the default or provide your own and you can update a client secret at any time. By updating a secret, old secrets become invalid.

This secret is used to add an authorization header, generated with HMAC-SHA256. With this header, the server can assert:

- The request came from an authorized client
- The request was not corrupted in transit

For more info about HMAC, see [Wikipedia](https://en.wikipedia.org/wiki/Hash-based_message_authentication_code) or Ruby's [OpenSSL::HMAC](https://ruby-doc.org/stdlib-2.4.0/libdoc/openssl/rdoc/OpenSSL/HMAC.html) support.

The Authorization header takes the form:

```ruby
"GraphQL::Pro #{client_name} #{hmac}"
```

[`graphql-pro-js`](http://github.com/rmosolgo/graphql-pro-js) adds this header to outgoing requests by using the provided `--client` and `--secret` values.

repository: taiki-t/graphql-ruby

repo_name: taiki-t/graphql-ruby


text:
# Contributing to RichWidgets [![Build Status](https://travis-ci.org/richwidgets/richwidgets.png?branch=master)](https://travis-ci.org/richwidgets/richwidgets) #

> A JavaScript project containing standalone javascript widgets based on Bootstrap, jQuery UI and other third-party widgets, styled with LESS.

## Hacking on RichWidgets

Begin by setting you your environment and making sure you can build RichWidgets as described the
[README.md](https://github.com/richwidgets/richwidgets/blob/master/README.md#build).

Widgets are highly visual constructs, and we are more effective in developing them when we get rapid feedback as we introduce changes.
Without this rapid feedback, we are like sculptors carving blindfolded!

Run the `grunt dev` command from the richwidgets root folder to start up a development version of the online demo with
integrated [live-reload](https://github.com/livereload/livereload-js).  You will see this demo runnin locally at:

> http://localhost:9000

As you make changes to the html, javascript, and LESS
source, the source will be compiled and the demo site will be updated.

Additionally, this development version of the demo uses unminified versions of the sources making it easier to debug your widget
in your browsers development tools.

The demo pages themselves use [handlebars.js](http://handlebarsjs.com/) for templating.  This allows us to keep our demo
code [DRY](http://en.wikipedia.org/wiki/Don%27t_Repeat_Yourself) by using the same code to both run the page sample as well as display
the code on the page.  Handlebars is easily extensible through it's [helper](http://assemble.io/docs/Custom-Helpers.html)
mechanism.  If you find yourself bending over backwards to make something work, consider writing such a helper.

## Publishing the demos

Publish the demos to github pages using the grunt task `grunt site`

## Coding Style

It helps to be consistent.  Here are the coding style conventions we have agreed to follow in this project.  this will be
updated as new discrepancies are discovered and cleaned up.  Wherever possible these conventions are being monitored by
jshint.

* Indentation
    * Use spaces (not tabs)
    * Indentation size is 2 spaces
* Filenames
    * All filenames will use a lowercase-hyphenated naming convention (e.g. `popup-panel.js`)
* LESS
  * CSS class names use lowercase-hyphenated naming convention (e.g. `popup-panel`)
  * Define variables centrally in `src/widgets/variables.less`
  * Define mixins centrally in `src/widgets/mixins.less`
* JavaScript
  * Functions and variables names use camelCase (e.g. `popupPanelWidth`, `showPopup()`)
  * Single quotes `'` for strings
  * do not use `self` for capturing a reference to `this`
      * use `widget` where it makes sense to do so, `that` otherwise
  * function white-spaces
      * named functions
          * `function name(param1, param2) {`
      * unnamed functions
          * `function (param1, param2) {`
* Widget Factory
  * use the `rich` jquery plugin namespace
  * Widget name prefix ***TBD***
  * Widget event prefix: stick with the default name of the widget (eg. no underscores or abbreviating)
  * initialize all options, using `null` as default value is required

## Creating a new widget

To create a new widget called "My widget", start by creating the files:

* `src/widget/<widget-family>/my-widget.less`
* `src/widget/<widget-family>/my-widget.js`

Create a demo for your widget in

* `src/demo/pages/<widget-family/my-widget.hbs`

The demo pages use [handlebars.js](http://handlebarsjs.com/), and will be added to the site navigation automatically.

## Testing

Read about our approach to testing in the [TESTS.md](https://github.com/richwidgets/richwidgets/blob/master/TESTS.md) guide.


text:
---
title: "Environment"
linkTitle: "Environment"
weight: 1
description: >
  Learn to build the development environment suggested by Enduro.
---

Enduro does not have a development mode with all dependencies mocked or stubbed.
The dependencies need to be installed for Enduro to work. To make this process
easier we offer an environment based on Docker Compose where we run all the
dependencies as Docker containers.

But for Enduro itself, we suggest to build it and run it locally because that
makes thing much simpler during development.

## Development dependencies

There are some dependencies that need to be installed:

* [Go][go],
* [Docker Compose][docker-compose],
* [Yarn][yarn], and
* [GNU Make][make]

This guide may not work for you if you manage Docker with `sudo`, see
[issue #118][issue-118] for more. It is possible to manage Docker as a non-root
user with some [extra configuration steps][docker-non-root].

## First steps

Spin up the environment with the following command:

    docker-compose up --detach

Cadence will crash right away because the database has not been set up properly.
Run the following command to introduce the MySQL tables needed by Cadence:

    make cadence-seed

In a minute or less, Cadence should be up again. Since Cadence is a multitenant
service, we need to create a Cadence domain for Enduro:

    make cadence-domain

Now we need to build some Go tools we're going to use during development:

    make tools

To build our first binary succesfully, we need to generate the database
migrations and the web interface with:

    make migrations ui

We're ready to run our first build:

    make

With Enduro running in the background, you should be able to access the web
interface via http://<IP_ADDRESS>:9000/ or connect to the API, e.g.:

    curl -v "<IP_ADDRESS>:9000/collection/"

## Set up MinIO for object storage

MinIO is one of the services installed automatically with Docker Compose. You
should be able to access the web file browser via http://<IP_ADDRESS>:7460 using
the following credentials:

* Access key: `minio`
* Secret key: `minio123`

Alternatively, you can [install][mc] the MinIO command-line client (mc) and
register the local instance with:

    $ mc config host add enduro http://<IP_ADDRESS>:7460 minio minio123
    Added `enduro` successfully.

We provide some default configuration so MinIO publishes events via our local
Redis instance. Validate the configuration with:

    $ mc admin config get enduro notify_redis
    notify_redis:1 format=access address=redis:6379 password= key=minio-events queue_dir=/tmp/events queue_limit=10000
    notify_redis enable=off format=namespace address= key= password= queue_dir= queue_limit=0

    $ mc event list enduro/sips
    arn:minio:sqs::1:redis   s3:ObjectCreated:*   Filter:

List the bucket with:

    $ mc ls enduro/sips
    [2020-04-29 13:28:32 PDT]  4.6KiB archivematica.png

## Start a transfer

Uploading a file into the `sips` bucket should trigger the workflow. From the
command-line interface, you can run the following:

    curl -s https://news.ycombinator.com/y18.gif | mc pipe enduro/sips/y18.gif

The workflow should be triggered automatically. It is okay to start Enduro later
since the event is buffered by Redis.

    make run

Additionally, you can visualize workflows and activities from Cadence Web. Try
opening the following link: http://<IP_ADDRESS>:7440/domain/enduro/workflows/.

## Development workflow

You can use the standard Go build workflows such `go build` or `go install` but
we provide a couple of shortcuts that use our custom build directory and flags:

    # Build the binary in ./build/enduro
    make enduro-dev

    # Build and run
    make run

    # Since `run` is the default, you can just type
    make

If you are looking for a source-code editor, Visual Studio Code has great Go
support. It uses the official language server, [gopls][gopls], which is it not
stable yet - it is recommendable to read their docs and keep it up to date.

Other useful commands are `make lint` and `make test`. This project uses
GolangCI-Lint, an aggregator of many linters. It is installed with `make tools`.
You can enable it in Visual Studio Code as follows:

```json
{
    "go.lintTool": "golangci-lint",
    "go.lintFlags": ["--fast"]
}
```

[docker-compose]: https://docs.docker.com/compose/install/
[mc]: https://docs.min.io/docs/minio-client-quickstart-guide.html
[go]: https://golang.org/doc/install
[gopls]: https://github.com/golang/tools/blob/master/gopls/README.md
[yarn]: https://classic.yarnpkg.com/en/docs/install/
[make]: https://www.gnu.org/software/make/
[issue-118]: https://github.com/artefactual-labs/enduro/issues/118
[docker-non-root]: https://docs.docker.com/engine/install/linux-postinstall/#manage-docker-as-a-non-root-user

repository: evelynPM/enduro

repo_name: evelynPM/enduro


text:
![header](assets/images/header.jpg)
# d3-challenge

# Project Overview

This project consists in creating a d3 visualization from U.S. census data, aggregated by state, that will allow the user to determine visually if a correlation exists between 2 variables

# Project breakdown

- The project is divided in 2 parts, part 1 requires a static graphic that will correlate 2 variables of my choosing
- Part 2 requires to allow the user to choose 2 variables and display the corresponding graph, this has to be achieved dynamically
- For this project I will implement the solution thinking ahead for part 2, this requires to write functions for pretty much every parameter that the graph needs, making the resulting app much more scalable

# Repository structure
````bash
│   commits.md
│   index.html
│   LICENSE
│   README.md
│
└───assets
    ├───css
    │       d3Style.css
    │       style.css
    │
    ├───data
    │       data.csv
    │
    ├───images
    │       header.jpg
    │       US-Healthcare.jpg
    │
    └───js
            .eslintrc.json
            app.js
````

# Screenshots

![screenshot1](assets/images/scsh1.png)
![screenshot1](assets/images/scsh2.png)


text:
# Change Log

The change log describes what is "Added", "Removed", "Changed" or "Fixed" between each release.

## 0.3.0

### Added

- Exception listener

### Changed

- We do not care about kernel:debug when evaluating `happyr_api.wsse.debug`.

## 0.2.0

- Removed controller and created `ResponseFactory`

## 0.1.0

First release

repository: GeorgosXL/ApiBundle

repo_name: GeorgosXL/ApiBundle


text:
# vCloud Tools Tester

The integration tests for [vCloud Tools](https://github.com/gds-operations/vcloud-tools) require a lot of parameters that are specific to your environment, and not ones you want to make public.

vCloud tools Tester makes it easy to run the vCloud Tools integration tests using a config file.

## Installation

Add this line to your application's Gemfile:

    gem 'vcloud-tools-tester'

And then execute:

    $ bundle

Or install it yourself as:

    $ gem install vcloud-tools-tester

## Testing

Run the default suite of tests (e.g. lint, unit, features):

    bundle exec rake

There are no integration tests for this project.

## Contributing

1. Fork it
2. Create your feature branch (`git checkout -b my-new-feature`)
3. Commit your changes (`git commit -am 'Add some feature'`)
4. Push to the branch (`git push origin my-new-feature`)
5. Create new Pull Request

repository: mikepea/vcloud-tools-tester

repo_name: mikepea/vcloud-tools-tester


text:
---
layout: post
title: "Elixir block keywords"
date: 2015-12-03 22:25
comments: true
categories:
  - Elixir
  - Macros
---

Exploring Elixir, I tried this:

``` elixir linenos:false
defmodule Example do
  def foobar(do: _, else: _) do
  end
end

Example.foobar do
 IO.puts "true"
else
 IO.puts "false"
end
```

And it worked. Well, in a sense. The code runs, but it outputs *both* "true" and "false".

What's going on here? Let's try another experiment:

``` elixir linenos:false
IO.inspect do
  "true"
else
  "false"
end

# => [do: "true", else: "false"]
```

[Turns out](https://groups.google.com/forum/#!topic/elixir-lang-talk/jVqCeLcaUV0/discussion) this is language-level syntactic sugar (e.g. [1](https://github.com/elixir-lang/elixir/blob/c37ea4e8740539918683eb03ca9fce28239a3cac/lib/elixir/src/elixir_tokenizer.erl#L1050-L1053), [2](https://github.com/elixir-lang/elixir/blob/c37ea4e8740539918683eb03ca9fce28239a3cac/lib/elixir/src/elixir_exp_clauses.erl), [3](https://github.com/elixir-lang/elixir/blob/c37ea4e8740539918683eb03ca9fce28239a3cac/lib/elixir/lib/macro.ex#L652)) that desugars to a plain keyword list. And that explains why the code above would output both "true" and "false" – it's equivalent to

``` elixir linenos:false
Example.foobar([do: IO.puts("true"), else: IO.puts("false")])
```

The keyword list is evaluated before it's even passed to the function, like any keyword list would be. That includes evaluating the `IO.puts` function calls.

Now that we have the full list of block keywords (from the Elixir source) we can go completely crazy:

``` elixir linenos:false
IO.inspect do
  "a"
else
  "b"
catch
  "c"
rescue
  "d"
after
  "e"
end

# => [do: "a", else: "b", catch: "c", rescue: "d", after: "e"]
```

So these are all available to your own functions, like in our `Example.foobar` example. But what use are they if every branch is evaluated all the time?

Macros to the rescue.

[Elixir macros](http://elixir-lang.org/getting-started/meta/macros.html) get access to the syntax tree of a piece of code, without the code being evaluated first. They can then slice and dice the code and return another syntax tree, that *will* be evaluated.

Elixir's own `if/do/else` is [just a macro](https://github.com/elixir-lang/elixir/blob/c37ea4e8740539918683eb03ca9fce28239a3cac/lib/elixir/lib/kernel.ex#L2321-L2341) using these keyword lists.

Just for fun, we could make a macro that randomly executes one of two branches, and then always runs the `after` branch:

``` elixir linenos:false
defmodule MyMacro do
  defmacro pick([do: option1, else: option2, after: after_block]) do
    :random.seed(:os.timestamp)
    option = Enum.random([option1, option2])
    [option, after_block]
  end
end

defmodule Example do
  require MyMacro

  def run do
    MyMacro.pick do
      IO.write "dog"
    else
      IO.write "cat"
    after
      IO.puts "!"
    end
  end
end

Example.run

# Outputs either of these:
# dog!
# cat!
```

I haven't used this myself, other than in silly experiments. I can picture it being handy for some DSLs, though. If you apply this to anything interesting, please do let me know in a comment!

repository: henrik/octopress

gha_language: Ruby

repo_name: henrik/octopress