text:
---
layout: default
---

# 1993-02-22 - Proposal - The 'Peterson.mil' Virus List

## Header Data

From<EMAIL_ADDRESS>To<EMAIL_ADDRESS>Message Hash: f9326d70e4817c1e1a8bff2d74cf88f91011219354f6cf4c954519215d17e3ad

Message ID<EMAIL_ADDRESS>Reply To: _N/A_

UTC Datetime: 1993-02-22 21:00:39 UTC

Raw Date: Mon, 22 Feb 93 13:00:39 PST

## Raw message

```
{% raw  %}From<EMAIL_ADDRESS>Date: Mon, 22 Feb 93 13:00:39 PST
To<EMAIL_ADDRESS>Subject: Proposal - The 'Peterson.mil' Virus List
Message-ID<EMAIL_ADDRESS>MIME-Version: 1.0
Content-Type: text/plain

Yo, Punks

I propose that we establish another list dedicated to the study of viruses
and name it after Keith Peterson. After all, why shouldn't he get some credit
for awakening a sleeping giant?

I want to know more about viruses so I can protect myself from
government-created viruses and so I can learn about the various mechanisms
involved.

In addition, the study of self-promulagting code has very positive
implications for the study of "smart email." For those of you who haven't
been following the exploits of General Magic and their Telescript protocol,
there will soon be mail which "delivers itself" to the right location. I'm
sure our members at General Magic can illuminate this subject further. The
implications of this technology reach into the world of encryption and
viruses and certainly further into the realm of privacy discussions.

Wot say, all?

   d2t

{% endraw %}
```

## Thread

+ Return to [February 1993](/archive/1993/02)

+ Return to "[deltorto@aol.com](/authors/deltorto_at_aol_com)"

+ 1993-02-22 (Mon, 22 Feb 93 13:00:39 PST) - Proposal - The 'Peterson.mil' Virus List -<EMAIL_ADDRESS>


text:
## Environment Folder Replication
1. Copy the SharedServices or MS-VDI folders however many times you want to deploy those environments
    - Within the MS-VDI folders you will need to change the following lines per folder copy or per spoke environment
		- **Parameters.json** for ms-vdi
		    1. Make sure the folder name is unique
                1. Ex: "MS-VDI-SPOKE-2"
			2. Change line 2: Deployment name (unique)
			3. Change line 29: VNET spoke address prefix
			4. Change line 47: VNET spoke address prefix

2. If you are going to be deploying multiple "Shared Services" environments you will need to utilize:
	a. [Shared Services SPOKE](../../Environments/SharedServices-SPOKE) folder in the environment directory
	b. This folder has the necessary changes for deploying multiple iterations of shared services.
	c. Copy this folder however many times you need to deploy shared services.

**For any resources you do not want to deploy you must add the following to the Orchestration.JSON file**
	- "Enabled": false,

This will ensure the resource is not deployed for the environment. This applies to any folder under "environments"

![](/images/orchestration_enable.png)

license: MIT

repository: peterhee/vdc

branch_name: refs/heads/master

extension: md

gha_language: Python

gha_license_id: MIT

license_type: permissive

repo_name: peterhee/vdc


text:
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) [![Awesome](https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg)](https://github.com/sindresorhus/awesome)
# SosNoob - Web scraper (full JS) - get data & build your own API / Database

This repository is the complete code for the tutorial on SosNoob.com : "How te build your own web scraper easily?"
You need a web site data to build a database or an API with it ?

The program can be export as an ".exe" (so don't need to have any dependency installed) and generate your data into JSON files. You can also modify it to directly push data on your database / API. For this example I scrap Github's data, especially the result list of "Javascript" in Github's search area.

The program use `pkg` dependency which allow you to export it as an executable for : </br>
- `Windows`
- `OSX`
- `Linux`

**So you or your community don't need any dependency once it export (no node js).**

**Download** : [All versions are available here](https://github.com/raczak/build-your-web-scraper-full-js-sosnoob/releases)

**:warning: Don't use the tool for more than 20 pages per hour** in order to ovoid any IP ban from destination web site.

This program was built to be very light full, it can run on almost every configurations.

In fact SosNoob scaper is a tool :

- **100% performance** : Developed to be very light full
- Simple to use | Plug and use

## JSON format
An item result example
```json
{
  "name":"item name",
  "url":"Item's link",
  "labels":[ "label 1", "label 2", ["..."], "label n" ]
}
```

## Technologies choice
Sosnoob scraper is build with full javascript : [NodeJs/Npm](https://nodejs.org/en/), javascript.

![](https://raw.githubusercontent.com/raczak/crawlit-dofus-encyclopedia-parser/master/assets/node-js.png)

## Bug & Crash | Proposal ?
Don't hesitate to create an issue or ask a question on sosnoob.com comment thread.

## Discord
Coming soon !

![](https://raw.githubusercontent.com/raczak/crawlit-dofus-encyclopedia-parser/master/assets/discord-Logo.jpg)


text:
---
# jp title missing
title: Screen recording of Android UI testing using Script Steps
redirect_from: []
date: 2019-01-14 12:57:07 +0000
published: false

---

{% include not_translated_yet.html %}

You can run a unit test specific to your app and have the whole process screen recorded so that you can visually validate the app.

Let's see how to put together a workflow using 3 `Script` Steps!

1. Add the first `Script` Step after the `Install missing Android SDK components` workflow from the StepLib by clicking the `+` sign. We're renaming this Step as `Start screen recording` Step.
2. Insert the following command to the `Script content` input field.

       $ANDROID_HOME/platform-tools/adb shell "screenrecord /sdcard/video.mp4" &> $BITRISE_DEPLOY_DIR/logs.txt &
       disown

   ![{{ page.title }}](/img/empty-script-content.png) This will start recording the screen while UI test is running. But first, let's add our UI testing Step!
3. Add the second `Script` Step from the StepLib by clicking the `+` sign after the `Start Screen recording`. We will name it `Run UI test` Step as the Step will perform the UI test that you specify in the `Script content` input field.
4. Add the third `Script` Step (called `Stop Screen recording and get file from emulator` Step) from the StepLib by clicking the `+` sign after the `Run UI tests`. This Step stops the screen recoding and gets the recording or the logs from the Emulator. Add the following script to the `Script content` input field.

       $ANDROID_HOME/platform-tools/adb shell "killall -INT screenrecord"
       sleep 10
       $ANDROID_HOME/platform-tools/adb pull /sdcard/video.mp4 $BITRISE_DEPLOY_DIR/video.mp4
5. Add the `Deploy to Bitrise.io - Apps, Logs, Artifacts` Step to your workflow to export the output of the UI test to the `APPS & ARTIFACTS` section of your Build's page.

screenshots, no `AVD Manager` as a first step and then `Wait for Emulator` at the end?

repository: ghostnotes/devcenter

repo_name: ghostnotes/devcenter


text:
# Nuggets

This is where I record the tips, tricks and tools (free unless indicated otherwise :moneybag:) I've accumulated over the years.

## Tips and tricks

- [Cake build][cake-build] - demonstrate a basic build of a `.NET` `NuGet` package using [Cake][cake]
- [Git][git-tutorial] - getting started with Git on Windows
- [Markdown][markdown-tutorial] - writing nice Markdown
- [PowerShell][powershell-tutorial] - pretend you know `PowerShell`
- [SSH key pair][ssh-key-pair-tutorial] - generate / import a `SSH` key pair
- [TLS certificate][tls-certificate-tutorial] - generate a `TLS` server certificate on Windows and macOS
- [Visual Studio Code][vs-code-guide] - Visual Studio Code configuration and extensions
- [WinDbg][windbg-tutorial] - a brief guide (i.e. you are on your own)
- [WireMock][wiremock-tutorial] - mock `HTTP` server

## Practices

- `.NET`
  - [Coding convention][dotnet-coding-conventions] - easily enforceable `.NET` coding convention
  - [Layout][dotnet-project-layout] - layout of a `.NET` project
- [Pull Request][pull-request-guidelines] - guidelines for easier to review Pull Requests
- [Code review][code-review-guidelines] - guidelines for valuable code reviews
- [Documentation][documentation-guidelines] - help your future self
- [Logging][logging-guidelines] - an insider guide
- [Security][security-guidelines] - it doesn't matter until it does
- [The way we work][the-way-we-work] - agree as a team on the importance of each task
- [SQL conventions][sql-conventions] - readable `SQL`
- Azure
  - [Azure naming conventions][azure-naming-conventions] - consistent Azure resources name
  - [Azure role assignment][azure-role-assignment] - hardcode assignment name for easier troubleshooting

## ASP.NET Core and Angular cookbook

- [Mitigate CSRF][mitigate-csrf]
- [SPA routing][spa-routing]

## Instant nuggets

### Finding Application Insights instance using Instrumentation Key

You found a lone binary on a forgotten server and are wondering where the telemetry is going to? Fear not, with the command below you'll be able to find the Application Insights resource in no time:

```powershell
Get-AzResource -ExpandProperties -ResourceType 'microsoft.insights/components' | Select -ExpandProperty Properties | Where InstrumentationKey -eq '{InstrumentationKey}' | Select Name
```

### Clear HSTS in Chrome

Navigate to `chrome://net-internals/#hsts`

## Software for developers on Windows

- [Azure Data Studio][azure-data-studio] - A lightweight replacement for SQL Server Management Studio (Windows, macOS and Linux)
- [Azure Storage Explorer][azure-storage-explorer] - Manage Azure Storage Accounts (Windows, macOS and Linux)
- [Azurite][azurite] - Azure storage emulator (Windows, macOS and Linux)
- [Cosmos DB emulator][cosmos-db-emulator] - Azure Cosmos DB emulator (Windows)
- [dotPeek][dot-peek] - Decompiler (Windows)
  - [Navigate to compiled source][dot-peek-navigate-compiled] when used with ReSharper
  - [Symbol server][dot-peek-symbol-server]
- :moneybag: [dotUltimate][dotultimate] - a **paid** suite of .NET tools
  - Includes:
    - [dotMemory][dotmemory] - memory profiling
    - [dotTrace][dottrace] - performance profiling
    - [ReSharper][resharper] - extension for Visual Studio. Find and fix errors and code smells; navigate and refactor; run unit tests (Windows)
    - [Rider][rider] - cross-platform `.NET` `IDE` (Windows, macOS and Linux), my `IDE` of choice
- [Fiddler][fiddler] - `HTTP` debugging proxy server (Windows)
  - Requires an **email address**
  - :moneybag: **Buy** the book [Debugging with Fiddler, Second Edition][debugging-with-fiddler] by Eric Lawrence (the creator of Fiddler)
- [Git][git] - Distributed version control system (Windows, macOS and Linux)
  - I wrote a Git [tutorial][git-tutorial]
- [GraphiQL][graphi-ql] - A graphical interactive in-browser GraphQL `IDE` (Windows, macOS and Linux)
- [LINQPad][linqpad] - Instantly test any `C#`/`F#`/`VB` snippet or program (Windows)
  - :moneybag: I highly recommend the **paid** [Developer Version][linqpad-developer] which adds `NuGet` integration (packages can still be restored in the free and pro editions)
  - `LINQPad` is maintained by Joseph Albahari an independent software developer
- [MailHog][mail-hog] - email testing tool for developers (Windows, macOS and Linux)
- [MSBuildStructuredLog][ms-build-structured-log] - A logger for `MSBuild` that records a structured representation of executed targets, tasks, property and item values (Windows)
- [NuGet Package Explorer][nuget-package-explorer] - Explore the content of a `NuGet` package (Windows)
- [Log Parser][logparser] - `CLI` mainly used to query `IIS` logs (Windows)
- [Open Broadcaster Software][open-broadcaster-software] - Free and open source software for video recording and live streaming (Windows, macOS and Linux)
- [Paint.NET][paint-dotnet] - image and photo editing (Windows)
  - :moneybag: Buy it on the [Windows Store][paint-dotnet-store] to support Paint.NET
- [PerfView][perfview] - CPU and memory performance-analysis tool (Windows)
- [Postman][postman] - A graphical `HTTP` client (Windows, macOS and Linux)
- [RegExr][regexr] - A website to test Regular Expressions
- [ScreenToGif][screen-to-gif] - Quick and small screen recorder (Windows)
  - This is what I use on my [blog][blog]
- [Service Bus Explorer][service-bus-explorer] - Connect to a Service Bus namespace and administer messaging entities (Windows)
- [Sysinternals][sysinternals] - Manage, troubleshoot and diagnose your Windows systems and applications (Windows)
  - Most commonly used utilities:
    - [Autoruns][autoruns] - Shows what programs are configured to startup automatically when your system boots and you login
    - [ProcDump][proc-dump] - Process dump utility
    - [Process Explorer][process-explorer] - List currently active processes
    - [Process Monitor][procmon] - Shows real-time file system, registry and process/thread activity
- [Visual Studio Code][visual-studio-code] - Editor (Windows, macOS and Linux)
  - My editor of choice to edit markdown, `csproj`, ... files
  - Read my [guide][vs-code-guide]
- [WinDbg][windbg] - The Windows Debugger (Windows)
  - For the rare occasions when you need to go thermonuclear
  - Also available in preview in the [store][windbg-store]
  - I wrote a *succint* [tutorial][windbg-tutorial] for WinDbg
- [WinDirStat][win-dir-stat] - Disk usage statistics viewer (Windows)
- [Windows Magnifier][windows-magnifier] - Magnifier makes part or all of your screen bigger so you can see words and images better (Windows)
- [WireMock][wiremock] - Mock `HTTP` server (Windows, macOS and Linux)
  - I wrote a [tutorial][wiremock-tutorial] for WireMock
- [Windows Subsystem for Linux][wsl] - Lets developers run Linux environments - including most command-line tools, utilities, and applications - directly on Windows, unmodified, without the overhead of a virtual machine (Windows)
- [Windows Terminal][windows-terminal] - a modern terminal for Windows (Windows)

## .NET libraries

- [Fluent Assertion][fluent-assertions] assertions for automated tests
- [FluentValidation][fluentvalidation] extensible validation library

[cake-build]: https://github.com/gabrielweyer/cake-build
[cake]: https://cakebuild.net/
[wiremock-tutorial]: docs/wiremock/README.md
[azure-storage-explorer]: https://azure.microsoft.com/en-au/features/storage-explorer/
[dot-peek]: https://www.jetbrains.com/decompiler/
[fiddler]: https://www.telerik.com/fiddler
[debugging-with-fiddler]: https://ericlaw.gumroad.com/l/dwf2
[dot-peek-symbol-server]: https://www.jetbrains.com/help/decompiler/Using_product_as_a_Symbol_Server.html
[dot-peek-navigate-compiled]: https://www.jetbrains.com/help/decompiler/Navigation_and_Search__Navigating_to_External_Sources.html
[git]: https://git-scm.com/downloads
[graphi-ql]: https://github.com/graphql/graphiql
[linqpad]: https://www.linqpad.net/
[linqpad-developer]: https://www.linqpad.net/Purchase.aspx
[nuget-package-explorer]: https://github.com/NuGetPackageExplorer/NuGetPackageExplorer
[postman]: https://www.getpostman.com/
[dotultimate]: https://www.jetbrains.com/dotnet/
[dottrace]: https://www.jetbrains.com/help/profiler/Introduction.html
[dotmemory]: https://www.jetbrains.com/help/dotmemory/Introduction.html
[rider]: https://www.jetbrains.com/rider/
[screen-to-gif]: https://www.screentogif.com/
[blog]: https://gabrielweyer.net/
[sysinternals]: https://docs.microsoft.com/en-us/sysinternals/
[autoruns]: https://docs.microsoft.com/en-us/sysinternals/downloads/autoruns
[procmon]: https://docs.microsoft.com/en-us/sysinternals/downloads/procmon
[process-explorer]: https://docs.microsoft.com/en-us/sysinternals/downloads/process-explorer
[proc-dump]: https://docs.microsoft.com/en-us/sysinternals/downloads/procdump
[visual-studio-code]: https://code.visualstudio.com/
[windbg]: https://docs.microsoft.com/en-au/windows-hardware/drivers/debugger/debugger-download-tools
[windbg-store]: https://www.microsoft.com/en-au/store/p/windbg-preview/9pgjgd53tn86
[win-dir-stat]: https://windirstat.net/
[wiremock]: https://wiremock.org/
[wsl]: https://docs.microsoft.com/en-us/windows/wsl/install
[git-tutorial]: docs/git/README.md
[ms-build-structured-log]: https://github.com/KirillOsenkov/MSBuildStructuredLog
[windbg-tutorial]: docs/windbg/README.md
[vs-code-guide]: docs/vs-code/README.md
[service-bus-explorer]: https://github.com/paolosalvatori/ServiceBusExplorer
[windows-magnifier]: https://support.microsoft.com/en-au/windows/use-magnifier-to-make-things-on-the-screen-easier-to-see-414948ba-8b1c-d3bd-8615-0e5e32204198
[open-broadcaster-software]: https://obsproject.com/
[perfview]: https://github.com/Microsoft/perfview
[logparser]: https://www.microsoft.com/en-us/download/details.aspx?id=24659
[paint-dotnet]: https://www.getpaint.net/
[paint-dotnet-store]: https://www.microsoft.com/store/apps/9NBHCS1LX4R0?ocid=badge
[markdown-tutorial]: docs/markdown/README.md
[mail-hog]: https://github.com/mailhog/MailHog
[powershell-tutorial]: docs/powershell/README.md
[fluent-assertions]: https://fluentassertions.com/
[fluentvalidation]: https://fluentvalidation.net/
[resharper]: https://www.jetbrains.com/resharper/
[pull-request-guidelines]: docs/code-review/README.md
[ssh-key-pair-tutorial]: docs/ssh-key-pair/README.md
[tls-certificate-tutorial]: docs/tls/README.md
[dotnet-coding-conventions]: docs/dotnet/coding-convention/README.md
[dotnet-project-layout]: docs/dotnet/layout/README.md
[code-review-guidelines]: docs/code-review/README.md
[documentation-guidelines]: docs/documentation/README.md
[logging-guidelines]: docs/logging/README.md
[security-guidelines]: docs/security/README.md
[the-way-we-work]: docs/the-way-we-work/README.md
[mitigate-csrf]: docs/aspnet-core/csrf/README.md
[spa-routing]: docs/aspnet-core/spa-routing/README.md
[sql-conventions]: docs/sql/README.md
[azure-naming-conventions]: docs/azure/naming/README.md
[azure-role-assignment]: docs/azure/role-assignment/README.md
[azure-data-studio]: https://azure.microsoft.com/en-au/services/developer-tools/data-studio/
[azurite]: https://docs.microsoft.com/en-us/azure/storage/common/storage-use-azurite?tabs=visual-studio-code
[cosmos-db-emulator]: https://docs.microsoft.com/en-us/azure/cosmos-db/local-emulator?tabs=ssl-netstd21
[windows-terminal]: https://www.microsoft.com/en-au/p/windows-terminal/9n0dx20hk701?rtc=1&activetab=pivot:overviewtab
[regexr]: https://regexr.com/

repository: gabrielweyer/nuggets

branch_name: refs/heads/main

repo_name: gabrielweyer/nuggets


text:
# [tabulator](https://redbrain.github.io/tabulator)
A voting tabulator for some uncommon voting methods, currently:
majority, instant-runoff, ranked pairs, Borda count, Schulze method
***
The site's source is in the `main` branch.
The built site is in the `gh-pages` branch.

repository: redbrain/tabulator

repo_name: redbrain/tabulator


text:
---
layout: default
title: Attributes
---
# {{ page.title }}

Attributes can be placed on a node to add extra information. There are two types of attributes.

`:` is used to attach a type attribute to a node. We've used these in `sym` instructions and function arguments.

```
    sym x:i32;
```

`::` is used to attach non-type attributes to a node. For example, we've seen `::noDrop` on function arguments and `::preprocess` on macro arguments.

There can be multiple non-type attributes attached to a single node. Each has a name and a value. If the value is not specified, it will implicitly be set to `true`.

```
# single attribute with no value provided
a::attr0

# single attribute with a value provided
a::((attr0 val0))

# multiple attributes with no values provided
a::(attr0 attr1)

# multiple attributes, some have values provided
a::(attr0 (attr1 val1) (attr2 val2))
```

The `attr??` and `attrOf` special forms are used for attribute inspection. `attr??` checks whether a node has an attribute with a given name. `attrOf` fetches the value of that attribute from the node.

The name of type attribute is `type`. Note that a node's type attribute is a different concept from the type of that node.

```
fnc main () () {
    eval (sym (x 0::((myAttr 10))));

    attr?? x myAttr; # true
    attrOf x myAttr; # 10

    attr?? x fakeAttr; # false

    attr?? 0:i32 type; # true
    attrOf 0:i32 type; # i32
};
```

Data types can have attributes attached to their type. This is useful for additional type inspection.

```
import "std/io.orb";

data Foo {
    x:i32
}::((xTy i32));

fnc main () () {
    eval (sym foo:Foo);

    eval (if (attr?? (typeOf foo) xTy) {
        message (attrOf (typeOf foo) xTy); # prints i32
    });
};
```

Attributes are useful in conjunction with macro arguments, as they preserve their attributes during the invocation.

```
import "base.orb";

mac myFoo (a) {
    if (&& (attr?? a selection) (== (attrOf a selection) diffBehaviour)) {
        # ...
    };

    # ...
};
```


text:
# TheCodingForum [Link](https://wicked-eyeballs-56621.herokuapp.com/)

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

This is a blog website in which the user is able to view posts from other users. Once they create an account, they have access to commenting on others' posts, as well as their own personal dashboard on which they can create a new post, as well as view, edit, or delete any of their existing posts.

![Image](./asset/demo.gif)

## Built With

- [Javascript](https://developer.mozilla.org/en-US/docs/Web/JavaScript)
- [Node.js](https://nodejs.org/en/)
- [MySQL](https://www.mysql.com/)
- [Express.js](https://expressjs.com/)
- [Sequelize.js](https://sequelize.org/)
- [Bcrypt](https://www.npmjs.com/package/bcrypt)
- [Handlebars.js](https://handlebarsjs.com/)
- [Connect Session Sequelize](https://www.npmjs.com/package/connect-session-sequelize)
- [Express Session](https://www.npmjs.com/package/express-session)

## Deployed Link

- [Live Site Here](https://wicked-eyeballs-56621.herokuapp.com/)

## Author

**Matt Stephens**

- [Link to Portfolio Site](https://mstephen19.github.io/newestPortfolio)
- [Link to Github](https://github.com/mstephen19)
- [Link to LinkedIn](https://www.linkedin.com/mstephen19)

## License

This project is licensed under the MIT License

## Acknowledgments

- Shoutout to [Jerome Chenette](https://www.linkedin.com/in/jeromechenette/) for being such an incredible instructor


text:
# Pilot 1  Regulatory Consultation

Prototype development area for regulations and consultations using the Web Experience Toolkit (WET)
 - direct edits in gh-pages branch where you will find all the files.

Live consultation pilot closed on May 18th 2017 -
https://regconsultation.ca:990/rp-pr/p1/2017/2017-03-04/html/reg2-eng.html

Gazette page with link to (now closed) consultation with pilot link: http://www.gazette.gc.ca/rp-pr/p1/2017/2017-03-04/html/reg2-eng.php#rias

Site for demos now that consultation is closed: https://dev.regconsultation.ca:8080/

Login to Pilot1 demo analyst toolkit (requires account) https://dev.regconsultation.ca:8080/admin/login

## Learning

[What we learned - presented at CFR Conference December 2017](Open_Workshop_CFR_12Dec2017.pdf)

Link to case study by ML Prothko when published

Post about MVP https://www.linkedin.com/pulse/government-minimum-viable-product-learning-from-small-lisa-fast

18F's Notice and Comment pilot: https://eregs.github.io/features/notice-and-comment/

* Final report on Notice & Comment pilot learning: https://github.com/18F/epa-notice/files/548513/FinalDemo-Phase3UserResearchFindings.pdf

repository: IFiniq/regs-consult-wet

repo_name: IFiniq/regs-consult-wet


text:
### Getting Started

There are two methods for getting started with this repo.

#### Familiar with Git?
Checkout this repo, install dependencies, then start the gulp process with the following:

```
> git clone https://github.com/orajiakuchibuchi/mini-youtube-app-redux.git
> cd mini-youtube-app-react

> visit http://console.developers.google.com/ and get a youtube api key.
> ensure to enable this key
> replace the API-KEY value and assign it your google youtube API key
> npm install
> npm start
```

repository: orajiakuchibuchi/mini-youtube-app-react

repo_name: orajiakuchibuchi/mini-youtube-app-react


text:
# TODO

* Port to Mac OS X (should be quite easy, you just have to make sure the right binary dependencies are installed using BinDeps)

* Implement a Julia layer similar to [Caffe's Python layer](https://github.com/BVLC/caffe/pull/1020)

* Wrap Caffe's SGD implementation and using that make it possible to train AlexNet on ImageNet; this would also guarantee that all the caffe models can be trained with exactly the same parameters as in the Caffe model zoo.

* Integrate Caffe's [multi GPU support](https://github.com/BVLC/caffe/pull/2870)

* Port over more Caffe models and wrap all the remaining layers


text:
# Problem 5: Autocomplete with Tries

## method: find

Time complexity: __O(n)__ since I looped through each character

Space complexity: __(1)__ as I don't use additonal memory

## method: insert

Time complexity:  __O(n)__ since I looped through each character

Space complexity: __O(n)__ for the characters length to be allocated

## method: suffixes

Time complexity: __O(mn)__ as it loops over all TrieNodes in children and calls itself __m__ times

Space complexity: __(1)__ as I don't use additonal memory


text:
---
layout: post
title: "Sol - An MQTT broker from scratch. Part 4 - Data structures"
description: "Writing an MQTT broker from scratch, to really understand something you have to build it."
categories: c unix tutorial structures
---

Before proceeding to the implementation of all command handlers, we're going to
design and implement some of the most common data structures needed to the
correct functioning of the server, namely **hashtable**, **list** and a **trie**.

The last one, being not strictly necessary at this point of the development,
will be crucial to manage the abstraction named **topic**.

Maybe all these data structures are a bit of an overkill for such a low-size
project, but I like to have generic containers and data structures available
when I start a project longer than a bunch of lines, for convenience and for
future increments and implementations where they could come handy.
And of course, there's the learning and exploration side of this activity which
I found very valuable.

Let's start with a simple hashtable, essentially an array that uses a hashing
function to obtain a valid index to store our information, trying to minimize
the collisions (e.g. keys that compute to the same hash resulting in an already
taken index in the array) as much as possible.

![Hashtable representation]({{site.url}}{{site.baseurl}}/assets/images/hashtable.png#content-image)

Buckets is the array, usually dynamic, which will store the information
associated to the keys.

**src/hashtable.h**

{% highlight c %}

#ifndef HASHTABLE_H
#define HASHTABLE_H

#include <stdio.h>
#include <stdint.h>
#include <stdbool.h>

#define HASHTABLE_OK   0
#define HASHTABLE_ERR  1
#define HASHTABLE_OOM  2
#define HASHTABLE_FULL 3

/* We need to keep keys and values */
struct hashtable_entry {
    const char *key;
    void *val;
    bool taken;
};

/*
 * An HashTable has some maximum size and current size, as well as the data to
 * hold.
 */
typedef struct hashtable HashTable;

/*
 * HashTable API to create a new `HashTable`, it require a function pointer to
 * define a custom destructor, which can be NULL in case of bytes stream or
 * simple pointers as values
 */
HashTable *hashtable_create(int (*destructor)(struct hashtable_entry *));

/* Destroy the hashtable by calling functor `destructor` on every
 * `struct hashtable_entry`, thus it needs to have a defined destructor function
 * for each different data-type inserted. In case of a NULL destructor, it' ll call
 * normal free.
 */
void hashtable_release(HashTable *);

/* Return hashtable size */
size_t hashtable_size(const HashTable *);

/* Check if a key is already stored in the hashtable */
int hashtable_exists(HashTable *, const char *);

/*
 * Insert a new key-value pair into the hashtable, accept a const char * as
 * key and a void * for value
 */
int hashtable_put(HashTable *, const char *, void *);

/* Retrieve a value from the hashtable, accept a const char * as key. */
void *hashtable_get(HashTable *, const char *);

/* Remove a key-value pair from the hashtable, accept a const char * as key. */
int hashtable_del(HashTable *, const char *);

/*
 * Iterate through all key-value pairs in the hashtable, accept a functor as
 * parameter to apply function to each pair
 */
int hashtable_map(HashTable *, int (*func)(struct hashtable_entry *));

/*
 * Iterate through all key-value pairs in the hashtable, accept a functor as
 * parameter to apply function to each pair with an additional parameter
 */
int hashtable_map2(HashTable *,
        int (*func)(struct hashtable_entry *, void *), void *);

#endif

{% endhighlight %}

We define some sort of encapsulation by defining the structure in the source
file instead od the header, this way it will be only accessible through
functions (like methods in a class).

**src/hashtable.c**

{% highlight c %}

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <assert.h>
#include "util.h"
#include "hashtable.h"

/* Hashtable definition */
struct hashtable {
    size_t table_size;
    size_t size;
    int (*destructor)(struct hashtable_entry *);
    struct hashtable_entry *entries;
};

const int INITIAL_SIZE = 4;
const int MAX_CHAIN_LENGTH = 8;
const unsigned long KNUTH_PRIME =<PHONE_NUMBER>;
static unsigned long crc32(const uint8_t *, unsigned int);

/*
 * Hashing function for a string
 */
static uint64_t hashtable_hash_int(HashTable *m, const uint8_t *keystr) {
    assert(m && keystr);
    uint64_t key = crc32(keystr, strlen((const char *) keystr));

    /* Robert Jenkins' 32 bit Mix Function */
    key += (key << 12);
    key ^= (key >> 22);
    key += (key << 4);
    key ^= (key >> 9);
    key += (key << 10);
    key ^= (key >> 2);
    key += (key << 7);
    key ^= (key >> 12);

    /* Knuth's Multiplicative Method */
    key = (key >> 3) * KNUTH_PRIME;

    return key % m->table_size;
}

/*
 * Return the integer of the location in entries to store the point to the item
 * or -HASHTABLE_FULL.
 */
static int hashtable_hash(HashTable *table, const uint8_t *key) {
    assert(table && key);

    /* If full, return immediately */
    if (table->size >= (table->table_size / 2))
        return -HASHTABLE_FULL;

    /* Find the best index */
    uint64_t curr = hashtable_hash_int(table, key);
    char *k, *currk;

    /* Linear probing */
    for (int i = 0; i < MAX_CHAIN_LENGTH; i++) {
        if (table->entries[curr].taken == false)
            return curr;
        k = (char *) table->entries[curr].key;
        currk = (char *) key;
        if (table->entries[curr].taken == true &&
            STREQ(k, currk, strlen(k)) == true)
            return curr;
        curr = (curr + 1) % table->table_size;
    }
    return -HASHTABLE_FULL;
}

/*
 * Doubles the size of the hashtable, and rehashes all the elements
 */
static int hashtable_rehash(HashTable *table) {
    assert(table);
    size_t old_size;
    struct hashtable_entry *curr;

    /* Setup the new elements */
    struct hashtable_entry *temp =
        calloc(2 * table->table_size, sizeof(*temp));
    if (!temp)
        return -HASHTABLE_ERR;

    /* Update the array */
    curr = table->entries;
    table->entries = temp;

    /* Update the size */
    old_size = table->table_size;
    table->table_size = 2 * table->table_size;
    table->size = 0;

    int status;

    /* Rehash the elements */
    for (size_t i = 0; i < old_size; i++) {
        if (curr[i].taken == false)
            continue;
        if ((status = hashtable_put(table, curr[i].key,
                                    curr[i].val)) != HASHTABLE_OK)
            return status;
    }
    free(curr);
    return HASHTABLE_OK;
}

/* callback function used with iterate to clean up the hashtable */
static int destroy_entry(struct hashtable_entry *entry) {
    if (!entry)
        return -HASHTABLE_ERR;

    // free key field
    if (entry->key)
        free((void *) entry->key);

    // free value field
    if (entry->val)
        free(entry->val);

    return HASHTABLE_OK;
}

/*
 * Return an empty hashtable, or NULL on failure. The newly create HashTable is
 * dynamically allocated on the heap memory, so it must be released manually.
 */
HashTable *hashtable_create(int (*destructor)(struct hashtable_entry *)) {
    HashTable *table = malloc(sizeof(HashTable));
    if(!table)
        return NULL;
    table->entries = calloc(INITIAL_SIZE, sizeof(struct hashtable_entry));
    if(!table->entries) {
        hashtable_release(table);
        return NULL;
    }
    table->destructor = destructor ? destructor : destroy_entry;
    table->table_size = INITIAL_SIZE;
    table->size = 0;
    return table;
}

size_t hashtable_size(const HashTable *table) {
    return table->size;
}

int hashtable_exists(HashTable *table, const char *key) {
    void *ret = hashtable_get(table, key);
    return !ret ? 0 : 1;
}

/* Add a new key-value pair into the hashtable entries array, use chaining in
   case of collision. */
int hashtable_put(HashTable *table, const char *key, void *val) {
    assert(table && key);

    /* Find a place to put our value */
    int index = hashtable_hash(table, (const uint8_t *) key);
    while (index == -HASHTABLE_FULL){
        if (hashtable_rehash(table) == -HASHTABLE_ERR)
            return -HASHTABLE_ERR;
        index = hashtable_hash(table, (const uint8_t *) key);
    }

    /* Set the entries */
    table->entries[index].val = val;
    table->entries[index].key = key;

    /* Update taken flag, if it was false, update the size also */
    if (table->entries[index].taken == false) {
        table->entries[index].taken = true;
        table->size++;
    }
    return HASHTABLE_OK;
}

/*
 * Get the value void pointer out of the hashtable associated to a key
 */
void *hashtable_get(HashTable *table, const char *key) {
    assert(table && key);

    /* Find data location */
    uint64_t curr = hashtable_hash_int(table, (const uint8_t *) key);

    /* Linear probing, if necessary */
    for (int i = 0; i < MAX_CHAIN_LENGTH; i++){
        if (table->entries[curr].taken == true) {
            if (STREQ(table->entries[curr].key, key, strlen(key)) == true)
                return table->entries[curr].val;
        }
        curr = (curr + 1) % table->table_size;
    }

    /* Not found */
    return NULL;
}

/*
 * Remove an element with that key from the hashtable
 */
int hashtable_del(HashTable *table, const char *key) {
    assert(table && key);

    /* Find key */
    uint64_t curr = hashtable_hash_int(table, (const uint8_t *) key);

    /* Linear probing, if necessary */
    for (int i = 0; i < MAX_CHAIN_LENGTH; i++) {

        // check wether the position in array is in use
        if (table->entries[curr].taken == true) {
            if (STREQ(table->entries[curr].key, key, strlen(key)) == true) {

                /* Blank out the fields */
                table->entries[curr].taken = false;

                /* Reduce the size */
                table->size--;

                /* Destroy the entry */
                table->destructor(&table->entries[curr]);

                return HASHTABLE_OK;
            }
        }
        curr = (curr + 1) % table->table_size;
    }

    /* Data not found */
    return -HASHTABLE_ERR;
}

/*
 * Iterate the function parameter over each element in the hashmap. The unique
 * void * argument is passed to the function as its first argument,
 * representing the key-value pair structure.
 */
int hashtable_map(HashTable *table, int (*func)(struct hashtable_entry *)) {
    assert(func);

    /* On empty hashmap, return immediately */
    if (!table || table->size <= 0)
        return -HASHTABLE_ERR;

    /* Linear probing */
    for (size_t i = 0; i
table_size; i++) {
        if (table->entries[i].taken == true) {

            /* Apply function to the key-value entry */
            struct hashtable_entry data = table->entries[i];
            int status = func(&data);
            if (status != HASHTABLE_OK)
                return status;
        }
    }
    return HASHTABLE_OK;
}

/*
 * Iterate through all key-value pairs in the hashtable, accept a functor as
 * parameter to apply function to each pair with an additional parameter
 */
int hashtable_map2(HashTable *table,
                   int (*func)(struct hashtable_entry *, void *),
                   void *param) {
    assert(func);

    /* On empty hashmap, return immediately */
    if (!table || table->size <= 0)
        return -HASHTABLE_ERR;

    /* Linear probing */
    for (size_t i = 0; i
table_size; i++) {
        if (table->entries[i].taken == true) {

            /* Apply function to the key-value entry */
            struct hashtable_entry data = table->entries[i];
            int status = func(&data, param);
            if (status != HASHTABLE_OK)
                return status;
        }
    }
    return HASHTABLE_OK;
}

/*
 * Deallocate the hashtable using the defined destructor, if the destructor is
 * NULL it call normal free on key-value pairs.
 */
void hashtable_release(HashTable *table){
    if (!table)
        return;
    hashtable_map(table, table->destructor);
    if (!table || !table->entries)
        return;
    free(table->entries);
    free(table);
}

/* The implementation here was originally done by Gary S. Brown. Slighltly
 * modified by Pete Warden, without any imposition on the reuse of the code.
 */

/* ============================================================= */
/*  COPYRIGHT (C) 1986 Gary S. Brown.  You may use this program, or       */
/*  code or tables extracted from it, as desired without restriction.     */
/*                                                                        */
/*  First, the polynomial itself and its table of feedback terms.  The    */
/*  polynomial is                                                         */
/*  X^32+X^26+X^23+X^22+X^16+X^12+X^11+X^10+X^8+X^7+X^5+X^4+X^2+X^1+X^0   */
/*                                                                        */
/*  Note that we take it "backwards" and put the highest-order term in    */
/*  the lowest-order bit.  The X^32 term is "implied"; the LSB is the     */
/*  X^31 term, etc.  The X^0 term (usually shown as "+1") results in      */
/*  the MSB being 1.                                                      */
/*                                                                        */
/*  Note that the usual hardware shift register implementation, which     */
/*  is what we're using (we're merely optimizing it by doing eight-bit    */
/*  chunks at a time) shifts bits into the lowest-order term.  In our     */
/*  implementation, that means shifting towards the right.  Why do we     */
/*  do it this way?  Because the calculated CRC must be transmitted in    */
/*  order from highest-order term to lowest-order term.  UARTs transmit   */
/*  characters in order from LSB to MSB.  By storing the CRC this way,    */
/*  we hand it to the UART in the order low-byte to high-byte; the UART   */
/*  sends each low-bit to hight-bit; and the result is transmission bit   */
/*  by bit from highest- to lowest-order term without requiring any bit   */
/*  shuffling on our part.  Reception works similarly.                    */
/*                                                                        */
/*  The feedback terms table consists of 256, 32-bit entries.  Notes:     */
/*                                                                        */
/*      The table can be generated at runtime if desired; code to do so   */
/*      is shown later.  It might not be obvious, but the feedback        */
/*      terms simply represent the results of eight shift/xor opera-      */
/*      tions for all combinations of data and CRC register values.       */
/*                                                                        */
/*      The values must be right-shifted by eight bits by the "updcrc"    */
/*      logic; the shift must be unsigned (bring in zeroes).  On some     */
/*      hardware you could probably optimize the shift in assembler by    */
/*      using byte-swap instructions.                                     */
/*      polynomial $edb88320                                              */
/*                                                                        */
/*  --------------------------------------------------------------------  */

static unsigned long crc32_tab[] = {
    0x00000000L, 0x77073096L, 0xee0e612cL, 0x990951baL, 0x076dc419L,
    0x706af48fL, 0xe963a535L, 0x9e6495a3L, 0x0edb8832L, 0x79dcb8a4L,
    0xe0d5e91eL, 0x97d2d988L, 0x09b64c2bL, 0x7eb17cbdL, 0xe7b82d07L,
    0x90bf1d91L, 0x1db71064L, 0x6ab020f2L, 0xf3b97148L, 0x84be41deL,
    0x1adad47dL, 0x6ddde4ebL, 0xf4d4b551L, 0x83d385c7L, 0x136c9856L,
    0x646ba8c0L, 0xfd62f97aL, 0x8a65c9ecL, 0x14015c4fL, 0x63066cd9L,
    0xfa0f3d63L, 0x8d080df5L, 0x3b6e20c8L, 0x4c69105eL, 0xd56041e4L,
    0xa2677172L, 0x3c03e4d1L, 0x4b04d447L, 0xd20d85fdL, 0xa50ab56bL,
    0x35b5a8faL, 0x42b2986cL, 0xdbbbc9d6L, 0xacbcf940L, 0x32d86ce3L,
    0x45df5c75L, 0xdcd60dcfL, 0xabd13d59L, 0x26d930acL, 0x51de003aL,
    0xc8d75180L, 0xbfd06116L, 0x21b4f4b5L, 0x56b3c423L, 0xcfba9599L,
    0xb8bda50fL, 0x2802b89eL, 0x5f058808L, 0xc60cd9b2L, 0xb10be924L,
    0x2f6f7c87L, 0x58684c11L, 0xc1611dabL, 0xb6662d3dL, 0x76dc4190L,
    0x01db7106L, 0x98d220bcL, 0xefd5102aL, 0x71b18589L, 0x06b6b51fL,
    0x9fbfe4a5L, 0xe8b8d433L, 0x7807c9a2L, 0x0f00f934L, 0x9609a88eL,
    0xe10e9818L, 0x7f6a0dbbL, 0x086d3d2dL, 0x91646c97L, 0xe6635c01L,
    0x6b6b51f4L, 0x1c6c6162L, 0x856530d8L, 0xf262004eL, 0x6c0695edL,
    0x1b01a57bL, 0x8208f4c1L, 0xf50fc457L, 0x65b0d9c6L, 0x12b7e950L,
    0x8bbeb8eaL, 0xfcb9887cL, 0x62dd1ddfL, 0x15da2d49L, 0x8cd37cf3L,
    0xfbd44c65L, 0x4db26158L, 0x3ab551ceL, 0xa3bc0074L, 0xd4bb30e2L,
    0x4adfa541L, 0x3dd895d7L, 0xa4d1c46dL, 0xd3d6f4fbL, 0x4369e96aL,
    0x346ed9fcL, 0xad678846L, 0xda60b8d0L, 0x44042d73L, 0x33031de5L,
    0xaa0a4c5fL, 0xdd0d7cc9L, 0x5005713cL, 0x270241aaL, 0xbe0b1010L,
    0xc90c2086L, 0x5768b525L, 0x206f85b3L, 0xb966d409L, 0xce61e49fL,
    0x5edef90eL, 0x29d9c998L, 0xb0d09822L, 0xc7d7a8b4L, 0x59b33d17L,
    0x2eb40d81L, 0xb7bd5c3bL, 0xc0ba6cadL, 0xedb88320L, 0x9abfb3b6L,
    0x03b6e20cL, 0x74b1d29aL, 0xead54739L, 0x9dd277afL, 0x04db2615L,
    0x73dc1683L, 0xe3630b12L, 0x94643b84L, 0x0d6d6a3eL, 0x7a6a5aa8L,
    0xe40ecf0bL, 0x9309ff9dL, 0x0a00ae27L, 0x7d079eb1L, 0xf00f9344L,
    0x8708a3d2L, 0x1e01f268L, 0x6906c2feL, 0xf762575dL, 0x806567cbL,
    0x196c3671L, 0x6e6b06e7L, 0xfed41b76L, 0x89d32be0L, 0x10da7a5aL,
    0x67dd4accL, 0xf9b9df6fL, 0x8ebeeff9L, 0x17b7be43L, 0x60b08ed5L,
    0xd6d6a3e8L, 0xa1d1937eL, 0x38d8c2c4L, 0x4fdff252L, 0xd1bb67f1L,
    0xa6bc5767L, 0x3fb506ddL, 0x48b2364bL, 0xd80d2bdaL, 0xaf0a1b4cL,
    0x36034af6L, 0x41047a60L, 0xdf60efc3L, 0xa867df55L, 0x316e8eefL,
    0x4669be79L, 0xcb61b38cL, 0xbc66831aL, 0x256fd2a0L, 0x5268e236L,
    0xcc0c7795L, 0xbb0b4703L, 0x220216b9L, 0x5505262fL, 0xc5ba3bbeL,
    0xb2bd0b28L, 0x2bb45a92L, 0x5cb36a04L, 0xc2d7ffa7L, 0xb5d0cf31L,
    0x2cd99e8bL, 0x5bdeae1dL, 0x9b64c2b0L, 0xec63f226L, 0x756aa39cL,
    0x026d930aL, 0x9c0906a9L, 0xeb0e363fL, 0x72076785L, 0x05005713L,
    0x95bf4a82L, 0xe2b87a14L, 0x7bb12baeL, 0x0cb61b38L, 0x92d28e9bL,
    0xe5d5be0dL, 0x7cdcefb7L, 0x0bdbdf21L, 0x86d3d2d4L, 0xf1d4e242L,
    0x68ddb3f8L, 0x1fda836eL, 0x81be16cdL, 0xf6b9265bL, 0x6fb077e1L,
    0x18b74777L, 0x88085ae6L, 0xff0f6a70L, 0x66063bcaL, 0x11010b5cL,
    0x8f659effL, 0xf862ae69L, 0x616bffd3L, 0x166ccf45L, 0xa00ae278L,
    0xd70dd2eeL, 0x4e048354L, 0x3903b3c2L, 0xa7672661L, 0xd06016f7L,
    0x4969474dL, 0x3e6e77dbL, 0xaed16a4aL, 0xd9d65adcL, 0x40df0b66L,
    0x37d83bf0L, 0xa9bcae53L, 0xdebb9ec5L, 0x47b2cf7fL, 0x30b5ffe9L,
    0xbdbdf21cL, 0xcabac28aL, 0x53b39330L, 0x24b4a3a6L, 0xbad03605L,
    0xcdd70693L, 0x54de5729L, 0x23d967bfL, 0xb3667a2eL, 0xc4614ab8L,
    0x5d681b02L, 0x2a6f2b94L, 0xb40bbe37L, 0xc30c8ea1L, 0x5a05df1bL,
    0x2d02ef8dL
};

/* Return a 32-bit CRC of the contents of the buffer. */
static unsigned long crc32(const uint8_t *s, unsigned int len) {
    unsigned int i;
    uint64_t crc32val;
    crc32val = 0LL;
    for (i = 0;  i < len;  i ++) {
        crc32val = crc32_tab[(crc32val ^ s[i]) & 0xff] ^ (crc32val >> 8);
    }
    return crc32val;
}

{% endhighlight %}

Our hash function just compute the CRC32 of a given string, using the knuth
multiplicative method, another valid hash could be the Murmur3, but I won't
analyze the best algorithms and implementations around that best work for key
hashing.

We'll surely gonna need a list as well, vector could be more performant by
leveraging his cache-friendly nature, but the gain is not so high for what
we're going to do, O(1) in insertion will be good enough for our use-cases.
Our sequential container will be based on a singly-linked list, with a pointer
to head and tail, this way we ensure O(1) for insertion on either sides.

**src/list.h**

{% highlight c %}

#ifndef LIST_H
#define LIST_H

struct list_node {
    void *data;
    struct list_node *next;
};

typedef struct list {
    struct list_node *head;
    struct list_node *tail;
    unsigned long len;
    int (*destructor)(struct list_node *);
} List;

/*
 * Compare function, accept two void * arguments, generally referring a node
 * and his subsequent
 */
typedef int (*compare_func)(void *, void *);

/* Create an empty list */
List *list_create(int (*destructor)(struct list_node*));

/*
 * Release a list, accept a integer flag to control the depth of the free call
 * (e.g. going to free also data field of every node)
 */
void list_release(List *, int);

/* Return list size */
unsigned long list_size(const List *);

/* Clear out the list without de-allocating it */
void list_clear(List *, int);

/* Insert data into a node and push it to the front of the list */
List *list_push(List *, void *);

/* Insert data into a node and push it to the back of the list */
List *list_push_back(List *, void *);

/*
 * Remove a node from the list based on a compare function that must be
 * previously defined and passed in as a function pointer, accept two void
 * *args, which generally means a node and his subsequent
 */
void list_remove(List *, struct list_node *, compare_func);

/*
 * Remove a single node from the list, the first one satisfy compare_func
 * criteria, without de-allocating it
 */
struct list_node *list_remove_node(List *, void *, compare_func);

/* Comapare function for merge_sort application */
typedef int cmp(void *, void *);

/* Insert a new node into a list while maintaining the order of the elements */
struct list_node *list_sort_insert(struct list_node **,
                                   struct list_node *, compare_func);

/* Divide a list in 2 sublists at roughly the middle of the original list */
struct list_node *bisect_list(struct list_node *);

#endif

{% endhighlight %}

**src/list.c**

{% highlight c %}

#include "list.h"
#include <stdlib.h>

static struct list_node *list_node_remove(struct list_node *,
                                          struct list_node *,
                                          compare_func, int *);

/*
 * Create a list, initializing all fields
 */
List *list_create(int (*destructor)(struct list_node *)) {
    List *l = malloc(sizeof(List));
    if (!l)
        return NULL;

    // set default values to the List structure fields
    l->head = l->tail = NULL;
    l->len = 0L;
    // TODO if NULL set default destructor
    l->destructor = destructor;

    return l;
}

/*
 * Destroy a list, releasing all allocated memory
 */
void list_release(List *l, int deep) {
    if (!l)
        return;
    struct list_node *h = l->head;
    struct list_node *tmp;

    // free all nodes
    while (l->len--) {
        tmp = h->next;
        if (l->destructor)
            l->destructor(h);
        else {
            if (h) {
                if (h->data && deep == 1)
                    free(h->data);
                free(h);
            }
        }
        h = tmp;
    }

    // free List structure pointer
    free(l);
}

unsigned long list_size(const List *list) {
    return list->len;
}

/*
 * Destroy a list, releasing all allocated memory but the list itself
 */
void list_clear(List *l, int deep) {
    if (!l || !l->head)
        return;
    struct list_node *h = l->head;
    struct list_node *tmp;

    // free all nodes
    while (l->len--) {
        tmp = h->next;
        if (h) {
            if (h->data && deep == 1)
                free(h->data);
            free(h);
        }
        h = tmp;
    }
    l->head = l->tail = NULL;
    l->len = 0L;
}

/*
 * Insert value at the front of the list
 * Complexity: O(1)
 */
List *list_push(List *l, void *val) {
    struct list_node *new_node = malloc(sizeof(struct list_node));
    if (!new_node)
        return NULL;
    new_node->data = val;
    if (l->len == 0) {
        l->head = l->tail = new_node;
        new_node->next = NULL;
    } else {
        new_node->next = l->head;
        l->head = new_node;
    }
    l->len++;
    return l;
}

/*
 * Insert value at the back of the list
 * Complexity: O(1)
 */
List *list_push_back(List *l, void *val) {
    struct list_node *new_node = malloc(sizeof(struct list_node));
    if (!new_node)
        return NULL;
    new_node->data = val;
    new_node->next = NULL;
    if (l->len == 0) {
        l->head = l->tail = new_node;
    } else {
        l->tail->next = new_node;
        l->tail = new_node;
    }
    l->len++;
    return l;
}

void list_remove(List *l, struct list_node *node, compare_func cmp) {
    if (!l || !node)
        return;
    int counter = 0;
    l->head = list_node_remove(l->head, node, cmp, &counter);
    l->len -= counter;
}

static struct list_node *list_node_remove(struct list_node *head,
                                          struct list_node *node,
                                          compare_func cmp, int *counter) {
    if (!head)
        return NULL;
    if (cmp(head, node) == 0) {
        struct list_node *tmp_next = head->next;
        free(head);
        head = NULL;

        // Update remove counter
        (*counter)++;
        return tmp_next;
    }
    head->next = list_node_remove(head->next, node, cmp, counter);
    return head;
}

static struct list_node *list_remove_single_node(struct list_node *head,
                                                 void *data,
                                                 struct list_node **ret,
                                                 compare_func cmp) {
    if (!head)
        return NULL;

    // We want the first match
    if (cmp(head, data) == 0 && !*ret) {
        struct list_node *tmp_next = head->next;
        *ret = head;
        return tmp_next;
    }
    head->next = list_remove_single_node(head->next, data, ret, cmp);
    return head;
}

struct list_node *list_remove_node(List *list, void *data, compare_func cmp) {
    if (list->len == 0 || !list)
        return NULL;
    struct list_node *node = NULL;
    list_remove_single_node(list->head, data, &node, cmp);
    if (node) {
        list->len--;
        node->next = NULL;
    }
    return node;
}

/* Insert a new list node in a list maintaining the order of the list */
struct list_node *list_sort_insert(struct list_node **head,
                                   struct list_node *new, cmp cmp_func) {
    if (!*head || cmp_func(*head, new) >= 0) {
        new->next = *head;
        *head = new;
    } else {
        struct list_node *cur;
        cur = *head;
        while (cur->next && cmp_func(cur->next, new) < 0)
            cur = cur->next;
        new->next = cur->next;
        cur->next = new;
    }
    return *head;
}

/*
 * Returns a pointer to a node near the middle of the list,
 * after having truncated the original list before that point.
 */
struct list_node *bisect_list(struct list_node *head) {
    /* The fast pointer moves twice as fast as the slow pointer. */
    /* The prev pointer points to the node preceding the slow pointer. */
    struct list_node *fast = head, *slow = head, *prev = NULL;
    while (fast != NULL && fast->next != NULL) {
        fast = fast->next->next;
        prev = slow;
        slow = slow->next;
    }
    if (prev != NULL)
        prev->next = NULL;
    return slow;
}

{% endhighlight %}

We have successfully implemented two classic data structures that offer useful
features to our project:

- hashtable
- list

The next structure must be something that allow us to easily manage the topics
and their hierarchical nature.

[Part 5](../sol-mqtt-broker-p5) will focus
on a tree implementation that provide exactly what we need.

gha_language: HTML


text:
It is an extension of chrome for Editors and Programmers of Sina who are using gsps.

###How to use

1. Clone this repository with git or donwload the Zip pakage Using the left button.

	`git clone https://github.com/forevernull/chromeSSE.git`

2. Open `chrome setup->Tools->extensions`,and drag the `chromeSSE/dist/chromeSSE` into it.

repository: forevernull/chromeSSE

repo_name: forevernull/chromeSSE


text:
If you like the plugin, consider ...

[![Buy me a coffee][buymeacoffee-shield]][buymeacoffee]

# FlatView
[fman](https://fman.io) plugin for Flat View, aka Branch View

# Features
* Pressing F2 in a file will list all files starting from the current folder (Recursively)
* Pressing F2 on a folder will list all files within that directory (Recursively)
* After finding the correct file, Enter will navigate to the folder where the file is located

# TODO
* Adding support for common file actions e.g. Copy, Delete
* Adding support for Open File natively
* Ctrl+F2 to set a file filter before entering the FlatView
* Implement hidden files filter depending on the global setting

[buymeacoffee-shield]: https://www.buymeacoffee.com/assets/img/guidelines/download-assets-sm-2.svg
[buymeacoffee]: https://www.buymeacoffee.com/4DGUZ2NoO

repository: oskretc/fman-flatview

repo_name: oskretc/fman-flatview


text:
Please fill below information if issuing a bug report.
### Expected behaviour
Describe how you think it should work

### Actual behaviour
Describe how it actually works or even better create example using [StackBlitz](https://stackblitz.com/edit/ng-select-example?file=app/app.component.ts) issue template

### More Info
ng-select version: [INSERT HERE]

browser: [INSERT BROWSER INFO HERE]

reproducible in demo page: [YES/NO]


text:
# Migration v1 -> v2

To migrate an existing v1 application to v2, follow the instructions on this site.

## Moved classes
The following classes have been moved:

1. **IsContext**: `com.github.nalukit.nalu.client.application.IsContext` -> `com.github.nalukit.nalu.client.context.IsContex`

2. **RouterStateEvent**: `com.github.nalukit.nalu.client.router.event` -> `com.github.nalukit.nalu.client.event`

## New required method
* **Shell:** you need to override the `detachShell`-method with the code needed to remove the shell.

## Removed features
The following features from version 1.x have been removed:

1. **errorRoute**: See error handling doc for more information on how to migrate ([Error Handling](https://github.com/NaluKit/nalu/wiki/12.-Error-Handling))
2. **plugins**: See Multi Module Project ([Multi Module Feature](https://github.com/NaluKit/nalu/wiki/15.-Multi-Module-Project))

repository: NaluKit/nalu

branch_name: refs/heads/dev

gha_language: Java

repo_name: NaluKit/nalu


text:
# Ethereum Swarm Beekeeper

**Ethereum Swarm Beekeeper** is tool used for testing of [Ethereum Swarm Bee](https://github.com/ethersphere/bee).

# Installing

```bash
make binary
cp dist/beekeeper /usr/local/bin/beekeeper
```

# Usage

**beekeeper** has following commands:

|command|description|
|-------|-----------|
| check | Run tests on Bee node(s) |
| help | Help about any command |
| version | Print version number |

## check

Command **check** runs test(s) on Bee node(s).
 Each test is implemented as a subcommand.

Available subcommands:

|subcommand|description|
|----------|-----------|
| peercount | Check node's peer count for all nodes in the cluster |

### peercount

**peercount** checks node's peer count for all nodes in the cluster.
It retrieves list of peers from node's Debug API (/peers endpoint),
and compares number of node's peers against expected peer count.
Expected peer count equals: *node-count + bootnode-count - 1*.

Example:
```bash
beekeeper --bootnode-count 1 --node-count 1 --namespace bee
```
 or, shorthand:
 ```bash
beekeeper -b 1 -c 1 -n bee
```

repository: svetomir/beekeeper

repo_name: svetomir/beekeeper


text:
**Supermarket Flowers双手简谱** 和五线谱完全对应。

_Supermarket Flowers_ 是由英国新生代歌手Ed Sheeran演唱的一首歌曲，是Ed Sheeran2017年新专辑《÷
(Deluxe)》中的第12首。专辑于2017年3月3日通过Asylum Records和大西洋唱片公司发行。该张专辑名称读作“divide”。

Ed Sheeran被誉为英国创作才子。作为一个优秀的唱作人，Ed Sheeran不只为自己创作，还为许多歌手写过歌。

歌词下方是 _Supermarket Flowers钢琴谱_ ，大家可以免费下载学习。

### Supermarket Flowers歌词：

I took the supermarket flowers from the windowsill
Threw the day-old tea from the cup
Packed up the photo album Matthew had made
Memories of a life that's been loved
Took the "Get Well Soon" cards and stuffed animals
Poured the old ginger beer down the sink
Dad always told me, "Don't you cry when you're down."
But, mum, there's a tear every time that I blink
Oh, I'm in pieces. It's tearing me up but I know
A heart that's broke is a heart that's been loved
So, I'll sing Hallelujah,
You were an angel in the shape of my mum
When I fell down you'd be there holding me up
Spread your wings as you go
And when God takes you back
He'll say, "Hallelujah, you're home."


text:
# Statly

TODO: Write a gem description

## Installation

Add this line to your application's Gemfile:

    gem 'statly'

And then execute:

    $ bundle

Or install it yourself as:

    $ gem install statly

## Usage

There are (will be) an array of different similarity classes you can use:

### Jaccard Index

```ruby
ids_a = [1, 2, 3, 4]
ids_b = [1, 2, 4]
Statly::Similarity::Jaccard.coefficient(ids_a, ids_b) # => 0.75
```

## Contributing

1. Fork it
2. Create your feature branch (`git checkout -b my-new-feature`)
3. Commit your changes (`git commit -am 'Add some feature'`)
4. Push to the branch (`git push origin my-new-feature`)
5. Create new Pull Request

repository: dewski/statly

repo_name: dewski/statly


text:
# VLMOcu : Very Large Matrix Operations Library with CUDA

## 0. Version
version 1.0

## 1. Support operations list
1. Element-wise matrix operations
      * Element-wise matrix addition
      * Element-wise matrix subtraction
      * Element-wise matrix multiplication
      * Element-wise matrix division
2. Matrix multiplication
3. Matrix transposition

## 2. Element-wise matrix operations
uni
## 3. Matrix multiplication
## 4. Matrix transposition

gha_language: Cuda


text:
### vCD Metadata Scripts

This folder provides 2 scripts to allow the backing up of vApp Metadata which runs on a Management VM as part of pre-script activity within Rubrik.

## Config

* Drop both scripts into a folder in a known location of the management VM
* Create a new folder called `Exports` in the same directory as both scripts
* Browse to the VM in Rubrik
* Click on the Elipses in the top right and select `Configure Pre/Post Scripts`
* In the `Pre-Backup Script`, add the path to the `backup_vcd_vapp_metadata.ps1` script e.g. `C:\Scripts\backup_vcd_vapp_metadata.ps1`
* Set a timeout window, suggested 30-180 seconds depending on amount of metadata
* Apply the Pre-Script

Once this VM is backed up, we will now store the current Metadata across the vCD Cell.

## Restore

Since we have backed up the metadata, we can restore the CSV from anypoint by either file level recovery for the date, or browsing the `Exports` folder.
* Run the `restore_vcd_vapp_metadata.ps1` script which will prompt a file open wizard; browse and select the CSV file to restore from
* It will now prompt for which metadata you want to restore, following the commands on-screen.

repository: KennyNeal/rubrik-scripts-for-powershell

repo_name: KennyNeal/rubrik-scripts-for-powershell


text:
---
title: Floating-Point Precision
tags:
  - floating-point
  - decimal
  - rounding
order: 39
date: '2019-10-07'
answers:
  - 'true'
  - 'false // correct'
---

Consider the following scenario. What gets logged?

```javascript
const a = 0.1;
const b = 0.2;
const c = 0.3;

console.log(a + b === c);
```

<!-- explanation -->

In computing, decimal numbers as we know them are most commonly represented using floating-point arithmetic. Floating-point numbers only approximate real numbers and cannot accurately represent numbers like `0.1`, `0.2` or `0.3` at all. In the case of `0.1 + 0.2`, this results in small rounding errors before we even get to the addition part. The final sum then equals to `0.30000000000000004`.

Similarly, the decimal system cannot represent `1/3` (one third) accurately. And if we took a rounded value like `0.333`, you wouldn't expect `0.333 + 0.333 + 0.333` to add up to `1`, either.

gha_language: JavaScript


text:
# Uzay

Uzay is a neatly written API using Express and Mongo (^3.2) to serve as the basic backend for my personal blog.

### Features

Here's a list of some features included in the bundle :

* __Highly modular__ : All code is written neatly as separate independent modules, providing high flexibility for debugging and testing.
* __Highly anonymous__ : No Passport or any similar policies used to authenticate users. Usage of alias is encouraged, however, a working email is also required once to avoid bullshit.
* __Highly secure__ : A single passphrase heavily encrypted assures that Bloggers identity remains safe and with the Blogger only.
* __User Friendly Install__ : Post Install Script helps you set up the passphrase in the same go, for once and for all. No manually editing of the files required. Deploy ahoy!
* __Minimalistic Admin Page__ : The Admin Page (/) provides a simple yet attractive UI where you can write your blog post.
* __Accept Public Upvotes__ : Because at times, we just need to say a simple "Thank You!". Why log into whatever ? Just say! This does exactly that.

And many more !

### Installation

You can use this for your own purposes as well. Fork this repository right away!

Also, since Uzay uses Mongo DB, you must have it [installed](https://docs.mongodb.com/manual/tutorial/install-mongodb-on-linux/) on your sytsem. Once installed, run the `mongod` service in your shell, and then follow the steps below.

```
git clone https://github.com/YOUR_USERNAME/uzay.git
cd uzay
npm install
```

That's it really ! It'll install, configure, and deploy too in the same go !

### License

MIT


text:
---
period: "Flow-3"
title: "Android/IOS-Development with React native"
date: "22-04-2020"
pageintro: |
  React Native Continued
  Native Device Features with React Native and Expo continued
---

### Before this lesson you should:

Complete all exercises related to this period

## Lecture(s)

<!--BEGIN lectures ##-->

[Complete the Team Finder App](https://docs.google.com/document/d/15ps9pyGyjlW3SZeGCEBKvxRqBJkJSUmDwQJqfAanog4/edit?usp=sharing)

<!--END lectures ##-->

## Exercises

<!--BEGIN exercises ##-->

<!--END exercises ##-->

## Videos recorded in the class

TBD

<!--BEGIN slides ##-->

<!--END slides ##-->


text:
# Graph Visualization with D3.js

The aim of this software is to provide a useful tool for visualising and analysing the result of different simulations based on graph. Once you run the simulation, you will be able to interact with the simulation in real time.

For this purpose, a model which tries to simulate the spread of information to comprehend the radicalism spread in a society is included. Whith all this, the main project goals could be divided in five as it is shown in the following.

* Simulate the spread of information through a network applied to radicalism.
* Visualize the results of the simulation.
* Interact with the simulation in real time.
* Extract data from the results.
* Show data in a right way for its research.

## Deploying the server

For deploying the application, you will only need to run the following command.

`python3 run.py [--name NAME] [--dump] [--port PORT] [--verbose]`

Where the options are detailed in the following table.

| Option | Description |
| --- | --- |
| `--name NAME` | The name of the simulation. It will appear on the app. |
| `--dump` | For dumping the results in server side. |
| `--port PORT` | The port where the server will listen. |
| `--verbose` | Verbose mode. |

> You can dump the results of the simulation in server side. Anyway, you will be able to download them in GEXF or JSON Graph format directly from the browser.

## Visualization Params

The configuration of the simulation is based on the simulator configuration. In this case, it follows the [SOIL](https://github.com/gsi-upm/soil) configuration syntax and for visualising the results in a more comfortable way, more params can be added in `visualization_params` dictionary.

* For setting a background image, the tag needed is `background image`. You can also add a `background_opacity` and `background_filter_color` if the image is so clear than you can difficult view the nodes.
* For setting colors to the nodes, you can do it based on their properties values. Using the `color` tag, you will need to indicate the attribute key and value, and then the color you want to apply.
* The shapes applied to a group of nodes are always the same. This means than it won't change dynamically, so you will have to indicate the property with the `shape_property` tag and add a dictionary called `shapes` in which for each value, you indicate the shape.
  All shapes have to had been downloaded before in SVG format and added to the server.

An example of this configuration applied to the TerroristNetworkModel is presented.

```yaml
visualization_params:
  # Icons downloaded from https://www.iconfinder.com/
  shape_property: agent
  shapes:
    TrainingAreaModel: target
    HavenModel: home
    TerroristNetworkModel: person
  colors:
    - attr_id: 0
      color: '#40de40'
    - attr_id: 1
      color: red
    - attr_id: 2
      color: '#c16a6a'
  background_image: 'map_4800x2860.jpg'
  background_opacity: '0.9'
  background_filter_color: 'blue'
```

repository: gsi-upm/soil

repo_name: gsi-upm/soil


text:
---
title: 3.1 - Authentication options
navPrev: 3-basic-security.html
navPrevText: 3 - Basic Security
navUp: 3-basic-security.html
navUpText: 3 - Basic Security
navNext: 3.2-basic-authorization.html
navNextText: 3.2 - Basic authorization
---

# 3.1 - Authentication options

This section describes the authentication options of ApacheDS 2.0. Anonymous and simple binds are supported, as well as SASL mechanisms. Configuring and using the first two of them is described below with the help of examples.

{{% toc %}}

## What is authentication?

*Authentication* is the process of determining whether someone (or something) in fact is what he/she/it asserts to be.

Within ApacheDS you will likely want to authenticate clients in order to check whether they are allowed to read, add or manipulate certain data stored within the directory. The latter, i.e. whether an authenticated client is permitted to do something, is deduced during *authorization*.

Quite often, the process of authentication is delegated to a directory service by other software components. Because in doing so, authentication data (e.g. username, password) and authorization data (e.g. group relationships) are stored and managed centrally in the directory, and all connected software solutions benefit from it. The integration sections of this guide provide examples for Apache Tomcat, Apache HTTP servers, and others.

ApacheDS 2.0 supports simple authentication and anonymous binds while storing passwords within _userPassword_ attributes in user entries. Passwords can be stored in clear text or one-way encrypted with a hash algorithm like MD5 or SHA1. Since version 1.5.1, SASL mechanism are supported as well. We start with anonymous binds.

## Simple binds

Authentication via simple bind is widely used. The method is supported by ApacheDS 2.0 for all person entries stored within any partition, if they contain a password attribute. How does it work? An LDAP client provides the DN of a user entry and a password to the server, the parameters of the bind operation. ApacheDS checks whether the given password is the same as the one stored in the _userpassword_ attribute of the given entry. If not, the bind operation fails (LDAP error code 49, LDAP_INVALID_CREDENTIALS), and the user is not authenticated.

### Using command line tools

Assume this entry from the Seven Seas partition is stored within the directory (only a fragment with the relevant attributes is shown).

	dn: cn=Horatio Hornblower,ou=people,o=sevenSeas
	objectclass: person
	objectclass: organizationalPerson
	cn: Horatio Hornblower
	sn: Hornblower
	userpassword: pass
	...

In the following search command, a user tries to bind with the given DN (option -D) but a wrong password (option -w). The bind fails and the command terminates without performing the search.

	$ ldapsearch -h zanzibar -p 10389 -D "cn=Horatio Hornblower,ou=people,o=sevenSeas" \\
		-w wrong -b "ou=people,o=sevenSeas" -s base "(objectclass=*)"
	ldap_simple_bind: Invalid credentials
	ldap_simple_bind: additional info: Bind failed: null

If the user provides the correct password during the call of the ldapsearch command, the bind operation succeeds and the seach operation is performed afterwards.

	$ ldapsearch -h zanzibar -p 10389 -D "cn=Horatio Hornblower,ou=people,o=sevenSeas" \\
    	-w pass -b "ou=people,o=sevenSeas" -s base "(objectclass=*)"
	version: 1
	dn: ou=people,o=sevenSeas
	ou: people
	description: Contains entries which describe persons (seamen)
	objectclass: organizationalUnit
	objectclass: top

### Binds from Java components using JNDI

Using JNDI, authentication via simple binds is accomplished by appropriate configuration. One option is to provide the parameters in a Hashtable object like this

```java
import java.util.Hashtable;

import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingEnumeration;
import javax.naming.NamingException;

public class SimpleBindDemo {

    public static void main(String[] args) throws NamingException {

        if (args.length < 2) {
            System.err.println("Usage: java SimpleBindDemo <userDN> <password>");
            System.exit(1);
        }

        Hashtable env = new Hashtable();
        env.put(Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.ldap.LdapCtxFactory");
        env.put(Context.PROVIDER_URL, "ldap://zanzibar:10389/o=sevenSeas");

        env.put(Context.SECURITY_AUTHENTICATION, "simple");
        env.put(Context.SECURITY_PRINCIPAL, args[0]);
        env.put(Context.SECURITY_CREDENTIALS, args[1]);

        try {
            Context ctx = new InitialContext(env);
            NamingEnumeration enm = ctx.list("");

            while (enm.hasMore()) {
                System.out.println(enm.next());
            }

            enm.close();
            ctx.close();
        } catch (NamingException e) {
            System.out.println(e.getMessage());
        }
    }
}
```

If the DN of a user entry and the fitting password are provided as command line arguments, the program binds successfully and performs a search:

```bash
$ java SimpleBindDemo "cn=Horatio Hornblower,ou=people,o=sevenSeas" pass
ou=people: javax.naming.directory.DirContext
ou=groups: javax.naming.directory.DirContext
```

On the other hand, providing an incorrect password results in a failed bind operation. JNDI maps it to a _NamingException_:

```bash
$ java SimpleBindDemo "cn=Horatio Hornblower,ou=people,o=sevenSeas" quatsch
[LDAP: error code 49 - Bind failed: null]
```

In real life, you obviously want to separate most of the configuration data from the source code, for instance with the help of the _jndi.properties_ file.

## Passwords stored one-way encrypted

If passwords are stored in the directory in clear like above, the administrator (_uid=admin,ou=system_) is able to read them. This holds true even if authorization is enabled. The passwords would also be visible in exported LDIF files. This is often unacceptable.

Not only the administrator will be able to read your password, or be visible in LDIF files, but if one does not use SSL, the password is transmitted in clear text above the wire...

### Passwords not stored in clear text

ApacheDS does also support simple binds, if user passwords are stored one-way encrypted. An LDAP client, which creates user entries, applies a hash-function (SHA for instance) to the user passwords beforehand, and stores the users with these fingerprints as _userpassword_ values (instead of the clear text values), for instance:

	dn: cn=Horatio Hornblower,ou=people,o=sevenSeas
	objectclass: person
	objectclass: organizationalPerson
	cn: Horatio Hornblower
	sn: Hornblower
	userpassword: {SHA}nU4eI71bcnBGqeO0t9tXvY1u5oQ=
	...

The value "{SHA}nU4eI71bcnBGqeO0t9tXvY1u5oQ=" means that _SHA_ (Secure Hash Algorithm) was applied to the password, and "nU4eI71bcnBGqeO0t9tXvY1u5oQ=" was the result (Base-64 encoded). Please note that it is not possible to calculate the source ("pass" in our case) back from the result. This is why it is called one-way encrypted -- it is rather difficult to decrypt it. One may guess many times, calculate the hash values (the algorithms are public) and compare the result. But this would take a long time, especially if you choose a more complex password than we did ("pass").

### But how to obtain the hash value for a password?

With some lines of code, it is quite easy to accomplish this task programatically in Java:

```java
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import sun.misc.BASE64Encoder;

public class DigestDemo {
    public static void main(String[] args) throws NoSuchAlgorithmException {
        String password = "pass";
        String algorithm = "SHA";

        // Calculate hash value
        MessageDigest md = MessageDigest.getInstance(algorithm);
        md.update(password.getBytes());
        byte[] bytes = md.digest();

        // Print out value in Base64 encoding
        BASE64Encoder base64encoder = new BASE64Encoder();
        String hash = base64encoder.encode(bytes);
        System.out.println('{'+algorithm+'}'+hash);
    }
}
```

The output is "{SHA}nU4eI71bcnBGqeO0t9tXvY1u5oQ=".

Another option is to use command line tools to calculate the hash value; the [OpenSSL](http://www.openssl.org) project provides such stuff. Furthermore many UI LDAP tools allow you to store passwords automatically encrypted with the hash algorithm of your choice. See below [Apache Directory Studio](https://directory.apache.org/studio/) as an example. The dialog automatically shows up if a _userPassword_ attribute is to be manipulated (added, changed).

![Password Edit](images/password-edit-ls.png)

### From an LDAP client point of view

From an LDAP client point of view, the behavior during authentication is the same as with passwords stored in clear. During a simple bind, a client sends DN and password (unencrypted, i.e. no hash algorithm applied) to the server. If ApacheDS detects, that the user password for the given DN is stored in the directory with a hash function applied, it calculates the hash value of the given password with the appropriate algorithm (this is why the algorithm is stored together with the hashed password). Afterwards it compares the result with the stored attribute value. In case of a match, the bind operation ends successfully:

	$ ldapsearch -h zanzibar -p 10389 -D "cn=Horatio Hornblower,ou=people,o=sevenSeas" \\
		-w pass -b "ou=people,o=sevenSeas" -s base "(objectclass=*)"
	version: 1
	dn: ou=people,o=sevenSeas
	ou: people
	description: Contains entries which describe persons (seamen)
	objectclass: organizationalUnit
	objectclass: top

Providing the hashed value of the _userPassword_ attribute instead of the original value will be rejected by ApacheDS:

	$ ldapsearch -h zanzibar -p 10389 -D "cn=Horatio Hornblower,ou=people,o=sevenSeas" \\
		-w "{SHA}nU4eI71bcnBGqeO0t9tXvY1u5oQ=" -b "ou=people,o=sevenSeas" -s base "(objectclass=*)"
	ldap_simple_bind: Invalid credentials
	ldap_simple_bind: additional info: Bind failed: null

This is intended. If someone was able to catch this value (from an LDIF export for instance), s/he must still provide the password itself in order to get authenticated.

Be Warned: Limited security added

Please note that storing user passwords one-way encrypted only adds limited security. During the bind operation, the credentials are still transmitted unencrypted, if no SSL/TLS communication is used (thus you should definitely consider to do so).

Furthermore, if someone gets an LDIF file with userpassword values digested with SHA etc., s/he may be able to determine some of the passwords with brute force. Calculation of hash functions can be done very fast, and the attacker can attempt millions of values with ease, without you getting notice of it. Therefore protect your data, even if one-way encryption is applied to the passwords!

## Anonymous binds

In some occasions it is appropriate to allow LDAP clients to permit operations without authentication. If data managed by the directory service is well known by all clients, it is not uncommon to allow search operations (not manipulation) within this data to all clients -- without providing credentials. An example for this are enterprise wide telephone books, if clients access the directory service from the intranet.

### Enable/disable anonymous binds

Anonymous access is enabled by default. Changing this is one of the basic configuration tasks (see [1.4.5 - Enable and disable anonymous access](1.4.5-anonymous-access.html)).

### Example: Server behavior with anonymous binds disabled

Assume anonymous binds are disabled and our sample partition _Seven Seas_ present in the server. Here is an example with a search operation performed by a command line tool as a client. It tries to connect anonymously (no DN and password given, i.e. options -D and -w missing) to the server. Afterwards the entry _ou=people,o=sevenSeas_ should be displayed.

See the command and the resulting error message provided by the server below

	$ ldapsearch -h zanzibar -p 10389 -b "ou=people,o=sevenSeas" -s one "(objectclass=*)"
	ldap_search: Insufficient access
	ldap_search: additional info: failed on search operation: Anonymous binds have been disabled!

### Example: Server behavior with anonymous binds enabled

Now the same command performed against ApacheDS 1.5 with anonymous access enabled as described above. The behavior is different -- the entry is visible.

	$ ldapsearch -h zanzibar -p 10389 -b "ou=people,o=sevenSeas" -s base "(objectclass=*)"
	version: 1
	dn: ou=people,o=sevenSeas
	ou: people
	description: Contains entries which describe persons (seamen)
	objectclass: organizationalUnit
	objectclass: top

### Other clients

The examples above have used a command line tool. Of course graphical tools and programmatical access (JNDI etc.) allow anonymous binds as well. Below is a screen shot from the configuration dialog of [Apache Directory Studio](https://directory.apache.org/studio/) as an example. During configuration of the connection data ("New LDAP Connection", for instance), the option _Anonymous Authentication_ leads to anonymous binds. Other UI tools offer this feature as well.

![Authentication options](images/authentication-options-ls.png)

**Use this feature wisely**

With anonymous access enabled it is not only possible to search the directory without providing username and password. With autorization disabled, anonymous users may also be able to modify data. It is therefore highly recommended to enable and configure the authorization subsystem as well. Learn more about authorization in the [3.2. Basic authorization] section.

## How to authenticate a user by uid and password?

If you want to use simple binds with user DN and password within a Java component, in order to authenticate users programatically, in practice one problem arises: Most users do not know their DN. Therefore they will not be able to enter it. And even if they know it, it would be frequently very laborious due to the length of the DN. It would be easier for a user if s/he only has to probvide a short, unique _ID_ and the password, like in this web form

![Confluence Logon](images/confluence-logon.png)

Usually the ID is an attribute within the user's entry. In our sample data (Seven Seas), each user entry contains the _uid_ attribute, for instance uid=hhornblo for Captain Hornblower:

	dn: cn=Horatio Hornblower,ou=people,o=sevenSeas
	objectclass: person
	objectclass: organizationalPerson
	objectclass: inetOrgPerson
	objectclass: top
	cn: Horatio Hornblower
	description: Capt. Horatio Hornblower, R.N
	givenname: Horatio
	sn: Hornblower
	uid: hhornblo
	mail<EMAIL_ADDRESS>	userpassword: {SHA}nU4eI71bcnBGqeO0t9tXvY1u5oQ=

But how to authenticate a user who provides "hhornblo"/"pass" instead of "cn=Horatio Hornblower,ou=people,o=sevenSeas"/"pass" with the help of ApacheDS?

### An algorithm

In order to accomplish this task programmatically, one option is to perform the following steps

#### Arguments
* _uid_ of a user (e.g. "hhornblo")
* _password_ proclaimed to be correct for the user

#### Steps

* Bind to ApacheDS anonymously, or with the DN of a technical user. In both cases it must be possible to search the directory afterwards (authorization has to be configured that way)
* Perform a search operation with an appropriate filter to find the user entry for the given ID, in our case "(&(objectClass=inetorgperson)(uid=hhornblo))"
	* If the search result is empty, the user does not exist -- terminate
	* If the search result contains more than one entry, the given ID is not unique, this is likely a data error within your directory
* Bind to ApacheDS with the DN of the entry found in the previous search, and the _password_ provided as argument
	* If the bind operation fails, the password is wrong, and the result is _false_ (not authenticated)
	* If the bind is successful, authenticate the user

### Sample code with JNDI

The algorithm described above is implemented by many software solutions which are able to integrate LDAP directories. You will learn more about some of them and their configuration options within a later section of this guide.

For illustration purposes, here is a simple Java program which performs the steps with the help of JNDI. It uses anonymous bind for the first step, hence it must be enabled (replace with a technical user, if it better meets your requirements).

```java
import java.util.Hashtable;
import javax.naming.Context;
import javax.naming.NamingEnumeration;
import javax.naming.NamingException;
import javax.naming.directory.DirContext;
import javax.naming.directory.InitialDirContext;
import javax.naming.directory.SearchControls;
import javax.naming.directory.SearchResult;

public class AdvancedBindDemo {

    public static void main(String[] args) throws NamingException {

        if (args.length < 2) {
            System.err.println("Usage: java AdvancedBindDemo <uid> <password>");
            System.exit(1);
        }

        Hashtable env = new Hashtable();
        env.put(Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.ldap.LdapCtxFactory");
        env.put(Context.PROVIDER_URL, "ldap://zanzibar:10389/");
        env.put(Context.SECURITY_AUTHENTICATION, "simple");

        String uid = args[0];
        String password = args[1];

        DirContext ctx = null;

        try {
            // Step 1: Bind anonymously
            ctx = new InitialDirContext(env);

            // Step 2: Search the directory
            String base = "o=sevenSeas";
            String filter = "(&(objectClass=inetOrgPerson)(uid={0}))";
            SearchControls ctls = new SearchControls();
            ctls.setSearchScope(SearchControls.SUBTREE_SCOPE);
            ctls.setReturningAttributes(new String[0]);
            ctls.setReturningObjFlag(true);
            NamingEnumeration enm = ctx.search(base, filter, new String[] { uid }, ctls);

            String dn = null;

            if (enm.hasMore()) {
                SearchResult result = (SearchResult) enm.next();
                dn = result.getNameInNamespace();

                System.out.println("dn: "+dn);
            }

            if (dn == null || enm.hasMore()) {
                // uid not found or not unique
                throw new NamingException("Authentication failed");
            }

            // Step 3: Bind with found DN and given password
            ctx.addToEnvironment(Context.SECURITY_PRINCIPAL, dn);
            ctx.addToEnvironment(Context.SECURITY_CREDENTIALS, password);
            // Perform a lookup in order to force a bind operation with JNDI
            ctx.lookup(dn);
            System.out.println("Authentication successful");

            enm.close();
        } catch (NamingException e) {
            System.out.println(e.getMessage());
        } finally {
            ctx.close();
        }
    }
}
```

Some example calls:

	$ java AdvancedBindDemo unknown sailor
	Authentication failed

	$ java AdvancedBindDemo hornblo pass
	dn: cn=Horatio Hornblower,ou=people,o=sevenSeas
	Authentication successful

	$ java AdvancedBindDemo hornblo quatsch
	dn: cn=Horatio Hornblower,ou=people,o=sevenSeas
	[LDAP: error code 49 - Bind failed: null]

The examples consist of an unknown user (an _inetOrgPerson_ entry with uid=unknown does not exist), a successful authenttication, and an attempt with an existing uid but a wrong password.

## Resources

* [RFC 2829](http://www.faqs.org/rfcs/rfc2829.html) Authentication Methods for LDAP
* [The Secure Hash Algorithm Directory](http://www.secure-hash-algorithm-md5-sha-1.co.uk/) MD5, SHA-1 and HMAC Resources

repository: rlenferink/directory-site

repo_name: rlenferink/directory-site


text:
---
title: Opcode Forage
searchTitle: Lua Opcode Forage
weight: 1
hidden: true
menuTitle: Opcode Forage
---
## Forage

This represents a Forage opcode
```lua
Opcode.Forage
```

repository: xackery/eqquestapi

repo_name: xackery/eqquestapi


text:
---
layout: post
title: Week 4!
---

## What did you do this past week?
I spent most of my week applying to different places and preparing for recruiting this week. I also spent some time looking at the project, but have not gotten a good start on it just yet. I ended up spending most of my time finishing up projects due for other classes, since they were due this week.

## What's in your way?
Being efficient with time and prioritizing. I am trying to split my time up between work, school and friends and it can get hard during your senior year.

## What will you do next week?
This week I plan on working on the 3 projects I have due on thursday. In terms of this class, my partner and I plan on meeting up 3 days this week so that we can finish it off before the due date, so hopefully that'll be enough to get it done. The rest of the time I'll be spending on catching up and finishing up the solo projects in other classes. After thursday, I'll be taking a long break before preparing for the tests next week.

## Describe your experience in the class so far.
The class is starting to get pretty interesting with things I didn't know about python. I consider myself pretty good with python, but I am learning a lot of the abstracted theoretical stuff in Python that I didn't know about till now. The projects in the class are also pretty interesting, albeit time consuming.

## Tip-of the week:
Learn from other people's mistakes on piazza before spending an hour on it of your own time.


text:
# ActivityStatisticsResultDto

## Properties

Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
**id** | **String** | The id of the activity the results are aggregated for. |  [optional]
**instances** | **Integer** | The total number of running process instances of this activity. |  [optional]
**failedJobs** | **Integer** | The total number of failed jobs for the running instances. **Note**: Will be `0` (not `null`), if failed jobs were excluded. |  [optional]
**incidents** | [**List<IncidentStatisticsResultDto>**](IncidentStatisticsResultDto.md) | Each item in the resulting array is an object which contains `incidentType` and `incidentCount`. **Note**: Will be an empty array, if `incidents` or `incidentsForType` were excluded. Furthermore, the array will be also empty if no incidents were found. |  [optional]


text:
#  rust - schedule

[Test-Online](https://play.rust-lang.org/)

# docs:
search doc for std lib.
- https://doc.rust-lang.org/std/process/struct.Child.html

qa.
- https://stackoverflow.com/questions/40392906/no-method-named-flush-found-for-type-stdiostdout-in-the-current-scope

search for google.
- stdout().flush()  method not found on `stdio::Stdout`

#

#
#
#

# refs
- https://zhuanlan.zhihu.com/p/66395385

gha_language: Rust


text:
## The 12 Silliest/Most Dangerous Things People Say About Stocks

- If it's gone down this much already, it can't go much lower
- You can always tell when a stock's hit bottom (catch a falling knife)
- If it's gone this high already, how can it possibly go higher
- It's only $3/share - what can I lose? anything can go to $0
- Eventually they always come back
- It's always darkest before dawn
- When it rebounds to $X, I'll sell
- Conservative stocks don't fluctuate much
- It's taking too long for anything to happen
    - most money made in 3rd or 4th year of owning a stock
- I didn't buy it, look at all the money i lost
- I missed that one, I'll catch the next one
    - often better to buy original company at higher price than "next one" at a bargain
- The stocks up I must be right/ the stocks down I must be right
    - Don't confuse prices w/ prospects


text:
# Simple Gallery

A gallery for viewing photos and videos.

A highly customizable gallery capable of displaying many different image and video types including SVGs, RAWs, panoramic photos and videos.

It is open source, contains no ads or unnecessary permissions.

Let's list some of its features worth mentioning:
1. Search
2. Slideshow
3. Notch support
4. Pinning folders to the top
5. Filtering media files by type
6. Recycle bin for easy file recovery
7. Fullscreen view orientation locking
8. Marking favorite files for easy access
9. Quick fullscreen media closing with down gesture
10. An editor for modifying images and applying filters
11. Password protection for protecting hidden items or the whole app
12. Changing the thumbnail column count with gestures or menu buttons
13. Customizable bottom actions at the fullscreen view for quick access
14. Showing extended details over fullscreen media with desired file properties
15. Several different ways of sorting or grouping items, both ascending and descending
16. Hiding folders (affects other apps too), excluding folders (affects only Simple Gallery)

The fingerprint permission is needed for locking either hidden item visibility, the whole app, or protecting files from being deleted.

This app is just one piece of a bigger series of apps. You can find the rest of them at https://www.simplemobiletools.com

License
-------
    Copyright 2016-present SimpleMobileTools

    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
    You may obtain a copy of the License at

       https://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License.

repository: salehaliii/Simple-Gallery

gha_language: Kotlin

repo_name: salehaliii/Simple-Gallery


text:
---
id: 6796
title: Corinne Leigh
date: 2021-04-07T02:36:50+00:00
author: victor
layout: post
guid: https://ukdataservers.com/corinne-leigh/
permalink: /04/07/corinne-leigh/
---

* some text
{: toc}

## Who is Corinne Leigh

YouTube personality best known for being part of the ThreadBanger YouTube channel that has racked up more than 3.8 million subscribers. She has a personal, self-titled YouTube channel that has more than one million subscribers. The ThreadBanger channel specializes in fashion DIY videos.

## Prior to Popularity

She got started on YouTube and social media in 2007. The first video on the ThreadBanger channel was “LA Graffiti Artist Interview / ‘Spray Can Michaelangelo.'”

## Random data

Some of the more popular DIY videos on the ThreadBanger channel include “DIY Galaxy Slime – Man Vs. Pin – Pinterest Test No. 40,” “How to make Cleaning Slime – Man Vs. Pin No. 16,” and “DIY Water Blob – Man Vs. Pin No. 25.”

## Family & Everyday Life of Corinne Leigh

She grew up on an Air Force base in New Jersey.  She and her husband Rob Czar, whom she runs ThreadBanger with, began dating around 2004.

## People Related With Corinne Leigh

She and Rob collaborated with Kevin Lieber on a video called “DIY Edible Water Bottles – Man Vs. Science No. 1 (w/Vsauce2).”


text:
# Search Service

The Search Service provides a direct querying controller which interfaces with our ELK stack.

## Install

```
Run `pnpm install` from the root.
```

## Usage

```
# normal mode
pnpm start
```

By default the server is running on http://localhost:4445/.

### Examples

- `GET /query?text=Telescope&filter=post&page=0` - Returns the search results of posts containing the keyword "Telescope"
- `GET /query?text=SenecaCDOT?filter=author&page=0` - Returns the search results of authors which relate to the keyword "SenecaCDOT"
- `GET /healthcheck` - returns `{ "status": "ok" }` if everything is running properly

## Docker / Docker-Compose

### Docker

- To build and tag: `docker build . -t telescope_search_svc:latest`
- To run locally: `docker run -p 4445:4445 telescope_search_svc:latest`

### Docker-Compose

_Commands to be run from the `~/src/api` directory_

- To build and run: `docker-compose -f docker-compose-api-production.yml up --build search`

repository: LuigiZaccagnini/telescope

repo_name: LuigiZaccagnini/telescope


text:
Examples PLab App and Arduino Integration
===
This folder contains examples that document the use of the PLab app for
controlling Bluetooth and Bluetooth LE units from your phone.

The app is a framework for constructing user interfaces fast through
[Processing][1]. The framework uses [Processing.js][2] to interpret Processing
code. The framework was originally created for the Programming Lab course at
[NTNU][3]. For more information, see [the project repo][4].

For examples that contain Arduino code it is assumed that the neccessary
libraries are installed. All (and more) are gathered in the course
[library folder][5]. In particular is the PLabBT library used.

The app has allways the file `InterfacesInc.pde` already loaded, so there is
NO NEED TO INCLUDE THIS FILE when running in the app.

Example Descriptions
---
### InterfacesInc
Serves as documentation for app functionality. Is not meant to run.

### Empty
Minimal code neccesary for successful setup of project.

To run in app, copy the following files to the Internet:
* `Empty.pde`

### MinimalBTExample
A small example that show how to use the Bluetooth capabilities through
Processing code. The Processing code can be run on a computer. Changes to
`AppReplacer.pde` neccessary to run on a particular computer should be limited
to changing the content of `portName` within `setupSerial()` to the correct
port on the intended computer.

To run in app, copy the following files to the Internet:
* `MinimalBTExample.pde`
Note: The line holding `setupSerial();` must be commented out to run properly
in app.

### Acceleration
An example that show how to use the app to report device acceleration.

To run in app, copy the following files to the Internet:
* `Acceleration.pde`

### AnalogValueExample
An example that reads an analog value from an Arduino, and reports it back to
the app.

To run in app, copy the following files to the Internet:
* `AnalogValueExamplei.pde`

### Orientation
An example that show how to use the app to report device orientation.

To run in app, copy the following files to the Internet:
* `Orientation.pde`

### RobotDriveOnOff
An example that shows how to control a Pololu Zumo robot remotely with the app.

To run in app, copy the following files to the Internet:
* `RobotDriveOnOff.pde`

### SimpleButtonExample
An example that shows how a push button may be implemented in the app, and
state reported to an Arduino through a Bluetooth device.

Two versions of Arduino code:
* One using `PLabBTSerial.h` located in our `PLabBT` library.
* One using only `SoftwareSerial.h` and a simple state machine to parse
incomming text

To run in app, copy the following files to the Internet:
* `SimpleButtonExample.pde`

### TwoButtonsExample
An example that show how two toggle buttons can be created, and their states
reported an Arduino through a Bluetooth device.

To run in app, copy the following files to the Internet:
* `TwoButtonsExample.pde`

### VibrateOnClick
An example to show how to control the phone vibration through the app.

To run in app, copy the following files to the Internet:
* `VibrateOnClick.pde`

[1]: https://processing.org			"Processing"
[2]: http://processingjs.org			"Processing.js"
[3]: http://www.ntnu.edu	"Norwegian University of Science and Technology - NTNU"
[4]: https://github.com/IDI-PLAB/plab-mobile-framework	"PLab Mobile Framework"
[5]: https://github.com/IDI-PLAB/plab-library	"PLab Libraries"

repository: rammt/Examples

repo_name: rammt/Examples


text:
# micronaut-transaction-question

Create application:
```
mn create-app issue --lang groovy --features application,http-client,http-server,jdbc-hikari,liquibase,logback,spock
```

## Question:

Based on the test `issue.web.rest.UserResourceSpec`.

The current error that is returned is that there is no user in the database
```
11:46:06.764 [pool-1-thread-4] ERROR i.m.h.s.netty.RoutingInBoundHandler - Unexpected error occurred: Query produced no result
io.micronaut.data.exceptions.EmptyResultException: Query produced no result
	at io.micronaut.data.runtime.intercept.DefaultFindOneInterceptor.intercept(DefaultFindOneInterceptor.java:59)
	at io.micronaut.data.intercept.DataIntroductionAdvice.intercept(DataIntroductionAdvice.java:79)
```

This behaviour comes from the user that has been created in the `given` part is not commited
```
def user = createUser(username)
```

To be sure that the issue comes from the commit :

* Go to `io.micronaut.data.jdbc.operations.DefaultJdbcRepositoryOperations` in the method `persistOne`.

* At this end create a breakpoint then run the test is debug.

* Force the connection to commit :

![Commit the connection](./docs/images/connection-commit.png)

* The test is now successful

![Successful test](./docs/images/successful-test.png)

repository: adbonnin/micronaut-transaction-question

repo_name: adbonnin/micronaut-transaction-question