text:
---
layout: post
title: "Mining Twitter data with R, TidyText, and TAGS"
modified: 2017-02-24 14:41:00 -0500
image:
  feature: waterfall.jpg
  teaser: waterfall-teaser.jpg
  credit: Pixabay
  creditlink: https://www.pexels.com/photo/time-lapse-photography-of-water-splashing-on-brown-rock-during-dytime-33104/
share: true
categories: [data science, R]
short_description: "Use TAGS archive, Google Drive, and TidyVerse tools in R to easily mine Twitter data."
---

One of the best places to get your feet wet with text mining is Twitter data. Though not as open as it used to be for developers, the Twitter API makes it incredibly easy to download large swaths of text from its public users, accompanied by substantial metadata. A treasure trove for data miners that is relatively easy to parse.

It's also a great source of data for those studying the distribution of (mis)information via digital media. This is something I've been working on a lot lately, both in independent projects and in preparation for my courses on Digital Storytelling, Digital Studies, and The Internet. It's amazing how much data you can get, and how detailed a picture it can paint about how citizens, voters, and activists find and disseminate information. Most recently, Bill Fitzgerald and I have begun discussing a project analyzing the distribution of (mis)information in extremist, so-called "alt-right" circles on Twitter, and comparing the language and information sources of left- and right-leaning communities on Twiter.

It turns out this is a really straightforward thing to do, thanks to Martin Hawksey's TAGS (Twitter Archiving Google Sheet) and Julia Silge's and David Robinson's TidyText package for R. In what follows, I'll walk through the process of setting up a TAGS archive, linking it to R, and mining it with TidyText (and other tools from the TidyVerse).

## Setting up a TAGS archive

There are some excellent tools for interacting with the Twitter API directly, but if what you want is a regularly updating archive of tweets that you can repeatedly mine and analyze, TAGS is definitely the way to go. All you need is a Google account, a Twitter account, and a copy of Martin Hawksey's Google Sheet, and you're good to go. You don't even need your own API key!

To set it up, make a copy of Martin's TAGS sheet in your Google account. Then follow the instructions on the setup page to get up and running. After entering your search terms, I recommend setting it up to update every hour *and* making a one-off collection to start ("Run now!"). For more information on setting it up, check out Martin's video:

## Connecting your Google sheet with R

If you're just planning on doing a one-time analysis of the tweets you archived, you can simply export your Google sheet as a CSV file (specifically, the Archive page), and read it into R with ```read.csv``` or ```read_csv```. However, if you want to keep the archive updating over time and check on it regularly with R (or maybe even build a Shiny App that automatically updates analyses and visualizations for you!), you'll need to publish your sheet to the web. Go to ```File >> Publish to the web...``` and publish the Archive page to the web as a CSV file. Be sure to check the box to automatically republish when changes are made. That way, when your Google sheet downloads new Twitter content each hour, it will also update the public CSV file.

When you click ```Publish```, it will give you a URL. Simply copy that URL and paste it into R:

~~~r
tweets <- read_csv('https://docs.google.com/spreadsheets/d/your-archive-page-id-here',
                   col_types = 'ccccccccccccciiccc')
~~~

The ```col_types``` will ensure that the long, numeric ID numbers import as characters, rather than convert to (rounded) scientific notation.

Now you have your data, updated every hour, accessible to your R script!

## Mining the tweets with TidyText (and dplyr and tidyr)

One of my favorite tools for text mining in R is TidyText. It was developed by a friend from grad school, Julia Silge, in collaboration with her (now) Stack Overflow colleague, David Robinson. It's a great extension to the TidyVerse data wrangling suite. (Also, you should pre-order their new book, *Text Mining with R: A Tidy Approach*.)

Let's walk through some of the things you can do with your Twitter archive using TidyText (and the TidyVerse in general). As an example, I'll reference my growing collection of tweets with the hashtag ```#americafirst```. (Note that though the tweets are technically public, I want to protect user privacy, so I won't be linking to my Google sheet here or providing anything other than aggregate results. If you want, you can reproduce the study with the code, though it will return a later batch of tweets.)

First, load the necessary libraries, import the data from your Google sheet, and append an R-friendly date column.

~~~r
library(tidyverse)
library(tidytext)
library(lubridate)
library(stringr)
library(httr)

google_sheet_csv <- '' #insert URL of published CSV file from TAGS archive Google sheet
tweets <- read_csv(google_sheet_csv,
                   col_types = 'ccccccccccccciiccc') %>%
  mutate(date = mdy(paste(substring(created_at, 5, 10), substring(created_at, 27 ,30))))

source_text <- '#americafirst'
minimum_occurrences <- 5 # minimum number of occurrences to include in output
~~~

This will give you a tibble (a tidy data frame) where each row is a tweet, and each column contains (meta)data for that tweet. I've also declared two variables that will help out in the parsing later.

To find the most frequent words, hashtags, or Twitter handles in the archive, we can pretty much lift the code out of Julia and David's ebook:

~~~r
reg_words <- "([^A-Za-z_\\d#@']|'(?![A-Za-z_\\d#@]))"
tidy_tweets <- tweets %>%
  filter(!str_detect(text, "^RT")) %>%
  mutate(text = str_replace_all(text, "https://t.co/[A-Za-z\\d]+|http://[A-Za-z\\d]+|&|<|>|RT|https", "")) %>%
  unnest_tokens(word, text, token = "regex", pattern = reg_words) %>%
  filter(!word %in% stop_words$word,
         str_detect(word, "[a-z]"))
~~~

This snippet of code filters out any tweets whose text begins with ```RT``` (retweets; delete or comment out that line to keep retweets in), removes URLs and certain characters that signal something other than natural language text from the tweets, and *tokenizes* the tweets into words. That means splitting the text by spaces, removing punctuation, converting all letters to lower-case, etc., and then applying all of the metadata for the tweet to each individual word. A tweet with 10 words is a single record in the ```tweets``` data frame, but after tokenizing will result in 10 records in the new ```tidy_tweets``` data frame, each with a different word, but identical metadata. After tokenizing, we filter out any records where the word is contained in a list of stop words (a, an, the, of, etc.) or where the "word" contains no letters (i.e., raw numbers).

What's unique about the way TidyText is tokenizing the Twitter data here is that it uses a regular expression to parse data in a Twitter-specific way. This regular expression includes all alphanumeric characters and the hash ```#``` and at-reply ```@``` symbols. (Julia and David drop those in their ebook analyses, but I want to filter out all Twitter handles for privacy, as well as apply special analysis to hashtags, so I'm leaving them in.) It doesn't always do this. For natural language, you can usually just tokenize by a pre-defined "word" concept, or n-gram. But this approach is necessary when parsing text that contains a lot of URLs and special symbols, as tweets do.

To find the most common words in this tweet corpus, we can use the following code to count occurrences of each unique word, filter out the most common (n > 150, in this case), and order them most to least frequent.

~~~r
tidy_tweets %>%
  count(word, sort=TRUE) %>%
  filter(n > 150) %>%
  mutate(word = reorder(word, n))
~~~

This produces:
~~~r
# A tibble: 31 × 2
               word     n
             <fctr> <int>
1     #americafirst  5593
2             #maga  2530
3            #trump   821
4            @potus   742
5  @realdonaldtrump   741
6       #trumptrain   666
7             trump   503
8               #2a   457
9    #draintheswamp   432
10    #buildthewall   408
# ... with 21 more rows
~~~

All hashtags and Twitter handles! Let's see what's most common if we omit hashtags and handles:

~~~r
tidy_tweets %>%
  count(word, sort=TRUE) %>%
  filter(substr(word, 1, 1) != '#', # omit hashtags
         substr(word, 1, 1) != '@') %>% # omit Twitter handles
  mutate(word = reorder(word, n))

# A tibble: 6,533 × 2
        word     n
      <fctr> <int>
1      trump   503
2      video   339
3  president   316
4    america   197
5     people   188
6    illegal   165
7   american   140
8       time   121
9  americans   114
10   country   109
# ... with 6,523 more rows
~~~

To make this count into a new data frame (to view the whole thing or save to a file), just assign it to a variable.

~~~r
word_count <- tidy_tweets %>%
  count(word, sort=TRUE) %>%
  filter(substr(word, 1, 1) != '#', # omit hashtags
         substr(word, 1, 1) != '@') %>% # omit Twitter handles
  mutate(word = reorder(word, n))
~~~

Or visualize it with ```ggplot2``` (part of the ```TidyVerse``` meta-package).

~~~r
tidy_tweets %>%
  count(word, sort=TRUE) %>%
  filter(substr(word, 1, 1) != '#', # omit hashtags
         substr(word, 1, 1) != '@', # omit Twitter handles
         n > 80) %>% # only most common words
  mutate(word = reorder(word, n)) %>%
  ggplot(aes(word, n, fill = word)) +
  geom_bar(stat = 'identity') +
  xlab(NULL) +
  ylab(paste('Word count (since ',
             min_date,
             ')', sep = '')) +
  ggtitle(paste('Most common words in tweets containing', source_text)) +
  theme(legend.position="none") +
  coord_flip()
~~~

Often times, looking for the most common bigrams (two-word phrases) is more instructive than individual words. Using TidyText to do this with Twitter data is a bit more complicated, as you need to parse by regex, rather than use the built-in n-gram option. Julia and David don't give an example in their ebook, but it's not too hard. We simply tokenize by regex like before, use dplyr's ```lead()``` function to append the following word to each record, and then ```unite()``` the two into a single bigram (assuming they both belong to the same tweet). Here's how to do that, as well as to remove bigrams containing hashtags, Twitter handles, raw numbers, stop words.

~~~r
tidy_bigrams <- tweets %>%
  filter(!str_detect(text, "^RT")) %>%
  mutate(text = str_replace_all(text, "https://t.co/[A-Za-z\\d]+|http://[A-Za-z\\d]+|&|<|>|RT|https", "")) %>%
  unnest_tokens(word, text, token = "regex", pattern = reg_words) %>%
  mutate(next_word = lead(word)) %>%
  filter(!word %in% stop_words$word, # remove stop words
         !next_word %in% stop_words$word, # remove stop words
         substr(word, 1, 1) != '@', # remove user handles to protect privacy
         substr(next_word, 1, 1) != '@', # remove user handles to protect privacy
         substr(word, 1, 1) != '#', # remove hashtags
         substr(next_word, 1, 1) != '#',
         str_detect(word, "[a-z]"), # remove words containing ony numbers or symbols
         str_detect(next_word, "[a-z]")) %>% # remove words containing ony numbers or symbols
  filter(id_str == lead(id_str)) %>% # needed to ensure bigrams to cross from one tweet into the next
  unite(bigram, word, next_word, sep = ' ') %>%
  select(bigram, from_user, date, id_str, user_followers_count, user_friends_count, user_location)
~~~

Now we can count and sort the bigrams.

~~~r
tidy_bigrams %>%
  count(bigram, sort=TRUE) %>%
  mutate(bigram = reorder(bigram, n))
~~~

This gives us:

~~~r
# A tibble: 4,857 × 2
                  bigram     n
                  <fctr> <int>
1         illegal aliens    77
2        calls president    59
3        president trump    57
4              fake news    51
5          maxine waters    38
6       sanctuary cities    38
7             anti trump    37
8        project veritas    37
9  enforcing immigration    31
10     homeland security    31
# ... with 4,847 more rows
~~~

And we can plot it like before.

~~~r
tidy_bigrams %>%
  count(bigram, sort=TRUE) %>%
  filter(n >= 30) %>%
  mutate(bigram = reorder(bigram, n)) %>%
  ggplot(aes(bigram, n, fill = bigram)) +
  geom_bar(stat = 'identity') +
  xlab(NULL) +
  ylab(paste('bigram count (since ',
             min_date,
             ')', sep = '')) +
  ggtitle(paste('Most common bigrams in tweets containing', source_text)) +
  theme(legend.position="none") +
  coord_flip()
~~~

We can do a lot with this word and bigram data, as Julia and David's ebook demonstrate. We can produce a network analysis of words (essentially a 2D visualization of a Markov model; we could also do this with user data), we can compare word or bigram frequency with another Twitter corpus, and we could search for the most common hashtags and handles in the corpus to find other terms to add to the search that generates the TAGS archive.

One thing Bill and I want to look at, though, is where some of the people using these hashtags are getting the information that they are sharing. In other words, we want to access *only* the URLs we filtered out at the beginning, and use them to find out which articles and which domains appear the most often. So let's go back to the beginning and do that.

We don't need to reimport the data, as the ```tweets``` tibble still contains the raw imported tweets with all the metadata. We just have to re-parse the tweets.

~~~r
reg <- "([^A-Za-z_\\d#@:/']|'(?![A-Za-z_\\d#@:/]))"
urls_temp <- tweets %>%
  unnest_tokens(word, text, token = "regex", pattern = reg, to_lower = FALSE) %>%
  mutate(word = str_replace_all(word, "https|//t|http|&|<|>", ""),
         word = str_replace_all(word, "co/", "https://t.co/")) %>%
  select(word) %>%
  filter(grepl('https://t.co/', word, fixed = TRUE)) %>%
  count(word, sort=TRUE) %>%
  mutate(word = reorder(word, n))
~~~

Note the differences in this parse code: the regular expression now includes the forward slash. Without it, we'd lose the URL strings. Also note the ```to_lower = FALSE``` option in ```unnest_tokens```. All URLs returned by the Twitter API use the t.co URL shortener, which uses case-sensitive strings for their URLs. Without ```to_lower = FALSE```, TidyText would make everything lower-case, and our URLs would be useless. The first mutate command should look familiar, except that it no longer removes whole URLs that contain ```https://t.co```. Instead, it removes only the ```https```. What we're left with, at this point, is a bunch of "words" of the form ```co/juMbl3OfCh@ract3r$```. We want to re-construct these into full URLs, which the next line does for us. Then we remove all metadata, and then all "words" that don't contain ```https://t.co/```. This will leave us *only* with t.co shortened URLs, count them, and sort them by frequency, saved in a new data frame ```urls_temp```.

But a bunch of t.co URLs isn't very helpful. We want the targets they point to! Thankfully, Hadley Wickham's ```httr``` package can do that for us! First, let's limit ourselves to the most common URLs. (Remember that ```minimum_occurrences = 5``` we set at the beginning? That will save us a lot of processing time, while only skipping over URLs that contribute little to the corpus data.)

~~~r
urls_common <- urls_temp %>%
  filter(n >= minimum_occurrences) %>%
  mutate(source_url = as.character(word)) %>%
  select(source_url, count = n)
~~~

Now we use ```httr``` to obtain the target URLs for those t.co URLs in the corpus that occur at least five times, and return an HTTP status code (so we can filter out any broken links). This will take some time, especially for a large corpus. If you regularly return to a large, growing corpus, I recommend saving the results to a CSV file, and then importing that CSV file and using ```anti_join()``` so you can limit your GET requests only to URLs you haven't already tracked down.

~~~r
url <- t(sapply(urls_common$source_url, GET)) %>%
  as_tibble() %>%
  select(url, status_code)
~~~

Then we can bind those results to the original URLs and their frequency count, and filter out any 404 statuses (page not found), and any URLs that point back to Twitter (like quoted tweets). Of course, you can leave those in if it makes sense for your project.

~~~r
url_list <- cbind(urls_common, unnest(url)) %>%
  as_tibble() %>%
  select(url, count, status_code) %>%
  filter(status_code != 404,
         url != 'https://t.co/',
         !grepl('https://twitter.com/', url))
~~~

Now we have a list of URLs, sorted by how frequently they occur in our corpus! A great way to identify the (mis)information sources in an online community.

If it's not articles but *sites* we're after, we can just parse the URL character string to get to the root domain of each URL and count those instead.

~~~r
# extract domains from URLs
extract_domain <- function(url) {
  return(gsub('www.', '', unlist(strsplit(unlist(strsplit(as.character(url), '//'))[2], '/'))[1]))
}

# count the frequency of a domain's occurrence in the most frequent URL list
domain_list <- url_list %>%
  mutate(domain = mapply(extract_domain, url)) %>%
  group_by(domain) %>%
  summarize(domain_count = sum(count)) %>%
  arrange(desc(domain_count))
~~~

That first function is really messy! It's simply splitting the URL by ```//``` and taking everything to the right of that, then splitting it by ```/``` and taking everything to the left of the first slash, then dropping ```www.```, if present.

This gives us a very interesting mix of domains found in the #americafirst tweets.

~~~r
# A tibble: 33 × 2
                     domain domain_count
                      <chr>        <int>
1             breitbart.com          446
2            patriotpost.us          139
3          pamelageller.com          137
4    NewsUnitedStates.co.vu          105
5               youtube.com          101
6          americaspaper.ml           69
7            USATrump.co.vu           34
8  overpassesforamerica.com           30
9               foxnews.com           26
10           whitehouse.gov           25
# ... with 23 more rows
~~~

Which we can also plot.

~~~r
domain_list %>%
  mutate(domain = reorder(domain, domain_count)) %>%
  ggplot(aes(domain, domain_count, fill = domain)) +
  geom_bar(stat = 'identity') +
  xlab(NULL) +
  ylab(paste('domain count (since ',
             min_date,
             ')', sep = '')) +
  ggtitle(paste('Most common domains in tweets containing', source_text)) +
  theme(legend.position="none") +
  coord_flip()
~~~

And if you follow further redirects for root domains, guess what you find! Those sites supposedly from Mali and Vanuatu? They redirect to adf.ly, whose motto is "Get paid to share your links on the Internet!" Add them together, and they're the second-highest frequency of the domains in this corpus. I smell something fishy. But that's for another day...

## On GitHub

I've put all of this code (and a little more) in a GitHub repository, which I will continue to expand as time permits and my projects progress. Feel free to download it, test it out, and even send a pull request if you add some cool functionality to it!

Happy Twitter mining!

Header image by Pixabay.


text:
---
layout: post
title: Session 17 Intro to Threads
readtime: true
date: Wed 07 Apr 2021 05:42:12 PM EDT
gh-repo: rhit-csse332/csse332
gh-badge: [star,watch,follow]
---

# Introduction and aim
In this session, we will concretely define threads and learn how to use the `pthreads` library. We
will conduct a speed test to compare multithreading vs multiprocessing. Finally, we will take a look
at what are some common issues with multi-threaded applications so we can set up our dive into
concurrency.

## Logistics
* [Slides](https://rosehulman-my.sharepoint.com/:p:/g/personal/noureddi_rose-hulman_edu/Edaz8g1YS5lEu3yXwdlaud8B2aAj7Gt8pVL9UAt9Gw47zA?e=bPennE)
* [Class notebook section](https://rosehulman-my.sharepoint.com/personal/noureddi_rose-hulman_edu/_layouts/OneNote.aspx?id=%2Fpersonal%2Fnoureddi_rose-hulman_edu%2FDocuments%2FClass%20Notebooks%2FCSSE%20332%20Operating%20Systems&wd=target%28_Content%20Library%2FSession%2017.one%7CDB74E55B-48A2-4446-BEBB-9E38D430D2D9%2F%29)
* [Source code]({{ site.gh_repository_url }}/tree/main/activities/17_threads)
* [Activity submission](https://moodle.rose-hulman.edu/mod/assign/view.php?id=2708154)
* [Clear/Muddy survey](https://moodle.rose-hulman.edu/mod/quiz/view.php?id=2785929)

In-class activity and clear/muddy survey are due on Friday before the start of session 18.

# Outline

|           Topic             |           Time              |
| :-------------------------- | --------------------------- |
| Motivation | 5 minutes |
| Intro to multi-threading and multi-threading vs multi-processing | 10 minutes |
| Intro to `pthreads` | 5 minutes |
| Activity | 10 minutes |
| __Break__ | 2 minutes |
| Concurrency issues | 15 minutes |
| Conclusion and moving forward | 3 minutes |

# Video Recording
It turned out that I forgot to hit record when we started the lecture (what a great way to start
things off Mohammad!!). So Dr Song has kindly provided his videos below:

license: MIT

branch_name: refs/heads/main

extension: md

gha_language: C

gha_license_id: MIT

license_type: permissive


text:
# Hello World Daemon

This project is as simple Python script which uses [python-daemon](https://pypi.python.org/pypi/python-daemon) to create a daemon which logs a message every 30 seconds.

branch_name: refs/heads/master


text:
# Introduction

## Background

It's always a best practice to have concatenation and minimization of resources (JavaScript, CSS) when deploy to production.
However it makes life harder when debug a issue at production.
In additions, people is likely to have some change in order to quick verify whether the fixing is right.

## Limitation of existing tools

- Why adding a parameter like `debug=true` to URL is not sufficient?
It enables to show un-minified version to help debug but can't allow to have a quick fix and verification.

- Why Charles (http://www.charlesproxy.com/), a commercial web debug proxy tool is not sufficient?
It's commercial and not programmable/highly config-able.

## Solution

Therefore, we'd like to create such tool, a proxy, to address this problem.

The tool will behavior as below

1. Started as standalone proxy server
2. It acts a middle-man proxy mostly. (Just fetch request from origin site and return the response)
3. When request match certain URL patterns, it respond with content from
    - Local files. Therefore, people will not only to serve un-minified file but also verify modification very quickly.
    - Mock services. In order to mock API requests from a predefined mock server.
    - Other similar environment. It could be useful while multiple QA boxes.

The tool will be able to start from command line and read a configuration file in order to figure out how to match URL to local files.

# Features

  - mock resource files (JS/CSS/Images) from local file or another URL
  - mock API response from local file or another URL
  - mock APE response with customized response header / body

# Build from source

  1. `mvn package`
    - It will generate kingfisher/target/kingfisher-release-2.0.0.zip
  2. unzip the release zip file  `unzip kingfisher-release-2.0.0.zip`
  3. Modify the configuration file : `sample.config.xml` accordingly
    - examples are provided, read it for details
  4. `sh run.sh` start the proxy!

# Install the root CA certification (Chrome)

  1. Open chrome://settings/
  2. Click show advanced settings
  3. Click button: manage certificates
  4. Drag the file (proxy_cert/cacert.pem) in.
  5. Find it (KingFisher CA), make it all trust
  6. Save

# Script Guidance

  1. Java Script
  2. embed variable:
     - context

      used to access http request

      [source](src/main/java/com/kingfisher/proxy/Context.java)
     - delegator

      used to generate response from local resource or remote URL resource

      [source](src/main/java/com/kingfisher/proxy/resolver/Delegator.java)
     - responseBuilder

      used to build response from sketch or existing response

      [source](src/main/java/com/kingfisher/proxy/util/HttpResponseBuilder.java)

# Other library worth to compare

- [WireMock](http://wiremock.org/)
- [hoxy](http://greim.github.io/hoxy/)
- [mock server](http://mock-server.com/)
- [browsermod proxy](http://bmp.lightbody.net/)

gha_language: Java


text:
## Discover Objects with Property of Type

This is an excercise of finding all the objects within an assembly that have a
property of a type.

The [test project includes classes][test_classes] that might come from an e-commerce platform
that reference each other in different ways (`Category`, `Order`, `Product`,
`ProductImage`, `RevisedProductListing`).

The classes are connected to each other with the following properties.

```c#
class Category				// (IEnumerable<Product> Products)
class Order        			// (ICollection<Product> OrderItems)
class Product      			// (Category Category, ICollection<ProductImage> Images)
class ProductImage 			// (Product Product)
class RevisedProductListing // (Product OldProduct, Product NewProduct)
```

## Usage

Objects can be discovered from all available assemblies or by specifying the assembly as the first parameter.

```c#
interface ObjectsWithPropertyType
{
    IList<Type> Discover<T>();
    IList<Type> Discover<T>(Assembly assembly);
    IDictionary<Type, IEnumberable<string>> DiscoverWithMapping<T>(Assembly assembly);
}
```

[test_classes]: https://github.com/BaylorRae/discover-projects-with-property/tree/master/DiscoverObjectsWithPropertyType.Tests/Classes


text:
The `java.util.ServiceLoader` class was added in JDK 6 to provide support
for the de-facto service provider loading mechanism that has been used by
Java extensions for many years.

A _service_ is defined as a set of interfaces and (often abstract) classes;
i.e. a service provides an API for some functionality.  An example of a
service is the JavaMail (`javax.mail`) API.  A service can have one or
more _providers_ that implement all or part of the API, perhaps in different
ways.

A service provider specifies some metadata in plain text files in
the `META-INF/services` class path container.  The name of each text
file specifies the fully-qualified class name of an interface or class
specified in the service API.  The file contents consist of a single line
of text that specifies the fully-qualified class name of a class that
implements or extends the type specified in the file name.  Each
implementation class must have a public no-arg constructor that is used
by the `ServiceLoader` to construct an instance of the class.

An application that wishes to use a service needs a mechanism to locate
a suitable provider for the service.  The `ServiceLoader` mechanism provides
underlying support for a _service locator_ object.  The locator is used to
find a suitable provider for a given service.  The `ServiceLoader` class
provides methods that can locate implementations of a given interface or
class in the service API on the application class path, using the metadata
specified by one or more service providers.  The service locator is
responsible for determining which provider to select when multiple providers
are available.

This demo is Maven project that consists of several submodules:

* service-loader-demo -- is the top-level project
* service-loader-api -- provides the service API for a simple service that
  provides some sort of character encoding/decoding capability
* service-provider-standard -- is a service provider implementation
* service-provider-alternate -- is a service provider implementation
* service-provider-main -- is a simple `main` that uses the
  `CodecLocator` defined in the service API to locate codecs for two different
  character encodings

repository: ceharris/service-loader-demo

repo_name: ceharris/service-loader-demo


text:
---
extends: _layouts.post
section: content
title: My First Laravel Package - laravel-silent-spam-filter
date: 2019-05-21
description: I created my first open-source Laravel package and I couldn't be happier.
categories: [Laravel]
featured: false
image: /assets/img/2019-05-21-my-first-laravel-package-silent-spam-filter.png
image_thumb: /assets/img/2019-05-21-my-first-laravel-package-silent-spam-filter.png
image_author:
image_author_url:
image_unsplash:
---

I've wanted to contribute to open-source for a long time but getting started is always the hardest part. PHP package development is one of those areas that sounds simple on the surface but if you want to do it properly you'll realize that there are many subtle layers to it, making it easy to find excuses not to do it at all.

So when I heard that [Marcel Pociot](https://twitter.com/marcelpociot) (a prominent open-source contributor and Laravel expert) was working on a course on [PHP package development](https://phppackagedevelopment.com/) I knew it was going to be worth every penny. And no, in case you're wondering, he's not paying me to say this, I'm just very happy with his course.

Owning the tools is cool but putting them to work is another story. Luckily, one thing I don't lack is a shortage of ideas. Initially I had a more ambitious package in mind (which will probably be the next one I build) but the need arose for a reusable and configurable piece of Laravel code, and after a Sunday of on-and-off coding I produced what I'm about to describe.

## The TL;DR version

If you want to hear my rambling history of how this package came about, hang around. Otherwise, here's the short version of what it does.

[laravel-silent-spam-filter](https://github.com/breadthe/laravel-silent-spam-filter) does a very simple thing. It analyzes a string for certain keywords or phrases and returns `true` if it finds those words or `false` otherwise. Sounds useless but there's more depth to it.

First, the keywords are configurable in a config file. When you publish the package config, a file will be created under `config/silentspam.php` that comes pre-loaded with just 2 entries:

```php
return [
    'blacklist' => [
        '(beautiful|best|sexy) girls',
        'girls in your (city|town)',
    ],
];
```

You can replace these and/or add your own. In this example, if a string contains any combination of "beautiful girls" or "best girls" or "sexy girls" or "girls in your city" or "girls in your town", it will be marked as spam.

Thus, the second feature is that you can use regex patterns to filter spam.

Third, you can use Laravel's facade to run the spam check, as shown below:

```php
SilentSpam::isSpam('Find sexy girls in your city'); // true
```

Finally, while the keywords in the config are global to the entire application, you can add additional keywords at runtime, before calling `isSpam`:

```php
SilentSpam::blacklist([
    'buy pills',
]);

SilentSpam::isSpam('Go to this site to buy pills'); // true
```

And if you feel lazy, there's also a `notSpam` command, which is exactly the opposite of `isSpam`.

```php
SilentSpam::notSpam('This is a normal message'); // true
```

## A spammy situation

One of my projects named [Sikrt](https://sikrt.com/) has apparently caught the attention of some spammers who've latched on to the public-facing contact form, despite being protected by Google's Recaptcha. I wasn't getting a lot of spam, but it was constantly trickling in, at a rate of 1-2 a day. Perhaps Recaptcha was actually doing its job, or else I would have been swamped? Who knows. The fact is that it still let some of it through.

Now, these spammers (bots really) operate under the premise that any public form might be attached to a commenting system or something similar (like on a blog for example). They'll fill in and submit the form with their garbage which almost always contains links to whatever they're trying to promote. Very often these messages will get passed through, ending up as "legit" content on that page. From there, Google will pick up the links, and a few visitors will click them. Sometimes the form contents are emailed to the site owner's address, from where additional mischief can occur.

Unfortunately for them, all my contact form does is save an entry in my database. I didn't build a more complex solution because I don't need it at the moment. I can check messages directly in the DB. So the spam doesn't end up anywhere productive. There's still the matter of clearing these out occasionally (as I said, the volume received is low).

I thought I would automate this clearing-out process without complex checks or third-party APIs.

But before that, I wanted to try yet another protection method: the honeypot. And what package would be better suited for this than one from Spatie (a heavy-duty Laravel contributor). I added [laravel-honeypot](https://github.com/spatie/laravel-honeypot) but unfortunately, just like Google Recapcha, it still lets some spam through. I was kinda expecting that. By now I would've thought that spammers have grown wise to this method and built smarter bots that can bypass it.

## The package idea

By analyzing the spam messages I had so far, I noticed some very obvious patterns and words that could be easily filtered by. It looks like all of it came from the same spamming "authority" so it was trivial to create a few very simple rules to handle those kinds of messages.

The way I decided to approach it was to simply not save anything in the database if it matched those patterns. The spammer would receive feedback that the form was submitted successfully, but the data would end up in a black hole.

Initially I built the pattern matching as a service in [Sikrt](https://sikrt.com/) itself, using TDD of course, but soon after launching it in production I decided it would make a great (if very basic) first package.

I wanted 2019 to be the year I released at least one open-source package and even though I had something more complex in mind, this was a great learning experience.

## Building the package

As I mentioned at the beginning, [PHP package development](https://phppackagedevelopment.com/) is amazing in guiding you step by step through the PHP package-building process, including Laravel-specifics. Hand-crafting still remains a little tedious, especially for someone who hasn't done this a hundred times before, but luckily the author has also built [Laravel Package Boilerplate](https://laravelpackageboilerplate.com/#/) which makes it trivial to scaffold the whole directory structure, along with all the bells and whistles he describes in the course.

Of course, as it oft happens in these pioneering moments, after moving my original logic to the package, I spent 3 hours trying to figure out why my tests were failing with a cryptic message, only to discover that I had the wrong namespace somewhere in my new code. Lesson learned.

My original spam filtering service worked something like this:

```php
use App\Services\SpamService;

// Silently reject spam messages
if ((new SpamService(config('spam.blacklist')))->notSpam(request('message'))) {
    // save the contact form data
}
```

My goal was to simplify the API by implicitly loading the blacklist from the config file, and also to be able to use Laravel's facade accessor.

Converting the code was simply a matter of copy-pasting it from the original service to the new package structure, replacing the calls with the facade accessor and making sure the tests still worked. Which, until I discovered the elusive mangled namespace, they didn't.

Overall, the experience was smoother than expected. Back in my [Sikrt](https://sikrt.com/) project codebase, it was even more simple to `composer require breadthe/laravel-silent-spam-filter` and swap everything out. And the app continued to work perfectly.

 ## In conclusion

 The package may be very basic but it's my package and I'm proud of it. Building it allowed me not just to dip my toes in this exciting new world, but also opened my eyes to what the package-building process entails.

 You may ask why this is strictly a Laravel package. For one thing, I'm deeply embedded in the Laravel ecosystem and wouldn't have anything else at the moment. For another, as simple as it is, I don't think there would be much value in creating a general PHP package. After all, the core functionality is just a regex check. But if you are still upset about the Laravel exclusivity, you are always invited to [contribute](https://github.com/breadthe/laravel-silent-spam-filter/blob/master/CONTRIBUTING.md).

It's a bit of a drug. I'm already brainstorming what the next one should be.

gha_language: CSS


text:
## Nintendo Labo Project

**Project description:** This was a short commisison to design enemy AI for a video game utilizing Nintendo Labo technologies. The game is in the style of a tower defense in which enemies attack fortresses one by one and the player must defend them by constructing physical defenses which are converted virtually.

**Tasks:** I created the default enemy AI for the project consisting of randomized movements and allowing enemies to duck for cover within the environment. The brown box is a randomly generated obstacle the enemies can use for cover and the grey box is the structure enemies will line up behind to slowly demolish it.
### Screenshots
---

<img src="images/NintendoLabo/LaboGif.gif" border="5"/>

license: Unlicense


text:
# Clima
> iOS app to display the weather by city name or  gps user location.

The purpose of this project is to show the basic functions of the following topics.

- Schemes (Change your API endpoint/environment using Xcode Configurations)
- Alamofire and SwiftyJSON for Http Requests
- Unit Test
- MVC structure

**Note: Do not forget to change the [OpenWeatherMap](http://openweathermap.org/api) key (OWM_API_KEY) from User-Defined settings by your own key**

Iphone 8             |  Iphone X
:-------------------------:|:-------------------------:
![](screenshots/iphone8-home.png)  | ![](screenshots/iphoneX-home-alert.png)
![](screenshots/iphone8-home.png) | ![](screenshots/iphoneX-home.png)
![](screenshots/iphone8-change-city.png) | ![](screenshots/iphoneX-change-city.png)

## Requirements

- iOS 8.0+
- Xcode 9
- Swift 4
- Cocoapods 1.3.+

## Meta

Jonathan Guillermo –<EMAIL_ADDRESS>
Distributed under the MIT license.

[Github - hernandezmarquina](https://github.com/hernandezmarquina)

repository: hernandezmarquina/Clima

repo_name: hernandezmarquina/Clima


text:
## Example library usage

The following C code snipped demonstrates, how library functions and primitives can be used for the transitive closure
evaluation of the directed graph, represented as an adjacency matrix with boolean values. The transitive closure
provides info about reachable vertices in the graph:

```c++
/**
 * Performs transitive closure for directed graph
 *
 * @param A Adjacency matrix of the graph
 * @param T Reference to the handle where to allocate and store result
 *
 * @return Status on this operation
 */
spbla_Status TransitiveClosure(spbla_Matrix A, spbla_Matrix* T) {
    spbla_Matrix_Duplicate(A, T);                       /* Duplicate A to result T */

    spbla_Index total = 0;
    spbla_Index current;

    spbla_Matrix_Nvals(*T, &current);                   /* Query current nvals value */

    while (current != total) {                          /* Iterate, while new values are added */
        total = current;
        spbla_MxM(*T, *T, *T, SPBLA_HINT_ACCUMULATE);   /* T += T x T */
        spbla_Matrix_Nvals(*T, &current);
    }

    return SPBLA_STATUS_SUCCESS;
}
```

The following Python code snippet demonstrates, how the library python wrapper can be used to compute the same
transitive closure problem for the directed graph within python environment:

```python
import pyspbla as sp

def transitive_closure(a: sp.Matrix):
    """
    Evaluates transitive closure for the provided
    adjacency matrix of the graph.

    :param a: Adjacency matrix of the graph
    :return: The transitive closure adjacency matrix
    """

    t = a.dup()  # Duplicate matrix where to store result
    total = 0  # Current number of values

    while total != t.nvals:
        total = t.nvals
        t.mxm(t, out=t, accumulate=True)  # t += t * t

    return t
```

repository: SparseLinearAlgebra/spbla

repo_name: SparseLinearAlgebra/spbla


text:
# Combine XLS Name Columns

Combines names in separate columns (John, Jane, Doe) and appends a new column to the spreadsheet containing "John and Jane Doe".

## Installation

1. Install `openpyxl`
    ```bash
    pip3 install openpyxl
    ```

## Usage

1. Edit the input file and header columns to match your spreadsheet, and the output file to whatever you want:
    ```python
    input_file = "input.xlsx"
    output_file = "output.xlsx"

    xls_headers = [["first name",False,-1],["spouse/partner name",False,-1],["last name",False,-1]]
    ```

2. Run the script:
    ```bash
    python3 xls-combine-name-cols.py
    ```

repository: sirken/xls-combine-name-columns

repo_name: sirken/xls-combine-name-columns


text:
# Letter #

Turn to 1 to begin.
<a name="section1">

## 1 ##
 LaTeX output (and this PDF output) from this book should be in letter size rather than the default A5 size (see command-line options used in letter.gamebook.options).

repository: lifelike/gamebookformat

repo_name: lifelike/gamebookformat


text:
# helpful-code

A repository to hold all sorts of skeletons, templates, and helpful snippets.

## MEAN Stack

A MEAN stack template exists in the MEAN directory. It includes the general template, as well as an example of a controller, directive, and a service.

repository: runnersaw/helpful-code

repo_name: runnersaw/helpful-code


text:
** Our Privacy Policy **
This privacy policy will explain how we use the personal data we collect from you when you use our services.

Topics:
- What data do we collect?
- How do we collect your data?
- How will we use your data?
- How do we store your data?
- What are your data protection rights?
- Changes to our privacy policy
- How to contact us
- How to contact the appropriate authorities

** What data do we collect? **
We collect the following data:
- Discord UUID and name
- User provided API Keys for retrieval of user permitted information from the Guild Wars 2 API
- Guild Wars 2 API data, based on the provided Guild Wars 2 API

** How do we collect your data? **
You directly provide us with most of the data we collect. We collect data and process data when you:
- Verify your Guild Wars 2 world association to grant/deny access to sections of the community voice & text channels, using your exchanged Guild Wars 2 API key.
  - We periodically re-verify your Guild Wars 2 world association using your provided API key
- Use text and voice channels the bot has access to.

** How will we use your data? **
We collect your data so that we can:
- Verify your Guild Wars 2 world association.
- Publicize additional personal information on our associated community discord channels, if consented by you via the supported commands.
- Collect statistical information on world transfers and server activity.

If you agree, we will share your data with the moderation teams of connected communities, which currently is limited to:
- Far Shiverpeaks Community Discord staff.

** How do we store your data? **
We securely store your data on servers hosted by *Hetzner Online GmbH* and all data in transit over the internet is encrypted using asymmetric encryption keys authorized by Let's Encrypt.

We will keep your Guild Wars 2 player data for as long as your Guild Wars 2 API key remains valid. An API Key can be made invalid by deleting it from your account at <https://account.arena.net/applications>. We will retain your data for 2 weeks after an API key has become invalid for the following reasons:
- To prevent users from avoiding moderation after breaking community guidelines or rules.
- To prevent accidental deletion of data when the Guild Wars 2 API experience issues.

Once the 2 weeks are up, your data will be deleted from our database.

**An exception exist** in our Legitimate Interest to prevent banned users from avoiding their restrictions by requiring their ban to be deleted and such a ban will **NOT** be automatically deleted after 2 weeks.

** What are your data protection rights? **
We would like to make sure you are fully aware of all of your data protection rights. Every user is entitled to the following:

- The right to access – You have the right to request us for copies of your personal data. We may charge you a small fee for this service.
- The right to rectification – You have the right to request that we correct any information you believe is inaccurate. You also have the right to request us to complete the information you believe is incomplete.
- The right to erasure – You have the right to request that we erase your personal data, under certain conditions.
- The right to restrict processing – You have the right to request that we restrict the processing of your personal data, under certain conditions.
- The right to object to processing – You have the right to object to our processing of your personal data, under certain conditions.
- The right to data portability – You have the right to request that we transfer the data that we have collected to another organization, or directly to you, under certain conditions.
- If you make a request, we have one month to respond to you. If you would like to exercise any of these rights, please contact us at our email<EMAIL_ADDRESS>
** Changes to our privacy policy **
We keep our privacy policy under regular review and places any updates on this web page. This privacy policy was last updated on 29 May 2021.

** How to contact us **
If you have any questions about our privacy policy, the data we hold on you, or you would like to exercise one of your data protection rights, please do not hesitate to contact us.
Email us at<EMAIL_ADDRESS>
** How to contact the appropriate authority **
Should you wish to report a complaint or if you feel that we have not addressed your concern in a satisfactory manner, you may contact the Danish Data Protection Agency - Datatilsynet https://www.datatilsynet.dk/english


text:
# docker-machine test environment

Run a Windows 10 VM to test `docker-machine` and nested boot2docker Linux VM's and Linux containers.

## Create environment

Adjust the versions in the Vagrantfile for the Windows VM to be used, VMware Workstation, docker-machine and the docker CLI. Then run

```
vagrant up
```

## Test to create a Docker machine

Now inside the VM - either with `vagrant rdp` or just using the desktop of the visible VM.

First you have to enter a valid VMware Workstation license or start the 30 days trial.

Then open a PowerShell terminal ( `Win`+ `R` and enter `powershell` ).

Then run this command and specify the version of the boot2docker URL.

```
docker-machine -D create -d vmwareworkstation --vmwareworkstation-boot2docker-url=https://github.com/boot2docker/boot2docker/releases/download/v18.06.1-ce/boot2docker.iso test
```

During the creation you can see the boot2docker VM by opening the vmx file:

```
start C:\Users\vagrant\.docker\machine\machines\test\test.vmx
```

If that works try if the shared folder is mounted in the boot2docker VM:

```
docker-machine ssh test ls /Users
```

If that also works try to run a container with a shared folder mounted:

```
docker-machine env test | iex
docker run --rm -v /Users:/Users alpine ls /Users
```

If that also works you found a working combination.

To try another boot2docker version destroy the current docker machine:

```
docker-machine rm -f test
```

## Current status

This combination works fine

- docker-machine 0.13.0 with `vmwareworkstation` driver
- boot2docker 18.06.1

Windows has two third-party docker-machine drivers: [`vmwareworkstation`](https://github.com/pecigonzalo/docker-machine-vmwareworkstation) and [`vmware`](https://github.com/machine-drivers/docker-machine-driver-vmware).

| shared folder status | docker-machine version | driver used | boot2docker |
|--|--|--|--|
| ✅ | 0.13.0 | vmwareworkstation | 18.06.1 |
| ✅ | 0.16.0 | vmwareworkstation | 18.06.1 |
| ❌ | 0.13.0 | vmware | 18.06.1 |
| ❌ | 0.16.0 | vmware | 18.06.1 |

Here is a combination that does not work:

```
choco install -y docker-machine -version 0.16.0 -force
docker-machine -D create -d vmware --vmware-boot2docker-url=https://github.com/boot2docker/boot2docker/releases/download/v18.06.1-ce/boot2docker.iso test
docker-machine ssh test ls /Users
```

This is the same behaviour as with the `vmwarefusion` driver on macOS. The code base for the `vmware` driver is probably very similar.

## See also

- https://github.com/boot2docker/boot2docker/pull/1350

repository: gavioto/docker-windows-box

gha_language: PowerShell

repo_name: gavioto/docker-windows-box


text:
**The project is no more active and replaced by https://github.com/junit-team/testng-engine**

# testng-junit5
Implementation of the JUnit5 Platform TestEngine interface

[![Build Status](https://travis-ci.com/testng-team/testng-junit5.svg?branch=master)](https://travis-ci.com/testng-team/testng-junit5)

## build

Launch Maven to build, format and run integration tests:

`mvn verify -P run-fmt,run-its`


text:
# sdr-flighttracking
This repository is a docker container automation for publicly available tools for aircraft tracking by capturing broadcasted ADS-B signals, tested and made for Raspberry Pi 2/3. All you need is Raspberry Pi 2/3 with latest Raspbian OS and RTL-SDR dongle (RTL283x).
I hope you know what you are doing, just check `docker-compose.yml` and enable/disable desired containers. Please keep in mind that receiving and/or sharing aircraft ADS-B signals may not be completely legal in some countries, always check local laws ;)

By default, it will run three containers: `dump1090-fa` to receive ADS-B signals, `flightaware` to report to flightaware.com and `flightradar24` to report to flightradar24.com. If you just want to receive ADS-B and don't want to share received data, you can just run the `dump1090-fa` container. You will have nice GUI with aircrafts drawn over the map available on port 8080.

## Install docker and docker-compose
```shell
$ curl -fsSL https://get.docker.com -o get-docker.sh
$ sudo sh ./get-docker.sh
$ apt install python-pip
$ pip install docker-compose
```

## Write credentials to env files
```shell
$ cat > flightradar24.env
FLIGHTRADAR24_KEY=xxx
$ cat > flightaware.env
FLIGHTAWARE_USER=user
FLIGHTAWARE_PASSWORD=password
FLIGHTAWARE_FEEDER_ID=feeder-id (if you have it)
```
## Run the whole thing
```shell
$ docker-compose up -d
```

## Check flight tracking
Access your Raspberry Pi on port 8080, and enjoy! Check logs of corresponding containers to know if communication with specific service is fine.

repository: rnurgaliyev/sdr-flighttracking

gha_language: Dockerfile

repo_name: rnurgaliyev/sdr-flighttracking


text:
## About Dashboard Maker
The goal of this project to make it so easy for developer to make an basic dashboard functionality and foucse on the project idea it self

## Goals
- user can have a full dashboard from the start.
- haveing a full employee system for the user from the start.

## License
Dashboard Maker is open-sourced software licensed under the [MIT license](https://opensource.org/licenses/MIT).


text:
---
layout: post
title: Task List 1
---

If you are on the coures, please use GitHub Classroom.
[Click](https://github.com/distributed-systems-wrust/base-l1-2019)

### In the class room

TP:

1. Create async hashing method (compute hash value for any string)
2. Do not use hashCode(), compute it by yourself.
3. Add call in the client

TN:

1. Create async method that takes array of strings and returns longest string
2. Add call in the client


text:
---
# An instance of the Accomplishments widget.
# Documentation: https://sourcethemes.com/academic/docs/page-builder/
widget: accomplishments

# This file represents a page section.
headless: true

# Order that this section appears on the page.
weight: 50

# Note: `` is used to add a 'soft' hyphen in a long heading.
title: 'Accomplishments'
subtitle:

# Date format
#   Refer to https://wowchemy.com/docs/customization/#date-format
date_format: Jan 2006

# Accomplishments.
#   Add/remove as many `item` blocks below as you like.
#   `title`, `organization`, and `date_start` are the required parameters.
#   Leave other parameters empty if not required.
#   Begin multi-line descriptions with YAML's `|2-` multi-line prefix.
item:
- date_end: ""
  date_start: "April - May 2020"
  description: ""
  organization: YantraByte
  title: Basics of Machine Learning and Data Science, YantraByte
  url: ""
- date_end: ""
  date_start: "July 2020"
  description: ""
  organization: Coursera
  title: Java Programming: Solving Problems with Software
- date_end: ""
  date_start: "June - July 2019"
  description: ""
  organization: Softpro India Ltd.
  title: Training on Web development with Django
  url: ""
- date_end: ""
  date_start: "Dec 2019"
  description: ""
  organization: Spoken Tutorial, IIT Bombay
  title: Python 3.4.3
  url: ""
- date_end: ""
  date_start: "June 2019"
  description: ""
  organization: Udemy
  title: Web Design for Web Developers
  url: ""
---


text:
Here is the list of the files in this folder:

| File | Description |
| -------- | ------- |
| Design | MIS and MG documents |
| DevelopmentPlan | Plans for the redevelopment process|
| FinalPresentation | Slides for final demonstration |
| ProblemStatement | Problem statements for the original project before redevelopment |
| SRS | Software Requirement Specifications for the application |
| TestPlan | Plans for testing the application |
| TestReport | The report after testing the system |

repository: ArfaAmer/FaultInOurPong

repo_name: ArfaAmer/FaultInOurPong


text:
# dataframe_sql

![CI](https://github.com/zbrookle/dataframe_sql/workflows/CI/badge.svg)
[![Downloads](https://pepy.tech/badge/dataframe-sql)](https://pepy.tech/project/dataframe-sql)
[![PyPI license](https://img.shields.io/pypi/l/dataframe_sql.svg)](https://pypi.python.org/pypi/dataframe_sql/)
[![PyPI status](https://img.shields.io/pypi/status/dataframe_sql.svg)](https://pypi.python.org/pypi/dataframe_sql/)
[![PyPI version shields.io](https://img.shields.io/pypi/v/dataframe_sql.svg)](https://pypi.python.org/pypi/dataframe_sql/)

## Installation

```bash
pip install dataframe_sql
```

## Usage

In this simple example, a DataFrame is read in from a csv and then using the query
function you can produce a new DataFrame from the sql query.

``` python
from pandas import read_csv
from dataframe_sql import register_temp_table, query

my_table = read_csv("some_file.csv")

register_temp_table(my_table, "my_table")

query("""select * from my_table""")
```

The package currently only supports pandas but there are plans to support dask and
rapids in the future.

## Execution plan

### Values of certain "random" variables
In certain places it was necessary to create functionality that pandas doesn't support
and in those cases, there may be strange variables that you come across in the
execution plan.

This is what their values are:

```python
FALSE_SERIES = Series(data=[False for _ in range(0, dataframe_size)]))
NONE_SERIES = Series(data=[None for _ in range(0, dataframe_size)]))
```

## SQL Syntax
The sql syntax for dataframe_sql is as follows:

Select statement:

```SQL
SELECT [{ ALL | DISTINCT }]
    { [ <expression> ] | <expression> [ [ AS ] <alias> ] } [, ...]
[ FROM <from_item>  [, ...] ]
[ WHERE <bool_expression> ]
[ GROUP BY { <expression> [, ...] } ]
[ HAVING <bool_expression> ]
```

Set operations:

```SQL
<select_statement1>
{UNION [DISTINCT] | UNION ALL | INTERSECT [DISTINCT] | EXCEPT [DISTINCT] | EXCEPT ALL}
<select_statment2>
```

Joins:

```SQL
INNER, CROSS, FULL OUTER, LEFT OUTER, RIGHT OUTER, FULL, LEFT, RIGHT
```

Order by and limit:

```SQL
<set>
[ORDER BY <expression>]
[LIMIT <number>]
```

Supported expressions and functions:
```SQL
+, -, *, /
```
```SQL
CASE WHEN <condition> THEN <result> [WHEN ...] ELSE <result> END
```
```SQL
SUM, AVG, MIN, MAX
```
```SQL
{RANK | DENSE_RANK} OVER([PARTITION BY (<expresssion> [, <expression>...)])
```
```SQL
CAST (<expression> AS <data_type>)
```
*Anything in <> is meant to be some string

*Anything in [] is optional

*Anything in {} is grouped together

### Supported Data Types for cast expressions include:
* VARCHAR, STRING
* INT16, SMALLINT
* INT32, INT
* INT64, BIGINT
* FLOAT16
* FLOAT32
* FLOAT, FLOAT64
* BOOL
* DATETIME64, TIMESTAMP
* CATEGORY
* OBJECT

*Data types in dataframe SQL support many different name for certain datatypes becuase
popular SQL data types are not implemented with common names in pandas and other
dataframe frameworks

**To make this less confusing all data types that are of the same size on the
backend are grouped together in this list

## Issues that come from Pandas

- No native cross join
- No rank over(order by ...)
- No straight aggregation without groupby object
- ** No pandas date object


text:
# Pee-Racer
    ## LET US MAKE URINATING HETEROCLITIC,SILLY AND AMAZING.

# MIT Media India Innovation Lab 2015 Workshop Project

We started working on our track synchronous tools. We were supposed to make a project that is heteroclitic, funny, silly and as well as having synchronicity in it. So first we tried to figure it out by thinking by individually in the group. Ideation of the project was the most difficult part of the project. We all four members of the group came with different ideas about the project. All the members presented different scenarios. Some ideas were funny but not having synchronicity within it. Some ideas were synchronous but not amazing.

After long brainstorm and discussion with our mentors and with my group, we decided to go with craziest one .we came up with one of the most unique, silly, funny idea having synchronicity in it that and we called it that pee racing.

# Ideation Part

    ## For Ideation we use Paper Prototyping

# Animation Video

[![Pee Racers](https://github.com/harshcrop/Pee-Racer/blob/master/assets/video.png)](https://www.youtube.com/watch?v=8Ct3ShK-exU)

# Project Details

When we generally go to toilets to pee, it is really boring. Many of the people of the world might have to find peeing very monotonous. So we came up with idea that let us make peeing more interesting and amazing. So we came up with an idea about pee racing that is really heteroclitic and funny with having synchronicity in it. This idea was exactly like what we needed exactly. That time we had that eureka feeling.

We decided to make a pee racing game.

Idea is simple to make this game. There will be one big screen that is projection on the wall. You will find two different cars on different tracks.  So when both of the people start urinating on some of the parts that will be in the sink their car will move ahead. That is what makes the game interesting and unique.

We made it by using different kinds of tools like Arduino, neodymium magnet,3D printer( to make the turbine), hall effect sensors. We used ceramic plastic to make turbine so that will not get corroded. All these tools are connected in a particular manner. First of all, the turbine will be connected with the neodymium magnets. Neodymium magnet give some sensing effect to the hall effect sensors. Those sensors are connected to the Arduino micro controller board.and Arduino is connected to the projector that can project a game on a screen.

The whole project will be working technically this way. When urine comes up on the turbine, the turbine will rotate. The turbine is having neodymium magnet on it.when it will rotate the magnetic field will change due to a rotation. The change in the magnetic field will be sensed by the hall effect sensors. Hall effect sensors are that transducer which changes outputs in terms of voltage by changing magnetic field. Hall effect sensors will change the given change in voltage to the Arduino. Arduino gives some output that will run through processing software. that processing software will give
us digital output on our projected screen.

  ## Project Images

# Technology

  1. Arduino Uno
  2. Presser Sensor
  3. Hall-effect sensor
  4. 3-D Printer For Parts Printing
  5. Processing Language Game Building
  6. Pow Toons For Create animation Video

# The MIT License (MIT)

Copyright (c) 2015 Harsh Shah

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

 IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

repository: harshcrop/Pee-Racer

repo_name: harshcrop/Pee-Racer


text:
# V-Calendar

> V-Calendar is a clean and lightweight plugin for building attributed calendars in Vue.js.

Visit https://vcalendar.netlify.com for demos and API reference. This plug-in is currently in beta state.

## Features

* Display clean and simple attributed calendars
* Built-in support for various attributes, including
  * highlighted regions
  * dot and bar indicators
  * day content styles (hovered and non-hovered)
* Apply attributes for multiple dates or date ranges (start & end dates)
* Semantic inspired popover navigation panel with month-level attribute indicators
* Date-picker supporting all native v-calendar props/events with various selection modes
  * single date
  * multiple dates
  * date range
* Extensive API with custom slot support
* Responsive and mobile-friendly
  * Handles taps for date selection
  * Handles swipes for month navigation

# Usage

## Calendar

All attributes for a calendar are supplied within an array.

A single attribute may consist of one of each of the following objects: higlights, dots, bars, content style, content hover style. Supply the attributes as an array for the `v-calendar` component.

Here is an example of a simple highlight with a content style.

```html
<v-calendar :attributes='attrs'>
</v-calendar>
```

```javascript
export const {
  data() {
    return {
      attrs: [
        {
          highlight: {
            backgroundColor: 'red',
            borderRadius: '5px'             // Only applied on highlighted end caps
          },
          contentStyle: {
            color: 'white'                  // Contrasts well with the red background
          },
          dates: [
            new Date(),                     // Use dates
            {                               // ...or date ranges
              start: new Date('1/21/83'),   // ...that start on my birthday :)
              end: new Date()
            }
          ]
          customData: myData                // Custom data to reference later
        }
      ]
    }
  }
};
```

The `dates` array specifies dates for which all components of the attribute appear. As you can see, both date objects and date range objects are allowed, where the latter requires start and end dates when needed. For date ranges, null values are allowed for infinite start and end dates, in which case those properties can be ommitted entirely.

### Date Patterns

One really neat feature is that you can target specific dates from within a parent range. This allows for creating complex date patterns that would be difficult to achieve otherwise.

For a simple example, let's say we want to display an attribute on the weekends. To do that, we configure the date like this:

```javascript
...
attrs: [
  {
    highlight: {...},
    contentStyle: {...},
    dates: [
      {
        start: null,      // From the beginning of time...
        end: null,        // ...to the end of time...
        weekdays: [1, 7]  // ...on Sundays and Saturdays
      }
    ]
  }
]
...
```

We can also target other specific day properties, like `days: [6, 15]` for the 6th and 15th of the month, `weeks: [-1]` for the last week of the month and even `ordinalWeekdays: { -1: [1] }` for the last Sunday of the month.

If supplying only a single numerical argument, like `weeks: [-1]`, we can nix the array and simplify to `weeks: -1`.

Additionally, if the date is applied over an infinite time scale (`start` and `end` dates are `null`) like the example before, we can remove `start` and `end` completely. And since our date object is the only one in the array, we can nix the array again like before.

```javascript
...
attrs: [
  {
    highlight: {...},
    contentStyle: {...},
    dates: { weekdays: [1, 7] } // Nice and tidy
  }
],
```

Let's consider another simple example of displaying dot indicators on the last Friday of every other month, starting on January 1st of 2018. We could do so like this.

```javascript
...
  attrs: [
    {
      dot: { backgroundColor: 'red' },
      dates: {
        start: new Date('1/1/2018'),
        monthlyInterval: 2,           // Every other month
        ordinalWeekdays: { [-1]: 6 }  // ...on the last Friday
      }
    }
  ]
...
```

Now, for some reason, we also want to display them on the 15th of every other month, so our first attempt might be to modify the dates to this:

```javascript
...
dates: {
  start: new Date('1/1/2018'),
  monthlyInterval: 2,           // Every other month
  ordinalWeekdays: { [-1]: 6 }, // ...on the last Friday
  days: 15                      // ...and on the 15th? (WRONG!)
},
...
```

But this would be **wrong**, because all component specifiers are conditionally *anded* with each other.

To evaluate a set of conditions *or* another set, we can break the sets of conditions out into an array assigned to the `on` property.

```javascript
...
dates: {
  start: new Date('1/1/2018'),
  monthlyInterval: 2,                 // Every other month
  on: [                               // ...on...
    { ordinalWeekdays: { [-1]: 6 } }, // ...the last Friday
    { days: 15 }                      // ...or the 15th of the month
  ]
}
...
```

Note how we kept the `monthlyInterval` condition outside of the others. Any conditions that should be **anded** with all the others can be extracted out of the array. This prevents unnecessary duplication of conditions within **or** subsets.

Here is a complete reference of date component specifiers available.

| Property | Type | Description | Range |
| --- | --- | --- | --- |
| `days` | Number, Array | Day number from the start or end of the month. | 1 to 31, -1 to -31 |
| `weekdays` | Number, Array | Day of the week. | 1: Sun to 7: Sat |
| `ordinalWeekdays` | Object (key: Number / value: Number, Array) | Weekday ordinal position from the start or end of the month. | key: 1 to 6, -1 to -6 / value: 1: Sun to 7: Sat |
| `weeks` | Number, Array | Week number from the start or end of the month. | 1 to 6, -1 to -6 |
| `months` | Number, Array | Months of the year. | 1 to 12 |
| `years` | Number, Array | Year numbers. | 4-digit integer |
| `dailyInterval` | Number | Interval number of days from the start date (or today when no start date provided). | n > 0 |
| `weeklyInterval` | Number | Interval number of weeks from the start date (or today). | n > 0 |
| `monthlyInterval` | Number | Interval number of months from the start date (or today). | n > 0 |
| `yearlyInterval` | Number | Interval number of years from the start date (or today). | n > 0 |

## Quick Start

[Vue.js](https://vuejs.org) version 2.4+ is required.

### 1 Install via npm

```bash
npm install v-calendar
```

### 2 Import and use VCalendar
```javascript
import Vue from 'vue';
import VCalendar from 'v-calendar';
import 'v-calendar/lib/v-calendar.min.css';

// Use v-calendar, v-date-picker & v-popover components
Vue.use(VCalendar);
```

### 3 Reference in your component templates
```html
<template>
  <v-calendar
    is-double-paned>
  </v-calendar>
  <v-date-picker
    mode='single'
    v-model='selectedValue'>
  </v-date-picker>
</template>
```
```javascript
<script>
export default {
  data() {
    return {
      selectedValue: new Date(),
    };
  },
};
</script>
```

### Or use a CDN
```html
<html>
  <head>
    <meta charset='utf-8'>
    <meta name='viewport' content='width=device-width, initial-scale=1, shrink-to-fit=no'>
    <meta http-equiv='x-ua-compatible' content='ie=edge'>
    <!--1. Link VCalendar CSS-->
    <link rel='stylesheet' href='https://unpkg.com/v-calendar/lib/v-calendar.min.css'>
  </head>
  <body>
    <div id='app'>
      <v-calendar></v-calendar>
      <v-date-picker :mode='mode' v-model='selectedDate'></v-date-picker>
    </div>
    <!--2. Link Vue Javascript-->
    <script src='https://unpkg.com/vue/dist/vue.js'></script>
    <!--3. Link VCalendar Javascript (Plugin automatically installed)-->
    <script src='https://unpkg.com/v-calendar'></script>
    <!--4. Create the Vue instance-->
    <script>
      new Vue({
        el: '#app',
        data: {
          // Data used by the date picker
          mode: 'single',
          selectedDate: null,
        }
      })
    </script>
  </body>
</html>
```

## License
[MIT](http://opensource.org/licenses/MIT)

Copyright (c) 2017-present, Nathan Reyes

repository: ggirodda/v-calendar

repo_name: ggirodda/v-calendar


text:
# TensorFlow Agents

**Reinforcement Learning with TensorFlow**

Agents makes designing, implementing and testing new RL algorithms easier, by
providing well tested modular components that can be modified and extended. It
enables fast code iteration, with good test integration and benchmarking.

To get started, we recommend checking out one of our [tutorials](/tutorials).

## Installation

TF-Agents publishes nightly and stable builds. For a list of releases read the
Releases section. The commands below cover installing
TF-Agents stable and nightly from [pypi.org](https://pypi.org) as well as from a
GitHub clone.

> :warning: If using Reverb (replay buffer), which is very common,
TF-Agents will only work with Linux.

> Note: Python 3.11 requires pygame 2.1.3+.

### Stable

Run the commands below to install the most recent stable release. API
documentation for the release is on
[tensorflow.org](https://www.tensorflow.org/agents/api_docs/python/tf_agents).

```shell
$ pip install --user tf-agents[reverb]

# Use this tag get the matching examples and colabs.
$ git clone https://github.com/tensorflow/agents.git
$ cd agents
$ git checkout v0.17.0
```

If you want to install TF-Agents with versions of Tensorflow or
[Reverb](https://github.com/deepmind/reverb) that are flagged as not compatible
by the pip dependency check, use the following pattern below at your own risk.

```shell
$ pip install --user tensorflow
$ pip install --user dm-reverb
$ pip install --user tf-agents
```

If you want to use TF-Agents with TensorFlow 1.15 or 2.0, install version 0.3.0:

```shell
# Newer versions of tensorflow-probability require newer versions of TensorFlow.
$ pip install tensorflow-probability==0.8.0
$ pip install tf-agents==0.3.0
```

### Nightly

Nightly builds include newer features, but may be less stable than the versioned
releases. The nightly build is pushed as `tf-agents-nightly`. We suggest
installing nightly versions of TensorFlow (`tf-nightly`) and TensorFlow
Probability (`tfp-nightly`) as those are the versions TF-Agents nightly are
tested against.

To install the nightly build version, run the following:

```shell
# `--force-reinstall helps guarantee the right versions.
$ pip install --user --force-reinstall tf-nightly
$ pip install --user --force-reinstall tfp-nightly
$ pip install --user --force-reinstall dm-reverb-nightly

# Installing with the `--upgrade` flag ensures you'll get the latest version.
$ pip install --user --upgrade tf-agents-nightly
```

### From GitHub

After cloning the repository, the dependencies can be installed by running `pip
install -e .[tests]`. TensorFlow needs to be installed independently: `pip
install --user tf-nightly`.

## Contributing

We're eager to collaborate with you! See
[`CONTRIBUTING.md`](https://github.com/tensorflow/agents/blob/master/CONTRIBUTING.md)
for a guide on how to contribute. This project adheres to TensorFlow's
[code of conduct](https://github.com/tensorflow/agents/blob/master/CODE_OF_CONDUCT.md).
By participating, you are expected to uphold this code.

## Releases

TF Agents has stable and nightly releases. The nightly releases are often fine
but can have issues due to upstream libraries being in flux. The table below
lists the version(s) of TensorFlow that align with each TF Agents' release.
Release versions of interest:

  * 0.16.0 is the first version to support Python 3.11.
  * 0.15.0 is the last release compatible with Python 3.7.
  * If using numpy < 1.19, then use TF-Agents 0.15.0 or earlier.
  * 0.9.0 is the last release compatible with Python 3.6.
  * 0.3.0 is the last release compatible with Python 2.x.

Release | Branch / Tag                                               | TensorFlow Version | dm-reverb Version
------- | ---------------------------------------------------------- | ------------------ | -----------
Nightly | [master](https://github.com/tensorflow/agents)             | tf-nightly         | dm-reverb-nightly
0.17.0  | [v0.17.0](https://github.com/tensorflow/agents/tree/v0.17.0) | 2.13.0           | 0.12.0
0.16.0  | [v0.16.0](https://github.com/tensorflow/agents/tree/v0.16.0) | 2.12.0           | 0.11.0
0.15.0  | [v0.15.0](https://github.com/tensorflow/agents/tree/v0.15.0) | 2.11.0           | 0.10.0
0.14.0  | [v0.14.0](https://github.com/tensorflow/agents/tree/v0.14.0) | 2.10.0           | 0.9.0
0.13.0  | [v0.13.0](https://github.com/tensorflow/agents/tree/v0.13.0) | 2.9.0            | 0.8.0
0.12.0  | [v0.12.0](https://github.com/tensorflow/agents/tree/v0.12.0) | 2.8.0            | 0.7.0
0.11.0  | [v0.11.0](https://github.com/tensorflow/agents/tree/v0.11.0) | 2.7.0            | 0.6.0
0.10.0  | [v0.10.0](https://github.com/tensorflow/agents/tree/v0.10.0) | 2.6.0            |
0.9.0   | [v0.9.0](https://github.com/tensorflow/agents/tree/v0.9.0) | 2.6.0              |
0.8.0   | [v0.8.0](https://github.com/tensorflow/agents/tree/v0.8.0) | 2.5.0              |
0.7.1   | [v0.7.1](https://github.com/tensorflow/agents/tree/v0.7.1) | 2.4.0              |
0.6.0   | [v0.6.0](https://github.com/tensorflow/agents/tree/v0.6.0) | 2.3.0              |
0.5.0   | [v0.5.0](https://github.com/tensorflow/agents/tree/v0.5.0) | 2.2.0              |
0.4.0   | [v0.4.0](https://github.com/tensorflow/agents/tree/v0.4.0) | 2.1.0              |
0.3.0   | [v0.3.0](https://github.com/tensorflow/agents/tree/v0.3.0) | 1.15.0 and 2.0.0.  |

## Principles

This project adheres to
[Google's AI principles](https://github.com/tensorflow/agents/blob/master/PRINCIPLES.md).
By participating, using or contributing to this project you are expected to
adhere to these principles.

## Citation

If you use this code, please cite it as:

```
@misc{TFAgents,
  title = {{TF-Agents}: A library for Reinforcement Learning in TensorFlow},
  author = {Sergio Guadarrama and Anoop Korattikara and Oscar Ramirez and
     Pablo Castro and Ethan Holly and Sam Fishman and Ke Wang and
     Ekaterina Gonina and Neal Wu and Efi Kokiopoulou and Luciano Sbaiz and
     Jamie Smith and Gábor Bartók and Jesse Berent and Chris Harris and
     Vincent Vanhoucke and Eugene Brevdo},
  howpublished = {\url{https://github.com/tensorflow/agents}},
  url = "https://github.com/tensorflow/agents",
  year = 2018,
  note = "[Online; accessed 25-June-2019]"
}
```

repository: tensorflow/agents

gha_language: Python

repo_name: tensorflow/agents


text:
# Welcome

This is a collection of Scripts used to extend the functionality of Jamf Pro’s management capabilities.

These Scripts are **not official Jamf products** and are provided without warranty. Do **not** contact Jamf Support to ask for help using these Scripts as scripting is ***unsupported*** except as a paid engagement with Professional Services.

There are four categories of Scripts on this GitHub: API Scripts, macOS Scripts, Extension Attributes, and Snippets. Each category is in a directory named the same, and there is a README inside each directory detailing its contents.

***API Scripts*** can be run on anything that can run Bash. They do not need to be deployed to computers enrolled in Jamf Pro.

***macOS Scripts*** are written to be deployed via a Jamf Pro Policy to a macOS device.

***Extension Attributes*** are used to collect additional Inventory information for computers enrolled in Jamf Pro.

***Snippets*** are little pieces of code that might be useful to integrate into other projects. By themselves, they're usually pretty pointless.

repository: iMatthewCM/Jamf-Scripts

gha_language: Shell

repo_name: iMatthewCM/Jamf-Scripts


text:
# Lacking Game Engine [![Go Report Card](https://goreportcard.com/badge/github.com/mokiat/lacking)](https://goreportcard.com/report/github.com/mokiat/lacking) [![Go Reference](https://pkg.go.dev/badge/github.com/mokiat/lacking.svg)](https://pkg.go.dev/github.com/mokiat/lacking)

![logo](logo.png)

A 3D game engine / game framework that lacks a lot of features (hence the name).

This project aims to replace [go-whiskey](https://github.com/mokiat/go-whiskey), which has now been archived, by going through a more pragmatic development approach.

This repository is a hobby project of mine that aims to support other hobby projects like [rally-mka](https://github.com/mokiat/rally-mka). There are no plans at the moment to have this become a serious project and as such it lacks proper testing and code cleanliness.

**WARNING:** If you do decide to base a project of yours off of this repository make sure you depend on a specific tagged version, as the `master` branch or new tagged versions are unlikely to be backward compatible.

repository: mokiat/lacking

gha_language: Go

repo_name: mokiat/lacking


text:
# pgyer plugin

[![fastlane Plugin Badge](https://rawcdn.githack.com/fastlane/fastlane/master/fastlane/assets/plugin-badge.svg)](https://rubygems.org/gems/fastlane-plugin-pgyer)

## modified the official fastlane plugin

This project is a [fastlane](https://github.com/fastlane/fastlane) plugin. To get started with `fastlane-plugin-pgyer`, add it to your project by running:

```bash
fastlane add_plugin pgyer
```

## About pgyer

This pluginin allow you distribute app automatically to [pgyer beta testing service](https://www.pgyer.com) in fastlane workflow.

## Modified return value

modified the return value to be the Unique URL of the package, like http://www.pgyer.com/app/qrcodeHistory/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

```


text:
---
layout: default
---

# 1995-12-01 - UK Labour Party policy on encryption

## Header Data

From: "Richard Martin"<EMAIL_ADDRESS>To<EMAIL_ADDRESS>Message Hash: 93fd8e1a9c760e80aac94e504f27838e180cdf1214eeae7446de18085710a07a

Message ID<EMAIL_ADDRESS>Reply To: _N/A_

UTC Datetime: 1995-12-01 00:31:39 UTC

Raw Date: Fri, 1 Dec 1995 08:31:39 +0800

## Raw message

```
{% raw  %}From: "Richard Martin"<EMAIL_ADDRESS>Date: Fri, 1 Dec 1995 08:31:39 +0800
To<EMAIL_ADDRESS>Subject: UK Labour Party policy on encryption
Message-ID<EMAIL_ADDRESS>MIME-Version: 1.0
Content-Type: text/plain

-----BEGIN PGP SIGNED MESSAGE-----

[see http://www.poptel.org.uk/labour-party/policy/info-highway/content.html
for full text]

> It is important that privacy is rigorously protected over the
> new networks, for both personal and commercial reasons. We do
> not accept the "clipper chip" argument developed in the United
> States for the authorities to be able to swoop down on any
> encrypted message at will and unscramble it.
>
> The only power we would wish to give to the authorities, in
> order to pursue a defined legitimate anti-criminal purpose,
> would be to enable decryption to be demanded under judicial
> warrant (in the same way that a warrant is required in order
> to search someone's home).
>
> Attempts to control the use of encryption technology are wrong
> in principle, unworkable in practice, and damaging to the
> long-term economic value of the information networks. There is
> no fundamental difference between an encrypted file and a
> locked safe. A safe may be effectively impregnable in that the
> effort taken to open it would destroy the contents. An
> encryption algorithm, similarly, may be effectively
> unbreakable.

Notes.

I've taken this entirely out of context*; cpunks will be glad to
find that the same document manages to hit at least two of the
horsemen of the infocalypse. A few paragraphs earlier is the
standard "must protect women, children, and ethnic minorities"
(paraphrased) argument.

Labour seems likely to form the next government. And even if they
don't, their stated policy might be a good thing to take to the
Tories, saying something along the lines of, "Labour has a clue,
where's yours?" [Note that I was unable to find the Tory point
of view.]

I found this stuff from Anne Campbell MP's home page.

http://www.worldserver.pipex.com/home/anne.campbell

* for brevity's sake.

richard

Richard Martin                           I DON'T SPEAK FOR ALIAS|WAVEFRONT
Alias|Wavefront - Toronto Office [Co-op Software Developer, Games Team]
rmartin<EMAIL_ADDRESS>    http://www.io.org/~samwise
Trinity College UofT ChemPhysCompSci 9T7+PEY=9T8 Shad Valley Waterloo 1992

-----BEGIN PGP SIGNATURE-----
Version: 2.6.2

iQCVAwUBML5COB1gtCYLvIJ1AQFZ1wP+Ox2fbOKsEbtl06GZugpB8ecHl5cGYHSQ
7sBro0QFr2v4KZOV6X435LXmAqSV8ML+BmVsT/1bnfUzsxeIQErhC0q1kQeA+hAX
h21SYfW6hN3h2L9aOtLr/1FDZkKAcXGDNFkvq/weKQdxCbtTw0Nl1ZQym3oaYdhT
MrjKi85fU44=
=CeYH
-----END PGP SIGNATURE-----

{% endraw %}
```

## Thread

+ Return to [December 1995](/archive/1995/12)

+ Return to "["Richard Martin" <rmartin@aw.sgi.com>](/authors/richard_martin_rmartin_at_aw_sgi_com_)"

+ 1995-12-01 (Fri, 1 Dec 1995 08:31:39 +0800) - UK Labour Party policy on encryption - _"Richard Martin"<EMAIL_ADDRESS>


text:
# Proccess-Authorization [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

A Simple tool to prevent others using your personal software or tools on regular startup.

## Install

- Download the current master branch.

  - Using **Git CLI**..

    ```shell
    > git clone https://github.com/raja-ravi-prakash/Proccess-Authorization
    ```

  - Use the provided [download](https://github.com/raja-ravi-prakash/Proccess-Authorization/archive/master.zip) option from github.

  - And you also need [GCC Compiler](https://sourceforge.net/projects/mingw/) to run C language parts, For instructions visit [this](https://www.youtube.com/watch?v=Ubfgi4NoTPk).

## How To

- From the root folder run **build.bat** which prompts you for the no.of programs names of them which you want to restrict.
- And it prompts you for a password, so you can use it later to deactivate current running session.

  ```shell
  > build # Example
    Enter no.of Programs : 3 # 3 Programs are given

    Enter Program names followed by Extensions
    Enter Program Name : Bluestacks.exe # valid extensions with valid names should be mentioned.
    Enter Program Name : chrome.exe
    Enter Program Name : notepad.exe
    # And this creates a dist folder with required files...

    Create Password: <your password>
  ```

  **Note :**

  - If you don't know the valid name with extension of your program (or) software quick way is , run it in your system manually and
    run this command in **cmd** (or) **powershell** .

    ```shell
    > tasklist
    ```

  - The above command shows you all the running process try and find your own.

  - Another way is look in the installation directory of your software you can find it .

## Configure

- If you want to reset and create a new build, below command does that for you. so, after for regeneration use **build**.

  ```shell
  > clean
  ```

- That's it on regular boot no one can run these files, for you to run it below can do that.

  ```shell
  > %ronin%__s <your_password> # without <>
  ```

- And fell free to contribue (or) create an issue if something doesn't work as expected.

## LICENSE

    MIT License

    Copyright (c) 2020 Raja Ravi Prakash

    Permission is hereby granted, free of charge, to any person obtaining a copy
    of this software and associated documentation files (the "Software"), to deal
    in the Software without restriction, including without limitation the rights
    to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
    copies of the Software, and to permit persons to whom the Software is
    furnished to do so, subject to the following conditions:

    The above copyright notice and this permission notice shall be included in all
    copies or substantial portions of the Software.

     IN NO EVENT SHALL THE
    AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
    OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
    SOFTWARE.


text:
---
breadcrumbs:
- - /nativeclient
  - Native Client
- - /nativeclient/reference
  - '4: Reference'
page_name: research-papers
title: Research Papers
---

Google's Native Client engineering team has published the following papers about
Native Client technology:

*   [Native Client: A Sandbox for Portable, Untrusted x86 Native
            Code](http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en/us/pubs/archive/34913.pdf),
            Appeared in the 2009 IEEE Symposium on Security and Privacy
            (<http://oakland09.cs.virginia.edu>), May 2009
*   [Adapting Software Fault Isolation to Contemporary CPU
            Architectures](http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en/us/pubs/archive/35649.pdf).
            Appeared in the 19th USENIX Security Symposium
            (<http://www.usenix.org/events/sec10/index.html>), August 2010;
            originally posted February 2010, revised June 2010
*   [PNaCl: Portable Native Client
            Executables](https://docs.google.com/a/chromium.org/viewer?a=v&pid=sites&srcid=Y2hyb21pdW0ub3JnfGRldnxneDo3ZmQ2NjI2NjFkM2EzYjFl),
            white paper, February 2010
*   [Language-Independent Sandboxing of Just-In-Time Compilation and
            Self-Modifying
            Code](http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en/us/pubs/archive/37204.pdf),
            Appear in the 2011 Conference on Programming Language Design and
            Implementation (<http://pldi11.cs.utah.edu>) June 2011.

repository: iridium-browser/iridium-browser

repo_name: iridium-browser/iridium-browser


text:
books
=====

command-line book inventory management

**WARNING**: this software is in development. Don't use it unless you know what you're doing. Which is also why I don't write installation instructions, yet.

[![Build Status](https://travis-ci.org/stchris/books.png?branch=master)](https://travis-ci.org/stchris/books)

Database
--------

When adding your first entry, a Sqlite database will be created in *~/.books/books.db*

Example usage
-------------

    $ books add
    Author: Frank Herbert
    Title: Dune
    Comments:

    $ books add
    Author: Stephen King
    Title: The Stand
    Comments:

    $ books ls
    {#1: 'Dune' by 'Frank Herbert', comments: }
    {#2: 'The Stand' by 'Stephen King', comments: }

    $ books ls sta
    {#2: 'The Stand' by 'Stephen King', comments: }

    $ books ls herb
    {#1: 'Dune' by 'Frank Herbert', comments: }

    $ books help
    USAGE: books COMMAND argument1 argument2 ...
    Available commands:
        ls - list books, pass search terms as arguments
        add - add books
        del - delete book (prompts for id)
        web - starts the built-in web server on port 8765
        help - display this text

    $ books web
    2015/05/02 13:32:49 Web server listening at http://<IP_ADDRESS>:8765
    2015/05/02 13:32:49 Press ^C to stop

repository: stchris/books

repo_name: stchris/books


text:
---
layout: post
title: Interfacing with an FPGA from Linux on ZYNQ
---

[Mitchell Orsucci](https://www.hackster.io/morsucci) has proposed method to control an FPGA from the Linux OS, where you are dynamically reconfigure the FPGA from Linux user-space. You can read more [here](https://www.hackster.io/morsucci/interfacing-with-an-fpga-from-linux-on-zynq-90ea3e)

![](https://hackster.imgix.net/uploads/attachments/346596/photo_sep_05_8_12_16_am_6v8Kkds7lH.jpg?auto=compress%2Cformat&w=900&h=675&fit=min)

This project represents the control of an FPGA from Linux user-space. This project utilizes a Digilent PmodOLED_RGB and a Digilent PmodCDC1, as well as the available inputs and outputs on the ArtyZ7-20 board.

The ArtyZ7-20 contains a Xilinx Zynq chip which contains a 650Mhz ARM dual-core processor as well as some FPGA fabric. An FPGA design can be instantiated using Xilinx Vivado. Additionally, using Xilinx Petalinux, a Linux kernel and root file-system can be obtained for the ARM processor. This allows us to run the Linux operating system, but still use the functionality of the FPGA. FPGA configurations can be loaded and changed dynamically without interrupting or crashing the running kernel.

![](https://hackster.imgix.net/uploads/attachments/346620/artydemobd_goKFeGnlyx.png?auto=compress%2Cformat&w=680&h=510&fit=max)

The code is available on [Github](https://github.com/mitchellorsucci/ArtyZ720)

gha_language: SCSS


text:
# Robot fleet simulation using concurrent gazebo instances

robot_fleet rospackage within this repo enables co-ordination between multiple [gazebo](http://gazebosim.org/tutorials?tut=ros_overview) instances to simulate a fleet of robots. We can compound this further by having multiple robots within same simulation as mentioned [here](https://answers.ros.org/question/41433/multiple-robots-simulation-and-navigation/). However, this current version expects a single robot with physics in each simulation, and uses [rosbridge](http://wiki.ros.org/rosbridge_suite) server/clients to communicate between the simulation instances.

## Requirements
* [ROS Melodic](http://wiki.ros.org/melodic) - Other ROS versions have not been tested.
* [Colcon](https://colcon.readthedocs.io) - Tested on colcon. catkin confirmed working on melodic only.
* [Gazebo9](http://gazebosim.org/blog/gazebo9) - Gazebo simulator.
* [Colcon bundle](https://github.com/colcon/colcon-bundle) - Enables packaging the application for running on RoboMaker.
* [boto3](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html) - AWS SDK for Python
* [awscli](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html) - AWS CLI setup on local machine with appropriate [credentials](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html)

## Usage

### Local Execution
This section is to run a single instace of the above application. This will NOT kick off multiple robots, but is a good way to see whats under the hood in this application. Has been testing on colcon/catkin on ROS Melodic, Ubuntu 18.04.
```
git clone https://github.com/aws-samples/multi-robot-fleet-sample-application.git
cd multi-robot-fleet-sample-application
./setup/ros_setup.bash
```

Running this file performs
   * new rosdep definition for - roslibpy
   * rosdep update
   * rosws update - pulls in husky and aws-robomaker-small-warehouse-world repositories
   * Applies minor patches reqd to work with AWS RoboMaker
   * rosdep install -y —from-path <path_of_repos>
   * colcon build; source install/setup.bash
   * colcon bundle

Set the appropriate environement variables required for the application
```
export ROBOT_NAME=server  # unique robot name
export ROSBRIDGE_STATE=SERVER  # SERVER or CLIENT
export ROSBRIDGE_IP=localhost  # localhost for SERVER. IP of rosbridge for CLIENT
export START_X=0  # start location of robot
export START_Y=0
export START_YAW=0
export HUSKY_REALSENSE_ENABLED=true
export HUSKY_LMS1XX_ENABLED=true
export USE_CUSTOM_MOVE_OBJECT_GAZEBO_PLUGIN=true  # set to true if you use custom plugin to move robot. False uses regular gazebo rostopics
```

To see the application running on your local machine, run the following command.
```
source simulation_ws/install/setup.bash
roslaunch robot_fleet robot_fleet_rosbridge.launch gui:=true
```

### Cloud Execution

To setup multi robot fleet simulation on your AWS account, run the following command **AFTER** running the **Local execution** section. The local ROS application need not be running for this.

```
./setup/aws_setup_sample.bash
```

Running this file performs

   * Deploying the cloudformation stack to setup the AWS environment
   * Creates and uploads the bundle file to AWS
   * Kicks off a lambda function to start the multi-robot-fleet on AWS
   * Picks the corresponding params for the simulations from setup/fleetLauncherApp/fleetLauncherLambda/event.json

![fleet_in_robomaker](readmeimages/fleet_robomaker.png)

*Figure 1. A multiple robot fleet running in AWS RoboMaker.*

#### Setting the parameters in the cloud-based simulation

To update the parameters, such as robot name (ROBOT) and starting positions, you can edit the event.json file at **fleetLaucherApp/fleetLauncherLambda/event.json**.

Here is the example JSON:

```
{
    "robots": [
        {
            "name": "robot1",
            "environmentVariables": {
                "START_X": "2",
                "START_Y": "2",
                "START_YAW": "3.143",
                "HUSKY_REALSENSE_ENABLED": "true",
                "HUSKY_LMS1XX_ENABLED": "true",
                "USE_CUSTOM_MOVE_OBJECT_GAZEBO_PLUGIN":"true"
            },
            "packageName": "robot_fleet",
            "launchFile": "robot_fleet_rosbridge.launch"
        }
        ...
    ],
    "server": {
        "name": "SERVER",
        "environmentVariables": {
            "START_X": "0",
            "START_Y": "0",
            "START_YAW": "0",
            "HUSKY_REALSENSE_ENABLED": "true",
            "HUSKY_LMS1XX_ENABLED": "true",
            "USE_CUSTOM_MOVE_OBJECT_GAZEBO_PLUGIN":"true"
        },
        "packageName": "robot_fleet",
        "launchFile": "robot_fleet_rosbridge.launch"
      }
  }
```

There are a group of optional attributes you can set if a ROSBridge server is already running or you want to customize the application details.

```
  "serverIP": "<ALREADY_RUNNING_ROSBRIDGE_SERVER>",
  "simulationJobParams": {
      "vpcConfig": {
          "subnets": [],
          "securityGroups": []
      },
      "iamRole": "<PRECREATED_IAM_ROLE>",
      "outputLocation": {
          "s3Bucket": "",
          "s3Key": ""
      }
  },
  "simulationApplicationArn": "<PRECREATED_SIMULATION_APPLICATION>"
```

#### Serverless Application Deployment

If you want to run the simulation launcher lambda function in AWS and link to an automated testing workflow, follow the README instructions [here](setup/fleetLauncherApp/README.md).

## How it works

* One gazebo instance acts as rosbridge_server and others as rosbridge clients
* We use rosbridge to pass current position information of robots in all gazebo instances
* For all other gazebo instances, we dynamically create and remove static robot models that represent the robot running its actual environment. We use gazebo plugin to move the static model in sync with its actual position information that we gather from rosbridge
* With RoboMaker, its easy to spin up the simulation with ROSBRIGE_SERVER, wait for its IP and pass that along for rest of CLIENT robots.

## Architecture

![multibot_image](readmeimages/multibot.png)

![cloud_architecture](readmeimages/cloud_architecture.png)

## Workshop for the Application

Follow the workshop built around this application by visiting our [workshop page](https://stage.robomakerworkshops.com/ws/multi_robot_fleet_simulations)

## Known issues

This package does not perform time synchronization, and simulation update lockstep between gazebo instances. Best results have been seen with homogeneous software stacks running on the gazebo instances.

## Security

See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information.

## License

This library is licensed under the MIT-0 License. See the LICENSE file.


text:
# Perl Post Processing Examples

## [pps_simple.pl](pps_simple.pl)

A simple example to show how to create and use post processing scripts. This script adds line numbers to the report before displaying it.

---

## [pps_DeckAccess-SSCtoCommonName.pl](pps_DeckAccess-SSCtoCommonName.pl)

This post-processing script, when assigned to a DeckAccess report which was computed to a Satellite Deck, will replace the SSC number with the common name as listed in the %ProgramData%\AGI\STK 12\Databases\Satellite\stkSatDbAll.sd file.

---

## [pps_LLAPos2GA.pl](pps_LLAPos2GA.pl)

This post-processing script will create an STK formatted Waypoint File (*.ga) from LLA data.

---

## [pps_Ordered_Lighting_Times.pl](pps_Ordered_Lighting_Times.pl)

This reformats the Lighting Times data providers into a single, time ordered report of all conditions (as opposed to each condition in a separate report section).

---

## [pps_TimeLLA-to-ephemeris.pl](pps_TimeLLA-to-ephemeris.pl)

This is an STK Post Processing script that works against any report where the first four columns are Time (UTCG), Lat(dec deg), Long (dec deg), Alt(m). Altitude is optional and will default to 0. The output is an STK formatted ephemeris file.

---

repository: AnalyticalGraphicsInc/STKCodeExamples

gha_language: Jupyter Notebook

repo_name: AnalyticalGraphicsInc/STKCodeExamples


text:
# Linknest Collection JSON

A plugin making every collection on Linknest avaliable as JSON. Making it easy to fetch from other apps. Making it easier to follow them from other apps even if you're not using the service.

#### ⚡ [Styleguides collection](https://linknest.com/urban/styleguides/)

#### ✅ [Styleguides collection JSON](https://linknest.com/urban/styleguides/json)

## Notes and usage

For use on Linknest but I'm sure you can fit this in your project aswell, or just pick pieces you want/like.

Usage:

1. ``composer install``
2. Install the plugin in wp-admin

## Related

### [Collection Feeds](https://github.com/Linknest/collectionfeeds)
A plugin giving the collections on Linknest custom RSS feeds. Making it easier to follow them even if you're not using the service.

## Linknest?

Linknest is a simple bookmarking service. Visual bookmarking made simple. (It's free [to use](https://linknest.com/login/?q=/login&action=register))

# License

MIT © [Urban Sanden](https://github.com/urre)

repository: Linknest/collectionjson

repo_name: Linknest/collectionjson