text:
---
sidebar_label: Query your Rollup
---

# 💬 Say gm world

Now, we're going to get our blockchain to say `gm world!` - in order to do so
you need to make the following changes:

- Modify a protocol buffer file
- Create a keeper query function that returns data

Protocol buffer files contain proto RPC calls that define Cosmos SDK queries
and message handlers, and proto messages that define Cosmos SDK types. The RPC
calls are also responsible for exposing an HTTP API.

The `Keeper` is required for each Cosmos SDK module and is an abstraction for
modifying the state of the blockchain. Keeper functions allow us to query or
write to the state.

## ✋ Create your first query

**Open a new terminal instance that is not the
same that you started the chain in.**

In your new terminal, `cd` into the `gm` directory and run this command
to create the `gm` query:

```bash
ignite scaffold query gm --response text -y
```

Response:

```bash
modify proto/gm/gm/query.proto
modify x/gm/client/cli/query.go
create x/gm/client/cli/query_gm.go
create x/gm/keeper/grpc_query_gm.go

🎉 Created a query `gm`.
```

What just happened? `query` accepts the name of the query (`gm`), an optional
list of request parameters (empty in this tutorial), and an optional
comma-separated list of response field with a `--response` flag (`text` in this
tutorial).

Navigate to the `gm/proto/gm/gm/query.proto` file, you’ll see that `Gm` RPC has
been added to the `Query` service:

<!-- markdownlint-disable MD010 -->
<!-- markdownlint-disable MD013 -->
```protobuf title="gm/proto/gm/gm/query.proto"
service Query {
  rpc Params(QueryParamsRequest) returns (QueryParamsResponse) {
    option (google.api.http).get = "/gm/gm/params";
  }
	rpc Gm(QueryGmRequest) returns (QueryGmResponse) {
		option (google.api.http).get = "/gm/gm/gm";
	}
}
```
<!-- markdownlint-enable MD013 -->
<!-- markdownlint-enable MD010 -->

The `Gm` RPC for the `Query` service:

- is responsible for returning a `text` string
- Accepts request parameters (`QueryGmRequest`)
- Returns response of type `QueryGmResponse`
- The `option` defines the endpoint that is used by gRPC to generate an HTTP API

## 📨 Query request and response types

In the same file, we will find:

- `QueryGmRequest` is empty because it does not require parameters
- `QueryGmResponse` contains `text` that is returned from the chain

```protobuf title="gm/proto/gm/gm/query.proto"
message QueryGmRequest {
}

message QueryGmResponse {
  string text = 1;
}
```

## 👋 Gm keeper function

The `gm/x/gm/keeper/grpc_query_gm.go` file contains the `Gm` keeper function that
handles the query and returns data.

<!-- markdownlint-disable MD013 -->
<!-- markdownlint-disable MD010 -->
```go title="gm/x/gm/keeper/grpc_query_gm.go"
func (k Keeper) Gm(goCtx context.Context, req *types.QueryGmRequest) (*types.QueryGmResponse, error) {
	if req == nil {
		return nil, status.Error(codes.InvalidArgument, "invalid request")
	}
	ctx := sdk.UnwrapSDKContext(goCtx)
	_ = ctx
	return &types.QueryGmResponse{}, nil
}
```
<!-- markdownlint-enable MD010 -->
<!-- markdownlint-enable MD013 -->

The `Gm` function performs the following actions:

- Makes a basic check on the request and throws an error if it’s `nil`
- Stores context in a `ctx` variable that contains information about the
environment of the request
- Returns a response of type `QueryGmResponse`

Currently, the response is empty and you'll need to update the keeper function.

Our `query.proto` file defines that the response accepts `text`. Use your text
editor to modify the keeper function in `gm/x/gm/keeper/grpc_query_gm.go` .

<!-- markdownlint-disable MD013 -->
<!-- markdownlint-disable MD010 -->
```go title="gm/x/gm/keeper/grpc_query_gm.go"
func (k Keeper) Gm(goCtx context.Context, req *types.QueryGmRequest) (*types.QueryGmResponse, error) {
	if req == nil {
		return nil, status.Error(codes.InvalidArgument, "invalid request")
	}
	ctx := sdk.UnwrapSDKContext(goCtx)
	_ = ctx
	return &types.QueryGmResponse{Text: "gm world!"}, nil
}
```
<!-- markdownlint-enable MD010 -->
<!-- markdownlint-enable MD013 -->

## 🟢 Start your Sovereign Rollup

We have a handy `init.sh` found in this repo
[here](https://github.com/celestiaorg/devrel-tools).

We can copy it over to our directory with the following commands:

```sh
# From inside the `gm` directory
cd ..
git clone https://github.com/celestiaorg/devrel-tools
cp devrel-tools/gm/init.sh gm/
cd gm/
```

This copies over our `init.sh` script to initialize our
gm rollup.

You can view the contents of the script to see how we
initialize the gm rollup.

:::danger caution

Before starting our rollup, we'll need to find and change
`FlagIAVLFastNode` to `FlagDisableIAVLFastNode`:

```go title="gm/cmd/gmd/cmd/root.go"
baseapp.SetIAVLDisableFastNode(cast.ToBool(appOpts.Get(server.FlagDisableIAVLFastNode))),
```

Also, if you are on macOS, you will need to install md5sha1sum:

```sh
brew install md5sha1sum
```

:::

You can initialize the script with the following command:

```sh
bash init.sh
```

With that, we have kickstarted our `gmd` network!

The `query` command has also scaffolded
`x/gm/client/cli/query_gm.go` that
implements a CLI equivalent of the gm query and mounted this command in
`x/gm/client/cli/query.go`.

In a separate window, run the following command:

```bash
gmd q gm gm
```

We will get the following JSON response:

```bash
text: gm world!
```

![4.png](/img/gm/4.png)

Congratulations 🎉 you've successfully built your first rollup and queried it!

repository: pynchmeister/docs

repo_name: pynchmeister/docs


text:
Helpful concepts
================

Here are a few concepts I've found helpful when learning how WebGL works.

The context object
------------------

The entire API is methods and constants on the WebGL context object, which you should call `gl`.
The API is not object-oriented, even though you do produce objects. The objects that get produced
don't have any useful methods. Instead, you need to pass them as argument back to `gl`.  These are written with all caps. You never use their value for anything,
you just pass them as arguments to `gl` methods.

In other OpenGL API implementations besides WebGL, the methods and constants are named more or less
the same thing, but instead of a context object, they have prefixes. So they look like
`glCreateShader` and `GL_VERTEX_SHADER`. That might help find information about the API online, but
keep in mind there might be subtle differences from the WebGL version.

Client and server
-----------------

You can think of everything that runs on the JavaScript side as being a "client", and everything
that happens behind the WebGL API as being a "server". It's generally (but not 100%) true that
the client runs on the CPU, and the server runs on the GPU. I'll usually refer to the client as
"JavaScript", and the server as "WebGL" - it should be clear from context what I actually mean.

When it comes to efficiency, both the client and the server take time to run, but where you really
want to look for bottlenecks is in client/server communications, i.e., WebGL API calls. If there's
a call that you're making every frame of your game, when you could just be making it just once,
it's probably worth changing it.

On the other hand, if you're not used to working with the GPU, you should probably avoid making
optimizations right away. It can be pretty counterintuitive what works and what doesn't, so I
suggest getting some experience first.

WebGL as a state machine
------------------------

There are very few methods that actually tell WebGL to draw anything, and looking at the draw call
itself doesn't really tell you what's being drawn. That all depends on the large amount of state
that WebGL has at any time, and there are a bunch of methods to set the state.

There are several getter and setter methods. They don't really appear in pairs, though, and often a
single getter corresponds to several setters, and you specify what you want to get or set with one
of those constants. The getters always start with "get", and the setters don't have a prefix. This
is probably because you're usually setting a lot more frequently than getting.

	gl.clearColor(0, 0, 0, 1)  // set the clear color
	var c = gl.getParameter(gl.COLOR_CLEAR_VALUE)  // get the clear color

Because of efficiency concerns, many people will keep a client-side copy of the relevent parts of
the WebGL state at any time, to avoid having to call the getters. But again, I wouldn't worry about
that yet.

Shaders
-------

I'm just saying this because it confused me for a long time: shaders do not, in general, have to do
with shading.

Shaders are programs that run when you call a draw method, in order to determine where the drawn
objects appear and what color they are. Shading is part of that, of course, but just one part. I
think the name is a bit of misnomer. To create shaders, you need to pass their source code as a
string to WebGL.

Shaders are written in GLSL ES (the OpenGL ES Shading Language), not JavaScript. GLSL is kind of a
cool language. It's really limited and it takes some getting used to, but it has a nifty syntax
for doing what it does.

repository: Eroid/glprog

branch_name: refs/heads/master

extension: md

license_type: permissive

repo_name: Eroid/glprog


text:
---
layout: post
title: Stories We Tell
summary: \
Stories stick. Our brains love stories. Simple and engaging, stories are easy \
to understand and remember. Stories blind us, though, to the complexities of \
the world, and in ourselves. By accepting a simple narrative, we look at only \
a single path through chaos.
status: draft
tags:
- opinion
- rationality
- thought
---

Stories stick. Our brains love stories. Simple and engaging, stories are easy \
to understand and remember. Stories blind us, though, to the complexities of \
the world, and in ourselves. By accepting a simple narrative, we look at only \
a single path through chaos.

<!-- EXCERPT -->

## Kinds of Stories

I'll define stories as a description of events that has some purpose, or point to it. The sequence of events should be consistent (a character can't die in one event, then take part in one that happens after[^0])

### Raw Stories

Some stories are exist purely to be a good

### Teaching Stories

- Explaining morals of a culture, Gilgamesh, the Bible
- Teaching gullible children, Cheinoos

Oral tradition
Literature
  -

## Tricky Stories

## Rationalization

Stories we tell ourselves can be the hardest to detect. It is easy for us to unconsciously accept the simplest story that explains events.

## Personal Narratives

These are the meta-stories that spin-off countless minor variations. My favorite is the Entrepreneur Memoir:

1. Alice had a great idea
2. "Common Sense Charlie" told Alice that she would fail
3. Alice was bold, and broke the rules, leading directly to her success[^4]
4.
5. Now when someone tells YOU your idea is not good, ignore them!

This is wrong on so many levels,[^3] but the story is repeated over and over again. Whenever someone describes the way that a company started

  - Rationalization
  - Trope Narratives projections onto well-known stories

## The Power of Stories

- Simple
- Easy to Understand
- Emotionally Evocative (The Things They Carried)

Stories simplify. Stories are simpler than experience, and are designed to be easy to digest. It's impossible to accurately convey the details of every event. Even when we tell stories from our point of view, we can't remember as vividly as we can experience. A lot of that detail isn't important, though. When we tell a story we pick out the important parts, and, like editing a move, trim, expand, and enhance each scene into a more engaging narrative. We want other people to enjoy our stories, or at least feel some emotional reaction to them.

"Emotionally evocative" is a necessary requirement of all stories.

### True War Stories

One of the most provocative articles I've read about stories[^1] is by Tim O'Brien. It has appeared as a standalone article "How to Tell a True War Story" and as a chapter in The Things They Carried. In it, Tim tries to describe what it means to tell the truth. What I got out of it is that he was trying to convey an emotional truth; to induce empathy. War is a beast whose scale and intensity is incomprehensible to outsiders. How can a story about war be understood, when the context of story is cannot be. The most true way to represent a war story may not be to present the events as they appear in a captain's log, but as they appear through the eye of the combatant, as they are remembered. A story, like a memory, is irrecoverably distorted[^2].

### Side Note on Powerful Rhetoric

Stories, like any form of convincing rhetoric, can cut multiple ways. The value of a story, or its truth, are independent of its power. Both real and fictional stories can be captivating, and there can be many more fictional stories than real ones.

I want to make a long, well-thought-out post about this topic. For now, I'll say that I used to get caught up thinking "but it shouldn't be that way!" Now, I try to accept that truthiness and rhetorical power are not naturally connected. If I accept that as immutable, it becomes my responsibility to use rhetoric effectively to spread the truth. Rhetoric becomes even more important, because if I ignore it, I can spew forth as much truth as I want, but it will never take root.

The fact that humans are so easily swayed by stories is not a _bad thing_. It's just a _thing_. If we understand the power of stories, we can identify when it is being used to spread hate and slander, and promote its use to spread goodness and knowledge. Only by understanding how stories captivate and

> TODO: Is there any concrete evidence for fake stories spreading faster? I've heard this said before, but I haven't seen anything conclusive

[^0] Unless it's Berrick Dondarian?
[^1] It might be the only article I've ever read that discusses stories directly. But still, it's really good!
[^2] I'm tempted to make a non-sequitor to information theory here. Once the bits are lost, there isn't a way to get them back.
[^3] Oh you _know_ a blog post/rant about this specific story is coming in the near future. Working title "Entrepreneurship Is a Circle Jerk"
[^4] Okay she failed a bit, but she failed _fast_ so it doesn't count. That was on purpose. It's definitely a different type of failure than what Common Sense Charlie was saying...right?

license: MIT

repository: skrulcik/bitspluscontext

repo_name: skrulcik/bitspluscontext


text:
# [String.format()]()
## JavaScript sprintf/console.log()/String.Format() inspired formatting library
## 997 bytes minified, 576 bytes gzipped.

String-format is a micro library that provides a very simple but very useful format() method to your string literals allowing you to access sprintf like formatting on the fly, e.g.

```js
'This is a string %s'.format('literal') // 'This is a string literal'
```

This product is the evolution of a tool that I've copied from project to project in one form or another over several years. In
true coder form, it's updated, unit tested, and documented on github right now because I'm procrastinating another project.

## Syntax

- `%s` : substitute a string
  - `%Ds` : limit the substituted string to 2 characters
  - `%D.Xs` : substr starting at index D and X characters long
- `%f` : substitute a floating point number, no leading zero
  - `%0f` : floating point with leading zero
  - `%.Df` : fixed to D decimal places
  - `%0.Df` : fixed to D decimal places with leading zero
  - `%,Df` : Using a comma will add a comma separator every 3 digits
- `%d` or `%i` : substitute an integer (zero decimal places)
  - `%,d` : add comma separator every 3 digits
- `%%` : literal % sign

## Examples

```javascript
'Github is %s'.format('awesome');                  // "Github is awesome"
'One answer may be %i'.format(42);                 // "One answer may be 42"
'Another answer may be %.5f'.format(Math.PI);      // "Another answer may be 3.14159"
'%.5f is not equal to %.5f'.format(22/7, Math.PI); // "3.14286 is not equal to 3.14159"
'PI minus 3 is %0.5f'.format(Math.PI - 3);         // "PI minus 3 is 0.14159"
'%,d is a really big number'.format(299792458);    // "299,792,458 is a really big number"
'%0,2f is a smaller number'.format(12021.12);      // "12,021.12 is a smaller number"
```

The format() method is on every string, not just literal strings.

```javascript
var header = 'Welcome, %s';
header.format("Bob")          // "Welcome, Bob"
```

## License

MIT/BSD License


text:
# Checking Time Consistency

Database transaction consistency is guaranteed by a logical clock and is not affected by OS time. However, OS time inconsistency will lead to problems, such as abnormal backend O&M and monitoring functions. Therefore, you are advised to monthly check time consistency among nodes.

## Procedure

1.  Log in as the OS user  **omm**  to any host in the GaussDB Kernel cluster.
2.  Create a configuration file containing information about nodes in the cluster.

    ```
    vim /tmp/mpphosts
    ```

    Add the host name of each node.

    ```
    plat1
    plat2
    plat3
    ```

3.  Save the configuration file.

    ```
    :wq!
    ```

4.  Run the following command and write the time on each node into the  **/tmp/sys\_ctl-os1.log**  file:

    ```
    for ihost in `cat /tmp/mpphosts`; do ssh -n -q $ihost "hostname;date"; done > /tmp/sys_ctl-os1.log
    ```

5.  Check time consistency between the nodes based on the command output. The time difference should not exceed 30s.

    ```
    cat /tmp/sys_ctl-os1.log
    plat1
    Thu Feb  9 16:46:38 CST 2017
    plat2
    Thu Feb  9 16:46:49 CST 2017
    plat3
    Thu Feb  9 16:46:14 CST 2017
    ```

gha_license_id: MIT


text:
---
layout: default
---

# 1994-05-23 - Re: Skipjack backdoor? NOT!

## Header Data

From<EMAIL_ADDRESS>(Michael Handler)

To<EMAIL_ADDRESS>Message Hash: 4c6b0ba99f56b5a1ba9f76b9aba84eb214d077c39d32fd578d8566304d2d772e

Message ID<EMAIL_ADDRESS>Reply To<EMAIL_ADDRESS>UTC Datetime: 1994-05-23 22:24:57 UTC

Raw Date: Mon, 23 May 94 15:24:57 PDT

## Raw message

```
{% raw  %}From<EMAIL_ADDRESS>(Michael Handler)
Date: Mon, 23 May 94 15:24:57 PDT
To<EMAIL_ADDRESS>Subject: Re: Skipjack backdoor? NOT!
In-Reply-To<EMAIL_ADDRESS>Message-ID<EMAIL_ADDRESS>MIME-Version: 1.0
Content-Type: text

> If they were that smart, they'd simply have made Skipjack weak enough to
> easily break, and not cause all the outcry by suggesting escrows.

	The thing is, the rest of the law enforcement agencies and the
government have wised up to how easily people can be monitored on the
Internet and other computer networks. Now, these other agencies want the
NSA's ability to monitor encrypted communications, but the NSA, in
typical spook fashion, is understandably reluctant to let other agencies
in on its deep dark secrets of cryptanalysis. So, they created the
Clipper scheme, which is a compromise.
	[1] It has a key-escrow scheme, so those other agencies, who
actually have to worry about little things like the validity of searches
and the Constitutionality of their actions, can go through proper
channels and legally obtain the plaintext of the messages.
	[2] My guess is that Skipjack is compromised in some way (not
releasing the algorithm was *really* suspicious) or that the NSA can
easily break 80-bit keys, so that the NSA can continue to have a leg up
on all of the other agencies (ie they don't have to deal with the key
escrow departments or other little trivial legal details).

	All IMAO, of course.

--
Michael Brandt Handler<EMAIL_ADDRESS>Philadelphia, PA, USA	      PGP v2.3a public key via server / finger / mail
"I am iron, I am steel, nobody can touch me when I'm on the wheel"  --  Curve

{% endraw %}
```

## Thread

+ Return to [May 1994](/archive/1994/05)

+ Return to<EMAIL_ADDRESS>(Michael Handler)](/authors/grendel_at_netaxs_com_michael_handler_)"
+ Return to "[rishab@dxm.ernet.in](/authors/rishab_at_dxm_ernet_in)"

+ 1994-05-23 (Mon, 23 May 94 04:14:17 PDT) - [Skipjack backdoor? NOT!](/archive/1994/05/95656e6015aac4cbceb350282d078cbfa8b9d88ff5d0455f8fbbaf9a7d438d79) -<EMAIL_ADDRESS>  + 1994-05-23 (Mon, 23 May 94 15:24:57 PDT) - Re: Skipjack backdoor? NOT! -<EMAIL_ADDRESS>(Michael Handler)_


text:
# plug.kak

plug.kak is a plugin manager for [Kakoune].

[Kakoune]: https://kakoune.org

Just a thin wrapper around the `require-module` command and `ModuleLoaded` hook
— with Git support — to structure your kakrc.

plug.kak does not know about plugins and Git until you use a `plug-install` command.

At its core plug.kak is just a translation of:

``` kak
plug my-module my-plugin.kak %{
  my-plugin-enable
}
```

to:

``` kak
hook global ModuleLoaded my-module %{
  my-plugin-enable
}

hook global KakBegin .* %{
  require-module my-module
}
```

## Installation

Run the following in your terminal:

``` sh
git clone https://github.com/alexherbo2/plug.kak ~/.config/kak/autoload/plugins/plug
```

## Usage

``` kak
require-module plug

# Let plug.kak manage itself.
plug plug https://github.com/alexherbo2/plug.kak %{
  # Upgrade plugins
  # Install plugins and build them.
  define-command plug-upgrade -docstring 'plug-upgrade' %{
    plug-install
    plug-execute lsp cargo build --release
  }
}

plug-core %{
  # Tools
  set-option global makecmd 'make -j 8'
  set-option global grepcmd 'rg --column --with-filename'
}

# A module without associated repository.
# Typically, a kak script living in your autoload.
plug-autoload my-module

# A local plugin, relative to your home directory.
plug my-module projects/my-plugin.kak %{
  my-plugin-enable
}

# A plugin without module.
plug-old state-save https://gitlab.com/Screwtapello/kakoune-state-save %{
  # Starting
  hook global KakBegin .* %{
    state-save-reg-load colon
    state-save-reg-load pipe
    state-save-reg-load slash
  }

  # Quitting
  hook global KakEnd .* %{
    state-save-reg-save colon
    state-save-reg-save pipe
    state-save-reg-save slash
  }
}
```

Run `plug-install` or `plug-upgrade` you just defined.

Install new plugins interactively with the `plug` user mode.

## Tips and tricks

### Disable a plugin

Prefix the `plug` command with `nop`:

``` kak
nop plug my-module projects/my-plugin.kak %{
  my-plugin-enable
}
```

### Access user modes

If you have `u` aliased to `enter-user-mode`, you can do:

``` kak
u plug
```

``` kak
alias global u enter-user-mode
```

license: Unlicense


text:
### [CVE-2021-21123](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-21123)
![](https://img.shields.io/static/v1?label=Product&message=Chrome&color=blue)
![](https://img.shields.io/static/v1?label=Version&message=%3C%2088.0.4324.96%20&color=brighgreen)
![](https://img.shields.io/static/v1?label=Vulnerability&message=Insufficient%20data%20validation&color=brighgreen)

### Description

Insufficient data validation in File System API in Google Chrome prior to 88.0.4324.96 allowed a remote attacker to bypass filesystem restrictions via a crafted HTML page.

### POC

#### Reference
No PoCs from references.

#### Github
- https://github.com/ARPSyndicate/cvemon
- https://github.com/Puliczek/CVE-2021-21123-PoC-Google-Chrome
- https://github.com/Puliczek/puliczek
- https://github.com/SYRTI/POC_to_review
- https://github.com/WhooAmii/POC_to_review
- https://github.com/adriacabeza/personal-stars
- https://github.com/anquanscan/sec-tools
- https://github.com/manas3c/CVE-POC
- https://github.com/nomi-sec/PoC-in-GitHub
- https://github.com/soosmile/POC
- https://github.com/taielab/awesome-hacking-lists
- https://github.com/trhacknon/Pocingit
- https://github.com/xdavidhu/awesome-google-vrp-writeups

repository: trickest/cve

branch_name: refs/heads/main

gha_language: HTML

repo_name: trickest/cve


text:
# Native Packager

This is a small experiment to enable creating standalone executables from any folder structure,
by providing a shell command to be used as an entry point. Currently the project works,
but many improvements are required to make it actually useful.

# To Do

* Unix file permissions/other metadata (for now this can be emulated using gzip)
* Windows support without cross compilers
* Documentation
* Testing

repository: PhilipRoman/native-packager

repo_name: PhilipRoman/native-packager


text:
Week-by-week notes
==================

Below are notes about things that I did week-by-week for the first few weeks of the project.  I have
tried to distill everything important here and place it into the `cassandra_notes.md` file.

Week of 2014-01-06
==================

Goals:

- Refactor as much HBase code as possible into o.k.s.impl.hbase
- Refactor the test code as well, making sure that unit tests still pass
- Create o.k.s.impl.cassandra package and start implementing stuff there
- Be able to install Kiji C* instances
  - Add command-line install, uninstall
  - Implement C* versions of KijiInstaller and the various metadata tables

New packages:

- schema.cassandra
- schema.impl.cassandra
- schema.impl.hbase

New files:

- o.k.s.tools.CassandraInstallTool
- o.k.s.tools.CassandraUninstallTool
- o.k.s.cassandra.KijiManagedCassandraTableName
- o.k.s.cassandra.CassandraKijiInstaller
    - Create keyspace based on KijiURI
    - Create system table, meta table, schema table there
    - For now will not use any factories for these.
    - Keep track of table names in Cassandra keyspaces
- o.k.s.impl.cassandra.CassandraSystemTable
- o.k.s.impl.cassandra.CassandraMetaTable
- o.k.s.impl.cassandra.CassandraSchemaTable

- New classes to provide similar functionality to HBaseAdmin and HTableInterface
  - o.k.s.impl.cassandra.CassandraAdmin
  - o.k.s.impl.cassandra.CassandraTableInterface

Interfacing with C* tables versus HBase tables
----------------------------------------------

The interface between a client program and an HBase table uses just `byte[]` arrays.  We can mimic
this interface in our C* code by using blobs everywhere.  Doing so, however, makes the CQL shell
somewhat useless, and will probably be strange for experienced C* users.

For now, I'll add lightweight classes that mimic `HBaseAdmin` and `HTableInterface`, but for C*,
just to minimize the amount of refactoring to get C* working.  In the future, we can probably
implement more of the Kiji table layouts with C* primitives, but doing so will require deeper code
changes.

Code reuse
----------

For a lot of KijiSchema components, we have an interface and then an HBase class that implements
that interface.  Many of the C* classes that we are implementing now will share a lot of code with
the HBase classes.  Do we want to add another level of indirection to allow us to not copy/paste
code between HBase and C*?  e.g., we could do

    KijiFoo [interface] -> AbstractKijiFoo [abstract class] -> HBaseKijiFoo / CassandraKijiFoo

Choosing at run-time between Cassandra and HBase implementations of Kiji components
-----------------------------------------------------------------------------------

One way to implement the run-time choice between creating or references Cassandra or HBase Kiji
implementations is to have the two use different URIs.  A current Kiji URI looks like:

    kiji://(zkHost,zkHost):zkPort/instance

Now, in addition to ZooKeeper hosts, we need to have to allow the user to specify contact points and
a port.  For now, we'll use the following format:

    kiji-cassandra://(zkHost,zkHost):zkPort/(cassandraHost,cassandraHost)/cassandraPort/instance

We shall assume that URIs that don't explicitly start with `kiji-cassandra://` are HBase URIs.

We want to provide an interface to the user in which he can can use the same code to open HBase- or
Cassandra-backed `Kiji` instances, `KijiTable`s, etc., selecting between the two implementations
just by changing the URI.

The point of entry for most anything a user would want to do with KijiSchema is the creation of a
`Kiji` instance.  Currently, there are two ways to get a `Kiji` instance:

- `Kiji.Factory.get().open(kijiURI, ...)`
- `Kiji.Factory.open(kijiURI, ...)`

The former method call we shall have continue to return an `HBaseKijiFactory`, since there is no URI
information present at the time we generate the factory.

The second method call shall do the following:

- Look at the URI to determine whether the factory is HBase- or Cassandra-backed
- Then use the highest-priority `HBaseKijiFactory` or `CassandraKijiFactory` available

Notes on updating system, schema, and meta tables
-------------------------------------------------

#### The system table

Updating the system table looks pretty straightforward.  We just need to create a table with keys
and values as blobs and convert between blobs and byte arrays and we can reuse all of the rest of
the code.

#### The schema tables

The schema tables, one of which stores schemas by IDs, the other of which stores by hashes, are also
pretty straightforward and we can reuse almost all of the code.  A few notes:

- Because the usage of the schema tables is more well-defined (users can put whatever they want into
  the system table), we can use a slightly more sane schema, i.e., everything does not have to be a
  blob.
- For now I'm going to punt on storing every version of every
- We have to implement counters in the C* style (putting them into a separate table)
- We don't need ZooKeeper locks.  Instead, we can put all of the schema table operations into a
  batch block.

#### The meta table

The meta table stores `KijiTableLayout` information for each user table.

There are two tables in here:

1. `HBaseTableLayoutDatabase` contains the layouts for each table (stored in Avro `TableLayoutDesc`
records)
2. `HBaseTableKeyValueDatabase` contains per-table key-value pairs.  I'm not sure what these are
used for beyond "column annotations."

#### Common notes for all of the tables

All three of the tables have the following common methods:

- `public static HTableInterface newFooTable(KijiURI, Configuration, HTableFactory)`.  Creates an
  `HTableInterface` pointing to the HBase table for the system table, schema ID table, schema hash
  table, or meta table.
    - Called internally from the constructor that uses the `HTableInterfaceFactory` (used to get a
      pointer to the `HTableInterface` for calling the other constructor)
    - Called internally from the `install` method for the same reason (get pointers to
      `HTableInterface` methods before calling constructor)
- Constructor `FooTable(KijiURI, Configuration, HTableInterfaceFactory)`.  As described above, uses
  the `HTableInterfaceFactory` and the `newFooTable` static method to get pointers to one or more
  `HTableInterface` objects.
- Constructor `FooTable(KijiURI, HTableInterface)`.  Just assigns the URI and table to members and
  performs some sanity checks (e.g., updating and checking `mState`).
- Installer `static void install(HBaseAdmin, KijiURI, Configuration, HTableInterfaceFactory, some
  map of properties)`.
  This method actually creates the table (instead of just getting a pointer to its
  `HTableInterface`).
- Installer `static void install(HBaseAdmin, KijiURI, Configuration, HTableInterfaceFactory)`.  Just
  calls the other installer, but with an empty map of properties.
- Uninstaller `static void uninstall(HBaseAdmin, KijiURI)`.  Deletes the table.

What calls what?

- As far as I can tell, the *only* place that calls `install` or `uninstall` is `KijiInstaller.`
- Although `newFooTable` is public, the *only* place I see it called is within the `FooTable` class,
  inside the constructor that uses the `HTableInterfaceFactory`.  (*TODO: CAN WE MAKE THIS METHOD
  PRIVATE?*)
- The constructors with the `HTableInterfaceFactory` get called from `HBaseKiji`.
- The constructors that take an `HTableInterface` as an argument get called only within `FooTable`
  and within the unit tests.

We'll keep all of these methods for the Cassandra implementations of these classes.  The main
difference is that we are using `CassandraAdmin` to fill in the role of `HBaseAdmin` and
`HTableInterfaceFactory`.  The methods will therefor look like the following:

- `public static CassandraTableInterface newFooTable(KijiURI, Configuration, CassandraAdmin)`
- Constructor `FooTable(KijiURI, Configuration, CassandraAdmin)`
- Constructor `FooTable(KijiURI, CassandraTableInterface)`
- Installer `static void install(CassandraAdmin, KijiURI, Configuration, map of properties)`
- Installer `static void install(CassandraAdmin, KijiURI, Configuration)`
- Uninstaller `static void uninstall(CassandraAdmin, KijiURI)`

Week of 2014-01-13
==================

Goals for this week:

- Add unit test support for what we did last week
- Add support for meta tables
- Refactor layout.impl code into layout.impl.hbase
- Refine the `CassandraAdmin` and `CassandraTableInterface` classes
  - Start adding more checking, reference counting, etc.
  - Add more documentation (especially with regard to whose responsibility it is to close which
    tables)
- Begin looking at how to implement Cassandra KijiTables.

New packages:
- o.k.s.layout.impl.cassandra

New files:

- o.k.s.cassandra.CassandraFactory
- o.k.s.cassandra.TestingCassandraFactory
- o.k.s.cassandra.CassandraKijiClientTest
- o.k.s.impl.cassandra.CassandraAdminFactory
- o.k.s.impl.cassandra.CassandraKiji
- o.k.s.impl.cassandra.CassandraKijiFactory
- o.k.s.impl.cassandra.CassandraKijiTable
- o.k.s.impl.cassandra.CassandraMetaTable
- o.k.s.impl.cassandra.CassandraTableKeyValueDatabase
- o.k.s.impl.cassandra.DefaultCassandraAdmin
- o.k.s.impl.cassandra.DefaultCassandraAdminFactory
- o.k.s.impl.cassandra.DefaultCassandraFactory
- o.k.s.impl.cassandra.TestingCassandraAdmin
- o.k.s.impl.cassandra.TestingCassandraAdminFactory
- o.k.s.layout.impl.cassandra.CassandraTableLayoutDatabase
- o.k.s.layout.impl.cassandra.CassandraTableSchemaTranslator
- o.k.s.security.CassandraKijiSecurityManager
- o.k.s.tools.CassandraCreateTableTool

Unit and integration testing
----------------------------

We are assuming that uses will have an environment with Cassandra, ZooKeeper, and HBase installed.
Ideally we can add C* nodes to whatever fake HBase we are using now.

For unit tests, we use Cassandra's `EmbeddedCassandraService.`  Much as the HBase implementation of
Kiji has a `TestingHBaseFactory`, we have a `TestingCassandraFactory.`  This factory returns a
`CassandraAdmin` object that will always use a singleton session with the
`EmbeddedCassandraService.`

We need to figure out how we want to clean up Cassandra keyspaces before starting unit tests.  We
don't want to carry over stale C* test sessions from test to test.

Note that Cassandra limits keyspaces to 48 characters (?!), so we have to truncate the names of the
Kiji instances used for tests.
([Apparently](https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/Schema.java#L49)
this is related to path-size limits in Windows!  Awesome!)

For integration tests, Joe recommends using a Vagrant virtual machine to set up a C* server.

Note that you can run just the C* unit tests at the command line with the command:

    mvn  -Dtest='*Cassandra*' test

Notes on updating the meta table
--------------------------------

(I did not finish the meta table updates last week.)

We will have two column families, one for the layout-specific metadata and one for user-defined
metadata.  In C*, a column family and a table are the same thing, so we'll also have two
`CassandraTableInterface` members of the `CassandraMetaTable` class (as opposed to the single
`HTableInterface` in `HBAseMetaTable`).

Within the `CassandraTableLayoutDatabase` I implemented straight insertions instead of the
"check-and-insert" that was present in the HBase implementation.  We should be able to implement the
"check-and-insert" stuff pretty easily with the CQL "lightweight transactions."

What to do about HBase "flush" calls?
-------------------------------------

Should these just be C* writes with higher consistency requirements?

Security
--------

I did not put `CassandraKijiSecurityManager` into
o.k.s.security.impl.cassandra.CassandraKijiSecurityManager because there are so many package-private
security things that it needs to access.  I assume this code is going to have to get rewritten
anyway, so I did the minimum to get it to compile for now.

Notes on support for creating C*-backed Kiji tables
---------------------------------------------------

Summary of stuff that we'll have to change:

#### Table layout descriptions

The current table layout (stored in `TableLayoutDesc` Avro records) has a lot of HBase-specific
stuff.  We may want to make a separate description for C*-backed Kiji tables.

HBase offers a very nice data structure, `HTableDescriptor`, that contains all of the layout
information for an HTable.  I'm not sure if we have anything similar for Cassandra.  The mapping
between Kiji tables and C* tables is not as straightforward as that between Kiji tables and HTables,
so we may need to make big changes in how we handle the Kiji / C* translation.

#### Entity ID factory

We don't have to do all of the hashing, etc. within Kiji anymore.  We could do so, and just make
every entity ID a C* blob, but doing so would really reduce the readability of the C* table to
anyone familiar with C* (and make debugging a lot harder).

#### Updating table layouts

All of the synchronization / update-passing stuff related to updating table layouts, e.g.,

- `InnerLayoutUpdater`
- `LayoutTracker`
- `LayoutCapsule`
- `LayoutConsumer` (I'm assuming a `LayoutConsumer` is a table reader or writer)

...can probably stay the same.  This is a pretty large portion of the code in `HBaseKijiTable.`

### Reader / Writer factories

These seem to allow the user to override the default encoding and decoding of data going to/from
tables.  I suggest punting on these for now.

### Table annotator?

This provides a way to work with the user-defined key-value pairs in the meta table.  Should be
pretty easy to get this to work with Cassandra.

### Bulk load

Loading HFiles into a C* does not make sense (AFAIK).  Possibly we will want to offer functionality
similar to this in the future, but with bulk loading of SSTables.

### Regions

Don't really make sense in the context of a C* implementation.

### `KijiTableWriter`

This looks pretty straightforward and should map well to C*.  It looks like this is where the rubber
meets the road and we really implement the Kiji / HBase interface for writes.  Here is where we will
put a lot of the C* code that implements writes to the C* tables.

In the `KijiTableWriter` we are mostly just dealing with puts of single cells, so we shouldn't have
to deal too much with the messiness of having one C* table per Kiji column family.

### `KijiTableReader`, `KijiRowData`, `KijiRowScanner`, `HBaseRequestAdapter`

This is where a lot of the action will happen.  We have to think carefully about how we encapsulate
the code that maps between Kiji and C*.

A lot of the code in `KijiTableReader` again concerns how to synchronize table layouts and handle
different cell decoders.  This code should not have to change.

So where does all of the C* / Kiji interface code go?
-----------------------------------------------------

Translations between C* and Kiji need to happen in the following places:

- Any updates to table layouts (including creating a table layout)
- Translating Kiji writes in C* writes
- Translating Kiji reads in C* queries
- Translating C* query results into Kiji read results
- (We also do some C* stuff when we create Kiji instances and create the system, meta, and schema
  tables, but that is decoupled from the other operations.  We should easily be able to change how
  we represent any of these extra tables in C* without changing anything else in Kiji.)

It would be good to keep all of this translation code as localized as possible so that we can easily
make changes to how we implement Kiji under the hood in C*.

Here is a list of places where we'll have to put changes:

- `CassandraKijiTableReader`
- `CassandraKijiTableWriter`
- `CassandraKijiRowData`
- `CassandraKijiRowScanner`
- `CassandraRequestAdapter`
- `CassandraKijiTable`
- `CassandraKiji` (this contains a lot of the code for implementing updates to table layout)
- `CassandraTableLayoutUpdater` (Seems to be used for ZooKeeper to broadcast table layout updates)
- `CassandraTableSchemaTranslator`
  - I don't know if we'll have the equivalent of `ColumnNameTranslator` because the mapping from
    Kiji column to C* will be so different from what you would get in HBase.
  - This is used only within `HBaseKiji`

Week of 2014-01-20
==================

Goals for this week:

- Continue to refine the `CassandraAdmin` and `CassandraTableInterface` classes
  - Reference counting
  - Documentation
- Start actually implementing C* Kiji tables
  - Support creating tables, but not altering them
  - Support simple reads (no row scanning)
  - Support writing data and simple deletes (delete a row or a single cell)

New classes and packages
------------------------

- `o.k.s.impl.cassandra.CassandraKijiTable`
- `o.k.s.impl.cassandra.CassandraKijiReaderFactory`
- `o.k.s.impl.cassandra.CassandraKijiWriterFactory`
- `o.k.s.impl.cassandra.CassandraKijiTableReader`
- `o.k.s.impl.cassandra.CassandraKijiTableReaderBuilder`
- `o.k.s.impl.cassandra.CassandraKijiTableWriter`
- `o.k.s.impl.cassandra.CassandraKijiTableWriter`
- `o.k.s.impl.cassandra.CassandraDataRequestAdapter`
- `o.k.s.impl.cassandra.CassandraKijiRowData`
- `o.k.s.CassandraKijiURI`

How to store Cassandra table layout information?
------------------------------------------------

There does not appear to be any Cassandra analogue (at least in the DataStax Java driver) for
HBase's `HTableDescriptor`, which contains a structured description of the layout of an HBase table.
We can sort of replace it with `TableMetadata`, which we can generate for an existig table, but we
cannot build or modify our own `TableMetadata` instances, and we cannot use a `TableMetadata`
instance to update the layout of an existing table.

Where to we use `HTableDescriptor` now?  (Omitting usages that are easy to replace)
- `HBaseKiji` for modifying table layouts
  - The table description comes from `HTableSchemaTranslator` translating a `TableLayoutDesc`
  - Code goes through every column in the table, making updates if the new column is different from
    the previous column
- `HBaseKiji` for creating a new table
  - The table description comes from `HTableSchemaTranslator` translating a `TableLayoutDesc`
- `HTableSchemaTranslator` translates from `TableLayoutDesc` to `HTableDescriptor`
- `HTableDescriptorComparator` compare two HTable layouts (checks to see if any layout updates are
  actually necessary)

It might be worthwhile copying the `TableMetadata` and associated code from the DataStax source,
stripping it down a bit, and just using that.

Total functionality for C* / Kiji interface
-------------------------------------------

- Creating and altering tables
  - Given a `TableLayoutDesc`, create a C* table or set of tables
  - Given a `TableLayoutDesc` and an existing C* table, alter the existing table to implement the
    new layout
- Reading data
  - Normal "get"
    - Given a `KijiDataRequest`, an `EntityId`, and a `KijiTableLayout`, create a C* `SELECT`
      statement to query the database.
    - Given the result of the query, create an instance of `KijiRowData`
  - "Bulk get"
    - Do the same (create query, return `KijiRowData`) given a list of `EntityId` instances
  - scan
    - Given a `KijiDataRequest`, a `KijiTableLayout`, and start and stop `EntityId`s, create a
      row-scanner `SELECT` statement
    - Return a `KijiRowScanner`
- Writing data
  - Given an `EntityId`, column family, column qualifier, timestamp, and value, write to the
    database
  - Given an `EntityId`, column family, qualifier, and amount, increment a counter by amount
  - Given a column family and qualifier, verify that the column is a counter
- Deleting data
  - Given an `EntityId`, delete a row
  - Given an `EntityId` and a timestamp, delete all values up to that timestamp
  - Given an `EntityId`, column family, and timestamp, delete all values in that family up to the
    timestamp
    - Requires a row lock for a map-type family in HBase
  - Given an `EntityId`, a family, a qualifier, and maybe a timestamp, delete values
- Row data
  - `KijiRowData` has lots and lots of different methods for viewing results, but most of them just
    use `mFilteredMap` (a map from family to qualifier to timestamp to value) in different ways.
  - `KijiRowScanner` is really just an iterator of `KijiRowData`.  Should not present any additional
    translation difficulties.

It might make sense to put all of this functionality (for now) into a `CassandraLayoutTranslator`
object or something like that.  Keeping all of the Kiji/C* mapping code in one place would make
iterating easier.

Layout capsules and other such stuff
------------------------------------

`o.k.s.impl.LayoutConsumer` has a signature that expects an
`o.k.s.impl.hbase.HBaseKijiTable.LayoutCapsule`.  We need to fix this.

Probably easiest to create a new `LayoutCapsule` interface and have the
`o.k.s.impl.hbase.HBaseKijiTable.LayoutCapsule` and
`o.k.s.impl.cassandra.CassandraKijiTable.LayoutCapsule` both implement it.  This should be easy,
just has a bunch of getters in it.

Implementing gets
-----------------

Reads are tricky for the C* Kiji because we do not have a 1:1 correspondence between C* tables and
Kiji tables.  For each Kiji read, we'll need to have one C* read per colum family.  How does this
change affect the code?  One proposal:

- `CassandraDataRequestAdapter` creates a list of Cassandra `Statement`s for each `DataRequest`
- The `CassandraKijiTableReader` issues those statements to a Cassandra `Session` and get back a
  list of Cassandra `ResultSet`s
- Combine those result sets into a `KijiRowData` instance
- We need to think a little bit about how to prepare these query statements only once, even across
  multiple `CassandraDataRequestAdapter` instances.

If everything fits into memory, then this works out great.  But what do we do about paging?

We will get back on DataStax `ResultSet` per query.  We can think of each `ResultSet` as being
potentially almost bottomless, since we could be querying a column family with lots and lots and
lots of qualifiers and versions.

If we are not using paging, then we can assume that everything fits into main memory and so we can
go through every `Row` in every `ResultSet` for a given get request and use the results to fully
populate the big nested `NavigableMap` in `KijiRowData`.

Implementing a row scanner
--------------------------

A row scanner provides an interable interface to a multi-row read from a Kiji table.  The results of
the multi-row read are *not* guaranteed to fit into main memory (that is kind of the point of the
row scanner!).

To implement a row scanner, we need to know that all of the `Row` objects in all of the `ResultSet`
objects that we get back from all of the various queries we perform on a C* table are grouped by
entity ID -- this is fine because our entity ID is our Cassandra partiion key.  The algorithm for a
row scanner therefore looks something like this:

    Perform a bunch of Cassandra queries
    Get back on ResultSet object per query (per fully-qualified column or per column family)
    do:
      create an empty KijiRowData
      eid = the earliest entity ID from all of the ResultSet objects
      for (each resultSet in all ResultSets)
        add all data for eid to the current KijiRowData
        (stop when you get to a new eid)
      emit the KijiRowData
    while we still have data left in the ResultSet objects

The tricky part is that we need a way to tell when we have gotten to the end of the data for a given
entity ID for a `ResultSet`, likely when we are using the `ResultSet` as an `Iterator<Row>`.  To
find that we've gotten the last piece of data for an entity ID, we'll have to read a piece of data
with the new entity ID, which means we will have pulled that data off of the iterator.  So we'll
need to know how to save the first `Row` of data for each `ResultSet` for each entity ID.  (Note:
Google's `common.collect` has a `PeekingIterator`, awesome!!!!!)

We can share the code that reads through a bunch of `ResultSet`s while the eids are the same and
creates a `KijiRowData` instance between the functions for gets and row scanners.  We could probably
even implement a get under the hood with a row scanner, since the underlying implementations won't
be very different now.

What if you want to mix paging with a row scanner?  What if a user is going through every row in his
table, and fetching data from one column family with a small amount of information, and another with
a potentially HUGE amount of information.  He uses paging on the second family, and, depending on
the contents of the first family, may or may not wish to page through all of the data in the second.
Can we supporting this use model without paging in the background through *all* of the data for the
second family?  We can certainly support this functionality if we do our own manual paging, instead
of using the paging that the DataStax driver allows.

DataStax driver paging
----------------------

Paging of results from the DataStax Java driver should happen automatically.  At most, we should
have to set the fetch size by using the `setFetchSize` method in class `Statement`.  There are also
methods like `getAvailableWithoutFetching`, `isFullyFetched`, and `fetchMoreResults` that we can use
to have more manual control over paging.

There is a little bit more information in
[this blog post](http://www.datastax.com/dev/blog/client-side-improvements-in-cassandra-2-0)
about Cassandra 2.0.

The API documentation for `fetchMoreResults` also has some example code (`fetchMoreResults` can be
used to prefetch more data).

Functionality needed for the phonebook tutorial
-----------------------------------------------

- Add C*-specific URIs
- The DDL shell may need to be modified to call C* code (need to check)
- Delete a Kiji instance
- `AddEntry.java` - Requires doing puts
- `Lookup.java` - Requires doing gets
- `StandalonePhonebookImporter` - Also just does puts
- `PhonebookImporter` - Use KijiMR, not ready yet
- `AddressFieldExtractor` - Appears to be deprecated, calls a method `getMostRecentValue` that I
  don't think exists anymore
- `IncrementTalkTime` - Uses counters, but generally seems deprecated for KijiMR stuff.
- `DeleteEntry` - Calls delete.

Prepared statements for different tables.
-----------------------------------------

For now, I am putting a Map from tableURI-to-prepared statements for reading, writing, etc. into
`CassandraAdmin`

Week of 2014-02-24
==================

Paging
------

We need to support paging in C* Kiji.  Milo started circulating a document with proposals for
KijiSchema 2.0, which will be the first version to support Cassandra.  Given that we are going to
change the major version of KijiSchema for C* support, we may not need to support 100% of the
current KijiSchema paging API (since we can change it in 2.0), but we should try.

The main trick with paging is likely that we cannot build up the entire
family-to-qualifier-to-timestamp-to-value map structure in `KijiRowData` in one go now, since all of
the cells may not fit into memory.  The current code will iterate through everything and create a
map with all of the data for the entire row.

In the current KijiSchema, it looks like paging works like this:

- The user specifies that he wants to use paging when he creates his data request
- The user then has to use "getPager" for the family or qualified column
- The pager has an interface that looks like an iterator over `KijiRowData`.

The user can also get different iterators over a column that hide the page-fetching process:

- `ColumnVersionIterator` iterates through the versions in a given column.
- `MapFamilyQualifierIterator` iterates through the qualifiers in a map-type family.
- `MapFamilyVersionIterator` iterates through the versions in all columns from a map-type family.

In all of these, the initial `KijiRowData` does not contain any values for any columns that have
paging enabled.  The subsequent calls to `ColumnVersionIterator`, `KijiPager`, etc. make multiple
get requests to get the paged data.

We may need to refactor how we organize the various calls to
`CassandraDataRequestAdapter#queryCassandraTables`.  We have the following cases:

- Standard scans and gets
  - Should *not* try to get any data from a Cassandra table for any paged columns.
  - The paged columns will get requested later by a `KijiPager`
- Paged gets (I don't think there are paged scans)
  - We should be able to verify that *every* column in a `KijiDataRequest` used for a paged request
    has paging turned on.
  - We can probably do other assertion checks for the `KijiDataRequest` for a paged operation, e.g.,
    that it should not have more than one column (or at least more than one family).
  - A paged get should (for now) return only a single ResultSet, not a collection of them.

##### Qualifier pager

The code for this is pretty simple, but would be a lot simpler if Cassandra had a way to filter out
duplicate results in a query on the server.  To get a list of all of the qualifiers for a column, we
have to do something like:

    SELECT qualifier, version FROM <table> WHERE key=<key> AND family=<family> AND qualifier <=
    <maxqual> AND qualifier >= <minqual>

We select `version` so that we can filter out any results for which the timestamp is out of the
bounds specified in the user's data request.  We could put the timestamp range in the query as we
have put the qualifier range into the query, starting with C* 2.0.6 (see this
[ticket](https://issues.apache.org/jira/browse/CASSANDRA-4851)).

In any case, we'll get back a separate qualifier result for every cell that has the qualifier that
we're looking for.

...

The iterator stuff was not that hard to get done.  The C* iterator code actually is simpler than the
HBase iterator code, just because the DataStax Java driver does so much of the work.

The HBase implementation of the map family version iterator works like this:

- Get a paged iterator of every qualifier in the map family
- For each qualifier, we a paged iterator over cells

If we can figure out a way to support max versions in our queries, then we can probably build one
entire big query for the entire multi-qualifier, multi-version pager.  Doing so would possibly allow
us to save a lot of RPCs, since many qualifiers may not have enough versions to max out a version
pager by themselves.

Refactoring the code to put all CQL into `CassandraAdmin`
---------------------------------------------------------

Over the last few weeks I've occasionally changed the way in which we map Kiji tables to Cassandra
tables.  These changes haven't been difficult to make, but as we implement more and more of the Kiji
functionality in C*, we are getting more and more classes that include CQL calls.  We also have more
and more statements that need to be prepared once and used many times.

To simplify some of the bookkeeping in C* Kiji (keeping track of what CQL statements we have cached,
for example) and to make any future changes to our underlying C* implementation easier, I'd like to
put all of the CQL commands into a single class, likely `CassandraAdmin` or maybe
`CassandraTableInterface` (which doesn't do anything right now).

A change like this might also allow us to refactor the boilerplate for translating column names into
separate, shared methods.

From where do we currently execute CQL statements?

- `CassandraAtomicKijiPutter`
  - Batch write based on accumulated key-value object.  This is a little bit tricky because we have
    to use a C* `Session` first to create the various INSERT statements (one per key-value object),
    and then again to create and execute the batch INSERT.
- `CassandraDataRequestAdapter`
  - This class has a method `queryCassandraTables`, which turns a data request into a read.
  - We have multiple flavors of SELECT statements, for paged and non-paged scans and gets.

I'm not sure if this is worthwhile.  I'll punt for now.

Counters
--------

Counters in CQL have to go into their own table.  So for every Kiji table, we'll have to create a C*
table for non-counter values and a C* table for counter values.  Then during every read, we have to
determine whether to read from the non-counter table, the counter table, or both.  We already have
mechanisms in place to combine multiple sets of `ResultSet` objects together to form `KijiRowData`,
etc., which is nice, but now we'll have to be able to handle reading back non-blob values (a counter
is a `bigint`).

During writes, we have to figure out whether we are writing to a counter.  If we are setting a
counter (instead of incrementing it), then we may have to do a read followed by a (negative)
increment.  Or a delete followed by an increment (to whatever the starting value is).

repository: wibiclint/kiji-schema

repo_name: wibiclint/kiji-schema


text:
# 0.2.0

Removes the `turnstile_client_factory`.
Simplifies the use of redis client in pyramid by relaying on single DSN resource, which
can configure all cases of that `py-redis` supports.

# 0.1.4

Add `RedisClientConfiguration` class. With it you may configure your client via URL or unix_socket_path.

# 0.1.3

Add `turnstile` module providing a special case `turnstile_client_factory`, which can be used as a drop in `turnstile.redis_client` entrypoint with the
[turnstile]() rate limiting middleware.

[turnstile]: https://github.com/klmitch/turnstile

# 0.1.2

Explicitly write the `redis.` namespace in the default settings dict keys.

# 0.1.1

Support a `REDIS_KEY` environment variable to make picking up vars from
providers like REDISTOGO and REDISCLOUD easier.

Make sure that DB and MAX_CONNECTIONS are ints.

# 0.1

Initial release.


text:
# mapsnap - Apple Maps Web Snapshots on Python

**mapsnap** generates static maps using Apple's MapKit Web Snapshots API

```python
>>> from mapsnap import MapSnap, Annotation, ColorScheme, MarkerStyle
>>> map_snapshot = MapSnap(
...     _key=YOUR_PRIVATE_KEY_PATH,
...     teamId=YOUR_TEAM_ID,
...     keyId=YOUR_KEY_ID,
...     center=[-21.208392, -47.788517],
...     size="414x200",
...     colorScheme=ColorScheme.DARK,
... )
>>> print(map_snapshot.url)
https://snapshot.apple-mapkit.com/api/v1/snapshot?teamId=YOUR_TEAM_ID&keyId=YOUR_KEY_ID&center=-21.208392%2C-47.788517&size=414x200&scale=2&colorScheme=dark&lang=en-US&signature=A_DYNAMIC_SIGNATURE_GENERATED_AUTOMATICALLY_BY_THIS_MODULE
>>>
>>> print(map_snapshot.url)
https://snapshot.apple-mapkit.com/api/v1/snapshot?teamId=YOUR_TEAM_ID&keyId=YOUR_KEY_ID&center=-21.208392%2C-47.788517&size=414x200&scale=2&colorScheme=dark&lang=en-US&signature=A_DYNAMIC_SIGNATURE_GENERATED_AUTOMATICALLY_BY_THIS_MODULE
>>>
```

The **map_snapshot** variable will now have the **url** attribute containing the fully signed path to the static map image that can be accessed over https by any client. The string representation of the **map_snapshot** returned instance will always return the map URL as well.

If you want to dig deeper on the attributes and parameters of the returned instance you just need to take a look at its representation by either calling it directly on the Python REPL or by calling repr() over it like so:

```python
>>> map_snapshot
MapSnap(_key=None, teamId=None, keyId=None, center=[-21.208392, -47.788517], z=None, spn=None, size='414x200', scale=2, t=None, colorScheme=<ColorScheme.DARK: 'dark'>, poi=None, lang='en-US', annotations=None, overlays=None, referer=None, expires=None, imgs=None)
>>> repr(map_snapshot)
"MapSnap(_key=None, teamId=None, keyId=None, center=[-21.208392, -47.788517], z=None, spn=None, size='414x200', scale=2, t=None, colorScheme=<ColorScheme.DARK: 'dark'>, poi=None, lang='en-US', annotations=None, overlays=None, referer=None, expires=None, imgs=None)"
>>> print(repr(map_snapshot))
MapSnap(_key=None, teamId=None, keyId=None, center=[-21.208392, -47.788517], z=None, spn=None, size='414x200', scale=2, t=None, colorScheme=<ColorScheme.DARK: 'dark'>, poi=None, lang='en-US', annotations=None, overlays=None, referer=None, expires=None, imgs=None)
```

> Notice that the **\_key**, **teamId** and **keyId** attributed are always nullified by design before returning the instance for security purposes (try to prevent exposure of the key content and IDs).

You can also call the **.png()** method which returns a PNG file object.
The sample below shows how you can fetch the PNG image directly and write it to a file:

```python
>>> with open("test.png", "wb") as fp:
...     fp.write(map_snapshot.png().read())
...
86737
```

and alas, we have a map image:

![Sample1](/docs/img/sample1.png)

> You can later convert the PNG file to other formats using an image module like [Pillow](https://pypi.org/project/Pillow/).

### Installing & Updating

**mapsnap** is available on **PyPI** and can be installed or updated with:

```shell
$ pip install -U mapsnap
```

If you're having installation problems due to ecdsa **gmpy2** dependency, try installing the **libgmp**, **libmpfr** and **libmpc**:

-   **(mac)** - `brew install gmp mpfr libmpc mpc`
-   **(linux)** - `sudo apt install -y libgmp-dev libmpfr-dev libmpc-dev`

### Setup

-   First you need to head to your [Apple Developer Account](https://developer.apple.com/account/)

    -   Then Go to [Certificates, Identifiers & Profiles](https://developer.apple.com/account/resources/certificates/list)
    -   Click on [Keys](https://developer.apple.com/account/resources/authkeys/list) on the left menu
    -   Click the [+ Sign next to Keys](https://developer.apple.com/account/resources/authkeys/add)
    -   Choose a name for your new Apple MapKit JS Key
    -   Mark the **MapKit JS** checkbox and Configure it
    -   Click **Continue**
    -   Click on **Register** at the top right
    -   **Download** your key and store it **safely**!!! Take Note of your Key ID

-   After that you **MAY** need to convert your .p8 key to a EC PEM format that python-ecdsa can use.
    -   Sometimes the .p8 keys works out of the box. If you're having signing issues, than try the next steps. First try to use the .p8 key you have downloaded directly from apple as the **\_key** parameter.
    -   You can do that using either:
        -   `$ openssl ec -in __[PATH_TO_ORIGINAL_KEY.p8]__ -out __[PATH_TO_OUTPUT_PEM_KEY.pem]__`
        -   `$ openssl pkcs8 -nocrypt -in __[PATH_TO_ORIGINAL_KEY.p8]__ -out __[PATH_TO_OUTPUT_PEM_KEY.pem]__`
    -   You need to set the path for the generated .pem as the **\_key** parameter of the **MapSnap** class instance

### Usage

You just need to import the **MapSnap** class and instance it. you should also import utilities that you will need to use like Annotations, Colors and others.

Every time you create a new instance of this class, a signed URL is automatically generated and you can access it directly over the instance **.url** attribute or download the PNG from Apple for usage as a regular Python file object by calling the **.png()** method on the instance itself.

##### Customizing the Maps

```python

```

##### Adding Annotations (Markers)

```python

```

##### Default Parameters and Attributes

| Parameter  | Type         | Description                                                                                |
| ---------- | ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **\_key**  | str or bytes | The path for the .p8 or .pem file containing your MapKit JS Key downloaded from Apple Developer Portal. **(ex: /opt/myapplekeys/AuthKey_XXXXXXXXXX.p8)** <br> This can also be a **bytes** object containing the raw contents of the key file.                         |
| **teamId** | str          | Your Apple Developer Team ID (you can view it on https://developer.apple.com after logging in near your company/developer name or by accessing the **Membership** option on the menu)                                                                                |
| **keyId**  | str          | This ID is shown to you when you create your key and is generally part of the key file name, for instance if the downloaded key file is AuthKey_XYZ712387.p8 your keyId is XYZ712387. (You can double check it by going to your details on the Apple Developer portal) |

The rest of the parameters are mapped directly from [Apple Maps Web Snapshots](https://developer.apple.com/documentation/snapshots) documentation and can be used as guided there.

I have tried to map as much as possible as dataclasses that can be imported from this module itself as shown in examples.

If you want to set a value for a parameter, you can check the mapped values currently implemented on [/mapsnap/utils.py](https://github.com/ygbr/mapsnap/blob/master/mapsnap/utils.py)

### Requirements

**mapsnap** requires Python 3.7 os higher since it uses [dataclasses](https://docs.python.org/3/library/dataclasses.html).

-   Python 3.7 (or higher)
-   ecdsa[gmpy2] (`pip install ecdsa[gmpy2]`)
-   requests (`pip install requests`)

### References

-   Official Apple Docs: https://developer.apple.com/documentation/snapshots

### Authors

-   [Ygor Lemos <ygbr@mac.com>](https://ygor.dev)

### Contributing

-   Follow the PEP8 specification
-   Write and Run Tests
-   Make sure to run pre-commit hooks

### LICENSE

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

### DISCLAIMERS AND TRADEMARKS

Apple, Apple Maps, MapKit and iOS are registered trademarks of [Apple Inc.](https://apple.com)

The Apple Maps Web Snapshots are currently considered **Beta Software** and subject to change.

repository: ygbr/mapsnap

gha_language: Python

repo_name: ygbr/mapsnap


text:
# rtmp-rtsp-stream-client-java

[![Android Arsenal](https://img.shields.io/badge/Android%20Arsenal-rtmp%20rtsp%20stream%20client%20java-green.svg?style=true)](https://android-arsenal.com/details/1/5333)
[![Release](https://jitpack.io/v/pedroSG94/rtmp-rtsp-stream-client-java.svg)](https://jitpack.io/#pedroSG94/rtmp-rtsp-stream-client-java)

Sponsored with 💖 &nbsp by

Enterprise Grade APIs for Feeds & Chat. Try the Android Chat tutorial 💬

Library for stream in RTMP and RTSP. All code in Java.

If you need a player see this project:

https://github.com/pedroSG94/vlc-example-streamplayer

## Wiki

https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/wiki

## Permissions:

```xml
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<!--Optional for play store-->
<uses-feature android:name="android.hardware.camera" android:required="false" />
<uses-feature android:name="android.hardware.camera.autofocus" android:required="false" />
```

## Compile

To use this library in your project with gradle add this to your build.gradle:

```gradle
allprojects {
  repositories {
    maven { url 'https://jitpack.io' }
  }
}
dependencies {
  implementation 'com.github.pedroSG94.rtmp-rtsp-stream-client-java:rtplibrary:2.0.7'
}

```

## Features:

- [x] Android min API 16.
- [x] Support [camera1](https://developer.android.com/reference/android/hardware/Camera.html) and [camera2](https://developer.android.com/reference/android/hardware/camera2/package-summary.html) API
- [x] Encoder type buffer to buffer.
- [x] Encoder type surface to buffer.
- [x] RTMP/RTSP auth.
- [x] Audio noise suppressor.
- [x] Audio echo cancellation.
- [x] Disable/Enable video and audio while streaming.
- [x] Switch camera while streaming.
- [x] Change video bitrate while streaming (API 19+).
- [X] Get upload bandwidth used.
- [X] Record MP4 file while streaming (API 18+).
- [x] H264, H265 and AAC hardware encoding.
- [x] Force H264 and AAC Codec hardware/software encoding (Not recommended).
- [x] RTSP TCP/UDP.
- [x] Stream from video and audio files like mp4, webm, mp3, etc (Limited by device decoders). [More info](https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/wiki/Stream-from-file)
- [x] Stream device display (API 21+).
- [X] Set Image, Gif or Text to stream on real time.
- [X] OpenGL real time filters. [More info](https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/wiki/Real-time-filters)
- [X] RTMPS and RTSPS.
- [X] RTSP H265 support (Waiting FLV official packetization to add RTMP support).

## Other related projects:

https://github.com/pedroSG94/RTSP-Server

https://github.com/pedroSG94/AndroidReStreamer

https://github.com/pedroSG94/Stream-USB-test

## Use example:

This code is a basic example.
I recommend you go to Activities in app module and see all examples.

### RTMP:

```java

//default

//create builder
RtmpCamera1 rtmpCamera1 = new RtmpCamera1(openGlView, connectCheckerRtmp);
//start stream
if (rtmpCamera1.prepareAudio() && rtmpCamera1.prepareVideo()) {
  rtmpCamera1.startStream("rtmp://yourEndPoint");
} else {
 /**This device cant init encoders, this could be for 2 reasons: The encoder selected doesnt support any configuration setted or your device hasnt a H264 or AAC encoder (in this case you can see log error valid encoder not found)*/
}
//stop stream
rtmpCamera1.stopStream();

//with params

//create builder
RtmpCamera1 rtmpCamera1 = new RtmpCamera1(openGlView, connectCheckerRtmp);
//start stream
if (rtmpCamera1.prepareAudio(int bitrate, int sampleRate, boolean isStereo, boolean echoCanceler,
      boolean noiseSuppressor) && rtmpCamera1.prepareVideo(int width, int height, int fps, int bitrate, int rotation)) {
  rtmpCamera1.startStream("rtmp://yourEndPoint");
} else {
 /**This device cant init encoders, this could be for 2 reasons: The encoder selected doesnt support any configuration setted or your device hasnt a H264 or AAC encoder (in this case you can see log error valid encoder not found)*/
}
//stop stream
rtmpCamera1.stopStream();

```

### RTSP:

```java

//default

//create builder
//by default TCP protocol.
RtspCamera1 rtspCamera1 = new RtspCamera1(openGlView, connectCheckerRtsp);
//start stream
if (rtspCamera1.prepareAudio() && rtspCamera1.prepareVideo()) {
  rtspCamera1.startStream("rtsp://yourEndPoint");
} else {
 /**This device cant init encoders, this could be for 2 reasons: The encoder selected doesnt support any configuration setted or your device hasnt a H264 or AAC encoder (in this case you can see log error valid encoder not found)*/
}
//stop stream
rtspCamera1.stopStream();

//with params

//create builder
RtspCamera1 rtspCamera1 = new RtspCamera1(openGlView, connectCheckerRtsp);
rtspCamera1.setProtocol(protocol);
//start stream
if (rtspCamera1.prepareAudio(int bitrate, int sampleRate, boolean isStereo, boolean echoCanceler,
      boolean noiseSuppressor) && rtspCamera1.prepareVideo(int width, int height, int fps, int bitrate, int rotation)) {
  rtspCamera1.startStream("rtsp://yourEndPoint");
} else {
 /**This device cant init encoders, this could be for 2 reasons: The encoder selected doesnt support any configuration setted or your device hasnt a H264 or AAC encoder (in this case you can see log error valid encoder not found)*/
}
//stop stream
rtspCamera1.stopStream();

```

gha_language: Java


text:
XKCDBook
========

Most people understand HTML/CSS/JS but have never worked with Chrome Extensions. I wanted to make it easier for others to get started with a relavent project. This is to show how HTML can be injected into the top of the group. I hope others will make more interesting things in forks.

#Instructions
![Instructions](instructions.png)
#Fork Ideas
- Upcoming Hackathons (Maybe even have a "Who's going?" that gets it's data from a Post with a poll.")
- Members Search (Powered by something like https://github.com/hshackers/docs/blob/master/members.csv)
- Member Chat
- Top HN Stories
- Yo Feed (e.g. YO HSHACKERS to be seen.)

#License
This is free and unencumbered software released into the public domain.

Anyone is free to copy, modify, publish, use, compile, sell, or
distribute this software, either in source code form or as a compiled
binary, for any purpose, commercial or non-commercial, and by any
means.

In jurisdictions that recognize copyright laws, the author or authors
of this software dedicate any and all copyright interest in the
software to the public domain. We make this dedication for the benefit
of the public at large and to the detriment of our heirs and
successors. We intend this dedication to be an overt act of
relinquishment in perpetuity of all present and future rights to this
software under copyright law.

IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.

For more information, please refer to <http://unlicense.org/>

repository: Hackathon-Hackers/XKCDBook

repo_name: Hackathon-Hackers/XKCDBook


text:
# A simple library to validate strings in the V language
The main purpose of this library is to validate the user input in the web applications.
Due to the nature of the input in the web (it's almost always strings), the main focus is on validating **strings**.

## Installation and Usage
<!-- Install the library with `v install endeveit.validate` -->
The library provides a number of validators and sanitizers

### Validators
Here's the full list of the currently available validators

Validator             | Description
----------------------|------------
is_bool               | checks if the string is a bool
is_eq                 | checks if the string is equal to another int, float or string value
is_ne                 | checks if the string is not equal to another int, float or string value
is_in                 | checks if the string is in an array of allowed values
is_gt                 | checks if the string is greater than another int, float or string value
is_ge                 | checks if the string is greater or equal to another int, float or string value
is_lt                 | checks if the string is less than another int, float or string value
is_le                 | checks if the string is less or equal to another int, float or string value
is_email              | checks if the string is a valid email address
is_fqdn               | checks if the string is a Fully Qualified Domain Name
is_float              | checks if the string is a float
is_int                | checks if the string is an integer
is_ip                 | checks if the string is an IP address (v4 or v6)
is_ipv4               | checks if the string is an IPv4 address
is_ipv6               | checks if the string is an IPv6 address
is_regex_match        | checks if the string matches the regular expression
is_regex_valid        | checks if regular expression syntax is correct
is_uuid               | checks if the string is a valid UUID
is_uuid_v3            | checks if the string is a valid UUID v3
is_uuid_v4            | checks if the string is a valid UUID v4
is_uuid_v5            | checks if the string is a valid UUID v5

### Sanitizers
Here's the full list of the currently available sanitizers

Sanitizer             | Description
----------------------|------------
to_bool               | converts the string to a boolean value
to_float              | converts the string to a float value
to_int                | converts the string to an integer value

repository: spytheman/v-validate

repo_name: spytheman/v-validate


text:
# [Exploring Code Project](https://theolivenbaum.medium.com/exploring-codeproject-with-curiosity-4f197ceaa831) 🚀

This repository contains the source code presented in the [Exploring CodeProject with Curiosity](https://theolivenbaum.medium.com/exploring-codeproject-with-curiosity-4f197ceaa831) article. It uses [Curiosity's](https://curiosity.ai) AI-powered search and it's [data connector](https://www.nuget.org/packages/Curiosity.Library) library to explore and search on CodeProject's articles, using machine-learning-based synonyms, and to suggest similar articles using word and graph embeddings.

![Graphs](https://raw.githubusercontent.com/curiosity-ai/codeproject-search/media/graphs.gif)

The data used for the demo was scraped from the [CodeProject](https://codeproject.com) website, and includes authors and their articles.

Check more details on our accompanying [blog post](https://theolivenbaum.medium.com/exploring-codeproject-with-curiosity-4f197ceaa831).

## Running Curiosity Locally

[Check our documentation](https://docs.curiosity.ai/en/articles/4449019-installation) to install a free instance of Curiosity on your computer or clould environment of preference.

Once you have your Curiosity instance up and running, check the [initial setup guide](https://docs.curiosity.ai/en/articles/4452603-initial-setup) and then you'll be ready to use the code in this repository.

## Data Ingestion

The code in this repository will crawl and ingest all articles from CodeProject to your Curiosity instance.

You'll need to generate an API token for your system, and pass it to the connector. Check the [documentation on how to create an API token](https://docs.curiosity.ai/en/articles/4453131-external-data-connectors).

```bash
git clone https://github.com/curiosity-ai/codeproject-search
cd codeproject-search
dotnet run {SERVER_URL} {AUTH_TOKEN}
```

You need to replace `{SERVER_URL}` with the address your server is listing to (usually `http://localhost:8080` if you're running it locally), and `{AUTH_TOKEN}` with the API token generated earlier.

repository: curiosity-ai/codeproject-search

repo_name: curiosity-ai/codeproject-search


text:
# BootstrapActiveNav

A Rails 3 view helper for generating Bootstrap navbar links with the active tag.

```ruby
gem "bootstrap_active_nav"
```

To use:

```ruby
<%= nav_link("Home", root_path) %>
```

Takes the same options as `link_to`, including blocks.

Released under an MIT License.

gha_language: Ruby


text:
# MyReads

![Screenshot](./src/images/screenshot.png)

MyReads is a bookshelf app that helps you keep track of your books by organizing them into your personal "bookshelves."

## Installation

Run the commands below and the app will run on [http://localhost:3000/](http://localhost:3000/)

```bash
npm install
npm start
```

## Usage

MyReads fetches data from [Udacity's React Books API](https://reactnd-books-api.udacity.com/) to connect the user with an expansive library of books.

The user can search for books and organize them into three different bookshelves:

- Currently Reading
- Want to Read
- Read

## Technologies

- Bootstrapped with create-react-app
- Styled with SASS
- Routed with React Router DOM in Development
- Used HashRouter for Production Routing

## Future Functionality

- Switch API to Google Books API to fetch data from a larger database of books
- Remove books from shelves
- Create and delete shelves
- Bulk move books
- Reviews and a rating system
- Create log-in system (Backend)

## License

[MIT](https://choosealicense.com/licenses/mit/)

repository: sharnajh/myreads

gha_language: JavaScript

repo_name: sharnajh/myreads


text:
---
title: Hide
introduction: |
  Utilities for hiding text with considerations for accessibility.
source: sky-toolkit-core/utilities/hide
dependencies:
  - sky-toolkit-core
layout: component
related:
  - sky-toolkit-core/tools/mixins
---

# Hide

## Hide Visually

Visually hide content, leaving it accessible to screen readers and ATs.

**Note**: This is also provided as a mixin in [sky-toolkit-core/tools/mixins](../../utilities/_hide.scss).

Further reading: [Jonathan Snook's 'Hiding Content for Accessibility'](http://snook.ca/archives/html_and_css/hiding-content-for-accessibility).

```html { "render": false }
<span class="u-hide-visually">I'm visually hidden</span>
```

---

## Hide Completely

Completely hide content both visually from the screen, and from screen readers
and ATs.

**Note**: This is also provided as a mixin in [sky-toolkit-core/tools/mixins](../../utilities/_hide.scss).

```html { "render": false }
<span class="u-hide-completely">I'm completely hidden</span>
```

repository: TobyPinfold/Sky-Test

gha_language: CSS

repo_name: TobyPinfold/Sky-Test


text:
nicefloats
==========

an erlang implementation of nice decimal representations of floating point numbers, based on "Printing FLoating-Point Numbers Quickly and Accurately" by Burger & Dybvig

### usage ###

`nicefloats:format(Float)`

### installation ###

`make` or `./rebar compile` to build nicefloats

`make test` or `./rebar eunit` to run tests

### notes ###

this project was originally a part of [jsx][1], and remnants of that projects history may linger. try to ignore it.

[1]: http://www.github.com/talentdeficit/jsx/

repository: liangchuan/lingtuan-xmpp

extension: markdown

repo_name: liangchuan/lingtuan-xmpp


text:
  ✨ Portfolio 2.0 ✨

![Image of Home Page](https://richelleb.dev/images/thumbnail.jpg)

## Allow Me to Reintroduce Myself
This project is the second iteration of my online portfolio. This time around, I wanted to venture into the New Modern Web by utilizing JAMstack architecture, the GatsbyJS ecosystem as well as CSS in JS with Styled Components.

In the end, I found that I was left with an incredibly fast and reliable site with improved scalability and security. It was a great first foray into the JAMstack workflow and I'm excited to dive deeper and build more things.

## Tech Stack

- [React](https://reactjs.org/)
- [GatsbyJS](https://www.gatsbyjs.com/)
- [Styled-Components](https://styled-components.com/)
- [Contentful](https://www.contentful.com/)
- [Netlify](https://www.netlify.com/)

## Next Steps

Now that basic functionality is complete, I'm working towards taking the project to the next level by adding:

- [x] Toggle for dark mode
- [ ] Testing with Jest
- [ ] Offline functionality

## Author
[@rchlblns](https://github.com/rchlblns)

## License
[MIT](https://choosealicense.com/licenses/mit/)


text:
# simple-semantic-config

[**semantic-release**](https://github.com/semantic-release/semantic-release) shareable config to publish npm packages with simple commit rules

[![Build Status](https://travis-ci.org/artemv/simple-semantic-config.svg?branch=master)](https://travis-ci.org/artemv/simple-semantic-config)

## Why this project exists
This is a shareable config to publish npm packages with simple commit rules bases on
[angular](https://github.com/conventional-changelog/conventional-changelog/tree/master/packages/conventional-changelog-angular)
preset of [commit-analyzer plugin](https://github.com/semantic-release/commit-analyzer#usage) with following change:
`docs`, `chore` and `refactor` commit types yield patch version.
Thus pretty much any correctly formatted commit message yields new version.

## Install

```bash
$ yarn add --dev "simple-semantic-config"
```

## Usage

The shareable config can be configured in the [**semantic-release** configuration file](https://github.com/semantic-release/semantic-release/blob/master/docs/usage/configuration.md#configuration) or package.json:

```json
{
  "release": {
    "extends": "simple-semantic-config"
  }
}
```

It can work in [monorepo semantic-release configuration](https://github.com/pmowrer/semantic-release-monorepo), too:
```json
{
  "release": {
    "extends": ["semantic-release-monorepo", "simple-semantic-config"]
  }
}
```

## semantic-release-pre

Figures out new version according to semantic-release configuration, and writes it to build/.version file.

Example: `BRANCH=master yarn simple-semantic-release-pre`

OR

`ALLOW_FAILING=false BRANCH=development yarn simple-semantic-release-pre`

OR, to write to tmp/.version file:

`OUT_DIR=tmp BRANCH=master yarn simple-semantic-release-pre`

This requires having write Git access to repo specified in package.json, you can set it by exporting GH_TOKEN env var with: `export GH_TOKEN=my-key`

When it's run in CircleCI build context it uses CIRCLE_BRANCH so specifying BRANCH is not needed there:
`yarn simple-semantic-release-pre` is enough.

`semantic-release` takes previous versions info from Github tags of `origin` remote, so when running it from local fork switch origin to upstream to get the correct new version:
```
git remote rm origin
git remote add origin git@github.com:upstream/my-project.git
BRANCH=master yarn simple-semantic-release-pre
git remote rm origin
git remote add origin git@github.com:my-account/my-project.git
```

repository: artemv/simple-semantic-config

gha_language: Shell

repo_name: artemv/simple-semantic-config


text:
## bKash Merchant Payment Gateway for WHMCS

bKash Mechant module allow you to integrate payment solutions with the WHMCS automation platform.

### Installation

```
 modules/gateways/
  |- callback/bkashlegacy.php
  |  bkashlegacy.php
```

## Requirements

Support all major WHMCS version including version 6, 7 & 8.

# FAQs

- How can I get the API credential?
    - Please contact your bKash account manger to get API.

## TODO

- Better support for any template
- Use bKash logo for payment button

repository: directionbd/bkash-merchant-for-whmcs

repo_name: directionbd/bkash-merchant-for-whmcs


text:
# html
My html projects.

Mozilla instructions for javascript:
https://developer.mozilla.org/en-US/docs/Learn/Getting_started_with_the_web/JavaScript_basics
***

DISCLAIMER:
As the code here is constantly changing and consists of mostly example projects for my personal learning experience, code may be incorrect or non-working anytime. I am an individual just messing around with computers.
But by browsing my repos', you may find some interesting programming methods that you may use as well.


text:
# 2022-01-27, stale prime version

## Summary

A bug in the federation-redirect health checks
kept the reference version from being updated,
preventing traffic from being sent to other federation members.

This effect lasted approximately nine hours before normal operation was restored without intervention.

![Grafana chart showing traffic to other federation members dropping for hours, then rising again](img/2022-01-27-pods.png)

## Timeline

All times in CET

### 2022-01-27 11:00

A routine bump of binderhub and reop2docker versions is deployed.
Traffic to federation members other than GKE begins to drop.

At this point, the prime cluster (GKE) is 'full' according to its pod quota.

Due to a bug in federation-redirect,
as long as a cluster is full,
its version is not checked.

However, the version of the prime cluster is used as a reference
when comparing with other federation members.

As long as the prime cluster stays 'full' (or unhealthy for any reason),
the reference version will be out of date,
Any actually up-to-date federation members will be considered to have mismatched version,
and will not receive any redirect traffic.

### 14:35

Federation members notice drop in traffic.

### 18:00

GKE load drops below quota,
allowing the prime version to be updated in the federation-redirector.
At this point, traffic to all federation members returns to normal.

### 2022-01-28 09:00

Investigations starts.

federation-redirect logs show that the version mismatch is the cause of the drop in traffic,
and that the 'prime' version is the one that's out-of-date.

Initial suspicion was an out-of-date binder pod was still receiving traffic.
Exploring logs in GCP logs explorer eventually eliminated this possibility.

Useful log query:

```
resource.type="k8s_container" resource.labels.cluster_name="prod"
resource.labels.container_name="binder"
"GET / "
```

It was useful to expand the "Log Fields" to see the counts by pod name within the time frame:

![GCP log fields showing log events per pod name](img/2022-01-27-log-fields.png)

A brief digression reduced trust in GCP logs because it is clear that the "Kubernetes Pod" resource logs are missing many entries:

```
resource.type="k8s_pod" resource.labels.cluster_name="prod"
resource.labels.pod_name=~"^binder.*"
```

### 2022-01-28 10:00

Underlying issue in federation-redirect is identified and [fixed][fix-pr].

## Lessons learnt

### What went well

1. Cluster healed itself when traffic dropped below the prime cluster's pod quota.
2. federation-redirect logs clearly showed it was the version mismatch preventing traffic.

### What went wrong

1. federation-redirect did not have the correct 'prime version' for 9 hours, due to load on the prime cluster ([fixed][fix-pr])
2. Response time was low. The issue went unnoticed for 3 hours,
   and without action for at least 9 hours (when it resolved itself).
   We do not currently have much int he way of proactive 'alerting' ([issue][alerting-issue]),
   so only when grafana or the status page is being watched do we notice issues like this.
3. The health of the cluster is not immediately visible except in logs or long-term traffic trends,
   which can take a while to be noticed ([issue][metrics-issue]).
   We do have most of these checks reimplemented on [the status page][status],
   but not the actual values as considered by the redirector (bugs and all).
   Notably, the status page does not include the version check,
   which caused this incident.

[status]: https://mybinder.readthedocs.io/en/latest/about/status.html

### Where we got lucky

1. Federation 'as a whole' handled the traffic okay,
   because the prime cluster can actually handle more traffic than its pod quota.
   That said, if we actually stopped routing traffic to the prime cluster while it is unhealthy,
   the issue would hav been resolved in minutes
   because it would not have stayed full.

## Action items

### Process improvements

1. Implement proactive alerts. This is a longstanding issue we have yet to address ([issue][alerting-issue]).

### Technical improvements

1. Fix bug preventing prime cluster's version from being updated while it is full ([pull request][fix-pr])
2. Add metrics for the federation-redirect service to improve visibility of federation health issues ([issue][metrics-issue])
3. Consider allowing the prime cluster to stop accepting traffic when it is unhealthy ([issue][full-issue])

[alerting-issue]: https://github.com/jupyterhub/mybinder.org-deploy/issues/611
[metrics-issue]: https://github.com/jupyterhub/mybinder.org-deploy/issues/2114
[fix-pr]: https://github.com/jupyterhub/mybinder.org-deploy/pull/2113
[full-issue]: https://github.com/jupyterhub/mybinder.org-deploy/issues/2115

gha_language: Jupyter Notebook


text:
---
title: Mushroom Cheese Toast [Recipe]
category: food
---

So I made an awesome mushroom cheese toast the other day and here is how to make it. Beware though, there’s tons of cheeeese and butter in here; not for you diet people!

Take a non-stick pan and throw in a decently sized chunk of butter on lowest flame for about a minute. Now, add mushrooms!

![](/assets/images/mushroom-cheese/1.JPG)

Cover the pan and let the mushrooms cook for about 5 minutes on lowest flame. Now time to add some cheeeese!

<!--more-->
![](/assets/images/mushroom-cheese/2.JPG)

Add some more butter and cover the pan again for 5 minutes. Add a little grated garlic, salt, and pepper. Now toast the bread (in the pan or grill or oven)!
![](/assets/images/mushroom-cheese/3.JPG)

Now pour the delicious mushroom-cheese goodies on the toast and voila!
![](/assets/images/mushroom-cheese/4.JPG)
![](/assets/images/mushroom-cheese/5.JPG)

There you go, have a delicious blast!

repository: oxalorg/personal-blog

repo_name: oxalorg/personal-blog


text:
# jQuery json-viewer

![npm](https://img.shields.io/npm/v/jquery.json-viewer.svg)
![npm](https://img.shields.io/npm/l/jquery.json-viewer.svg)
![npm](https://img.shields.io/npm/dt/jquery.json-viewer.svg)

*json-viewer* is a jQuery plugin for easily displaying JSON objects by transforming them into HTML.

Features:
- Syntax highlighting
- Collapse and expand child nodes on click
- Clickable links
- Easily readable and minimal DOM structure

Check out the [demo page](http://rawgit.com/abodelot/jquery.json-viewer/master/demo.html)!

## Usage

Import `jquery.json-viewer.js` and `jquery.json-viewer.css` in your application.

Copy sources from git repository, or use npm:

```sh
npm install jquery.json-viewer
```

Then just call the `jsonViewer()` method and pass your JSON data in argument:
```html
<pre id="json-renderer"></pre>
```

```js
var data = {
  "foobar": "foobaz"
};
$('#json-renderer').jsonViewer(data);
```

## Options

The `jsonViewer` method accepts an optional `options` hash as a second argument:

- `collapsed` (boolean, default: `false`): all nodes are collapsed at html generation.
- `rootCollapsable` (boolean, default: `true`): allow root element to be collasped.
- `withQuotes` (boolean, default: `false`): all JSON keys are surrounded with double quotation marks (`{"foobar": 1}` instead of `{foobar: 1}`).
- `withLinks` (boolean, default: `true`): all values that are valid links will be clickable, if `false` they will only be strings.

Example:

```js
$('#json-renderer').jsonViewer(data, {collapsed: true, withQuotes: true, withLinks: false});
```

## About

- Author: [Alexandre Bodelot](https://github.com/abodelot)
- License: [MIT License](http://opensource.org/licenses/MIT)


text:
# @velcro/bundler

![npm (scoped)](https://img.shields.io/npm/v/@velcro/bundler?style=flat-square)
![NPM](https://img.shields.io/npm/l/@velcro/bundler?style=flat-square)

Build a module graph of modules and serialize these to different formats.

[API Docs](https://github.com/ggoodman/velcro/tree/v0.45.0/docs/bundler.md)

repository: evanw/velcro

repo_name: evanw/velcro


text:
## Installation
You can install **`PolyFuzz`** via pip:

```bash
pip install polyfuzz
```

This will install the base dependencies and excludes any deep learning/embedding models.

If you want to be making use of 🤗 Transformers, install the additional additional `Flair` dependency:

```bash
pip install polyfuzz[flair]
```

## Getting Started

The main goal of `PolyFuzz` is to allow the user to perform different methods for matching strings.
We start by defining two lists, one to map from and one to map to. We are going to be using `TF-IDF` to create
n-grams on a character level in order to compare similarity between strings.

We only have to instantiate `PolyFuzz` with `TF-IDF` and match the lists:

```python
from polyfuzz import PolyFuzz

from_list = ["apple", "apples", "appl", "recal", "house", "similarity"]
to_list = ["apple", "apples", "mouse"]

model = PolyFuzz("TF-IDF").match(from_list, to_list)
```

The resulting matches can be accessed through `model.get_matches()`:

```python
>>> model.get_matches()
         From      To    Similarity
0       apple   apple    1.000000
1      apples  apples    1.000000
2        appl   apple    0.783751
3       recal    None    0.000000
4       house   mouse    0.587927
5  similarity    None    0.000000

```

**NOTE**: When instantiating `PolyFuzz` we also could have used "EditDistance" or "Embeddings" to quickly
access Levenshtein and FastText (English) respectively.

### Group Matches
We can group the matches `To` as there might be significant overlap in strings in our to_list.
To do this, we calculate the similarity within strings in to_list and use `single linkage` to then
group the strings with a high similarity.

When we extract the new matches, we can see an additional column `Group` in which all the `To` matches were grouped to:

```python
>>> model.group(link_min_similarity=0.75)
>>> model.get_matches()
	      From	To		Similarity	Group
0	     apple	apple	1.000000	apples
1	    apples	apples	1.000000	apples
2	      appl	apple	0.783751	apples
3	     recal	None	0.000000	None
4	     house	mouse	0.587927	mouse
5	similarity	None	0.000000	None
```

As can be seen above, we grouped apple and apples together to `apple` such that when a string is mapped to `apple` it
will fall in the cluster of `[apples, apple]` and will be mapped to the first instance in the cluster which is `apples`.

### Precision-Recall Curve
Next, we would like to see how well our model is doing on our data. We express our results as
**`precision`** and **`recall`** where precision is defined as the minimum similarity score before a match is correct and
recall the percentage of matches found at a certain minimum similarity score.

Creating the visualizations is as simple as:

```
model.visualize_precision_recall()
```

![](tfidf.png)


text:
# GSP092 —— Monitoring and Logging for Cloud Functions

    Table of Contents
    （🔎 Click to expand／collapse）

- [GSP092 —— Monitoring and Logging for Cloud Functions](#gsp092--monitoring-and-logging-for-cloud-functions)
  - [Overview](#overview)
  - [Viewing Cloud Function logs & metrics in Cloud monitoring](#viewing-cloud-function-logs--metrics-in-cloud-monitoring)
  - [Create Logs-based Metric](#create-logs-based-metric)
  - [Create Charts on The Monitoring Overview Window](#create-charts-on-the-monitoring-overview-window)
  - [References](#references)

## Overview

You can view your [Cloud Functions](https://cloud.google.com/functions) with their execution times,execution counts, and memory usage in the Cloud Console using [Cloud Monitoring](https://cloud.google.com/monitoring), where you can set up custom alerting on these metrics.

## Viewing Cloud Function logs & metrics in Cloud monitoring

## Create Logs-based Metric

Logs-based metrics are Cloud Monitoring metrics that are based on the content of log entries.

## Create Charts on The Monitoring Overview Window

## References


text:
# Unit 1 - Module 6: Control flow

| Example | Description | Compiles? |
|---------|-------------|-----------|
| [01-19-if-else](01-19-if-else) | Using an `if`/`else` block | ✅ |
| [02-02-loop](02-02-loop) | Using `loop`; use `CTRL-C` to exit! | ✅ |
| [02-17-break-loop](02-17-break-loop) | Using `break` to exit a `loop` | ✅ |
| [03-09-while](03-09-while) | Using `while`; has the same behavior as the `break` example | ✅ |
| [04-04-for](04-04-for) | Using `for` to iterate over each number in a range | ✅ |
| [05-25-match-on-integer](05-25-match-on-integer) | Using a `match` expression to test the value of an integer | ✅ |
| [06-10-match-on-tuple](06-10-match-on-tuple) | Matching on a tuple of integers, using underscore to ignore some values and using a vertical bar for OR | ✅ |
| [07-15-non-exhaustive-match](07-15-non-exhaustive-match) | A non-exhaustive `match` expression that Rust catches for us | ❌ |
| [07-42-adding-catchall](07-42-adding-catchall) | Fixing the previous example by adding a catchall case with underscore | ✅ |

gha_license_id: NOASSERTION