text:
---
layout: documentation
title: Optimizing Performance
---

# Optimizing Performance

When writing rules, the most common performance pitfall is to traverse or copy
data that is accumulated from dependencies. When aggregated over the whole
build, these operations can easily take O(N^2) time or space. To avoid this, it
is crucial to understand how to use depsets effectively.

This can be hard to get right, so Bazel also provides a memory profiler that
assists you in finding spots where you might have made a mistake. Be warned:
The cost of writing an inefficient rule may not be evident until it is in
widespread use.

## Contents
{:.no_toc}

* ToC
{:toc}

## Use depsets

Whenever you are rolling up information from rule dependencies you should use
[depsets](lib/depset.html). Only use plain lists or dicts to publish information
local to the current rule.

A depset represents information as a nested graph which enables sharing.

Consider the following graph:

```
C -> B -> A
D ---^
```

Each node publishes a single string. With depsets the data looks like this:

```
a = depset(direct=['a'])
b = depset(direct=['b'], transitive=[a])
c = depset(direct=['c'], transitive=[b])
d = depset(direct=['d'], transitive=[b])
```

Note that each item is only mentioned once. With lists you would get this:

```
a = ['a']
b = ['b', 'a']
c = ['c', 'b', 'a']
d = ['d', 'b', 'a']
```

Note that in this case `'a'` is mentioned four times! With larger graphs this
problem will only get worse.

Here is an example of a rule implementation that uses depsets correctly to
publish transitive information. Note that it is OK to publish rule-local
information using lists if you want since this is not O(N^2).

```
MyProvider = provider()

def _impl(ctx):
  my_things = ctx.attr.things
  all_things = depset(
      direct=my_things,
      transitive=[dep[MyProvider].all_things for dep in ctx.attr.deps]
  )
  ...
  return [MyProvider(
    my_things=my_things,  # OK, a flat list of rule-local things only
    all_things=all_things,  # OK, a depset containing dependencies
  )]
```

See the [depset overview](depsets.md) page for more information.

### Avoid calling `depset.to_list()`

You can coerce a depset to a flat list using
[`to_list()`](lib/depset.html#to_list), but doing so usually results in O(N^2)
cost. If at all possible, avoid any flattening of depsets except for debugging
purposes.

A common misconception is that you can freely flatten depsets if you only do it
at top-level targets, such as an `<xx>_binary` rule, since then the cost is not
accumulated over each level of the build graph. But this is *still* O(N^2) when
you build a set of targets with overlapping dependencies. This happens when
building your tests `//foo/tests/...`, or when importing an IDE project.

**Note**: Today it is possible to flatten depsets implicitly by iterating over
the depset the way you would a list, tuple, or dictionary, or by taking the
depset's size via `len()`. This functionality is [deprecated](../skylark/backward-compatibility.html#depset-is-no-longer-iterable).
and will be removed.

### Avoid calling `len(depset)`

It is O(N) to get the number of items in a depset. It is however
O(1) to check if a depset is empty. This includes checking the truthiness
of a depset:

```
def _impl(ctx):
  args = ctx.actions.args()
  files = depset(...)

  # Bad, has to iterate over entire depset to get length
  if len(files) != 0:
    args.add("--files")
    args.add_all(files)

  # Good, O(1)
  if files:
    args.add("--files")
    args.add_all(files)
```

As mentioned above, support for `len(<depset>)` is deprecated.

## Use `ctx.actions.args()` for command lines

When building command lines you should use [ctx.actions.args()](lib/Args.html).
This defers expansion of any depsets to the execution phase.

Apart from being strictly faster, this will reduce the memory consumption of
your rules -- sometimes by 90% or more.

Here are some tricks:

* Pass depsets and lists directly as arguments, instead of flattening them
yourself. They will get expanded by `ctx.actions.args()` for you.
If you need any transformations on the depset contents, look at
[ctx.actions.args#add](lib/Args.html#add) to see if anything fits the bill.

* Are you passing `File#path` as arguments? No need. Any
[File](lib/File.html) is automatically turned into its
[path](lib/File.html#path), deferred to expansion time.

* Avoid constructing strings by concatenating them together.
The best string argument is a constant as its memory will be shared between
all instances of your rule.

* If the args are too long for the command line an `ctx.actions.args()` object
can be conditionally or unconditionally written to a param file using
[`ctx.actions.args#use_param_file`](lib/Args.html#use_param_file). This is
done behind the scenes when the action is executed. If you need to explicitly
control the params file you can write it manually using
[`ctx.actions.write`](lib/actions.html#write).

Example:

```
def _impl(ctx):
  ...
  args = ctx.actions.args()
  file = ctx.declare_file(...)
  files = depset(...)

  # Bad, constructs a full string "--foo=<file path>" for each rule instance
  args.add("--foo=" + file.path)

  # Good, shares "-foo" among all rule instances, and defers file.path to later
  args.add("--foo")
  args.add(file)

  # Use format if you prefer ["--foo=<file path>"] to ["--foo", <file path>]
  args.add(format="--foo=%s", value=file)

  # Bad, makes a giant string of a whole depset
  args.add(" ".join(["-I%s" % file.short_path for file in files])

  # Good, only stores a reference to the depset
  args.add_all(files, format_each="-I%s", map_each=_to_short_path)

# Function passed to map_each above
def _to_short_path(f):
  return f.short_path
```

## Transitive action inputs should be depsets

When building an action using [ctx.actions.run](lib/actions.html?#run), do not
forget that the `inputs` field accepts a depset. Use this whenever inputs are
collected from dependencies transitively.

```
inputs = depset(...)
ctx.actions.run(
  inputs = inputs,  # Do *not* turn inputs into a list
  ...
)
```

## Performance profiling

To profile your code and analyze the performance you have two options:

- use the `--profile` flag and the `analyze-profile` command, or
- use the new `--experimental_generate_json_trace_profile` flag and load the
  resulting JSON profile in `chrome://tracing` (recommended).

### --profile and analyze-profile

This profiling method consists of two steps, first you have to execute your
build/test with the `--profile` flag, for example

```
$ bazel build --nobuild --profile=/tmp/prof //path/to:target
```

The file generated (in this case `/tmp/prof`) is a binary file, which can be
postprocessed and analyzed by the `analyze-profile` command:

```
$ bazel analyze-profile /tmp/prof
```

By default, it prints summary analysis information for the specified profile
datafile. This includes cummaltive statistics for different task types for each
build phase and an analysis of the critical path.

The first section of the default output is an overview of the time spent
on the different build phases:

```
=== PHASE SUMMARY INFORMATION ===

Total launch phase time          978 ms    2.67%
Total init phase time            178 ms    0.49%
Total loading phase time         381 ms    1.04%
Total analysis phase time       2.618 s    7.15%
Total preparation phase time    46.0 ms    0.13%
Total execution phase time     32.366 s   88.46%
Total finish phase time         19.3 ms    0.05%
Total run time                 36.586 s  100.00%
```

The following sections show the execution time of different tasks happening
during that particular phase, for example:

```
=== EXECUTION PHASE INFORMATION ===

Total preparation time                   46.0 ms
Total execution phase time              32.366 s
Total time finalizing build              19.3 ms

Actual execution time                   32.366 s

Total time (across all threads) spent on:
              Type    Total    Count     Average
            ACTION    0.01%      131     0.44 ms
    ACTION_EXECUTE   31.72%      131     2.635 s
              INFO    0.00%       10     1.32 ms
          VFS_STAT    0.02%    32187     0.01 ms
           VFS_DIR    0.01%     4401     0.02 ms
      VFS_READLINK    0.00%        6     0.01 ms
           VFS_MD5    0.50%     4652     1.16 ms
        VFS_DELETE    0.02%     9336     0.03 ms
          VFS_OPEN    0.01%     4706     0.01 ms
          VFS_READ    0.06%   182413     0.00 ms
         VFS_WRITE    0.01%    17691     0.00 ms
              WAIT   16.12%      544      322 ms
     SKYFRAME_EVAL    2.97%        1    32.343 s
       SKYFUNCTION   48.59%    57301     9.23 ms
```

The last section shows the critical path:

```
Critical path (31.410 s):
    Id        Time Percentage   Description
132283     1.219 s    3.88%   action 'Executing genrule //src bazel-bin'
132282    13.591 s   43.27%   action 'Executing genrule //src package-zip'
132281     1.263 s    4.02%   action 'Executing genrule //src install_base_key-file'
132280      381 ms    1.21%   action 'Building deploy jar src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar'
132279     1.10 ms    0.00%   runfiles for //src/main/java/com/google/devtools/build/lib bazel/BazelServer
132278      398 ms    1.27%   action 'Building Java resource jar'
132277     8.292 s   26.40%   action 'Building src/main/java/com/google/devtools/build/lib/libbazel-rules-class.jar (133 source files) and running annotation processors (AutoCodecProcessor, OptionProcessor, AutoAnnotationProcessor, AutoValueProcessor)'
...
```

You can use the following options to display more detailed information:

- `--dump=text`: Print all recorded tasks in the order they occurred.
- `--dump=raw`: Use this for automated analysis with scripts.
- `--html`: Writes a file called `<profile-file>.html` in the directory of the
  profile file. Open it in you browser to see a Gantt type chart that displays
  time on the horizontal axis and threads of execution along the vertical axis.
- `--html_details`: Renders a more detailed execution chart. Beware that this
  increases file size and browser load considerably.

### JSON profile

When you add the flag `--experimental_generate_json_trace_profile` to your Bazel
invocations, Bazel will write a JSON profile in [Chrome Trace
Format](https://docs.google.com/document/d/1CvAClvFfyA5R-PhYUmn5OOQtYMH4h6I0nSsKchNAySU/preview).

We suggest to use the flag together with the following flags:
- `--experimental_json_trace_compression`: The profile will be compressed with
  gzip.
- `--experimental_profile_cpu_usage`: Bazel will measure and include local CPU
  usage in the JSON profile.

Example usage:
```
$ bazel build --experimental_generate_json_trace_profile --experimental_profile_cpu_usage //third_party/zlib/...
INFO: Writing tracer profile to '/home/johndoe/.cache/bazel/_bazel_twerth/f01bc937da326f5bb0feb15c854c110c/command.profile'
INFO: Invocation ID: 34995931-ef6f-4838-9ab1-9a0bc39a8712
INFO: Analysed 3 targets (7 packages loaded, 316 targets configured).
INFO: Found 3 targets...
INFO: Elapsed time: 1.223s, Critical Path: 0.27s
INFO: 17 processes: 17 linux-sandbox.
INFO: Build completed successfully, 20 total actions
```

The resulting profile file
(`/home/johndoe/.cache/bazel/_bazel_twerth/f01bc937da326f5bb0feb15c854c110c/command.profile`
in this case; can be configured by the `--profile=<path>` flag) can then be
loaded and viewed in Chrome. For this, open `chrome://tracing` in a new tab,
click `Load` and pick the profile file.

Example profile:
<img src="profile.png" alt="Example Profile" />

The top row ('cpu counters') shows the local CPU usage, which is high in this
build during the analysis phase and then gets lower during execution.
The second row ('Critical Path') refers to the critical path of the build, that
is even with infinite parallelism the build would not be faster than the actions
in this path.
The third row ('grpc-command-1') displays everything that's happening on
Bazel's main thread, giving a high level overview of what Bazel is doing.
The remaining rows show what the worker threads are doing.

You can interact with the profile, for example zoom in, inspect particular tasks,
filter for task descriptions and select multiple tasks to get an overview. Press
`?` to get an overview of what you can do.

When analyzing these kind of profiles look for the following:
- Slow analysis phase (RunAnalysisPhase), especially on incremental builds. This
  can be a sign of a poor rule implementation, for example one that flattens
  nested sets.
- Individual slow actions, especially those on the critical path. It might be
  possible to split large actions into multiple smaller actions or reduce the
  dependencies to speed them up.
- Bottlenecks, that is a small number of threads is busy while all others are
  idleing waiting for the result. Optimizing this will most likely require
  touching the rule implementations to introduce more parallelism.

Note that we filter out fast tasks and certain task types completely to keep the
profile files small enough to render fast in the Chrome Trace Viewer.

## Memory Profiling

Bazel comes with a built-in memory profiler that can help you check your rule's
memory use. If there is a problem you can dump the heap to find the
exact line of code that is causing the problem.

### Enabling Memory Tracking

You must pass these two startup flags to *every* Bazel invocation:

  ```
  STARTUP_FLAGS=\
  --host_jvm_args=-javaagent:$(BAZEL)/third_party/allocation_instrumenter/java-allocation-instrumenter-3.0.1.jar \
  --host_jvm_args=-DRULE_MEMORY_TRACKER=1
  ```
  **NOTE**: The bazel repository comes with an allocation instrumenter.
  Make sure to adjust '$(BAZEL)' for your repository location.

These start the server in memory tracking mode. If you forget these for even
one Bazel invocation the server will restart and you will have to start over.

### Using the Memory Tracker

Let's have a look at the target `foo` and see what it's up to. We add
`--nobuild` since it doesn't matter to memory consumption if we actually build
or not, we just have to run the analysis phase.

```
$ bazel $(STARTUP_FLAGS) build --nobuild //foo:foo
```

Let's see how much memory the whole Bazel instance consumes:

```
$ bazel $(STARTUP_FLAGS) info used-heap-size-after-gc
> 2594MB
```

Let's break it down by rule class by using `bazel dump --rules`:

```
$ bazel $(STARTUP_FLAGS) dump --rules
>

RULE                                 COUNT     ACTIONS          BYTES         EACH
genrule                             33,762      33,801    291,538,824        8,635
config_setting                      25,374           0     24,897,336          981
filegroup                           25,369      25,369     97,496,272        3,843
cc_library                           5,372      73,235    182,214,456       33,919
proto_library                        4,140     110,409    186,776,864       45,115
android_library                      2,621      36,921    218,504,848       83,366
java_library                         2,371      12,459     38,841,000       16,381
_gen_source                            719       2,157      9,195,312       12,789
_check_proto_library_deps              719         668      1,835,288        2,552
... (more output)
```

And finally let's have a look at where the memory is going by producing a
`pprof` file using `bazel dump --skylark_memory`:

```
$ bazel $(STARTUP_FLAGS) dump --skylark_memory=$HOME/prof.gz
> Dumping skylark heap to: /usr/local/google/home/$USER/prof.gz
```

Next, we use the `pprof` tool to investigate the heap. A good starting point is
getting a flame graph by using `pprof -flame $HOME/prof.gz`.

  You can get `pprof` from https://github.com/google/pprof.

In this case we get a text dump of the hottest call sites annotated with lines:

```
$ pprof -text -lines $HOME/prof.gz
>
      flat  flat%   sum%        cum   cum%
  146.11MB 19.64% 19.64%   146.11MB 19.64%  android_library <native>:-1
  113.02MB 15.19% 34.83%   113.02MB 15.19%  genrule <native>:-1
   74.11MB  9.96% 44.80%    74.11MB  9.96%  glob <native>:-1
   55.98MB  7.53% 52.32%    55.98MB  7.53%  filegroup <native>:-1
   53.44MB  7.18% 59.51%    53.44MB  7.18%  sh_test <native>:-1
   26.55MB  3.57% 63.07%    26.55MB  3.57%  _generate_foo_files /foo/tc/tc.bzl:491
   26.01MB  3.50% 66.57%    26.01MB  3.50%  _build_foo_impl /foo/build_test.bzl:78
   22.01MB  2.96% 69.53%    22.01MB  2.96%  _build_foo_impl /foo/build_test.bzl:73
   ... (more output)
```

repository: chenshuo/bazel

repo_name: chenshuo/bazel


text:
# QSSG: Development Plan

## Code
- Dependency sorting for contentTree.
	- Implement as a visitor?
	- Ask plugins for dependencies

- How to handle mtime for composites and templates?

## Examples
Many concepts would best be explained with a few good examples.

Showcase:
 - build sitemap from visitor
 - create blog with generated RSS feed
 - composite to concat and minify JS & CSS

## Documentation

- design & philosophy
- usage overview
- build process

- API
  - site
  - plugins
  - tree, context tree, composite tree
  - content & visitor

## Test Suite
Create using Mocha? Should `fs` be mocked for test?

license: MIT

repository: shanewholloway/node-qssg

branch_name: refs/heads/master

extension: md

license_type: permissive

repo_name: shanewholloway/node-qssg


text:
# Flask-todo
A todo app written in python's flask.

> This app is under MIT licence, so you can use/tweak it as much as you want. However be warned
>that it's not secured, which means you haev to implement password encryption,
>protect authentication etc.

This ap is mainly for learning purposes. Both to practice my effectiveness and for beginners
to learn from example

### Version
1.0

### Installation

First make a virtual environment in the base folder **(where this file is)** and activate it.

Then run this to install all the required libraries (including flask):

```sh
$ pip install -r requirements.txt
```

Run this to generate app.db:
```sh
$ python db_create.py
```

Use this command every time you make change in *app/models.py*:
```sh
$ python db_update.py
$ python db_migrate.py
```

To run the app after you've created a database use:
```sh
$ python run.py
```
And you are ready to go. Just go to <IP_ADDRESS>:5000

I feel like the rest is self-explanatory. Huge thanks to [Miguel Grinberg](https://github.com/miguelgrinberg) for database scripts.

### Remember to have fun
---

### Todos

 - Heavily comment the code

repository: stefanbro/flask-todo

repo_name: stefanbro/flask-todo


text:
First of all, thank you for contributing. It’s appreciated.

1. Clone the repo and install dependencies with `npm install`.
2. Make a GitHub issue before doing significant amount of work.
3. Run `npm test` to lint and test.
4. Reference the issue’s number in your commit. E.g.: “Did this #12”
5. Make a pull request.

repository: staltz/switch-path

gha_language: TypeScript

gha_license_id: MIT

repo_name: staltz/switch-path


text:
# SaveFilesToDisk
Oracle APEX plugin that saves files uploaded with file browse item to disk

PLUGIN INSTALLATION

First your DBA should grant:

grant EXECUTE on "SYS"."UTL_FILE" to <schema>;

grant EXECUTE on "SYS"."DBMS_CRYPTO" to <schema>;

Next you need to create SaveToDisk and optionally SaveToDiskWithLog function (*.fnc create scripts) to some schema available to APEX workspace. Most usually this would-be APEX parsing schema.

We reference this schema as <schema>.

Prerequisite for creating SaveToDiskWithLog is installed Logger (https://github.com/OraOpenSource/Logger) on <schema>.

Import plugin to your app. Execution Function Name is by default #OWNER#.SaveToDisk. If <schema> is not APEX parsing schema change it to <schema>.SaveToDisk. If you want to use extensive login with Logger (https://github.com/OraOpenSource/Logger) change it to #OWNER#.SaveToDiskWithLog or <schema>.SaveToDiskWithLog.

PLUGIN USE

Create File Browse page item on APEX app page and choose APEX_APPLICATION_TEMP_FILES for storage.

Create page process of type plugin.

Choose process Point: On Submit - After Computations and Validations (default).

File Browse Item: Name of File Browse item (e.g. P1_UPLOAD) which holds filenames of uploaded files. Storage must be APEX_APPLICATION_TEMP_FILES table. Plugin move files from APEX_APPLICATION_TEMP_FILES table to server disk.

Destination Linebreak: Line break in text files after text move. Can be CR LF (Windows) or LF (Linux, MAC)

Max Filesize: Maximum allowed size in bytes. Abbreviations K,KB,M,MB,G,GB are allowed. Examples of valid formats: 100000, 150 K, 3,76 M, 1,5 GB, 1.5G.

Move format:

database directory#source filename#regexp_pattern#regexp_replace#Overwrite old file(Y/N)#Binary or text move(B/T)#Charset#plsqlblock

database directory#source filename#regexp_pattern#regexp_replace#Overwrite old file(Y/N)#Binary or text move(B/T)#Charset#plsqlblock

....

User <schema> needs read, write privileges on each database directory. (e.g. grant READ, WRITE on directory "DIR" to <schema>);

See Move format plugin attribute help or move_format_help.md for more info.

Item with Filenames: Select page item which will save filenames of uploaded files separated with colon. Usually you want to use this item value in process after SaveFilesToDisk process.

Item with Hash values: Select page item to store Hash values of type RAW separated with colon. This is useful when you want to track files changes.

Text area item with log: Select item which will hold plugin log. Display Only or read only Text Area is usual choice. It's highly recommended to display plugin log if you have multiple files and/or complex move format.

Conditions: Select Condition Type "Value of Item/Column in Expression Is NOT NULL" and for Expression1 set the same File Browse item you set at Plug-in settings (e.g. P1_UPLOAD). This step is optional.

For the rest you can left defaults.

SAMPLE APPLICATION

Install sample application fSampleApp.sql, change SaveToDiskMulti plugin process attribute "Move Format" to suit your configuration. You can change also "Destination Linebreak" and "Max Filesize" attributes. Similarly change SaveToDisk plugin process attribute Database directory and possibly "Filename" and "Max Filesize".

repository: mirmas/SaveFilesToDisk

repo_name: mirmas/SaveFilesToDisk


text:
# Frontend Test Description
The project goal is to build a small part of an E-commerce App using VueJs with the following screens: Register, Login, List products and a modal with product details.

## Implementation Details

### Backend API
The frontend app **MUST** consume the following API: https://test-api.updivision.work/docs/1.0/overview

### Registration page
The registration page/form will have the following fields: Name, Email, Education start date, Education end date, Password, Confirm password and a check for “I agree terms and conditions”.

All the fields are required and should display the error messages from the API. Optionally should be validated in the frontend (rules: Minimum 2 characters for name, Valid email, Education start date < Education end date, Password with min 6 characters characters and check to contain at least one uppercase, one lowercase and a number).

### Login page
The login page will contain only email and password and should display the error messages from the API.

For the optional validation the same rules as for registration are applied.

### Product listing page
Display a list of products with name, image, price and details button. When the button is pressed a pop-up will be opened that contains the product description and add to cart button. To view the products list you must be authenticated.

Optionally the product listing will have pagination / infinite loading.

The cart button doesn’t need to work.

### Wireframes
The wireframes should be used **only as a visual guideline** for the implementation.

## Note
The usage of component libraries and tools from the VUE ecosystem is allowed and encouraged.
## Install
npm i
## Run the project via command
vite


text:
---
title: "Default Endpoints"
category: "Features"
order: 11
---

apiato comes shipped with many **useful** API endpoints to help speeding up the development process.

To see all the endpoints in a beautiful documentation run the `php artisan apidocjs:generate` command, after installing the apidocjs tool. See the [API Docs Generator](http://apiato.io/C.features/api-docs-generator/) page for more details.

## Screenshot of the default API Documentation:

![](https://s19.postimg.org/nwxrejwyb/d17df37-_Screen__Shot_2017-02-16_at_5.08.54__PM.png)


text:
---
layout: creature
name: "Spider"
tags: [diminuta, bestia, cr0, manual-de-monstruos]
cha: 2 (-4)
wis: 10 (0)
int: 1 (-5)
con: 8 (-1)
dex: 14 (+2)
str: 2 (-4)
size: Tiny beast
alignment: no alineado
challenge: "0 (10 XP)"
senses: "darkvision 30 ft."
skills: "Sigilo +4"
speed: "20 pies (6 mts), climb 20 ft."
hit_points: "1 (1d4-1)"
armor_class: "12"
---

***Spider Climb.*** The spider can climb difficult surfaces, including upside down on ceilings, without needing to make an ability check.

***Web Sense.*** While in contact with a web, the spider knows the exact location of any other creature in contact with the same web.

***Web Walker.*** The spider ignores movement restrictions caused by webbing.

### Acciones

***Mordisco.*** Ataque de arma cuerpo a cuerpo: +4 al ataque, alcance 5 pies (1.5 mts), una criatura. Impacto: 1 piercing damage, and the target must succeed on a DC 9 Constitution saving throw or take 2 (1d4) poison damage.


text:
---
layout: post
title: Specifications 3: The DSL Strikes Back
date_created: 13th of March, 2015
location: London, UK
author_name: Benjamin Hodgson
author_title: Machine Data and Big Learning Specialist
description: Building a Domain-Specific Language on top of composite specifications
---

tl;dr
-----

You should consider building a domain-specific
language for certain parts of your code.
You can end up with very clear code that can be read
by non-programmers.

Designing a domain-specific language
------------------------------------

In the [last post](Composite-specifications.md),
we developed a few classes that allowed us to combine small,
atomic specifications into more complex ones.
The resulting code was very terse, but not particularly meaningful
to a non-programmer. Our design is DRY (**D**on't **R**epeat **Y**ourself)
but not DAMP (**D**escriptive **A**nd **M**eaningful **P**hrases).

Let's remedy that and build a domain-specific language that resembles English.
The end result is going to look like this:

```csharp
var renameDocumentRule =
  (new UserHasReadPermission().And(new UserCreatedTheDocument())
                              .AndNot(new DocumentIsLocked()))
  .Or(new UserIsWorkspaceManager());
```

The DSL uses _method chaining_ (sometimes known as a _fluent interface_)
to drastically increase the readability of the code;
while the example I started with in the [first post](All-about-security.md)
had _no obvious bugs_, this version has _obviously no bugs_. It's a huge difference!

We're going to set ourselves the goal of building this DSL
on top of the specifications library we developed in the last post,
_without_ changing any of the code we've already written.

Extension methods
-----------------

Me when I use Java

We want to write `And` and `Or` as [_infix_](http://en.wikipedia.org/wiki/Infix_notation)
methods because that's how English works, but `ISpecification`
doesn't contain methods of that name.
In Java (the horror!), you'd need to convert the interface
to an abstract base class and implement `And` and `Or` there.

_Combining_ specifications has nothing to do with _being_ a specification,
so `And` and `Or` don't belong in an abstract base class.
What's more, a class can only have one parent class (though it can implement many interfaces),
so requiring specifications to inherit from an abstract base class is restrictive.

Luckily, we can use C#'s
[extension methods](https://msdn.microsoft.com/en-GB/library/bb383977.aspx)
(indicated by the `this` keyword) to surgically implant new methods
onto our interface from the outside, without changing everything that implements it.

```csharp
static ISpecification<T>; And<T>;(
    this ISpecification<T>; left,
    ISpecification<T>; right)
{
  return new AndSpecification<T>;(left, right);
}
static ISpecification<T>; Or<T>;(
    this ISpecification<T>; left,
    ISpecification<T>; right)
{
  return new OrSpecification<T>;(left, right);
}
```

These methods operate on an `ISpecification<T>`
and return another `ISpecification<T>`.
This means you can call `And` on the result of another call to `And`,
chaining the methods indefinitely.
Incidentally, this is how LINQ works - methods like `Select`
and `Where` are defined as extension methods which take an `IEnumerable`
and return a new `IEnumerable`, allowing you to chain them to one another.

Prefix functions
----------------

To make our DSL read like English, we want to implement a [_prefix_](http://en.wikipedia.org/wiki/Polish_notation)
function called `Not`. But C#'s scoping rules present a sticking point
- if `Not` isn't defined in the current class, we can't use its name unqualified.

We could bring `Not` into scope by defining it in a base class which
all users of our DSL have to inherit from, but that would be
stretching the meaning of inheritance, and has the annoying restriction
that our users can't inherit from any other classes.
If we're willing to give up on the idea of a prefix function,
a more sanitary option is to define new extension methods which
negate their argument:

```csharp
static ISpecification<T>; AndNot<T>;(
    this ISpecification<T>; left,
    ISpecification<T>; right)
{
  return new AndSpecification<T>;(
      left,
      new NotSpecification<T>;(right));
}
static ISpecification<T>; OrNot<T>;(
    this ISpecification<T>; left,
    ISpecification<T>; right)
{
  return new OrSpecification<T>;(
      left,
      new NotSpecification<T>;(right));
}
```

But extension methods need an instance of `ISpecification` to chain from.
What if the specification you want to negate isn't part of a method chain?
The best we can do is write a static method with the
least disruptive name we can think of:

```csharp
public static class Not
{
  public ISpecification<T>; This<T>;(ISpecification<T>; spec)
  {
    return new NotSpecification<T>;(spec);
  }
}

// example
var documentIsNotLocked = Not.This(new DocumentIsLocked());
```

Users still have the option of defining a local function called `Not`
as an alias for `Not.This`. C# 6's upcoming "using static"
feature would solve this problem entirely - I could define a class
containing a static `Not` method and users of the DSL would be able to
import the name directly.

When to use a DSL
-----------------

We've defined a simple, readable domain-specific language for composing
specifications using C#. When does it make sense to build a fluent DSL?

* **Use a DSL for self-contained sections of your business logic**.
    Huddle's specification DSL is limited in scope to
    permission-checking. It would be a mistake to use a fluent
    interface for _all_ usages of Boolean logic in your program!
* **Express small domains**.
    Domain-specific languages are not general-purpose.
    They're at their most effective when there are only a few
    operations within your DSL. Don't fall into the trap of trying to
    create a fully-fledged extensible programming language -
    leave that to the professionals.
* **Mathematical structures are good**.
    It's quite difficult to change the syntax or semantics of
    a domain-specific language because everyone would have to re-learn it.
    This makes them a good fit to model unchanging mathematical
    theories such as the Boolean logic we've been modelling in this example.
    An advantage of building on mathematics is that other people
    have already done the hard work - you'll be able to draw from
    long-established definitions, theorems, and beautiful structures when
    designing and optimising your DSL.
* **Use a DSL for rapidly-changing parts of your system**.
    This is the flip-side of the last point. If the business-oriented details
    of your code change with the winds, the simple and flexible framework
    provided by a DSL can help you immensely.
* **Use a DSL when business-people want to read the code**.
     by other members
    of the team. I can open up the file containing our fluently-defined
    permissions rules and I don't need to explain the code!
    This makes everyone feel good.

In [the next post](Specification-visitor.md),
we'll enterprise-ify this code even more, to make it easier to add new
ways to evaluate specifications.

In this series
--------------

1. [All about security](All-about-security.md)
2. [The power of Composite Specifications](Composite-specifications.md)
3. **Specifications 3: The DSL Strikes Back**
4. [Knock knock. Who's there? AbstractSpecificationNodeVisitorImpl](Specification-visitor.md)

branch_name: refs/heads/main


text:
---
layout: post
title: "IPSEC VPN for connection multiple AWS VPC's"
date: 2014-02-16 15:45:49 -0500
comments: true
categories: [vpc, aws, ipsec, vpn]
---

Getting a VPN working to connect multiple regions is actually pretty simple.  I'll lay out the step's I used to get this working and hopefully it helps someone else.

* Setup security groups

{% highlight bash %}
ec2-create-group ipsec_east -d "Ipsec East" -c <vpc_id> --region us-east-1
ec2-create-group ipsec_west -d "Ipsec West" -c <vpc_id> --region us-west-2
ec2-authorize ipsec_east --protocol udp -p 4500 --region us-east-1 --cidr <IP_ADDRESS>/0
ec2-authorize ipsec_east --protocol udp -p 500 --region us-east-1 --cidr <IP_ADDRESS>/0
ec2-authorize ipsec_west --protocol udp -p 4500 --region us-west-2 --cidr <IP_ADDRESS>/0
ec2-authorize ipsec_west --protocol udp -p 500 --region us-west-2 --cidr <IP_ADDRESS>/0
# Note you can set the cidr address to be the other elastic IP, this would probably be the more secure thing to do, but for demonstrative purposes I didn't
{% endhighlight %}

* Deploy 2 small instances. I used two CentOS 6.4 [AMI's](https://aws.amazon.com/marketplace/pp/B00DGYP804/ref=sp_mpg_product_title?ie=UTF8&sr=0-4).  Use the security groups from step 1, and in my example I am deploying one in east and one in west-2:

{% highlight bash %}
ec2-run-instances ami-bf5021d6 -t t1.small -s subnet-xxxxxxxx -k my-key-pair -g sg-xxxxxxxx --region us-east-1
ec2-run-instances ami-b3bf2f83 -t t1.small -s subnet-xxxxxxxx -k my-key-pair -g sg-xxxxxxxx --region us-west-1
{% endhighlight %}

* Allocate two Elastic IP's, one in east and one in west

{% highlight bash %}
ec2-allocate-address -d vpc --region region us-east-1
ec2-allocate-address -d vpc --region region us-west-2
{% endhighlight %}

* Assign elastic IP's from step 3 to the instances created in step 1.

{% highlight bash %}
ec2-associate-address -i <instance_id> --region us-east-1
ec2-associate-address -i <instance_id> --region us-west-2
{% endhighlight %}

* Turn off source destination checks for each instance

{% highlight bash %}
ec2-modify-instance-attribute <instance_id> --source-dest-check false --region us-east-1
ec2-modify-instance-attribute <instance_id> --source-dest-check false --region us-west-2
{% endhighlight %}

* Edit /etc/sysctl.conf and comment out, or add, the following lines:
{% highlight bash %}
net.ipv4.ip_forward=1
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.eth0.send_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.eth0.accept_redirects = 0
{% endhighlight %}

* Enable sysctl changes from the previous step
    `sudo sysctl -p /etc/sysctl.conf`

* install openswan
    `yum -y install openswan`

* Uncoment ipsec.d setting inside __/etc/ipsec.conf__
    `sed -i -e "s_#include /etc/_include /etc/_" /etc/ipsec.conf`

So for the remainder of this example we are going to esablish some "facts" to help.

{% highlight ruby %}
# Instance West:
west_elastic_ip: <IP_ADDRESS>
west_private_ip: <IP_ADDRESS>
west_vpc_subnet_cidr: <IP_ADDRESS>/16

# Instance East:
east_elastic_ip: <IP_ADDRESS>
east_private_ip: <IP_ADDRESS>
east_vpc_subnet_cidr: <IP_ADDRESS>/16
{% endhighlight %}

* So using these values we'll go ahead and create the tunnel configs on each instance:

## west-east.conf
### /etc/ipsec.d/west-east.conf
{% highlight ruby %}
conn west-east
   authby=secret
   auto=start
   type=tunnel
   left=<IP_ADDRESS>
   leftid=<IP_ADDRESS>
   leftsubnet=<IP_ADDRESS>/16
   right=<IP_ADDRESS>
   rightsubnet=<IP_ADDRESS>/16
   ike=aes256-sha1;modp2048
   dpddelay=30
   dpdtimeout=120
   dpdaction=restart

{% endhighlight %}

## east-west.conf
### /etc/ipsec.d/east-west.conf

{% highlight ruby %}
conn east-west
   authby=secret
   auto=start
   type=tunnel
   left=<IP_ADDRESS>
   leftid=<IP_ADDRESS>
   leftsubnet=<IP_ADDRESS>/16
   right=<IP_ADDRESS>
   rightsubnet=<IP_ADDRESS>/16
   ike=aes256-sha1;modp2048
   dpddelay=30
   dpdtimeout=120
   dpdaction=restart
{% endhighlight %}

- Next we'll want to go ahead and setup the shared secret between the two instances.

  - ### West
  `echo '<IP_ADDRESS> <IP_ADDRESS>: PSK "super_secure_key"' > /etc/ipsec.secrets`

  - ### East
  `echo '<IP_ADDRESS> <IP_ADDRESS>: PSK "super_secure_key"' > /etc/ipsec.secrets`

- Verify config, and make sure there are no FAILED items

{% highlight bash %}
[root@vpn1 ~]# sudo ipsec verify
Checking your system to see if IPsec got installed and started correctly:
Version check and ipsec on-path                               [OK]
Linux Openswan U2.6.32/K2.6.32-358.el6.x86_64 (netkey)
Checking for IPsec support in kernel                          [OK]
 SAref kernel support                                         [N/A]
 NETKEY:  Testing for disabled ICMP send_redirects            [OK]
NETKEY detected, testing for disabled ICMP accept_redirects   [OK]
Checking that pluto is running                                [OK]
 Pluto listening for IKE on udp 500                           [OK]
 Pluto listening for NAT-T on udp 4500                        [OK]
Two or more interfaces found, checking IP forwarding          [OK]
Checking NAT and MASQUERADEing                                [OK]
Checking for 'ip' command                                     [OK]
Checking /bin/sh is not /bin/dash                             [OK]
Checking for 'iptables' command                               [OK]
Opportunistic Encryption Support                              [DISABLED]
[root@vpn1 ~]#
{% endhighlight %}

- chkconfig and start services on both servers
{% highlight bash %}
sudo service ipsec status
sudo chkconfig ipsec on
sudo service ipsec stop
sudo service ipsec start
{% endhighlight %}

- Setup routes in each VPC to route traffic over the VPN instance.  Some illustrations to help with this part can be found [here](http://aws.amazon.com/articles/5472675506466066#_Toc331599186)

{% highlight bash %}
ec2-create-route <route_table_id> -r <IP_ADDRESS>/16  -i <west_instance_id> --region us-west-2
ec2-create-route <route_table_id> -r <IP_ADDRESS>/16 -i <east_instance_id> --region us-east-1
{% endhighlight %}

- Profit

At this point you should be able to connect to an instance in west and ssh to an instance in east.  Make sure your security groups in east have SSH open for the new origin IP ( <IP_ADDRESS>/32 ).

## References:

* http://www.onepwr.org/2012/08/20/link-amazon-vpcs-over-a-ipsec-site-to-site-vpn/
* https://gist.github.com/winhamwr/2871257
* http://aws.amazon.com/articles/5472675506466066

extension: markdown


text:
---
collegename: "COCHIN UNIVERSITY OF SCIENCE AND TECHNOLOGY"
hubname: "Kochi Hub"
slug: "/kochi-hub/cochin-university-of-science-and-technology"
---

## The WIE AG ExeCom Training

```Date: 25 April 2020```

```Attendance: 17```

The WIE AG ExeCom training session was handled by Ms. Mini Ulanat, the WIE Chair of IEEE Kerala Section. All the new WIE members became aware of the opportunities in WIE, the various travel grants, funding, WIE awards, the importance of proper office reporting and documentation of events.

## CUSATWEB Webinar Series

A platform opened by IEEE CUSAT SB & WIE AG for the CUSAT community to share their technical knowledge with their colleagues in the form of webinars. 8 webinars were conducted under the webinar series.

| Topics |
|---------|
| System Design for Mission100kft Payload |
| Wave Star Energy – an Eco-friendly Solution to the Energy Crisis |
| Basics of Networking and Protocols |
| Introduction to UI design and Prototyping |
| Are Hackathons for you? |
| Introduction to Android App Development |
| Introduction to MATLAB |
| Computer Vision |

Topics of Webinars

## INWED 20 Photography Contest

```Date:  25 June 2020```

```Attendance:  16```

IEEE CUSAT WIE AG celebrated this year’s International Women in Engineering Day by conducting a Photography contest with the theme ‘The women Engineer in you’.


text:
# rememberthemilk-rs

Rust crate for the [Remember The Milk](https://www.rememberthemilk.com) API.

This includes a library implementing the implementing the main API methods and
a command-line utility to make use of it.  It is currently very incomplete, but
can do the basics of listing and adding tasks.

I expect the functionality to fill out over time as I want or need them, and
my priorities are influenced by polite feature or pull requests.

# Contributing

Bug reports, feature requests, general queries, and pull requests welcome.  The
CI runs cargo test, cargo fmt and cargo clippy, but these are not a
prerequisite for me accepting a PR - in general if the PR is otherwise good I'm
happy to just fix those up myself before merging.

For any larger changes, I suggest discussing in an issue first to check it's in
the right direction before wasting time.

repository: jugglerchris/rememberthemilk-rs

gha_language: Rust

repo_name: jugglerchris/rememberthemilk-rs


text:
  Carraway

Tool to simplify building a [Gatsby](https://www.gatsbyjs.org/) site. For more information see [Motivation](#motivation) and [Implementation](#implementation) below.

## Installation

```bash
# using npm:

npm install gatsby carraway

# using yarn:

yarn add gatsby carraway
```

## Usage

Create a `gatsby.config.json` file in your repos root or add a `gatsby` field to your `package.json`, with the following structure:

```json
{
  "siteMetadata": {},
  "plugins": []
}
```

This will act the basis for the Gatsby site Carraway builds.

### Building Site

To build your Gatsby site with Carraway, run:

```bash
carraway build
```

### Development Mode

To run your Gatsby site in development mode with Carraway, run:

```bash
carraway develop
```

### Cleaning Up

To clean up the artifacts created by Gatsby and Carraway, run:

```bash
carraway clean
```

## Motivation

While building [my website](https://github.com/keplersj/website), [the website for my surname](https://github.com/keplersj/surname-site), the [AtomLinter website](https://github.com/AtomLinter/atomlinter.github.io), and [AtomBuild website](https://github.com/AtomBuild/atombuild.github.io) I became very interested with idea of separating the content of a website from the code that renders it to a webpage.

Gatsby does a lot of the work for us by letting us isolate how data and content is handled from the React code that renders it. The framework then takes this one step further by allowing you to isolate entire website themes, and slotting in content and data.

Carraway aims to take this functionality to the next step with the goal of creating Gatsby sites with repos that contain no code, just the site's data, content, and configuration.

Sites using Carraway will still be using Gatsby, and you will have to configure it as such. However instead of configuring Gatsby in a `gatsby-config.js` at the repo's root, Carraway sites will be configured using JSON in a `gatsby.config.json` file at the repo's root or under the `gatsby` field in the repo's `package.json`.

### Why Carraway?

In _The Great Gatsby_ the reader follows the tale of narrator Nick Carraway as he recalls his summer with the titular character Jay Gatsby. Through the course of the book Nick befriends Gatsby and helps him attain a lifelong muse.

Similar to this dynamic of classic American literature, our Carraway simply hopes to make things easier for us and Gatsby.

## Implementation

Much like the character its named after, this package is rather simple.

Carraway creates a `gatsby-config.js` for Gatsby to interact with, and invokes Gatsby for you. This lets us declare our site's configuration somewhere else, while providing it to Gatsby when Gatsby needs it.

## Prior Art

- [Gatsby](https://www.gatsbyjs.org/) - powerful static-site generating framework
- [Docz](https://www.docz.site/) - builds a Gatsby site based on repo content

## License

Copyright [Kepler Sticka-Jones](https://keplersj.com) 2019. Licensed ISC.

license: ISC

repository: keplersj/carraway

gha_license_id: ISC

repo_name: keplersj/carraway


text:
# Multiple Account Restore

<PageHeader />

The M-A-R (multiple account restore) utility can be used to restore data in the format of a multiple account save (known as M-A-S on Reality systems). M-A-R invokes the ACCOUNT-RESTORE utility to restore each account.

```
M-A-R {-Options}
```

| Option | Explanation |
| --- | --- |
| -Q | question before restore of each account |
| -V | verify only, no restore |

Any other options specified are passed to the ACCOUNT-RESTORE process.

The M-A-R command attempts to restore all the accounts on the attached tape into the current working directory. The first tape block on the tape is expected to be a T-DUMP of the RESTORE-LIST record. This record is used to restore each Account from the M-A-S.

If a SYSTEM file has been configured then it will be checked before restoring to ensure that the Account as specified by the current value in the RESTORE-LIST, does not already exist. If the Account does exist in the SYSTEM file then the Account will be skipped using a T-FWD command. If the Account does NOT exist in the SYSTEM file but DOES exist in the current directory then the Account will be skipped. Also if the Account Name exists in the current directory as a hash or Unix/NT file then the Account will skipped. If the Account exists as an entry in the SYSTEM file OR as a directory in the current directory AND the Account name appears as an attribute in the **FORCE.RESTORELIST** record in the configured SYSTEM file, then an attempt to restore the Account will occur. Once restored the Account MD will be updated via the UpdateMD command and an entry added to the SYSTEM file.

Two control records can be configured in the SYSTEM file, **FORCE.RESTORELIST**, as previously mentioned, will cause any Accounts specified as an attribute in this record to be restored. A second control record, **SKIP.RESTORELIST**, again configured in the SYSTEM file, will cause all Accounts specified as an attribute in the record to be skipped.

To restore from a Reality M-A-S Save, a T-ATT followed by the M-A-R is all that should be required.

[Back to Articles](./../README.md)

<PageFooter />

repository: josepheshaw/docs

repo_name: josepheshaw/docs


text:
---
# Documentation: https://sourcethemes.com/academic/docs/managing-content/

title: "Faculty Showcase Parallel Computing Research at IPDPS 2019"
subtitle: ""
summary: ""
authors: []
tags: []
categories: []
date: 2019-05-16T14:56:40-04:00
lastmod: 2019-05-16T14:56:40-04:00
featured: false
draft: false

# Featured image
# To use, add an image named `featured.jpg/png` to your page's folder.
# Focal points: Smart, Center, TopLeft, Top, TopRight, Left, Right, BottomLeft, Bottom, BottomRight.
image:
  caption: ""
  focal_point: ""
  preview_only: false

# Projects (optional).
#   Associate this post with one or more of your projects.
#   Simply enter your project's folder or file name without extension.
#   E.g. `projects = ["internal-project"]` references `content/project/deep-learning/index.md`.
#   Otherwise, set `projects = []`.
projects: []
---

Researchers from the School of Computational Science and Engineering (CSE) will present seven papers at the 33rd IEEE International Parallel and Distributed Processing Symposium (IPDPS 2019) in Rio De Janeiro, Brazil, May 20-24.

“IPDPS is one of the premier parallel and distributed computing conferences in the world that provides broad coverage of all areas in high performance computing (HPC) and parallel computing,” said CSE Professor Ümit V. Çatalyürek, one of the leaders of Georgia Tech’s participation at this year’s symposium.

“With seven papers out of 103 in the main conference alone, and many other invited talks, plenary panels and committee services, CSE researchers not only continue to push the research frontier in HPC and parallel processing, but also demonstrate their commitment to serve the parallel processing community at large,” he said.

Several Georgia Tech faculty, including Çatalyürek, are serving on IPDPS committees and workshops.

* Srinivas Aluru – HiCOMB Workshop Steering Committee
* **David A. Bader**– IPDPS Steering Committee, HiCOMB Workshop Steering Committee
* Ümit V. Çatalyürek - Technical Committee on Parallel Processing (TCPP) Chair, IPDPS Steering Committee, Parallel and Distributed Scientific and Engineering Computing Panelist, Algorithms Primary Committee
* Ada Gavrilovska– Regular Program Committee, HPBDC Workshop Panel Moderator
* Vivek Sarkar– Regular Program Committee
* Richard Vuduc – iWAPT Program Committee, iWAPT Steering Committee

IPDPS also serves as the flagship activity of the TC on Parallel Processing (TCPP) which presents another addition to the Georgia Tech at IPDPS roster. CSE Professor and Institute for Data Engineering and Science Co-executive Director Srinivas Aluru was named as this year’s IEEE TCPP Outstanding Service Award winner in recognition of his professional service roles that have had a major impact on the parallel processing research community at large.

Georgia Tech’s presence at IPDPS this year includes 10 researchers from CSE and three researchers from the School of Computer Science (SCS).

Georgia Tech’s research:

* A scalable clustering-based task scheduler for homogeneous processors using DAG partitioning
* Accelerating Sequence Alignment to Graphs
* Asynchronous Multigrid Methods
* Efficient Architecture-Aware Acceleration of BWA-MEM for Multicore Systems
* Load-Balanced Sparse MTTKRP on GPUs
* Overlapping Communications with Other Communications and its Application to Distributed Dense Matrix Computations
* ParILUT -  A Parallel Threshold ILU for GPUs

https://www.cc.gatech.edu/news/621726/faculty-showcase-parallel-computing-research-ipdps-2019

gha_language: Shell


text:
  Piston

A high performance general purpose code execution engine.

---

  About •
  Public API •
  Getting Started •
  Usage •
  Supported Languages •
  Principles •
  Security •
  License

---

# About

Piston is a high performance general purpose code execution engine. It excels at running untrusted and
possibly malicious code without fear from any harmful effects.

It's used in numerous places including:
* [EMKC Challenges](https://emkc.org/challenges),
* [EMKC Weekly Contests](https://emkc.org/contests),
* [Engineer Man Discord Server](https://discord.gg/engineerman),
* [I Run Code (Discord Bot)](https://github.com/engineer-man/piston-bot) bot as well as 1300+ other servers
and 100+ direct integrations.

To get it in your own server, go here: https://emkc.org/run.

# Public API

- Requires no installation and you can use it immediately.
- Reference the Versions/Execute sections below to learn about the request and response formats.

When using the public Piston API, use the base URL:

```
https://emkc.org/api/v1/piston
```

#### GET
```
https://emkc.org/api/v1/piston/versions
```
#### POST
```
https://emkc.org/api/v1/piston/execute
```

> Important Note: The Piston API is rate limited to 5 requests per second. If you have a need for more requests than that
and it's for a good cause, please reach out to me (EngineerMan#0001) on [Discord](https://discord.gg/engineerman)
so we can discuss potentially getting you an unlimited key.

# Getting Started

### Host System Package Dependencies

* NodeJS
* lxc
* libvirt

If your OS is not documented below, please open pull requests with the correct commands for your OS.

CentOS / RHEL

```sh
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.37.2/install.sh | bash
nvm install --lts
nvm use --lts

yum install -y epel-release
yum install -y lxc lxc-templates debootstrap libvirt
systemctl start libvirtd
```

Ubuntu (18.04)

```sh
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.37.2/install.sh | bash
nvm install --lts
nvm use --lts

apt install -y lxc lxc-templates debootstrap libvirt0
```

Arch Linux

```sh
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.37.2/install.sh | bash
nvm install --lts
nvm use --lts

pacman -S lxc libvirt unzip
```

#### After system dependencies are installed, clone this repository:

```sh
# clone and enter repo
git clone https://github.com/engineer-man/piston
```

#### Installation (simple)

- Install additional dependencies python3, pip and distrobuilder
- `cd container && ./build.sh`
- Wait, it may take up to an hour.
- `lxc-create -n piston -t local -- --metadata meta.tar.xz --fstree rootfs.tar.xz`
- `cd lxc && ./start`
- Good to go!

#### Installation (advanced)

- See `var/install.txt` for how to build the container manually

#### CLI Usage
- `cli/execute [language] [file path] [args]`

# Usage

### CLI

```sh
lxc/execute [language] [file path] [args]
```

### API
To use the API, it must first be started. Please note that if root is required to access
LXC then the API must also be running as root. To start the API, run the following:

```
cd api
./start
```

For your own local installation, the API is available at:

```
http://<IP_ADDRESS>:2000
```

#### Versions Endpoint
`GET /versions`
This endpoint will return the supported languages along with the current version and aliases. To execute
code for a particular language using the `/execute` endpoint, either the name or one of the aliases must
be provided.
```json
HTTP/1.1 200 OK
Content-Type: application/json

[
    {
        "name": "awk",
        "aliases": ["awk"],
        "version": "1.3.3"
    },
    {
        "name": "bash",
        "aliases": ["bash"],
        "version": "4.4.20"
    },
    {
        "name": "c",
        "aliases": ["c"],
        "version": "7.5.0"
    }
]
```

#### Execute Endpoint
`POST /execute`
This endpoint requests execution of some arbitrary code.
- `language` (**required**) The language to use for execution, must be a string and supported by Piston (see list below).
- `source` (**required**) The source code to execute, must be a string.
- `stdin` (*optional*) The text to pass as stdin to the program. Must be a string or left out of the request.
- `args` (*optional*) The arguments to pass to the program. Must be an array or left out of the request.
```json
{
    "language": "js",
    "source": "console.log(process.argv)",
    "stdin": "",
    "args": [
        "1",
        "2",
        "3"
    ]
}
```
A typical response upon successful execution will contain the `language`, `version`, `output` which
is a combination of both `stdout` and `stderr` but in chronological order according to program output,
as well as separate `stdout` and `stderr`.
```json
HTTP/1.1 200 OK
Content-Type: application/json

{
    "ran": true,
    "language": "js",
    "version": "12.13.0",
    "output": "[ '/usr/bin/node',\n  '/tmp/code.code',\n  '1',\n  '2',\n  '3' ]",
    "stdout": "[ '/usr/bin/node',\n  '/tmp/code.code',\n  '1',\n  '2',\n  '3' ]",
    "stderr": ""
}
```
If a problem exists with the request, a `400` status code is returned and the reason in the `message` key.
```json
HTTP/1.1 400 Bad Request
Content-Type: application/json

{
    "message": "Supplied language is not supported by Piston"
}
```

# Supported Languages
`awk`,
`bash`,
`brainfuck`,
`c`,
`cpp`,
`clojure`,
`crystal`,
`csharp`,
`d`,
`dash`,
`deno`,
`elixir`,
`emacs`,
`elisp`,
`go`,
`haskell`,
`java`,
`jelly`,
`julia`,
`kotlin`,
`lisp`,
`lolcode`,
`lua`,
`nasm`,
`nasm64`,
`nim`,
`node`,
`osabie`,
`paradoc`,
`perl`,
`php`,
`python2`,
`python3`,
`ruby`,
`rust`,
`scala`,
`swift`,
`typescript`,
`zig`,

# Principle of Operation
Piston utilizes LXC as the primary mechanism for sandboxing. There is a small API written in Node which takes
in execution requests and executes them in the container. High level, the API writes
a temporary source and args file to `/tmp` and that gets mounted read-only along with the execution scripts into the container.
The source file is either ran or compiled and ran (in the case of languages like c, c++, c#, go, etc.).

# Security
LXC provides a great deal of security out of the box in that it's separate from the system.
Piston takes additional steps to make it resistant to
various privilege escalation, denial-of-service, and resource saturation threats. These steps include:
- Disabling outgoing network interaction
- Capping max processes at 64 (resists `:(){ :|: &}:;`, `while True: os.fork()`, etc.)
- Capping max files at 2048 (resists various file based attacks)
- Mounting all resources read-only (resists `sudo rm -rf --no-preserve-root /`)
- Cleaning up all temp space after each execution (resists out of drive space attacks)
- Running as a variety of unprivileged users
- Capping runtime execution at 3 seconds
- Capping stdout to 65536 characters (resists yes/no bombs and runaway output)
- SIGKILLing misbehaving code

# License
Piston is licensed under the MIT license.

repository: getoffmyhack/piston

repo_name: getoffmyhack/piston


text:
# NIH COMmunity: COVID-19 Ongoing Monitoring - Phase 1

[NIH-supported](https://www.nih.gov/news-events/news-releases/nih-awards-contracts-develop-innovative-digital-health-technologies-covid-19) effort to rapidly integrate data from multiple sources, including surveys and wearable sensors, to identify individuals who may have undiagnosed COVID-19.

This repository provides ML models for detecting the onset of COVID-19-like symptoms from wearable data and a survey data-based model with the potential to differentiate flu-like symptoms from COVID-19.

### Content

1. _Installation_ - see package [requirements](./requirements.txt) (install via `pip` or `conda`)

2. See [example notebook](./0__example_notebook.ipynb) which includes a walk-through of training and evaluating a COVID-19 onset detection model using synthetic data
    * Invokes functions from [utils.py](./utils.py)

3. Trained [wearable model](./trained_models/wearable_model.json) for reapplication to new datasets -- this can be subsituted in the example notebook above and reapplied to actual data

### Example

* **Synthetic data** is used to illustrate the model training and evaluation pipeline. Using the [default generation function and parameters](https://github.com/evidation-opensource/nih-community/blob/219f2d50b174edb31896f212a1d15287519e3b89/utils.py#L40) provided, discriminating signal around an Influenza-like Illness (**ILI** e.g. `Flu` or `COVID-19`) event onset can be seen in the normalized heart-rate derived features.

![Signal around event onset](./images/signal_around_onset.png)

* An `XGBoost` model with these 4 features and 5 days on lagging data is trained to disciminate between Healthy (`Class-0`) and COVID-19 (`Class-1`) days. Missing data is also handled by the model (potentially _informative missingness_) and performance decay (towards the end of the x-axis) with all-missing inputs can be seen.

![Missingness performance](./images/missingness_performance.png)

* Validation AUROC is used to select a model after [hyperparameter tuning](https://github.com/evidation-opensource/nih-community/blob/219f2d50b174edb31896f212a1d15287519e3b89/utils.py#L170) and a threshold (for prediction of COVID-19 events) is selected based on a `95% specificify` cut-off

```
95% Specificity cutoff = 0.8778
              precision    recall  f1-score   support

           0       0.85      0.95      0.90       385
           1       0.83      0.57      0.68       154

    accuracy                           0.84       539
   macro avg       0.84      0.76      0.79       539
weighted avg       0.84      0.84      0.83       539
```

![Validation ROC](./images/validation_ROC.png)

* **Cumulative recall** of individual participants around event onset is used to evaluate performance of the COVID-19 detection model. However, a model trained without explicitly accounting for other ILI events is suseptible to confusing non-COVID-19 ILI events (red and blue lines in plot below) as COVID-19 (orange line)

![Cumulative recall around onset](./images/cumulative_recall_around_onset.png)

repository: erchiw/nih-community

repo_name: erchiw/nih-community


text:
---
title: 'JS: Get Account Comments'
position: 9
description: "_By the end of this tutorial you should know how to retrieve account comments from the steem blockchain_"
layout: full
---
Full, runnable src of [Get Account Comments](https://github.com/steemit/devportal-tutorials-js/tree/master/tutorials/09_get_account_comments) can be downloaded as part of the [JS tutorials repository](https://github.com/steemit/devportal-tutorials-js).

This tutorial will show how to fetch comments made by a specific account (in this case `@steemitblog`) by demonstrating how to use the `get_state` api function call. We will also demonstrate the most commonly used fields from the response object as well as how to parse the body of each comment.

## Intro

We are using the `get_state` function with `dsteem` that returns the current state of the network as well as additional content. Each content body is written in markdown and could be submitted to the blockchain by many different applications built on top of Steem. For that reason we are using the `remarkable` npm package to parse markdown in a readable format.

## Steps

1.  [**App setup**](#app-setup) Configuration of `dsteem` to use the proper connection and network.
1.  [**Query**](#query) Query the path which we want to extract from Steem blockchain state.
1.  [**Formatting**](#formatting) Formatting the JSON object to be viewed in a simple user interface.

#### 1. App setup

Below we have `dsteem` pointing to the main network with the proper chainId, addressPrefix and connection server.
There is a `public/app.js` file which holds the Javascript segment of this tutorial. In the first few lines we define and configure library and packages.

```javascript
const dsteem = require('dsteem');
let opts = {};
//connect to production server
opts.addressPrefix = 'STM';
opts.chainId =
    '0000000000000000000000000000000000000000000000000000000000000000';
//connect to server which is connected to the network/production
const client = new dsteem.Client('https://api.steemit.com');

const Remarkable = require('remarkable');
const md = new Remarkable({ html: true, linkify: true });
```

`remarkable` is assigned to the variable `md` with linkify and html options, allowing us to parse markdown links and html properly.

#### 2. Query

Next, we have the `main` function which runs when the page is loaded.

```javascript
// query string, fetching comments made by @steemitblog account
const query = '/@steemitblog/comments';

client.database.call('get_state', [query]).then(result => {
    // work with state object
});
```

`query` is the path from where want to extract Steem blockchain state. In our example we are querying `comments` from the `@steemitblog` account. The result will be the current state object with various information as well as the `content` property holding the content of the query.

The following is an example of the returned object:

```json
{
    "current_route":"/@steemitblog/comments",
    "props":{
        "head_block_number":22307429,
        "head_block_id":"01546265c9dc3e761add4c4b652743e3c640fa19",
        "time":"2018-05-10T12:15:30",
        "current_witness":"smooth.witness",
        "total_pow":514415,
        "num_pow_witnesses":172,
        "virtual_supply":"271970374.699 STEEM",
        "current_supply":"268140818.508 STEEM",
        "confidential_supply":"0.000 STEEM",
        "current_sbd_supply":"13342173.771 SBD",
        "confidential_sbd_supply":"0.000 SBD",
        "total_vesting_fund_steem":"191002132.498 STEEM",
        "total_vesting_shares":"388786707656.308148 VESTS",
        "total_reward_fund_steem":"0.000 STEEM",
        "total_reward_shares2":"0",
        "pending_rewarded_vesting_shares":"366359809.533218 VESTS",
        "pending_rewarded_vesting_steem":"178575.754 STEEM",
        "sbd_interest_rate":0,
        "sbd_print_rate":10000,
        "maximum_block_size":65536,
        "current_aslot":22373110,
        "recent_slots_filled":"340282366920938463463374607431768211455",
        "participation_count":128,
        "last_irreversible_block_num":22307411,
        "vote_power_reserve_rate":10,
        "average_block_size":14881,
        "current_reserve_ratio":200000000,
        "max_virtual_bandwidth":"264241152000000000000"
    },
    "tag_idx":{
        "trending":["","life","photography","steemit","kr","introduceyourself","bitcoin","art","travel","cryptocurrency","spanish","food","steem","blog","funny","news","nature","colorchallenge","dtube","indonesia","story","cn","money","music","writing","crypto","contest","busy","health","poetry","meme","video","utopian-io","photo","new","love","blockchain","deutsch","dmania","science","technology","aceh","entertainment","gaming","politics","myanmar","esteem","sports","fun","tr"]
    },
    "tags":{},
    "content":{
        "steemitblog/afm007-re-steemitblog-devportal-update-3-ux-improvements-more-javascript-tutorials-and-more-20180509t050215510z":{
            "id":47669989,
            "author":"steemitblog",
            "permlink":"afm007-re-steemitblog-devportal-update-3-ux-improvements-more-javascript-tutorials-and-more-20180509t050215510z",
            "category":"steem",
            "parent_author":"afm007",
            "parent_permlink":"devportal-update-3-ux-improvements-more-javascript-tutorials-and-more","title":"","body":"I want to learn the Python language.",
            "json_metadata":"{''}",
            "last_update":"2018-05-09T05:02:15",
            "created":"2018-05-09T05:02:15",
            "active":"2018-05-09T05:02:15",
            "last_payout":"1970-01-01T00:00:00",
            "depth":1,
            "children":0,
            "net_rshares":1057692008,
            "abs_rshares":1057692008,
            "vote_rshares":1057692008,
            "children_abs_rshares":0,
            "cashout_time":"2018-05-16T05:02:15",
            "max_cashout_time":"1969-12-31T23:59:59",
            "total_vote_weight":32523,
            "reward_weight":10000,
            "total_payout_value":"0.000 SBD",
            "curator_payout_value":"0.000 SBD",
            "author_rewards":0,
            "net_votes":1,
            "root_author":"steemitblog",
            "root_permlink":"devportal-update-3-ux-improvements-more-javascript-tutorials-and-more",
            "max_accepted_payout":"1000000.000 SBD",
            "percent_steem_dollars":10000,
            "allow_replies":true,
            "allow_votes":true,
            "allow_curation_rewards":true,
            "beneficiaries":[],
            "url":"/steem/@steemitblog/devportal-update-3-ux-improvements-more-javascript-tutorials-and-more#@afm007/afm007-re-steemitblog-devportal-update-3-ux-improvements-more-javascript-tutorials-and-more-20180509t050215510z",
            "root_title":"DevPortal Update #3: UX Improvements, More Javascript Tutorials and More!",
            "pending_payout_value":"0.005 SBD",
            "total_pending_payout_value":"0.000 STEEM",
            "active_votes":[{"voter":"afm007","weight":17182,"rshares":1057692008,"percent":10000,"reputation":"855556264424","time":"2018-05-09T05:18:06"}],
            "replies":[],
            "author_reputation":"855556264424",
            "promoted":"0.000 SBD",
            "body_length":0,
            "reblogged_by":[]
        },
        "steemitblog/re-steemitblog-devportal-update-3-ux-improvements-more-javascript-tutorials-and-more-20180509t045305223z":{
            "id":47669080,
            "author":"steemitblog",
            "permlink":"re-steemitblog-devportal-update-3-ux-improvements-more-javascript-tutorials-and-more-20180509t045305223z",
            "category":"steem",
            "parent_author":"andreina89",
            "parent_permlink":"devportal-update-3-ux-improvements-more-javascript-tutorials-and-more",
            "title":"",
            "body":"Excellent post very interesting friend, thanks",
            "json_metadata":"{\"tags\":[\"steem\"],\"app\":\"steemit/0.1\"}",
            "last_update":"2018-05-09T04:53:21",
            "created":"2018-05-09T04:53:21",
            "active":"2018-05-09T04:53:27",
            "last_payout":"1970-01-01T00:00:00",
            "depth":1,
            "children":1,
            "net_rshares":0,
            "abs_rshares":0,
            "vote_rshares":0,
            "children_abs_rshares":0,
            "cashout_time":"2018-05-16T04:53:21",
            "max_cashout_time":"1969-12-31T23:59:59",
            "total_vote_weight":0,
            "reward_weight":10000,
            "total_payout_value":"0.000 SBD",
            "curator_payout_value":"0.000 SBD",
            "author_rewards":0,
            "net_votes":0,
            "root_author":"steemitblog",
            "root_permlink":"devportal-update-3-ux-improvements-more-javascript-tutorials-and-more",
            "max_accepted_payout":"1000000.000 SBD",
            "percent_steem_dollars":10000,
            "allow_replies":true,
            "allow_votes":true,
            "allow_curation_rewards":true,
            "beneficiaries":[],
            "url":"/steem/@steemitblog/devportal-update-3-ux-improvements-more-javascript-tutorials-and-more#@andreina89/re-steemitblog-devportal-update-3-ux-improvements-more-javascript-tutorials-and-more-20180509t045305223z",
            "root_title":"DevPortal Update #3: UX Improvements, More Javascript Tutorials and More!",
            "pending_payout_value":"0.000 SBD",
            "total_pending_payout_value":"0.000 STEEM",
            "active_votes":[],
            "replies":[],
            "author_reputation":"174938588721",
            "promoted":"0.000 SBD",
            "body_length":0,"reblogged_by":[]
        },
        {"etc.":"etc."}
    },
    "accounts":{
        "steemitblog:{
            "etc.":"etc."
        },
    },
    "witnesses":{},
    "discussion_idx":{},
    "witness_schedule":{
        "id":0,
        "current_virtual_time":"326078326927286190874576091",
        "next_shuffle_block_num":22307439,
        "current_shuffled_witnesses":["furion","someguy123","good-karma","blocktrades","smooth.witness"],
        "num_scheduled_witnesses":21,
        "top19_weight":1,
        "timeshare_weight":5,
        "miner_weight":1,
        "witness_pay_normalization_factor":25,
        "median_props":{
            "account_creation_fee":"0.100 STEEM",
            "maximum_block_size":65536,
            "sbd_interest_rate":0
        },
        "majority_version":"0.19.3",
        "max_voted_witnesses":20,
        "max_miner_witnesses":0,
        "max_runner_witnesses":1,
        "hardfork_required_witnesses":17
    },
    "feed_price":{
        "base":"3.484 SBD",
        "quote":"1.000 STEEM"
    },
    "error":""
}
```

#### 3. Formatting

Next we will format the above object properly to view in a simple user interface. From the above object, we are only interested in the `content` object which holds the data we require.

```javascript
if (
    !(
        Object.keys(result.content).length === 0 &&
        result.content.constructor === Object
    )
) {
    var comments = [];
    Object.keys(result.content).forEach(key => {
        const comment = result.content[key];
        const parent_author = comment.parent_author;
        const parent_permlink = comment.parent_permlink;
        const created = new Date(comment.created).toDateString();
        const body = md.render(comment.body);
        const netvotes = comment.net_votes;
        comments.push(
            `<div class="list-group-item list-group-item-action flex-column align-items-start">\
            <div class="d-flex w-100 justify-content-between">\
              <h6<EMAIL_ADDRESS>              <small class="text-muted">${created}</small>\
            </div>\
            <p class="mb-1">${body}</p>\
            <small class="text-muted">▲ ${netvotes}, Replied to: @${parent_author}/${parent_permlink}</small>\
          </div>`
        );
    });
    document.getElementById('comments').style.display = 'block';
    document.getElementById('comments').innerHTML = comments.join('');
}
```

We first check if `content` is not an empty object. We then iterate through each object in `content` and extract:

*   `parent_author`
*   `parent_permlink`
*   and the post or comment the `@steemitblog` account is replying to

We format `created` date and time, parse `body` markdown and get `net_votes` on that comment.
Each line is then pushed and displayed separately.

### To Run the tutorial

1.  clone this repo
1.  `cd tutorials/09_get_account_comments`
1.  `npm i`
1.  `npm run dev-server` or `npm run start`
1.  After a few moments, the server should be running at [http://localhost:3000/](http://localhost:3000/)

---

repository: steemit/devportal

branch_name: refs/heads/develop

gha_language: CSS

repo_name: steemit/devportal


text:
---
type: rule
title: SharePoint - Do you review usage?
uri: sharepoint-usage
authors:
  - title: Jean Thirion
    url: https://www.ssw.com.au/people/jean-thirion
related:
  - rules-to-better-google-analytics-reports
created: 2021-11-17T05:37:10.823Z
guid: b757f0c9-a2ed-4acc-8fa9-47577557a1c3
---
Usage analytics is quite important - it allows you to identify popular and unused pages or sites, but also helps measuring users' engagement.

There are 3 places to check for Usage reports in SharePoint:

<!--endintro-->

1. Page Usage

![Figure: Access page usage from any page by clicking "Analytics"](sharepoint-page-usage.jpg)

2. Site Usage

Site owners can access global site usage reports:

![Figure: Access site level usage via "Settings" | "Site Usage"](sharepoint-site-usage.jpg)

3. Tenant Usage

This is by far the most interesting and import report. You need special (tenant-level) permissions to access those usage reports

![Figure: Access tenant level usage via "Office 365 Admin Center" | "Reports" | "Usage" | "View More" (under SharePoint section)](sharepoint-tenant-usage.jpg)

---

Now enjoy the full power of SharePoint usage analytics !

![Figure: Plenty of very useful reports in this dashboard](sharepoint-tenant-usage-home.jpg)

::: info
You will need SysAdmin help to get access to those reports.
:::

gha_language: TSQL


text:
---
name: Thrax
rarity: 5
series: ds9
memory_alpha:
bigbook_tier: 9
events: 7
in_portal: true
date: 04/04/2019
obtained: Mega
mega: true
published: true
---

Red's mega welcoming party went something like "Uuggghh another CMD/DIP/SEC?" and his daily work has predictably not been great since then, but as we navigate WRG's collection fun house, Thrax's rather alluring trait set may be his redemption arc. Seven non-HFS traits is a bounty and while they've not yet paid off on the weekend, many of them are not so large that they can't catch bonuses, especially as a sidecar to a new collection.

The only quantifiable, daily use attribute worth discussing at this point is his usable SEC proficiency. Maybe new players get some one star use out of that (along with his SEC/Investigator node unlock in the Celestial Temple), but with so much skill set competition and a lack of 45%+ crit gauntlets, Thrax won't hold up in that area as your roster grows.

repository: joshurtree/datacore-website

repo_name: joshurtree/datacore-website


text:
# 自动浇花套件V2.0

SKU:KIT0003

<img src="https://github.com/jeff111203/ext.zdjh-master/blob/master/arduinoC/_images/featured.png" width="450" height="300" align=center>

# 用户库教程

教程链接：[Mind+官方文档](http://wiki.dfrobot.com.cn/index.php?title=(SKU:KIT0003)%E8%87%AA%E5%8A%A8%E6%B5%87%E8%8A%B1%E5%A5%97%E4%BB%B6V2.0#.E9.85.8D.E9.80.81.E6.B8.85.E5.8D.95).

---------------------------------------------------------

## Table of Contents

* [URL](#url)
* [Summary](#summary)
* [Blocks](#blocks)
* [License](#license)
* [Supported targets](#Supportedtargets)

## URL
project URL: ```https://github.com/jeff111203/ext.zdjh-master```

## Summary
Based on the design of Arduino Leonardo controller, the current temperature and humidity and soil humidity can be monitored in real time by using soil humidity sensor and air temperature and humidity sensor. The humidity threshold can be set manually. When the humidity is lower than a certain value, the water pump will be started automatically. It is suitable for different kinds of plants. On board an XBee interface, which can plug in WiFi, Bluetooth, ZigBee and other wireless devices, connect to the local gateway, so as to realize the remote control of the Internet of things. Even if you are not at home, you can automatically take care of your flowers and plants to avoid your worries.

## Blocks

![image](https://github.com/DFRobot/ext-oled12864/blob/master/arduinoC/_images/blocks.png)

Get a purchase connection: [store](https://www.dfrobot.com.cn/index.php).

## License

MIT

## Supported targets

MCU                | JavaScript    | Arduino   | MicroPython    | Remarks
------------------ | :----------: | :----------: | :---------: | -----
leonardo        |             |       √       |             |


text:
---
layout: article
title: "Audiovisual performance with Bay Dam"
image:
  teaser: grigri-0.png
---

During the residence in Madrid of the artist and programmer Bay Dam on the occasion of the Grigri pixel 2017, AVfloss has collaborated in the workshop and the production of a video mapping performance in EVA Arganzuela, the old fruit market of Legazpi.

This project is the collaborative work of the participants in the workshop "video mapping with MAPMAP" that Bai Dam did the week before the performance.

Subsequently, a Hackathon was held for the development of the free software program MAPMAP.

Part of the group preparing the performance the previous day

Bay Dam at the workshop

gha_language: SCSS


text:
# PelotonHashTable
Implementations of hash tables for CMUDB/peloton to validate a series of assumptions and implementations

There are currently three implementations in this repo:

HashTable_OA_KVL: Open addressing with Key-Value-List to hold duplicated values for the same key
HashTable_CA_CC: Closed addressing with collision chain as collision resolution strategy
HashTable_CA_SCC: Closed addressing with collision chain, but unlike the previous one, it does not chain all buckets together for easiness of deleting entries (so this hash table does not support removal, but it is faster)


text:
# Intro to Combining Shiny & R Markdown

Original Report here:

https://beta.rstudioconnect.com/content/2671/Combining-Shiny-R-Markdown.html

Example code and links for Intro to Combining Shiny & R Markdown.

Requires the following packages from CRAN:

```r
install.packages(c("rmarkdown","flexdashboard", "shiny"))
```

An up-to-date version of RStudio is also recommended.

R 3.2.2 used for examples.

## Tools

I will compile my thesis using `bookdown`. Here is a minimal example of a book based on R Markdown and **bookdown** (https://github.com/rstudio/bookdown).


text:
## Prerequisites

Download [löve2d](https://love2d.org/)

Download a text editor, for example [vscode](https://code.visualstudio.com/) or [sublime text](https://www.sublimetext.com/)

Make sure you can run [löve](https://love2d.org/wiki/Getting_Started). You can test with the folder 1.

## Goal

We are going to build the classic game Pong step by step.

## How to use

Each numbered step is complete in itself. It will have an example of how to do something and will then have a challenge for you to do.

## Research

https://en.wikipedia.org/wiki/Pong

https://www.youtube.com/watch?v=fiShX2pTz9A

repository: madboy/from-rectangle-to-pong

repo_name: madboy/from-rectangle-to-pong


text:
+++
date = "2013-03-23T11:40:41+01:00"
title = "Index Attribute in EF 6.1"
tags = ["Entity Framework", "EF6.1", ".NET"]
menu = ""
 banner = "banners/entity-framework.png"
+++

Early last week Entity Framework 6.1 [went RTM](http://blogs.msdn.com/b/adonet/archive/2014/03/17/ef6-1-0-rtm-available.aspx).

Amongst the usual performance improvements and a bunch of other stuff that will probably come in useful was the addition of [Index Attributes](http://msdn.microsoft.com/en-US/data/jj591583#Index), which allow the developer to create indexes by annotating their code first models.

This attribute allows the creation of unique indexes as well as supporting multi-column indexes.

A project I'm working on at the moment has the following entity

    public class Pledge
    {
	    public int Id {get;set;}
        public int AttendeeId {get;set;}
        public int ProjectId {get;set;}
        public int EventId {get;set;}

        //other properties
    }

A Pledge must be unique to an Attendee, an Event and a Project

Prior to EF 6.1, there was no way to annotate your properties and have EF create the database with the indexes already set up. You would have had to create your database then write a migration specfically for the index, like this.

	public partial class AddUniqueIndexToPledge : DbMigration
    {
        public override void Up()
        {
            CreateIndex("dbo.Pledge",new[]{"AttendeeId","ProjectId", "EventId"},true);
        }

        public override void Down()
        {
            DropIndex("dbo.Pledge", new[]{"AttendeeId","ProjectId", "EventId"});
        }
    }

Not so difficult, but it's still an extra step rather than just creating the database with the indexes already added.

Now with Entity Framework 6.1 we can do the below

    public class Pledge
    {
        public int Id {get;set;}
        [Index("IX_AttendeeEventProjectIndex", IsUnique = true, IsClustered = false)
        public int AttendeeId {get;set;}
        [Index("IX_AttendeeEventProjectIndex", IsUnique = true, IsClustered = false)
        public int ProjectId {get;set;}
        [Index("IX_AttendeeEventProjectIndex", IsUnique = true, IsClustered = false)
        public int EventId {get;set;}

    }

Which will create the following index

    CREATE UNIQUE NONCLUSTERED INDEX [IX_AttendeeEventProjectIndex] ON [dbo].[Pledges]
    (
    [AttendeeId] ASC,
    [EventId] ASC,
    [ProjectId] ASC
    )WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
    GO

repository: MrBliz/DevBlog

repo_name: MrBliz/DevBlog


text:
# NatML

![NatML](.media/wall.png)

NatML is a high-performance cross-platform machine learning runtime for Unity Engine. Designed for app and game developers, NatML completely removes the need to have any experience with machine learning in order to take advantage of the feautres that they can provide. Features include:

- **Bare Metal Performance**. NatML takes advantage of hardware machine learning accelerators, like CoreML on iOS and macOS, NNAPI on Android, and DirectML on Windows. As a result, it is [multiple times faster](https://github.com/natsuite/ML-Bench) than Unity's own Barracuda engine.

- **Extremely Easy to Use**. NatML exposes machine learning models with simple classes that return familiar data types, with all conversions to and from the model handled for you. No need to write pre-processing scripts or shaders, wrangle tensors, or anything of that sort.

- **Full Support for ONNX**. NatML supports the full ONNX specification, with all layers supported on the CPU, and a large number supported on dedicated hardware accelerators. You can drag and drop any ONNX model from anywhere and run it, no conversions needed.

- **Cross Platform**. NatML supports Android, iOS, macOS, and Windows alike. As a result, you can build your app once, test it in the Editor, and deploy it to the device all in one seamless workflow.

- **Growing Ecosystem**. NatML is designed with a singular focus on applications. As a result, there is a growing ecosystem of predictor packages for ML models and applications that run on NatML. [Check out NatML Hub](https://hub.natml.ai).

- **Augmented Reality**. NatML is particularly suited for augmented reality because it delegates work to ML accelerators, freeing up the GPU to render your app smoothly.

- **Lightweight Package**. NatML is distributed in a self-contained package, with no external dependencies. As a result, you can simply import the package and get going--no setup necessary.

## Installing NatML
Add the following items to your Unity project's `Packages/manifest.json`:
```json
{
  "scopedRegistries": [
    {
      "name": "NatSuite Framework",
      "url": "https://registry.npmjs.com",
      "scopes": ["api.natsuite"]
    }
  ],
  "dependencies": {
    "api.natsuite.natml": "1.0.5"
  }
}
```

## Discover ML Models on Hub
**[Create an account on NatML Hub](https://hub.natml.ai/)** to find and download ML predictors to use in your project!

![NatML Hub](.media/hub.png)

You can also [upload your models to Hub](https://hub.natml.ai/predictor/create) and make them private or public. [Check out the online documentation](https://docs.natml.ai/advanced/authoring) for information on authoring predictors.

## Using ML Models in Three Simple Steps
You will always use NatML in three steps. First, create a **model** from model data:
```csharp
// Fetch model data from NatML Hub
var modelData = await MLModelData.FromHub("@author/some-model");
// Deserialize the model
var model = modelData.Deserialize();
```

Then create a **predictor** to make predictions with the model:
```csharp
// Create a predictor for the model
var predictor = new SomePredictor(model);
```

Finally, make predictions with the predictor:
```csharp
// Make prediction on an image
Texture2D input = ...;
var someOutput = predictor.Predict(input);
```

Different predictors accept and produce different data types, but the usage pattern will always be the same.

___

## Requirements
- Unity 2019.2+

## Supported Platforms
- Android API Level 24+
- iOS 13+
- macOS 10.15+ (Intel only)
- Windows 10+, 64-bit only

## Resources
- Join the [NatSuite community on Discord](https://discord.gg/y5vwgXkz2f).
- See the [NatML documentation](https://docs.natml.ai).
- See more [NatSuite projects on GitHub](https://github.com/natsuite).
- Read the [NatSuite blog](https://blog.natml.ai/).
- Discuss [NatML on Unity Forums](https://forum.unity.com/threads/open-beta-natml-machine-learning-runtime.1109339/).
- Contact us at [hi@natsuite.io](mailto:hi@natsuite.io).

Thank you very much!

repository: prakhar-s-srivastava/NatML

repo_name: prakhar-s-srivastava/NatML


text:
# Lua Toolbox

## Presentation

Inspired by [The Ruby Toolbox](https://www.ruby-toolbox.com/).

## Usage

Edit config.moon, then start with:

    make dev

Make sure you have a local Redis server running on port 6379, then open http://localhost:8080.

For more, see Makefile and config.moon.

## Copyright

Copyright (c) 2013 Pierre Chapuis

repository: haiiev/lua-toolbox

repo_name: haiiev/lua-toolbox


text:
## Getting started

_Wombat Dressing Room_ allows you to create secure tokens for publishing to the
npm registry from automated systems. Follow the steps below to generate a token:

### Generating per-package publication tokens

1. click [Manage tokens](/_/manage-tokens), under _Actions_.
2. enter a _package name_, in the text area provided.
3. click `Create Token`.
4. use the token generated to publish to the corresponding _package name_.

### Generating 24 hour and release-backed tokens

1. set `Wombat Dressing Room` as your default registry, `npm config set registry {registry-href}`.
2. `npm login`.
3. Click the `Create release-backed publication token` or
   `Create 24 hour token` button.
4. Your `~/.npmrc` will now be populated with a token which you can either use
   locally, or copy for use in automation tasks.

repository: SeppPenner/wombat-dressing-room

repo_name: SeppPenner/wombat-dressing-room


text:
# [postcss][postcss]-reduce-transforms [![Build Status](https://travis-ci.org/ben-eb/postcss-reduce-transforms.svg?branch=master)][ci] [![NPM version](https://badge.fury.io/js/postcss-reduce-transforms.svg)][npm] [![Dependency Status](https://gemnasium.com/ben-eb/postcss-reduce-transforms.svg)][deps]

> Reduce transform functions with PostCSS.

## Install

With [npm](https://npmjs.org/package/postcss-reduce-transforms) do:

```
npm install postcss-reduce-transforms --save
```

## Example

This module will reduce transform functions where possible. For more examples,
see the [tests](src/__tests__/index.js).

### Input

```css
h1 {
    transform: rotate3d(0, 0, 1, 20deg);
}
```

### Output

```css
h1 {
    transform: rotate(20deg);
}
```

## Usage

See the [PostCSS documentation](https://github.com/postcss/postcss#usage) for
examples for your environment.

## Contributing

Pull requests are welcome. If you add functionality, then please add unit tests
to cover it.

## License

MIT © [Ben Briggs](http://beneb.info)

[ci]:      https://travis-ci.org/ben-eb/postcss-reduce-transforms
[deps]:    https://gemnasium.com/ben-eb/postcss-reduce-transforms
[npm]:     http://badge.fury.io/js/postcss-reduce-transforms
[postcss]: https://github.com/postcss/postcss

repository: jenkins-zh/jenkins-zh

gha_language: HTML

repo_name: jenkins-zh/jenkins-zh


text:
# with-nuxt

> vue-datocms demo with Nuxt

## Build Setup

``` bash
# install dependencies
$ npm install

# serve with hot reload at localhost:3000
$ npm run dev

# build for production and launch server
$ npm run build
$ npm run start

# generate static project
$ npm run generate
```

For detailed explanation on how things work, check out [Nuxt.js docs](https://nuxtjs.org).

## To test locally

Get to the main `vue-datocms` folder and run:
``` bash
$ npm link
```

Then in the `examples/with-nuxt` folder run:
``` bash
$ npm link vue-datocms
```
to use the code from the above folder in the example project.

repository: maria-arce/vue-datocms

repo_name: maria-arce/vue-datocms


text:
# ansible-role-tshark

Installs `tshark` and manages privileged group

# Requirements

None

# Role Variables

| Variable | Description | Default |
|----------|-------------|---------|
| `tshark_group` | name of group allowed to run `dumpcap(1)` | `{{ __tshark_group }}` |
| `tshark_log_dir` | directory where `tshark_group` can write to | `/var/log/tshark` |
| `tshark_package` | package name of `tshark` | `{{ __tshark_package }}` |
| `tshark_group_members` | a dict of additional members of `tshark_group` (see below) | `{}` |

## `tshark_group_members`

The key is the user name. The value is another dict with `state` as only valid
key. `state` must be either `present` or `absent`. An example:

```yaml
tshark_group_members:
  vagrant:
    state: present
```

## Debian

| Variable | Default |
|----------|---------|
| `__tshark_group` | `wireshark` |
| `__tshark_package` | `tshark` |

## FreeBSD

| Variable | Default |
|----------|---------|
| `__tshark_group` | `network` |
| `__tshark_package` | `tshark` |

## OpenBSD

| Variable | Default |
|----------|---------|
| `__tshark_group` | `_wireshark` |
| `__tshark_package` | `tshark` |

## RedHat

| Variable | Default |
|----------|---------|
| `__tshark_group` | `wireshark` |
| `__tshark_package` | `wireshark` |

# Dependencies

None

# Example Playbook

```yaml
- hosts: localhost
  roles:
    - ansible-role-tshark
  vars:
    tshark_group_members:
      vagrant:
        state: present
```

# License

```
Copyright (c) 2017 Tomoyuki Sakurai<EMAIL_ADDRESS>
Permission to use, copy, modify, and distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.

 IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
```

# Author Information

Tomoyuki Sakurai<EMAIL_ADDRESS>
This README was created by [qansible](https://github.com/trombik/qansible)

repository: reallyenglish/ansible-role-tshark

gha_language: Ruby

repo_name: reallyenglish/ansible-role-tshark


text:
## Setting
```bash
cd orb_slam2_ros
echo "export ORBSLAM2_LIBRARY_PATH=${ORB_SLAM2_PATH}/ORB_SLAM2" >> ~/.bashrc
source ~/.bashrc
cd bash_scripts
chmod +x launch_kinect2.sh
```

## Run
```bash
rosnode cleanup
roslaunch orb_slam2_ros lcr_realsense.launch
```

If you get errors like `Failed to load nodelet '/camera/realsense2_camera` or `An exception has been thrown: Time is out of dual 32-bit range`, one possible reason is that the realsense driver node is not killed properly previously and you can use `rosnode list` to see the nodes that are running. You may see that even though you killed the camera driver's launch file with `ctrl+c`, the nodes (`/camera/realsense2_camera, /camera/realsense2_camera_manager`) are still running. So you need to call `rosnode cleanup` to kill the nodes and then you will be able to launch the camera driver again.

repository: facebookresearch/pyrobot

gha_language: Python

repo_name: facebookresearch/pyrobot


text:
The problem is described in the [javadoc] for `Files`.

## `Files.newDirectoryStream`

> When not using the try-with-resources construct, then directory stream's close
> method should be invoked after iteration is completed so as to free any
> resources held for the open directory.

## `Files.list`

> The returned stream encapsulates a `DirectoryStream`. If timely disposal of
> file system resources is required, the try-with-resources construct should be
> used to ensure that the stream's close method is invoked after the stream
> operations are completed.

## `Files.walk`

> The returned stream encapsulates one or more `DirectoryStreams`. If timely
> disposal of file system resources is required, the try-with-resources
> construct should be used to ensure that the stream's close method is invoked
> after the stream operations are completed. Operating on a closed stream will
> result in an `IllegalStateException`.

## `Files.find`

> The returned stream encapsulates one or more `DirectoryStreams`. If timely
> disposal of file system resources is required, the try-with-resources
> construct should be used to ensure that the stream's close method is invoked
> after the stream operations are completed. Operating on a closed stream will
> result in an `IllegalStateException`.

## `Files.lines`

> The returned stream encapsulates a `Reader`. If timely disposal of file system
> resources is required, the try-with-resources construct should be used to
> ensure that the stream's close method is invoked after the stream operations
> are completed.

[javadoc]: https://docs.oracle.com/javase/8/docs/api/java/nio/file/Files.html

## The solution

To ensure the stream is closed, always use try-with-resources. For example, when
using `Files.lines`, do this:

```java
String input;
try (Stream<String> stream = Files.lines(path)) {
  input = stream.collect(Collectors.joining(", "));
}
```

Not this:

```java
// the Reader is never closed!
String input = Files.lines(path).collect(Collectors.joining(", ");
```

## What about methods that return closeable streams? {#must-be-closed}

Methods that return `Stream`s that encapsulate a closeable resource can be
annotated with `com.google.errorprone.annotations.MustBeClosed` to ensure their
callers remember to close the stream.

repository: PicnicSupermarket/error-prone

gha_language: Java

repo_name: PicnicSupermarket/error-prone


text:
---
ref: mao23taps
title: "TAPS: Connecting Certified and Adversarial Training"
authors: Yuhao Mao, Mark Niklas Müller, Marc Fischer, Martin Vechev
year: 2023
month: 05
venue: arXiv
projects: safeai
bibtex: '@article{mao23taps
			author       = {Yuhao Mao and
							Mark Niklas M{\"{u}}ller and
							Marc Fischer and
							Martin T. Vechev},
			title        = {{TAPS:} Connecting Certified and Adversarial Training},
			year         = {2023},
			doi          = {10.48550/arXiv.2305.04574},
			eprinttype    = {arXiv},
			eprint       = {2305.04574}}'
paper: https://arxiv.org/abs/2305.04574
# code: https://github.com/eth-sri/sabr
# slides: https://files.sri.inf.ethz.ch/website/slides/mueller2023sabr_slides.pdf
#poster: https://files.sri.inf.ethz.ch/website/slides/mueller2021boosting_poster.pdf
image: assets/images/taps.png
#talk: https://iclr.cc/virtual/2021/poster/3359
---

Training certifiably robust neural networks remains a notoriously hard problem.
While adversarial training optimizes under-approximations of the worst-case loss, which leads to insufficient regularization for certification, sound certified training methods, optimize loose over-approximations, leading to over-regularization and poor (standard) accuracy.
In this work, we propose TAPS, an (unsound) certified training method that combines IBP and PGD training to optimize more precise, although not necessarily sound, worst-case loss approximations, reducing over-regularization and increasing certified and standard accuracies.
Empirically, TAPS achieves a new state-of-the-art in many settings, e.g., reaching a certified accuracy of 22% on TinyImageNet for L-inf-perturbations with radius eps=1/255.