text:
# Project3:  Final Phantasy
https://dry-mountain-58151.herokuapp.com/

An RPG with a sign up, store, and combat.

## Getting Started

Sign up/Log in. Purhcase your equipment from the store.  Fight monsters, make money and either buy more equipment to continune fighting monsters or proceed to the the boss fight.

### Prerequisites

None.

### Installing

No installation required.  Click, type, and submit.

## Running the tests

Test by project group (Himadri, KaiWei, Richard, Garrett and myself (Yev)).

### Break down into end to end tests

I tested functionality, database pulls and saves, server, combat functionality and any bugs that might or did arise.  Some issues still remain and I will continue to work on it to improve the functionality.

### And coding style tests

Style tests, floating using react-bootstrap.  Richard Biala and Garrett Hiebert were responsible for styles and they did most of the work if not all of the work with styles.  I did minimal styling pertainign to stage1 and boss fights - the user and mosnter cards and action buttons.

## Deployment

React-app, browser deployed.

## Built With

VS Code, React App, JS, React-Bootsrap, CSS, and lots of Heart and Sweat.

## Contributing

Group Project Consisted of:
- Biala, Richard
- Hiebert, Garrett
- Rokhin, Yevgeniy (me)
- Shen, KaiWei,
- Sinha, Himadri

## Versioning

1.0

## Authors

Group Project Consisted of:
- Biala, Richard
- Hiebert, Garrett
- Rokhin, Yevgeniy (me)
- Shen, KaiWei,
- Sinha, Himadri

## License

This project is licensed under the MIT License - see the [LICENSE.md](LICENSE.md) file for details

## Acknowledgments

I wanted to thank my group and Himadri for hosting the original project under his github account.  It was hard work with some bumps in the road, but we overcame most of the challenges.  Goal is to improve it in the near future.

repository: korkdaork/FinalPhantasy

repo_name: korkdaork/FinalPhantasy


text:
**MasterControl Documentation**

MasterControl is a program that facilitates sending TCP commands to
other devices, the execution of other programs, and the sending of
keystrokes to the system. This is presented to the user as a series of
button groups for easy execution.

**Requirements**

Java Virtual Machine

Jaxen Xpath library for Java (Included)

**Compiling MasterControl**

Compilation of the application requires Apache Ant. Build and run the application with the following command:

`ant build`

To view other options, just run:

`ant`

**Usage**

Use Java to execute the file MasterControl.jar:

`java -jar MasterControl.jar`

**User Interface**

Below is a screen capture that details the primary portion of the user
interface:

The sections and the buttons within them can be customized from
“File->Modify Configuration” menu.

**Configuration**

Choosing the “File->Modify Configuration” menu item displays a screen
that can be used to configure the appearance of the primary portion of
the user interface. Below is the configuration screen for the example in
the previous screen capture:

**Configuration Nodes**

The configuration is displayed in a tree view on the left pane of the
window (called the “Configuration pane”) comprised of items called
nodes. Each node may or may not have child nodes. For example, the root
node, labeled “Configuration”, has two child nodes labeled “TCP Clients”
and “User Interface”. On the other hand, the selected node, labeled
“Command – exec ‘notepad.exe’” has no child nodes. The configuration
structure can be altered by adding, reordering, and removing nodes.
Right clicking on a node causes a context menu to popup, having these
three options:

-   Add Child – Adds a child node to the selected node. The new child will be the last node of all selected node’s children. For example, if the node “Button – Notepad” was currently selected, the new child would appear after the node, “Command – exec ‘notepad.exe’”.

-   Add Sibling – Adds a child node to the parent node of the selected node. The new child will appear after the selected node. For example, if the node “Button – Notepad” was currently selected, a new node would appear before the node “Button – Media Player” as a child of the node “Section – Extra”.

-   Remove Item – Removes the current node from the tree. Some nodes cannot be removed.

Nodes can be rearranged, as long as they will still have the same parent.
To move a node, left click on the node and drag it to its new position.
When the mouse button is released, the node will be moved.

**Node Attributes**

Each node may or may not have attributes. When a node is selected, its
attributes are displayed in the right pane of the window (called the
“Attributes pane”). They can be altered by typing in or selecting a new
value. The changes will be saved and reflected once the Configuration
screen has been closed. Attributes cannot be added or removed.

**Configuration Management**

To create a new configuration, select “New Configuration” from the File
menu. Configurations can be loaded and saved using the “Load
Configuration” and “Save Configuration As…” File menu items
respectively.

The configuration file used by Master Control when it first starts can
be specified on the command line using the following format:

java –jar MasterControl.jar YourConfigFileHere.dat

where “YourConfigFileHere.dat” is the name of the configuration file you
wish to use. If a configuration file is not specified, the default
configuration file “settings.dat” is used.

**Configuration Format**

For information on the MasterControl Configuration Format can be viewed [here](http://htmlpreview.github.io/?https://github.com/aphillip/MasterControl/blob/master/MasterControlConfigurationFormat.html).

**TCPCommunicator**

TCPCommunicator is a java class that can act as a TCP server and a TCP client.  It can be used as a dummy server for MasterControl’s tcpsend command.  It is located in the src/test folder.

Compile it with `javac TCPCommunicator.java` and run with `java TCPCommunicator` to see usage instructions.

repository: aphillip/MasterControl

branch_name: refs/heads/master

extension: md

license_type: permissive

repo_name: aphillip/MasterControl


text:
# Create a new project (.NET Core)

> .NET Core also runs on non-Windows and non-Visual Studio environment, please check [this other tutorial for MacOS](https://github.com/augustogoncalves/dotnetcoreheroku). Windows OS still required to compile the plugin.

Go to menu **File** >> **New** >> **Project** and select **.NET Core** >> **ASP.NET Core Web Application**. For this sample, let's name it **forgesample**. On the next dialog, select **Empty** and select **Web API**.

Install the Autodesk Forge NuGet package: right-click on the project (**Solution Explorer**), select **Manage NuGet Package**, then on **Browse** search for **Autodesk.Forge** and install `Autodesk.Forge` and `Autodesk.Forge.DesignAutomation`, where the first will be used to upload input and output results to OSS and the second to use Design Automation.

> Please check **Include prerelease** while searching for packages. Design Automation v3 is in beta 5 (as of Jun/2019).

![](_media/netcore/create_project_da.gif)

!> If **Web** & **ASP.NET** project types are not available, please review [Tools](environment/tools/net) section

Right-click on the project, go to **Properties**, then under **Debug** tab see the **Environment Variables** section. `ASPNETCORE_ENVIRONMENT` should be already defined, so add:

- `ASPNETCORE_URLS`: use `http://localhost:3000`
- `FORGE_CLIENT_ID`: use your id here
- `FORGE_CLIENT_SECRET`: use your secret here
- `FORGE_WEBHOOK_URL`: use the **ngrok** forwarding URL from previous step

You may also check **Launch browser** and specify the **App URL**. Finally, as this is running locally, uncheck **Enable SSL** option. It should look like as shown below.

![](_media/netcore/env_vars_da.png)

Now open the **Program.cs** and add the following namespaces:

```csharp
using Autodesk.Forge.Core;
using Autodesk.Forge.DesignAutomation;
```

Then replace the **Program.cs** `Main()` method content with the following. This tells our application to load Forge Client ID & Secret from the environment variables defined above.

```csharp
CreateWebHostBuilder(args).ConfigureAppConfiguration(builder =>
{
    builder.AddForgeAlternativeEnvironmentVariables();
}).ConfigureServices((hostContext, services) =>
{
    services.AddDesignAutomation(hostContext.Configuration);
}).Build().Run();
```

Now open the **Startup.cs** and replace the content of the `Startup` class with the following code, which enables static file server (HTML & JS) and [SignalR](https://docs.microsoft.com/en-us/aspnet/core/signalr/introduction?view=aspnetcore-2.2), used to push notifications to the client.

```csharp
// This method gets called by the runtime. Use this method to add services to the container.
// For more information on how to configure your application, visit https://go.microsoft.com/fwlink/?LinkID=398940
public void ConfigureServices(IServiceCollection services)
{
    services.AddMvc();
    services.AddSignalR();
}

// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
    app.UseFileServer();
    app.UseMvc();
}
```

Finally, create a **Controllers** folder, which will later host the WebAPI Controllers.

## OAuthController.cs

We'll need an `access token` to read & write input & output files to OSS Buckets. Under **Controllers** folder, create a `OAuthController.cs` file with the following content:

```csharp
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Autodesk.Forge;

namespace forgesample.Controllers
{
    [ApiController]
    public class OAuthController : ControllerBase
    {
        // As both internal & public tokens are used for all visitors
        // we don't need to request a new token on every request, so let's
        // cache them using static variables. Note we still need to refresh
        // them after the expires_in time (in seconds)
        private static dynamic InternalToken { get; set; }

        /// <summary>
        /// Get access token with internal (write) scope
        /// </summary>
        public static async Task<dynamic> GetInternalAsync()
        {
            if (InternalToken == null || InternalToken.ExpiresAt < DateTime.UtcNow)
            {
                InternalToken = await Get2LeggedTokenAsync(new Scope[] { Scope.BucketCreate, Scope.BucketRead, Scope.BucketDelete, Scope.DataRead, Scope.DataWrite, Scope.DataCreate, Scope.CodeAll });
                InternalToken.ExpiresAt = DateTime.UtcNow.AddSeconds(InternalToken.expires_in);
            }

            return InternalToken;
        }

        /// <summary>
        /// Get the access token from Autodesk
        /// </summary>
        private static async Task<dynamic> Get2LeggedTokenAsync(Scope[] scopes)
        {
            TwoLeggedApi oauth = new TwoLeggedApi();
            string grantType = "client_credentials";
            dynamic bearer = await oauth.AuthenticateAsync(
              GetAppSetting("FORGE_CLIENT_ID"),
              GetAppSetting("FORGE_CLIENT_SECRET"),
              grantType,
              scopes);
            return bearer;
        }

        /// <summary>
        /// Reads appsettings from web.config
        /// </summary>
        public static string GetAppSetting(string settingKey)
        {
            return Environment.GetEnvironmentVariable(settingKey);
        }
    }
}
```

Project is ready!

Next: [Basic app UI](designautomation/html/)

license: MIT

gha_license_id: MIT


text:
# fix-upwork-interview-ui
Upwork has a massive UI problem where you can click on the side of a modal and the modal is closed and all your work is lost.

This script fixes the UI problem by coloring the outside of the modal so you know not to click on it.

The accept interview modal on upwork presents as a modal on the left side with a faded background. The modal appears to slide out of the browser on the right side of the screen and remains attached to the right of the screen with the modal background stretching all the way across the screen to the right side of the browser window. I often find myself clicking to the right of the modal, which causes the modal to close. There is no visual indicator to the right of the modal that you are actually clicking on the background - instead the background color makes me think I can safely click on the modal without closing it.

This creates a really poor user experience where I find myself losing my work.

This is a tapermonkey script which fixes the modal.

Here is a video demonostrating the effect:

https://www.youtube.com/watch?v=_nwshgT6GGk

You'll notice the change just before the 5 second mark.

repository: kheera/fix-upwork-interview-ui

repo_name: kheera/fix-upwork-interview-ui


text:
# [Glossary](#glossary)

## [Term](#term)

GIVEN a term '[Term][1]'
AND option 'indexFile' is './sub/index.md'
AND the glossary file is in './glossary.md'
AND config option 'linking' is 'relative'
THEN the term MUST be linked with a path '../glossary.md#term'.

[1]: #term "GIVEN a term 'Term'
AND option 'indexFile' is './sub/index.md'
AND the glossary file is in './glossary.md'
AND config option 'linking' is 'relative'
THEN the term MUST be linked with a path '../glossary.md#term'."

repository: about-code/glossarify-md

gha_language: JavaScript

repo_name: about-code/glossarify-md


text:
# WebSocket Driver

This example demonstrates using a WebSocket to drive the main component in a Halogen application.

The visible result is unimpressive as we're just using an echo service and pushing in all the messages at the start, but the setup would be mostly the same for a real world case.

## Building

You can build this example from the root of the Halogen project:

```sh
npm install
npm run example-driver-websockets
```

This will bundle a runnable JS file, `example.js`, in the `examples/driver-websockets/dist` directory. You can view the running application by opening the corresponding `index.html` file.

repository: purescript-halogen/purescript-halogen

gha_language: PureScript

repo_name: purescript-halogen/purescript-halogen


text:
# Tecan Infinite Reader
## Introduction
This is used to process the xlsx file output by Tecan Microplate Reader (Currently Support Tecan Infinite 500).
The Infinite 500 outputs the reading data as a xlsx file, which includes multiple worksheets within one file.
This kind of file makes the data hard to organize (cannot be read directly by R), which is an obstacle to researchers! (At least in the lab i am working it is)
This project use Rscript to read all the data (Reading and Environment/Strain notation) from each worksheet in one file, and put the data into the dataframe, named by the Strain Notation.
## Requirement
R version 3.4.3
R Studio
R package 'gdata'
R package 'readxl'
## Let's Begin!
This is how the output file from Microplate Reader looks like:
![alt text](https://github.com/Zha0rong/Tecan_Infinite_Reader/blob/master/media/Test%20File.png?raw=true)
The test file is generated using RNG, the environment and notation are randomly assigned.
You can see that here the environment/notation is 'Drosophila-AspN/GluN' (I have no idea what this is).
Now you run the R script. Remember to change the file\_path to the file path of your file.
Then the data frame will be generated, like this:
![alt text](https://github.com/Zha0rong/Tecan_Infinite_Reader/blob/master/media/Sampleoutput1.png?raw=true)
![alt text](https://github.com/Zha0rong/Tecan_Infinite_Reader/blob/master/media/Sampleoutput2.png?raw=true)
As you can see, the data is extracted from the xlsx worksheet and named as the notation and environment.
I also attached the Environment/Notation to the end of each row. If you want to combine all the data together for the analysis you would not be confused!

The test case is uploaded too.

Thank you!
## Help needed.
If you find some other model of Tecan microplate readers have different forms of output , and you want this project to work for those readers, please contact me and send me a sample output. I will check the sample and update this project shortly.


text:
# Example

* This tool does operations below.
  - multiply given 2 numbers.
  - add given 2 numbers.

## Usage

```console
$ go run main.go multiply 2 3
6

$ go run main.go add 1 2
3
```

## Development

```console
wat2wasm example.wat
```

repository: syumai/wasmexec

repo_name: syumai/wasmexec


text:
# C#-Unity-Game---FlyRush-3D
An interesting 3D game project where I learnt 3D modelling in blender, importing them into Unity and making all of them work together with C#

FlyRush 3D is a platformer unity 3D game where the player can fly through endless terrain levels with 11 different character combinations. The goal is to dodge obstacles and to gain as much as coins and diamonds and finish the level

Tech stack Used: Google Android SDK, Unity Engine, Blender for 3D models

This project was made By Abbilashh Senthilkumar @abilashsenth / @thenextbiggeek

# Whats new with flyRush

Nothing, actually. I wanted to make an addictive game which also contributed to some learning experience with c# and unity. And I was able to achieve that goal within 3 months from scratch
FlyRush 3D has 5 different maps such as golf course, mountain range, Ice land, Futuristic city and Desert land

# Theres around 11 3D modelled characters for the game
Flyrush 3D has a few nice characters such as DJI Drone, UFO, Avatar's Glider, Helicopter each animated by myself with Unity Keyframing tech

# Some more nice gameplay screenshots

branch_name: refs/heads/main


text:
# Overview

Search index has multiple efficient index schemas to cope with complex queries in big data scenarios.

## Background

Data tables in Tablestore are typical distributed NoSQL data structures. The tables support the storage and reads or writes of large-scale data such as monitoring data and log data.

Tablestore introduced the search index feature to meet requirements for the following queries: complex queries such as query based on non-primary key columns and Boolean query, and query based on primary keys including single-row read and range read.

Search indexes support a variety of queries types based on inverted indexes and column-oriented storage, including query based on non-primary key columns, full-text query, prefix query, fuzzy query, Boolean query, nested query, and geo query.

## Index differences

Aside from queries based on primary keys in data tables, Tablestore provides two index schemas for accelerated queries: global secondary index and search index. The following table describes the differences among the three types of indexes.

|Index type|Description|Scenario|
|----------|-----------|--------|
|Primary key|A data table is similar to a large map. Data tables only support queries based on primary keys.|You can determine a complete primary key or the prefix of a primary key.|
|Global secondary index|You can create one or more index tables and perform queries based on the primary key columns of the index tables.|Only a few columns are queried and you can determine the required columns in advance. You can also determine a complete primary key or the prefix of a primary key.|
|Search index|Search index uses inverted indexes, Bkd-trees, and column-oriented storage for various query scenarios.|All query and analysis scenarios that the primary key and the global secondary index do not support.|

## API operations

Search index provides the common query \(Search\) and data export \(ParallelScan\) API operations.

Most functions of the two operations are the same. However, to improve the performance and throughput, ParallelScan does not provide some functions of the Search operation. The following table compares the two API operations.

|API operation|Description|
|-------------|-----------|
|Search|Supports all functions of search index.-   Query: query based on non-primary key columns, full-text query, prefix query, fuzzy query, Boolean query, nested query, and geo query.
-   Collapse \(distinct\)
-   Sorting
-   Aggregation
-   Count |
|ComputeSplits+ParallelScan|Exports data for multiple concurrent queries.ParallelScan can use concurrent queries to provide better performance and up to five times the throughput of a single Search request.

-   Query: query based on non-primary key columns, full-text query, prefix query, fuzzy query, Boolean query, nested query, and geo query.
-   Multiple concurrent queries in a single request |

## Precautions

-   Index synchronization

    If you have created a search index for a data table, data is written to the data table first. After the data is written to the data table, a success message is returned. At the same time, another asynchronous thread reads the newly written data from the data table and writes the data to the search index. The write performance of Tablestore is not affected when data is being synchronized between the data table and search index.

    Typically, the latency of search index is less than 10 seconds. You can view the latency in real time in the Tablestore console.

-   TTL

    Search index does not support the time to live \(TTL\) feature. You cannot create a search index within a data table that has the TTL parameter set.

    If you want to retain the data only for a period of time and the time field does not need to be updated, you can implement the TTL feature by splitting a data table into several time-based tables. This solution is implemented based on the following principles and rules, and has the following advantages.

    -   Principle: Split a table based on fixed periods of time, such as day, week, month, or year, and create a search index for each table. In this way, tables for the specified periods of time are retained.

        For example, if you want to retain data for six months, you can store the data for each month in its own data table \(such as table\_1, table\_2, table\_3, table\_4, table\_5, and table\_6\) and create a search index for each one. Each data table and search index stores only the data for a single month. Then, you only need to delete data tables that are over six months old to implement the same function as TTL.

        When you query data by using search index, you only need to query one table if data in the required time range is in that table. If data in the required time range is included in multiple tables, you need to query all these tables and then combine the query results.

    -   Rule: The size of a single table or index cannot exceed 50 billion rows. Search index provides the optimal query performance if the size of a single table or index does not exceed 20 billion rows.
    -   Advantages:
        -   You can adjust the data storage duration based on the number of data tables retained.
        -   Query performance is directly proportional to data volume. When a data table is split into multiple tables, the data volume of each table has an upper limit. This helps ensure better query performance and avoid low query delays or timeouts.
-   Max versions

    You cannot create a search index in a table where you have specified the max versions parameter.

    You can customize the timestamp whenever you write data to a data table that only allows a single version. If you first write data with a major version number and then data with a minor version number, the index of the data with the major version number may be overwritten by that of the data with the minor version number.

## Features

Search index can solve complex query problems in big data scenarios. Other systems such as databases and search engines can also solve data query problems. The following figure shows the differences between Tablestore and databases and search engines.

Tablestore can provide all of the features of databases and search engines except join operations, transactions, and relevance of search results. Tablestore also has high data reliability of databases and supports advanced queries of search engines. Because of this, Tablestore can be used to replace the common architecture of `database + search engine`. If you do not need join operations, transactions, or relevance of search results, we recommend that you use search index of Tablestore.

![Differences between Tablestore and databases and search engines](https://static-aliyun-doc.oss-cn-hangzhou.aliyuncs.com/assets/img/en-US/9818958951/p55724.png)

repository: hglakfs/ots

repo_name: hglakfs/ots


text:
# ANGEL
[![Test and Coverage (Ubuntu)](https://github.com/GiulioRossetti/ANGEL/actions/workflows/test_ubuntu.yml/badge.svg)](https://github.com/GiulioRossetti/ANGEL/actions/workflows/test_ubuntu.yml)
[![codecov](https://codecov.io/gh/GiulioRossetti/ANGEL/branch/master/graph/badge.svg?token=3YJOEVK02B)](https://codecov.io/gh/GiulioRossetti/ANGEL)
[![PyPI download month](https://img.shields.io/pypi/dm/angel-cd.svg?color=blue&style=plastic)](https://pypi.python.org/pypi/angel-cd/)

Community discovery in complex networks is an interesting problem with a number of applications, especially in the knowledge extraction task in social and information networks.
However, many large networks often lack a particular community organization at a global level.
In these cases, traditional graph partitioning algorithms fail to let the latent knowledge embedded in modular structure emerge, because they impose a top-down global view of a network.
We propose here a simple local-first approach to community discovery, namely **Angel**, able to unveil the modular organization of real complex networks.
This is achieved by democratically letting each node vote for the communities it sees surrounding it in its limited view of the global system, i.e. its ego neighborhood, using a label propagation algorithm; finally, the local communities are merged into a global collection.

Moreover, we provide also an evolution of Angel, namely **ArchAngel**, designed to extract community from evolving network topologies.

**Note:** Angel has been integrated within [CDlib](http://cdlib.readthedocs.io) a python package dedicated to community detection algorithms, check it out!

## Installation
You can easily install the updated version of Angel (and Archangel) by using pip:

```bash
pip install angel_community
```

or using conda

```bash
conda install -c giuliorossetti angel_community
```

## Implementation details

*Required input format(s)*

Angel:
.ncol edgelist (nodes represented with integer ids).

```
node_id0    node_id1
```

ArchAngel:
Extended .ncol edgelist (nodes represented with integer ids).

```
node_id0    node_id1	snapshot_id
```

# Execution
Angel is written in python and requires the following package to run:
- python 3.x
- python-igraph
- networkx
- tqdm

## Angel

```python
import angel as a
an = a.Angel(filename, threshold=0.4, min_comsize=3, outfile_name="angel_communities.txt")
an.execute()
```

Where:
* filename: edgelist filename
* threshold: merging threshold in [0,1]
* min_com_size: minimum size for communities
* out_filename: desired filename for the output

or alternatively

```python
import angel as a
an = a.Angel(graph=g, threshold=0.4, min_com_size=3, out_filename="communities.txt")
an.execute()
```

Where:
* g: an igraph.Graph object

## ArchAngel

```python
import angel as a
aa = a.ArchAngel(filename, threshold=0.4, match_threshold=0.4, min_com_size=3, outfile_path="./")
aa.execute()
```

Where:
* filename: edgelist filename
* threshold: merging threshold in [0,1]
* match_threshold: cross-time community matching threshold in [0, 1]
* min_com_size: minimum size for communities
* outfile_path: path for algorithm output files

repository: GiulioRossetti/ANGEL

repo_name: GiulioRossetti/ANGEL


text:
# ASP.NET CORE

* API Browser - https://docs.microsoft.com/en-us/dotnet/api/ - try webhook v dbdataadapter on .NET Standard 2.0
* Choosing between .NET Coe and .NET Framework for server apps <https://docs.microsoft.com/en-us/dotnet/standard/choosing-core-framework-server>
* Docs - https://docs.microsoft.com/en-us/aspnet/core/
* Docs - samples - https://github.com/aspnet/Docs/tree/master/aspnetcore
* DodyG - Awesome .NET Core - https://github.com/dodyg/awesome-dotnet-core
* Features of ASP.NET Core 2.0 - https://blogs.msdn.microsoft.com/webdev/2017/08/25/asp-net-core-2-0-features-1/
* Hosting > IIS - https://docs.microsoft.com/en-us/aspnet/core/publishing/iis
* Migration - https://docs.microsoft.com/en-us/aspnet/core/migration/
* Roadmap - https://github.com/aspnet/Home/wiki/Roadmap
* Session state - https://docs.microsoft.com/en-us/aspnet/core/fundamentals/app-state?tabs=aspnetcore2x
* Sql Server based distributed cache for ASP.NET Core - http://gunnarpeipman.com/2017/04/aspnet-core-sql-server-cache/
* Using memory cache in ASP.NET Core - alternative to ObjectCache - local and distributed - http://gunnarpeipman.com/2017/04/aspnet-core-memory-cache/
* .NET Standard - with version support chart- https://docs.microsoft.com/en-us/dotnet/standard/net-standard

## BLOGS AND WEBSITES

* Gunnar Peipman - http://gunnarpeipman.com/aspnet/

## CONTAINERS

* .NET Microservices for containerized .net applications <https://blogs.msdn.microsoft.com/cesardelatorre/2017/05/10/free-ebookguide-on-net-microservices-architecture-for-containerized-net-applications/>
* github repo .NET Microservices for Containerized .NET Apps <https://github.com/dotnet-architecture/eShopOnContainers>
* EBook - .NET Core Framework Containers <https://docs.microsoft.com/en-us/dotnet/standard/microservices-architecture/net-core-net-framework-containers/>
* Ebook - .NET Core Framework Containers - Container Scenarios Chapter <https://docs.microsoft.com/en-us/dotnet/standard/microservices-architecture/net-core-net-framework-containers/net-core-container-scenarios>

## LEARN

* Build and deploy a multi-container application in Azure Container Service - ASP.NET Core / K8S / Docker / (ebook)(march 2018) https://azure.microsoft.com/en-us/resources/build-and-deploy-a-multi-container-application-in-azure-container-service/en-us/
* Create a web app with aspnetcore (mslearn) - https://docs.microsoft.com/en-us/learn/modules/create-razor-pages-aspnet-core/?view=aspnetcore-6.0
* dotnetfoundation training - https://github.com/dotnet-presentations
* LearnRazorPages - https://www.learnrazorpages.com
* MVA - ASP.NET CORE Learning Path - https://mva.microsoft.com/learning-path/asp-net-core-6
* Workshop - from Shayne Boyer and krew - teach you ASP.NET Core by building a complete conference management website from scratch - https://github.com/dotnet-presentations/aspnetcore-app-workshop - updates: https://tattoocoder.com/aspnetcore-workshop-update/

## PERFORMANCE

* Bing runs on .NET Core 2.1 / MVC Razor - performance gains noted <https://blogs.msdn.microsoft.com/dotnet/2018/08/20/bing-com-runs-on-net-core-2-1/>
* .NET Core 2.0 Improvements versus 4.7 - https://blogs.msdn.microsoft.com/dotnet/2017/06/07/performance-improvements-in-net-core/
* Managed runtime performance comparisons - http://anthonylloyd.github.io/blog/2017/12/30/dotnetcore-performance-part2
* AWS Lambda perf of Node.js, Python, Java, C# .NET Core, and GO - https://read.acloud.guru/comparing-aws-lambda-performance-of-node-js-python-java-c-and-go-29c1163c2581
* Benchmarks v node and others - https://github.com/aspnet/benchmarks
*  Iris go versus .NET Core Kestrel in terms of performance (August 2017) - https://hackernoon.com/iris-go-vs-net-core-kestrel-in-terms-of-http-performance-806195dc93d5
* TechEmpower Plaintext benchmark - https://www.techempower.com/benchmarks/#section=data-r15&hw=ph&test=plaintext

## RAZOR

* MSDN Mag - Quick intro to Razor (Sept 2017) <https://msdn.microsoft.com/en-us/magazine/mt842512>
* Creating Razor Pages - https://docs.microsoft.com/en-us/aspnet/core/tutorials/razor-pages/
* Intro to Razor Pages - https://docs.microsoft.com/en-us/aspnet/core/mvc/razor-pages/index?tabs=visual-studio
* MapPath equiv in Razor <https://stackoverflow.com/questions/47893477/how-do-you-map-path-in-razor-for-net-core-2-0-mvc>
* Routing primer (tutorial missing layout pages, watchout) <https://tahirnaushad.com/2017/08/20/asp-net-core-mvc-routing/>, github <https://github.com/TahirNaushad/Fiver.Mvc.RazorPages/tree/master/Fiver.Mvc.RazorPages/Pages/Movies>
* Syntax on W3C - https://www.w3schools.com/asp/razor_syntax.asp
* Startup class in Razor - https://www.learnrazorpages.com/startup
* Quick Tutorial <https://tahirnaushad.com/2017/08/29/asp-net-core-2-0-razor-pages/>

## SAMPLES

* ASP .NET Doc Samples - must have! - https://github.com/aspnet/Docs
* DodyG - 100+ samples for aspcore fundamentals - https://github.com/dodyg/practical-aspnetcore/
* Music Store - https://github.com/aspnet/MusicStore

## WebSocket

* Edge Developer Guide - WebSocket call (javascript client side) - https://docs.microsoft.com/en-us/microsoft-edge/dev-guide/networking-and-connectivity/websocket
* Server side socket in core - https://docs.microsoft.com/en-us/aspnet/core/fundamentals/websockets
* Server side socket sample - https://github.com/aspnet/Docs/tree/master/aspnetcore/fundamentals/websockets/sample

## MISC

* .NET 5 and ASP.NET Core - https://devblogs.microsoft.com/aspnet/announcing-asp-net-core-in-net-5/
* Analyzers - Top 10 Static Code Analyzers for ASP.NET Core - https://bharatdwarkani.medium.com/top-10-code-quality-static-analysers-for-asp-net-core-1660ad7a8d61
* Auth - Google Auth with ASP.NET Core - http://ruimourato.com/2016/02/12/using-google-auth-middleware-aspnetcore.html
* AWS UI - React.js framework - https://www.npmjs.com/package/@awsui/components-react
* Case Study > Bing runs on .NET Core 2.1 / MVC Razor <https://blogs.msdn.microsoft.com/dotnet/2018/08/20/bing-com-runs-on-net-core-2-1/>
* Case Study > Stack overflow moving to .NET Core <https://twitter.com/Nick_Craver/status/1031858480888639488>
* Choosing between Core and .NET - https://docs.microsoft.com/en-us/dotnet/standard/choosing-core-framework-server
* Configuring Session Providers - https://docs.microsoft.com/en-us/aspnet/core/fundamentals/app-state#configuring-session
* ASP.NET Core Dockerized app using VS Code - by wally - https://medium.com/@waelkdouh/developing-a-dockerized-asp-net-core-application-using-visual-studio-code-6ccfc59d6f6
* Event sourcing - https://docs.microsoft.com/en-us/azure/architecture/patterns/event-sourcing
* HostedService Sample - https://www.stevejgordon.co.uk/asp-net-core-2-ihostedservice
* HtmlHelpers listing - https://docs.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.mvc.viewfeatures.htmlhelper?view=aspnetcore-2.0
* HtmlHelpers - https://www.mikesdotnetting.com/article/184/html-helpers-for-forms-in-razor-web-pages
* HtmlHelpers - tutorials - http://www.tutorialsteacher.com/mvc/html-helpers
* Loggings > Introduction to Logging in ASP.NET Core - https://docs.microsoft.com/en-us/aspnet/core/fundamentals/logging
* Middleware > Great overview (1.1) - https://www.jeffogata.com/asp-net-core-middleware/
* Mocking in .NET Core > https://docs.microsoft.com/en-us/aspnet/core/mvc/controllers/testing
* Node.js -> Calling Node.js from ASP.NET Core - https://blogs.msdn.microsoft.com/waws/2017/10/23/calling-node-js-from-asp-net-core/
* Porting > Configuration - https://docs.microsoft.com/en-us/aspnet/core/migration/configuration
* Porting > MVC - https://docs.microsoft.com/en-us/aspnet/core/migration/mvc
* Porting > Third party dependencies - https://docs.microsoft.com/en-us/dotnet/articles/core/porting/third-party-deps
* Porting > .NET Portability Analyzer - https://docs.microsoft.com/en-us/dotnet/standard/portability-analyzer
* MVC compared to Razor - <https://jonhilton.net/razor-pages-or-mvc-a-quick-comparison/>
* React and ASP.NET Core - https://blogs.msdn.microsoft.com/whereismysolution/2018/01/23/some-cool-stuff-around-asp-net-core-2-0-and-reactjs/
* Secrets - Configuration by environment <https://docs.microsoft.com/en-us/aspnet/core/fundamentals/configuration/index?tabs=basicconfiguration&view=aspnetcore-2.2&viewFallbackFrom=aspnetcore-2.0>
* SignalR > ASP.NET Core 2.0 Announced - Sept 2017 - https://blogs.msdn.microsoft.com/webdev/2017/09/14/announcing-signalr-for-asp-net-core-2-0/
* SignalR > Roadmap - part of ASP .NET Core - https://github.com/aspnet/Home/wiki/Roadmap
* SignalR > Future of (Sept 17 2018) - includes SignalR Service <https://blogs.msdn.microsoft.com/webdev/2018/09/17/the-future-of-asp-net-signalr/>
* SPA - templates - https://blogs.msdn.microsoft.com/webdev/2017/02/14/building-single-page-applications-on-asp-net-core-with-javascriptservices/
* Using Windows Auth in ASP.NET Core via http.sys in containers - https://artisticcheese.wordpress.com/2017/09/10/using-windows-authentication-in-asp-net-core-via-http-sys-server/
* Windows Service > Hosting an ASP.NET Core app in a Windows Service - https://docs.microsoft.com/en-us/aspnet/core/hosting/windows-service

SEE ALSO - .NET Core section in this doc - databases included here - https://github.com/jhealy/awesome/blob/master/dnc.md

repository: jhealy/awesome

gha_language: Shell

repo_name: jhealy/awesome


text:
---
title: PostCSS
description: Using the PostCSS plugin to transform CSS
docs: plugins/postcss.ts
---

The `PostCSS` plugin load and transform your CSS files using
[PostCSS](https://postcss.org/) processor. This plugin **is disabled by
default** so to enable it you have to import and use it in the `_config.js`
file:

```js
import postcss from "lume/plugins/postcss.ts";

site.use(postcss());
```

## Configuration

This plugin accepts a configuration object. The available options are:

- `extensions`: Array with the extensions of the files that this plugin will
  load. By default is `[".css"]`.
- `plugins`: Array with the PostCSS plugins that you want to use.
- `keepDefaultPlugins`: Set `true` to append your plugins to the defaults,
  instead of replacing them.
- `sourceMap`: Set `true` to generate a source map file that will be saved in
  the same place but with the `.map` extension appended. For example, the file
  `my/styles.css` will generate the sourcemap file `my/styles.css.map`.
- `includes`: An array of directories to search for the `@import`ed files. By
  default is `_includes`. Set `false` to disable it.

## PostCSS Plugins

PostCSS uses the following plugins by default:

- [postcss-nesting](https://github.com/lumeland/postcss-nesting) to give support
  to nested rules.
- [postcss_autoprefixer](https://deno.land/x/postcss_autoprefixer) to add
  automatically the vendor prefixes.

Use the property `plugins` to replace them. For example, to use the
[font-format-keywords](https://deno.land/x/postcss_font_format_keywords) plugin:

```js
import postcss from "lume/plugins/postcss.ts";
import postcssFontFormatKeywords from "https://deno.land/x/postcss_font_format_keywords/mod.js";

site.use(postcss({
  plugins: [postcssFontFormatKeywords()],
}));
```

This will override the default plugins with yours. If you only want to add more
plugins without remove the defaults, use the `keepDefaultPlugins` option:

```ts
// Add more postcss plugins without override the defaults
site.use(postcss({
  plugins: [postcssFontFormatKeywords()],
  keepDefaultPlugins: true,
}));
```

## Includes

In addition to the default plugins, PostCSS use also
[postcss-import](https://deno.land/x/postcss_import), to inline the local
`@imports` looking in the `_includes` directory.

```css
/* Import the CSS file from _includes/css/reset.css */
@import "css/reset.css";

/* Import the relative CSS file */
@import "./variables.css";
```

For convenience, this plugin won't be removed with your plugins (even if
`keepDefaultPlugins` is set to `false`). But you can change the `_includes`
directory or disable completelly with the `includes` option:

```ts
// Change the includes folder of CSS to _styles
site.use(postcss({
  includes: "_styles",
}));
```

```ts
// Disable the includes (the local @import's won't be inlined)
site.use(postcss({
  includes: false,
}));
```

## The `postcss` filter

This plugin also register the `postcss` filter so you can transform css code in
the template engines. For example:

```html
{% set css %}
  body::after {
    content: "Hello, the CSS world!";
  }
{% endset %}

<style>
  {{- css | postcss | safe -}}
</style>
```


text:
# Docker of kobas

This docker contains an Ubuntu 14.04 system and pre-installed kobas. Provides an environment for running kobas in instance.

### Running method
---

To build the docker image locally, just run:

```
./build [your image name]
# example: ./build readline/kobas
```

---

To start a container for kobas analysis, run:
```
# Run it in a interactive mode
docker run -it -v /path/to/your/files:/path/to/mount/your/files readline/kobas /bin/bash
```

---

# Assistance scripts:

### /opt/dbPrepare.sh

To download required sqlite3 database for your analysis. It requires the organism code which you can find in the db.list. This is essential for kobas analysis.

```bash
/opt/dbPrepare.sh ko
```

### /opt/blastdbPrepare.sh

To download fasta file for your analysis and make blastdb. It requires the organism code which you can find in the db.list

You can run blast in the docker, but you'd better run blast in a parallel multi-node environment, out side of docker. If so, this step can skip for kobas analysis.

### Infomations about kobas

version kobas2.0-20150126

> KOBAS 2.0 is an update of KOBAS (KEGG Orthology Based Annotation System). KOBAS is the first software to identify statistically significantly enriched pathways using hypergeometric test. It has been successfully used in pathway analysis in plants, animals and bacteria.

> KOBAS 2.0, which annotates an input set of genes with pathways, diseases, and GO terms based on mapping to genes with known annotation. It allows for both ID mapping and cross-species sequence similarity mapping. It then performs statistical tests to identify statistically significantly enriched pathways, diseases, and GO terms. KOBAS 2.0 incorporates knowledge across over 2969 species from 6 pathway databases (KEGG PATHWAY, PID, BioCarta (from PID), Reactome, BioCyc, and PANTHER), 5 human disease databases (OMIM, KEGG DISEASE, FunDO, GAD, and NHGRI GWAS Catalog), and Gene Ontology.

[kobas paper](http://www.ncbi.nlm.nih.gov/pubmed/21715386)


text:
---
layout: post
date:	2016-05-05 22:13:00
title:  Musing on migration
tags: [archived-posts]
categories: archives
permalink: /:categories/:year/:month/:day/:title/
---
I just flew back from St.Louis to Bangalore.

As I flew home through the air, high above the Earth's surface, I mused upon the fact that millions of birds do this every year, without aluminum tubes ,warmth, and regular food to cosset them. I think human beings are the most fragile and vulnerable of the earth's
creatures,but the aids they've built to protect themselves wind up destroying so much else!

It took me hardly any time to pack my suitcase, but the second one took DnA a long time as they chose important things to send with me, ahead of their own journey to live in my city for (at least) a couple of years.

But not all of us have trouble packing up for long trips. Here's K2, who took only a few minutes:

<lj-embed id="1389"/>


text:
## real world problem ##
#1- Audit logins to web service: Question description (Java 8)

# Problem Description As It was given to me:
From the call:  This question focuses on multi-threading and execution service.   Candidates can write their own separate thread or they can have Java provide the thread through an execution service.
ABC Corp has a popular rest API that is used by millions of customers.
ABC Corp has a change requirement to audit every login to this API.
To facilitate this, a method writeAuditLog has been implemented. This method takes two parameters, a log message, and a user id. The log message should be of the format "Login success at <iso_date_timestamp>".
This audit feature has to be implemented without impacting the response time of the existing api. The writes to the audit database take between 1 and 500 milliseconds.
Your task is to implement the auditLogin method with minimal impact to the existing response time of the API.

Discussion:
From what I get from that fragile description, it seems that we have the following variables:
1. A generic log message indicating successful login
2. A user identification
3. An ISO timestamp.

This sounds like a simple problem. I think the best approach would be to off load the actual
logging of the message to some parallely running process via some type of queing mechanism.
Once the successful login occurs, the method is passed the relevant information. That method
in turn will just dump the message into a queue and be done with it. Some other thread or even
multiple threads will read the queue and process the message.

So we will have the writeAuditLog method defined:

public static StatusLogAudit writeAuditLog( String logmessage, String userId );

Maybe define an enum describing the result of the method invocation:
enum StausLogAudit {
successQueueWrite, unsuccesQueueWrite }

The item to be queue should be a class also:
interface AuditQueue {
public String getUserId();
public String getLogmessage();
public Date getLogtime();
}

and the actual class object:
class LoginAudit implements AuditQueue { ....... }

Generally, this method should never fail and if it fails backend processes will handle
any types of errors that might occur.

The real work wold occur via a thread that will continually poll the queue looking for
work to do. One such work item would be to log a message to the database.

A simple simulation should demonstrate this approach using a simple java BlockingQueue.
In this case the Blocking Queue would be of unlimited size so as to never Block during
queue insertion. This could be extended to multiple queues so that the contention of too many requests hitting the queue at the same time could be held to a minimum. On the back end you would also have many threads reading the queue looking for work items to process.

In the real world, the queue would most likely be a real queue such as RabbitMQ , ActiveMG or some other type of distributed queing system

gha_language: Java


text:
---
tags:
- Uncategorized
date: 2004-03-25T06:54:47Z

title: Reaching the limits
url: /chronicle/2004/03/25/reaching-the-limits/
wordpress_id: 391
wordpress_url: http://www.j5studios.com/chronicle/index.php/60
---

Ever wonder what the upper limits of the VX file system is (Sun Solaris 8 accessing in this case)?  That is, how many links can you have in one directory (a link is roughly a file or folder)?  Well, I found out yesturday in very hard fashion on a major work order system we built for a large wireless telecom.

The system has been in use for some three years, and has worked very well, providing much needed process improvements.  For some time we've pushed for an archive feature that fell on deaf ears. Yesturday however, they hit the limit folder inode limit (32767) with a lovely error message that I couldn't figure out (the web app only returned a CFdirectory error and doesn't give you much to go on) which means the system could not create new work orders (I can't explain how the system works, and why a folder is needed, but trust me, it is important and just can't be turned off).  I had never heard of this limit, and it took me some time to figure out that fixing the limit is not really not possible without risking file system corruption.

We did find a solution (requiring cleaver symlinking and clearing out empty folders).  But if you ever see the error message "cannot create directory: too many links" you might find yourself in a bind.

repository: justinribeiro/blog-pwa

extension: markdown

repo_name: justinribeiro/blog-pwa


text:
## 617. Merge Two Binary Trees
You are given two binary trees root1 and root2.

Imagine that when you put one of them to cover the other, some nodes of the two trees are overlapped while the others are not. You need to merge the two trees into a new binary tree. The merge rule is that if two nodes overlap, then sum node values up as the new value of the merged node. Otherwise, the NOT null node will be used as the node of the new tree.

Return the merged tree.

Note: The merging process must start from the root nodes of both trees.

It's a easy question. But if one of the child treenode is null, we just need to link the parent node the other one. Otherwise, we just get the value and add it to the first node cause we will return the first node. So what we need to do is to let the node.left = merge(node1.left, node2.left), same as the right node.


text:
= yaml =
title: Home
layout: home
toc: false
= yaml =

         Create mobile and distributed applications that work even without an Internet connection.

         Get Started with Syncbase

         Syncbase is a mobile database that supports peer-to-peer synchronization. It works just as well when devices are not connected to the Internet.

         Develop for Android

         Develop for iOS (coming soon)

         Build Distributed Apps with Vanadium Core

      Vanadium Core is the discovery, RPC, and security layer underlying Syncbase.
      It enables building secure, distributed applications that can run anywhere.

         Develop in Go


text:
---
layout: page
title: Grants and Awards
---

* Best Paper Award, at the IEEE/WIC/ACM Web Intelligence conference (WI'2015) for the paper: [Leveraging Temporal Query-Term Dependency for Time-Aware Information Access](https://www.irit.fr/publis/SIG/2015_WI_MTB.pdf). 173 submitted papers, 43 accepted long papers. (Singapore, December 6-9, 2015).

* Student Travel Grant Award from IEEE TCII to attend the IEEE/WIC/ACM Web Intelligence (WI'2015) conference (through a competitive selection process).

* Student Travel Grant Award from User Modelling Inc. to attend the 22nd Conference on User modelling, Adaptation and Personalization (UMAP'2014), (Aalborg, Denmark, July  7-11, 2014).


text:
# Prepare Code Climate Test Reporting as a Github Action

Prepares the CodeClimate Test Reporting

It looks like it's not doing anything at all. I remember it once communicated
with the server, but now that they re-implemented the reporter in Golang, the
code it invokes is practically empty... but it's still a
[documented step](https://docs.codeclimate.com/docs/configuring-test-coverage#quick-guide)
in the official documentation, so maybe they are still planning to do something
in it.

## Usage

```yaml
      - name: Prepare Code Climate Test Reporting
        uses<EMAIL_ADDRESS>        with:
          # Optionally debug
          debug: true
```


text:
# EfficientNet For TensorFlow 2.4

This repository provides a script and recipe to train the EfficientNet model on Large Scale Imagenet Dataset for varios EfficientNet architectures.
The content of the repository is tested by NVIDIA and part of the repository [Deep Learning Examples](https://github.com/NVIDIA/DeepLearningExamples)

## Table Of Contents

- [Setup](#setup)
    * [Requirements](#requirements)
- [Quick Start Guide](#quick-start-guide)
- [Advanced](#advanced)
    * [Scripts and sample code](#scripts-and-sample-code)
    * [Parameters](#parameters)
    * [Command-line options](#command-line-options)
    * [Getting the data](#getting-the-data)
    * [Training process](#training-process)
    * [Inference process](#inference-process)
- [Release notes](#release-notes)
    * [Changelog](#changelog)
    * [Known issues](#known-issues)

## Setup

The following section lists the requirements that you need to meet in order to start training the EfficientNet model.

### Requirements

This repository contains Dockerfile which extends the TensorFlow NGC container and encapsulates some dependencies. Aside from these dependencies, ensure you have the following components:
-   [NVIDIA Docker](https://github.com/NVIDIA/nvidia-docker)
-   [TensorFlow 20.08-py3] NGC container or later
-   Supported GPUs:
    - [NVIDIA Volta architecture](https://www.nvidia.com/en-us/data-center/volta-gpu-architecture/)
    - [NVIDIA Turing architecture](https://www.nvidia.com/en-us/geforce/turing/)
    - [NVIDIA Ampere architecture](https://www.nvidia.com/en-us/data-center/nvidia-ampere-gpu-architecture/)

For more information about how to get started with NGC containers, see the following sections from the NVIDIA GPU Cloud Documentation and the Deep Learning Documentation:
-   [Getting Started Using NVIDIA GPU Cloud](https://docs.nvidia.com/ngc/ngc-getting-started-guide/index.html)
-   [Accessing And Pulling From The NGC Container Registry](https://docs.nvidia.com/deeplearning/frameworks/user-guide/index.html#accessing_registry)
-   [Running TensorFlow](https://docs.nvidia.com/deeplearning/frameworks/tensorflow-release-notes/running.html#running)

As an alternative  to the use of the Tensorflow2 NGC container, to set up the required environment or create your own container, see the versioned [NVIDIA Container Support Matrix](https://docs.nvidia.com/deeplearning/frameworks/support-matrix/index.html).

For multi-node, the sample provided in this repository requires [Enroot](https://github.com/NVIDIA/enroot) and [Pyxis](https://github.com/NVIDIA/pyxis) set up on a [SLURM](https://slurm.schedmd.com) cluster.

## Quick Start Guide

To train your model using mixed or TF32 precision with Tensor Cores or using FP32, perform the following steps using the default parameters of the EfficientNet model on the ImageNet dataset. For the specifics concerning training and inference, see the [Advanced](#advanced) section.

1. Clone the repository.

	```
    git clone https://github.com/codesteller/train-efficientnet-with-imagenet.git efficientnet-train

    cd efficientnet-train
	```

2. Download and prepare the dataset.
    * Download ImageNet Dataset from the [link](https://www.image-net.org/download.php) to a folder "~/Downloads/imagenet12"
        ```
        $ ls -l ~/Downloads/imagenet12
        ILSVRC2012_img_test_v10102019.tar
        ILSVRC2012_img_train_t3.tar
        ILSVRC2012_img_train.tar
        ILSVRC2012_img_val.tar
        ```

    * To prepare the dataset
        1.  Process and Prepare Imagenet Dataset
        ```
            mkdir -p /imagenet/raw-data
            cp /imagenet/ILSVRC2012_img_*.tar /imagenet/raw-data/
            ./download_and_preprocess_imagenet.sh /imagenet
        ```

3. Start EfficientNet Docker built on top of the NGC container for training. Configure the dataset path accordingly
           ```
           bash startDocker.sh
           ```

4. Run training - change parameters in the file `train_efficientnet.sh` appropriately file the architecture, the dataset path.
           ```
           bash ./scripts/docker/launch.sh
           ```

5. Start training.

## Advanced

The following sections provide greater details of the dataset and running training

### Parameters

Important parameters for training are listed below with default values.

- `mode` (`train_and_eval`,`train`,`eval`,`prediction`) - the default is `train_and_eval`.
- `arch` - the default is `efficientnet-b0`
- `model_dir` - The folder where model checkpoints are saved (the default is `/workspace/output`)
- `data_dir` - The folder where data resides (the default is `/data/`)
- `augmenter_name` - Type of Augmentation (the default is `autoaugment`)
- `max_epochs` - The number of training epochs (the default is `300`)
- `warmup_epochs` - The number of epochs of warmup (the default is `5`)
- `train_batch_size` - The training batch size per GPU (the default is `32`)
- `eval_batch_size` - The evaluation batch size per GPU (the default is `32`)
- `lr_init` - The learning rate for a batch size of 128, effective learning rate will be automatically scaled according to the global training batch size (the default is `0.008`)

The main script `main.py` specific parameters are:
```
 --model_dir MODEL_DIR
                        The directory where the model and training/evaluation
                        summariesare stored.
  --save_checkpoint_freq SAVE_CHECKPOINT_FREQ
                        Number of epochs to save checkpoint.
  --data_dir DATA_DIR   The location of the input data. Files should be named
                        `train-*` and `validation-*`.
  --mode MODE           Mode to run: `train`, `eval`, `train_and_eval`, `predict` or
                        `export`.
  --arch ARCH           The type of the model, e.g. EfficientNet, etc.
  --dataset DATASET     The name of the dataset, e.g. ImageNet, etc.
  --log_steps LOG_STEPS
                        The interval of steps between logging of batch level
                        stats.
  --use_xla             Set to True to enable XLA
  --use_amp             Set to True to enable AMP
  --num_classes NUM_CLASSES
                        Number of classes to train on.
  --batch_norm BATCH_NORM
                        Type of Batch norm used.
  --activation ACTIVATION
                        Type of activation to be used.
  --optimizer OPTIMIZER
                        Optimizer to be used.
  --moving_average_decay MOVING_AVERAGE_DECAY
                        The value of moving average.
  --label_smoothing LABEL_SMOOTHING
                        The value of label smoothing.
  --max_epochs MAX_EPOCHS
                        Number of epochs to train.
  --num_epochs_between_eval NUM_EPOCHS_BETWEEN_EVAL
                        Eval after how many steps of training.
  --steps_per_epoch STEPS_PER_EPOCH
                        Number of steps of training.
  --warmup_epochs WARMUP_EPOCHS
                        Number of steps considered as warmup and not taken
                        into account for performance measurements.
  --lr_init LR_INIT     Initial value for the learning rate.
  --lr_decay LR_DECAY   Type of LR Decay.
  --lr_decay_rate LR_DECAY_RATE
                        LR Decay rate.
  --lr_decay_epochs LR_DECAY_EPOCHS
                        LR Decay epoch.
  --weight_decay WEIGHT_DECAY
                        Weight Decay scale factor.
  --weight_init {fan_in,fan_out}
                        Model weight initialization method.
  --train_batch_size TRAIN_BATCH_SIZE
                        Training batch size per GPU.
  --augmenter_name AUGMENTER_NAME
                        Type of Augmentation during preprocessing only during
                        training.
  --eval_batch_size EVAL_BATCH_SIZE
                        Evaluation batch size per GPU.
  --resume_checkpoint   Resume from a checkpoint in the model_dir.
  --use_dali            Use dali for data loading and preprocessing of train
                        dataset.
  --use_dali_eval       Use dali for data loading and preprocessing of eval
                        dataset.
  --dtype DTYPE         Only permitted
                        `float32`,`bfloat16`,`float16`,`fp32`,`bf16`
```

### Command-line options

To see the full list of available options and their descriptions, use the `-h` or `--help` command-line option, for example:
`python main.py --help`

### Getting the data

Refer to the [TFDS ImageNet readme](https://github.com/tensorflow/datasets/blob/master/docs/catalog/imagenet2012.md) for manual download instructions.
To train on ImageNet dataset, pass `$path_to_ImageNet_tfrecords` to `$data_dir` in the command-line.

Name the TFRecords in the following scheme:

- Training images - `/data/train-*`
- Validation images - `/data/validation-*`

### Training process

The training process can start from scratch, or resume from a checkpoint.

By default, bash script `scripts/{B0, B4}/training/{AMP, FP32, TF32}/convergence_8x{A100-80G, V100-16G, V100-32G}.sh` will start the training process from scratch with the following settings.
   - Use 8 GPUs by Horovod
   - Has XLA enabled
   - Saves checkpoints after every 5 epochs to `/workspace/output/` folder
   - AMP or FP32 or TF32 based on the folder `scripts/{B0, B4}/training/{AMP, FP32, TF32}`

To resume from a checkpoint, include `--resume_checkpoint` in the command-line and place the checkpoint into `--model_dir`.

#### Multi-node

Multi-node runs can be launched on a Pyxis/enroot Slurm cluster (see [Requirements](#requirements)) with the `run_{B0, B4}_multinode.sub` script with the following command for a 4-node NVIDIA DGX A100 example:

```
PARTITION=<partition_name> sbatch N 4 --ntasks-per-node=8 run_B0_multinode.sub
PARTITION=<partition_name> sbatch N 4 --ntasks-per-node=8 run_B4_multinode.sub
```

Checkpoint after `--save_checkpoint_freq` epochs will be saved in `checkpointdir`. The checkpoint will be automatically picked up to resume training in case it needs to be resumed. Cluster partition name has to be provided `<partition_name>`.

Note that the `run_{B0, B4}_multinode.sub` script is a starting point that has to be adapted depending on the environment. In particular, variables such as `--container-image` handle the container image to train using and `--datadir` handle the location of the ImageNet data.

Refer to the files contents to see the full list of variables to adjust for your system.

### Inference process

Validation is done every epoch and can be also run separately on a checkpointed model.

`bash ./scripts/{B0, B4}/evaluation/evaluation_{AMP, FP32, TF32}_8x{A100-80G, V100-16G, V100-32G}.sh`

Metrics gathered through this process are as follows:

```
- eval_loss
- eval_accuracy_top_1
- eval_accuracy_top_5
- avg_exp_per_second_eval
- avg_exp_per_second_eval_per_GPU
- avg_time_per_exp_eval : Average Latency
- latency_90pct : 90% Latency
- latency_95pct : 95% Latency
- latency_99pct : 99% Latency
```

To run inference on a JPEG image, you have to first store the checkpoint in the `--model_dir` and store the JPEG images in the following directory structure:

    ```
    infer_data
    |   ├── images
    |   |   ├── image1.JPEG
    |   |   ├── image2.JPEG
    ```

Run:
`bash ./scripts/{B0, B4}/inference/inference_{AMP, FP32, TF32}.sh`

## Performance

The performance measurements in this document were conducted at the time of publication and may not reflect the performance achieved from NVIDIA’s latest software release. For the most up-to-date performance measurements, go to [NVIDIA Data Center Deep Learning Product Performance](https://developer.nvidia.com/deep-learning-performance-training-inference).

### Benchmarking

The following section shows how to run benchmarks measuring the model performance in training and inference modes.

#### Training performance benchmark

Training benchmark for EfficientNet-B0 was run on NVIDIA DGX A100 and NVIDIA DGX-1 V100 16GB.

To benchmark training performance with other parameters, run:

`bash ./scripts/B0/training/{AMP, FP32, TF32}/train_benchmark_8x{A100-80G, V100-16G}.sh`

Training benchmark for EfficientNet-B4 was run on NVIDIA DGX A100 and NVIDIA DGX-1 V100 32GB.

`bash ./scripts/B4/training/{AMP, FP32, TF32}/train_benchmark_8x{A100-80G, V100-16G}.sh`

## Release notes

### Changelog

March 2021
- Initial release
July 2021
- Repository Forked and customized

### Known issues

- EfficientNet-B0 does not improve training speed by using AMP as compared to FP32, because of the CPU bound Auto-augmentation.


text:
---
layout: post
last_modified_at: 2021-03-29
title: Flowers names and spellings part 3
youtubeId: vfA8LM1JBlo
---

In this video, you will learn the spelling and pronounciation of the following flowers names/sightwords.

1) Gladiolus

2) Oleander

3) Tiger Lilly

4) Daisy

5) Canna

6) Cactus

7) Zinnia

8) Calla lily

9) Bougainvillea

{% include youtubePlayer.html id=page.youtubeId %}

[Next]({{ site.baseurl }}{% link  split3/_posts/2020-05-12-Dinosaurs names and spellings part 6.md%})

url: https://raw.githubusercontent.com/gbuk21/sightswords.github.io/979317ae8b148b2f74f0973b9b8e9d6547c71871/split3/_posts/2020-05-08-Flowers names and spellings part 3.md


text:
## Nodes Backlog

If you are looking to contribute a Node, below are some ideas:

**Solr**

* Node for loading data into Solr
* Node for querying data from Solr

**NLP Nodes**

It would be great to have NLP integrated into the system. We plan to use OpenNLP.

* Sentence Detector
* Tokenizer
* Name Finder
* Document Categorizer
* Part-of-Speech Tagger
* Chunker
* Parser

**OCR Nodes**

Optical Character Recognition is useful and we plan to integrate it into the system.

**Nodes for various data formats**

Logs

* SysLog Parser
* Apache Logs Parser

Healthcare
* HL7 Parser

Networking
* Netflow
* J-Flow

## Nodes Implemented

The following Nodes have been implemented till now. They reside under :

https://github.com/FireProjects/fire/tree/master/core/src/main/java/fire/nodes

#### Dataset Nodes

* NodeDatasetFileOrDirectoryCSV.java : Reads in a CSV file
* NodeDatasetFileOrDirectoryText.java : Reads in Text file as lines
* NodeDatasetFileOrDirectoryParquet.java : Reads in a Parquet file
* NodeDatasetPDFImage.java : Reads in a PDF file as images

Ghostscript is used to extract the images from the PDFs. So Ghostscript needs to be installed on the system.
Ghost4J is included in the pom.xml

To install ghostscript on mac : brew install ghostscript

#### Save Nodes

* NodeSave.java : Saves the dataframe onto HDFS

#### Predictive Modeling Nodes

* **NodeLinearRegression.java** : Linear Regression
* **NodeLinearRegressionWithSGD.java** : Linear Regression with SGD
* **NodeLogisticRegression.java** : Logistic Regression
* **NodeDecisionTree.java** : Decision Tree
* **NodeDatasetSplit.java** : Splits an incoming dataset for train and test
* **NodeKMeans.java** : KMeans Clustering
* **NodeALS.java** : ALS
* **NodeModelScore.java** : Scores a given model and test dataset
* **NodeSummaryStatistics.java** : Summary Statistics

#### ETL Nodes

* **NodeJoin.java** : Joins the incoming datasets on the given keys
* **NodeFilter.java** : Filters the incoming dataset on the conditition provided

#### HBase Nodes

* **NodeHBase.java** : Loads data into HBase

#### File Ingestion Nodes

* **CompactTextFiles.java** : Compacts a set of small text files into larger ones

#### Utility Nodes

* **NodePrintFirstNRows.java** : Prints the first N rows of a dataset

#### Streaming Nodes

* **NodeStreamingSocketTextStream.java** : Reads text data from a socket
* **NodeStreamingKafka.java** : Reads text data from Apache Kafka
* **NodeStreamingFlume.java** : Reads text data from Apache Flume

repository: ashokvytla/fire

repo_name: ashokvytla/fire


text:
# ttl
Things To Learn

This repository contains my personal learning list mainly concantrated around Scala and Rust languages.

### Scala Ttl
- [ ] [Flyway Db Migration](https://github.com/flywaydb/flywaydb)
- [x] [Akka Streams](https://github.com/akka/akka)
- [x] [Slick FRM](https://github.com/slick/slick)
- [ ] [Cornichon](https://github.com/agourlay/cornichon)
- [ ] scala.meta

### Rust Ttl
- [ ] [Tokio](https://github.com/tokio-rs/tokio)
- [ ] [Diesel](https://github.com/diesel-rs/diesel)
- [ ] [Rocket](https://github.com/SergioBenitez/rocket)

### Db Ttl
- [ ] [Cassandra](https://github.com/apache/Cassandra)

### DevOps Ttl
- [ ] [Ansible Provisioning](https://github.com/ansible/ansible)
- [x] [Docker](https://github.com/docker)

### API Programming
- [ ] [GraphQL](http://graphql.org)

### Messaging Ttl
- [x] [Kafka](http://kafka.apache.org)

### AI-ML Ttl
- [ ] Reinforcement Learning
- [ ] RNN

### UI Ttl
- [ ] d3.js

repository: orhanbalci/ttl

repo_name: orhanbalci/ttl


text:
Stacktrace Support {#stacktrace-support}
===
<!--
 Note: License header cannot be first, as doxygen does not generate
 cleanly if it before the '==='
-->
<!--
 Licensed to the Apache Software Foundation (ASF) under one or more
 contributor license agreements.  See the NOTICE file distributed with
 this work for additional information regarding copyright ownership.
 The ASF licenses this file to You under the Apache License, Version 2.0
 (the "License"); you may not use this file except in compliance with
 the License.  You may obtain a copy of the License at

	http://www.apache.org/licenses/LICENSE-2.0

 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
-->
When debugging a code base and an assertion is hit, it is often useful to
have a stacktrace as part of an assertion in order for you to tell where
you are in the code to know why it is buggy.  Generating a stacktrace can
be done with [Boost Stacktrace](https://www.boost.org/doc/libs/1_81_0/doc/html/stacktrace.html),
or using the [stacktrace](https://en.cppreference.com/w/cpp/header/stacktrace) header if you are using a C++23 compatible compiler.

In order to enable stacktraces when using the `LOG4CXX_ASSERT` family of macros,
simply define `LOG4CXX_ENABLE_STACKTRACE` in your buildsystem.  If you are using a
compiler that does not support C++17 and the `__has_include` macro, Boost Stacktrace
must be installed and available on your system.  If your compiler supports the
`__has_include` macro, then it will search for Boost Stacktrace, followed by searching
for `<stacktrace>`.  Both implementations will insert an
entry into the MDC named `stacktrace` that may then be inserted into log
statements.  When using the [PatternLayout](@ref log4cxx.PatternLayout), this
may be accomplished by using the `%%X{stacktrace}` conversion pattern.

## Putting the stacktrace into the MDC

If you want a stacktrace in any part of your code(not just on assertions),
the following snippet of code may be used to insert a stacktrace into the
current MDC:

~~~{.cpp}
::log4cxx::MDC mdc_("stacktrace", LOG4CXX_EOL + boost::stacktrace::to_string(boost::stacktrace::stacktrace()));
~~~

This may be inserted at any point in your application, giving you access
to the current stacktrace in any log statement, not just in assert statements.


text:
# Running an analysis starting from multiple inputs
<!-- WARNING -->
<!-- Do not edit this file from within the docs.nextstrain.org repository. -->
<!-- It is fetched from another repository to be included in the docs.nextstrain.org build. -->
<!-- So, if you edit it after it is fetched into docs.nextstrain.org, your changes will be lost. -->
<!-- Instead, edit this file in its own repository and commit your changes there. -->
<!-- For more details on this (temporary) implementation, see https://github.com/nextstrain/docs.nextstrain.org#fetching-of-documents-from-other-repositories -->
<!-- This file is fetched from: https://github.com/nextstrain/ncov/blob/master/docs/running.md -->
<!-- WARNING -->
<!-- WARNING -->
<!-- WARNING -->

A common use case is to have a set (or sets) of SARS-CoV-2 sequences which you wish to analyse together.
For instance, you may have a set of freshly generated genomes which you wish to analyse in the context of a larger, worldwide set of genomes such as those found on GISAID.
This tutorial works through such a scenario.

We have partitioned the data contained within the main example dataset into two sets:
1. An "Australian" dataset, containing 91 genomes from Victoria, Australia. These were genomes uploaded to NCBI from [Torsten Seemann et al.,](https://www.doherty.edu.au/people/associate-professor-torsten-seemann) but are used in this tutorial to represent a small subset of genomes which may not yet be public.
2. A "worldwide" dataset for context. Often this would be the entire NCBI/GISAID dataset, but here only includes 327 genomes for speed and data-sharing reasons.

Our aim is to produce an analysis of the 91 Australian genomes with select worldwide genomes for context. To achieve this, we wish to apply different input-dependent filtering, subsampling and colouring steps.

## Overview of the files used in this tutorial

The **sequences and metadata** for this tutorial are in `data/example_multiple_inputs.tar.xz` and must be decompressed via `tar xf data/example_multiple_inputs.tar.xz --directory data/`.

You should now see the following starting files:
```sh
data/example_metadata_aus.tsv           # Aus data (n=91) from Seemann et al.
data/example_sequences_aus.fasta
data/example_metadata_worldwide.tsv     # Worldwide, contextual data (n=327)
data/example_sequences_worldwide.fasta
```

The files are small enough to be examined in a text editor -- the format of the worldwide metadata is similar to the `nextmeta.tsv` file which you may download from GISAID, whereas the format of the Australian metadata is more limited, only containing sampling date and geographic details, which may be more realistic for a newly generated sequencing run.
Note: see `data/example_metadata.tsv` for the full metadata of these Australian samples, we've intentionally restricted this here to mimic a real-world scenario.

The **build-specific configs** etc are in `my_profiles/example_multiple_inputs`

```sh
my_profiles/example_multiple_inputs/config.yaml
my_profiles/example_multiple_inputs/builds.yaml # this is where the input files and parameters are specified
my_profiles/example_multiple_inputs/my_auspice_config.json
```

## Setting up the config

You can define a single input dataset in `builds.yaml` as follows.

```yaml
inputs:
  - name: my-data
    metadata: "data/metadata.tsv"
    sequences: "data/sequences.fasta"
```

For multiple inputs, you can add another entry to the `inputs` config list.
Here, we will give them the names "aus" and "worldwide":

```yaml
# my_profiles/example_multiple_inputs/builds.yaml
inputs:
  - name: "aus"
    metadata: "data/example_metadata_aus.tsv"
    sequences: "data/example_sequences_aus.fasta"
  - name: "worldwide"
    metadata: "data/example_metadata_worldwide.tsv"
    sequences: "data/example_sequences_worldwide.fasta"
```

### Snakemake terminology

Inside the Snakemake rules, we use a wildcard `origin` to define different starting points.
For instance, if we ask for the file `results/aligned_worldwide.fasta` then `wildcards.origin="worldwide"` and we expect that the config has defined
a sequences input as shown above.

## How is metadata combined?

The different provided metadata files (for `aus` and `worldwide`, defined above) are combined during the pipeline, and the combined metadata file includes all columns present across the different metadata files.
Looking at the individual TSVs, the `worldwide` metadata contains many more columns than the `aus` metadata does, so we can expect the the `aus` samples to have many empty values in the combined metadata.
In the case of **conflicts**, the order of the entries in the YAML matters, with the last value being used.

Finally, we use one-hot encoding to express the origin of each row of metadata.
This means that **extra columns** will be added for each input (e.g. `aus` and `worldwide`), with values of `"yes"` or `"no"`, representing which samples are contained in each set of sequences.
We are going to use this to our advantage, by adding a coloring to highlight the source of sequences in auspice via `my_profiles/example_multiple_inputs/my_auspice_config.json`:

```json
"colorings": [
  {
    "key": "aus",
    "title": "Source: Australia",
    "type": "boolean"
  }
],
"display_defaults": {
  "color_by": "aus"
}
```

# Input-specific filtering parameters

The first stage of the pipeline performs filtering, masking and alignment  (note that this is different to subsampling).
If we have multiple inputs, this stage of the pipeline is done independently for each input.
The parameters used for filtering steps are typically defined by the "filter" dict in the `builds.yaml`, with sensible defaults provided (see `defaults/parameters.yaml`).
For multiple inputs, we can overwrite these for each input.

As an example, in this tutorial let's ensure we include all the `aus` samples, even if they may be partial genomes etc

```yaml
# my_profiles/example_multiple_inputs/builds.yaml
filter:
  aus:
    min_length: 5000 # Allow shorter (partial) genomes
    skip_diagnostics: True # skip diagnostics (which can remove genomes) for this input
```

# Subsampling parameters

The second stage of the pipeline subsamples the (often large) dataset.
By this stage, the multiple inputs will have been combined into a unified alignment and metadata file (see above), however we may utilise the fact that the combined metadata has additional columns to represent which samples came from which input source (the columns `aus` and `worldwide`).
This allows us to have per-input subsampling steps.

In this example, we want to produce a dataset which contains:
1. _All_ of the samples from the `aus` input (i.e. all of the Australian genomes)
2. A worldwide sampling which prioritises sequences close to (1)
3. A random, background worldwide sampling

```yaml
# my_profiles/example_multiple_inputs/builds.yaml
builds:
  multiple-inputs:
    subsampling_scheme: custom-scheme # use a custom subsampling scheme defined below

# STAGE 2: Subsampling parameters
subsampling:
  custom-scheme:
    # Use metadata key to include ALL from `input1`
    allFromAus:
      exclude: "--exclude-where 'aus!=yes'" # subset to sequences from input `aus`
    # Proximity subsampling from `worldwide` input to provide context
    worldwideContext:
      exclude: "--exclude-where 'aus=yes'" # i.e. subset to sequences _not_ from input `aus`
      group_by: "year"    # NOTE: `augur filter` needs this to use `max_sequences` (TODO)
      max_sequences: 100
      priorities:
        type: "proximity"
        focus: "allFromAus"
    worldwideBackground:
      exclude: "--exclude-where 'aus=yes'"
      group_by: year month
      seq_per_group: 5
```

## Run the build

The following commands will run this tutorial

```sh
tar xf data/example_multiple_inputs.tar.xz --directory data/ # make sure you have input files!
snakemake --profile my_profiles/example_multiple_inputs -f auspice/ncov_multiple-inputs.json
```

The resulting JSON can be dropped onto [auspice.us](https://auspice.us) for visualization.

The following figure shows the graph (DAG) of steps which Snakemake will run to produce the target auspice JSON.
You can generate this yourself via
`snakemake --profile my_profiles/example_multiple_inputs -f auspice/ncov_multiple-inputs.json --dag | dot -Tpdf > dag.pdf`.

![snakemake-graph](images/multiple_inputs_dag.png)

## Extra examples

### What if I need to preprocess input files beforehand?

A common use case may be that some of your input sequences and/or metadata may require preprocessing before the pipeline even starts, which will be use-case specific.
To provide an example of this, let's imagine the situation where we haven't uncompressed the starting files, and our "custom preprocessing" step will be to decompress them.
In other words, our preprocessing step will replace the need to run `tar xf data/example_multiple_inputs.tar.xz --directory data/`.

We can achieve this by creating a snakemake rule which produces all of (or some of) the config-specified input files:

```python
# my_profiles/example_multiple_inputs/rules.smk
rule make_starting_files:
    message:
        """
        Creating starting files for the multiple inputs tutorial by decompressing {input.archive}
        """
    input:
        archive = "data/example_multiple_inputs.tar.xz"
    output:
        # Note: the command doesn't use these, but adding them here makes snakemake
        # aware that this rule produces them
        aus_meta = "data/example_metadata_aus.tsv",
        aus_seqs = "data/example_sequences_aus.fasta",
        world_meta = "data/example_metadata_worldwide.tsv",
        world_seqs = "data/example_sequences_worldwide.fasta"
    shell:
        """
        tar xf {input.archive} --directory data/
        """
```

And then making our build aware of these custom rules:
```yaml
# my_profiles/example_multiple_inputs/builds.yaml
custom_rules:
  - my_profiles/example_multiple_inputs/rules.smk
```

### What about if my starting files are stored remotely?

Currently we can handle files stored on S3 buckets rather than remotely by simply declaring this as the input location:

```yaml
# your pipeline's builds.yaml config
inputs:
  - name: worldwide
    metadata: "s3://your_bucket_name/metadata.tsv"
    sequences: "s3://your_bucket_name/sequences.fasta.xz"
```

> If your S3 bucket is private, make sure you have the following env variables set: `$AWS_SECRET_ACCESS_KEY` and `$AWS_ACCESS_KEY_ID`.

> You may use `.xz` or `.gz` compression - we automatically infer this from the filename suffix.

repository: neherlab/ncov-europe

gha_language: Python

repo_name: neherlab/ncov-europe


text:
# pg_serializer: Very fast JSON serializer using Django and PostgreSQL

[![PyPI version](https://badge.fury.io/py/pg-serializer.svg)](https://pypi.org/project/pg-serializer/)
[![Python Version](https://img.shields.io/badge/python-3.8-blue.svg)](https://docs.python.org/3/whatsnew/3.8.html)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
[![pipeline status](https://gitlab.com/kozlek/pg_serializer/badges/master/pipeline.svg)](https://gitlab.com/kozlek/pg_serializer/-/commits/master)
[![coverage report](https://gitlab.com/kozlek/pg_serializer/badges/master/coverage.svg)](https://gitlab.com/kozlek/pg_serializer/-/commits/master)

## Overview

> Django REST framework is a powerful and flexible toolkit for building Web APIs.

Django Rest Framework is a great framework for building REST APIs on top of Django and
it fits 99% of use cases.
However, JSON serialization is done in Python and it can be very slow for
some `list` endpoints that return a lot of records.

`pg_serializer` aims to soften this 1% issue by providing a way to do JSON serialization
inside your PostgreSQL database. Indeed, PostgreSQL has a built-in support for JSON
since version 9.2, and it is far more faster than Python's one.

## Installation

```bash
pip install pg-serializer
```

## Basic usage

`pg_serializer` features an automatic `ModelSerializer` like DRF.

```python
from django.contrib.auth.models import User

import pg_serializer

class UserSerializer(pg_serializer.ModelSerializer):
    class Meta:
        model = User
        fields = (
            "id",
            "username",
            "email",
            "first_name",
            "last_name",
            "is_staff",
            "date_joined",
        )

json_str = UserSerializer(User.objects.all()).json
```

It also has preliminary support for ForeignKey relation using the Django `__` separator.

```python
import pg_serializer

class OrderSerializer(pg_serializer.ModelSerializer):
    buyer = pg_serializer.StringField(source="buyer__username")

    class Meta:
        model = Order
        fields = (
            "transaction_id",
            "transaction_time",
            "buyer",
            "is_gift",
            "shipping_date",
            "additional_data",
        )

json_str = OrderSerializer(Order.objects.all()).json
```

Full examples are available inside `tests/models.py` and `tests/serializers.py`.

## Disclaimer ⚠

- `pg_serializer` is not designed to replace DRF serializers, only to speed up
some endpoints when performance is becoming an issue
- `pg_serializer` is still in alpha development: bugs can occur and
it doesn't support all Django fields and relations

## Roadmap

- reinforce the test suite
- implement (and document) custom datetime and decimal formatting
- document how to create custom serializer fields
- full support for ArrayField and JSONField
- complex queryset operations (GROUP BY, ...) support


text:
# timeout-promise-queue
[![NPM Version](https://badge.fury.io/js/timeout-promise-queue.png)](https://npmjs.org/package/timeout-promise-queue)
[![Build Status](https://travis-ci.org/ericprud/timeout-promise-queue.svg?branch=master)](https://travis-ci.org/ericprud/timeout-promise-queue)
[![Coverage Status](https://coveralls.io/repos/github/ericprud/timeout-promise-queue/badge.svg?branch=master)](https://coveralls.io/github/ericprud/timeout-promise-queue?branch=master)

Promise queue with timeouts and promise cleanup after expiration.

# Motivation
Reduce threads associated with forked processes by throttling the creating of processes.

## Installation

`timeout-promise-queue` can be installed using `npm`:

```
npm install timeout-promise-queue
```

## Interface
 - `PromiseQueue(Number maxConcurrent): Queue`
 - `Queue#add(Function promseGenerator, [Number timeout, [rejection]]): Promise` - calls function to generate a promise to add to the queue
   - `promiseGenerator([EventEmitter cancel]): Promise - generate a promise to add to the queue
   - if `timeout` is defined, `promseGenerator` is called with an EventEmitter which will get a `'timeout'` event if `timeout` is exceeded.
   If `rejection` is not defined, timeouts call `reject(Error('timeout of ' + timeout + ' exceeded'))`
   If `rejection` is defined, it generates the timeout error. `rejection` must be one of:
     - if `rejection` is function, timouts call `reject(rejection())`.
     - otherwise timouts call `reject(rejection)`.
 - `Queue#size(): Number` - returns number of pending promises

# Use

```js
const Queue = require('timeout-promise-queue').PromiseQueue(25) // allow 25 concurrent promises
Promise.all([
  Queue.add(cancel => cancellable(cancel)),
  Queue.add(cancel => cancellable(cancel)),
  Queue.add(cancel => cancellable(cancel))...,
]).then(l => {console.log('done') })
```

repository: ericprud/timeout-promise-queue

repo_name: ericprud/timeout-promise-queue


text:
# RobotS

[![Build Status](https://travis-ci.org/gamazeps/RobotS.svg?branch=travis__test)](https://travis-ci.org/gamazeps/RobotS)
[![Coverage Status](https://coveralls.io/repos/gamazeps/RobotS/badge.svg?branch=master&service=github)](https://coveralls.io/github/gamazeps/RobotS?branch=master)

Robots is a pure rust actor system library, it takes a strong inspiration from [erlang](https://www.erlang.org/) and
[akka](http://akka.io).

Documentation can be found [here](http://gamazeps.github.io/RobotS/).

## Maintained status

I am no longer working on this crate, so features will probably not be added by me.

If you want to add features feel free to submit a PR, I will try to review it within the week.

Bugs and errors will nevertheless be corrected if an issue is opened, provided they are not too long
to correct.

## Build it

This library can be built on a stable rustc (starting at 1.4.0) version.

A simple
```bash
cargo build
```
should be enough.

## Usage

### Initiate the ActorSystem

In order to use this crate you first need to create an ActorSystem and give it some threads to work
with.

```rust
extern crate robots;
use robots::actors::ActorSystem;

fn main() {
    let actor_system = ActorSystem::new("test".to_owned());
    actor_system.spawn_threads(1);
}
```

You can also shut down the actor system by calling the `shutdown` method on it, it will  stop all
threads and terminate all the actors and their allocated ressources (if you implemented their drop
properly).

That's nice but you need to populate it with some Actors.

### Implement an Actor

In order to define an actor you need to:

  * Define a struct and have it implement the `Actor` trait.
  * Have a function with a single argument to create it (a `tuple` will do that if you need multiple
    values, or `()` if you need none).

```rust
use robots::actors::{Actor, ActorCell, ActorContext};

struct Dummy;

impl Actor for Dummy {
    fn receive(&self, _message: Box<Any>, _context: ActorCell) {}
}

impl Dummy {
    fn new(_: ()) -> Dummy {
        Dummy
    }
}
```

### Instantiate an Actor

Now let's see how to instantiate such an actor.

Actors are created with a `Props`, this is a structure containing a constructor for this actor and
the argument for it.

There are two ways to instantiate an actor, it can either be the child of another user created actor
or be the child of the root actor for user actors.

The first way looks like:

```rust
let props = Props::new(Arc::new(Dummy::new), ());
let greeter = context.actor_of(props, "dummy".to_owned());
```

The second looks like:

```rust
let props = Props::new(Arc::new(Dummy::new), ());
let _actor = actor_system.actor_of(props, "dummy".to_owned());
```

The `actor_of` method will give you an `ActorRef` to the created actor.

Note that the first way to create actors is much faster than the second one (about 10 times), so it
should only be used if you want to create a new actor hierachy.

### Handle messages

The actor will receive its messages in the form of `Box<Any>`, this allows the actor to handle
multiple types of message which can be usefull in many cases (forwarding messages for example).

In order to have a concrete type from a `Box<Any>`, you will need to downcast it like in the
following example.

```rust
impl Actor for Printer {
    fn receive(&self, message: Box<Any>, _context: ActorCell) {
        if let Ok(message) = Box::<Any>::downcast::<String>(message) {
            println!("{}", *message);
        }
    }
}
```

As you can see this is rather easy.

If you think that using Box<Any> is very bad and that someone should do terrible things to me, check
[this post](http://gamazeps.github.io/posts/robots_any.html) before :)

### ActorContext methods

Now let's see how to use the context argument.

This gives most of the communication methods and features expected of an actor:

```rust
/// Returns an ActorRef to the Actor.
fn actor_ref(&self) -> ActorRef;

/// Spawns a child actor.
fn actor_of(&self, props: Arc<ActorFactory>, name: String) -> Result<ActorRef, &'static str>;

/// Sends a Message to the targeted ActorRef.
fn tell<MessageTo: Message>(&self, to: ActorRef, message: MessageTo);

/// Creates a Future, this Future will send the message to the targetted ActorRef (and thus be
/// the sender of the message).
fn ask<MessageTo: Message>(&self, to: ActorRef, message: MessageTo, future_name: String) -> ActorRef;

/// Completes a Future.
fn complete<MessageTo: Message>(&self, to: ActorRef, complete: MessageTo);

/// Tells a future to forward its result to another Actor.
/// The Future is then dropped.
fn forward_result<T: Message>(&self, future: ActorRef, to: ActorRef);

/// Tells a future to forward its result to another Future that will be completed with this
/// result.
/// The Future is then dropped.
fn forward_result_to_future<T: Message>(&self, future: ActorRef, to: ActorRef);

/// Sends the Future a closure to apply on its value, the value will be updated with the output
/// of the closure.
fn do_computation<T: Message, F: Fn(Box<Any + Send>, ActorCell) -> T + Send + Sync + 'static>
    (&self, future: ActorRef, closure: F);

/// Requests the targeted actor to stop.
fn stop(&self, actor_ref: ActorRef);

/// Asks the father of the actor to terminate it.
fn kill_me(&self);

/// Returns an Arc to the sender of the message being handled.
fn sender(&self) -> ActorRef;

/// Father of the actor.
fn father(&self) -> ActorRef;

/// Children of the actor.
fn children(&self) -> HashMap<Arc<ActorPath>, ActorRef>;

/// Lifecycle monitoring, list of monitored actors.
fn monitoring(&self) -> HashMap<Arc<ActorPath>, (ActorRef, FailureHandler)>;

/// Actors monitoring this actor.
fn monitored_by(&self) -> Vec<ActorRef>;

/// Monitor an actor with the given handler.
fn monitor(&self, actor: ActorRef, handler: FailureHandler);

/// Logical path to the actor, such as `/user/foo/bar/baz`
fn path(&self) -> Arc<ActorPath>;

/// Future containing an Option<ActorRef> with an ActtorRef to the Actor with the given logical
/// path.
///
/// The future will have the path: `$actor/$name_request`
fn identify_actor(&self, logical_path: String, request_name: String) -> ActorRef;

/// Sends a control message to the given actor.
fn tell_control(&self, actor: ActorRef, message: ControlMessage);

/// Puts the actor in a state of failure with the given reason.
fn fail(&self, reason: &'static str);
```

## Logging

RobotS logs informations on the `info` channel of the `log` crate.

If you have never used this crate you can simply activate the logs by adding the following lines to
your executable:

```rust
extern crate env_logger;

// Your use and extern crates...

fn main() {
    env_logger::init().unwrap();

    // Your main code.
}
```

You will need to set the `RUST_LOG` environment variable to `robots=info`. For example when launching the factorial
example we see:

```
➜  RobotS git:(master) ✗ RUST_LOG=robots=info cargo run --example=factorial
   Compiling RobotS v0.3.0 (file:///home/gamazeps/dev/RobotS)
     Running `target/debug/examples/factorial`
INFO:robots::actors::actor_system: Created cthulhu
INFO:robots::actors::actor_ref: /user receiving a system message
INFO:robots::actors::actor_system: Created /user actor
INFO:robots::actors::actor_ref: /system receiving a system message
INFO:robots::actors::actor_system: Created /system actor
INFO:robots::actors::actor_system: Launched the first thread
INFO:robots::actors::actor_system: Created the channel to get an ActorRef from a root actor
INFO:robots::actors::actor_ref: / is sending a message to /system
INFO:robots::actors::actor_ref: /system receiving a message
INFO:robots::actors::actor_ref: /user handling a message
INFO:robots::actors::actor_ref: /system handling a message
INFO:robots::actors::actor_ref: /system handling a message
INFO:robots::actors::actor_cell: creating actor /system/name_resolver
INFO:robots::actors::actor_ref: /system/name_resolver receiving a system message
INFO:robots::actors::actor_system: Created the /system/name_resolver actor
INFO:robots::actors::actor_ref: /system/name_resolver handling a message
INFO:robots::actors::actor_system: Created the channel to get an ActorRef from a root actor
INFO:robots::actors::actor_ref: / is sending a message to /user
INFO:robots::actors::actor_ref: /user receiving a message
INFO:robots::actors::actor_ref: /user handling a message
INFO:robots::actors::actor_cell: creating actor /user/sender
INFO:robots::actors::actor_ref: /user/sender receiving a system message
INFO:robots::actors::actor_ref: /system/name_resolver receiving a message
INFO:robots::actors::actor_ref: /user/sender handling a message
INFO:robots::actors::actor_system: Created the channel to get an ActorRef from a root actor
INFO:robots::actors::actor_ref: /system/name_resolver handling a message
INFO:robots::actors::actor_ref: / is sending a message to /user
INFO:robots::actors::actor_ref: /user receiving a message
....
```

This is a bit verbose but allows you to monitor the proper execution of your program.

## Contributing

All contribution are welcome, if you have a feature request don't hesitate to open an issue !

## Features

  * Actor communication in a local context.
  * Actor supervision with an actor hierarchy (each actor supervises its children).
  * Failure handling with explicit reaosns and handlers.
  * Ask pattern using Futures for asynchronous requests.
  * Name resolving (obtaining an ActorRef from a logical path).
  * Logging.

## TODO

  * Network communication in a transparent manner.
  * Investigate the performances to shave some microseconds.
  * Your crazy ideas ?

## Benches

Some crude benchs were written for actor creation and local message passing.

Here are the result on my Intel(R) Core(TM) i5-3317U CPU @ 1.70GHz

```
test create_1000_actors                   ... bench:   3,341,549 ns/iter (+/- 173,594)
test send_1000_messages                   ... bench:   1,217,572 ns/iter (+/- 196,141)
```

Nevertheless, I have not yet managed to get akka working on my computer so I don't have anything to
bench against so do what you want with these benches.

gha_language: Rust


text:
+++
date = "2016-04-27T10:45:00-07:00"
draft = false
image = "/images/sunrise.jpg"
title = "At Sunrise in the Desert"
description = "A perfect time to get married"
+++

As the sun rising begins a new day, we thought the metaphor fit well for us
starting our new lives together.

![](/images/sunrise-big.jpg)

Here's the tenative plan.

A small wedding with about 2 dozen guests including 4 bridesmaids and 4 groomsmen,
located somewhere in Las Vegas, outdoors, where the sun can rise in the background as
we say our vows. We will have breakfast and drinks for the reception, which may
end between 10am and noon.

Sunrise on the dates in the [date poll](/post/date-poll/) is between 6:20am and
6:30am. Please vote for a date if you haven't yet.


text:
# Code Challenge: Class 26

## Repeated Word

### Author: Shubham Majumdar

### Links and Resources
* [submission PR](https://github.com/smajumdar22/data-structures-and-algorithms/pull/70)

# Challenge Summary
Write a function that accepts a lengthy string parameter.
Without utilizing any of the built-in library methods available to your language, return the first word to occur more than once in that provided string.
Return the new tree.
## Challenge Description
Takes an input string
Converting the string into lowercase.
Using two loops to find duplicate words. First loop will select a word and  and inner loop will compare the word selected by outer loop with rest of the words.the second word
If a match found, then increment the count by 1 and set the duplicates of word to '0' to avoid counting it again.
After the inner loop, if count of a word is greater than 1 which signifies that the word has duplicates in the string.On getting the first match lop is broken and that word is returned.In case of no repetade words NOne is returned.

## Approach & Efficiency

## Solution
![UML Diagram](whiteboard.jpg)

#### How to initialize/run your server app (where applicable)
* `npm start`

#### Tests
* How do you run tests?
npm test

gha_language: HTML