text:
# Resumo SDIS

## Communication channels

- **Connection-based** - the processes must setup the channel before exchanging
  data.
- **Connectionless** - the processes need not set up the channel, can exchange
  data immediately.

- IP + port: identificam processo (actually endpoint, proc. pode mudar).

### UDP + TCP

```txt
| Property                            | UDP     | TCP    |
|=====================================|=========|========|
| Abstraction                         | Message | Stream |
| Connection-based                    | N       | Y      |
| Reliability(no loss or duplication) | N       | Y      |
| Order                               | N       | Y      |
| Flow control                        | N       | Y      |
| Number of recipients                | 1-n     | 1      |
```

- UDP supports multicast. A successful receive() always receives the message
  atomically.
- TCP channels are bidirectional. A read() might get only part of the stream (no
  separation).

---

## Multicast communication

- Broadcast efficiently on a point-to-point network
  - spanning tree that includes all nodes of the network.
- Multicast efficiently on a point-to-point network?
  - spanning tree that includes: the sender, the n receivers, the nodes between
    them (to ensure we have one tree).

### Switch Trees

Optimal algorithms are not pratical (SPT, MST). We change the topology of the
multicast tree: taking into account resource constraints and improving
performance (cost).
Limitation: assumes that the multicast tree has been previously created (child
becomes child of root on join).

Node parent switches can be performed to any node that isn't the node's child:

- switch-sibling - switch to sibling.

- switch-one-hop - switch to sibling or grandparent (can implement all
  topologies in two-hop while taking more iterations).

- switch-two-hop - switch to node 1 ou 2 hops away.

- switch-any - go anywhere that isn't parent or child.

  ![image-20210615201708189](/home/joao/.config/Typora/typora-user-images/image-20210615201708189.png)

Decisão entre candidatos pode ser feita com base no custo da árvore e/ou no
delay até a root.

### Banana Tree Protocol (BTP)

- One-hop.
- Node joins and becomes child of root. If no tree, becomes root.
- Node asks new parent for permission to switch parent (request can be rejected).
- If node fails, its children become child of root (its tree partitions). Can become child of grandparent to avoid overloading root.
- Cycles can be generated:
  - Concurrent attempts by different nodes to switch their parents. When in process of switching, reject all requests to becomes parent.
  - Switch based on outdated topological information. May be prevented by including topological info in the auth request (in one-hop algorithm, the parent of the request node is enough).

### Epidemic algorithms

Update the info by passing it to some neighbor nodes. These will pass it on to their neighbors in a lazy way (not immediately). Eventually, all nodes with copies of that piece of info will update it.

- Robust: tolerates crashes, even if each node has only a partial view of the system.
- Highly scalable (sync between nodes is local) if we assume that any node can randomly select any other node (not very scalable, because we would need to know about every other node).

#### Anti-Entropy

Periodically node P randomly chooses node Q for exchanging messages. The probability of a node missing a message tends to 0.

- Push gets slower at the end (less probability of finding node that hasn't seen the message).
- Pull gets faster at the end (more probability of choosing a node with the message).
- Push-Pull combines the advantages of both (P and Q exchange messages, so they have the same messages after the exchange).

#### Gossiping

P loses the motivation to disseminate a message (based on a **probability p_stop**), if it tries to disseminate it to another node Q that already knows it. Efficient, but we **can't be sure that everyone receives the message**.

---

## Remote Procedure Call (RPC)

### Transparency

- Platform heterogeneity:
  - Different architectures and/or representations for composite data-structures.
  - Solution: standard format in the wires (less efficient but only needs 2 conversions); or receiver-makes-right.

- Addresses as args. Use call-by-copy/restore:
  - Works in most cases.
  - Complex: same address can be passed in different arguments.
  - Inefficient for complex data structures (like trees).

### RPC Semantics

- **At-least-once**: keep retransmitting until we obtain a response (be careful with non-idempotent requests). Suitable if requests are idempotent (message loss).
- **At-most-once**: not trivial for UDP. RPC can report an error on TCP connection break. Appropriate when requests are not idempotent (message loss).
- **Exactly-once**: not always possible to ensure this semantics (**Note: policies no slide 20 de RPC.pdf**).

---

## Clients and Servers (processing)

The goal is to overlap I/O with processing.

### Thread-based concurrency

- appears simple (need only grantee isolation in access to shared data).
- Could use only monitors and synchronized methods in Java and condition variables (modularity problems and possibility of deadlocks).
- Performance may suffer: the larger the critical sections, less concurrency (but main reason for concurrency is performance).
- Each thread requires resources. Increasing the number of threads, eventually leads to lower throughput. Context switching (kernel-level threads) can introduce latency.

### Event-based concurrency

- Structure of the code becomes very different.
- No errors like race conditions (may be elusive)
- Lack of support by debugging tools.
- Leads to poorly structured code: lack of atomicity (with multiple cores, we can have race conditions, even if there is no preemption).

### Leases

Server leases a resource to a client for only a finite time interval: upon its expiration, the resource may be taken away (unless the lease is renewed).

---

## Security and Cryptography

**Unconditionally secure**: an attacker cannot extract information about the plaintext by observing the cyphertext.

History shows that published algorithms are broken mostly because of the length of the keys (and not vulnerabilities). With unpublished algorithms, e.g.: DeCSS, the history is different.

### CIA triad

- **Confidentiality**: prevent unauthorized access;
- **Integrity**: prevent unauthorized modification;
- **Availability**: prevent authorized access from being denied

### Encryption/Decryption Algorithms

- Symmetrical, or with a shared key: the keys for encrypting and decrypting are the same. These key is shared by all principal authorized to access info, so it must be known to those principals only;
- Asymmetric, or with public key: the keys for encrypting/decrypting are different.

### Encryption with shared key: DES

Replaced by AES. Works by combining 2 basic operations on bit blocks: permutation of bits in a block + substitution of 6-bit sub-blocks with 4-bit sub-blocks. The basic algorithm operates on 64-bit blocks that are transformed in blocks with the same length. The encryption takes 16 rounds, each taking a different 48-bit key that is generated from the 56-bit master key.

### Public key encryption: RSA

Based on modular arithmetic (p and q are prime numbers):

- n = p * q and z = (p - 1) (q - 1)
- d * e = 1 mod z
- For any x (0 <= x < n): x^(d*e) = x mod n

The encryption key, Ke = (e, n), must be public. The decryption key, Kd = (d, n), must be secret.

* Pick 2 very large prime numbers, p and q
* Compute n and z
* Select e (it can be small)
* Compute d using Euclides algorithm: e * d = 1 mod z

### Cipher Block Modes of operation

- Block ciphers encrypt fixed-size blocks: use padding and split data in blocks
- Stream ciphers operate directly on sequences of bytes of arbitrary length

![image-20210616090451034](/home/joao/.config/Typora/typora-user-images/image-20210616090451034.png)

Can be problematic because identical blocks are encrypted in identical cyphered-blocks.

![image-20210616090504043](/home/joao/.config/Typora/typora-user-images/image-20210616090504043.png)

We can use an **Initialization Vector (IV)**. This is a (pseudo-)random number.

### Cryptographic hash functions

- Compression - maps input of arbitrary length into a fixed-length hash-value.
- Ease of computation.
- Non reversible (one-way) - given y, it is computationally infeasible to find a value x such that y=h(x)
- Weakly collision resistant - given x, it is infeasible to find a different value x' such that h(x)=h(x')
- Strongly collision resistant - infeasible to find two different x and x' such that h(x)=h(x')

#### MD5

Needs message to be multiple of 512-bit (uses padding if necessary). Makes k stages (number of 512-bit blocks). Input is 512-bit block and 128-bit number. Output is 128-bit number. Each stage makes 4 passes over message block.

### Authentication with Hash Functions

By adding a key to the message/data, hash functions can be used to authenticate the sender and to check message integrity: in this case hash functions are known as **MAC**.

In this case, the hash functions must be **computational resistant**: for any unknown key, k, given the values (x, h(k, x)), it is computationally infeasible to compute h(k, y) for a different value y.

Keys must be shared by all comm. ends: MAC is not a digital signature.

### Digital Signature

Based on asymmetric encryption systems, e.g.: RSA. Allows third-parties to identify the sender (not only the receiver can do it now). Basically, the encryption of a message with the senders private key van be considered as signature. In practice, we first compute the hash of the data to sign, and then encrypt that hash value (output is digital signature).

---

## Secure communication channels

- Authentication of comm parties
  - On the web, usually only one of the entities is authenticated.
  - It's common for the channel set up phase to include the establishment of a session key.
  - Password are not appropriate for authentication while setting up a secure channel: use challenge/response protocols.
- Integrity/Authenticity: messages were not modified in transit

### Shared key authentication protocols

Assume Alice and Bob are the two ends of a communication channel and share a secret key, K_A,B.

![image-20210616094742564](/home/joao/.config/Typora/typora-user-images/image-20210616094742564.png)

![image-20210616094757068](/home/joao/.config/Typora/typora-user-images/image-20210616094757068.png)

**Reflection attack**: We trick B into sending use the response to its own challenge (challenge in message 2, we attack in message 3 and 4 and then answer in 5). We can avoid this by making the protocol asymmetrical (one party gets the even challenge and the other, the odd).

![image-20210616095130490](/home/joao/.config/Typora/typora-user-images/image-20210616095130490.png)

### Mediated authentication

Each pair of principals sharing a key is not scalable. We need a Key Distribution Center (KDC) which must be trusted by all principals. The KDC shares a secret key with each principal and generates secret keys for sharing between principals that wish to communicate securely.

![image-20210616095655573](/home/joao/.config/Typora/typora-user-images/image-20210616095655573.png)

This is not complete, because A and B must first prove they know K_A,B before starting the communication (A can receive key and start to communicate before B has received its key).

#### Needham-Schroeder's protocol

![image-20210616100048213](/home/joao/.config/Typora/typora-user-images/image-20210616100048213.png)

- Nonce (R_A1) is used to prevent **replay attacks**: C intercepts A message to KDC and sends a copy of it (replay) after the communication is over to act as A.
- The KDC includes B's identity in the response, to prevent C from impersonating B, by replacing B with C, in message 1.
- Messages 3 + 4 allow A to auth B.
- Messages 4 + 5 allow B to auth A.
- K_B,KDC(A, K_A,B) in message 2 is known as **ticket** to Bob.
- If an attacker learns about Ka,b, he can replay-attack. We solve this by requiring A to ask for a challenge Kb,kdc(Rb1). This is a random number that will be sent Kb,kdc(A, Ka,b, Rb1) to B on first comm for validation.

### Public-key authentication

A and B know public keys of one another. Private keys are known exclusively by the respective principal.

![image-20210616103313279](/home/joao/.config/Typora/typora-user-images/image-20210616103313279.png)

In addition to authenticating both principals, this protocol also negotiates a **session key**.

**Authenticated encryption**:

Channel should be able to detect modification of the messages. **Encrypt-then-MAC (EtM)**: first encrypt the message then compute MAC (different keys should be used, but there are approaches that use a single key).

### Session key

- Public key encryption is less efficient
- Keys wear out with use: the more ciphertext an attacker has, the more likely the attack will succeed.
- Using the same key over multiple sessions, makes replay attacks more likely
- If session key is compromised, the attacker will not be able to decrypt messages exchanged in other sessions.
- Channels can change keys in the middle of a session to prevent replay attacks in long running sessions.

### Diffie-Hellman Key-Agreement protocol

n is a large prime number and g a number less than n. These can be public. Each principal picks a secret large number x and y, and executes the following protocol:

![image-20210616105347384](/home/joao/.config/Typora/typora-user-images/image-20210616105347384.png)

An attacker overhearing the communication can't compute the session key, because the discrete logarithm is considered to be computationally intractable (no efficient algorithm is known to compute a from g^a mod n, knowing only n and g).

Algorithm is vulnerable to man in the man-in-the-middle attacks. To defend:

- A and B need to publish g^x mod n.
- Use authenticated DH: either share secret key between A and B or public-keys (and private).

### Perfect Forward Secrecy

An attacker will not be able to decrypt a recorded session even if:

- breaks into both A and B
- steals their long-term secrets

as long as A and B delete their secret numbers.

### Session key for unidirectional Authentication

Servers do not authenticate clients. Client generates (randomly) the session key and sends it to the server encrypted with the server's public key. Client and server can execute the Diffie-Hellman, but only the server's messages are authenticated.

- Client is guaranteed that it has set up a secure channel with the server.
- Server has no idea who the client is, but knows it is always the same client on the other end of the channel.

### Public key certificates

Public-key certificates/digital certificate contains: subject's name, public key that matches private key, signature by CA, and name of CA.

---

## Fault tolerance

### Triple modular redundancy (TMR)

- Each node is triplicated and works in parallel.
- The output of each module is connected to a voting element, also triplicated, whose output is the majority of its inputs.

### Reliability and availability

Reliability R(t): the probability that a has not failed until time t (often characterized by the MTTF).

Availability: a = MTTF / (MTTF + MTTR).

### Failure models

- **crash**: does not respond to any input after some instant
- **omission**: a component does not respond to some of its inputs
- **timing/performance**: a component does not respond on time
- **byzantine/arbitrary**: component behaves in a totally arbitrary way

---

## Leader Election

coordinator/leader roles lead to simpler and more efficient algorithms. Only one node is elected the coordinator. All nodes know the identity of the coordinator.

### Garcia-Molina's algorithms

To achieve fault tolerance, we can:

- mask failures (only approach if we need continuous operation)
- reorganize the system (simpler)

- **State**: DOWN, ELECTION, NORMAL
- **Coordinator**: coordinator according to node.

### Specification of election

Assuming that the communication system does not fail, has an upper bound on time to deliver a message, and incoming messages arrive with no delay.

At any time instant, for any 2 nodes, if they are both in NORMAL state, then they agree on the coordinator: S(i).s == S(j).s == NORMAL => S(i).c == S(j).c

If no failures occur during election, the protocol will eventually transform a system in any state to a state where:

- there is a node i such that S(i).s == NORMAL && S(i).c == i
- all other non-faulty nodes such that j != i have S(j).s == NORMAL && S(j).c == i

### Leader election vs mutual exclusion

- in an election fairness is not important (all we need is that one node becomes the leader)
- Election must deal properly with the failure of the leader (mutual exclusion assume that a process in a critical section does not fail)
- All nodes need to learn who the coordinator is

### Bully election algorithm

- Smaller identifier means we are stronger.
- Look around for anyone stronger (the leader): ARE-U-THERE messages are answered with YES by stronger nodes (they initiate a new election when this happens). We wait 2 * T for this answer. If challenge is answered, we back off and start a timeout to detect failure of the challenger.
- If there isn't anyone, impose self as leader and tell everyone: HALT message to weaker nodes. Weaker nodes set state to ELECTION and stop any elections they started. T time units later, the node will send NEW-LEADER to these weaker nodes (they set state to NORMAL and register new leader).
- ARE-U-NORMAL message by leader checks if all remote nodes are NORMAL. Start new election if anyone reports anything different from NORMAL. This also allows nodes to detect leader failures.

### Groups

Menos assumptions (time upper bound e instant reply).

- se somos os 2 NORMAL e estamos no mesmo grupo, então temos o mesmo coordinator
- se es NORMAL e o teu leader é o i, entao o i está no teu grupo

### Invitation algorithm

All nodes initially create a singleton group, of which it is the coordinator. Periodically, coordinators try to merge their group with other groups in order to form larger groups.

- ARE-U-THERE message to detect if group leader is still alive. If not, create new own singleton group.
- ARE-U-LEADER messages to find other groups. If any reply, start merge algorithm after random delay.
- INVITATION message to all leaders that responded and members of own group. Upon receiving INVITATION, leaders forward it to their group members. Everyone with INVITATION responds with ACCEPT. After some time, send READY message to all ACCEPTS (if a node doesn't receive the READY after timeout, initiates a new election).
- Only answer ARE-U-LEADER and INVITATION if NORMAL. Only allowed to participate in 1 election at a time.

---

## DNS (is iterative)

- name - up.pt ou fe.up.pt
- domain(subdomain) - up.pt(fe.up.pt)
- zone - never overlap, region managed by an administrative authority, can be just a subdomain.
- dns server - each zone has a primary server that is responsible for the info of that zone. This server can contain the info of more than one zone. For availability reasons, this info needs to be replicated in at least 1 secondary server.

### Resource records

- name/value - not necessarily host name and IP
- type - how the value should be interpreted (depends on the class)
- class - specifies the name space (usually IN)
- TTL

### Name resolution

- Resolution is based on longest prefix matching
- Every server must keep (NS, A) RR pairs for each of its direct subdomains.
- Answers provided using caches are said to be non-authoritative.

### Replication

Rely heavily on replication: performance + availability. Problems of inconsistency. Zone info is updated/added at only the primary server. Replicas don't need to be updated synchronously. The use of stale data is usually detected upon use.

To detect changes to a zone's RR, each zone has a Start of Authority (SOA) RR with the following fields: Serial (32 bits that identify a version; increased on update) and Refresh (32 bits that identify maximum time in seconds between update attempts).

### Update detection and zone propagation

Detect update by comparing serial field of the secondary with the primary. Poll primary server for updates or get notifications (not yet approved). Zone transfer sends AXFR query requesting for the transfer of an entire zone (incremental (only changes) not yet approved).

DNS requires zone transfers to use TCP. All other queries usually use UDP (although TCP is possible).

---

## Chord

### Distributed Hash Table (DHT)

Key-Value pairs are stored in a potentially very large number of nodes. Provides a single operation: lookup(key) returns the address of the node responsible for the key.

### Chord

- m-bit identifiers in a ring (mod 2^m). m=7 => 2^7=128 identifiers
- The node responsible for key k is the successor of key k, succ(k): smallest id that is larger or equal to k (modular arithmetic)

### Fingers

- node stores info about successor (next node in ring).
- each node knows more about nodes closer to it than about nodes further away.
- Key resolution requires O(log(N)) steps, where N is the number of nodes in the system.

Finger table is an array of m pointers to nodes: succ((n + 2^(i-1)) mod 2^m). First finger is the successor. Chord works correctly iff FT[0] is correct.

### Other issues

- Every node needs to keep info about its predecessor: periodically node asks its successor about its predecessor, p. If p is between self and successor, update successor to p and notify p about choosing it as the new successor (p will set the new predecessor).
- Periodically update elements in FT one at a time.
- Periodically check if predecessor is still alive
- Node can keep a list of r successors: if node fails, the next node can replace it.
- Cryptographic hash functions help achieve tolerance against denial-of-service attacks.

### Virtual topology issues

Chord and P2P systems use an overlay network. If topology is oblivious to the underlying physical network, routing may be inefficient. Solution 1: assign identifiers so that the overlay topology is close to that of the underlying physical topology (not possible in chord). Solution 2: route based on the underlying topology. Keep several nodes per interval and resolve to the closest one. Solution 3: pick neighbors according to the underlying topology (not possible in chord).

---

## Names

Are layers of indirection.

### Identifier is a name with 3 properties

- refer to one entity at most
- an entity has at most one identifier
- an identifier always refers to the same entity (never reused)

### Binding

Binding is a mapping from a name to an object/entity. Name resolution is the process of finding a binding for a name.

A name is always resolved in the context of its name space.

### Name resolution

In small scale, can use a single server (e.g.: rmiregistry).

![image-20210616141146735](/home/joao/.config/Typora/typora-user-images/image-20210616141146735.png)

We can use 3 different strategies. Recursive allows for caching at server, but requires servers to keep state and makes it harder to set values of timeouts. Transitive makes it harder for the client to set a timeout.

### Closure mechanism

Allows one to get a context to use to resolve a name. Selecting the initial node in a name space from which name resolution is to start. Ad-hoc and simple.

---

## Clock synchronization

### Local clocks

Generate periodic interrupts. OS uses these interrupts to keep the local time. These clocks have a drift (dH(t)/dt - 1) wrt a perfect clock. Even if this drift bounded, unless clocks synchronize, their values may diverge forever.

### Clock sync (CS)

- External CS - sync against an external time reference (R) - |Ci(t) - R(t)| < a (accuracy), para todo i. This implies internal clock sync too.
- Internal CS - sync among local clocks in the system (not necessarily related to real time) - |Ci(t) - Cj(t)| < pi (precision), para todos i, j.

### CS Centralized Algorithm

Each process has a local clock:

- Master/server clock - provides the time reference (periodically read the local time and broadcast a SYNC message).
- Slave/client clock - synchronize with the master/server clock (update clock upon reception of a SYNC message).

Local clock drifts are bounded. System is synchronous: there are known bounds (both lower and upper) for comm delays; the time a process requires to take a step is negligible with respect to comm delays.

Accuracy: |Ci(t) - Cm(t)| < a for all correct processes i, where Cm(t) is the master's clock.

### Master time estimation

There is a delay to notify slaves. Delay can be estimated min < delay < max (max is known because system is synchronous). To minimize error, use midpoint of [Cm(t) + min, Cm(t) + max]: Cs(t) = Cm(t) + (min + max)/2

### SYNC message period

To ensure accuracy: |Cm(t) - Cs(t)| < a. Because the clock drifts are bounded.

The clock skew lower bound is determined by the delay jitter.

### Message delay estimation

Christian approach (use round-trip-delay): [t + min, t + round - min] -> round/2 - min

Reduce error by time stamping with the local clocks the sending/receiving of messages.

![image-20210616152846395](/home/joao/.config/Typora/typora-user-images/image-20210616152846395.png)

Precision depends on: rate synchronization, symmetry assumption, time stamp accuracy (the lower in the protocol stack, the better).

### Precision Time Protocol

**Syntonization**: ensure master and slave have the same clock rate. Sync message requires the ability to insert the timestamp into the message on the fly. Follow_up message requires an additional message (optionally if hardware doesn't support inserting the timestamp in Sync on-the-fly):

- T1^(k + 1) - T1^k = T2^(k + 1) - T2^k
- T2^(k + 1) - T1^(k + 1) = T2^k - T1^k

**Offset and delay estimation**: Delay_Req + Delay_Resp to obtain round-trip-delay (T3 and T4).

![image-20210616154003065](/home/joao/.config/Typora/typora-user-images/image-20210616154003065.png)

### Local clock correction

- Instantaneous: correct offset only at every sync point (may lead to non-monotonic clocks).
- Amortized: over the sync period adjust both a and b: Cs(t) = a * t + b

Each clock between resync points can be seen as a different clock. Therefore, clock correction can be seen as jumping from one clock to the next.

### NTP

Synchronization hierarchical networks. Servers at the top are primary (lowest stratum), next stratum (level) are secondary, and so on... Leaves are clients. Subnet can be reconfigured in case of failure: secondary server switches primary server; primary becomes secondary when UTC source becomes unreachable.

Uses UDP. Server multicasts its time to clients (low precision but OK for LANs). Request-reply (similar to Cristian's algorithm). Symmetrical (servers swap their times) (used by servers at the lowest strata, ensures highest precision, decentralized algorithm).

### At-most-once messaging

Remember all messages received and discard duplicates. But can't remember all: forget far away past: session-based (handshake after a while), synchronized clocks (all nodes have a synchronized clocks with accuracy a). Each message has connection id (selected by the sender) and must be unique. Synchronized clocks improve performance at the cost of occasionally rejecting a message.

### Leases

Ensure some property during a limited time (lease duration). Absolute duration require synchronized clocks, relative requires synchronized rates. Can be renewed.

### Kerberos

Uses Needham-Schroeder's but instead of their solution for replay attacks, uses synchronized clocks (and to prevent the use of keys for too long).

### Synchronized rates

Whenever possible, we should use synchronized rates. They work for most applications and require little communication. Synchronized clocks are more powerful than rates but support more algorithms (can provide warning when clocks get out of sync).

---

## Events and logical clocks

We often don't need SC, we just need to order events.

### Happened-before relation (->) (captures potential causality)

1. if e and e' happen in that order: e -> e'
2. if e is sending of message m and e' is the corresponding reception: e -> e'
3. if e -> e' and e' -> e'', then e -> e'' (transitive)

a -/-> e /\ e -/-> a  => concurrent a || e

### Lamport clocks

Logical clock used to assign (Lamport) timestamps to events. Each process has its own Lamport clock Li.

Lamport condition: e -> e' => L(e) < L(e') or L(e) >= L(e') => e -/-> e'

### Clock condition

- **C1:** e -> e' in the same process => L(e) < L(e')
- **C2:** e is send msg by i and e' is receive by j => L(e) < L(e')

Note: free-running physical clocks cannot be used as Lamport clocks (problem with C2).

To satisfy the condition a Lamport clock must be updated **before** assigning its value to the event (incrementing a Lamport clock is not an event):

- LC1 if e is not the receiver of a message, just increment Li
- LC2 if e is the receiver of a message m: Li = max(Li, Ts(m)) + 1 (requires sending the timestamp of the sending of event in every message)

### Extended Lamport timestamp

Order the pair (L(e), i), where i is the process where e happens. Somewhat arbitrary but total.

A process executes its command timestamped T when it has learned of all commands issued by all other processes with timestamps less than or equal to T.

### Vector clocks

The main limitation of Lamport clocks is that: **L(e) < L(e') =/=> e -> e'**. We can only conclude only if HB does not hold.

Each process pi keeps its own vector Vi, which it updates as follows:

- VC1 if e is not the receiver of a message, just increment Vi[i]
- VC2 if e is the receiver of a message m:
  - Vi[i] = Vi[i] + 1
  - Vi[j] = max(Vi[j], TS(m)[j])

Initially Vi[j] = 0, for all j. The timestamp of the sending event is piggybacked on the corresponding message (TS(m) = V(send(m))).

Vi[i] is the number of events timestamped by pi. Vi[j] is the number of events in pj that pi knows about.

![image-20210616171525153](/home/joao/.config/Typora/typora-user-images/image-20210616171525153.png)

---

## Atomic commitment

- all processes that decide, must decide the same value
- the decision of a process is final (can't change)
- If some process decides commit, all processes must vote commit
- If a all processes voted commit and there are no failures, then all processes must decide commit
- If in an execution containing only failures that the can be tolerated, all are repaired and no new failures occur (for sufficiently long), then all processes eventually reach a decision.

### Two-phase commit (may block even when there is no failure on all sites)

1 coordinator + several participants. Coordinator can be participant too.

- Phase 1: upon request from the app
  - coordinator sends a VOTE-REQUEST to each participant and wait for reply
  - participant upon receiving a VOTE-REQUEST sends vote: VOTE-COMMIT/YES or VOTE-ABORT/NO
- Phase 2: once coordinator determines that it is time to decide
  - Coordinator decides/sends GLOBAL-COMMIT (everyone voted commit) or GLOBAL-ABORT (else)
  - Participant decides according to the message received from the coordinator

![image-20210616173414489](/home/joao/.config/Typora/typora-user-images/image-20210616173414489.png)

#### Termination protocol

Participant will communicate with other participants to decide the outcome: someone voted abort; someone knows the decision; or just wait.

#### Recovery

Need to keep logs in stable storage of the state of the protocol. 2PC satisfies assertions even in the presence of non-byzantine node failures and communication faults (including partitions).

The main problem with this protocol is that it may require participants to block (wait longer than a comm timeout) (problem less likely in 3PC).

### Independent recovery and blocking

No AC protocol that always allows for local recovery (without comm with other procs). No AC protocol that never blocks in the presence of comm failures or failures on all other processes.

### Three-Phase commit

Adds the PRECOMMIT phase to 2PC. PRECOMMIT states ensure the non-blocking cond:

- no process can commit while another proc is in an uncertain state (INIT, WAIT, READY)
- proc in PRECOMMIT will decide commit (not uncertain) unless the coordinator fails.

This cond is necessary and sufficient to prevent blocking unless all processes fail:

- if there's a majority, see majority. If READY decide ABORT; if PRECOMMIT decide PRECOMMIT.
- processes may block if there is no majority.

---

## State replication with primary backup

### Basic algorithm

- one server is the **primary** and the remaining are **backups**.
- the client send requests to the primary only: it executes the requests, updates the state of the **backups** and responds to the clients (orders the different client requests).
- If the primary fails, a failover occurs and one of the backups becomes the new primary (may use leader election).

### Failure detection

- send I'M ALIVE messages periodically or acknowledgment messages (only reliable if the system is synchronous).
- Failover: select new primary.

### Primary failure

- Crashes after sending response to client - transparent to client unless message is lost.
- Crashes before sending update to backups - no backup receives the update; if client retransmits the request, it will be handled as a new request by the new primary.
- Crashes after sending update (and before sending response) (must ensure update delivery atomicity):
  - all backups receive update - if client retransmits request, new primary will respond (update message must include response if operation is non-idempotent).
  - not all receive update - backups will be in inconsistent state.

### Recovery

When a replica recovers, its state is stale (can't apply updates and send ACKs to primary). Solution is to use a state transfer protocol to bring the state of the backup in sync with that of the primary:

- resending missing UPDATES
- transferring the state itself

In both cases, the recovering replica can buffer the UPDATE messages received from the primary and process them when its state is sufficiently up to date.

### Non-blocking algorithm

Waiting for backup ACKs increases latency. Solution is allowing primary to send response to client before receiving ACKs from backups.

---

## Consensus

- all procs get a set of values, V, as input
- each proc chooses a **single** value, v, from V (decision is irreversible)
- all procs must choose the same val (agreenment)
- the value must be in the input set (validity)
- in a failure free execution, eventually all procs chose a value (termination)

### Synod

- Proposers - send proposals to the acceptors. Proposal is a pair, (n, v), containing an unique number (proposal identifier) and v (some value from V).
- Acceptors - accept the proposals. A proposal is accepted if it is valid. A value is chosen if a majority of acceptors accepts a proposal with a given value.
- Learners - learn the chosen value.

2 phases:

1. find a number that makes the proposal likely to be accepted (and the value that has been chosen, if any)
2. submit a proposal

#### Synod phase 1

- **Proposer** - selects a proposal number n and sends a PREPARE(n) request to a majority of acceptors
- **Acceptor** - upon receiving a PREPARE where n is larger than any previous PREPARE request to which it has already responded, responds with:
  - a promise to not accept any more proposals numbered less than n + highest-numbered proposal (if any) that it has accepted.
  - else, the acceptor doesn't need to respond

#### Synod phase 2

- **Proposer** - If receives response to its PREPARE request (numbered n) from a majority of acceptors, then it sends an ACCEPT(n ,v) request to each of those acceptors for a proposal numbered n with value v, where v is:
  - value of the highest-numbered proposal among the responses received in phase 1
  - or is the proposer's input value (if the responses didn't report values)
- **Acceptor** - if receives an ACCEPT request for a proposal numbered n, it accepts the proposal unless it has already responded to a PREPARE request having a number greater than n.

![image-20210617110124848](/home/joao/.config/Typora/typora-user-images/image-20210617110124848.png)

#### Learners

- acceptor can notify all learners: minimizes learning delay, but too much traffic
- A learner may learn from another learner: acceptors notify the learner leader (leader selection) or set of distinguished learners
- Learner can ask the acceptors what proposals they accepted, or ask proposer to make another proposal so we can see what happens

### Paxos

Each process plays the role of proposer, acceptor and learner. The chosen one is the _**distinguished**_ PROPOSER AND LEARNER.

Stable storage is used to keep state used by acceptors and that must survive crashes:

- the largest number of any PREPARE to which it responded
- the highest-numbered proposal it has accepted
- updates to stable storage must occur before sending the response

Uniqueness of the proposal numbers is ensured by using a pair (cnt, id). Each proposer stores in stable storage the highest-numbered proposal it has tried to issue.

### State machine replication with Paxos

Paxos decides which command should be n: the ith instance is used to decide the ith command in the sequence; different executions may run concurrently but must ensure that each command is executed only once.

A single server is elected to be the leader (distinguished proposer). When leader receives command, it chooses a sequence number for the command and proposes the execution of that command. Leader may not succeed if it fails or another server believes itself to be leader.

1. leader executes phase 1 for all instances (using a single very short message (same proposal number, 1, for all instances))
2. acceptor will respond with a simple OK message
3. for each client command (received by the leader), it can run phase 2 for the next instance
4. a leader may start phase 2 for instance i before it learns the value chosen by instance i - 1 (an implementation may bound the number of pending phase 2 instances). This can lead to problems with leader failure, because new leader might have learned cmd for instance i, but not for instance i - k.
5. server may execute command i if it learned the command chosen in the ith instance and has executed all commands up to command i.

If failure of a leader is a rare event, the cost of executing a state machine command is essentially the cost of executing only phase 2: this is the best we can hope for; Paxos is almost optimal.

Election of a single leader is needed only to ensure progress.

---

## Group based communication

### Group membership (which processes belong to the group)

Outputs a **view** of the group. Each view is a set of processes with an identifier (vi) (alternatively, a process can get a new identifier every time it join the group). If a process has a view, all procs in that view must have agreed to join the view.

Type:

- Primary component - ensures that at "any time" there is at most one view (the views are totally ordered; achieved by requiring a view to comprise a majority of the procs)
- Partitionable - allows the existence of more than one view at any time.

Interface:

- join/leave - used by procs to request to join/leave groups
- new-view - used to notify a view change, either in response to: voluntary request (join/leave); unexpected events (failure or recovery of procs).

Failure detection: needs not be reliable (a process may be expelled from a group by mistake).

### Reliable broadcast in static groups

- Validity - if a correct proc broadcasts m, then all correct procs in the group deliver m eventually
- Agreement - if a correct proc delivers m, then all correct procs in the group deliver m eventually
- Integrity - a proc delivers m at most once and only if it was previously broadcasted by another proc (in closed groups, broadcaster must be a group member too).

Failure assumptions: procs may fail by crash and recover; any network partition eventually heals.

### Reliable broadcast in dynamic groups

Validity and agreement change from **all correct procs** to **all correct procs in that view**.

Validity and agreement conflict with: groups as a mean to keep track of the state of procs in the system in a coordinated way; impossiblity of distinguishing between slow, failed, and unreachable procs.

### View/Virtual Synchronous Multicast

**Virtual synchrony**: if a proc changes from view V to V', it delivers the same set of messages in view V (variation of agreement).

**Self delivery**: if a correct proc p broadcasts message m in view v, then it delivers m in that view (variation of validity)

Assume:

- point-to-point channels
- reliable channels - if the procs at the end of a point-to-point channel are correct, then a msg send in one end will be delivered at the other.
- FIFO channels - msgs delivered in order
- Procs fail by crash only.

Sender crash in middle of multicast:

- deliver the msg **only if** all correct procs receive it (increases delivery latency).
- deliver the msg immediately - upon view change, procs that survive the current view must send each other the messages they have delivered that may have not been received by other view members.

A message m is **stable** for proc p, if p knows that all other procs in the view have received it:

- keep a copy of the messages delivered until they become stable.
- upon view change: resend all non-stable messages to the remaining procs; wait for the reception of non-stable msgs from other procs (and deliver them if haven't done it done); change to the new view. **OR** a proc is elected as coordinator (doesn't require extra message because all group members are known so we can use a rule like selecting member with smallest id); each proc sends its non-stable msg to the coordinator.

Procs send a FLUSH message after sending all non-stable msgs. Upon receiving a FLUSH from each proc in the current and next view, a proc may change its view. Procs initiate a new view change if a proc crashes during this protocol.

### PBR implementation with View Sync

Primary can be determined from the view membership without further communication. Primary sends the updates to the backups using view sync multicast.

- Upon receiving a request, the primary:
  - executes it;
  - generates and update representing the change in the state;
  - multicasts the UPDATE to the current view, Vi.
- Upon receiving an UPDATE, the backup:
  - updates its state accordingly
  - sends its acknowledgment to the sender

Upon failure, a new view is generated (and a new primary is generated if the primary failed).

Ensures UPDATE delivery atomicity to replicas that move from one view to the next: either all replicas that move from that view to the next deliver the UPDATE or none does it.

Upon view change, new members must sync their state: still need a state transfer protocol. At most-once semantics: if uncertain, cache the response (retransmit response if client retransmits the request).

repository: JoaoCostaIFG/SDIS

repo_name: JoaoCostaIFG/SDIS


text:
# The Challenge:
"The challenge is to integrate NASA data, ground-based air quality data, and citizen science data to create an air quality surface that displays the most accurate data for a location and time. Create algorithms that select or weight the  best data from several sources for a specific time and location, and display that information" [*1]

--------------------------------------------------------------------------------------------------------
# Hawa
Hawa is one of the NASA challenges for 2019. It is an algorithm that integrates the most accurate data that indicates more efficiency about the Air quality for a specific location and time:

-----------------------------------------------------------------------------------------------------------
# Quesitions to answer:
1. [As long as we deal with an existed data? who is the resource of the input data? Who is going to benefit from our output?]
>>>

2.[ Why air quality? Why the world care about the air quality? what is the core value of this algorithm?]
>>>---------------------
"Air pollution is associated with almost 5 million annual deaths worldwide as well as harm to countless others through negative health impacts from asthma and diabetes to cardiovascular disease an cancer. The World Bank estimates that air pollutio cosets the global economy $225 billion in lost labor income and $5 trillion in welfare losses annually" [*2] [**]

3. [Those who will use Hawa algorithm, what are the addressed problem in real world that they want to solve using Hawa Algorithm?]
>>>

4. [How is Hawa algorithm going to be different form the existed methodologies> Can we address what companies, organizations, or existed industries and agencies that have a strong technology that similar or better than Hawa algorithm in any way? ]
*note: this question indicates the energy and passion that push us to keep Hawa uniqueness, success, cooperation, and partnership
>>>

5. [From a scientific stand point, what are the main factors that the air quality relying on?]
>>>

----------------------------------------------------------------------------------------------
# Resources:
[*1] Sustainability at the U.S. Department of state:
https://www.state.gov/communications-outreach-and-partnerships/#space-apps
> "more than 100 U.S. experts from municipal, state, and Federal governments and universities have been paired with embassies and consulates to help Fellows have helped build capacity and understanding about how to understand air quality data, both within the embassy community and with host governments."

[*2] https://2019.spaceappschallenge.org/challenges/living-our-world/surface-air-quality-mission/details

-----------------------------------------------------------------------------------------------------
# information for the presentation:
Air pollution is associated with almost 5 million annual deaths worldwide as well as harm to countless others through negative health impacts from asthma and diabetes to cardiovascular disease an cancer. The World Bank estimates that air pollutio cosets the global economy $225 billion in lost labor income and $5 trillion in welfare losses annually" [*2]
Yet real-time, realiable air quality data are not avaliable in most of the world, leaving people with a significant and harmful knowledge gap.

----------------------------------------------------------
# Sample Data for the Surface-to-Air Quality Mission:
( The following data has been compiled with satellite data, local meteorological data, and ground reference monitor data for the sites in three different location; they are:
a. Los Angeles, CA, USA
b. Delhi, India
c. Addis Ababa, Ethiopia

  # Aerosol Otical Depth (AOD) Data:
  AOD is a unitless quaitity representing the opacity of the atmosphere form ground to dspace at the fiven location and time. Each measurement line includes the AOD form the 10km x 10 Km grid pint closest to the ground sites as wll as the average .
  Aerosol optical depth is a measure of the extinction of the solar beam by dust and haze. In other words, particles in the atmosphere (dust, smoke, pollution) can block sunlight by absorbing or by scattering light. ... An average aerosol optical depth for the U.S. is 0.1 to 0.15.https://www.esrl.noaa.gov/gmd/grad/surfrad/aod/

  https://www.state.gov/wp-content/uploads/2019/10/AOD_DATA.zip
  includes satellite derived Aerosol optical Depth produced from NASA MODIS instruments on the Terra(morning) and Aqua(afternoon) satellites using NASA'S Deep Blue and Dark target Algorithm. https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20140012657.pdf

  ** What does the Terra satellite do?
In December 1999, NASA launched the Terra satellite as the flagship mission of the Earth Observing System to answer these questions. The Terra satellite carries five instruments that observe Earth's atmosphere, ocean, land, snow and ice, and energy budget https://www.nasa.gov/mission_pages/terra/spacecraft/index.html

  ** Aqua is an Earth Science satellite mission that collects information on our water systems. The satellite has six different Earth-observing instruments on board and streams approximately 89 gigabytes of data per day. https://www.nasa.gov/mission_pages/aqua/index.html

  For each location, there are two files: those ending with MOD04: ARE FROM THE MORNING satellite and those with MYD04 are fromthe afternoon satellite

  Tiny solid and liquid particles suspended in the atmosphere are called aerosols. Windblown dust, sea salts, volcanic ash, smoke from wildfires, and pollution from factories are all examples of aerosols. Depending upon their size, type, and location, aerosols can either cool the surface, or warm it. They can help clouds to form, or they can inhibit cloud formation. And if inhaled, some aerosols can be harmful to people's health.https://earthobservatory.nasa.gov/global-maps/MODAL2_M_AER_OD

  An optical thickness of less than 0.1 (palest yellow) indicates a crystal clear sky with maximum visibility, whereas a value of 1 (reddish brown) indicates very hazy conditions.
High aerosol amounts are linked to different process in different places and times of year. High aerosol amounts occur over South America from July through September. This pattern is due to land clearing and agricultural fires that are widespread across the Amazon Basin and Cerrado regions during the dry season.

Readme: (product name):
- MYD04: this represents MODIS-AQUA satellite (afternoon overpass)
-MOD04:  this represents MODIS-TERRA Satellite (morning overpass)
data ordered in each file:
- YYYY, MM, DD, latitude, longitude , AOD1, AD3 STD3
where AOD1: Aerosal optical Depth at 550 nm for nearest gird to ground location
AOD3:  Aeorsal optical depth at 55 nm

# Meteorological Data:
The file: https://www.state.gov/wp-content/uploads/2019/10/MERRA2.zip
this contains 365 meterological data files corresponding to each day of 2018. within each files are 24 hourly measurements for each of the 22 statioin locations. each hourly measurements includes locaiton, date, time, water vapor, temperature, wind speed, and trace gas concentrations.
MERRA2 Data Columns

"""
**Station – Name of ground monitor for data row
Lat – Latitude (degrees north) of station
Lon – Longitude (degrees east) of station
SRadius – Search radius (km) for nearest MERRA grid point to station
MERRALat – Latitude (degrees north) of nearest MERRA grid point to station
MERRAlon – Longitude (degrees east) of nearest MERRA grid point to station
IDXi – I index of MERRA grid point
IDXj – J index of MERRA grid point
PS – Surface pressure (Pa)
QV10m – Specific humidity at 10 m above surface (kg/kg)   	(multiplied by 1000.0)
Q500 - Specific humidity at 500 mbar pressure (kg/kg) 		(multiplied by 1000.0)
Q850 – Specific humidity at 850 mbar pressure (kg/kg) 		(multiplied by 1000.0)
T10m – Temperature at 10 m above surface (Kelvin)
T500 – Temperature at 500 mbar pressure (Kelvin)
T850 – Temperature at 850 mbar pressure (Kelvin)
Wind – Surface wind speed (m/s)
BCSMASS – Black Carbon mass concentration at surface (μg/m3)
DUSMASS25 – Dust surface mass PM 2.5 concentration at surface (μg/m3)
OCSMASS – Organic carbon mass concentration at surface (μg/m3)
SO2SMASS – Sulphur dioxide mass concentration at surface (μg/m3)
SO4SMASS – Sulphate aerosol mass concentration at surface (μg/m3)
SSSMASS25 – Sea Salt surface mass concentration PM 2.5 (μg/m3)
TOTEXTTAU – Total aerosol extinction AOT @ 550 nm (unitless)
UTC_DATE – YearMonthDay (GMT date)
UTC_TIME – Time of sample (hours) (GMT time)
"""

These data were produced by NASA using satellite obervasions merged with atmosphered model forecasts in the reanalysis to provide the best estimates in the retorspective sense of atmospheric quantitie. these data are typically used to quantity and the near surfce particulate matter quantities such as PM2.5 And PM10

PM2.5 refers to atmospheric particulate matter (PM) that have a diameter of less than 2.5 micrometers, which is about 3% the diameter of a human hair.

# Refernce Ground Monitor Data:
https://www.state.gov/wp-content/uploads/2019/10/Reference_Monitor_Data.zip
contains historical measurements of ground pollutants at each of the 22 locations for various time periods between 2016 and 2019. Each files contains mesuremennt of the PM2.5, P.M10, and trace gas pollutants for time periods and sampling intervals that vary by site.Not all sites have all data for the full period..

these data were obtained from the open AQ website.Other historical da

# How is Air Quality Measured?
- Air quality is measured with Air Quality index, or AQI. it works like thermometer that runs form 0 to 500 degress. However, instead of showing changes in the temperature, the AQI is a way of showing changes in the amount of pollution in the air.https://www.epa.gov/sites/production/files/2014-05/documents/zell-aqi.pdf

# PSI ( Pollutant standards index)
https://www.haze.gov.sg/docs/default-source/faq/computation-of-the-pollutant-standards-index-(psi).pdf

repository: Zaid-msstate/Hawa

branch_name: refs/heads/master

extension: md

license_type: permissive

repo_name: Zaid-msstate/Hawa


text:
---
title: Managing listing data in Console
slug: console-manage-listings
updated: 2022-04-13
category: concepts-management
ingress: Manage, edit, add, and browse listing information in Console
published: true
---

The Listings tab includes all information about your users. The data is
presented in a list of listing cards. To view a listing’s information,
click on their card and it will expand to show more information.

## Listing card

Each listing has a card with all the information about them. Each
listing card is divided into sections.

![Listing card](./listing-card.png)

**The left side of the card has** general information and quick links.
You can see the main listing picture (and the listing author profile
picture) and a count of the number of pictures added if they have any.
You can navigate directly to the listing author user card from the
author profile picture. You find the listing title and the price. You
can also easily navigate to a filtered view of all transactions and
reviews where that listing is involved in the corresponding transaction
or review section.

You also have an options menu on the top right corner where you can find
quick actions links to close the listing or delete the listing
permanently. If the listing is pending approval, the only option
available is to delete the listing.

**The right side of the card has** the Summary section, which contains
default information about the listing, like the Listing title, Listing
description, Listing ID, and the listing’s creation date. These fields
are not changeable or deletable, but most of their contents can be
edited directly on this page. You cannot, however, make changes to the
listing creation date and the listing ID, which is a unique identifier
within the Flex database.

Below the summary, you have separate sections for each type of extended
data. Extended data is a Flex feature that allows you to customize and
collect user, listing, and transaction data specific to your
marketplace. You can read
[an introduction to extended data](/concepts/extended-data-introduction/)
to learn more. All extended data can be edited directly through Console.
You can change existing field values or even add new extended data
fields.

At the bottom of the listing card, you have the Events section. It
contains a Flex CLI command that you can use to view all the events
related to this listing. Read more about
[viewing events with Flex CLI](/how-to/view-events-with-flex-cli/) or
visit the [Events reference in our Docs site](/references/events/).

### Listing approval

If you have
[the listing approval feature](/operator-guides/concepts/#listing-pending-approval)
enabled in your marketplace, you will find the “approve listing” button
at the bottom of the displayed section of the listing card.

## Browse and search Listings

You can scroll through listings cards at a rate of 100 per page, or use
the search function to find specific listings via title, description, or
ID. You can also filter listings by status: pending approval, closed,
published, or draft. You can combine the filters with the search
function.

## Download listings’ information

You also have the option to download your listing data as a CSV, which
you can import to a third-party system.

repository: sharetribe/flex-docs

gha_language: JavaScript

gha_license_id: NOASSERTION

repo_name: sharetribe/flex-docs


text:
---
title: Blog roundup - February 4, 2015
date: 2015-02-04 12:41:42
author: rbowen
---

Here's what RDO engineers have been writing about over the past week.

If you're writing about RDO, or about OpenStack on CentOS, Fedora or RHEL, and you're not on my list, please let me know!

**An atomic upgrade process for OpenStack compute nodes**, by Steven Dake

> I have been working with container technology since September 2014, sorting out how they are useful in the context of OpenStack.  This led to my involvement in the Kolla project, a project to containerize OpenStack as well as Magnum, a project to provide containers as a service.  Containers are super useful as  an upgrade tool for OpenStack, and the main topic of this blog post.

... Read more at [http://tm3.org/blog84](http://tm3.org/blog84)

**Notes from a talk on “Advanced snapshots with libvirt and QEMU”**, by Kashyap Chamarthy

> I just did a talk at a small local conference (Infrastructure.Next, co-located with cfgmgmtcamp.eu) about Advanced Snapshots in libvirt and QEMU.

... Read more at [http://tm3.org/blog85](http://tm3.org/blog85)

license: MIT

repository: gbraad/redhat-openstack-website

gha_language: HTML

repo_name: gbraad/redhat-openstack-website


text:
# HW2-Question 4

> Question 4

Write a function swap that will swap the values of the inputs (two integers).  Implement this using pass by reference feature using 1. references 2. pointers

### Runtime Analysis

This question is asking to use reference and pointer to complete swap.
For both methods, we just need to exchange the value, so the runtime should be [`O(1)`]

### Run main:

You can run this using `blaze`:

```bash
bazel run src/main:main
```

### Run Tests:

You can run unit tests using [`blaze`](installing-bazel):

```bash
bazel test tests:tests
```

branch_name: refs/heads/main


text:
# An Akka Persistence Plugin for Mongo

[![Build Status](https://travis-ci.org/tradecloud/akka-persistence-mongo.svg?branch=master)](https://travis-ci.org/tradecloud/akka-persistence-mongo) [![Maven Central](https://maven-badges.herokuapp.com/maven-central/nl.tradecloud/akka-persistence-mongo_2.12/badge.svg)](https://maven-badges.herokuapp.com/maven-central/nl.tradecloud/akka-persistence-mongo_2.12) [![License](http://img.shields.io/:license-mit-blue.svg)](http://doge.mit-license.org)

A replicated [Akka Persistence](http://doc.akka.io/docs/akka/current/scala/persistence.html) journal backed by [MongoDB Casbah](http://mongodb.github.io/casbah/).

## Prerequisites

### Release

| Technology | Version                          |
| :--------: | -------------------------------- |
| Scala      | 2.11.12, 2.12.4 - Cross Compiled |
| Akka       | 2.5.9 or higher                  |
| Mongo      | 2.6.x or higher                  |

## Installation

### SBT

It can then be included as dependency:

```scala
libraryDependencies += "nl.tradecloud" %% "akka-persistence-mongo" % "1.0.1"
```

### Build Locally

You can build and install the plugin to your local Ivy cache. This requires sbt 1.0.4 or above.

```scala
sbt publishLocal
```

## Mongo Specific Details

Both the [Journal](#journal-configuration) and [Snapshot](#snapshot-configuration) configurations use the following Mongo components for connection management and write guarantees.

### Mongo Connection String URI

The [Mongo Connection String URI Format](http://docs.mongodb.org/manual/reference/connection-string/) is used for establishing a connection to Mongo. Please **note** that while some of the components of the connection string are **[optional]** from a Mongo perspective, they are **[required]** for the Journal and Snapshot to function properly. Below are the required and optional components.

#### Standard URI connection scheme

```scala
mongodb://[username:password@]host1[:port1][,host2[:port2],...[,hostN[:portN]]][/[database][.collection][?options]]
```

#### Required

| Component     | Description                                                                                |
| :------------ | ------------------------------------------------------------------------------------------- |
| `mongodb:// ` | The prefix to identify that this is a string in the standard connection format.             |
| `host1`       | A server address to connect to. It is either a hostname, IP address, or UNIX domain socket. |
| `/database  ` | The name of the database to use.                                                            |
| `.collection` | The name of the collection to use.                                                          |

#### Optional

| Component | Description |
| :-------- | ----------- |
| `username:password@` | If specified, the client will attempt to log in to the specific database using these credentials after connecting to the mongod instance. |
| `:port1` | The default value is :27017 if not specified.                                               |
| `hostN` | You can specify as many hosts as necessary. You would specify multiple hosts, for example, for connections to replica sets. |
| `:portN` | The default value is :27017 if not specified. |
| `?options` | Connection specific options. See [Connection String Options](http://docs.mongodb.org/manual/reference/connection-string/#connections-connection-options) for a full description of these options. |

### Mongo Write Concern

The [Write Concern Specification](https://docs.mongodb.org/manual/reference/write-concern/) describes the guarantee that MongoDB provides when reporting on the success of a write operation. The strength of the write concern determine the level of guarantee. When a `PersistentActor's` `persist` or `persistAsync` method completes successfully, a plugin must ensure the `message` or `snapshot` has persisted to the store. As a result, this plugin implementation enforces Mongo [Journaling](https://docs.mongodb.org/manual/core/journaling/) on all write concerns and requires all mongo instance(s) to enable journaling.

| Options | Description |
| :------------ | ----------- |
| `woption`     | The `woption` requests acknowledgment that the write operation has propagated to a specified number of mongod instances or to mongod instances with specified tags.  Mongo's `wOption` can be either an `Integer` or `String`, and this plugin implementation supports both with `woption`.

If `woption` is an `Integer`, then the write concern requests acknowledgment that the write operation has propagated to the specified number of mongod instances. Note: The `woption` cannot be set to zero.

If `woption` is a `String`, then the value can be either `"majority"` or a `tag set name`. If the value is `"majority"` then the write concern requests acknowledgment that write operations have propagated to the majority of voting nodes. If the value is a `tag set name`, the write concern requests acknowledgment that the write operations have propagated to a replica set member with the specified tag. The default value is an `Integer` value of `1`. |
| `wtimeout`    | This option specifies a time limit, in `milliseconds`, for the write concern. If you do not specify the `wtimeout` option, and the level of write concern is unachievable, the write operation will **block** indefinitely. Specifying a `wtimeout` value of `0` is equivalent to a write concern without the `wTimeout` option. The default value is `10000` (10 seconds). |

## Journal Configuration

### Activation

To activate the journal feature of the plugin, add the following line to your Akka `application.conf`. This will run the journal with its default settings.

```scala
akka.persistence.journal.plugin = "casbah-journal"
```

### Connection

The default `mongo-url` is a `string` with a value of:

```scala
casbah-journal.mongo-url = "mongodb://localhost:27017/store.messages"
```

This value can be changed in the `application.conf` with the following key:

```scala
casbah-journal.mongo-url

# Example
casbah-journal.mongo-url = "mongodb://localhost:27017/employee.events"
```

See the [Mongo Connection String URI](#mongo-connection-string-uri) section of this document for more information.

### Write Concern

The default `woption` is an `Integer` with a value of:

```scala
casbah-journal.woption = 1
```

This value can be changed in the `application.conf` with the following key:

```scala
casbah-journal.woption

# Example
casbah-journal.woption = "majority"
```

### Write Concern Timeout

The default `wtimeout` is an `Long` in milliseconds with a value of:

```scala
casbah-journal.wtimeout = 10000
```

This value can be changed in the `application.conf` with the following key:

```scala
casbah-journal.wtimeout

# Example
casbah-journal.wtimeout = 5000
```

See the [Mongo Write Concern](#mongo-write-concern) section of this document for more information.

### Reject Non-Serializable Objects

This plugin supports the rejection of `non-serializable` journal messages. If `reject-non-serializable-objects` is set to `true` and a message is received who's payload cannot be `serialized` then it is rejected.

If set to `false` (the default value) then the `non-serializable` payload of the message becomes `Array.empty[Byte]` and is persisted.

the default `reject-non-serializable-objects` is a `Boolean` with the value of:

```scala
casbah-journal.reject-non-serializable-objects = false
```

This value can be changed in the `application.conf` with the following key:

```scala
casbah-journal.reject-non-serializable-objects

# Example
casbah-journal.reject-non-serializable-objects = true
```

## Snapshot Configuration

### Activation

To activate the snapshot feature of the plugin, add the following line to your Akka `application.conf`. This will run the snapshot-store with its default settings.

```scala
akka.persistence.snapshot-store.plugin = "casbah-snapshot"
```

### Connection

The default `mongo-url` is a `string` with a value of:

```scala
casbah-snapshot.mongo-url = "mongodb://localhost:27017/store.snapshots"
```

This value can be changed in the `application.conf` with the following key:

```scala
casbah-snapshot.mongo-url

# Example
casbah-snapshot.mongo-url = "mongodb://localhost:27017/employee.snapshots"
```

See the [Mongo Connection String URI](#mongo-connection-string-uri) section of this document for more information.

### Write Concern

The default `woption` is an `Integer` with a value of:

```scala
casbah-snapshot.woption = 1
```

This value can be changed in the `application.conf` with the following key:

```scala
casbah-snapshot.woption

# Example
casbah-snapshot.woption = "majority"
```

### Write Concern Timeout

The default `wtimeout` is an `Long` in milliseconds with a value of:

```scala
casbah-snapshot.wtimeout = 10000
```

This value can be changed in the `application.conf` with the following key:

```scala
casbah-snapshot.wtimeout

# Example
casbah-snapshot.wtimeout = 5000
```

See the [Mongo Write Concern](#mongo-write-concern) section of this document for more information.

### Snapshot Load Attempts

The snapshot feature of the plugin allows for the selection of the youngest of `{n}` snapshots that match an upper bound specified by configuration. This helps where a snapshot may not have persisted correctly because of a JVM crash. As a result an attempt to load the snapshot may fail but an older may succeed.

The default `load-attempts` is an `Integer` with a value of:

```scala
casbah-snapshot.load-attempts = 3
```

This value can be changed in the `application.conf` with the following key:

```scala
casbah-snapshot.load-attempts

# Example
casbah-snapshot.load-attempts = 5
```

## Status

* All operations required by the Akka Persistence [journal plugin API](http://doc.akka.io/docs/akka/current/scala/persistence.html#Journal_plugin_API) are supported.
* All operations required by the Akka Persistence [Snapshot store plugin API](http://doc.akka.io/docs/akka/current/scala/persistence.html#Snapshot_store_plugin_API) are supported.
* Tested against [Plugin TCK](http://doc.akka.io/docs/akka/current/scala/persistence.html#Plugin_TCK).
* Plugin uses [Asynchronous Casbah Driver](http://mongodb.github.io/casbah/3.1/)
* Message writes are batched to optimize throughput.

## Performance

Minimal performance testing is included against a **native** instance. In general the journal will persist around 8,000 to 10,000 messages per second.

## Sample Applications

The [sample applications](https://github.com/ironfish/akka-persistence-mongo-samples) are now located in their own repository.

## Change Log

### 1.0.0-SNAPSHOT

* Upgrade `Akka` 2.4.1.
* Upgrade `Casbah` to `Async` driver 3.1.
* Supports latest [Plugin TCK](http://doc.akka.io/docs/akka/current/scala/persistence.html#Plugin_TCK).

### 0.7.6

* Upgrade `sbt` to 0.13.8.
* Upgrade `Scala` cross-compilation to 2.10.5 & 2.11.7.
* Upgrade `Akka` to 2.3.12.

### 0.7.5

* Upgrade `sbt` to 0.13.7.
* Upgrade `Scala` cross-compilation to 2.10.4 & 2.11.4.
* Upgrade `Akka` to 2.3.7.
* Examples moved to their own [repository](https://github.com/ironfish/akka-persistence-mongo-samples).
* Removed `logback.xml` in `akka-persistence-mongo-casbah` as it was not needed.
* Added `pomOnly()` resolution to `casbah` dependency, fixes #63.

### 0.7.4

* First release version to Maven Central Releases.
* Upgrade `Sbt` to 0.13.5.
* Upgrade `Scala` cross-compilation to 2.10.4 & 2.11.2.
* Upgrade `Akka` to 2.3.5.
* Added exception if `/database` or `.collection` are not accessible upon boot. Thanks @Fristi.
* Modified snapshot feature for custom serialization support. Thanks @remcobeckers.

### 0.7.3-SNAPSHOT

* Upgrade `Sbt` to 0.13.4.
* Upgrade `Scala` cross-compilation to 2.10.4 & 2.11.2.
* Upgrade `Akka` to 2.3.4.
* `@deprecated` write confirmations, `CasbahJournal.writeConfirmations`, in favor of `AtLeastOnceDelivery`.
* `@deprecated` delete messages, `CasbahJournal.deleteMessages`, per akka-persistence documentation.

## Author / Maintainer

* [Duncan DeVore (@ironfish)](https://github.com/ironfish/)

## Contributors

* [Heiko Seeberger (@hseeberger)](https://github.com/hseeberger)
* [Sean Walsh (@SeanWalshEsq)](https://github.com/sean-walsh/)
* [Al Iacovella](https://github.com/aiacovella/)
* [Remco Beckers](https://github.com/remcobeckers)
* [Mark Fristi](https://github.com/Fristi)

repository: tradecloud/akka-persistence-mongo

gha_language: Scala

repo_name: tradecloud/akka-persistence-mongo


text:
3.26 - John Henry Thompson - Invent Your Future

[Yoga](../../../yoga.md) > [Patanjali's Yoga Sutras - A Comparison](../../patanjani.md) > [Book 3](../book-3.md) >

### 3.26

**भुवज्ञानं सूर्येसंयमात् ॥२६॥**
bhuva-jñānaṁ sūrye-saṁyamāt ||26||

[\[RS\] 3.26 Meditation (samyama) on the sun gives rise to knowledge (jnana) of the ethereal and physical worlds.](http://www.ashtangayoga.info/philosophy/yoga-sutra-patanjali/chapter-3/item/bhuva-jnanam-surye-sanyamat-26/)

[\[JW\] 3.26 As a result of constraint upon the sun \[there arises the intuitive\] knowledge of the cosmic-spaces (bhuvana).](http://books.google.com/books?id=YzFImjtOxUwC&pg=PA254&ci=180%2C164%2C726%2C61&source=bookclip)

[\[SS\]](http://www.amazon.com/Yoga-Sutras-Patanjali-Commentary-Satchidananda/dp/0932040381) 3.27 By samyama on the sun, knowledge of the entire solar system is obtained. \[p189\]

[\[EB\]](http://www.amazon.com/Yoga-Sutras-Patanjali-Translation-Commentary/dp/0865477361/ref=sr_1_1?ie=UTF8&s=books&qid=1250508322&sr=1-1) 3.26 By performing samyama on the sun arises knowledge of the different realms in the universe. \[p353\]

[<Page 3.25](325.md)  [Page 3.27>](327.md)

(**भुवन**, bhuvana) = world; realm; ethereal and physical worlds
(**ज्ञान**, jñāna) = knowledge; understanding
(**सूर्ये**, sūrye) = concerning the sun; concerning surya
(**संयमात्**, saṁyamāt) = deep contemplation; meditation

gha_license_id: MIT


text:
# My Project

This is an example project that can be used as a starting point to create your own Vaadin application with Spring Boot.
It contains all the necessary configuration and some placeholder files to get you started.

The project is a standard Maven project, so you can import it to your IDE of choice. [Read more how to set up a development environment](https://vaadin.com/docs/v14/flow/installing/installing-overview.html) for Vaadin projects (Windows, Linux, macOS).

This project was created from https://start.vaadin.com.

## Running the Application
There are two ways to run the application:
 - To run from the command line, use `mvn` and open [http://localhost:8080](http://localhost:8080) in your browser.
 - Another way is to to run the `Application` class directly from your IDE.

#### Intellij IDEA
- On the right side of the window, select Maven --> Plugins--> `spring-boot` --> `spring-boot:run` goal
- Optionally, you can disable tests by clicking on a `Skip Tests mode` blue button.

Clicking on the green run button will start the application.

After the application has started, you can view your it at http://localhost:8080/ in your browser.

#### Eclipse
- Right click on a project folder and select `Run As` --> `Maven build..` . After that a configuration window is opened.
- In the window set the value of the **Goals** field to `spring-boot:run`
- You can optionally select `Skip tests` checkbox
- All the other settings can be left to default

Once configurations are set clicking `Run` will start the application

## Project structure

- `MainView.java` in `src/main/java` contains the navigation setup. It uses [App Layout](https://vaadin.com/components/vaadin-app-layout).
- `views` package in `src/main/java` contains the server-side Java views of your application.
- `views` folder in `frontend/src/` contains the client-side JavaScript views of your application.

## What next?

[vaadin.com](https://vaadin.com) has lots of material to help you get you started:

- Follow the tutorials in [vaadin.com/tutorials](https://vaadin.com/tutorials). Especially [vaadin.com/tutorials/getting-started-with-flow](https://vaadin.com/tutorials/getting-started-with-flow) is good for getting a grasp of the basic Vaadin concepts.
- Read the documentation in [vaadin.com/docs](https://vaadin.com/docs).
- For a bigger Vaadin application example, check out the Full Stack App starter from [vaadin.com/start](https://vaadin.com/start).

license: Unlicense

repository: eriklumme/reactive-vaadin-demo

repo_name: eriklumme/reactive-vaadin-demo


text:
[![Visual Studio Marketplace](https://img.shields.io/visual-studio-marketplace/i/ins0mniaque.ILSupport.svg)](https://marketplace.visualstudio.com/items?itemName=ins0mniaque.ILSupport) [![License](https://img.shields.io/github/license/ins0mniaque/ILSupport.svg)](LICENSE)

IL Support extension
====================

IL Support is an extension for Visual Studio 2017 and 2019 that provides syntax highlighting for the IL (Intermediate Language) and project templates for C#, F# and Visual Basic that support embedding and calling IL code. You can download it through the Extension Manager, or on the [Visual Studio Marketplace](https://marketplace.visualstudio.com/items?itemName=ins0mniaque.ILSupport).

Syntax Highlighting
-------------------

IL Support includes a simple syntax highlighter for IL files (.il). It adds 3 configurable custom classifications (Instruction, Directive and Directive (Security)), in addition to using the standard classifications.

Project Templates
-----------------

IL Support includes multiple project templates that support embedding and calling IL code in addition to another language (C#, F# and Visual Basic). The generated projects have no dependencies and should build on any Visual Studio 2010/2012/2013/2015/2017/2019 installation, even without this extension installed. They support all the project properties and configurations, build incrementally, and allow breakpoints and debugging step by step in both the main language and the IL.

Every non-empty project contains a simple example showing how to embed IL.

The following projects are included:

  * Visual C#
    * Empty Project with IL Support
    * Class Library with IL Support
    * Portable Class Library with IL Support
    * Console Application with IL Support
    * Windows Forms Application with IL Support
    * WPF Application with IL Support
  * Visual F#
    * F# Application with IL Support
    * F# Library with IL Support
    * F# Portable Library with IL Support
  * Visual Basic
    * Empty Project with IL Support
    * Class Library with IL Support
    * Portable Class Library with IL Support
    * Console Application with IL Support
    * Windows Forms Application with IL Support
    * WPF Application with IL Support

Modifying your own projects to allow IL embedding
-------------------------------------------------

#.NET Framework

  1. Edit your project file and insert [this](https://raw.github.com/ins0mniaque/ILSupport/master/IL%20Support.ProjectTemplates/IL%20Support.targets) __before__ the following section (which should be at the end)
```xml
<!-- To modify your build process, add your task inside one of the targets below and uncomment it.
     Other similar extension points exist, see Microsoft.Common.targets.
<Target Name="BeforeBuild">
</Target>
<Target Name="AfterBuild">
</Target>
-->
```

  2. You're done! Create a new project with IL Support if you need an example of how to embed IL.

#.NET Standard/Core

With the new Visual Studio `.csproj` format, the targets are slightly different.  Complete the steps for .NET Framework, except use [this](https://raw.github.com/ins0mniaque/ILSupport/master/IL%20Support.ProjectTemplates/IL%20Support.Standard.targets) anywhere in your project file

How does it work ?
------------------

The project modification adds 5 simple steps :

  1. Hide IL files from the compiler
  2. Let the project compile
  3. Decompile the resulting assembly into IL
  4. Comment out all methods that were marked with the [forwardref](http://msdn.microsoft.com/en-us/library/system.runtime.compilerservices.methodimploptions%28v=vs.110%29.aspx) method implementation option
  5. Recompile the commented out IL with all the other IL files

License
=======

IL Support is licensed under the MIT License. This means you can do whatever you want as long as you include the original copyright.

See the [LICENSE.md file](https://github.com/ins0mniaque/ILSupport/blob/master/LICENSE.md) for the full license text.

gha_language: C#