text:
# Abstract groups

```agda
module group-theory.groups where
```

<details><summary>Imports</summary>

```agda
open import foundation.action-on-identifications-binary-functions
open import foundation.action-on-identifications-functions
open import foundation.binary-embeddings
open import foundation.binary-equivalences
open import foundation.cartesian-product-types
open import foundation.dependent-pair-types
open import foundation.embeddings
open import foundation.equivalences
open import foundation.function-extensionality
open import foundation.function-types
open import foundation.homotopies
open import foundation.identity-types
open import foundation.injective-maps
open import foundation.involutions
open import foundation.propositions
open import foundation.sets
open import foundation.subtypes
open import foundation.universe-levels

open import group-theory.monoids
open import group-theory.products-of-elements-monoids
open import group-theory.semigroups

open import lists.concatenation-lists
open import lists.lists

open import structured-types.pointed-types
open import structured-types.pointed-types-equipped-with-automorphisms
```

</details>

## Idea

An **abstract group** is a group in the usual algebraic sense, i.e., it consists
of a set equipped with a unit element `e`, a binary operation `x, y ↦ xy`, and
an inverse operation `x ↦ x⁻¹` satisfying the group laws

```text
  (xy)z = x(yz)      (associativity)
     ex = x          (left unit law)
     xe = x          (right unit law)
   x⁻¹x = e          (left inverse law)
   xx⁻¹ = e          (right inverse law)
```

## Definition

### The condition that a semigroup is a group

```agda
is-group' :
  {l : Level} (G : Semigroup l) → is-unital-Semigroup G → UU l
is-group' G is-unital-Semigroup-G =
  Σ ( type-Semigroup G → type-Semigroup G)
    ( λ i →
      ( (x : type-Semigroup G) →
        Id (mul-Semigroup G (i x) x) (pr1 is-unital-Semigroup-G)) ×
      ( (x : type-Semigroup G) →
        Id (mul-Semigroup G x (i x)) (pr1 is-unital-Semigroup-G)))

is-group :
  {l : Level} (G : Semigroup l) → UU l
is-group G =
  Σ (is-unital-Semigroup G) (is-group' G)
```

### The type of groups

```agda
Group :
  (l : Level) → UU (lsuc l)
Group l = Σ (Semigroup l) is-group

module _
  {l : Level} (G : Group l)
  where

  semigroup-Group : Semigroup l
  semigroup-Group = pr1 G

  set-Group : Set l
  set-Group = pr1 semigroup-Group

  type-Group : UU l
  type-Group = pr1 set-Group

  is-set-type-Group : is-set type-Group
  is-set-type-Group = pr2 set-Group

  has-associative-mul-Group : has-associative-mul type-Group
  has-associative-mul-Group = pr2 semigroup-Group

  mul-Group : type-Group → type-Group → type-Group
  mul-Group = pr1 has-associative-mul-Group

  ap-mul-Group :
    {x x' y y' : type-Group} (p : Id x x') (q : Id y y') →
    Id (mul-Group x y) (mul-Group x' y')
  ap-mul-Group p q = ap-binary mul-Group p q

  mul-Group' : type-Group → type-Group → type-Group
  mul-Group' x y = mul-Group y x

  commute-Group : type-Group → type-Group → UU l
  commute-Group x y = mul-Group x y ＝ mul-Group y x

  associative-mul-Group :
    (x y z : type-Group) →
    Id (mul-Group (mul-Group x y) z) (mul-Group x (mul-Group y z))
  associative-mul-Group = pr2 has-associative-mul-Group

  is-group-Group : is-group semigroup-Group
  is-group-Group = pr2 G

  is-unital-Group : is-unital-Semigroup semigroup-Group
  is-unital-Group = pr1 is-group-Group

  monoid-Group : Monoid l
  pr1 monoid-Group = semigroup-Group
  pr2 monoid-Group = is-unital-Group

  unit-Group : type-Group
  unit-Group = pr1 is-unital-Group

  is-unit-Group : type-Group → UU l
  is-unit-Group x = Id x unit-Group

  is-prop-is-unit-Group : (x : type-Group) → is-prop (is-unit-Group x)
  is-prop-is-unit-Group x = is-set-type-Group x unit-Group

  is-unit-group-Prop : type-Group → Prop l
  pr1 (is-unit-group-Prop x) = is-unit-Group x
  pr2 (is-unit-group-Prop x) = is-prop-is-unit-Group x

  left-unit-law-mul-Group :
    (x : type-Group) → Id (mul-Group unit-Group x) x
  left-unit-law-mul-Group = pr1 (pr2 is-unital-Group)

  right-unit-law-mul-Group :
    (x : type-Group) → Id (mul-Group x unit-Group) x
  right-unit-law-mul-Group = pr2 (pr2 is-unital-Group)

  pointed-type-Group : Pointed-Type l
  pr1 pointed-type-Group = type-Group
  pr2 pointed-type-Group = unit-Group

  has-inverses-Group : is-group' semigroup-Group is-unital-Group
  has-inverses-Group = pr2 is-group-Group

  inv-Group : type-Group → type-Group
  inv-Group = pr1 has-inverses-Group

  left-inverse-law-mul-Group :
    (x : type-Group) → Id (mul-Group (inv-Group x) x) unit-Group
  left-inverse-law-mul-Group = pr1 (pr2 has-inverses-Group)

  right-inverse-law-mul-Group :
    (x : type-Group) → Id (mul-Group x (inv-Group x)) unit-Group
  right-inverse-law-mul-Group = pr2 (pr2 has-inverses-Group)

  inv-unit-Group :
    Id (inv-Group unit-Group) unit-Group
  inv-unit-Group =
    ( inv (left-unit-law-mul-Group (inv-Group unit-Group))) ∙
      ( right-inverse-law-mul-Group unit-Group)
```

## Properties

### Multiplication by `x` from the left is an equivalence

```agda
module _
  {l : Level} (G : Group l)
  where

  left-div-Group : type-Group G → type-Group G → type-Group G
  left-div-Group x y = mul-Group G (inv-Group G x) y

  is-section-left-div-Group :
    (x : type-Group G) → (mul-Group G x ∘ left-div-Group x) ~ id
  is-section-left-div-Group x y =
    ( inv (associative-mul-Group G _ _ _)) ∙
    ( ( ap (mul-Group' G y) (right-inverse-law-mul-Group G x)) ∙
      ( left-unit-law-mul-Group G y))

  is-retraction-left-div-Group :
    (x : type-Group G) → (left-div-Group x ∘ mul-Group G x) ~ id
  is-retraction-left-div-Group x y =
    ( inv (associative-mul-Group G _ _ _)) ∙
    ( ( ap (mul-Group' G y) (left-inverse-law-mul-Group G x)) ∙
      ( left-unit-law-mul-Group G y))

  is-equiv-mul-Group : (x : type-Group G) → is-equiv (mul-Group G x)
  is-equiv-mul-Group x =
    is-equiv-has-inverse
      ( left-div-Group x)
      ( is-section-left-div-Group x)
      ( is-retraction-left-div-Group x)

  equiv-mul-Group : (x : type-Group G) → type-Group G ≃ type-Group G
  pr1 (equiv-mul-Group x) = mul-Group G x
  pr2 (equiv-mul-Group x) = is-equiv-mul-Group x

  is-equiv-left-div-Group : (x : type-Group G) → is-equiv (left-div-Group x)
  is-equiv-left-div-Group x =
    is-equiv-has-inverse
      ( mul-Group G x)
      ( is-retraction-left-div-Group x)
      ( is-section-left-div-Group x)

  equiv-left-div-Group : (x : type-Group G) → type-Group G ≃ type-Group G
  pr1 (equiv-left-div-Group x) = left-div-Group x
  pr2 (equiv-left-div-Group x) = is-equiv-left-div-Group x
```

### Multiplication by `x` from the right is an equivalence

```agda
  right-div-Group : type-Group G → type-Group G → type-Group G
  right-div-Group x y = mul-Group G x (inv-Group G y)

  is-section-right-div-Group :
    (x : type-Group G) → (mul-Group' G x ∘ (λ y → right-div-Group y x)) ~ id
  is-section-right-div-Group x y =
    ( associative-mul-Group G _ _ _) ∙
    ( ( ap (mul-Group G y) (left-inverse-law-mul-Group G x)) ∙
      ( right-unit-law-mul-Group G y))

  is-retraction-right-div-Group :
    (x : type-Group G) → ((λ y → right-div-Group y x) ∘ mul-Group' G x) ~ id
  is-retraction-right-div-Group x y =
    ( associative-mul-Group G _ _ _) ∙
    ( ( ap (mul-Group G y) (right-inverse-law-mul-Group G x)) ∙
      ( right-unit-law-mul-Group G y))

  is-equiv-mul-Group' : (x : type-Group G) → is-equiv (mul-Group' G x)
  is-equiv-mul-Group' x =
    is-equiv-has-inverse
      ( λ y → right-div-Group y x)
      ( is-section-right-div-Group x)
      ( is-retraction-right-div-Group x)

  equiv-mul-Group' : (x : type-Group G) → type-Group G ≃ type-Group G
  pr1 (equiv-mul-Group' x) = mul-Group' G x
  pr2 (equiv-mul-Group' x) = is-equiv-mul-Group' x

  is-equiv-right-div-Group :
    (x : type-Group G) → is-equiv (λ y → right-div-Group y x)
  is-equiv-right-div-Group x =
    is-equiv-has-inverse
      ( mul-Group' G x)
      ( is-retraction-right-div-Group x)
      ( is-section-right-div-Group x)

  equiv-right-div-Group :
    (x : type-Group G) → type-Group G ≃ type-Group G
  pr1 (equiv-right-div-Group x) y = right-div-Group y x
  pr2 (equiv-right-div-Group x) = is-equiv-right-div-Group x
```

### Multiplication is a binary equivalence and a binary embedding

```agda
  is-binary-equiv-mul-Group : is-binary-equiv (mul-Group G)
  pr1 is-binary-equiv-mul-Group = is-equiv-mul-Group'
  pr2 is-binary-equiv-mul-Group = is-equiv-mul-Group

  is-binary-emb-mul-Group : is-binary-emb (mul-Group G)
  is-binary-emb-mul-Group =
    is-binary-emb-is-binary-equiv is-binary-equiv-mul-Group

  is-emb-mul-Group : (x : type-Group G) → is-emb (mul-Group G x)
  is-emb-mul-Group x = is-emb-is-equiv (is-equiv-mul-Group x)

  is-emb-mul-Group' : (x : type-Group G) → is-emb (mul-Group' G x)
  is-emb-mul-Group' x = is-emb-is-equiv (is-equiv-mul-Group' x)

  is-injective-mul-Group : (x : type-Group G) → is-injective (mul-Group G x)
  is-injective-mul-Group x = is-injective-is-equiv (is-equiv-mul-Group x)

  is-injective-mul-Group' : (x : type-Group G) → is-injective (mul-Group' G x)
  is-injective-mul-Group' x = is-injective-is-equiv (is-equiv-mul-Group' x)
```

### Transposition laws for equalities in groups

```agda
  transpose-eq-mul-Group :
    {x y z : type-Group G} →
    (mul-Group G x y ＝ z) → (x ＝ mul-Group G z (inv-Group G y))
  transpose-eq-mul-Group {x} {y} {z} refl =
    inv (is-retraction-right-div-Group y x)

  inv-transpose-eq-mul-Group :
    {x y z : type-Group G} →
    (x ＝ mul-Group G z (inv-Group G y)) → (mul-Group G x y ＝ z)
  inv-transpose-eq-mul-Group {._} {y} {z} refl =
    is-section-right-div-Group y z

  transpose-eq-mul-Group' :
    {x y z : type-Group G} →
    Id (mul-Group G x y) z → Id y (mul-Group G (inv-Group G x) z)
  transpose-eq-mul-Group' {x} {y} {z} refl =
    inv (is-retraction-left-div-Group x y)

  inv-transpose-eq-mul-Group' :
    {x y z : type-Group G} →
    Id y (mul-Group G (inv-Group G x) z) → (mul-Group G x y ＝ z)
  inv-transpose-eq-mul-Group' {x} {y} {._} refl =
    is-section-left-div-Group x _
```

### Distributivity of inverses over multiplication

```agda
  distributive-inv-mul-Group :
    (x y : type-Group G) →
    Id
      ( inv-Group G (mul-Group G x y))
      ( mul-Group G (inv-Group G y) (inv-Group G x))
  distributive-inv-mul-Group x y =
    transpose-eq-mul-Group
      ( ( transpose-eq-mul-Group
          ( ( associative-mul-Group G (inv-Group G (mul-Group G x y)) x y) ∙
            ( left-inverse-law-mul-Group G (mul-Group G x y)))) ∙
        ( left-unit-law-mul-Group G (inv-Group G y)))
```

### Inverting elements of a group is an involution

```agda
  inv-inv-Group :
    (x : type-Group G) → Id (inv-Group G (inv-Group G x)) x
  inv-inv-Group x =
    is-injective-mul-Group
      ( inv-Group G x)
      ( ( right-inverse-law-mul-Group G (inv-Group G x)) ∙
        ( inv (left-inverse-law-mul-Group G x)))
```

### Inverting elements of a group is an equivalence

```agda
  is-equiv-inv-Group : is-equiv (inv-Group G)
  is-equiv-inv-Group = is-equiv-is-involution inv-inv-Group

  equiv-equiv-inv-Group : type-Group G ≃ type-Group G
  pr1 equiv-equiv-inv-Group = inv-Group G
  pr2 equiv-equiv-inv-Group = is-equiv-inv-Group
```

### Two elements `x` and `y` are equal iff `x⁻¹y=1`

```agda
  eq-is-unit-left-div-Group :
    {x y : type-Group G} → is-unit-Group G (left-div-Group x y) → x ＝ y
  eq-is-unit-left-div-Group {x} {y} H =
    ( inv (right-unit-law-mul-Group G x)) ∙
    ( inv-transpose-eq-mul-Group' (inv H))

  is-unit-left-div-eq-Group :
    {x y : type-Group G} → x ＝ y → is-unit-Group G (left-div-Group x y)
  is-unit-left-div-eq-Group refl = left-inverse-law-mul-Group G _
```

### Two elements `x` and `y` are equal iff `xy⁻¹=1`

```agda
  eq-is-unit-right-div-Group :
    {x y : type-Group G} → is-unit-Group G (right-div-Group x y) → x ＝ y
  eq-is-unit-right-div-Group H =
    inv (inv-transpose-eq-mul-Group (inv H)) ∙ left-unit-law-mul-Group G _

  is-unit-right-div-eq-Group :
    {x y : type-Group G} → x ＝ y → is-unit-Group G (right-div-Group x y)
  is-unit-right-div-eq-Group refl = right-inverse-law-mul-Group G _
```

### The inverse of `x⁻¹y` is `y⁻¹x`

```agda
  inv-left-div-Group :
    (x y : type-Group G) →
    inv-Group G (left-div-Group x y) ＝ left-div-Group y x
  inv-left-div-Group x y =
    equational-reasoning
      inv-Group G (left-div-Group x y)
      ＝ left-div-Group y (inv-Group G (inv-Group G x))
        by distributive-inv-mul-Group (inv-Group G x) y
      ＝ left-div-Group y x
        by ap (left-div-Group y) (inv-inv-Group x)
```

### The inverse of `xy⁻¹` is `yx⁻¹`

```agda
  inv-right-div-Group :
    (x y : type-Group G) →
    inv-Group G (right-div-Group x y) ＝ right-div-Group y x
  inv-right-div-Group x y =
    equational-reasoning
      inv-Group G (right-div-Group x y)
      ＝ right-div-Group (inv-Group G (inv-Group G y)) x
        by distributive-inv-mul-Group x (inv-Group G y)
      ＝ right-div-Group y x
        by ap (mul-Group' G (inv-Group G x)) (inv-inv-Group y)
```

### The multiple of `x⁻¹y` and `y⁻¹z` is `x⁻¹z`

```agda
  mul-left-div-Group :
    (x y z : type-Group G) →
    mul-Group G (left-div-Group x y) (left-div-Group y z) ＝ left-div-Group x z
  mul-left-div-Group x y z =
    equational-reasoning
      mul-Group G (left-div-Group x y) (left-div-Group y z)
      ＝ mul-Group G (inv-Group G x) (mul-Group G y (left-div-Group y z))
        by associative-mul-Group G (inv-Group G x) y (left-div-Group y z)
      ＝ left-div-Group x z
        by ap (left-div-Group x) (is-section-left-div-Group y z)
```

### The multiple of `xy⁻¹` and `yz⁻¹` is `xz⁻¹`

```agda
  mul-right-div-Group :
    (x y z : type-Group G) →
    mul-Group G (right-div-Group x y) (right-div-Group y z) ＝
    right-div-Group x z
  mul-right-div-Group x y z =
    equational-reasoning
      mul-Group G (right-div-Group x y) (right-div-Group y z)
      ＝ mul-Group G x (mul-Group G (inv-Group G y) (right-div-Group y z))
        by associative-mul-Group G x (inv-Group G y) (right-div-Group y z)
      ＝ right-div-Group x z
        by ap (mul-Group G x) (is-retraction-left-div-Group y (inv-Group G z))
```

### For any semigroup, being a group is a property

```agda
abstract
  all-elements-equal-is-group :
    {l : Level} (G : Semigroup l) (e : is-unital-Semigroup G) →
    all-elements-equal (is-group' G e)
  all-elements-equal-is-group
    ( pair G (pair μ associative-G))
    ( pair e (pair left-unit-G right-unit-G))
    ( pair i (pair left-inv-i right-inv-i))
    ( pair i' (pair left-inv-i' right-inv-i')) =
    eq-type-subtype
      ( λ i →
        prod-Prop
          ( Π-Prop (type-Set G) (λ x → Id-Prop G (μ (i x) x) e))
          ( Π-Prop (type-Set G) (λ x → Id-Prop G (μ x (i x)) e)))
      ( eq-htpy
        ( λ x → equational-reasoning
          i x
          ＝ μ e (i x)
            by inv (left-unit-G (i x))
          ＝ μ (μ (i' x) x) (i x)
            by ap (λ y → μ y (i x)) (inv (left-inv-i' x))
          ＝ μ (i' x) (μ x (i x))
            by associative-G (i' x) x (i x)
          ＝ μ (i' x) e
            by ap (μ (i' x)) (right-inv-i x)
          ＝ i' x
            by right-unit-G (i' x)))

abstract
  is-prop-is-group :
    {l : Level} (G : Semigroup l) → is-prop (is-group G)
  is-prop-is-group G =
    is-prop-Σ
      ( is-prop-is-unital-Semigroup G)
      ( λ e →
        is-prop-all-elements-equal (all-elements-equal-is-group G e))

is-group-Prop : {l : Level} (G : Semigroup l) → Prop l
pr1 (is-group-Prop G) = is-group G
pr2 (is-group-Prop G) = is-prop-is-group G
```

### Any idempotent element in a group is the unit

```agda
module _
  {l : Level} (G : Group l)
  where

  is-idempotent-Group : type-Group G → UU l
  is-idempotent-Group x = Id (mul-Group G x x) x

  is-unit-is-idempotent-Group :
    {x : type-Group G} → is-idempotent-Group x → is-unit-Group G x
  is-unit-is-idempotent-Group {x} p =
    is-injective-mul-Group G x (p ∙ inv (right-unit-law-mul-Group G x))
```

### Multiplication of a list of elements in a group

```agda
module _
  {l : Level} (G : Group l)
  where

  mul-list-Group : list (type-Group G) → type-Group G
  mul-list-Group = mul-list-Monoid (monoid-Group G)

  preserves-concat-mul-list-Group :
    (l1 l2 : list (type-Group G)) →
    Id
      ( mul-list-Group (concat-list l1 l2))
      ( mul-Group G (mul-list-Group l1) (mul-list-Group l2))
  preserves-concat-mul-list-Group =
    distributive-mul-concat-list-Monoid (monoid-Group G)
```

### Any group element yields a type equipped with an automorphism

```agda
module _
  {l : Level} (G : Group l) (g : type-Group G)
  where

  pointed-type-with-aut-Group : Pointed-Type-With-Aut l
  pr1 pointed-type-with-aut-Group = pointed-type-Group G
  pr2 pointed-type-with-aut-Group = equiv-mul-Group G g
```

### Equip a type with a structure of group

```agda
structure-group :
  {l1 : Level} → UU l1 → UU l1
structure-group X =
  Σ (structure-semigroup X) (λ p → is-group (compute-structure-semigroup X p))

compute-structure-group :
  {l1 : Level} → (X : UU l1) → structure-group X → Group l1
pr1 (compute-structure-group X (p , q)) = compute-structure-semigroup X p
pr2 (compute-structure-group X (p , q)) = q
```

repository: UniMath/agda-unimath

gha_language: Agda

repo_name: UniMath/agda-unimath


text:
# API Review 07/21/2022

## System.Diagnostics.CodeAnalysis.UnscopedRefAttribute

**Approved** | [#runtime/72074](https://github.com/dotnet/runtime/issues/72074#issuecomment-1191764092) | [Video](https://www.youtube.com/watch?v=E5djVCbZxo4&t=0h0m0s)

* We'd like to align the names between the synthesized attribute, the language keyword, and this new attribute
    - Could we go for `[ScopedRef]` and `[UnscopedRef]`?
* We don't believe we should have `AttributeTargets.Struct`
    - This should be rare, so consumers should be selective anyway

```C#
namespace System.Diagnostics.CodeAnalysis;

[AttributeUsage(AttributeTargets.Method |
                AttributeTargets.Property |
                AttributeTargets.Parameter,
                AllowMultiple = false,
                Inherited = false)]
public sealed class UnscopedRefAttribute : Attribute
{
    public UnscopedRefAttribute();
}
```

## Reconcile naming: RegexGeneratorAttribute to RegexAttribute

**Approved** | [#runtime/62123](https://github.com/dotnet/runtime/issues/62123#issuecomment-1191789133) | [Video](https://www.youtube.com/watch?v=E5djVCbZxo4&t=0h38m29s)

* We'd prefer `GeneratedRegexAttribute`

```diff
 namespace System.Text.RegularExpressions;

 [AttributeUsage(AttributeTargets.Method, AllowMultiple = false, Inherited = false)]
-public partial class RegexGeneratorAttribute : Attribute
+public partial class GeneratedRegexAttribute : Attribute
 {
     // ....
 }
```

## Introduce Commands and DataContext to WinForms to adapt modern Binding scenarios

**Approved** | [#winforms/4895](https://github.com/dotnet/winforms/issues/4895#issuecomment-1191833180) | [Video](https://www.youtube.com/watch?v=E5djVCbZxo4&t=1h6m58s)

* We'd prefer not having the classes `CommandComponent` and `CommandControl`
    - Instead just follow the WinForms model of having a naming convention
* We still need `BindableComponent` because `Component` is lower than WinForms (where `IBindableComponent` lives).
* We'd like the designer to mark the generated `InitializeComponent` with `[RequiresPreviewFeatures(Url = "<placeholder>")]` when these are used

```C#
namespace System.Windows.Forms;

[RequiresPreviewFeatures(Url = "<placeholder>")]
public abstract class BindableComponent : Component, IBindableComponent
{
    public event EventHandler? BindingContextChanged;
    public ControlBindingsCollection? DataBindings { get; }
    public BindingContext? BindingContext { get; set; }
    protected virtual void OnBindingContextChanged(EventArgs e);
}

// Suppress warning
public abstract class Control : BindableComponent // Used to be Component
{
}

public partial class ButtonBase
{
    [RequiresPreviewFeatures(Url = "<placeholder>")]
    public event EventHandler? CommandCanExecuteChanged;
    [RequiresPreviewFeatures(Url = "<placeholder>")]
    public event EventHandler? CommandChanged;
    [RequiresPreviewFeatures(Url = "<placeholder>")]
    public event EventHandler? CommandParameterChanged;

    [RequiresPreviewFeatures(Url = "<placeholder>")]
    public ICommand? Command { get; set; }
    [RequiresPreviewFeatures(Url = "<placeholder>")]
    public object? CommandParameter { get; set; }

    [RequiresPreviewFeatures(Url = "<placeholder>")]
    protected virtual void OnCommandChanged(EventArgs e);
    [RequiresPreviewFeatures(Url = "<placeholder>")]
    protected virtual void OnCommandCanExecuteChanged(EventArgs e);
    [RequiresPreviewFeatures(Url = "<placeholder>")]
    protected virtual void OnCommandParameterChanged(EventArgs e);
    [RequiresPreviewFeatures(Url = "<placeholder>")]
    protected virtual void OnRequestCommandExecute(EventArgs e);
}

public abstract partial class ToolStripItem : BindableComponent // Used to be Component
{
    [RequiresPreviewFeatures(Url = "<placeholder>")]
    public event EventHandler? CommandCanExecuteChanged;
    [RequiresPreviewFeatures(Url = "<placeholder>")]
    public event EventHandler? CommandChanged;
    [RequiresPreviewFeatures(Url = "<placeholder>")]
    public event EventHandler? CommandParameterChanged;

    [RequiresPreviewFeatures(Url = "<placeholder>")]
    public ICommand? Command { get; set; }
    [RequiresPreviewFeatures(Url = "<placeholder>")]
    public object? CommandParameter { get; set; }

    [RequiresPreviewFeatures(Url = "<placeholder>")]
    protected virtual void OnCommandChanged(EventArgs e);
    [RequiresPreviewFeatures(Url = "<placeholder>")]
    protected virtual void OnCommandCanExecuteChanged(EventArgs e);
    [RequiresPreviewFeatures(Url = "<placeholder>")]
    protected virtual void OnCommandParameterChanged(EventArgs e);
    [RequiresPreviewFeatures(Url = "<placeholder>")]
    protected virtual void OnRequestCommandExecute(EventArgs e);
}
```

license: MIT

repository: dotnet/apireviews

branch_name: refs/heads/main

extension: md

gha_license_id: MIT

license_type: permissive

repo_name: dotnet/apireviews


text:
---
layout: post
title: 'Double encoding: URI and HTML encoding'
date: 2013-03-04 16:30:18.000000000 -08:00
type: post
parent_id: '0'
published: true
password: ''
status: publish
categories:
- Code
tags: []
meta:
  _edit_last: '1'
  _syntaxhighlighter_encoded: '1'
  _wpas_done_all: '1'
  _oembed_6073140669e937827d5610641b40cdf4: "{{unknown}}"
  _su_title: ''
  _jetpack_related_posts_cache: a:1:{s:32:"8f6677c9d6b0f903e98ad32ec61f8deb";a:2:{s:7:"expires";i:1561033504;s:7:"payload";a:3:{i:0;a:1:{s:2:"id";i:4262;}i:1;a:1:{s:2:"id";i:3942;}i:2;a:1:{s:2:"id";i:4725;}}}}
  _oembed_dcc58c858c7066130a1a8a543b535881: "{{unknown}}"
  _oembed_da0fe90d451adcefeea5c7bb0f51ab34: "{{unknown}}"

permalink: "/2013/03/04/double-encoding-uri-and-html-encoding/"
---
URL's have specific characters that are special, like `%` and `&` that if you need to use as part of your GET URI then you need to [encode](http://www.w3schools.com/tags/ref_urlencode.asp) them. For example:

[html]
http://localhost?key=this & that&key2=value2
[/html]

It's obvious that this URL is invalid, `this & that` has both spaces and a special character `&`. In fact, you may have even noticed it's invalid in your browser:

[caption id="attachment\_3126" align="alignnone" width="432"] ![Screenshot of what you see above. Yes, I know it's a little meta](http://onoffswitch.net/wp-content/uploads/2013/03/invalidurl2.png) Screenshot of what you see above. Yes, I know it's a little meta[/caption]

To get around this you can URI encode your URL's which will convert special characters to ASCII:

[html]
http://localhost?key=this%20%26%20that&key2=value2
[/html]

But there is also [HTML encoding](http://en.wikipedia.org/wiki/Character_encodings_in_HTML), which means escaping special HTML characters when you are putting in text into html. For example:

[html]
\
6 \> 7\

[/html]

Doesn't work, since `>` is a special character. To get around this, you need to HTML encode your text. HTML encoding uses special characters to indicate escaping of text. HTML encoding the above example would make your text

[html]
\
6 > 7\

[/html]

But sometimes you want to put in some HTML dynamically into a page that also contains a URL. Here you need to double encode (encode the URI and the HTML). The ordering here matters. Let's take this HTML text as the source:

[html]
\
[/html]

And lets encode it first with HTML then with URI:

[html]
html encoded: \
uri encoded: \
[/html]

But what if we do it the other way around?

[html]
uri encoded: \
html encoded: \
[/html]

See the difference? Look at where the apostrophe would be

[code]
invalid: %26apos;
valid: '
[/code]

The first example will give you an invalid URL but the second example is the URL you want. URL and HTML encodings aren't interchangable, they are used for specific scenarios and sometimes need to be used together (in the right order).

repository: devshorts/onoffswitch

branch_name: refs/heads/gh-pages

gha_language: HTML

repo_name: devshorts/onoffswitch


text:
# RxPermissions
![device-2015-10-22-005828](https://cloud.githubusercontent.com/assets/1311551/10652589/30502624-7858-11e5-8139-771378cea7e3.png)
An easy way to request Android M permission using RxJava

Requesting permissions now is very easy, you can do it with one line.
Here is how you do it.

# Step 1: Dependencies
To use the library you have to add the dependence in your app, do it in your `build.gradle` file.
`
```groovy
dependencies {
	compile 'com.devdoo.rxpermissions:lib:1.0.2'
}
```
# Step 2: Add the permission
Add the permission you want to request in `AndroidManifest.xml`
Like in this example:
```xml
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.devdoo.sample" >
    <uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />
    <uses-permission android:name="android.permission.CAMERA" />
    <uses-permission android:name="android.permission.READ_CONTACTS" />
    <uses-permission android:name="android.permission.READ_CALENDAR" />

    <application
        android:allowBackup="true"
        android:icon="@mipmap/ic_launcher"
        android:label="@string/app_name"
        android:supportsRtl="true"
        android:theme="@style/AppTheme" >
        <activity
            android:name=".MainActivity"
            android:label="@string/app_name"
<EMAIL_ADDRESS>>
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />

                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
    </application>

</manifest>
```
# Step 4: Requesting permissions

`request` to request one or more permission and receive only one return callback.

```java
	public void readContacts(View v) {
		RxPermission.with(getFragmentManager()).request(permission.READ_CONTACTS)
				.subscribe(isGranted -> showResult(v, isGranted));
	}

	public void readCalendar(View v) {
		RxPermission.with(getFragmentManager()).request(permission.READ_CALENDAR)
				.subscribe(isGranted -> showResult(v, isGranted));
	}
```
`request` allows to receive a callback for the global state: if all permission are granted
the boolean value isGranted is true, if one permission is not granted isGranted is false.
NB: Only one result is received.

```java
	public void requestAll(View v) {
		RxPermission.with(getFragmentManager())
				.request(permission.CAMERA,
						permission.ACCESS_FINE_LOCATION,
						permission.READ_CONTACTS,
						permission.READ_CALENDAR)
				.subscribe(isGranted -> showResult(v, isGranted));
	}
```

`requestEach` allows to get the result of the request for each permission.
In this case you will receive 4 individual result.

```java
	public void requestEach(View v) {
		RxPermission.with(getFragmentManager())
				.requestEach(permission.CAMERA,
						permission.ACCESS_FINE_LOCATION,
						permission.READ_CONTACTS,
						permission.READ_CALENDAR)
				.subscribe(this::showResult);
	}
```

# Credits
This library was inspired by https://github.com/tbruyelle/RxPermissions, but it uses a bit different
design that uses headless Fragment to request Android M run-time permissions

repository: marius-m/RxPermissions

branch_name: refs/heads/master

repo_name: marius-m/RxPermissions


text:
# CMPDRP

CMPDRP is a simple way to view website design comps in a desktop browser.  Images are aligned to the top of the window and horizontally centered (horizontal scrolling is disabled) because that's how I need it right now.

Note: You are viewing files on _your own machine_. Files are _NOT_ uploaded to any server. No one else can see what you are seeing (unless they are standing behind you).

[http://cmpdrp.com/](http://cmpdrp.com/)

## TO DO

Things that could be better or added...

* Show a message if not compatible file format

* Drag a directory of images into window? (not sure if this is possible for JS security reasons)

* Separate key input to navigate index thumbs/list and select (sort of have this...)

* Show read/load progress

* Drag thumbs in index overlay to re-arrange

* Image detail display? (file name, dimensions, file size, etc.)

* Handle duplicate files better (jumpToFile will always go to first match if duplicate file names)

* Scale thumbnails up/down

* Better file name tooltip on thumb

## ABOUT

This is a simple tool, built quickly to solve a specific use case I encountered. It's not meant to solve all problems for everyone, but I am happy to share it as a reference or starting point for similar applications.

Lucas J. Shuman
[http://lucasishuman.com](http://lucasishuman.com)

repository: seanahern/cmpdrp

gha_language: JavaScript

repo_name: seanahern/cmpdrp


text:
# Finance Bundle

*By [endroid](https://endroid.nl/)*

[![Latest Stable Version](http://img.shields.io/packagist/v/endroid/finance-bundle.svg)](https://packagist.org/packages/endroid/finance-bundle)
[![Build Status](https://github.com/endroid/finance-bundle/workflows/CI/badge.svg)](https://github.com/endroid/finance-bundle/actions)
[![Total Downloads](http://img.shields.io/packagist/dt/endroid/finance-bundle.svg)](https://packagist.org/packages/endroid/finance-bundle)
[![Monthly Downloads](http://img.shields.io/packagist/dm/endroid/finance-bundle.svg)](https://packagist.org/packages/endroid/finance-bundle)
[![License](http://img.shields.io/packagist/l/endroid/finance-bundle.svg)](https://packagist.org/packages/endroid/finance-bundle)

## Installation

Use [Composer](https://getcomposer.org/) to install the library. Symfony Flex
will set up the configuration and routing for you.

``` bash
$ composer require endroid/finance-bundle
```

## Versioning

Version numbers follow the MAJOR.MINOR.PATCH scheme. Backwards compatibility
breaking changes will be kept to a minimum but be aware that these can occur.
Lock your dependencies for production and test your code when upgrading.

## License

This source code is subject to the MIT license bundled in the file LICENSE.

repository: abandroid/finance-bundle

repo_name: abandroid/finance-bundle


text:
# Pre-work - *Myblog*

**Myblog** is a Ruby on Rails blog application.

Submitted by: **Minh-Thien Le**

Time spent: **30** hours spent in total

URL: **https://dry-retreat-4413.herokuapp.com/**

## User Stories

The following **required** functionality is complete:

* [x] User can create a new post, formatted using the Markdown language.
* [x] User can edit an existing post.
* [x] There is one post that introduces the App Creator with name, picture.

The following **optional** features are implemented:
* [x] User can delete stories, with an alert that asks the user to confirm.
* [x] User can enter a search term and see all posts with titles that contain the search term.
* [ ] User can add "tags" to a post, and filter posts by tag.
* [x] There is a "navbar" that is responsive to window size similar to http://v4-alpha.getbootstrap.com/examples/navbar/. **NOTE**: This is currently buggy in Bootstrap v4 Alpha. For a workaround, look at the CodePen associater dhere: https://github.com/twbs/bootstrap/issues/18263.
* [ ] User can see how many views a post has.
* [x] User can leave a comment on a post.

The following **additional** features are implemented:

- [x] Basic validation for required fields (e.g: blog title)
- [x] Display timestamp for each blog entry or comment (last updated field)
- [ ] Pagination
- [ ] Login page, user authentication.

To be fixed:
- [ ] Navigation bar: when item selected, it must be highlighted.
- [ ] Navigation bar: add dropdown in navbar to group the Article related items: "List articles", "New article"

## Video Walkthrough

Here's a walkthrough of implemented user stories:

![Video Walkthrough](https://github.com/thienleminh1982/myblog/blob/master/myblog_walkthrough.gif)

GIF created with [LiceCap](http://www.cockos.com/licecap/).

## Notes

- Must install the postgresql locally properly, also edit the "database.yml" with development and test entry.
  Test the app to work locally first.
  => Lesson-learned: follow the principle to make the dev and prod environment as similar as possible.

- Heroku does not support sqlite3. Thefore, have to remove sqlite3 from "gemfile"
    https://devcenter.heroku.com/articles/sqlite3
    http://stackoverflow.com/questions/7963561/heroku-cannot-run-git-push-heroku-master
    http://stackoverflow.com/questions/10455527/sqlite3-h-missing-when-pushing-rails-app-to-heroku
- Gem 'spring' on "gemfile" causes crash when deployed the app to heroku:
    http://stackoverflow.com/questions/33793812/error-when-loading-heroku-app
- For Rails beginners, have to read a lot on the documentation to really understand what is done behind the scene.

## License

    Copyright 2015 Minh-Thien Le

    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
    You may obtain a copy of the License at

        http://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License.


text:
---
layout: post
title:  "The open-approach to Operationalize Machine Learning"
date:   2019-08-02 09:01:33 +0000
subtitle: Constructing an open-source ecosystem to govern the complexity of Machine Learning services

tags: [opsml, ml ops, machine learning deployments, machine-learning, operations, machine learning platform, machine learning operationalization]
---
This blog post discusses the operationalization of machine learning systems. From an engineering perspective, these systems pose enormous challenges due to the high variability of requirements that different use cases have. As the machine-learning maturity of organizations evolves, the complexity required to manage these systems grows exponentially; The proliferation of ML frameworks makes it hard to abstract layers of common functionality and, consequently, automating the whole process becomes even harder.

As regulators move towards giving consumers the "right to explainability" and the "right to model retraining", compliance requirements bring a new dimension of complexity to the processes.

Ideally, organizations should have standardized processes by which they can deliver ML services. This requires an ML ecosystem/platform that considers the following (nonexhaustive) properties:

* **Scalability** - Working with samples and small datasets is sometimes enough in the initial research phase. However, teams eventually require to have access to distributed data processing and model training. Furthermore, we might need to data pipelines and models at scale. The underlying platform-infrastructure is responsible to provide this capability.

* **Framework Agnostic** - Teams should have the flexibility to use any required tool; this is particularly relevant as frameworks become more specialized. This also minimizes code-rewrites between research and operations which is highly error-prone.

* **Modular** - The platform should provide primitive components that can be used by the engineering teams to deliver ML on a standardized way. We must identify commonalities for modularization.

*  **Governance** - From a technical point of view, we need enough evidence to prove that the delivery process produces artifacts that are reproducible, traceable, and verifiable.
Further, there are complex ethical, legal, conduct and reputational issues associated with the use of personal
data. For example, are data being used unfairly to exclude individuals or groups, or to promote unjustifiably
privileged access for others? Controls are needed to ensure data is treated fairly.

We are going to propose a technology stack for an ML-platform solution that can describe workflows respecting these desired properties. The following diagram illustrates the different components/primitives involved, which we are going to discuss:

![platform]({{ "/img/ops-ml/platform-components-2.png" | absolute_url}})

## ML Primitives

Lets briefly describe the responsibilities of each modular component. We'll also look at the current technology landscape and suggest tools that can solve the challenges posed by each layer.

### Model Training

Training machine learning models can be a highly time-consuming task. It can also be demanding from a resource allocation perspective, since bigger datasets and sophisticated models might require considerable amounts of processing power, which sometimes needs to come in specialized form, e.g. GPUs, TPU, etc.

As *Auto Machine Learning* technology improves, the process by which we select a  model, along with its tunned hyperparameters, is becoming more standardized. This wide search space is being explored by various methods like Bayesian Optimization or Genetic Programming. However, these algorithms can take a long time to finish their search.

Most of the times they aren't as simple as fitting one model on the dataset; they are considering multiple machine learning algorithms (random forests, linear models, SVMs, etc.) in a pipeline with multiple preprocessing steps (missing value imputation, scaling, PCA, feature selection, etc.), the hyperparameters for all of the models and preprocessing steps, as well as multiple ways to ensemble or stack the algorithms within the pipeline.

As an example consider a Genetic Programming library to evolve an ML pipeline that considers various preprocessing steps and multiple machine learning algorithms; Assume a configuration of `100 generations` with `100 population size`; This would result in 10,000 model configurations to evaluate with 10-fold cross-validation, which means that roughly 100,000 models are fit and evaluated on the training data. This is a time-consuming procedure, even for simpler models like decision trees.

We need a tool to allows us to orchestrate a machine learning training pipeline at scale. [Kubeflow Pipelines](https://www.kubeflow.org/docs/pipelines/overview/pipelines-overview/) is such a tool, belonging to Google's [Kubeflow](https://www.kubeflow.org/) ML ecosystem, a project dedicated to making deployments of machine learning (ML) workflows on Kubernetes.

Kubeflow Pipelines is Kubernetes-native workflow orchestrator that builds on top of [Argo Workflows](https://argoproj.github.io/argo/); it isn't a typical orchestrator in the sense that it knows the structure of a typical ML workflow and threfore it can provide features that leverage on that knowledge. It has a built-in `Experiment` concept that can be used to run a workflow with a set of parameters so that we can then track the associated performance metrics. It also allows a workflow to output
different artifacts that are used to assess the model performance/behavior.

Here's an illustration of a workflow:

![platform]({{ "/img/ops-ml/kubeflow-1.png" | absolute_url}})

Every step runs in its own container, allowing us to: isolate software dependencies; optimize required resources for each step, and; easily parallelize parts of the workflow.

### Model Serving

Once a training process results in a satisfactory model, the later should be made available at a central repository to facilitate its integration into a service.

We need a framework agnostic tool to rapidly deploy pipelines for model inference at scale. Note that, in the most general case, the transformations that occur at *training time* are not necessarily the same that we can afford to perform at *inference time*. There is some differentiation between transformation *training* and *testing* data; this is because training sets are typically enhanced by processing the full dataset all at once to extract meaningful information, while at *inference time* the data must be processed row by row (assuming a real-time system), leading to some information loss when compared to the *training phase*.

Moreover, we need a tool that provides us the flexibility to create inference pipelines that go beyond the traditional pattern of receiving the data and calling the model's API. Here's an illustration of a possible inference pipeline:

![platform]({{ "/img/ops-ml/seldon-222.png" | absolute_url }})

In this example the service receives some input data; transforms it to extract the relevant features; enriches the data with a query to some database to provide extra context; and then asks a prediction from multiple models.

Being able to create arbitrarily complex inference graphs creates the opportunity to generalize common behavior like creating generic steps for outlier detection or performing multi-armed bandits tests.

[Seldon](https://github.com/SeldonIO/seldon-core) is a model serving framework that supports model deployment on a Kubernetes cluster. It allows for the creation of inference graphs, where each step runs as a different container.

Every graph has a model orchestration pod at the top with a standardized API contract for both REST and gRPC. When it receives a request (or a batch of requests) it then triggers the execution of the graph. The of the wiring between the steps is handled by Seldon.

We can also leverage Kubernetes inherent scalability capability to serve models at scale. Here's the relevant part of a Seldon's graph definition:

```yaml
hpaSpec:
  minReplicas: 1
  maxReplicas: 4
    metrics:
      - type: Resource
        resource:
          name: cpu
          targetAverageUtilization: 0.80
```

This, in turn, will create a Horizontal Pod Autoscaler (HPA) rule in Kubernetes that monitors the service metrics. Once the metric threshold fails to be respected, a service replica is created to cope with the increased volume of traffic.

```bash
❯ kubectl get hpa --all-namespaces

NAMESPACE      NAME                          REFERENCE                                TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
jx-staging     manticore-manticore-a0b1623   Deployment/manticore-manticore-a0b1623   39%/80%   1         4         1          28d
```

To expose the models to consumers outside of the cluster Seldon integrates with [istio](https://istio.io/). Every time that the Seldon Kubernetes Operator detects a request for a Seldon deployment, it injects the necessary instructions to wire the service to `istio`. `L7 Routing` capability is also available via the integration with `istio`.

Each deployment exposes two default endpoints: `predict` and `feedback`:

![platform]({{ "/img/ops-ml/seldon-31.png" | absolute_url }})

The `predict` is the one used to to retrieve an inference. We can use the `feedback` endpoint to capture responses when these are made available. The later is important for:

* Deciding which model should serve more traffic when running inference graphs with A/B or multi-armed bandits, given the current performance;
* Models supporting *online-training*;
* Capturing new labeled data to be used in retraining jobs;

### Model Observability

Once a model is live, we do need to be able to monitor technical metrics like the number of requests/sec or the response latency.

However, data science teams should have to observe more functional metrics like the real-time performance of the model, the bias in the response or the distribution of the input data. This would provide useful insights to the teams to detect possible drifts in data distribution, or on how to make the model fairer.

Seldon provides an awesome integration with [Grafana](https://grafana.com/dashboards) and [Prometheus](https://prometheus.io/). For each Seldon Deployment, the Seldon's Service Orchestrator exposes a set of default Prometheus metrics which are then displayed in a central Grafana dashboard.

Here's an illutratrion of what this looks like:

![platform]({{ "/img/ops-ml/dashboard-1.png" | absolute_url }})

Custom metrics can be easily exposed and integrated in these dashboards. Seldon takes care of all of the wiring and communication among the services. At the dashboard level, we can select different deployments, versions or even particular containers that form an inference graph.

### Governance

A model's behavior is affected not only by its code and configuration but also by data consumed at training time. The following is a nonexhaustive list of what is needed to achieve governance when deploying ML services:

* Every Machine Learning Asset should be versioned. This includes code, data, configuration, models, inference graphs, etc.
* There should be visibility over the different experiments performed and associated metrics;
* Every model should be reproducible;
* We should be able to verify the results of a Machine Learning model to ensure we are only integrating models with acceptable performance;
* Rollback mechanisms should be in place;

An ML model is not much different from data: a set of learned weights along with some metadata. Any artifact storage solution should suffice to be used as a model repository. Nexus or an object-storage like MongoDB can be used as a model repository.

Experiment tracking can be done at the orchestrator level, e.g. Kubeflow Pipelines. If a more specialized service is needed, a [ML Flow Tracking](https://www.mlflow.org/docs/latest/tracking.html) service can be used. The later is particularly relevant if we also need to track experiments made by data scientists at *exploration phase*, i.e. when they are exploring the data and testing out ideas. MLFlow provides a simple `python`/`R` library that can be integrated into a data scientist exploration environment to track performance metrics and parameters used.

e.g. (from `python`):

```python
import mlflow

...

with mlflow.start_run():

    lr = ElasticNet(alpha=alpha, l1_ratio=l1_ratio, random_state=42)
    lr.fit(train_x, train_y)

    predicted_qualities = lr.predict(test_x)

    (rmse, mae, r2) = eval_metrics(test_y, predicted_qualities)

    # Print out ElasticNet model metrics
    print("Elasticnet model (alpha=%f, l1_ratio=%f):" % (alpha, l1_ratio))
    print("  RMSE: %s" % rmse)
    print("  MAE: %s" % mae)
    print("  R2: %s" % r2)

    # Set tracking_URI first and then reset it back to not specifying port
    # Note, we had specified this in an earlier cell
    mlflow.set_tracking_uri(mlflow_tracking_URI)

    # Log mlflow attributes for mlflow UI
    mlflow.log_param("alpha", alpha)
    mlflow.log_param("l1_ratio", l1_ratio)
    mlflow.log_metric("rmse", rmse)
    mlflow.log_metric("r2", r2)
    mlflow.log_metric("mae", mae)
    mlflow.sklearn.log_model(lr, "model")
```

With this service we can centralize the information related to every ML experiment:

* Experiments performed by the CICD infrastructure as a response to a data/code change;
* Experiments performed by data scientists while they are exploring new solutions;

The `MLFlow` artifact store can also be used as a model repository.

Treating a data change in the same way as a code change requires tooling to support some kind of formal *data versioning*.
[Pachyderm](https://www.pachyderm.com/) is a Kubernetes framework that can help with the engineering side of ML workflows. It can be used as kind of *git for data*.

This also means that our training workflows cannot explicitly consume the data when an execution is triggered. The consumption must be declarative and done via `git`.
By doing so, the CICD pipeline can respond in the same way to changes in both code and data.

Pachyderm provides an intuitive solution for data versioning. The following snippets illustrate the concept and the typical workflow when working with Pachyderm (via the `patchctl` CLI):

Create a data repository:

```bash
pachctl create repo product-recommendation
```

Commit data to master branch:

```bash
pachctl put file -r product-recommendation@master:/data -f data/
```

We can then list all of the different data versions of the `master` branch:

```bash
❯ pachctl list commit product-recommendation

REPO                   BRANCH COMMIT                           PARENT STARTED    DURATION           SIZE
product-recommendation master 70c9c944ed5f4d7286d5f16461d07c91 <none> 3 days ago Less than a second 52.4MiB
```

Note that `master` is just a pointer to the latest commit hash. To illustrate this consider the following example:

```bash
❯ pachctl list commit product-recommendation
REPO                   BRANCH COMMIT                           PARENT STARTED      DURATION           SIZE
product-recommendation master e37161e974e8482bbccc89e01b414f7c <none> 26 hours ago Less than a second 3.768KiB
product-recommendation master c3f0233361fd43f39f532ec6859001c8 <none> 26 hours ago 1 second           3.768KiB
```

The folllowing commands are equivalent:

```bash
pachctl get file product-recommendation@master:/data/ --recursive --output .
pachctl get file product-recommendation@e37161e974e8482bbccc89e01b414f7c:/data/ --recursive --output .
```

Every change must have associated a particular version of the code and data. This is why the data cannot be consumed directly and must be done via `git`.

A simple way would be to have the data version as a configuration in the repository:

`/configuration/data.json`

```json
{
  "train": {
      "commit": "befcd45a69704b3094f10a031ffaf398"
  }
}
```

The CICD pipeline would be triggered in two ways:

* 1 - Code Change

The team changes the logic of the model training process, e.g. new features, more complex models, etc.

* 2 - Data Changes

Every time data changes, a pipeline must be triggered that will version it in Pachyderm and commit a change to the relevant repositories by changing `train.commit` in `/configuration/data.json`; this, in turn, will trigger an execution of the training process.

### Data Exploration

Teams require the ability to create data exploration environments, from which they can access the data to research possible solutions to the problems they are working on.
Special purpose hardware like GPUs should also be made available on-demand when needed.

Different flavors of these environments, with different technological stacks, can provide development velocity and flexibility to different teams.

[JupyterHub](https://jupyterhub.readthedocs.io/) or [Anaconda](https://www.anaconda.com/) are well-known tools that provide exploratory user-sessions on top of Kubernetes.

### CICD

[Jenkins X](https://jenkins.io/projects/jenkins-x/) is Kubernetes native and can be used for integration and deployment. It creates a concise system where the integrations pipelines run inside of the same cluster. It supports environment promotion, with multiple deployment strategies, via [GitOps](https://www.weave.works/technologies/gitops/), which gives us the ability to perform declarative application deployments via Git.

## Workflow Diagram

The following diagram depicts the end-to-end workflow. Every step then is detailed below.

![platform]({{ "/img/ops-ml/opsml-workflow-2.png" | absolute_url }})

Once a project is integrated into the platform, the integration pipeline is triggered either via a code (1) or a data change (2).

At building stage (3) we build every step of the Kubeflow ML training workflow as well as every step of the Seldon's Inference Graph; we should have unit tests for every individual component and more comprehensive tests for the model that will compare the model's predictions against a known set of cases (4); if the model does not perform successfully, the integration pipeline fails.

We then publish every build artifact to their respective repositories (5):
* Push all of the images belonging to the Kubeflow training pipeline and Seldon's Inference Graph to a registry.
* Publish a new version of a Helm Chart containing the new version of the service (Seldon Deployment Object).
* Push the new model version to the model repository;
* Deploy the new version of the training pipeline to the Kubeflow server.

The staging environment repository should then be updated with the new version of the application (6). The deployment is performed declaratively via the Jenkins X Kubernetes operator, which synchronizes the repository state with the cluster applicational state.

A new version of the service is then deployed (7), consisting of the following assets:

* Kubeflow workflow hat represents the training process and can be triggered manually with new inputs;
* A model inference graph that can serve requests (or batches of requests) via `REST` or `gRPC`;
* Horizontal Pod Autoscaler (HPA) rule to ensure the scalability of the service;
* A Grafana dashboard with default visualizations like the number of requests/sec, model latency or model performance (if signal available);
* An Istio VirtualService;

The model is exposed to be consumed outside of the cluster through a Load Balancer (8) (either L4 or L7).

After this the service is ready to be tested at the `staging` environment. It can then be easily promoted to the next environment once the tests at the `staging` environment are complete.

extension: markdown


text:
SublimeHaskell Editing Guide
============================

<!-- MarkdownTOC -->

- Commands
    - `stack exec `
- Snippets
- Key Bindings
    - Comments
- Customizing The Build System
    - cabal
    - hdevtools
    - Using build system results
- Using other useful projects with SublimeText
    - Jump-to-definition
    - Inside your project: hasktags

<!-- /MarkdownTOC -->

## Commands

These are the _SublimeHaskell_ commands accessible via the command pane (<ctrl>-<shift>-P on Windows and Linux, <cmd>-<shift>-P on Mac). Commands are prefixed by "SublimeHaskell:".

| Command                             | Descr        |
| ----------------------------------- | ------------ |
| ...: Apply to selection             |              |
| ...: Apply to selection list        |              |
| ...: Auto fix                       |              |
| ...: Browse declarations            |              |
| ...: Browse module                  |              |
| ...: Build                          |              |
| ...: Cabal: List                    |              |
| ...: Check                          | Send the buffer's contents to the attached backend and check syntax and type consistency. The error window reports all errors and warnings found. |
| ...: Check & Lint                   | Conducts a `...: Check` and, if the Haskell source has no errors, also passes the source through `hlint` to suggest potential improvements. |
| ...: Clean                          | Invokes the build system's "clean" command. |
| ...: Configure                      | For the `cabal` builder only: (re-)configures the project associated with the currently active buffer. |
| ...: Eval selection                 |              |
| ...: Expand selection to Expression |              |
| ...: Find declarations              |              |
| ...: Go to declaration              |              |
| ...: Go to module                   |              |
| ...: Go to next error               |              |
| ...: Go to previous error           |              |
| ...: Hayoo                          |              |
| ...: HIndent                        | Invokes the `hindent` indenter on the selected region, or the entire buffer if no selected region. |
| ...: Insert import for symbol       |              |
| ...: Insert type                    |              |
| ...: Install                        |              |
| ...: Lint                           |              |
| ...: Rebuild                        | Invoke the build system to `clean` followed by a `build`, e.g., `cabal clean` followed by `cabal build`. |
| ...: Reinspect all                  | Rescan the current windows, collect their respective `stack` or `cabal` projects, and scan projects' packages and source files for symbols. Used to seed and generate autocompletion lists. |
| ...: Run                            |              |
| ...: Scan docs and infer types      |              |
| ...: Search declarations everywhere |              |
| ...: Show error panel               |              |
| ...: Show symbol info               |              |
| ...: Show type                      |              |
| ...: Show types                     |              |
| ...: Show/hide all types            |              |
| ...: Stack Exec                     | Prompt for a command to execute as `stack exec <command>` and display its output in the `build` output window. |
| ...: Stylish                        | Invokes the `stylish-haskell` indenter on the selected region, or the entire buffer if no selected region. |
| ...: Test                           |              |
| ...: Toggle symbol info panel       |              |
| ...: Typecheck build (no codegen)   |              |

### `stack exec <command>`

`--` is inserted into the command list if `<command>` contains options (arguments that start with `-`) and you haven't specified `--` already. This comes in handy if you enter the following line into the `stack exec` command prompt:

```shell
-no-ghc-packages -- cabal --help
```

If you include the `--`, the `stack exec` command will leave the command list alone. However, if you simply type:

```shell
cabal --help
```

then `stack exec` inserts a `--` at the beginning of the command, e.g., `stack exec -- cabal --help`. If `stack exec` didn't, you would end up seeing the output of `stack exec`'s help because `stack exec` interprets options before `--` as its own.

## Snippets

Snippets are a way to quickly insert code by typing all or part of the snippet trigger and pressing `TAB`. For example, typing `data<TAB>` inserts the snippet for a plain (not GADT) data type:

```Haskell
data Type = Type  deriving (Eq, Show)
```

| Snippet trigger | What it inserts                                                                           |
| --------------- | ----------------------------------------------------------------------------------------- |
| `data`          | Data type and constructor, where the `Type` and the first constructor have the same name. |
| `impq`          | Double import: Two import lines, one unqualified and the other unqualified.               |
| `if`            | Inserts an `if` expression.                                                               |
| `inline`        | `{-# INLINE function #-}` pragma.                                                         |
| `lang`          | Inserts a GHC `{-# LANGUAGE pragma #-}` pragma.                                           |
| `newtype`       | Inserts a `newtype` declaration.                                                          |
| `opts`          | Inserts a GHC `{-# OPTIONS_GHC options #-}` option comment.                               |
| `qualified`     | Inserts a qualified import.                                                               |
| `record`        | Record data type and constructor.                                                         |

Double import snippet example:
```Haskell
import           Data.Text (Text, fun)
import qualified Data.Text as T
```

Record snippet example:
```Haskell
data Record = Record
  { field :: Type
  } deriving (Eq, Show)
```

## Key Bindings

### Comments

| Key            | Action                                 |
| -------------- | -------------------------------------- |
| `Ctrl-/`       | Inserts a Haskell line comment `-- `   |
| `Ctrl-Shift-/` | Inserts a Haskell block comment `{--}` |

## Customizing The Build System

You don't have to use SublimeHaskell's built-in build functionality. If you prefer, you can disable them in the settings, and use the plain Sublime Build Systems:

### cabal

Save this to your `~/.config/sublime-text-2/Packages/User/cabal-custom.sublime-build` to make a custom `cabal` build system:

```json
{
  "cmd": ["cabal build --ghc-options='-O0 -hidir o0 -odir o0'"],  // append lib:myPackage or myexecutable here to only build certain cabal targets
  "shell": true,
  "file_regex": "^(\\S*?):(\\d+):(\\d+):$"  // this matches the output of ghc
}
```

For more options, [look here](http://docs.sublimetext.info/en/latest/reference/build_systems.html).

### hdevtools

Save this to your `~/.config/sublime-text-2/Packages/User/hdevtools.sublime-build` to make `hdevtools` a build system:

```json
{
  "cmd": ["/home/USERNAME/.cabal/bin/hdevtools", "check", "-g", "-Wall", "$file"],
  "file_regex": "^(.*?):(\\d+):(\\d+):",
  "selector": "source.haskell"
}
```

### Using build system results

You can then build with `Ctrl-B` and jump between the errors with (Shift-)`F4`.

It is also useful to add this to your key bindings to redisplay the error panel at any time:

```json
  { "keys": ["ctrl+alt+b"], "command": "show_panel", "args": {"panel": "output.exec"} }
```

## Using other useful projects with SublimeText

### Jump-to-definition

There are two kinds of jump-to-definition: Inside your project and outside your project.
In any case, install the Sublime [`CTags`](https://github.com/SublimeText/CTags) package via Package Control,
and `cabal install hasktags`.

`CTags` expects the extended exuberant ctags format.

### Inside your project: hasktags

* In your project, `hasktags --ctags --extendedctag .`
* You can now jump to definitions inside your project (`Ctrl-T, Ctrl-T` is the default keybinding)

repository: SublimeHaskell/SublimeHaskell

gha_language: Python

repo_name: SublimeHaskell/SublimeHaskell


text:
# K-P-S-T
### Karplus-Strong synthesis using a loop as plectrum

https://youtu.be/PFloWFe5UR4?t=6m8s

The reverb used in the patch is reverbCH~ by Scott Nordlund
http://www.pdpatchrepo.info/hurleur/reverbCH.zip

repository: lazzarello/K-P-S-T

repo_name: lazzarello/K-P-S-T


text:
# Easy Docker
An easy way to maintain PHP version with some essential config like as virtual Host, Mysql, Redis and more. It's only for localhost :)

## Getting started

1. Clone the easy-docker github repository
```Bash
git clone https://github.com/rinkurock/easy-docker ~/easy-docker
```

2.  Copy .env.example to .env
```Bash
cp .env.example .env
```

4. Copy docker-compose.yml.example to docker-compose.yml
```Bash
cp docker-compose.yml.example docker-compose.yml
```

5. Edit  APPLICATION=/your-root-project-directory & database setting on .env file
```Bash
nano .env
```

6. Run docker-compose
```Bash
docker-compose up -d
```

7. Go to <http://localhost:9999> on browser

## Setup Virtual Host for porject
Run:

```Bash
sudo ./v-host.sh
```

and follow the instructions on command line

* sudo for only addning local domain address on /etc/hosts file on your system.

You can check it by run:
```Bash
cat /etc/hosts
```

Run:

```Bash
docker-compose ps
```
#### Accessing Globally
Sometimes you may want to `docker-compose` up your `easy-docker` machine from anywhere on your filesystem. You can do this on Unix systems by adding a Bash function to your Bash profile. These scripts will allow you to run any `docker-compose` command from anywhere on your system and will automatically point that command to your `easy-docker` installation:

```
function easydocker() {
    ( cd ~/easy-docker && docker-compose $* )
}
```

## Features
* Vertual Host with nginx
* PHP FPM
* PHP version 5.5.34, 5.6, 7.0, 7.1, 7.1.3, 7.2, 7.3.22, 7.4.13, 8.1.0 etc
* MySql 5.7, 8.0.19
* Postgres
* Adminer
* Redis
* Mongo
* Consul
* RabbitMq
* Elastic Search
* Kibana

## Contribution
All issues, PRs and advices are more than welcome to discuss about :).

## License
MIT

repository: rinkurock/easy-docker

gha_language: Dockerfile

gha_license_id: NOASSERTION

repo_name: rinkurock/easy-docker


text:
---
layout: post
comments: true
title: "Mesa e Cadeira with Rafael Rozendaal"
description: "So you think you can do it?"
category:
tags: []
---
{% include JB/setup %}

To create - individually or collectively - browser compositions that explore the arrangement of elements in time and space.

That's what [Rafael Rozendaal](http://www.newrafael.com), leader of the 8th edition of [Mesa & Cadeira](http://www.mesaecadeira.org/en/) workshop, challenged us to do.

What the fuck! I haven't felt that way since I first read [Macunaíma](http://en.wikipedia.org/wiki/Macuna%C3%ADma_%28novel%29).

But wait! That's the kind of thing that puts your brain to work.

Let's sketch!

My first one was like this.

    A snake game with many simultaneous canvases.

 So the "same" snake would look different depending on the frame you put it in.

   It's really important to make people feel safe, so the ideas keep flowing.

Next day two winning ideas were chosen by voting. That's when the [iwannabealone.com](http://www.iwannabealone.com) insanity started.

A website for people craving to stay out of the crowd.

### The inception

Exploring the idea of composition for different screen sizes, we decided that we would build a website where each user arrival would change website's appearance to reflect the number of watchers.

    First idea sketch. Starting on the top left with one user.

That's very raw. From that we jumped to something more visually interesting, like this

    Figuring out how it would looks like at the end.

But it is still just one way of splitting the screen into N parts. It looked **ok** by the moment. Soon we would discover it was **not**.

### Adding interaction

Then we decided to add some hot sauce. Following Rafael's [Finger Battle](http://www.fingerbattle.us) game, we thought that it would be nice if people fought to be alone, having to kick each others asses out.

That required the game to have some sort of click/tap interaction, what leads to a whole new number of problems to be solved.

### Problems (or we thought they were)

* how to split the screen into N, from 1 to 20? How does it looks like when we have 2? And 3? And 4, 5, 6… How does the transition looks like?

We were thinking that our last sketch was enough, but to split a screen into 5 pieces with equal area does not have a single solution. Some sketches later we arrived at the following stage

    Some of the possible strategies to split the screen.

The first is plain boring. Through it away!
Can you explain the second algorithmically?
The third one is just perfect. Like slicing a squared pizza! (actually it required some thought. You can see the [code](https://github.com/rcillo/iwannabealone). It's actually simple: base times height divided by two...

Rafael showed up in the middle of the discussion and throught this [Voronoi](http://en.wikipedia.org/wiki/Voronoi_diagram) stuff on us. Are you kiding?

* What's the maximum number of concurrent players?

It doesn't metter! We lost some precious time discussing it. Should we create rooms when more than 20 users enter at the same time? Should there be a waiting line? I had never seem more than 10 users logged in at the same time.

* What's the minimum area comfortable to tap on mobile devices?

Like the last question, this problem really exists. But it's a latent problem. It's going to be there sleeping like a beast until awaken by the noise of the crowd, which may **never** happen.

* Is it technologically viable?

Yes.

* Is network going to be a problem?

Probably. But after we built it, I've already played it on the aweful brazilian 3G.

### Fake problems

Some problems are just distractions. They seem very plausible. Few people would disagree they exist.

But should we **care** about then?

Simplicity. It's the hardest achieving ideal.

### A metaphor

I really cannot see the future.

Yeah… you may be thinking it's obvious, right?

So you are also thinking it's impossible to visualize the final product of an idea, obviously.

That's the point. An idea is like a mirage.  You are really thirst. You look at the horizon and under the hottest sun on earth your mind makes you see a oasis. Water is just over there. Then you start to get close and realize it's farther than you thought. So you walk desperately to get to it. When you get there. You find a fallen aircraft. That might not be water but it may still be useful, you know? So you can keep yourself alive.

Being humble about what to expect is vital when it comes to software products. There will probably be no oasis.

### The real problems

- What is this colored pizza?
- How does someone discovers it's interactive? (touch devices have no click cursor)
- Why it's moving? (recently arrived user trying to figure out what's going on)
- Let's click on the different slice written YOU! (killing himself)
- I've tapped till the other colors disappeared. So what? (winners cannot see that the opponent has being kicked out with a nice game over message).
- Why sometimes I start the game with less than 50% percent of the screen? (confusing game scoring logic)

Original Finger Battle game had two fingers illustrated on the splash screen. That says more than it looks like. The call for action button "play" too. Being able to see the other person tapping the blue and see the immediate feedback of loosing area also helps to understand the game dinamics.

Borrowing ideas is harder than it seems.

Context is really important for comprehension.

### Bugs

Should every bug be fixed? I really think the answer is **no**. Here's the list of fixed and [deliberately] unfixed ones.

- _fixed:_ My color changes when a new user comes to the game.
- _ignored:_ The 'YOU' word hides partially sometimes.
- _ignored:_ Sometimes 2 users with the same color share the screen making it looks like there is just one (very rare).
- _unfixed:_ After winning, the word 'You' does not change to 'You are alone.'. Would be nice to fix, but... read [this](http://en.wikipedia.org/wiki/Wabi-sabi).
- _easter egg:_ If I kill myself I do not loose the game and can stay there forever like if I had incarnated into another color.

### The final product

    You are alone screen.

    Game dynamics example with 4 players.

    That's it.

### Conclusion

Understanding the real problems is hard. Simplicity is the ability to answer to them first.

Being there already doesn't solve the problem, only make you more aware of it.

gha_language: CSS


text:
---
layout: post
title:  "使用Celery和Redis自动化Django任务队列"
categories: Django
tags:   Django  Celery  Redis
---

* content
{:toc}

## Automate the Django Task Queue with Celery and Redis
`原始链接，http://www.craigderington.me/configure-django-task-queue-with-celery-and-redis/`

Task Queue: a robust, asynchronous job handler and message broker enabling you to run and consume back-end functions based on system events rather than human intervention. A Django and Celery powered application can quickly respond to user requests while the task automation manager handles longer running requests through the use of a task queuing system.

>Celery uses a message broker to pass messages between your application and the Celery worker process. In this tutorial, I am going to use Redis as the message broker and Celery as the task manager. As Redis is a persistent, in-memory data structure server, it should not be used to manage application state; as the state may not be maintained during a reboot or other system type event requiring the server to be re-started.

## 开始前的准备
I am using a clean AWS instance running Ubuntu 14.04 in a free T2-micro tier for this demonstration. The first step is to add Celery to our Django application inside our virtualenv. Our project directory is located at /var/www/ and the Django project is called django_dev.

## 安装Redis
At our Ubuntu terminal, update the package manager.

```shell
$ sudo apt-get update
$ sudo apt-get install redis-server
```

Then check the Redis version to ensure the server is listening.

```shell
$ redis-server --version
Redis server v=2.8.4 sha=0000000:0...
```

Is Redis up?

```shell
$ redis-cli
> PING
PONG
```

Next, activate the virtual environment and install Celery.

```shell
/var/www/django_dev~$ source .env/bin/activate
(.env)/var/www/django_dev~$ pip install celery[redis]
(...)
Successfully installed pytz celery billiard kombu redis anyjson amqp
Cleaning up...
```

## 对Django应用添加Celery支持
We have to add several Celery specific settings to our Django applications settings.py. I prefer to keep my settings outside of the named inner project directory.

```
config/settings/common.py
```

```
# Celery
BROKER_URL = 'redis://localhost:6379/0'
CELERY_ACCEPT_CONTENT = ['json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
```

Next, create celery.py and save it in your config.settings directory. This will start Celery and create a Celery application.

```python
from __future__ import absolute_import

import os
from celery import Celery
from config.settings import common as settings

# set Django settings module for celery program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'config.settings')

app = Celery('hello_django')

# Using a string here means the worker will not have to
# pickle the object when using Windows.
app.config_from_object('config.settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)
```

To ensure that the Celery application starts everytime the Django app is started, we have to add an `__init.py__` to our project folder.

```python
from __future__ import absolute_import
from .celery import app as celery_app
```

### 创建第一个Celery任务
Add a new app to our Django project and ass some tasks. First, create the application.

```shell
(.env)/var/www/django_dev~$ django-admin startapp taskapp
```

Add the new app to our config settings common.py file.

```python
LOCAL_APPS = (
    ...
    'taskapp',
)
```

Now, in our taskapp app folder, add a new file tasks.py and add the following code to create the first Celery task.

```python
from __future__ import absolute_import
from celery import shared_task

@shared_task
def tasktest(param):
    return 'The task executed with argument "%s" ' % param
```

If you have followed this project setup, your projct directory will look like this...

```shell
/var/www/django_dev
|--config
|  | -- settings
|       | -- common.py
|       | -- local.py
|       | -- production.py
|       | -- dev.py
|  | -- wsgi.py
|  | -- celery.py
|  | -- urls.py
| -- manage.py
| -- taskapp
|    | -- __init__.py
|    | -- models.py
|    | -- tests.py
|    | -- views.py
|    | -- urls.py
|    | -- tasks.py
```

## 测试任务应用
In a real production environment, you would want your Celery task queue daemonized, however, for our demonstration, we will just call it from the command line and inspect the output.

```shell
$ export PYTHONPATH=/var/www/django_dev/config.settings:$PYTHONPATH
$ /var/www/django_dev/bin/celery --app=config.settings.celery:app worker --loglevel=INFO
-------------------------- celery@django v3.1.20

...

-- * - ********** [config]
-- * - **********.> app:         taskapp
-- * - **********.> transport:   redis://localhost:6379/0
-- * - **********.> results:     disabled
-- * - **********.> concurrency: 2 (prefork)
...
---- ************ [queues]
-----------------.> celery     exchange=celery(direct) key=celery

[tasks]
. taskapp.tasks.test

[2016-03-01 08:45:30, 849: INFO/MainProcess] Connected to redis://localhost:6379/0
[2016-03-01 08:45:30, 852: INFO/MainProcess] mingle:  searching forneighbors
[2016-03-01 08:45:31, 236: INFO/MainProcess] mingle:  all alone
[2016-03-01 08:45:31, 428: WARNING/MainProcess] celery@django ready
```

As long as the configuration settings are all OK, you will be greeted by a welcome screen like above. The [tasks] section will list all tasks discovered in all apps within your Django project directory.

## Queuing Up Tasks for Execution
In a separate terminal instance, start your project's virtual environment and launch a new task.

```shell
/var/www/django_dev~$ source .env/bin/activate
(.env)/var/www/django_dev~$ cd taskapp
(.env)/var/www/django_dev/taskapp~$ python manage.py shell
Python 2.7.6 (default June 22 2015 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" formore information
(InteractiveConsole)
>>> from taskapp.tasks import tasktest
>>> tasktest.delay('This is only a test.')
<AsyncResult:  67a43cf9-0b3f-4792-e746-2f3aa35e5b17
```

Now if you look at the Celery task terminal, you will notice task workers have started:

```shell
[2016-03-01 10:30:24, 624: [INFO/MainProcess] Received task taskapp.tasks.tasktest[67a43cf9-0b3f-4792-e746-2f3aa35e5b17]
[2016-03-01 10:30:24, 673: [INFO/MainProcess] Task taskapp.tasks.tasktest[67a43cf9-0b3f-4792-e746-2f3aa35e5b17]
```

## Celery Daemons & Supervisord
Most Django devs run Celery as a daemon using a nifty little tool called supervisord that allows workers to be restarted in the event of a system reboot. To install supervisor, enter this command the terminal prompt.

```shell
sudo apt-get install supervisor
```

As soon as supervisor is installed, you can add programs to it's configuration file so it knows to watch those processes for changes and update the terminal accordingly.

Supervisors config file is located at /etc/supervisor/conf.d. For our example, we will create a supervisor config file to watch our Celery processes.

```
[program:  taskapp-celery
command=/var/www/django_dev/bin/celery --app=taskapp.celery:app worker -- loglevel=INFO
directory=/var/www/django_dev/taskapp
user=tech-walker
stdout_logfile=/var/www/django_dev/logs/celery-worker.log
stderr_logfile=/var/www/django_dev/logs/celery-worker.log
autostart=true
autorestart=true
startsecs=10

; need to wait for any tasks still running
stopwaitsecs=900

; send SIGKILL to destroy all processes as a group
killasgroup=true

; if rabbitmq is supervised, set it to a higher priority
priority=998
```

This is a simple sample config file provided by the Celery team.

Now, create a file to store the application's log messages.

```shell
/var/www/django_dev:~$ mkdir logs
/var/www/django_dev:~$ touch logs/celery-worker.log
```

Send a command to supervisord to re-read the config file and update the process.

```shell
$ sudo supervisorctl reread
taskapp-celery:  available
$ sudo supervisorctl update
taskapp-celery:  added process group
```

Monitor the output of the Celery task process by examining the log file.

```shell
$ tail -f /var/www/django_dev/logs/celery-worker.log
```

For more information on configuring Celery and options for monitoring the task queue status, check out the Celery User Guide.

## Wrap Up
Using Redis with Celery running in the application background is an easy way to automate many of the processes required to keep your application humming along with very little overhead. Use supervisord to monitor the task queue. In some of my recent projects, I use Celery tasks to make RESTful calls against a Django REST API, which in turn triggers both GET and POST methods to specified API endpoints to distribute data to multiple servers.


text:
# Laravel Filter

This package helps developers build query filters.

## Translation

[中文文档](README-cn.md)

## Installation

This package can be installed via composer:

```sh
composer require huang-yi/laravel-filter
```

## Register service provider (Optional)

You need to register the service provider manually if your Laravel version is less than 5.5:

```php
<?php // File: config/app.php

return [
    'providers' => [
        HuangYi\Filter\FilterServiceProvider::class,
    ],
];

```

## Configurations (Optional)

Run this command to publish the configuration file:

```
$ php artisan vendor:publish --provider="HuangYi\Filter\FilterServiceProvider"
```

`key`: The key to get filter rules from query string.

`log_level`: The log level.

`parser`: The filter rules parser.

## Generating filters

To create a new filter, use the make:filter Artisan command. This command will
create a new filter class in the `app/Filters` directory and name the filter in
the `app/filters.php` file.

```sh
php artisan make:filter user.gender
```

```php
<?php // File: app/Filters/User/Gender.php

namespace App\Filters\User;

use HuangYi\Filter\Contracts\FilterContract;
use Illuminate\Database\Eloquent\Builder;

class Gender implements FilterContract
{
    /**
     * Apply filter to eloquent builder.
     *
     * @param \Illuminate\Database\Eloquent\Builder $query
     * @param mixed $value
     * @return void
     */
    public function apply(Builder $query, $value)
    {
        $query->where('gender', $value);
    }
}

```

```php
<?php // File: app/filters.php

use HuangYi\Filter\Facades\Filter;

Filter::name('user.gender', App\Filters\User\Gender::class);

```

## Using filters

This package provides a trait `HuangYi\Filter\HasFilter`.

```php
<?php

namespace App\Http\Controllers;

use App\User;
use HuangYi\Filter\HasFilter;

class UserController extends Controller
{
    use HasFilter;

    /**
     * User list.
     *
     * @return \Illuminate\Database\Eloquent\Builder
     */
    public function index()
    {
        $query = User::query();

        $this->applyFilters($query);

        $users = $query->get();

        return $users;
    }
}

```

## Client query

We should Base 64 encode a JSON representation of the desired filters into the
`filters` query string variable.

```javascript
let Base64 = require('js-base64').Base64;
let axios = require('axios');

let filters = JSON.stringify([
    {
        name: "user.gender",
        value: "female"
    },
    {
        name: "user.cities",
        value: [1, 2, 3]
    }
]);

filters = Base64.encode(filters);

axios.get('/users', {
    filters: filters
}).then((response) => {
    console.log(response);
});
```

## License

Laravel Filter is licensed under [The MIT License (MIT)](http://opensource.org/licenses/MIT).

repository: huang-yi/laravel-filter

repo_name: huang-yi/laravel-filter


text:
---
layout: default
title: Routes
categories: [Routing]
---

# Routes

Routes are filters for URIs that you're interested in. They map certain URIs to instance methods (actions). Every route has a rule tree. A rule imposes conditions on URIs that they must fulfill in order to get processed. Rules can have sub-rules. For a rule to match when it has at least one sub-rule, at least one of the sub-rules has to match.

All routes match in declaration order.

Routes can be fordidden. URIs that match forbidden rules are never processed.

## Example

Suppose you want to route the following URI:
  `https://example.com/resource?foo=bar&page=42`

## Declaration styles

The following four snippets all set up the same routes:

{% highlight ruby %}
class DummyJob < Wayfarer::Job
  route.draw :foo, uri: "https://example.com"
  route.draw :bar, uri: "https://w3c.org"

  def foo; end
  def bar; end
end
{% endhighlight %}

{% highlight ruby %}
class DummyJob < Wayfarer::Job
  routes do
    draw :foo, uri: "https://example.com"
    draw :bar, uri: "https://w3c.org"
  end

  def foo; end
  def bar; end
end
{% endhighlight %}

{% highlight ruby %}
class DummyJob < Wayfarer::Job
  routes do
    draw :foo do
      uri "https://example.com"
    end

    draw :bar do
      uri "https://w3c.org"
    end
  end

  def foo; end
  def bar; end
end
{% endhighlight %}

{% highlight ruby %}
class DummyJob < Wayfarer::Job
  draw uri: "https://example.com"
  def foo; end

  draw uri: "https://w3c.org"
  def bar; end
end
{% endhighlight %}

# Declaring routes

In your Job classes, you declare a set of __routes__ that map URIs to __instance methods__. A route consists of __rules__ that match URIs you are interested in. Rules can be __forbidden__ as well. URIs that match no route or match forbidden rules are ignored. Rules themselves can have sub-rules. A rule with sub-rules matches if the rule itself matches and at least one of its sub-rules matches.

repository: fuzzygroup/wayfarer

repo_name: fuzzygroup/wayfarer


text:
# bsencode in Augmented Backus-Naur Form

The goal of this section is a description of bsencode in ABNF. ABNF is a
language defined in [RFC 4234](https://www.ietf.org/rfc/rfc4234). In order to
describe bsencode, it will be necessary to use a "prose description" to define
bsencode's `bstr` type -- this appears to be a fundamental limitation of ABNF,
that ABNF cannot encode netstrings.

After the grammar is defined, I will talk in the following section about some
modifications of ABNF that could describe netstrings. This is not a serious
proposal for any of those modifications, but just a discussion of what is
perhaps still missing from ABNF.

## Defining BENCODE-STR

Before we begin, I will have to describe netstrings in RFC 4234's ABNF syntax.
Netstrings appear in many places, including BitTorrent's bencode format, which
bsencode derives from. It's a neat idea: a set of ASCII digits encode a decimal
number N, followed by a `:` character in case the string starts with decimal
digits as well. This character is then followed by N arbitrary bytes.

The clearest description of this in RFC 4234-compliant notation is:

    BENCODE-STR = "0:" 0OCTET / "1:" 1OCTET / "2:" 2OCTET / "3:" 3OCTET /
                  "4:" 4OCTET / "5:" 5OCTET / "6:" 6OCTET / "7:" 7OCTET /
                  "8:" 8OCTET / "9:" 9OCTET / "10:" 10OCTET / <...>

This uses the angle-bracketed "prose description, to be used as last resort"
mentioned in  RFC 4234 section 4 to indicate that the pattern continues
*ad infinitum*.

The official [netstring](http://cr.yp.to/proto/netstrings.txt) description
involves using `BENCODE-STR ","` to encode a netstring; bencode just uses
`BENCODE-STR`. My language bsencode breaks with both of these: because I wanted
digits to encode integers, and I wanted a simple parser, I elected to begin a
netstring with the special character `'` -- a quote, because it is after all a
string; it is a single-quote because someone may wish to add JSON-style double
quoted strings sometime in the future.

## The rest of the grammar

Given `BENCODE-STR`, I can define the rest of the core bsencode grammar with
only the normal ABNF rules as:

      expr    = bstr / list / symbol

      ; Syntax Types: binary strings, lists, and symbols.
      bstr    = "'" BENCODE-STR     ; '23:An example byte string.

      list    = array / bin / date / dict / float / regex

      symbol  = true / false / null / int

      ; List Types: arrays and diverse objects
      array   = "(" [expr *(" " expr)] ")"

      bin     = "(" s-bin " " bstr ")"

      date    = "(" s-date " " datestr ")"

      dict    = "(" s-dict *key-val ")"

      float   = "(" s-float " '8:" 8OCTET ")"

      regex   = "(" s-regex " " bstr " " reflags ")"

      ; Symbol Types: subset of JSON.
      true   = %x<IP_ADDRESS>        ; true

      false  = %x66.61.6c.73.65     ; false

      null   = %x6e.75.6c.6c        ; null

      int     = "0" / nz_int        ; Integer. Excludes -0.

      ; Auxiliary Definitions

        ; nonzero integer type used for ints.
      nz_int  = ["-"] dig1-9 *DIGIT

        ; dictionary key-value pairs are wrapped in parens
      key-val = " (" bstr " " expr ")"

        ; regex flags admit optionally g, i, and m in that order
      reflags = ":" [%x67] [%x69] [%x6d]

        ; date format, subset of RFC 3339
      datestr = 4DIGIT "-" 2DIGIT "-" 2DIGIT s-T
                2DIGIT ":" 2DIGIT ":" 2DIGIT "." 3DIGIT s-Z

      ; Character Definitions
      dig1-9  = %x31-39             ; 1-9

      s-T     = %x54                ; T

      s-Z     = %x5a                ; Z

      s-bin   = %x62.69.6e          ; bin

      s-date  = %x<IP_ADDRESS>       ; date

      s-dict  = %x<IP_ADDRESS>       ; dict

      s-float = %x66.6c.6f.61.74;   ; float

      s-regex = %x<IP_ADDRESS>.78    ; regex

# Alternate definitions of BENCODE-STR

My above definition describes `BENCODE-STR` by appealing to an infinite
hierarchy of simple rules. Unfortunately, the syntax I've used is basically
unusable by an ABNF parser, even though it's clear and unambiguous to humans
and easy for our computers to parse. *Can* ABNF handle netstrings? My gut
instinct is "no".

ABNF doesn't quite have proper arithmetic, by which we could convert one form
of a number into another, and netstrings encode the same number *N* in two ways
 -- as ASCII decimal digits and as the length of a string. The only "glue" that
ABNF provides is concatenation, which cannot be undone; since it cannot be
undone, I believe the only recursive decomposition allowed is:

    BENCODE-STR =/ <m> BENCODE-STR <m * 10^n octets>

But ABNF also doesn't have the arithmetic needed to describe 10^n octets. So it
seems it doesn't have the expressive capability to describe netstrings at all.
I present this as a sort of conjecture, because I'm not completely certain
about the middle steps -- does ABNF really *need* to "glue" these two different
representations together?

And if ABNF *doesn't* work, then what *does*?

## A minimal solution

The simplest way to do this is to add onto ABNF the rules:

    num-val /= "$"
    repeat  /= "#" / ("#*" *DIGIT) / (*DIGIT "*#")

We interpret them as follows: a rule containing `#` or `$` represents a family
of rules for each integer 0, 1, 2, ...,  with that integer standing in for the
symbols `#` and `$` throughout the rule -- that is, this integer is the same
for all `#` and for all `$`. The difference between these symbols is that `$`
is the ASCII representation of the integer, while `#` is the integer itself
appearing in a repetition clause. Then `BENCODE-STR` can be written as:

    BENCODE-STR = $ ":" #OCTET

It more or less manufactures the rules that we need in order to express the
infinite hierarchy above; a more sophisticated approach might allow these to
bind to separate variables too -- `$(n)` might always be the same number as
`#(n)` but distinct from `$(m)` -- but we do not need this complexity for this
particular task.

## With local ABNF rules

The first proposal only adds the bare minimum of new ideas to effect netstrings
in ABNF and therefore feels mathematically unsatisfying, so I came up with the
idea of adding a new "glue" in the form of ABNF with local rule bindings. I
shall call this LABNF for the purposes of this section; if ABNF grammars were
translated into function calls, LABNF would allow those calls to have proper
function arguments.

To get LABNF, I add two rules to the ABNF description of ABNF:

    rule /= rulename 1*(":" rulename) defined-as elements c-nl
    element /= rulename 1*(":" group)

In other words, on the left hand side of a rule, a rule can now be specified
as a set of rule names separated by colons, and an element can now be specified
as a rulename followed by colons and groups. (I use groups here to avoid
remembering associativity or precedence rules.)

The leftmost name is the root rule name, and the other names bind local rules.
When the rule is referenced as an element, it must be as the root name followed
(in the above syntax) with one group for each local rule. The local rules get
bound to the groups exactly like normal ABNF rule definitions, except that this
binding only takes effect within the scope of the rule being defined.

In LABNF, the definition for `BENCODE-STR` would be described as:

    BS:num:str = num ":" str /
            BS:(num "0"):(10str 0OCTET) / BS:(num "1"):(10str 1OCTET) /
            BS:(num "2"):(10str 2OCTET) / BS:(num "3"):(10str 3OCTET) /
            BS:(num "4"):(10str 4OCTET) / BS:(num "5"):(10str 5OCTET) /
            BS:(num "6"):(10str 6OCTET) / BS:(num "7"):(10str 7OCTET) /
            BS:(num "8"):(10str 8OCTET) / BS:(num "9"):(10str 9OCTET)

    BENCODE-STR = "0:" /
            BS:("1"):(1OCTET) / BS:("2"):(2OCTET) / BS:("3"):(3OCTET) /
            BS:("4"):(4OCTET) / BS:("5"):(5OCTET) / BS:("6"):(6OCTET) /
            BS:("7"):(7OCTET) / BS:("8"):(8OCTET) / BS:("9"):(9OCTET)

This builds a decimal recursively as `<n> <d>` where `<d>` is a single digit,
in parallel it builds up an octet rule which repeats `10 * n + d` times.

I present LABNF as a curiosity rather than a serious proposal. I don't know of
other situations where LABNF is more userful than ABNF for a practical problem,
and the `<...>` syntax is clearer in this particular case, even though a
computer cannot use it.

repository: crdrost/bsencode

repo_name: crdrost/bsencode


text:
# @umijs/plugin-access

[![codecov](https://codecov.io/gh/umijs/plugin-access/branch/master/graph/badge.svg)](https://codecov.io/gh/umijs/plugin-access)
[![NPM version](https://img.shields.io/npm/v/@umijs/plugin-access.svg?style=flat)](https://npmjs.org/package/@umijs/plugin-access)
[![CircleCI](https://circleci.com/gh/umijs/plugin-access/tree/master.svg?style=svg)](https://circleci.com/gh/umijs/plugin-access/tree/master)
[![GitHub Actions status](https://github.com/umijs/plugin-access/workflows/Node%20CI/badge.svg)](https://github.com/umijs/plugin-access)
[![NPM downloads](http://img.shields.io/npm/dm/@umijs/plugin-access.svg?style=flat)](https://npmjs.org/package/@umijs/plugin-access)

Umi plugin for access management.

## Prerequisites

Before using this plugin, you need install and enable [@umijs/plugin-initial-state](https://www.npmjs.com/package/@umijs/plugin-initial-state) and [@umijs/plugin-model](https://www.npmjs.com/package/@umijs/plugin-model).

## Install

```bash
# or yarn
$ npm install @umijs/plugin-access --save
```

## Usage

Getting started in 3 steps.

### 1. Configure in `.umirc.js`

**Caution**: `@umijs/plugin-access`, `@umijs/plugin-initial-state` and `@umijs/plugin-model` must be in this order.

```js
export default {
  plugins: [
    ['@umijs/plugin-access'],
    ['@umijs/plugin-initial-state'],
    ['@umijs/plugin-model'],
  ],
};
```

### 2. Export `getInitialState()` function in `src/app.js`

You can fetch some data asynchronously or synchronously then return whatever value in the `getInitialState()` function, the returned value would be saved as initial state (basic information) by umi. For example:

```js
// src/app.js

export async function getInitialState() {
  const { userId, fole } = await getCurrentRole();
  return {
    userId,
    role,
  };
}
```

### 3. Create `src/access.js` and defaultly export a function to define the access feature of your application

With the initial state (basic information) prepared, you can define the access feature of your application, like "can create something", "can't update something", just return the definition in the function:

```js
// src/access.js

export default function(initialState) {
  const { userId, role } = initialState; // the initialState is the returned value in step 2

  return {
    canReadFoo: true,
    canUpdateFoo: role === 'admin',
    canDeleteFoo: foo => {
      return foo.ownerId === userId;
    },
  };
}
```

### 4. Consume the access feature definition

After step 3, now you get the access feature definition of your application, then you can use the definition in your component:

```jsx
import React from 'react';
import { useAccess, Access } from 'umi';

const PageA = props => {
  const { foo } = props;
  const access = useAccess(); // members of access: canReadFoo, canUpdateFoo, canDeleteFoo

  if (access.canReadFoo) {
    // Do something...
  }

  return (
    <div>
      <Access
        accessible={access.canReadFoo}
        fallback={<div>Can not read foo content.</div>}
      >
        Foo content.
      </Access>
      <Access
        accessible={access.canUpdateFoo}
        fallback={<div>Can not update foo.</div>}
      >
        Update foo.
      </Access>
      <Access
        accessible={access.canDeleteFoo(foo)}
        fallback={<div>Can not delete foo.</div>}
      >
        Delete foo.
      </Access>
    </div>
  );
};
```

You can use the `access` instance to control the execution flow, use `<Access>` component to control the rendering, when `accessible` is true, children is rendered, otherwise `fallback` is rendered.

**Full example can find in [./example](https://github.com/umijs/plugin-access/tree/master/example).**

## Options

* `options.showWarning`

A boolean value, default to be `true`. When `showWarning` is `true`, this plugin would check if `src/access.js` is exist and defaultly exports a function, if no function exported, a warning info would be shown, otherwise if `showWarning` is `false`, no warning info would be shown.

## LICENSE

MIT

repository: umijs/plugin-access

gha_language: TypeScript

repo_name: umijs/plugin-access


text:
# Developers Guide

This is a guide for developers hacking on AutobahnPython.

## Python 2 / 3

AutobahnPython supports both Python 2 and Python 3.

* http://python-future.org/compatible_idioms.html

## Release Process

1. Travis is fully green
2. Run AutobahnTestsuite
3. Build and spellcheck the docs
4. Update the [doc/changelog.rst](doc/changelog.rst)
5. Tag the release
6. Publish the package to PyPi
7. Publish the docs to the Autobahn Web site
8. Announce on mailing list

## Roadmap

### 0.10.6

[Milestone for 0.10.6](https://github.com/tavendo/AutobahnPython/milestones/0.10.6)

* Automatic reconnection
* Configurable WAMP connecting transports
* WAMP Connection abstration
* Authentication

**0.10** will get into "maintenance mode" after **0.10.6**. We'll have a maintenance branch for that over some time. The new development will be based on **0.11** (see below).

### 0.11.0

[Milestone for 0.11.0](https://github.com/tavendo/AutobahnPython/milestones/0.11.0)

* new main development line with new, long-term API

repository: shuyunqi/AutobahnPython

repo_name: shuyunqi/AutobahnPython