text: HAUSDORFF DIMENSION OF KUPERBERG MINIMAL SETS 53 8. Symbolic dynamics of propellers In the previous section we developed a symbolic dynamics of the Kuperberg pseudogroup Ψ acting on a transverse section O+(Θ(y)) ∩ S of an orbit. In this section, we will develop 1 similar symbolic dynamics for Ψ acting on a transverse section of a particular orbit surface. We will choose the same transversal S as deﬁned in Equation 14. The orbit surface we will consider is O+(γ); this is a double propeller in the sense of Deﬁnition 5.2, where γ is deﬁned 1 in Equation 16. It is comprised of two single propellers O+(γu) and O+(γl), in the sense of 1 1 Deﬁnition 5.1. (cid:91) The intersection O+(γ) ∩ S has a well-deﬁned level decomposition (see Deﬁnition 6.11) 1 ∞ O+(γ) ∩ S = O+(γ) ∩ S, where O+(γ) ∩ S = {y ∈ O+(γ) ∩ S : n (y) = k}, 1 1 k 1 k 1 γ k=0 and (cid:104)Φ, Θ(cid:105) acts on this intersection as in Section : (30) Φ : O+(γ) ∩ S (cid:55)→ O+(γ) ∩ S 1 k 1 k Θ : O+(γ) ∩ S (cid:55)→ O+(γ) ∩ S 1 k 1 k+1 In this section we will present the symbolic dynamics of the Kuperberg pseudogroup on the transverse intersection O+(γ) ∩ S. We have an analogue of Equation 28 for the orbit of 1 (cid:91) γ: (31) O+(γ) ∩ S = g(γ). 1 g∈Ψ1,1 Using this we will enumerate the curves by level according to a sequence space, and show that Ψ = Ψ acts faithfully on this sequence space. In the process we will compute explicit 1,1 parametrizations of the intersection curves comprising each level set, and compute bounds on their escape times. We begin with the level-zero curve γ parametrized in Equation 20: γ(s) = (2, β, −1 + s), with s ∈ [−R, R]. We refer to the midpoint γ(0) as the vertex of γ, which is the intersection l ∩ S of the special orbit with S. The only curve of level zero in O+(γ) ∩ S is γ, so O+(γ) ∩ S = {γ}. 1 1 0 N Let γ1 = Θ(γ), and for all i1 ∈ let γi1 = (Φi1−1Θ)(γ). By Equation 30, γi1 has level one for all i . We use the assumptions we made in Section to ﬁnd an explicit parametrization of 1 γ as follows. i1

text: HAUSDORFF

text: DIMENSION

text: OF

text: KUPERBERG

text: MINIMAL

text: SETS

label: section

text: Symbolic

text: dynamics

text: of

text: propellers

text: In

text: the

text: previous

text: section

text: we

text: developed

text: a

text: symbolic

text: dynamics

text: of

text: the

text: Kuperberg

text: pseudogroup

text: Ψ

text: acting

text: on

text: a

text: transverse

text: section

text: O+(Θ(y))

text: S

text: of

text: an

text: orbit.

text: In

text: this

text: section,

text: we

text: will

text: develop

text: similar

text: symbolic

text: dynamics

text: for

text: Ψ

text: acting

text: on

text: a

text: transverse

text: section

text: of

text: a

text: particular

text: orbit

text: surface.

text: We

text: will

text: choose

text: the

text: same

text: transversal

text: S

text: as

text: deﬁned

text: in

text: Equation

text: The

text: orbit

text: surface

text: we

text: will

text: consider

text: is

text: O+(γ);

text: this

text: is

text: a

text: double

text: propeller

text: in

text: the

text: sense

text: of

text: Deﬁnition

text: where

text: γ

text: is

text: deﬁned

text: in

text: Equation

text: It

text: is

text: comprised

text: of

text: two

text: single

text: propellers

text: O+(γu)

text: and

text: O+(γl),

text: in

text: the

text: sense

text: of

text: Deﬁnition

text: The

text: intersection

text: O+(γ)

text: S

text: has

text: a

text: well-deﬁned

text: level

text: decomposition

text: (see

text: Deﬁnition

text: O+(γ)

text: S

text: O+(γ)

text: S,

text: where

text: O+(γ)

text: S

text: {y

text: O+(γ)

text: S

text: n

text: (y)

text: k},

text: k

text: k

text: γ

text: and

text: acts

text: on

text: this

text: intersection

text: as

text: in

text: Section

text: Φ

text: O+(γ)

text: S

text: O+(γ)

text: S

text: k

text: k

text: Θ

text: O+(γ)

text: S

text: O+(γ)

text: S

text: k

text: In

text: this

text: section

text: we

text: will

text: present

text: the

text: symbolic

text: dynamics

text: of

text: the

text: Kuperberg

text: pseudogroup

text: on

text: the

text: transverse

text: intersection

text: O+(γ)

text: S.

text: We

text: have

text: an

text: analogue

text: of

text: Equation

text: for

text: the

text: orbit

text: of

text: γ:

text: O+(γ)

text: S

text: g(γ).

text: Using

text: this

text: we

text: will

text: enumerate

text: the

text: curves

text: by

text: level

text: according

text: to

text: a

text: sequence

text: space,

text: and

text: show

text: that

text: Ψ

text: Ψ

text: acts

text: faithfully

text: on

text: this

text: sequence

text: space.

text: In

text: the

text: process

text: we

text: will

text: compute

text: explicit

text: parametrizations

text: of

text: the

text: intersection

text: curves

text: comprising

text: each

text: level

text: set,

text: and

text: compute

text: bounds

text: on

text: their

text: escape

text: times.

text: We

text: begin

text: with

text: the

text: level-zero

text: curve

text: γ

text: parametrized

text: in

text: Equation

text: γ(s)

text: β,

text: −1

text: s),

text: with

text: s

text: [−R,

text: R].

text: We

text: refer

text: to

text: the

text: midpoint

text: as

text: the

text: vertex

text: of

text: γ,

text: which

text: is

text: the

text: intersection

text: l

text: S

text: of

text: the

text: special

text: orbit

text: with

text: S.

text: The

text: only

text: curve

text: of

text: level

text: zero

text: in

text: O+(γ)

text: S

text: is

text: γ,

text: so

text: O+(γ)

text: S

text: {γ}.

text: N

text: Let

text: γ1

text: Θ(γ),

text: and

text: for

text: all

text: let

text: By

text: Equation

text: has

text: level

text: one

text: for

text: all

text: i

text: We

text: use

text: the

text: assumptions

text: we

text: made

text: in

text: Section

text: to

text: ﬁnd

text: an

text: explicit

text: parametrization

text: of

text: γ

text: as

text: follows.


text: 2 Xu, Yue & Chen 2. The Island Model In the island model, a region remains neutral if the available number of ionizing pho- tons, either produced inside or outside, is not enough to ionize all hydrogen atoms in the region. The condition for island formation consists of two terms: Ω N m ξf (δ ; M, z) + m back H < 1, (2.1) coll M Ω MX (1 + n¯ ) b H rec where the ﬁrst term on the L.H.S. accounts for the ionizing photons produced by collapsed objects inside, and the second term accounts for the contribution by the ionizing background around it. ζ is the ionizing eﬃciency parameter, f is the collapsed fraction, coll and N is the number of consumed background ionizing photons. Assuming Gaussian back density ﬂuctuations, this condition can be written as a constraint on the density contrast of the region, i.e. the island barrier. Assuming spherical islands, and that the consumption rate of background ionizing photons is proportional to their surface area, we derive the basic island barrier shown with green curves in the left panel of Fig. 1. Diﬀerent from the bubble barrier, the island barrier bends down at small scales because of the ionizing background. One can derive the mass distribution and the volume fraction of neutral regions by solving for the ﬁrst-down-crossing distribution of random trajectories with respect to the island barrier. Note that there is a “bubbles-in-island” eﬀect, as there might be self-ionized regions inside a host island. The distribution of bubbles inside islands are obtained by computing the conditional probability distribution of the trajectories. The average bubbles-in-island fraction can be derived by integrating over all possible bubble sizes for a given island, and this fraction is limited by a percolation threshold p in order to deﬁne the bona ﬁde c islands, in the sense that an island is not percolated through by bubbles inside. This percolation criterion of q < p acts as an additional barrier for ﬁnding islands (plotted B c with the red curves in the left panel of Fig. 1), which combines with the basic island barrier to deﬁne the host islands. The percolation threshold also demarcates the scope of application of both bubble model and island model. The bubble model can be reliably applied before bubble ﬁlling factor becomes larger than p , while the island model should c be used below the redshift at which the island ﬁlling factor equals to p . c With the combined island barrier, the island model predicts the size distribution and evolution of islands after the islands become isolated. As the redshift decreases, small islands disappear rapidly while the large ones shrinks. Eventually, all these under-dense islands are ionized, only compact neutral regions such as galaxies or minihalos remain. 3. The IslandFAST Model The semi-numerical simulation islandFAST is developed from the 21cmFAST (Mesinger et al. 2011), but extended to treat the late EoR by the island model (see Xu et al. (2017) for details). Compared with the 21cmFAST, the major diﬀerences of the islandFAST are the two-step ﬁltering algorithm in generating the ionization ﬁeld in order to take bubbles-in-island eﬀect into account, and the self-consistent treatment for the ionizing background accounting for various absorption systems. The mean free path of ionizing photons and the corresponding intensity of the ionizing background is obtained by an iterative procedure. Adaptive redshift steps are applied in order to achieve convergence in the ionization ﬁeld and the intensity of the ionizing background. We ﬁnd that the late reionization proceeds very fast, and the bubbles-in-island eﬀect is quite obvious, showing complex morphology of the ionization ﬁeld (Fig. 2). In the central panel of Fig. 1 we show the comoving size distributions of neutral islands for ##LTLine## ##LTLine##

label: paragraph

text: Xu,

text: Yue

text: Chen

text: The

text: Island

text: Model

text: In

text: the

text: island

text: model,

text: a

text: region

text: remains

text: neutral

text: if

text: the

text: available

text: number

text: of

text: ionizing

text: pho-

text: tons,

text: either

text: produced

text: inside

text: or

text: outside,

text: is

text: not

text: enough

text: to

text: ionize

text: all

text: hydrogen

text: atoms

text: in

text: the

text: region.

text: The

text: condition

text: for

text: island

text: formation

text: consists

text: of

text: two

text: terms:

text: Ω

label: equation

text: N

text: m

text: ξf

text: (δ

text: M,

text: z)

text: m

text: back

text: H

text: coll

text: M

text: Ω

text: MX

text: n¯

text: b

text: H

text: rec

text: where

text: the

text: ﬁrst

text: term

text: on

text: the

text: L.H.S.

text: accounts

text: for

text: the

text: ionizing

text: photons

text: produced

text: by

text: collapsed

text: objects

text: inside,

text: and

text: the

text: second

text: term

text: accounts

text: for

text: the

text: contribution

text: by

text: the

text: ionizing

text: background

text: around

text: it.

text: ζ

text: is

text: the

text: ionizing

text: eﬃciency

text: parameter,

text: f

text: is

text: the

text: collapsed

text: fraction,

text: coll

text: and

text: N

text: is

text: the

text: number

text: of

text: consumed

text: background

text: ionizing

text: photons.

text: Assuming

text: Gaussian

text: back

text: density

text: ﬂuctuations,

text: this

text: condition

text: can

text: be

text: written

text: as

text: a

text: constraint

text: on

text: the

text: density

text: contrast

text: of

text: the

text: region,

text: i.e.

text: the

text: island

text: barrier.

text: Assuming

text: spherical

text: islands,

text: and

text: that

text: the

text: consumption

text: rate

text: of

text: background

text: ionizing

text: photons

text: is

text: proportional

text: to

text: their

text: surface

text: area,

text: we

text: derive

text: the

text: basic

text: island

text: barrier

text: shown

text: with

text: green

text: curves

text: in

text: the

text: left

text: panel

text: of

text: Fig.

text: Diﬀerent

text: from

text: the

text: bubble

text: barrier,

text: the

text: island

text: barrier

text: bends

text: down

text: at

text: small

text: scales

text: because

text: of

text: the

text: ionizing

text: background.

text: One

text: can

text: derive

text: the

text: mass

text: distribution

text: and

text: the

text: volume

text: fraction

text: of

text: neutral

text: regions

text: by

text: solving

text: for

text: the

text: ﬁrst-down-crossing

text: distribution

text: of

text: random

text: trajectories

text: with

text: respect

text: to

text: the

text: island

text: barrier.

text: Note

text: that

text: there

text: is

text: a

text: “bubbles-in-island”

text: eﬀect,

text: as

text: there

text: might

text: be

text: self-ionized

text: regions

text: inside

text: a

text: host

text: island.

text: The

text: distribution

text: of

text: bubbles

text: inside

text: islands

text: are

text: obtained

text: by

text: computing

text: the

text: conditional

text: probability

text: distribution

text: of

text: the

text: trajectories.

text: The

text: average

text: bubbles-in-island

text: fraction

text: can

text: be

text: derived

text: by

text: integrating

text: over

text: all

text: possible

text: bubble

text: sizes

text: for

text: a

text: given

text: island,

text: and

text: this

text: fraction

text: is

text: limited

text: by

text: a

text: percolation

text: threshold

text: p

text: in

text: order

text: to

text: deﬁne

text: the

text: bona

text: ﬁde

text: c

text: islands,

text: in

text: the

text: sense

text: that

text: an

text: island

text: is

text: not

text: percolated

text: through

text: by

text: bubbles

text: inside.

text: This

text: percolation

text: criterion

text: of

text: q

text: p

text: acts

text: as

text: an

text: additional

text: barrier

text: for

text: ﬁnding

text: islands

text: (plotted

text: B

text: c

text: with

text: the

text: red

text: curves

text: in

text: the

text: left

text: panel

text: of

text: Fig.

text: which

text: combines

text: with

text: the

text: basic

text: island

text: barrier

text: to

text: deﬁne

text: the

text: host

text: islands.

text: The

text: percolation

text: threshold

text: also

text: demarcates

text: the

text: scope

text: of

text: application

text: of

text: both

text: bubble

text: model

text: and

text: island

text: model.

text: The

text: bubble

text: model

text: can

text: be

text: reliably

text: applied

text: before

text: bubble

text: ﬁlling

text: factor

text: becomes

text: larger

text: than

text: p

text: while

text: the

text: island

text: model

text: should

text: c

text: be

text: used

text: below

text: the

text: redshift

text: at

text: which

text: the

text: island

text: ﬁlling

text: factor

text: equals

text: to

text: p

text: c

text: With

text: the

text: combined

text: island

text: barrier,

text: the

text: island

text: model

text: predicts

text: the

text: size

text: distribution

text: and

text: evolution

text: of

text: islands

text: after

text: the

text: islands

text: become

text: isolated.

text: As

text: the

text: redshift

text: decreases,

text: small

text: islands

text: disappear

text: rapidly

text: while

text: the

text: large

text: ones

text: shrinks.

text: Eventually,

text: all

text: these

text: under-dense

text: islands

text: are

text: ionized,

text: only

text: compact

text: neutral

text: regions

text: such

text: as

text: galaxies

text: or

text: minihalos

text: remain.

text: The

text: IslandFAST

text: Model

text: The

text: semi-numerical

text: simulation

text: islandFAST

text: is

text: developed

text: from

text: the

text: (Mesinger

text: et

text: al.

text: but

text: extended

text: to

text: treat

text: the

text: late

text: EoR

text: by

text: the

text: island

text: model

text: (see

text: Xu

text: et

text: al.

text: for

text: details).

text: Compared

text: with

text: the

text: the

text: major

text: diﬀerences

text: of

text: the

text: islandFAST

text: are

text: the

text: two-step

text: ﬁltering

text: algorithm

text: in

text: generating

text: the

text: ionization

text: ﬁeld

text: in

text: order

text: to

text: take

text: bubbles-in-island

text: eﬀect

text: into

text: account,

text: and

text: the

text: self-consistent

text: treatment

text: for

text: the

text: ionizing

text: background

text: accounting

text: for

text: various

text: absorption

text: systems.

text: The

text: mean

text: free

text: path

text: of

text: ionizing

text: photons

text: and

text: the

text: corresponding

text: intensity

text: of

text: the

text: ionizing

text: background

text: is

text: obtained

text: by

text: an

text: iterative

text: procedure.

text: Adaptive

text: redshift

text: steps

text: are

text: applied

text: in

text: order

text: to

text: achieve

text: convergence

text: in

text: the

text: ionization

text: ﬁeld

text: and

text: the

text: intensity

text: of

text: the

text: ionizing

text: background.

text: We

text: ﬁnd

text: that

text: the

text: late

text: reionization

text: proceeds

text: very

text: fast,

text: and

text: the

text: bubbles-in-island

text: eﬀect

text: is

text: quite

text: obvious,

text: showing

text: complex

text: morphology

text: of

text: the

text: ionization

text: ﬁeld

text: (Fig.

text: In

text: the

text: central

text: panel

text: of

text: Fig.

text: we

text: show

text: the

text: comoving

text: size

text: distributions

text: of

text: neutral

text: islands

text: for

text: ##LTLine##

text: ##LTLine##


text: Wigner functions on non-standard symplectic vector spaces Nuno Costa Dias∗ Joa˜o Nuno Prata† May 30, 2020 Abstract We consider the Weyl quantization on a ﬂat non-standard symplec- tic vector space. We focus mainly on the properties of the Wigner func- tions deﬁned therein. In particular we show that the sets of Wigner functions on distinct symplectic spaces are diﬀerent but have non- empty intersections. This extends previous results to arbitrary dimen- sion and arbitrary (constant) symplectic structure. As a by-product we introduce and prove several concepts and results on non-standard symplectic spaces which generalize those on the standard symplectic space, namely the symplectic spectrum, Williamson’s theorem and Narcowich-Wigner spectra. We also show how Wigner functions on non-standard symplectic spaces behave under the action of an arbi- trary linear coordinate transformation. MSC [2000]: Primary 35S99, 35P05; Secondary 35S05, 47A75 Keywords: Weyl quantization; Wigner functions; symplectic spaces; sym- plectic spectrum; Williamson’s theorem; Narcowich-Wigner spectra 1 Introduction In this work we return to the research programme initiated in [2, 3]. There we considered the Weyl-Wigner formulation of quantum mechanics based ∗ ncdias@meo.pt † joao.prata@mail.telepac.pt 1 ##LTLine##

text: Wigner

label: title

text: functions

text: on

text: non-standard

text: symplectic

text: vector

text: spaces

text: Nuno

label: author

text: Costa

text: Dias∗

text: Joa˜o

text: Nuno

text: Prata†

text: May

text: Abstract

text: We

label: abstract

text: consider

text: the

text: Weyl

text: quantization

text: on

text: a

text: ﬂat

text: non-standard

text: symplec-

text: tic

text: vector

text: space.

text: We

text: focus

text: mainly

text: on

text: the

text: properties

text: of

text: the

text: Wigner

text: func-

text: tions

text: deﬁned

text: therein.

text: In

text: particular

text: we

text: show

text: that

text: the

text: sets

text: of

text: Wigner

text: functions

text: on

text: distinct

text: symplectic

text: spaces

text: are

text: diﬀerent

text: but

text: have

text: non-

text: empty

text: intersections.

text: This

text: extends

text: previous

text: results

text: to

text: arbitrary

text: dimen-

text: sion

text: and

text: arbitrary

text: (constant)

text: symplectic

text: structure.

text: As

text: a

text: by-product

text: we

text: introduce

text: and

text: prove

text: several

text: concepts

text: and

text: results

text: on

text: non-standard

text: symplectic

text: spaces

text: which

text: generalize

text: those

text: on

text: the

text: standard

text: symplectic

text: space,

text: namely

text: the

text: symplectic

text: spectrum,

text: Williamson’s

text: theorem

text: and

text: Narcowich-Wigner

text: spectra.

text: We

text: also

text: show

text: how

text: Wigner

text: functions

text: on

text: non-standard

text: symplectic

text: spaces

text: behave

text: under

text: the

text: action

text: of

text: an

text: arbi-

text: trary

text: linear

text: coordinate

text: transformation.

text: MSC

text: Primary

text: Secondary

text: Keywords:

text: Weyl

text: quantization;

text: Wigner

text: functions;

text: symplectic

text: spaces;

text: sym-

text: plectic

text: spectrum;

text: Williamson’s

text: theorem;

text: Narcowich-Wigner

text: spectra

text: Introduction

text: In

text: this

text: work

text: we

text: return

text: to

text: the

text: research

text: programme

text: initiated

text: in

text: There

text: we

text: considered

text: the

text: Weyl-Wigner

text: formulation

text: of

text: quantum

text: mechanics

text: based

text: ncdias@meo.pt

text: joao.prata@mail.telepac.pt

text: ##LTLine##


text: In the sequel, we shall assume that det Ω = 1, det S = ±1, (58) (cid:110) (cid:111) (cid:98) for all S ∈ D(n; ω). This is because diﬀerent values of the determinant of Ω simply amount to a rescaling of the variables Ξ (or, equivalently α ≤ ≤ 1 α 2n of a rescaling of Planck’s constant). Upon exponentiation of (57), we obtain the non-standard Heisenberg- Weyl displacement operators: (cid:98) (cid:98) Dω(ξ) = eiω(ξ,Ξ), (59) satisfyin(cid:98)g the (cid:98)commutation rel(cid:98)ations (cid:98) (cid:98) Dω(ξ)Dω(ζ) = e i ω(ξ,ζ)Dω(ξ + ζ) = eiω(ξ,ζ)Dω(ζ)Dω(ξ). (60) 2 As before, the elements o(cid:98)f the form (cid:98) Uω(ξ, τ) = eiτDω(ξ), (61) constitute a unitary irreducib(cid:18)le representation of the m(cid:19)odiﬁed Heisenberg H group ω(n) with multiplication (cid:48) (cid:48) (cid:48) (cid:48) 1 (cid:48) (ξ, τ) · (ξ , τ ) = ξ + ξ , τ + τ + ω(ξ, ξ ) . (62) (cid:98) 2 Notice that if S ∈ D(n; ω) and Z obey the Heisenberg-Weyl algebra (32), then (cid:98) (cid:98) Ξ = SZ (63) satisfy (5(cid:98)7). Consequently, we may write (cid:98) (cid:98) (cid:98) (cid:98) − − Dω(ξ) = eiω(ξ,Ξ) = eiω(ξ,SZ) = eiσ(S 1ξ,Z) = Dσ(S 1ξ). (64) 15 ##LTLine## ##LTLine##

text: In

text: the

text: sequel,

text: we

text: shall

text: assume

text: that

text: det

text: Ω

text: det

text: S

text: for

text: all

text: S

text: D(n;

text: ω).

text: This

text: is

text: because

text: diﬀerent

text: values

text: of

text: the

text: determinant

text: of

text: Ω

text: simply

text: amount

text: to

text: a

text: rescaling

text: of

text: the

text: variables

text: Ξ

text: (or,

text: equivalently

text: α

text: α

text: of

text: a

text: rescaling

text: of

text: Planck’s

text: constant).

text: Upon

text: exponentiation

text: of

text: we

text: obtain

text: the

text: non-standard

text: Heisenberg-

text: Weyl

text: displacement

text: operators:

text: Dω(ξ)

text: eiω(ξ,Ξ),

text: the

text: Dω(ξ)Dω(ζ)

text: e

text: i

text: ω(ξ,ζ)Dω(ξ

text: ζ)

text: eiω(ξ,ζ)Dω(ζ)Dω(ξ).

text: As

text: before,

text: the

text: elements

text: the

text: form

text: Uω(ξ,

text: τ)

text: eiτDω(ξ),

text: constitute

text: a

text: unitary

text: representation

text: of

text: the

text: Heisenberg

text: H

text: group

text: ω(n)

text: with

text: multiplication

text: (ξ,

text: τ)

text: (ξ

text: τ

text: ξ

text: ξ

text: τ

text: τ

text: ω(ξ,

text: ξ

text: Notice

text: that

text: if

text: S

text: D(n;

text: ω)

text: and

text: Z

text: obey

text: the

text: Heisenberg-Weyl

text: algebra

text: then

text: Ξ

text: SZ

text: satisfy

text: Consequently,

text: we

text: may

text: write

text: Dω(ξ)

text: eiω(ξ,Ξ)

text: eiω(ξ,SZ)

text: eiσ(S

text: Dσ(S

text: ##LTLine##

text: ##LTLine##


text: Before we proceed, let us recapitulate the concept of Hamard-Schur (HS) C product. Let A = [[A ]] , B = [[B ]] ∈ M(k × n; ). Then the HS product ij ij C C C is the map ♦ : M(k × n; ) × M(k × n; ) → M(k × n; ) A♦B = [[A B ]] . (85) ij ij According to Schur’s Theorem, the HS product has the property of being P C a closed operation in the convex cone × ( ) of positive k × k matrices, i.e.: k k P C P C P C ♦ : × ( ) × × ( ) → × ( ). (86) k k k k k k R R Theorem 10 Let f, g ∈ L1( 2n) be real-valued functions on 2n, such that F(f), F(g) are of the (α, Σ) and of the (β, Υ)-positive type, respectively, and 1 det(αΣ + βΥ) (cid:54)= 0. Moreover, deﬁne γ := (det(αΣ + βΥ))2n to be a real (2n)-th root of that determinant (this is always possible since real antisym- (cid:16) (cid:17) metric matrices have non-negative determinants; they are equal to the square of a polynomial of its entries - the Pfaﬃan). Then the Fourier transform of the convolution f (cid:63) g is of the γ, αΣ + β Υ -positive type. For later conve- γ (cid:18) γ (cid:19) nience we write: α β (α, Σ) ⊕ (β, Υ) = γ, Σ + Υ . (87) γ γ R Proof. Since f, g ∈ L1( 2n), the Fourier transform of the convolution amounts to the pointwise product F(f (cid:63) g) = F(f) · F(g). Thus: (cid:16) (cid:17) (cid:16) · (cid:17) F(f (cid:63) g)(aj − ak)ei2γ (αγ ak Σ+βγ Υ)aj = (88) · · = F(f)(aj − ak)ei2αak Σaj F(g)(aj − ak)ei2β ak Υaj . The last expression is the (jk)-th component of the HS product of two positive matrices and the result follows. Henceforth, we shall use the notation G to represent the phase-space A Gaussian with covariance matrix A: 22 ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine##

text: Before

text: we

text: proceed,

text: let

text: us

text: recapitulate

text: the

text: concept

text: of

text: Hamard-Schur

text: (HS)

text: C

text: product.

text: Let

text: A

text: [[A

text: B

text: [[B

text: M(k

text: n;

text: Then

text: the

text: HS

text: product

text: ij

text: ij

text: C

text: C

text: C

text: is

text: the

text: map

text: M(k

text: n;

text: M(k

text: n;

text: M(k

text: n;

text: A♦B

text: [[A

text: B

text: ij

text: ij

text: According

text: to

text: Schur’s

text: Theorem,

text: the

text: HS

text: product

text: has

text: the

text: property

text: of

text: being

text: P

text: C

text: a

text: closed

text: operation

text: in

text: the

text: convex

text: cone

text: of

text: positive

text: k

text: k

text: matrices,

text: i.e.:

text: k

text: k

text: P

text: C

text: P

text: C

text: P

text: C

text: k

text: k

text: k

text: k

text: k

text: k

text: R

text: R

text: Theorem

text: Let

text: f,

text: g

text: be

text: real-valued

text: functions

text: on

text: such

text: that

text: F(f),

text: F(g)

text: are

text: of

text: the

text: (α,

text: Σ)

text: and

text: of

text: the

text: (β,

text: Υ)-positive

text: type,

text: respectively,

text: and

text: det(αΣ

text: βΥ)

text: Moreover,

text: deﬁne

text: γ

text: (det(αΣ

text: to

text: be

text: a

text: real

text: root

text: of

text: that

text: determinant

text: (this

text: is

text: always

text: possible

text: since

text: real

text: antisym-

text: metric

text: matrices

text: have

text: non-negative

text: determinants;

text: they

text: are

text: equal

text: to

text: the

text: square

text: of

text: a

text: polynomial

text: of

text: its

text: entries

text: the

text: Pfaﬃan).

text: Then

text: the

text: Fourier

text: transform

text: of

text: the

text: convolution

text: f

text: g

text: is

text: of

text: the

text: γ,

text: αΣ

text: β

text: Υ

text: -positive

text: type.

text: For

text: later

text: conve-

text: γ

text: γ

text: nience

text: we

text: write:

text: α

text: β

text: (α,

text: Σ)

text: (β,

text: Υ)

text: γ,

text: Σ

text: Υ

text: γ

text: γ

text: R

text: Proof.

text: Since

text: f,

text: g

text: the

text: Fourier

text: transform

text: of

text: the

text: convolution

text: amounts

text: to

text: the

text: pointwise

text: product

text: F(f

text: g)

text: F(f)

text: F(g).

text: Thus:

text: F(f

text: g)(aj

text: (αγ

text: ak

text: Σ+βγ

text: Υ)aj

text: F(f)(aj

text: Σaj

text: F(g)(aj

text: ak

text: Υaj

text: The

text: last

text: expression

text: is

text: the

text: (jk)-th

text: component

text: of

text: the

text: HS

text: product

text: of

text: two

text: positive

text: matrices

text: and

text: the

text: result

text: follows.

text: Henceforth,

text: we

text: shall

text: use

text: the

text: notation

text: G

text: to

text: represent

text: the

text: phase-space

text: A

text: Gaussian

text: with

text: covariance

text: matrix

text: A:

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##


text: [9] F. Bayen, M. Flato, C. Fronsdal, A. Lichnerowicz, D. Sternheimer: De- formation theory and quantization I. Deformations of symplectic struc- tures. Ann. Phys. (N.Y.) 111 (1978) 61-110. [10] J. Bellissard, A. van Elst, H. Schulz-Baldes: The non-commutative ge- ometry of the quantum hall eﬀect. J. Math. Phys. 35 (1994) 5373-5451. [11] K. Bolonek, P. Kosinski: On uncertainty relations in noncommutative quantum mechanics. Phys. Lett. B 547 (2002) 51-54. [12] A. Borowiec, A. Pachol: Twisted bialgebroids from a Drinfeld twist. J. Phys. A: Math. Theor. 50 (2017) 055205. [13] A. Bracken, G. Cassinelli, J. Wood: Quantum symmetries and the Weyl- Wigner product of group representations. J. Phys. A: Math. Gen. 36(4) (2003) 1033. [14] S.L. Braunstein, P. van Loock: Quantum information with continuous variables. Rev. Mod. Phys. 77 (2005) 513-577. [15] T. Bro¨cker, R.F. Werner: Mixed states with positive Wigner functions. J. Math. Phys. 36 (1995) 62. [16] A. Cannas da Silva: Lectures on symplectic geometry. Springer (2001). [17] S.M. Carroll, J.A. Harvey, V.A. Kostelecky´, C.D. Lane, T. Okamoto: Noncommutative ﬁeld theory and Lorentz violation. Phys. Rev. Lett. 87 (2001) 141601. [18] S.H.H. Chowdhuri, and S.T. Ali: Wigner functions for noncommuta- tive quantum mechanics: a group representation based construction. J. Math. Phys. 56 (2015) 122102. [19] S.H.H. Chowdhuri, and S.T. Ali: Triply extended group of translations R of 4 as deﬁning group of NCQM: relations to various gauges. J. Phys. A: Math. Theo. 47 (2014) 085301. [20] L. Cohen, P. Loughlin, G. Okopal: Exact and approximate moments of a propagating pulse. J. Mod. Optics 55 (2008) 3349-3358. [21] A. Connes: Noncommutative Geometry. London-New York: Academic Press, 1994. 50

label: reference

text: F.

text: Bayen,

text: M.

text: Flato,

text: C.

text: Fronsdal,

text: A.

text: Lichnerowicz,

text: D.

text: Sternheimer:

text: De-

text: formation

text: theory

text: and

text: quantization

text: I.

text: Deformations

text: of

text: symplectic

text: struc-

text: tures.

text: Ann.

text: Phys.

text: (N.Y.)

text: J.

text: Bellissard,

text: A.

text: van

text: Elst,

text: H.

text: Schulz-Baldes:

text: The

text: non-commutative

text: ge-

text: ometry

text: of

text: the

text: quantum

text: hall

text: eﬀect.

text: J.

text: Math.

text: Phys.

text: K.

text: Bolonek,

text: P.

text: Kosinski:

text: On

text: uncertainty

text: relations

text: in

text: noncommutative

text: quantum

text: mechanics.

text: Phys.

text: Lett.

text: B

text: A.

text: Borowiec,

text: A.

text: Pachol:

text: Twisted

text: bialgebroids

text: from

text: a

text: Drinfeld

text: twist.

text: J.

text: Phys.

text: A:

text: Math.

text: Theor.

text: A.

text: Bracken,

text: G.

text: Cassinelli,

text: J.

text: Wood:

text: Quantum

text: symmetries

text: and

text: the

text: Weyl-

text: Wigner

text: product

text: of

text: group

text: representations.

text: J.

text: Phys.

text: A:

text: Math.

text: Gen.

text: S.L.

text: Braunstein,

text: P.

text: van

text: Loock:

text: Quantum

text: information

text: with

text: continuous

text: variables.

text: Rev.

text: Mod.

text: Phys.

text: T.

text: Bro¨cker,

text: R.F.

text: Werner:

text: Mixed

text: states

text: with

text: positive

text: Wigner

text: functions.

text: J.

text: Math.

text: Phys.

text: A.

text: Cannas

text: da

text: Silva:

text: Lectures

text: on

text: symplectic

text: geometry.

text: Springer

text: S.M.

text: Carroll,

text: J.A.

text: Harvey,

text: V.A.

text: Kostelecky´,

text: C.D.

text: Lane,

text: T.

text: Okamoto:

text: Noncommutative

text: ﬁeld

text: theory

text: and

text: Lorentz

text: violation.

text: Phys.

text: Rev.

text: Lett.

text: S.H.H.

text: Chowdhuri,

text: and

text: S.T.

text: Ali:

text: Wigner

text: functions

text: for

text: noncommuta-

text: tive

text: quantum

text: mechanics:

text: a

text: group

text: representation

text: based

text: construction.

text: J.

text: Math.

text: Phys.

text: S.H.H.

text: Chowdhuri,

text: and

text: S.T.

text: Ali:

text: Triply

text: extended

text: group

text: of

text: translations

text: R

text: of

text: as

text: deﬁning

text: group

text: of

text: NCQM:

text: relations

text: to

text: various

text: gauges.

text: J.

text: Phys.

text: A:

text: Math.

text: Theo.

text: L.

text: Cohen,

text: P.

text: Loughlin,

text: G.

text: Okopal:

text: Exact

text: and

text: approximate

text: moments

text: of

text: a

text: propagating

text: pulse.

text: J.

text: Mod.

text: Optics

text: A.

text: Connes:

text: Noncommutative

text: Geometry.

text: London-New

text: York:

text: Academic

text: Press,


text: [46] L. Ho¨rmander: The analysis of linear partial diﬀerential operators I. Berlin-Heidelberg-New York: Springer-Verlag, 1983. [47] P.A. Horvathy: The noncommutative Landau problem. Ann. Phys. (N.Y.) 299 (2002) 128. [48] R.L. Hudson: When is the Wigner quasi-probability density non- negative? Rep. Math. Phys. 6 (1974) 249-252. [49] A.J.E.M. Janssen: A note on Hudson’s Theorem about functions with nonnegative Wigner distributions. SIAM J. Math. Anal. 15 (1984) 170. [50] A.J.E.M. Janssen: Positivity of weighted Wigner distributions. SIAM J. Math. Anal. 12 (1981) 752. [51] J.-J. Jiang, S.H.H. Chowdhuri: Deformation of noncommutative quan- tum mechanics. Arxiv: math-ph: 1603.05805. ∗ [52] D. Kastler, The C -algebras of a free boson ﬁeld. Commun. Math. Phys. 1 (1965) 14. [53] M. Kontsevich: Deformation quantization of Poisson manifolds. Lett. Math. Phys. 66 (2003) 157. [54] J. Leray: Lagrangian analysis and quantum mechanics. A mathematical structure related to asymptotic expansions and the Maslov index. MIT Press, Cambridge, Mass. (1981). [55] P.L. Lions, T. Paul: Sur les mesures de Wigner. Rev. Mat. Iberoamer. 9 (1993) 553-618. [56] R.G. Littlejohn: The semiclassical evolution of wave packets. Phys. Rep. 138 (1986) 193. [57] P. Loughlin, L. Cohen: Approximate wavefunction from approximate non-representable Wigner distributions. J. Mod. Optics 55 (2008) 3379- 3387. ∗ [58] G. Loupias, S. Miracle-Sole: C -alg`ebres des syst`emes canoniques. Ann. Inst. H. Poincar´e 6 (1967) 39. 53

text: L.

text: Ho¨rmander:

text: The

text: analysis

text: of

text: linear

text: partial

text: diﬀerential

text: operators

text: I.

text: Berlin-Heidelberg-New

text: York:

text: Springer-Verlag,

text: P.A.

text: Horvathy:

text: The

text: noncommutative

text: Landau

text: problem.

text: Ann.

text: Phys.

text: (N.Y.)

text: R.L.

text: Hudson:

text: When

text: is

text: the

text: Wigner

text: quasi-probability

text: density

text: non-

text: negative?

text: Rep.

text: Math.

text: Phys.

text: A.J.E.M.

text: Janssen:

text: A

text: note

text: on

text: Hudson’s

text: Theorem

text: about

text: functions

text: with

text: nonnegative

text: Wigner

text: distributions.

text: SIAM

text: J.

text: Math.

text: Anal.

text: A.J.E.M.

text: Janssen:

text: Positivity

text: of

text: weighted

text: Wigner

text: distributions.

text: SIAM

text: J.

text: Math.

text: Anal.

text: J.-J.

text: Jiang,

text: S.H.H.

text: Chowdhuri:

text: Deformation

text: of

text: noncommutative

text: quan-

text: tum

text: mechanics.

text: Arxiv:

text: math-ph:

text: D.

text: Kastler,

text: The

text: C

text: -algebras

text: of

text: a

text: free

text: boson

text: ﬁeld.

text: Commun.

text: Math.

text: Phys.

text: M.

text: Kontsevich:

text: Deformation

text: quantization

text: of

text: Poisson

text: manifolds.

text: Lett.

text: Math.

text: Phys.

text: J.

text: Leray:

text: Lagrangian

text: analysis

text: and

text: quantum

text: mechanics.

text: A

text: mathematical

text: structure

text: related

text: to

text: asymptotic

text: expansions

text: and

text: the

text: Maslov

text: index.

text: MIT

text: Press,

text: Cambridge,

text: Mass.

text: P.L.

text: Lions,

text: T.

text: Paul:

text: Sur

text: les

text: mesures

text: de

text: Wigner.

text: Rev.

text: Mat.

text: Iberoamer.

text: R.G.

text: Littlejohn:

text: The

text: semiclassical

text: evolution

text: of

text: wave

text: packets.

text: Phys.

text: Rep.

text: P.

text: Loughlin,

text: L.

text: Cohen:

text: Approximate

text: wavefunction

text: from

text: approximate

text: non-representable

text: Wigner

text: distributions.

text: J.

text: Mod.

text: Optics

text: G.

text: Loupias,

text: S.

text: Miracle-Sole:

text: C

text: -alg`ebres

text: des

text: syst`emes

text: canoniques.

text: Ann.

text: Inst.

text: H.

text: Poincar´e


text: Journal of Machine Learning Research 0 (0000) 0-00 Submitted 0/00; Published 0/00 Communication Optimality Trade-oﬀs For Distributed Estimation Anit Kumar Sahu anits@andrew.cmu.edu Department of Electrical and Computer Engineering Carnegie Mellon University Pittsburgh, PA 15213, USA Dusan Jakovetic djakovet@uns.ac.rs Department of Mathematics and Informatics Faculty of Sciences, University of Novi Sad 21000 Novi Sad, Serbia Soummya Kar soummyak@andrew.cmu.edu Department of Electrical and Computer Engineering Carnegie Mellon University Pittsburgh, PA 15213, USA Editor: Abstract This paper proposes Communication eﬃcient REcursive Distributed estimatiOn algo- CREDO CREDO rithm, , for networked multi-worker setups without a central master node. is designed for scenarios in which the worker nodes aim to collaboratively estimate a vec- tor parameter of interest using distributed online time-series data at the individual worker nodes. The individual worker nodes iteratively update their estimate of the parameter by assimilating latest locally sensed information and estimates from neighboring worker nodes exchanged over a (possibly sparse) time-varying communication graph. The underly- ing inter-worker communication protocol is adaptive, making communications increasingly (probabilistically) sparse as time progresses. Under minimal conditions on the inter-worker information exchange network and the sensing models, almost sure convergence of the esti- mate sequences at the worker nodes to the true parameter is established. Further, the paper CREDO characterizes the performance of in terms of asymptotic covariance of the estimate sequences and speciﬁcally establishes the achievability of optimal asymptotic covariance. (cid:16) (cid:17) C The analysis reveals an interesting interplay between the algorithm’s communication cost CREDOt (over t time-steps) and the asymptotic covariance. Most notably, it is shown that (cid:0) (cid:1) C− may be designed to achieve a Θ 2+ζ decay of the mean square error (ζ > 0, arbi- t C− trarily small) at each worker node, which signiﬁcantly improves over the existing Θ 1 CREDtO rates. Simulation examples on both synthetic and real data sets demonstrate ’s communication eﬃciency. Keywords: Distributed Estimation, Statistical Inference, Stochastic Approximation, Networks, Optimization (cid:13) c 0000 Anit Kumar Sahu, Dusan Jakovetic and Soummya Kar. License: CC-BY 4.0, see https://creativecommons.org/licenses/by/4.0/. Attribution requirements are provided at http://jmlr.org/papers/v0/00000.html.

text: Journal

text: of

text: Machine

text: Learning

text: Research

text: Submitted

text: Published

text: Communication

text: Optimality

text: Trade-oﬀs

text: For

text: Distributed

text: Estimation

text: Anit

text: Kumar

text: Sahu

text: anits@andrew.cmu.edu

text: Department

text: of

text: Electrical

text: and

text: Computer

text: Engineering

text: Carnegie

text: Mellon

text: University

text: Pittsburgh,

text: PA

text: USA

text: Dusan

text: Jakovetic

text: djakovet@uns.ac.rs

text: Department

text: of

text: Mathematics

text: and

text: Informatics

text: Faculty

text: of

text: Sciences,

text: University

text: of

text: Novi

text: Sad

text: Novi

text: Sad,

text: Serbia

text: Soummya

text: Kar

text: soummyak@andrew.cmu.edu

text: Department

text: of

text: Electrical

text: and

text: Computer

text: Engineering

text: Carnegie

text: Mellon

text: University

text: Pittsburgh,

text: PA

text: USA

text: Editor:

text: Abstract

text: This

text: paper

text: proposes

text: Communication

text: eﬃcient

text: REcursive

text: Distributed

text: estimatiOn

text: algo-

text: CREDO

text: CREDO

text: rithm,

text: for

text: networked

text: multi-worker

text: setups

text: without

text: a

text: central

text: master

text: node.

text: is

text: designed

text: for

text: scenarios

text: in

text: which

text: the

text: worker

text: nodes

text: aim

text: to

text: collaboratively

text: estimate

text: a

text: vec-

text: tor

text: parameter

text: of

text: interest

text: using

text: distributed

text: online

text: time-series

text: data

text: at

text: the

text: individual

text: worker

text: nodes.

text: The

text: individual

text: worker

text: nodes

text: iteratively

text: update

text: their

text: estimate

text: of

text: the

text: parameter

text: by

text: assimilating

text: latest

text: locally

text: sensed

text: information

text: and

text: estimates

text: from

text: neighboring

text: worker

text: nodes

text: exchanged

text: over

text: a

text: (possibly

text: sparse)

text: time-varying

text: communication

text: graph.

text: The

text: underly-

text: ing

text: inter-worker

text: communication

text: protocol

text: is

text: adaptive,

text: making

text: communications

text: increasingly

text: (probabilistically)

text: sparse

text: as

text: time

text: progresses.

text: Under

text: minimal

text: conditions

text: on

text: the

text: inter-worker

text: information

text: exchange

text: network

text: and

text: the

text: sensing

text: models,

text: almost

text: sure

text: convergence

text: of

text: the

text: esti-

text: mate

text: sequences

text: at

text: the

text: worker

text: nodes

text: to

text: the

text: true

text: parameter

text: is

text: established.

text: Further,

text: the

text: paper

text: CREDO

text: characterizes

text: the

text: performance

text: of

text: in

text: terms

text: of

text: asymptotic

text: covariance

text: of

text: the

text: estimate

text: sequences

text: and

text: speciﬁcally

text: establishes

text: the

text: achievability

text: of

text: optimal

text: asymptotic

text: covariance.

text: C

text: The

text: analysis

text: reveals

text: an

text: interesting

text: interplay

text: between

text: the

text: algorithm’s

text: communication

text: cost

text: CREDOt

text: (over

text: t

text: time-steps)

text: and

text: the

text: asymptotic

text: covariance.

text: Most

text: notably,

text: it

text: is

text: shown

text: that

text: C−

text: may

text: be

text: designed

text: to

text: achieve

text: a

text: Θ

text: decay

text: of

text: the

text: mean

text: square

text: error

text: (ζ

text: arbi-

text: t

text: C−

text: trarily

text: small)

text: at

text: each

text: worker

text: node,

text: which

text: signiﬁcantly

text: improves

text: over

text: the

text: existing

text: Θ

text: CREDtO

text: rates.

text: Simulation

text: examples

text: on

text: both

text: synthetic

text: and

text: real

text: data

text: sets

text: demonstrate

text: ’s

text: communication

text: eﬃciency.

text: Keywords:

text: Distributed

text: Estimation,

text: Statistical

text: Inference,

text: Stochastic

text: Approximation,

text: Networks,

text: Optimization

text: c

text: Anit

text: Kumar

text: Sahu,

text: Dusan

text: Jakovetic

text: and

text: Soummya

text: Kar.

text: License:

text: CC-BY

text: see

text: Attribution

text: requirements

text: are

text: provided

text: at


text: Communication Optimality Trade-offs For Distributed Estimation Benchmark 101 CREDO Benchmark CREDO 101 100 100 0 0.5 1 1.5 2 2.5 3 0 0.5 1 1.5 2 2.5 3 Iterations in log Communication Cost in log10 10 (b) Comparison of Test Error: Communication (a) Comparison of Test Error: Number of cost per node Iterations CREDO Figure 4: Abalone Dataset: Comparison of the and benchmark estimators 100 100 Benchmark Benchmark CREDO CREDO 10-1 10-1 10-2 10-2 0 0.5 1 1.5 2 2.5 3 0 0.5 1 1.5 2 2.5 3 Iterations in log10 Communcation cost in log10 (a) Comparison of Test Error: Number of (b) Comparison of Test Error: Communication Iterations cost per node CREDO Figure 5: Bank Dataset: Comparison of the and benchmark estimators 19 rorrE tseT rorrE tseT rorrE tseT rorrE tseT ##LTFigure## ##LTFigure## ##LTFigure## ##LTFigure## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine##

text: Communication

text: Optimality

text: Trade-offs

text: For

text: Distributed

text: Estimation

text: Benchmark

font: default

text: CREDO

text: Benchmark

text: CREDO

text: Iterations

text: in

text: log

text: Communication

text: Cost

text: in

text: (b)

text: Comparison

label: caption

text: of

text: Test

text: Error:

text: Communication

text: (a)

text: Comparison

text: of

text: Test

text: Error:

text: Number

text: of

text: cost

text: per

text: node

text: Iterations

text: CREDO

text: Figure

text: Abalone

text: Dataset:

text: Comparison

text: of

text: the

text: and

text: benchmark

text: estimators

text: Benchmark

text: Benchmark

text: CREDO

text: CREDO

text: Iterations

text: in

text: Communcation

text: cost

text: in

text: (a)

text: Comparison

text: of

text: Test

text: Error:

text: Number

text: of

text: (b)

text: Comparison

text: of

text: Test

text: Error:

text: Communication

text: Iterations

text: cost

text: per

text: node

text: CREDO

text: Figure

text: Bank

text: Dataset:

text: Comparison

text: of

text: the

text: and

text: benchmark

text: estimators

text: rorrE

text: tseT

text: rorrE

text: tseT

text: rorrE

text: tseT

text: rorrE

text: tseT

text: ##LTFigure##

text: ##LTFigure##

text: ##LTFigure##

text: ##LTFigure##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##


text: Sahu, Jakovetic and Kar {F } where the ﬁltration may be taken to be the natural ﬁltration generated by the t random observations, the random(cid:18)Laplacians i.e., (cid:19) (cid:110) (cid:111) − F { } { } t 1 = σ y (s) N , L(s) , (20) t n n=1 s=0 ≥ which is the σ-algebra induced by the observation processes. For t t , it can be shown 1 that, (cid:16) (cid:17) (cid:0) (cid:1) (cid:62) − ⊗ − − (cid:62) 2 z (t) I β L I α G Σ 1G z(t) NM t M t H H ≤ − (cid:107) (cid:107) (1 c α )2 z(t) 2 . (21) 4 t We use the following inequalities so as to analyze the recursion in (19). (cid:20) (cid:21) (cid:16) (cid:17) (cid:101) (cid:107) (cid:107) z(cid:62)(t(cid:20))(cid:13)(cid:13)E ∗ L(t(cid:16)) ⊗ I 2 z(t) ≤ c5(cid:17)(cid:13)(cid:13)zC(cid:21)⊥ 2 (cid:13) θ M (t +(cid:13)1)τ1+(cid:15) E (cid:16) (cid:0) − (cid:1) − (cid:62) ⊗ ∗ (cid:17) 2 ≤ ∗ G Σ 1 y(t) G 1 θ c θ H H N 6 (cid:0) (cid:1) (cid:62) ⊗ − (cid:62) z (t) β L I + α G Σ 1G z(t) t M t H H ≥ (cid:107) (cid:107) (cid:107) (cid:107) β λ L zC⊥ 2 + c α z(t) 2 . (22) t 2 7 t Using the inequalities derived in (22), we have, (cid:18) (cid:19) E |F ≤ [V (t + 1) ] (1 + c α2(t))V (t) t 8 − − c (cid:107) (cid:107) c β 5 zC⊥ 2 + c α2(t). (23) 9 t (t + 1)τ1+(cid:15) 6 ∃ ∀ ≥ ≥ As c5 goes to zero faster than β , t such that t t , β c5 . By the above (t+1)τ1+(cid:15) ∀ ≥ t 2 2 t (t+1)τ1+(cid:15) construction we obtain t t , 2 (cid:98) E |F ≤ ∗[V (t + 1) ] (1 + α2(t))V (t) + α2, (24) θ (cid:81)t t (cid:98) √ ∞ { } where α(t) = c α . The product (1 + α2) exists for all t. Now let W (t) be such 6 t s=t s that 22 ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine##

text: Sahu,

text: Jakovetic

text: and

text: Kar

text: {F

text: where

text: the

text: ﬁltration

text: may

text: be

text: taken

text: to

text: be

text: the

text: natural

text: ﬁltration

text: generated

text: by

text: the

text: t

text: random

text: observations,

text: the

text: i.e.,

text: F

text: t

text: σ

text: y

text: (s)

text: N

text: L(s)

text: t

text: n

text: which

text: is

text: the

text: σ-algebra

text: induced

text: by

text: the

text: observation

text: processes.

text: For

text: t

text: t

text: it

text: can

text: be

text: shown

text: that,

text: z

text: (t)

text: I

text: β

text: L

text: I

text: α

text: G

text: Σ

text: z(t)

text: NM

text: t

text: M

text: t

text: H

text: H

text: c

text: α

text: z(t)

text: t

text: We

text: use

text: the

text: following

text: inequalities

text: so

text: as

text: to

text: analyze

text: the

text: recursion

text: in

text: I

text: z(t)

text: θ

text: M

text: (t

text: E

text: G

text: Σ

text: y(t)

text: G

text: θ

text: c

text: θ

text: H

text: H

text: N

text: z

text: (t)

text: β

text: L

text: I

text: α

text: G

text: Σ

text: z(t)

text: t

text: M

text: t

text: H

text: H

text: β

text: λ

text: L

text: zC⊥

text: c

text: α

text: z(t)

text: t

text: t

text: Using

text: the

text: inequalities

text: derived

text: in

text: we

text: have,

text: E

text: |F

text: [V

text: (t

text: c

text: (t)

text: t

text: c

text: c

text: β

text: zC⊥

text: c

text: t

text: (t

text: As

text: goes

text: to

text: zero

text: faster

text: than

text: β

text: t

text: such

text: that

text: t

text: t

text: β

text: By

text: the

text: above

text: t

text: t

text: construction

text: we

text: obtain

text: t

text: t

text: E

text: |F

text: ∗[V

text: (t

text: (t)

text: θ

text: t

text: where

text: α(t)

text: c

text: α

text: The

text: product

text: exists

text: for

text: all

text: t.

text: Now

text: let

text: W

text: (t)

text: be

text: such

text: t

text: s=t

text: s

text: that

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##


text: Sahu, Jakovetic and Kar (cid:80) (cid:80) (cid:2) (cid:3) { } R {F } Lemma .8 (Lemma 10 in Dubins and Freedman (1965)). Let J(t) be an -valued - E |F ≥ t+1 adapted process such that [J(t) ]] = 0 a.s. for each t 1. Then the sum ≥ J(t) t E |F t 0 exists and is ﬁnite a.s. on the set where ≥ J(t)2 is ﬁnite. t 0 t Lemma .6 establishes the almost sure convergence of the averaged estimate sequence { } xavg(t) to the true underlying parameter. We now establish the order optimal conver- gence of the estimate sequence in terms of t. { } Proof of Theorem 4.1. We ﬁrst analyze the rate of convergence of the process zδ as de- { } t veloped in Lemma .6 and note that the rate of convergence of the process zδ suﬃces for { } t { } the rate of convergence of the process z . For each δ > 0, recall the process zδ as in ∈ t t (59)-(64). Let τ [0, 1/2) be suc(cid:16)h that, (cid:13) (cid:13) (cid:17) (cid:13) (cid:13) (cid:13) (cid:13) P lim (t + 1)τ zδ = 0 = 1. (74) θ →∞ t t (cid:101) It is to be n(cid:101)oted that such a τ always exists from Lemma .6. We now focus on showing ∈ that there exists τ such that τ < τ < 1/2 for which the assertion holds. Deﬁne τ (τ, 1/2) and µ = 1(τ + τ). Then,(cid:13)for e(cid:13)ach δ > 0, (cid:13) (cid:13) (cid:13) (cid:13) 2 (cid:13) (cid:13) (cid:13) (cid:13) (cid:13) (cid:13) (cid:13) (cid:13) (cid:13) (cid:13) (cid:13) (cid:13) 2 ≤ (cid:107) − (cid:107) 2 2 zδ I α Γ 2 zδ + α2 Uδ t+1 (cid:16) (cid:17) M t t t t (cid:107) (cid:107) + α2 J 2 t t (cid:13) (cid:13)(cid:62)(cid:16) (cid:13) (cid:13) (cid:17) (cid:13) (cid:13) − (cid:13) (cid:13) + 2α (cid:13)zδ (cid:13) (I α Γ) J(cid:13) (cid:13) t t M t t (cid:107) − (cid:107) (cid:107) (cid:107) + 2α Uδ I α Γ zδ + α J . (75) t t M t t t t (cid:13) (cid:13) (cid:0) (cid:1) (cid:13) (cid:13) (cid:0) (cid:1) (cid:13) (cid:13) (cid:13) (cid:13) { } We have that, 1 > τ + 1 + 1, hence the process Uδ may be chosen such that, − 1 2+(cid:15)1 2 − t Uδ = o (t + 1) 1/2 . M(cid:13)ore(cid:13)over, as zδ (cid:13)= o(cid:13) (t +(cid:16)1) τ , we have(cid:17), t (cid:13) (cid:13) t (cid:13) (cid:13) (cid:13) (cid:13) (cid:13) (cid:13) (cid:107) − (cid:107) − − 2α Uδ I α Γ zδ = o (t + 1) 3/2 τ . (76) t t M t t From Assumption M(cid:16)1, we have that, (cid:13) (cid:13)(cid:17) (cid:13) (cid:13) (cid:13) (cid:13) P − − lim (t + 1) 1/2 (cid:15) Jδ = 1, for each (cid:15) > 0, (77) θ →∞ t t and hence we conclude that(cid:13) (cid:13) (cid:16) (cid:17) (cid:13) (cid:13) (cid:13) (cid:13) (cid:107) (cid:107) − − 2α2 Uδ J = o (t + 1) 3/2 τ . (78) t t t 32 ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine##

text: Sahu,

text: Jakovetic

text: and

text: Kar

text: R

text: {F

text: Lemma

text: (Lemma

text: in

text: Dubins

text: and

text: Freedman

text: Let

text: J(t)

text: be

text: an

text: -valued

text: E

text: |F

text: adapted

text: process

text: such

text: that

text: [J(t)

text: a.s.

text: for

text: each

text: t

text: Then

text: the

text: sum

text: J(t)

text: t

text: E

text: |F

text: t

text: exists

text: and

text: is

text: ﬁnite

text: a.s.

text: on

text: the

text: set

text: where

text: is

text: ﬁnite.

text: t

text: t

text: Lemma

text: establishes

text: the

text: almost

text: sure

text: convergence

text: of

text: the

text: averaged

text: estimate

text: sequence

text: xavg(t)

text: to

text: the

text: true

text: underlying

text: parameter.

text: We

text: now

text: establish

text: the

text: order

text: optimal

text: conver-

text: gence

text: of

text: the

text: estimate

text: sequence

text: in

text: terms

text: of

text: t.

text: Proof

text: of

text: Theorem

text: We

text: ﬁrst

text: analyze

text: the

text: rate

text: of

text: convergence

text: of

text: the

text: process

text: zδ

text: as

text: de-

text: t

text: veloped

text: in

text: Lemma

text: and

text: note

text: that

text: the

text: rate

text: of

text: convergence

text: of

text: the

text: process

text: zδ

text: suﬃces

text: for

text: t

text: the

text: rate

text: of

text: convergence

text: of

text: the

text: process

text: z

text: For

text: each

text: δ

text: recall

text: the

text: process

text: zδ

text: as

text: in

text: t

text: t

text: Let

text: τ

text: be

text: that,

text: P

text: lim

text: (t

text: zδ

text: θ

text: t

text: t

text: It

text: is

text: to

text: be

text: that

text: such

text: a

text: τ

text: always

text: exists

text: from

text: Lemma

text: We

text: now

text: focus

text: on

text: showing

text: that

text: there

text: exists

text: τ

text: such

text: that

text: τ

text: τ

text: for

text: which

text: the

text: assertion

text: holds.

text: Deﬁne

text: τ

text: (τ,

text: and

text: µ

text: τ).

text: δ

text: zδ

text: I

text: α

text: Γ

text: zδ

text: α2

text: Uδ

text: M

text: t

text: t

text: t

text: t

text: α2

text: J

text: t

text: t

text: 2α

text: (I

text: α

text: Γ)

text: t

text: t

text: M

text: t

text: t

text: 2α

text: Uδ

text: I

text: α

text: Γ

text: zδ

text: α

text: J

text: t

text: t

text: M

text: t

text: t

text: t

text: t

text: We

text: have

text: that,

text: τ

text: hence

text: the

text: process

text: Uδ

text: may

text: be

text: chosen

text: such

text: that,

text: t

text: Uδ

text: o

text: (t

text: as

text: zδ

text: (t

text: τ

text: we

text: t

text: t

text: 2α

text: Uδ

text: I

text: α

text: Γ

text: zδ

text: o

text: (t

text: τ

text: t

text: t

text: M

text: t

text: t

text: From

text: Assumption

text: we

text: have

text: that,

text: P

text: lim

text: (t

text: Jδ

text: for

text: each

text: θ

text: t

text: t

text: and

text: hence

text: we

text: conclude

text: Uδ

text: J

text: o

text: (t

text: τ

text: t

text: t

text: t

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##


text: Communication Optimality Trade-offs For Distributed Estimation (cid:101) (cid:101) Since, 2µ = τ + τ an(cid:88)d τ < 1/2, we (cid:13)have(cid:13)the following(cid:13)con(cid:13)clusions (cid:13) (cid:13) (cid:13) (cid:13) (cid:13) (cid:13) (cid:13) (cid:13) (cid:88)(t + 1)2µα (cid:13)(cid:13)Uδ (cid:13)(cid:13)(cid:107)I − α Γ(cid:107) zδ < ∞ ≥ t (cid:13) t (cid:13) M t t t 0 (cid:107) (cid:107) ∞ (cid:88)(t + 1)2µαt2 (cid:13)(cid:13)Uδt(cid:13)(cid:13) Jt < ≥ (cid:13) (cid:13) t 0 (cid:88)(t + 1)2µα2 Uδ 2 < ∞ t t ≥ t 0 (cid:107) (cid:107) ∞ (t + 1)2µα2 J 2 < . (79) t t ≥ t 0 { } F With the above development in place, let W δ denote the -adapted sequence given t t+1 by (cid:16) (cid:17) (cid:62) − (cid:2) (cid:12) (cid:3) Wtδ = αt zδt (I αtΓ) Jt, (80) (cid:12) E F where θ Wtδ t =(cid:20)(cid:16)0 an(cid:17)d (cid:12)(cid:12)for (cid:21)t chos(cid:0)en suﬃciently(cid:1) large, we have that, (cid:12) (cid:12) Eθ W(cid:20)tδ 2 Ft =(cid:16) o (cid:17)(t (cid:12)(cid:12)(cid:12)+ 1)(cid:21)−2−2τ(cid:0) (cid:1) (cid:12) ⇒ E(cid:16) (cid:17) 2 F − − (t + 1)4µ W δ = o (t + 1) 2 2τ+4µ θ (cid:101) t t − = o (t + 1) 2+2(cid:104)τ . (cid:0) (cid:1) (cid:12)(cid:12) (cid:105) (81) (cid:101) (cid:12) (cid:80) Since, 2τ < 1, the sequence E (t + 1)4µ W δ 2 F is summable and by Lemma .8, θ t t → → ∞ ≥ (t + 1)2µW δ exists. It may be shown that as α 0 as t , t 0 t t (cid:107) − (cid:107) ≤ − I α Γ 2 1 c α , (82) t 1 t where c = λ (Γ).(cid:13)Then(cid:13), from (75), we(cid:13)hav(cid:13)e, 1 min (cid:13) (cid:13) (cid:13) (cid:13) (cid:13) (cid:13) (cid:13) (cid:13) 2 ≤ − 2 − zδ (1 c α ) zδ + d (t + 1) 2µ, (83) t+1 1 t t t (cid:80) − where the term d (t + 1) 2µ represents all the residual terms in (75). The fact that t ≥ ≥ lim →∞ t d exists and is ﬁnite in conjunction with c α (t+1) 1 2µ (from Assump- t s=0 s 1 t tion M4) brings (83) under the purview of Lemma .7 and yields 33 ##LTLine## ##LTLine## ##LTLine##

text: Communication

text: Optimality

text: Trade-offs

text: For

text: Distributed

text: Estimation

text: Since,

text: 2µ

text: τ

text: τ

text: τ

text: we

text: α

text: zδ

text: t

text: t

text: M

text: t

text: t

text: t

text: Jt

text: t

text: Uδ

text: t

text: t

text: t

text: (t

text: J

text: t

text: t

text: t

text: F

text: With

text: the

text: above

text: development

text: in

text: place,

text: let

text: W

text: δ

text: denote

text: the

text: -adapted

text: sequence

text: given

text: t

text: by

text: Wtδ

text: αt

text: zδt

text: (I

text: αtΓ)

text: Jt,

text: E

text: F

text: where

text: θ

text: Wtδ

text: t

text: large,

text: we

text: have

text: that,

text: Eθ

text: Ft

text: o

text: F

text: (t

text: W

text: δ

text: o

text: (t

text: θ

text: t

text: t

text: o

text: (t

text: Since,

text: 2τ

text: the

text: sequence

text: E

text: (t

text: W

text: δ

text: F

text: is

text: summable

text: and

text: by

text: Lemma

text: θ

text: t

text: t

text: (t

text: δ

text: exists.

text: It

text: may

text: be

text: shown

text: that

text: as

text: α

text: as

text: t

text: t

text: t

text: t

text: I

text: α

text: Γ

text: c

text: α

text: t

text: t

text: where

text: c

text: λ

text: from

text: min

text: zδ

text: c

text: α

text: zδ

text: d

text: (t

text: t

text: t

text: t

text: where

text: the

text: term

text: d

text: (t

text: 2µ

text: represents

text: all

text: the

text: residual

text: terms

text: in

text: The

text: fact

text: that

text: t

text: lim

text: t

text: d

text: exists

text: and

text: is

text: ﬁnite

text: in

text: conjunction

text: with

text: c

text: α

text: 2µ

text: (from

text: Assump-

text: t

text: s

text: t

text: tion

text: brings

text: under

text: the

text: purview

text: of

text: Lemma

text: and

text: yields

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##


text: Sahu, Jakovetic and Kar (cid:13) (cid:13) (cid:13) (cid:13) (cid:13) (cid:13) 2 ∞ lim sup(t + 1)2µ zδ < a.s., (84) →∞ t t (cid:13) (cid:13) (cid:13) (cid:13) which leads to the conclusion that there exists τ with τ < τ < µ, such that (t + → → ∞ 1)τ zδ 0 as t . The fact that the above development holds for all δ > 0, we t (cid:107) (cid:107) → → ∞ conclude that (t + 1)τ z (cid:16) 0 as t . Hence, for every(cid:17)τ for which t P (cid:107) − (cid:107) θ l→im∞(t + 1)τ xavg(t) θ = 0 = 1 (85) t ∈ holds, then there exists τ (τ, 1/2) for which the convergence continues to hold. Finally, an application of indu(cid:16)ction yields the result (cid:17) P (cid:107) − (cid:107) ∀ ∈ θ l→im∞(t + 1)τ xavg(t) θ = 0 = 1, τ [0, 1/2) (86) t The above result in conjunction with Lemma .5 and the usage of triangle inequality ∀ ∈ yields τ [0, 1/2) (cid:13) (cid:13) (cid:13) (cid:13) (cid:13) (cid:13) (cid:13) (cid:13) (cid:107) − (cid:107) ≤ − (t + 1)τ xn(t) θ (t + 1)τ xavg(t) θ − + (t + 1)τ xn(t) xavg(t) ⇒ (cid:107) − (cid:107) lim (t + 1)τ x (t) θ = 0 a.s. (87) →∞ n t Proof of Theorem 4.2. Proceeding as in(cid:0)proof of Lemma(cid:1).2, we have, for t large enough E |F ≤ − [V (t + 1) ] 1 2c α + c α2 V (t) + c α2 θ t 4 t 7 t 6 t ≤ V (t) + c α2, (88) 6 t − as for t large enough, c4α(cid:16)t +c(cid:0)7αt2 < 0(cid:1). Before proceedin(cid:17)g further, we note that, from (21), (cid:18) (cid:19) (cid:62) ⊗ (cid:0) (cid:1) − (cid:62) x β L I + α G Σ 1G x t M t H H (cid:62) (cid:16) β ⊗ − (cid:17) (cid:62) = α x t L I + G Σ 1G x t α M H H t ≥ (cid:62) ⊗ − (cid:62) ≥ α x (L I ) + G Σ 1G x c α , (89) t M H H 4 t 34 ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine##

text: Sahu,

text: Jakovetic

text: and

text: Kar

text: lim

text: sup(t

text: zδ

text: a.s.,

text: t

text: t

text: which

text: leads

text: to

text: the

text: conclusion

text: that

text: there

text: exists

text: τ

text: with

text: τ

text: τ

text: µ,

text: such

text: that

text: (t

text: zδ

text: as

text: t

text: The

text: fact

text: that

text: the

text: above

text: development

text: holds

text: for

text: all

text: δ

text: we

text: t

text: conclude

text: that

text: (t

text: z

text: as

text: t

text: Hence,

text: for

text: for

text: which

text: t

text: P

text: θ

text: l→im∞(t

text: xavg(t)

text: θ

text: t

text: holds,

text: then

text: there

text: exists

text: τ

text: (τ,

text: for

text: which

text: the

text: convergence

text: continues

text: to

text: hold.

text: Finally,

text: an

text: application

text: of

text: yields

text: the

text: result

text: P

text: θ

text: l→im∞(t

text: xavg(t)

text: θ

text: τ

text: t

text: The

text: above

text: result

text: in

text: conjunction

text: with

text: Lemma

text: and

text: the

text: usage

text: of

text: triangle

text: inequality

text: yields

text: τ

text: (t

text: xn(t)

text: θ

text: (t

text: xavg(t)

text: θ

text: (t

text: xn(t)

text: xavg(t)

text: lim

text: (t

text: x

text: (t)

text: θ

text: a.s.

text: n

text: t

text: Proof

text: of

text: Theorem

text: Proceeding

text: as

text: of

text: we

text: have,

text: for

text: t

text: large

text: enough

text: E

text: |F

text: [V

text: (t

text: α

text: c

text: α2

text: V

text: (t)

text: c

text: α2

text: θ

text: t

text: t

text: t

text: t

text: V

text: (t)

text: c

text: t

text: as

text: for

text: t

text: large

text: enough,

text: Before

text: further,

text: we

text: note

text: that,

text: from

text: x

text: β

text: L

text: I

text: α

text: G

text: Σ

text: x

text: t

text: M

text: t

text: H

text: H

text: β

text: α

text: x

text: t

text: L

text: I

text: G

text: Σ

text: x

text: t

text: α

text: M

text: H

text: H

text: t

text: α

text: x

text: (L

text: I

text: G

text: Σ

text: x

text: c

text: α

text: t

text: M

text: H

text: H

text: t

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##


text: Communication Optimality Trade-offs For Distributed Estimation where (cid:16)(cid:0) (cid:1) (cid:17) ⊗ − (cid:62) c = λ L I + G Σ 1G . (90) 4 min M H H Thus, we have th(cid:13)(cid:13)at (cid:0) (cid:1) (cid:13)(cid:13) (cid:13) (cid:13) − ⊗ − − (cid:62) ≤ − I β L I α G Σ 1G 1 c α , (91) NM t M t H H 4 t ≥ {F } for all t t , where t is chosen to be appropriately large. Now, consider the - 1{ } 1 t adapted process V (t) deﬁned as follows 1 (cid:88) ∞ V (t) = V (t)(cid:88)+ c α2 1 6 s s=t ∞ − = V (t) + c (t + 1) 2, (92) 8 s=t { − } for appropriately chosen positive constant c .Since, (t + 1) 2 is summable, the process { } 8 { } V (t) is bounded from above. Moreover, it also follows that V (t) ≥ is a supermartin- 1 1 t t1 gale and hence converges a.s. to a ﬁnite random variable. By deﬁnition from (29), we also { } ∗ have that V (t) converges to a non-negative ﬁnite random variable V . Finally, from (88), we have that, E ≤ − E − [V (t + 1)] (1 c α ) [V (t)] + c (t + 1) 2 θ 4 t θ 8 ⇒ ≤ − E − E [V (t + 1)] (1 c α ) [V (t)] + c α (t + 1) 1 (93) θ 4 t θ 10 t ≥ { } for t t . The summability of α in conjunction with assumption M4 ensures that the 1{ } t sequence V (t) then falls under the purview of Lemma .7, and we have (cid:18) (cid:19) E ∞ lim sup(t + 1) [V (t + 1)] < →∞ θ t ⇒ E 1 [V (t)] = O . (94) θ t Furthermore, from (92), we also have that (cid:18) (cid:19) E ≤ E c6π2 [V (t)] [V (t)] + θ 1 θ 6 ⇒ E (cid:107) − (cid:107) 1 [ x (t) θ 2] = O . (95) θ n t 35 ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine##

text: Communication

text: Optimality

text: Trade-offs

text: For

text: Distributed

text: Estimation

text: where

text: c

text: λ

text: L

text: I

text: G

text: Σ

text: min

text: M

text: H

text: H

text: Thus,

text: we

text: have

text: I

text: β

text: L

text: I

text: α

text: G

text: Σ

text: c

text: α

text: NM

text: t

text: M

text: t

text: H

text: H

text: t

text: {F

text: for

text: all

text: t

text: t

text: where

text: t

text: is

text: chosen

text: to

text: be

text: appropriately

text: large.

text: Now,

text: consider

text: the

text: t

text: adapted

text: process

text: V

text: (t)

text: deﬁned

text: as

text: follows

text: V

text: (t)

text: V

text: c

text: α2

text: s

text: s=t

text: V

text: (t)

text: c

text: (t

text: s=t

text: for

text: appropriately

text: chosen

text: positive

text: constant

text: c

text: .Since,

text: (t

text: is

text: summable,

text: the

text: process

text: V

text: (t)

text: is

text: bounded

text: from

text: above.

text: Moreover,

text: it

text: also

text: follows

text: that

text: V

text: (t)

text: is

text: a

text: supermartin-

text: t

text: gale

text: and

text: hence

text: converges

text: a.s.

text: to

text: a

text: ﬁnite

text: random

text: variable.

text: By

text: deﬁnition

text: from

text: we

text: also

text: have

text: that

text: V

text: (t)

text: converges

text: to

text: a

text: non-negative

text: ﬁnite

text: random

text: variable

text: V

text: Finally,

text: from

text: we

text: have

text: that,

text: E

text: E

text: [V

text: (t

text: c

text: α

text: [V

text: (t)]

text: c

text: (t

text: θ

text: t

text: θ

text: E

text: E

text: [V

text: (t

text: c

text: α

text: [V

text: (t)]

text: c

text: α

text: (t

text: θ

text: t

text: θ

text: t

text: for

text: t

text: t

text: The

text: summability

text: of

text: α

text: in

text: conjunction

text: with

text: assumption

text: ensures

text: that

text: the

text: t

text: sequence

text: V

text: (t)

text: then

text: falls

text: under

text: the

text: purview

text: of

text: Lemma

text: and

text: we

text: have

text: E

text: lim

text: sup(t

text: [V

text: (t

text: θ

text: t

text: E

text: [V

text: (t)]

text: O

text: θ

text: t

text: Furthermore,

text: from

text: we

text: also

text: have

text: that

text: E

text: E

text: [V

text: (t)]

text: [V

text: (t)]

text: θ

text: θ

text: E

text: x

text: (t)

text: θ

text: O

text: θ

text: n

text: t

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##


text: Sahu, Jakovetic and Kar (cid:0) (cid:1) − − (cid:62) aI Γ I 1 PMP = + 2a . (104) 2N 4N { } { } Now from Lemma .5, we have that the processes x (t) and x (t) are indistinguishable n avg in the (t + 1)1/2 tim(cid:16)e scal(cid:13)e, which is formalized as follows: (cid:13) (cid:17) (cid:13) (cid:13) √ √ P (cid:16) (cid:13)(cid:13) − − (cid:13)(cid:13) (cid:17) − lim t + 1 (x (t) θ) t + 1 (x (t) θ) = 0 θ →∞ n avg t √ P − = lim t + 1 (x (t) x (t)) = 0 = 1. (105) θ →∞ (cid:8) n avg (cid:9) (cid:8) (cid:9) t √ √ − − Thus, the diﬀerence of the sequences t + 1 (x (t) θ) and t + 1 (x (t) θ) con- → ∞ n avg verges a.s. to zero as t and hence we hav(cid:32)e, (cid:0) (cid:1) (cid:33) √ − − − D⇒ N aI Γ I 1 t + 1 (x (t) θ) = 0, + 2a . (106) n 2N 4N References Delve datasets. http://www.cs.toronto.edu/~delve/data/datasets.html. Libsvm regression datasets. https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/ datasets/regression.html. Dragana Bajovi´c, Jos´e M. F Moura, Jo˜ao Xavier, and Bruno Sinopoli. Distributed in- ference over directed networks: Performance limits and optimal design. arXiv preprint arXiv:1504.07526, 2015. V. S. Borkar. Stochastic Approximation: A Dynamical Systems Viewpoint. Cambridge University Press, Cambridge, UK, 2008. Paolo Braca, Stefano Marano, and Vincenzo Matta. Enforcing consensus while monitoring the environment in wireless sensor networks. IEEE Transactions on Signal Processing, 56(7):3375–3380, 2008. Federico S Cattivelli and Ali H Sayed. Diﬀusion lms strategies for distributed estimation. IEEE Transactions on Signal Processing, 58(3):1035–1048, 2010. Jie Chen, C´edric Richard, and Ali H Sayed. Multitask diﬀusion adaptation over networks. IEEE Transactions on Signal Processing, 62(16):4129–4144, 2014. 38 ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine##

text: Sahu,

text: Jakovetic

text: and

text: Kar

text: aI

text: Γ

text: I

text: PMP

text: Now

text: from

text: Lemma

text: we

text: have

text: that

text: the

text: processes

text: x

text: (t)

text: and

text: x

text: (t)

text: are

text: indistinguishable

text: n

text: avg

text: in

text: the

text: (t

text: which

text: is

text: formalized

text: as

text: follows:

text: P

text: lim

text: t

text: (x

text: (t)

text: θ)

text: t

text: (x

text: (t)

text: θ)

text: θ

text: n

text: avg

text: t

text: P

text: lim

text: t

text: (x

text: (t)

text: x

text: (t))

text: θ

text: n

text: avg

text: t

text: Thus,

text: the

text: diﬀerence

text: of

text: the

text: sequences

text: t

text: (x

text: (t)

text: θ)

text: and

text: t

text: (x

text: (t)

text: θ)

text: con-

text: n

text: avg

text: verges

text: a.s.

text: to

text: zero

text: as

text: t

text: and

text: hence

text: we

text: D⇒

text: N

text: aI

text: Γ

text: I

text: t

text: (x

text: (t)

text: θ)

text: n

text: References

text: Delve

text: datasets.

text: Libsvm

text: regression

text: datasets.

text: datasets/regression.html.

text: Dragana

text: Bajovi´c,

text: Jos´e

text: M.

text: F

text: Moura,

text: Jo˜ao

text: Xavier,

text: and

text: Bruno

text: Sinopoli.

text: Distributed

text: in-

text: ference

text: over

text: directed

text: networks:

text: Performance

text: limits

text: and

text: optimal

text: design.

text: arXiv

text: preprint

text: V.

text: S.

text: Borkar.

text: Stochastic

text: Approximation:

text: A

text: Dynamical

text: Systems

text: Viewpoint.

text: Cambridge

text: University

text: Press,

text: Cambridge,

text: UK,

text: Paolo

text: Braca,

text: Stefano

text: Marano,

text: and

text: Vincenzo

text: Matta.

text: Enforcing

text: consensus

text: while

text: monitoring

text: the

text: environment

text: in

text: wireless

text: sensor

text: networks.

text: IEEE

text: Transactions

text: on

text: Signal

text: Processing,

text: Federico

text: S

text: Cattivelli

text: and

text: Ali

text: H

text: Sayed.

text: Diﬀusion

text: lms

text: strategies

text: for

text: distributed

text: estimation.

text: IEEE

text: Transactions

text: on

text: Signal

text: Processing,

text: Jie

text: Chen,

text: C´edric

text: Richard,

text: and

text: Ali

text: H

text: Sayed.

text: Multitask

text: diﬀusion

text: adaptation

text: over

text: networks.

text: IEEE

text: Transactions

text: on

text: Signal

text: Processing,

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##


text: Sahu, Jakovetic and Kar Qiang Liu and Alexander T Ihler. Distributed estimation, information loss and exponential families. In Advances in Neural Information Processing Systems, pages 1098–1106, 2014. C. G. Lopes and A. H. Sayed. Diﬀusion least-mean squares over adaptive networks: For- mulation and performance analysis. IEEE Transactions on Signal Processing, 56(7): 3122–3136, July 2008. Chenxin Ma and Martin Tak´aˇc. Partitioning data on features or samples in communication- eﬃcient distributed optimization? arXiv preprint arXiv:1510.06688, 2015. Chenxin Ma, Virginia Smith, Martin Jaggi, Michael Jordan, Peter Richtarik, and Martin Takac. Adding vs. averaging in distributed primal-dual optimization. In International Conference on Machine Learning, pages 1973–1982, 2015. A. Nedic and A. Ozdaglar. Distributed subgradient methods for multi-agent optimization. IEEE Transactions on Automatic Control, 54(1):48 61, Jan. 2009. Angelia Nedi´c, Alex Olshevsky, and C´esar A Uribe. Nonasymptotic convergence rates for cooperative learning over time-varying directed graphs. arXiv preprint arXiv:1410.1977, 2014. M.B. Nevel’son and R.Z. Has’minskii. Stochastic Approximation and Recursive Estimation. American Mathematical Society, Providence, Rhode Island, 1973. S. S. Ram, A. Nedic, and V. V. Veeravalli. Incremental stochastic subgradient algorithms for convex optimization. SIAM Journal on Optimization, 20(2):691–717, June 2009. S.S. Ram, A. Nedi´c, and V.V. Veeravalli. Distributed stochastic subgradient projection algorithms for convex optimization. Journal of optimization theory and applications, 147 (3):516–545, 2010a. S.S. Ram, V.V. Veeravalli, and A. Nedic. Distributed and recursive parameter estimation in parametrized linear state-space models. to appear in IEEE Transactions on Automatic Control, 55(2):488– 492, February 2010b. Benjamin Recht, Christopher Re, Stephen Wright, and Feng Niu. Hogwild: A lock-free approach to parallelizing stochastic gradient descent. In Advances in neural information processing systems, pages 693–701, 2011. Anit Kumar Sahu and Soummya Kar. Distributed sequential detection for Gaussian shift- in-mean hypothesis testing. IEEE Transactions on Signal Processing, 64(1):89–103, 2016. Anit Kumar Sahu and Soummya Kar. Recursive distributed detection for composite hypoth- esis testing: Nonlinear observation models in additive gaussian noise. IEEE Transactions on Information Theory, 63(8):4797–4828, 2017. Konstantinos Tsianos, Sean Lawlor, and Michael G Rabbat. Communication/computation tradeoﬀs in consensus-based distributed optimization. In Advances in neural information processing systems, pages 1943–1951, 2012. 40

text: Sahu,

text: Jakovetic

text: and

text: Kar

text: Qiang

text: Liu

text: and

text: Alexander

text: T

text: Ihler.

text: Distributed

text: estimation,

text: information

text: loss

text: and

text: exponential

text: families.

text: In

text: Advances

text: in

text: Neural

text: Information

text: Processing

text: Systems,

text: pages

text: C.

text: G.

text: Lopes

text: and

text: A.

text: H.

text: Sayed.

text: Diﬀusion

text: least-mean

text: squares

text: over

text: adaptive

text: networks:

text: For-

text: mulation

text: and

text: performance

text: analysis.

text: IEEE

text: Transactions

text: on

text: Signal

text: Processing,

text: July

text: Chenxin

text: Ma

text: and

text: Martin

text: Tak´aˇc.

text: Partitioning

text: data

text: on

text: features

text: or

text: samples

text: in

text: communication-

text: eﬃcient

text: distributed

text: optimization?

text: arXiv

text: preprint

text: Chenxin

text: Ma,

text: Virginia

text: Smith,

text: Martin

text: Jaggi,

text: Michael

text: Jordan,

text: Peter

text: Richtarik,

text: and

text: Martin

text: Takac.

text: Adding

text: vs.

text: averaging

text: in

text: distributed

text: primal-dual

text: optimization.

text: In

text: International

text: Conference

text: on

text: Machine

text: Learning,

text: pages

text: A.

text: Nedic

text: and

text: A.

text: Ozdaglar.

text: Distributed

text: subgradient

text: methods

text: for

text: multi-agent

text: optimization.

text: IEEE

text: Transactions

text: on

text: Automatic

text: Control,

text: Jan.

text: Angelia

text: Nedi´c,

text: Alex

text: Olshevsky,

text: and

text: C´esar

text: A

text: Uribe.

text: Nonasymptotic

text: convergence

text: rates

text: for

text: cooperative

text: learning

text: over

text: time-varying

text: directed

text: graphs.

text: arXiv

text: preprint

text: M.B.

text: Nevel’son

text: and

text: R.Z.

text: Has’minskii.

text: Stochastic

text: Approximation

text: and

text: Recursive

text: Estimation.

text: American

text: Mathematical

text: Society,

text: Providence,

text: Rhode

text: Island,

text: S.

text: S.

text: Ram,

text: A.

text: Nedic,

text: and

text: V.

text: V.

text: Veeravalli.

text: Incremental

text: stochastic

text: subgradient

text: algorithms

text: for

text: convex

text: optimization.

text: SIAM

text: Journal

text: on

text: Optimization,

text: June

text: S.S.

text: Ram,

text: A.

text: Nedi´c,

text: and

text: V.V.

text: Veeravalli.

text: Distributed

text: stochastic

text: subgradient

text: projection

text: algorithms

text: for

text: convex

text: optimization.

text: Journal

text: of

text: optimization

text: theory

text: and

text: applications,

text: S.S.

text: Ram,

text: V.V.

text: Veeravalli,

text: and

text: A.

text: Nedic.

text: Distributed

text: and

text: recursive

text: parameter

text: estimation

text: in

text: parametrized

text: linear

text: state-space

text: models.

text: to

text: appear

text: in

text: IEEE

text: Transactions

text: on

text: Automatic

text: Control,

text: February

text: Benjamin

text: Recht,

text: Christopher

text: Re,

text: Stephen

text: Wright,

text: and

text: Feng

text: Niu.

text: Hogwild:

text: A

text: lock-free

text: approach

text: to

text: parallelizing

text: stochastic

text: gradient

text: descent.

text: In

text: Advances

text: in

text: neural

text: information

text: processing

text: systems,

text: pages

text: Anit

text: Kumar

text: Sahu

text: and

text: Soummya

text: Kar.

text: Distributed

text: sequential

text: detection

text: for

text: Gaussian

text: shift-

text: in-mean

text: hypothesis

text: testing.

text: IEEE

text: Transactions

text: on

text: Signal

text: Processing,

text: Anit

text: Kumar

text: Sahu

text: and

text: Soummya

text: Kar.

text: Recursive

text: distributed

text: detection

text: for

text: composite

text: hypoth-

text: esis

text: testing:

text: Nonlinear

text: observation

text: models

text: in

text: additive

text: gaussian

text: noise.

text: IEEE

text: Transactions

text: on

text: Information

text: Theory,

text: Konstantinos

text: Tsianos,

text: Sean

text: Lawlor,

text: and

text: Michael

text: G

text: Rabbat.

text: Communication/computation

text: tradeoﬀs

text: in

text: consensus-based

text: distributed

text: optimization.

text: In

text: Advances

text: in

text: neural

text: information

text: processing

text: systems,

text: pages


text: Sahu, Jakovetic and Kar tion cost has not been addressed in the aforementioned references. The well studied class of distributed estimation algorithms in the consensus+innovations framework Kar and Moura (2011); Kar et al. (2013) characterize the algorithm parameters, under which estimate sequences optimal in the sense of asymptotic covariance can be ob- tained. However, the inter-agent message passing and the associated communication cost is not taken into account in the aforementioned algorithms. The lack of exploration into the dimension of communication cost in the context of distributed estimation algorithms in the consensus+innovations framework motivated us to develop a stochastic communication protocol in this paper, that exploits the redundancy in inter-agent message passing while not compromising on the optimality aspect of the estimate sequence. Hence, in order to test the eﬃcacy of our stochastic message-passing protocol, we take the distributed estimation algorithm proposed in Kar and Moura (2011); Kar et al. (2013) as the primary benchmark. 2.2 Preliminaries: Oracle and Distributed Estimation In this section we go over the preliminaries of oracle and distributed estimation. Oracle Estimation: In the setup described above in (1), if a hypothetical oracle node having access to the data samples at all the nodes at all times were to conduct the parameter estimation in an iterative manner, it would do so in the following way: (cid:88) x (t + 1) = x (t) c c N (cid:124) a (cid:62) − (cid:123)(cid:122) − (cid:125) + H Σ 1 (y (t) H x (t)), t + 1 n n n n c n=1 Global Innovation where a is a positive constant. It is well known from standard stochastic approximation { } results (see, for example, Nevel’son and Has’minskii (1973)) that the sequence x (t) gen- c erated from the update above converges almost surely to the true parameter θ. Moreover, { } (cid:16) (cid:17) the sequence x (t) is asymptotically normal, i.e, c √ − D⇒ N − (cid:80) t + 1 (x (t) θ) = 0, (NΓ) 1 , c (cid:62) − where Γ = 1 N H Σ 1H . The above established asymptotic normality also points to N n=1 n n n the conclusion that the mean square error (MSE) decays as Θ(1/t). However, such an oracle based scheme may not be implementable in our distributed multi- worker setting with time-varying sparse inter-worker interaction primarily due to the fact that the desired global innovation computation requires instantaneous access to the entire set of network sensed data at all times at the oracle. Distributed Estimation: Distributed estimation scenarios where the global model information is not available at each worker, makes it necessary to communicate at a properly crafted rate. An aptly chosen communication rate would then ensure information ﬂow among the worker nodes so that every worker is able to estimate the parameter of interest. If in the case of a distributed setup, a worker n in the network were to replicate the centralized update by replacing the global innovation in accordance with its local innovation (i.e., based on its local sensed data 6 ##LTLine## ##LTLine## ##LTLine##

text: Sahu,

text: Jakovetic

text: and

text: Kar

text: tion

text: cost

text: has

text: not

text: been

text: addressed

text: in

text: the

text: aforementioned

text: references.

text: The

text: well

text: studied

text: class

text: of

text: distributed

text: estimation

text: algorithms

text: in

text: the

text: consensus+innovations

text: framework

text: Kar

text: and

text: Moura

text: Kar

text: et

text: al.

text: characterize

text: the

text: algorithm

text: parameters,

text: under

text: which

text: estimate

text: sequences

text: optimal

text: in

text: the

text: sense

text: of

text: asymptotic

text: covariance

text: can

text: be

text: ob-

text: tained.

text: However,

text: the

text: inter-agent

text: message

text: passing

text: and

text: the

text: associated

text: communication

text: cost

text: is

text: not

text: taken

text: into

text: account

text: in

text: the

text: aforementioned

text: algorithms.

text: The

text: lack

text: of

text: exploration

text: into

text: the

text: dimension

text: of

text: communication

text: cost

text: in

text: the

text: context

text: of

text: distributed

text: estimation

text: algorithms

text: in

text: the

text: consensus+innovations

text: framework

text: motivated

text: us

text: to

text: develop

text: a

text: stochastic

text: communication

text: protocol

text: in

text: this

text: paper,

text: that

text: exploits

text: the

text: redundancy

text: in

text: inter-agent

text: message

text: passing

text: while

text: not

text: compromising

text: on

text: the

text: optimality

text: aspect

text: of

text: the

text: estimate

text: sequence.

text: Hence,

text: in

text: order

text: to

text: test

text: the

text: eﬃcacy

text: of

text: our

text: stochastic

text: message-passing

text: protocol,

text: we

text: take

text: the

text: distributed

text: estimation

text: algorithm

text: proposed

text: in

text: Kar

text: and

text: Moura

text: Kar

text: et

text: al.

text: as

text: the

text: primary

text: benchmark.

text: Preliminaries:

text: Oracle

text: and

text: Distributed

text: Estimation

text: In

text: this

text: section

text: we

text: go

text: over

text: the

text: preliminaries

text: of

text: oracle

text: and

text: distributed

text: estimation.

text: Oracle

text: Estimation:

text: In

text: the

text: setup

text: described

text: above

text: in

text: if

text: a

text: hypothetical

text: oracle

text: node

text: having

text: access

text: to

text: the

text: data

text: samples

text: at

text: all

text: the

text: nodes

text: at

text: all

text: times

text: were

text: to

text: conduct

text: the

text: parameter

text: estimation

text: in

text: an

text: iterative

text: manner,

text: it

text: would

text: do

text: so

text: in

text: the

text: following

text: way:

text: x

text: (t

text: x

text: (t)

text: c

text: c

text: N

text: a

text: H

text: Σ

text: (y

text: (t)

text: H

text: x

text: (t)),

text: t

text: n

text: n

text: n

text: n

text: c

text: Global

text: Innovation

text: where

text: a

text: is

text: a

text: positive

text: constant.

text: It

text: is

text: well

text: known

text: from

text: standard

text: stochastic

text: approximation

text: results

text: (see,

text: for

text: example,

text: Nevel’son

text: and

text: Has’minskii

text: that

text: the

text: sequence

text: x

text: (t)

text: gen-

text: c

text: erated

text: from

text: the

text: update

text: above

text: converges

text: almost

text: surely

text: to

text: the

text: true

text: parameter

text: θ.

text: Moreover,

text: the

text: sequence

text: x

text: (t)

text: is

text: asymptotically

text: normal,

text: i.e,

text: c

text: D⇒

text: N

text: t

text: (x

text: (t)

text: θ)

text: (NΓ)

text: c

text: where

text: Γ

text: N

text: H

text: Σ

text: The

text: above

text: established

text: asymptotic

text: normality

text: also

text: points

text: to

text: N

text: n

text: n

text: n

text: the

text: conclusion

text: that

text: the

text: mean

text: square

text: error

text: (MSE)

text: decays

text: as

text: However,

text: such

text: an

text: oracle

text: based

text: scheme

text: may

text: not

text: be

text: implementable

text: in

text: our

text: distributed

text: multi-

text: worker

text: setting

text: with

text: time-varying

text: sparse

text: inter-worker

text: interaction

text: primarily

text: due

text: to

text: the

text: fact

text: that

text: the

text: desired

text: global

text: innovation

text: computation

text: requires

text: instantaneous

text: access

text: to

text: the

text: entire

text: set

text: of

text: network

text: sensed

text: data

text: at

text: all

text: times

text: at

text: the

text: oracle.

text: Distributed

text: Estimation:

text: Distributed

text: estimation

text: scenarios

text: where

text: the

text: global

text: model

text: information

text: is

text: not

text: available

text: at

text: each

text: worker,

text: makes

text: it

text: necessary

text: to

text: communicate

text: at

text: a

text: properly

text: crafted

text: rate.

text: An

text: aptly

text: chosen

text: communication

text: rate

text: would

text: then

text: ensure

text: information

text: ﬂow

text: among

text: the

text: worker

text: nodes

text: so

text: that

text: every

text: worker

text: is

text: able

text: to

text: estimate

text: the

text: parameter

text: of

text: interest.

text: If

text: in

text: the

text: case

text: of

text: a

text: distributed

text: setup,

text: a

text: worker

text: n

text: in

text: the

text: network

text: were

text: to

text: replicate

text: the

text: centralized

text: update

text: by

text: replacing

text: the

text: global

text: innovation

text: in

text: accordance

text: with

text: its

text: local

text: innovation

text: (i.e.,

text: based

text: on

text: its

text: local

text: sensed

text: data

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##