text: )A˚ ( F )A˚ ( F λ λ A&A proofs: manuscript no. ﬁnal_dif − ± calculation, and 0.16 0.11 in−NLTE. The λ4934 Å line 1.1 leads to an NLTE abundance of 0.09 for fodd = 0.46 and 0.21 for fodd = 0.18. This rules out a low fodd. Our analysis FeI FeI FeI is also in agreement with the results of Hayek et al. (2009), − 1.0 FeI BaII who determined log(cid:15)(Ba) = 0.14. ii) Since the atmosphere of very metal-poor stars can be sub- 0.9 BaII ject to deviation from LTE, we performed NLTE calcula- tions, following the procedure of Mashonkina et al. (1999). 0.8 BaII It turns out that the departure from LTE is very small for the Ba ii 4934 Å line, w−ith a NLTE abundance correction 0.7 o−f ∆NLTE = log εNLTE log εLTE = 0.00 for ET0381 and 0.01 dex for scl_03_059 (Table 6). As to the subordinate lines, the NLTE calculation leads to higher abundances by 0.6 0.05 dex to 0.1 dex, resulting in even larger discrepancies ET0381 with the Ba ii 4934 Å line. 0.5 iii) Another source of uncertainty arises from the atmo- 4933.7 4934.2 6141.3 6142.1 6496.5 6497.0 λ (A˚ ) spheric parameters. The consequence of a variation of each of T , log g, or v has been estimated (Table 6). The “to- eﬀ t tal” quantities correspond to the total impact of varying stel- 1.1 lar parameters, computed as the quadratic sum of the three FeI BaII FeI sources of uncertainties. For ET0381, these uncertainties 1.0 BaII produce very similar abundance shifts for the Ba ii 4934 Å 0.9 BaII and the subordinate lines, and, therefore cannot explain the diﬀerence between them. For scl_03_059, the abundance de- 0.8 BaII rived from the Ba ii 4934 Å line is slightly more aﬀected than the subordinate lines, however, not large enough yet to ex- 0.7 plain the observed discrepancy. iv) We did not use any 3D hydrodynamical model atmo- 0.6 spheres to derive the abundances of barium. This element is 0.5 only detected in the Ba ii majority species, and the detected lines arise either from the ground or low-excitation levels. scl 03 059 0.4 Dobrovolskas et al. (2013) predicted that the (3D-1D) abun- dance corrections for metal-poor giant stars are small for the − − 0.3 4933.7 4934.2 5853.5 5854λ.0(A˚61)41.3 6142.1 6496.5 6497.0 B0.a0idi eExe,xrces=pe0ctiavnedly2, ienVthelin5e0s2:0(/32D.5/−12Dm) o=del0an.0d5(3dDex−1aDnd) − − − Fig. 4. Synthesis of the Ba lines for ET0381 (upper panel) and = 0.10 dex and 0.02 dex in the 5020/2.5/ 3 model. The scl_03_059 (lower panel). A value of fodd = 0.18 is taken for the Baii Ba ii 5853, 6141, 6496 Å lines arise from the Eexc= 0.6-0.7 4934Å. The observed spectra are shown in black. The red continuous, eV level. Interpolating between Eexc = 0 and 2 eV in Dobro- green dashed, and green dotted curves correspond to the theoretical volskas et al. (2013), one sees that the diﬀerence between spectra of the full blend, pure Baii lines, and Fei blending lines, re- the Ba ii 4934 Å and the Ba ii subordinate lines would not spectively. be reduced since the (3D-1D) abundance correction is more negative for the Ba ii 4934 Å than for the other lines. It turns out that none of the above plausible sources of errors possible fractions, fodd = 0.46 (Travaglio et al. 1999), 0.522 is suﬃcient to explain the diﬀerence between the subordinate (Sneden et al. 1996), 0.601 (Bisterzo et al. 2014), and 0.72 and the Ba ii 4934 Å lines. Figure 4 displays the result of our (McWilliam 1998). We ﬁrst considered f = 0.46 and 0.72 to derive the Ba ii NLTE synthesis assuming fodd = 0.18 for the Ba II 4934 Å odd line. All Ba lines, which have very diﬀerent strengths, are abundance from the λ4934Å line. In both cases and for both well reproduced whether they are blended with Fe i lines ET0381 and scl_03_059, we found signiﬁcant discrepancies or not. In conclusion, one can assert that, for ET0381 and between the abundances derived from the subordinate lines scl_03_059, consistency between the diﬀerent lines of Ba ii and from the resonance line (Table 6). Further investigation can only be achieved by applying a low fraction of the odd-A shows that these diﬀerences are removed for fodd = 0.18, i.e. Ba isotopes, f between 0.18 and 0.11. At ﬁrst glance this the solar Ba isotope mixture (Lodders et al. 2009) or even odd result would imply a dominant contribution of the s-process fodd = 0.11, which is predicted by Bisterzo et al. (2014) for to barium. The implications and alternative scenarios are dis- pure s-process production. cussed further in Section . We performed a series of tests to estimate how much of the diﬀerences between the subordinate and the Ba ii 4934 Å lines found for the r-process Ba isotope mixtures could sim- ply arise from uncertainties in our analysis: 5. Discussion i) We tested the exact same methodology on HE1219-0312, 5.1. Comparison samples: References, symbols, and colour a well-known r-process enhanced star with [Eu/Fe]= +1.4 codes in ﬁgures (Hayek et al. 2009). Using the spectrum reduced by Norbert Christlieb, we ﬁnd a mean abundance−of the±Ba ii 5853 Å, In all ﬁgures, the grey points show the location of RGB stars 6141 Å, and 6496 Å, lines to be log ε= 0.06 0.03 in LTE in the Milky Way halo (Honda et al. 2004; Cayrel et al. 2004; Article number, page 6 of 21page.21 ##LTFigure## ##LTFigure## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine##

text: )A˚

text: F

text: )A˚

text: F

text: λ

text: λ

text: A&A

text: proofs:

text: manuscript

text: no.

label: list

font: VMYLUC+txsy

text: calculation,

text: and

text: in−NLTE.

text: The

text: Å

text: line

text: leads

text: to

text: an

text: NLTE

text: abundance

text: of

text: for

text: fodd

font: XNCHCA+rtxr

text: and

text: for

text: fodd

text: This

text: rules

text: out

text: a

text: low

text: fodd.

text: Our

text: analysis

text: FeI

text: FeI

text: FeI

text: is

text: also

text: in

text: agreement

text: with

text: the

text: results

text: of

text: Hayek

text: et

text: al.

text: FeI

text: BaII

text: who

text: determined

text: ii)

text: Since

text: the

text: atmosphere

text: of

text: very

text: metal-poor

text: stars

text: can

text: be

text: sub-

text: BaII

text: ject

text: to

text: deviation

text: from

text: LTE,

text: we

text: performed

text: NLTE

text: calcula-

text: tions,

text: following

text: the

text: procedure

text: of

text: Mashonkina

text: et

text: al.

text: BaII

text: It

text: turns

text: out

text: that

text: the

text: departure

text: from

text: LTE

text: is

text: very

text: small

text: for

text: the

text: Ba

text: ii

text: Å

text: line,

text: w−ith

text: a

text: NLTE

text: abundance

text: correction

text: o−f

text: ∆NLTE

text: log

text: εNLTE

text: log

text: εLTE

text: for

text: and

text: dex

text: for

text: (Table

text: As

text: to

text: the

text: subordinate

text: lines,

text: the

text: NLTE

text: calculation

text: leads

text: to

text: higher

text: abundances

text: by

text: dex

text: to

text: dex,

text: resulting

text: in

text: even

text: larger

text: discrepancies

text: with

text: the

text: Ba

text: ii

text: Å

text: line.

text: iii)

text: Another

text: source

text: of

text: uncertainty

text: arises

text: from

text: the

text: atmo-

text: λ

text: (A˚

text: spheric

text: parameters.

text: The

text: consequence

text: of

text: a

text: variation

text: of

text: each

text: of

text: T

text: log

text: g,

text: or

text: v

text: has

text: been

text: estimated

text: (Table

text: The

text: “to-

text: eﬀ

text: t

text: tal”

text: quantities

text: correspond

text: to

text: the

text: total

text: impact

text: of

text: varying

text: stel-

text: lar

text: parameters,

text: computed

text: as

text: the

text: quadratic

text: sum

text: of

text: the

text: three

text: FeI

text: BaII

text: FeI

text: sources

text: of

text: uncertainties.

text: For

text: these

text: uncertainties

text: BaII

text: produce

text: very

text: similar

text: abundance

text: shifts

text: for

text: the

text: Ba

text: ii

text: Å

text: BaII

text: and

text: the

text: subordinate

text: lines,

text: and,

text: therefore

text: cannot

text: explain

text: the

text: diﬀerence

text: between

text: them.

text: For

text: the

text: abundance

text: de-

text: BaII

text: rived

text: from

text: the

text: Ba

text: ii

text: Å

text: line

text: is

text: slightly

text: more

text: aﬀected

text: than

text: the

text: subordinate

text: lines,

text: however,

text: not

text: large

text: enough

text: yet

text: to

text: ex-

text: plain

text: the

text: observed

text: discrepancy.

text: iv)

text: We

text: did

text: not

text: use

text: any

text: hydrodynamical

text: model

text: atmo-

text: spheres

text: to

text: derive

text: the

text: abundances

text: of

text: barium.

text: This

text: element

text: is

text: only

text: detected

text: in

text: the

text: Ba

text: ii

text: majority

text: species,

text: and

text: the

text: detected

text: lines

text: arise

text: either

text: from

text: the

text: ground

text: or

text: low-excitation

text: levels.

text: scl

text: Dobrovolskas

text: et

text: al.

text: predicted

text: that

text: the

text: abun-

text: dance

text: corrections

text: for

text: metal-poor

text: giant

text: stars

text: are

text: small

text: for

text: the

text: Fig.

text: Synthesis

text: of

text: the

text: Ba

text: lines

text: for

text: (upper

text: panel)

text: and

text: dex

text: and

text: dex

text: in

text: the

text: model.

text: The

text: (lower

text: panel).

text: A

text: value

text: of

text: fodd

text: is

text: taken

text: for

text: the

text: Baii

text: Ba

text: ii

text: Å

text: lines

text: arise

text: from

text: the

text: Eexc=

text: The

text: observed

text: spectra

text: are

text: shown

text: in

text: black.

text: The

text: red

text: continuous,

text: eV

text: level.

text: Interpolating

text: between

text: Eexc

text: and

text: eV

text: in

text: Dobro-

text: green

text: dashed,

text: and

text: green

text: dotted

text: curves

text: correspond

text: to

text: the

text: theoretical

text: volskas

text: et

text: al.

text: one

text: sees

text: that

text: the

text: diﬀerence

text: between

text: spectra

text: of

text: the

text: full

text: blend,

text: pure

text: Baii

text: lines,

text: and

text: Fei

text: blending

text: lines,

text: re-

text: the

text: Ba

text: ii

text: Å

text: and

text: the

text: Ba

text: ii

text: subordinate

text: lines

text: would

text: not

text: spectively.

text: be

text: reduced

text: since

text: the

text: abundance

text: correction

text: is

text: more

text: negative

text: for

text: the

text: Ba

text: ii

text: Å

text: than

text: for

text: the

text: other

text: lines.

text: It

text: turns

text: out

text: that

text: of

text: the

text: above

text: plausible

text: sources

text: of

text: errors

text: possible

text: fractions,

text: fodd

text: (Travaglio

text: et

text: al.

text: is

text: suﬃcient

text: to

text: explain

text: the

text: diﬀerence

text: between

text: the

text: subordinate

text: (Sneden

text: et

text: al.

text: (Bisterzo

text: et

text: al.

text: and

text: and

text: the

text: Ba

text: ii

text: Å

text: lines.

text: Figure

text: displays

text: the

text: result

text: of

text: our

text: (McWilliam

text: We

text: ﬁrst

text: considered

text: f

text: and

text: to

text: derive

text: the

text: Ba

text: ii

text: NLTE

text: synthesis

text: assuming

text: fodd

text: for

text: the

text: Ba

text: II

text: Å

text: odd

text: line.

text: All

text: Ba

text: lines,

text: which

text: have

text: very

text: diﬀerent

text: strengths,

text: are

text: abundance

text: from

text: the

text: line.

text: In

text: both

text: cases

text: and

text: for

text: both

text: well

text: reproduced

text: whether

text: they

text: are

text: blended

text: with

text: Fe

text: i

text: lines

text: and

text: we

text: found

text: signiﬁcant

text: discrepancies

text: or

text: not.

text: In

text: conclusion,

text: one

text: can

text: assert

text: that,

text: for

text: and

text: between

text: the

text: abundances

text: derived

text: from

text: the

text: subordinate

text: lines

text: consistency

text: between

text: the

text: diﬀerent

text: lines

text: of

text: Ba

text: ii

text: and

text: from

text: the

text: resonance

text: line

text: (Table

text: Further

text: investigation

text: can

text: only

text: be

text: achieved

text: by

text: applying

text: a

text: low

text: fraction

text: of

text: the

text: odd-A

text: shows

text: that

text: these

text: diﬀerences

text: are

text: removed

text: for

text: fodd

text: i.e.

text: Ba

text: isotopes,

text: f

text: between

text: and

text: At

text: ﬁrst

text: glance

text: this

text: the

text: solar

text: Ba

text: isotope

text: mixture

text: (Lodders

text: et

text: al.

text: or

text: even

text: odd

text: result

text: would

text: imply

text: a

text: dominant

text: contribution

text: of

text: the

text: s-process

text: fodd

text: which

text: is

text: predicted

text: by

text: Bisterzo

text: et

text: al.

text: for

text: to

text: barium.

text: The

text: implications

text: and

text: alternative

text: scenarios

text: are

text: dis-

text: pure

text: s-process

text: production.

text: cussed

text: further

text: in

text: Section

text: We

text: performed

text: a

text: series

text: of

text: tests

text: to

text: estimate

text: how

text: much

text: of

text: the

text: diﬀerences

text: between

text: the

text: subordinate

text: and

text: the

text: Ba

text: ii

text: Å

text: lines

text: found

text: for

text: the

text: r-process

text: Ba

text: isotope

text: mixtures

text: could

text: sim-

text: ply

text: arise

text: from

text: uncertainties

text: in

text: our

text: analysis:

text: Discussion

text: i)

text: We

text: tested

text: the

text: exact

text: same

text: methodology

text: on

text: Comparison

text: samples:

text: References,

text: symbols,

text: and

text: colour

text: a

text: well-known

text: r-process

text: enhanced

text: star

text: with

text: [Eu/Fe]=

text: codes

text: in

text: ﬁgures

text: (Hayek

text: et

text: al.

text: Using

text: the

text: spectrum

text: reduced

text: by

text: Norbert

text: Christlieb,

text: we

text: ﬁnd

text: a

text: mean

text: abundance−of

text: the±Ba

text: ii

text: Å,

text: In

text: all

text: ﬁgures,

text: the

text: grey

text: points

text: show

text: the

text: location

text: of

text: RGB

text: stars

text: Å,

text: and

text: Å,

text: lines

text: to

text: be

text: log

text: ε=

text: in

text: LTE

text: in

text: the

text: Milky

text: Way

text: halo

text: (Honda

text: et

text: al.

text: Cayrel

text: et

text: al.

text: Article

text: number,

text: page

text: of

text: ##LTFigure##

text: ##LTFigure##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##


text: A&A proofs: manuscript no. ﬁnal_dif selection of the sample, varying from 90% to 15% of stars with There are two ways to produce the low [α/Fe] ratios observed high carbonicity ([C/Fe] ratios, Carollo et al. (2012)) in individ- for ET0381 and scl051_05. One is to increase [Fe/H] at a given ual samples to 46% when combining them all. α-element abundance by the ejecta of SNeIa and the second is to Another feature in calculating the statistics of carbonicity in lower the abundance of α-elements at a ﬁxed iron abundance, diﬀerent galaxies is related to the spatial distribution. Carollo by varying the ratio between faint and massive SNeII. In the et al. (2012) ﬁnd that the fraction of CEMP stars is larger by a following we investigate the two possibilities, concentrating on ∼ − ≤ factor of 1.5 between the inner and the outer halo for 3 ET0381 for which we have the most detailed chemical informa- ≤ − ∼ [Fe/H] 2.5, rising from 20% to 30%. Lee et al. (2013) tion•: further notice that by restricting stars to within a <5kpc distance If ET0381 were polluted by the ejecta of SNeIa, the sec- from the Galactic mid-plane, the fraction of CEMP stars does not ond nucleosynthetic signature of such an event would be a large increase with decreasing metallicity. What part of the Milky Way production of nickel. We can assume that before being polluted halo should the samples in dSphs, which are spatially randomly the ISM from which ET0381 arose was of classical composi- distributed, be compared with? tion, i.e. on the [α/Fe] plateau formed by the−SNeII±products. In conclusion, the comparison of the CEMP star fraction in Its initial m−etallic±ity [Fe/H]i would ∼then be 3.31 0.2 (for dSphs and the Milky Way is promising to unveil both the origin [Mg/H]=− 2.91± 0.2, i.e. [Mg/Fe] 0.4), corresponding to of carbon and the conditions of formation of the diﬀerent sys- [Ni/H]i= 3.31 0.2. The W7 model of SNeIa nucleosynthe- tems. However, it seems to require further investigation at least sis, which is widely used in galactic chemical evolution mod- along the RGBs in classical dwarfs, reaching larger gravities. els (Nomoto et al. 1984; Iwamoto et al. 1999) and also in the 5.4. The α elements work of Travaglio et al. (2005) varying the metallicity, pre- dicts 58Ni (a stable isotope of Ni) to be overproduced by a Figures 8 and 9 display [Mg/Fe], [Si/Fe], [Ca/Fe], and factor of 1 to ∼3 relative to iron, compared to the solar abun- [Ti/Fe] as a function of [Fe/H]. For these elements as for the dances. If Ni were produced just as much as Fe, the corre- others, we present the results of our analysis together with earlier − sponding ﬁnal abundance [Ni/H]= 2.44 would already exceed works on Sculptor Tafelmeyer et al. (2010); Frebel et al. (2010a); the observations. Chromium is also largely produced by SNeIa. Starkenburg et al. (2013). Following the same line of reasoning as for Ni, from an ini- − ± 5.4.1. Global features tial [Cr/H]i= 3.61 0.2, cor−respon±ding to the classical Milky The bulk (80%) of the metal-poor and extremely metal-poor Way halo level of a [Fe/H]= 3.31 0.2 star, with a SNeI pro- ducing at least twice as much Cr than Fe compared to the so- stars observed so far in Sculptor follow the main trend of the − ± Milky Way halo stars for the four α-elements presented here, lar−abunda±nces, the ﬁnal [Cr/H] should be 2.44 0.2 instead i.e. they mostly have supersolar abundance ratios. The scatter of 2.75 0.1. Additionally, the knee of the [−α/Fe] versus the of this plateau is also very similar in the two galaxies. There [Fe/H] relation in Sculptor is found at [Fe/H]> 2 (Tolstoy et al. 2009), hence at higher metallicity than the two outliers shown are three outliers with subsolar ratios, ET0381, scl051_05, and ∼ − here. This means that we are not witnessing a major onset of the scl_11_1_4296. The star ET0381 at [Fe/H] 2.5 is the only one SNeIa explosions. Therefore, ET0381 would need to form from with consistent low ratios for the four elements and which can an unusual type of SNeIa, since the rest of the chemical evolu- robustly be attributed to an interstellar medium inhomogeneity, tion of Sculptor looks very chemically homogeneous. The last although it is likely also the case of scl051_05. Indeed Starken- piece of evidence comes from the existing observations of stars burg et al. (2013) could not measure Si, but Ti, Mg, and Ca were − with metallicity larger than 2, i.e. past the metallicity at which securely derived from 2 to 5 lines. The star scl_11_1_4296 from the SNeIa are the major contributors to the chemical evolution of Simon et al. (2015) exhibits low [α/Fe] in three of the four el- Sculptor (Tolstoy et al. 2009). Shetrone et al. (2003) measured ements, but a normal, Milky Way-like [Ti ii/Fe]. Interestingly, the uncertainty on [Fe/H] for this star is similar to the under- the abundance of cobalt in four such stars, and they have [Co/Fe] close to or above solar, far above the present value for ET0381. abundance in Mg, Si, and Ca. Moreover, its abundance of Ti ii is derived from 11 lines as opposed to a single line in Si and In summary, although there are no models of metal-poor Ca, and two lines for Mg. This all calls for conﬁrmation of these SNeI that can deﬁnitely infer the amount of iron peak elements ﬁrst determinations. Finally, scl07-50 (Tafelmeyer et al. 2010) they would release, for now there is a body of corroborating ev- has a subsolar [Ca/Fe] ratio, but here again the Ca abundance is idence that a pollution by their sole ejecta is unlikely to explain derived from a single resonance line. A forthcoming paper ded- the c•hemical features of ET0381. icated to accurate NLTE corrections will provide a better view, The more massive the SNeII, the larger their yield of α- with a higher value, on the intrinsic Ca abundance of this star elements. Therefore, ET0381 (and possibly scl051_05) could (Mashonkina et al. 2015). probably arise from a pocket of interstellar medium that had not In conclusion, so far robust evidence for the existence of been polluted (or at least that was under-polluted) by the most pockets of chemical inhomogeneity in the early days of Sculp- massive of the SNeII. tor comes from two stars out of 14, scl051_05 and ET0381; the A quick calculation with the yields of Tsujimoto et al. (1995) rest of the metal-poor population appears as homogeneous as the shows that missing the ejecta of the stars more massive than Milky Way halo within the observational uncertainties. Indeed, 20M(cid:12) leads to subsolar [α/Fe]. With such a truncated IMF, it a supersolar plateau in [α/Fe] is expected when stars form out of takes only 30 Myr for a single stellar population born f−rom an interstellar medium in which the ejecta of the massive stars, a gas clump 50− times more massive to reach [Fe/H] = 2.5 in numbers following a classical initial mass function (IMF), and [Mg/Fe] = 0.5. Although it is beyond the scope of the are suﬃciently well mixed (Pagel 2009). This means that what- present paper to run full and detailed simulations to understand ever the process that led to this homogeneity, it was the same in how pockets of interstellar medium can have discordant abun- Sculptor and in the Milky Way halo, or at least, the processes at dance ratios in an otherwise homogeneous galaxy, we ran a few play were scalable to the diﬀerent sizes/masses of the systems. tests with the chemo-dynamical tree-SPH code GEAR (Revaz 5.4.2. The origin of the outsiders & Jablonka 2012). We follo×wed at very high spatial resolution (30 pc), the evolution of a 8 108M(cid:12) total mass system, forming Article number, page 8 of 21page.21

text: A&A

label: paragraph

text: proofs:

text: manuscript

text: no.

text: selection

text: of

text: the

text: sample,

text: varying

text: from

text: to

text: of

text: stars

text: with

text: There

text: are

text: two

text: ways

text: to

text: produce

text: the

text: low

text: [α/Fe]

text: ratios

text: observed

text: high

text: carbonicity

text: ([C/Fe]

text: ratios,

text: Carollo

text: et

text: al.

text: in

text: individ-

text: for

text: and

text: One

text: is

text: to

text: increase

text: [Fe/H]

text: at

text: a

text: given

text: ual

text: samples

text: to

text: when

text: combining

text: them

text: all.

text: α-element

text: abundance

text: by

text: the

text: ejecta

text: of

text: SNeIa

text: and

text: the

text: second

text: is

text: to

text: Another

text: feature

text: in

text: calculating

text: the

text: statistics

text: of

text: carbonicity

text: in

text: lower

text: the

text: abundance

text: of

text: α-elements

text: at

text: a

text: ﬁxed

text: iron

text: abundance,

text: diﬀerent

text: galaxies

text: is

text: related

text: to

text: the

text: spatial

text: distribution.

text: Carollo

text: by

text: varying

text: the

text: ratio

text: between

text: faint

text: and

text: massive

text: SNeII.

text: In

text: the

text: et

text: al.

text: ﬁnd

text: that

text: the

text: fraction

text: of

text: CEMP

text: stars

text: is

text: larger

text: by

text: a

text: following

text: we

text: investigate

text: the

text: two

text: possibilities,

text: concentrating

text: on

text: factor

text: of

text: between

text: the

text: inner

text: and

text: the

text: outer

text: halo

text: for

text: for

text: which

text: we

text: have

text: the

text: most

text: detailed

text: chemical

text: informa-

text: [Fe/H]

text: rising

text: from

text: to

text: Lee

text: et

text: al.

text: tion•:

text: further

text: notice

text: that

text: by

text: restricting

text: stars

text: to

text: within

text: a

text: distance

text: If

text: were

text: polluted

text: by

text: the

text: ejecta

text: of

text: SNeIa,

text: the

text: sec-

text: from

text: the

text: Galactic

text: mid-plane,

text: the

text: fraction

text: of

text: CEMP

text: stars

text: does

text: not

text: ond

text: nucleosynthetic

text: signature

text: of

text: such

text: an

text: event

text: would

text: be

text: a

text: large

text: increase

text: with

text: decreasing

text: metallicity.

text: What

text: part

text: of

text: the

text: Milky

text: Way

text: production

text: of

text: nickel.

text: We

text: can

text: assume

text: that

text: before

text: being

text: polluted

text: halo

text: should

text: the

text: samples

text: in

text: dSphs,

text: which

text: are

text: spatially

text: randomly

text: the

text: ISM

text: from

text: which

text: arose

text: was

text: of

text: classical

text: composi-

text: distributed,

text: be

text: compared

text: with?

text: tion,

text: i.e.

text: on

text: the

text: [α/Fe]

text: plateau

text: formed

text: by

text: the−SNeII±products.

text: In

text: conclusion,

text: the

text: comparison

text: of

text: the

text: CEMP

text: star

text: fraction

text: in

text: Its

text: initial

text: m−etallic±ity

text: [Fe/H]i

text: would

text: ∼then

text: be

text: (for

text: dSphs

text: and

text: the

text: Milky

text: Way

text: is

text: promising

text: to

text: unveil

text: both

text: the

text: origin

text: [Mg/H]=−

text: i.e.

text: [Mg/Fe]

text: corresponding

text: to

text: of

text: carbon

text: and

text: the

text: conditions

text: of

text: formation

text: of

text: the

text: diﬀerent

text: sys-

text: [Ni/H]i=

text: The

text: model

text: of

text: SNeIa

text: nucleosynthe-

text: tems.

text: However,

text: it

text: seems

text: to

text: require

text: further

text: investigation

text: at

text: least

text: sis,

text: which

text: is

text: widely

text: used

text: in

text: galactic

text: chemical

text: evolution

text: mod-

text: along

text: the

text: RGBs

text: in

text: classical

text: dwarfs,

text: reaching

text: larger

text: gravities.

text: els

text: (Nomoto

text: et

text: al.

text: Iwamoto

text: et

text: al.

text: and

text: also

text: in

text: the

text: The

text: α

text: elements

text: work

text: of

text: Travaglio

text: et

text: al.

text: varying

text: the

text: metallicity,

text: pre-

text: dicts

text: (a

text: stable

text: isotope

text: of

text: Ni)

text: to

text: be

text: overproduced

text: by

text: a

text: Figures

text: and

text: display

text: [Mg/Fe],

text: [Si/Fe],

text: [Ca/Fe],

text: and

text: factor

text: of

text: to

text: ∼3

text: relative

text: to

text: iron,

text: compared

text: to

text: the

text: solar

text: abun-

text: [Ti/Fe]

text: as

text: a

text: function

text: of

text: [Fe/H].

text: For

text: these

text: elements

text: as

text: for

text: the

text: dances.

text: If

text: Ni

text: were

text: produced

text: just

text: as

text: much

text: as

text: Fe,

text: the

text: corre-

text: others,

text: we

text: present

text: the

text: results

text: of

text: our

text: analysis

text: together

text: with

text: earlier

text: sponding

text: ﬁnal

text: abundance

text: [Ni/H]=

text: would

text: already

text: exceed

text: works

text: on

text: Sculptor

text: Tafelmeyer

text: et

text: al.

text: Frebel

text: et

text: al.

text: the

text: observations.

text: Chromium

text: is

text: also

text: largely

text: produced

text: by

text: SNeIa.

text: Starkenburg

text: et

text: al.

text: Following

text: the

text: same

text: line

text: of

text: reasoning

text: as

text: for

text: Ni,

text: from

text: an

text: ini-

text: Global

text: features

text: tial

text: [Cr/H]i=

text: cor−respon±ding

text: to

text: the

text: classical

text: Milky

text: The

text: bulk

text: of

text: the

text: metal-poor

text: and

text: extremely

text: metal-poor

text: Way

text: halo

text: level

text: of

text: a

text: [Fe/H]=

text: star,

text: with

text: a

text: SNeI

text: pro-

text: ducing

text: at

text: least

text: twice

text: as

text: much

text: Cr

text: than

text: Fe

text: compared

text: to

text: the

text: so-

text: stars

text: observed

text: so

text: far

text: in

text: Sculptor

text: follow

text: the

text: main

text: trend

text: of

text: the

text: Milky

text: Way

text: halo

text: stars

text: for

text: the

text: four

text: α-elements

text: presented

text: here,

text: lar−abunda±nces,

text: the

text: ﬁnal

text: [Cr/H]

text: should

text: be

text: instead

text: i.e.

text: they

text: mostly

text: have

text: supersolar

text: abundance

text: ratios.

text: The

text: scatter

text: of

text: Additionally,

text: the

text: knee

text: of

text: the

text: [−α/Fe]

text: versus

text: the

text: of

text: this

text: plateau

text: is

text: also

text: very

text: similar

text: in

text: the

text: two

text: galaxies.

text: There

text: [Fe/H]

text: relation

text: in

text: Sculptor

text: is

text: found

text: at

text: [Fe/H]>

text: (Tolstoy

text: et

text: al.

text: hence

text: at

text: higher

text: metallicity

text: than

text: the

text: two

text: outliers

text: shown

text: are

text: three

text: outliers

text: with

text: subsolar

text: ratios,

text: and

text: here.

text: This

text: means

text: that

text: we

text: are

text: not

text: witnessing

text: a

text: major

text: onset

text: of

text: the

text: The

text: star

text: at

text: [Fe/H]

text: is

text: the

text: only

text: one

text: SNeIa

text: explosions.

text: Therefore,

text: would

text: need

text: to

text: form

text: from

text: with

text: consistent

text: low

text: ratios

text: for

text: the

text: four

text: elements

text: and

text: which

text: can

text: an

text: unusual

text: type

text: of

text: SNeIa,

text: since

text: the

text: rest

text: of

text: the

text: chemical

text: evolu-

text: robustly

text: be

text: attributed

text: to

text: an

text: interstellar

text: medium

text: inhomogeneity,

text: tion

text: of

text: Sculptor

text: looks

text: very

text: chemically

text: homogeneous.

text: The

text: last

text: although

text: it

text: is

text: likely

text: also

text: the

text: case

text: of

text: Indeed

text: Starken-

text: piece

text: of

text: evidence

text: comes

text: from

text: the

text: existing

text: observations

text: of

text: stars

text: burg

text: et

text: al.

text: could

text: not

text: measure

text: Si,

text: but

text: Ti,

text: Mg,

text: and

text: Ca

text: were

text: with

text: metallicity

text: larger

text: than

text: i.e.

text: past

text: the

text: metallicity

text: at

text: which

text: securely

text: derived

text: from

text: to

text: lines.

text: The

text: star

text: from

text: the

text: SNeIa

text: are

text: the

text: major

text: contributors

text: to

text: the

text: chemical

text: evolution

text: of

text: Simon

text: et

text: al.

text: exhibits

text: low

text: [α/Fe]

text: in

text: three

text: of

text: the

text: four

text: el-

text: Sculptor

text: (Tolstoy

text: et

text: al.

text: Shetrone

text: et

text: al.

text: measured

text: ements,

text: but

text: a

text: normal,

text: Milky

text: Way-like

text: [Ti

text: ii/Fe].

text: Interestingly,

text: the

text: uncertainty

text: on

text: [Fe/H]

text: for

text: this

text: star

text: is

text: similar

text: to

text: the

text: under-

text: the

text: abundance

text: of

text: cobalt

text: in

text: four

text: such

text: stars,

text: and

text: they

text: have

text: [Co/Fe]

text: close

text: to

text: or

text: above

text: solar,

text: far

text: above

text: the

text: present

text: value

text: for

text: abundance

text: in

text: Mg,

text: Si,

text: and

text: Ca.

text: Moreover,

text: its

text: abundance

text: of

text: Ti

text: ii

text: is

text: derived

text: from

text: lines

text: as

text: opposed

text: to

text: a

text: single

text: line

text: in

text: Si

text: and

text: In

text: summary,

text: although

text: there

text: are

text: no

text: models

text: of

text: metal-poor

text: Ca,

text: and

text: two

text: lines

text: for

text: Mg.

text: This

text: all

text: calls

text: for

text: conﬁrmation

text: of

text: these

text: SNeI

text: that

text: can

text: deﬁnitely

text: infer

text: the

text: amount

text: of

text: iron

text: peak

text: elements

text: ﬁrst

text: determinations.

text: Finally,

text: (Tafelmeyer

text: et

text: al.

text: they

text: would

text: release,

text: for

text: now

text: there

text: is

text: a

text: body

text: of

text: corroborating

text: ev-

text: has

text: a

text: subsolar

text: [Ca/Fe]

text: ratio,

text: but

text: here

text: again

text: the

text: Ca

text: abundance

text: is

text: idence

text: that

text: a

text: pollution

text: by

text: their

text: sole

text: ejecta

text: is

text: unlikely

text: to

text: explain

text: derived

text: from

text: a

text: single

text: resonance

text: line.

text: A

text: forthcoming

text: paper

text: ded-

text: the

text: c•hemical

text: features

text: of

text: icated

text: to

text: accurate

text: NLTE

text: corrections

text: will

text: provide

text: a

text: better

text: view,

text: The

text: more

text: massive

text: the

text: SNeII,

text: the

text: larger

text: their

text: yield

text: of

text: α-

text: with

text: a

text: higher

text: value,

text: on

text: the

text: intrinsic

text: Ca

text: abundance

text: of

text: this

text: star

text: elements.

text: Therefore,

text: (and

text: possibly

text: could

text: (Mashonkina

text: et

text: al.

text: probably

text: arise

text: from

text: a

text: pocket

text: of

text: interstellar

text: medium

text: that

text: had

text: not

text: In

text: conclusion,

text: so

text: far

text: robust

text: evidence

text: for

text: the

text: existence

text: of

text: been

text: polluted

text: (or

text: at

text: least

text: that

text: was

text: under-polluted)

text: by

text: the

text: most

text: pockets

text: of

text: chemical

text: inhomogeneity

text: in

text: the

text: early

text: days

text: of

text: Sculp-

text: massive

text: of

text: the

text: SNeII.

text: tor

text: comes

text: from

text: two

text: stars

text: out

text: of

text: and

text: the

text: A

text: quick

text: calculation

text: with

text: the

text: yields

text: of

text: Tsujimoto

text: et

text: al.

text: rest

text: of

text: the

text: metal-poor

text: population

text: appears

text: as

text: homogeneous

text: as

text: the

text: shows

text: that

text: missing

text: the

text: ejecta

text: of

text: the

text: stars

text: more

text: massive

text: than

text: Milky

text: Way

text: halo

text: within

text: the

text: observational

text: uncertainties.

text: Indeed,

text: leads

text: to

text: subsolar

text: [α/Fe].

text: With

text: such

text: a

text: truncated

text: IMF,

text: it

text: a

text: supersolar

text: plateau

text: in

text: [α/Fe]

text: is

text: expected

text: when

text: stars

text: form

text: out

text: of

text: takes

text: only

text: Myr

text: for

text: a

text: single

text: stellar

text: population

text: born

text: f−rom

text: an

text: interstellar

text: medium

text: in

text: which

text: the

text: ejecta

text: of

text: the

text: massive

text: stars,

text: a

text: gas

text: clump

text: times

text: more

text: massive

text: to

text: reach

text: [Fe/H]

text: in

text: numbers

text: following

text: a

text: classical

text: initial

text: mass

text: function

text: (IMF),

text: and

text: [Mg/Fe]

text: Although

text: it

text: is

text: beyond

text: the

text: scope

text: of

text: the

text: are

text: suﬃciently

text: well

text: mixed

text: (Pagel

text: This

text: means

text: that

text: what-

text: present

text: paper

text: to

text: run

text: full

text: and

text: detailed

text: simulations

text: to

text: understand

text: ever

text: the

text: process

text: that

text: led

text: to

text: this

text: homogeneity,

text: it

text: was

text: the

text: same

text: in

text: how

text: pockets

text: of

text: interstellar

text: medium

text: can

text: have

text: discordant

text: abun-

text: Sculptor

text: and

text: in

text: the

text: Milky

text: Way

text: halo,

text: or

text: at

text: least,

text: the

text: processes

text: at

text: dance

text: ratios

text: in

text: an

text: otherwise

text: homogeneous

text: galaxy,

text: we

text: ran

text: a

text: few

text: play

text: were

text: scalable

text: to

text: the

text: diﬀerent

text: sizes/masses

text: of

text: the

text: systems.

text: tests

text: with

text: the

text: chemo-dynamical

text: tree-SPH

text: code

text: GEAR

text: (Revaz

text: The

text: origin

text: of

text: the

text: outsiders

text: Jablonka

text: We

text: follo×wed

text: at

text: very

text: high

text: spatial

text: resolution

text: pc),

text: the

text: evolution

text: of

text: a

text: total

text: mass

text: system,

text: forming

text: Article

text: number,

text: page

text: of


text: 2 NICOLAS CHOPIN AND JAMES RIDGWAY We would also like to discuss how Bayesian computation algorithms should be compared. One obvious criterion is the error versus CPU time trade-oﬀ; this im- plies discussing which posterior quantities one may be need to approximate. A related point is whether the considered method comes with a simple way to evalu- ate the numerical error. Other criteria of interest are: (a) how easy to implement is the considered method? (b) how generic is it? (does changing the prior or the link function require a complete rewrite of the source code?) (c) to which extent does it require manual tuning to obtain good performances? (d) is it amenable to parallelisation? Points (a) and (b) are rarely discussed in Statistics, but relate to the important fact that, the simpler the program, the easier it is to maintain, and to make it bug-free. Regarding point (c), we warn beforehand that, as a matter of principle, we shall refuse to manually tune an algorithm on a per dataset basis. Rather, we will discuss, for each approach, some (hopefully reasonable) general re- cipe for how to choose the tuning parameters. This has two motivations. First, human time is far more valuable that computer time: Cook (2014) mentions that one hour of CPU time is today three orders of magnitude less expensive than one hour of pay for a programmer (or similarly a scientist). Second, any method re- quiring too much manual tuning through trial and error may be practically of no use beyond a small number of experts. Finally, we also hope this paper may serve as an up to date review of the state of Bayesian computation. We believe this review to be timely for a number of reas- ons. First, as already mentioned, because Bayesian computation seems to develop currently in several diﬀerent directions. Second, and this relates to criterion (d), the current interest in parallel computation (Lee et al., 2010; Suchard et al., 2010) may require a re-assessment of Bayesian computational methods: method A may perform better than method B on a single core architecture, while performing much worse on a parallel architecture. Finally, although the phrase ‘big data’ seems to be a tired trope already, it is certainly true that datasets are getting bigger and bigger, which in return means that statistical methods needs to be evaluated on bigger and bigger datasets. To be fair, we will not really consider in this work the kind of huge datasets that pertain to ‘big data’, but we will at least strive to move away from the kind of ‘ridiculously small’ data encountered too often in Bayesian computation papers. The paper is structured as follows. Section 2 covers certain useful preliminaries on binary regression models. Section 3 discusses fast approximations, that is, de- terministic algorithms that oﬀer an approximation of the posterior, at a lower cost than sampling-based methods. Section 4 discusses ‘exact’, sampling-based meth- ods. Section 5 is the most important part of the paper, as it contains an extensive numerical comparison of all these methods. Section 6 discusses variable selection. Section 7 discusses our ﬁndings, and their implications for both end users and Bayesian computation experts. 2. Preliminaries: binary regression models The likelihood of a binary regression model have the generic expression (cid:89) D| nD (2.1) p( β) = F(y βT x ) i i i=1 D ∈ {− } where the data consist of n responses y 1, 1 and n vectors x of p i i covariates, and F is some CDF (cumulative distribution function) that transforms the linear form y βT x into a probability. Taking F = Φ, the standard normal CDF, i i − gives the probit model, while taking F = L, the logistic CDF, L(x) = 1/ (1 + e x),

text: NICOLAS

text: CHOPIN

text: AND

text: JAMES

text: RIDGWAY

text: We

text: would

text: also

text: like

text: to

text: discuss

text: how

text: Bayesian

text: computation

text: algorithms

text: should

text: be

text: compared.

text: One

text: obvious

text: criterion

text: is

text: the

text: error

text: versus

text: CPU

text: time

text: trade-oﬀ;

text: this

text: im-

text: plies

text: discussing

text: which

text: posterior

text: quantities

text: one

text: may

text: be

text: need

text: to

text: approximate.

text: A

text: related

text: point

text: is

text: whether

text: the

text: considered

text: method

text: comes

text: with

text: a

text: simple

text: way

text: to

text: evalu-

text: ate

text: the

text: numerical

text: error.

text: Other

text: criteria

text: of

text: interest

text: are:

text: (a)

text: how

text: easy

text: to

text: implement

text: is

text: the

text: considered

text: method?

text: (b)

text: how

text: generic

text: is

text: it?

text: (does

text: changing

text: the

text: prior

text: or

text: the

text: link

text: function

text: require

text: a

text: complete

text: rewrite

text: of

text: the

text: source

text: code?)

text: (c)

text: to

text: which

text: extent

text: does

text: it

text: require

text: manual

text: tuning

text: to

text: obtain

text: good

text: performances?

text: (d)

text: is

text: it

text: amenable

text: to

text: parallelisation?

text: Points

text: (a)

text: and

text: (b)

text: are

text: rarely

text: discussed

text: in

text: Statistics,

text: but

text: relate

text: to

text: the

text: important

text: fact

text: that,

text: the

text: simpler

text: the

text: program,

text: the

text: easier

text: it

text: is

text: to

text: maintain,

text: and

text: to

text: make

text: it

text: bug-free.

text: Regarding

text: point

text: (c),

text: we

text: warn

text: beforehand

text: that,

text: as

text: a

text: matter

text: of

text: principle,

text: we

text: shall

text: refuse

text: to

text: manually

text: tune

text: an

text: algorithm

text: on

text: a

text: per

text: dataset

text: basis.

text: Rather,

text: we

text: will

text: discuss,

text: for

text: each

text: approach,

text: some

text: (hopefully

text: reasonable)

text: general

text: re-

text: cipe

text: for

text: how

text: to

text: choose

text: the

text: tuning

text: parameters.

text: This

text: has

text: two

text: motivations.

text: First,

text: human

text: time

text: is

text: far

text: more

text: valuable

text: that

text: computer

text: time:

text: Cook

text: mentions

text: that

text: one

text: hour

text: of

text: CPU

text: time

text: is

text: today

text: three

text: orders

text: of

text: magnitude

text: less

text: expensive

text: than

text: one

text: hour

text: of

text: pay

text: for

text: a

text: programmer

text: (or

text: similarly

text: a

text: scientist).

text: Second,

text: any

text: method

text: re-

text: quiring

text: too

text: much

text: manual

text: tuning

text: through

text: trial

text: and

text: error

text: may

text: be

text: practically

text: of

text: no

text: use

text: beyond

text: a

text: small

text: number

text: of

text: experts.

text: Finally,

text: we

text: also

text: hope

text: this

text: paper

text: may

text: serve

text: as

text: an

text: up

text: to

text: date

text: review

text: of

text: the

text: state

text: of

text: Bayesian

text: computation.

text: We

text: believe

text: this

text: review

text: to

text: be

text: timely

text: for

text: a

text: number

text: of

text: reas-

text: ons.

text: First,

text: as

text: already

text: mentioned,

text: because

text: Bayesian

text: computation

text: seems

text: to

text: develop

text: currently

text: in

text: several

text: diﬀerent

text: directions.

text: Second,

text: and

text: this

text: relates

text: to

text: criterion

text: (d),

text: the

text: current

text: interest

text: in

text: parallel

text: computation

text: (Lee

text: et

text: al.,

text: Suchard

text: et

text: al.,

text: may

text: require

text: a

text: re-assessment

text: of

text: Bayesian

text: computational

text: methods:

text: method

text: A

text: may

text: perform

text: better

text: than

text: method

text: B

text: on

text: a

text: single

text: core

text: architecture,

text: while

text: performing

text: much

text: worse

text: on

text: a

text: parallel

text: architecture.

text: Finally,

text: although

text: the

text: phrase

text: ‘big

text: data’

text: seems

text: to

text: be

text: a

text: tired

text: trope

text: already,

text: it

text: is

text: certainly

text: true

text: that

text: datasets

text: are

text: getting

text: bigger

text: and

text: bigger,

text: which

text: in

text: return

text: means

text: that

text: statistical

text: methods

text: needs

text: to

text: be

text: evaluated

text: on

text: bigger

text: and

text: bigger

text: datasets.

text: To

text: be

text: fair,

text: we

text: will

text: not

text: really

text: consider

text: in

text: this

text: work

text: the

text: kind

text: of

text: huge

text: datasets

text: that

text: pertain

text: to

text: ‘big

text: data’,

text: but

text: we

text: will

text: at

text: least

text: strive

text: to

text: move

text: away

text: from

text: the

text: kind

text: of

text: ‘ridiculously

text: small’

text: data

text: encountered

text: too

text: often

text: in

text: Bayesian

text: computation

text: papers.

text: The

text: paper

text: is

text: structured

text: as

text: follows.

text: Section

text: covers

text: certain

text: useful

text: preliminaries

text: on

text: binary

text: regression

text: models.

text: Section

text: discusses

text: fast

text: approximations,

text: that

text: is,

text: de-

text: terministic

text: algorithms

text: that

text: oﬀer

text: an

text: approximation

text: of

text: the

text: posterior,

text: at

text: a

text: lower

text: cost

text: than

text: sampling-based

text: methods.

text: Section

text: discusses

text: ‘exact’,

text: sampling-based

text: meth-

text: ods.

text: Section

text: is

text: the

text: most

text: important

text: part

text: of

text: the

text: paper,

text: as

text: it

text: contains

text: an

text: extensive

text: numerical

text: comparison

text: of

text: all

text: these

text: methods.

text: Section

text: discusses

text: variable

text: selection.

text: Section

text: discusses

text: our

text: ﬁndings,

text: and

text: their

text: implications

text: for

text: both

text: end

text: users

text: and

text: Bayesian

text: computation

text: experts.

text: Preliminaries:

text: binary

text: regression

text: models

text: The

text: likelihood

text: of

text: a

text: binary

text: regression

text: model

text: have

text: the

text: generic

text: expression

text: D|

text: nD

text: p(

text: β)

text: F(y

text: βT

text: x

text: i

text: i

text: D

text: where

text: the

text: data

text: consist

text: of

text: n

text: responses

text: y

text: and

text: n

text: vectors

text: x

text: of

text: p

text: i

text: i

text: covariates,

text: and

text: F

text: is

text: some

text: CDF

text: (cumulative

text: distribution

text: function)

text: that

text: transforms

text: the

text: linear

text: form

text: y

text: βT

text: x

text: into

text: a

text: probability.

text: Taking

text: F

text: Φ,

text: the

text: standard

text: normal

text: CDF,

text: i

text: i

text: gives

text: the

text: probit

text: model,

text: while

text: taking

text: F

text: L,

text: the

text: logistic

text: CDF,

text: L(x)

text: e

text: x),


text: 12 NICOLAS CHOPIN AND JAMES RIDGWAY Algorithm 1 Hastings-Metropolis iteration Input: β (cid:48) Output: β ∼ | 1: Sample β(cid:63) κ(β(cid:63) β). ∧ 2: With probability 1 r, D| | p(β(cid:63))p( β(cid:63))κ(β β(cid:63)) r = D| | , p(β)p( β)κ(β(cid:63) β) (cid:48) (cid:48) set β = β(cid:63); otherwise set β = β. In practice, this usually does not work better than importance sampling based on the same proposal, hence this strategy is hardly used. | A more usual strategy is to set the proposal kernel to a random walk: κ(β(cid:63) β) = N (β, Σ ). It is well known that the choice of Σ is critical for good perform- p prop prop ance. For instance, in the univariate case, if Σ is too small, the chain moves prop slowly, while if too large, proposed moves are rarely accepted. A result from the optimal scaling literature (e.g. Roberts and Rosenthal, 2001) is that, for a N (0, I ) target, Σ = (λ2/p)I with λ = 2.38 is asymptotically p p →pr∞op p optimal, in the sense that as p , this choice leads to the fastest exploration. Since the posterior of a binary regression model is reasonably close to a Gaussian, we adapt this result by taking Σ = (λ2/p)Σ in our simulations, where Σ is the prop q q covariance matrix of a (Laplace or EP) Gaussian approximation of the posterior. This strategy seems validated by the fact we obtain acceptance rates close to the optimal rate, as given by Roberts and Rosenthal (2001). O The bad news behind this optimality result is that the chain requires (p) steps O to move a (1) distance. Thus random walk exploration tends to become slow for large p. This is usually cited as the main motivation to develop more elaborate MCMC strategies, such as HMC, which we cover in the following section. Hamiltonian Monte Carlo (HMC, also known as Hybrid Monte Carlo, Duane et al., 1987) is a new type of MCMC algorithm, where one is able to perform several steps in the parameter space before determining if the new position is accepted or not. Consequently, HMC is able to make much bigger jumps in the parameter space than standard Metropolis algorithms. See Neal (2010) for an excellent introduction. ∼ |D ∼ − Consider the pair (β, α), where β p(β ), and α N (0, M 1), thus with {− } p joint un-normalised density exp H(β, α) , with 1 − { D| } H(β, α) = E(β) + αT Mα, E(β) = log p(β)p( β) . 2 The physical interpretation of HMC is that of a particle at position β, with velo- city α, potential energy E(β), kinetic energy 1αT Mα, for some mass matrix M, 2 and therefore total energy given by H(β, α). The particle is expected to follow a trajectory such that H(β, α) remains constant over time. In practice, HMC proceeds as follows: ﬁrst, sample a new velocity vector, ∼ − α N (0, M 1). Second, move the particle while keeping the Hamiltonian H con- p stant; in practice, discretisation must be used, so L steps of step-size (cid:15) are performed through leap-frop steps; see Algorithm 2 which describes one such step. Third, the new position, obtained after L leap-frog steps is accepted or rejected according to ∧ { − } probability 1 exp H(β, α) H(β(cid:63), α(cid:63)) ; see Algorithm 3 for a summary. The validity of the algorithm relies on the fact that a leap-frog step is “volume pre- → serving”; that is, the deterministic transformation (β, α) (β , α ) has Jacobian 1 1 one. This is why the acceptance probability admits this simple expression. ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine##

text: NICOLAS

text: CHOPIN

text: AND

text: JAMES

text: RIDGWAY

text: Algorithm

text: Hastings-Metropolis

label: caption

text: iteration

text: Input:

text: β

text: Output:

text: β

text: Sample

text: β).

text: With

text: probability

text: r,

text: D|

text: r

text: D|

text: p(β)p(

text: β)

text: set

text: β

text: otherwise

text: set

text: β

text: β.

text: In

text: practice,

text: this

text: usually

text: does

text: not

text: work

text: better

text: than

text: importance

text: sampling

text: based

text: on

text: the

text: same

text: proposal,

text: hence

text: this

text: strategy

text: is

text: hardly

text: used.

text: A

text: more

text: usual

text: strategy

text: is

text: to

text: set

text: the

text: proposal

text: kernel

text: to

text: a

text: random

text: walk:

text: β)

text: N

text: (β,

text: Σ

text: It

text: is

text: well

text: known

text: that

text: the

text: choice

text: of

text: Σ

text: is

text: critical

text: for

text: good

text: perform-

text: p

text: prop

text: prop

text: ance.

text: For

text: instance,

text: in

text: the

text: univariate

text: case,

text: if

text: Σ

text: is

text: too

text: small,

text: the

text: chain

text: moves

text: prop

text: slowly,

text: while

text: if

text: too

text: large,

text: proposed

text: moves

text: are

text: rarely

text: accepted.

text: A

text: result

text: from

text: the

text: optimal

text: scaling

text: literature

text: (e.g.

text: Roberts

text: and

text: Rosenthal,

text: is

text: that,

text: for

text: a

text: N

text: I

text: target,

text: Σ

text: with

text: λ

text: is

text: asymptotically

text: p

text: p

text: →pr∞op

text: p

text: optimal,

text: in

text: the

text: sense

text: that

text: as

text: p

text: this

text: choice

text: leads

text: to

text: the

text: fastest

text: exploration.

text: Since

text: the

text: posterior

text: of

text: a

text: binary

text: regression

text: model

text: is

text: reasonably

text: close

text: to

text: a

text: Gaussian,

text: we

text: adapt

text: this

text: result

text: by

text: taking

text: Σ

text: in

text: our

text: simulations,

text: where

text: Σ

text: is

text: the

text: prop

text: q

text: q

text: covariance

text: matrix

text: of

text: a

text: (Laplace

text: or

text: EP)

text: Gaussian

text: approximation

text: of

text: the

text: posterior.

text: This

text: strategy

text: seems

text: validated

text: by

text: the

text: fact

text: we

text: obtain

text: acceptance

text: rates

text: close

text: to

text: the

text: optimal

text: rate,

text: as

text: given

text: by

text: Roberts

text: and

text: Rosenthal

text: O

text: The

text: bad

text: news

text: behind

text: this

text: optimality

text: result

text: is

text: that

text: the

text: chain

text: requires

text: (p)

text: steps

text: O

text: to

text: move

text: a

text: distance.

text: Thus

text: random

text: walk

text: exploration

text: tends

text: to

text: become

text: slow

text: for

text: large

text: p.

text: This

text: is

text: usually

text: cited

text: as

text: the

text: main

text: motivation

text: to

text: develop

text: more

text: elaborate

text: MCMC

text: strategies,

text: such

text: as

text: HMC,

text: which

text: we

text: cover

text: in

text: the

text: following

text: section.

text: Hamiltonian

text: Monte

text: Carlo

text: (HMC,

text: also

text: known

text: as

text: Hybrid

text: Monte

text: Carlo,

text: Duane

text: et

text: al.,

text: is

text: a

text: new

text: type

text: of

text: MCMC

text: algorithm,

text: where

text: one

text: is

text: able

text: to

text: perform

text: several

text: steps

text: in

text: the

text: parameter

text: space

text: before

text: determining

text: if

text: the

text: new

text: position

text: is

text: accepted

text: or

text: not.

text: Consequently,

text: HMC

text: is

text: able

text: to

text: make

text: much

text: bigger

text: jumps

text: in

text: the

text: parameter

text: space

text: than

text: standard

text: Metropolis

text: algorithms.

text: See

text: Neal

text: for

text: an

text: excellent

text: introduction.

text: |D

text: Consider

text: the

text: pair

text: (β,

text: α),

text: where

text: β

text: p(β

text: and

text: α

text: N

text: M

text: thus

text: with

text: p

text: joint

text: un-normalised

text: density

text: exp

text: H(β,

text: α)

text: with

text: D|

text: H(β,

text: α)

text: E(β)

text: αT

text: Mα,

text: E(β)

text: log

text: p(β)p(

text: β)

text: The

text: physical

text: interpretation

text: of

text: HMC

text: is

text: that

text: of

text: a

text: particle

text: at

text: position

text: β,

text: with

text: velo-

text: city

text: α,

text: potential

text: energy

text: E(β),

text: kinetic

text: energy

text: Mα,

text: for

text: some

text: mass

text: matrix

text: M,

text: and

text: therefore

text: total

text: energy

text: given

text: by

text: H(β,

text: α).

text: The

text: particle

text: is

text: expected

text: to

text: follow

text: a

text: trajectory

text: such

text: that

text: H(β,

text: α)

text: remains

text: constant

text: over

text: time.

text: In

text: practice,

text: HMC

text: proceeds

text: as

text: follows:

text: ﬁrst,

text: sample

text: a

text: new

text: velocity

text: vector,

text: α

text: N

text: M

text: Second,

text: move

text: the

text: particle

text: while

text: keeping

text: the

text: Hamiltonian

text: H

text: con-

text: p

text: stant;

text: in

text: practice,

text: discretisation

text: must

text: be

text: used,

text: so

text: L

text: steps

text: of

text: step-size

text: are

text: performed

text: through

text: leap-frop

text: steps;

text: see

text: Algorithm

text: which

text: describes

text: one

text: such

text: step.

text: Third,

text: the

text: new

text: position,

text: obtained

text: after

text: L

text: leap-frog

text: steps

text: is

text: accepted

text: or

text: rejected

text: according

text: to

text: probability

text: exp

text: H(β,

text: α)

text: see

text: Algorithm

text: for

text: a

text: summary.

text: The

text: validity

text: of

text: the

text: algorithm

text: relies

text: on

text: the

text: fact

text: that

text: a

text: leap-frog

text: step

text: is

text: “volume

text: pre-

text: serving”;

text: that

text: is,

text: the

text: deterministic

text: transformation

text: (β,

text: α)

text: (β

text: α

text: has

text: Jacobian

text: one.

text: This

text: is

text: why

text: the

text: acceptance

text: probability

text: admits

text: this

text: simple

text: expression.

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##


text: LEAVE PIMA INDIANS ALONE 3 leads to the logistic model. Other choices could be considered, such as e.g. the CDF of a Student distribution (robit model) to better accommodate outliers. We follow Gelman et al. (2008)’s recommendation to standardise the predictors in a preliminary step: non-binary predictors have mean 0 and standard deviation 0.5, binary predictors have mean 0 and range 1, and the intercept (if present) is set to 1. This standardisation facilitates prior speciﬁcation: one then may set up a “weakly informative” prior for β, that is a proper prior that assigns a low probability that the marginal eﬀect of one predictor is outside a reasonable range. Speciﬁcally, we shall consider two priors p(β) in this work: (a) the default prior recommended by Gelman et al. (2008), a product of independent Cauchys with centre 0 and scale 10 for the constant predictor, 2.5 for all the other predictors (henceforth, the Cauchy prior); and (b) a product of independent Gaussians with mean 0 and standard deviation equal to twice the scale of the Cauchy prior (henceforth the Gaussian prior). Of course, other priors could be considered, such as e.g. Jeﬀreys’ prior (Firth, 1993), or a Laplace prior (Kabán, 2007). Our main point in considering the two priors above is to determine to which extent certain Bayesian computation methods may be prior-dependent, either in their implementation (e.g. Gibbs sampling) or in their performance, or both. In particular, one may expect the Cauchy prior to be more diﬃcult to deal with, given its heavy tails. We explain in this section how to quickly compute the mode, and the Hessian at the mode, of the posterior: ˆ D| |D p(β)p( β) D D| p(β ) = D , p( ) = p(β)p( β) dβ, p( ) R d D where p(β) is one of the two priors presented in the previous section, and Z( ) is the marginal likelihood of the data (also known as the evidence). These quantities will prove useful later, in particular to tune certain of the considered methods. The two ﬁrst derivatives of the log-posterior density may be computed as: ∂ |D ∂ ∂ D| log p(β ) = log p(β) + log p( β), ∂β ∂β ∂β ∂2 |D ∂2 ∂2 D| log p(β ) = log p(β) + log p( β) ∂β∂βT ∂β∂βT ∂β∂βT (cid:88) where ∂ D| nD (cid:48) log p( β) = (cid:88) (log F) (y βT x )y x ∂β i i i i i=1 ∂2 D| nD (cid:48)(cid:48) log p( β) = (log F) (y βT x )x xT ∂β∂βT i i i i i=1 (cid:48) (cid:48)(cid:48) and (log F) and (log F) are the two ﬁrst derivatives of log F. Provided that log F is concave, which is the case for probit and logit regressions, the Hessian of the log-likelihood is clearly a negative deﬁnite matrix. Moreover, if we consider the Gaussian prior, then the Hessian is of the log-posterior is also negative (as the sum of two negative matrices, as Gaussian densities are log-concave). We stick to the Gaussian prior for now. This suggests the following standard approach to compute the MAP (maximum a posterior) estimator, that is the point β that maximises the posterior density |D MAP p(β ): to use Newton-Raphson, that is, to iterate ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine##

text: LEAVE

text: PIMA

text: INDIANS

text: ALONE

text: leads

text: to

text: the

text: logistic

text: model.

text: Other

text: choices

text: could

text: be

text: considered,

text: such

text: as

text: e.g.

text: the

text: CDF

text: of

text: a

text: Student

text: distribution

text: (robit

text: model)

text: to

text: better

text: accommodate

text: outliers.

text: We

text: follow

text: Gelman

text: et

text: al.

text: recommendation

text: to

text: standardise

text: the

text: predictors

text: in

text: a

text: preliminary

text: step:

text: non-binary

text: predictors

text: have

text: mean

text: and

text: standard

text: deviation

text: binary

text: predictors

text: have

text: mean

text: and

text: range

text: and

text: the

text: intercept

text: (if

text: present)

text: is

text: set

text: to

text: This

text: standardisation

text: facilitates

text: prior

text: speciﬁcation:

text: one

text: then

text: may

text: set

text: up

text: a

text: “weakly

text: informative”

text: prior

text: for

text: β,

text: that

text: is

text: a

text: proper

text: prior

text: that

text: assigns

text: a

text: low

text: probability

text: that

text: the

text: marginal

text: eﬀect

text: of

text: one

text: predictor

text: is

text: outside

text: a

text: reasonable

text: range.

text: Speciﬁcally,

text: we

text: shall

text: consider

text: two

text: priors

text: p(β)

text: in

text: this

text: work:

text: (a)

text: the

text: default

text: prior

text: recommended

text: by

text: Gelman

text: et

text: al.

text: a

text: product

text: of

text: independent

text: Cauchys

text: with

text: centre

text: and

text: scale

text: for

text: the

text: constant

text: predictor,

text: for

text: all

text: the

text: other

text: predictors

text: (henceforth,

text: the

text: Cauchy

text: prior);

text: and

text: (b)

text: a

text: product

text: of

text: independent

text: Gaussians

text: with

text: mean

text: and

text: standard

text: deviation

text: equal

text: to

text: twice

text: the

text: scale

text: of

text: the

text: Cauchy

text: prior

text: (henceforth

text: the

text: Gaussian

text: prior).

text: Of

text: course,

text: other

text: priors

text: could

text: be

text: considered,

text: such

text: as

text: e.g.

text: Jeﬀreys’

text: prior

text: (Firth,

text: or

text: a

text: Laplace

text: prior

text: (Kabán,

text: Our

text: main

text: point

text: in

text: considering

text: the

text: two

text: priors

text: above

text: is

text: to

text: determine

text: to

text: which

text: extent

text: certain

text: Bayesian

text: computation

text: methods

text: may

text: be

text: prior-dependent,

text: either

text: in

text: their

text: implementation

text: (e.g.

text: Gibbs

text: sampling)

text: or

text: in

text: their

text: performance,

text: or

text: both.

text: In

text: particular,

text: one

text: may

text: expect

text: the

text: Cauchy

text: prior

text: to

text: be

text: more

text: diﬃcult

text: to

text: deal

text: with,

text: given

text: its

text: heavy

text: tails.

text: We

text: explain

text: in

text: this

text: section

text: how

text: to

text: quickly

text: compute

text: the

text: mode,

text: and

text: the

text: Hessian

text: at

text: the

text: mode,

text: of

text: the

text: posterior:

text: ˆ

text: D|

text: |D

text: p(β)p(

text: β)

text: D

text: D|

text: p(β

text: D

text: p(

text: p(β)p(

text: β)

text: dβ,

text: p(

text: R

text: d

text: D

text: where

text: p(β)

text: is

text: one

text: of

text: the

text: two

text: priors

text: presented

text: in

text: the

text: previous

text: section,

text: and

text: Z(

text: is

text: the

text: marginal

text: likelihood

text: of

text: the

text: data

text: (also

text: known

text: as

text: the

text: evidence).

text: These

text: quantities

text: will

text: prove

text: useful

text: later,

text: in

text: particular

text: to

text: tune

text: certain

text: of

text: the

text: considered

text: methods.

text: The

text: two

text: ﬁrst

text: derivatives

text: of

text: the

text: log-posterior

text: density

text: may

text: be

text: computed

text: as:

text: |D

text: D|

text: log

text: p(β

text: log

text: p(β)

text: log

text: p(

text: β),

text: ∂β

text: ∂β

text: ∂β

text: ∂2

text: |D

text: ∂2

text: ∂2

text: D|

text: log

text: p(β

text: log

text: p(β)

text: log

text: p(

text: β)

text: ∂β∂βT

text: ∂β∂βT

text: ∂β∂βT

text: where

text: D|

text: nD

text: log

text: p(

text: β)

text: (log

text: F)

text: (y

text: βT

text: x

text: )y

text: x

text: ∂β

text: i

text: i

text: i

text: i

text: ∂2

text: D|

text: nD

text: log

text: p(

text: β)

text: (log

text: F)

text: (y

text: βT

text: x

text: )x

text: xT

text: ∂β∂βT

text: i

text: i

text: i

text: i

text: and

text: (log

text: F)

text: and

text: (log

text: F)

text: are

text: the

text: two

text: ﬁrst

text: derivatives

text: of

text: log

text: F.

text: Provided

text: that

text: log

text: F

text: is

text: concave,

text: which

text: is

text: the

text: case

text: for

text: probit

text: and

text: logit

text: regressions,

text: the

text: Hessian

text: of

text: the

text: log-likelihood

text: is

text: clearly

text: a

text: negative

text: deﬁnite

text: matrix.

text: Moreover,

text: if

text: we

text: consider

text: the

text: Gaussian

text: prior,

text: then

text: the

text: Hessian

text: is

text: of

text: the

text: log-posterior

text: is

text: also

text: negative

text: (as

text: the

text: sum

text: of

text: two

text: negative

text: matrices,

text: as

text: Gaussian

text: densities

text: are

text: log-concave).

text: We

text: stick

text: to

text: the

text: Gaussian

text: prior

text: for

text: now.

text: This

text: suggests

text: the

text: following

text: standard

text: approach

text: to

text: compute

text: the

text: MAP

text: (maximum

text: a

text: posterior)

text: estimator,

text: that

text: is

text: the

text: point

text: β

text: that

text: maximises

text: the

text: posterior

text: density

text: |D

text: MAP

text: p(β

text: to

text: use

text: Newton-Raphson,

text: that

text: is,

text: to

text: iterate

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##


text: LEAVE PIMA INDIANS ALONE 23 No algorithm seems to vastly outperform the others consistently across the three datasets. If anything, RWMH seems to show consistently best or second best performance. Still, these results oﬀer the following insights. Again, we see that Gibbs sampling, despite being a specialised algorithm, does not outperform signiﬁcantly more generic algorithms. Recall that the probit/Gaussian scenario is very favourable to Gibbs sampling; in other scenarios (results not shown), Gibbs is strongly dominated by other algorithms. More surprisingly, RWHM still performs well despite the high dimension. In addition, RHHM seems more robust than SMC to an imperfect calibration; see the DNA example, where the error of the EP approximation is greater. On the other hand, SMC is more amenable to parallelisation, hence on a parallel architecture, SMC would be likely to outperform the other approaches. 6. Variable selection We discuss in this section the implications of our ﬁndings on variable selection. The standard way to formalise variable selection is to introduce as a parameter the ∈ { } binary vector γ 0, 1 p, and to deﬁne(cid:89)the likelihood D| nD p( β, γ) = F(y βT x ) i γ γ,i i=1 | | where β (resp. x ) is the vector of length γ that one obtains by excluding γ γ,i from β (resp. x ) the components j such that γ = 0. Several priors may be i j considered for this problem (Chipman et al., 2001), but for simplicity, we will take p(β, γ) = p(β)p(γ) where p(β) is either the Cauchy prior or the Gaussian prior discussed in Section ??, and p(γ) is the uniform distribution with respect to the { } − set 0, 1 p, p(γ) = 2 p. Computationally, variable selection is more challenging than parameter estima- |D tion, because the posterior p(β, γ ) is a mixture of discrete and continuous com- ponents. If p is small, one may simply perform a complete enumeration: for all the D| 2p possible values of γ, approximate p( γ) using e.g. importance sampling. If p is large, one may adapt the approach of Schäfer and Chopin (2011), as described in the next sections. ∼ In linear regression, y = βT x + ε , ε N (0, σ2), the marginal likelihood D| i γ γ,i i i 1 p( γ) is available in close form (for a certain class of priors). Schäfer and Chopin (2011) use this property to construct a tempering SMC sampler, which trans- |D itions from the prior p(γ) to the posterior p(γ ), through the tempering sequence ∝ D| πt(γ) p(γ)p( γ)δt, with δt growing from 0 to 1. This algorithm has the same structure as Algorithm 4 (with the obvious replacements of the β’s by γ’s and so on.) The only diﬀerence is the MCMC step used to diversify the particles after res- ampling. Instead of a random walk step (which would be ill-deﬁned on a discrete space), Schäfer and Chopin (2011) use a Metropolis step based on an independent proposal, constructed from a sequence of nested logistic regressions: proposal for ﬁrst component γ is Bernoulli, proposal for second component γ , conditional on 1 2 γ , corresponds to a logistic regression with γ and an intercept as covariates, and 1 1 so on. The parameters of these p successive regressions are simply estimated from the current particle system. Schäfer and Chopin (2011) show that their algorithm signiﬁcantly outperform several MCMC samplers on datasets with more than 100 covariates.

text: LEAVE

text: PIMA

text: INDIANS

text: ALONE

text: No

text: algorithm

text: seems

text: to

text: vastly

text: outperform

text: the

text: others

text: consistently

text: across

text: the

text: three

text: datasets.

text: If

text: anything,

text: RWMH

text: seems

text: to

text: show

text: consistently

text: best

text: or

text: second

text: best

text: performance.

text: Still,

text: these

text: results

text: oﬀer

text: the

text: following

text: insights.

text: Again,

text: we

text: see

text: that

text: Gibbs

text: sampling,

text: despite

text: being

text: a

text: specialised

text: algorithm,

text: does

text: not

text: outperform

text: signiﬁcantly

text: more

text: generic

text: algorithms.

text: Recall

text: that

text: the

text: probit/Gaussian

text: scenario

text: is

text: very

text: favourable

text: to

text: Gibbs

text: sampling;

text: in

text: other

text: scenarios

text: (results

text: not

text: shown),

text: Gibbs

text: is

text: strongly

text: dominated

text: by

text: other

text: algorithms.

text: More

text: surprisingly,

text: RWHM

text: still

text: performs

text: well

text: despite

text: the

text: high

text: dimension.

text: In

text: addition,

text: RHHM

text: seems

text: more

text: robust

text: than

text: SMC

text: to

text: an

text: imperfect

text: calibration;

text: see

text: the

text: DNA

text: example,

text: where

text: the

text: error

text: of

text: the

text: EP

text: approximation

text: is

text: greater.

text: On

text: the

text: other

text: hand,

text: SMC

text: is

text: more

text: amenable

text: to

text: parallelisation,

text: hence

text: on

text: a

text: parallel

text: architecture,

text: SMC

text: would

text: be

text: likely

text: to

text: outperform

text: the

text: other

text: approaches.

text: Variable

text: selection

text: We

text: discuss

text: in

text: this

text: section

text: the

text: implications

text: of

text: our

text: ﬁndings

text: on

text: variable

text: selection.

text: The

text: standard

text: way

text: to

text: formalise

text: variable

text: selection

text: is

text: to

text: introduce

text: as

text: a

text: parameter

text: the

text: binary

text: vector

text: γ

text: p,

text: and

text: to

text: likelihood

text: D|

text: nD

text: p(

text: β,

text: γ)

text: F(y

text: βT

text: x

text: i

text: γ

text: γ,i

text: where

text: β

text: (resp.

text: x

text: is

text: the

text: vector

text: of

text: length

text: γ

text: that

text: one

text: obtains

text: by

text: excluding

text: γ

text: γ,i

text: from

text: β

text: (resp.

text: x

text: the

text: components

text: j

text: such

text: that

text: γ

text: Several

text: priors

text: may

text: be

text: i

text: j

text: considered

text: for

text: this

text: problem

text: (Chipman

text: et

text: al.,

text: but

text: for

text: simplicity,

text: we

text: will

text: take

text: p(β,

text: γ)

text: p(β)p(γ)

text: where

text: p(β)

text: is

text: either

text: the

text: Cauchy

text: prior

text: or

text: the

text: Gaussian

text: prior

text: discussed

text: in

text: Section

text: and

text: p(γ)

text: is

text: the

text: uniform

text: distribution

text: with

text: respect

text: to

text: the

text: set

text: p,

text: p(γ)

text: p.

text: Computationally,

text: variable

text: selection

text: is

text: more

text: challenging

text: than

text: parameter

text: estima-

text: |D

text: tion,

text: because

text: the

text: posterior

text: p(β,

text: γ

text: is

text: a

text: mixture

text: of

text: discrete

text: and

text: continuous

text: com-

text: ponents.

text: If

text: p

text: is

text: small,

text: one

text: may

text: simply

text: perform

text: a

text: complete

text: enumeration:

text: for

text: all

text: the

text: D|

text: possible

text: values

text: of

text: γ,

text: approximate

text: p(

text: γ)

text: using

text: e.g.

text: importance

text: sampling.

text: If

text: p

text: is

text: large,

text: one

text: may

text: adapt

text: the

text: approach

text: of

text: Schäfer

text: and

text: Chopin

text: as

text: described

text: in

text: the

text: next

text: sections.

text: In

text: linear

text: regression,

text: y

text: βT

text: x

text: ε

text: ε

text: N

text: the

text: marginal

text: likelihood

text: D|

text: i

text: γ

text: γ,i

text: i

text: i

text: p(

text: γ)

text: is

text: available

text: in

text: close

text: form

text: (for

text: a

text: certain

text: class

text: of

text: priors).

text: Schäfer

text: and

text: Chopin

text: use

text: this

text: property

text: to

text: construct

text: a

text: tempering

text: SMC

text: sampler,

text: which

text: trans-

text: |D

text: itions

text: from

text: the

text: prior

text: p(γ)

text: to

text: the

text: posterior

text: p(γ

text: through

text: the

text: tempering

text: sequence

text: D|

text: πt(γ)

text: p(γ)p(

text: γ)δt,

text: with

text: δt

text: growing

text: from

text: to

text: This

text: algorithm

text: has

text: the

text: same

text: structure

text: as

text: Algorithm

text: (with

text: the

text: obvious

text: replacements

text: of

text: the

text: β’s

text: by

text: γ’s

text: and

text: so

text: on.)

text: The

text: only

text: diﬀerence

text: is

text: the

text: MCMC

text: step

text: used

text: to

text: diversify

text: the

text: particles

text: after

text: res-

text: ampling.

text: Instead

text: of

text: a

text: random

text: walk

text: step

text: (which

text: would

text: be

text: ill-deﬁned

text: on

text: a

text: discrete

text: space),

text: Schäfer

text: and

text: Chopin

text: use

text: a

text: Metropolis

text: step

text: based

text: on

text: an

text: independent

text: proposal,

text: constructed

text: from

text: a

text: sequence

text: of

text: nested

text: logistic

text: regressions:

text: proposal

text: for

text: ﬁrst

text: component

text: γ

text: is

text: Bernoulli,

text: proposal

text: for

text: second

text: component

text: γ

text: conditional

text: on

text: γ

text: corresponds

text: to

text: a

text: logistic

text: regression

text: with

text: γ

text: and

text: an

text: intercept

text: as

text: covariates,

text: and

text: so

text: on.

text: The

text: parameters

text: of

text: these

text: p

text: successive

text: regressions

text: are

text: simply

text: estimated

text: from

text: the

text: current

text: particle

text: system.

text: Schäfer

text: and

text: Chopin

text: show

text: that

text: their

text: algorithm

text: signiﬁcantly

text: outperform

text: several

text: MCMC

text: samplers

text: on

text: datasets

text: with

text: more

text: than

text: covariates.


text: LEAVE PIMA INDIANS ALONE 25 (cid:89) (cid:8) (cid:9) p − (cid:28) p(β) = λN (β ; 0, v2) + (1 λ)N (β ; 0, v2) , v2 v2 1 j 0 1 j 1 0 1 j=1 ∈ where λ (0, 1), v2 and v2 are ﬁxed hyper-parameters. This prior generates a 0 1 continuous posterior (without point masses at β = 0), which is easier to sample j from than the discrete-continuous mixture obtained in the standard formulation of Bayesian variable selection. It would be interesting to see to which extent our discussion and ﬁndings extend to this particular type of posteriors; see for instance Hernández-Lobato et al. (2013) for how to deal with such priors in EP. 7. Conclusion and extensions Our ﬁrst and perhaps most important message to end users is that Bayesian com- putation (for binary regression) is now suﬃciently fast for routine use: if the right approach is used, results may be obtained near instantly on a standard computer, at least on simple datasets. Concretely, as far as binary regression is concerned, our main recommendation is to always use EP. It is very fast, and its approximation error is negligible in most cases (for such models). EP requires some expertise to implement, but the second author will release shortly a R package that computes the EP approximation for any logit or probit model. The only drawback of EP is the current lack of theoretical support. We learnt however while ﬁnishing this manuscript that Simon Barthelmé and Guillaume Dehaene (personal communication) established that the error rate O − of EP is (nD2) in certain models (where nD is the sample size). This seems to explain why EP often performs so well. In case one wishes to assess the EP error, by running in a second step some exact algorithm, we would recommend to use the SMC approach outlined in Section ?? (i.e. with initial particles simulated from the EP approximation). Often, this SMC sampler will reduce to a single importance sampling step, and will perform extremely well. Even when it does not, it should provide decent performance, especially if run on (and implemented for) a parallel architecture. Alternatively, on a single-core machine, random walk Metropolis is particularly simple to implement, and performs surprisingly well on high-dimensional data (when properly calibrated using EP). Our main message to Bayesian computation scientists was already in the title of this paper: leave Pima Indians alone, and more generally, let’s all refrain from now on from using datasets and models that are too simple to serve as a reasonable benchmark. To elaborate, let’s distinguish between specialised algorithms and generic al- gorithms. For algorithms specialised to a given model and a given prior (i.e. Gibbs samplers), the choice of a “benchmark” reduces to the choice of a dataset. It seems unfortunate that such algorithms are often showcased on small datasets (20 covariates or less), for which simpler, more generic methods perform much better. As a matter of fact, we saw in our simulations that even for bigger datasets Gibbs sampling does not seem to oﬀer better performance than generic methods. For generic algorithms (Metropolis, HMC, and so on), the choice of a benchmark amounts to the choice of a target distribution. A common practice in papers pro- posing some novel algorithm for Bayesian computation is to compare that algorithm

text: LEAVE

text: PIMA

text: INDIANS

text: ALONE

text: p

text: p(β)

text: λN

text: (β

text: λ)N

text: (β

text: j

text: j

text: where

text: λ

text: and

text: are

text: ﬁxed

text: hyper-parameters.

text: This

text: prior

text: generates

text: a

text: continuous

text: posterior

text: (without

text: point

text: masses

text: at

text: β

text: which

text: is

text: easier

text: to

text: sample

text: j

text: from

text: than

text: the

text: discrete-continuous

text: mixture

text: obtained

text: in

text: the

text: standard

text: formulation

text: of

text: Bayesian

text: variable

text: selection.

text: It

text: would

text: be

text: interesting

text: to

text: see

text: to

text: which

text: extent

text: our

text: discussion

text: and

text: ﬁndings

text: extend

text: to

text: this

text: particular

text: type

text: of

text: posteriors;

text: see

text: for

text: instance

text: Hernández-Lobato

text: et

text: al.

text: for

text: how

text: to

text: deal

text: with

text: such

text: priors

text: in

text: EP.

text: Conclusion

text: and

text: extensions

text: Our

text: ﬁrst

text: and

text: perhaps

text: most

text: important

text: message

text: to

text: end

text: users

text: is

text: that

text: Bayesian

text: com-

text: putation

text: (for

text: binary

text: regression)

text: is

text: now

text: suﬃciently

text: fast

text: for

text: routine

text: use:

text: if

text: the

text: right

text: approach

text: is

text: used,

text: results

text: may

text: be

text: obtained

text: near

text: instantly

text: on

text: a

text: standard

text: computer,

text: at

text: least

text: on

text: simple

text: datasets.

text: Concretely,

text: as

text: far

text: as

text: binary

text: regression

text: is

text: concerned,

text: our

text: main

text: recommendation

text: is

text: to

text: always

text: use

text: EP.

text: It

text: is

text: very

text: fast,

text: and

text: its

text: approximation

text: error

text: is

text: negligible

text: in

text: most

text: cases

text: (for

text: such

text: models).

text: EP

text: requires

text: some

text: expertise

text: to

text: implement,

text: but

text: the

text: second

text: author

text: will

text: release

text: shortly

text: a

text: R

text: package

text: that

text: computes

text: the

text: EP

text: approximation

text: for

text: any

text: logit

text: or

text: probit

text: model.

text: The

text: only

text: drawback

text: of

text: EP

text: is

text: the

text: current

text: lack

text: of

text: theoretical

text: support.

text: We

text: learnt

text: however

text: while

text: ﬁnishing

text: this

text: manuscript

text: that

text: Simon

text: Barthelmé

text: and

text: Guillaume

text: Dehaene

text: (personal

text: communication)

text: established

text: that

text: the

text: error

text: rate

text: O

text: of

text: EP

text: is

text: in

text: certain

text: models

text: (where

text: nD

text: is

text: the

text: sample

text: size).

text: This

text: seems

text: to

text: explain

text: why

text: EP

text: often

text: performs

text: so

text: well.

text: In

text: case

text: one

text: wishes

text: to

text: assess

text: the

text: EP

text: error,

text: by

text: running

text: in

text: a

text: second

text: step

text: some

text: exact

text: algorithm,

text: we

text: would

text: recommend

text: to

text: use

text: the

text: SMC

text: approach

text: outlined

text: in

text: Section

text: (i.e.

text: with

text: initial

text: particles

text: simulated

text: from

text: the

text: EP

text: approximation).

text: Often,

text: this

text: SMC

text: sampler

text: will

text: reduce

text: to

text: a

text: single

text: importance

text: sampling

text: step,

text: and

text: will

text: perform

text: extremely

text: well.

text: Even

text: when

text: it

text: does

text: not,

text: it

text: should

text: provide

text: decent

text: performance,

text: especially

text: if

text: run

text: on

text: (and

text: implemented

text: for)

text: a

text: parallel

text: architecture.

text: Alternatively,

text: on

text: a

text: single-core

text: machine,

text: random

text: walk

text: Metropolis

text: is

text: particularly

text: simple

text: to

text: implement,

text: and

text: performs

text: surprisingly

text: well

text: on

text: high-dimensional

text: data

text: (when

text: properly

text: calibrated

text: using

text: EP).

text: Our

text: main

text: message

text: to

text: Bayesian

text: computation

text: scientists

text: was

text: already

text: in

text: the

text: title

text: of

text: this

text: paper:

text: leave

text: Pima

text: Indians

text: alone,

text: and

text: more

text: generally,

text: let’s

text: all

text: refrain

text: from

text: now

text: on

text: from

text: using

text: datasets

text: and

text: models

text: that

text: are

text: too

text: simple

text: to

text: serve

text: as

text: a

text: reasonable

text: benchmark.

text: To

text: elaborate,

text: let’s

text: distinguish

text: between

text: specialised

text: algorithms

text: and

text: generic

text: al-

text: gorithms.

text: For

text: algorithms

text: specialised

text: to

text: a

text: given

text: model

text: and

text: a

text: given

text: prior

text: (i.e.

text: Gibbs

text: samplers),

text: the

text: choice

text: of

text: a

text: “benchmark”

text: reduces

text: to

text: the

text: choice

text: of

text: a

text: dataset.

text: It

text: seems

text: unfortunate

text: that

text: such

text: algorithms

text: are

text: often

text: showcased

text: on

text: small

text: datasets

text: covariates

text: or

text: less),

text: for

text: which

text: simpler,

text: more

text: generic

text: methods

text: perform

text: much

text: better.

text: As

text: a

text: matter

text: of

text: fact,

text: we

text: saw

text: in

text: our

text: simulations

text: that

text: even

text: for

text: bigger

text: datasets

text: Gibbs

text: sampling

text: does

text: not

text: seem

text: to

text: oﬀer

text: better

text: performance

text: than

text: generic

text: methods.

text: For

text: generic

text: algorithms

text: (Metropolis,

text: HMC,

text: and

text: so

text: on),

text: the

text: choice

text: of

text: a

text: benchmark

text: amounts

text: to

text: the

text: choice

text: of

text: a

text: target

text: distribution.

text: A

text: common

text: practice

text: in

text: papers

text: pro-

text: posing

text: some

text: novel

text: algorithm

text: for

text: Bayesian

text: computation

text: is

text: to

text: compare

text: that

text: algorithm