text: 8 S. P. C. Peters et al. Figure 4. Polar plots for two of our galaxies, IC1515 (upper) and UGC12208 (lower). On the left the polar plot before rectiﬁcation and on the right after rectiﬁcation according to the procedure described in section at a surface brightness of 26 mag/arcsec2. The minor axis ± is at position angles 0 and π. in the inner, brighter parts of the galaxy, well above the We then set out to quantify our results by measuring pixel-to-pixel noise, and as such, they are visible in all four the parameters of each feature. We deﬁned ﬁt-regions to the types of proﬁles. The rectiﬁed polar proﬁles are intended proﬁle on both sides of a feature (shown in Table 2). In each for detecting truncations. We classiﬁed truncations of the of these regions we have a set of radii R and observed surface − proﬁles, when at levels fainter than 26 mag arcsec 2 the brightness levels µ (R), to which we ﬁt a linear relation obs proﬁle deﬁnitely changes slope and drops towards the sky with scale length h and central brightness µ at R = 0 of 0 before disappearing in the noise. NGC 1090 has a break at form − 26.1 mag arcsec 2, but has a very minor change in slope. We do not classify it as a truncation, but a break instead. R Haloes are the opposite of truncations; when rather than µ(R; µ , h) = µ + 1.086 . (3) 0 0 h steepening, the proﬁle becomes less steep. Haloes should be visible in both the rectiﬁed polar proﬁles and in the ellipse proﬁles. As we expect truncations and haloes to occur at The ﬁts were performed on the ellipse-ﬁtting proﬁles, the outer edges of the proﬁles, deep below the pixel-to-pixel as the EP proﬁles in general do not extend suﬃciently far noise, the PAS and EP methods are not used. out, the PAS proﬁles suﬀer from convolution and the RPP To ensure unbiased results from the data, our team split results have been produced by stretching and compressing up into groups. Each group went through all of the proﬁles the data. and classiﬁed what they believed to be breaks, truncations We give the ﬁtting algorithm the freedom to estimate and/or haloes. After that, we joined up again to discuss our the noise by itself, in a method inspired by the use of the ﬁndings and agree on any diﬃcult cases. jitter parameter in Hou et al. (2012). We assume that the (cid:13) c 2015 RAS, MNRAS 000, 1–44 ##LTFigure## ##LTFigure## ##LTFigure## ##LTFigure## ##LTLine##

text: S.

text: P.

text: C.

text: Peters

text: et

text: al.

text: Figure

text: Polar

text: plots

text: for

text: two

text: of

text: our

text: galaxies,

text: (upper)

text: and

text: (lower).

text: On

text: the

text: left

text: the

text: polar

text: plot

text: before

text: rectiﬁcation

text: and

text: on

text: the

text: right

text: after

text: rectiﬁcation

text: according

text: to

text: the

text: procedure

text: described

text: in

text: section

text: at

text: a

text: surface

text: brightness

text: of

text: The

text: minor

text: axis

text: is

text: at

text: position

text: angles

text: and

text: π.

text: in

text: the

text: inner,

text: brighter

text: parts

text: of

text: the

text: galaxy,

text: well

text: above

text: the

text: We

text: then

text: set

text: out

text: to

text: quantify

text: our

text: results

text: by

text: measuring

text: pixel-to-pixel

text: noise,

text: and

text: as

text: such,

text: they

text: are

text: visible

text: in

text: all

text: four

text: the

text: parameters

text: of

text: each

text: feature.

text: We

text: deﬁned

text: ﬁt-regions

text: to

text: the

text: types

text: of

text: proﬁles.

text: The

text: rectiﬁed

text: polar

text: proﬁles

text: are

text: intended

text: proﬁle

text: on

text: both

text: sides

text: of

text: a

text: feature

text: (shown

text: in

text: Table

text: In

text: each

text: for

text: detecting

text: truncations.

text: We

text: classiﬁed

text: truncations

text: of

text: the

text: of

text: these

text: regions

text: we

text: have

text: a

text: set

text: of

text: radii

text: R

text: and

text: observed

text: surface

text: proﬁles,

text: when

text: at

text: levels

text: fainter

text: than

text: mag

text: arcsec

text: the

text: brightness

text: levels

text: µ

text: (R),

text: to

text: which

text: we

text: ﬁt

text: a

text: linear

text: relation

text: obs

text: proﬁle

text: deﬁnitely

text: changes

text: slope

text: and

text: drops

text: towards

text: the

text: sky

text: with

text: scale

text: length

text: h

text: and

text: central

text: brightness

text: µ

text: at

text: R

text: of

text: before

text: disappearing

text: in

text: the

text: noise.

text: NGC

text: has

text: a

text: break

text: at

text: form

text: mag

text: arcsec

text: but

text: has

text: a

text: very

text: minor

text: change

text: in

text: slope.

text: We

text: do

text: not

text: classify

text: it

text: as

text: a

text: truncation,

text: but

text: a

text: break

text: instead.

text: R

text: Haloes

text: are

text: the

text: opposite

text: of

text: truncations;

text: when

text: rather

text: than

text: µ(R;

text: µ

text: h)

text: µ

text: h

text: steepening,

text: the

text: proﬁle

text: becomes

text: less

text: steep.

text: Haloes

text: should

text: be

text: visible

text: in

text: both

text: the

text: rectiﬁed

text: polar

text: proﬁles

text: and

text: in

text: the

text: ellipse

text: proﬁles.

text: As

text: we

text: expect

text: truncations

text: and

text: haloes

text: to

text: occur

text: at

text: The

text: ﬁts

text: were

text: performed

text: on

text: the

text: ellipse-ﬁtting

text: proﬁles,

text: the

text: outer

text: edges

text: of

text: the

text: proﬁles,

text: deep

text: below

text: the

text: pixel-to-pixel

text: as

text: the

text: EP

text: proﬁles

text: in

text: general

text: do

text: not

text: extend

text: suﬃciently

text: far

text: noise,

text: the

text: PAS

text: and

text: EP

text: methods

text: are

text: not

text: used.

text: out,

text: the

text: PAS

text: proﬁles

text: suﬀer

text: from

text: convolution

text: and

text: the

text: RPP

text: To

text: ensure

text: unbiased

text: results

text: from

text: the

text: data,

text: our

text: team

text: split

text: results

text: have

text: been

text: produced

text: by

text: stretching

text: and

text: compressing

text: up

text: into

text: groups.

text: Each

text: group

text: went

text: through

text: all

text: of

text: the

text: proﬁles

text: the

text: data.

text: and

text: classiﬁed

text: what

text: they

text: believed

text: to

text: be

text: breaks,

text: truncations

text: We

text: give

text: the

text: ﬁtting

text: algorithm

text: the

text: freedom

text: to

text: estimate

text: and/or

text: haloes.

text: After

text: that,

text: we

text: joined

text: up

text: again

text: to

text: discuss

text: our

text: the

text: noise

text: by

text: itself,

text: in

text: a

text: method

text: inspired

text: by

text: the

text: use

text: of

text: the

text: ﬁndings

text: and

text: agree

text: on

text: any

text: diﬃcult

text: cases.

text: jitter

text: parameter

text: in

text: Hou

text: et

text: al.

text: We

text: assume

text: that

text: the

text: c

text: RAS,

text: MNRAS

text: ##LTFigure##

text: ##LTFigure##

text: ##LTFigure##

text: ##LTFigure##

text: ##LTLine##


text: predictive text systems, recommender systems and so on, the space to a point where it can be easily processed. Each di- model embodies the biases and norms of the majority. mension in the vector represent a linguistic context, and the It is not always easy to see biases and norms when they are representation should preserve characteristics of the original taken for granted and pervade your reality. Fortunately, for word (Goldberg and Levy 2014). many of the computational assistance tools based on massive Such mappings have been achieved using various tech- datasets there is a way to drastically highlight or foreground niques, such as neural networks (Bengio, Ducharme, and the biases in the dataset, namely to train the models on a Vincent 2003), principal component analysis (Lebret and completely different dataset. In this paper we explore the Collobert 2013), and probabilistic models (Globerson et role of biases inherent in training data in predictive text al- al. 2007). A popular method is skip-gram with negative- gorithms through creating a system trained not on “neutral” sampling training, a context-predictive approach imple- text but on the works of Chuck Tingle. mented in word2vec models (Mikolov et al. 2013). On Chuck Tingle is a renowned Hugo award nominated au- the other hand, global vectors (GloVe) is a context-count thor of fantastic gay erotica. His work can be seen as erot- word embedding technique (Pennington, Socher, and Man- ica, science ﬁction, absurdist comedy, political satire, met- ning 2014). GloVe captures the probability of a word ap- aliterature, or preferably all these things and more at the pearing in a certain context in relation to the remaining text. same time. The books frequently feature gay sex with uni- Neural Networks and Recurrent Neural Networks corns, dinosaurs, winged derrie`res, chocolate milk cowboys, Neural networks (NN) are a machine learning tech- and abstract entities such as Monday or the very story you nique originally inspired by the way the human brain func- are reading right now. The bizarre plotlines feature various tions (Hornik, Stinchcombe, and White 1989). The basic landscapes, from paradise islands and secretive science labs, unit of a NN is a neuron. Neurons receive vectors as inputs, to underground clubs and luxury condos inside the protago- and output values by applying a non linear function to the nist’s own posterior. The corpus of Chuck Tingle’s collected multiplication of said vectors and a set of weights. They are works is a good choice to train our models on precisely be- usually grouped in layers, and neurons in the same layer can- cause they so egregiously violate neutral text conventions, not be connected to each other. Neurons in a given layer are not only in terms of topics, but also narrative structure, word fully connected to all neurons in the following layer. NNs choice and good taste. They are also surprisingly consistent can be trained using the backpropagation algorithm. Back- in style, despite the highly varied subjects. Finally, Chuck propagation updates the network weights by taking small Tingle is a very proliﬁc author, providing us with a large steps in the direction of minimizing the error measured by corpus to train our models on. In fact, the consistency and the network. idiosyncracy of his literary style together with his marvelous A recurrent neural network (RNN) is a special case of productivity has led more than one observer to speculate neural network. In a RNN, the output of each layer depends about whether Chuck Tingle is actually a computer program, not only on the input to the layer, but also on the previous an irony not lost on us. output. RNNs are trained using backpropagation through In this paper, we ask the question what would happen if time (BPTT) (Werbos 1990), an algorithm that unfolds the our writing support systems did not assume that we wanted recursive nature of the network for a given amount of steps, to write like normal people, but instead assumed that we and applies a generic backpropagation to the unfolded RNN. wanted to write like Chuck Tingle. We train a deep neural Unfortunately, BPTT doesn’t suit vanilla RNNs when they net based on Long Short-Term Memory and word-level em- run for large amount of steps (Hochreiter 1998). One solu- beddings to predict Chuck Tingle’s writings, and using this tion for this problemis the use of Long Short-Term Memory model we build a couple of tools (a predictive text system (LSTM). LSTMs were introduced by Sepp Hochreiter and and a reimagining of literary classics) that assists you with Ju¨rgen Schmidhuber ( 1997), and introduces a memory unit. getting your text exactly right, i.e. to write just like Chuck The memory unit acts as a storage device for the previous in- Tingle would have. put values. The input is added to the old memory state using A secondary goal of the research is to investigate how well gates. These gates control the percentage of new values con- we can learn to generate text that mimics the style of Chuck tributing to the memory unit with respect to the old stored Tingle from his collected works. The more general question values. Using gates helps to sustain constant optimization is that of generative modeling of literary style using modern through each time step. machine learning methods. The highly distinctive style of Natural Language Generation Tingle’s writing presumably makes it easy to verify whether the generated text adheres to his style. Natural language generation approaches can be divided Background into two categories: Rule- or template-based and machine learning (Tang et al. 2016). Rule-based (or template- This work builds on a set of methods from modern ma- based) approaches (Cheyer and Guzzoni 2014; Mirkovic chine learning, in particular in the form of deep learning. and Cavedon 2011) were considered norm for most sys- Word Embedding tems, with rules/templates handmade. However, these tend Word embedding is a technique for converting words into to be too specialized, not generalizing well to different do- a n-dimensional vector of real numbers, capable of captur- mains, and a large amount of templates is necessary to gen- ing probabilistic features of the words in the current text. erate quality text even on a small domain. Some effort The primary goal is to reduce the dimensionality of the word has been made towards generating the template based on

text: predictive

label: paragraph

text: text

text: systems,

text: recommender

text: systems

text: and

text: so

text: on,

text: the

text: space

text: to

text: a

text: point

text: where

text: it

text: can

text: be

text: easily

text: processed.

text: Each

text: di-

text: model

text: embodies

text: the

text: biases

text: and

text: norms

text: of

text: the

text: majority.

text: mension

text: in

text: the

text: vector

text: represent

text: a

text: linguistic

text: context,

text: and

text: the

text: It

text: is

text: not

text: always

text: easy

text: to

text: see

text: biases

text: and

text: norms

text: when

text: they

text: are

text: representation

text: should

text: preserve

text: characteristics

text: of

text: the

text: original

text: taken

text: for

text: granted

text: and

text: pervade

text: your

text: reality.

text: Fortunately,

text: for

text: word

text: (Goldberg

text: and

text: Levy

text: many

text: of

text: the

text: computational

text: assistance

text: tools

text: based

text: on

text: massive

text: Such

text: mappings

text: have

text: been

text: achieved

text: using

text: various

text: tech-

text: datasets

text: there

text: is

text: a

text: way

text: to

text: drastically

text: highlight

text: or

text: foreground

text: niques,

text: such

text: as

text: neural

text: networks

text: (Bengio,

text: Ducharme,

text: and

text: the

text: biases

text: in

text: the

text: dataset,

text: namely

text: to

text: train

text: the

text: models

text: on

text: a

text: Vincent

text: principal

text: component

text: analysis

text: (Lebret

text: and

text: completely

text: different

text: dataset.

text: In

text: this

text: paper

text: we

text: explore

text: the

text: Collobert

text: and

text: probabilistic

text: models

text: (Globerson

text: et

text: role

text: of

text: biases

text: inherent

text: in

text: training

text: data

text: in

text: predictive

text: text

text: al-

text: al.

text: A

text: popular

text: method

text: is

text: skip-gram

text: with

text: negative-

text: gorithms

text: through

text: creating

text: a

text: system

text: trained

text: not

text: on

text: “neutral”

text: sampling

text: training,

text: a

text: context-predictive

text: approach

text: imple-

text: text

text: but

text: on

text: the

text: works

text: of

text: Chuck

text: Tingle.

text: mented

text: in

text: models

text: (Mikolov

text: et

text: al.

text: On

text: Chuck

text: Tingle

text: is

text: a

text: renowned

text: Hugo

text: award

text: nominated

text: au-

text: the

text: other

text: hand,

text: global

text: vectors

text: (GloVe)

text: is

text: a

text: context-count

text: thor

text: of

text: fantastic

text: gay

text: erotica.

text: His

text: work

text: can

text: be

text: seen

text: as

text: erot-

text: word

text: embedding

text: technique

text: (Pennington,

text: Socher,

text: and

text: Man-

text: ica,

text: science

text: ﬁction,

text: absurdist

text: comedy,

text: political

text: satire,

text: met-

text: ning

text: GloVe

text: captures

text: the

text: probability

text: of

text: a

text: word

text: ap-

text: aliterature,

text: or

text: preferably

text: all

text: these

text: things

text: and

text: more

text: at

text: the

text: pearing

text: in

text: a

text: certain

text: context

text: in

text: relation

text: to

text: the

text: remaining

text: text.

text: same

text: time.

text: The

text: books

text: frequently

text: feature

text: gay

text: sex

text: with

text: uni-

text: Neural

text: Networks

text: and

text: Recurrent

text: Neural

text: Networks

text: corns,

text: dinosaurs,

text: winged

text: derrie`res,

text: chocolate

text: milk

text: cowboys,

text: Neural

text: networks

text: (NN)

text: are

text: a

text: machine

text: learning

text: tech-

text: and

text: abstract

text: entities

text: such

text: as

text: Monday

text: or

text: the

text: very

text: story

text: you

text: nique

text: originally

text: inspired

text: by

text: the

text: way

text: the

text: human

text: brain

text: func-

text: are

text: reading

text: right

text: now.

text: The

text: bizarre

text: plotlines

text: feature

text: various

text: tions

text: (Hornik,

text: Stinchcombe,

text: and

text: White

text: The

text: basic

text: landscapes,

text: from

text: paradise

text: islands

text: and

text: secretive

text: science

text: labs,

text: unit

text: of

text: a

text: NN

text: is

text: a

text: neuron.

text: Neurons

text: receive

text: vectors

text: as

text: inputs,

text: to

text: underground

text: clubs

text: and

text: luxury

text: condos

text: inside

text: the

text: protago-

text: and

text: output

text: values

text: by

text: applying

text: a

text: non

text: linear

text: function

text: to

text: the

text: nist’s

text: own

text: posterior.

text: The

text: corpus

text: of

text: Chuck

text: Tingle’s

text: collected

text: multiplication

text: of

text: said

text: vectors

text: and

text: a

text: set

text: of

text: weights.

text: They

text: are

text: works

text: is

text: a

text: good

text: choice

text: to

text: train

text: our

text: models

text: on

text: precisely

text: be-

text: usually

text: grouped

text: in

text: layers,

text: and

text: neurons

text: in

text: the

text: same

text: layer

text: can-

text: cause

text: they

text: so

text: egregiously

text: violate

text: neutral

text: text

text: conventions,

text: not

text: be

text: connected

text: to

text: each

text: other.

text: Neurons

text: in

text: a

text: given

text: layer

text: are

text: not

text: only

text: in

text: terms

text: of

text: topics,

text: but

text: also

text: narrative

text: structure,

text: word

text: fully

text: connected

text: to

text: all

text: neurons

text: in

text: the

text: following

text: layer.

text: NNs

text: choice

text: and

text: good

text: taste.

text: They

text: are

text: also

text: surprisingly

text: consistent

text: can

text: be

text: trained

text: using

text: the

text: backpropagation

text: algorithm.

text: Back-

text: in

text: style,

text: despite

text: the

text: highly

text: varied

text: subjects.

text: Finally,

text: Chuck

text: propagation

text: updates

text: the

text: network

text: weights

text: by

text: taking

text: small

text: Tingle

text: is

text: a

text: very

text: proliﬁc

text: author,

text: providing

text: us

text: with

text: a

text: large

text: steps

text: in

text: the

text: direction

text: of

text: minimizing

text: the

text: error

text: measured

text: by

text: corpus

text: to

text: train

text: our

text: models

text: on.

text: In

text: fact,

text: the

text: consistency

text: and

text: the

text: network.

text: idiosyncracy

text: of

text: his

text: literary

text: style

text: together

text: with

text: his

text: marvelous

text: A

text: recurrent

text: neural

text: network

text: (RNN)

text: is

text: a

text: special

text: case

text: of

text: productivity

text: has

text: led

text: more

text: than

text: one

text: observer

text: to

text: speculate

text: neural

text: network.

text: In

text: a

text: RNN,

text: the

text: output

text: of

text: each

text: layer

text: depends

text: about

text: whether

text: Chuck

text: Tingle

text: is

text: actually

text: a

text: computer

text: program,

text: not

text: only

text: on

text: the

text: input

text: to

text: the

text: layer,

text: but

text: also

text: on

text: the

text: previous

text: an

text: irony

text: not

text: lost

text: on

text: us.

text: output.

text: RNNs

text: are

text: trained

text: using

text: backpropagation

text: through

text: In

text: this

text: paper,

text: we

text: ask

text: the

text: question

text: what

text: would

text: happen

text: if

text: time

text: (BPTT)

text: (Werbos

text: an

text: algorithm

text: that

text: unfolds

text: the

text: our

text: writing

text: support

text: systems

text: did

text: not

text: assume

text: that

text: we

text: wanted

text: recursive

text: nature

text: of

text: the

text: network

text: for

text: a

text: given

text: amount

text: of

text: steps,

text: to

text: write

text: like

text: normal

text: people,

text: but

text: instead

text: assumed

text: that

text: we

text: and

text: applies

text: a

text: generic

text: backpropagation

text: to

text: the

text: unfolded

text: RNN.

text: wanted

text: to

text: write

text: like

text: Chuck

text: Tingle.

text: We

text: train

text: a

text: deep

text: neural

text: Unfortunately,

text: BPTT

text: doesn’t

text: suit

text: vanilla

text: RNNs

text: when

text: they

text: net

text: based

text: on

text: Long

text: Short-Term

text: Memory

text: and

text: word-level

text: em-

text: run

text: for

text: large

text: amount

text: of

text: steps

text: (Hochreiter

text: One

text: solu-

text: beddings

text: to

text: predict

text: Chuck

text: Tingle’s

text: writings,

text: and

text: using

text: this

text: tion

text: for

text: this

text: problemis

text: the

text: use

text: of

text: Long

text: Short-Term

text: Memory

text: model

text: we

text: build

text: a

text: couple

text: of

text: tools

text: (a

text: predictive

text: text

text: system

text: (LSTM).

text: LSTMs

text: were

text: introduced

text: by

text: Sepp

text: Hochreiter

text: and

text: and

text: a

text: reimagining

text: of

text: literary

text: classics)

text: that

text: assists

text: you

text: with

text: Ju¨rgen

text: Schmidhuber

text: and

text: introduces

text: a

text: memory

text: unit.

text: getting

text: your

text: text

text: exactly

text: right,

text: i.e.

text: to

text: write

text: just

text: like

text: Chuck

text: The

text: memory

text: unit

text: acts

text: as

text: a

text: storage

text: device

text: for

text: the

text: previous

text: in-

text: Tingle

text: would

text: have.

text: put

text: values.

text: The

text: input

text: is

text: added

text: to

text: the

text: old

text: memory

text: state

text: using

text: A

text: secondary

text: goal

text: of

text: the

text: research

text: is

text: to

text: investigate

text: how

text: well

text: gates.

text: These

text: gates

text: control

text: the

text: percentage

text: of

text: new

text: values

text: con-

text: we

text: can

text: learn

text: to

text: generate

text: text

text: that

text: mimics

text: the

text: style

text: of

text: Chuck

text: tributing

text: to

text: the

text: memory

text: unit

text: with

text: respect

text: to

text: the

text: old

text: stored

text: Tingle

text: from

text: his

text: collected

text: works.

text: The

text: more

text: general

text: question

text: values.

text: Using

text: gates

text: helps

text: to

text: sustain

text: constant

text: optimization

text: is

text: that

text: of

text: generative

text: modeling

text: of

text: literary

text: style

text: using

text: modern

text: through

text: each

text: time

text: step.

text: machine

text: learning

text: methods.

text: The

text: highly

text: distinctive

text: style

text: of

text: Natural

text: Language

text: Generation

text: Tingle’s

text: writing

text: presumably

text: makes

text: it

text: easy

text: to

text: verify

text: whether

text: the

text: generated

text: text

text: adheres

text: to

text: his

text: style.

text: Natural

text: language

text: generation

text: approaches

text: can

text: be

text: divided

text: Background

text: into

text: two

text: categories:

text: Rule-

text: or

text: template-based

text: and

text: machine

text: learning

text: (Tang

text: et

text: al.

text: Rule-based

text: (or

text: template-

text: This

text: work

text: builds

text: on

text: a

text: set

text: of

text: methods

text: from

text: modern

text: ma-

text: based)

text: approaches

text: (Cheyer

text: and

text: Guzzoni

text: Mirkovic

text: chine

text: learning,

text: in

text: particular

text: in

text: the

text: form

text: of

text: deep

text: learning.

text: and

text: Cavedon

text: were

text: considered

text: norm

text: for

text: most

text: sys-

text: Word

text: Embedding

text: tems,

text: with

text: rules/templates

text: handmade.

text: However,

text: these

text: tend

text: Word

text: embedding

text: is

text: a

text: technique

text: for

text: converting

text: words

text: into

text: to

text: be

text: too

text: specialized,

text: not

text: generalizing

text: well

text: to

text: different

text: do-

text: a

text: n-dimensional

text: vector

text: of

text: real

text: numbers,

text: capable

text: of

text: captur-

text: mains,

text: and

text: a

text: large

text: amount

text: of

text: templates

text: is

text: necessary

text: to

text: gen-

text: ing

text: probabilistic

text: features

text: of

text: the

text: words

text: in

text: the

text: current

text: text.

text: erate

text: quality

text: text

text: even

text: on

text: a

text: small

text: domain.

text: Some

text: effort

text: The

text: primary

text: goal

text: is

text: to

text: reduce

text: the

text: dimensionality

text: of

text: the

text: word

text: has

text: been

text: made

text: towards

text: generating

text: the

text: template

text: based

text: on


text: Grammar Coherence Interesting ∗ ∗ CT vs DT 16/23 19/27 17/31 ∗∗ ∗∗ ∗∗ CT vs Markov 29/31 31/33 26/33 ∗∗ ∗∗ DT vs Markov 17/21 21/27 11/19 Table 1: Table shows the result of the user study where CT is Chuck Tingle’s original text, Markov is the Markov chain generated text, and DT is the DeepTingle generated text. The superscript indicate the p-value from using binomial ∗ ∗∗ test. indicated that the p-value is less than 5%, while indicates the p-value is less than 1%. the sequence sizes against different percentages of missing words from the input text. It shows that the more words we have the better the results except for sizes 3 and 4. At these sizes, 20% missing data means nothing change. We chose size 6 as it is higher than the others, and at the same time won’t compromise the neural network speed. User Study We performed a user study to compare the generated text Figure 6: Screenshot of Predictive Tingle. Shows the input by DeepTingle to Chuck Tingle’s original text. Addition- box with an example text, and a sugestion of the next word. ally, we wanted to conﬁrm if a neural network would actu- ally have an advantage over a simpler representation, such as a Markov chain model. We trained a Markov chain on the The ﬁnal part of the tools is Tingle Classics, shown in Fig- same data set, and chose the state size to be 3 as it empiri- ure 7. From top to bottom, the screen shows the tool’s name cally achieved the best results without losing generalization and description, followed by a list of books, to be selected ability. by the user. A button, ”Generate!”, triggers the word gen- In the user study, the user is presented with two pieces of eration. A line, right bellow the bottom, shows the original text of equal length picked randomly from any of the 3 cate- initial line for the book selected. Two conﬁgurations options gories of text (Chuck Tingle’s original text, DeepTingle text, can be found in sequence: the option of toggle substitution and Markov chain text). The user has to answer 3 questions: on and off, and the amount of words to generate. Finally, the “Which text is more grammatically correct?”; “Which text story generated is outputted at the very bottom of the page. is more interesting?”; and “Which text is more coherent?’. If substitution is selected, a preprocessing of the initial The user could pick one of four options: “Left text is better”, line is made, transforming every word in the original text “Right text is better”, “Both are the same”, or “None”. that doesn’t appear in the Tingle corpus, into a Tingle word. We collected approximately 146 different comparisons. Thus, it guarantees that every word in the input vector ap- Table 1 presents the results of comparisons, excluding all pears in the Tingle corpus. If substitution is not used, words choices for “Both are the same” or “None of them”. The not in the Tingle corpus are skipped. For example, if the values represent the fraction of times the ﬁrst text is voted sentence is “Hello, my name is Inigo Montoya”, and nei- over the second one. Results show that using neural net- ther “Inigo” nor “Montoya” belong in the corpus, the vec- works for text prediction produce more coherent and gram- tor would shift to embed only “Hello, my name is” (notice matically correct text than Markov chain, but less so than that the comma is considered a word). This may result in the original text, which is reasonable considering the latter diverging stories, as shown in Examples 3 and 4. Both is written and reviewed by a human. are generated from the same line (“Call me Ishmael”, from Predictive Tingle Moby-Dick, by Herman Melville), but the ﬁrst doesn’t use Figure 6 shows a screenshot of the system: On top we substitution, while the second does. have a brief description of what Predictive Tingle is. Right Conclusion and Future Work below, a text ﬁeld where the user can write text. To the text This paper proposes a two-part system, composed of a ﬁeld’s right, a purple suggestion button that is updated ev- deep neural network trained over a speciﬁc literary corpus ery time the user presses the spacebar. In this example, the and a writing assistance tool built on the network. Our cor- user wrote “It was raining in New York”, and pressed enter pus consists solely of works by renowned author Chuck Tin- consecutively, allowing the system to ﬁnish the input. The gle. This corpus represents a large set of stories, diverse in outcome was “It was raining in New York city. It’s not long setting and context, but similar in structure. Its controver- before the familiar orgasmic sensations begin to bubble up sial themes negates the “neutral’ norm of writing assistance within me once again, spilling out through my veins like sim- tools currently available. We trained a six layer architecture, mering erotic venom.” using GloVe embeding, LSTMs, dense and softmax layers, Tingle Classics capable of word sequence prediction. Our system allows ##LTFigure## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine##

text: Grammar

label: table

text: Coherence

text: Interesting

text: CT

text: vs

text: DT

text: CT

text: vs

text: Markov

text: DT

text: vs

text: Markov

text: Table

text: Table

label: caption

text: shows

text: the

text: result

text: of

text: the

text: user

text: study

text: where

text: CT

text: is

text: Chuck

text: Tingle’s

text: original

text: text,

text: Markov

text: is

text: the

text: Markov

text: chain

text: generated

text: text,

text: and

text: DT

text: is

text: the

text: DeepTingle

text: generated

text: text.

text: The

text: superscript

text: indicate

text: the

text: p-value

text: from

text: using

text: binomial

text: test.

text: indicated

text: that

text: the

text: p-value

text: is

text: less

text: than

text: while

text: indicates

text: the

text: p-value

text: is

text: less

text: than

text: the

text: sequence

text: sizes

text: against

text: different

text: percentages

text: of

text: missing

text: words

text: from

text: the

text: input

text: text.

text: It

text: shows

text: that

text: the

text: more

text: words

text: we

text: have

text: the

text: better

text: the

text: results

text: except

text: for

text: sizes

text: and

text: At

text: these

text: sizes,

text: missing

text: data

text: means

text: nothing

text: change.

text: We

text: chose

text: size

text: as

text: it

text: is

text: higher

text: than

text: the

text: others,

text: and

text: at

text: the

text: same

text: time

text: won’t

text: compromise

text: the

text: neural

text: network

text: speed.

text: User

text: Study

text: We

text: performed

text: a

text: user

text: study

text: to

text: compare

text: the

text: generated

text: text

text: Figure

text: Screenshot

text: of

text: Predictive

text: Tingle.

text: Shows

text: the

text: input

text: by

text: DeepTingle

text: to

text: Chuck

text: Tingle’s

text: original

text: text.

text: Addition-

text: box

text: with

text: an

text: example

text: text,

text: and

text: a

text: sugestion

text: of

text: the

text: next

text: word.

text: ally,

text: we

text: wanted

text: to

text: conﬁrm

text: if

text: a

text: neural

text: network

text: would

text: actu-

text: ally

text: have

text: an

text: advantage

text: over

text: a

text: simpler

text: representation,

text: such

text: as

text: a

text: Markov

text: chain

text: model.

text: We

text: trained

text: a

text: Markov

text: chain

text: on

text: the

text: The

text: ﬁnal

text: part

text: of

text: the

text: tools

text: is

text: Tingle

text: Classics,

text: shown

text: in

text: Fig-

text: same

text: data

text: set,

text: and

text: chose

text: the

text: state

text: size

text: to

text: be

text: as

text: it

text: empiri-

text: ure

text: From

text: top

text: to

text: bottom,

text: the

text: screen

text: shows

text: the

text: tool’s

text: name

text: cally

text: achieved

text: the

text: best

text: results

text: without

text: losing

text: generalization

text: and

text: description,

text: followed

text: by

text: a

text: list

text: of

text: books,

text: to

text: be

text: selected

text: ability.

text: by

text: the

text: user.

text: A

text: button,

text: ”Generate!”,

text: triggers

text: the

text: word

text: gen-

text: In

text: the

text: user

text: study,

text: the

text: user

text: is

text: presented

text: with

text: two

text: pieces

text: of

text: eration.

text: A

text: line,

text: right

text: bellow

text: the

text: bottom,

text: shows

text: the

text: original

text: text

text: of

text: equal

text: length

text: picked

text: randomly

text: from

text: any

text: of

text: the

text: cate-

text: initial

text: line

text: for

text: the

text: book

text: selected.

text: Two

text: conﬁgurations

text: options

text: gories

text: of

text: text

text: (Chuck

text: Tingle’s

text: original

text: text,

text: DeepTingle

text: text,

text: can

text: be

text: found

text: in

text: sequence:

text: the

text: option

text: of

text: toggle

text: substitution

text: and

text: Markov

text: chain

text: text).

text: The

text: user

text: has

text: to

text: answer

text: questions:

text: on

text: and

text: off,

text: and

text: the

text: amount

text: of

text: words

text: to

text: generate.

text: Finally,

text: the

text: “Which

text: text

text: is

text: more

text: grammatically

text: correct?”;

text: “Which

text: text

text: story

text: generated

text: is

text: outputted

text: at

text: the

text: very

text: bottom

text: of

text: the

text: page.

text: is

text: more

text: interesting?”;

text: and

text: “Which

text: text

text: is

text: more

text: coherent?’.

text: If

text: substitution

text: is

text: selected,

text: a

text: preprocessing

text: of

text: the

text: initial

text: The

text: user

text: could

text: pick

text: one

text: of

text: four

text: options:

text: “Left

text: text

text: is

text: better”,

text: line

text: is

text: made,

text: transforming

text: every

text: word

text: in

text: the

text: original

text: text

text: “Right

text: text

text: is

text: better”,

text: “Both

text: are

text: the

text: same”,

text: or

text: “None”.

text: that

text: doesn’t

text: appear

text: in

text: the

text: Tingle

text: corpus,

text: into

text: a

text: Tingle

text: word.

text: We

text: collected

text: approximately

text: different

text: comparisons.

text: Thus,

text: it

text: guarantees

text: that

text: every

text: word

text: in

text: the

text: input

text: vector

text: ap-

text: Table

text: presents

text: the

text: results

text: of

text: comparisons,

text: excluding

text: all

text: pears

text: in

text: the

text: Tingle

text: corpus.

text: If

text: substitution

text: is

text: not

text: used,

text: words

text: choices

text: for

text: “Both

text: are

text: the

text: same”

text: or

text: “None

text: of

text: them”.

text: The

text: not

text: in

text: the

text: Tingle

text: corpus

text: are

text: skipped.

text: For

text: example,

text: if

text: the

text: values

text: represent

text: the

text: fraction

text: of

text: times

text: the

text: ﬁrst

text: text

text: is

text: voted

text: sentence

text: is

text: “Hello,

text: my

text: name

text: is

text: Inigo

text: Montoya”,

text: and

text: nei-

text: over

text: the

text: second

text: one.

text: Results

text: show

text: that

text: using

text: neural

text: net-

text: ther

text: “Inigo”

text: nor

text: “Montoya”

text: belong

text: in

text: the

text: corpus,

text: the

text: vec-

text: works

text: for

text: text

text: prediction

text: produce

text: more

text: coherent

text: and

text: gram-

text: tor

text: would

text: shift

text: to

text: embed

text: only

text: “Hello,

text: my

text: name

text: is”

text: (notice

text: matically

text: correct

text: text

text: than

text: Markov

text: chain,

text: but

text: less

text: so

text: than

text: that

text: the

text: comma

text: is

text: considered

text: a

text: word).

text: This

text: may

text: result

text: in

text: the

text: original

text: text,

text: which

text: is

text: reasonable

text: considering

text: the

text: latter

text: diverging

text: stories,

text: as

text: shown

text: in

text: Examples

text: and

text: Both

text: is

text: written

text: and

text: reviewed

text: by

text: a

text: human.

text: are

text: generated

text: from

text: the

text: same

text: line

text: (“Call

text: me

text: Ishmael”,

text: from

text: Predictive

text: Tingle

text: Moby-Dick,

text: by

text: Herman

text: Melville),

text: but

text: the

text: ﬁrst

text: doesn’t

text: use

text: Figure

text: shows

text: a

text: screenshot

text: of

text: the

text: system:

text: On

text: top

text: we

text: substitution,

text: while

text: the

text: second

text: does.

text: have

text: a

text: brief

text: description

text: of

text: what

text: Predictive

text: Tingle

text: is.

text: Right

text: Conclusion

text: and

text: Future

text: Work

text: below,

text: a

text: text

text: ﬁeld

text: where

text: the

text: user

text: can

text: write

text: text.

text: To

text: the

text: text

text: This

text: paper

text: proposes

text: a

text: two-part

text: system,

text: composed

text: of

text: a

text: ﬁeld’s

text: right,

text: a

text: purple

text: suggestion

text: button

text: that

text: is

text: updated

text: ev-

text: deep

text: neural

text: network

text: trained

text: over

text: a

text: speciﬁc

text: literary

text: corpus

text: ery

text: time

text: the

text: user

text: presses

text: the

text: spacebar.

text: In

text: this

text: example,

text: the

text: and

text: a

text: writing

text: assistance

text: tool

text: built

text: on

text: the

text: network.

text: Our

text: cor-

text: user

text: wrote

text: “It

text: was

text: raining

text: in

text: New

text: York”,

text: and

text: pressed

text: enter

text: pus

text: consists

text: solely

text: of

text: works

text: by

text: renowned

text: author

text: Chuck

text: Tin-

text: consecutively,

text: allowing

text: the

text: system

text: to

text: ﬁnish

text: the

text: input.

text: The

text: gle.

text: This

text: corpus

text: represents

text: a

text: large

text: set

text: of

text: stories,

text: diverse

text: in

text: outcome

text: was

text: “It

text: was

text: raining

text: in

text: New

text: York

text: city.

text: It’s

text: not

text: long

text: setting

text: and

text: context,

text: but

text: similar

text: in

text: structure.

text: Its

text: controver-

text: before

text: the

text: familiar

text: orgasmic

text: sensations

text: begin

text: to

text: bubble

text: up

text: sial

text: themes

text: negates

text: the

text: “neutral’

text: norm

text: of

text: writing

text: assistance

text: within

text: me

text: once

text: again,

text: spilling

text: out

text: through

text: my

text: veins

text: like

text: sim-

text: tools

text: currently

text: available.

text: We

text: trained

text: a

text: six

text: layer

text: architecture,

text: mering

text: erotic

text: venom.”

text: using

text: GloVe

text: embeding,

text: LSTMs,

text: dense

text: and

text: softmax

text: layers,

text: Tingle

text: Classics

text: capable

text: of

text: word

text: sequence

text: prediction.

text: Our

text: system

text: allows

text: ##LTFigure##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##


text: be inﬂuenced by knowledge of white-room decisions as I have suggested. By way of analogy, suppose we see Kim treating another person kindly, and this other person happens to stand to her left. Clearly, this will increase our credence that Kim would treat other people who stand to her left kindly. But it would be preposterous to not also increase our credence that Kim would treat people who stand to her right kindly, unless we have reason to believe that there is a fundamental asymmetry between left and right (e.g., if we know Kim does not hear well with her right ear and this causes her great frustration). The situation is similar in our context: unless we have a particular reason to believe that the color of the room is relevant to the decisions (as in the ﬁrst example of a neurological examination), the Dutch book goes through.8 4.3 Why susceptibility to (certain) Dutch books poses a problem Does susceptibility to diachronic Dutch books really indicate irrationality? This question has been discussed at length in earlier work [Hitchcock, 2004, Briggs, 2010], and I do not have much to add that is new, but it is worth revisiting the key points here. Some Dutch book arguments have been made that require the bookie to have information that Beauty does not. For exam- ple, consider the following Dutch book argument against a thirder, presented by Hitchcock [2004] precisely in order to highlight this issue. On Sunday, Beauty is oﬀered a bet that pays out 30 on Heads, which costs 15. Then, on Monday, Beauty is oﬀered a bet that pays out 30 on Tails, which costs 20. The argument is that she is willing to accept both bets – in particular, she is willing to accept the latter because she at that point believes the probability of Tails is 2/3 – but accepting both bets is sure to result in a loss of 5. The problem is that oﬀering the second bet requires the bookie to know something that Beauty does not, namely, that it is Monday and not Tuesday. If he did not know this, he could end up selling the second bet to her twice in the Tails world, which would cause the Dutch book to fail.9 Now, susceptibility to being Dutch-booked by a bookie that has additional information does not seem to indicate a failure of rationality. After all, consider the extreme case where a deceptive bookie already knows the outcomes of the bets in advance. In such a case it is not surprising that he can choose which bet to oﬀer to ensure himself a proﬁt. (For a concrete example, see Briggs [2010].) Another take on this is that if Beauty is astute, then the event of being oﬀered a bet in fact provides her with additional information, which she should take into account when deciding whether to accept. If our thirder Beauty above knows that the second bet is only oﬀered to her on Mon- 8One might also suppose that knowledge of her decision in a white room makes Beauty only (say) 99% conﬁdent in what her decision would be in a black room, where the remaining 1% is intended to capture a small probability that the room color is somehow relevant to the decision. It is easy to see that the Dutch book still goes through under these conditions. 9As already pointed out by Hitchcock, to be precise, what information the bookie has is not exactly what is at issue. If the bookie does not know what day it is, but someone else prevents the bookie from oﬀering the second bet on Tuesday to make the Dutch book work, this is just as problematic. The point is that the process as a whole by which bets are oﬀered to Beauty cannot use information that is unavailable to Beauty. 11 ##LTLine##

text: be

text: inﬂuenced

text: by

text: knowledge

text: of

text: white-room

text: decisions

text: as

text: I

text: have

text: suggested.

text: By

text: way

text: of

text: analogy,

text: suppose

text: we

text: see

text: Kim

text: treating

text: another

text: person

text: kindly,

text: and

text: this

text: other

text: person

text: happens

text: to

text: stand

text: to

text: her

text: left.

text: Clearly,

text: this

text: will

text: increase

text: our

text: credence

text: that

text: Kim

text: would

text: treat

text: other

text: people

text: who

text: stand

text: to

text: her

text: left

text: kindly.

text: But

text: it

text: would

text: be

text: preposterous

text: to

text: not

text: also

text: increase

text: our

text: credence

text: that

text: Kim

text: would

text: treat

text: people

text: who

text: stand

text: to

text: her

text: right

text: kindly,

text: unless

text: we

text: have

text: reason

text: to

text: believe

text: that

text: there

text: is

text: a

text: fundamental

text: asymmetry

text: between

text: left

text: and

text: right

text: (e.g.,

text: if

text: we

text: know

text: Kim

text: does

text: not

text: hear

text: well

text: with

text: her

text: right

text: ear

text: and

text: this

text: causes

text: her

text: great

text: frustration).

text: The

text: situation

text: is

text: similar

text: in

text: our

text: context:

text: unless

text: we

text: have

text: a

text: particular

text: reason

text: to

text: believe

text: that

text: the

text: color

text: of

text: the

text: room

text: is

text: relevant

text: to

text: the

text: decisions

text: (as

text: in

text: the

text: ﬁrst

text: example

text: of

text: a

text: neurological

text: examination),

text: the

text: Dutch

text: book

text: goes

text: Why

text: susceptibility

text: to

text: (certain)

text: Dutch

text: books

text: poses

text: a

text: problem

text: Does

text: susceptibility

text: to

text: diachronic

text: Dutch

text: books

text: really

text: indicate

text: irrationality?

text: This

text: question

text: has

text: been

text: discussed

text: at

text: length

text: in

text: earlier

text: work

text: [Hitchcock,

text: Briggs,

text: and

text: I

text: do

text: not

text: have

text: much

text: to

text: add

text: that

text: is

text: new,

text: but

text: it

text: is

text: worth

text: revisiting

text: the

text: key

text: points

text: here.

text: Some

text: Dutch

text: book

text: arguments

text: have

text: been

text: made

text: that

text: require

text: the

text: bookie

text: to

text: have

text: information

text: that

text: Beauty

text: does

text: not.

text: For

text: exam-

text: ple,

text: consider

text: the

text: following

text: Dutch

text: book

text: argument

text: against

text: a

text: thirder,

text: presented

text: by

text: Hitchcock

text: precisely

text: in

text: order

text: to

text: highlight

text: this

text: issue.

text: On

text: Sunday,

text: Beauty

text: is

text: oﬀered

text: a

text: bet

text: that

text: pays

text: out

text: on

text: Heads,

text: which

text: costs

text: Then,

text: on

text: Monday,

text: Beauty

text: is

text: oﬀered

text: a

text: bet

text: that

text: pays

text: out

text: on

text: Tails,

text: which

text: costs

text: The

text: argument

text: is

text: that

text: she

text: is

text: willing

text: to

text: accept

text: both

text: bets

text: in

text: particular,

text: she

text: is

text: willing

text: to

text: accept

text: the

text: latter

text: because

text: she

text: at

text: that

text: point

text: believes

text: the

text: probability

text: of

text: Tails

text: is

text: but

text: accepting

text: both

text: bets

text: is

text: sure

text: to

text: result

text: in

text: a

text: loss

text: of

text: The

text: problem

text: is

text: that

text: oﬀering

text: the

text: second

text: bet

text: requires

text: the

text: bookie

text: to

text: know

text: something

text: that

text: Beauty

text: does

text: not,

text: namely,

text: that

text: it

text: is

text: Monday

text: and

text: not

text: Tuesday.

text: If

text: he

text: did

text: not

text: know

text: this,

text: he

text: could

text: end

text: up

text: selling

text: the

text: second

text: bet

text: to

text: her

text: twice

text: in

text: the

text: Tails

text: world,

text: which

text: would

text: cause

text: the

text: Dutch

text: book

text: to

text: Now,

text: susceptibility

text: to

text: being

text: Dutch-booked

text: by

text: a

text: bookie

text: that

text: has

text: additional

text: information

text: does

text: not

text: seem

text: to

text: indicate

text: a

text: failure

text: of

text: rationality.

text: After

text: all,

text: consider

text: the

text: extreme

text: case

text: where

text: a

text: deceptive

text: bookie

text: already

text: knows

text: the

text: outcomes

text: of

text: the

text: bets

text: in

text: advance.

text: In

text: such

text: a

text: case

text: it

text: is

text: not

text: surprising

text: that

text: he

text: can

text: choose

text: which

text: bet

text: to

text: oﬀer

text: to

text: ensure

text: himself

text: a

text: proﬁt.

text: (For

text: a

text: concrete

text: example,

text: see

text: Briggs

text: Another

text: take

text: on

text: this

text: is

text: that

text: if

text: Beauty

text: is

text: astute,

text: then

text: the

text: event

text: of

text: being

text: oﬀered

text: a

text: bet

text: in

text: fact

text: provides

text: her

text: with

text: additional

text: information,

text: which

text: she

text: should

text: take

text: into

text: account

text: when

text: deciding

text: whether

text: to

text: accept.

text: If

text: our

text: thirder

text: Beauty

text: above

text: knows

text: that

text: the

text: second

text: bet

text: is

text: only

text: oﬀered

text: to

text: her

text: on

text: Mon-

text: might

text: also

text: suppose

text: that

text: knowledge

text: of

text: her

text: decision

text: in

text: a

text: white

text: room

text: makes

text: Beauty

text: only

text: (say)

text: conﬁdent

text: in

text: what

text: her

text: decision

text: would

text: be

text: in

text: a

text: black

text: room,

text: where

text: the

text: remaining

text: is

text: intended

text: to

text: capture

text: a

text: small

text: probability

text: that

text: the

text: room

text: color

text: is

text: somehow

text: relevant

text: to

text: the

text: decision.

text: It

text: is

text: easy

text: to

text: see

text: that

text: the

text: Dutch

text: book

text: still

text: goes

text: through

text: under

text: these

text: conditions.

text: already

text: pointed

text: out

text: by

text: Hitchcock,

text: to

text: be

text: precise,

text: what

text: information

text: the

text: bookie

text: has

text: is

text: not

text: exactly

text: what

text: is

text: at

text: issue.

text: If

text: the

text: bookie

text: does

text: not

text: know

text: what

text: day

text: it

text: is,

text: but

text: someone

text: else

text: prevents

text: the

text: bookie

text: from

text: oﬀering

text: the

text: second

text: bet

text: on

text: Tuesday

text: to

text: make

text: the

text: Dutch

text: book

text: work,

text: this

text: is

text: just

text: as

text: problematic.

text: The

text: point

text: is

text: that

text: the

text: process

text: as

text: a

text: whole

text: by

text: which

text: bets

text: are

text: oﬀered

text: to

text: Beauty

text: cannot

text: use

text: information

text: that

text: is

text: unavailable

text: to

text: Beauty.

text: ##LTLine##


text: W (1/4) WB (1/2) B (1/4) Sunday bet 1’ (W): 11 bet 1’ (W): -10 bet 1’ (W): 11 bet 1’ (B): 11 bet 1’ (B): -10 bet 1’ (B): 11 Monday bet 2 (W): -24 bet 2 (W): 9 (no bet for W) (no bet for B) bet 2 (B): 9 bet 2 (B): -24 total gain from accepting all bets -2 -2 -2 Figure 6: The table shows which bet is oﬀered when and (in parentheses) to whom, as well as the net gain from accepting the bet in the corresponding possible world, for the Dutch book adapted to the two-Beauties variant. · total gain of 2 9 = 18; she places credence 1/3 in the world W, where Black will not get to act, so that the loss from accepting the bet is only 24. Because · − · − (once again) (2/3) 18 (1/3) 24 = 12 8 = 4 > 0, she accepts the bet (and Black will do so as well, by symmetry). Does the fact that the two Beauties together are susceptible to a Dutch book indicate that they are irrational? This is certainly not as well established as it is for the case of a Dutch book for a single agent (as summarized in Subsection ), and some skepticism is in order. For example, it is well known in game theory that rational behavior by multiple agents can result in an outcome that is strongly Pareto dominated, i.e., there exists another outcome that all agents would strictly prefer. The Prisoner’s Dilemma is the standard example. However, such examples rely on the agents having diﬀerent preferences. In contrast, the two Beauties above have the exact same preferences. Also, it seems that the key properties that make a Dutch book convincing, as discussed in Subsection , still hold here. It is true that the bookie will have more information than either single Beauty alone. However, this is easily ﬁxed by stipulating that there are two bookies, also with a joint account, each of whom is assigned to sleep with and oﬀer bets to one of the Beauties. Then, again, being oﬀered a bet does not provide either Beauty with more information, and this remains true even if the bookies’ joint betting strategy is common knowledge at the outset. It appears, then, that the Dutch book argument presented in this paper deals a serious blow to evidential decision theory. Of course, evidential decision theory is often applied in settings where a decision provides evidence not about past or future decisions that are similar, or about decisions by another similar agent, but rather about something relevant in the environment – various types of brain lesion, a demon’s decision, etc. It does not appear that this Dutch book argument can be applied in such cases, so perhaps the evidential decision theorist can retreat to an appropriately restricted version of the theory (though it is not immediately clear whether and how this can be coherently done). Even upon such retreat, I believe the glancing blow to what remains of the theory should be cause for concern for anyone sympathetic to it. 13 ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine## ##LTLine##

text: W

text: WB

text: B

text: Sunday

text: bet

text: 1’

text: (W):

text: bet

text: 1’

text: (W):

text: bet

text: 1’

text: (W):

text: bet

text: 1’

text: (B):

text: bet

text: 1’

text: (B):

text: bet

text: 1’

text: (B):

text: Monday

text: bet

text: (W):

text: bet

text: (W):

text: (no

text: bet

text: for

text: W)

text: (no

text: bet

text: for

text: B)

text: bet

text: (B):

text: bet

text: (B):

text: total

text: gain

text: from

text: accepting

text: all

text: bets

text: Figure

text: The

text: table

text: shows

text: which

text: bet

text: is

text: oﬀered

text: when

text: and

text: (in

text: parentheses)

text: to

text: whom,

text: as

text: well

text: as

text: the

text: net

text: gain

text: from

text: accepting

text: the

text: bet

text: in

text: the

text: corresponding

text: possible

text: world,

text: for

text: the

text: Dutch

text: book

text: adapted

text: to

text: the

text: two-Beauties

text: variant.

text: total

text: gain

text: of

text: she

text: places

text: credence

text: in

text: the

text: world

text: W,

text: where

text: Black

text: will

text: not

text: get

text: to

text: act,

text: so

text: that

text: the

text: loss

text: from

text: accepting

text: the

text: bet

text: is

text: only

text: Because

text: (once

text: again)

text: she

text: accepts

text: the

text: bet

text: (and

text: Black

text: will

text: do

text: so

text: as

text: well,

text: by

text: symmetry).

text: Does

text: the

text: fact

text: that

text: the

text: two

text: Beauties

text: together

text: are

text: susceptible

text: to

text: a

text: Dutch

text: book

text: indicate

text: that

text: they

text: are

text: irrational?

text: This

text: is

text: certainly

text: not

text: as

text: well

text: established

text: as

text: it

text: is

text: for

text: the

text: case

text: of

text: a

text: Dutch

text: book

text: for

text: a

text: single

text: agent

text: (as

text: summarized

text: in

text: Subsection

text: and

text: some

text: skepticism

text: is

text: in

text: order.

text: For

text: example,

text: it

text: is

text: well

text: known

text: in

text: game

text: theory

text: that

text: rational

text: behavior

text: by

text: multiple

text: agents

text: can

text: result

text: in

text: an

text: outcome

text: that

text: is

text: strongly

text: Pareto

text: dominated,

text: i.e.,

text: there

text: exists

text: another

text: outcome

text: that

text: all

text: agents

text: would

text: strictly

text: prefer.

text: The

text: Prisoner’s

text: Dilemma

text: is

text: the

text: standard

text: example.

text: However,

text: such

text: examples

text: rely

text: on

text: the

text: agents

text: having

text: diﬀerent

text: preferences.

text: In

text: contrast,

text: the

text: two

text: Beauties

text: above

text: have

text: the

text: exact

text: same

text: preferences.

text: Also,

text: it

text: seems

text: that

text: the

text: key

text: properties

text: that

text: make

text: a

text: Dutch

text: book

text: convincing,

text: as

text: discussed

text: in

text: Subsection

text: still

text: hold

text: here.

text: It

text: is

text: true

text: that

text: the

text: bookie

text: will

text: have

text: more

text: information

text: than

text: either

text: single

text: Beauty

text: alone.

text: However,

text: this

text: is

text: easily

text: ﬁxed

text: by

text: stipulating

text: that

text: there

text: are

text: two

text: bookies,

text: also

text: with

text: a

text: joint

text: account,

text: each

text: of

text: whom

text: is

text: assigned

text: to

text: sleep

text: with

text: and

text: oﬀer

text: bets

text: to

text: one

text: of

text: the

text: Beauties.

text: Then,

text: again,

text: being

text: oﬀered

text: a

text: bet

text: does

text: not

text: provide

text: either

text: Beauty

text: with

text: more

text: information,

text: and

text: this

text: remains

text: true

text: even

text: if

text: the

text: bookies’

text: joint

text: betting

text: strategy

text: is

text: common

text: knowledge

text: at

text: the

text: outset.

text: It

text: appears,

text: then,

text: that

text: the

text: Dutch

text: book

text: argument

text: presented

text: in

text: this

text: paper

text: deals

text: a

text: serious

text: blow

text: to

text: evidential

text: decision

text: theory.

text: Of

text: course,

text: evidential

text: decision

text: theory

text: is

text: often

text: applied

text: in

text: settings

text: where

text: a

text: decision

text: provides

text: evidence

text: not

text: about

text: past

text: or

text: future

text: decisions

text: that

text: are

text: similar,

text: or

text: about

text: decisions

text: by

text: another

text: similar

text: agent,

text: but

text: rather

text: about

text: something

text: relevant

text: in

text: the

text: environment

text: various

text: types

text: of

text: brain

text: lesion,

text: a

text: demon’s

text: decision,

text: etc.

text: It

text: does

text: not

text: appear

text: that

text: this

text: Dutch

text: book

text: argument

text: can

text: be

text: applied

text: in

text: such

text: cases,

text: so

text: perhaps

text: the

text: evidential

text: decision

text: theorist

text: can

text: retreat

text: to

text: an

text: appropriately

text: restricted

text: version

text: of

text: the

text: theory

text: (though

text: it

text: is

text: not

text: immediately

text: clear

text: whether

text: and

text: how

text: this

text: can

text: be

text: coherently

text: done).

text: Even

text: upon

text: such

text: retreat,

text: I

text: believe

text: the

text: glancing

text: blow

text: to

text: what

text: remains

text: of

text: the

text: theory

text: should

text: be

text: cause

text: for

text: concern

text: for

text: anyone

text: sympathetic

text: to

text: it.

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##

text: ##LTLine##


text: Acknowledgments I thank the anonymous reviewers for many useful comments that have helped to signiﬁcantly improve the paper. References Frank Arntzenius. Reﬂections on Sleeping Beauty. Analysis, 62(1):53–62, 2002. Nick Bostrom. Anthropic Bias: Observation Selection Eﬀects in Science and Philosophy. Routledge, 2002. Nick Bostrom. Sleeping Beauty and Self-location: A Hybrid Model. Synthese, 157(1):59–78, 2007. Rachael Briggs. Putting a value on Beauty. In Tamar Szab´o Gendler and John Hawthorne, editors, Oxford Studies in Epistemology: Volume 3, pages 3–34. Oxford University Press, 2010. Kai Draper and Joel Pust. Diachronic Dutch Books and Sleeping Beauty. Syn- these, 164(2):281–287, 2008. Adam Elga. Self-locating belief and the Sleeping Beauty problem. Analysis, 60 (2):143–147, 2000. Adam Elga. Defeating Dr. Evil with self-locating belief. Philosophy and Phe- nomenological Research, 69(2):383–396, 2004. Joseph Y. Halpern. Sleeping Beauty reconsidered: Conditioning and reﬂection in asynchronous systems. In Tamar Szab´o Gendler and John Hawthorne, editors, Oxford Studies in Epistemology: Volume 3, pages 111–142. Oxford University Press, 2006. Christopher Hitchcock. Beauty and the bets. Synthese, 139(3):405–420, 2004. Christopher J. G. Meacham. Sleeping beauty and the dynamics of de se beliefs. Philosophical Studies, 138(2):245–269, 2008. John Pittard. When Beauties disagree: Why halfers should aﬃrm robust per- spectivalim. In Tamar Szab´o Gendler and John Hawthorne, editors, Oxford Studies in Epistemology: Volume 5. Oxford University Press, 2015. Forth- coming. Wolfgang Schwarz. Belief update across ﬁssion. British Journal for the Philos- ophy of Science, 2014a. Forthcoming. Wolfgang Schwarz. Lost memories and useless coins: Revisiting the absent- minded driver, 2014b. Michael G. Titelbaum. The relevance of self-locating beliefs. Philosophical Review, 117(4):555–605, 2008. 14

text: Acknowledgments

label: section

text: I

text: thank

text: the

text: anonymous

text: reviewers

text: for

text: many

text: useful

text: comments

text: that

text: have

text: helped

text: to

text: signiﬁcantly

text: improve

text: the

text: paper.

text: References

label: reference

text: Frank

text: Arntzenius.

text: Reﬂections

text: on

text: Sleeping

text: Beauty.

text: Analysis,

text: Nick

text: Bostrom.

text: Anthropic

text: Bias:

text: Observation

text: Selection

text: Eﬀects

text: in

text: Science

text: and

text: Philosophy.

text: Routledge,

text: Nick

text: Bostrom.

text: Sleeping

text: Beauty

text: and

text: Self-location:

text: A

text: Hybrid

text: Model.

text: Synthese,

text: Rachael

text: Briggs.

text: Putting

text: a

text: value

text: on

text: Beauty.

text: In

text: Tamar

text: Szab´o

text: Gendler

text: and

text: John

text: Hawthorne,

text: editors,

text: Oxford

text: Studies

text: in

text: Epistemology:

text: Volume

text: pages

text: Oxford

text: University

text: Press,

text: Kai

text: Draper

text: and

text: Joel

text: Pust.

text: Diachronic

text: Dutch

text: Books

text: and

text: Sleeping

text: Beauty.

text: Syn-

text: these,

text: Adam

text: Elga.

text: Self-locating

text: belief

text: and

text: the

text: Sleeping

text: Beauty

text: problem.

text: Analysis,

text: Adam

text: Elga.

text: Defeating

text: Dr.

text: Evil

text: with

text: self-locating

text: belief.

text: Philosophy

text: and

text: Phe-

text: nomenological

text: Research,

text: Joseph

text: Y.

text: Halpern.

text: Sleeping

text: Beauty

text: reconsidered:

text: Conditioning

text: and

text: reﬂection

text: in

text: asynchronous

text: systems.

text: In

text: Tamar

text: Szab´o

text: Gendler

text: and

text: John

text: Hawthorne,

text: editors,

text: Oxford

text: Studies

text: in

text: Epistemology:

text: Volume

text: pages

text: Oxford

text: University

text: Press,

text: Christopher

text: Hitchcock.

text: Beauty

text: and

text: the

text: bets.

text: Synthese,

text: Christopher

text: J.

text: G.

text: Meacham.

text: Sleeping

text: beauty

text: and

text: the

text: dynamics

text: of

text: de

text: se

text: beliefs.

text: Philosophical

text: Studies,

text: John

text: Pittard.

text: When

text: Beauties

text: disagree:

text: Why

text: halfers

text: should

text: aﬃrm

text: robust

text: per-

text: spectivalim.

text: In

text: Tamar

text: Szab´o

text: Gendler

text: and

text: John

text: Hawthorne,

text: editors,

text: Oxford

text: Studies

text: in

text: Epistemology:

text: Volume

text: Oxford

text: University

text: Press,

text: Forth-

text: coming.

text: Wolfgang

text: Schwarz.

text: Belief

text: update

text: across

text: ﬁssion.

text: British

text: Journal

text: for

text: the

text: Philos-

text: ophy

text: of

text: Science,

text: Forthcoming.

text: Wolfgang

text: Schwarz.

text: Lost

text: memories

text: and

text: useless

text: coins:

text: Revisiting

text: the

text: absent-

text: minded

text: driver,

text: Michael

text: G.

text: Titelbaum.

text: The

text: relevance

text: of

text: self-locating

text: beliefs.

text: Philosophical

text: Review,


text: (cid:80) theorist who is a halfer will accept the bet if Cr (W)N X > 0. W u W W In contrast, a causal decision theorist deciding on the same bet will not let − the other N 1 bets that she is oﬀered in world W aﬀect her decision, so that W (cid:80) N X is replaced by X in the above. However, if she is a thirder rather than W W W a halfer, then her credence in world W will be proportional to Cr (W)N rather u W than Cr (W). Hence, again, she accepts the bet if Cr (W)N X > 0, u W u W W the only diﬀerence being that the factor N comes from the credence in this W case. Briggs proves that betting in this way (“betting at thirder odds”) leaves Beauty immune to Dutch books. All of this makes sense when Beauty always has the same information upon awakening. When this is not the case, we should ﬁrst enrich the notation a bit. What is relevant is not the total number of centers N in a world, but W rather the number of centers NI consistent with the current information I. W For example, in the WBG variant, it does not suﬃce to know that N = 2; WG rather, we need that Nwhite = 1, Ngrey = 1, and Nblack = 0. Then, one might WG WG WG suppose that with information I, the credence in some world W (that is not yet ruled out by I) is Cr (W) for the halfer and proportional to Cr (W)NI (cid:80) u u W for the thirder. (Note that this would be consistent with the credences in the WBG variant.) The causal decision theorist who is a thirder would then accept the bet if Cr (W)NI X > 0. Now, what about the evidential decision W u W W theorist who is a halfer? Suppose it were the case that now accepting (rejecting) the bet with information I leads her to believe that she always accepts (rejects) it with information I, but does not inﬂuence her beliefs about what she would do given any other information.7 Then, in world W, she believes her net payout if she rejects the bet is c (i.e., whatever she expects to get from any bets accepted (cid:80) when she has information other than I), and her net payout if she accepts the bet is NI X + c. The c term cancels out, and hence, again, she will accept W W the bet if Cr (W)NI X > 0. So the argument would appear to carry W u W W through. The problem is that, as I have argued (and will argue further in Subsection ), it is unreasonable to suppose that the decision made with the current information does not aﬀect beliefs about decisions made with slightly diﬀerent information! If it does aﬀect them, then the equivalence argument falls apart: we can no longer cancel out the c term in the above because it now depends on the decision made with information I, and as a result the condition for accepting a bet changes in the case of the evidential decision theorist who is a halfer. This is what allows the Dutch book for the WBG variant. (It is worth emphasizing again that in the WBG variant, the credences of 1/3 placed in each of the remaining possible worlds do not seem in question, suggesting that the problem in fact lies with evidential decision theory, not with halﬁng – at least as far as this particular Dutch book is concerned.) Thus, a causal decision theorist who is a thirder might analyze the eviden- tial decision theorist’s susceptibility to the Dutch book as follows. When an evidential decision theorist considers the payoﬀ she expects to get from a given 7Again, note that she should always be oﬀered the same bet whenever she has information I; otherwise, the bet oﬀered would in fact give her additional information. See Subsection for further discussion. 9 ##LTLine##

text: theorist

text: who

text: is

text: a

text: halfer

text: will

text: accept

text: the

text: bet

text: if

text: Cr

text: (W)N

text: X

text: W

text: u

text: W

text: W

text: In

text: contrast,

text: a

text: causal

text: decision

text: theorist

text: deciding

text: on

text: the

text: same

text: bet

text: will

text: not

text: let

text: the

text: other

text: N

text: bets

text: that

text: she

text: is

text: oﬀered

text: in

text: world

text: W

text: aﬀect

text: her

text: decision,

text: so

text: that

text: W

text: N

text: X

text: is

text: replaced

text: by

text: X

text: in

text: the

text: above.

text: However,

text: if

text: she

text: is

text: a

text: thirder

text: rather

text: than

text: W

text: W

text: W

text: a

text: halfer,

text: then

text: her

text: credence

text: in

text: world

text: W

text: will

text: be

text: proportional

text: to

text: Cr

text: (W)N

text: rather

text: u

text: W

text: than

text: Cr

text: (W).

text: Hence,

text: again,

text: she

text: accepts

text: the

text: bet

text: if

text: Cr

text: (W)N

text: X

text: u

text: W

text: u

text: W

text: W

text: the

text: only

text: diﬀerence

text: being

text: that

text: the

text: factor

text: N

text: comes

text: from

text: the

text: credence

text: in

text: this

text: W

text: case.

text: Briggs

text: proves

text: that

text: betting

text: in

text: this

text: way

text: (“betting

text: at

text: thirder

text: odds”)

text: leaves

text: Beauty

text: immune

text: to

text: Dutch

text: books.

text: All

text: of

text: this

text: makes

text: sense

text: when

text: Beauty

text: always

text: has

text: the

text: same

text: information

text: upon

text: awakening.

text: When

text: this

text: is

text: not

text: the

text: case,

text: we

text: should

text: ﬁrst

text: enrich

text: the

text: notation

text: a

text: bit.

text: What

text: is

text: relevant

text: is

text: not

text: the

text: total

text: number

text: of

text: centers

text: N

text: in

text: a

text: world,

text: but

text: W

text: rather

text: the

text: number

text: of

text: centers

text: NI

text: consistent

text: with

text: the

text: current

text: information

text: I.

text: W

text: For

text: example,

text: in

text: the

text: WBG

text: variant,

text: it

text: does

text: not

text: suﬃce

text: to

text: know

text: that

text: N

text: WG

text: rather,

text: we

text: need

text: that

text: Nwhite

text: Ngrey

text: and

text: Nblack

text: Then,

text: one

text: might

text: WG

text: WG

text: WG

text: suppose

text: that

text: with

text: information

text: I,

text: the

text: credence

text: in

text: some

text: world

text: W

text: (that

text: is

text: not

text: yet

text: ruled

text: out

text: by

text: I)

text: is

text: Cr

text: (W)

text: for

text: the

text: halfer

text: and

text: proportional

text: to

text: Cr

text: (W)NI

text: u

text: u

text: W

text: for

text: the

text: thirder.

text: (Note

text: that

text: this

text: would

text: be

text: consistent

text: with

text: the

text: credences

text: in

text: the

text: WBG

text: variant.)

text: The

text: causal

text: decision

text: theorist

text: who

text: is

text: a

text: thirder

text: would

text: then

text: accept

text: the

text: bet

text: if

text: Cr

text: (W)NI

text: X

text: Now,

text: what

text: about

text: the

text: evidential

text: decision

text: W

text: u

text: W

text: W

text: theorist

text: who

text: is

text: a

text: halfer?

text: Suppose

text: it

text: were

text: the

text: case

text: that

text: now

text: accepting

text: (rejecting)

text: the

text: bet

text: with

text: information

text: I

text: leads

text: her

text: to

text: believe

text: that

text: she

text: always

text: accepts

text: (rejects)

text: it

text: with

text: information

text: I,

text: but

text: does

text: not

text: inﬂuence

text: her

text: beliefs

text: about

text: what

text: she

text: would

text: do

text: given

text: any

text: other

text: Then,

text: in

text: world

text: W,

text: she

text: believes

text: her

text: net

text: payout

text: if

text: she

text: rejects

text: the

text: bet

text: is

text: c

text: (i.e.,

text: whatever

text: she

text: expects

text: to

text: get

text: from

text: any

text: bets

text: accepted

text: when

text: she

text: has

text: information

text: other

text: than

text: I),

text: and

text: her

text: net

text: payout

text: if

text: she

text: accepts

text: the

text: bet

text: is

text: NI

text: X

text: c.

text: The

text: c

text: term

text: cancels

text: out,

text: and

text: hence,

text: again,

text: she

text: will

text: accept

text: W

text: W

text: the

text: bet

text: if

text: Cr

text: (W)NI

text: X

text: So

text: the

text: argument

text: would

text: appear

text: to

text: carry

text: W

text: u

text: W

text: W

text: through.

text: The

text: problem

text: is

text: that,

text: as

text: I

text: have

text: argued

text: (and

text: will

text: argue

text: further

text: in

text: Subsection

text: it

text: is

text: unreasonable

text: to

text: suppose

text: that

text: the

text: decision

text: made

text: with

text: the

text: current

text: information

text: does

text: not

text: aﬀect

text: beliefs

text: about

text: decisions

text: made

text: with

text: slightly

text: diﬀerent

text: information!

text: If

text: it

text: does

text: aﬀect

text: them,

text: then

text: the

text: equivalence

text: argument

text: falls

text: apart:

text: we

text: can

text: no

text: longer

text: cancel

text: out

text: the

text: c

text: term

text: in

text: the

text: above

text: because

text: it

text: now

text: depends

text: on

text: the

text: decision

text: made

text: with

text: information

text: I,

text: and

text: as

text: a

text: result

text: the

text: condition

text: for

text: accepting

text: a

text: bet

text: changes

text: in

text: the

text: case

text: of

text: the

text: evidential

text: decision

text: theorist

text: who

text: is

text: a

text: halfer.

text: This

text: is

text: what

text: allows

text: the

text: Dutch

text: book

text: for

text: the

text: WBG

text: variant.

text: (It

text: is

text: worth

text: emphasizing

text: again

text: that

text: in

text: the

text: WBG

text: variant,

text: the

text: credences

text: of

text: placed

text: in

text: each

text: of

text: the

text: remaining

text: possible

text: worlds

text: do

text: not

text: seem

text: in

text: question,

text: suggesting

text: that

text: the

text: problem

text: in

text: fact

text: lies

text: with

text: evidential

text: decision

text: theory,

text: not

text: with

text: halﬁng

text: at

text: least

text: as

text: far

text: as

text: this

text: particular

text: Dutch

text: book

text: is

text: concerned.)

text: Thus,

text: a

text: causal

text: decision

text: theorist

text: who

text: is

text: a

text: thirder

text: might

text: analyze

text: the

text: eviden-

text: tial

text: decision

text: theorist’s

text: susceptibility

text: to

text: the

text: Dutch

text: book

text: as

text: follows.

text: When

text: an

text: evidential

text: decision

text: theorist

text: considers

text: the

text: payoﬀ

text: she

text: expects

text: to

text: get

text: from

text: a

text: given

text: note

text: that

text: she

text: should

text: always

text: be

text: oﬀered

text: the

text: same

text: bet

text: whenever

text: she

text: has

text: information

text: I;

text: otherwise,

text: the

text: bet

text: oﬀered

text: would

text: in

text: fact

text: give

text: her

text: additional

text: information.

text: See

text: Subsection

text: for

text: further

text: discussion.

text: ##LTLine##