\documentclass[twocolumn]{article}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage[english]{babel}
\usepackage{ifpdf,amsmath,amsthm,amssymb,amsfonts,newtxtext,newtxmath} 
\usepackage{array,graphicx,dcolumn,multirow,hevea,abstract,hanging}
\usepackage[labelfont=sc,textfont=sf]{caption}
\usepackage[hyperfootnotes=false,breaklinks=true]{hyperref} % was dvipdfmx
\usepackage{multirow}
\usepackage{color,colortbl}
\definecolor{Gray}{gray}{0.8}
\urlstyle{rm}
\usepackage[hyphenbreaks]{breakurl}
%\usepackage{natbib} % must come afer hyperfootnotes
\usepackage{booktabs} % \toprule \midrule \bottomrule \cmidrule(lr){a-b}
% define centered and ragged columns:
%\newcommand{\dd}[1]{\mathrm{d}#1}
\newcolumntype{L}[1]{>{\raggedright\arraybackslash }p{#1}} % can use m{}
\newcolumntype{C}[1]{>{\centering\arraybackslash }p{#1}}
\newcolumntype{R}[1]{>{\raggedleft\arraybackslash }p{#1}}
\newcolumntype{d}[1]{D{.}{.}{#1}} % d{3.2} for 3 places on l, 2 on r
\newcommand{\mc}{\multicolumn}
\topmargin=-.3in \oddsidemargin=-.1in \evensidemargin=-.1in \textheight=9in \textwidth=6.8in
\setlength\tabcolsep{1mm}
\setlength\columnsep{5mm}
\setlength\abovecaptionskip{.5ex}
\setlength\belowcaptionskip{.5ex}
\setlength\belowbottomsep{.3ex}
\setlength\lightrulewidth{.04em}
\renewcommand\arraystretch{1.2}
\renewcommand{\topfraction}{1}
\renewcommand{\textfraction}{0}
\renewcommand{\floatpagefraction}{.9}
% \renewcommand{\baselinestretch}{1.00} \large\normalsize % for fixing spaces
\widowpenalty=1000
\clubpenalty=1000
\setlength{\parskip}{0ex}
\let\tempone\itemize
\let\temptwo\enditemize
\let\tempthree\enumerate
\let\tempfour\endenumerate
\renewenvironment{itemize}{\tempone\setlength{\itemsep}{0pt}}{\temptwo}
\renewenvironment{enumerate}{\tempthree\setlength{\itemsep}{0pt}}{\tempfour}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\setcounter{page}{217} % start with first page

\title{Predictably intransitive preferences}

\author{
David J. Butler\thanks{Department of Accounting, Finance and Economics, Griffith Business School, Griffith University, Gold Coast, Queensland}
\and 
 Ganna Pogrebna\thanks{Corresponding author: Department of Economics, Birmingham Business School, University of Birmingham, JG Smith Building, Birmingham, B15 2TT, email: gpogrebna@gmail.com} \;\,
  \thanks{Alan Turing Institute, 96 Euston Road, Kings Cross, London, NW1 2DB}\footnotemark[2] % indicates same as 2nd thanks
}


\date{} % leave empty
\begin{document} % goes here

% fill in short title
\newcommand{\jref}{http://journal.sjdm.org/vol13.3.html}
\newcommand{\jhead}{Judgment and Decision Making, Vol.~13, No.~3}
\newcommand{\jdate}{May 2018}
\pagestyle{myheadings} \markright{\protect\small \href{\jref}{\jhead},
  \jdate \hfill Predictably intransitive preferences \qquad}
\begin{htmlonly}
\href{\jref}{\jhead}, \jdate, pp.\ 217-236
\end{htmlonly}
%\begin{latexonly}
\twocolumn[
\vspace{-.3in}
{\small \href{\jref}{\jhead}, \jdate, pp.\ 217--236}
%\end{latexonly}

\maketitle

%\begin{latexonly}
\vspace{-3mm}
\begin{onecolabstract}
%\end{latexonly}

  The transitivity axiom is common to nearly all descriptive and
  normative utility theories of choice under risk. Contrary to both
  intuition and common assumption, the little-known 'Steinhaus-Trybula
  paradox' shows the relation 'stochastically greater than' will not
  always be transitive, in contradiction of Weak Stochastic
  Transitivity. We bespoke-design pairs of lotteries inspired by the
  paradox, over which individual preferences might cycle. We run an
  experiment to look for evidence of cycles, and violations of
  expansion/contraction consistency between choice sets. Even after
  considering possible stochastic but transitive explanations, we show
  that cycles can be the modal preference pattern over these simple
  lotteries, and we find systematic violations of
  expansion/contraction consistency.

\smallskip
\noindent
Keywords: intransitivity, cycles, lotteries, experiment, expansion consistency
%\begin{latexonly}
\end{onecolabstract}\bigskip
]
%\end{latexonly}

{\renewcommand{\thefootnote}{}
\footnotetext{ % note blank lines above and below acknowledgment

We would like to thank seminar participants at Warwick Business School, University of East Anglia, University of California, Irvine, Cal State Fullerton, University of Arizona, Curtin University, Murdoch University, Sydney University, Griffith University, University of Queensland and Queensland University of Technology for comments on earlier versions of this manuscript. David Butler acknowledges the support of the Australian Research Council (grant: DP1095681). Ganna Pogrebna acknowledges financial support from RCUK/EPSRC grants EP/N028422/1 and EP/P011896/1.

Copyright: \copyright\ 2018.
The authors license this article under the terms of the
\href{http://creativecommons.org/licenses/by/3.0/}{Creative Commons
  Attribution 3.0 License.}
}}

\saythanks

\setlength{\baselineskip}{12pt plus.2pt}

\section{Introduction}

Researchers have questioned the adequacy of Expected Utility Theory
(EUT) as an account of choice under risk since Allais (1953) presented
his famous 'paradox' examples. Economists question one axiom of EUT
less than most: transitivity. Bar-Hillel \& Margalit (1988) quote Luce
\& Raiffa's (1957) definition of transitivity as "if A is preferred in
the paired comparison (A, B) and B is preferred in the paired
comparison (B. C), then A is preferred in the paired comparison (A,
C)" (Luce \& Raiffa, 1975, p.\ 16). Notice that the binary comparison
{A, C} is therefore superfluous: a rational chooser can rely on
transitivity to deliver the best option. Choice cycles cannot occur
unless this chooser is exactly indifferent between A, B and C, or
makes a mistake. In short, economists regard transitivity as a
defining characteristic of rational choice.

In light of this consensus, Butler \& Blavatskyy (2018) propose the
following scenario. A fund manager offers a reward to the broker who
selects one portfolio that outperforms the others over the following
year. The decision maker's (DM) preference then is to maximise the
probability of earning the greater sum. Suppose there are three
(statistically independent) portfolios; portfolio $A$ yields \$4m with
probability $2/3$ and \$1m with probability $1/3$; portfolio $B$
yields \$3m for sure; and portfolio $C$ yields \$5m with probability
$1/3$ and \$2m with probability $2/3$.
 
Suppose the fund manager begins by comparing $\left\{A,B\right\}$; she
will choose portfolio $A$ because $A$ yields a higher outcome than B
with probability $2/3$. Next she compares $\left\{B,C\right\}$; she
chooses portfolio B because B will yield a higher outcome than C also
with probability $2/3$. Then as a rational decision maker relying on
transitivity for choice set $\left\{A,C\right\}$, she selects
portfolio A over C. However, her faith in transitivity is
disadvantageous. Had she not relied upon transitivity, her revealed
preference in $\left\{A,C\right\}$ would have been for C, because C
yields a higher outcome than A with probability $5/9$, or 55.5\%. The
advantageous preference ordering across the set of pairwise choices is
the cycle $A\succ B\succ C\succ A$, contradicting the transitivity
axiom.\footnote{ We discuss her preference order over the ternary
  choice set $\left\{A,B,C\right\}$ in the next section.} While some
may say this makes probable winner preferences whether induced or
elicited unreasonable (e.g., Pratt, 1972), this paper takes a
different view (e.g., Blyth, 1972; Bar-Hillel \& Margalit,
1988). First, businesses do employ these kinds of incentives, which
for pairwise decisions can lead to the preference (and choice) cycle
in our example. Second, no utility theories with a transitivity axiom
currently come with a warning that they cannot account for probable
winner or related preferences, in which case it is not clear why we
should deem their preferences unworthy of maximization.

This cycle in our example is an illustration of a paradox first
described by Steinhaus and Trybula (Steinhaus \& Trybula, 1959). As a
mathematical puzzle their paradox, which we denote STP for short, has
inspired a small literature in applied statistics (e.g., from Usiskin
1964 to Conrey et al., 2016). We may state it as follows: let choice
objects $A$, $B$, $C$ be independent random variables and let
$Pr(A\succ B)$ denote the probability of choosing $A$ over $B$. It is
possible for $Pr(A\succ B)$, $Pr(B \succ C) $and $Pr(C\succ A)$ to all
exceed 50\%, given a preference for the winner, contrary to Weak
Stochastic Transitivity (WST).

Steinhaus \& Trybula proved that, for three choice objects, each with
three equiprobable attributes, the theoretical maximum 'minimum'
(max-min) winning probability is $(\sqrt{5}-1)/2$ or 61.8\%. It is
because this value exceeds 50\% that preference cycles may arise. In
our earlier example, the smallest of the three 'winning' probabilities
is 55.5\%. While we focus in the rest of this paper on preferences
over simple lotteries, we should not forget the relevance of these
objects to real economic decisions for choice under risk. Steinhaus \&
Trybula gave an application to testing the relative strength of
randomly selected steel bars $A$, $B$, $C$, for which successive
comparisons could exhibit a cycle: $A$ stochastically stronger than
$B$, $B$ than $C$ but with $C$ stochastically stronger than A. Other
examples are not difficult to imagine.

Given the STP relies on a preference for the most probable winner, as
incentivised by the fund manager in our earlier example, does it have
much relevance for decision theory and individual preferences more
generally? This paper suggests that the answer is yes, even though the
STP has passed mostly unnoticed in the decision theory literature
(exceptions include Butler \& Hey, 1987; Anand, 1993; Blavatskyy,
2006; Rubinstein \& Segal, 2012). Even if very few individual
preferences over lottery pairs are simply for the probable winner, the
STP still serves as an important demonstration that imposing
transitivity on an unrestricted domain of preference profiles will
sometimes result in an inferior choice. We conjecture the STP can also
serve as a heuristic in constructing new lotteries, over which a
broader range of preferences may cycle, a claim we return to in
section on Inferences for Lottery Design and an Experiment.

Kahneman (2012) reminds us that \textit{"The errors of a theory are
  rarely found in what it asserts explicitly; they hide in what it
  ignores or tacitly assumes"}. Transitivity must hold either if a
value attaches to each option without reference to other alternatives
(choice-set independence), or if an equivalent value results after
comparing and contrasting the attributes of the available choice
options. The latter process points us towards the flaw in how
transitivity is applied to multi-attribute choice. This process will
produce an equivalent value only if utility is sufficiently 'linear in
the differences' between the options' attribute values; see Tversky
(1969); Fishburn (1982); or Loomes \& Sugden, (1982) for details. The
STP relies on an extreme example of a non-linear additive difference
choice rule, for which a larger difference in an attribute's magnitude
carries no extra weight.

However, individuals often form valuations of options in a
comparative, context-dependent manner rather than attracting a
context-independent value (Russo \& Dosher, 1983; Arieli et al, 2009;
Noguchi \& Stewart, 2014). Evidence from eye-tracking experiments
shows clear empirical evidence against choice-set independence, at
least when expected utilities are sufficiently 'close' to prompt a DM
to compare the attributes of the alternatives.\footnote{Evidence from
  response time's shows fast decisions when one option is clearly
  better. Response times lengthen, as the DM needs to accumulate more
  evidence to trigger a choice.} Non-linearity sufficient to produce a
preference cycle (dependent on the relative size of the attributes)
may then occur.

The rest of the paper is organised as follows. In the next Section, we
discuss inferences for lottery design and describe our
experiment. After that, we present the experimental results and
conclude with a general discussion

\section{Inferences for Lottery Design and Experiment}

\subsection{General Implications}

Now we transition to using the STP objects to design our own lotteries
without either inducing or assuming probable winner preferences. Let
us consider decision making under risk when choice alternatives are
lotteries --- i.e. probability distributions over a nonempty finite
set of outcomes. A decision maker faces a set of choice alternatives
that contains at least two distinct elements. Next, the DM chooses the
choice alternative that yields a strictly greater, context
independent, expected utility. Often, no such choice alternative is
present, so she compares the attributes of the available options to
recognize where her preference lies. This step is required to avoid
arbitrary choice; a growing body of evidence shows preferences are
often known imperfectly (inter alia, Butler \& Loomes, 2007).

Descriptively, the consensus is that true intransitive preference
cycles are vanishingly infrequent. Evidence once taken to indicate
systematic intransitivity (Tversky, 1969; Loomes, Starmer \& Sugden,
1991) has since been either reinterpreted as not reflecting
fundamental intransitivity (inter alia: Starmer \& Sugden, 1993) or
found by newer statistical methods to be compatible with noisy but
transitive responses (Baillon et al, 2015; Birnbaum \& Diecidue,
2015). In his highly influential 1969 article 'Intransitivity of
Preference', Amos Tversky lamented \textit{"...in the absence of a
  model that guides the construction of the alternatives, one is
  unlikely to detect consistent violations of weak stochastic
  transitivity (WST)"}.

% figure 1
\begin{figure}[t!]\centering
\includegraphics[width=\columnwidth]{fig1.pdf}
\caption{Experimental Flow.}
\end{figure}

One reason why experiments to date have only rarely found evidence of
intransitive behaviour is lack of guidance from theory to select
suitable lottery parameters. This lack of guidance is probably a
result of the assumption any utility function must apply to all
lottery pairs. However, this presumes there is no 'black hole' in
parameter space from which question sets can trigger preferences of a
different kind. Drawing on the STP as a heuristic, we may address the
problem Tversky faced and bespoke design candidate lotteries.

Let each consequence represent a sum of money, in \pounds, a very
familiar, directly comparable outcome for which magnitudes are easily
interpretable by our subjects. For simplicity and comparability,
the probability of each outcome in our design is always $1/3$, $2/3$
or $1$. We imposed some filters to guide our selection of
triples. Since expected value differences between each choice object,
within a given triple, reach a maximum of \pounds 4
${}^2{\mskip -5mu/\mskip -3mu}_3$ $\pm$ \pounds 1
${}^2{\mskip -5mu/\mskip -3mu}_3$ $\pm$ 35.7\%, our first filter is to
focus on sets with larger EV differences (see Figure 1 for the
distribution). We avoid constructing triples informed by sets with
equal or near equal EV's to avoid tipping the balance towards a cycle
simply through noise.

Next, the second filter reduces cognitive load by requiring each of
the three STP objects to reuse integers such that there are no more
than two different amounts as consequences (e.g., 5, 2 and 2); we
therefore exclude any triple with three different money consequences
on any lottery. It is also important to keep the presentation of the
number of attributes in each object equal rather than coalescing
identical outcomes. This is because past experiments have found the
contrast between coalesced and non-coalesced outcomes (also known as
event-splitting; Starmer and Sugden, 1993) can be confused as evidence
for intransitivity (Birnbaum \& Schmidt, 2008; Baillon et al,
2015). To control for this we display each consequence even when all
three lead to the same sum. We then made a number of modifications
that increased the prizes on offer and then allowed for risk aversion
in our experiment (risk-aversion plays no role for probable winner
preferences). We make no claim this step in the parameter selection
process involves more than a mix of informed guesswork and personal
judgment.

Finally, as a third filter, to mimic the preference reversal (PR)
phenomenon problems (Lichtenstein \& Slovic, 1971) we decided to focus
on the STP triples which have expected values strictly in the
following order: $\$ \succ P \succ CE$. Here, \$ is a dollar bet
(lottery which yields a large outcome with low probability), P is a
probability bet (lottery which gives a small outcome with large
probability) and CE is certainty. If we have 3 lotteries: $X$, $Y$ and
$Z$, we will assume that $Z$ stands for the dollar bet, $X$ - for the
probability bet, and $Y$ for certainty (degenerate lottery).

%Table 1
\begin{table*}[t!]\centering
\caption{Binary choices and intransitivity.}
\begin{tabular}{*{6}{|C{.4in}}|C{.9in}|C{.9in}|C{.9in}|C{.9in}|}\hline
{\ }\newline\newline Triple & {\ }\newline\newline Option &{\ }\newline\newline p1=1/3 &
{\ }\newline\newline p2=1/3 &{\ }\newline\newline p3=1/3 &{\ }\newline\newline EV &N intransitivities in Repetition 1 (frequency) &
N intransitivities in Repetition 2 (frequency) &
N consistently intransitive with the same pattern (frequency) & 
N consistently intransitive including different patterns (frequency) \\ \hline
 & X & 12 & 12 & 3 & 9.00 &  &  &  &  \\
 & Y & 8 & 8 & 8 & 8.00 &  &  &  &  \\
{1} & Z & 24 & 4 & 4 & 10.67 & {0.22} & {0.21} & {0.10} & {0.11} \\ \hline
 & X & 11 & 11 & 1 & 7.67 & & & & \\
 & Y & 5 & 5 & 5 & 5.00 & & & & \\
{2} & Z & 20 & 2 & 2 & 8.00 & {0.11} & {0.25} & {0.05} & {0.05} \\ \hline
 & X & 12 & 12 & 2 & 8.67 &  &  &  &  \\
 & Y & 8 & 8 & 8 & 8.00 &  &  &  &  \\
{3} & Z & 20 & 4 & 4 & 9.33 & {0.29} & {0.26} & {0.12} & {0.15} \\ \hline
 & X & 15 & 15 & 3 & 11.00 & & & & \\
 & Y & 10 & 10 & 10 & 10.00 & & & & \\
{4} & Z & 27 & 5 & 5 & 12.33 & {0.59} & {0.60} & {0.26} & {0.34} \\ \hline
 & X & 15 & 15 & 6 & 12.00 &  &  &  &  \\
 & Y & 11 & 11 & 11 & 11.00 &  &  &  &  \\
{5} & Z & 28 & 6 & 6 & 13.33 & {0.23} & {0.21} & {0.09} & {0.09} \\ \hline
 & X & 17 & 17 & 5 & 13.00 & & & & \\
 & Y & 12 & 12 & 12 & 12.00 & & & & \\
{6} & Z & 30 & 6 & 6 & 14.00 & {0.19} & {0.16} & {0.09} & {0.09} \\ \hline
 & X & 9 & 9 & 3 & 7.00 &  &  &  &  \\
 & Y & 6 & 6 & 6 & 6.00 &  &  &  &  \\
{7} & Z & 16 & 4 & 4 & 8.00 & {0.35} & {0.24} & {0.12} & {0.13} \\ \hline
 & X & 15 & 15 & 5 & 11.67 & & & & \\
 & Y & 10 & 10 & 10 & 10.00 & & & & \\
{8} & Z & 30 & 3 & 3 & 12.00 & {0.27} & {0.15} & {0.10} & {0.11} \\ \hline
 & X & 12 & 12 & 0 & 8.00 &  &  &  &  \\
 & Y & 7 & 7 & 7 & 7.00 &  &  &  &  \\
{9} & Z & 28 & 0 & 0 & 9.33 & {0.08} & {0.04} & {0.01} & {0.01} \\ \hline
 & X & 14 & 14 & 2 & 10.00 & & & & \\
 & Y & 8 & 8 & 8 & 8.00 & & & & \\
{10} & Z & 21 & 6 & 6 & 11.00 & {0.23} & {0.30} & {0.17} & {0.17} \\ \hline
 & X & 14 & 14 & 2 & 10 &  &  &  &  \\
 & Y & 24 & 5 & 5 & 11.33 &  &  &  &  \\
{11} & Z & 8 & 8 & 9 & 8.33 & {0.23} & {0.20} & {0.07} & {0.09} \\ \hline
\end{tabular}
\end{table*}

An important implication of our design choices is that the direction
of cycles may be subject to two opposing forces, in aggregate, because
the direction of cycles for standard PR lotteries is opposite to the
'probable winner' cycle. However, this should not stop systematic
cycles appearing at the individual level, if some people exhibit one
tendency more strongly. This in mind, we can now put forward our first
testable hypothesis. Birnbaum \& Schmidt (2008) succinctly state the
currently dominant view regarding the evidence for intransitive
preferences: \textit{"...we think the burden of proof should shift to
  those who argue that intransitive models are descriptive of more
  than five percent of the population"}.

\smallskip
\noindent
\textbf{Hypothesis 1}: Drawing on the STP ingredients, we can design
sets of lotteries for which cycles will occur with significantly
greater frequency than 5\%.
\smallskip

We can see the lotteries we designed in the left hand columns (columns
1--6) of Table 1.

In the spirit of Allais' famous example, consider one such 'bespoke'
lottery set, and assume your preferred lottery is incentivized. The
three choice objects are statistically independent lotteries: each
outcome is a monetary amount with a one-third probability
attached. For each of the binary choice sets $\left\{X,Y\right\}$;
$\left\{Y,Z\right\}$; $\left\{Z,X\right\}$, viewed separately, we ask
the reader to consider her preference, ideally looking only at each
decision in isolation. In combination, there are eight possible binary
preference patterns, of which just two are intransitive. Consider $X$
versus $Y$, where $X$ provides \pounds 15 with probability 1/3,
\pounds 15 with probability 1/3, or \pounds 3 with probability 1/3;
and $Y$ yields \pounds 10 for certain - i.e., \pounds 10 with
probability 1/3, \pounds 10 with probability 1/3, or \pounds 10 with
probability 1/3 (see Table 1). Suppose $Y \succ X$. Now compare $Y$
which gives \pounds 10 for certain and $Z$ which provides \pounds 27
with probability 1/3, \pounds 5 with probability 1/3, or \pounds 5
with probability 1/3. Perhaps, here $Z \succ Y$. Finally, compare $Z$
which yields \pounds 27 with probability 1/3, \pounds 5 with
probability 1/3, or \pounds 5 with probability 1/3 and $X$ which
provides \pounds 15 with probability 1/3, \pounds 15 with probability
1/3, or \pounds 3 with probability 1/3.\footnote{In order to make sure
  that subjects in our experiment understand probabilities, we use
  display with 3 differently colored marbles for each lottery
  option. We provide screenshots of this display in later sections of
  this paper.}

In this case, maybe you found $X \succ Z$. If you prefer $Y \succ X$,
$Z \succ Y$ and $X \succ Z$, you have exhibited the preference cycle
$X \succ Z \succ Y \succ X$, the modal preference pattern for our
subjects. The opposite cycle here is $X \succ Y \succ Z \succ X$; we
found these two intransitive patterns together exceeded, by a small
majority, the six transitive patterns combined. Referring back to our
opening example, suppose the consequences on each of $X$, $Y$ and $Z$
refer to investment returns on three portfolios and the probabilities
are the historical frequencies. A consumer's binary preferences over
those risk-return combinations might potentially cycle also with
implications for the structure of portfolios in finance.

\subsection{Design Implications from Models of Probabilistic Choice}

Although choice is often stochastic, and an intransitive cycle may
arise from transitive latent preference due to noise, distinguishing
structurally intransitive latent preferences from stochastic
transitivity in experiments is not straightforward. How frequently can
intransitive cycles arise for individuals with transitive preferences,
but who choose probabilistically? For example, individuals may have
transitive core preferences but choice probabilities are determined by
embedding these preferences into a model of random errors (e.g.,
Blavatskyy, 2014). Such a modelling approach can generate a
statistically significant asymmetry between the two possible cycle
directions, but it cannot generate a proportion of intransitive cycles
above 25\% of all observed choice patterns, for any triple.

% below changed Loomes & Sugden 1995 to 1987.

A more promising model of probabilistic choice for rationalizing
intransitive cycles is the random preference approach (e.g., Loomes \&
Sugden, 1995). As the extreme example, let us consider an individual
who has three transitive preference orderings $X \succ Y \succ Z$,
$Z \succ X \succ Y$ and $Y \succ Z \succ X$ with each ordering equally
likely to be drawn when a choice is to be made. It is straightforward
to see that in a direct binary choice between $X$ and $Y$, this
individual chooses $X$ with probability 2/3. Likewise, in a direct
binary choice between $Y$ and $Z$, this individual chooses $Y$ with
probability 2/3. Finally, in a direct binary choice between $X$ and
$Z$, this individual chooses $Z$ with probability 2/3, thereby
violating weak stochastic transitivity. Thus, a model of random
transitive preferences generates a maximum of $(2^3)/(3^3) = 8/27$
(29.6\%) intransitive choice cycles. This limit involves a strong
asymmetry between the two possible intransitive patterns; the maximum
frequency of a particular cycle given random sampling is 1/4; see
Rubinstein \& Segal (2012) for proofs of these propositions.

However, a model of random transitive preferences has another testable
implication so far overlooked by a literature focused on binary choice
sets. When comparing binary choice data with the choice data from a
ternary set, we derive a new set of constraints that any stochastic
but exclusively transitive preferences must meet. In such models of
stochastic choice, the probability of choosing $X$ from the ternary
set $\left\{X,Y,Z\right\}$ is given by the probability that a decision
maker draws a preference order in which $X$ is preferred to $Y$ and
$X$ is preferred to $Z$. In contrast, for a direct binary choice
between $X$ and $Y$, this decision maker chooses $X$ with a
probability that is equal to the probability that he or she draws a
preference order in which $X$ is preferred to $Y$ (but $X$ may or may
not be preferred to $Z$). Similarly, for a direct binary choice
between $X$ and $Z$, this decision maker chooses $X$ with a
probability that is equal to the probability that he or she draws a
preference order in which $X$ is preferred to $Z$ (but $X$ may or may
not be preferred to $Y$). Hence, any model of random transitive
preferences must make the following testable hypotheses. If any one of
the three hypotheses fails to hold, no model of stochastic transitive
preferences can be consistent with the data.

\smallskip
\noindent
\textbf{Hypothesis 2}: The probability of choosing X from the ternary set $\left\{X,Y,Z\right\}$ cannot exceed
\begin{align}
{min\left\{P(X,Y),P(X,Z)\right\}=min\left\{P(X,Y),1-P(Z,X)\right\}}
\end{align} 
\textbf{Hypothesis 3}: The probability of choosing Y from the ternary set $\left\{X,Y,Z\right\}$ cannot exceed
\begin{align}
{min\left\{(Y,X),P(Y,Z)\right\}=min\left\{1-P(X,Y),P(Y,Z)\right\}}
\end{align} 
\textbf{Hypothesis 4}: The probability of choosing Z from the ternary set $\left\{X,Y,Z\right\}$ cannot exceed \begin{align}
{min\left\{P(Z,X),P(Z,Y)\right\}=min\left\{P(Z,X),1-P(Y,Z)\right\}}
\end{align} 
Since, by definition, the probabilities of choosing $X$, $Y$ and $Z$ from the ternary set $\left\{X,Y,Z\right\}$ must sum up to one, we have the following implication of any model of stochastic but transitive preferences:
\begin{align}
{min\left\{P(X,Y),1-P(Z,X)\right\}+}\nonumber\\
{+min\left\{1-P(X,Y),P(Y,Z)\right\}+}\nonumber\\
{+ min\left\{P(Z,X),1-P(Y,Z)\right\}>1}
\end{align} 
A decision maker who violates weak stochastic transitivity, such that $P(X,Y)>0.5$, $P(Y,Z)>0.5$ and $P(Z,X)>0.5$, must still satisfy the inequality
\begin{align}
{1-P(Z,X) + [1-P(X,Y)] + [1-P(Y,Z)] >1}
\end{align} 
which can be simplified as a triangle inequality
\begin{align}
{P(X,Y) + P(Y,Z) + P(Z,X) < 2}
\end{align} 
The triangle inequalities (5) and (6), it is usually argued, produce a
stronger test than WST to separate genuine intransitivity from
stochastic transitivity. However, Birnbaum (2011) showed that the
triangle inequalities could be satisfied even by underlying
intransitive preferences. Furthermore, recent work by Müller-Trede et
al.\ (2015) demonstrates how these inequalities may be violated even
when underlying preferences are 100\% transitive. Their experiment
also shows clear violations of these inequalities.

% Figure 2
\begin{figure*}[t!]\centering
\includegraphics[width=5in]{fig2.pdf}
\caption{Binary choice display.}
\end{figure*}

In other words, the triangle inequalities for stochastic transitive
preferences can be satisfied when preferences are intransitive and
violated when preferences are transitive, raising a concern that they
are not as useful for identifying true intransitive preference cycles
as generally believed, though see Cavagnaro \& Davis-Stober (2014) for
an alternative view. For these reasons, among others, our experiment
was not designed specifically to test the triangle inequalities, which
ideally would require multiple repetitions of the same lottery pairs
for every person. However, we can test H2-H4 below, for each triple
violating WST. We also follow Birnbaum \& Diecidue (2015) and repeat
each set of choices once after a distractor task, which facilitates
additional methods of separating noise from true preferences.

Consider a decision maker who: a) makes a direct binary choice between
choice alternatives $X$ and $Y$; and b) ranks choice alternatives $X$
and $Y$ as part of a ternary choice set in terms of their
desirability. Ignoring the ranking of $Z$, this decision maker can
reveal four different preference patterns:
\begin{itemize}
	\item $X \succ Y$ and $X$ is ranked more desirable than $Y$ (revealed preferences i);
	\item $Y \succ X$ and $Y$ is ranked more desirable than $X$ (revealed preferences ii);
	\item $X \succ Y$ and $Y$ is ranked more desirable than $X$ (revealed preferences iii);
	\item $Y \succ X$ and $X$ is ranked more desirable than $Y$ (revealed preferences iv).
\end{itemize}

Revealed preferences i and ii are both consistent with the
independence of irrelevant alternatives axiom. Revealed preferences
iii and iv are inconsistent with this axiom. Thus, if preferences of a
decision maker satisfy contraction and expansion consistency, we
should observe patterns i and ii and not patterns iii and iv. Our
experiment investigates.

\subsection{Experimental Design}

% figure 3
\begin{figure*}[t!]\centering
\includegraphics[width=5in]{fig3.pdf}
\caption{Ternary Choice Display.}
\end{figure*}

In total 100 subjects (all undergraduate students at the
University of Warwick) were invited to take part in the experiment. We
programmed the experiment using the Qualtrics software and consisted
of 100 questions divided into 5 parts (see Figure 1).

Earlier tests for preference cycles primarily used state-contingent
consequences in matrix-style displays. Those displays facilitate
between-act comparisons and enhance the possibility of, for instance,
anticipated regret when consequences are state-contingent and thus the
potential for cycles. Our design maintains statistical independence
between the choice objects such that any observed preference cycles
are more likely to be rooted in description-invariant, intransitive
latent preferences. Finally, we include a 'standard PR' control set
(Triple 9) to compare to our 'new PR' gambles that is the focus of our
experiment.

In Part 1, we broke up the 11 triples into binary choices between
individual lotteries and asked subjects to answer 33 questions (3
binary choice questions per each triple). Table 1 provides a detailed
list of all triples. We present each binary choice in the format shown
in Figure 2 with two options --- Left and Right. Each option shows a
lottery with 3 equiprobable outcomes.

% removed "below" from "below in Figure 2

All binary choice questions gave subjects four different options
on a slider. The initial starting point for each slider was "No
preference". However, subjects were not able to proceed by leaving
the slider in the original position (i.e., the choice of "No
preference" was not allowed).  Subjects were able to move the
slider to the right and opt for "Slightly prefer Right" or "Strongly
prefer Right" or, alternatively, to move the slider to the left and
choose "Slightly prefer Left" or "Strongly prefer Left". Irrespective
of whether a subject indicated slight or strong preference, we
used only revealed preferences for "Left" or "Right" in the payoff
calculations and we do not report the strength of preference results
here. We randomized all 33 questions for each individual
separately. In Part three, we repeated all 33 binary questions again
but presented them in a different random order to each subject.

% table 2
\begin{table*}[t!]\centering
\caption{Frequency of intransitive cycles obtained from binary choices, in percent.}
\begin{tabular}{cccccccccccc}\toprule
Triple & Profile & 121 & \cellcolor{Gray}123 & 131 & 133 & 221 & 223 & \cellcolor{Gray}231 & 233 & INTR & TR \\\midrule
 & R1 & 9 & \cellcolor{Gray}2 & 5 & 12 & 29 & 8 & \cellcolor{Gray}20 & 15 & 22 & 78 \\
\multirow{-2}{*}{1} & R2 & 13 & \cellcolor{Gray}3 & 6 & 10 & 27 & 9 & \cellcolor{Gray}18 & 14 & 21 & 79 \\
 & R1 & 30 & \cellcolor{Gray}9 & 25 & 17 & 7 & 8 & \cellcolor{Gray}2 & 2 & 11 & 89 \\
\multirow{-2}{*}{2} & R2 & 25 & \cellcolor{Gray}17 & 23 & 5 & 2 & 10 & \cellcolor{Gray}8 & 10 & 25 & 75 \\
 & R1 & 16 & \cellcolor{Gray}11 & 5 & 0 & 31 & 12 & \cellcolor{Gray}18 & 7 & 29 & 71 \\
\multirow{-2}{*}{3} & R2 & 14 & \cellcolor{Gray}10 & 5 & 8 & 25 & 13 & \cellcolor{Gray}16 & 9 & 26 & 74 \\
 & R1 & 11 & \cellcolor{Gray}0 & 6 & 6 & 9 & 5 & \cellcolor{Gray}59 & 4 & 59 & 41 \\
\multirow{-2}{*}{4} & R2 & 8 & \cellcolor{Gray}18 & 5 & 3 & 6 & 14 & \cellcolor{Gray}42 & 4 & 60 & 40 \\
 & R1 & 8 & \cellcolor{Gray}3 & 5 & 17 & 25 & 7 & \cellcolor{Gray}20 & 15 & 23 & 77 \\
\multirow{-2}{*}{5} & R2 & 13 & \cellcolor{Gray}5 & 6 & 12 & 31 & 8 & \cellcolor{Gray}16 & 9 & 21 & 79 \\
 & R1 & 15 & \cellcolor{Gray}4 & 8 & 7 & 32 & 9 & \cellcolor{Gray}15 & 10 & 19 & 81 \\
\multirow{-2}{*}{6} & R2 & 15 & \cellcolor{Gray}4 & 5 & 11 & 33 & 12 & \cellcolor{Gray}12 & 8 & 16 & 84 \\
 & R1 & 3 & \cellcolor{Gray}7 & 15 & 16 & 11 & 9 & \cellcolor{Gray}28 & 11 & 35 & 65 \\
\multirow{-2}{*}{7} & R2 & 7 & \cellcolor{Gray}1 & 17 & 19 & 16 & 10 & \cellcolor{Gray}23 & 7 & 24 & 76 \\
 & R1 & 12 & \cellcolor{Gray}14 & 5 & 8 & 40 & 3 & \cellcolor{Gray}13 & 5 & 27 & 73 \\
\multirow{-2}{*}{8} & R2 & 13 & \cellcolor{Gray}7 & 7 & 9 & 47 & 7 & \cellcolor{Gray}8 & 2 & 15 & 85 \\
 & R1 & 17 & \cellcolor{Gray}4 & 3 & 1 & 57 & 10 & \cellcolor{Gray}4 & 4 & 8 & 92 \\
\multirow{-2}{*}{9} & R2 & 15 & \cellcolor{Gray}3 & 2 & 1 & 57 & 19 & \cellcolor{Gray}1 & 2 & 4 & 96 \\
 & R1 & 4 & \cellcolor{Gray}0 & 16 & 16 & 8 & 7 & \cellcolor{Gray}23 & 26 & 23 & 77 \\
\multirow{-2}{*}{10} & R2 & 5 & \cellcolor{Gray}1 & 10 & 13 & 12 & 10 & \cellcolor{Gray}29 & 20 & 30 & 70 \\
 & R1 & 8 & \cellcolor{Gray}22 & 13 & 13 & 15 & 21 & \cellcolor{Gray}1 & 7 & 23 & 77 \\
\multirow{-2}{*}{11} & R2 & 10 & \cellcolor{Gray}10 & 9 & 11 & 18 & 14 & \cellcolor{Gray}10 & 18 & 20 & 80 \\\bottomrule
\mc{12}{L{4in}}{Notes: R1 --- Repetition 1; R2 --- Repetition 2; INRT - intransitive preferences; TR - transitive preferences.}
\end{tabular}
\end{table*}

In Part 2, we asked subjects to make two choices in each ternary
set, for the most and next most preferred object for each of the 11
triples (see Figure 3).

We sought to maximize the similarity to the binary choice task and so
we incentivized the choice of best and next best in each ternary set
to obtain the full ordering in the triple. We explained that we would
draw two of the three lotteries at random and they would play out
whichever of the two they had positioned higher, if the ternary set is
selected for payment. We use the standard 'random lottery incentive
system' for the binary choices. Our aim was to keep to a minimum any
extraneous 'choice versus ranking' disparities, to allow as clean a
test as possible of the expansion consistency property and the
transitive random preference restrictions, H2--4.

We randomized the order in which the ternary sets appeared, as well as
the order in which the lotteries appeared on each screen. To avoid
lazy acceptance of the default ordering, it was not possible to accept
the default ranking. If the default was by chance preferred, they
first had to move away from the default then move back to it by
deliberate choice.

In Part 3, subjects were offered a Distractor Task in order to create
a break between the two repetitions of binary and ternary choice
tasks. The distractor task consisted of 12 risky choices using a
different display, the results of which are not reported here. In Part
4, all 11 ternary choice set problems were repeated in a different
random order.

We asked subjects to complete all tasks in the experiment online in a
5-day window.  All 100 subjects received an invitation to the
laboratory to play out their decisions for real money. Each subject
drew a question number (between 1 and 100) at random and received
payment based on his/her choice in that question. In each question, we
looked at the lottery option chosen by the subject and played out that
lottery according to the description on the experimental display
(e.g., Figure 2 and Figure 3). Immediately after the draw, subjects
received their payoff in cash. All 100 subjects turned up to play out
the lottery and receive their winnings. Finally, all subjects
completed a detailed online survey covering questions such as
domain-specific risk attitudes and a variety of demographic variables
not reported on here.

% figure 4
\begin{figure*}[t!]\centering
\includegraphics[width=5in]{fig4.pdf}
\caption{Histogram of Cycle Frequency by Individual.}
\end{figure*}

\section{Results}

\subsection{Descriptive Statistics}

Table 2 reports the frequency of intransitive cycles for each of the
eleven triples obtained from binary choices. Other than the PR control
(Triple 9) we find the proportion of preference cycles averaged across
both repetitions ranges from a low of 18\% (Triples 2 and 6) to a high
of 59\% (Triple 4).\footnote{Table A in the
  \href{http://journal.sjdm.org/17/17912b/supp.pdf}{Online
    Supplementary Material} provides detailed summary statistics
  including the frequency of intransitive cycles for each of the
  eleven triples.}

In contrast the PR 'control' triple found just 6\% intransitive
patterns. Other than the control triple, the proportion of cycles for
each triple is strongly significantly greater than 5\%, using Fisher's
exact test, supporting H1. The average proportion intransitive in the
first block was 27.1\% followed by 25.8\% for the second block, giving
an overall intransitive proportion of 26.5\%. If learning occurred
between blocks, it did not reduce the occurrence of cycles noticeably,
giving our first clue that error may not be the main cause of
cycles. Of the eight possible preference orderings (two intransitive,
six transitive), the mode was intransitive for three of the ten
triples and an intransitive pattern was runner-up in a further five
triples.

At the individual level, between 30\% and 85\% of subjects cycled in
each of the ten triples either once or on both repetitions. For
instance, in triple 4 alone, 85 of the 100 individuals cycled at least
once and 34 cycled on both occasions. It appears that a significant
minority, a plurality, even an occasional majority, exhibit
intransitive choice cycles, for these statistically independent pairs
of simple, incentivized lotteries. Across the ten triples,
\textit{every single one of the 100 subjects} cycled at least once. A
Spearman correlation of the number of cycles by individual between
repetitions was $+0.93$, suggesting again that the cycles we observe
are not simply random errors but latent intransitive
preferences. Figure 4 shows the histogram of cycles by individual.

The predominant direction of cycles for our lotteries is consistent
with 'reverse' cycles, rather than the 'probable winner'
cycles.\footnote{The 'probable winner' cycle refers to a case when
  $CE \succ \$-bet \succ P \succ CE$ and the 'reverse' refers to a
  case when $\$-bet \succ CE \succ P-bet \succ \$-bet$.} The 26\%
average figure breaks down 19:7 in favour of the 'reverse'
direction. For the eight triples where an intransitive pattern is
either the mode or runner-up, six follow the 'reverse' direction and
two the 'probable winner' direction. This likely reflects our choice
of expected value rankings for $X$, $Y$ and $Z$, as we noted
previously. Unlike the STP, the random lottery incentive system does
not induce a particular preference pattern; it elicits preferences
rather than prescribes them.

% figure 5
\begin{figure*}[t!]\centering
\includegraphics[width=4.5in]{fig5.pdf}
\caption{Individual Inconsistency versus Frequency of Cycles.}
\end{figure*}

\subsection{Noisy Transitivity or Noisy Intransitivity?}

A reasonable approximation to a true, 'error free' proportion for each
preference pattern is to identify those subjects making the same three
binary choices within a triple on both repetitions. To do so means
they avoid six possible choice errors, for each 'true' preference
pattern. Across the ten sets of triples (excluding the control), the
diagonals for each triple in Table 2 reveal this occurs on 366
occasions out of 1000. Of these, 117 were of one of the two
intransitive orderings and 249 were for one of the six transitive
orderings. Thus, the share of revealed consistently intransitive
preference patterns among all revealed consistent preference patterns
was 32\% (117 out of 366). To get a sense of how striking this finding
is, a recent and unusually careful and thorough investigation of
intransitive choice patterns was able to conclude: "...very few people
repeat the same intransitive pattern on two replications of the same
test. In other words, most violations that have been observed can be
attributed to error rather than to true intransitivity" (Birnbaum \&
Diecidue, 2015). Yet we find that on average a typical intransitive
pattern is 41\% ($117/2$ vs $249/6$) more likely to replicate than a
typical transitive pattern.

Delving deeper, we see from the middle panels in Table 2 that the
intransitive proportion of \textit{consistently} revealed patterns
across the triples varies from 17.8\% (triple 8) to 83.9\% (triple
4). The modal consistently revealed patterns are intransitive for
triples 4, 7, 10 and 11 and runner-up for Triples 1, 3, 5 and 6; that
is, eight of the ten triples have an intransitive modal or second
modal consistently revealed preference pattern. Therefore,
consistently intransitive preferences appear to be revealed relatively
\textit{more frequently} than intransitive preferences that may or may
not replicate. This result suggests that noise diminishes (rather than
increases) intransitive preferences in revealed choice patterns. In
other words, as the noise washes out, cyclical choice patterns
increase their share of the total, a finding replicated at the
individual level, as we next show. As a comparison, in our control,
triple 9, we find the opposite: just 1 consistently intransitive
person but 51 \textit{consistently} transitive people, a 98\%
transitive share, in line with the consensus view on the rarity of
cycles.

Another way to check whether fundamental intransitivity or noise is
driving the data is to divide the subjects into two equal-size groups
by rate of choice switching between repetitions. On inspection, we
find a threshold of 10 or fewer inconsistencies separates 51
individuals with fewer and 49 individuals with more inconsistencies
(or stochastic preferences). In Figure 5, we plot the number of cycles
exhibited by each individual against the number of his or her choice
switches between repetitions.

The graph shows a broad tendency for more cycles among the more
consistent individuals, with cycles decreasing as the rate of choice
switching increases. The most consistent group of 51 subjects
committed an average 6.16 cycles (out of a maximum possible 20), which
is 33\% more than the group of 49 noisier individuals who averaged
4.63 cycles. This conclusion is not dependent on the threshold of 10,
as is clear from Figure 5, offering further evidence that true
intransitivity rather than noise is responsible for most of the
observed cycles.\footnote{The caveat is there may be a modest uptick
  of cycles for the most inconsistent of all; but inspection of the
  graph shows just two out of 100 individuals drive the uptick, so it
  may not be reliable.}

Weak stochastic transitivity (WST), requires $Pr (C \succ A)$ to be at
least as large as the minimum of $Pr (B \succ A)$ and
$Pr (C \succ B)$, a requirement diametrically at odds with the STP. As
noted earlier, violations of WST can also result if subjects have
random preferences over exclusively transitive preference
orders. However, we also showed an overlooked implication of this
claim is the set of constraints we identified in H2-H4. The frequency
with which each lottery is chosen from the respective ternary choice
set must satisfy each of H2-H4. In other words, any random preference
model over transitive orderings capable of violating WST must meet
H2-H4. If it does not, latent intransitive preferences are presumably
the only remaining explanation. Our data shows that WST is violated
for Triples 1, 3, 4, 5, and 7 (see Table 3).\footnote{See Table B in
  the \href{http://journal.sjdm.org/17/17912b/supp.pdf}{Online Supplementary Material} for more details.}

% table 3
\begin{table*}[t!]\centering
\caption{Intransitive cycles across binary and ternary choices.}
\begin{tabular}{ccccc@{\qquad}cccccc}\toprule
% \multirow{2}{*}{Triple} & \multirow{2}{*}{Option} & \multirow{2}{*}{p1=1/3} & \multirow{2}{*}{p2=1/3} & \multirow{2}{*}{p3=1/3} & \mc{6}{c}{Total} \\\cmidrule{6-11}
%  & & & & &$X \succ Y$ & $Y \succ X$ & $Y \succ Z$ & $Z \succ Y$ & $X \succ Z$ & $Z \succ X$ \\\midrule
 & & & & & \mc{6}{c}{Total} \\\cmidrule{6-11}
Triple & Option & p1=1/3 & p2=1/3 & p3=1/3 & $X \succ Y$ & $Y \succ X$ & $Y \succ Z$ & $Z \succ Y$ & $X \succ Z$ & $Z \succ X$ \\\midrule
{1} & X & 12 & 12 & 3 & 60 & 140 & 100 & 100 & 127 & 73 \\
 & Y & 8 & 8 & 8 & 43 & 81 & 76 & 53 & 95 & 41 \\
 & Z & 24 & 4 & 4 & 72\% & 58\% & 76\% & 53\% & 75\% & 56\% \\\midrule
{2} & X & 11 & 11 & 1 & 152 & 49 & 108 & 92 & 122 & 78 \\
 & Y & 5 & 5 & 5 & 104 & 27 & 49 & 59 & 87 & 48 \\
 & Z & 20 & 2 & 2 & 69\% & 55\% & 45\% & 64\% & 71\% & 62\% \\\midrule
{3} & X & 12 & 12 & 2 & 69 & 131 & 132 & 68 & 130 & 70 \\
 & Y & 8 & 8 & 8 & 35 & 84 & 83 & 27 & 107 & 52 \\
 & Z & 20 & 4 & 4 & 51\% & 64\% & 63\% & 40\% & 82\% & 74\% \\\midrule
{4} & X & 15 & 15 & 3 & 57 & 143 & 71 & 129 & 146 & 54 \\
 & Y & 10 & 10 & 10 & 37 & 97 & 49 & 32 & 106 & 20 \\
 & Z & 27 & 5 & 5 & 65\% & 68\% & 69\% & 25\% & 73\% & 37\% \\\midrule
{5} & X & 15 & 15 & 6 & 69 & 131 & 100 & 100 & 124 & 76 \\
 & Y & 11 & 11 & 11 & 33 & 91 & 72 & 37 & 105 & 51 \\
 & Z & 28 & 6 & 6 & 48\% & 69\% & 72\% & 37\% & 85\% & 67\% \\\midrule
{6} & X & 17 & 17 & 5 & 69 & 131 & 124 & 76 & 135 & 65 \\
 & Y & 12 & 12 & 12 & 38 & 71 & 80 & 27 & 103 & 36 \\
 & Z & 30 & 6 & 6 & 55\% & 54\% & 65\% & 36\% & 76\% & 55\% \\\midrule
{7} & X & 9 & 9 & 3 & 85 & 115 & 64 & 136 & 120 & 80 \\
 & Y & 6 & 6 & 6 & 50 & 55 & 37 & 87 & 47 & 45 \\
 & Z & 16 & 4 & 4 & 59\% & 48\% & 58\% & 64\% & 39\% & 56\% \\\midrule
{8} & X & 15 & 15 & 5 & 75 & 125 & 143 & 57 & 145 & 55 \\
 & Y & 10 & 10 & 10 & 38 & 58 & 61 & 29 & 93 & 29 \\
 & Z & 30 & 3 & 3 & 51\% & 46\% & 43\% & 51\% & 64\% & 53\% \\\midrule
{9} & X & 12 & 12 & 0 & 46 & 154 & 182 & 18 & 156 & 44 \\
 & Y & 7 & 7 & 7 & 27 & 86 & 97 & 9 & 76 & 24 \\
 & Z & 28 & 0 & 0 & 59\% & 56\% & 53\% & 50\% & 49\% & 55\% \\\midrule
{10} & X & 14 & 14 & 2 & 65 & 135 & 47 & 153 & 107 & 93 \\
 & Y & 8 & 8 & 8 & 34 & 76 & 31 & 86 & 76 & 67 \\
 & Z & 21 & 6 & 6 & 52\% & 56\% & 66\% & 56\% & 71\% & 72\% \\\midrule
{11} & X & 14 & 14 & 2 & 96 & 104 & 118 & 82 & 84 & 116 \\
 & Y & 8 & 8 & 9 & 44 & 59 & 32 & 32 & 33 & 73 \\
 & Z & 24 & 5 & 5 & 46\% & 57\% & 27\% & 39\% & 39\% & 63\% \\\bottomrule
\mc{11}{L{5in}}{Notes: In each Triple in Columns 6-11: row 1 shows data for binary choice task; row 2 shows data for ternary choice task; row 3 shows (data from ternary/data from binary) in \%.}
\end{tabular}\medskip
\end{table*}

% table 4
\begin{table*}[t!]\centering
\caption{Testing Hypotheses 2--4.}
%\setlength{\arrayrulewidth}{.1em}
\begin{tabular}{c@{\quad}L{1.5in}L{1.5in}L{1.5in}}\toprule
\mc{1}{c}{Triples} & \mc{1}{c}{Hypothesis 2} & \mc{1}{c}{Hypothesis 3} & \mc{1}{c}{Hypothesis 4} \\ \hline
1 WST & H2: Pr(X) \textless 0.36\newline Data: Pr(X) = 0.32 & \cellcolor{Gray}H3: Pr(Y) \textless 0.3\newline Data: Pr(Y) = 0.39 & H4: Pr(Z) \textless 0.5\newline Data: Pr(Z) = 0.29 \\\midrule
3 WST & H2: Pr(X) \textless 0.35\newline Data: Pr(X) = 0.37 & \cellcolor{Gray}H3: Pr(Y) \textless 0.34\newline Data: Pr(Y) = 0.43 & H4: Pr(Z) \textless 0.35\newline Data: Pr(Z) = 0.29 \\ \midrule
4 WST & \cellcolor{Gray}H2: Pr(X) \textless 0.27\newline Data: Pr(X) = 0.34 & \cellcolor{Gray}H3: Pr(Y) \textless 0.285\newline Data: Pr(Y) = 0.485 & H4: Pr(Z) \textless 0.355\newline Data: Pr(Z) = 0.175 \\ \midrule
5 WST & H2: Pr(X) \textless 0.38\newline Data: Pr(X) = 0.23 & \cellcolor{Gray}H3: Pr(Y) \textless 0.34\newline Data: Pr(Y) = 0.54 & H4: Pr(Z) \textless 0. 5\newline Data: Pr(Z) = 0.23 \\ \midrule
7 WST & H2: Pr(X) \textless 0.4\newline Data: Pr(X) = 0.23 & H3: Pr(Y) \textless 0.42\newline Data: Pr(Y) = 0.315 & \cellcolor{Gray}H4: Pr(Z) \textless 0.32\newline Data: Pr(Z) = 0.455 \\ \bottomrule
\mc{4}{l}{Notes: Shaded cells highlight cases when a hypothesis is rejected by the data.} 
\end{tabular}
\end{table*}

Triple 4 exhibits the strongest violation followed closely by Triple
3. Averaged across both repetitions, in triple 4 we found:
$Pr (Y \succ X)$= 71.5\%; $Pr (Z \succ Y)$ = 64.5\% and also that
$Pr (Z \succ X)$ = 27\%. At 37.5 percentage points, this is a
strikingly strong violation of WST. It is also a violation of 'simple
scalability' (Tversky \& Russo, 1968). Table 4 shows the results of
the hypothesis tests.

In summary, for each triple where WST did not hold for the binary
choice sets, any transitive random preference model must satisfy H2-H4
in the ternary sets. This is essential if the WST violations in the
binary sets were a result of stochastic but transitive latent
preferences. Taken as a whole, the tests reported above to separate
noisy but transitive latent preferences from underlying intransitivity
lean heavily in favor of the latter proposition.

Finally, triple 8 was one of two triples where intransitive patterns
were relatively infrequent. The lotteries comprising triple 8 were
designed to be a test of one intriguing 'ingredient' in the STP
recipe: a higher minimum consequence for Z than for X. A small pilot
experiment had previously identified triple 4 as particularly prone to
exhibit cycles (19 of 27 subjects cycled). We decided to make as few
changes as possible to the lottery pairs of triple 4 when swapping the
lowest payoff in X with that in Z. This change then required an
increase in the maximum payoff in Z to keep the expected value above
that for X.

The combined effect of these two changes is to drastically reduce the
number of observed cycles, from 119 of 200 in triple 4 down to 42 of
200 in triple 8 (triple 8 resembles, but does not strictly satisfy,
the STP). Even more striking is the reduction in the intransitive
share of consistently revealed patterns, from 83.9\% in triple 4 to
17.8\% in triple 8. We conjecture that if researchers adopt this new
PR design, they may find even stronger reversals than those that have
comprised the PR paradox to date.

\subsection{Testing Expansion Consistency and IIA}

The second purpose for eliciting preferences in the respective ternary
choice sets is to test for expansion consistency. Table 3 presents the
results for all decisions, contrasting the binary and ternary choices,
by triple. Table 5 reports separately the most preferred elements in
all the ternary sets.

% Table 5
\begin{table*}[t!]\centering
\caption{Average ternary top preferences by triple.}
\begin{tabular}{c@{\qquad}c@{\qquad}c@{\qquad}c@{\qquad}cccc}\toprule
Triple & Option & p1=1/3 & p2=1/3 & p3=1/3 & X in a 1st choice & Y in a 1st choice & Z in a 1st choice\\\midrule
1 & X & 12 & 12 & 3 & 32\% & 39\% & 29\% \\
 & Y & 8 & 8 & 8 & & & \\
 & Z & 24 & 4 & 4 & & & \\\midrule
2 & X & 11 & 11 & 1 & 49\% & 14.5\% & 36.5\% \\
 & Y & 5 & 5 & 5 & & & \\
 & Z & 20 & 2 & 2 & & & \\\midrule
3 & X & 12 & 12 & 2 & 43\% & 28\% & 29\% \\
 & Y & 8 & 8 & 8 & & & \\
 & Z & 20 & 4 & 4 & & & \\\midrule
4 & X & 15 & 15 & 3 & 34\% & 48.5\% & 17.5\% \\
 & Y & 10 & 10 & 10 & & & \\
 & Z & 27 & 5 & 5 & & & \\\midrule
5 & X & 15 & 15 & 6 & 23\% & 54\% & 23\% \\
 & Y & 11 & 11 & 11 & & & \\
 & Z & 28 & 6 & 6 & & & \\\midrule
6 & X & 17 & 17 & 5 & 33\% & 41.5\% & 25.5\% \\
 & Y & 12 & 12 & 12 & & & \\
 & Z & 30 & 6 & 6 & & & \\\midrule
7 & X & 9 & 9 & 3 & 23\% & 31.5\% & 45.5\% \\
 & Y & 6 & 6 & 6 & & & \\
 & Z & 16 & 4 & 4 & & & \\\midrule
8 & X & 15 & 15 & 5 & 33.5\% & 38\% & 28.5\% \\
 & Y & 10 & 10 & 10 & & & \\
 & Z & 30 & 3 & 3 & & & \\\midrule
9 & X & 12 & 12 & 0 & 44\% & 42\% & 14\% \\
 & Y & 7 & 7 & 7 & & & \\
 & Z & 28 & 0 & 0 & & & \\\midrule
10 & X & 14 & 14 & 2 & 34\% & 30\% & 36\% \\
 & Y & 8 & 8 & 8 & & & \\
 & Z & 21 & 6 & 6 & & & \\\midrule
11 & X & 14 & 14 & 2 & 25.5\% & 31\% & 43.5\% \\
 & Y & 8 & 8 & 9 & & & \\
 & Z & 24 & 5 & 5 & & & \\\bottomrule
\end{tabular}
\end{table*}

The rightmost panels in Table 3 show the following. The binary choices
in repetition 1 were consistent with binary choices in repetition 2
for $2043/3000$ or 68.1\% of decisions (excluding the control), which
we take as the benchmark. Aggregating across these 10 sets of triples,
for those choosing $X \succ Z$ in the binary set, 852 of 1240, or
68.7\% maintain the ranking in the ternary set. The binary/ternary
comparison for $X \succ Z$ is the same as the benchmark consistency
supporting (stochastic) expansion consistency when $Y$ is
included. However, this is just one of six binary preference ranks
tested and is the only one to match the binary/binary consistency.

For the binary preference $Y \succ X$, 699 of 1204 decisions, maintain
this ranking in the ternary set, or just 58.1\%. This is statistically
significantly lower than 68.1\%. Finally there are 993 binary choices
of $Z \succ Y$; of these just 469, or 47.2\%, maintain that rank in
the respective ternary comparisons so expansion consistency is
rejected. This is the lowest consistency rate of the six binary
ranks. It seems that preference for the riskiest option over the
certainty reverses when the intermediate option is included in the
set.

The other four comparisons are all around 58\% consistent. Hence, most
binary/ternary comparisons show clear evidence, beyond noise, of
set-dependent preferences, inconsistent with expansion to, or
contraction from, the ternary set. While some of the violations
reflect the well-known compromise effect, the largest inconsistency
above cannot be explained by any of the three known effects:
attraction, similarity or compromise.  We can also test whether
preferences revealed from a binary choice are consistent with
preferences revealed from rankings within a ternary set at a more
fine-grained level using the Conlisk (1989) test. This test compares
the relative frequency of two inconsistent choice patterns. The first
pattern is when a decision maker chooses $X$ over $Y$ in a binary
choice but ranks $Y$ over $X$ in a ternary set. The second pattern is
when a decision maker chooses $Y$ over $X$ in a binary choice but
ranks $X$ over $Y$ in a ternary set. If these two choice patterns are
due to indifferences, random errors, imprecision, noise or
indecisiveness, they should occur, a priori, with equal or similar
frequencies. In contrast, a decision maker who systematically reveals
one choice pattern significantly more often than the other (for
example, by following the most probable winner) is unlikely to simply
reflect indifference or noise. The Conlisk's (1989) test formalizes
this idea.

Table 6 presents the results of Conlisk (1989) test comparing the
consistency of binary choices with ternary rankings.

% table 6
\begin{table}[t!]\centering
\caption{Conlisk z (p-value) comparing binary choices $X$ vs $Y$, $Y$ vs $Z$ and $X$ vs $Z$ with the corresponding rankings from a ternary $\left\{X,Y,Z\right\}$.}
\begin{tabular}{C{.5in}C{.8in}C{.8in}C{.8in}}\toprule
Triple & Y vs Z\newline Conlisk z\newline (p-value) & X vs Y\newline
Conlisk z\newline (p-value) & X vs Z\newline Conlisk z\newline (p-value) \\\midrule
1 & $-$2.7749\newline (0.0028) & $-$5.1114\newline (0.0000) & 0.0000\newline (0.5000) \\\midrule
2 & 2.7550\newline (0.0029) & 3.0725\newline (0.0011) & 0.6192\newline (0.2679) \\\midrule
3 & 0.8427\newline (0.1997) & $-$1.4484\newline (0.0738) & 0.7801\newline (0.2177) \\\midrule
4 & $-$7.8478\newline (0.0000) & $-$3.2774\newline (0.0005) & 0.6966\newline (0.2430) \\\midrule
5 & $-$3.7896\newline (0.0001) & $-$0.4579\newline (0.3235) & $-$0.9041\newline (0.1830) \\\midrule
6 & $-$0.5175\newline (0.3024) & $-$3.1050\newline (0.0010) & 0.3833\newline (0.3508) \\\midrule
7 & $-$2.5583\newline (0.0053) & $-$2.6017\newline (0.0046) & 3.7758\newline (0.0001) \\\midrule
8 & 5.5142\newline (0.0000) & $-$3.0000\newline (0.0013) & 3.0023\newline (0.0013) \\\midrule
9 & 9.3944\newline (0.0000) & $-$5.6441\newline (0.0000) & 6.6093\newline (0.0000) \\\midrule
10 & $-$6.0806\newline (0.0000) & $-$3.0104\newline (0.0013) & 0.6613\newline (0.2542) \\\midrule
11 & 3.1553\newline (0.0008) & 0.7099\newline (0.2389) & 0.8245\newline (0.2048) \\\bottomrule
\end{tabular}
\end{table}

A significant positive or negative z value indicates that
inconsistencies between binary choices and ternary rankings are not
due to indifferences, random errors, imprecision or noise. In triples
7-10 binary choice is always statistically significantly different (at
5\% significance level) from ternary rankings for all three
comparisons ($X$ vs $Y$, $Y$ vs $Z$ and $X$ vs $Z$). For triple 3
binary choices are not statistically significantly different from
ternary rankings (for all three comparisons). For other triples, there
are significant differences for some but not all combinations
(typically $X$ vs $Y$ and $Y$ vs $Z$ are significantly different but
$X$ vs $Z$ is not). Thus, for all triples but triple 3 there appear to
be significant inconsistencies between binary choice and ternary
rankings that go beyond imprecision or noise.

More specifically, there is an asymmetry when $Z$ is involved; when
the binary choice favors $Z$ (over $X$ or over $Y$) the ranking is
more likely to be overturned in the ternary set than if $Z$ is
disfavored in the binary set (see Figure 6). These results appear to
confirm a preponderance of set-dependent preferences, in violation of
expansion-consistency, as predicted earlier.

% figure 6
\begin{figure*}[t!]\centering
\includegraphics[width=6in]{fig6.pdf}
\caption{Set-Dependent Preferences.}
\end{figure*}

The effect captured in Figure 6 suggests that when there is a straight
choice between our modified \$-bet and certainty, the majority of
subjects (129 of 200 people) prefer the new \$-bet. Of those 129, 97
reversed their preference when new P-bet was included in the choice
set. The set-dependent anchoring and adjustment effect (Slovic \&
Lichtenstein, 1983) may help explain Figure 6. Specifically, % omitted 1979, not in references
when people consider the new \$-bet versus certainty, they tend to
focus on the stake rather than probability and concentrate on the most
extreme outcome.\footnote{Kim, Seligman \& Kable (2012) show in an
  eye-tracking study that when either \$-bet or P-bet is compared to
  certainty; people tend to pay more attention to stakes rather than
  probabilities which is consistent with our findings. Kim et al.\
  (2012) also find that when \$-bet is compared with P-bet in a
  straight binary choice, people tend to focus on probabilities rather
  than stakes.}  However, when we add the new P-bet to the choice set,
our results suggest that subjects tend to concentrate on probabilities
rather than stakes. They pay attention to the most extreme probability
of 1 (certainty) and, hence, tend to choose certainty over the new
\$-bet.

\section{Conclusion: Predictably Intransitive?}

Where do our arguments leave the most popular utility theories of
choice under risk (which all include transitivity)? The
Steinhaus-Trybula Paradox shows that for multi-attribute risky choice
objects, which we evaluate in binary and ternary choice sets, relying
on transitivity can fail to select the most advantageous
lotteries. The domain of application for transitive theories excludes
choice-rules that are common and cognitively plausible and can violate
transitivity where utility differences between options are not too
large. It is likely that ternary choice sets using smaller differences
in expected values than we used would show even greater rates of
intransitive binary comparisons.

An innovative feature of our experiment was eliciting preferences in
the ternary choice sets as well as the constituent binary sets. This
allows us to investigate the predicted set-dependence of preferences
and test transitive but random preference theory as a possible
explanation of cycles. Results support our conjectures that the cycles
reflect latent intransitive preference rather than noisy
implementation of transitive preferences. We saw that although very
little solid evidence for true intransitive preferences existed prior
to our experiment, this 'absence of evidence' should not be mistaken
for 'evidence of the absence' of preference cycles. This paper has
identified limits to the descriptive invocation of transitivity. Our
findings also point to a deeper underlying process at work in choice
under risk, part of the growing evidence (Louie, Glimcher \& Webb, 2015) that
choices often bear the stamp of other options in the choice set, as
well as latent preferences (Noguchi \& Stewart, 2014).

One implication of our arguments and our experimental results is that
rather than modelling individuals as possessing a core utility
function (transitive or intransitive), many typically transitive
individuals are the same people who violate transitivity in the
circumstances we identify. This suggests neither a transitive nor
intransitive 'core' utility function can accurately describe
preferences over all lotteries a person may encounter. Stewart,
Reimers and Harris (2015) recently concluded, \textit{"The shape of % was 2014
  the revealed utility ... function is, at least in part, a property
  of the question set and not the individual"}, in line with a
constructed-preference paradigm. Our results point towards the
constructed-preference paradigm as the more promising way forward.

\section*{References}

\begin{hangparas}{1em}{1}

Allais, M. (1953). Le comportement de l'homme rationnel devant 
le risque: critique des postulats et axiomes de l'ecole Americaine. 
\textit{Econometrica, 21}, 503-546. %could not find the second initial
  
Anand, P. (1993). The philosophy of intransitive preference.
\textit{Economic Journal, 103}, 337-346. %could not find the second initial

Arieli, A., Ben-Ami, Y., \& Rubinstein, A. (2009). Fairness motivations 
and procedures of choice between lotteries as revealed through eye movements.
Unpublished manuscript.

% Arrow, K. J. (1951). \textit{Social Choice and Individual Values}, Yale University Press, New York.

Baillon, A., Bleichrodt, H., \& Cillo, A. (2015). A tailor-made test of intransitive choice.
\textit{Operations Research, 63}, 198--211. % no second initials for these authors

Bar-Hillel, M., \& Margalit, A. (1988). How vicious are cycles of intransitive choice?
\textit{Theory and Decision, 24}(2), 119--145. % no second initials for these authors 

Birnbaum, M. H. (2011). Testing mixture models of transitive preference:
comment on Regenwetter, Dana and Davis-Stober. \textit{Psychological
  Review, 118}, 675--683. %second initial added for Birnbaum

Birnbaum, M. H., \& Diecidue, E. (2015). Testing a class of models that includes majority 
rule and regret theories: transitivity, recycling and restricted branch independence. \textit{Decision, 1}, 145--190.

% Birnbaum, M. H., \& Gutierrez, R. J. (2007). Testing for intransitivity of preferences predicted by 
% a lexicographic semiorder. \textit{Organizational Behavior and Human Decision Processes, 104}, 97--112.

Birnbaum, M. H., \& Schmidt, U. (2008). An experimental investigation of violations of transitivity 
in choice under uncertainty. \textit{Journal of Risk and Uncertainty, 37}, 77--91.

Blavatskyy, P. R. (2006). Axiomatization of a preference for a probable winner. 
\textit{Theory and Decision, 60}, 17--33.

Blavatskyy, P. R. (2014). Stronger utility. 
\textit{Theory and Decision, 76}, 265--286.

% Block, H. D., \& Marschak, J. (1960). Random orderings and stochastic theories of response. 
% in I. Olkin, ed., \textit{Contributions to Probability and
%   Statistics}, pp.\ 97--132. Stanford University Press, Stanford.

Blyth, C. R. (1972). Some probability paradoxes in choice from among random alternatives. 
\textit{Journal of the American Statistical Association, 67}(338), 366--373.

Butler, D. J. \& Blavatskyy, P. R. (2018). Normative limits to expected utility theory's transitivity axiom. 
Unpublished manuscript.

Butler, D. J., \& Hey, J. (1987). Experimental economics: an introductory survey. 
\textit{Empirica: Austrian Economic Papers, 2}, 157--186. 

Butler, D. J., \& Loomes, G. (2007). Imprecision as an account of the preference 
reversal phenomenon. \textit{American Economic Review, 97}(1), 277--297.

% Butler, D. J., Isoni, A., \& Loomes, G. (2013). What lies beneath? Systematic intransitive 
% tendencies revealed by strength of preference judgments. \textit{mimeo}.

% Butler, D. J., Isoni, A., Loomes, G., \& Tsutsui, K. (2014). On the measurement of 
% strength of preference in units of money. \textit{Economic Record, 90}, 1--15.

Cavagnaro, D. R., \& Davis-Stober, C. P. (2014). Transitive in our preferences, but transitive
in different ways: an analysis of choice variability. \textit{Decision, 1}, 102--122.

Conlisk, J. (1989) Three variants on the Allais example.
\textit{American Economic Review 79}(3), 392--407 % no second initial

Conrey, B., Gabbard, J., Grant, K., Liu, A., \& Morrison, K. E (2016). Intransitive dice. 
\textit{Mathematics Magazine, 89},133--143.

% Cubitt, R. P., Monroe, A., \& Starmer, C. (2004). Testing explanations of preference reversal. 
% \textit{Economic Journal, 114}, 709--726.

% Davidson, D., McKinsey, J. C. C., \& Suppes, P. (1955) Outlines of a formal theory of value, I.
% \textit{Philosophy of Science, 22}, 140--160.

Fishburn, P. C. (1982). Non-transitive measurable utility. 
\textit{Journal of Mathematical Psychology, 26}, 31--67.

% Fishburn, P. C. (1991). Non-transitive preferences in decision theory. 
% \textit{Journal of Risk and Uncertainty, 4}, 113--134.

% Huber, J., Payne, J. W., \& Puto, C. (1982). Adding asymmetrically
% dominated alternatives: violations of regularity and the similarity
% hypothesis. \textit{Journal of Consumer Research, 9}, 90--98.

% Iverson, G., \& Falmagne, J.-C. (1985). Statistical issues in measurement. 
% \textit{Mathematical Social Sciences, 10}, 131--153.

Kahneman, D. (2012). \textit{Thinking, fast and slow}. Penguin, London \& New York.

Kim, B. E., Seligman, D., \& Kable, J. W. (2012). Preference reversals
in decision making under risk are accompanied by changes in attention
to different attributes.  \textit{Frontiers in Neuroscience, 6}(109),
1--10.

Lichtenstein, S. \& Slovic, P., (1971). Reversals of preference
between bids and choices in gambling decisions. \textit{Journal of
  Experimental Psychology, 89}, 46--55.

% Kontek, K., \& Lewandowski, M. (in press). Range-Dependent Utility.
% \textit{Management Science}.

% not cited

Loomes, G., \& Sugden, R. (1982). Regret theory: an alternative theory of rational 
choice under uncertainty. \textit{Economic Journal, 92}, 805--824.

Loomes, G., \& Sugden, R. (1987). Some implications of a more general form of 
regret theory. \textit{Journal of Economic Theory, 41}, 270--287.

Loomes, G., Starmer, C., \& Sugden, R. (1991). Observing violations of transitivity 
by experimental methods. \textit{Econometrica, 59}, 425--439.

% Louie, K., Haw, M. W., \& Glimcher, P. W. (2013). Normalization is a general neural 
% mechanism for context-dependent decision making. 
% \textit{Proceedings of the National Academy of Sciences, 110}, 6139--6144.

Louie, K., Glimcher, P. W., \& Webb, R. (2015). Adaptive neural coding:
from biological to behavioural decision-making. \textit{Current
  Opinion in Behavioral Sciences, 5}, 91--99.

Luce, R. D., \& Raiffa, H. (1957). \textit{Games and Decisions}. Wiley, New York.

% Morrison, H. W. (1962). Intransitivity of paired comparison choices. Unpublished 
% Doctoral dissertation, University of Michigan. 

Müller-Trede, J., Sher, S., \& McKenzie, C. R. M. (2015). Transitivity in
context: a rational analysis of intransitive choice and
context-sensitive preference. \textit{Decision, 1}, 1--26.

Noguchi, T., \& Stewart, N. (2014). In the attraction, compromise and
similarity effects, alternatives are repeatedly compared in pairs on
single dimensions. \textit{Cognition, 132}, 44--56.

Pratt, J. W. (1972) Comment. \textit{Journal of the American
  Statistical Association, 67}, 378--379.

% Ranyard, R. H. (1977). Risky decisions which violate transitivity and
% double cancellation.  \textit{Acta Psychologica, 41}, 449--459.

% Regenwetter, M., Dana, J., \& Davis-Stober, C. P. (2011). Transitivity of
% preferences.  \textit{Psychological Review, 118}, 42--56.

Rubinstein, A., \& Segal, U. (2012). On the likelihood of cyclic comparisons. 
\textit{Journal of Economic Theory, 147}, 2483--2491.

Russo, J. E., \& Dosher, B. A. (1983). Strategies for multi-attribute binary
choice.  \textit{Journal of Experimental Psychology: Learning, Memory
  and Cognition, 9}, 676--696.
 
Slovic, P., \& Lichtenstein, S. (1983). Preference reversals: a
broader perspective.  \textit{American Economic Review, 73}(4),
596--605.

Starmer, C., \& Sugden, R. (1993). Testing for juxtaposition and
event-splitting effects.  \textit{Journal of Risk and Uncertainty, 6},
235--254.

Steinhaus, H., \& Trybula, S. (1959). On a paradox in applied
probabilities.  \textit{Bulletin of the Polish Academy of Sciences,
  7}, 67--69.

Stewart, N., Reimers, S., \& Harris, A. J. L. (2015). On the origin of
utility, weighting and discount functions: how they get their shapes
and how to change their shapes.  \textit{Management Science, 61},
687--705.

Tversky, A. (1969). Intransitivity of preferences.
\textit{Psychological Review, 76}, 31--48.

Tversky, A., \& Russo, J. E. (1969). Substitutability and similarity in
binary choices.  \textit{Journal of Mathematical Psychology, 6},1--12.

Usiskin, Z. (1964). Max-min probabilities in the voting paradox.
\textit{Annals of Mathematical Statistics, 35}, 857--862.

\vfill
\break

\end{hangparas}

\onecolumn

%\bigskip
\section*{Appendix}
\begin{figure}[h]\centering
\includegraphics[width=6.5in]{fig7.pdf}
\caption*{Figure A1: Experimental Instructions: Screenshot 1.}
\end{figure}

\begin{figure}[t!]\centering
\includegraphics[width=5in]{fig8.pdf}
\caption*{Figure A2: Experimental Instructions: Screenshot 2.}
\end{figure}

\begin{figure}[t!]\centering
\includegraphics[width=5in]{fig9.pdf}
\caption*{Figure A3: Experimental Instructions: Screenshot 3.}
\end{figure}

\begin{figure}[t!]\centering
\includegraphics[width=6in]{fig10.pdf}
\caption*{Figure A4: Experimental Instructions: Screenshot 4.}
\end{figure}

\begin{figure}[t!]\centering
\includegraphics[width=6.5in]{fig11.pdf}
\caption*{Figure A5: Experimental Instructions: Screenshot 5.}
\end{figure}

\end{document}
