\documentclass[twocolumn]{article}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage[english]{babel}
\usepackage{ifpdf,amsmath,amsthm,amssymb,amsfonts,newtxtext,newtxmath} 
\usepackage{array,graphicx,dcolumn,multirow,hevea,abstract,hanging}
\usepackage[labelfont=sc,textfont=sf]{caption}
\usepackage[hyperfootnotes=false,breaklinks=true]{hyperref} % was dvipdfmx
\urlstyle{rm}
\usepackage[hyphenbreaks]{breakurl}
%\usepackage{natbib} % must come afer hyperfootnotes
\usepackage{booktabs} % \toprule \midrule \bottomrule \cmidrule(lr){a-b}
% define centered and ragged columns:
\newcolumntype{L}[1]{>{\raggedright\arraybackslash }p{#1}} % can use m{}
\newcolumntype{C}[1]{>{\centering\arraybackslash }p{#1}}
\newcolumntype{R}[1]{>{\raggedleft\arraybackslash }p{#1}}
\newcolumntype{d}[1]{D{.}{.}{#1}} % d{3.2} for 3 places on l, 2 on r
\newcommand{\mc}{\multicolumn}
\topmargin=-.3in \oddsidemargin=-.1in \evensidemargin=-.1in \textheight=9in \textwidth=6.8in
\setlength\tabcolsep{1mm}
\setlength\columnsep{5mm}
\setlength\abovecaptionskip{.5ex}
\setlength\belowcaptionskip{.5ex}
\setlength\belowbottomsep{.3ex}
\setlength\lightrulewidth{.04em}
\renewcommand\arraystretch{1.2}
\renewcommand{\topfraction}{1}
\renewcommand{\textfraction}{0}
\renewcommand{\floatpagefraction}{.9}
% \renewcommand{\baselinestretch}{1.00} \large\normalsize % for fixing spaces
\widowpenalty=1000
\clubpenalty=1000
\setlength{\parskip}{0ex}
\let\tempone\itemize
\let\temptwo\enditemize
\let\tempthree\enumerate
\let\tempfour\endenumerate
\renewenvironment{itemize}{\tempone\setlength{\itemsep}{0pt}}{\temptwo}
\renewenvironment{enumerate}{\tempthree\setlength{\itemsep}{0pt}}{\tempfour}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\setcounter{page}{305} % start with first page

\title{Still no compelling evidence that Americans overestimate upward socio-economic mobility rates: Reply to Davidai \& Gilovich (2018)}

\author{Sondre S. Nero\thanks{University of Chicago Booth School of
    Business.} \and
  Lawton K. Swan \thanks{Department of Psychology, University of Florida. Email:
    lkswan@ufl.edu.} \and John R. Chambers \thanks{Independent
    scholar} \and Martin Heesacker\thanks{University of Florida.}
}

\date{} % leave empty
\begin{document} % goes here

% fill in short title
\newcommand{\jref}{http://journal.sjdm.org/vol13.3.html}
\newcommand{\jhead}{Judgment and Decision Making, Vol.~13, No.~3}
\newcommand{\jdate}{May 2018}
\pagestyle{myheadings} \markright{\protect\small \href{\jref}{\jhead},
  \jdate \hfill Response to Davidai and Gilovich \qquad}
\begin{htmlonly}
\href{\jref}{\jhead}, \jdate, pp.\ 305-308
\end{htmlonly}
%\begin{latexonly}
\twocolumn[	
\vspace{-.5in}
{\small \href{\jref}{\jhead}, \jdate, pp.\ 305--308}
%\end{latexonly}

\maketitle

%\begin{latexonly}
\vspace{-3mm}
\begin{onecolabstract}
%\end{latexonly}

Davidai and Gilovich (2018) contend that (a) Americans tend to think about their nation's income distribution in terms of quintiles (fifths), and (b) when Americans' perceptions of socio-economic mobility rates are measured properly (e.g., by asking online survey respondents to guess upward-mobility rates across quintiles), a trend of overestimation (too much optimism concerning the number of people who manage to transcend poverty) will emerge. In this reply, we hail Davidai and Gilovich's new data as novel, important, and relevant to the former (a), but we doubt that they can support the latter (b) claim about population-level (in)accuracy. Namely, we note that even if mobility-rate perceptions could be measured perfectly, inferences about the accuracy of those perceptions still depend on a \textit{particular} comparator—a point-estimate of the "true" rate of upward social mobility in the U.S. against which survey respondents' guesses are evaluated—that is \textit{itself }an error-prone estimate. Applying different established comparators to survey respondents' guesses changes both the direction and magnitude of previously observed overestimation effects. We conclude with a challenge: researchers who wish to compute the average distance between socio-economic perceptions and socio-economic reality must first select and justify a fair comparator. 

\smallskip
\noindent
Keywords: social mobility, inequality, political ideology, lay beliefs.
%\begin{latexonly}
\end{onecolabstract}\bigskip
]
%\end{latexonly}

{\renewcommand{\thefootnote}{}
\footnotetext{ % note blank lines above and below acknowledgment


Copyright: \copyright\ 2018.
The authors license this article under the terms of the
\href{http://creativecommons.org/licenses/by/3.0/}{Creative Commons
  Attribution 3.0 License.}
}
}

\saythanks

\setlength{\baselineskip}{12pt plus.2pt}

Three years ago, in the January 2015 issue of \textit{Perspectives on Psychological Science}, Davidai and Gilovich argued that Americans tend to overestimate socio-economic prospects for the poor. Their evidence: when asked to estimate the likelihood that a child born into a low-income family in the United States would secure a higher social class ranking in adulthood, participants in Davidai and Gilovich’s study (\textit{N} = 3,324 online survey respondents) exaggerated the \textit{actual} likelihood of ascent by an average of ten percentage points. Four weeks later, in the February 2015 issue of \textit{Psychological Science}, we (Chambers, Swan \& Heesacker, 2015) argued \textit{precisely the opposite} — our research participants (\textit{N} = 865) had apparently \textit{underestimated} the very same upward-mobility statistic.

Seven months ago, here in \textit{Judgment and Decision Making}, we
(Swan, Chambers, Heesacker \& Nero, 2017) recanted — after (a)
cataloging the slew of ostensibly superficial differences between the
two teams’ methodologies and (b) presenting evidence that \textit{just
  one} of those differences could account for the discrepancy between
the two teams’ findings, we concluded that the probability of a false
positive result produced by systematic measurement error, such as
question-wording artifacts, was high enough to undermine \textit{both}
teams’ claims about \textit{population-level perceptual
  (in)accuracy}.\footnote{Note that we were not advocating for the
  retraction of either the Davidai and Gilovich (2015) or Chambers et
  al.\ (2015) papers — both addressed a wide range of additional
  questions related to perceptions of socio-economic mobility (e.g.,
  the relationship between political ideology and mobility
  perceptions) that withstood our (2017) inquisition.}

Now, Davidai and Gilovich (2018; this issue) suggest that the error
was endemic exclusively to \textit{our} questionnaire wording. In
brief: Davidai and Gilovich (2015) had asked their participants to
estimate socio-economic mobility rates by dividing the population into
\textit{quintiles}, whereas we (Chambers et al., 2015) had asked had
our participants to divide into \textit{tertiles}. When we inverted
the quintile-tertile factor in a new set of experiments (Swan et al.,
2017; \textit{N} = 1,142) \textit{while holding all other differences
  in wording constant}, the overestimation and underestimation effects
observed respectively by Davidai and Gilovich (2015) and Chambers et
al.\ (2015) either vanished or reversed (see Table 3 in Swan et al.,
2017). Our quintile-tertile-inversion results could not bear on the
question of which format is more likely to capture respondents’
\textit{true} perceptions, but we did speculate that the tertile
format may better reflect how people naturally think about income
distributions (in terms of the \textit{upper}, \textit{middle}, and
\textit{lower classes}).

We may stand corrected. In three new studies (\textit{N} = 392),
Davidai and Gilovich (2018) found that most people sampled divided the
income distribution pie into \textit{five} (or more, not three)
slices. Consequently, on Davidai and Gilovich’s account, when we
(Chambers et al., 2015) asked our study participants to estimate
upward mobility from the \textit{bottom third} — a non-intuitive
category label — we exposed ourselves to a set of tertile-specific
measurement vulnerabilities. Namely, in two additional studies
(\textit{N} = 304), Davidai and Gilovich (2018) found that people tend
to (a) assume that a three-rung ladder is \textit{ipso facto} harder
to climb than a five-rung ladder; and (b) estimate less upward social
mobility when a questionnaire first asks about (primes the concept of)
\textit{im}mobility (as ours did).\footnote{In our original study
  (Chambers et al., 2015), we first asked participants to estimate
  “the percentage of children born to parents in the BOTTOM 3rd of the
  income distribution who stayed in the bottom third of the income
  distribution (i.e., lower class), like their parents.”} Citing
additional evidence that people normally conceptualize movement in
rankings as the likelihood of rising up rather than as the likelihood
of staying put (Davidai \& Gilovich, 2016), Davidai and Gilovich
(2018) concluded that their original (2015) quintile prompts elicited
reliable and valid responses (whereas our original tertile prompts did
not).

Although we find Davidai and Gilovich's (2018) new data to be
interesting, we do not believe that they support their central claim:
that the average American overestimates the actual number of people
who manage to transcend childhood poverty in the United States. We
dedicated most of the text in our (2017) critique to an experimental
demonstration of the quintile-tertile effect, but, as we noted in our
(2017) introduction, the \textit{key} distinction between the Davidai
and Gilovich (2015) and Chambers et al. (2015) investigations was
simpler: \textit{the two teams had applied different true-mobility
  rate comparators}. Whereas Davidai and Gilovich (2015) had weighed
their participants’ guesses against data from the Panel Study of
Income Dynamics (PSID) as reported by the Pew Economic Mobility
Project (2012), we (Chambers et al., 2015) had relied on tax-record
data curated by Chetty, Hendren, Kline, Saez and Turner
(2014).\footnote{Note that the Pew Research Center in 2015 updated its
  sampling methods to bring them more into line with those used by
  Chetty et al.} The distance between the two comparators matters:
when one contrasts Chetty et al.’s true-mobility-rate-point-estimate
(rather than Pew’s) with Davidai and Gilovich’s (2015)
participant-guess-averages, the overestimation-of-mobility effect
disappears entirely. To wit, if Davidai and Gilovich had used Chetty
et al.’s (2014) data to compute true-mobility-rate benchmarks in their
original (2015) study, they would have discovered that their
participants perceived social mobility trends with dead-on
accuracy.\footnote{Pew (2012) reported that 43\% of children born into
  the bottom quintile of the U.S. income distribution remained there
  as adults. Chetty et al.\ (2014) estimated the same quantity to be
  33\%. Davidai and Gilovich’s (2015) participants’ guesses were, on
  average, right on the money (\textit{M} = 33\%).} Thus, even if one
accepts that quintile measures yield psychometrically superior data,
Davidai and Glivoch’s (2018) overestimation claim still depends on the
belief that Pew’s (2012) point estimates are more accurate than Chetty
et al.’s (2014).

Our (2017) discovery of elicitation-method sensitivity (the
quintile-tertile effect) alone would not have led us to conclude that
the quest to quantify population-level perceptual (in)accuracy is
hopeless. But reckoning with \textit{both} of these problems — (a)
comparators with unknown error margins weighed against participants’
responses to (b) estimation prompts with a high potential for subtle
systematic bias — did move us to emphasize the formidable
methodological challenges that face researchers who approach this
topic. The first problem (selecting the correct comparator) does not
appear to have a single econometric solution (e.g., see Bloome, 2015;
and Warren, 2014). Comparators should therefore be accompanied by
clear justifications and statements of limitations.

Davidai and Gilovich (2018) seem to believe that they have solved the
second problem, that of ensuring that mobility-perception prompts
avoid hidden question-wording confounds. We respectfully disagree —
even if the evidence continues to favor the use of quintile response
formats, other possible complications remain. For instance, Davidai
and Gilovich’s (2015) questionnaire apparently asked participants to
forecast mobility rates \textit{moving forward} — the likelihood that
a person born today \textit{would} end up in a given income group
\textit{in the future}. Both Pew (2012) and Chetty et al. (2014)
approximated upward social mobility trends \textit{in the past}. Thus,
unless one assumes that intergenerational elasticity in earnings will
remain stable over the next several decades (this seems highly
unlikely to us; see Chetty et al., 2014; and Mishel, Bivens, Gould \&
Shierholz, 2012), arguably there is no valid comparator for Davidai
and Gilovich’s (2015) data. It may be that most participants in
Davidai and Gilovich’s (2015) study ignored these temporal cues and
estimated mobility rates in the past, but without a direct empirical
test, in our view it remains prudent to doubt Davidai and Gilovich’s
(2018) broad conclusion that “Americans indeed overestimate the
[current] degree of mobility in the United States.”

Of course, there are other ways to assess trends in population-level
perceptual (in)accuracy. Consider, for instance, Kraus and Tan (2015)
— two weeks \textit{after} Davidai and Gilovich (2015) and two weeks
\textit{before} us (Chambers et al., 2015), Kraus and Tan
\textit{also} published a paper (in the \textit{Journal of
  Experimental Social Psychology}) on the topic of how Americans
perceive the degree of socio-economic mobility in the United
States. Like Davidai and Gilovich (2015), Kraus and Tan asked their
online survey respondents (Study 1 \textit{N} = 751) to guess the
future social class rankings of people born into the bottom income
quintile, and, like Davidai and Gilovich, Kraus and Tan observed undue
optimism. (See also Kraus, 2015.) But Kraus and Tan (2015) also asked
their study participants to estimate several \textit{more specific}
social mobility statistics. For instance: how many people out of 100
during a ten-year time period from 1997--2006 would move up from the
bottom 20\% of income (a) by working 1000 extra hours or (b) with some
kind of college degree? People overestimated on both counts.

Should we treat Kraus and Tan’s (2015) findings as a line of
converging independent evidence that, despite the methodological
concerns we have raised about Davidai and Gilovich’s (2015) and
Chambers et al.’s (2015) studies, Americans really do overestimate
social class mobility? Again, the comparator problem rears its head —
Kraus and Tan drew their benchmark values from yet another data source
(Mishel, Bivens, Gould \& Shierholz, 2012). Kraus and Tan's
real-mobility-rate estimates were more pessimistic: citing data from
the PSID, they claim that fewer than 1\% of people born into the
bottom 20\% will rise to the top 20\% in adulthood. (Chetty et
al. estimated the number closer to 8\%.) The consistency of Kraus and
Tan’s participants’ responses across a diverse array of questions
\textit{may well signify an underlying socio-cognitive
  phenomenon},\footnote{However, subtle framing effects may have
  influenced Kraus and Tan’s (2015) participants’ responses — for
  instance, asking participants solely about movement from the very
  bottom to the very top could have primed the concept of
  mobility. (Davidai \& Gilovich, 2018, documented a related framing
  effect.)}  but, in our view, without a clear justification for the
use of these data as the comparator, the title of Kraus and Tan’s
(2015) paper, \textit{Americans Overestimate Social Class Mobility}
(see also Kraus’ 2015 pre-registered replication report,
\textit{Americans Still Overestimate Social Class Mobility}) remains
debatable.\footnote{Though, as we noted in our (2017) article, ``Kraus
  and Tan’s 2015 contribution focused chiefly on comparing mobility
  perception averages between groups (e.g., between people of high
  versus low subjective socio-economic status). The question-wording
  measurement error that we identified in [our] studies need not bear
  on most of Kraus and Tan’s results, which do not require that the
  dependent measure be externally valid'' (p.\ 513).}

Of course, there are still \textit{other} ways to assess trends in
population-level perceptual (in)accuracy. In a final pair of studies
(\textit{N} = 206), Davidai and Gilovich (2018) observed that
Americans tend to rank the United States too favorably among a group
of fifteen countries with varied social class mobility rates. Does the
preponderance of evidence \textit{now} support the broad claim that
Americans’ overestimate social class mobility? Alas, although we may
update our priors, our skepticism endures. In this case, we note that
people’s estimates of social mobility rates \textit{within} a country
may be orthogonal to their estimates of social mobility rankings
\textit{between} countries. One can imagine, for instance, that while
most Americans believe their country boasts more socio-economic
mobility than most others (which is demonstrably not the case),
opportunities for advancement are still unfairly constrained (which is
demonstrably the case). Moreover, in our 2017 critique we developed
our own novel measure of beliefs about economic mobility — when we
asked our participants (\textit{N} = 722) to pick from between three
graphical representations of (pictures of ladders overlaid with)
different social-mobility-rate distributions, the results could not
have been clearer: the vast majority of our participants (72\%)
\textit{underestimated} even when they were (a) presented with
quintiles and (b) could have selected a distribution that corresponds
exactly to the average participant guesses (overestimates) from
Davidai and Gilovich’s (2015) original study. Davidai and Gilovich
(2018) did not mention or discuss this finding. Yet another team of
researchers (Alesina, Stantcheva \& Teso, 2018) asked a slightly
different question, used Chetty et al.,’s (2014) data as their
comparator, and documented a trend of \textit{accuracy}.

When we advised our judgment-and-decision making colleagues to shift
away from the question of population-level (in)accuracy, we were
admitting our own discouragement in the face of so many granular
complexities. Davidai and Gilovich respectfully disagreed, and we are
glad they did. Indeed, effective policy development depends on our
ability to discern these trends. And so, in the end, we concur — the
quest to measure distortions in Americans’ socio-economic-mobility
perceptions deserves scientific attention.\footnote{Clearly many
  people do overestimate social mobility rates (even if a majority of
  Americans do not), and those perceptions likely have democratic
  consequences. It would be useful to know who those people are (see
  Kraus \& Tan, 2015 for some answers). It would also be useful to
  know why, if a majority of Americans really do perceive mobility
  rates accurately (Alesina et al., 2018), they seem to care
  relatively little about the fact that opportunities for social
  advancement are so unevenly (unfairly) distributed (Gallup, 2016).}
Our step-one proposal is to select and justify a fair comparator (with
appropriate error bars).

\section*{References} 

\begin{hangparas}{1em}{1}

Alesina, A., Stantcheva, S., \& Teso, E. (2018). Intergenerational mobility and preferences for redistribution. \textit{American Economic Review, 108}(2), 521--554. \url{https://www.aeaweb.org/articles?id=10.1257/aer.20162015}

Bloome, D. (2015). Income inequality and intergenerational income in the United States. \textit{Social Forces, 93}(3), 1047--1080. \url{https://doi.org/10.1093/sf/sou092}
 
Chambers, J. R., Swan, L. K., \& Heesacker, M. (2015). Perceptions of U.S. social mobility are divided (and distorted) along ideological lines. \textit{Psychological Science, 26}(4), 413--423. \url{http://doi.org/10.1177/0956797614566657} 

Chetty R., Hendren N., Kline P., Saez E., Turner N. (2014). \textit{The Equality of Opportunity Project — Online Data Table 1: National 100 by 100 transition matrix}. Retrieved from \url{http://www.equality-of-opportunity.org/index.php/data} 

Davidai, S., \& Gilovich, T. (2015). Building a more mobile America — one income quintile at a time.\textit{ Perspectives on Psychological Science, 10}(1), 60--71. \url{http://doi.org/10.1177/1745691614562005}

Davidai, S., \& Gilovich, T. (2016). The tide that lifts all focal boats: Asymmetric predictions of ascent and descent in rankings. \textit{Judgment and Decision-Making, 11}(1), 7--20. \url{http://journal.sjdm.org/15/151021/jdm151021.html}

Davidai, S., \& Gilovich, T. (2018). How should we think about Americans’ beliefs about economic mobility? \textit{Judgment and Decision-Making, 13},(3). 

Gallup (2016). \textit{What do you think is the most important problem facing the country today?} Retrieved from: \url{http://www.gallup.com/poll/1675/most-important-problem.aspx}

Kraus, M. W. (2015). Americans still overestimate social class mobility: A pre-registered self-replication. \textit{Frontiers in Psychology, 6}, 1709. \url{https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4637403}

Kraus, M. W., \& Tan, J. J. X. (2015). Americans overestimate social class mobility. \textit{Journal of Experimental Social Psychology, 58}, 101--111. \url{http://doi.org/10.1016/j.jesp.2015.01.005}

Mishel, L., Bivens, J., Gould, E., \& Shierholz, H. (2012). \textit{The state of working America} (12th edition). New York: Cornell University Press. \url{http://www.stateofworkingamerica.org/subjects/overview/?reader}

\vfill
\break

Pew Economic Mobility Project. (2012). \textit{Pursuing the American dream: Economic mobility across generations}. Retrieved from \url{http://www.pewstates.org/uploadedFiles/PCS_Assets/2012/Pursuing_American_Dream.pdf}.

Pew Research Center. (2015). \textit{Economic mobility in the United States}. Retrieved from \url{http://www.pewtrusts.org/~/media/assets/2015/07/fsm-irs-report_artfinal.pdf}.

Swan, L. K., Chambers, J. R., Heesacker, M., \& Nero, S. S. (2017). How should we measure Americans’ perceptions of socio-economic mobility? \textit{Judgment and Decision-Making, 12}(5), 507--515. \url{http://journal.sjdm.org/16/16506b/jdm16506b.html}

Warren, J. R. (2014). Potential data sources for a new study of social mobility in the United States. \textit{The Annals of the American Academy of Political and Social Science, 657}(1), 208--246. \url{https://doi.org/10.1177/0002716214552773}

\end{hangparas}

\end{document}
