Judgment and Decision Making, Vol. 15, No. 5, September 2020, pp. 685-703

We examine a Query Theory account of risky choice framing effects — when risky choices are framed as a gain, people are generally risky averse but, when an equivalent choice is framed as a loss, people are risk seeking. Consistent with Query Theory, frames affected the structure of participants’ arguments: gain frame participants listed arguments favoring the certain option earlier and more often than loss frame participants. These argumentative shifts mediated framing effects; manipulating participants initial arguments attenuated them. While emotions, as measured by PANAS, were related to frames but not related to choices, an exploratory text analysis of the affective valence of arguments was related to both. Compared to loss-frame participants, gain-frame participants expressed more positive sentiment towards the certain option than the risky option. This relative-sentiment index predicted choices by itself but not when included with structure of arguments. Further, manipulated initial arguments did not significantly affect participant’s relative sentiment. Prior to changing choices, risky choice frames alter both the structure and emotional valence of participants’ internal arguments.

Keywords: risky choice framing; query theory; choice processing

1  Introduction

We argue that when evaluating a risky decision, like choosing to settle a lawsuit or negotiating a contract, a person considers a series of arguments in favor of and against the options. Specifically, we examine how the structure of these arguments is affected by how the risky choice is framed — presenting equivalent risky decisions framed as either gains or losses. Consistent with Query Theory, we posit that risky choice frames differentially shift participants’ arguments to favor one option. When framed as a gain, a certain option attracts participants arguments favoring it, perhaps because it has more positive emotional valence than the risky alternative. But when framed as a loss, the certain option repels arguments favoring it because it has a more negative emotional valence than the risky alternative. According to Query Theory (Johnson, Häubl & Keinan, 2007; Weber et al., 2007), initial arguments in favor of an option underlies value construction: the initially favored option crowds out arguments for the other option, leading people to generate more arguments in favor of the early favorite, thus choosing it more often.

In our studies, we find that the structure of arguments is related to choice: risky choice frames affect which option receives arguments in its favor, which mediates the effect of frame on choice. In addition, we find that emotions, as measured by PANAS (Positive And Negative Affect Schedule), are predicted by frame but not predictive of choice. Given this unexpected result, we performed an exploratory sentiment analysis, which measured the affective valence of participant’s arguments. Unlike PANAS, this sentiment index was predicted by frame and predictive of choices. These results suggest that, on the way to changing choices, risky choice frames change both the structure of participants’ internal arguments their arguments’ emotional valence.

Risky choice framing effects appear in a variety of domains: financial decisions (Levin, Schneider & Gaeth, 1998), medical decisions by both patients and doctors (McNeil, Pauker, Sox & Tversky, 1982), and negotiations (Neale & Bazerman, 1985). Given the impact of risky choice frames in disparate domains, it is important to understand the underlying psychological mechanisms to debias decisions.

2  Literature Review

Risky choice frames can change what people chose (Levin et al., 1998). When risky choices are framed as gains, participants select the certain option more often than when risky choices are framed as losses (Kahneman & Tversky, 1979; Tversky & Kahneman, 1981), even when both frames are logically equivalent. For instance, in the Unusual Disease Problem,¹ participants chose between two public health programs for the treatment of a disease: a certain option — e.g., where 200 of the 600 people will live — and a risky option — e.g., where there is a 1/3 probability that all 600 will live and a 2/3 probability that no one will live. When the treatment outcomes are framed as gains — e.g., 200 people will be saved — participants choose the certain option more often than when logically equivalent treatment outcomes are framed as losses — e.g., 400 people will die. Given that frames alter decisions, we can infer that they also alter participants’ cognitive and/or emotional processes preceding a decision.

Risky choice framing effects were first explained by the Prospect Theory value function, which predicts risk aversion for gains but risk seeking for losses (Kahneman & Tversky, 1979). That is, for gains people prefer a sure gain to an equal expected value risky gain, but for losses people prefer a risky option to an equal expected value sure loss. Applied to the Unusual Disease Problem, in the gain frame risk aversion predicts choice of the certain option, but in the loss frame risk seeking predicts choice of the risky option.

Different risk preferences for gains and losses, however, seems inconsistent with choice patterns in variants of the Unusual Disease Problem. For instance, the framing effect diminishes when the risky options are truncated, e.g., participants are shown “a 1/3 probability that all 600 will be saved” but not a “2/3 probability that no one will be saved”. While Prospect Theory does not explain this result, other theories do (Kühberger, 1998; Kühberger & Gradl, 2013; Mandel, 2014; Rahimi-Golkhandan, Garavito, Reyna-Brainerd & Reyna, 2017; Tombu & Mandel, 2015). A widely cited encoding account of the Unusual Disease Problem — Fuzzy Trace Theory (Rahimi-Golkhandan et al., 2017) — suggests that people encode only the gist of the options. In the gain frame, the gist of the certain option is “some people are saved” while the gist of the un-truncated risky option is “some people are saved, or no one is saved.” When comparing these two gists, “some people are saved” is better than “some people are saved, or no one is saved”, making the certain option more desirable. In the loss frame, however, the gist of the certain option — “some people will die” — is less desirable than the gist of the risky option — “some people will die or no one will die” (Reyna & Brainerd, 1991). Fuzzy Trace Theory explains the effect truncating risky options by people encoding a different gist leading to different choices. Changing the gist of certain option also changes choices (Kühberger, 1995). Risky choice framing effects depend on how the options are described.

While Fuzzy Trace Theory and other theories explain choices on variants of the Unusual Disease Problem, we argue that there are multiple distinguishing features of Query Theory that make it an attractive explanation. First, it explains a broad set framing effects in other domains — attribute framing (Hardisty, Johnson & Weber, 2010), the endowment effect (Johnson, Häubl & Keinan, 2007), and the accelerate/delay effect in intertemporal choice (Weber et al., 2007) — making it a parsimonious explanation. Second, it allows for individual differences in framing effects: labeling an environmentally friendly option as either a tax or an offset affected Republicans’ attention but not Democrats’ (Hardisty, Johnson & Weber, 2010). Third, unlike other explanations of the Unusual Disease Problem, it makes novel predictions about how shifting initial the order of arguments will affect choice. Fourth, our measure of participants’ internal argumentative processes — thought listings — provides, via natural language processing, a granular measure of the emotional valence of participants’ arguments.

Query Theory assumes people evaluate options by sequential queries of memory, retrieving arguments in favor of one option then arguments in favor of the other option. Importantly, the option with early arguments in its favor is more likely to be chosen. Initially retrieving arguments in favor of an option interferes with the subsequent retrieval of arguments in favor of the other option. Differences in argument retrieval are related to choice: if someone retrieves arguments in favor of one program earlier, there will be more arguments for choosing it, making it will more likely be chosen. Therefore, a key question in Query Theory is: which option has initial arguments in its favor. The answer depends on the problem (Weber & Johnson, 2009). For instance, the status quo affects thought order (Johnson et al., 2007; Weber et al., 2007). When making an intertemporal choice where the immediate option was the status quo, people queried arguments for taking the immediate option earlier than the delayed option. Further, arguments in favor of the status quo were more accessible: participants were faster to recognize queries, which were consistent with the status quo than thoughts inconsistent with the status quo. Which option has initial arguments in its favor varies based on individual differences: labeling a carbon user fee as an offset or a tax shifted the structure of Republicans’ arguments but not Democrats’ (Hardisty, Johnson & Weber, 2010).

Given that the Unusual Disease Problem does not have an obvious status quo and does not alter attribute labels, we hypothesize that another element of the options drives initial arguments: the emotional valence of the choice options (Gonzalez, Dana, Koshino & Just, 2005; Tombu & Mandel, 2015; Willemsen, Böckenholt & Johnson, 2011). In the gain frame, the certain option (“saved” has a positive valence) has a more positive emotional valence than the risky option (the positive valence of “saved” is negated by “no one”). Therefore, the certain option attracts participants initial arguments. But in the loss frame the certain option (“die” has a negative valence) has more negative emotional valence than the risky option (“die” is negated by “no one”) meaning arguments should shift towards the risky option (Kühberger & Gradl, 2013; Tombu & Mandel, 2015; Wallin, Paradis & Katsikopoulos, 2016).

Further, risky choice framing effects are related to specific emotions participants are experiencing. For instance, the emotions experienced by participants moderated the effect of frames: participants with elevated levels of distress were more likely to show risky choice framing effects compared to those who were experiencing low levels of distress (Druckman & McDermott, 2008). Manipulated emotions also related to risky choice framing effects (Druckman & McDermott, 2008). People manipulated to feel distressed showed a risky choice framing effect but the effect was attenuated in those manipulated to feel enthusiasm.

Other work suggests that manipulating emotions may also manipulate which option is favored by early arguments. For instance, sadness and its associated sense of loss triggers an implicit goal of reward replacement (Lerner, Small & Loewenstein, 2004). This implicit goal relates to intertemporal choices between monetary amounts received at different points in time. Participants who watched a sad movie clip made more impatient intertemporal choices than participants who watched a neutral or a disgusting movie clip (Lerner, Li & Weber, 2013). Consistent with participants’ emotions shifting argument order, participants manipulated to experience sadness were more likely to list arguments in favor of the immediate reward option than those manipulated to experience disgust or neutral-feelings. That is, emotions changed choices by changing which option received favorable initial arguments.

Instead of focusing on manipulated emotions unrelated to the choice, we focus on observing the emotions that result from risky choice frames. This shifted focus is due to the fact that we hypothesize that initial arguments in the UDP are due to the emotional valence of the choices. We measure the effect of frames on emotions in two ways (Watson, Clark & Tellegen, 1988): First, using items from the PANAS (Watson, Clark & Tellegen, 1988) that relate to gambling behavior (Raghunathan & Pham, 1999) and risky choice framing effects (Druckman & McDermott, 2008).

And second, we measured the valence of participant’s arguments via exploratory text analysis. Using natural language, text analyses predict movie ratings from review text, depressive tendencies in patients from their essays, and investing behavior from twitter activity (Chatterjee & Perrizo, 2016; Da, Engelberg & Gao, 2015; Rude, Gortner & Pennebaker, 2004; Turney, 2002). Relatedly, the perceived risk of an activity is predicted by its semantic relatedness to emotional words, e.g., the more semantically related an activity is to fearful words the higher its perceived risk (Bhatia, 2018). Measuring only the emotional valence of participant’s thoughts misses out on multiple emotional dimensions, e.g., how certain, how in control, and how much responsibility people feel (Lerner, et al., 2015). However, emotional valence, by itself, is related to risky choice framing effects (Tombu & Mandel, 2015; Wallin et al., 2016). For instance, framing effects were attenuated, holding the risky choice constant, when the gain framed certain option had a negative valence and the loss framed certain option had a positive valence (Tombu & Mandel, 2015). While the emotional valence of participant’s thoughts is only a single dimension of emotions, prior research suggests it captures choice relevant information.

In sum, prior work on framing effects suggests that risky choice frames affect the structure and emotional valence of participants arguments. Our work has three goals: first and foremost, to test a Query Theory account of risky choice framing effects; second, to examine how risky choice frames affect participant’s reported emotions; and third, to present an exploratory text analysis of the content of participant’s arguments. Studies 1 and 2 showed that early arguments favoring an option, as predicted by Query Theory, were related to risky choices. Studies 3 and 4 replicated Studies 1 and 2 in a more vivid scenario that attempted to increase participant’s emotional reactions to the stimuli. Further, these studies showed that a conventional measure of specific emotions — items from the PANAS scale — were altered by gain/loss framing but were not related to choices or the structure of participants arguments. However, a series of exploratory analyses showed that the sentiment of participants arguments was altered by frames and related to both choices and the structure of participants arguments — e.g., which option was the early favorite and which option had more thoughts favoring it. We jointly analyzed emotional valence and the structure of participants arguments in a path analysis. This model suggested that risky choice frames alter both the sentiment of participant’s arguments and the argument’s structure but that the sentiment of arguments accounts for no additional variance beyond the structure of arguments. Another analysis suggested that frames affected the emotional valence of participants thoughts before affecting the structure of their argument.

3  Studies

3.1  Hypotheses

In Studies 1 and 2, we focused on how of risky choice frames affect participants’ arguments and subsequent choices. That is, these studies test if risky choice framing effects can be accounted for by Query Theory. Prior Query Theory studies showed that, while deliberating their decision, certain aspects of an option lead participants to argue in its favor earlier and more often (Johnson et al., 2007; Weber et al., 2007). In the Unusual Disease problem, we hypothesize that the certain option is initially considered in the gain frame (200 people saved) because certain gains are attractive, but not in the loss frame (400 people die) because certain losses are aversive making the risky prospect to become the initial leader.

Hypothesis 1:

Frames shift the structure of participant’s arguments.

The framing of the problem — as a gain or a loss — alters the order in which people consider options. Specifically, in the gain frame participants should list arguments favoring the certain option — those that make a case for the certain option being good or the risky option being bad — earlier and more often relative to the risky option than participants in the loss frame.

Whereas prior work demonstrates that risky choice frames direct attention in the form of acquiring information about options, we investigate how frames direct arguments participants make while deliberating (Willemsen et al., 2011).

Consistent with prior Query Theory papers we hypothesize that frames affect arguments and choices.

Hypothesis 2:

The structure of participant’s arguments predicts choices and mediates the effect of frames on choice.

Framing alters initial arguments and the number of arguments, which then alter choice. The effect of frame on choice will be mediated by differences in the thought listings. Study 1 tested hypotheses 1 and 2.

3.2  Study 1: A Query Theory Account of the Unusual Disease Problem

3.2.1  Method

Participants

We recruited 180 participants (93 female), who ranged in age from 19 to 65 (M = 33.7, SD = 10.84), 143 reported an education level of some college or higher, from Amazon Mechanical Turk, an online labor pool. Participants received $1 for completing the study. To ensure participants read the instructions and prompts carefully, we included an attention check question. Participants who failed the attention check (10 participants) were not included in the final analysis.

Procedure

Participants first listed thoughts about purchasing a vehicle to familiarize themselves with the thought listing interface (Hardisty et al., 2010; Johnson et al., 2007; Weber et al., 2007). Next, participants read either a gain or loss framed version of the Unusual Disease problem. As opposed to having 600 expected fatalities, we changed this to 720 to make the proportions more difficult to calculate.²

The exact wording used for the gain and loss frames follows:

Imagine that the U.S. is preparing for the outbreak of an unusual Asian Disease, which is expected to kill 720 people. Two alternative programs to combat the disease have been proposed. Assume that the exact scientific estimate of the consequences of the programs are as follows (participants did not see the text in square brackets):

[Gain Frame:]

If Program A is adopted, 240 people will be saved. [Certain Option]

If Program B is adopted, there is 1/3 probability that 720 people will be saved, and 2/3 probability that no people will be saved. [Risky Option]

[Loss Frame:]

If Program A is adopted 480 people will die. [Certain Option]

If Program B is adopted there is 1/3 probability that nobody will die, and 2/3 probability that 720 people will die. [Risky Option]

After participants read the Unusual Disease problem, they were instructed: “Before you indicate your preference for these programs, please tell us everything you are thinking of as you consider this decision between Program A and Program B. We would like you to list any thoughts, both positive and negative, that you might have about this decision. We will ask you to enter your thoughts one at a time.” On average, participants listed 3.94 thoughts (SD = 1.96). After their thought listing, participants choose one of the two programs: Program A (the certain option) or Program B (the risky option). Next, participants were shown their previously written thoughts, one at a time, and asked to place them in one of five categories: “An advantage of Program A”, “A disadvantage of Program A”, “An advantage of Program B”, “A disadvantage of Program B”, or “neither”. Finally, participants provided demographic information.

3.2.2  Results

Choices.

Replicating prior Unusual Disease studies, gain-frame participants were more likely to choose the certain option (71%) than participants in the loss frame (55%) (β = 0.69, SE = .33, p = 03).³

Thought Listings.

As in prior Query Theory studies, we combined thoughts participants rated as “An advantage of Program A”, and “A disadvantage of Program B” into pro-certain thoughts. Similarly, we combined thoughts participants rated as “An advantage of Program B”, and “A disadvantage of Program A” into pro-risky thoughts. (Thoughts participants rated as “neither” were not used, as they are not choice relevant arguments.)

As with prior studies, we characterized the structure of participants’ thought listings using the Standardized Median Rank Difference (SMRD) and the Balance of Thoughts (Johnson et al., 2007). SMRD was calculated as 2(MR_r−MR_c)/n, where MR_c is the mean rank pro-certain thoughts and MR_r is the mean rank of pro-risky thoughts and n is the total number of pro-certain and pro-risky thoughts. SMRD ranges from +1 (all pro-certain thoughts listed before all pro-risky thoughts) to −1 (pro-certain thoughts listed after all pro-risky thoughts). A SMRD of zero corresponds to pro-risky and pro-certain thoughts having the same median rank. We used SMRD instead of other measures — e.g., which option a participant’s first thought argued in favor of — because it has been used in all prior Query Theory papers and gives a broader view of the structure of participants arguments. Balance of Thoughts was simply the difference between the number of pro-certain and pro-risky thoughts. For example, if a participant listed 2 more pro-certain thoughts than pro-risky thoughts, then their Balance of Thoughts was +2. For the twelve participants rated all thoughts as only belonging to the “neither” category we set both SMRD and Balance of Thoughts to 0.

Consistent with Hypothesis 1 — Frames shift the structure of participant’s arguments — participants had a higher SMRD — i.e., they listed pro-certain arguments earlier — in the gain frame (M = .40) than in the loss frame (M = .14) (t(168) = 1.96, p = .052, Cohen’s D = 0.30). Participants in the gain frame had a significantly more positive Balance of Thoughts (M = 1.18) than participants in the loss frame (M = 0.41) (t(159.12) = 2.5, p = .013, Cohen’s D = 0.39). Further, as predicted by Query Theory, SMRD and Balance of Thoughts were correlated (r = .59, t(168) = 9.53, p < .001).

Mediation.

Hypothesis 2 predicts that the structure of arguments mediates the effect of frame on choice. To simplify our mediation and prediction analysis, we created a Structure of Thoughts index that z-scored both SMRD and Balance of Thoughts and took their average. Structure of Thoughts — which combines rank and order information — predicted choice (β = 1.56, SE = .25, p < .001).⁴

To fit the mediation model, we used the lavaan R package (Rosseel, 2012) with 5000 bootstrapped iterations. We used a probit model because the dependent variable, choice, is binary (Imai, Keele & Tingley, 2010).

Figure 1A shows the results of the mediation model. Consistent with hypothesis 2, there was a significant effect of frame on Structure of Thoughts (β = 0.33, SE = 0.13, p = .013). Also, the effect of Structure of Thoughts on choice was significant (β = 0.30, SE = 0.032, p < .001). After including the indirect path, the effect of frame was not significant (β = 0.06, SE = 0.06, p = .33). Importantly, the indirect effect from frame through Structure of Thoughts to choice was significant and positive (β = 0.098, SE = 0.039, p = .01). The significant indirect path suggests that the structure of participant’s arguments mediates risky choice framing effects.

---

Figure 1: Path models predicting choice by frame for Study 1 (Panel A) and the vivid stimuli of Study 3 (Panel B). Direct effects without mediator are in parentheses. * p < .05 *** p < .001.

---

3.3  Study 2: Manipulated Thought Order Changes Choices

While Study 1 established that Query Theory processes mediate risky choice framing effects, it did not manipulate initial arguments. Study 2 directly tests hypothesis 3.

Hypothesis 3:

Manipulating the order in which participants attend to arguments will attenuate framing effects.

Specifically, when participants are forced to consider the options in an unnatural order — in the gain frame considering the risky option first, in the loss frame considering the certain option first — the effect of frame on choice will be attenuated. Consistent with prior studies, we predicted attenuation not reversal of framing effects because the frame induced natural order is difficult for participants to suppress and therefore still operates (Johnson et al., 2007; Weber et al., 2007).

3.3.1  Method

Participants

We recruited 584 participants (263 female), ranging between the ages of 18 and 72, (M = 33.26, SD = 10.46) with 124 having an education level of at least some college from Amazon Mechanical Turk. An attention check question removed 50 participants.

Procedure

Participants saw either the gain-framed or loss-framed choice and were instructed to first generate thoughts supporting the certain option followed by thoughts supporting the risky option, or vice versa. Otherwise the materials and procedures were identical to Study 1. Based on the results of Study 1, we categorized listing the arguments supporting choice of the certain first as the “natural” order for the gain frame and the “unnatural” order for the loss frame. In contrast, we considered listing arguments supporting the risky option first as the “natural” order for the loss frame and the “unnatural” order for the gain frame. That is, participants saw one of four between-subject conditions (2 frames by 2 argument-orders).

3.3.2  Results

To test the effect of manipulated argument order, we performed logistic regression with choice predicted by frame, argument listing order (natural or unnatural), and their interaction. (We used contrast coding for this analysis).

We replicated the standard risky choice framing effect. Gain-frame participants were more likely to choose the certain option (71.4%) than loss-frame participants (42.6%) (β = 0.61, SE = .09, p < .001). Analyses of simple effects showed that the effect of frame was significant both in the natural order (β = 1.76, SE = 0.27, p < .0001) and unnatural order (β = 0.70, SE = 0.25, p = 006). As shown in Figure 2A, however, there was a significant interaction: the effect of frame was larger in the natural order than in the unnatural order (β = 0.26, SE = 0.09, p = .004). Consistent with Hypothesis 3 and prior Query Theory studies, the effect of frame on choice was attenuated when participants listed arguments in an unnatural order.

Because the natural order in the gain frame and the unnatural order in the loss frame both involve listing pro-certain thoughts first and the unnatural gain frame and natural loss frame both involve listing pro-risky thoughts first, we expected no main effect of unnatural versus natural thought order. As expected, there was no main effect of thought order on choice (β = 0.04, SE = .09, p = .66).

---

Figure 2: Percentage of certain option choices for the natural and unnatural thought listing conditions in Study 2 (A) and Study 4 (B). Error-bars are 95% Confidence Intervals.

---

3.4  Study 3: Emotional Influences on Framing

Studies 1 and 2 demonstrated that the structure of participant’s arguments relates to framing effects; however, both did not explore how emotions relate to choices (e.g., Druckman & McDermott 2008) or the structure of arguments. Study 3 tested whether experienced emotions — as measured by the PANAS scale — related to framing effects.

Additionally, we created a new, more vivid, version of the Unusual Disease Problem. While prior work has manipulated the emotional valence of the two choice options by changing their description — in the gain frame adding “400 lives will not be saved” to the certain option — we attempted to increase participants’ overall level of emotional engagement of the scenario. Prior work suggests that people with reduced emotional reactions display smaller risky choice framing effects than people with normal emotional reactions (De Martino, Harrison, Knafo, Bird & Dolan, 2008). Given this result, we theorized that increasing participants’ emotional reactions via more vivid stimuli would enhance the effect of frames. As shown in Figure 3, we increased vividness and emotional engagement in three ways. First, we changed “unusual Asian Disease” to “Ebola”, which was salient due to a recent, at the time of the study, outbreak of Ebola in West Africa. Second, we indicated that the individuals infected by the virus were children. Third, we added photos: the gain frame showed a child recovering in a hospital bed, and the loss frame a child’s coffin being lowered into the ground. (While this may seem like a confound, the decision maker can logically infer in the gain frame that 480 children will die — and that there will be coffins — even though her attention — either verbally or by visual — is being focused on the children that are saved.)

---

Figure 3: Vivid, emotionally engaging gain-framed risky choice (Left) and loss-framed risky choice (Right)

---

3.4.1  Method

Participants

We recruited 258 (129 female) respondents, ranging between 18 and 72 years old (M = 34.77, SD = 10.86) with 163 reporting an education level of some college or higher, from Amazon Mechanical Turk. An attention check question removed 13 participants.

Procedure

The procedure was nearly identical to Study 1. We wanted to assess the emotional impact of the frame, so we measured the change in participants’ emotions using a subset of a modified version of the PANAS scale, as used by Lerner et al. (2013). Participants rated on a scale from 0 (Do not feel even the slightest bit) to 8 (Feel the emotion more strongly than ever) each of the following emotions both before and after making their decision: Positive Emotions (Amused, Cheerful, Happy); Neutral Emotions (Neutral, Unemotional); and negative emotions (Afraid, Angry, Bored, Depressed, Disgusted, Fearful, Furious, Gloomy, Indifferent, Mad, Nauseated, Nervous, Repulsed, Sad). Having a measure of these emotions both prior to and after making the choice allowed us to test if the two frames of the vivid Unusual Disease Problem differentially changed participants’ emotions. That is, we were concerned not with how participants were feeling prior to making the decision — incidental emotions — but were concerned with how risky choice frames shifted the choice relevant emotions — integral emotions (Lerner, Li, Valdesolo & Kassam, 2015). Integral emotions are important for the Unusual Disease Problem because, as we argued earlier, the valence of the options may focus participant’s attention.

Four participants who passed the attention check did not rate emotions both prior to and after their choice were removed from these analyses.

3.4.2  Results

Choices

As in Studies 1 and 2, the percentage of participants who chose the certain option was higher in the gain frame (M = 74%) than in the loss frame (M = 44%) (β = 1.25, SE = .27, p < .001).

Thought Listing

Consistent with Hypothesis 1 and Study 1, participants in the gain frame had a higher SMRD (M = .45) than in the loss frame (M = .01) (t(241.81) = 4.24, p < .001, Cohen’s D = 0.54). That is, they listed thoughts about the certain option earlier in the gain frame. Participants also listed significantly more thoughts in favor of the certain option relative to the risky option in the gain frame (M = 1.26) than in the loss frame (M = 0.15) (t(242.3) = 4.09, p < .001, Cohen’s D = 0.52). SMRD and Balance of Thoughts were again correlated (r = .61, t(243) = 12.08, p < .001).

Mediation

Consistent with the Study 1 results, the Structure of Thoughts index — the combination of SMRD and Balance of Thoughts — significantly predicted choice (β = 1.74, SE = 0.23, p < .001).

Consistent with Hypothesis 2, the results of the Study 1 mediation replicated. As seen in Figure 1B, there was a significant effect of frame on Structure of Thoughts (β = 0.52, SE = 0.11, p < .001). Also, the effect of Structure of Thoughts on choice was significant (β = 0.29, SE = 0.25, p < .001). After including the indirect path, the effect of frame was still significant (β = 0.14, SE =0.06, p = .02). Importantly, the indirect effect from frame through Structure of Thoughts to choice was, once again, significant and positive (β = 0.151, SE = 0.035, p < .001). When the sparse stimuli of the Unusual Disease problem were made more vivid, Query Theory processes still mediated the effect of frame on choice. However, the failure of Query Theory processes to completely mediate the effect of frame suggests that frames also change other processes.

PANAS Analysis

Prior work found that specific emotions, as measured by the PANAS items, predicted choices in the Unusual Disease problem (Druckman & McDermott, 2008). (Note: Traditionally PANAS items of a certain valence are combined). For example, participants’ anger correlated with choosing the risky option, independent of frame. Their analysis, however, only investigated how incidental emotions predicted choice, not how frames change emotions. We adapted the methods used by Druckman and McDermott to explore whether frames changed the PANAS emotions and how such changes would relate to choice (Lerner et al., 2015).

To test for the effect of frame on specific emotions, we used a multilevel model — with varying intercepts for each participant — predicting the difference between post- and pre-choice emotions. The 19 emotions were dummy coded. That is, in the data each participant had 19 rows one for each emotion they rated with emotion type indexed by a dummy variable and another dummy variable indicating which frame the participant saw. For ease of exposition, we report the results of an ANOVA with type 3 sums of squares (The Sums of Squares for each variable were entered after all other variables, including interactions.) (Table A1 presents all coefficients.) Answering the Unusual Disease problem did not change all emotions equally, different emotions had different means (χ²(18)=1848.21, p < .001). Critically, changes in participants’ specific emotions were not altered by which frame they saw: there was no main effect of frame (χ²(1)=0.26, p = .61) or interaction between frame and emotion type (χ²(18)=17.60, p = .48).

Moreover, specific emotions measured by the PANAS did not predict choice. Specifically, we ran a regression that predicted choice with continuous predictors — post-choice emotion level minus pre-choice emotion level — for each of the 19 emotions. Because of the large number of predictors in this model, we used a regularized regression model — LASSO — to determine which emotions related to choice (Hastie, Tibshirani & Friedman, 2009). The LASSO shrinks coefficients based on cross validated prediction error. Unlike other regularization techniques, the LASSO allows for coefficient estimates to become zero making it a principled way to select model features. As seen in Appendix B, Figure A1, the LASSO model regularized the coefficients for each emotion to 0: no emotion predicted choice.

In sum, Study 3 replicates Study 1 and demonstrates that specific emotions, as measured with the PANAS scale, did not differ between frames and were unrelated to choice.

3.5  Study 4: Manipulated Thought Order Changes Choices But Not Emotions

Study 2 showed that an instruction that changed the structure of participant’s arguments influenced the effect of frame on choice. Study 4 replicates Study 2 — where initial arguments were manipulated — with Study 3’s vivid stimuli and the PANAS measures. That is, Study 4 tests whether the manipulated thought order of Study 2 changes emotions, as measured by the PANAS.

3.5.1  Method

Participants

For Study 4, we recruited 593 participants (310 female), age range from 18 to 74 (M = 34.71, SD = 11.62) with 182 with at least some college education, from Amazon Mechanical Turk. Participants who failed an attention check question (17 in total) were removed from the analyses.

Procedure

All procedures were identical to Study 2 except participants saw the vivid stimuli and provided the PANAS ratings as in Study 3.

3.5.2  Results

Choice

The choice results mirrored those of Study 2: the effect of frame was attenuated in the unnatural order compared to the natural order. As in Study 2, we performed a logistic regression with choice predicted by frame, thought order, and their interaction. Gain-frame participants were more likely to choose the certain option (70.0%) than loss-frame participants (43.6%) (β = 0.57, SE = .09, p < .001). Consistent with Figure 2B, the effect was of frame was significant in the natural order (β = 1.65, SE = 0.26, p < .0001) and unnatural order (β = 0.61, SE = 0.24, p = .001). However, the effect of frame depended on thought order (β = 0.26, SE = 0.09, p = .004). Consistent with the Study 2 results and Hypothesis 3, the effect of frame on choice was attenuated when participants listed thoughts in an unnatural order. As seen in Figure 2, the interaction between frame and thought order are nearly identical across Studies 2 and 4. As with Study 2, there was no main effect of thought order (β = 0.08, SE = .09, p = .36). When stimuli are more vivid, manipulated initial arguments still attenuates the effect of frame on choice.

PANAS analyses:

We ran a similar model as in the Study 3 PANAS analyses, that asked the additional question: does manipulated thought order change rated emotions? To examine this, we ran a multilevel linear regression, which predicted, for each of the 19 emotions, the difference between post and pre-test emotion by type of emotion, frame, thought listing order, and all two way interactions between the three variables and included a varying intercept for each participant. (We did not include a three way interaction because it frustrates the interpretation of the two way interactions). That is, in the data each participant had 19 rows one for each emotion they rated with emotion type indexed by a dummy variable, and two additional dummy coded variables (e.g., coded as 0 and 1), one for which frame the participant saw and one for which thought order the particpant was instructed to use. (Two participants did not rate the PANAS items both prior to and after their choice and were removed from this analysis). We report the results of an ANOVA with type 3 sums of squares because the interactions are of primary importance: we want to see if Thought Listing Order differentially affects changes in specific emotions. The two two-way interaction between Thought Listing Order and emotion type was not significant(χ²(18) = 27.77, p = .066).⁵. Thus, the effect of manipulated argument order on choices does not appear to be accompanied by a change in specific reported emotions.

As with the Study 3 analyses, when predicting changes in emotions there was a significant effect of specific emotion type — Amused, Cheerful, Happy, Afraid, Angry, etc. — (χ²(18) = 1818.02, p < .001). The effect of frame was not significant (χ²(1) = 0.98, p = .32). Unlike Study 3, there was an interaction between specific emotion and frame (χ²(18) = 74.91, < .001). As can be seen in the Table A2, loss frames induced larger decreases in happiness than gain frames; however, no change in emotion was related to choice.

The conclusion of these results corroborates the conclusion from the Study 3 results: that changes in emotions measured by PANAS do not mediate the effect of frame on choice. Unlike Study 3, emotions, as measured by the PANAS, were associated with the effect of frame but, like Study 3, were unrelated to choice (as seen in Figure A2). More importantly, this study demonstrates that exogenously manipulating thought order, which attenuates the framing effect, does so without changing experienced emotions measured by the PANAS.

4  Exploratory Sentiment Analysis

We argue that the PANAS is a problematic measure of emotions for two main reasons. First, the PANAS measures cannot distinguish if a change in emotion is associated with one option or the other. Feeling a negatively valenced emotion towards the risky option and a positively valenced emotion toward the certain option should drive choices towards the certain option. Second, emotions are evanescent and the PANAS is temporally separated from choice processes. Participants’ emotional reactions may have been over by the time we administered the PANAS (Ericsson & Simon, 1980). To address these two problems, we conducted an exploratory sentiment analysis of the participants’ type-aloud thoughts, which reflected choice-relevant recall. Thought listings are collected during the decision, addressing temporal separation, and each thought is coded by the participant as pertaining to the risky or certain option, allowing us to code which option was the subject of any potentially emotional content of each statement.

The sentiment analyses, however, have a clear weakness: they measure only emotional valence. Measuring only a single dimension of emotions is problematic because emotions with the same valence have different effects on risk taking. Prior work, however, only measured overall emotions; we measure valence about options. The analyses attempt to answer an empirical question: is the emotional valence of participant’s arguments predicted by frames and predictive of choice?

To test this question, we pooled data from Studies 1 and 3 — the two experiments which allowed subjects to adopt their own thought order. We also conducted a related analysis examining the effects of manipulated argument order on valence.

4.1  Method

We measured the sentiment — “the expression of subjectivity as either a positive or negative opinion” (Taboada, 2016, p. 326) — of participants written arguments using text analysis. We used the sentimentr package, which outperforms many other packages on common natural language processing benchmarks (Rinker, 2019). The analysis produces a unidimensional score with 0 representing a neutral valence. Positive valences are represented by positive numbers and negative valences by negative numbers. The degree of sentiment is represented by the distance from zero: “Too many will die” had a sentiment of −1.38 but, the less extreme, “I do not like gambling” had a sentiment of −0.11. The sentimentr package uses regular expressions to parse sentence boundaries and a popular sentiment dictionary (Hu & Liu, 2004) to determine both the sentiment of words and valence shifters. Valence shifters can flip the sentiment of a sentence. For instance, one participant wrote: “go with plan b because there is a possibility of nobody dying”. A context free sentiment analysis technique would assign the thought a negative sentiment, because it contains only one negative word: dying. In contrast, a sentence level sentiment analysis codes this thought as positive: the negative sentiment of “dying” is negated by “nobody”. An accurate measure of sentiment requires placing emotionally laden words in a broader, sentence level, context.

Using the sentence level sentiment of participants arguments, we developed two measures: one that differentiates between the sentiment of thoughts about the certain option and the sentiment of thoughts about the risky option — the relative-sentiment index — and one that adds all thought sentiments together regardless of which option the thought was about — the overall-sentiment index. The relative-sentiment index takes, for each participant, the average sentiment of their arguments about the certain option minus the average sentiment of their arguments about the risky option.⁶ If there were no differences in sentiment between arguments about the certain and risky option, the relative-sentiment index was 0. The overall-sentiment index adds the sentiments of arguments about the certain option to the sentiments of arguments about the risky option. For instance, a participant whose arguments about the certain option had a mean sentiment of +1 and arguments about risky option had a mean sentiment +2, would have an overall-sentiment index of 1+2 = 3.

While the two sentiment indexes use the same data, we predicted their relationship with choices would differ. The overall-sentiment index only measures the valence of people’s overall emotions and emotions with the same valence have different impacts on risky decision making: anger can lead to risk taking, but distress can lead to risk aversion (Druckman & McDermott, 2008; Lerner et al., 2015). Therefore, we did not predict that the overall-sentiment index would relate to choices. Because the relative-sentiment index compares differences in emotional valence about options, we predicted that it would relate to choices: expressing more positive sentiment about the certain option than the risky option should lead to choices about the certain option.

4.2  Results

Using the combined Studies 1 and 3 data, a linear regression showed that the relative-sentiment index was related to choice (β = −0.62, SE = 0.35, p < .001). Another linear regression showed that frames influenced the relative-sentiment index (β = 0.32 SE = 0.15, p < .033), but that vividness, Study 3 had vivid stimuli but Study 1, did not, (β = 0.058, SE = 0.14, p = .67) and the interaction between frame and vividness (β = 0.07, SE = 0.20, p = .7) did not. Even though frames do not change overall reported experienced emotions as measured by the PANAS scale, this result suggests that frames change option-evaluation related emotional valence.

In the Study 3 data, the relative-sentiment index was not related to the PANAS measures: a LASSO regression that predicted the relative-sentiment index with the PANAS scales represented as continuous predictors (post-choice emotion level minus pre-choice emotion level) for each of the 19 emotions regularized all parameters to zero. None of the 19 emotions predicted the relative-sentiment index. Also, relative-sentiment index predicted choice when included in the same regression as PANAS emotions (Figure A3). These results suggest that the relative-sentiment index captures unique variance in emotional processing.

Unlike the relative-sentiment index, the overall-sentiment index was not related to frames or choice. In a simple regression predicting choice, the overall-sentiment index was not significant (β = 0.01, SE = 0.1, p = .9). A linear regression predicting overall-sentiment index showed no main significant effect of frame (β = 0.17, SE = 0.16, p = .26), vividness (β = −0.23, SE = 0.14, p = .097), or interaction between frame and vividness (β = 0.30, SE = 0.2, p = .13).

Like the relative-sentiment index, the overall-sentiment index for the Study 3 data was unrelated to the PANAS. A LASSO regression regularized all parameters to zero: none of the 19 emotions predicted the overall-sentiment index.

Taken together, these results suggest that combining the emotional valence of participant’s arguments in a relative manner captures choice-relevant information, but combining valence in an absolute manner does not. While both sentiment indices were unrelated to specific emotions measured by PANAS, we do not consider this an issue. The PANAS measures are more temporally removed from choice than the relative-sentiment index; the PANAS measures emotions experienced but the relative-sentiment index captures which option the emotional valence was about.

4.2.1  Analyses of Both Relative-Sentiment Index and Structure of Thoughts

Given that relative-sentiment index and Structure of Thoughts are both calculated on all of participants thoughts, we cannot make causal claims about which came first with our data. However, exploratory path analyses allow us to examine whether relative-sentiment-index had an effect on choice that was distinct from the effect of frame and structure of thoughts. That is, when included in the same mediation model, does both the Structure of Thoughts and relative-sentiment index mediate choice?

We fit the path analysis using the lavaan R package with 5000 bootstrapped samples (Rosseel, 2012). Again, we used a probit model for regressions, which had choice as the dependent variable. Given that our vivid stimuli did not directly alter the content or structure of participant’s arguments, we used the combined Study 1 and 3 data but did not include the effect of stimuli vividness.⁷

Figure 4 displays the path model (Table A3 has all parameter estimates). Consistent with the results of Studies 1 and 3, the path from frame to Structure of Thoughts, the path from Structure of Thoughts to choice, and the direct path from frame to choice were all significant. Further, there is no direct effect of relative-sentiment index on choice (β = 0.02, SE = 0.02, p = .219). When included in a multiple mediation with the Structure of Thoughts, relative-sentiment index does not mediate the effect of frame on choice.

---

Figure 4: Path model for multiple mediation analysis. Paths where p > .05 are gray, paths where p < .05 are black. *** p < .001, ** p < .01, * p < .05.

---

4.2.2  Reversing Argument Order Does Not Affect Relative-Sentiment Index

While the path model cannot test whether the emotional valence of arguments changes the structure of thoughts or vice versa, we reanalyze the Study 2 data for a more data driven test of this hypothesis. Specifically, if manipulated argument order also changed the relative-sentiment index, this would be evidence that altering the structure of participants arguments also changes the emotional content.

To examine this hypothesis, we ran an exploratory linear regression on the Study 2 data, which predicted the relative-sentiment index by frame, thought order, and their interaction.⁸ There was significant main effect of frame on relative-sentiment index (β = 0.23, SE = 0.042, p < .0001). However, there was no significant effect of thought order (β = 0.06, SE = 0.042, p = .171) or interaction between thought order and frame (β = −0.063, SE = 0.042, p = .162). The significant effect of frame on relative-sentiment index suggests that frames still affected the emotional valence of participants’ arguments, even when thought order is imposed. (This result holds for the combined Studies 1-3 data Appendix C — Figure A4). Also, a separate regression showed that the relative-sentiment index still related to the Balance of Thoughts (β = 0.19, SE = 0.043, p < .0001). That is, even when initial arguments were exogenously manipulated, the emotional valence of thoughts still related to the structure of arguments. Taken together, these results provide suggestive evidence that the structure of arguments does not exert a strong influence on the arguments emotional valence.

5  Discussion

Across four studies, we documented how risky choice frames affect the structure and content of participants arguments for and against options and how these measures relate to choices. Replicating prior work, gain-frame participants choose the certain option more often than loss-frame participants. Consistent with Query Theory, gain-frame participants listed arguments in favor of the certain option earlier and more often than loss-frame participants; this difference in argument structure mediated the effect of frame on choice. Consistent with a causal role of early arguments, when participants were instructed to list their thoughts in an unnatural order in Studies 2 and 4 (i.e., in the opposite order than the one identified to occur naturally/without instruction), the effect of frame on choice was attenuated. Comparing Studies 1 and 2 to Studies 3 and 4 suggests that our vivid stimuli did not appreciably change the structure or content of participants’ arguments nor their choices. More importantly, the PANAS measures of emotions were not influenced by frame or the manipulated argument order.

However, exploratory analyses showed that the sentiment of participants’ thoughts during option evaluation were influenced by frame. When included in a mediation analysis with the structure of thoughts, the sentiment of participant’s thoughts did not predict choice. Other analyses showed that exogenously manipulating argument structure towards an option did not change the expressed emotional valence. These results have multiple theoretical implications.

5.1  Theoretical Implications

Our results suggest that risky choice frames shift the structure of participants arguments. This result is consistent with Query Theory but not other theories.

While Query Theory naturally accounts for manipulated thought order attenuating framing effects, Fuzzy Trace Theory does not appear to be able to do so. Fuzzy Trace Theory suggests that people encode the gist of the options and compare them to one another — e.g., “some people are saved” is better than “some people are saved, or no people are saved”. To account for the results of Studies 2 and 4, Fuzzy Trace Theory would need to assume that gist encoding depends on the order in which the two options are considered.

Other results fit will with Query Theory: Prior evidence suggests that time pressure increases risky choice framing effects (Guo, Trueblood & Diederich, 2017). If people are serially evaluating arguments for and against options, then the options with initial arguments in their favor — those which are initially attractive — will be more likely to be chosen when there is time pressure, as time pressure may suspend evidence collection before arguments for the initial leader are exhausted and never switch to consideration of the alternative choice alternative. Based on our reading, Fuzzy Trace Theory would not predict these results without an additional assumption: peoples’ mental representations become more verbatim — they encode exact numeric information — the longer people deliberate and thus are less likely to show framing effects.

While Query Theory and Fuzzy Trace theory explanations are distinct, they are not mutually exclusive nor incompatible. Query Theory is based on attention to arguments and memory; Fuzzy Trace Theory is based on mental representation. The mental representations posited by Fuzzy Trace theory may be at a lower level of cognition than the type aloud protocol could measure.⁹ That is, mental representations consistent with Fuzzy Trace Theory may direct attention to arguments in favor of certain options. However, Fuzzy Trace processes are not sufficient to account for our results.

Our work also speaks to emotional accounts of the Unusual Disease problem. Specifically, we failed to replicate prior work showing that emotions, as measured by the PANAS, related to choice framing effects (Druckman & McDermott, 2008). However, our methodology differed. We measured how frames changed emotions, but Druckman and McDermott measured emotions after participants completed the Unusual Disease Problem. We performed an additional analysis looking at only the post choice emotions on the Study 3 data: the LASSO regression regularized all emotions and all interactions between emotions and frames to 0. Also, the subset of emotions used was different: prior work included enthusiasm and distress. While we did not include emotions related to enthusiasm, we did include emotions similar to distress: nervous, depressed, and afraid. Further, their analysis included multiple covariates — risk aversion tendency, gender, student status, expertise, and interactions between these covariates and frame — but ours did not. The inconsistencies between Druckman and McDermott’s results and ours may be due to differences in measurement and analyses. Future work should explore why these results were inconsistent.

Our exploratory sentiment analyses suggest that risky choice frames affect the valence of arguments expressed by the participants; related work explains framing effects by the emotional valence inherent in the choice options — e.g., the Explicated Valence Account (Tombu & Mandel, 2015). In further exploratory analyses, we investigated if the relative-sentiment index applied to choice options — that is, not the relative-sentiment of arguments, but the relative-sentiment of the choice options — can explain choices. When the description of the risky option is changed the relative-sentiment index makes predictions consistent with Fuzzy Trace Theory and the Explicated Valence Account. Moreover, as seen in Appendix C, when the certain option is made slightly risky the relative-sentiment index of the options correctly predicts choices but Fuzzy Trace Theory and the Explicated Valence Account do not (Tombu & Mandel, 2015).

Other, closely related work, measures the relationship between the content of arguments to their structure (Zhao, Richie & Bhatia, 2020). Instead of using sentiment analysis and relative-sentiment index to describe the contents of thoughts, they used semantic space models and computational memory models. Consistent with our account, they find that the structure of sequential arguments is best fit by a computational model closely related to Query Theory.

Our work has multiple limitations. We did not manipulate option order which may have introduced a very small main effect increase in choices of Program A, the first option considered if reading order were the only determinant of processing order, but this is the same main effect under both frames and gets canceled out when we compare choices under the two frames. Consistent with this explanation, other work found that “the effect of option order on choice…appears to be very weak” (Mandel & Kapler, 2018, p. 10). Nonetheless, future research should examine if option order relates to affect.

Also, by manipulating thought order in Studies 2 and 4 we may have, inadvertently, signaled to participants that they should choose the initially considered option. This demand effect, however, does not account for results from other, closely related, Query Theory studies. Specifically, arguments in favor of the option found to be the early leader were more accessible, as measured by decision times, than arguments in favor of the other option (Weber, et al., 2007). Other work has shown that participants who do not want to follow the externally imposed thought order do not do so (Hardisty, Johnson, Weber, 2010). Given these results, we are not concerned about exogenously imposed argument orders causing demand effects.

Relatedly, listing arguments in an unnatural order may make the arguments disfluent — e.g., subjectively difficult to process (Alter & Oppenheimer, 2009). This disfluency may lead to less articulated preferences, making choices regress towards a 50-50 split between options. While disfluency is a plausible explanation for the reversal study, it does not explain the mediation results or the enhanced accessibility of frame consistent arguments (Weber et al., 2007). We argue that Query theory is an attractive account of risky choice framing effects but, like any complicated psychological phenomenon, they likely arise for multiple reasons (McGuire, 1997).

Our measure — relative-sentiment index — captures emotional valence only, missing out on many dimensions of emotions. relative-sentiment index’s relationship to frames and choices, highlights the tradeoff between precisely measuring a specific emotion separated from choice and measuring only an aspect of an emotions — e.g., the valence — in close proximity to the decision making process. Most prior studies have measured specific emotions, likely leading to theoretical accounts of emotions and decision making that focus on the overall feelings of the decision-maker (Lerner et al., 2015). We argue that work should acknowledge that measuring specific emotions is often decoupled from actual choice processes and highlight the limitations imposed by many measures of emotions.

Beyond generating insight into how the structure and content of participant’s arguments relate to risky choice framing effects, our results also give insight into other framing effects: the endowment effect, default effects in intertemporal choice, and tax versus offset framing. To our knowledge, ours is the first study to measure participants’ relative emotional valence about the options in close temporal proximity to choice. Our results suggest that other framing effects may be driven, in part, by the relative emotional valence about options attracting initial arguments. For example, conservatives consider “tax” to be a dirty word, so their arguments about an expensive plane ticket with a carbon tax may have been more negatively valence than a plane ticket with a carbon offset (Hardisty et al., 2010).

5.2  Conclusion

Taken together, our results suggest that how risky outcomes are framed affects both the structure and content of peoples’ internal deliberation. This result has numerous real-world and managerial implications. For instance, in a negotiations context, it suggests that how a choice between agreeing to an offered contract (a certain option) vs. going out on strike (a risky option) is framed (as options in the gain domain or options in the loss domain) may lead to negotiators assigning different emotional valences to the options and altering the structure of their internal deliberations. Our results suggest that debiasing of risky choice framing effects requires additional understanding how frames affect the content and structure of internal arguments.

References

Appendix A: Studies 1 and 3 Alternate Mediation Analyses

We tested the sensitivity of our mediation analyses by using a different linking function: the logit. Specifically, we used the methodology outlined in Imai, Keele, and Yamamoto (2010), which makes less restrictive assumptions than other mediation models. In sum, the conclusions of our mediational analyses are not affected by model specification.

Study 1

We tested if the Structure of Thought index mediated the effect of frame on choice. Because our treatment is binary (gain versus loss frame), outcome is binary (choice of the certain option or the risky option), and mediator is continuous (Structure of Thoughts), the assumptions of standard linear structural equation model approach using the product of coefficients do not hold (Imai, Keele, & Yamamoto, 2010). Instead, we estimated the average causal mediation effect (ACME), Average Direct Effect (ADE) and the Total Effect, which are conceptually similar to the indirect effect, the direct effect (c’) and the total effect (c), respectively, in traditional mediation (Baron & Kenny, 1986).

We used mediation package in R (Tingley, Yamamoto, and Hirose 2014; R Core Team 2014) with 1000 bootstrapped samples. The total effect of frame was positive (.16) with a bootstrapped 95% CI which excludes zero [.02, .31], p < .05. The ACME of the Structure of Thoughts was positive (0.09) with a bootstrapped 95% CI which excludes zero [.02, .16], p = .02. Controlling for the Structure of Thoughts, the direct effect of frame was no longer significant (.07) with a bootstrapped 95% CI which includes zero [−.06, .2], p = .29.

Study 3

Once again, the Structure of Thought index completely mediated the effect of frame on choice, using the same analysis described for Study 1. The total effect of frame was 0.28 with a bootstrapped 95% CI which excludes zero [.16, .40], p < .001. The ACME of the Structure of Thought index was positive 0.15 with a bootstrapped 95% CI which excludes zero [.08, .22], p < .001. When controlling for the effect of the Structure of Thought index, the direct effect of frame was positive 0.13 with a bootstrapped 95% CI which did not include 0 [0.02, .25], p < .05. The proportion mediated, the proportion of the total variance accounted by the indirect path, was .53 with a bootstrapped 95% CI which excludes zero. [0.32, 0.88] (Ditlevsen et al. 2005; MacKinnon, Lockwood, and Brown 2007; Freedman and Graubard 1992).

Appendix B: Study 3 and 4 PANAS results

Study 3

Specific Emotions

Table A1 presents the coefficients from the multilevel model predicting the change in participants specific emotions as measured by PANAS items. Each emotion shifted in a sensible direction: e.g., after answering the Unusual Disease Problem, participants felt more afraid than they had before, but less amused. However, there is no effect of frame or any interaction between emotion and frame.

---

Table A1: Study 3, difference in the PANAS predicted by type of emotion, frame, and their interaction. Note we ran the multilevel model without an intercept to ease coefficient interpretation.

	Post Test minus Pre-Test
Predictors	Estimates	CI				p
Afraid	2.62	2.17	to	3.08		<0.001
Amused	-2.01	-2.47	to	-1.55		<0.001
Angry	1.82	1.37	to	2.28		<0.001
Bored	-1.23	-1.68	to	-0.77		<0.001
Cheerful	-3.25	-3.71	to	-2.79		<0.001
Depressed	1.97	1.51	to	2.42		<0.001
Disgusted	1.63	1.18	to	2.09		<0.001
Fearful	2.71	2.25	to	3.17		<0.001
Furious	1.42	0.96	to	1.87		<0.001
Gloomy	2.11	1.65	to	2.57		<0.001
Happy	-3.56	-4.02	to	-3.10		<0.001
Indifferent	-1.17	-1.62	to	-0.71		<0.001
Mad	1.84	1.38	to	2.30		<0.001
Nauseated	1.03	0.57	to	1.48		<0.001
Nervous	2.02	1.56	to	2.47		<0.001
Neutral	-2.27	-2.72	to	-1.81		<0.001
Repulsed	1.22	0.76	to	1.67		<0.001
Sad	3.18	2.73	to	3.64		<0.001
Unemotional	-0.99	-1.45	to	-0.53		<0.001
Loss frame	0.17	-0.48	to	0.81		0.61
Angry:Loss	-0.22	-1.05	to	0.60		0.594
Bored:Loss	-0.59	-1.41	to	0.24		0.163
Cheerful:Loss	-0.59	-1.41	to	0.24		0.164
Depressed:Loss	0.15	-0.68	to	0.97		0.729
Disgusted:Loss	0.45	-0.37	to	1.28		0.281
Fearful:Loss	0.16	-0.66	to	0.99		0.696
Furious:Loss	-0.1	-0.92	to	0.73		0.817
Gloomy:Loss	-0.01	-0.84	to	0.81		0.977
Happy:Loss	-0.4	-1.23	to	0.42		0.339
Indifferent:Loss	-0.48	-1.31	to	0.35		0.254
Mad:Loss	-0.32	-1.15	to	0.50		0.442
Nauseated:Loss	0.2	-0.63	to	1.02		0.644
Nervous:Loss	-0.02	-0.85	to	0.81		0.963
Neutral:Loss	-0.49	-1.32	to	0.34		0.247
Repulsed:Loss	0.18	-0.65	to	1.00		0.675
Sad:Loss	-0.09	-0.91	to	0.74		0.836
Unemotional:Loss	0.02	-0.81	to	0.85		0.959

---

---

Table A2: Study 4, difference in the PANAS predicted by type of emotion, frame, and their interaction. Note we ran the multilevel model without an intercept to ease coefficient interpretation.

	Post Test minus Pre-Test
Predictors	Estimates	CI				p
Afraid	1.89	1.60	to	2.17		<0.001
Amused	-1.76	-2.05	to	-1.48		<0.001
Angry	1.02	0.74	to	1.30		<0.001
Bored	-1.29	-1.57	to	-1.01		<0.001
Cheerful	-3.03	-3.31	to	-2.75		<0.001
Depressed	1.34	1.06	to	1.63		<0.001
Disgusted	1.3	1.01	to	1.58		<0.001
Fearful	1.89	1.60	to	2.17		<0.001
Furious	0.96	0.68	to	1.24		<0.001
Gloomy	1.43	1.15	to	1.71		<0.001
Happy	-3.33	-3.61	to	-3.05		<0.001
Indifferent	-1.23	-1.51	to	-0.95		<0.001
Mad	0.95	0.67	to	1.23		<0.001
Nauseated	0.73	0.45	to	1.01		<0.001
Nervous	1.39	1.11	to	1.67		<0.001
Neutral	-1.95	-2.23	to	-1.67		<0.001
Repulsed	0.99	0.71	to	1.27		<0.001
Sad	2.11	1.83	to	2.39		<0.001
Unemotional	-0.75	-1.03	to	-0.47		<0.001
Loss frame	0.31	-0.09	to	0.71		0.127
Amused:Loss	-0.42	-0.94	to	0.10		0.111
Angry:Loss	0.14	-0.37	to	0.66		0.584
Bored:Loss	-0.61	-1.13	to	-0.09		0.021
Cheerful:Loss	-0.7	-1.22	to	-0.18		0.008
Depressed:Loss	0.39	-0.13	to	0.90		0.142
Disgusted:Loss	0.34	-0.18	to	0.86		0.195
Fearful:Loss	0.21	-0.31	to	0.72		0.434
Furious:Loss	0.08	-0.43	to	0.60		0.749
Gloomy:Loss	0.38	-0.14	to	0.90		0.148
Happy:Loss	-0.75	-1.27	to	-0.24		0.004
Indifferent:Loss	-0.26	-0.77	to	0.26		0.332
Mad:Loss	0.08	-0.44	to	0.59		0.769
Nauseated:Loss	0.14	-0.38	to	0.65		0.608
Nervous:Loss	-0.18	-0.70	to	0.34		0.499
Neutral:Loss	-0.4	-0.91	to	0.12		0.13
Repulsed:Loss	0.15	-0.37	to	0.67		0.567
Sad:Loss	0.36	-0.16	to	0.88		0.17
Unemotional:Loss	-0.45	-0.97	to	0.07		0.089

---

Emotions to choice

To ask whether specific emotions, as measured by the PANAS, were related to choice we ran a LASSO regression. First, we z-scored all differences in emotions. We then ran a 10-fold cross-validation analysis with deviance as our cross-validation criterion using the glmnet package in R (Hastie, Tibshirani, & Wainwright, 2015).¹⁰ Figure A1 shows the how cross-validation deviance evolves as the penalty parameter, λ , increases. The minimum deviance occurs when all parameters in the regression are regularized to 0. Therefore, changes in emotions as measured by the PANAS do not predict choice.

---

Figure A1: Study 3 emotions LASSO regression. Binomial deviance as a function of the penalty parameter λ . The top of the graph displays the number of non-zero model parameters for a given λ . The vertical dotted line indicates the minimum binomial deviance.

---

Study 4

Specific Emotions

Table A2 presents the multilevel model predicting the difference in emotion by the type of emotion, frame, and the interaction between frame and type of emotion. As with Study 3, the change in each type of emotion was in a sensible direction. Unlike Study 3, however, there were interactions between frame and the type of emotion. Specifically, loss-frame participants had a larger negative shift in their reported Boredom, Cheerfulness, and Happiness than gain-frame participants.

Predicting Choice

We used the same LASSO modeling framework as in Study 3. Figure A2 shows the how cross-validation deviance evolves as the penalty parameter, λ , increases. While the minimum deviance occurs when all but 4 parameters in the regression are regularized to 0, when using a λ value within 1 standard error of the minimum deviance — a common rule for regularized regressions (Hastie et al., 2009) — all parameters in the regression are regularized to 0. Therefore, we conclude that changes in specific emotions as measured by the PANAS do not predict choice.

---

Figure A2: Study 4 emotions LASSO regression. Binomial deviance as a function of the penalty parameter λ . The number of parameters in the model is on the top of the graph. The dotted lines indicate the λ value which yields the minimum deviance (left) and the λ value for the one-standard-error rule (right).

---

Appendix C: Additional Sentiment Analyses

Combined relative-sentiment index and PANAS analyses

Choice predicted by PANAS and relative-sentiment index

Beyond being distinct from other emotions, the relative-sentiment index also is unique in predicting choice. Once again, we performed a LASSO regression, which included all 19 emotions and the relative-sentiment index. The regression was specified as before. As seen in Figure A3, using the one-standard-error rule, only the relative-sentiment index remained non-zero. That is, the only measure of emotion which related to choice was the relative-sentiment index.

---

Figure A3: Study 3 emotions and relative-sentiment index LASSO regression. Binomial deviance as a function of the penalty parameter λ . The number of parameters in the model is on the top of the graph. The dotted lines indicate the λ value which yields the minimum deviance (left) and the λ value for the one-standard-error rule (right).

---

Vividness and Frame Regression models

We ran a regression to test if our vivid stimuli related to relative-sentiment index. In the regression model with an interaction between frame and vividness, gain-frame participants had a higher relative-sentiment index (M = 0.19) than loss-frame participants (M = −0.18) (β = 0.32, 95% CI [0.03, 0.62], t(407) = 2.14, p = .033). The vividness of stimuli was not related to relative-sentiment index (β = 0.06, 95% CI [-0.21, 0.33], t(407) = 0.43, p = .669). Further, there was no interaction between frame and vividness (β = 0.07, 95% CI [-0.31, 0.46], t(407) = 0.37, p = .709).

The model predicting relative-sentiment index with frame, vividness, and their interaction had a higher (worse) BIC (1180.05) and AIC (1159.95) than the model without the interaction BIC (1174.17); AIC (1158.10).

Combined Studies 1, 2, and 3 Sentiment analyses

To determine if the effect of frame on sentiment depended on thought order type — natural, unnatural, or unimposed (Studies 1 and 3) — we combined the data from Studies 1-3. Using an ANOVA which predicted the relative-sentiment index, there was a significant main effect of frame (F(1, 939) = 39.75, p < .001). The effect of thought order was marginally significant (F(2, 939) = 2.77, p = .063). We performed simple effects to determine if either imposed thought order differed from the unimposed thought order. Within the gain frame there was no difference between the unimposed sentiment and the imposed natural order (β = −0.14, SE = 0.11, p = .43) or between the unimposed order and the imposed unnatural thought order (β = −0.13, SE = 0.11, p = .45). Similarly, in the loss frame there was no difference between the unimposed sentiment and the imposed natural order (β = -0.02, SE = 0.11, p = .97) or between the unimposed order and the imposed unnatural thought order (β = −0.22, SE = 0.11, p = .08). The interaction between thought order and frame (F(2, 939) = 0.775, p = .46) was not significant (See Figure A4). That is, the imposed orders changed cognitive processes but not emotional processes.

---

Figure A4: Relative-sentiment index based on frame and thought order using combined Study 1, 2, and 3 data.

---

Path Analysis Table

Table A3 presents all estimated coefficients for the relative-sentiment index and Structure of Thoughts path analysis.

---

Table A3: Relative-sentiment index and Structure of Thoughts path model coefficients.

Dependent Variable	Predictor	Slope	SE	z	p
Choice	Frame	0.1	0	2.41	0.02
	Structure of Thoughts	0.28	0	13.1	<.001
	Relative-sentiment index	0.02	0	1.16	0.248
Structure of Thoughts	Frame	0.44	0.1	5.25	<.001
Relative Sentiment	Frame	0.37	0.1	3.86	<.001

---

Sentiment analysis of choice options.

Calculating the relative-sentiment index of the choice options makes predictions that Fuzzy Trace Theory and the Explicated Valence account do not. Specifically, when the certain option was made slightly risky — “there is a 1/3 probability that 400 people will be saved and a 2/3 probability that 100 people will be saved” in the gain frame — the effect of frame doubled (DeKay, Rubinchik & De Boeck, 2019). Fuzzy Trace theory predicts that the gists will be the same for the slightly risky option and the certain option — in the gain frame the gist of the certain option is “Some People are Saved” — meaning it predicts similarly sized framing effects. The Explicated Valence Account posits that the valence of choice options drives framing effects: whichever choice option has a higher valence is preferred (Tombu & Mandel, 2015). However, the Explicated Valence Account does not quantify valence. Therefore, the Explicated Valence of the slightly risky option is the same as the Explicated Valence as the traditional certain option, meaning it also predicts similarly sized framing effects. However, the relative-sentiment index of choice options correctly predicts a larger framing effect for the slightly risky option. In the gain frame, the slightly risky option has a higher sentiment than the regular certain option: “save” is written two times as opposed to one time, yielding an relative-sentiment index which favors the slightly risky option more than the certain option. For the loss frame, the slightly risky option is more negative than the certain option, yielding an relative-sentiment index which favors the risky option. Unlike other theories, the relative-sentiment index applied to choice options correctly predicts a larger framing effect for the slightly risky option.

The relative-sentiment index applied to choice options also accounts for findings traditionally explained via Fuzzy Trace Theory. When the risky choice is presented as only a non-zero-complement — “1/3 chance that 600 people are saved”/“2/3 chance that 600 die” — framing effects are smaller than when the both outcomes of the risky choice are explicitly stated. Fuzzy Trace Theory explains this result by the gists people encode: “1/3 chance that 600 people are saved” becomes “some people are saved” which is equivalent to the gist of the certain option “some people are saved”. Our account, however, explains non-zero compliment by the shift in the relative emotions of the options: the sentiment of “1/3 chance that 600 people are saved” is nearly the same as the sentiment of certain option, meaning initial arguments should not be driven to one option or the other. The predictions of relative-sentiment index applied to choice options should be tested in future research.

*

Carnegie Mellon University. Email: dwall@andrew.cmu.edu. https://orcid.org/0000-0001-6897-3077.

#

Columbia University. https://orcid.org/0000-0003-2420-7578.

$

Columbia University. https://orcid.org/0000-0001-7797-8347.

X

Princeton University. https://orcid.org/0000-0002-1678-3631.

Work supported by the US National Science Foundation through grant number 153104. We thank participants in the Preferences as Memories Lab meetings at Columbia University and Nathaniel Posner for comments on an earlier version of this manuscript. A portion of this paper was presented as “Whether You Are Framed Depends On How You Ask (Yourself): Query Theory and the Asian Disease Problem” at the 2014 Society for Judgment and Decision Making Conference in Long Beach, California.

Data and analysis files for all studies are at: https://osf.io/5frzx/

Copyright: © 2020. The authors license this article under the terms of the Creative Commons Attribution 3.0 License.

1

We use the more contemporary name instead of Asian Disease Problem.

2

Recent evidence suggests that this change in the number of people affected by the disease does not change the size of framing effects (Diederich, Wyszynski, & Ritov, 2018).

3

For this and all future logistic regressions β ’s are presented in log odds.

4

Both SMRD and Balance of Thoughts predicted choices independently.

5

All of the 19 planned contrasts assessing the difference between gain and loss frames for each specific emotions were not significant

6

This coding scheme is similar to prior accounts of risky choice framing based on the sentiment of choice options: 400 people dying has a negative sentiment but 1/3rd chance of nobody dying and a 2/3rd chance of everyone dying has a neutral sentiment, meaning the risky option has a higher sentiment than the certain option (Tombu & Mandel, 2015; Wallin et al., 2016).

7

The conclusions are identical regardless of whether the vividness factor was included or not.

8

Due to a minor coding error of the thought ratings, we could not use the Study 4 data to calculate the relative-sentiment index. Due to the large sample in Study 2 and the similar choice results in Studies 2 and 4, we do not find this concerning.

9

We also attempted to calculate the number of verbatim and gist representations based on participants thoughts. Of the 1643 thoughts, 211 explicitly mentioned a number related to the problem — 720, 480, 240, 1/3, 2/3, 33, 66 — suggesting a verbatim representation and 93 had words related to a gist encoding — some, few. However, these are poor proxies for verbatim and gist representations: multiple thoughts compared numbers across options and multiple thoughts which had explicit numbers also included words related to gists. Therefore, we did not pursue these analyses further.

10

Deviance is calculated as minus twice the log-likelihood on the cross-validated holdout sample (Hastie et al., 2009).

This document was translated from L^AT_EX by H^EV^EA.