Judgment and Decision Making, Vol. 11, No. 1, January 2016, pp. 114-120
Cognitive reflection as a predictor of susceptibility to behavioral anomalies
To study the effect of cognitive reflection on behavioral anomalies, we used the cognitive reflection test to measure cognitive reflection. The study was conducted on 395 Iranian university students and shows that subjects with lower cognitive reflection are significantly more likely to exhibit the conjunction fallacy, illusion of control, overconfidence, base rate fallacy, and conservatism. In addition, test scores are correlated with risk preferences. The results do not show any relationship between cognitive reflection and self-serving bias or status quo bias. We also find that gender is significantly related to illusion of control and self-serving bias.
Keywords: cognitive reflection, behavioral finance, decision making, behavioral anomalies, gender.
After Frederick (2005) introduced the Cognitive Reflection Test (CRT), a number of researchers studied the relationship between cognitive reflection and behavioral biases. Frederick (2005) shows that individuals with high CRT scores are generally more willing to be patient and willing to gamble in domain of gains; Benjamin, Brown and Shapiro (2006) and Slonim, Carlson and Bettinger (2007) found similar relationships. Oechssler, Roider and Schmitz (2009) found that subjects with low CRT scores exhibited the conjunction fallacy and conservatism more often than subjects with high CRT scores. They specifically posit that “…people with higher cognitive abilities might save more and receive higher expected returns; potentially leading them to play a more pronounced role in financial markets than subjects with lower cognitive abilities”. Hoppe and Kusterer (2011) find that CRT is predictive of susceptibility to the base rate fallacy and conservatism. They postulate that CRT is instrumental only when there is a correct solution for the bias through analytical calculations. Moreover, Hoppe and Kusterer report that the endowment effect is not correlated with CRT scores. Albaity, Rahman and Shahidul (2014) also replicate the relationship between cognitive reflection and behavioral biases with Malaysian subjects. The main finding of this literature is that cognitive reflection is predictive of susceptibility to behavioral anomalies, though cognitive reflection is not the only predictor.
This study investigates the relationship between cognitive reflection and behavioral anomalies in Iran. It includes the status quo bias, illusion of control, and self-serving bias. We use the CRT (Frederick, 2005) as a measure of cognitive reflection, as it is hypothesized to measure the System 2 decision-making abilities in dual process theory. This theory differentiates between two types of thinking: System 1 thinking or intuition, which corresponds to intuitive judgments that are fast and automatic, and System 2 thinking or reasoning, which applies to the class of problems which require reasoning and are subject to conscious judgments (Kahneman, 2003). The current study replicates the earlier studies and their main findings regarding the conjunction fallacy (Hoppe & Kusterer, 2011; Oechssler et al., 2009), overconfidence (Hoppe & Kusterer, 2011), the base rate fallacy (Hoppe & Kusterer, 2011), conservatism (Albaity et al., 2014; Hoppe & Kusterer, 2011; Oechssler et al., 2009), and risk preferences (Frederick, 2005; Oechssler et al., 2009). The results show that these biases are significantly related to CRT scores. Regarding overconfidence, subjects with higher CRT scores have a significantly more precise self-assessment (Hoppe & Kusterer, 2011), and subjects with lower test scores tend to be significantly overconfident. There is no evidence that self-serving bias and CRT scores are correlated. Further, CRT scores do not indicate any relationship with time preference and the status quo bias.
This paper introduces a new topic, which is the relationship between gender and behavioral anomalies. The results show that female subjects are significantly more likely to exhibit illusion of control and the self-serving bias in decision making contexts.
Proportion by score “High” “Low” Location Mean CRT Score 3 2 1 0 N Sharif University of Technology 2.42 60.4% 22.4% 16.4% 0.7% 134 Amirkabir University of Technology 2.17 46.1% 33.9% 11.3% 8.7% 115 Allameh Tabataba’i University 1.18 19.9% 18.5% 23.3% 38.4% 146 Overall 1.89 41.3% 24.3% 17.5% 17.0% 395
Question Reflective Impulsive Others Bat and balls 68.9% 26.1% 5.1% Widgets 61.5% 29.4% 9.1% Lily pads 59.5% 25.8% 14.7%
Four hundred and ten university students from three universities (Allameh Tabataba’i University, Amirkabir University of Technology, and Sharif University of Technology) participated in the research and completed a questionnaire (15 students did not complete the questionnaire, so the results are based on 395 subjects). I distributed a questionnaire with the CRT and tests of cognitive biases. (The Appendix provides an English translation.) The data were collected during February to June, 2015. The average age of the subjects was 22.4 (SD=3.62), and 63% of subjects were male (249 subjects). Additionally, 66.3% were bachelor’s students, 29.6% master’s students, and the remaining 4.1% were doctoral students. Subjects who correctly answered 2 or 3 CRT questions were paid 30 thousand rials, with 259 subjects receiving this payment.
The CRT (Frederick, 2005) is a three-item test with two types of answers for each question: a reflective answer and an impulsive answer. The CRT includes the following questions.
The average number of correct CRT responses (in this study) is 1.89, which ranks our subjects among MIT and Princeton subjects based on Frederick’s (2005) sample. In our sample, 41.3% of subjects answered all three questions correctly, 24.3% answered two questions correctly, 17.5% answered only one question correctly, and 17% did not answer any questions correctly (see Table 1).
Table 2 shows the distribution of answers separately for each question. Male subjects received higher average test scores (2.06) than female subjects (1.06; p < 0.001, MWU test), which is similar to the findings in Frederick (2005), Hoppe and Kusterer (2011), and Oechssler et al. (2009).
CRT group Category Item High Low p Risk % prefer 10k rials for sure to 75% chance of winning 20k rials 49.4% 72.8% <0.001 % prefer –1m rials for sure to 75% chance of –2m rials 88.4% 80.9% <0.049 Time % prefer to take the prize now rather with 5% interest next month 41.7% 50.0% n.s. Conjunction fallacy % think Linda is a bank teller and feminist than a bank teller 49.4% 69.1% <0.001 Self-serving bias Average score for three questions 4.84 4.96 n.s. Status quo bias % prefer to hold the current shares 32.8% 48.5% n.s. Illusion of control % prefer to do it myself 62.2% 83.8% <0.001 Overconfidence % overconfident 47.1% 66.2% <0.001 % correct self-assessed 34.4% 16.9% <0.001 % underconfident 18.5% 16.9% n.s. Base rate fallacy Avg. estimate, Dick is an engineer (correct prob.: 30%) 37.4% 40.0% <0.025 Conservatism % prefer to wait rather than sending sell order immediately 64.9% 84.6% <0.001
We divide subjects into two groups based on their CRT score, with the “high” group answering 2 or 3 questions correctly (259 subjects) and the “low” group answering 1 or 0 questions correctly (136 subjects).1 The high group is regarded as reflective decision makers and the low group as impulsive decision makers. The summary of the results by these two groups is shown in Table 3.
We asked subjects two questions related to risky choice. In question 1, which is in the domain of gains, they have to choose between a sure payment of x, and a lottery with a 75% probability of getting 2x and 25% of getting 0. In question 2, which is in the domain of losses, they have to choose between paying –x or a lottery with a 75% probability of paying –2x.
On question 1 (gains), 49.4% of the high group and 72.8% of the low group chose the sure payment (p<0.001, Fisher test). On question 2 (losses), 88.4% of the high group and 80.9% of low group chose the sure payment (p=0.049). The CRT score (0–3) correlated negatively with risk aversion in gains (p<.001, based on Kendall’s τ) and positively with risk aversion in losses (p=.019). In this study 44.8% of the high group and 58.8% of the low group chose the sure payment in both gain and loss domains, thus showing higher risk aversion as compared to Oechssler et al. (2009).
Subjects were asked about their preference in choosing to receive a prize now or after a month with 5% interest. We found that 58.3% of the high group and 50% of the low group chose to wait (were patient). This reflects a non-significant relationship between time preference and cognitive reflection in our study, which conflicts with previous studies’ (Benjamin et al., 2006; Frederick, 2005; Slonim et al., 2007) findings that subjects with high CRT scores are more patient. (The result was also non-significant using the CRT score.)
Subjects were asked the “Linda” question from Tversky and Kahneman (1983): “Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations.” Then, they were asked to indicate which of the following statements is more likely to be true: (1) Linda is a bank teller. (2) Linda is a bank teller and is active in the feminist movement”.
49.4% of the high group and 69.1% of the low group choose the biased answer (Linda is a bank teller and is active in the feminist movement), with this group difference being significant based on a Fisher exact test (p<.001; and the results was also significant at p<.001 for the Kendall correlation with the CRT score). Tversky and Kahneman (1983) found that 85% of the subjects showed the conjunction fallacy, while in our sample, 56.2% of all subjects showed it. However, the experiments are not comparable because Tversky and Kahneman provided more options, thus disguising the critical comparison.
The self-serving bias is the belief that individuals tend to ascribe success to their own abilities and efforts while simultaneously ascribing failure to external factors (Campbell & Sedikides, 1999). To study the self-serving bias, we use a reduced and modified version of the attributional style questionnaire from Dykema, Bergbower, Doctora and Peterson (1996). Responses are reported on a 10-point scale, where 0 means “the problem or situation has occurred solely by myself” and 10 means “the problem or situation has occurred solely by circumstances”. In each of the questions, subjects encounter a problem or a bad event, and they decide whether the cause of the problem is internal or external. We use the scale to study differences between the two CRT groups in attributing problems to internal or external factors.
The average score of the three questions is used as the measure of susceptibility to self-serving bias. The questions are as follows:
Although the average score of the high CRT group (4.84) is lower than the average score of low CRT group (4.92), the difference is not statistically significant.
In choosing among alternatives, individuals display a bias toward maintaining the status quo alternative (Samuelson & Zeckhauser, 1988). To study the bias, we use a slightly modified question based on Samuelson and Zeckhauser (1988). Subjects were presented with the following scenario: “You are a serious follower of the financial news. Recently you realized that you inherited 10,000 shares of an auto company from your uncle. You are deliberating whether to leave the stocks intact or to change it by investing in other securities. Your options are to:
Individuals who choose option 1 show the status quo bias. Of our subjects, 32.8% of the high group and 48.5% chose this option. However, neither this difference nor the correlation betwen CRT score and the bias is significant.
Langer (1975) stated that “…people behave as if they have control over [an] uncontrollable event even when the fact that success or failure depends on chance is salient”. This behavior is called the illusion of control. To study it, we use the following question: “In playing games that use dice, which condition would you prefer? (1) I throw the dice myself, (2) it makes no difference whether others throw the dice for me, or I do it myself”. Subjects in the high CRT group are considerably less susceptible to this bias. 62.2% of the high group and 83.8% of the low group show the bias (p<0.001, Fisher test; also p<.001 by Kendall correlation with CRT score). Overall, 69.6% of all subjects exhibited illusion of control on this question.
Overconfidence involves people’s inclination to overestimate their own abilities to successfully perform a particular task (Brenner, Koehler, Liberman & Tversky, 1996). To study overconfidence, we asked subjects five questions related to general knowledge. After answering the questions, subjects were asked to estimate the number of correct answers.2 For each CRT group, we divided the subjects into three subgroups: those who were overconfident, those who had correct self-assessments, and those who were underconfident. Overconfident (underconfident) refers to those whose estimated number of correct answers was more (less) than the actual number of correct answers, and correct self-assessments represent those whose reported number of correct answers was equal to the actual number of correct answers.
Male (N=249) Female (N=146) p Self-serving bias 4.72 5.15 <0.005 Illusion of control 65.5% 76.7% <0.023
As shown in Table 3, subjects in the low CRT group tend to overestimate the number of correct answers significantly more often than subjects in the high CRT group. That is, 66.2% of the low group and 47.1% of the high group exhibit overconfidence (p<0.001, Fisher test). Compared to the low CRT group (16.9%), a larger proportion of subjects in the high CRT group (34.4%) correctly estimate the number of correct answers (p < 0.001, Fisher test). Overconfidence is thus more prevalent in subjects with low cognitive reflection. However, this result is due entirely to the correlation between CRT score and actual number correct (r = 0.31, p<.001), plus the absence of any correlation between CRT score and confidence (judged number correct, r = 0.02). When confidence is regressed on both CRT.score and actual number correct, it is highly influence by number correct (coefficient 0.24, p<.001), but the negative effect of CRT.score is not significant (coefficient –0.06, p=0.207), as it could be if reflection actually reduced overconfidence. It is also worthy of note that, in many previous studies, people were more overconfident when facing acculturated test (like the one we used) than in sensory and perceptual tasks (where they often exhibited underconfidence). Thus, the overconfidence found in this study may depend on the use of an acculturated test.
The base rate fallacy refers to a tendency to ignore statistical information of prior probabilities in favor of the specific evidence concerning an individual case (Kahneman & Tversky, 1973). To examine the base rate fallacy, we use a modified question based on the paper by Tversky and Kahneman (1974). Subjects answered the following question: “There are 100 officers in a commercial building, including 70 lawyers and 30 engineers. We interviewed a person from the building. Dick is a 30-year-old man. He is married with no children, a man of high ability and motivation, he promises to be quite successful in his field. He is well liked by his colleagues. What is the probability that he is an engineer?”. Our correct answer is 30%. The high CRT group reported an average guess of 37.4% which is significantly lower than the 40% from the low CRT group (p<0.025, Mann-Whitney U test; also p=.002 when tested with a Kendall correlation with the CRT score). The high group was more likely to give the correct answer (68.0% vs. 51.5%) and less likely to give the most common incorrect answer, 50%, which ignores base rates completely (7.7% vs. 24.3%).
Conservatism refers to behavior where one is slow to change beliefs in the face of new evidence. For example, investors subject to conservatism might disregard the information in a company’s earnings (or some other public) announcement. This might happen because the investor believes that the information is true only temporarily, or it might happen because the individual clings to his/her prior estimates of earnings (Barberis, Shleifer & Vishny, 1998). Therefore, the investor might inadequately adjust his/her valuation of shares in response to the announcement. To study this bias, we used the following question: “You have a friend who works in a brokerage firm, and he is active in stock trades. Recently he suggested to you to buy a pharmaceutical company’s stock, and he said that, these days, he receives numerous buy order for that stock. Furthermore, you found other information which supports the company’s profitability. Based on these facts, you bought 20,000 shares yesterday. Today, you heard news about fire in one of the company’s warehouses, news that your friend confirmed. What is your reaction? a) I wait for later news to get assured about the events that happened in the company, and after that I will decide, b) I send sell order to my broker immediately.”
In the study, 64.9% of the high CRT group and 84.5% of the low CRT group exhibited conservatism. This difference is significant at p<0.001 (Fisher test; also significant at p<.001 by a Kendall correlation with CRT score). In all, 71.6% of subjects in the study exhibited conservatism, though the bias is less prevalent in the high group. It is worth noting that, in many previous studies, the measure of conservatism was the original measure that Edwards (1968) used which involved numerical probabilities. Conversely, in this study, the measure is more related to perception (or intuition).
The results show that two biases, illusion of control and the self-serving bias, are correlated with gender. As shown in Table 4, female subjects tend to attribute problems to external factors more often, as compared to male subjects. The average score for female subjects is 5.15, and the average score for male subjects is 4.72 (p<0.005, U test). Further, female subjects exhibit an illusion of control more often than male subjects (76.7% vs. 65.5%; p<0.023, Fisher test). We did not find any other gender differences in the biases that were measured.
This study revealed that cognitive reflection can predict the occurrence of behavioral anomalies, with these findings being similar to earlier studies. To measure cognitive reflection, we used the CRT (Frederick, 2005) to split subjects into two groups based on their CRT scores. We looked at the relationship between subjects’ CRT scores and risk and time preferences, finding that subjects with low CRT scores are risk averse in domain of gains. The results did not indicate any relationship between cognitive reflection and time preference. We also found that subjects with low CRT scores are more likely to exhibit the conjunction fallacy, illusion of control, overconfidence, base rate fallacy and conservatism. We found no relationship between cognitive reflection and either the self-serving bias or the status quo bias. Results additionally showed that the self-serving bias and illusion of control are exhibited more often among female subjects.
By replicating and extending previous results in Iran, the present study shows that they occur in other cultural contexts than where they were first found. Also noteworthy is that, in present-day Iran, the CRT is not so well known as it is in the English-speaking world, so most subjects were, we can assume, unfamiliar with it.
Cognitive reflection is one of many factors that can predict the incidence of behavioral anomalies. This study also used gender as a predictor, though it was related to only two anomalies that involved self-estimation. Study limitations include the fact that it was limited to university students, along with the fact that only seven biases were studied. Future studies could additionally consider the biases of financial investors.
Albaity, M., Rahman, M., & Shahidul, I. (2014). Cognitive reflection test and behavioral biases in Malaysia. Judgment and Decision Making, 9, 149–151.
Barberis, N., Shleifer, A., & Vishny, R. (1998). A model of investor sentiment. Journal of Financial Economics, 49(3), 307–343.
Benjamin, D. J., Brown, S. A., & Shapiro, J. M. (2006, May 5). Who is behavioral? Cognitive ability and anomalous preferences. Working paper, Harvard University.
Brenner, A. L., Koehler, J. D., Liberman, V., & Tversky, A. (1996). Overconfidence in probability and frequency judgment: A critical examination. Organizational Behavior and Human Decision Processes, 65(3), 212–219.
Campbell, W. K., & Sedikides, C. (1999). Self-threat magnifies the self-serving bias: A meta-analytic integration. Review of General Psychology, 3(1), 23–43.
Dykema, J., Bergbower, K., Doctora, J. D., & Peterson, C. (1996). An attributional style questionnaire for general use. Journal of Psychoeducational Assessment, 14, 100–108.
Edwards, W. (1968). Conservatism in human information processing. Formal Representation of Human Judgment, 17, 17–52.
Frederick, S. (2005). Cognitive reflection and decision making. Journal of Economic Perspectives, 19, 25–42.
Hoppe, E. I., & Kusterer, D. J. (2011). Behavioral biases and cognitive reflection. Economics Letters, 110(2), 97–100.
Kahneman, D. (2003). A perspective on judgment and choice: Mapping bounded rationality. American Psychologist, 58(9), 697.
Kahneman, D., & Tversky, A. (1973). On the psychology of prediction. Psychological Review, 80(4), 237.
Langer, E. J. (1975). The illusion of control. Journal of Personality and Social Psychology, 32, 311–328.
Oechssler, J., Roider, A., & Schmitz, P. W. (2009). Cognitive abilities and behavioral biases. Journal of Economic Behavior & Organization, 72(1), 147–152.
Samuelson, W., & Zeckhauser, R. (1988). Status quo bias in decision making. Journal of Risk and Uncertainty, 1, 7–59.
Slonim, R., Carlson, J., & Bettinger, E. (2007). Possession and discounting behavior. Economics Letters, 97(3), 215–221.
Tversky, A., & Kahneman, D. (1974). Judgement under uncertainty: Heuristics and biases. Science, 185(4157), 1124–1131.
Tversky, A., & Kahneman, D. (1983). Extensional versus intuitive reasoning: The conjunction fallacy in probability judgement. Psychological Review, 90(4), 293.
a) when you feel sick and tired most of the time.
b) when you have financial problems.
c) when you have a serious argument with someone in your family.
a) Hold the stocks, which have an expected return of 28 percent.
b) Invest in a pharmaceutical company stock which has 0.5 chance of doubling in value and 0.5 chance of declining 30% in value.
c) Invest in government bond that will yield a tax-free return of 24%.
Copyright: © 2016. The authors license this article under the terms of the Creative Commons Attribution 3.0 License.
This document was translated from LATEX by HEVEA.