Judgment and Decision Making, vol. 5, no. 3, June 2010, pp. 182-192
Correlations of cognitive reflection with judgments and choicesGuillermo Campitelli* 1,2, and Martín Labollita3 |
We investigated the role of individual differences in cognitive reflection in decision making. We measured the performance of 157 participants in the cognitive reflection test (Frederick, 2005) and a number of decision-making tasks. We examined the relation of cognitive reflection with performance in tasks that assess correspondence (as distinct from coherence), such as predicting the ratings of chess players. We found significant correlations between cognitive reflection and all the estimation measures in correspondence tasks. Our results suggest that cognitive reflection is a thinking disposition that includes more characteristics than originally proposed by Frederick (2005). We proposed that cognitive reflection is related to the concept of actively open-minded thinking (Baron, 1985, 2008). We concluded that cognitive reflection is a thinking disposition that interacts with knowledge, domain-specific heuristics and characteristics of the environment and that it may play an essential role in the adaptation of the decision maker to different environments and situations.
Keywords: cognitive reflection, open-minded thinking, heuristics, decision making.
Humans often violate the tenets of rationality as ruled by logic, statistics, expected utility theory or other normative models of rational decision making (Tversky & Kahneman, 1974, 1983; Simon, 1955). Instead of making decisions or judgments based on those models, humans tend to use heuristics that sometimes lead them to commit systematic errors or biases (Tversky & Kahneman, 1974, 1983). Nonetheless, not all humans are biased, some of them perform as if they use normative models of rationality (Stanovich & West, 1998, 1999, 2000). Numerous studies have shown that adherence to normative principles and cognitive abilities or thinking dispositions are positively related (Bruine de Bruin et al., 2007; Cokely & Kelley, 2009; Frederick, 2005; Parker & Fischhoff, 2005; Peters & Levin, 2008; Oechssler et al., 2009; Stanovich & West, 1998, 1999, 2000). This article aims to investigate the role of a thinking disposition — cognitive reflection — in decision making.
Frederick (2005) defined cognitive reflection as the “ability or disposition to resist reporting the response that first comes to mind” (pp. 35), and developed the cognitive reflection test (CRT) in order to measure this thinking disposition. Frederick also proposed a more general conception of cognitive reflection based on dual-system theories (e.g., Kahneman & Frederick, 2002; Sloman, 1996; Stanovich & West, 2000). Briefly, dual-system theories propose that humans have two cognitive systems: System 1’s processes are quick, effortless, intuitive and heuristic, and System 2’s processes are slow, effortful, reflective and rule-based. Based on a default-interventionist conception of System 2 (Evans, 2008), Frederick (2005) identified two characteristics of System 2 related to cognitive reflection: its capacity to monitor System 1’s outputs and its capacity to override System 1’s functioning.
Research has shown that individual differences in CRT play a role in individual’s decisions and judgments on tasks where their behavior could be compared to predictions of normative models (Cokely & Kelley, 2009; Frederick, 2005; Oechssler et al., 2009). Those studies found that CRT was positively related to choices predicted by expected utility theory (see von Neumann & Morgenstern, 1947) in risky choice tasks. Note, however, that Frederick (2005) also found that, in the domain of gains, high CRT scorers chose more risky options than low CRT scorers even when the expected value of the risk option was lower than that of the safe option. In intertemporal choice Frederick (2005) showed that CRT was positively related with choosing patient options (i.e., high CRT scorers had lower discount rates than that of low CRT scorers) and Oechssler et al. (2009) found and almost significant difference in the same direction. Furthermore, Oechssler et al. (2009) found that high CRT scorers were better than low CRT scorers at avoiding logical fallacies and also they were less overconfident.
This article is concerned with three topics related to cognitive reflection:
(1) Do individual differences in cognitive reflection, as measured by CRT, influence behavior in correspondence decision-making tasks (those in which accuracy is measured by an external criterion rather than by internal consistency)?
(2) Does the CRT measure only the “ability or disposition to resist reporting the response that first comes to mind”? Or does it also measure a broader disposition such as actively open-minded thinking (Baron, 1985, 2008)?
(3) Does general knowledge account for the influence of cognitive reflection on decision making? Or does cognitive reflection influence decision making independently from general knowledge?
Correspondence tasks are those that test the correspondence of participants’ estimations with facts of the world (see Dunwoody, 2009, and Hammond, 1996, for a detailed explanation of the correspondence, as well as the coherence, criteria of assessment in judgment and decision making). For example, participant’s estimations of number of inhabitants in cities are compared to actual number of inhabitants in cities. The theoretical interest of correspondence tasks is that good performance in them is typically explained by participants’ use of fast-and-frugal heuristics (Gigerenzer & Goldstein, 1996; Gigerenzer et al., 1999) and not by adherence to normative principles. As far as we are aware, the present study is the first one in relating cognitive reflection, as measured by CRT, with correspondence decision-making tasks. Previous research has shown that high CRT scorers engage in heuristic behavior in tasks that investigate participants’ judgments but not whether these judgments match facts of the world (Cokely et al., 2009). Cokely and Kelley (2009) found that high performance in CRT was related to more elaborative heuristic search. Based on these results we predicted that cognitive reflection and performance would be positively related in correspondence tasks in which the use of heuristics is helpful.
The second topic we investigated was whether CRT only measures a disposition to refrain from reporting an intuitive response or that, as suggested by Cokely and Kelley (2009), CRT also measures reflectiveness or thoroughness in decision making (Baron, 1985, 1990). If this is the case, cognitive reflection, as measured by CRT, would be a more general thinking disposition analogous to Baron’s (1985, 2008) concept of actively open-minded thinking. The third topic was concerned with alternative explanations of the role of CRT in decision-making tasks. Since CRT positively correlates with different types of general knowledge — e.g., numeracy (Cokely & Kelley, 2009; for the concept of numeracy see Lipkus et al., 2001; Peters & Levin, 2008) and academic achievement (Frederick, 2005) — it could be argued that the relation between CRT and decision making is explained by individual differences in general knowledge. We tested this by using tasks that did not require numeracy skills. Moreover, in one of these tasks participants could only use their general knowledge indirectly (see Procedure for a detailed explanation).
The main goal of this study was to investigate the role of cognitive reflection in correspondence decision-making tasks. Additionally, we aimed to replicate previous findings on the role of cognitive reflection in intertemporal choice and risky choice. In the correspondence tasks we used a familiar domain (cities) and an unfamiliar domain (chess).
One hundred and fifty seven volunteers from Buenos Aires metropolitan area participated in the study. The average age of the sample was 24.4 (S.D.: 5.4; range 16–43). One hundred and ten of the participants were female.
Participants filled in a 4-section booklet containing the following tasks: CRT, intertemporal choice and risky choice tasks, estimation of population of cities, and estimation of rating of chess players. The titles presented in the booklet for each of the sections were “Problems”, “Preferences”, “Cities” and “Chess players”. The “Problems” section contained the three CRT problems, as shown in Frederick (2005). The Preferences section contained some of the intertemporal choice and risky choice questions reported by Frederick (2005). The Cities section contained 4 tables with 5 columns (see Appendix 1 for one of the tables). The first column had the name of cities (worldwide cities in tables 1 and 3, and Argentine cities in tables 2 and 4). The rest of the columns had to be filled out by participants. In the second column participants had to indicate whether they were aware of the existence of each city. The third column was labeled “country” in tables 1 and 3, and “province” in tables 2 and 4. Participants had to write down the name of the country or that of the Argentine province where the city is located. In the fourth and fifth columns participants had to estimate the population size of each city. In the fourth column, participants had to sort each city onto one of eight possible categories of number of inhabitants. (See Appendix 1 for the range of number of inhabitants in each category.) In the fifth column participants had to estimate the exact number of inhabitants of the city. The number of cities in each table was 16, 11, 11, and 10. In Tables 3 and 4 we randomly chose cities from the top 100 most populated cities (top 100 worldwide cities and top 100 Argentine cities, respectively). In Tables 1 and 2 we used known cities such as Paris (high population) and Vatican City (low population) and less known cities such as Dhaka (high population) and Kwinana (low population).
The “Chess Players” section contained 4 tables with 5 columns. (See Appendix 2 for one of the tables.) The first column had the name of chess players (worldwide chess players in tables 1 and 3, and Argentine chess players in tables 2 and 4). The rest of the columns had to be filled out by participants. In the second column participants had to indicate whether they knew each chess player. The third column was labeled “country”; participants had to write down the nationality of the worldwide players (they did not have to fill out this column in tables 2 and 4). In the fourth and fifth columns participants had to estimate the international rating of each chess player. In the fourth column, participants had to sort each chess player onto one of ten possible categories of chess rating. (See Appendix 2 for the range of ratings within each category.) In order to give a reference to the participants, some categories contained labels such as “top 30 players of the world” (see Appendix 2). In the fifth column participants had to estimate the exact rating of each chess player. The number of chess players in each table was 17, 10, 10, and 10. In Tables 3 and 4 we randomly chose chess players from the top 100 players in the international rating (worldwide top 100 and Argentine top 100, respectively). In Tables 1 and 2 we did the same but we also added non-Slav players with high international rating and the two Argentine players with Slav surnames and high rating.
Some of the booklets were filled out in groups and some of them individually. The instructions were written on the booklet and researchers also briefed the participants. It was emphasized that all the questions had to be answered and all the cells on the tables had to be filled out. In very few cases, participants handed in the booklet with a few empty cells. In these cases, researchers asked the participants to fill out the complete booklet. As a consequence of this procedure, there were no missing values. There was no time limit to complete the task.
In the cities task participants could make their estimations using their geographical knowledge directly or indirectly. For example, their knowledge that Paris has a couple of million of inhabitants and that Vatican City is the smallest city on Earth, directly leads to an approximately accurate estimation of their population. On the other hand, vague knowledge that Chinese cities are highly populated may lead to high population estimations on unknown cities the name of which seem Chinese. We refer to this use of knowledge in estimations as “indirect” because this knowledge is not on the value of the criterion but on a somehow related cue. In the chess task, the only way of using knowledge is indirect. Knowledge that Russians (or more generally, Slavs) are good at chess, may lead to high rating estimations on players whose surname seem Slav. The correlation between surname (being Slav or non-Slav) and international rating was .49.
From the Problems section we obtained the number of CRT problems correctly solved (range 0 to 3). The questions in the Preferences section were grouped into four categories. In the first category — intertemporal choice (ITC) — we calculated the proportion of patient choices. In the other three categories — risky choices where the expected value of the gamble was higher than the value of the certain choice, in the domain of gains (HEVg); risky choices where the expected value of the gamble was lower than value of the certain choice, in the domain of gains (LEVg); risky choices where the expected value of the gamble was lower than the value of the certain option, in the domain of losses (LEVl) — we calculated the proportion of risky choices.
In the Cities and Chess Players sections, we measured participants’ general knowledge and participant’s ability to make estimations of facts in a relatively familiar (cities) and in a completely unfamiliar (chess) domain. In the Cities section, we used percentage of recognized cities (RecCi) and percentage of correct geographical localizations (countries in worldwide cities, and provinces in Argentine cities; LocCi) as measures of general knowledge. Percentage of correct estimations of the population category (Epop), and correlation between the log population size reported in all cities and actual log population size of those cities (rLgPop) were used as measures of estimation of facts in a relatively familiar domain. In the Chess Players section, we used percentage of recognized chess players (RecCh) as a measure of domain-specific knowledge of chess. Percentage of correct nationalities (NatCh), percentage of correct estimations of rating category (Erat) and correlation between log rating reported for all chess players and actual log rating of those players (rLgRat) were used as measures of ability to use knowledge indirectly to make estimations of facts in an unknown domain. We also analyzed the relation of the variables sex and age with cognitive reflection.
We computed bivariate correlations among all the variables. In this study we focused on the correlations of CRT with intertemporal choice, with risky choice and with performance in correspondence tasks. Based on Frederick’s (2005) results we predicted a positive correlation between CRT and ITC, HEVg and LEVg, and a negative correlation in CRT-LEVl. Based on Cokely et al.’s (2009) and in Cokely and Kelley’s (2009) results we predicted a positive correlation of CRT with the five estimation variables: NatCh, Epop, Erat, rLgPop, rLgRat.
Table 1: Bivariate correlations, means, and s.d.’s for cognitive reflection, intertemporal choice and risky choice tasks. Pearson’s r correlation coefficient above the diagonal and uncorrected p values below the diagonal. CRT= cognitive reflection test; ITC = intertemporal choice; HEVg, LEVg and LEVl = risky choices in items where the expected value of the risky option was higher than that of the certain option (domain of gains), lower than the certain option (domain of gains) and lower than the certain option (domain of losses), respectively.
The average number of correct CRT answers was 0.656 (S.D.= 0.9). 6% of the participants solved the three problems correctly, 11% of the participants solve 2 out of 3 problems correctly, 24% of the participants solve 1 problem correctly, and 59% of the participants solve no problems correctly. The average number of intuitive answers was 2.038 (S. D.= 0.9). This result corroborates Frederick’s (2005) observation that CRT problems prompt an intuitive (wrong) answer.
Frederick (2005) reported that the average CRT was significantly higher in men (Mean = 1.47) than in women (Mean = 1.03). In the present study the average CRT scores were 0.80 (S.D.= .92) for men, and .59 (S.D.= .91) for women. This sex difference in CRT is in the same direction of Frederick’s (2005) result; however, this difference was not significant: t (155) = 1.36, p = .175. Frederick also reported that women committed more intuitive errors than men. We did not find support to such claim. The average number of intuitive errors in men was 1.95 (S.D. = .98), and that of women was 2.07 (S.D. = .93); the comparison between means was not significant: t(155) = .688, p = .486. The correlation between age and cognitive reflection was almost significant (CRT-Age: r(155) = .152, p = .0574).
Table 2: Bivariate correlations, means, and s.d.’s for cognitive reflection and correspondence tasks. Pearson’s r correlation coefficient above the diagonal and uncorrected p values below the diagonal. RecCi = recognition of cities; RecCh = recognition of chess players; LocCi = geographical localization of cities; NatCh = nationality of chess players; Epop = estimation of population (category); Erat = estimation of rating of chess players (category); rLgPop = correlation between log actual and log reported population size; rLgRat = correlation between log actual and log reported rating of chess players.
Table 1 shows the bivariate correlations among intertemporal choice and risky choice tasks, and between these tasks and CRT. It also shows means and standard deviations of each variable. The sample of the present study could be considered impatient and safe. The majority of choices were impatient in the intertemporal choice items (75.6% impatient vs. 24.4% patient) and safe in the risky choice items (35.3% risky vs. 64.7% safe; 28.6% risky vs. 71.4% safe; and 33.1% risky vs. 66.9% safe in HEVg, LEVg and LEVl, respectively). As shown in Table 1, the prediction of positive correlation between ITC and CRT was not corroborated (r(155) = -.012; p = .8814). A possible explanation of this result is that the previous three years’ inflation rates in Argentina were 10.7%, 25.7%, 23%, for 2006, 2007 and 2008 (Bevacqua & Salvatore, 2009). Such high inflation rates might have caused a high discount rate in all participants.
In accordance with previous research (Frederick, 2005; Oechssler et al., 2009), cognitive reflection was positively related to risk taking when expected value of gains on the gamble was higher than the value of the safe option: CRT-HEVg (r(155) = .258; p=.0011). Frederick (2005) also found a tendency of high CRT scorers to gamble more than low CRT scorers in the domain of gains, even when gambles had a lower expected value than the value of the safe option. We also found a positive significant correlation (CRT-LEVg: r(155)= .134; p = .0355). Frederick (2005) also found that low CRT scorers gambled significantly more than high CRT scorers in gambles with lower expected values than the values of the safe options, indicating that high CRT scorers were more prone to accept losses. We found a tendency to accept losses in the whole sample. Therefore, we did not found a negative correlation between CRT and LEVl: CRT-LEVl (r(155) = .037; p = .6455).
Table 2 shows bivariate correlations among correspondence tasks and between these tasks and CRT. It also shows means and standard deviations of each variable. On average, participants reported recognizing 60.8% of the cities and 0.8% of the chess players. This result supports our classification of the cities task as familiar and the chess task as an unfamiliar. Cognitive reflection was significantly correlated to recognition of worldwide cities (CRT-RecCi, r(155) = .180, p = .0241), and, expectedly, was not related with recognition of chess players (CRT-RecCh, r(155) = .013; p = .8716). Participants geographically located cities correctly in 49.1% of the cases. There are two possible explanations of this 11.7% decrease from recognition to geographical localization of cities. First, some participants may be loose at reporting which cities they recognized. Second, some participants may have incomplete knowledge of cities (i.e., they know that a city with a particular name exists, but they do not know in which country it is located). The correlation between cognitive reflection and geographical localization of cities was highly significant (CRT-LocCi, r(155)= .446, p= .0000). This indicates that participants scoring high in CRT have a more complete geographical knowledge of the world than low CRT scorers. Interestingly, participants were able to correctly indicate the nationality of chess players in 17.6% of the cases, and the correlation of this variable with cognitive reflection was significant (CRT-NatCh, r(155)= .267, p= .0007). This result indicates that participants with higher cognitive reflection, instead of guessing, were able to use pieces of general knowledge (e.g., surnames ending “ov” tend to be Russian or Slav) to estimate unknown facts in unfamiliar domains (e.g., Malakhov is a Russian chess player).
Regarding the estimation measures on the domain of cities, participants accurately sorted cities onto categories of number of inhabitants in 20.3% of the cases. This was significantly better than chance (12.5% ): t(156) = 13.4, p = .0000. The average correlation between reported city size and actual city size was .386. This average correlation was significantly different than chance (average zero correlation): t(156) = 23.7, p = .0000. This result indicates that participants were able to estimate the population size of cities above chance, based on their geographical knowledge of the world. Performance in these tasks could be, at least in part, accounted for by either exact factual knowledge of city populations (e.g., the knowledge that Buenos Aires has roughly 3 million inhabitants) or by estimations based on fast and frugal heuristics such as the recognition heuristic (Goldstein & Gigerenzer, 2002). For example, participants that recognized Los Angeles and did not recognize Quezon City may have estimated that the former has more inhabitants than the latter. Cognitive reflection was related to performance in these estimation tasks (CRT-Epop: r(155)= .245, p= .0020; CRT-rLgPop: r(155)= .172, p= .0312). These results support the idea that cognitive reflection is related to possession (and use) of more general knowledge (in this case, geographical knowledge). It remains to be established whether cognitive reflection is also related to better use of fast and frugal heuristics. The following analysis of estimation performance in the domain of chess provides relevant information about this issue.
Participants correctly allocated players onto chess rating categories in 13.0% of the cases. This was significantly better than chance (10%): t(156) = 6.6, p = .0000. The average correlation between reported rating of chess players and actual rating was .213. This average correlation was significantly different than chance (average zero correlation): t(156) = 9.0, p = .0000. This result indicates that participants were able to estimate the rating of chess players above chance. This is remarkable because participants were not familiar with the domain of chess and they did not know the chess players. How could they perform above chance? Participants could only use heuristics based on knowledge on related cues. For example, knowledge that Russians are good at chess could be used to estimate higher ratings for players the surnames of whom seem Russian (e.g., Guseinov). These measures on estimation also correlated significantly with cognitive reflection (CRT-Erat: r(155) = .226, p = .0044; CRT-rLgRat: r(155) = .274, p = .0005). There is a possibility that the few participants that knew some of the chess players (there was a 0.8% average recognition of chess players) were also high CRT scorers and that this would account for the relation between CRT and estimation of chess ratings. However, as shown in Figure 1, the correlation between RecCh and CRT was not significant (r (155) = .013, p = .8716) and the partial correlations of CRT with Erat and with rLgrat controlling for RecCh were almost the same as the zero-order correlations (r (154) = .227, p = .004, and r (154) = .274, p = .001, respectively).
This result suggests that cognitive reflection is related to the use of heuristics based on knowledge on cues that are somehow related to the criterion.
The link between cognitive reflection, as measured by CRT, and decision making was established in risky choices and intertemporal choices (Cokely & Kelley, 2009; Frederick, 2005). Also, cognitive reflection was found to be related with the avoidance of biases (Oechssler et al., 2009). The first goal of this article was to investigate the role of cognitive reflection in correspondence decision-making tasks. Results supported our hypothesis that CRT would be positively correlated with 5 estimation measures in correspondence tasks. This finding extends the scope of the influence of cognitive reflection in decision-making tasks.
Frederick’s (2005) explanation of performance in CRT is based on a default-interventionist type of dual-system theory of cognitive processing. Frederick suggests that when participants try to solve CRT problems, their System 1 generates a quick, intuitive option. System 2’s duty is to monitor System 1’s output and to override its functioning if necessary. Most people tend to answer the intuitive option because of System 2’s “laziness”. However, there are some reflective individuals whose System 2 overrides System 1’s functioning, find out that the intuitive option is wrong, and carry out the appropriate computations to solve problems. A similar explanation could be offered for the role of cognitive reflection in risky choices. System 1 may quickly propose a biased option. An industrious System 2 overrides the heuristic process that generated this option and engages in rule-based thinking.
However, the observed good performance in the chess task requires an explanation that should include more factors. In accordance with Frederick’s (2005) explanation, in the chess task, System 1 may suggest guessing and System 2 may override System 1 and not give up too soon. However, in this case, the next step is not to engage in rule-based thinking. Instead, performance above chance is explained by initiation of heuristic processing (i. e., searching for superficial cues and use of heuristics based on general knowledge of the world). This analysis accords with Cokely and Kelley’s (2009) suggestion that CRT measures reflectiveness — i.e., careful, thorough, elaborative search and less impulsiveness (Baron, 1985, 2008). Think aloud protocols presented in Cokely and Kelley’s (2009) study suggest that even normative answers in risky choice tasks are not accounted for by participant’s calculations of expected value but by their elaborate heuristic search. Cokely et al. (2009) found that high CRT scorers used heuristics in situations where no normative models of rational decision making were useful, even when using these heuristics not necessarily increased their performance. Baron’s (1985, 2008) concept of actively open-minded thinking refers, among other things, to the disposition for searching for more possibilities before making an inference. Participants in the chess task on the present study, in Cokely et al. (2009) and in Cokely and Kelley (2009) seem to do just that. In the present study participants search for cues provided by the surnames of the chess players, in Cokely et al. (2009) they search for superficial cues (e.g., superficial features of a Euro note, physical aspect of an expert), and in Cokely and Kelley (2009) they search for reasons to choose options.
Frederick (2005) acknowledged that there are alternative explanations for the observed correlations between CRT and decision-making tasks. CRT may be measuring other skills rather than cognitive reflection; for example, numeracy or the skills needed for academic achievement. There are three facts that may lead to interpret the effect of CRT on decision making as modulated by numeracy: (a) finding the solution of CRT problems requires mathematical knowledge; (b) previous studies showed significant correlations of CRT with numeracy and with SAT math scores; (c) investigations of relation of CRT with decision making typically used tasks with mathematical components (e.g., risky choice, intertemporal choice). Although they had numbers, the correspondence tasks used in the present study did not require mathematical calculations. Therefore, at least in the present study, the influence of cognitive reflection in decision making was not mediated by numeracy.
Similar reasons could be put forward to interpret the effect of CRT on decision making as mediated by individual differences in general knowledge, as measured be academic achievement: (a) finding the solution of CRT problems requires general knowledge (e.g., semantic knowledge of the terms used in the CRT problems); (b) previous studies showed a significant correlation between CRT and academic achievement (i.e., ACT and SAT scores); (c) investigations of relation of CRT with decision making typically used tasks in which general knowledge directly helps in solving them. The role of general knowledge (i. e., geographical knowledge of nationality of types of surnames) in the chess task used in the present study could not have been used straightforwardly to perform above chance. Participants had to actively decide that this apparently irrelevant knowledge could be useful, to combine this knowledge with another piece of knowledge sometimes reported be media (i. e., Russians have been dominating chess for a long time) and to use this to make their estimations. Thus, there is a hint in the present study that, as suggested by Frederick (2005), individual differences in cognitive reflection independently influence performance in decision-making tasks.
Research has shown that CRT correlates significantly with academic achievement, intellectual abilities, need for cognition, numeracy, working memory, elaborative heuristic search, avoidance of biases, normative intertemporal choices and normative risky choices (see Cokely & Kelley, 2009; Frederick, 2005; Oechssler et al., 2009). It has also been shown that in risky choices, cognitive reflection is not always related to normative choices, but it is also related to a preference for risk (Frederick, 2005). The finding in the present study that CRT correlates significantly with performance in a task without mathematics and in which general knowledge has only an indirect influence, provides some support for Frederick’s (2005) claim that the correlations between CRT and decision-making tasks are not only explained by individual differences in general knowledge or cognitive abilities. Our results also suggest that cognitive reflection does not only refer to the ability to monitor and to override heuristic processes, but it also refers to the disposition to initiate heuristic processing in appropriate situations.
Based on these results we proposed that cognitive reflection, as measured by CRT, is related to Baron’s (2008) broader concept of actively open-minded thinking. Stanovich and West (1998, 1999) used a scale that measured actively open-minded thinking. They found that individual differences in this thinking disposition were related to individual differences in a number of decision-making tasks (but not in correspondence decision-making tasks). A possible avenue of further studies would be the investigation of the relation between CRT and the actively open-minded thinking scale used by Stanovich and West (1998, 1999), and whether this scale is also related to good performance in correspondence tasks. These studies would strengthen (or weaken) our proposal of linking cognitive reflection to actively open-minded thinking.
Our study suggests that cognitive reflection is a thinking disposition that interacts with knowledge, preferences, heuristics and characteristics of the environment. There are several proposals on the adaptability of the decision maker (e.g., Beach & Mitchell, 1978; Bröder, 2003; Gigerenzer & Selten, 2001; Payne et al, 1993). Based on the results of our study, we suggest that cognitive reflection plays an important role in the adaptation of individuals to different environments and situations.
Beach, L. R., & Mitchell, T. R. (1978). A contingency model for the selection of decision strategies. Academic of Management Review, 3, 439–449.
Baron, J. (1985). Rationality and intelligence. New York: Cambridge University Press.
Baron, J. (1990). Reflectiveness and rational thinking: Response to Duemler and Mayer (1988). Journal of Educational Psychology, 82, 391–392.
Baron, J. (2008). Thinking and deciding. New York: Cambridge University Press. (4th edition).
Bevacqua, G., & Salvatore, N. (2009). Buenos Aires City. Tasas de inflación anual. Retrieved from http://bsascity.googlepages.com/inflation on December 16th 2009.
Bröder, A. (2003) Decision making with the adaptive toolbox: Influence of environmental structure, personality, intelligence, and working memory load. Journal of Experimental Psychology: Learning, Memory and Cognition, 29, 611–625.
Bruine de Bruin, W., Parker, A. M., Fischhoff, B. (2007). Individual differences in adult decision-making competence. Journal of Personality and Social Psychology, 92, 938–956.
Cokely, E. T., & Kelley, C. M. (2009). Cognitive abilities and superior decision making under risk: A protocol analysis and process model evaluation. Judgment and Decision Making, 4, 20–33.
Cokely, E. T., Parpart, P., & Schooler, L. J. (2009). On the link between cognitive control and heuristic processes. In N.A. Taatgen & H. van Rijn (Eds.), Proceedings of the 31st Annual Conference of the Cognitive Science Society (pp.2926–2931). Austin, TX: Cognitive Science Society.
Dunwoody, P. T. (2009). Theories of truth as assessment criteria in judgment and decision making. Judgment and Decision Making, 4, 116–125.
Evans, J. St. B. T. (2008). Dual-processes accounts of reasoning. Annual Review of Psychology, 59, 255–278.
Frederick, S. (2005). Cognitive reflection and decision making. Journal of Economic Perspectives, 19, 25–42.
Gigerenzer, G., & Goldstein, D. G. (1996). Reasoning the fast and frugal way: Models of bounded rationality. Psychological Review, 103, 650–669.
Gigerenzer, G., & Selten, R. (Eds.) (2001). Bounded rationality: The adaptive toolbox. Cambridge, MA: MIT Press.
Gigerenzer, G., Todd, P., & ABC Research Group (1999). Simple heuristics that make us smart. New York: Oxford University Press.
Goldstein, D., & Gigerenzer, G. (2002) Models of ecological rationality: The recognition heuristic. Psychological Review, 109, 75–90.
Hammond, K. R. (1996). Human judgment and social policy: Irreducible uncertainty, inevitable error, unavailable injustice. New York: Oxford University Press.
Kahneman, D., & Frederick, S. (2002). Representativeness revisited: Attribute substitution in intuitive judgment. In T. Gilovich, D. Griffin & D. Kahneman (Eds.), Heuristics and biases: The psychology of intuitive judgment. New York: Cambridge University Press.
Lipkus, I. M., Samsa, G., & Rimer, B. K. (2001). General performance on a numeracy scale among highly educated samples. Medical Decision Making, 21, 37–44.
Oechssler, J., Roider, A., & Schmitz, P. W. (2009). Cognitive abilities and behavioural biases. Journal of Economic Behavior & Organization, 72, 147–152.
Parker, A. M., & Fischhoff, B. (2005). Decision-making competence: External validation through an individual-differences approach. Journal of Behavioral Decision Making, 18, 1–27.
Payne, J. W., Bettman, J. R., & Johnson, E. J. (1993). The adaptive decision maker. Cambridge, England: Cambridge University Press.
Peters, E., & Levin, I. P. (2008). Dissecting the risky-choice framing effect: Numeracy as an individual-difference factor in weighting risky and riskless options. Judgment and Decision Making, 3, 435–448.
Simon, H. A. (1955). A behavioral model of rational choice. Quarterly Journal of Economics, 59, 99–118.
Sloman, S. (1996). The empirical case for two systems of reasoning. Psychological Bulletin, 119, 3–22.
Stanovich, K. E., & West, R. F. (1998). Individual differences in rational thought. Journal of Experimental Psychology: General, 127, 161–188.
Stanovich, K. E., & West, R. F. (1999). Discrepancies between normative and descriptive models of decision making and the Understanding/Acceptance principle. Cognitive Psychology, 38, 349–385.
Stanovich, K. E., & West, R. F. (2000). Individual differences in reasoning: Implications for the rationality debate? Behavioral and Brain Sciences, 23, 645–726.
Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185, 1124–1131.
Tversky, A., & Kahneman, D. (1983). Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment. Psychological Review, 90, 293–315.
von Neumann, J., & Morgenstern, O. (1947). Theory of games and economic behavior (2nd ed.). Princeton: Princeton University Press.
Categories:
a) less than 50.000 inhabitants
b) between 50.000 and 100.000 inhab.
c) between 100.001 and 250.000 inhab.
d) between 250.001 and 500.000 inhab.
e) between 500.001 and 1.000.000 inhab.
f) between 1.000.001 and 2.500.000 inhab.
g) between 2.500.001 and 5.000.000 inhab.
h) more than 5.000.000 inhab.
Foreign cities.
| Do you know it? | Country | Number of inhabitants | ||
| YES or NO | (category) | (in numbers) | ||
| Inchon | ||||
| Wuhan | ||||
| Belo Horizonte | ||||
| Cairo | ||||
| Pyongyang | ||||
| Casablanca | ||||
| Santiago | ||||
| Singapore | ||||
| Los Angeles | ||||
| Quezon City | ||||
| Dhaka | ||||
| Percentage of correct answers in each column | % | % | -------------- | |
Note. The data obtained in the last row were not reported in this study.
Categories of Elo chess rating
| a) less than 2350 Elo points | |
| National Masters | b) 2350–2400 Elo points |
| c) 2401–2450 Elo points | |
| International Masters | d) 2451–2500 Elo points |
| e) 2501–2550 Elo points | |
| International Grand Masters | f) 2551–2600 Elo points |
| g) 2601–2650 Elo points | |
| Top 80 players in the world | h) 2651–2700 Elo points |
| Top 30 players in the world | i) 2701–2750 Elo points |
| Top 10 players in the world | j) more than 2750 Elo points |
Foreign chess players.
| Do you know him? | Country | ELO | ||
| YES or NO | (category) | (in numbers) | ||
| Bologan | ||||
| Guseinov | ||||
| Ivanchuk | ||||
| Milov | ||||
| Dominguez | ||||
| Bu | ||||
| Nisipeanu | ||||
| Naiditsch | ||||
| Mamedyarov | ||||
| Graf | ||||
| Percentage of correct answers in each column | % | % | -------------- | |
Note. The data obtained in the last row were not reported in this study.
This document was translated from LATEX by HEVEA.