Kenneth R. Hammond’s contributions to the study of judgment and decision making

Kenneth R. Hammond (1917–2015) made several major contributions to the science of human judgment and decision making. As a student of Egon Brunswik, he kept Brunswik’s legacy alive – advancing his theory of probabilistic functionalism and championing his method of representative design. Hammond pioneered the use of Brunswik’s lens model as a framework for studying how individuals use information from the task environment to make clinical judgments, which was the precursor to much ‘policy capturing’ and ‘judgment analysis’ research. Hammond introduced the lens model equation to the study of judgment processes, and used this to measure the utility of different forms of feedback in multiple-cue probability learning. He extended the scope of analysis to contexts in which individuals interact with one another – introducing the interpersonal learning and interpersonal conflict paradigms. Hammond developed social judgment theory which provided a comprehensive quantitative approach for describing and improving judgment processes. He proposed cognitive continuum theory which states that quasi-rationality is an important middle-ground between intuition and analysis and that cognitive performance is dictated by the match between task properties and mode of cognition. Throughout his career, Hammond moved easily from basic laboratory work to applied settings, where he resolved policy disputes, and in doing so, he pointed to the dichotomy between theories of correspondence and coherence. In this paper, we present Hammond’s legacy to a new generation of judgment and decision making scholars.

Keywords: lens model, policy capturing, cognitive feedback, interpersonal learning, interpersonal conflict, social judgment theory, cognitive continuum theory

1 Introduction

Kenneth R. Hammond [1917–2015] is perhaps one of the most important figures in the history of the psychology of human judgment and decision making. Although his influence has already been considerable, we anticipate that it will grow over time because his contributions build on a coherent and comprehensive theoretical framework, which lays a foundation for the development of a cumulative body of theory and research. It may be challenging, however, for future generations of scholars to access the breadth and depth of Hammond’s work, which spanned over seven decades, and is documented in over 100 articles, seven authored books, and five edited volumes. In this paper, our goal is to review and synthesize Hammond’s major contributions to the study of human judgment and decision making.

Hammond made at least eight major contributions that we will discuss. First, he virtually single-handedly kept Egon Brunswik’s legacy alive in psychology (Hammond, 1966; Hammond & Stewart, 2001b). Hammond built on Brunswik’s theory of probabilistic functionalism and his lens model, and he championed Brunswik’s method of representative design. Second, Hammond pioneered the use of Brunswik’s lens model as a framework for studying how individuals use information from the task environment to make clinical judgments (Hammond, 1955). This was the precursor to what is known today as ‘policy capturing’ and ‘judgment analysis’ research. Third, Hammond formulated the lens model equation which provided a quantitative tool for modeling and analyzing both the task environment side and human judgment side of the lens model for studying judgment processes (Hammond, Hursch & Todd, 1964; Hursch, Hammond & Hursch, 1964). Fourth, Hammond employed this equation when examining multiple-cue probability learning, making contributions regarding the effects of different forms of feedback on learning processes (Hammond & Summers, 1965; Todd & Hammond, 1965). Fifth, Hammond extended the application of the lens model to contexts in which individuals interacted with one another – introducing both the interpersonal learning paradigm (Hammond, Wilkins & Todd, 1966a) and the interpersonal conflict paradigm (Hammond, 1965; 1972; 1973; Hammond, Todd, Wilkins, & Mitchell, 1966b). Sixth, Hammond and his colleagues developed social judgment theory (SJT), which integrated prior work involving the lens model into a comprehensive quantitative approach for describing judgment processes and exploring means for improving cognitive performance (Hammond, Stewart, Brehmer & Steinmann, 1975). Seventh, grounded in, but going beyond Brunswikian concepts and SJT, Hammond developed cognitive continuum theory (Hammond, 1996a; 2000a; 2001a; Hammond, Hamm, Grassia & Pearson, 1987). He proposed that cognition moves along an intuitive-analytical continuum, with quasi-rationality as a middle-ground, and that cognitive tasks induce different modes of cognition, implying that cognitive performance is dictated by the match between properties of the task and mode of cognition. Finally, Hammond moved seemingly effortlessly from conducting basic work in the laboratory to solving applied policy-related problems (Hammond, 1996a; 2007; Hammond & Adelman, 1976). In doing so, he drew attention to the dichotomy between theories of correspondence, which focus on empirical reality, and theories of coherence, which focus on internal consistency, as well as the issues of separating facts from values and balancing type I and II errors in judgment under uncertainty. In the remainder of this paper, we discuss each of Hammond’s major contributions in more detail.

2 Keeping Egon Brunswik’s Legacy Alive

Hammond was key to keeping alive and advancing the legacy of the Austro-Hungarian psychologist Egon Brunswik who worked largely in the field of perception (1940, 1943, 1944, 1952, 1955c, 1955a, 1956; Tolman & Brunswik, 1935; for a review of Brunswik’s ideas, see Dhami, Hertwig & Hoffrage, 2004). Specifically, Hammond (1966) edited The Psychology of Egon Brunswik, which appeared just over a decade after Brunswik’s untimely death. This volume contained a eulogy by Edward Tolman; essays on Brunswik’s contributions to the science of psychology by such luminaries as Donald Campbell, Jane Loevinger, Fritz Heider, Lee Cronbach, and Roger Barker; four key papers by Brunswik himself; and a bibliography of Brunswik’s published works. Second, Hammond co-edited The Essential Brunswik (Hammond & Stewart, 2001b), which reprinted eighteen of Brunswik’s major papers accompanied by original commentaries, along with over two dozen papers discussing Brunswik’s theoretical and methodological contributions to psychology and describing applications of Brunswikian psychology to substantive problems. This volume also provided a complete annotated list of Brunswik’s publications. Third, Hammond established The Brunswik Society, with its associated annual international meeting and newsletter. This provided a setting for intellectual exchange among the next generation of neo-Brunswikian scholars.

In addition to his concrete efforts in archiving, preserving and curating Brunswik’s legacy, Hammond also built on Brunswik’s work. He was not deterred by the fact that Brunswik’s ideas were resisted and ignored by his contemporaries. Rather, Hammond vigorously championed Brunswik’s theory of probabilistic functionalism and lens model, as well as his method of representative design.

Brunswik (1943, 1952) believed that an organism functions to achieve a distal variable in its environment. This is done by using multiple, proximal cues that may be inter-correlated and are ultimately fallible. He argued that psychological processes are adapted to the probabilistic environments in which an organism functions. Brunswik (1957) saw the organism and environment as equal partners. Hammond and Stewart (2001a) suggest that the clearest statement of this fundamental thesis can be found in Brunswik’s (1957, p. 5) last paper, where he wrote:

The theory of probabilistic functionalism had analytic and methodological corollaries. Brunswik (1944, 1955c, 1956) argued that in order to understand how the organism has adapted to its environment, researchers should employ the method of representative design ideally in the form of randomly sampling stimuli from a defined population to which the researcher wants to generalize. This contrasts from the commonly used systematic design (i.e., manipulation and control of selected variables) in psychology. In addition, Brunswik (1943) noted that in order to measure an individual’s degree of achievement, the researcher needs to perform data analysis at the idiographic level. This departs from nomothetic analysis, which is most commonly used in the law finding tradition.

Hammond (1966, p. 16) characterized Brunswik’s theory of probabilistic functionalism as “an organic whole — a history, comprehensive theory, and a methodology.” He took the lead in extending probabilistic function so that it applied to judgment and decision making processes as well as to perception, which was Brunswik’s original and primary focus. In the last few years of his life, Brunswik introduced the concepts of quasirationality (Brunswik, 1952) and ratiomorphic models of perception and thinking (Brunswik, 1955b; 1956, pp. 89–93). These nascent efforts were also advanced by Hammond, and four decades after Brunswik’s death, Hammond (2001a) chronicled his own success in applying the lens model to clinical judgment, conducting research on learning under uncertainty, developing a mathematical representation of the lens model, expanding this model to studying interpersonal learning and conflict, and developing cognitive continuum theory.¹ We will discuss these contributions in more detail in subsequent sections of the paper. But, first, we will consider Hammond’s efforts to translate Brunswik’s method of representative design and his frustrated struggle to advocate Brunswik’s ideals for a scientific psychology.

As Hammond (2001b, 2001c) lamented, representative design is not well-understood in psychology (see also Dhami, 2011; Dhami et al., 2004; Gigerenzer, 2001). Hammond (2001b; p. 134) pointed out that it does not refer to:

Hammond was tenacious in his promotion of representative design. It was a key element of his first paper (Hammond, 1948), a key element of his last (posthumous) paper (Hammond & Lang, 2017) and many papers in between (e.g., Hammond, 1996b). In particular, he clarified and further developed Brunswik’s ideas regarding representative design by distinguishing between substantive situational sampling and formal situational sampling (Hammond, 1966, 1972).

Formal situational sampling refers to the formal properties of the task (i.e., number of cues, their values, distributions, inter-correlations, and ecological validities), irrespective of content. The formal properties define the universe of stimulus (or situation) populations. For instance, cue number, values, and distributions range from 0 to infinity, and the ecological validities of cues and their inter-correlations range from -1 to 1. Any population of situations lies within these boundaries. A researcher who uses formal situational sampling can sample various combinations of formal properties. Formal situational sampling permits the construction and presentation of stimuli that are formally representative of the natural stimulus population. In order to discover what the formal properties of the task are, it is often necessary to conduct a task analysis prior to the study. This may involve interviews with those who are familiar or experienced with the task, observations of individuals performing the task, document analyses of past case records and a review of the extant literature on the task.

An example of the use of formal situational sampling can be found in Hammond et al.’s (1987) study of 21 expert highway engineers judging highway safety. The distal criterion was the rate of accidents divided by the number of miles travelled, averaged over 7 years, for each of 40 highways. The highways were described in terms of 10 cues (e.g., lane width) that highway safety experts identified as essential for judging road safety. The values, inter-correlations, distributions, and ecological validities of the eight cues were deduced from highway department records, and properties of two cues were measured by the experimenters from visual inspection of videotapes of each highway. The researchers were primarily interested in examining how presentation of the task affects the mode of cognition used (e.g., analytical versus intuitive), and so the cue information for each highway was presented via filmstrips, bar graphs, and formulas.

Despite Hammond’s proposal to use formal situational sampling, as Brehmer (1979, p. 198) argued, there is “no easy road to success”. The number of all possible combinations of variables may be extremely large, and the researcher needs to know which combinations should be studied. The problem of defining a reference class or sampling frame remains. In a review, Dhami et al. (2004) revealed that when trying to capture judgment processes, few researchers had used representative design either in terms of sampling real stimuli from the environment (for some exceptions, see Kirlic, 2006) or in terms of constructing stimuli to be formally representative of the environment. Hammond (1996b, p. 245) himself confessed that he had not always been faithful to the method of representative design and explained that this was because, like Brunswik, if he had done so, “I would become isolated and ostracized.”²

3 Clinical Judgment, Policy Capturing and Judgment Analysis

Hammond (1955) was the first to use the lens model outside the study of perception. He pioneered the use of Brunswik’s (1952) lens model (see Figure 1) in the study of clinical judgment, employing it as a framework for studying how individuals use information from the task environment to make clinical judgments. This work was the precursor to ‘policy capturing’/‘judgment analysis’ research mentioned earlier.

In his 1955 landmark paper, Hammond shifted attention away from the practice of solely studying the accuracy of clinical judgment to also explaining how clinicians achieve accuracy.³ This was a fundamental shift in focus because “it is suggested that the clinician not be considered a reader of instruments, but an instrument to be understood in terms of a probability model” (Hammond, 1955, p. 262). Hammond argued that the clinician and the patient are two different, but interacting systems that should be considered as a whole, and so research should examine the relations between a clinician and his or her environment (i.e., patients). He further pointed out that the clinician’s judgment process is often ‘quasi-rational’ and difficult to communicate because it is a result of the process of vicarious functioning. The term quasi-rational was first used by Brunswik (1952) to describe cognitive processes that were intermediate between intuitive and analytical poles, and Hammond elaborated this concept in other work (Hammond, 1996a; Hammond et al., 1987), which we will return to later.

Brunswik (1956, 1957) used the concept of vicarious functioning to describe how, in an uncertain environment in which no proximal cue is a perfectly valid and reliable indicator of the distal state, organisms learn to rely on multiple cues of partial but imperfect validity and reliability. Vicarious functioning is essential in clinical judgment because clients may present a set of symptoms that may change over time or may present symptoms different from those presented by another client who is suffering the same problem. Hammond also argued that a clinician’s capacity for dealing with the intersubstitutability of cues, over encounters with a series of patients, should be studied using representative design.

Hammond’s points are illustrated in two studies conducted by his students (Herring, 1954; Todd, 1954, see also Hammond, 1955). In the first study, Todd (1954) asked 10 clinicians to judge the intelligence (as measured by an IQ test) of 78 patients using a Rorschach test. The IQ test score was the objective outcome criterion. Achievement was measured in terms of the correlation between the clinician’s judgments made over a set of clients and their IQ test scores. Emulating Brunswik’s (1940) approach, correlational statistics were used to capture the relationships between cues and judgments (cue utilization), cues and the environmental criterion (ecological validity), as well as inter-cue correlations.

Hammond noted that the clinicians’ performance improved when they had access to more information. The median correlation between judgments and IQ scores rose from 0.47 to 0.64 when clinicians had access to the verbal protocol data from the Rorschach. Then, using the four most valid cues as predictors, a multiple linear regression equation was computed for the environment (i.e., capturing the relations between the patients’ IQ scores and the four Rorschach cues) and separate equations were computed for each clinician (i.e., capturing the relations between each clinician’s judgments of IQ and the four Rorschach cues). Each model revealed the relative weights attached to the cues. The model of the environment was then compared to each clinician’s model.⁴ The match between the two models explained how the clinicians attained their level of achievement. The multiple R for the model of the environment was 0.479 and the median correlation between the clinicians’ judgments of IQ and the IQ test score was 0.470. The ecological validities of the four cues in the environment, each clinician’s utilization validities of the four cues, the inter-correlations among the four cues in the environment and as used by each clinician, were also elicited. There were variations among the clinicians in terms of the cues they used. “Certain clinicians were found to be using invalid cues, others neglected the valid ones” (Hammond, 1955, p. 261).

In his study of clinical judgment, Hammond also demonstrated that the multiple linear regression model could predict human (clinical) judgment. A model was developed for each clinician on 39 patients and cross-validated on a further 39 patients (i.e., it made predictions of each clinician’s judgments on a set of new cases). The median correlation between the models’ predictions and the clinicians’ judgments on the new cases was 0.85. Hammond (1955, p. 261) concluded that “evidently the multiple correlation model which predicts that the clinician combines the data from the Rorschach in a linear, additive fashion is a good one – it predicts quite successfully in comparison with most psychological efforts.”

In the second study, Herring (1954) asked clinicians to judge patients’ responses to surgical anesthesia using their psychological test results. No objective outcome criterion was available and so analysis of the environment side of the lens model and consequently achievement was not possible. Instead, Hammond demonstrated how the correspondence or agreement between the judgments of two clinicians (a medic and a psychologist) could be studied by examining the match between their regression models. Brunswik (1956, p. 30) had also noted that correlations could be used to measure “agreement among judges.”

In his first successful application of the regression model to clinical judgment, Hammond (1955, p. 261) was cautious in noting that the analyses merely demonstrated that it was possible to construct “some probability model” that captured the essential elements of the lens model. Over time, however, the similarity between the multiple linear regression model of the environment and of the clinician’s judgments led Hammond et al. (1964, p. 444) to conclude that “The clinicians’ inferential processes were nearly identical with the multiple-regression procedure both in function and in content.” When proposing further studies, they stated that “We are confident that … such studies will find small differences between the cognitive processes of the clinician, or any human subject, and the multiple-regression equation” (p. 452). Later, however, Hammond (1996b, pp. 244–245) confessed that: “a […] sin of commission on my part was to overemphasise the role of the multiple regression (MR) technique as a model for organising information from multiple fallible indicators into a judgment. There is nothing within the framework of the Lens Model that demands that MR be the one and only model of that organising process.” Researchers have now successfully developed and tested non-statistical alternatives to the regression model within the lens model context (e.g., Dhami & Ayton, 2001; Dhami & Harries, 2001; Gigerenzer & Goldstein, 1996).

Hammond’s ground-breaking 1955 paper presaged two related, but distinguishable threads of work. In one thread, exemplified by the Todd study, the focus is on modelling both the human judgment and environment side of the lens model, as well as the interaction between them, consistent with the fundamental tenets of Brunswik’s probabilistic functionalism. In the other thread, reflected by the Herring study, the emphasis on the environmental system recedes (because it is unknowable, unavailable or not of interest), and the spotlight is on how individual judges weight and combine information to make judgments. This latter situation represents what has been called the single-system design (Cooksey, 1996; Hammond, 2001a) of the lens model (see Figure 2) and is associated with the policy-capturing/judgment-analysis approach (Brehmer & Brehmer, 1988; Stewart, 1988), which we discuss in more detail below.

In the single-systems design there is no outcome criterion, or at least none that is immediately available, so researchers simply describe an individual’s judgment policy. As Cooksey (1996) points out, the single-system design is not truly Brunswikian because when the environment is unknown or unmeasurable, it does not allow for an examination of the interrelationships between the judge’s cognitive system and the task system. Nonetheless, Hammond (2001a) noted that the single-system design is the most widely used version of the lens model. Dhami et al. (2004) also observed that much of the subsequent research within SJT, developed by Hammond and his colleagues, which we discuss later, uses this design.

The single-system design may be appropriate because an outcome criterion is unavailable, for a variety of valid reasons, as enumerated by Cooksey (1996). First, an outcome criterion may not be useful or appropriate because no definitive, correct answer is available or ascertainable. In order to overcome problems in obtaining an outcome criterion, some studies have used expert judgments to provide proxies for environmental criterion measures (e.g., Adelman & Mumpower, 1979; Hammond & Adelman, 1976; Mumpower & Adelman, 1980). Second, an outcome criterion may be difficult to obtain due to concerns with ethics or legality. Third, an outcome criterion may be unavailable during the study period. Fourth, studies using hypothetical cases or cases that represent future situations, by their very nature preclude the use of an outcome criterion. Finally, an outcome criterion may not be included because it is irrelevant to the research goal since researchers wish solely to study agreement.

As mentioned above, analyses of how judges weight and combine information to make judgments in the single-system design have come to be known to as policy capturing and judgment analysis both within and outside⁵ the Brunswikian and SJT traditions. A large body of research has emerged, and in Brehmer and Joyce’s (1988) edited volume called Human Judgment: The SJT View that summarizes advances since Hammond et al.’s (1975) introduction of SJT, there is both a chapter on judgment analysis (Stewart, 1988) and a chapter on policy capturing (Brehmer & Brehmer, 1988). The former chapter acknowledges that the topic of that chapter is sometimes also called ‘policy capturing’ and the latter chapter similarly acknowledges that the focus of the chapter is sometimes referred to as ‘judgment analysis’.⁶

Judgment analysis and policy capturing have been fully explicated by a number of scholars (Brehmer & Brehmer, 1988; Cooksey, 1996; Hammond et al., 1975; Stewart, 1988). A study by Dhami and Ayton (2001) provides an example of the primary features of this approach. They studied British judges’ bail decisions. Through a postal survey, 81 judges were presented with a set of 41 hypothetical cases. These were described in terms of nine cues (e.g., seriousness of offence, community ties). The cues were identified following a review of the literature on bail, legal guidelines and training, interviews with judges, and observations of bail hearings. Each cue itself varied (e.g., the strength of community ties cue was dichotomous and described as strong or weak). Other cues (e.g., length of adjournment) were held constant and provided background information to all the cases. The judges were first asked to make a decision on each case (i.e., bail unconditionally, bail with conditions, and remand in custody). They were then asked to rate how confident they were in their decision. After making their decisions, the judges were asked to rank the cues in order of the importance attached to them when making their decisions. Seven of the 41 cases were duplicates used to measure test-retest consistency in decisions, and seven were holdouts used for model validation. Finally, studies interested in achievement would also document the outcome (or criterion) for each case, although this was not possible for Dhami and Ayton (2001).

Once the judgment data has been collected, judgment policies are captured for each individual.⁷ Traditionally, policies are captured using multiple linear regression statistics (Cooksey, 1996; Hammond et al., 1975; Stewart, 1988). The criterion (dependent) variable is the judgment and the cues are the predictor (independent) variables. An individual’s judgments are regressed on the cues. This procedure yields a weighted linear model that can be cross-validated, and which describes an individual’s judgment policy in terms of statistically significant cues in the model, relative cue weights, the form of the function relating the cues to the judgments (e.g., linear), the rule used to integrate the cues into a judgment (i.e., additive), and an individual’s predictability as measured by the model (i.e., R₂). Inter-individual differences (or agreement) in decisions and policies may then be examined. Participants’ ‘insight’ into their decision making may also be examined by comparing their self-reported policies with their model. Finally, intra-individual inconsistency in making decisions may be studied by comparing the decisions made in the test-retest situations.

Although there are exceptions, as Brehmer and Brehmer (1988) point out, reviews of judgment analysis and policy capturing research have generally concluded that the studies have yielded consistent findings, irrespective of the number and type of decision makers sampled, and the nature and content of the judgment tasks involved (Brehmer, 1994; Brehmer & Brehmer, 1988; Camerer, 1981; Cooksey, 1996; Libby & Lewis, 1982; Hammond et al., 1975; Karelaia & Hogarth, 2008; Slovic & Lichtenstein, 1971).⁸ Studies have typically found that the multiple linear regression model is a descriptively valid model as it provides an adequate fit to individuals’ judgment data. Researchers have suggested that a high R₂ implies that judgments are the result of a linear additive process (Hammond et al., 1964. However, as we pointed out earlier, Hammond recognized the paramorphic nature of the regression model, and other researchers have since successfully captured human judgment policies using non-statistical and even non-compensatory models (e.g., Dhami & Ayton, 2001; Dhami & Harries, 2001, 2010). Gerd Gigerenzer and his colleagues initiated a program of research modeling the task environment, in which they demonstrate the descriptive and predictive utility of simple (non-statistical and sometimes non-compensatory) models, called ‘fast and frugal’ heuristics (e.g., Gigerenzer, Todd & the ABC Research Group, 1999; see also Gigerenzer, Hertwig & Pachur, 2011).

The regression model indicates the number of cues used by an individual (i.e., those that have statistically significant beta weights), the relative weights of the cues (i.e., beta weights) and the direction in which cues were used (i.e., the sign of the beta weight). It has typically been found that regression models contain on average three cues (Brehmer, 1994; Slovic & Lichtenstein, 1971). Most studies have reported that participants do not use all of the cues available. Studies have often found that a few cues are weighted more heavily than the others. These findings on cue use are compatible with the more recent research demonstrating the descriptive validity of non-compensatory models that imply judgments may be based on only one cue (e.g., Dhami & Ayton, 2001; Dhami & Harries, 2001; Gigerenzer & Goldstein, 1996).

Hammond et al. (1975) proposed that consistency should be measured in terms of the variance of judgments made in a test-retest situation. Studies using such measures of consistency have often correlated the two sets of decisions, although other indices of agreement may also be used (e.g., Gillis, Lipkin & Moran, 1981). It has generally been found that correlations are moderate for the majority of participants in a study. Note that inconsistency implies some degree of judgment inaccuracy.

Researchers have also compared the captured policies with individuals’ stated policies, as elicited by various direct report methods.⁹ The subjective cue weights may be compared with statistical weights derived from the regression model, the fit of models containing each set of weights may be compared, or the predictions made by the two sets of weights may be compared (Reilly & Doherty, 1992). According to some, such comparisons provide a measure of an individual’s insight into his or her judgment policy (e.g., Ullman & Doherty, 1984). Todd (1954; see also Hammond, 1955 and Summers, Taliaferro & Fletcher, 1970) had found that his participants were not able to accurately articulate the weights they attached to the cues. However, acknowledging that direct methods may provide an unreliable and invalid method for demonstrating self-insight because of the difficulties in introspection and articulating policies, Reilly and Doherty (1989, 1992) proposed an alternative method, i.e., policy recognition. Here, participants are asked to identify their own policy, defined in terms of cue weights, from a set of other policies. This method indicates a greater degree of insight, as participants were quite successful in recognizing their own policies.

4 The Lens Model Equation

Hammond and his colleagues introduced the lens model equation (Hammond et al., 1964; Hursch et al., 1964) to the study of judgment processes. This equation provided the requisite quantitative tool for modeling and analyzing both the environment side and the human judgment side of Brunswik’s lens model. The lens model equation is the analytic complement of the conceptual framework articulated in the lens model. The original formulation by Hursch et al. (1964) was simplified by Tucker (1964), whose version is shown in Equation 1 (see also Karelaia & Hogarth, 2008).

Here, r_a represents achievement, and is measured by the correlation between the judgments and the criterion.

R_e represents the predictability of the environment and thus the upper limit of achievement. It is measured by the linear multiple correlation between the cues and the criterion.

R_s represents an individual’s ability to utilize his or her knowledge of the task in a consistent manner, and is measured by the linear multiple correlation between the cues and the judgments.

G represents the match of the linear components of the two models, namely the model of the environment and of the individual. It is measured by the correlation between the linearly predictable variance in the environment and the linearly predictable variance in the individual’s judgments.

C represents the non-linear component of achievement, and is measured by the correlation between the residuals from the linear regressions of the environment and the individual.

Each of the major statistical terms of the lens model equation can be directly translated into the conceptual framework of Brunswik’s lens model itself. Achievement (r_a) is determined by an individual’s ability to detect and utilize both linear and non-linear patterns in the environment. Stewart (1988) notes that in most judgment tasks, the non-linear component is so small as to be negligible. If the non-linear component is large, it indicates that the multiple linear regression model is not capturing all of the consistent variation in judgment.

Task predictability (R_e) is measured in terms of the linear multiple correlation between the cues and the criterion. Perfect achievement is not possible when R_e is less than 1. It is not possible to be always correct in a world that is not perfectly consistent. Task predictability thus sets an upper mathematical bound on achievement; r_a ≤ R_e. Brehmer (1970, 1972, 1973b) demonstrated that not only does R_e set an upper bound on r_a, but that lower levels of task predictability tend to induce sub-optimal levels of achievement. This happens because less predictable tasks tend to elicit lower levels of cognitive control, R_s. Brehmer (1976) suggests that judges become less consistent as they modify their judgment policies in order to try to achieve higher levels of achievement, but due to task uncertainty they receive noisy feedback, making it difficult to assess whether changes in judgment policy have led to improved performance. As summarized by Stewart, Roebber, and Bosart (1997, p. 206), “Not only does task predictability place an upper bound on potential accuracy …, but a number of studies have found evidence that the reliability of judgment is lower for less predictable tasks …Judges respond to unpredictable tasks by behaving less predictably themselves.”

Cognitive control (R_s) refers to the relationship between an individual’s judgments and environmental cues. It is measured in terms of the linear multiple correlation between cues and judgments. Building on earlier work, Hammond and Summers (1972) proposed that performance in cognitive tasks involves two distinct processes: acquisition of knowledge and cognitive control over knowledge already acquired. Just as task predictability places an upper bound on achievement, so does cognitive control. Cognitive control sets an upper mathematical bound on achievement; r_a ≤ R_s. In subsequent work, Hammond and colleagues (Hammond et al., 1975) distinguished between cognitive control and consistency. Cognitive control refers to the similarity between an individual’s judgments and predictions of the best-fitting model of that individual. Consistency refers to the similarity between repeated judgments made on identical cases. Consistency places an upper bound on cognitive control.

Knowledge (G) is assessed in terms of the correlation between the best fitting model of the human system and the best fitting model of the environmental system. It represents the achievement that would be attained if both the model of the judge and the task were executed with perfect control. In general, G will be maximized when the judge’s cue utilizations (weights and function forms) parallel the ecological validities of the environmental cues, presuming that the judge uses the same organizing principle to combine the information into an overall judgment as is found in the environmental system. As Karelaia and Hogarth (2008) note, however, it is possible for judges to obtain high levels of G, even when the cue utilizations do not precisely match the ecological validities. This can happen when inter-cue correlations are high.

Finally, C is also a measure of knowledge. It represents the consistent non-linear variation shared between the individual and environment that is not captured by the regression models.

Cooksey (1996, p. 165, words in square brackets added) sums up the situation nicely:

In the first application of the lens model equation, Hammond et al. (1964) reanalyzed data from a study by Grebstein (1963), who compared naïve, semi-sophisticated and sophisticated clinicians’ predictions of 30 patients’ IQ test scores using 10 cues from patients’ Rorschach tests. Grebstein had concluded that performance did not increase with experience and that there was room for improvement. Hammond et al. (1964) used the lens model equation to determine the upper limit of achievement for this task and found that R_e was 0.79. They also demonstrated that the three groups of clinicians did not differ in terms of R_s, or C as all groups were highly linear. The three groups did, however, differ in terms of G because clinicians in the more sophisticated groups exhibited a greater match between the ecological validities of the cues and their utilization validities.

Once individuals’ judgment policies have been captured, they can be compared with the model of the task, in order to examine individual achievement. As mentioned, the upper limit of achievement is dependent upon that afforded by the task. Libby and Lewis (1982) concluded that studies in accounting (in particular prediction of business failure and prediction of security return) reported high levels of achievement. Overall, studies comparing judgments against an outcome criterion have demonstrated fairly high levels of achievement or judgmental accuracy as measured by the lens model equation; r_a averaged 0.56 among the 249 studies in a meta-analytic review by Karelaia and Hogarth (2008).

Since the first studies were conducted over 50 years ago, an extensive body of research has used or referred to the lens model equation. The precise number of studies is difficult to estimate accurately, but efforts to compile bibliographies (Holzworth, 1999) and to conduct meta-analyses (Karelaia & Hogarth, 2008) provide evidence that there are hundreds of such studies, at a minimum. Methodologically, Castellan (1973, 1992), Cooksey and Freebody (1985), Stewart (1976, 1988, 1990) and Stewart and Lusk (1994), among others, have made important modifications, suggested significant extensions, or examined the statistical and empirical behavior of parameters of the lens model equation.

By extending Brunswik’s lens model from the study of perception to judgment, and then formulating the lens model equation, Hammond opened up the possibility of new research approaches and agendas. These tools could be used to study how individuals learn to make judgments in probabilistic multiple-cue learning situations. We discuss this topic in the next session. Second, these tools offered the opportunity to study inter-individual (dis)agreement in judgment, interpersonal learning, and interpersonal conflict. This topic is discussed in the subsequent section.

5 Multiple-Cue Probability Learning

The lens model equation enabled Hammond to undertake a program of research on multiple-cue probability learning in which he and others made numerous contributions. Holzworth (1999) provides a good overview of these efforts:

In one of the earliest multiple-cue probability learning studies, Hammond and Summers (1965) gave three groups different amounts of information about the task in addition to different amounts/forms of outcome feedback (i.e., no information, information that the task contained linear and non-linear cue-criterion function forms, and information that in addition identified the linear and non-linear cues). They asked individuals to predict a criterion value from two cues, one linearly related to the criterion and one non-linearly related. All groups showed learning over five blocks of 20 trials and all groups learned to use the linear cue more efficiently than the non-linear cue. However, individuals in the group given the most information showed a greater degree of achievement and were more likely to learn to use the non-linear cue. The two groups who were provided information about the task were also better at learning the ecological validities of the cues.

Holzworth’s 1999 annotated bibliography identified 315 studies of single- and multiple-cue probability learning conducted up to that date. Several important empirical regularities have emerged from the literature. For instance, people can learn positive cue-criterion relations faster than negative ones; they can slowly learn to track changes in relative cue weights over time; they can learn to use cues faster than learning function forms; and they typically do not use cue redundancies effectively (Klayman, 1988; Slovic & Lichtenstein, 1971). Generally speaking, the findings from the multiple-cue probability learning paradigm indicate that individuals can learn about the formal properties of the task such as cue validities and linearity of cues and adapt to them, although learning tends to be better and faster when task predictability (R_e) is higher (Brehmer, 1973a; 1974). It has also been shown that people find it more difficult to learn non-linear cue-criterion relations than linear ones and they experience difficulty in applying knowledge about non-linear relations consistently (Deane, Hammond & Summers, 1972).

Hammond initiated a line of research within the multiple-cue probability learning paradigm that proved to be of particular consequence. This concerned the distinction between, and differential effects of, outcome versus cognitive feedback. It was clear from Hammond and Summers’ (1965) study that providing information about properties of the task in addition to traditional outcome feedback improved learning. Thus, Todd and Hammond (1965) developed the procedure of cognitive feedback (sometimes also known as lens model feedback).¹⁰ This involves providing information about the formal properties of the task, the individual’s judgment policy (i.e., utilization validities, R_s, consistency and cue-judgment function forms), and the match between properties of the environment and the individual’s judgment policy (i.e., achievement, G and C; Balzer, Doherty & O’Connor, 1989; Doherty & Balzer, 1988).

Todd and Hammond (1965) provided participants with feedback on their level of achievement, cue utilization validities and the ecological validities of the cues, for each of eight blocks of 25 trials. They found that cognitive feedback led to significantly greater achievement than did outcome feedback,¹¹ and concluded that cognitive feedback enables people to compare their understanding of the task and discover where they were using cues inappropriately. This study was followed by a series of additional ones which revealed that participants given only outcome feedback learned less rapidly than those given cognitive feedback (Deane et al., 1972; Hammond, 1971; Hammond & Boyle, 1971; Hammond, Summers & Deane, 1973).

Other research has found that providing both cognitive feedback in the context of outcome feedback may actually impair performance (Holzworth & Doherty, 1976); that learning is slow and difficult with outcome feedback alone (Brehmer, 1980; Klayman, 1988);¹² and that cognitive feedback is superior to no feedback at all (Doherty & Balzer, 1988). Unlike cognitive feedback, in stable environments, outcome feedback does not provide information useful for making future judgments.

The usefulness of cognitive feedback, however, may need to be buttressed by other information. Balzer et al. (1989) concluded that providing people with information about the characteristics of their own judgment performance but not about the appropriateness of that performance does little to improve performance compared to providing information about the characteristics of the task. More recently, Karelaia and Hogarth (2008) concluded that information about the task was more useful than feedback about the judgment policy alone, or simply outcome feedback. As Brehmer (1979) points out, in real world conditions feedback is not always available and people are not consciously trying to learn the task. In fact, analyzing feedback requires cognitive resources, which may be lacking in complex tasks (Harvey, 2011).

6 Interpersonal Learning and Interpersonal Conflict

Among Hammond’s most significant contributions was his extension of the lens model to social contexts in which individuals learn from or about one another, or are in judgment conflict with one another. Brunswik’s probabilistic functionalism is based on the double-system lens case, where the two focal systems are the individual and the environment. This asocial focus is unsurprising given that Brunswik was primarily concerned with perception. For Brunswik (1956, p. 35), the study of agreement was the only time when more than one individual was “brought into the picture.”

In an innovative and revolutionary development, Hammond moved the lens model into a social context, introducing both the interpersonal learning (IPL) paradigm (Hammond, 1972; Hammond et al., 1966a; see also Earle, 1973) and interpersonal conflict (IPC) paradigm (Hammond, 1965, 1973; Hammond et al., 1966b). The triple-system lens model for the IPL and IPC paradigms is shown in Figure 3. In Hammond’s (2001a, p. 471) own view, the triple-system case represented “perhaps the most important expansion of Brunswik’s lens model. It is possibly one that Brunswik did not anticipate.”

In both the IPL and IPC paradigms, the judges face a common task, where there is an environmental criterion with which multiple, proximal cues are imperfectly associated. Judges attempt to infer the value of the criterion on the basis of available cues. The IPL and IPC paradigms share much in common. The main differences lie in the specific elements of the triple-system case where attention is focused. In the IPL paradigm, the focus may be either on learning about or learning from the other person, whereas in the IPC paradigm, the focus is on reducing disagreement with the other person. We present these two paradigms below.

Interpersonal learning about the other person is focused on understanding the other person and developing the ability to predict how he or she will respond to the task. The ability to predict accurately the other person’s judgments requires an understanding of the weights, function forms, and organizing principle that the other person uses to combine information from cues into a judgment. Prediction accuracy is also constrained by the degree of cognitive control the other person exercises in making judgments. The other person’s R_s plays the same role as R_e does in the two-system case, constraining the upper limit of achievement.

In one of the original interpersonal learning studies, Hammond et al. (1966b) found that, on average, paired participants were able to predict one another’s responses quite well, as well as one another’s differential cue weights, linear (although not to the extent hypothesized), and non-linear cue use. Participants were also more likely to learn about the other person if that person was more reliable, and through interaction each pair’s policy similarity increased.

Some research on interpersonal learning about the other person has focused on the ability to predict another’s judgments when there is no common environmental criterion, for instance, in negotiation, when the relevant judgments are about the desirability of potential settlements (Balke, Hammond & Meyer, 1973; Miller, 1973; Mumpower, Sheffield, Darling & Milter, 2004).

Research on interpersonal learning from the other person focuses on understanding how a judge learns about a task by observing the judgments that another person makes. Clearly, interpersonal learning from the other person is not independent of interpersonal learning about the other person — to learn about the task from another person, one needs to learn how the other person interacts with the task. The other person must convey relevant information about the task for interpersonal learning to be better than individual-only task learning.

Hammond (1972) found that function forms affected cue utilization validities where, for instance, after interpersonal learning, individuals trained to use a non-linear cue could give up reliance on that cue and learn to use a linear cue faster than individuals trained to use a linear cue. In a set of three experiments, Earle (1973) reported that participants taught to rely solely on linear cues required interpersonal learning from participants using non-linear cues in order to switch to using non-linear ones, but not vice versa. Interpersonal learning was also necessary for learning negative linear or non-linear rules.

Research has also investigated the effects of task characteristics on interpersonal learning and found the effects to be similar to those found in the multiple-cue probability learning paradigm, with the exception that non-linear policies are easier to learn through interpersonal learning (Hammond et al., 1975). Cognitive feedback has also been shown to be useful in interpersonal learning tasks (Balke et al., 1973; Miller, 1973).

Judges may find themselves in false agreement, impeding interpersonal learning, when they reach similar judgments despite different polices because of high levels of inter-cue correlations (Mumpower & Hammond, 1974). On the other hand, lack of cognitive control means that parties may disagree in practice even though they agree in principle, referred to as false disagreement (Dhami & Olsson, 2008). These latter observations lead us to describe Hammond’s IPC paradigm.

Research in the IPC paradigm is motivated by Hammond’s observation (1965, 1973; Hammond et al., 1966b) that not all conflict and disagreement stem from differences in incentives or motivations. Cognitive conflict may arise due to differences in what people “believe to be the efficient, just, and moral ways to solve their problems” (Hammond, 1973, p. 189). As Hammond and Grassia (1985, p. 233) put it “People dispute many things besides differential gain, or who gets what. They also disagree about (a) the facts (what is, what was), (b) the future (what will be), (c) values (what ought to be), and (d) action (what to do).”

As the triple-system lens model in Figure 3 illustrates, there are four primary potential sources of cognitive conflict. Judges may disagree about the relative importance (i.e., relative weights) of the proximal cues; they may disagree about the relationship between values of the cues and values of the criterion (i.e., the appropriate function forms for cues); they may disagree about the appropriate rule for combining information from cues into a judgment (i.e., the organizing principle); and they may disagree because one or both judges are unable to make judgments in a consistent way (i.e., imperfect cognitive control).

In a standard interpersonal conflict experiment (Brehmer, 1976; Cooksey, 1996; Hammond, 1965; 1973), participants may be selected either because they already have conflicting judgment policies or they may be taught to develop conflicting policies. Unaware that they have different policies, participants are brought together and asked to co-operate on solving a set of problems where cues are probabilistically related to the criterion. On each trial, they consider the available information and make judgments about the criterion variable individually and then communicate these to one another. If they disagree they must discuss the problem until they reach an acceptable joint response. They are then asked to reconsider their original decisions, and these revisions remain private. Finally, they are presented with the correct solution. Conflict or agreement is therefore defined objectively as the actual differences in the judgments made by the two individuals.

Conflict may be due to systematic and non-systematic cognitive differences in the way people perform the task. Systematic differences refer to features of judgment policies such as the relative cue weights and non-systematic differences refer to the idea that people may exercise imperfect cognitive control in the application of their policies (Mumpower & Stewart, 1996). Research has shown that, over a series of trials, participants unlearned their conflicting policies and developed similar ones, however, conflict persisted because individuals simultaneously became more inconsistent in applying their revised policies. Such non-systematic differences accounted for more conflict than did systematic differences in judgment policies (Brehmer, 1976). The aforementioned findings have been replicated using different types of participants and task conditions (Hammond & Brehmer, 1973).

Brehmer’s (1976) research demonstrated how the degree and nature of conflict (i.e., whether it is due to systematic or non-systematic differences) is affected by task conditions. For example, he showed that policy consistency is lower in less predictable tasks leading to less agreement; that non-linear cues lead to lower consistency but do not affect policy similarity; that when the task contains linear cues and people only have to use one cue they are more consistent but their policy similarity is unaffected; that when the task contains both linear and non-linear cues and people only have to use one cue their policy similarity and consistency is higher; and that inter-cue correlations lead to less policy similarity.

Hammond and Brehmer (1973) applied the technique of cognitive feedback and developed a cognitive aid to conflict resolution called POLICY (originally called COGNOGRAPH). This was an interactive computer program that enabled people to express their policies, compare them, change them, and discover the effects of such changes on cognitive conflict. The emphasis is on teaching consistent new policies. It has been found that cognitive feedback helps to speed conflict reduction (Balke et al., 1973).

In sum, research in the IPC paradigm has demonstrated that agreement could be studied in the same way as achievement. Although the study of two cognitive systems has become popular, and despite the potential for theoretical and methodological advance of the concept of cognitive conflict, Dhami and Olsson’s (2008) review reveals that research on cognitive conflict using the lens model has declined sharply. It has been replaced by research on what is called ‘task conflict’ conducted by scholars interested in group conflict. Unfortunately, the latter has less theoretical precision and methodological rigor than Hammond’s IPC paradigm.

7 Social Judgment Theory

In 1975, after two decades of work building on Brunswik’s ideas and advancing them in the study of clinical judgment, multiple-cue probability learning, interpersonal learning and interpersonal conflict, Hammond and his colleagues (Hammond et al., 1975) synthesized the key elements of this work into a unified framework, which they called social judgment theory (SJT). SJT is not a theory providing testable hypotheses about the nature of human judgment, but a meta-theory that provides a framework to guide research in this endeavor.

The basic concepts of SJT are all foreshadowed by the two decades of work that preceded it, and so we shall not discuss them in detail here. Suffice it to say that in SJT equal status is accorded to both the human and environment, there are parallel concepts depicting each side of the lens, and the distinction between surface (proximal cues) and depth (the environmental criterion and judgment about the environmental criterion) is essential.

Social judgment theorists, as they have come to be known, study “life relevant” issues (Hammond et al., 1975, p. 276), and so much of SJT research is applied. There are four basic goals of SJT research: (a) to analyze judgment tasks and processes; (b) to analyze the structure of achievement and agreement; (c) to understand how humans learn to achieve and agree; and (d) to find methods for improving achievement and agreement (Brehmer & Joyce, 1988; Hammond et al., 1975). SJT research, thus, aims to describe behavior before prescribing changes to improve it. The model of the environment serves as a benchmark, indicating how judgment can be improved (Brehmer & Joyce, 1988; Hammond et al., 1964). Cognitive performance may be enhanced by cognitive feedback and cognitive (decision) aids (Hammond et al., 1975).

Several types of judgment situations are distinguished in SJT research. These are the double-systems design, single-system design, triple-systems design, the N-systems design and the hierarchical design (see also Hammond, 1972; Hammond et al., 1975). The first refers to Brunswik’s (1952) original lens model as shown in Figure 1, and involves an analysis of the interaction between an individual and a task. As mentioned earlier, this framework is used to study achievement and multiple-cue probability learning.

The other judgment situations represent modifications to the original lens model. As mentioned earlier, the single-system design refers simply to an individual’s judgment policy, with no reference to achievement in the task (see Figure 2). The triple-systems design involves one task and two individuals (see Figure 3). It is used to study interpersonal learning and interpersonal conflict. The N-systems design involves more than one individual and may or may not include an analysis of the task. Research on policy formation is conducted within this framework (e.g., Adelman, Stewart, & Hammond, 1975; Stewart & Gelberd, 1976). Finally, there are judgment situations in which the cues themselves may be judgments made at earlier stages of the judgment process either by the same or different judges. Hammond et al. (1975, p. 286) refer to such situations as “hierarchical judgment models”. Here, an outcome criterion is often unavailable and each stage is analyzed separately.¹³

In SJT research, judgment data is elicited over a series of trials and is analyzed at the level of the individual (Hammond et al., 1975). Hammond et al. (1975, p. 278) state that “the judgment data are analyzed in terms of multiple regression statistics.” Thus, correlational statistics and models such as multiple linear regression are used to describe and explain cognitive performance. However, as Brehmer (1979, p. 199) pointed out:

SJT is also committed to representative design defined in terms of formal situational sampling (Brehmer, 1979; Cooksey, 1996; Hammond et al., 1975; Hammond & Wascoe, 1980). Here, all of the relevant cues, cue values, cue distributions, inter-cue correlations and ecological validities of cues should be representative of those that exist in the natural version of the task. Hammond et al. (1975) recognized that under representative conditions, the presence of inter-cue correlations may make it difficult to ascertain the relative independent effects of each cue upon judgments. As an alternative to representative design, they recommended multi-method analyses where techniques such as predicting each cue from the others and successive omission of cues may be used.¹⁴ However, Dhami et al. (2004) have revealed that researchers typically only study single-system cases and do not use any form of representative design. Consequently, little is known about judgmental achievement for many tasks, and the lack of representative stimulus sampling threatens the validity of the findings reported.

Hammond et al. (1975, p. 304) stated that “the unique contribution of SJT has been to bring the theory, quantitative procedures, results of research, and technological innovations (externalization of judgment policies by means of interactive computer graphics) to bear on social policy formation outside the laboratory.” Over the past four decades since its launch, SJT has inspired hundreds of applied and basic studies of human judgment (for collections of such studies see Brehmer & Joyce, 1988; Dhami et al., 2004; Karelaia & Hogarth, 2008), as well as some more recent approaches to the psychology of human judgment and decision making (e.g., Gigerenzer et al., 1999; Juslin et al., 2000). One applied area of special note has been medical decision making (e.g., González-Vallejo, Sorum, Stewart, Chessare & Mumpower, 1998; Poses, Cebul, Wigton, et al., 1992; Tape, Kripal & Wigton, 1992; Way, Allen, Mumpower et al., 1998; Wigton, 1988; 1996). As Wigton (1996) observed, SJT is particularly well suited to the study of medical judgments because they characteristically involve decision making under uncertainty with inevitable error and an abundance of fallible cues. SJT has been useful in establishing variation among decision makers’ judgments and in their weighting of clinical information.

8 Cognitive Continuum Theory

Later in his career, still faithful to Brunswik’s ideas but going beyond them, Hammond developed cognitive continuum theory (CCT, Hammond, 1996a, 2000a, 2001a; Hammond et al., 1987). The key propositions of CCT include the premises that cognition moves on an intuitive-analytical continuum; that quasi-rationality represents a commonly used and important middle-ground on this continuum; that cognitive tasks induce different modes of cognition; and that the upper level of cognitive performance is dictated by the match between properties of the task and mode of cognition.

More specifically, CCT states that there are modes of cognition that lie in-between intuition and analysis (see also Dhami, Belton & Goodman-Delahunty, 2015; Dhami & Thomson, 2012). Intuition (often also referred to as System 1, experiential, heuristic, and associative thinking) is generally considered to be an unconscious, implicit, automatic, holistic, fast process, with great capacity, requiring little cognitive effort. By contrast, analysis (often also referred to as System 2, rational, and rule-based thinking) is generally characterized as a conscious, explicit, controlled, deliberative, slow process that has limited capacity and is cognitively demanding. The modes of cognition that lie in-between are quasirational.

As Hammond (2010, p. 331) points out, the term ‘quasi’ does not mean that quasirational modes of cognition are the result of “improper cognitive activity”. In addition, Hammond (1996, pp. 166–167, brackets added) takes pains to differentiate quasirationaity from Herbert Simon’s (1957) concept of bounded rationality, which he states “means that cognitive activity has neither the time nor resources to explore […] completely the “problem space” of the task. The problem space that is explored, however, is explored in a rational or analytical fashion.” For Hammond, quasirationality is distinct from rationality. It comprises different combinations of intuition and analysis, and so may sometimes lie closer to the intuitive end of the cognitive continuum and at other times closer to the analytic end.

Brunswik (1943, 1952) pointed to the adaptive nature of perception (and cognition).¹⁵ For Hammond (1996a, 2000b), modes of cognition are determined by properties of the task (and/or expertise with the task). Task properties include, for example, the amount of information, its degree of redundancy, format, and order of presentation, as well as the decision maker’s familiarity with the task, opportunity for feedback, and extent of time pressure. The cognitive mode induced will depend on the number, nature and degree of task properties present. A task comprising either intermediate levels of, or a combination of, those properties inducing pure intuition or pure analysis will induce quasirationality. Depending on task properties, quasirationality may imply a combination where there is greater use of intuition than analysis, or vice versa. Movement along the cognitive continuum is characterized as oscillatory or alternating, thus allowing different forms of compromise between intuition and analysis (i.e., different forms of quasirationality).

According to Hammond, cognitive tasks can be differentiated from one another with regard to their properties as well as the mode of cognition they induce (see also Dhami & Thomson, 2012). Studies support the idea that different task properties induce different modes of cognition (Dunwoody, Haarbauer, Mahan, Marino & Chu-Chun, 2000; Hamm, 1988; Hammond et al., 1987; see also Mahan, 1994). Success on a task inhibits movement along the cognitive continuum (or change in cognitive mode) while failure stimulates it. Hamm (1988) additionally demonstrated that cognitive mode can shift during a task.

Brunswik (1943, 1952, 1956) stressed the significance of the correspondence or fit between properties of the task and the mode of cognition. Hammond (1988) predicted that judgment performance is contingent on the degree of correspondence between task properties and mode of cognition. The key implication is that pure analysis may be neither necessary nor sufficient for ceiling-level performance. Evidence suggests that task characteristics are important in determining the upper bound of performance (e.g., Seifert & Hadida, 2013; Rusou, Zakay & Usher, 2013) and that achievement is greater when the cognitive mode matches that induced by the task (e.g., Dunwoody et al., 2000; Hammond et al., 1987).

Although there is a growing body of evidence on the nature and performance of intuitive versus analytic cognition (e.g., Dunwoody et al., 2000; Hammond et al., 1987; Mahan, 1994; Marewski & Mehlhorn, 2011), there is a distinct dearth of research on the operation and outcomes of quasirationality, even though, as some have argued, this is the most common mode of cognition in many consequential tasks (Dhami et al., 2015; Dhami & Thomson, 2012). In their efforts to identify the processes involved in intuitive versus analytic cognition, Glöckner and his colleagues have found some similarities and differences between these two modes of cognition (e.g., Glöckner & Betsch, 2008a; 2008b; 2012; Jekel, Glöckner, Fiedler & Bröder, 2012; Horstmann, Ahlgrimm, & Glöckner, 2009). Their findings suggest that intuition and analysis may operate in an integrative fashion and thus potentially shed light on different forms of quasirationality. For instance, quasirationality may allow individuals to use a lot of information quickly. Other work measuring the performance of different modes of cognition, for example by Blattberg and Hoch (1990), has demonstrated that a quasirational model which combined managerial intuition (expertise) and statistical analysis repeatedly outperformed purely intuitive and statistical models in five forecasting tasks (see also Ganzach, Kluger & Klayman, 2000).

Hammond (2007) returned to the themes of analysis and intuition and the cognitive continuum in his last book entitled Beyond Rationality: The Search for Wisdom in a Troubled Time, published at age 92. At the heart of his argument is the proposition that the key to wisdom lies in being able to match modes of cognition to properties of the task. In brief, for most judgment and decision making tasks, quasirationality is what is required for wisdom. According to Hammond (2007, p. 237), “…the tactics that most of us use most of the time are neither fully intuitive nor fully analytical: they are a compromise that contains some of each; how much of each depends on the nature of the task and on the knowledge the person making the judgment brings to the task.”

9 Solving Applied Problems

Finally, it is notable that Hammond was as comfortable conducting basic research in the laboratory as he was in solving applied problems, especially in the policy context. In fact, an emphasis on application to social policy formation was part of the original declaration of SJT (Hammond et al., 1975). A number of SJT inspired policy applications were undertaken on topics ranging from citizen participation in community planning, through water resource planning, to air pollution management, and so forth (e.g., Brady & Rappoport, 1973; Flack & Summers, 1971; Hammond, Mumpower & Smith, 1977; Mosier, Skitka, Heers & Burdick, 1998; Mumpower, Veirs & Hammond, 1979; Stewart & Gelberd, 1972). Perhaps the best known and most influential of these applications is Hammond and Adelman’s (1976) ‘Denver Bullet Study.’

The policy issue in the Denver Bullet Study concerned the type of bullet that should be used by the Denver (Colorado, USA) City Police. In 1974, the Denver Police Department (DPD) decided to change its handgun ammunition because it was argued that conventional round-nosed bullets provided insufficient ‘stopping effectiveness’ (i.e., the ability to incapacitate and thus to prevent the person shot from firing back at the police or others). The DPD chief recommended using a hollow-point bullet, claiming that such bullets flattened on impact, thus decreasing penetration, increasing stopping effectiveness, and decreasing ricochet potential. This claim was challenged by, for example, the American Civil Liberties Union and minority groups. Opponents stated that the new bullets were in fact outlawed ‘dum-dum’ bullets which were more injurious than the round-nosed bullet and so should be barred from use. There were public hearings, debates and disputes, and appeals by both sides to ballistics experts for scientific information and support. Disputants focused on evaluating the merits of specific alternative bullets—confounding the physical effect of the bullets with social policy implications. As Hammond and Adelman (1976) realized, they confounded questions of value — what the bullet should accomplish with questions of fact — concerning ballistic characteristics of specific bullets. Arguments favored one option or another, but obscured the basis for a preference.

Hammond and Adelman (1976) stated that policy makers inadvertently had adopted the role of (unqualified) ballistics experts, and ballistics experts inadvertently had adopted the role of (poor) policy makers. Hammond and Adelman intervened to first discover the important policy dimensions from the policy makers’ viewpoint and then elicited ballistics experts’ ratings of the bullets on these dimensions. The relevant dimensions were stopping effectiveness, probability of serious injury, and probability of harm to bystanders. The experts’ ratings of the bullets on the last two dimensions were almost perfectly confounded. The probability of serious harm to bystanders is highly related to the penetration of the bullet, whereas the probability of the bullet effectively stopping someone from returning fire is highly related to the width of the entry wound. Giving equal weights to these dimensions, and combining these weights with the experts’ technical judgments, led Hammond and Adelman (1976, p. 395) to identify a bullet that “has greater stopping effectiveness and is less apt to cause injury (and is less apt to threaten bystanders) than the standard bullet then in use by the DPD.” The bullet they recommended was accepted by both the DPD and the Denver City Council and put into operation.

According to Adelman (1988, p. 443), the above study illustrates the significant contribution of SJT in identifying the importance of the “separation of facts and values” in resolving social policy disputes. Adelman identifies five key points to be derived from the Denver Bullet Study and similar applications. First, social polices comprise three types of judgments: (a) value judgments about what ought to be, (b) factual judgments about what is or will be, and (c) evaluative judgments that integrate value and factual judgments into a final policy decision. Second, policy makers should be responsible for value judgments and technical experts for factual judgments. Third, methods exist to build quantitative models for both value and factual judgments. Fourth, analytical methods can and should be used to combine value and factual judgments so that alternatives can be systematically evaluated. Fifth, cognitive feedback can be used to make the implications of value judgments and factual judgments explicit.

In 1996, Hammond published a book entitled Human Judgment and Social Policy: Irreducible Uncertainty, Inevitable Error, Unavoidable Injustice which attempted to understand the policy formation process.¹⁶ He included Brunswikian and SJT research as well as other work, notably, Ward Edwards’ (1954, 1961) contributions using Bayesian-themed behavioral decision theory. This 1996 book emphasized two key themes that Hammond had been considering for a number of years, but which he was able to more fully address here. The first theme was the distinction between theories of truth that emphasized coherence competence and theories of truth that emphasized correspondence competence. The former is exemplified in Ward Edward’s Bayesian approach (and later in Kahneman and Tversky’s ‘heuristics and biases’ approach, e.g., Kahneman, Slovic & Tversky, 1982) which accentuate normative rationality defined in terms of internal and logical consistency. The latter is exemplified by the Brunswikian and SJT approaches in which empirical accuracy defines achievement. The issue, according to Hammond, was whether in a policy context, it was more important to be rational (internally and logically consistent) or to be empirically accurate. Hammond’s treatment of the strengths and limitations of the two approaches is too nuanced and sophisticated to review fully here, but his conclusions are guardedly optimistic that in the realm of policy we can strike a balance between coherence and correspondence. The key to achieving this balance lies in how we think about error, which was the second theme.

Hammond (1996a) emphasized the duality of error. He noted that, although this concept had been recognized long before, it was not until Neyman and Pearson (1933) that duality of error (Type I and Type II error) was formally introduced with a mathematical treatment. The concern with error was also an important part of the Brunswikian legacy. Brunswik (1956) demonstrated that the error distributions for intuitive and analytical processes were quite different. Intuitive processes led to distributions in which there were few precisely correct responses but also few large errors, whereas with analysis there were often many precisely correct responses but occasional large errors. According to Hammond, duality of error inevitably occurs whenever decisions must be made in the face of irreducible uncertainty, or uncertainty that cannot be reduced at the moment action is required. Thus, there are two potential mistakes that may arise — false positives (Type I errors) and false negatives (Type II errors) — whenever policy decisions involve dichotomous choices, such as whether to admit or reject college applications, claims for welfare benefits, and so on. Hammond (1996a) argued that any policy problem involving irreducible uncertainty has the potential for dual error, and consequently unavoidable injustice in which mistakes are made that favor one group over another.

In this work, Hammond made a remarkable and influential advance to the Brunswikian and SJT approaches, pushing those ideas all the way from perception through thinking to policy formation. He identified two tools of particular value for analyzing policy making in the face of irreducible environmental uncertainty and duality of error. These were Signal Detection Theory (SDT; Tanner & Swets, 1954; Swets, 1992) and the Taylor-Russell (1939) paradigm. Others have extended these ideas in a number of substantive policy areas such as college admissions (Mumpower, Nath & Stewart, 2002), child welfare services (Mumpower, 2010; Mumpower & McClelland, 2014), international policy (Dunwoody & Hammond, 2006), and mammography (Stewart & Mumpower, 2004). In fact, Hammond’s 1996 book, published almost 50 years after his first academic paper, is now his most cited work.

10 Closing Remarks

In closing, it is worth noting that Hammond made several attempts, albeit with mixed success, to integrate Brunswikian and SJT perspectives with other normative, prescriptive, and descriptive theories of human judgment and decision making. One of the first efforts in this regard was the book published in 1980 entitled Human Judgment and Decision Making: Theories, Methods, and Procedures (Hammond, McClelland & Mumpower, 1980). This grew out of the Twelfth Annual Conference on Human Judgment, held in April 1978, in Boulder, Colorado. The conference Hammond organized brought together 25 prominent scholars who represented approaches to the study of judgment and decision making that focused more on analytic thought to those that focused more on intuition.¹⁷ Six approaches were included as follows: Decision Theory, Behavioral Decision Theory, Psychological Decision Theory, Social Judgment Theory, Information Integration Theory, and Attribution Theory. These were compared and contrasted in terms of six categories: origins, scope, intended function, principal concepts, loci of concepts, and intended uses. The book remains widely cited, although it is unclear how much direct influence it had on the field of judgment and decision making, and some reviewers were unconvinced that integration was possible (e.g., Phillips, 1982).

Hammond was not deterred from continuing his efforts to explicate his own work and the work of others in the field of judgment and decision making. In 1981, he was part of a team that produced a glossary intended to help decipher the idiosyncratic terminology used by different approaches and disambiguate usage of the same or similar terms to mean different things (Anderson, Deane, Hammond et al., 1981). The team defined each major concept in judgment and decision making, and provided its source, usage, current status and relationship with other concepts. Indeed, students entering the field and scholars outside the field still find this glossary useful, although it now needs to be updated to encompass the continued evolution and addition of approaches over the decades since the book was published.

In this paper, we have discussed the major and distinctive contributions that Hammond made to the science of human judgment and decision making throughout his illustrious career. In sum, his ground-breaking research extended Egon Brunswik’s theory of probabilistic functionalism and lens model framework to the study of human judgment and decision making. Through SJT, Hammond proposed a general framework for the study of judgment processes. This enabled analysis of individual judgment as well as judgment in situations where individuals interact with one another. This framework also allowed researchers to describe and explain judgmental achievement as well as explore ways to improve it. Going beyond Brunswik’s ideas, Hammond introduced CCT which is the first comprehensive theory in psychology of the relation between task properties and cognition. Hammond’s commitment to scholarship was matched by his desire to improve our social world; evidenced by his efforts to apply research to the improvement of public policy decisions.

The importance of Hammond’s contribution to the psychology of judgment and decision making continues to be manifest in the work of his students and colleagues in such disparate fields as educational research, clinical decision making, healthcare and medical decision making, social work, social perception, group decision making and negotiation, accounting and auditing, law, public policy analysis, human factors, and human technology interaction. Like Hammond, many of these researchers have engaged in both basic and applied research. We have to thank Hammond for creating a path on which we could travel during our academic careers. We hope the present paper will inspire new scholars to take their own intellectual journeys building upon, and advancing, Hammond’s legacy.

Brief Biography of Kenneth R. Hammond

Kenneth R. Hammond was born in San Francisco, California in 1917. He died in Boulder, Colorado, in 2015. He received his BA, MA, and PhD from the University of California, Berkeley. In 1948, he accepted an appointment in the Department of Psychology at the University of Colorado, Boulder, where he taught until his retirement in 1987. There, he co-founded both the Institute of Behavioral Science and the Center for Research on Judgment and Policy. He was a visiting professor at the Universities of Hawaii, Berkeley, and Arizona, as well as a visiting scholar at the International Institute for Applied Systems Analysis and the Rockefeller Foundation Bellagio Center. His research was supported by the National Science Foundation, the U. S. Public Health Service, the Army Research Institute, the Office of Naval Research, the Commonwealth Fund, and other private foundations. In 1982, he was awarded an honorary doctorate from the University of Uppsala, Sweden. In 1983, he founded the Brunswik Society which continues to serve as a forum for the discussion of Brunswikian ideas among researchers from around the world. In 1987–88, Hammond served as the second President of the Society for Judgment and Decision Making, a Society that some recall he helped to establish.

References

Adelman, L. (1988). Separation of facts and values. In B. Brehmer & C. R. B. Joyce (Eds.), Human judgment: The SJT view (pp. 443–464). New York: North-Holland.

Adelman, L., & Mumpower, J. L. (1979). The analysis of expert judgment. Technological Forecasting and Social Change, 15, 191–204.

Adelman, L., Stewart, T. R, & Hammond, K. R. (1975). A case history of the application of social judgment theory in policy formation. Policy Sciences, 6, 137–159.

Anderson, B. F., Deane. D. H., Hammond, K. R., McClelland. G. H., & Shanteau. J. C. (1981). Concepts in judgment and decision research: Definitions, sources, interrelations, comments. New York: Praeger.

Balke, W. M., Hammond, K. R., & Meyer, G. D. (1973). An alternate approach to labor-management relations. Administrative Science Quarterly, 18, 311–327.

Balzer, W. K., Doherty, M. E., & O’Connor, R. (1989). Effects of cognitive feedback on performance. Psychological Bulletin, 106(3), 410–433.

Blattberg, R. C., & Hoch, S. J. (1990). Database models and managerial intuition: 50% model+ 50% manager. Management Science, 36(8), 887–899.

Bottenberg, R. A., & Christal, R. E. (1961). An iterative technique for clustering criteria which retains optimum predictive efficiency. (WADD-TN-61–30, ASTIA Document AD-261 615). Lackland Air Force Base, TX: Personnel Research Laboratory. Wright Air Development Division.

Brady, D., & Rappoport, L. (1973). Policy capturing in the field: The nuclear safeguards problem. Organizational Behavior and Human Performance, 9(2), 253–266.

Brehmer, B. (1970). Inference behavior in a situation where the cues are not reliably perceived. Organizational Behavior and Human Performance, 5, 330–347.

Brehmer, B. (1972). Cue utilization and cue configurations in multiple-cue probability learning. Organizational Behavior and Human Performance, 8, 286–296.

Brehmer, B. (1973a). Effects of task predictability and cue validity on interpersonal learning of inference tasks involving both linear and nonlinear relations. Organizational Behavior and Human Performance, 10, 24–46.

Brehmer, B. (1973b). Effects of cue validity on interpersonal learning of inference tasks with linear and nonlinear cues. American Journal of Psychology, 86, 29–48.

Brehmer, B. (1974). Effects of cue validity and task predictability on interpersonal learning of linear inference tasks. Organizational Behavior and Human Performance, 12(1), 17–29.

Brehmer, B. (1976). Social judgment theory and the analysis of interpersonal conflict. Psychological Bulletin, 83(6), 985–1003.

Brehmer, B. (1979). Preliminaries to a psychology of inference. Scandinavian Journal of Psychology, 20, 193–210.

Brehmer, B. (1980). In one word: Not from experience. Acta Psychologica, 45, 223–241.

Brehmer, B. (1994). The psychology of linear judgment models. Acta Psychologica, 87, 137–154.

Brehmer, A., & Brehmer, B. (1988). What have we learned about human judgment from thirty years of policy capturing. In B. Brehmer, & C. R. B. Joyce (Eds.), Human judgment: The SJT view (pp. 75–114). New York: North-Holland.

Brehmer, B., & Joyce, C. R. B. (1988). Human judgment: The SJT view. New York: North-Holland.

Brunswik, E. (1940). Thing constancy as measured by correlation coefficients. Psychological Review, 47, 69–78.

Brunswik, E. (1943). Organismic achievement and environmental probability. Psychological Review, 50, 255–272.

Brunswik, E. (1944). Distal focussing of perception: Size constancy in a representative sample of situations. Psychological Monographs, 56(254), 1–49.

Brunswik, E. (1952). The conceptual framework of psychology. Chicago, IL: University of Chicago Press.

Brunswik, E. (1955a). In defense of probabilistic functionalism: A reply. Psychological Review, 55, 236–242.

Brunswik, E. (1955b). “Ratiomorphic” models of perception and thinking. Acta Psychologica, 11, 108–109.

Brunswik, E. (1955c). Representative design and probabilistic theory in a functional psychology. Psychological Review, 55, 193–217.

Brunswik, E. (1956). Perception and the representative design of experiments. Berkeley, CA: University of California Press.

Brunswik, E. (1957). Scope and aspects of the cognitive problem. In H. Gruber, K. R. Hammond, & R. Jessor (Eds.), Contemporary approaches to cognition (pp. 5–31). Cambridge, MA: Harvard University Press.

Camerer, C. F. (1981). General conditions for the success of bootstrapping models. Organizational Behavior and Human Performance, 27, 411–422.

Castellan, N. J. (1973). Comments on the “lens model” equation and the analysis of multiple-cue judgment tasks. Psychometrika, 38, 87–100.

Castellen, N. J. (1992). Relations between linear models: Implications for the lens model. Organizational Behavior and Human Decision Processes, 51, 364–381.

Christal, R. E. (1963). JAN: A technique for analyzing group judgment. (TDR-63–3 AD-402 813). Lackland Air Force Base, TX: Personnel Research Laboratory. Aerospace Medical Division.

Cook, R. L., & Stewart, T. R. (1975). A comparison of seven methods for obtaining subjective descriptions of judgmental policy. Organizational Behavior and Human Performance, 123, 31–45.

Cooksey, R. W. (1996). Judgment analysis: Theories, methods, and applications. New York: Academic Press.

Cooksey, R. W., & Freebody, P. (1985). Generalized multivariate lens model analysis for complex human inference tasks. Organizational Behavior and Human Performance, 35, 46–72.

Deane, D. H., Hammond, K. R., & Summers, D. (1972). Acquisition and application of knowledge in complex inference tasks. Journal of Experimental Psychology, 92, 20–26.

Dhami, M. K. (2011). Representative design: A challenge for scientific psychology. In K. O. Moore, & N. P. Gonzalez (Eds.), Handbook on psychology of decision-making. Nova Science Publishers, Inc.

Dhami, M. K., & Ayton, P. (2001). Bailing and jailing the fast and frugal way. Journal of Behavioral Decision Making, 14(2), 141–168.

Dhami, M. K., Belton, I., & Goodman-Delahunty, J. (2015). Quasirational models of sentencing. Journal of Applied Research in Memory and Cognition, 4(3), 239–247.

Dhami, M. K., & Harries, C. (2001). Fast and frugal versus regression models of human judgement. Thinking & Reasoning, 7(1), 5–27.

Dhami, M. K., & Harries, C. (2010). Information search in heuristic decision making. Applied Cognitive Psychology, 24(4), 571–586.

Dhami, M. K., Hertwig, R., & Hoffrage, U. (2004). The role of representative design in an ecological approach to cognition. Psychological Bulletin, 130(6), 959–988.

Dhami, M. K., & Olsson, H. (2008). Evolution of the interpersonal conflict paradigm. Judgment and Decision Making, 3(7), 547–569.

Dhami, M. K., & Thomson, M. E. (2012). On the relevance of Cognitive Continuum Theory and quasirationality for understanding management judgment and decision making. European Management Journal, 30(4), 316–326.

Doherty, M. E., & Balzer, W. K. (1988). Cognitive feedback. In B. Brehmer & C. R. B. Joyce (Eds.), Human judgment: The SJT view (pp. 163–197). Amsterdam, Netherlands: North Holland.

Dunwoody, P. T., Haarbauer, E., Mahan, R. P., Marino, C., & Chu-Chun, T. (2000). Cognitive adaptation and its consequences: A test of cognitive continuum theory. Journal of Behavioral Decision Making, 13(1), 35–54.

Dunwoody, P. T., & Hammond, K. R. (2006). The policy of pre-emption and its consequences: Iraq and beyond. Peace and Conflict: Journal of Peace Psychology, 12(1), 19–35.

Earle, T. C. (1973). Interpersonal learning. In L. Rappoport & D. Summers (Eds.), Human judgment and social interaction (pp. 240–266). New York: Holt, Rinehart, & Winston.

Edwards, W. (1954). The theory of decision making. Psychological Bulletin, 41, 380–417.

Edwards, W. (1961). Behavioral decision theory. Annual Review of Psychology, 12(1), 473–498.

Flack, J. E., & Summers, D. A. (1971). Computer aided conflict resolution in water resource planning: An illustration. Water Resources Research, 7, 1410–1414.

Ganzach, Y., Kluger, A. N., & Klayman, N. (2000). Making decisions from an interview: Expert measurement and mechanical combination. Personnel Psychology, 53(1), 1–20.

Gigerenzer, G. (2001). Ideas in exile: The struggles of an upright man. In K. R. Hammond & T. R. Stewart (Eds.), The essential Brunswik (pp. 445–452). Oxford: Oxford University Press.

Gigerenzer, G., & Goldstein, D. G. (1996). Reasoning the fast and frugal way: models of bounded rationality. Psychological Review, 103(4), 650–659.

Gigerenzer, G., Hertwig, R., & Pachur, T. (Eds.), (2011). Heuristics: The foundations of adaptive behavior. New York: Oxford University Press.

Gigerenzer, G., Hoffrage, U., & Kleinbolting, H. (1991). Probabilistic mental models: A Brunswikian theory of confidence. Psychological Review, 98, 506–528.

Gigerenzer, G., Todd, P. M., & ABC Research Group, T. (1999). Simple heuristics that make us smart. New York: Oxford University Press.

Gillis, J. S., Lipkin, J. O., & Moran, T. J. (1981). Drug therapy decisions: A social judgment analysis. The Journal of Nervous and Mental Disease, 169(7), 439–447.

Glöckner, A., & Betsch, T. (2008a). Modeling option and strategy choices with connectionist networks: Towards an integrative model of automatic and deliberate decision making. Judgment and Decision Making, 3, 215–228.

Glöckner, A., & Betsch, T. (2008b). Multiple-reason decision making based on automatic processing. Journal of Experimental Psychology: Learning, Memory, and Cognition, 34, 1055–1075.

Glöckner, A., & Betsch, T. (2012). Decisions beyond boundaries: When more information is processed faster than less. Acta Psychologica, 139, 532–542.

González-Vallejo, C., Sorum, P. C., Stewart, T. R., Chessare, J. B., & Mumpower, J. L. (1998). Physicians’ diagnostic judgments and treatment decisions for acute otitis media in children. Medical Decision Making, 18(2), 149–162.

Grebstein, L. (1963). Relative accuracy of actuarial prediction, experienced clinicians, and graduate students in a clinical judgment task. The Journal of Consulting Psychology, 37, 127–132.

Hamm, R. M. (1988). Clinical intuition and clinical analysis: Expertise and the cognitive continuum. In J. Dowie & A. Elstein (Eds.), Professional judgment: A reader in clinical decision making (pp. 78–105). Cambridge: Cambridge University Press.

Hammond, K. R. (1948). Subject and object sampling. Psychological Bulletin, 45, 530–533.

Hammond, K. R. (1954). Representative design vs. systematic design in clinical psychology. Psychological Bulletin, 51, 150–159.

Hammond, K. R. (1955). Probabilistic functioning and the clinical method. Psychological Review, 62, 255–262.

Hammond, K. R. (1965). New directions in research on conflict resolution. Journal of Social Issues, 21, 44–66.

Hammond, K. R. (1966). Probabilistic functionalism: Egon Brunswik’s integration of history, theory, and method of psychology. In K. R. Hammond (Ed.), The psychology of Egon Brunswik (pp. 15–80). New York: Holt, Rinehart, & Winston.

Hammond, K. R. (1971). Computer graphics as an aid to learning. Science, 172, 903–908.

Hammond, K. R. (1972). Inductive knowing. In J. Royce & W. Rozeboom (Eds.), The psychology of knowing (pp. 285–320). New York: Gordon and Breach.

Hammond, K. R. (1973). The cognitive conflict paradigm. In L. Rappoport & D. Summers (Eds.), Human judgment and social interaction (pp. 188–205). New York: Holt, Rinehart, & Winston.

Hammond, K. R. (1988). Judgment and decision making in dynamic tasks. University of Colorado, Boulder, Center for Research on Judgment and Policy. Retrieved July 31, 2017, from http://www.dtic.mil/get-tr-doc/pdf?AD=ADA199907.

Hammond, K. R. (1996a). Human judgment and social policy: Irreducible uncertainty, inevitable error, unavoidable injustice. New York: Oxford University Press.

Hammond, K. R. (1996b). Upon reflection. Thinking and Reasoning, 2, 239–248.

Hammond, K. R. (2000a). Coherence and correspondence theories in judgment and decision making. In T. Connolly, H. R. Arkes, & K. R. Hammond (Eds.), Judgment and decision making: An interdisciplinary reader, 53–65.

Hammond, K. R. (2000b). Judgments under stress. New York: Oxford University Press.

Hammond, K. R. (2001a). Expansion of Egon Brunswik’s psychology, 1955–1995. In K. R. Hammond & T. R. Stewart (Eds.), The essential Brunswik (pp. 464–480). Oxford: Oxford University Press.

Hammond, K. R. (2001b). Explaining to the nobility. In K. R. Hammond & T. R. Stewart (Eds.), The essential Brunswik (pp. 134–135). Oxford: Oxford University Press.

Hammond, K. R. (2001c). Representative design in action in the middle of the twentieth century. In K. R. Hammond & T. R. Stewart (Eds.), The essential Brunswik (pp. 67–68). Oxford: Oxford University Press.

Hammond, K. R. (2007). Beyond rationality: The search for wisdom in a troubled time. Oxford: Oxford University Press.

Hammond, K. R. (2010). Intuition, no! …quasirationality, yes! Psychological inquiry, 21(4), 327–337.

Hammond, K. R., & Adelman, L. (1976). Science, values, and human judgment. Science, 194, 389–396.

Hammond, K. R., & Boyle, P. J. R. (1971). Quasirationality, quarrels, and new conceptions of feedback. Bulletin of the British Psychological Society, 24, 103–113.

Hammond, K. R., & Brehmer, B. (1973). Quasi-rationality and distrust: Implications for international conflict. In L. Rappoport & D. Summers (Eds.), Human judgment and social interaction (pp. 338–391). New York: Holt, Rinehart & Winston.

Hammond, K. R. & Grassia, J. (1985). The cognitive side of conflict: From theory to resolution of policy disputes. In S. Oskamp (Ed.), Applied social psychology annual. Vol. 6 International conflict and national public policy issues (pp. 233–254). Beverly Hills, CA: Sage.

Hammond, K. R., Hamm, R. M., Grassia, J., & Pearson, T. (1987). Direct comparison of the efficacy of intuitive and analytical cognition in expert judgment. IEEE Transactions on Systems, Man, and Cybernetics, 17, 753–770.

Hammond, K. R., Hursch, C. J., & Todd, F. J. (1964). Analyzing the components of clinical inference. Psychological Review, 71, 438–456.

Hammond, K. R., & Lang, Z. R. (2017). Confronting inductive inference (at last): Concepts from aeronautical engineering can lead to advances in social psychology. Paper presented at 25th International Meeting of the Brunswik Society, Vancouver, British Columbia, Canada, November.

Hammond, K. R., McClelland, G. H., & Mumpower, J. (1980). Human judgment and decision making: Theories, methods, procedures. New York: Praeger.

Hammond, K. R., Mumpower, J. L., & Smith, T. R. (1977). Linking environmental models with models of human judgment: A symmetrical decision aid. IEEE Transactions on Systems, Man, and Cybernetics, 7, 358–367.

Hammond, K. R., & Stewart, T. R. (2001a). Introduction. In K. R. Hammond & T. R. Stewart (Eds.), The essential Brunswik: Beginnings, Explications, Applications (pp. 3–11). New York: Oxford University Press.

Hammond, K. R., & Stewart, T. R. (2001b). The essential Brunswik. Oxford: Oxford University Press.

Hammond, K. R., Stewart, T. R., Brehmer, B., & Steinmann, D. O. (1975). Social judgment theory. In M. Kaplan & S. Schwartz (Eds.), Human judgment and decision processes (pp. 271–312). San Diego, CA: Academic Press.

Hammond, K. R., & Summers, D. A. (1965). Cognitive dependence on linear and non-linear cues. Psychological Review, 72, 215–224.

Hammond, K. R., & Summers, D. A. (1972). Cognitive control. Psychological Review, 79, 58–67.

Hammond, K. R., Summers, D. A., & Deane, D. H. (1973). Negative effects of outcome feedback in multiple cue probability learning. Organizational Behavior and Human Performance, 9, 30–34.

Hammond, K. R., & Wascoe, N. (1980). Realizations of Brunswik’s representative design. San Francisco: Jossey-Bass.

Hammond, K. R., Wilkins, M. M., & Todd, F. J. (1966a). A research paradigm for the study of interpersonal learning. Psychological Bulletin, 65, 221–232.

Hammond, K. R., Todd, F. J., Wilkins, M. M., & Mitchell, M. (1966b). Cognitive conflict between persons: Applications of the “lens model” paradigm. Journal of Experimental Social Psychology, 2, 343–360.

Harvey, N. (2011). Learning judgment and decision making from feedback. In M. K. Dhami, A. Schlottmann, & M. Waldmann (Eds.). Judgment and decision making as a skill: Learning, development, and evolution (pp. 406–464). Cambridge: Cambridge University Press.

Herring, F. H. (1954). A psychological study of patient response during surgery and anesthesia. Unpublished doctoral dissertation, University of Colorado.

Holaday, B. E. (1933). Die Gro¨_enkonstanz der Sehdinge bei Variation der inneren und a¨u_eren Wahrnehmungsbedingungen [Size constancy for visual objects under variation of internal and external perceptual conditions]. Achiv fu¨r die Gesamte Psychologie, 88, 419–486.

Holzworth, R. J. (1999). Annotated Bibliography of Cue Probability Learning Studies. The Brunswik Society. Retrieved on July 6, 2017, from http://www.brunswik.org/resources/mcplbib.doc.

Holzworth, R. J., & Doherty, M. E. (1976). Feedback effects in a metric multiple cue probability learning task. Bulletin of the Psychonomic Society, 8, 1–3.

Horstmann, N., Ahlgrimm, A., & Glöckner, A. (2009). How distinct are intuition and deliberation? An eye-tracking analysis of instruction-induced decision modes. Judgment and Decision Making, 4, 335–354.

Hursch, C. J., Hammond, K. R., & Hursch, J. (1964). Some methodological considerations in multiple-cue probability studies. Psychological Review, 71, 42–60.

Jekel, M., Glöckner, A., Fiedler, S., & Bröder, A. (2012). The rationality of different kinds of intuitive decision processes. Synthese, 189, 147–160.

Juslin, P., Winman, A., & Olsson, H. (2000). Naive empiricism and dogmatism in confidence research: A critical examination of the hard-easy effect. Psychological Review, 107, 384–396.

Kirlik, A. (Ed.), Adaptive perspectives on human-technology interaction. Oxford: Oxford University Press.

Libby, R., & Lewis, B. L. (1982). Human information processing research in accounting: The state of the art 1982. Accounting, Organisations, and Society, 7, 231–285.

Kahneman, D., Slovic, P., & Tversky, A. (1982). Judgments under uncertainty: Heuristics and biases. Cambridge: Cambridge University Press.

Karelaia, N., & Hogarth, R. M. (2008). Determinants of linear judgment: a meta-analysis of lens studies. Psychological Bulletin, 134(3), 404–426.

Klayman, J. (1988). Cue discovery in probabilistic environments: Uncertainty and experimentation. Journal of Experimental Psychology: Learning, Memory, and Cognition, 14(2), 317–330.

Mahan, R. P. (1988). Stress-induced strategy shifts toward intuitive cognition: A cognitive continuum framework approach. Human Performance, 7(2), 85–118.

Marewski, J. N., & Melhorn, K. (2011). Using the ACT-R architecture to specify 39 quantitative process models of decision making. Judgment and Decision Making, 6(6), 439–519.

Meehl, P. E. (1954). Clinical versus statistical prediction: A theoretical analysis and a review of the evidence. Minneapolis: University of Minnesota Press.

Miller, M. (1973). Interpersonal learning: Laboratory and field investigations. In L. Rappoport & D. Summers (Eds.), Human judgment and social interaction (pp. 267–294). New York: Holt, Rinehart, & Winston.

Mosier, K. L., Skitka, L. J., Heers, S., & Burdick, M. (1998). Automation bias: Decision making and performance in high-tech cockpits. The International Journal of Aviation Psychology, 8(1), 47-63.

Mumpower, J. L. (2010). Disproportionality at the “front end” of the child welfare services system: An analysis of rates of referrals, “hits,” “misses,” and “false alarms”. Journal of Health and Human Services Administration, 33(3), 364–405.

Mumpower, J., & Adelman, L. (1980). The application of Brunswikian methodology to policy formation. In K. R. Hammond, & Wascoe, N. E. (Eds.). Realizations of Brunswik’s representative design (pp. 49–60). San Francisco: Jossey-Bass.

Mumpower, J. L. & Hammond, K. R. (1974). Entangled task dimensions: An impediment to interpersonal learning. Organizational Behavior and Human Performance, 11, 377–389.

Mumpower, J. L., & McClelland, G. H. (2014). A signal detection theory analysis of racial and ethnic disproportionality in the referral and substantiation processes of the US child welfare services system. Judgment and Decision Making, 9(2), 114.

Mumpower, J. L., Nath, R., & Stewart, T. R. (2002). Affirmative action, duality of error, and the consequences of mispredicting the academic performance of African American college applicants. Journal of Policy Analysis and Management, 21(1), 63–77.

Mumpower, J. L., Sheffield, J., Darling, T. A., & Milter, R. G. (2004). The accuracy of post-negotiation estimates of the other negotiator’s payoff. Group Decision and Negotiation, 13(3), 259–290.

Mumpower, J. L., & Stewart, T. R. (1996). Expert judgment and expert disagreement. Thinking and Reasoning, 2, 191–212.

Mumpower, J. L., Veirs, V., & Hammond, K. R. (1979). Scientific information, social values, and policy formation: The application of simulation models and judgment analysis to the Denver regional air pollution problem. IEEE Transactions on Systems, Man, and Cybernetics, 9, 464–474.

Newton, J. R. (1965). Judgment and feedback in a quasi-clinical situation. Journal of Personality and Social Psychology, 1, 336–342.

Neyman, J., & Pearson, E. S. (1933). On the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society of London. Series A, 231, 289–337.

Phillips, L. (1982). Judgment and choice: An integration? Contemporary Psychology, 27(4), 312–313.

Poses, R. M., Cebul, R. D., Wigton, R. S., Centor, R. M., Collins, J., & Fleischli, G. (1992). Controlled trial using computerized feedback to improve physicians’ diagnostic judgments. Academic Medicine, 67, 345–347.

Reilly, B., & Doherty, M. E. (1989). A note on the assessment of self-insight in judgment research. Organizational Behavior and Human Decision Processes, 44, 123–131.

Reilly, B. A., & Doherty, M. E. (1992). The assessment of self-insight in judgment policies. Organizational Behavior and Human Decision Processes, 53, 285–309.

Rusou, Z., Zakay, D., & Usher, M. (2013). Pitting intuitive and analytical thinking against each other: The case of transitivity. Psychonomic Bulletin and Review, 20, 608–614.

Seifert, M., & Hadida, A. L. (2013). On the relative importance of linear model and human judge(s) in combined forecasting. Organizational Behavior and Human Decision Processes, 120(1), 24–36.

Simon, H. A. (1957). Models of man: Social and rational. New York: Wiley.

Simon, H. A. (1991). Models of my life. New York: Basic Books.

Slovic, P., & Lichtenstein, S. (1971). Comparison of Bayesian and regression approaches to the study of information processing in judgment. Organizational Behavior and Human Performance, 6(6), 649–744.

Stewart, T. R. (1976). Components of correlation and extensions of the lens model equation. Psychometrika, 41(1), 101–120.

Stewart, T. R. (1988). Judgment analysis. In B. Brehmer, & C. R. B. Joyce (Eds.), Human judgment: The SJT view (pp. 41–74). New York: North-Holland.

Stewart, T. R. (1990). A decomposition of the correlation coefficient and its use in analysing forecasting skill. Weather and Forecasting, 5, 661–666.

Stewart, T. R., & Gelberd, L. (1976). Analysis of judgment policy: A new approach for citizen participation in planning. Journal of the American Institute of Planners, 42, 33–41.

Stewart, T. R., & Lusk, C.M. (1994). Seven components of judgmental forecasting skill: Implications for research and the improvement of forecasts. Journal of Forecasting, 13, 575–599.

Stewart, T. R., & Mumpower, J. L. (2004). Detection and selection decisions in the practice of screening mammography. Journal of Policy Analysis and Management, 23(4), 908–920.

Stewart, T. R., Roebber, P. J., & Bosart, L. F. (1997). The importance of the task in analyzing expert judgment. Organizational Behavior and Human Decision Processes, 69(3), 205–219.

Summers, D. A., Taliaferro, J. D., & Fletcher, D. J. (1970). Subjective vs objective description of judgment policy. Psychonomic Science, 18, 249–250.

Swets, J. A. (1992). The science of choosing the right decision threshold in high stakes diagnoses. American Psychologist, 47, 522–532.

Tanner, W. P., Jr., & Swets, J. A. (1954). A decision making theory of visual detection. Psychological Review, 61, 401–409.

Tape, T. G., Kripal, J., & Wigton, R. S. (1992). Comparing methods of learning clinical prediction from case simulations. Medical Decision Making, 12, 213–221.

Taylor, H. C., & Russell, J. T. (1939). The relationship between validity coefficients to the practical application of tests in selection. Journal of Applied Psychology, 23, 565–578.

Todd, F. J. (1954). A methodological study of clinical judgment. Unpublished doctoral dissertation, University of Colorado.

Todd, F. J., & Hammond, K. R. (1965). Differential feedback in two multiple-cue probability learning tasks. Behavioral Science, 10(4), 429–435.

Tucker, L. R. (1964). A suggested alternative formulation in the developments by Hursch, Hammond and Hursch, and by Hammond, Hursch and Todd, Psychological Review, 71, 528–530.

Tolman, E., & Brunswik, E. (1935). The organism and the causal texture of the environment. Psychological Review, 13, 300–304.

Ullman, D. G., & Doherty, M. E. (1984). Two determinants of the diagnosis of hyperactivity: The child and the clinician. In M. Woraich & D. K. Routh (Eds.), Advances in behavioral pediatrics (pp. 167–219). Greenwich, CT: JAI Press.

Wallace, H. A. (1923). What is in the corn judge’s mind? Journal of the American Society of Agronomy, 42, 43–77.

Way, B. B., Allen, M. H., Mumpower, J. L., Stewart, T. R., & Banks, S. M. (1998). Interrater agreement among psychiatrists in psychiatric emergency assessments. American Journal of Psychiatry, 155(10), 1423–1428.

Wigton, R. S. (1988). Use of linear models to analyze physicians’ decisions. Medical Decision Making, 8, 241–252.

Wigton, R. S. (1996). Social judgment theory and medical judgment. Thinking & Reasoning, 2, 175–190.

Department of Psychology, Middlesex University, The Burroughs, Hendon, London NW4 4BT, UK. E-mail: m.dhami@mdx.ac.uk.

Bush School of Government and Public Service, Texas A&M University, College Station, Texas, USA.

Hammond continued to perform idiographic analysis (but see Hammond et al., 1975), as Brunswik suggested, despite its time-consuming and somewhat tedious nature.

Hammond was thanked by researchers who later drew on Brunswik’s idea of representative stimulus sampling to critique the extant work on cognitive biases such as overconfidence (e.g., Gigerenzer, Hoffrage & Kleinbolting, 1991; see also Juslin, Winman & Olsson, 2000). Simply put, it was found that biases were reduced if researchers employed representative rather than systematic stimulus sampling.

At about the same time, Meehl (1954) published an influential book comparing clinical versus actuarial prediction, concluding that statistical predictions typically outperformed clinical judgment.

Whereas Meehl (1954) had compared the clinician to a statistical model of the environment, Hammond (1955), compared a statistical model of the clinician to a statistical model of the environment.

There is a large body of basic and applied psychological research that arose independently, and has little or no apparent connection to Brunswik or Hammond. Some have argued that the history of policy capturing research dates back to Wallace (1923) who is often credited with pioneering such efforts in his study of corn judges in which he tried to capture ‘what is in the corn judge’s mind?’ The term policy capturing was coined by Bottenberg and Christal (1961), and refers to the analysis of judgment data using multiple regression techniques. Judgment analysis (originally called JAN) was coined by Christal (1963), and refers to the combined use of policy capturing and policy clustering methods.

Cooksey (1996, pp. 57–58) provides an informative discussion of the points of intersection and divergence between the strands of policy capturing and judgment analysis research closely linked to the Brunswikian and SJT tradition and those that are outside that tradition.

In some studies, aggregate or composite policies may be captured (Hammond et al., 1975), although this departs from Brunswik’s (1943) call for analysis to be at the idiographic level.

Slovic and Lichtenstein (1971) reviewed studies conducted in what they called the correlational and analysis of variance paradigms. Hammond et al. (1975) reviewed experimental studies in the multiple-cue probability learning, and the interpersonal learning and interpersonal conflict paradigms. Libby and Lewis (1982) reviewed studies conducted in the domains of accounting and auditing. The reviews by Brehmer and Brehmer (1988) and Brehmer (1994) focused on studies involving participants experienced with the judgment task being studied. Finally, Cooksey (1996) reviewed studies employing the single-systems, double-systems, triple-systems, N-systems and hierarchical designs (we shall say more about these designs later).

The policy captured by the regression model is also referred to as a tacit, implicit or objective policy, and the individual’s own statement of policy is also called his or her explicit or subjective policy. Cook and Stewart (1975) found little difference amongst seven different direct report procedures, namely distributing 100 points among cues, rating cues on a 100 point scale, paired comparison ratings of cues, ratio comparison ratings of cues, the number of times cues were influential, aggregation of judgments made using each cue one at a time on each case, and aggregation of ratings of each cue’s contribution to the judgment of each case.

Cognitive feedback is the term used when information about past events is provided and cognitive feedforward refers to information about future events (Doherty & Balzer, 1988).

Newton (1965) had demonstrated that the sole provision of cognitive feedback (i.e., ecological validities of cues and the utilization validities) could significantly improve performance.

Unfortunately, this latter situation reflects how learning may take place outside the laboratory (Anderson, Deane, Hammond, McClelland & Shanteau, 1981).

Stewart (1976) developed a hierarchical formulation of the lens model equation which made it possible to isolate the contributions of different sets of variables.

Holaday (1933) had first shown how an exemplary stimulus could be systematically stripped of its complexity through ‘successive omission’ of cues, and Brunswik (1956) viewed this as a hybrid design—mixing elements of systematic and representative design.

Hammond (1996) points out that Simon (1991) similarly viewed humans as ‘adaptive systems’ whose efforts to understand their environments are constrained, and consequently limit any potential for optimization.

This book won the 1997 Outstanding Research Publication Award from the American Educational Research Association.

For a photograph of attendees at the conference, who included pioneers of judgment and decision making research such as Norman Anderson, Hutton Baron, Robyn Dawes, Michael Doherty, Ward Edwards, Hillel Einhorn, Baruch Fischhoff, Kenneth Hammond, Marty Kaplan, Ralph Keeney, Sarah Lichtenstein, Ned Jones, Mel Manis, Gary McClelland, Lee Ross, James Shanteau, Kelly Shaver, Paul Slovic, Thomas Stewart, Amos Tversky, and others, see http://www.brunswik.org/photos/bldrgroup.html.