Decision Science News

TRICKING PEOPLE INTO THINKING YOUR SCIENTIFIC SHOTS NEVER MISS

You visit the farm of a Texan, Joe, who claims to be a sharpshooter. When walking past his barn, you see a chalk target drawn on the wall with a bunch of tightly-grouped bullet holes in the bullseye. After observing that Joe can’t shoot well at all, you realize that he drew the bullseye after firing the shots.

The tale of the Texas sharpshooter resonates with JDM (Judgment and Decision-Making) research on perceiving illusory patterns, and a topic of recent interest, detecting bogus experimental results. In a recent paper, Ulrich Schimmack talks about multi-study research papers in this way. When you see 10 studies in a single paper that confirm a hypothesis, can you conclude that the basic effect is replicable and robust?

One problem in science is that reading a research article is a bit like visiting Joe’s farm. Readers only see the final result, without knowing how the final results were created. Is Joe a sharpshooter who drew a target and then fired 10 shots ar the target? Or was the target drawn after the fact?
-Schimmack, in press

We have been looking into the roots of the Texas Sharpshooter vignette in academic writing. The earliest and most common “initial” cite we found after a quick search was Grufferman’s from 1977, with no claims that this is the earliest use:

Here it is.

There have been several dramatic time-space clusters of leukemia reported in which, following an initial observation of two or more cases in a locality, a time unit and geographical area are selected so as to best define a time-space cluster. Such a posteriori clusters are analogous to the story of the Texas sharpshooter who would shoot his rifle at the side of a barn and then carefully draw a target around each bullet-hole so that each bullet-hole passed exactly through the center of the “bull’s-eye.” Although a posteriori clusters do serve to demonstrate that cases can cluster in time and space, they do not allow for determining whether this is more than a chance occurrence.

-Grufferman (1977)

REFERENCES
Grufferman S. (1977). Clustering and aggregation of exposures in Hodgkin’s disease. Cancer 39, 1829-1833

Schimmack, U. (in press). The Ironice Effect of Significant Results on the Credibility of Multiple Study Articles. Psychological Methods.

Photo credit: http://www.flickr.com/photos/24730945@N03/4130123404/sizes/l/

EVOKING FRANKLIN TO GET PEOPLE TO BUY

In response to last week’s post about Franklin’s rule, your loyal Editor’s mother sends along this passage from the story “Can I Just Sit Here for a While?” from Ron Hansen’s Nebraska: Stories (also published in the Atlantic Monthly).

In the story, the salesman is telling an acquaintance that he “discovered a gimmick, a tool which handn’t failed him yet. It was called the Benjamin Franklin close.”

Say you get a couple who’re wavering over the purchase of a car. You take them into your office and close the door and say, ‘Do you know what Benjamin Franklin would do in situations like this?’ That’s a toughie for them so you let them off the hook. You take out a tablet and draw a line down the center of the page, top to bottom. ‘Benjamin Franklin,’ you say, ‘would list all the points in favor of buying this car and then he’d list whatever he could against it. Then he’d total things up.’ The salesman handles all the benefits. You begin by saying, “So okay, you’ve said your old car needs an overhaul. That’s point one. You’ve said you want a station wagon for the kids; that’s point two. You’ve told me that a particular shade of brown is your favorite.’ And so on. Once you’ve tabulated your pitches, you flip the tablet around and hand across the pen. ‘Okay,’ you tell them. ‘Now Benjamin Franklin would write down whatever he had against buying that car.’ And you’re silent. As noiseless as you can be. You don’t say boo to them. They stare at that blank side of the paper and they get flustered. They weren’t expecting this at all. Maybe the wife will say, ‘We can’t afford it,’ and the husband will hurry up and scribble that down. Maybe he’ll say, “It’s really more than we need for city driving.’ He’ll glance at you for approval but you won’t even nod your head. You’ve suddenly turned to stone. Now they’re struggling. They see two reasons against and twelve reasons for. You decide to help them. You say, ‘Was it the color you didn’t like?’ Of course not, you dope. You put that down as point three in favor. But the wife will say, ‘Oh no, I like that shade of brown a lot.’ You sit back in your chair and wait. You wait four or five minutes if you have to, until they’re really uncomfortable, until you’ve got them feeling like bozos. Then you take the tablet from them and make a big show of making the tally. They think you’re an idiot anyway; counting out loud won’t surprise them. And when you’ve told them they have twelve points in favor, two points against, you sit back in your chair and let that sink in. You say, ‘What do you think Benjamin Franklin would do in this situation?; You’ve got them cornered and they know it and they can’t think of any way out because there’s only one way and they never consider it. Pressed against the wall like that the only solution is for the man or woman to say, I-Just-Don’t-Feel-Like-It-Now.’ All the salesman can do is recapitulate. If they want to wait, if the vibes don’t feel right, if they don’t sense it’s the appropriate thing to do, they’ve got him. I just don’t feel like it now. There’s no way to sell against that.

Mom writes “I hope you found this an interesting use (misuse?) of old Ben Franklin’s technique!”.

Despite all our decision science researching, we’ve never come across the idea of using a (unit) weighted rule as a sales tactic. You’d think it wouldn’t really work, as the customer could always generate reasons against buying. We wonder if this works because of social pressure against listing things like “I don’t trust: this guy / this dealership / the stuff he’s telling me / that quoted price as all-inclusive”. If such things aren’t listed, the tally will favor buying over not buying.

FRANKLIN’S RULE

Ben Franklin had views on how to make a decision. In a letter to Joesph Preistley, he wrote

To Joseph Priestley

London, September 19, 1772

Dear Sir,

In the Affair of so much Importance to you, wherein you ask my Advice, I cannot for want of sufficient Premises, advise you what to determine, but if you please I will tell you how.

When these difficult Cases occur, they are difficult chiefly because while we have them under Consideration all the Reasons pro and con are not present to the Mind at the same time; but sometimes one Set present themselves, and at other times another, the first being out of Sight. Hence the various Purposes or Inclinations that alternately prevail, and the Uncertainty that perplexes us.

To get over this, my Way is, to divide half a Sheet of Paper by a Line into two Columns, writing over the one Pro, and over the other Con. Then during three or four Days Consideration I put down under the different Heads short Hints of the different Motives that at different Times occur to me for or against the Measure. When I have thus got them all together in one View, I endeavour to estimate their respective Weights; and where I find two, one on each side, that seem equal, I strike them both out: If I find a Reason pro equal to some two Reasons con, I strike out the three. If I judge some two Reasons con equal to some three Reasons pro, I strike out the five; and thus proceeding I find at length where the Ballance lies; and if after a Day or two of farther Consideration nothing new that is of Importance occurs on either side, I come to a Determination accordingly.

And tho’ the Weight of Reasons cannot be taken with the Precision of Algebraic Quantities, yet when each is thus considered separately and comparatively, and the whole lies before me, I think I can judge better, and am less likely to take a rash Step; and in fact I have found great Advantage from this kind of Equation, in what may be called Moral or Prudential Algebra.

Wishing sincerely that you may determine for the best, I am ever, my dear Friend,

Yours most affectionately

B. Franklin

A fair bit of academic research has been done on the quality of Franklin’s rule for making decisions. See for example:

* Gigerenzer, G. & Goldstein, D. G. (1999). Betting on one good reason: The Take The Best heuristic. In Gigerenzer, G., Todd, P. M. & the ABC Research Group, Simple Heuristics That Make Us Smart. New York: Oxford University Press.

* Czerlinski, J., Gigerenzer, G., & Goldstein, D. G. (1999). How good are simple heuristics? In Gigerenzer, G., Todd, P. M. & the ABC Research Group, Simple Heuristics That Make Us Smart. New York: Oxford University Press. [Download]

For analysis of an even simpler, unweighted variant, see:

* Dawes, R. M. The robust beauty of improper linear models in decision making. American Psychologist, 1979, 34, 571-582.

Source of Franklin quote: There is a copy of this quote online at http://www.procon.org/view.background-resource.php?resourceID=1474 which cites:

* Mr. Franklin: A Selection from His Personal Letters. Contributors: Whitfield J. Bell Jr., editor, Franklin, author, Leonard W. Labaree, editor. Publisher: Yale University Press: New Haven, CT 1956.

Image credit: http://en.wikipedia.org/wiki/Benjamin_Franklin_Medal_%28American_Philosophical_Society%29

FAST POLLING AND CELEBRITY WRANGLERS

We recently connected through London’s Heathrow Airport, and learned a couple things.

1. Fast, easy customer satisfaction surveys are possible. As you finish the security check at this airport, you walk past these machines (above) that ask you to rate how your experience by simply pushing one button. The machines are placed right in the middle of your path, not off to the side, so you can vote as you walk by without stopping. If you want to fill in a comment card, they had those and pens, too. We really like the idea of getting lots of data in a way that doesn’t slow people down or compromise anonymity. They could use the responses to figure out when the experience is the worst and take measures to fix it. The only negative here is that we think these machines are only at the “fast track” lanes (for frequent fliers), making it a bit classist.

2. On the inter-terminal bus, we eavesdropped on two uniformed celebrity wranglers, whose job it is to look after VIPs as they pass through Heathrow. Apparently, VIPs (actors, pop stars, politicians, etc.) get escorted from place to place, are popped into their first class seats just before take off, and get to hang out in some private lounge before flights. As you might imagine, wranglers talk about which celebrities are naughty or nice. Allegedly, Smokey Robinson is “sooooo nice” while will.i.am was a “stuck-up little ____”.

SOCIETY FOR JUDGMENT AND DECISION MAKING NEWSLETTER

Just a reminder that the quarterly Society for Judgment and Decision Making newsletter can be downloaded from the SJDM site:

http://sjdm.org/newsletters/

It features jobs, conferences, announcements, and more.

Enjoy!
Decision Science News / SJDM Newsletter Editor

DIFFERENT CITIES TELL DIFFERENT STORIES

Last spring we looked at the state of the housing bubble in the US. The question on readers’ minds then was “where is it going next”? Since Decision Science News is looking for a place to buy, it is on our minds as well.

It’s been more than a year, so let’s have a look. Above, we see the plot for all cities. We realize the colors are hard to follow, so if you want to track your city, download the spreadsheet.

In January 2011, the average index value across the cities represented here was 127, in April 2012 (the latest data we have) it was 124. The “composite 10” score in January 2011 was 154, it went to 148. Similarity the “composite 20” value went from 141 to 136. So, things have continued to drop a bit.

All depends on the local market, however. This is an Olympic year, so we really should highlight a few exceptional stories (the same ones we profiled in 2011):

Want to reproduce these graphs yourself? Go right ahead! Here’s the code. Plots are made with R and Hadley Wickham‘s ggplot2.


library(ggplot2)
library(reshape)
## Read in data, available from:
#www.standardandpoors.com/indices/sp-case-shiller-home-price-indices/en/us/?indexId=spusa-cashpidff--p-us----
#Delete the 2nd row and make 1st col 1st row say YEAR
dat=read.csv("CSHomePrice_History.csv")
mdf=melt(dat,id.vars="YEAR")
mdf$Date=as.Date(paste("01-",mdf$YEAR,sep=""),"%d-%b-%y")
names(mdf)=c("MonthYear","City","IndexValue","Date")
mdf$yr=format(mdf$Date,"%Y")
mdf=subset(mdf,yr>1999)
ggplot(data=mdf,aes(x=Date,y=IndexValue)) + geom_line(aes(color=City),size=1.25) +
scale_x_date("Year", minor_breaks="years") + scale_y_continuous("Case Schiller Index")
sm=subset(mdf,City %in% c('NY.New.York','FL.Miami','CA.Los Angeles','MI.Detroit',
'TX.Dallas','IL.Chicago','DC.Washington'))
sm$City=droplevels(sm$City)
ggplot(data=sm,aes(x=Date,y=IndexValue)) + geom_line(aes(color=City),size=1.5) +
scale_x_date("Year", minor_breaks="years") + scale_y_continuous("Case Schiller Index")


USING TIME, IN ADDITION TO IMPRESSIONS, FOR INTERNET ADVERTISING

From Goldstein, Suri, McAfee (2011)

Last week it was announced that Facebook is rotating its ads after a certain time of exposure.

In 2011 and 2012, Sid Suri, Preston McAfee, and Dan Goldstein published a couple papers putting forth and improving the idea of time-based internet advertising:

• Goldstein, Daniel G., R. Preston McAfee, & Siddharth Suri. (2012). Improving the effectiveness of time-based display advertising. Proceedings of the 13th ACM Conference on Electronic Commerce (EC’12) [Winner: Best Paper Award]
• Goldstein, Daniel G., R. Preston McAfee, & Siddharth Suri. (2011). The effects of exposure time on memory of display advertisements. Proceedings of the 12th ACM Conference on Electronic Commerce (EC’11)

How does it work? Well, there are a couple primary forces at play. First, the longer an ad is in view, the more likely people are to remember it later. This can be seen in the graph above, based on an experiment in which the time an ad was in view was manipulated randomly. Second, ads that appear soon after the page loads are more likely to be remembered than ads that appear a longer time after the page loads. This can be seen below.

From Goldstein, Suri, McAfee (2012)

These data are based on an experiment in which an ad was made to appear during certain seconds after the page loaded.

The vertical axis shows the probability of recognizing an ad after the experiment. The horizontal axis shows the number of seconds that have elapsed after a page loads. For example, the short line running from 0 to 10 seconds shows the probability of recognition of an ad that was present for the first ten seconds after the page loaded. The short line running from 10 to 20 seconds shows the probability of recognition of an ad that was shown for the second 10 seconds after the page loaded. Clearly, the ad that was shown immediately after the page loads is more likely to be remembered. We hypothesize that people are visually scanning the page soon after it loads and that after a while, they start reading and do not notice ads anymore. (The vertical line segments are standard error bars)

The long line that hits the vertical axis at about .4 shows roughly the recognition rate you get by leaving an ad up for the first 20 seconds after the page loads. The line at top shows the recognition rate you get by advertising in the first 10 seconds on one page load and the second 10 seconds on another page load (the sum of the bottom two lines). This shows that two ten-second exposures are better than one 20-second exposure, even if one of the ten-second exposures takes place in the less desirable second slot (seconds 10 to 20 after the page loads). To learn more, check out Goldstein, Suri and McAfee (2012).

NEWS REFERENCES

Facebook now replacing ads on static pages

Facebook begins rotating ads on static pages if users don’t interact

** AMA MARKETING JOB MARKET NEWS **

Schools that are interviewing at the AMA and are interested in an excellent and computationally-skilled rookie should arrange to interview Yvetta Simonyan who is completing her PhD under myself and others at London Business School.

Plots in this post made with Hadley Wickham’s ggplot2 in R

GAMES BUILT INTO EXERCISE MACHINES

Decision Science News was back in its former home of Berlin this week.

The hotel they put us up in provided free entry to a fancy gym next door. This gym had rowing machines. In the past, we’ve always found it hard to resist trying rowing machines, but always found them too boring to use for long.

But this rowing machine, the clever Concept2, had built in games.

In the fish game, above left, you regulate the speed of your rowing to eat smaller fish and avoid sharks.

In the dart game, which is better, an even speed of rowing steers virtual darts into a target.

Both games provide you a score, which if you are human, you may find hard to resist improving. We figure the games added 20 minutes to our workout. And we’re going back tomorrow and try them again.

Call it gamification, incentives for exercise, or whatever; we like the combination of games and exercise machines.

Maybe we should just take up a sport?

SLIPS IN HEURISTIC USE WITH AGE

This week, an interesting paper about how heuristics, which have low cognitive demands, can nonetheless become less effective as cognitive decline sets in.

REFERENCE
Pachur, Thorsten; Mata, Rui; Schooler, Lael J. (2009). Cognitive aging and the adaptive use of recognition in decision making. Psychology and Aging, 24(4), 901-915.

ABSTRACT

The recognition heuristic, which predicts that a recognized object scores higher on some criterion than an unrecognized one, is a simple inference strategy and thus an attractive mental tool for making inferences with limited cognitive resources-for instance, in old age. In spite of its simplicity, the recognition heuristic might be negatively affected in old age by too much knowledge, inaccurate memory, or deficits in its adaptive use. Across 2 studies, we investigated the impact of cognitive aging on the applicability, accuracy, and adaptive use of the recognition heuristic. Our results show that (a) young and old adults’ recognition knowledge was an equally useful cue for making inferences about the world; (b) as with young adults, old adults adjusted their use of the recognition heuristic between environments with high and low recognition validities; and (c) old adults, however, showed constraints in their ability to adaptively suspend the recognition heuristic on specific items. Measures of fluid intelligence mediated these age-related constraints.

HOW TO PREDICT THE LIFETIMES OF (SOME) THINGS

In 1969, John Gott looked at the Berlin Wall and asked himself how long it would stand.

The wall was 8 years old at the time.

He could be witnessing something that would stand many years to come or something that would fall down tomorrow.

In certain domains, you can make the inference that you are observing a thing at a random point in its lifetime, and this is what Gott did.

He figured that if he’s witnessing it 50% of the way through its lifetime, then it would stand another 8 years (Because it has been up 8 years and is halfway through its life).

He figured that if he’s witnessing it 5% of the way through its lifetime, then it would stand another 152 years. (Because if 8 years is 5% of its life, then 160 years is 100% of its life. Since it has been up for 8 years, it has 152 years left).

He figured that if he’s witnessing it 95% of the way through its lifetime, then it would stand another 5 months. (Because if 8 years is 5% of its life then 8.42 years is 100% of its life, Since it has already been up 8 years, it has .42 years, or five months, left).

In this way, Gott came up with a principle for estimating confidence intervals for the lifetimes of certain classes of things. A 95% CI for the lifetime of the Berlin Wall would be 8.42 to 160 years, which contains the age of the wall (about 28 years) when it came down.

Gott’s principle was also used to predict the predict the closing dates of 44 Broadway and Off-Broadway shows, and was about 95% correct.

So there you have it, a heuristic for predicting the lifetimes of things. It doesn’t apply everywhere (e.g., it doesn’t work on human lives), but it’s kind of fun.

REFERENCES
Gott, J.R. (1993). Implications of the Copernican principle for our future prospects. Nature, 363, 315–319.

Gott, J.R. (1994). Future prospects discussed. Nature, 368, 108.

Photo credit: http://www.flickr.com/photos/siyublog/1982035178/