{"id":3583,"date":"2012-08-29T13:43:34","date_gmt":"2012-08-29T17:43:34","guid":{"rendered":"http:\/\/www.decisionsciencenews.com\/?p=3583"},"modified":"2012-09-01T14:33:38","modified_gmt":"2012-09-01T18:33:38","slug":"the-texas-sharpshooter-story","status":"publish","type":"post","link":"https:\/\/www.decisionsciencenews.com\/?p=3583","title":{"rendered":"The Texas sharpshooter story"},"content":{"rendered":"

TRICKING PEOPLE INTO THINKING YOUR SCIENTIFIC SHOTS NEVER MISS<\/p>\n

\n\"\"<\/a>\n<\/p>\n

You visit the farm of a Texan, Joe, who claims to be a sharpshooter. When walking past his barn, you see a chalk target drawn on the wall with a bunch of tightly-grouped bullet holes in the bullseye. After observing that Joe can’t shoot well at all, you realize that he drew the bullseye after firing the shots.<\/p>\n

The tale of the Texas sharpshooter resonates with JDM (Judgment and Decision-Making) research on perceiving illusory patterns, and a topic of recent interest, detecting bogus experimental results. In a recent paper<\/a>, Ulrich Schimmack talks about multi-study research papers in this way. When you see 10 studies in a single paper that confirm a hypothesis, can you conclude that the basic effect is replicable and robust?<\/p>\n

One problem in science is that reading a research article is a bit like visiting Joe’s farm. Readers only see the final result, without knowing how the final results were created. Is Joe a sharpshooter who drew a target and then fired 10 shots ar the target? Or was the target drawn after the fact?
\n-Schimmack, in press<\/p><\/blockquote>\n

We have been looking into the roots of the Texas Sharpshooter vignette in academic writing. The earliest and most common “initial” cite we found after a quick search was Grufferman’s from 1977, with no claims that this is the earliest use:<\/p>\n

Here it is.<\/p>\n

There have been several dramatic time-space clusters of leukemia reported in which, following an initial observation of two or more cases in a locality, a time unit and geographical area are selected so as to best define a time-space cluster. Such a posteriori clusters are analogous to the story of the Texas sharpshooter who would shoot his rifle at the side of a barn and then carefully draw a target around each bullet-hole so that each bullet-hole passed exactly through the center of the “bull\u2019s-eye.” Although a posteriori clusters do serve to demonstrate that cases can cluster in time and space, they do not allow for determining whether this is more than a chance occurrence.<\/p>\n

-Grufferman (1977)<\/p><\/blockquote>\n

REFERENCES
\nGrufferman S. (1977). Clustering and aggregation of exposures in Hodgkin’s disease. Cancer 39, 1829-1833<\/p>\n

Schimmack, U. (in press). The Ironice Effect of Significant Results on the Credibility of Multiple Study Articles. Psychological Methods.<\/p>\n

Photo credit: http:\/\/www.flickr.com\/photos\/24730945@N03\/4130123404\/sizes\/l\/<\/font><\/p>\n","protected":false},"excerpt":{"rendered":"

TRICKING PEOPLE INTO THINKING YOUR SCIENTIFIC SHOTS NEVER MISS You visit the farm of a Texan, Joe, who claims to be a sharpshooter. When walking past his barn, you see a chalk target drawn on the wall with a bunch of tightly-grouped bullet holes in the bullseye. After observing that Joe can’t shoot well at […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false}}},"categories":[5,16],"tags":[433,20,386,432,82,203,431],"class_list":["post-3583","post","type-post","status-publish","format-standard","hentry","category-articles","category-ideas","tag-bias","tag-decision-making","tag-effect","tag-fallacy","tag-jdm","tag-judgment","tag-texas-sharpshooter"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p4LKj-VN","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=\/wp\/v2\/posts\/3583","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3583"}],"version-history":[{"count":14,"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=\/wp\/v2\/posts\/3583\/revisions"}],"predecessor-version":[{"id":3607,"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=\/wp\/v2\/posts\/3583\/revisions\/3607"}],"wp:attachment":[{"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3583"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=3583"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=3583"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}