{"id":399,"date":"2008-08-14T17:18:25","date_gmt":"2008-08-14T16:18:25","guid":{"rendered":"http:\/\/www.decisionsciencenews.com\/?p=399"},"modified":"2008-08-21T11:52:29","modified_gmt":"2008-08-21T10:52:29","slug":"mind-your-ps","status":"publish","type":"post","link":"https:\/\/www.decisionsciencenews.com\/?p=399","title":{"rendered":"Mind your Ps"},"content":{"rendered":"<p style=\"text-align: left;\">GET A FEELING FOR ACCIDENTAL SIGNIFICANCE<\/p>\n<p style=\"text-align: center;\"><a href=\"http:\/\/www.jerrydallal.com\/LHSP\/multtest.htm\" target=\"_blank\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-400 aligncenter\" title=\"20pvals\" src=\"http:\/\/www.decisionsciencenews.com\/wp-content\/uploads\/2008\/08\/20pvals.png\" alt=\"\" width=\"420\" height=\"333\" \/><\/a><\/p>\n<p style=\"text-align: center;\"><em>Click through to site to try<br \/>\n<\/em><\/p>\n<p style=\"text-align: left;\">We were exploring <a href=\"http:\/\/www.jerrydallal.com\/\" target=\"_blank\">Jerry Dallal&#8217;s site<\/a> and came across this <a href=\"http:\/\/www.jerrydallal.com\/LHSP\/multtest.htm\">cute gizmo<\/a> linked to as &#8220;a valuable lesson&#8221;.<\/p>\n<p style=\"text-align: left;\">Clicking the button simulates running 20 significance tests, each of which has a 5% chance of coming up significant when no effect is present. Underneath Jerry writes, &#8220;The chance that nothing is significant is only 0.3585, so don&#8217;t give up hope!&#8221;<\/p>\n<p style=\"text-align: left;\">This is probably a good thing to show to new grad students, whom I suspect get a bit too excited over significant results. We wonder how long it takes new scientists to realize that all that glitters is not meaningful.\u00a0 Simply administering, say, the same 30 question survey to 4 different randomly-assigned groups should be enough to teach this lesson, so one would think that researchers ought to learn this quickly.<\/p>\n<p style=\"text-align: left;\">We notice that seasoned researchers, who are generally comfortable dismissing insignificant significance, fall into two camps. The first camp waves the results away as noise. The second camp believes that there was an underlying effect, but dismisses it as stemming from an ignore-worthy flaw in the design: &#8220;we&#8217;re seeing this because that group answered first thing in the morning&#8221;, &#8220;&#8230;right after lunch&#8221;, &#8220;&#8230;on Monday&#8221;, &#8220;&#8230;at the end of class&#8221;, etc. They never seem to say &#8220;we&#8217;re seeing this because a bunch of people who\u00a0 answer that way got randomly put in that group&#8221;.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>GET A FEELING FOR ACCIDENTAL SIGNIFICANCE Click through to site to try We were exploring Jerry Dallal&#8217;s site and came across this cute gizmo linked to as &#8220;a valuable lesson&#8221;. Clicking the button simulates running 20 significance tests, each of which has a 5% chance of coming up significant when no effect is present. Underneath [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false}}},"categories":[4,6,2,15],"tags":[],"class_list":["post-399","post","type-post","status-publish","format-standard","hentry","category-encyclopedia","category-profiles","category-research-news","category-tools"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p4LKj-6r","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=\/wp\/v2\/posts\/399","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=399"}],"version-history":[{"count":13,"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=\/wp\/v2\/posts\/399\/revisions"}],"predecessor-version":[{"id":406,"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=\/wp\/v2\/posts\/399\/revisions\/406"}],"wp:attachment":[{"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=399"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=399"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=399"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}