{"id":4026,"date":"2013-04-02T10:03:30","date_gmt":"2013-04-02T14:03:30","guid":{"rendered":"http:\/\/www.decisionsciencenews.com\/?p=4026"},"modified":"2013-04-02T10:03:30","modified_gmt":"2013-04-02T14:03:30","slug":"who-would-have-thought-turkers-could-do-this","status":"publish","type":"post","link":"https:\/\/www.decisionsciencenews.com\/?p=4026","title":{"rendered":"Who would have thought Turkers could do this?"},"content":{"rendered":"<p>THE LUCID PERCEPTION OF THE CROWD<\/p>\n<p style=\"text-align: center;\"><a href=\"http:\/\/www.decisionsciencenews.com\/wp-content\/uploads\/2013\/04\/writing-1-distort-2.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-4027\" alt=\"writing-1-distort-2\" src=\"http:\/\/www.decisionsciencenews.com\/wp-content\/uploads\/2013\/04\/writing-1-distort-2.png\" width=\"485\" height=\"388\" \/><\/a><\/p>\n<p>This is one of the most amazing things we&#8217;ve seen all week. What does the text block above say? You have no idea, right?<\/p>\n<p>The folks running the <a href=\"http:\/\/groups.csail.mit.edu\/uid\/deneme\/\">Deneme <\/a>blog over at MIT gave this task to experimental participants in Amazon&#8217;s <a href=\"http:\/\/www.decisionsciencenews.com\/2010\/10\/18\/a-guide-to-conducting-behavioral-research-on-amazons-mechanical-turk\/\">Mechanical Turk<\/a> labor market. By doing it in an iterative way, people <a href=\"http:\/\/groups.csail.mit.edu\/uid\/deneme\/?p=329\">were able to transcribe the blurry mess to English with only one mistake<\/a>.<\/p>\n<p>The iterative method is as follows:<\/p>\n<p>* One person gives the transcription a try<br \/>\n* The next person sees the previous tries and tries to improve upon it<br \/>\n* Repeat until done<\/p>\n<p>(We&#8217;re not sure if the workers can see all the previous attempts or just the last one).<\/p>\n<p>This is what the iterative method came up with:<\/p>\n<blockquote><p>I had intended to hit the nail, but I\u2019m not a very good aim it seems and I ended up hitting my thumb. This is a common occurence I know, but it doesn\u2019t make me feel any less ridiculous having done it myself. My new strategy will involve lightly tapping the nail while holding it until it is embedded into the wood enough that the wood itself is holding it straight and then I\u2019ll remove my hand and pound carefully away. We\u2019ll see how this goes.<\/p><\/blockquote>\n<p>&#8230;and the only error was that &#8217;embedded&#8217; should have been &#8216;wedged&#8217;.<\/p>\n<p>Remarkably, for the passage above, people got it. And in a non-iterative version, in which each person had no help from previous workers, everyone gave up. <a href=\"http:\/\/groups.csail.mit.edu\/uid\/deneme\/wp-content\/uploads\/2009\/08\/results1.html\">Details are here<\/a>. That said in <a href=\"http:\/\/groups.csail.mit.edu\/uid\/deneme\/wp-content\/uploads\/2009\/08\/results2.html\">two<\/a> <a href=\"http:\/\/groups.csail.mit.edu\/uid\/deneme\/wp-content\/uploads\/2009\/08\/results3.html\">other<\/a> texts, neither the iterative or non-iterative method worked, but of course they only tried once per text.<\/p>\n<p>We are curious to know how they&#8217;ve improved upon this very promising method.<\/p>\n<p>P.S. We&#8217;ve found this task to be slightly easier when you scroll the blurry text up and down a bit in the browser. Try it on the <a href=\"http:\/\/groups.csail.mit.edu\/uid\/deneme\/?p=329\">full sized image on the original blog post<\/a>.<\/p>\n<p>P.P.S. When you iterate but only look at the last attempt (instead of the original plus the last attempts) things get worse, as in the game of telephone. Check out this cool example of how a <a href=\"http:\/\/boingboing.net\/2011\/02\/18\/straight-line-traced.html\">straight line morphs into craziness<\/a> by error propagation.<\/p>\n<p style=\"text-align: center;\"><a href=\"http:\/\/www.decisionsciencenews.com\/wp-content\/uploads\/2013\/04\/before.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-4030\" alt=\"before\" src=\"http:\/\/www.decisionsciencenews.com\/wp-content\/uploads\/2013\/04\/before.png\" width=\"480\" height=\"412\" \/><\/a><\/p>\n<p style=\"text-align: center;\"><a href=\"http:\/\/www.decisionsciencenews.com\/wp-content\/uploads\/2013\/04\/after.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-4031\" alt=\"after\" src=\"http:\/\/www.decisionsciencenews.com\/wp-content\/uploads\/2013\/04\/after.png\" width=\"480\" height=\"411\" \/><\/a><\/p>\n<p>P.P.P.S. I remember my dad, a psychology professor, showing me results of a line copying experiment he did on his students at Carnegie Mellon in the 1970s. He passed a deck of index cards around the class. Each student looked at the image on the top of the deck, moved it to the bottom, and tried to reproduce it on the blank card on top, and then passed it on. The results were similarly crazy.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This is one of the most amazing things we&#8217;ve seen in a while. What does the text block above say? You have no idea, right?<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false}}},"categories":[16,2],"tags":[538,544,312,543,72,532,540,536,539,542,36,545,546,537,541],"class_list":["post-4026","post","type-post","status-publish","format-standard","hentry","category-ideas","category-research-news","tag-amazon","tag-blurry","tag-cognition","tag-deneme","tag-error","tag-experiments","tag-iterative","tag-mechanical","tag-methods","tag-mit","tag-psychology","tag-text","tag-transcription","tag-turk","tag-turkit"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p4LKj-12W","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=\/wp\/v2\/posts\/4026","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=4026"}],"version-history":[{"count":10,"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=\/wp\/v2\/posts\/4026\/revisions"}],"predecessor-version":[{"id":4039,"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=\/wp\/v2\/posts\/4026\/revisions\/4039"}],"wp:attachment":[{"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=4026"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=4026"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.decisionsciencenews.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=4026"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}