Who would have thought Turkers could do this?
Subscribe to Decision Science News by Email (one email per week, easy unsubscribe)
THE LUCID PERCEPTION OF THE CROWD
This is one of the most amazing things we’ve seen all week. What does the text block above say? You have no idea, right?
The folks running the Deneme blog over at MIT gave this task to experimental participants in Amazon’s Mechanical Turk labor market. By doing it in an iterative way, people were able to transcribe the blurry mess to English with only one mistake.
The iterative method is as follows:
* One person gives the transcription a try
* The next person sees the previous tries and tries to improve upon it
* Repeat until done
(We’re not sure if the workers can see all the previous attempts or just the last one).
This is what the iterative method came up with:
I had intended to hit the nail, but I’m not a very good aim it seems and I ended up hitting my thumb. This is a common occurence I know, but it doesn’t make me feel any less ridiculous having done it myself. My new strategy will involve lightly tapping the nail while holding it until it is embedded into the wood enough that the wood itself is holding it straight and then I’ll remove my hand and pound carefully away. We’ll see how this goes.
…and the only error was that ’embedded’ should have been ‘wedged’.
Remarkably, for the passage above, people got it. And in a non-iterative version, in which each person had no help from previous workers, everyone gave up. Details are here. That said in two other texts, neither the iterative or non-iterative method worked, but of course they only tried once per text.
We are curious to know how they’ve improved upon this very promising method.
P.S. We’ve found this task to be slightly easier when you scroll the blurry text up and down a bit in the browser. Try it on the full sized image on the original blog post.
P.P.S. When you iterate but only look at the last attempt (instead of the original plus the last attempts) things get worse, as in the game of telephone. Check out this cool example of how a straight line morphs into craziness by error propagation.
P.P.P.S. I remember my dad, a psychology professor, showing me results of a line copying experiment he did on his students at Carnegie Mellon in the 1970s. He passed a deck of index cards around the class. Each student looked at the image on the top of the deck, moved it to the bottom, and tried to reproduce it on the blank card on top, and then passed it on. The results were similarly crazy.
Or you could just zoom out (ctrl minus on your keyboard) a handful of times and read the text in the image pretty clearly immediately. No Turks needed!
April 2, 2013 @ 10:22 am
Phil – Or you could just stand up and take a few steps back from the monitor. Same effect.
April 2, 2013 @ 10:34 am
CAPTHGHA: “Completely Automated Public Turing test to tell a Human and Groups of Humans Apart”
April 2, 2013 @ 10:38 pm
A slightly different twist on the same sort of idea is to get a group of people to use one finger only to lift a meter-long piece of wood and keep it on a level. It usually ends up accelerating towards the ceiling pretty rapidly as people try to ‘compensate’ their end of the piece of wood. I’m sure there is something profound to draw from this, but most of all it’s fun to try!
April 3, 2013 @ 8:23 am
I have no idea how you guys are reading this by simply standing back from the monitor or reducing the size. It’s still mostly unintelligible to me. Did you find it that easy after you knew what the text was? I can see it if I know what’s there but without previous knowledge I wouldn’t have a clue.
What’s also interesting about this is how much the first pass must matter. With this much text, one can start to glean the words from the context – the exact opposite of the game of Telephone, where without context you’ve only got what you thought you heard to go on.
April 3, 2013 @ 8:28 pm
Also interesting, the one mistake actually improves on the original.
April 3, 2013 @ 8:29 pm
Is this an aprils fool joke?
April 4, 2013 @ 7:24 am
In my note card experiment I showed the students the effects of “leveling” and “sharpening” in rumor transmission: details thought to be unimportant tend to drop out and details thought to be important become exaggerated. Thus an accidental partial thumb print became an elaborate “grid” pattern. Also, there is an anchoring effect: when the first card is simply a dot in dead center slight movement away from the center in subsequent iterations tended to make the final card show it migrating toward a corner–an “anchor” point.
April 10, 2013 @ 12:53 pm