[ View menu ]

May 9, 2018

The CREATE (Crowdsourcing Evidence, Argumentation, Thinking and Evaluation) program

Filed in Ideas ,Research News
Subscribe to Decision Science News by Email (one email per week, easy unsubscribe)


IARPA (the Intelligence Advanced Research Projects Activity) seeks thousands of people to test the analytic methods developed for the CREATE (Crowdsourcing Evidence, Argumentation, Thinking and Evaluation) program.

CREATE is a multi-million-dollar, multi-year R\&D effort to improve the core process of intelligence analysis for national security: making well-reasoned inferences from incomplete information. If effective, CREATE analytic tools could greatly assist other disciplines that depend on good reasoning, including law, medicine, and public policy.

The Create Better Reasoning Study seeks people to test innovative reasoning and problem solving tools. Ideal participants like to Learn new tools and techniques to evaluate and share evidence Develop well-reasoned, evidence-based solutions to complex problems Explain their reasoning clearly and convincingly

Participants will spend 2-3 hours/week for 9 weeks working alone or in teams to discuss, develop and document solutions to challenging problems. Along the way, you will learn and use special software that helps with CRowdsourcing Evidence, Analysis, Thinking and Evaluation (CREATE).

You can join CREATE if you
Are 18 or older
Have at least some college education
Read and write English fluently
Have reliable internet access and will accept cookies from the study website

Your contribution could shape the future of intelligence and problem solving tools.

Ready to Join?

Here is the signup page.

May 1, 2018

Nineteen vs twenty

Filed in Ideas
Subscribe to Decision Science News by Email (one email per week, easy unsubscribe)


We were getting an egg and cheese on a roll at a deli the other day and started chatting with the cashier about politics and he said that two people were like 19 vs 20.

We said, “19 vs 20, what’s that mean?”

He said, “It means basically the same thing. Different but not really different”

We said, “Where do they say that?”

He said, “Pakistan”

As we continued walking to work, we were thinking “this is the best saying ever.”

In the psychology of sensation and perception, differences in sensation of 5% (e.g., 19 vs 20) are often not detectable. People can’t reliably tell if a rock that weighs 19 pounds is heavier or lighter than a rock that weighs 20 pounds. If the threshold for detecting differences is k, k is called the Weber fraction. Here’s are some estimated Weber fractions (as percentages) for a number of stimulus dimensions:



As we can see, in many domains, k is greater than 5%, meaning that for many things, the 19 vs 20 difference is not detectable.

Next, we wanted to see if we could find some web pages on this topic, you know, for blogging.

Searching on “19 vs 20” in English was not a successful search strategy.

We thought, well, maybe if it’s said in Pakistan, it’s said in India, too? We asked a couple of Indian colleagues in the lab and bingo, both knew the phrase. In Hindi, it’s “unees bees ka fark”:

unees means ‘nineteen’

bees means ‘twenty’

ka fark means ‘the difference between’

My colleague Chinmay Singh adds the following

There is question on Quora about this topic:

The exact words can be loosely translated as “difference of nineteen twenty”, as in the difference between two things being compared is the same as that between nineteen and twenty.

Please note there is subtle difference between this idiom and the English variant “six of one, half a dozen of the other”. The Hindi phrase acknowledges that the difference actually exists, but should be ignored because it is (statistically) insignificant. The English phrase suggests that it’s only a matter of saying it differently as the two choices are identical otherwise.


  • While we wish the Weber fraction was named for JDM researcher Elke Weber, it’s actually named for Ernst Heinrich Weber who died unfortunately in 1878.
  • The ideas of the just noticeable difference and Weber fraction are useful but false models that have been supplanted. When we wrote a term paper on the topic for Stephen Stigler’s History of Statistics class, we learned about the 1885 attack on the just noticeable difference (unterschiedsschwelle) by Charles Sanders Peirce and Joseph Jastrow.
  • As the image above shows, “nineteen twenty” is the name of an awesome looking movie as well.

April 25, 2018

Foundations of Utility and Risk Conference June 25-28, 2018

Filed in Conferences
Subscribe to Decision Science News by Email (one email per week, easy unsubscribe)


The upcoming Foundations of Utility and Risk Conference (FUR 2018) is being held ​at the University of​ York, UK from Monday 25 June to Thursday 28 June.

FUR conferences were started in Oslo in 1982 by Ole Hagen and Maurice Allais, and have been held every two years since. It is a highly prestigious conference with many distinguished presenters and participants. Each year FUR invites abstracts for presentations and posters on any topic related to risk, and, more generally, decision theory.

We are writing to you now as the deadline for the second round of submissions – Tuesday the 1st of May – is approaching fast. The first round has already passed – we had almost 200 submissions and we have accepted a sizeable majority of them. The programme is filling up nicely. We will communicate decisions on the second round by the 7th of May, in time for you to register before the early registration deadline of the 14th of May (after which registrations go up from £300 to £400 (full) and from £200 to £300 (student)), and also in time for you to take advantage of pre-booked hotel rooms offered by our Registration Agency, Mosaic (available up to the 10th of May). There will be a third round of submissions opening on Thursday 8 May and closing on Monday 21 May, with decisions communicated by Thursday 31 May – but this will be too late to register at the lower fees. We advise you to submit soon.

​We have distinguished plenary speakers: Lara Buchak (UC Berkeley), Ido Erev (Technion – Israel Institute of Technology) and Ariel Rubinstein (Tel Aviv University, New York University) as well as our ‘Rising Star​’ Terri Kneeland (University College London). There will also be Round Table discussions on specialised topics chaired by: Ani Guerdjikova (University of Grenoble Alpes): Ambiguity in Dynamic Interactions: Can Economists Give Precise Advice with Imprecise Probabilities?; Paulo Natenzon (Washington University in Saint Louis): Attention: Limited?; and Wlodek Rabinowicz (Lund University): Dynamic Decisions beyond EU.

We are trying to make the conference as interdisciplinary as possible, with economics, philosophy and psychology being covered by the invited speakers. Other disciplines, related to the conference theme, will also be welcome.

You can register now for the conference, and find out more about submission of abstracts online.

April 18, 2018

What is the average temperature on earth?

Filed in Encyclopedia
Subscribe to Decision Science News by Email (one email per week, easy unsubscribe)


We were up at Yale the other week guest lecturing for Shane Frederick, who always hits us up with trivia questions. Here’s one question he asked us:

What’s the average temperature on the earth?

That is, what’s the average surface temperature over all places on the earth (inhabited or not), over all times of day, over all days of the year. Roughly speaking, what’s the average value on a graph like this? [Y axis values are temperatures that we’ve hidden]

Spoiler alert: If you want to see the Y-axis values for the graph, click on it.

As the average temperature is changing and fuzzy statistic, so we’ll accept a rough guess via a multiple choice item.

Which of A – J represents the average surface temperature on the earth (in Fahrenheit)?
A) 36
B) 43
C) 50
D) 57
E) 64
F) 72
G) 79
H) 86
I) 93
J) 100

Or if you are more comfortable with Celsius:
A) 2
B) 6
C) 10
D) 14
E) 18
F) 22
G) 26
H) 30
I) 34
J) 38

Want the answer? Go to this page and scroll down to the 2nd to last line, where it says “For the global mean, the most trusted models produce a value of roughly …”

As insurance against that page moving, we’ll leave an Internet Archive link here.

April 11, 2018

Pre-Conference on Digitized Behavior at BDRM, Boston, June 7, 2018

Filed in Conferences
Subscribe to Decision Science News by Email (one email per week, easy unsubscribe)


There will be a pre-conference on Digitized Behavior at Boston University, right before the 2018 BDRM conference.

Each talk examines how digitization changes behavior and business.

Adrian Ward, University of Texas at Austin
Alix Barasch, New York University
Andrey Fradkin, Massachusetts Institute of Technology
Avi Goldfarb, University of Toronto
Berkeley Dietvorst, University of Chicago
Daniella Kupor, Boston University
Eva Buechel, University of South Carolina
Fleura Bhardi, City University of London
Frederic Brunel, Boston University
Iyad Rahwan, Massachusetts Institute of Technology
Kostadin Kushlev, University of Virginia
Leslie John, Harvard Business School
Molly Crockett, Yale University
Sandra Matz, Columbia University
Todd Rogers, Harvard Kennedy School
Veronica Marotta, Carnegie Mellon University

The preconference runs from 9am to 4pm on Thursday, June 7th, 2018 in the auditorium of the Questrom School of Business, 595 Commonwealth Ave, Boston, MA 02215.

Lunch will be provided. How cool is that!?

Registration is free while space is available. You can register now here.

The preconference is organized by the Marketing Department at the Questrom School of Business. Please direct questions to nishibun at bu.edu.

April 3, 2018

How large is the great Pacific garbage patch?

Filed in Encyclopedia ,Ideas
Subscribe to Decision Science News by Email (one email per week, easy unsubscribe)


This Nature Scientific Reports article gives an update on the size of the Great Pacific Garbage Patch. It is estimated to be four to 16 times larger than previously estimated: 1.6 million square kilometers.

While the article does show the patch along with latitude and longitude lines (above) and the Hawaiian islands, it doesn’t provide much help communicating how large it is with familiar reference objects. (Incidentally, our research has found that people seriously underestimate the size of Hawaii).

How large is 1.6 million square kilometers? It’s about

  • As big as Alaska
  • One-fifth as big as the contiguous United States
  • Half as big as India
  • As big as Iran

Open to suggestions for how to put into perspective the estimated mass of the plastic: 80,000 metric tons.


Lebreton et al. (2018). Evidence that the Great Pacific Garbage Patch is rapidly accumulating plastic. Scientific Reports.

h/t Stefano Puntoni

March 29, 2018

The logic of the talking dog joke

Filed in Ideas
Subscribe to Decision Science News by Email (one email per week, easy unsubscribe)


We were looking up the wording of a joke we wanted to tell our kids:

A guy has a talking dog. He brings it to a talent scout. “This dog can speak English,” he claims to the unimpressed agent. “Okay, Sport,” the guys says to the dog, “what’s on the top of a house?” “Roof!” the dog replies. “Oh, come on…” the talent agent responds. “All dogs go ‘roof’.” “No, wait,” the guy says. He asks the dog “what does sandpaper feel like?” “Rough!” the dog answers. The talent agent gives a condescending blank stare. He is losing his patience. “No, hang on,” the guy says. “This one will amaze you. ” He turns and asks the dog: “Who, in your opinion, was the greatest baseball player of all time?” “Ruth!” goes the dog. And the talent scout, having seen enough, boots them out of his office onto the street. And the dog turns to the guy and says “Maybe I shoulda said DiMaggio?”

Hilarious, right? You’re welcome. For some reason, we remember this being told by Gabe Kaplan in an episode of Welcome Back Kotter but we can’t find evidence of this online.

The reason for this post, however, was the web page we found the wording on. Professor of Computer Science / Cognitive Science professor Justin Li thinks about the joke in terms of the Wason selection task. Have a look.

It does seem the talent agent is wary of the reasoning fallacy called affirming the consequent, the Wikipedia example of which is:

If Bill Gates owns Fort Knox then Bill Gates is rich.
Bill Gates is rich
Ergo, Bill Gates owns Fort Knox

In the context of this joke, you don’t want to reason

If the dog can speak English, the dog can answer the questions
The dog can answer the questions
Ergo, the dog can speak English

March 22, 2018

Payments to authors based on the journals in which they publish

Filed in Gossip ,Ideas ,Research News
Subscribe to Decision Science News by Email (one email per week, easy unsubscribe)


Table 5 Comparison of Average Amount of Cash Awards for a Paper Published in Selected Journals

When we first saw this image on Facebook, we thought it was a joke. But then we downloaded the paper and found that indeed, there’s a practice of paying authors based on the journal that publishes the paper.

“In summary, Chinese universities differentiate the amount of cash reward based on the JIF and JCR Quartile of journals in which the awarded papers are published. The average amount of cash award has increased over the past 10 years, except that the amount awarded to papers published in journals with low JIF has decreased. Publications in Nature and Science are awarded the largest amount of cash reward.” – Page 13

We’re now waiting for someone to write a web app that scrapes CVs and puts dollar values on them.

Quan, Wei, Bikun Chen, and Fei Shu (working paper). Publish or impoverish: An investigation of the monetary reward system of science in China (1999-2016).

h/t Andreas Ortmann

March 15, 2018

2018 Consumer Financial Protection Bureau Research Conference

Filed in Conferences
Subscribe to Decision Science News by Email (one email per week, easy unsubscribe)


On May 3–4, 2018, the CFPB will host its third research conference on consumer finance at the Crystal Gateway Marriott in Arlington, Va.

The goal of the conference is to highlight research on the topic of consumer finance that can inform researchers and policymakers. The conference will focus on high-quality consumer finance research, with academic and government researchers presenting their research papers.

Crystal Gateway Marriott
1700 Jefferson Davis Hwy
Arlington, Va. 22202

Conference agenda


For logistical details and information about past conferences, please visit the conference webpage

March 8, 2018

Sweater weather

Filed in Ideas
Subscribe to Decision Science News by Email (one email per week, easy unsubscribe)


We came across this weather.com story on what people consider “sweater weather” in different states, and thought that the topic seemed just silly enough to be of interest to decision science news readers.

Not much value we can add here. How about some stats?

Mean sweater weather: 58.6 F (14.8 C)
Max sweater weather: Arizona, Nevada at 65 F (18.3 C)
Min sweater weather: South Dakota at 51 F (10.6 C)
Standard deviation: 2.8 F (1.6 C)


New Jersey,60,15.6
North Carolina,60,15.6
South Carolina,60,15.6
West Virginia,60,15.6
New Mexico,59,15
New York,58,14.4
Rhode Island,58,14.4
New Hampshire,55,12.8
North Dakota,55,12.8
South Dakota,51,10.6