\documentclass[twocolumn]{article}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage[english]{babel}
\usepackage{amsmath,amsthm,amssymb,amsfonts,newtxtext,newtxmath} 
\usepackage{array,graphicx,dcolumn,multirow,hevea,abstract,hanging}
\usepackage[labelfont=sc,textfont=sf]{caption}
\usepackage[hyperfootnotes=false,breaklinks=true]{hyperref} % was dvipdfmx
\urlstyle{rm}
\usepackage[hyphenbreaks]{breakurl}
\usepackage{booktabs} % \toprule \midrule \bottomrule \cmidrule(lr){a-b}
% define centered and ragged columns:
\newcolumntype{L}[1]{>{\raggedright\arraybackslash }p{#1}} % can use m{}
\newcolumntype{C}[1]{>{\centering\arraybackslash }p{#1}}
\newcolumntype{R}[1]{>{\raggedleft\arraybackslash }p{#1}}
\newcolumntype{d}[1]{D{.}{.}{#1}} % d{3.2} for 3 places on l, 2 on r
\newcommand{\mc}{\multicolumn}
\topmargin=-.3in \oddsidemargin=-.1in \evensidemargin=-.1in \textheight=9in \textwidth=6.8in
\setlength\tabcolsep{1mm}
\setlength\columnsep{5mm}
\setlength\abovecaptionskip{.5ex}
\setlength\belowcaptionskip{.5ex}
\setlength\belowbottomsep{.3ex}
\setlength\lightrulewidth{.04em}
\renewcommand\arraystretch{1.2}
\renewcommand{\baselinestretch}{1.00} \large\normalsize 
\widowpenalty=1000
\clubpenalty=1000
\setlength{\parskip}{0ex}
\let\tempone\itemize
\let\temptwo\enditemize
\let\tempthree\enumerate
\let\tempfour\endenumerate
\renewenvironment{itemize}{\tempone\setlength{\itemsep}{-5pt}}{\temptwo}
\renewenvironment{enumerate}{\tempthree\setlength{\itemsep}{-5pt}}{\tempfour}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\setcounter{page}{314} % start with first page

\title{Perceptions of water systems}

\author{
Shahzeen Z. Attari\thanks{School of Public and Environmental Affairs, Indiana
University Bloomington, 1315 East Tenth Street, Bloomington, IN
47405. Email: sattari@indiana.edu.}
\and
Kelsey Poinsatte-Jones\thanks{School of Public and Environmental Affairs, Indiana
University Bloomington}
\and
Kelsey Hinton\footnotemark[2]
}

\date{} % leave empty
\begin{document} % goes here

% change next 3 lines each issue
\newcommand{\jref}{http://journal.sjdm.org/vol12.3.html}
\newcommand{\jhead}{Judgment and Decision Making, Vol.~12, No.~3}
\newcommand{\jdate}{May 2017}
\pagestyle{myheadings} \markright{\protect\small \href{\jref}{\jhead},
  \jdate \hfill Perceptions of water systems \qquad}
\begin{htmlonly}
\href{\jref}{\jhead}, \jdate, pp.\ 314-327
\end{htmlonly}
%\begin{latexonly}
\twocolumn[
\vspace{-.5in}
{\small \href{\jref}{\jhead}, \jdate, pp.\ 314--327}
%\end{latexonly}

\maketitle

%\begin{latexonly}
\vspace{-3mm}
\begin{onecolabstract}
%\end{latexonly}
  Public understanding of the water system is vital in confronting
  contemporary water challenges, as public support is necessary for
  implementing measures to address shortages and repair
  infrastructure.  In this study, university student participants (N =
  457) were asked to \textit{draw} diagrams illustrating how water
  reaches the tap in an average home in the U.S. and is then returned
  to the natural environment. We also conducted an expert elicitation
  (N = 15) to create a simplified, accurate diagram by which to code
  each student drawing. Results showed major gaps in understanding,
  where 29\% of the student participants did not draw a water
  treatment plant, 64\% did not draw a wastewater treatment plant, and
  1 in 5 participants depicted untreated wastewater returning to the
  natural environment. For the majority of non-environmental students,
  the water system stops at the home. These gaps reveal a critical
  area for public environmental education efforts.

\smallskip
\noindent
Keywords: systems thinking, drinking water, student drawings,
expert elicitation, folk science
%\begin{latexonly}
\end{onecolabstract}\bigskip
]
%\end{latexonly}

{\renewcommand{\thefootnote}{}

\footnotetext{

This research was supported by funding from Indiana University’s School
of Public and Environmental Affairs (SPEA) and Indiana University’s
Office of Sustainability. We thank each of our fifteen water experts
for taking part in the expert elicitation: Majed Akhter, Rachel Atz,
Stephen (Chip) Glaholt, Henk Haitjema, Chad Jafvert, John Langley, Mark
Menefee, Kimberly Novick, Flynn Picardal, Linda Prokopy, Amisha Shah,
Joseph Shaw, Todd Royer, Adam Ward, and Jeffery White. We thank each of
our participants and thank Carissa Knox for help with this project and
Ashley Clark for comments on the survey. Finally we are grateful to
David Krantz, David Landy, Tyler Marghetis, Elisabeth Andrews and
reviewers for comments that have improved this paper.

Copyright: \copyright\ 2017.
The authors license this article under the terms of the
\href{http://creativecommons.org/licenses/by/3.0/}{Creative Commons
  Attribution 3.0 License.}
}}

\saythanks

\section{Introduction}

As climate change increases the likelihood of competition for water
among agricultural, municipal, and environmental users, the need to
confront growing risks related to water quality, quantity, and an
increasingly fragile infrastructure becomes more urgent (Famiglietti,
2014; Gleick, 1993; Kundzewicz et al., 2008; Melillo, 2014; U.S. Global
Change Program, 2014). Adapting to these risks requires political will
and public support for strategies such as decreasing water use,
accepting recycled water (converting wastewater into water that can be
used for other purposes), and paying for infrastructure improvements.
In this study, we investigate how university students think about the
entire system of water supply, delivery, and treatment in order to
uncover any prevailing misperceptions about how this vital and complex
system functions. Understanding how these young adults coarsely
interpret the complex reality of the water system is an important step
for environmental education, and can help identify ways to protect our
water system without overloading cognition.

Although functional and effective water systems are taken for granted
(Lund, 2015), many recent events have placed water quality, quantity,
and infrastructure at risk. For example, in January 2014, an industrial
solvent spilled into West Virginia’s Elk river and contaminated 15\% of
the state’s potable water, causing a “do not use” order on residential
water for ten days (Whelton et al., 2015). Related to quantity,
California’s Governor Brown issued an emergency executive order on
April 2015 mandating a 25\% statewide reduction in urban potable water
use to address diminishing water supplies caused by the ongoing drought
(Brown, 2015). The American Society of Civil Engineers gave a D+ grade
to the U.S. for the state of its infrastructure (ASCE, 2017), and the
U.S. Environmental Protection Agency (EPA) estimates that approximately
\$130 million is needed to maintain existing water infrastructure (EPA,
2003; Tanellari, Bosch, Boyle \& Mykerezi, 2015). Repairing the aging
water infrastructure should be, according to the EPA, “one of our
Nation’s top water priorities” (EPA, 2012a).

To address these increased risks, different approaches to water
management are being considered, such as reusing wastewater to augment
freshwater sources (Po, Nancarrow \& Kaercher, 2003; Rozin, Haddad,
Nemeroff \& Slovic, 2015; Vörösmarty et al., 2010; Wester et al.,
2015). Public perception is a significant barrier to implementing
wastewater reuse for potable water on a municipal level. Some potential
users report feelings of disgust, i.e. the “yuck factor,” associated
with what is referred to as “toilet to tap” water (Rozin et al., 2015).
These users likely do not understand that communities with surface
water sources are already participating in this cycle: upstream
communities discharge their treated wastewater into nearby creeks,
rivers, and lakes, which then become part of the water supply for
downstream communities (EPA, 2012b; Hartley, 2006). Another potential
strategy for addressing shortages is changing water prices to reflect
water scarcity, which can effectively decrease urban water use
(Olmstead \& Stavins, 2009), but public acceptance for such policies is
also lacking.

To understand how residential water users think about water delivery,
we take an approach that utilizes mental models and systems thinking.
While some research communities use these terms to explain cognitive
processes, and others apply them to the understanding of system
dynamics (Forrester, 1961; Meadows \& Wright, 2008; Sterman \&
Sweeney, 2002), we use mental models and systems thinking to represent
participant understanding of how a system works.

A large body of research uses structured interviews to elicit mental
models from experts and novices to see where there are gaps in
understanding (Morgan, Fischhoff, Bostrom \& Atman, 2002), and these
methods have been used in a variety of environmental domains. For
example, Bostrom, Morgan, Fischhoff \& Read (1994) and Read et al.\
(1994) highlighted that participants in their initial studies of
mental models of climate change tended to confuse stratospheric ozone
depletion with the greenhouse effect and weather with climate. Their
follow-up work found that some of the initial misperceptions were
corrected over time, such as participants not mentioning ozone
depletion as a direct cause of climate change (Reynolds, Bostrom, Read
\& Morgan, 2010).

Systems thinking research has also been used to investigate resource
conservation and understanding. Kempton (1986) investigated mental
models of how thermostats work, and found that 25-50\% of Americans are
“valve theorists” (thermostat controls the amount of heat) rather than
the more accurate “feedback theorists” (thermostat senses the
temperature and turns the furnace on or off to maintain an even
temperature). This inaccuracy can lead to significant increases in
energy use, as valve theorists are inclined to use a higher heat
setting to heat up a house faster, an inefficient practice. The work of
Gentner and Gentner (1982), tested how using different analogical
models for electrical circuits (as flowing fluids or crowds of moving
objects) could improve participant performance on solving electricity
problems. Thus, identifying lay mental models can reveal surprising
misperceptions that have implications for evidence-based policies.

How important is systems thinking for environmental education? Do
citizens need to understand the basic structure, function, and
interconnections in resource systems to make informed decisions at the
individual and societal level? Orion (2002) argues that a scientific
understanding of how these system work and are interconnected is a
necessary foundation for a citizen’s ability to contribute to
environmental preservation. Resource conservation decisions can range
from adopting effective conserving behaviors in the home (Inskeep \&
Attari, 2014) to accepting alternative price structures for resources
(Olmstead, Hanemann \& Stavins, 2007) to altering voting behavior
related to environmental management. If citizens believe that water
magically comes out of the tap and electricity from a wall socket –
without understanding, at least in some sense, the integrated system
behind the tap and wall – then, if parts of that unseen but expected
system weaken, reacting and adapting to those weaknesses could be slow
and potentially catastrophic. A recent example is the lead
contamination water crisis in Flint, Michigan, where introducing a more
corrosive water source into an aging water system without corrosion
control led to elevated blood lead levels in children (Hanna-Attisha,
LaChance, Sadler \& Champney Schnepp, 2016). Thus, without some
application of systems thinking to vital resources, identifying and
responding to system-level problems will be a formidable challenge. 

In our study we elicit water system drawings from expert and university
student participants. We use drawings rather than other means (such as
open-ended structured interviews or closed-ended surveys) to gain a
view into participants’ mental constructs. This use of participant
drawings to gauge system thinking has been previously applied to
environmental questions. Focusing on system dynamics, Sterman (2008)
asked university student participants to sketch out the emissions
trajectory that would be required to stabilize carbon dioxide
concentration. He showed that 84\% of his student participants drew
patterns of emissions that violated the principles of accumulation; if
emissions followed the modal path drawn by these students,
concentrations would continue to rise rather than stabilize. Sterman
(2008) went on to argue that the general public’s intuitive
understanding of systems dynamics (including stocks and flows) is poor,
but improving this understanding is vital for effective action on
important issues such as stabilizing atmospheric carbon dioxide
concentration. 

Closely related to the research endeavor at hand, elicited drawings
have also been used to examine how students understand the water cycle
(Assaraf, Eshach, Orion \& Alamour, 2012; Assaraf \& Orion, 2005; Bar,
1989; Dove, Everett \& Preece, 1999; Shepardson, Wee, Priddy,
Schellenberger \& Harbor, 2007a). Assaraf and Orion (2005) asked high
school juniors to draw the water cycle, finding that many students
focus exclusively on the atmospheric component of the cycle
(evaporation, condensation, and rainfall) while ignoring the
groundwater component. 

Reasons why people fail to accurately understand natural and
anthropogenic systems may include lack of everyday experience with
separate system components in addition to limited comprehension of the
complex structural, functional, and behavioral elements that interact
to form these systems (Hmelo-Silver, Marathe \& Liu, 2007). For
example, people rarely see the underground water pipeline
infrastructure for drinking water and wastewater treatment plants. Even
students who take courses related to water systems may have difficulty
comprehending the full set of processes, as systems thinking is rarely
reflected in water-related higher education textbooks. The water system
is frequently presented in these textbooks as disaggregated processes
(e.g., water treatment or wastewater treatment processes) rather than
interrelated and integrated parts of a functioning whole (J. E. Drinan
\& F. Spellman, 2012; EPA, 1991; National Research Council, 2006); see
SI Text for detailed examples). In K-12 education, water tends to
appear in science classes idiosyncratically (Gunckel, Covitt, Salinas,
\& Anderson, 2012; Sadler, Nguyen \& Lankford, 2016), with some focus
on the water cycle, but little attention to the built water system. 

Here, we move beyond investigating understanding of the water cycle and
focus instead on student understanding of the system of interconnected
processes required to deliver potable water to the home and manage that
water once it leaves the home. By eliciting drawings, we gain the
advantage of allowing participants to reflect on their mental models
and represent them non-verbally. Drawings also enable the presentation
of concepts independent from scientific terminology, allowing
participants to have more confidence in their answers (Fischer \&
Young, 2007). 

\subsection{Research questions}

Our research explores the following questions: 
\begin{enumerate}
\item How do university students’ mental models of the water system
match those of water experts? Where along the system does student
understanding fail?
\item Do environmental students have a more accurate understanding of
the water system?
\item Which risks are most salient to students – those related to water
quality, quantity, or infrastructure?
\end{enumerate}
Here we investigate these research questions by (a) constructing a
diagram of the drinking water system that reflects the consensus of
expert water scientists, (b) using the consensus expert diagram to code
students drawings of the entire drinking water system, and (c)
identifying factors that predict student accuracy in understanding the
drinking water system, including perceptions of water-related risks. 

Understanding and identifying the prevailing misperceptions related to
water systems thinking could lead to improved educational tools and
increased public support for adaptation to risks related to water
quality, quantity, and infrastructure. This information may also guide
acceptance of technologies, such as recycled water, and more effective
and efficient water pricing mechanisms to increase adaptive capacity. 

\section{Methods}

\subsection{Participants }

\textit{Expert Participants:} Between September and December of 2014,
Indiana water experts (\textit{N} = 15) were interviewed in person for
an expert elicitation. Experts included professors at Indiana
University and Purdue University as well as professionals who run
public water utilities in the state. Our participants had expertise in
the following areas: environmental engineering, water supply and
wastewater treatment, nutrient transport in water, environmental
toxicology, water quality, watershed hydrology, groundwater modeling,
water management, politics of water development, and natural resource
science. 

\textit{Student Participants:} In April 2014, students participated in
an in-class paper survey administered at Indiana University
Bloomington (\textit{N }= 578).  Email requests to conduct the survey
at the beginning of class were sent to more than 15 professors at the
University, chosen based on environmental versus non-environmental
course offerings, and in some cases, chosen based on whether the
authors knew the professors to make this request.  The courses taught
by the professors ranged from environmental science to performing
arts. The survey was then conducted in 19 separate classes. The
surveys were distributed during class within a two-week
period. Although survey responses were collected from 578
participants, only 457 participants completed the first question
(instructing them to draw a water system), thus we restrict our
analysis to this subsample. Students may have chosen not to complete
the survey because it was voluntary and uncompensated. For this study
sample (\textit{N} = 457), the median age of participants was 21 years
old, 50.1\% were female, and 9.7\% were international students. The
median level of education was a third-year undergraduate (junior) in
college. Approximately 84.6\% of the participants were at the
undergraduate level, 14.5\% were at the master’s level, and 1.1\% were
at the doctoral level. Given that this is a sample of convenience, our
figures indicate some selection bias, although the demographic
characteristics are somewhat representative of the students at the
university: 10.7\% masters, 9.7\% doctoral, and 10\% international
(Indiana University School of Education, 2014; University
Institutional Research and Reporting, 2013).

\subsection{Expert elicitation}

We examined many environmental engineering and water textbooks to
search for an accurate systems drawing by which to score the student
drawings (see this
\href{http://journal.sjdm.org/17/17124/supp.pdf}{supplement}).
None of the drawings we found were adequate for our project because
textbooks tend to focus on component processes (such as the specific
processes involved in wastewater treatment) rather than how all the
natural and human processes fit together in an integrated system
(Drinan \& Spellman, 2012). We therefore created our own drinking
water system diagram by interviewing 15 Indiana water experts. Experts
were visited in their offices or were requested to visit our lab one
at a time. Each expert was given the following instructions verbally:
\begin{quote}
Please draw a diagram illustrating your understanding of the processes
by which clean water reaches the tap in the average home in the United
States. Please draw how water reaches the home from its original
sources and is then returned to the natural environment.
\end{quote}

Diagrams were drawn on either on a piece of paper or on a white-erase
board.\footnote{ We began taking photographs of these drawings during
our meeting with the fourth expert, thus we have records of 12 expert
diagrams.} The experts were then asked to explain how their system
diagrams worked and to justify why each process was included. After the
first expert elicitation, we produced a working version of the
simplified diagram. Each of the subsequently interviewed experts were
asked to critique the simplified expert diagram at the end of their
elicitation, resulting in some changes in the level of detail in the
simplified diagram. After all the expert elicitations were complete, we
produced a final version of the simplified diagram, then re-visited two
of the experts interviewed earlier to verify the processes included in
the simplified diagram. 

\subsection{Student survey}

The first question in the paper survey prompted participants to draw a
diagram of a typical U.S. drinking water system: 
\begin{quote}
Please \textit{draw a diagram} illustrating your understanding of the
processes by which \textit{clean} water reaches the tap in the average
home in the United States. Please draw how water reaches the home from
its original source(s) and is then returned to the natural environment.
Show all of the processes that the water goes through. You are
encouraged to label your drawing and add any explanations you believe
will help convey your understanding of the water system.
\end{quote}
The rest of the page was blank, allowing participants space to draw
their diagrams. 

Following the diagram elicitation, participants were asked if they
believed there are risks associated with drinking water quality. If
participants answered yes, they were asked to identify the greatest
risk they perceived to be associated with drinking water quality. The
next two questions followed this same format, focusing on perceived
risks associated with water quantity and water infrastructure.
Participants were also asked to indicate how adequate they found
drinking water in terms of quality, quantity, and infrastructure using
a 5-point rating scale, ranging from highly inadequate to highly
adequate.

Participants were then asked whether the water they received from
their tap was generally safe or likely to contain harmful
contaminants. If participants identified their water as likely to
contain contaminants, they were asked to list any contaminants that
came to mind.  Participants were then asked to identify the frequency
with which they thought about home water use using a 5-point rating
scale: never, rarely, occasionally, a moderate amount, and a great
deal.  Participants were asked to list which activities made them most
aware of the quantity of water used in the home.

Participants were asked to characterize whether they would describe
their local water supply as safe to drink using a 5-point scale: far
below average, below average, average, above average, and far above
average. Participants were then asked to list the factors that caused
them to think about water quality in their home. 

The next set of questions assessed the information available to the
participants with regard to their local water supply. Participants were
asked how difficult it was for them to find out information about their
local water supply system, then asked to indicate whether they obtain
system information from the following sources: public utilities, news
articles, personal observation, class, word of mouth, water bill, none,
or other. Participants were also asked to indicate whether they had
observed the following system parts in the past year: water main,
storage tank, surface runoff, drinking water treatment, surface water,
reservoir, pump station, wastewater treatment, water meter, or none. 

Finally, participants were asked to identify their local drinking
water source. Demographic questions concluded the survey.  The entire
survey is presented in a \href{http://journal.sjdm.org/17/17124/finalsurvey.pdf}{supplement}.

\begin{figure*}[t!]\centering
\includegraphics[width=5in,clip=true]{fig1.eps}
\caption*{Figure 1. An expert elicited drawing of an average U.S. drinking water
system showing an integration of the natural and human system.}
\end{figure*} 

\section{Results}

\begin{figure*}[t!]\centering
\includegraphics[width=6.1in,clip=true]{fig2.eps}
\caption*{Figure 2. Simplified drinking water system diagram created
  after expert elicitation. The seven major categories for analysis
  are listed on the top of the diagram. (Note that leaks occur
  throughout the water system, but are shown only in one place on our
  simplified system diagram.)}
\end{figure*} 

\subsection{Expert elicitation and accurate systems diagram}

Experts were asked to draw their representations of the entire drinking
water system. Experts varied in their interpretations of these
instructions. Some experts focused on specific details of water
treatment and wastewater treatment, while others focused on how their
expertise related to the water system. Figure 1 shows an example of one
of the expert drawings, chosen for legibility and clarity in including
details of both natural and human processes. 

Although there were differences in how the experts interpreted the
question, the expert drawings are broadly similar and converged,
leading to the consensus diagram shown in Figure 2. One coder coded
each of the 12 expert drawings that we photographed. Ten experts
illustrated at least seven common major components of drinking water
systems: water source, drinking treatment, distribution, household
use, collection, wastewater treatment, and return to environment. One
expert drew only six major components, missing “return to
environment,” and another drew a diagram that focused exclusively on
that expert’s specialization, without drawing the larger water
system. Thus the consensus diagram comprises seven major categories of
processes involved in getting water from the environment to the
average household in the U.S. and back to the environment. A second
\href{http://journal.sjdm.org/17/17124/supp.pdf}{supplement} shows
three additional expert drawings.

Figure 2 shows that drinking water sources are divided into surface
water (such as lakes and rivers) and ground water. Drinking water can
be publically supplied by utilities, as is the case for 85\% of
households in the U.S., or privately supplied via onsite household
treatment of groundwater, as is the case for 15\% of households in the
U.S. (U.S. Geological Survey, 2014). In the public system, water is
pumped from surface water, ground water, or a combination of the two,
and then sent to a drinking water treatment plant where it undergoes
treatment. Public water systems use a variety of methods to treat
water; the most common steps include coagulation and flocculation
(which enhances the ability of the treatment process to remove
particles), sedimentation (allows particles to settle), filtration
(removes the particles), and disinfection (kills any remaining germs).

\pagebreak

Treated drinking water then leaves the treatment plant to enter the
city distribution network. Once the water reaches the household, it is
used in indoor and outdoor applications. In the U.S., each person uses
about 80--100 gallons of water per day across the two categories of
indoor and outdoor use (USGS, 2015). In a non-representative sample of
737 homes in the U.S., DeOreo, Mayer, Dziegielewski \& Kiefer (2016)
found that average indoor water use was 58.6 gallons per capita per
day (gpcd). The authors disaggregated total indoor water use via flow
meters, finding the major water use applications to be toilets (14.2
gpcd), showers (11.1 gpcd), faucets (11.1 gpcd), clothes washers (9.6
gpcd), and leaks (7.9 gpcd). The remaining water is used outdoors,
primarily for irrigation, and then returns, untreated, to the natural
environment (Stephens, 2010). Water that has been used indoors is sent
to a collection system before it is treated. In a public system, this
treatment can be achieved via a combined sewer system or a separate
sanitary system. In a combined sewer system, wastewater and storm
water are collected in the same pipes (Drinan \& Spellman,
2012). Public systems send collected wastewater to a wastewater
treatment plant, where it usually undergoes preliminary treatment
(removal of large solids and trash), primary treatment (separates out
solids and grease from the liquid), secondary treatment (biological
treatment of wastewater to break down organic material), and
disinfection treatment (typically adding chlorine to kill any
remaining harmful microorganisms). The treated water is then
discharged back into the natural environment.

\begin{figure*}[t!]\centering
\begin{tabular}{cc}
\textbf{A} & \textbf{C} \\
\includegraphics[width=3.75in,clip=true]{fig3a.eps} &
\includegraphics[width=3in,clip=true]{fig3c.eps}\\[2ex]
\mc{2}{c}{\textbf{B}}\\
\mc{2}{c}{\includegraphics[width=3.5in,clip=true]{fig3b.eps}}\\
\end{tabular}
\caption*{Figure 3. Panels A, B, and C show examples of student drawings ranging
  from near perfect sophistication and understanding (panel A) to a less
  complete understanding (panel C). Panel A shows a drawing that scored a
  6 out of 7 for inclusion of water system components (water source,
  drinking treatment, distribution, household use, collection system,
  wastewater treatment). The drawing in Panel B scored a 4 (featured
  components are water source, household use, collection system,
  wastewater treatment). Finally, panel C scored a 3 as it only includes
  water source, distribution, and household use. (Images were
  edited for ease of viewing without altering any content.)}
\end{figure*}

\subsection{Analysis of student drawings}

Although there are many possible approaches to analyzing the drawings
produced by the student participants, we restricted our main analysis
to whether or not the drawings included each of the seven major
categories identified by the experts (listed across the top of Figure
2). Two separate coders coded each participant drawing. The coders
assigned a 1 or 0 score for whether or not each of seven major
categories identified as vital in the expert elicitation consensus
model (Figure 2) were included in each of the student diagrams. After
coding a set of diagrams, the two coders compared scores to settle on
an agreed-upon final score. When coding the first subset of diagrams,
the coders noticed a series of systematic misperceptions, specifically
relating to not including a water treatment plant or wastewater
treatment plant, including filtration as the only form of treatment,
and wastewater being returned untreated to the natural environment.
Separate codes for these specific misperceptions were created for
analysis. Figure 3 shows three examples of student drawings that cover
a range of degrees of understanding of the built water system. 

\begin{table}[t]\centering
\caption*{Table 1. Descriptive statistics for entire sample.}
\begin{tabular}{lc@{\quad}cc}\toprule
                            &   M &     SD &    Range \\\midrule
 Sum major score &              4.18 & 1.57 & 1--7 \\
 Risk quality &                 0.77 & 0.42 & 0--1 \\ 
 Risk quantity &                0.62 & 0.49 & 0--1 \\ 
 Risk infrastructure &          0.53 & 0.50 & 0--1 \\ 
 Observed sum of visual cues  & 3.53 & 2.34 & 0--9 \\ 
 Environmental student &        0.14 & 0.35 & 0--1 \\ 
 Undergraduate &                0.85 & 0.36 & 0--1 \\ 
 International student &        0.10 & 0.30 & 0--1 \\ 
 Male &                         0.50 & 0.50 & 0--1 \\ 
 Age  &                         21.4 & 3.83 & 18--67\\\bottomrule
\end{tabular}
\end{table}

Overall, the drawings revealed mental models that ranged from nearly
accurate to a complete lack of understanding of how the system works. A
perfect score of 7 (showing all categories) was only obtained by 7.2\%
of participants in our sample. On average, students identified
approximately four categories in their diagrams out of seven categories
(mean number of categories = 4.18, SE = 0.07) – see Table 1. 

Students were categorized into “environmental” if their degree required
natural resource management, environmental, or water-related courses
and “non-environmental” otherwise. Based on our classification, 14\% (N
= 65) were environmental students and 86\% (N = 392) of our sample was
non-environmental. On average, environmental students identified
significantly more major categories out of seven (M = 5.32, SE = 0.17)
than non-environmental students (M = 3.99, SE = 0.08). The difference
in the sum of major categories for the two groups differed
significantly (pooled t-test \textit{t}(455)= 6.65, \textit{p
}{\textless}0.0001). 

Figure 4 shows the percentage of students that included each major
category by environmental and non-environmental groups. Pairwise
\textit{t}{}-tests revealed significant differences between
environmental and non-environmental students for all categories shown
in Figure 4 except household use, \textit{P}s {\textless}0.05. 

Figure 4 shows that for many students, the drinking water system stops
at the home (with low inclusion of collection system, wastewater
treatment, and return to the environment). We grouped the first three
categories together (source, drinking treatment, and distribution) to
create “before home” (M = 2.25, SE = 0.04) and compared to the last
three categories (collection system, wastewater treatment, and return
to environment) “after home” (M = 1.00, SE = 0.05). Before home and
after home variables showed significant differences between
environmental and non-environmental students (\textit{P}s
{\textless}0.001). We note that this finding may indicate that students
read the first sentence of the drawing instructions (“Please
\textit{draw a diagram} illustrating your understanding of the
processes by which \textit{clean} water reaches the tap in the average
home in the United States.”) and completed their illustration without
reading the second sentence of the instructions (“…and is then returned
to the natural environment”). 

\begin{figure}[t]
\includegraphics[width=\columnwidth]{fig4.eps}
\caption*{Figure 4. Percentage of students to include each major category in their
water system diagram, separated by environmental (N = 65) and
non-environmental (N = 392) students. Error bars correspond to 1
standard error of the mean in each direction.}
\end{figure}

% \begin{tabular}{l*{10}{C{.5in}}}\toprule
%  & Sum major score & Risk quality & Risk quantity & Risk infrastructure & \multicolumn{2}{c}{Observed
% sum of visual cues} & Environmental student & Undergraduate & International student & Male & Age\\\midrule
% M & 4.18 & 0.77 & 0.62 & 0.53 & 3.53 & 0.14 & 0.85 & 0.10 & 0.50 & 21.4\\
% SD & 1.57 & 0.42 & 0.49 & 0.50 & 2.34 & 0.35 & 0.36 & 0.30 & 0.50 & 3.83\\
% Range & 1 – 7 & 0 – 1 & 0 – 1 & 0 – 1 & 0 – 9 & 0 – 1 & 0 – 1 & 0 – 1 & 0 – 1 & 18 – 67\\\midrule
% \end{tabular}


Other faulty perceptions include 29\% of the participants not including
a water treatment plant, 19.7\% of participants indicating filtration
as the only form of water treatment before the water reached the home,
and 64\% not including a wastewater treatment plant. Approximately 1 in
5 participants (21\%) depicted untreated wastewater returning to the
environment. 

\subsection{Correlates of accuracy}

To investigate which factors correlate with including more components in
the system diagram, we created a “sum major score” variable, which is
equal to the number of system components (out of a maximum of seven)
depicted in each student-drawn diagram. Table 1 provides descriptive
statistics of variables of interest, including sum major score. 

As shown in Table 1, the variables “risk quality,” “risk quantity,” and
“risk infrastructure” are binary, indicating whether or not the
participant perceived risks associated with these categories. The
variable “observed sum of visual cues” is an aggregate score that
indicates how many of the following nine components participants had
seen in the past year: water main, storage tank, surface runoff,
drinking water treatment, surface water, reservoir, pump station,
wastewater treatment, and water meter. We also constructed binary
variables indicating undergraduate status (otherwise masters-level or
doctoral students), international student status, and gender, along
with a continuous variable for age. 

\begin{table*}[t]
\caption*{Table 2. Correlation matrix to show individual relationships.}
\begin{tabular}{l*{9}{d{2.4}}}\toprule
%Variables & Sum major score & Risk quality & Risk quantity & Risk infrastructure & \multicolumn{1}{c}{Observed
%sum of visual cues} & Environmental student & Undergraduate & \multicolumn{1}{c}{International
%student} & Male & Age\\\midrule
& \mc{1}{c}{1} & \mc{1}{c}{2} & \mc{1}{c}{3} & \mc{1}{c}{4} & \mc{1}{c}{5} & \mc{1}{c}{6} & \mc{1}{c}{7}
  & \mc{1}{c}{8} & \mc{1}{c}{9} \\
1 Sum major score & 1 &  &  &  &  &  &  &  &  \\
2 Risk quality & 0.13^{**}& 1 &  &  &  &  &  &  &  \\
3 Risk quantity & 0.19^{***}& 0.30^{***}& 1 &  &  &  &  &  &  \\
4 Risk infrastructure & 0.25^{***}& 0.33^{***}& 0.36^{***}& 1 &  &  &  &  & \\
5 Observed sum of visual cues & 0.25^{***}& 0.15^{**}& 0.23^{***}& 0.22^{***}& 1 &  &  &  &  \\
6 Environmental student & 0.30^{***}& 0.16^{***}& 0.23^{***}& 0.29^{***}& 0.20
*** & 1 &  &  &  \\
7 Undergraduate & -0.20^{***}& -0.02 & -0.14^{**}& -0.14^{**}& -0.10^{*}& -0.44^{***}& 1 &  &  \\
8 International student & -0.08 & -0.07 & -0.15^{**}& -0.13^{**}& -0.19^{***}& -0.01 & -0.10^{*}& 1 &  \\
9 Male & 0.04 & -0.01 & 0.03 & 0.11^{*}& 0.22^{***}& 0.06 & -0.00 & -0.05 & 1 \\
10 Age & 0.18^{***}& 0.04 & 0.11^{*}& 0.16^{***}& 0.20^{***}& 0.19^{***}& -0.53^{***}& 0.05 & 0.09 \\\bottomrule
\mc{10}{l}{$^*$ p<.05; $^{**}$ p<.01;  $^{***}$ p<.001.}
\end{tabular}
\end{table*}

Table 2 shows a correlation matrix for all the variables presented in
Table 1. Perceived risk quality, risk quantity, risk infrastructure,
observed sum of visual cues, environmental, graduate, and older
students all correlated positively to the sum major score. 

\subsection{Risk perceptions}

\begin{figure}[t]
\includegraphics[width=\columnwidth]{fig5.eps}
\caption*{Figure 5. Percentage of participants believing that there are risks
associated with water quality, water quantity, and water
infrastructure, categorized by environmental and non-environmental
majors. Error bars correspond to 1 standard error of the mean in each
direction.}
\end{figure}
 
\subsubsection{Closed-ended responses related to risk}

As shown in Figure 4 and Table 2, “environmental student” is
significantly correlated to students including more components in
their drinking water system diagrams. To investigate environmental
students further, we analyzed how having an environmental education
relates to risk perceptions.  Figure 5 shows percentage of
participants who believe there are risks related to water quality,
quantity, and infrastructure for environmental and non-environmental
students. Environmental students have higher perceptions of risk for
quantity, quality, and infrastructure in comparison to
non-environmental majors. Mean scores of water quality risk are
highest among risk categories for both groups of students, suggesting
that drinking water quality is the top area of concern for
students. Non-environmental students’ mean scores for risk related to
quantity and infrastructure drop-off significantly compared to those
of environmental students, suggesting non-environmental students are
less concerned about these categories of risk.

Additionally, a series of closed-ended questions asked participants to
identify and rate a variety of attributes related to their water
supply. The first question asked participants how often they think
about the quantity of water they use in their home. About 33\% of
participants stated that they never or rarely (annually) think about
water quantity used in their home, compared to 37\% who report they
think of water quantity a moderate amount (weekly) or a great deal
(daily). 

Participants were also asked to what extent they would describe their
local water supply as safe to drink. An overwhelming majority of
participants (92\%) chose average, above average, or far above average
in describing their local water supply as safe to drink (Lake Monroe in
Bloomington, IN). 

Finally, participants were asked how they would rate the level of
difficulty they encounter in finding information about where their
water comes from. The majority of participants (58\%) perceive
information on their local water source to be very difficult,
difficult, or moderately difficult to find. 

\begin{table*}[t]
\caption*{Table 3. The top five open-ended responses for risks related to water
quality, quantity and infrastructure, shown as percentages of total
responses.}
\begin{tabular}{L{1.6in}L{.5in}L{1.6in}L{.5in}L{1.6in}L{.5in}}\toprule
\mc{2}{c}{Quality (N = 352)} & \mc{2}{c}{Quantity (N = 280)} &
 \mc{2}{c}{Infrastructure (N = 237)} \\\cmidrule(lr){1-2}\cmidrule(lr){3-4}\cmidrule(lr){5-6}
Cleanliness & 27.8 & Limited supply/\newline inadequate storage & 38.2 &
Inadequate treatment/ contamination & 23.2\\
Synthetic chemicals & 15.3 & Inefficient use & 27.1 & Age of
infrastructure & 19.8\\
Issues with infrastructure & 10.5 & Drought & 11.1 & Inefficiency of system & 13.1\\
Microorganisms & 8.8 & Contaminants / toxins & 4.3 & Not sure / don’t understand & 9.3\\
Negative health effects & 4.8 & Social justice & 3.2 & Lack of inspection/ maintenance/regulation & 7.2\\\bottomrule
\end{tabular}
\end{table*}

\subsubsection{Open-ended responses related to risk}

The first set of open-ended questions asked participants to identify the
greatest risk associated with water quality, quantity, and/or
infrastructure, if they perceived risks to exist in any of these areas.
Two coders developed a coding scheme for the open-ended questions, with
between 11 and 20 categories for each question. The two independent
coders coded a maximum of three responses for each open-ended question
(three “greatest risk” for each of quality, quantity, and
infrastructure). Here we restrict our analysis to the first “greatest
risk” mentioned by the participant. Assessment of inter-rater
reliability suggests a high level of agreement between the two raters,
with $\kappa$ ranging between 0.81 and 0.88 across the three
open-ended questions for the first risk coded (Landis \& Koch, 1977). 

Results for the top five open-ended responses for participants who
believed there were risks associated with quality, quantity, and
infrastructure mentioned in their open-ended response are shown
in Table 3. Close to 30\% of the participants believe water quality
risks to be associated with cleanliness (this category included
responses using words such as cleanliness, purity, contamination, and
pollution). Synthetic chemicals (15\%) was the second most common risk
identified and included words such as pesticides, fertilizers, and
Atrazine (a herbicide commonly used in the Midwest; EPA, 2013). More
than half of the participants associated drinking water quantity risks
with limited supply/inadequate storage (38\%) and inefficient use
(27\%). Water infrastructure received the lowest number of “yes”
responses indicating perceived risk among participants (Figure 5).
Of the participants who perceived a risk associated with drinking water
infrastructure, the top response was inadequate treatment and
contamination, followed by age of infrastructure. These results imply
that participants believe that the risks associated with water
infrastructure overlap with those for quality and quantity, which is
also evident in the mention of infrastructure issues as risks to water
quality and inadequate storage as a risk to quantity. Worthy of note is
that the risks listed below in Table 3 are mostly causes, with very few
effects of the risks mentioned. 

About 34\% of participants agreed with the statement “water I get from
my tap is likely to contain some harmful contaminants.” Participants
who agreed with the statement were asked to list any contaminants that
came to mind. The top three responses were heavy metals including
arsenic, zinc, copper, lead, iron, and mercury (16\%); not sure / I
don’t know (14\%); and contaminants associated with treatment
including chlorine, chloramines, and bleach (10\%).\footnote{ There
  were reported high levels of disinfection byproducts in Bloomington
  Indiana’s drinking water supply at the time that this study was
  conducted.}

Finally, participants were asked to identify their local water source,
where the correct answer is a reservoir called Lake Monroe. Of those
who answered the question and did not leave it blank (N = 412), only
18.8\% of the participants listed this source correctly. Approximately
1 in 5 (19.9\%) of the participants stated they did not know their
local water source, and 13.6\% stated that their local water source was
their house (sink, fridge, and tap), which may reflect an alternative
reading of the question. 

\subsection{Alternative wording}

After analysis of the first study was complete, we wanted to understand
how the details in our drawing elicitation guided our participant
drawings. Specifically, “how water reaches the home from its original
source(s) and is then returned to the natural environment” may have
caused student participants to think of and include specific features of
the water system in their drawings that they would not have originally
included without such detailed wording. Thus, to test the effect of the
original wording used for the drawing elicitation, we simplified the
wording and elicited responses from another group of primarily
non-environmental students (\textit{N = }51). The wording in this
version of the survey was as follows:
\begin{quote}
Please draw all processes by which drinking water reaches the average
home in the United States and all processes after it is used. You are
encouraged to label your drawing.
\end{quote}

Restricting our main original sample to undergraduates (N = 387), the
mean sum of major categories included in diagrams drawn in response to
the original question was 4.05 versus 2.94 in this alternative wording
group. In response to the reworded question, we find that many more
participants drew bottled water as their main water source (14\% in
this sample compared to 2.3\% in our total undergraduate main sample).
These results imply that the level of detail specified in the main
study was necessary to effectively guide participants to draw the
drinking water system. 

\section{Discussion}

Drinking water is the most essential among all resources, and yet water
is often taken for granted in highly developed contexts. In the U.S.,
most people expect to have immediate access to safe drinking water,
which appears from the tap as if by “magic”. Hidden from view, however,
are the multiple parts of this complex and vital system that must work
seamlessly together for drinking water to be made readily available.
Given the present risks related to water quality, quantity, and
infrastructure, informed decisions concerning drinking water and the
policies and management structures that support it are critically
important. 

Our results show that most university students are unable to produce a
complete or accurate depiction of the built water
system. Environmental students are more likely than others to include
a higher number of the seven major categories of the drinking water
system in their drawings (i.e., water source, drinking treatment,
distribution, household use, collection, wastewater treatment, and
return to environment). In general, students who have higher
perceptions of risk related to infrastructure and those that have
noticed more visual cues related to the drinking water system have
more complete knowledge about the water system. However, only 7.2\% of
participants had a perfect score of seven out of seven water system
categories, 45\% of which were environmental students, and 5.5\% of
participants only included one of the seven categories in their
diagram, all of who were non-environmental students.

Our results also highlight many significant misperceptions. We find
that 19.7\% of our participants indicated filtration as the only form
of water treatment before the water reached the home. Additionally,
29\% of the participants did not draw a water treatment plant and 64\%
did not draw a wastewater treatment plant. One in five participants
had untreated water returning to the natural environment. Perhaps some
students would have benefitted from a follow-up question asking, “What
do you think happens after the home?” to distinguish between
participants’ lack of knowledge and their lack of response.

\renewcommand{\baselinestretch}{1.04} \large\normalsize 

In relation to risk, a high percentage of environmental student
participants believed that there are risks related to water quality,
quantity, and infrastructure. Non-environmental participants had a
significantly lower perception of risk than environmental participants.
Non-environmental participants had the highest perception of risk for
water quality, followed by water quantity, and lastly for water
infrastructure. This finding may shed light on why the public is
seemingly uninterested in water infrastructure issues. 

\subsection{Implications}

Water systems thinking has implications for multiple audiences. Here, we
focus on lay people and public policy makers. We posit that a greater
understanding of the water system among lay people will be necessary in
order for municipalities and larger levels of government to confront
new and exceedingly complex risks to the water system, given that tax
payers must approve price hikes to address water shortages and
infrastructure upkeep (Olmstead et al., 2007; Sabelli, 2006). 

Systems thinking is also important in order for public policy makers to
more fully understand which parts of the integrated system are most at
risk. When systems thinking is limited, catastrophes ensue. With
respect to water, lead contamination in Flint, Michigan is a clear
example of systems thinking failure. Another example in the
environmental domain is the Deepwater Horizon oil spill of 2010, during
which managers misunderstood pressure data, which begun a catastrophic
domino effect that led to the eventual explosion and spill. A similar
lack of systems thinking can be seen in the Challenger disaster of
1986, in which O-ring failure, bad weather, and “go fever” contributed
to the death of seven space shuttle crewmembers. Capacity to avoid such
catastrophes through better decision making can improve to the extent
that these systems and potential pitfalls are better understood as an
integrated whole (Hammond, 1996). 

Our work contributes to the existing body of work focusing on systems
thinking about the water cycle. For example, Dove, Everett, and Preece
(1999) found that children draw rivers in rural rather than urban
environments, Shepardson et al.\ (2007b) identified significant
misperceptions about watersheds (such as a watershed is a “shed” to
store water), and Covitt, Gunkel and Anderson (2009) show that although
students have some understanding of water systems, most students fail
to trace water through the complete system correctly. Our study
contributes to this body of research by (a) creating an accurate expert
model of the water system that practitioners can use as part of a
communication and education outreach strategy; (b) showing that systems
thinking falls off rapidly for non-environmental students when
attempting to model the water system beyond the home, a misperception
that needs to be corrected given that the cost of treating wastewater
far exceeds that of treating drinking water; and (c) risk perceptions
of infrastructure are the lowest for non-environmental students,
potentially because infrastructure is an “out of sight, out of mind”
investment. 

This study also contributes to contemporary research in the field of
water education and understanding. To our knowledge, ours is the first
paper that investigates systems thinking of the natural and
anthropogenic system together, rather than primarily focusing on the
natural water cycle (Assaraf et al., 2012). By eliciting and analyzing
drawings from university students, we are able to gauge students’
mental models of our current water system compared to an accurate
expert-elicited model, without relying on technical terms. 

Our research can also be used as a teaching tool. Some scholars have
argued that all students ought to learn about the water system (Sadler
et al., 2016). Educators focusing on the environment in general or
water resources specifically could ask students to draw these diagrams
before and after a course to test how understanding improves over the
course’s duration. The Next Generation Science Standards, a multi-state
effort to create new education standards, focuses on three themes:
science and engineering practice, disciplinary core ideas, and
crosscutting concepts (NRC, 2013). Under the heading of cross cutting
concepts, understanding patterns, cause and effect, scales, systems and
system models, structure and function, and how change moves through a
system all relate to systems thinking (NRC, 2013). Thus, we argue that
understanding the complex water system and focusing on systems thinking
is crucial for STEM disciplines as well as an educated electorate. 

\renewcommand{\baselinestretch}{1.00} \large\normalsize 

\subsection{Limitations}

This study has several limitations. Among them are (1) using a
convenience sample; (2) not asking follow-up questions; (3)
constructing a consensus expert water system diagram that has not been
independently validated; (4) focusing on correlations, which indicate
associations, not causation; and (5) the problematic assumption that
better system education will produce better problem solving. We examine
each of these limitations below.

\paragraph{(1) Convenience sample:} We use a convenience sample of students in a
large Midwestern University. Students in more drought-ridden areas or
places where more time and effort are expended to collect water for
residential use could have more accurate systems understanding.
Additionally, students may be well educated but likely have limited
real-world experience with the water system, such as not having to pay
their own water bills. This limitation may affect the generalizability
of our results. 

\paragraph{(2) Method of eliciting systems thinking:} Although eliciting students’
drawings serves as a powerful tool to gauge their level of
understanding, methods that use open-ended structured interviews could
allow for a deeper understanding of misperceptions, due to the ability
to ask probing follow-up questions when misperceptions are revealed.
Capturing these deeper causal explanations could be invaluable
information for environmental educators seeking to correct the
misperceptions identified in our work. Deeper explanations could allow
us to theorize how to effectively improve understanding of systems by
assessing which explanations relate to accuracy (Keil, 2006; Lombrozo,
2006, 2007). Our study also uses a relative limited approach to systems
thinking, as we capture interrelations between system processes but
miss key features such as systems dynamics, time delays, or feedback.
In the future, this limitation can be overcome by using structured
interviews or other probes to more deeply understand water systems
thinking.

\paragraph{(3) Validity of our expert water diagram:} We interviewed 15 Indiana
water experts with backgrounds in water science and management to
construct our consensus diagram. Future work can elicit drawings from a
new group of independent experts to gauge whether or not the seven
features we included in our consensus diagram are also included in
their individual diagrams. The accurate water system diagram could have
added layers of complexity including feedback loops and other human
components such as water managers and engineers who manage and build
the water system. 

\paragraph{(4) Correlational study showing associations, not
  causation:} In this research we show that students specializing to
some extent in environmental sciences identify more of the water
system components shown in the expert portrayal of the water system,
and that these students believe that there are more risks to the water
system. This work shows association, but does not demonstrate
causation. It is plausible that students with more prior awareness are
drawn to environmental sciences. One way to tease this limitation
apart would be to design a module for an introductory undergraduate
course to teach systems thinking, and compare diagrams from students
who were and were not exposed to the module.

\paragraph{(5) The problematic assumption that better systems
  education will produce better problem solving:} This last limitation
is much harder to contend with and will need more follow-up work to
overcome. It may be the case that people using a vital resource system
do not \textit{need} to know how that system or component of the
system functions in order to rely on and use it. In fact, Keil (2003)
argues that there may be benefits to using folk science to understand
and interact with the increasingly complex world around us, where we
know what we need to know in order to function effectively in everyday
life.

\subsection{Conclusion}

Our research reveals specific systematic misperceptions of how the
water system works by eliciting and analyzing drawings from university
students in a convenience sample. Some highlights of the findings for
non-environmental students include the water system halting at the
home and low perceptions of risks related to water
infrastructure. There are many research questions that stem from this
work. Future research could investigate \textit{why} specific
misperceptions exist, \textit{how} to correct faulty water systems
thinking, and to \textit{what} \textit{extent} does correcting systems
thinking lead to changes in willingness to accept adaptation
strategies, such as wastewater reuse, and to accept changes in water
prices to account for water scarcity and fortification of our
crumbling infrastructure.

\section*{References}
\begin{hangparas}{1em}{1}

ASCE. (2017). 2017 Report card for America's
Infrastructure. Retrieved from
\url{http://www.infrastructurereportcard.org/}.

Assaraf, O. B.-Z., Eshach, H., Orion, N., \& Alamour, Y. (2012).
Cultural differences and students’ spontaneous models of the water
cycle: A case study of Jewish and Bedouin children in Israel.
\textit{Cultural Studies of Science Education, 7}(2), 451--477. 

Assaraf, O. B.-Z., \& Orion, N. (2005). Development of system thinking
skills in the context of earth system education. \textit{Journal of
Research in Science Teaching, 42}(5), 518--560. 

Bar, V. (1989). Children's views about the water cycle.
\textit{Science Education, 73}(4), 481--500. 

Bostrom, A., Morgan, M., Fischhoff, B., \& Read, D. (1994). What do
people know about global climate change? 1. Metal models. \textit{Risk
Analysis, 14}(6), 959--970. 

Brown, E. G. (2015). \textit{Executive Order No. B-29-15}. Retrieved
from \url{https://www.gov.ca.gov/docs/4.1.15\_Executive\_Order.pdf}.

Covitt, B. A., Gunckel, K. L., \& Anderson, C. W. (2009).
Students' developing understanding of water in
environmental systems. \textit{The Journal of Environmental Education,
40}(3), 37--51. 

DeOreo, W. B., Mayer, P. W., Dziegielewski, B., \& Kiefer, J. (2016).
\textit{Residential End Uses of Water, Version 2}: Water Research
Foundation.

Dove, J., Everett, L., \& Preece, P. (1999). Exploring a hydrological
concept through children's drawings.
\textit{International Journal of Science Education, 21}(5), 485--497. 

Drinan, J. E., \& Spellman, F. (2012). \textit{Water and wastewater
treatment: a guide for the nonengineering professional}: Crc Press.

EPA. (1991). \textit{Manual of Individual and Non-Public Water Supply
Systems}. Washington D.C.: EPA.

EPA. (2003). \textit{Drinking Water Infrastructure Needs Survey and
Assessment}. (EPA 812-R-97-001). Washington, D.C.

EPA. (2012a). Aging Water Infrastructure. Retrieved from
\url{https://cfpub.epa.gov/si/si_public_record_Report.cfm?dirEntryId=224905}

EPA. (2012b). \textit{Guidelines for Water Reuse}. Retrieved from 

EPA. (2013). Atrazine Updates. Retrieved from
\url{https://watereuse.org/wp-content/uploads/2015/04/epa-2012-guidelines-for-water-reuse.pdf}

Famiglietti, J. S. (2014). The global groundwater crisis. \textit{Nature
Clim. Change, 4}(11), 945--948. \url{http://dx.doi.org/10.1038/nclimate2425}.

Fischer, A., \& Young, J. C. (2007). Understanding mental constructs of
biodiversity: implications for biodiversity management and
conservation. \textit{Biological Conservation, 136}(2), 271--282. 

Forrester, J. (1961). \textit{Industrial Dynamics}. Cambridge: MIT
Press.

Gentner, D., \& Gentner, D. R. (1983). Flowing waters or teeming
crowds: Mental models of electricity. In D. Gentner \& A. L. Stevens
(Eds.), \textit{Mental models}, pp. 99--129. Hillsdale, NJ: Erlbaum.

Gleick, P. H. (1993). \textit{Water in crisis: a guide to the
world's fresh water resources}: Oxford University
Press, Inc.

Gunckel, K. L., Covitt, B. A., Salinas, I., \& Anderson, C. W. (2012). A
learning progression for water in socio-ecological systems.
\textit{Journal of Research in Science Teaching, 49}(7), 843--868. 

Hammond, K. R. (1996). \textit{Human judgement and social policy:
Irreducible uncertainty, inevitable error, unavoidable injustice}:
Oxford University Press.

Hanna-Attisha, M., LaChance, J., Sadler, R. C., \& Champney Schnepp, A.
(2016). Elevated blood lead levels in children associated with the
Flint drinking water crisis: a spatial analysis of risk and public
health response. \textit{American journal of public health, 106}(2),
283-290. 

Hartley, T. W. (2006). Public perception and participation in water
reuse. \textit{Desalination, 187}(1–3), 115--126.
\url{http://dx.doi.org/10.1016/j.desal.2005.04.072}.

Hmelo-Silver, C. E., Marathe, S., \& Liu, L. (2007). Fish swim, rocks
sit, and lungs breathe: Expert-novice understanding of complex systems.
\textit{The Journal of the Learning Sciences, 16}(3), 307--331. 

Indiana University School of Education. (2014). International Student
Resources. Retrieved from
\url{http://education.indiana.edu/graduate/international.html}

Inskeep, B. D., \& Attari, S. Z. (2014). The Water Short List: The Most
Effective Actions US Households Can Take to Curb Water Use.
\textit{Environment: Science and Policy for Sustainable Development,
56}(4), 4--15. 

Keil, F. C. (2003). Folkscience: Coarse interpretations of a complex
reality. \textit{Trends in Cognitive Sciences, 7}(8), 368--373. 

Keil, F. C. (2006). Explanation and understanding. \textit{Annu. Rev.
Psychol., 57}, 227--254. 

Kempton, W. (1986). Two theories of home heat control. \textit{Cognitive
Science, 10}(1), 75--90. 

Kundzewicz, Z., Mata, L., Arnell, N. W., Doll, P., Jimenez, B., Miller,
K., \dots Shiklomanov, I. (2008). The implications of projected climate
change for freshwater resources and their management.
\textit{Hydrological sciences journal, 53}(1), 3--10. 

Landis, J., \& Koch, G. (1977). The measurement of observer agreement
for categorical data. \textit{Biometrics, 33}(1), 159--174. 

Lombrozo, T. (2006). The structure and function of explanations.
\textit{Trends in Cognitive Sciences, 10}(10), 464--470. 

Lombrozo, T. (2007). Simplicity and probability in causal explanation.
\textit{Cognitive Psychology, 55}(3), 232--257. 

Lund, J. R. (2015). Integrating social and physical sciences in water
management. \textit{Water Resources Research, 51}, 5905--5918. 

Meadows, D. H., \& Wright, D. (2008). \textit{Thinking in systems: A
primer}: Chelsea Green Publishing.

Melillo, J., Richmond, TC., Yoho, GW. (2014). Climate Change Impacts in
the United States: The Third National Climate Assessment. (pp.\ 844).
Washington, D.C.: US Global Change Research Program.

Morgan, G., Fischhoff, B., Bostrom, A., \& Atman, C. (2002).
\textit{Risk Communication: A Mental Models Approach}: Cambridge
University Press.

National Research Council. (2006). \textit{Drinking Water Distribution
Systems: Assessing and Reducing Risks}: The National Academies Press.

NRC. (2013). \textit{Next Generation Science Standards}. Retrieved from
Washington DC: 

Olmstead, S. M., Hanemann, W. M., \& Stavins, R. N. (2007). Water demand
under alternative price structures. \textit{Journal of Environmental
Economics and Management, 54}(2), 181--198. 

Olmstead, S. M., \& Stavins, R. N. (2009). Comparing price and nonprice
approaches to urban water conservation. \textit{Water Resources
Research, 45}(4). 

Orion, N. (2002). An Earth systems curriculum development model
\textit{Global science literacy} (pp.\ 159-168): Springer.

Po, M., Nancarrow, B. E., \& Kaercher, J. D. (2003). \textit{Literature
review of factors influencing public perceptions of water reuse}.
Retrieved from 
\url{http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.197.423&rep=rep1&type=pdf}.

Read, D., Bostrom, A., Morgan, M. G., Fischhoff, B., \& Smuts, T.
(1994). What do people know about global climate change? 2. Survey
studies of educated laypeople. \textit{Risk Analysis, 14}(6), 971--982. 

Reynolds, T. W., Bostrom, A., Read, D., \& Morgan, M. G. (2010). Now
what do people know about global climate change? Survey studies of
educated laypeople. \textit{Risk Analysis, 30}(10), 1520--1538. 

Rozin, P., Haddad, B., Nemeroff, C., \& Slovic, P. (2015). Psychological
aspects of the rejection of recycled water: Contamination, purification
and disgust. \textit{Judgment and Decision Making, 10}(1), 50--63. 

Sabelli, N. H. (2006). Complexity, technology, science, and education.
\textit{The Journal of the Learning Sciences, 15}(1), 5--9. 

Sadler, T. D., Nguyen, H., \& Lankford, D. (2017).  Water systems
understandings: a framework for designing instruction and considering
what learners know about water. \textit{Wiley Interdisciplinary
  Reviews: Water, 4}(1).

Shepardson, D. P., Wee, B., Priddy, M., Schellenberger, L., \& Harbor,
J. (2007a). What is a watershed? Implications of student conceptions
for environmental science education and the national science education
standards. \textit{Science Education, 91}(4), 554--578. 

Stephens, D. (2010). \textit{The Role of Decentralized Artificial
Recharge Systems in Water Resources Management}. 55th Annual NM Water
Conference, How Will Institutions Evolve to Meet Our Water Needs in the
Next Decade? 

Sterman, J. D. (2008). Risk communication on climate: mental models and
mass balance. \textit{Science, 322}(5901), 532--533. 

Sterman, J. D., \& Sweeney, L. B. (2002). Cloudy skies: assessing public
understanding of global warming. \textit{System Dynamics Review,
18}(2), 207--240. 

Tanellari, E., Bosch, D., Boyle, K., \& Mykerezi, E. (2015). On
consumers' attitudes and willingness to pay for
improved drinking water quality and infrastructure. \textit{Water
Resources Research, 51}(1), 47--57. \url{http://dx.doi.org/10.1002/2013WR014934}.

U.S. Geological Survey. (2014). Total Water Use in the United States,
2005. Retrieved from
\url{http://water.usgs.gov/edu/wateruse-total.html}

U.S. Global Change Program. (2014). The National Climate Assessment
Report: Executive Summary. Retrieved from
\url{http://nca2014.globalchange.gov/report}

University Institutional Research and Reporting. (2013). Indiana
University Bloomington Campus 2013-2014 Fact Book. Retrieved from
\url{https://www.iu.edu/\~uirr/reports/standard/factbook/}

USGS. (2015). Total water use. Retrieved from
\url{http://water.usgs.gov/watuse/wuto.html}

Vörösmarty, C. J., McIntyre, P., Gessner, M. O., Dudgeon, D., Prusevich,
A., Green, P., \dots Liermann, C. R. (2010). Global threats to human
water security and river biodiversity. \textit{Nature, 467}(7315),
555-561. 

\vfill
\break

Wester, J., Timpano, K. R., Çek, D., Lieberman, D., Fieldstone, S. C.,
\& Broad, K. (2015). Psychological and social factors associated with
wastewater reuse emotional discomfort. \textit{Journal of Environmental
Psychology, 42}, 16--23. 

Whelton, A. J., McMillan, L., Connell, M., Kelley, K. M., Gill, J. P.,
White, K. D., \dots Novy, C. (2015). Residential Tap Water
Contamination Following the Freedom Industries Chemical Spill:
Perceptions, Water Quality, and Health Impacts. \textit{Environmental
Science \& Technology, 49}(2), 813--823. \url{http://dx.doi.org/10.1021/es5040969}.

\end{hangparas}

\end{document}
