Seminar explores communication of uncertainty and risk
By Kelly Lenox
Objectivity is the touchstone for scientists of every discipline. But when civic responsibility calls for decision making in the context of uncertainty, as it does for scientists in NTP, researchers must grapple with how to effectively communicate their findings.
For insight into how to better express conclusions on potential health concerns associated with exposure to environmental substances, NTP turned to two experts in psychology — David Budescu, Ph.D., of Fordham University, and Thomas Wallsten, Ph.D., of the University of Maryland (UM). Budescu and Wallsten discussed this issue at a seminar March 11 at NIEHS.
For NTP, an important part of reviewing literature-based health assessments (see story) involves examining how to best communicate the results of those assessments, according to Kristina Thayer, Ph.D., director of the NTP Office of Health Assessment and Translation. NTP stakeholders include decision-makers in the public and private sectors, as well as individuals with concerns about their own environmental health.
A concern with levels of concern
“When the literature base is well developed, it can result in a formal NTP conclusion referred to as a level of concern (LOC), which integrates what is known about health hazard with the extent of human exposure,” Thayer said.
LOC conclusions are narrative rather than quantitative, and that has resulted in some confusion, Thayer continued. “For example, researchers and the public have expressed difficulty in distinguishing between a substance being of ‘some concern’ and being ‘of concern,’” she explained.
Mary Wolfe, Ph.D., director of the NTP Office of Liaison, Policy, and Review, further detailed the challenge of LOC terms. “Do we have too many categories? Should we use numbers along with the words?” she asked. “It’s about how we frame the outcome of an evaluation.”
Use both verbal and numerical terms
“Decision-makers are best served when the terms used in communicating uncertainty match the ambiguity of the event and the uncertainty of its occurrence,” said Budescu (see text box).
As an example of ways to better deal with ambiguity, he pointed to an international study (http://www.ncbi.nlm.nih.gov/pubmed/19207697?dopt=Abstract) on communicating uncertainty about climate change. In 25 countries, researchers compared the understanding of two forms of conveying uncertainty — verbal terms, and verbal combined with numerical terms. The verbal terms, such as “very likely,” were linked to a separate table listing the numerical value of that likelihood. The verbal combined with numerical terms, such as “very likely (greater than 90 percent probability),” included the numerical value alongside.
Researchers were surprised that, even though a reference table of values was provided, understanding of the verbal terms alone varied considerably among readers. “The presentation of verbal with numerical information achieves better understanding of the terms,” Budescu said, noting that understanding improved both across countries surveyed and across all terms used.
Obtain and encode expert judgment
Wallsten discussed a process for obtaining and communicating expert judgment — another technique for communicating risk in cases of inadequate data. He outlined an example in which a small panel of 5-7 highly respected scientists was selected using an open and transparent process. The preparation of panel members included enhancing their sensitivity to sources of uncertainty.
The process included numerically encoding the subjective judgments of the expert panel, for example, the likely dose-response rates after exposure to an air pollutant. “Encoding the judgment calls can be useful in representing the extent of consensus or disagreement among the experts,” said Wallsten. This step echoed Budescu’s finding of the importance of using numerical values when communicating uncertainty.
“Subjectivity cannot be avoided, but it can be translated to a level of concern via an open and principled process,” Wallsten concluded.
Distinguishing sources of uncertainty
Budescu categorized three sources of imprecision when communicating probability.
- The ambiguity of the event itself - for example, “drawing a queen from a deck of cards” is a less ambiguous event than “a drop in the market in the near future.”
- The type of uncertainty, or the degree to which the probability of an event can be quantified - for example, “34 percent more likely” is more precisely quantified than “more likely.”
- The language of communication - for example, “drop 6 cents per share” is more precise than “drop 10-20 percent in value.”