Embracing the uncertainties

These are, safe to say, uncertain times.

The confirmed global cases of illness from coronavirus are approaching 1.5 million, and reported deaths are well into the six figures, but what are the true rates of infection and mortality?

We don’t know.

Last week, Dr Robert Redfield, head of the Centres for Disease Control and Prevention, said that up to 25% of people infected with coronavirus show no symptoms. But at the White House coronavirus task force briefing on Sunday, Dr Anthony Fauci, head of the National Institute of Allergy and Infectious Diseases at the National Institutes of Health, gave a markedly broader range.

“It’s somewhere between 25 and 50%,” Fauci said. “And trust me, that is an estimate. I don’t have any scientific data yet. You know when we’ll get the scientific data? When we get those antibody tests out there.”

This type of uncertainty about facts, numbers and science is called epistemic uncertainty. It is caused by a lack of knowledge about the past and the present — “our ignorance,” said David Spiegelhalter, a statistician and chair of the Winton Centre for Risk and Evidence Communication at the University of Cambridge.

Science is full of epistemic uncertainty. Circling the unknowns, inching toward truth through argument and experiment is how progress is made. But science is often expected to be a monolithic collection of all the right answers. As a result, some scientists — and the politicians, policymakers and journalists who depend on them — are reluctant to acknowledge the inherent uncertainties, worried that candour undermines credibility.

Then there are scientists like Fauci, who has also acknowledged uncertainty about matters like the time needed to flatten the curve, to develop an antibody test and to find a vaccine.

“I will say what’s true, and whatever happens, happens,” he told Vanity Fair.

What happens when scientists do acknowledge uncertainty is the question behind a study, published March 23 in the Proceedings of the National Academy of the Sciences. It explored “The Effects of Communicating Uncertainty on Public Trust in Facts and Numbers.”

“The accusations of a post-truth society, and claims that the public ‘had had enough of experts,’ prompted us to investigate whether trust in ‘experts’ was lowered by their openly admitting uncertainty about what they know,” said Spiegelhalter, one of the principal investigators.

The study’s findings suggest that being transparent about uncertainty does not harm the public’s trust in the facts or in the source.

“These results indicate that people ‘can handle the truth’ about the level of certainty or uncertainty of scientific facts and knowledge,” said Anne Marthe van der Bles, a psychologist at the University of Groningen, who is the lead author and an affiliate with the Cambridge research team.

Using online surveys, the study measured reactions to uncertainty expressed in statements about various subjects: the number of tigers left in India, the increase in global average surface temperature between 1880 and 2010 and unemployment figures in the United Kingdom. The survey was replicated “in the wild” with a field study on the BBC News website. The researchers tested uncertainties presented quantitatively, with a numerical range or percentage; and more qualitatively, using a word such as “estimated” or “approximately.”

The precise numerical statements were more effective both in conveying uncertainty and in maintaining trust. (There was in fact a minor reduction in trust, but the researchers deemed the effect so small as to be trivial.)

“I find it heartening, and very good advice,” said Ed Humpherson, the director general of the Office for Statistics Regulation in the UK. “Being trustworthy depends not on conveying an aura of infallibility, but on honesty and transparency.”

Starting on March 19, the experiment was replicated in several countries with statements about the severity of COVID-19, the disease caused by the new coronavirus. Early results confirm the paper’s findings.

“So, where there is uncertainty around — for example, the death rates from COVID-19 — people shouldn’t feel concerned about communicating this to the public,” said Alex Freeman, a co-author, and the executive director of the Winton Centre. “It may be important to do so.”

The public, in turn, must be open to considering and adapting to new evidence, said Lorraine Daston, a historian of science at the Max Planck Institute for the History of Science in Berlin. “We the public must expect scientific views on the nature of the virus and how best to combat it to change as more evidence comes in, and be prepared to change our conduct accordingly,” she said.

VARAITIES OF UNCERTAINTY

The Cambridge team has been exploring uncertainty in its many forms for a while now. Last year, they published a theory paper, reviewing related research. At a conference of uncertainty quantification specialists about two years ago, Freeman asked attendees to write definitions of uncertainty on Post-it notes and stick them on the wall. “Every one was different,” she said. “I have to say my favourite was: ‘Anything and everything that can **** up a decision’ [insert descriptor of your choice].”

The recent study, funded by the Nuffield Foundation, focused on people’s reactions to epistemic uncertainty: things we don’t know about the past and present but in theory could come to know, through measurement. The team is now researching perceptions of aleatory uncertainty — unknowns about the future due to randomness, indeterminacy, chance or luck. (In Latin, alea means dice or gambling.)

Most uncertainty is a mix of epistemic and aleatory elements. For instance: How many more people will get COVID-19? And once transmission is suppressed below the R0=1 threshold (the reproduction number required to rapidly reduce the number of cases to low levels), how will we avoid a rebound?

Common wisdom from the psychologist’s perspective is that people do not like uncertainty, especially about the future, and that it generates a negative response. (Psychologists call this “ambiguity aversion.”) From the statistician’s perspective, the hypothesis is that people have a positive reaction and trust information more when the communicator is being open about uncertainties in facts and figures.

“The motivation was to try to adjudicate between these competing hypotheses,” said Sander van der Linden, a principal investigator, and a psychologist and the director of the Cambridge Social Decision-Making Lab. “Ultimately we didn’t find support for the notion that communicating uncertainty enhances public trust, but it also didn’t substantially undermine it.”

MODELS ARE THE MAPS, NOT THE TERRITORY

Either way, there is little to assuage our most pressing existential uncertainty: When will the pandemic end?

In the early days of the outbreak, when data was beginning to emerge from China, we were in a state of “deep uncertainty,” also known as “radical uncertainty” or “Knightian uncertainty.” (The economist Frank Knight distinguished between risk and uncertainty about a century ago.) Deep uncertainty is the quagmire of unknown unknowns; there are no constraints.

Risk is yet another type of uncertainty, usually pertaining to things in the future that might turn out badly. Risk encompasses the known unknowns and can be calculated with probabilities. As more reliable data comes in, said Spiegelhalter, “the COVID-19 pandemic is rapidly becoming a constrained problem.”

Earlier in March, he sought to tame the uncertainty and the fear by investigating exactly how much “normal risk” infection with coronavirus represents.

We face normal risk daily — “We are all going to die sometime,” said Spiegelhalter. And the odds increase from one day to the next, with age.

Working with the latest (albeit uncertain) data about COVID-19 mortality rates, he found that getting infected essentially compressed a year’s worth of normal risk into a couple of weeks. His risk of dying in the next year, as a 66-year-old man, was about 1.5%. “Very roughly, getting COVID-19 seems to be like packing that much risk into the time that you are ill,” he said. “Of course, if you survive, you still have your standard ration of risk to deal with.”

Statistical science, he said, “is a machine, in a sense, to turn the variability that we see in the world — the unpredictability, the enormous amount of scatter and randomness that we see around us — into a tool that can quantify our uncertainty about facts and numbers and science.”

But as he acknowledged in his book, “The Art of Statistics,” models “are simplifications of the real world — they are the maps not the territory.” (This is reminiscent of Jorge Luis Borges’ story, “On Exactitude in Science,” about a map growing as large as the territory it was meant to represent.)

The limitations and uncertainties inevitably get exploited in politicised narratives, and entangled in misinformation and disinformation, making it all the more important to confront them head on.

Neil Ferguson, who heads the team of epidemiologists at Imperial College London that produced the influential March 16 report modelling the virus’ spread, said that it is the role of modelers, in presenting the projections, to at once indicate the uncertainties “and to evaluate effectively how the uncertainties — the extent of asymptomatic infection, for instance; the overall lethality of this virus — might change the conclusions, particularly qualitatively rather than quantitatively.”

In communicating uncertain or ambiguous results for governments, Ferguson usually includes “confidence bounds” or “error bars.” But in doing a radio interview, for instance, he tends to opt for more qualitative descriptions. He was surprised to hear that the study’s results indicated that quantitative statements are more effective in conveying uncertainty even with a more general audience. “Maybe I’ll take that on board,” he said.

The study is, in a sense, proof of the value in acknowledging these unknowns. As Spiegelhalter noted, it is “an empirical test of humility.”

c.2020 The New York Times Company