The College Fix

‘A travesty unfolding right before our eyes’

The COVID-19 model developed by the University of Washington’s Institute for Health Metrics and Evaluation, which owns the website HealthData.org, is popular with the White House despite wildly overprojecting hospitalizations from the novel coronavirus.

Epidemiologists are now calling on policymakers to stop giving credence to the model (above), according to life-sciences news site STAT.

A colleague of the UW researchers has gone so far as to call the model’s use in public policy “a travesty unfolding right before our eyes”:

“That the IHME model keeps changing is evidence of its lack of reliability as a predictive tool,” said epidemiologist Ruth Etzioni of the Fred Hutchinson Cancer Center, home to several of the researchers who created the model, and who has served on a search committee for IHME. …

The chief reason the IHME projections worry some experts, Etzioni said, is that “the fact that they overshot” — initially projecting up to 240,000 U.S. deaths, compared with fewer than 70,000 now — “will be used to suggest that the government response prevented an even greater catastrophe, when in fact the predictions were shaky in the first place.”

That could produce misplaced confidence in the effectiveness of the social distancing policies, which in turn could produce complacency about what might be needed to keep the epidemic from blowing up again.

University of Sydney statistician Sally Cripps called out the model for poor performance “even when it predicts the number of next-day deaths: The true number of next-day deaths has been outside the 95% intervals 70% of the time.”

Harvard epidemiologist Marc Lipsitch was milder in his criticism, saying that “most of us in the infectious disease epidemiology field” do not think the so-called Murray model is “well suited” to COVID-19 projections.

MORE: ‘Murray model’ still wildly overestimates hospital rate after revision

STAT explains how the UW model,  “differs from those used by almost all other epidemiologists.”

The SEIR model (susceptible, exposed, infected, recover) and “agent-based models,” which simulate interactions through massive computing power, “have often nailed projections of, for instance, U.S. cases of seasonal flu.”

The UW model, however, bases its U.S. projections on analysis of data from original COVID-19 hotspots in China, Italy and Spain:

It then produces a graph showing the number of deaths rising and falling as the epidemic exploded and then dissipated in those cities, resulting in a bell curve. Then (to oversimplify somewhat) it finds where U.S. data fits on that curve. The death curves in cities outside the U.S. are assumed to describe the U.S., too, with no attempt to judge whether countermeasures —lockdowns and other social-distancing strategies — in the U.S. are and will be as effective as elsewhere, especially Wuhan.

A SEIR model by MIT researchers has been “pretty much on the money,” in contrast, projecting a plateau of 600,000 coronavirus cases and then a trickle of new cases each day. A competing UW model also projected new cases “peaking in mid-April,” which seems to have been validated.

The UW model took another hit from Imperial College London researchers in the Annals of Internal Medicine, saying its projections are based “on a statistical model with no epidemiologic basis”:

“This appearance of certainty is seductive when the world is desperate to know what lies ahead,” Britta Jewell of Imperial College and her colleagues wrote in their Annals paper. But the IHME model “rests on the likely incorrect assumption that effects of social distancing policies are the same everywhere.”

Read the article.

MORE: ICL researchers estimate tens of millions of undiagosed European COVID cases

IMAGE: HealthData.org screenshot

Like The College Fix on Facebook / Follow us on Twitter