Junk Science Rules

Did you hear about the headline which claimed that every year 250,000 people in the U.S. die after misdiagnosis in the emergency room? Here is an article from CNN:

https://www.cnn.com/2022/12/15/health/hospital-misdiagnoses-study/index.html

Here is the amazing part. The statistic was extrapolated from the death of one man in a Canadian emergency room more than a decade ago. Yes, you read that right.

When I saw the headline it really seemed like BS so I blew it off. The details, however, on how they got these numbers are disturbing. I am going to highlight what Dr. Kristen Panthagani, an emergency physician at Yale, said in an opinion piece in the Wall Street Journal:

The figure comes from a Dec. 15 report by the Agency for Health Care Research and Quality. The AHRQ, part of the U.S. Department of Health and Human Services, produces research to improve healthcare in America. 

The evidence doesn’t support the magnitude of error suggested by the AHRQ report. The inaccurate estimate would put emergency-room misdiagnoses in the top five causes of death in the U.S., with 1 of every 500 ER patients dying because of physician error. There is no reliable evidence to suggest anything close to this. These estimates undermine the patient-physician relationship and the dwindling trust in public-health reporting.

The statistical methods used to arrive at the report’s estimate of 250,000 deaths are very bad, resulting in inaccurate findings that exaggerate potential harm in ERs. The estimate was derived from a single study that included only 503 patients discharged from two Canadian emergency rooms from August to December 2004. Researchers found that among the 503 patients, one person unexpectedly died related to a delay in diagnosis by an ER physician. The patient had signs of an aortic dissection—a tear in the major vessel that carries blood from the heart. For reasons we don’t know, the diagnosis was delayed for seven hours.

The goal of the Canadian study was to measure all kinds of medical errors, not to estimate the death rate from erroneous or late diagnoses. The sample size wasn’t big enough for that. Had nobody in the study sample died, would that mean that ERs never make fatal errors? Obviously not. 

The AHRQ report misused this single death to estimate a death rate across the entire U.S. Dividing one death by 503 patients, the researchers estimate a death rate of 0.2%. They then multiply 0.2% by total annual ER visits in the U.S.—130 million—and come up with 250,000 deaths.

This is not how epidemiology works. One cannot extrapolate a death rate across a nation from a single death. 

A standard analysis also reports a confidence interval—a statistical estimate of the uncertainty around the finding. The smaller the sample size, the wider the confidence interval, indicating more uncertainty in the estimate.

The AHRQ report calculated a confidence interval for the 0.2% death rate and found the “possible” range was anywhere from 0.005% to 1.1%, or 6,500 to 1.4 million deaths. The researchers dismissed that interval as “implausibly wide” and arbitrarily went with a new range—0.1% to 0.4%, 130,000 deaths to 520,000 deaths. This arbitrary range is also statistically invalid—statistical parameters come from objective analysis of the data, not guesstimates.

The AHRQ report tried to justify the new range by citing other sources, none of which hold up to scrutiny or were intended to measure ER misdiagnosis-related deaths in the U.S. The sources included a small study from Switzerland (33 deaths) that found an association with increased mortality when the ER diagnosis and final diagnosis didn’t match—but didn’t review the medical charts. Another was a study of Medicare data that, if interpreted as ER physician error, effectively requires ER doctors to be clairvoyant. A third was a small study in Spain of patients returning to the ER after discharge that reported three deaths—again, way too small to calculate a nation-wide death rate. 

Finally, the AHRQ report cited an estimate of U.S. hospital (not ER-specific) fatal misdiagnosis rates that is more than two decades old. It was based on a systematic review of autopsy studies from the last century that projected a “modern” estimate for the year 2000.

The estimate of 250,000 deaths a year from ER misdiagnosis is statistically specious speculation, not a carefully measured fact. It pulls data from very small studies, the majority of which are from other countries and past decades, and none of which were designed to assess the frequency of ER misdiagnosis-related death. The report then inappropriately extrapolates those numbers across an entire nation.

This, my friends, is JUNK SCIENCE and only helps to perpetuate the trend of patients NOT trusting doctors. Do not listen to the corrupt media who never questions anything anymore.

Get our awesome newsletter by signing up here. It’s FREE!!! And we don’t share your email with anyone.