The VA Proves Why Rating Systems and Quality Metrics Don’t Work

Want some more proof how the system can be gamed or rigged?  Or how about bogus ratings and metrics?  Well, here is the VA doing its thing again:

When the Department of Veterans Affairs released the annual ratings of its hospitals this fall, the facility in Atlanta dropped to the bottom, while the one in West Haven, Conn., shot to the top. It was something of a mystery as to why.

The Atlanta hospital was downgraded to one star from three on the agency’s five-star scale, even though there had been only a “trivial change” in its quality data from the year before, according to the department. The Connecticut hospital climbed to five stars from three, even though numerous operations had to be performed elsewhere or canceled at the last minute because of problems with sterilization of surgical tools, according to an internal assessment and other accounts cited by Senator Richard Blumenthal in a letter to the agency.

Veterans Affairs set up the rating system in 2012 in the hope of pushing its hospitals to improve, and it has been increasingly aggressive in using the ratings to hold hospital managers accountable. Leaders with low ratings can be ousted, as happened last week in Atlanta, where the chief of staff and heads of the emergency department, primary care and clinical access services were removed because of low scores.

But former senior officials at the agency and experts in health care metrics say the system can be confusing, and so arbitrary that hospitals may gain and lose stars based only on statistical error. More than a dozen hospitals improved care but lost stars; another did not improve and gained one.

What is most worrisome to some experts is the role that the star ratings now play in grading performance of hospitals and their managers. They say it creates an incentive to conceal problems rather than grapple with them, in order to collect bonuses or sidestep penalties.

You can find the rest of the article here in the NY Times.  The question is whether we can truly measure quality?


Will people game the system to make themselves look better?


The article is proof of everything I have been blogging about over the last decade.

Get our awesome newsletter by signing up here. We don’t give your email out and we don’t spam you.)
123690cookie-checkThe VA Proves Why Rating Systems and Quality Metrics Don’t Work