The VA Proves Why Rating Systems and Quality Metrics Don’t Work
Want some more proof how the system can be gamed or rigged? Or how about bogus ratings and metrics? Well, here is the VA doing its thing again:
When the Department of Veterans Affairs released the annual ratings of its hospitals this fall, the facility in Atlanta dropped to the bottom, while the one in West Haven, Conn., shot to the top. It was something of a mystery as to why.
The Atlanta hospital was downgraded to one star from three on the agency’s five-star scale, even though there had been only a “trivial change” in its quality data from the year before, according to the department. The Connecticut hospital climbed to five stars from three, even though numerous operations had to be performed elsewhere or canceled at the last minute because of problems with sterilization of surgical tools, according to an internal assessment and other accounts cited by Senator Richard Blumenthal in a letter to the agency.
Veterans Affairs set up the rating system in 2012 in the hope of pushing its hospitals to improve, and it has been increasingly aggressive in using the ratings to hold hospital managers accountable. Leaders with low ratings can be ousted, as happened last week in Atlanta, where the chief of staff and heads of the emergency department, primary care and clinical access services were removed because of low scores.
But former senior officials at the agency and experts in health care metrics say the system can be confusing, and so arbitrary that hospitals may gain and lose stars based only on statistical error. More than a dozen hospitals improved care but lost stars; another did not improve and gained one.
What is most worrisome to some experts is the role that the star ratings now play in grading performance of hospitals and their managers. They say it creates an incentive to conceal problems rather than grapple with them, in order to collect bonuses or sidestep penalties.
You can find the rest of the article here in the NY Times. The question is whether we can truly measure quality?
No.
Will people game the system to make themselves look better?
Yes.
The article is proof of everything I have been blogging about over the last decade.
Get our awesome newsletter by signing up here. We don’t give your email out and we don’t spam you.)
I’m living this now. I’m at the Hampton VA which just went from a 1 star to 2 stars. I see no difference in Primary Care. I am the 6th provider on this panel in 3 years and they burned me out in 8 months. I asked for help from the Chief of staff and Department Head and they said they would get right on it. I work 11 hour days and I still can’t finish all the work. The assistant to the Chief of staff, Mrs Durio, wanted me to stay late to do a computer class. I told her I already work 11 hours a day. She told me that’s no excuse. I will definitely send her a Be Well email when I leave.
In the book Extreme Ownership, the authors talk about how APPROPRIATE incentives can drive positive changes and how complex, poorly thought out incentives don’t. The VA is all about complex incentive programs that pay lip service to “performance improvement” without actually DRIVING performance improvement. It’s old news… “Same as it ever was… Same as it ever was…”
The VA just needs to be watched closely and contained. They provide some good service to veterans and that is their ONLY reason for being.
Spot on, Doug, spot on!????????
I work for VA and my location is one of the best. I have been to several as a a vet and they are all unique. I realized the bogusness of this system at a recent meeting where we were told we went from 5 to 4 stars because we didn’t improve. We were doing great! Maintaining should equate to a sustained score as well. Another example of bean counters trying to practice medicine. Running a company making widgets is not the same as running a medical establishment.