Hospital ratings for kids a roll of the dice: study
URL of this page: http://www.nlm.nih.gov/medlineplus/news/fullstory_116199.html(*this news item will not be available after 12/05/2011)
Tuesday, September 6, 2011
By Frederik Joelving
NEW YORK (Reuters Health) - A key component of children's hospital ratings may be statistically unreliable, fueling concerns over popular rankings used by millions of Americans every year, according to a new study.
Researchers found that death rates across hospitals, which carry heavy weight in commercial rankings like the U.S. News & World Report Best Hospitals, are mostly indistinguishable from a statistical point of view.
"It's like taking a stack of cards and shuffling them almost at random to put them in a rank order," said Dr. Chris Feudtner of The Children's Hospital of Philadelphia, whose findings appear in the journal Pediatrics.
"You are chasing after something that is being buffeted around by chance," he told Reuters Health.
Hospital death rates have become an increasingly popular yardstick for the quality of care that patients receive around the country and have been made publicly available by the Department of Health and Human Services.
But critics like Feudtner say they may backfire if hospitals start focusing on improving a number instead of patient care. For instance, he said, there are examples of hospitals sending critically ill people home to die, so that they wouldn't drive up the death rates.
"The more powerful these measures become in the public arena, the more self-serving they become," said Feudtner. His own hospital consistently ranks among the top three best children's hospitals in U.S. News & World Report's rankings.
The new study, based on more than 475,000 children discharged from 42 children's hospitals in 2008, suggests the measures may also give a false sense of precision where little exists.
Ranking the 42 hospitals according to their adjusted death rates -- that is, taking into account how sick the kids were -- is easy enough. But each estimate comes with some uncertainty due to chance, or what is known as a confidence interval. If a number is within the confidence interval of another number, researchers agree there is no way to know if they are really different.
Feudtner and his colleagues found that half of the confidence intervals overlapped at least 22 of the 42 possible ranking positions. That means a hospital ranked number 15 might as well have been ranked number 37.
The death rates for hospitals at the top and the bottom were more robust, but still overlapped five other positions, so that number 2 could easily have been number 7 and vice versa.
Avery Comarow, the Health Rankings editor at the U.S. News & World Report, said he did not agree that the company's rankings may reflect chance.
"The study implies that all we look at is mortality," he told Reuters Health. "In fact even in the adults ranking it is less than a third of the score and in pediatrics it is as low as 20 percent."
Still, he said mortality is "one of many key factors" in the rankings.
He also criticized the new study for being too broad and including all patients instead of only the sickest ones, who may be more indicative of a hospital's true quality of care.
"We only look at the difficult or complex cases," said Comarow. "With those cases there is going to be a much wider range from hospital to hospital."
He acknowledged, however, that narrowing the estimates to the sickest patients will give less trustworthy statistics.
"But that is a trade-off that we are more than willing to make," Comarow explained.
Another measure that factors into the U.S. News & World Report rankings is a hospital's reputations among doctors. But that measure also makes some experts feel uneasy.
"I think it is time to acknowledge that this ranking offers very little in the way of valuable information," wrote Paul Levy, former president and CEO of Beth Israel Deaconess Medical Center in Boston, in a recent blog post.
"It is mainly a vehicle for advertisements from the pharmaceutical industry, who know that this issue of the magazine gets a lot of attention and high circulation. As you flip through to each specialty, you are blasted with ads for drugs related to syndromes within that specialty."
Instead of using rankings, Feudtner said, researchers should try to identify a few of the best hospitals and tease apart how they achieve their outcomes.
"We want to find people who are running ahead of the pack and find out how to copy them, but beyond that the rankings appear questionable," he told Reuters Health.
As for patients, Feudtner said they are unlikely to be harmed by picking hospitals that have done better on the rankings.
"But they have to be aware that if the rankings are not that different they should be allowed to think about other things -- how close is it to home, will my family be able to visit me?"
SOURCE: http://bit.ly/qUzbx3 Pediatrics, online September 2, 2011.
NEW YORK (Reuters Health) - A key component of children's hospital ratings may be statistically unreliable, fueling concerns over popular rankings used by millions of Americans every year, according to a new study.
Researchers found that death rates across hospitals, which carry heavy weight in commercial rankings like the U.S. News & World Report Best Hospitals, are mostly indistinguishable from a statistical point of view.
"It's like taking a stack of cards and shuffling them almost at random to put them in a rank order," said Dr. Chris Feudtner of The Children's Hospital of Philadelphia, whose findings appear in the journal Pediatrics.
"You are chasing after something that is being buffeted around by chance," he told Reuters Health.
Hospital death rates have become an increasingly popular yardstick for the quality of care that patients receive around the country and have been made publicly available by the Department of Health and Human Services.
But critics like Feudtner say they may backfire if hospitals start focusing on improving a number instead of patient care. For instance, he said, there are examples of hospitals sending critically ill people home to die, so that they wouldn't drive up the death rates.
"The more powerful these measures become in the public arena, the more self-serving they become," said Feudtner. His own hospital consistently ranks among the top three best children's hospitals in U.S. News & World Report's rankings.
The new study, based on more than 475,000 children discharged from 42 children's hospitals in 2008, suggests the measures may also give a false sense of precision where little exists.
Ranking the 42 hospitals according to their adjusted death rates -- that is, taking into account how sick the kids were -- is easy enough. But each estimate comes with some uncertainty due to chance, or what is known as a confidence interval. If a number is within the confidence interval of another number, researchers agree there is no way to know if they are really different.
Feudtner and his colleagues found that half of the confidence intervals overlapped at least 22 of the 42 possible ranking positions. That means a hospital ranked number 15 might as well have been ranked number 37.
The death rates for hospitals at the top and the bottom were more robust, but still overlapped five other positions, so that number 2 could easily have been number 7 and vice versa.
Avery Comarow, the Health Rankings editor at the U.S. News & World Report, said he did not agree that the company's rankings may reflect chance.
"The study implies that all we look at is mortality," he told Reuters Health. "In fact even in the adults ranking it is less than a third of the score and in pediatrics it is as low as 20 percent."
Still, he said mortality is "one of many key factors" in the rankings.
He also criticized the new study for being too broad and including all patients instead of only the sickest ones, who may be more indicative of a hospital's true quality of care.
"We only look at the difficult or complex cases," said Comarow. "With those cases there is going to be a much wider range from hospital to hospital."
He acknowledged, however, that narrowing the estimates to the sickest patients will give less trustworthy statistics.
"But that is a trade-off that we are more than willing to make," Comarow explained.
Another measure that factors into the U.S. News & World Report rankings is a hospital's reputations among doctors. But that measure also makes some experts feel uneasy.
"I think it is time to acknowledge that this ranking offers very little in the way of valuable information," wrote Paul Levy, former president and CEO of Beth Israel Deaconess Medical Center in Boston, in a recent blog post.
"It is mainly a vehicle for advertisements from the pharmaceutical industry, who know that this issue of the magazine gets a lot of attention and high circulation. As you flip through to each specialty, you are blasted with ads for drugs related to syndromes within that specialty."
Instead of using rankings, Feudtner said, researchers should try to identify a few of the best hospitals and tease apart how they achieve their outcomes.
"We want to find people who are running ahead of the pack and find out how to copy them, but beyond that the rankings appear questionable," he told Reuters Health.
As for patients, Feudtner said they are unlikely to be harmed by picking hospitals that have done better on the rankings.
"But they have to be aware that if the rankings are not that different they should be allowed to think about other things -- how close is it to home, will my family be able to visit me?"
SOURCE: http://bit.ly/qUzbx3 Pediatrics, online September 2, 2011.
Reuters Health
(c) Copyright Thomson Reuters 2011. Check for restrictions at: http://about.reuters.com/fulllegal.asp
- More Health News on:
- Children's Health
- Health Facilities
- Patient Safety
No hay comentarios:
Publicar un comentario