If you could hire sick children as mystery shoppers, you could learn a lot about care quality and variability in the health system. Imagine sending a child with a common ailment, such as an ear infection, to see a series of unwitting pediatricians, then comparing their performance. Which pediatricians prescribed antibiotics, which ones chose the most appropriate type for that child’s infection, and how many of them fell short of meeting recommendations for when and how to prescribe antimicrobial drugs?
Following these guidelines consistently is key, both for avoiding overtreatment and side effects for children and for preventing resistant bacterial strains. But because individual pediatricians generally work alone, they may not recognize if their regular prescribing habits are less than ideal.
“Even though clinicians often work in large practice groups, the day-to-day practice of medicine in primary care is usually solitary, and that lends itself to variability,” said Jeffrey Gerber, MD, PhD, an attending physician, medical director of the antimicrobial stewardship program, and associate director of the Center for Pediatric Clinical Effectiveness at Children’s Hospital of Philadelphia. “Primary care antibiotic prescribing is probably one of the best templates to look at that because it’s a common, bread-and-butter scenario we can capture in large numbers, and there are only a few diagnoses for which antibiotics are prescribed.”
Since an actual “mystery shopper” study would clearly not be practical or appropriate for the care of real sick children, Dr. Gerber and colleagues used statistical analysis of prescribing data to model that type of scenario instead. Using data from electronic health records across CHOP’s Pediatric Research Consortium (PeRC) primary care practices representing more than 100,000 patient visits for common childhood infections in one year, they adjusted for patient differences so they were statistically the same, with the physician practice as the only variable. This effort was part of a study to identify whether and where there was room for improvement.
The article describing this effort, published in the Journal of the Pediatric Infectious Diseases Society in December 2015, has earned Dr. Gerber the Caroline B. Hall Clinically Innovative Research Award from the Pediatric Infectious Diseases Society (PIDS) Education and Research Foundation, presented during IDWeek 2016, a combined annual scientific meeting of PIDS and several other professional organizations focused on infectious diseases held in October. In recognition of this project and his work more broadly, Dr. Gerber was also selected as Penn Medicine’s 2016 recipient of the Marjorie A. Bowman New Investigator Award for achievements in the health evaluation sciences, which will be presented Nov. 9.
The study’s adjusted analysis showed a threefold variation in whether antibiotics were prescribed, ranging from 18 percent to 36 percent of patient visits among different practices. There was nearly a fourfold variation in prescribing broad-spectrum antibiotics (which are not recommended as a first-line choice for most common infections), ranging from 15 to 58 percent of visits across practices. No matter how they sliced the data — looking only at visits when ear infections were diagnosed, or only sinusitis, for example — the story was the same. Some practices prescribed antibiotics in general, and broad-spectrum antibiotics in particular, far more often than others did. The analyses excluded well visits, patients with chronic conditions, and patients who had been prescribed another antibiotic on a recent prior visit.
For Dr. Gerber, this study was only a first step in more ways than one. The project began while he was wrapping up his fellowship and moving toward his current faculty position, in which he is also an assistant professor of Pediatrics and Epidemiology at the Perelman School of Medicine at the University of Pennsylvania. Working with senior colleagues including Louis Bell, MD, Alex Fiks, MD, MSCE, Ron Keren, MD, MPH, and Theoklis Zaoutis, MD, MSCE, he led the study of prescribing variability which was awarded funding from the U.S. Agency for Healthcare Research and Quality (AHRQ).
“That convinced us that there was an opportunity to try to create an intervention to improve this,” Dr. Gerber said.
In that intervention, participating physicians could see their own and their peers’ rates of antibiotic prescribing based on their electronic health record data. Dr. Gerber and colleagues reasoned that if more physicians could see beyond the walls of their solitary exam room to the prescribing rates of their most adherent colleagues, they would respond to that positive influence. The cluster randomized trial showed that physician practices that received this intervention had substantial improvements in their prescribing compared to other practices. The results were so noteworthy that Dr. Gerber and colleagues published their findings in the Journal of the American Medical Association (JAMA) in 2013, before attempting to publish the initial study of variability.
This next step was also not the end of the story. After physicians in the intervention trial stopped receiving feedback reports, their prescribing practices reverted back to their previous rates — a finding the team reported in a subsequent research letter in JAMA. Recognizing that continuous feedback was key to the intervention’s success, Dr. Gerber partnered with Dr. Keren, who is vice president of quality and chief quality officer at CHOP, and Lisa Biggs, MD, associate chief medical officer of the CHOP Care Network, to translate the intervention into a quality-improvement project.
“It’s an idea that has gone from an exploratory research project to research intervention to something that is now embedded into the culture of CHOP as an ongoing quality initiative thanks to Ron and Lisa’s leadership and the hard work of their group,” Dr. Gerber said. “In every physician practice, we’re now seeing almost 90 percent first-line prescribing of narrow-spectrum antibiotics.”
The project’s influence is extending beyond CHOP as well. The team received additional grant funding from AHRQ to support disseminating the intervention to other hospitals and health systems. Dr. Gerber and colleagues have created a version of the intervention and data collection protocol that can work with any electronic health record. Any practice with a clinician champion or staff who are able to work with data analysis can download the tools for free from the AHRQ website.
“That all started with this first paper,” Dr. Gerber said. “I’m really honored and pleased that PIDS recognized this as important work to help improve outpatient antimicrobial stewardship. We need to do better as researchers to bring the science of stewardship to the implementation stage and disseminate more broadly into primary care, emergency departments, and hospitals across the country.”