User login
Impact of the Choosing Wisely® Campaign Recommendations for Hospitalized Children on Clinical Practice: Trends from 2008 to 2017
The Choosing Wisely® Campaign (CWC) was launched in 2012. This ongoing national initiative encourages conversations among patients and clinicians about the need —or the lack thereof—for frequent tests, treatments, and procedures in healthcare. More than 80 professional societies have developed short lists of evidence-based recommendations aimed at avoiding unnecessary, “low-value” care. More than 550 recommendations are currently available.1 The Society of Hospital Medicine (SHM) Pediatric Committee published a list of five recommendations for the CWC in 2013.2
After seven years, the campaign has posted several success stories highlighting the increase in clinicians’ awareness about the recommendations. Several local, regional, and national initiatives and quality improvement (QI) projects have been inspired by the CWC and its tenants.1,3 However, limited research has been performed on the true impact of these recommendations on avoiding “low-value” services. A more comprehensive approach is required to “measure wisely” the impact of the campaign on bedside clinical practice.4 Stakeholders in healthcare value have been challenged to collaborate in creating high-impact lists of “low-value” interventions and designing effective tools to measure their impact on clinical practice and costs.5
We initially developed a report card with five metrics derived from the CWC-SHM pediatric recommendations to help individual institutions and group practices to measure their performance and benchmark their results with peers.6 The report card is available for hospital members of the Children’s Hospital Association (CHA).7
The current study analyzes the frequency of utilization and trends of five metrics included in the CHA/Pediatric Health Information System® (PHIS) CWC report card in tertiary children’s hospitals in the United States. We analyzed data from five years before and five years after the CWC-PHM recommendations were published in 2013. We hypothesize that the publication and dissemination of the CWC-PHM recommendations—the intervention—will result in either an immediate decrease in the use of the “low-value” services studied and/or a change in the trend of utilization over time.
METHODS
Study Design
We conducted an observational, longitudinal retrospective study aimed at evaluating the impact of the CWC-PHM recommendations on clinical practice in tertiary children’s hospitals in the US.
Study Population
The population included inpatient and observation stays for children aged 0-18 years admitted to the 36 children’s hospitals consistently providing data from 2008 to 2017 to the PHIS administrative database (CHA, Lenexa, Kansas). This database contains inpatient, emergency department, ambulatory, and observation encounter–level data from more than 50 not-for-profit, tertiary care pediatric hospitals and accounts for ~20% of all pediatric hospitalizations in the US every year.
A joint effort between the CHA and the participating hospitals ensures the quality of the data submitted, as previously described.8 These data are subjected to a routine quality check with each submission and within each report. Data were fully deidentified for this study. In total, 36 PHIS hospitals met the strict quality standards for inclusion of submitted data. The remaining hospitals were excluded because they did not have complete data or had incomplete billing information.
For external benchmarking purposes, PHIS participating hospitals provide encounter data, including demographics, diagnoses, and procedures (International Classification of Diseases versions 9 and 10).9,10 The transition from ICD-9 to ICD-10 in the US took place during the study period. However, the CHA completed a process of translating and mapping all ICD-9 codes to every possible equivalent ICD-10 code in the PHIS database. Thus, the change from ICD-9 to ICD-10 should not have had any significant effect on population definition and data analytics, including trend analysis.
For each condition, the study population was divided into the following two cohorts for comparison of the trends: all admissions from January 1, 2008 to December 31, 2012 (before) and all admissions from January 1, 2013 to December 31, 2017 (after) the CWC-PHM recommendations were published.
This study was determined to be nonhuman subject research and was therefore exempted by Nicklaus Children’s Hospital Human Research Protection Program.
Outcomes
The outcomes for this study were the percentages of patients receiving the not-recommended “low-value” services targeted by the CWC-PHM recommendations. For this purpose, four of the five recommendations were translated into the following five metrics, operationalized in the PHIS database and displayed in the “Choosing Wisely” report card:6
1. Percentage of patients with uncomplicated asthma receiving chest radiograph (CXR).
2. Percentage of patients with uncomplicated bronchiolitis receiving CXR.
3. Percentage of patients with uncomplicated bronchiolitis receiving bronchodilators.
4. Percentage of patients with lower respiratory tract infection (LRTI) receiving systemic corticosteroids (relievers).
5. Percentage of patients with uncomplicated gastroesophageal reflux (GER) receiving acid suppressor therapy.
The fifth recommendation—limiting the use of continuous pulse oximetry unless the patient is receiving supplemental oxygen—could not be operationalized in the PHIS database because of inconsistent reporting of these resources.6
The resulting percentages represent nonadherence to the recommendations, suggesting overuse of the specific “low-value” intervention. As such, a decreasing trend over time is the desired direction of improvement.
The definition of “uncomplicated” conditions and the metrics are presented in Table 1. A complete list of the inclusion and exclusion criteria to define “uncomplicated” conditions and the complete list of the clinical translation codes used in PHIS to identify the “low-value” services are presented as an electronic supplement.
Statistical Analyses
We compared the demographic and clinical characteristics of the various cohorts before and after the release of the CWC-PHM recommendations—the intervention—using chi-square statistics. To assess the individual hospital-level trends over time for each measure, we modeled the patient-level data of each hospital using generalized linear mixed effects models with a binomial distribution. These models were adjusted for patient demographic and clinical factors that were found to be significantly different (P < .01) before and after the intervention on bivariate analyses. From these models, we generated adjusted estimates for the quarterly percentages for each hospital. We then conducted an interrupted time series (ITS) using these estimates to compare trends in the five years before (2008-2012) and five years after (2013-2017) the publication of the CWC-PHM recommendations. For the ITS analysis, we used a generalized linear mixed effects model with the quarterly adjusted hospital-level utilization rates of “low-value” services for each cohort as the unit of analysis and a random intercept for each hospital. The model used an autoregressive(1) covariance structure to account for autocorrelation. The ITS allowed us to test our hypothesis by assessing the following two important features: (a) if a significant decrease occurred right after the CWC-PHM recommendations were published (level-change) and/or (b) if the intervention altered the secular trend (slope-change). All statistical analyses were performed using SAS v. 9.4 (SAS Institute, Cary, North Carolina), and P values <.01 were considered to be statistically significant.
RESULTS
Table 2 presents the demographic characteristics of the cohorts before (2008-2012) and after (2013-2017) the publication of the CWC-PHM recommendations. Hospitalizations due to asthma represented the largest cohort with 142,067 cases, followed by hospitalizations due to bronchiolitis with 94,253 cases. Hospitalizations due to GER comprised the smallest cohort with 13,635 cases. Most of the children had government insurance and had “minor” severity according to the All Patient Revised Diagnosis Related Group (APR-DRG) system.
We found statistically significant differences in most of the demographic characteristics for the cohorts when comparing cases before and after the introduction of the CWC-PHM recommendations.
After adjusting for demographic characteristics, we estimated the percentages of the utilization of the “low-value” services from 2008 to 2017. We observed a steady decrease in overutilization of all services over time. The absolute percentage decrease was more evident in the reduction of the utilization of relievers by 36.6% and that of CXR by 31.5% for bronchiolitis. We also observed a 20.8% absolute reduction in the use of CXR for asthma.
The use of systemic steroids in LRTI revealed the lowest utilization among the “low-value” services studied, with 15.1% in 2008 and 12.2% in 2017, a 2.9% absolute reduction. However, the prescription of acid suppressors for GER showed the highest utilization among all the overuse metrics studied, ie, 63% in 2008 and 48.9% in 2017, with an absolute decrease of 24.1%. The yearly adjusted estimated percentages of utilization for each “low-value” service are presented in Appendix Table A.
Table 3 and the Figure (attached as supplemental online graphic) respectively present the risk-adjusted ITS parameter estimates and the graphic representation before and after the inception of the CWC-PHM recommendations for the trend analysis.
During the five years preceding the intervention (2008-2012), a statistically significant decrease (P < .01) was already noted in the trend of utilization of relievers and CXR in bronchiolitis and CXR in asthma. However, we found no significant change in the trend of the use of systemic corticosteroids in cases with LRTI or the use of acid suppression therapy for GER.
The immediate effect of the intervention is represented by the level change. We found a statistically significant (P < .01) reduction according to the CWC-PHM recommendations only for the use of CXR in hospitalized children with uncomplicated asthma.
During the five years after the CWC-PHM recommendations were published (2013-2017), a sustained, significant decrease in the trend of the use of CXR in asthma and bronchiolitis and the use of relievers in bronchiolitis (P < .01) was observed. However, there was no significant change in the trend of the use of systemic corticosteroids in cases with LRTI or in the use of acid suppression therapy for GER during this period.
Comparison of the trends before and after the publication of the CWC-PHM recommendations revealed that only the decreasing trend in the use of relievers for bronchiolitis over time significantly correlated with the campaign (P < .01).
DISCUSSION
We found a steady reduction in the frequency of overutilization of five “low-value” services described in the CWC-PHM recommendations from 2008 to 2017 in 36 tertiary children’s hospitals in the US. This trend was more evident in the utilization of relievers and CXR for bronchiolitis. The ITS analysis demonstrated that immediately after the publication of the CWC-PHM recommendations, only the use of CXR for asthma decreased significantly. Then, only the use of relievers for bronchiolitis decreased significantly over time in comparison with the secular trend.
These results support our hypothesis for two of the five metrics studied, suggesting that the publication of the CWC-PHM recommendations had a modest impact in clinical practices related to those services in tertiary children’s hospitals.
These findings align with a limited number of published studies that have consistently found a modest decrease in the use of “low-value” services before 201211-13 and a limited impact of the CWC in clinical practices on the use of “low-value” services after the inception of the campaign.14-17
For instance, in a cross-sectional analysis of the 1999 and 2009 samples of ambulatory care practices in the US, only two of 11 overuse quality indicators showed improvement.11 The authors recognized that reducing inappropriate care will require the same attention to guideline development and performance measurement that was directed at reducing the underuse of needed therapies. However, determining whether a patient received inappropriate care generally requires a much more detailed analysis of clinical information than what is required for assessments of underuse.11
Another study designed claims-based algorithms to measure the prevalence of 11 Choosing Wisely-identified “low-value” services in fee-for-service Medicare patients aged >65 years from 2006 to 2011.12 The annual prevalence of selected CWC “low-value” services ranged from 1.2% (upper urinary tract imaging in men with benign prostatic hyperplasia) to 46.5% (preoperative cardiac testing for low-risk, noncardiac procedures). The study concluded that identifying and measuring “low-value” health services is a prerequisite for improving quality and eliminating waste.12
In pediatric medicine, the authors investigated a large cohort of infants aged one to 24 months hospitalized with bronchiolitis to 41 tertiary children’s hospitals reporting data to the PHIS database from 2004 to 2012.13 The trend analysis revealed a decrease in the utilization of diagnostics and treatment interventions before the publication of the American Academy of Pediatrics 2006 Bronchiolitis Guidelines.18 There was an additional reduction in the use of CXR, steroids, and bronchodilators after the publication of the guidelines.13
After the CWC was launched in 2012, several surveys have demonstrated a tangible increase in awareness of the CWC and its goals, mostly among primary care physicians and subspecialists. Clinicians who were aware of the campaign found the recommendations to be useful as a legitimate source of guidance and were more likely to reduce the indication of unnecessary care and “low-value” clinical services included in the CWC.1,3,19,20
Few studies in adults have focused on measuring the trends in overuse metrics derived from the CWC recommendations.14-16 The initial studies have found limited reduction on the use of “low-value” care after the inception of the CWC. They suggest that clinician education, awareness, and public promotion alone do not appear to be sufficient to achieve widespread changes in clinical practice. Additional interventions are necessary for the wider implementation and success of the CWC recommendations.11,14,15,19,21,22
However, a more recent study was conducted in 91 academic centers from 2013 through 2016, before and after the publication of a CWC recommendation on the use of troponin-only testing for the diagnosis of acute myocardial infarction. Hospitals with low rates of troponin-only testing before the publication of the recommendation demonstrated a statistically significant increase over time in the rate of adherence. The authors postulated that the impact of the CWC might have been significant because of the increase in the institutional and provider attention to “high-value” care as a result of the campaign.16
In pediatrics, a cross-sectional study defined 20 “low-value” services from a list of more than 400 items from the CWC and other sources of highly regarded, evidence-based pediatrics healthcare recommendations. The list included six diagnostic tests, five imaging tests, and nine prescription drugs ordered in a robust cohort of 4.4 million children nationwide in 2014. The study concluded that approximately one in 10 children received a “low-value” service. The majority (59.4%) were related to prescription drugs, specifically the inappropriate use of antibiotics for a variety of conditions. The estimated combined cost of these unnecessary services was approximately $27 million, with one-third of the cost being paid out of pocket, arguing for significant financial harm. However, this study did not perform a trend analysis.17
Our results are comparable with these studies, reporting an initial increase in awareness and beliefs, followed by progressive changes in clinical practice among pediatric hospital-based clinicians in delivering evidence-based, high-value care after the CWC.
The attribution of the steady reduction in the absolute percentages of overuse/waste in the five metrics related to the CWC observed in this study, including the significant changes noted in two of the overuse indicators after the publication of the CWC-PHM recommendations, should be interpreted with caution. For example, the significant decrease in the use of “low-value” services in bronchiolitis could be attributed to multiple factors such as national guidelines released in 2014 after the campaign,23 national multicenter QI collaborative projects,24,25 and multiple local QI efforts.26,27 The increase in the awareness and impact of the CWC recommendations among pediatric providers could also be a contributing factor, but this association cannot be established in the light of our findings.
On the other hand, despite extensive evidence for the lack of efficacy and the potential harm associated with the use of acid suppressors for uncomplicated GER in infants,28-30 the frequency of this “low-value” therapeutic intervention remains high (~50%). The trend in utilization was not impacted by the CWC-PHM recommendations. This finding could be explained by several factors, including the possibility that several hospitalized patients may suffer from GER disease requiring acid suppressors. Another possibility is that acid suppressors are generally prescribed as an outpatient medication, and physicians treating inpatients may be reluctant to discontinue it during hospitalization. Nevertheless, this recommendation represents a target for review, update, and QI interventions in the near future.
The delivery of inappropriate “low-value” care represents the most significant dimension of waste in healthcare.31 The development of quality measures of “low-value” services representing overuse and waste is the most needed step toward assessing the magnitude of the problem. Overuse metrics could be incorporated into QI interventions to decrease the provision of such services. However, systematic efforts aimed at developing quality indicators of overuse based on the CWC recommendations have been limited. To our knowledge, this is the first study on the trends of metrics derived from the CWC recommendations in pediatric medicine.
Future research is needed to develop overuse metrics further to assess the specific outcomes related to the implementation of the CWC. How much has clinical practice changed as a result of the campaign? What are the outcomes and savings attributable to these efforts? These are critical questions for the immediate future that should be answered to sustain the ongoing efforts and results and to validate that the efforts are worthwhile.
This study has several limitations. First, this is a retrospective and observational study. It cannot prove a direct causal relationship between the publication of the CWC-PHM and the observed trends, as other potential factors may have contributed to the outcomes. Second, in administrative databases, the data quality is dependent on proper documentation and coding that may vary among reporting institutions. These data lack clinical information, and a fair assessment of “appropriateness” could be questioned. In addition, the study included only 36 academic, tertiary children’s hospitals. Because approximately two-thirds of all pediatric hospitalizations in the US occur in community settings,32 this study may not fully represent clinical practice in the majority of pediatric hospitalizations in the US. Finally, the validity of the ITS analysis has inherent limitations due to the variability of the data in some metrics that may affect the power of the analysis. This fact could lead to inaccurate conclusions regarding intervention effectiveness due to the data-driven model applied, as well as the lack of control for other time-varying confounders.33
CONCLUSIONS
After seven years, the CWC faces important challenges. Critical to the success of the campaign is to “measure wisely” by developing quality indicators of overuse and operationalizing them into administrative and clinical data sources to assess the impact on clinical practice. Our study highlights some limited but steady reduction in the use of some “low-value” services before the campaign. It also demonstrates a modest impact of the campaign on clinical practices in tertiary care children’s hospitals in the US. Clinicians and institutions still have a long way to go in reducing the use of “low-value” interventions in pediatric medicine. These observations challenge us to step up our efforts to implement QI interventions aimed at incorporating these professional, society-endorsed recommendations into our clinical practice.
Acknowledgments
The authors thank Dr. Kristine De La Torre and Dr. Jennifer McCafferty-Fernandez and the Research Institute of Nicklaus Children’s Hospital for medical writing assistance. They also acknowledge Tatiana Consuegra, library technician, for her clerical assistance in the preparation and submission of this article.
1. Choosing Wisely. Choosing Wisely Campaign Official Site. http://www.choosingwisely.org/. Accessed May 2019.
2. Quinonez RA, Garber MD, Schroeder AR, et al. Choosing wisely in pediatric hospital medicine: five opportunities for improved healthcare value. J Hosp Med. 2013;8(9):479-485. https://doi.org/10.1002/jhm.2064.
3. ABIM Foundation CR. Choosing Wisely: A Special Report on the First Five Years. http://www.choosingwisely.org/choosing-wisely-a-special-report-on-the-first-five-years/. Updated 2017. Accessed May 2019.
4. Wolfson D, Santa J, Slass L. Engaging physicians and consumers in conversations about treatment overuse and waste: a short history of the choosing wisely campaign. Acad Med. 2014;89(7):990-995. https://doi.org/10.1097/ACM.0000000000000270.
5. Morden NE, Colla CH, Sequist TD, Rosenthal MB. Choosing wisely—the politics and economics of labeling low-value services. N Engl J Med. 2014;370(7):589-592. https://doi.org/10.1056/NEJMp1314965.
6. Reyes M, Paulus E, Hronek C, et al. Choosing wisely campaign: Report card and achievable benchmarks of care for children’s hospitals. Hosp Pediatr. 2017;7(11):633-641. https://doi.org/10.1542/hpeds.2017-0029.
7. Report Cards. Choosing Wisely Measures - Pediatric Hospital Medicine Detail Reports. Children’s Hospital Association Web site. https://www.childrenshospitals.org/. Accessed May 2019.
8. Mongelluzzo J, Mohamad Z, Ten Have TR, Shah SS. Corticosteroids and mortality in children with bacterial meningitis. JAMA. 2008;299(17):2048-2055. https://doi.org/10.1001/jama.299.17.2048.
9. Buck CJ. 2013 ICD 9 CM for Physicians, Volumes 1 & 2. Chicago, IL: American Medical Association; 2013.
10. Buck CJ. 2018 ICD-10-CM for Physicians. Chicago, IL: American Medical Association; 2018.
11. Kale MS, Bishop TF, Federman AD, Keyhani S. Trends in the overuse of ambulatory health care services in the United States. JAMA Inter Med. 2013;173(2):142-148. https://doi.org/10.1001/2013.jamainternmed.1022.
12. Colla CH, Morden NE, Sequist TD, Schpero WL, Rosenthal MB. Choosing wisely: Prevalence and correlates of low-value health care services in the United States. J Gen Intern Med. 2015;30(2):221-228. https://doi.org/10.1007/s11606-014-3070-z
13. Parikh K, Hall M, Teach SJ. Bronchiolitis management before and after the AAP guidelines. Pediatrics. 2014;133(1): e1-7. https://doi.org/10.1542/peds.2013-2005.
14. Rosenberg A, Agiro A, Gottlieb M, et al. Early trends among seven recommendations from the Choosing Wisely campaign. JAMA Inter Med. 2015;175(12):1913-1920. https://doi.org/10.1001/jamainternmed.2015.5441.
15. Reid RO, Rabideau B, Sood N. Low-value health care services in a commercially insured population. JAMA Inter Med. 2016;176(10):1567-1571. https://doi.org/10.1001/jamainternmed.2016.5031.
16. Prochaska MT, Hohmann SF, Modes M, Arora VM. Trends in troponin-only testing for AMI in academic teaching hospitals and the impact of choosing wisely(R). J Hosp Med. 2017;12(12):957-962. https://doi.org/10.12788/jhm.2846.
17. Chua KP, Schwartz AL, Volerman A, Conti RM, Huang ES. Use of low-value pediatric services among the commercially insured. Pediatrics. 2016;138(6):e20161809. https://doi.org/10.1542/peds.2016-1809.
18. American Academy of Pediatrics Subcommittee on Diagnosis and Management of Bronchiolitis. Diagnosis and management of bronchiolitis. Pediatrics. 2006;118(4):1774-1793.
19. Colla CH, Kinsella EA, Morden NE, Meyers DJ, Rosenthal MB, Sequist TD. Physician perceptions of Choosing Wisely and drivers of overuse. Am J Manag Care. 2016;22(5):337-343.
20. PerryUndem Research/Communication AF. DataBrief: Findings from a National Survey of Physicians. http://www.choosingwisely.org/wp-content/uploads/2017/10/Summary-Research-Report-Survey-2017.pdf. Updated 2017.
21. Wolfson D. Choosing wisely recommendations using administrative claims data. JAMA Inter Med. 2016;176(4):565. https://doi.org/10.1001/jamainternmed.2016.0357.
22. Heekin AM, Kontor J, Sax HC, Keller M, Wellington A, Weingarten S. Choosing wisely clinical decision support adherence and associated patient outcomes. Am J Manag Care. 2018;24(8):361-366.
23. Ralston SL, Lieberthal AS, Meissner HC, et al. Clinical practice guideline: the diagnosis, management, and prevention of bronchiolitis. Pediatrics. 2014;134(5):e1474-e502. https://doi.org/10.1542/peds.2014-2742.
24. Ralston SL, Garber MD, Rice-Conboy E, et al. A multicenter collaborative to reduce unnecessary care in inpatient bronchiolitis. Pediatrics. 2016;137(1):e20150851. https://doi.org/10.1542/peds.2015-0851.
25. Mussman GM, Lossius M, Wasif F, et al. Multisite emergency department inpatient collaborative to reduce unnecessary bronchiolitis care. Pediatrics. 2018;141(2):e20170830. https://doi.org/10.1542/peds.2017-0830.
26. Mittal V, Hall M, Morse R, et al. Impact of inpatient bronchiolitis clinical practice guideline implementation on testing and treatment. J Pediatr. 2014;165(3):570-576. https://doi.org/10.1016/j.jpeds.2014.05.021.
27. Tyler A, Krack P, Bakel LA, et al. Interventions to reduce over-utilized tests and treatments in bronchiolitis. Pediatrics. 2018;141(6):e20170485. https://doi.org/10.1542/peds.2017-0485.
28. Rosen R, Vandenplas Y, Singendonk M, et al. Pediatric gastroesophageal reflux clinical practice guidelines: joint recommendations of the North American Society for Pediatric Gastroenterology, Hepatology, and Nutrition and the European Society for Pediatric Gastroenterology, Hepatology, and Nutrition. J Pediatr Gastroenterol Nutr. 2018;66(3):516-554. https://doi.org/10.1097/MPG.0b013e3181b7f563.
29. Eichenwald EC, COMMITTEE ON FETUS AND NEWBORN. Diagnosis and management of gastroesophageal reflux in preterm infants. Pediatrics. 2018;142(1):e20181061. https://doi.org/10.1542/peds.2018-1061
30. van der Pol RJ, Smits MJ, van Wijk MP, Omari TI, Tabbers MM, Benninga MA. Efficacy of proton-pump inhibitors in children with gastroesophageal reflux disease: a systematic review. Pediatrics. 2011;127(5):925-935. https://doi.org/10.1542/peds.2010-2719.
31. IOM Report: Estimated $750B Wasted Annually In Health Care System. Kaiser Health News Web site. https://khn.org/morning-breakout/iom-report/. Updated 2012. Accessed May 2019.
32. Leyenaar JK, Ralston SL, Shieh M, Pekow PS, Mangione‐Smith R, Lindenauer PK. Epidemiology of pediatric hospitalizations at general hospitals and freestanding children’s hospitals in the United States. J Hosp Med. 2016;11(11):743-749. https://doi.org/10.1002/jhm.2624.
33. Bernal JL, Cummins S, Gasparrini A. Interrupted time series regression for the evaluation of public health interventions: a tutorial. Int J Epidemiol. 2017;46(1):348-355. https://doi.org/10.1093/ije/dyw098.
The Choosing Wisely® Campaign (CWC) was launched in 2012. This ongoing national initiative encourages conversations among patients and clinicians about the need —or the lack thereof—for frequent tests, treatments, and procedures in healthcare. More than 80 professional societies have developed short lists of evidence-based recommendations aimed at avoiding unnecessary, “low-value” care. More than 550 recommendations are currently available.1 The Society of Hospital Medicine (SHM) Pediatric Committee published a list of five recommendations for the CWC in 2013.2
After seven years, the campaign has posted several success stories highlighting the increase in clinicians’ awareness about the recommendations. Several local, regional, and national initiatives and quality improvement (QI) projects have been inspired by the CWC and its tenants.1,3 However, limited research has been performed on the true impact of these recommendations on avoiding “low-value” services. A more comprehensive approach is required to “measure wisely” the impact of the campaign on bedside clinical practice.4 Stakeholders in healthcare value have been challenged to collaborate in creating high-impact lists of “low-value” interventions and designing effective tools to measure their impact on clinical practice and costs.5
We initially developed a report card with five metrics derived from the CWC-SHM pediatric recommendations to help individual institutions and group practices to measure their performance and benchmark their results with peers.6 The report card is available for hospital members of the Children’s Hospital Association (CHA).7
The current study analyzes the frequency of utilization and trends of five metrics included in the CHA/Pediatric Health Information System® (PHIS) CWC report card in tertiary children’s hospitals in the United States. We analyzed data from five years before and five years after the CWC-PHM recommendations were published in 2013. We hypothesize that the publication and dissemination of the CWC-PHM recommendations—the intervention—will result in either an immediate decrease in the use of the “low-value” services studied and/or a change in the trend of utilization over time.
METHODS
Study Design
We conducted an observational, longitudinal retrospective study aimed at evaluating the impact of the CWC-PHM recommendations on clinical practice in tertiary children’s hospitals in the US.
Study Population
The population included inpatient and observation stays for children aged 0-18 years admitted to the 36 children’s hospitals consistently providing data from 2008 to 2017 to the PHIS administrative database (CHA, Lenexa, Kansas). This database contains inpatient, emergency department, ambulatory, and observation encounter–level data from more than 50 not-for-profit, tertiary care pediatric hospitals and accounts for ~20% of all pediatric hospitalizations in the US every year.
A joint effort between the CHA and the participating hospitals ensures the quality of the data submitted, as previously described.8 These data are subjected to a routine quality check with each submission and within each report. Data were fully deidentified for this study. In total, 36 PHIS hospitals met the strict quality standards for inclusion of submitted data. The remaining hospitals were excluded because they did not have complete data or had incomplete billing information.
For external benchmarking purposes, PHIS participating hospitals provide encounter data, including demographics, diagnoses, and procedures (International Classification of Diseases versions 9 and 10).9,10 The transition from ICD-9 to ICD-10 in the US took place during the study period. However, the CHA completed a process of translating and mapping all ICD-9 codes to every possible equivalent ICD-10 code in the PHIS database. Thus, the change from ICD-9 to ICD-10 should not have had any significant effect on population definition and data analytics, including trend analysis.
For each condition, the study population was divided into the following two cohorts for comparison of the trends: all admissions from January 1, 2008 to December 31, 2012 (before) and all admissions from January 1, 2013 to December 31, 2017 (after) the CWC-PHM recommendations were published.
This study was determined to be nonhuman subject research and was therefore exempted by Nicklaus Children’s Hospital Human Research Protection Program.
Outcomes
The outcomes for this study were the percentages of patients receiving the not-recommended “low-value” services targeted by the CWC-PHM recommendations. For this purpose, four of the five recommendations were translated into the following five metrics, operationalized in the PHIS database and displayed in the “Choosing Wisely” report card:6
1. Percentage of patients with uncomplicated asthma receiving chest radiograph (CXR).
2. Percentage of patients with uncomplicated bronchiolitis receiving CXR.
3. Percentage of patients with uncomplicated bronchiolitis receiving bronchodilators.
4. Percentage of patients with lower respiratory tract infection (LRTI) receiving systemic corticosteroids (relievers).
5. Percentage of patients with uncomplicated gastroesophageal reflux (GER) receiving acid suppressor therapy.
The fifth recommendation—limiting the use of continuous pulse oximetry unless the patient is receiving supplemental oxygen—could not be operationalized in the PHIS database because of inconsistent reporting of these resources.6
The resulting percentages represent nonadherence to the recommendations, suggesting overuse of the specific “low-value” intervention. As such, a decreasing trend over time is the desired direction of improvement.
The definition of “uncomplicated” conditions and the metrics are presented in Table 1. A complete list of the inclusion and exclusion criteria to define “uncomplicated” conditions and the complete list of the clinical translation codes used in PHIS to identify the “low-value” services are presented as an electronic supplement.
Statistical Analyses
We compared the demographic and clinical characteristics of the various cohorts before and after the release of the CWC-PHM recommendations—the intervention—using chi-square statistics. To assess the individual hospital-level trends over time for each measure, we modeled the patient-level data of each hospital using generalized linear mixed effects models with a binomial distribution. These models were adjusted for patient demographic and clinical factors that were found to be significantly different (P < .01) before and after the intervention on bivariate analyses. From these models, we generated adjusted estimates for the quarterly percentages for each hospital. We then conducted an interrupted time series (ITS) using these estimates to compare trends in the five years before (2008-2012) and five years after (2013-2017) the publication of the CWC-PHM recommendations. For the ITS analysis, we used a generalized linear mixed effects model with the quarterly adjusted hospital-level utilization rates of “low-value” services for each cohort as the unit of analysis and a random intercept for each hospital. The model used an autoregressive(1) covariance structure to account for autocorrelation. The ITS allowed us to test our hypothesis by assessing the following two important features: (a) if a significant decrease occurred right after the CWC-PHM recommendations were published (level-change) and/or (b) if the intervention altered the secular trend (slope-change). All statistical analyses were performed using SAS v. 9.4 (SAS Institute, Cary, North Carolina), and P values <.01 were considered to be statistically significant.
RESULTS
Table 2 presents the demographic characteristics of the cohorts before (2008-2012) and after (2013-2017) the publication of the CWC-PHM recommendations. Hospitalizations due to asthma represented the largest cohort with 142,067 cases, followed by hospitalizations due to bronchiolitis with 94,253 cases. Hospitalizations due to GER comprised the smallest cohort with 13,635 cases. Most of the children had government insurance and had “minor” severity according to the All Patient Revised Diagnosis Related Group (APR-DRG) system.
We found statistically significant differences in most of the demographic characteristics for the cohorts when comparing cases before and after the introduction of the CWC-PHM recommendations.
After adjusting for demographic characteristics, we estimated the percentages of the utilization of the “low-value” services from 2008 to 2017. We observed a steady decrease in overutilization of all services over time. The absolute percentage decrease was more evident in the reduction of the utilization of relievers by 36.6% and that of CXR by 31.5% for bronchiolitis. We also observed a 20.8% absolute reduction in the use of CXR for asthma.
The use of systemic steroids in LRTI revealed the lowest utilization among the “low-value” services studied, with 15.1% in 2008 and 12.2% in 2017, a 2.9% absolute reduction. However, the prescription of acid suppressors for GER showed the highest utilization among all the overuse metrics studied, ie, 63% in 2008 and 48.9% in 2017, with an absolute decrease of 24.1%. The yearly adjusted estimated percentages of utilization for each “low-value” service are presented in Appendix Table A.
Table 3 and the Figure (attached as supplemental online graphic) respectively present the risk-adjusted ITS parameter estimates and the graphic representation before and after the inception of the CWC-PHM recommendations for the trend analysis.
During the five years preceding the intervention (2008-2012), a statistically significant decrease (P < .01) was already noted in the trend of utilization of relievers and CXR in bronchiolitis and CXR in asthma. However, we found no significant change in the trend of the use of systemic corticosteroids in cases with LRTI or the use of acid suppression therapy for GER.
The immediate effect of the intervention is represented by the level change. We found a statistically significant (P < .01) reduction according to the CWC-PHM recommendations only for the use of CXR in hospitalized children with uncomplicated asthma.
During the five years after the CWC-PHM recommendations were published (2013-2017), a sustained, significant decrease in the trend of the use of CXR in asthma and bronchiolitis and the use of relievers in bronchiolitis (P < .01) was observed. However, there was no significant change in the trend of the use of systemic corticosteroids in cases with LRTI or in the use of acid suppression therapy for GER during this period.
Comparison of the trends before and after the publication of the CWC-PHM recommendations revealed that only the decreasing trend in the use of relievers for bronchiolitis over time significantly correlated with the campaign (P < .01).
DISCUSSION
We found a steady reduction in the frequency of overutilization of five “low-value” services described in the CWC-PHM recommendations from 2008 to 2017 in 36 tertiary children’s hospitals in the US. This trend was more evident in the utilization of relievers and CXR for bronchiolitis. The ITS analysis demonstrated that immediately after the publication of the CWC-PHM recommendations, only the use of CXR for asthma decreased significantly. Then, only the use of relievers for bronchiolitis decreased significantly over time in comparison with the secular trend.
These results support our hypothesis for two of the five metrics studied, suggesting that the publication of the CWC-PHM recommendations had a modest impact in clinical practices related to those services in tertiary children’s hospitals.
These findings align with a limited number of published studies that have consistently found a modest decrease in the use of “low-value” services before 201211-13 and a limited impact of the CWC in clinical practices on the use of “low-value” services after the inception of the campaign.14-17
For instance, in a cross-sectional analysis of the 1999 and 2009 samples of ambulatory care practices in the US, only two of 11 overuse quality indicators showed improvement.11 The authors recognized that reducing inappropriate care will require the same attention to guideline development and performance measurement that was directed at reducing the underuse of needed therapies. However, determining whether a patient received inappropriate care generally requires a much more detailed analysis of clinical information than what is required for assessments of underuse.11
Another study designed claims-based algorithms to measure the prevalence of 11 Choosing Wisely-identified “low-value” services in fee-for-service Medicare patients aged >65 years from 2006 to 2011.12 The annual prevalence of selected CWC “low-value” services ranged from 1.2% (upper urinary tract imaging in men with benign prostatic hyperplasia) to 46.5% (preoperative cardiac testing for low-risk, noncardiac procedures). The study concluded that identifying and measuring “low-value” health services is a prerequisite for improving quality and eliminating waste.12
In pediatric medicine, the authors investigated a large cohort of infants aged one to 24 months hospitalized with bronchiolitis to 41 tertiary children’s hospitals reporting data to the PHIS database from 2004 to 2012.13 The trend analysis revealed a decrease in the utilization of diagnostics and treatment interventions before the publication of the American Academy of Pediatrics 2006 Bronchiolitis Guidelines.18 There was an additional reduction in the use of CXR, steroids, and bronchodilators after the publication of the guidelines.13
After the CWC was launched in 2012, several surveys have demonstrated a tangible increase in awareness of the CWC and its goals, mostly among primary care physicians and subspecialists. Clinicians who were aware of the campaign found the recommendations to be useful as a legitimate source of guidance and were more likely to reduce the indication of unnecessary care and “low-value” clinical services included in the CWC.1,3,19,20
Few studies in adults have focused on measuring the trends in overuse metrics derived from the CWC recommendations.14-16 The initial studies have found limited reduction on the use of “low-value” care after the inception of the CWC. They suggest that clinician education, awareness, and public promotion alone do not appear to be sufficient to achieve widespread changes in clinical practice. Additional interventions are necessary for the wider implementation and success of the CWC recommendations.11,14,15,19,21,22
However, a more recent study was conducted in 91 academic centers from 2013 through 2016, before and after the publication of a CWC recommendation on the use of troponin-only testing for the diagnosis of acute myocardial infarction. Hospitals with low rates of troponin-only testing before the publication of the recommendation demonstrated a statistically significant increase over time in the rate of adherence. The authors postulated that the impact of the CWC might have been significant because of the increase in the institutional and provider attention to “high-value” care as a result of the campaign.16
In pediatrics, a cross-sectional study defined 20 “low-value” services from a list of more than 400 items from the CWC and other sources of highly regarded, evidence-based pediatrics healthcare recommendations. The list included six diagnostic tests, five imaging tests, and nine prescription drugs ordered in a robust cohort of 4.4 million children nationwide in 2014. The study concluded that approximately one in 10 children received a “low-value” service. The majority (59.4%) were related to prescription drugs, specifically the inappropriate use of antibiotics for a variety of conditions. The estimated combined cost of these unnecessary services was approximately $27 million, with one-third of the cost being paid out of pocket, arguing for significant financial harm. However, this study did not perform a trend analysis.17
Our results are comparable with these studies, reporting an initial increase in awareness and beliefs, followed by progressive changes in clinical practice among pediatric hospital-based clinicians in delivering evidence-based, high-value care after the CWC.
The attribution of the steady reduction in the absolute percentages of overuse/waste in the five metrics related to the CWC observed in this study, including the significant changes noted in two of the overuse indicators after the publication of the CWC-PHM recommendations, should be interpreted with caution. For example, the significant decrease in the use of “low-value” services in bronchiolitis could be attributed to multiple factors such as national guidelines released in 2014 after the campaign,23 national multicenter QI collaborative projects,24,25 and multiple local QI efforts.26,27 The increase in the awareness and impact of the CWC recommendations among pediatric providers could also be a contributing factor, but this association cannot be established in the light of our findings.
On the other hand, despite extensive evidence for the lack of efficacy and the potential harm associated with the use of acid suppressors for uncomplicated GER in infants,28-30 the frequency of this “low-value” therapeutic intervention remains high (~50%). The trend in utilization was not impacted by the CWC-PHM recommendations. This finding could be explained by several factors, including the possibility that several hospitalized patients may suffer from GER disease requiring acid suppressors. Another possibility is that acid suppressors are generally prescribed as an outpatient medication, and physicians treating inpatients may be reluctant to discontinue it during hospitalization. Nevertheless, this recommendation represents a target for review, update, and QI interventions in the near future.
The delivery of inappropriate “low-value” care represents the most significant dimension of waste in healthcare.31 The development of quality measures of “low-value” services representing overuse and waste is the most needed step toward assessing the magnitude of the problem. Overuse metrics could be incorporated into QI interventions to decrease the provision of such services. However, systematic efforts aimed at developing quality indicators of overuse based on the CWC recommendations have been limited. To our knowledge, this is the first study on the trends of metrics derived from the CWC recommendations in pediatric medicine.
Future research is needed to develop overuse metrics further to assess the specific outcomes related to the implementation of the CWC. How much has clinical practice changed as a result of the campaign? What are the outcomes and savings attributable to these efforts? These are critical questions for the immediate future that should be answered to sustain the ongoing efforts and results and to validate that the efforts are worthwhile.
This study has several limitations. First, this is a retrospective and observational study. It cannot prove a direct causal relationship between the publication of the CWC-PHM and the observed trends, as other potential factors may have contributed to the outcomes. Second, in administrative databases, the data quality is dependent on proper documentation and coding that may vary among reporting institutions. These data lack clinical information, and a fair assessment of “appropriateness” could be questioned. In addition, the study included only 36 academic, tertiary children’s hospitals. Because approximately two-thirds of all pediatric hospitalizations in the US occur in community settings,32 this study may not fully represent clinical practice in the majority of pediatric hospitalizations in the US. Finally, the validity of the ITS analysis has inherent limitations due to the variability of the data in some metrics that may affect the power of the analysis. This fact could lead to inaccurate conclusions regarding intervention effectiveness due to the data-driven model applied, as well as the lack of control for other time-varying confounders.33
CONCLUSIONS
After seven years, the CWC faces important challenges. Critical to the success of the campaign is to “measure wisely” by developing quality indicators of overuse and operationalizing them into administrative and clinical data sources to assess the impact on clinical practice. Our study highlights some limited but steady reduction in the use of some “low-value” services before the campaign. It also demonstrates a modest impact of the campaign on clinical practices in tertiary care children’s hospitals in the US. Clinicians and institutions still have a long way to go in reducing the use of “low-value” interventions in pediatric medicine. These observations challenge us to step up our efforts to implement QI interventions aimed at incorporating these professional, society-endorsed recommendations into our clinical practice.
Acknowledgments
The authors thank Dr. Kristine De La Torre and Dr. Jennifer McCafferty-Fernandez and the Research Institute of Nicklaus Children’s Hospital for medical writing assistance. They also acknowledge Tatiana Consuegra, library technician, for her clerical assistance in the preparation and submission of this article.
The Choosing Wisely® Campaign (CWC) was launched in 2012. This ongoing national initiative encourages conversations among patients and clinicians about the need —or the lack thereof—for frequent tests, treatments, and procedures in healthcare. More than 80 professional societies have developed short lists of evidence-based recommendations aimed at avoiding unnecessary, “low-value” care. More than 550 recommendations are currently available.1 The Society of Hospital Medicine (SHM) Pediatric Committee published a list of five recommendations for the CWC in 2013.2
After seven years, the campaign has posted several success stories highlighting the increase in clinicians’ awareness about the recommendations. Several local, regional, and national initiatives and quality improvement (QI) projects have been inspired by the CWC and its tenants.1,3 However, limited research has been performed on the true impact of these recommendations on avoiding “low-value” services. A more comprehensive approach is required to “measure wisely” the impact of the campaign on bedside clinical practice.4 Stakeholders in healthcare value have been challenged to collaborate in creating high-impact lists of “low-value” interventions and designing effective tools to measure their impact on clinical practice and costs.5
We initially developed a report card with five metrics derived from the CWC-SHM pediatric recommendations to help individual institutions and group practices to measure their performance and benchmark their results with peers.6 The report card is available for hospital members of the Children’s Hospital Association (CHA).7
The current study analyzes the frequency of utilization and trends of five metrics included in the CHA/Pediatric Health Information System® (PHIS) CWC report card in tertiary children’s hospitals in the United States. We analyzed data from five years before and five years after the CWC-PHM recommendations were published in 2013. We hypothesize that the publication and dissemination of the CWC-PHM recommendations—the intervention—will result in either an immediate decrease in the use of the “low-value” services studied and/or a change in the trend of utilization over time.
METHODS
Study Design
We conducted an observational, longitudinal retrospective study aimed at evaluating the impact of the CWC-PHM recommendations on clinical practice in tertiary children’s hospitals in the US.
Study Population
The population included inpatient and observation stays for children aged 0-18 years admitted to the 36 children’s hospitals consistently providing data from 2008 to 2017 to the PHIS administrative database (CHA, Lenexa, Kansas). This database contains inpatient, emergency department, ambulatory, and observation encounter–level data from more than 50 not-for-profit, tertiary care pediatric hospitals and accounts for ~20% of all pediatric hospitalizations in the US every year.
A joint effort between the CHA and the participating hospitals ensures the quality of the data submitted, as previously described.8 These data are subjected to a routine quality check with each submission and within each report. Data were fully deidentified for this study. In total, 36 PHIS hospitals met the strict quality standards for inclusion of submitted data. The remaining hospitals were excluded because they did not have complete data or had incomplete billing information.
For external benchmarking purposes, PHIS participating hospitals provide encounter data, including demographics, diagnoses, and procedures (International Classification of Diseases versions 9 and 10).9,10 The transition from ICD-9 to ICD-10 in the US took place during the study period. However, the CHA completed a process of translating and mapping all ICD-9 codes to every possible equivalent ICD-10 code in the PHIS database. Thus, the change from ICD-9 to ICD-10 should not have had any significant effect on population definition and data analytics, including trend analysis.
For each condition, the study population was divided into the following two cohorts for comparison of the trends: all admissions from January 1, 2008 to December 31, 2012 (before) and all admissions from January 1, 2013 to December 31, 2017 (after) the CWC-PHM recommendations were published.
This study was determined to be nonhuman subject research and was therefore exempted by Nicklaus Children’s Hospital Human Research Protection Program.
Outcomes
The outcomes for this study were the percentages of patients receiving the not-recommended “low-value” services targeted by the CWC-PHM recommendations. For this purpose, four of the five recommendations were translated into the following five metrics, operationalized in the PHIS database and displayed in the “Choosing Wisely” report card:6
1. Percentage of patients with uncomplicated asthma receiving chest radiograph (CXR).
2. Percentage of patients with uncomplicated bronchiolitis receiving CXR.
3. Percentage of patients with uncomplicated bronchiolitis receiving bronchodilators.
4. Percentage of patients with lower respiratory tract infection (LRTI) receiving systemic corticosteroids (relievers).
5. Percentage of patients with uncomplicated gastroesophageal reflux (GER) receiving acid suppressor therapy.
The fifth recommendation—limiting the use of continuous pulse oximetry unless the patient is receiving supplemental oxygen—could not be operationalized in the PHIS database because of inconsistent reporting of these resources.6
The resulting percentages represent nonadherence to the recommendations, suggesting overuse of the specific “low-value” intervention. As such, a decreasing trend over time is the desired direction of improvement.
The definition of “uncomplicated” conditions and the metrics are presented in Table 1. A complete list of the inclusion and exclusion criteria to define “uncomplicated” conditions and the complete list of the clinical translation codes used in PHIS to identify the “low-value” services are presented as an electronic supplement.
Statistical Analyses
We compared the demographic and clinical characteristics of the various cohorts before and after the release of the CWC-PHM recommendations—the intervention—using chi-square statistics. To assess the individual hospital-level trends over time for each measure, we modeled the patient-level data of each hospital using generalized linear mixed effects models with a binomial distribution. These models were adjusted for patient demographic and clinical factors that were found to be significantly different (P < .01) before and after the intervention on bivariate analyses. From these models, we generated adjusted estimates for the quarterly percentages for each hospital. We then conducted an interrupted time series (ITS) using these estimates to compare trends in the five years before (2008-2012) and five years after (2013-2017) the publication of the CWC-PHM recommendations. For the ITS analysis, we used a generalized linear mixed effects model with the quarterly adjusted hospital-level utilization rates of “low-value” services for each cohort as the unit of analysis and a random intercept for each hospital. The model used an autoregressive(1) covariance structure to account for autocorrelation. The ITS allowed us to test our hypothesis by assessing the following two important features: (a) if a significant decrease occurred right after the CWC-PHM recommendations were published (level-change) and/or (b) if the intervention altered the secular trend (slope-change). All statistical analyses were performed using SAS v. 9.4 (SAS Institute, Cary, North Carolina), and P values <.01 were considered to be statistically significant.
RESULTS
Table 2 presents the demographic characteristics of the cohorts before (2008-2012) and after (2013-2017) the publication of the CWC-PHM recommendations. Hospitalizations due to asthma represented the largest cohort with 142,067 cases, followed by hospitalizations due to bronchiolitis with 94,253 cases. Hospitalizations due to GER comprised the smallest cohort with 13,635 cases. Most of the children had government insurance and had “minor” severity according to the All Patient Revised Diagnosis Related Group (APR-DRG) system.
We found statistically significant differences in most of the demographic characteristics for the cohorts when comparing cases before and after the introduction of the CWC-PHM recommendations.
After adjusting for demographic characteristics, we estimated the percentages of the utilization of the “low-value” services from 2008 to 2017. We observed a steady decrease in overutilization of all services over time. The absolute percentage decrease was more evident in the reduction of the utilization of relievers by 36.6% and that of CXR by 31.5% for bronchiolitis. We also observed a 20.8% absolute reduction in the use of CXR for asthma.
The use of systemic steroids in LRTI revealed the lowest utilization among the “low-value” services studied, with 15.1% in 2008 and 12.2% in 2017, a 2.9% absolute reduction. However, the prescription of acid suppressors for GER showed the highest utilization among all the overuse metrics studied, ie, 63% in 2008 and 48.9% in 2017, with an absolute decrease of 24.1%. The yearly adjusted estimated percentages of utilization for each “low-value” service are presented in Appendix Table A.
Table 3 and the Figure (attached as supplemental online graphic) respectively present the risk-adjusted ITS parameter estimates and the graphic representation before and after the inception of the CWC-PHM recommendations for the trend analysis.
During the five years preceding the intervention (2008-2012), a statistically significant decrease (P < .01) was already noted in the trend of utilization of relievers and CXR in bronchiolitis and CXR in asthma. However, we found no significant change in the trend of the use of systemic corticosteroids in cases with LRTI or the use of acid suppression therapy for GER.
The immediate effect of the intervention is represented by the level change. We found a statistically significant (P < .01) reduction according to the CWC-PHM recommendations only for the use of CXR in hospitalized children with uncomplicated asthma.
During the five years after the CWC-PHM recommendations were published (2013-2017), a sustained, significant decrease in the trend of the use of CXR in asthma and bronchiolitis and the use of relievers in bronchiolitis (P < .01) was observed. However, there was no significant change in the trend of the use of systemic corticosteroids in cases with LRTI or in the use of acid suppression therapy for GER during this period.
Comparison of the trends before and after the publication of the CWC-PHM recommendations revealed that only the decreasing trend in the use of relievers for bronchiolitis over time significantly correlated with the campaign (P < .01).
DISCUSSION
We found a steady reduction in the frequency of overutilization of five “low-value” services described in the CWC-PHM recommendations from 2008 to 2017 in 36 tertiary children’s hospitals in the US. This trend was more evident in the utilization of relievers and CXR for bronchiolitis. The ITS analysis demonstrated that immediately after the publication of the CWC-PHM recommendations, only the use of CXR for asthma decreased significantly. Then, only the use of relievers for bronchiolitis decreased significantly over time in comparison with the secular trend.
These results support our hypothesis for two of the five metrics studied, suggesting that the publication of the CWC-PHM recommendations had a modest impact in clinical practices related to those services in tertiary children’s hospitals.
These findings align with a limited number of published studies that have consistently found a modest decrease in the use of “low-value” services before 201211-13 and a limited impact of the CWC in clinical practices on the use of “low-value” services after the inception of the campaign.14-17
For instance, in a cross-sectional analysis of the 1999 and 2009 samples of ambulatory care practices in the US, only two of 11 overuse quality indicators showed improvement.11 The authors recognized that reducing inappropriate care will require the same attention to guideline development and performance measurement that was directed at reducing the underuse of needed therapies. However, determining whether a patient received inappropriate care generally requires a much more detailed analysis of clinical information than what is required for assessments of underuse.11
Another study designed claims-based algorithms to measure the prevalence of 11 Choosing Wisely-identified “low-value” services in fee-for-service Medicare patients aged >65 years from 2006 to 2011.12 The annual prevalence of selected CWC “low-value” services ranged from 1.2% (upper urinary tract imaging in men with benign prostatic hyperplasia) to 46.5% (preoperative cardiac testing for low-risk, noncardiac procedures). The study concluded that identifying and measuring “low-value” health services is a prerequisite for improving quality and eliminating waste.12
In pediatric medicine, the authors investigated a large cohort of infants aged one to 24 months hospitalized with bronchiolitis to 41 tertiary children’s hospitals reporting data to the PHIS database from 2004 to 2012.13 The trend analysis revealed a decrease in the utilization of diagnostics and treatment interventions before the publication of the American Academy of Pediatrics 2006 Bronchiolitis Guidelines.18 There was an additional reduction in the use of CXR, steroids, and bronchodilators after the publication of the guidelines.13
After the CWC was launched in 2012, several surveys have demonstrated a tangible increase in awareness of the CWC and its goals, mostly among primary care physicians and subspecialists. Clinicians who were aware of the campaign found the recommendations to be useful as a legitimate source of guidance and were more likely to reduce the indication of unnecessary care and “low-value” clinical services included in the CWC.1,3,19,20
Few studies in adults have focused on measuring the trends in overuse metrics derived from the CWC recommendations.14-16 The initial studies have found limited reduction on the use of “low-value” care after the inception of the CWC. They suggest that clinician education, awareness, and public promotion alone do not appear to be sufficient to achieve widespread changes in clinical practice. Additional interventions are necessary for the wider implementation and success of the CWC recommendations.11,14,15,19,21,22
However, a more recent study was conducted in 91 academic centers from 2013 through 2016, before and after the publication of a CWC recommendation on the use of troponin-only testing for the diagnosis of acute myocardial infarction. Hospitals with low rates of troponin-only testing before the publication of the recommendation demonstrated a statistically significant increase over time in the rate of adherence. The authors postulated that the impact of the CWC might have been significant because of the increase in the institutional and provider attention to “high-value” care as a result of the campaign.16
In pediatrics, a cross-sectional study defined 20 “low-value” services from a list of more than 400 items from the CWC and other sources of highly regarded, evidence-based pediatrics healthcare recommendations. The list included six diagnostic tests, five imaging tests, and nine prescription drugs ordered in a robust cohort of 4.4 million children nationwide in 2014. The study concluded that approximately one in 10 children received a “low-value” service. The majority (59.4%) were related to prescription drugs, specifically the inappropriate use of antibiotics for a variety of conditions. The estimated combined cost of these unnecessary services was approximately $27 million, with one-third of the cost being paid out of pocket, arguing for significant financial harm. However, this study did not perform a trend analysis.17
Our results are comparable with these studies, reporting an initial increase in awareness and beliefs, followed by progressive changes in clinical practice among pediatric hospital-based clinicians in delivering evidence-based, high-value care after the CWC.
The attribution of the steady reduction in the absolute percentages of overuse/waste in the five metrics related to the CWC observed in this study, including the significant changes noted in two of the overuse indicators after the publication of the CWC-PHM recommendations, should be interpreted with caution. For example, the significant decrease in the use of “low-value” services in bronchiolitis could be attributed to multiple factors such as national guidelines released in 2014 after the campaign,23 national multicenter QI collaborative projects,24,25 and multiple local QI efforts.26,27 The increase in the awareness and impact of the CWC recommendations among pediatric providers could also be a contributing factor, but this association cannot be established in the light of our findings.
On the other hand, despite extensive evidence for the lack of efficacy and the potential harm associated with the use of acid suppressors for uncomplicated GER in infants,28-30 the frequency of this “low-value” therapeutic intervention remains high (~50%). The trend in utilization was not impacted by the CWC-PHM recommendations. This finding could be explained by several factors, including the possibility that several hospitalized patients may suffer from GER disease requiring acid suppressors. Another possibility is that acid suppressors are generally prescribed as an outpatient medication, and physicians treating inpatients may be reluctant to discontinue it during hospitalization. Nevertheless, this recommendation represents a target for review, update, and QI interventions in the near future.
The delivery of inappropriate “low-value” care represents the most significant dimension of waste in healthcare.31 The development of quality measures of “low-value” services representing overuse and waste is the most needed step toward assessing the magnitude of the problem. Overuse metrics could be incorporated into QI interventions to decrease the provision of such services. However, systematic efforts aimed at developing quality indicators of overuse based on the CWC recommendations have been limited. To our knowledge, this is the first study on the trends of metrics derived from the CWC recommendations in pediatric medicine.
Future research is needed to develop overuse metrics further to assess the specific outcomes related to the implementation of the CWC. How much has clinical practice changed as a result of the campaign? What are the outcomes and savings attributable to these efforts? These are critical questions for the immediate future that should be answered to sustain the ongoing efforts and results and to validate that the efforts are worthwhile.
This study has several limitations. First, this is a retrospective and observational study. It cannot prove a direct causal relationship between the publication of the CWC-PHM and the observed trends, as other potential factors may have contributed to the outcomes. Second, in administrative databases, the data quality is dependent on proper documentation and coding that may vary among reporting institutions. These data lack clinical information, and a fair assessment of “appropriateness” could be questioned. In addition, the study included only 36 academic, tertiary children’s hospitals. Because approximately two-thirds of all pediatric hospitalizations in the US occur in community settings,32 this study may not fully represent clinical practice in the majority of pediatric hospitalizations in the US. Finally, the validity of the ITS analysis has inherent limitations due to the variability of the data in some metrics that may affect the power of the analysis. This fact could lead to inaccurate conclusions regarding intervention effectiveness due to the data-driven model applied, as well as the lack of control for other time-varying confounders.33
CONCLUSIONS
After seven years, the CWC faces important challenges. Critical to the success of the campaign is to “measure wisely” by developing quality indicators of overuse and operationalizing them into administrative and clinical data sources to assess the impact on clinical practice. Our study highlights some limited but steady reduction in the use of some “low-value” services before the campaign. It also demonstrates a modest impact of the campaign on clinical practices in tertiary care children’s hospitals in the US. Clinicians and institutions still have a long way to go in reducing the use of “low-value” interventions in pediatric medicine. These observations challenge us to step up our efforts to implement QI interventions aimed at incorporating these professional, society-endorsed recommendations into our clinical practice.
Acknowledgments
The authors thank Dr. Kristine De La Torre and Dr. Jennifer McCafferty-Fernandez and the Research Institute of Nicklaus Children’s Hospital for medical writing assistance. They also acknowledge Tatiana Consuegra, library technician, for her clerical assistance in the preparation and submission of this article.
1. Choosing Wisely. Choosing Wisely Campaign Official Site. http://www.choosingwisely.org/. Accessed May 2019.
2. Quinonez RA, Garber MD, Schroeder AR, et al. Choosing wisely in pediatric hospital medicine: five opportunities for improved healthcare value. J Hosp Med. 2013;8(9):479-485. https://doi.org/10.1002/jhm.2064.
3. ABIM Foundation CR. Choosing Wisely: A Special Report on the First Five Years. http://www.choosingwisely.org/choosing-wisely-a-special-report-on-the-first-five-years/. Updated 2017. Accessed May 2019.
4. Wolfson D, Santa J, Slass L. Engaging physicians and consumers in conversations about treatment overuse and waste: a short history of the choosing wisely campaign. Acad Med. 2014;89(7):990-995. https://doi.org/10.1097/ACM.0000000000000270.
5. Morden NE, Colla CH, Sequist TD, Rosenthal MB. Choosing wisely—the politics and economics of labeling low-value services. N Engl J Med. 2014;370(7):589-592. https://doi.org/10.1056/NEJMp1314965.
6. Reyes M, Paulus E, Hronek C, et al. Choosing wisely campaign: Report card and achievable benchmarks of care for children’s hospitals. Hosp Pediatr. 2017;7(11):633-641. https://doi.org/10.1542/hpeds.2017-0029.
7. Report Cards. Choosing Wisely Measures - Pediatric Hospital Medicine Detail Reports. Children’s Hospital Association Web site. https://www.childrenshospitals.org/. Accessed May 2019.
8. Mongelluzzo J, Mohamad Z, Ten Have TR, Shah SS. Corticosteroids and mortality in children with bacterial meningitis. JAMA. 2008;299(17):2048-2055. https://doi.org/10.1001/jama.299.17.2048.
9. Buck CJ. 2013 ICD 9 CM for Physicians, Volumes 1 & 2. Chicago, IL: American Medical Association; 2013.
10. Buck CJ. 2018 ICD-10-CM for Physicians. Chicago, IL: American Medical Association; 2018.
11. Kale MS, Bishop TF, Federman AD, Keyhani S. Trends in the overuse of ambulatory health care services in the United States. JAMA Inter Med. 2013;173(2):142-148. https://doi.org/10.1001/2013.jamainternmed.1022.
12. Colla CH, Morden NE, Sequist TD, Schpero WL, Rosenthal MB. Choosing wisely: Prevalence and correlates of low-value health care services in the United States. J Gen Intern Med. 2015;30(2):221-228. https://doi.org/10.1007/s11606-014-3070-z
13. Parikh K, Hall M, Teach SJ. Bronchiolitis management before and after the AAP guidelines. Pediatrics. 2014;133(1): e1-7. https://doi.org/10.1542/peds.2013-2005.
14. Rosenberg A, Agiro A, Gottlieb M, et al. Early trends among seven recommendations from the Choosing Wisely campaign. JAMA Inter Med. 2015;175(12):1913-1920. https://doi.org/10.1001/jamainternmed.2015.5441.
15. Reid RO, Rabideau B, Sood N. Low-value health care services in a commercially insured population. JAMA Inter Med. 2016;176(10):1567-1571. https://doi.org/10.1001/jamainternmed.2016.5031.
16. Prochaska MT, Hohmann SF, Modes M, Arora VM. Trends in troponin-only testing for AMI in academic teaching hospitals and the impact of choosing wisely(R). J Hosp Med. 2017;12(12):957-962. https://doi.org/10.12788/jhm.2846.
17. Chua KP, Schwartz AL, Volerman A, Conti RM, Huang ES. Use of low-value pediatric services among the commercially insured. Pediatrics. 2016;138(6):e20161809. https://doi.org/10.1542/peds.2016-1809.
18. American Academy of Pediatrics Subcommittee on Diagnosis and Management of Bronchiolitis. Diagnosis and management of bronchiolitis. Pediatrics. 2006;118(4):1774-1793.
19. Colla CH, Kinsella EA, Morden NE, Meyers DJ, Rosenthal MB, Sequist TD. Physician perceptions of Choosing Wisely and drivers of overuse. Am J Manag Care. 2016;22(5):337-343.
20. PerryUndem Research/Communication AF. DataBrief: Findings from a National Survey of Physicians. http://www.choosingwisely.org/wp-content/uploads/2017/10/Summary-Research-Report-Survey-2017.pdf. Updated 2017.
21. Wolfson D. Choosing wisely recommendations using administrative claims data. JAMA Inter Med. 2016;176(4):565. https://doi.org/10.1001/jamainternmed.2016.0357.
22. Heekin AM, Kontor J, Sax HC, Keller M, Wellington A, Weingarten S. Choosing wisely clinical decision support adherence and associated patient outcomes. Am J Manag Care. 2018;24(8):361-366.
23. Ralston SL, Lieberthal AS, Meissner HC, et al. Clinical practice guideline: the diagnosis, management, and prevention of bronchiolitis. Pediatrics. 2014;134(5):e1474-e502. https://doi.org/10.1542/peds.2014-2742.
24. Ralston SL, Garber MD, Rice-Conboy E, et al. A multicenter collaborative to reduce unnecessary care in inpatient bronchiolitis. Pediatrics. 2016;137(1):e20150851. https://doi.org/10.1542/peds.2015-0851.
25. Mussman GM, Lossius M, Wasif F, et al. Multisite emergency department inpatient collaborative to reduce unnecessary bronchiolitis care. Pediatrics. 2018;141(2):e20170830. https://doi.org/10.1542/peds.2017-0830.
26. Mittal V, Hall M, Morse R, et al. Impact of inpatient bronchiolitis clinical practice guideline implementation on testing and treatment. J Pediatr. 2014;165(3):570-576. https://doi.org/10.1016/j.jpeds.2014.05.021.
27. Tyler A, Krack P, Bakel LA, et al. Interventions to reduce over-utilized tests and treatments in bronchiolitis. Pediatrics. 2018;141(6):e20170485. https://doi.org/10.1542/peds.2017-0485.
28. Rosen R, Vandenplas Y, Singendonk M, et al. Pediatric gastroesophageal reflux clinical practice guidelines: joint recommendations of the North American Society for Pediatric Gastroenterology, Hepatology, and Nutrition and the European Society for Pediatric Gastroenterology, Hepatology, and Nutrition. J Pediatr Gastroenterol Nutr. 2018;66(3):516-554. https://doi.org/10.1097/MPG.0b013e3181b7f563.
29. Eichenwald EC, COMMITTEE ON FETUS AND NEWBORN. Diagnosis and management of gastroesophageal reflux in preterm infants. Pediatrics. 2018;142(1):e20181061. https://doi.org/10.1542/peds.2018-1061
30. van der Pol RJ, Smits MJ, van Wijk MP, Omari TI, Tabbers MM, Benninga MA. Efficacy of proton-pump inhibitors in children with gastroesophageal reflux disease: a systematic review. Pediatrics. 2011;127(5):925-935. https://doi.org/10.1542/peds.2010-2719.
31. IOM Report: Estimated $750B Wasted Annually In Health Care System. Kaiser Health News Web site. https://khn.org/morning-breakout/iom-report/. Updated 2012. Accessed May 2019.
32. Leyenaar JK, Ralston SL, Shieh M, Pekow PS, Mangione‐Smith R, Lindenauer PK. Epidemiology of pediatric hospitalizations at general hospitals and freestanding children’s hospitals in the United States. J Hosp Med. 2016;11(11):743-749. https://doi.org/10.1002/jhm.2624.
33. Bernal JL, Cummins S, Gasparrini A. Interrupted time series regression for the evaluation of public health interventions: a tutorial. Int J Epidemiol. 2017;46(1):348-355. https://doi.org/10.1093/ije/dyw098.
1. Choosing Wisely. Choosing Wisely Campaign Official Site. http://www.choosingwisely.org/. Accessed May 2019.
2. Quinonez RA, Garber MD, Schroeder AR, et al. Choosing wisely in pediatric hospital medicine: five opportunities for improved healthcare value. J Hosp Med. 2013;8(9):479-485. https://doi.org/10.1002/jhm.2064.
3. ABIM Foundation CR. Choosing Wisely: A Special Report on the First Five Years. http://www.choosingwisely.org/choosing-wisely-a-special-report-on-the-first-five-years/. Updated 2017. Accessed May 2019.
4. Wolfson D, Santa J, Slass L. Engaging physicians and consumers in conversations about treatment overuse and waste: a short history of the choosing wisely campaign. Acad Med. 2014;89(7):990-995. https://doi.org/10.1097/ACM.0000000000000270.
5. Morden NE, Colla CH, Sequist TD, Rosenthal MB. Choosing wisely—the politics and economics of labeling low-value services. N Engl J Med. 2014;370(7):589-592. https://doi.org/10.1056/NEJMp1314965.
6. Reyes M, Paulus E, Hronek C, et al. Choosing wisely campaign: Report card and achievable benchmarks of care for children’s hospitals. Hosp Pediatr. 2017;7(11):633-641. https://doi.org/10.1542/hpeds.2017-0029.
7. Report Cards. Choosing Wisely Measures - Pediatric Hospital Medicine Detail Reports. Children’s Hospital Association Web site. https://www.childrenshospitals.org/. Accessed May 2019.
8. Mongelluzzo J, Mohamad Z, Ten Have TR, Shah SS. Corticosteroids and mortality in children with bacterial meningitis. JAMA. 2008;299(17):2048-2055. https://doi.org/10.1001/jama.299.17.2048.
9. Buck CJ. 2013 ICD 9 CM for Physicians, Volumes 1 & 2. Chicago, IL: American Medical Association; 2013.
10. Buck CJ. 2018 ICD-10-CM for Physicians. Chicago, IL: American Medical Association; 2018.
11. Kale MS, Bishop TF, Federman AD, Keyhani S. Trends in the overuse of ambulatory health care services in the United States. JAMA Inter Med. 2013;173(2):142-148. https://doi.org/10.1001/2013.jamainternmed.1022.
12. Colla CH, Morden NE, Sequist TD, Schpero WL, Rosenthal MB. Choosing wisely: Prevalence and correlates of low-value health care services in the United States. J Gen Intern Med. 2015;30(2):221-228. https://doi.org/10.1007/s11606-014-3070-z
13. Parikh K, Hall M, Teach SJ. Bronchiolitis management before and after the AAP guidelines. Pediatrics. 2014;133(1): e1-7. https://doi.org/10.1542/peds.2013-2005.
14. Rosenberg A, Agiro A, Gottlieb M, et al. Early trends among seven recommendations from the Choosing Wisely campaign. JAMA Inter Med. 2015;175(12):1913-1920. https://doi.org/10.1001/jamainternmed.2015.5441.
15. Reid RO, Rabideau B, Sood N. Low-value health care services in a commercially insured population. JAMA Inter Med. 2016;176(10):1567-1571. https://doi.org/10.1001/jamainternmed.2016.5031.
16. Prochaska MT, Hohmann SF, Modes M, Arora VM. Trends in troponin-only testing for AMI in academic teaching hospitals and the impact of choosing wisely(R). J Hosp Med. 2017;12(12):957-962. https://doi.org/10.12788/jhm.2846.
17. Chua KP, Schwartz AL, Volerman A, Conti RM, Huang ES. Use of low-value pediatric services among the commercially insured. Pediatrics. 2016;138(6):e20161809. https://doi.org/10.1542/peds.2016-1809.
18. American Academy of Pediatrics Subcommittee on Diagnosis and Management of Bronchiolitis. Diagnosis and management of bronchiolitis. Pediatrics. 2006;118(4):1774-1793.
19. Colla CH, Kinsella EA, Morden NE, Meyers DJ, Rosenthal MB, Sequist TD. Physician perceptions of Choosing Wisely and drivers of overuse. Am J Manag Care. 2016;22(5):337-343.
20. PerryUndem Research/Communication AF. DataBrief: Findings from a National Survey of Physicians. http://www.choosingwisely.org/wp-content/uploads/2017/10/Summary-Research-Report-Survey-2017.pdf. Updated 2017.
21. Wolfson D. Choosing wisely recommendations using administrative claims data. JAMA Inter Med. 2016;176(4):565. https://doi.org/10.1001/jamainternmed.2016.0357.
22. Heekin AM, Kontor J, Sax HC, Keller M, Wellington A, Weingarten S. Choosing wisely clinical decision support adherence and associated patient outcomes. Am J Manag Care. 2018;24(8):361-366.
23. Ralston SL, Lieberthal AS, Meissner HC, et al. Clinical practice guideline: the diagnosis, management, and prevention of bronchiolitis. Pediatrics. 2014;134(5):e1474-e502. https://doi.org/10.1542/peds.2014-2742.
24. Ralston SL, Garber MD, Rice-Conboy E, et al. A multicenter collaborative to reduce unnecessary care in inpatient bronchiolitis. Pediatrics. 2016;137(1):e20150851. https://doi.org/10.1542/peds.2015-0851.
25. Mussman GM, Lossius M, Wasif F, et al. Multisite emergency department inpatient collaborative to reduce unnecessary bronchiolitis care. Pediatrics. 2018;141(2):e20170830. https://doi.org/10.1542/peds.2017-0830.
26. Mittal V, Hall M, Morse R, et al. Impact of inpatient bronchiolitis clinical practice guideline implementation on testing and treatment. J Pediatr. 2014;165(3):570-576. https://doi.org/10.1016/j.jpeds.2014.05.021.
27. Tyler A, Krack P, Bakel LA, et al. Interventions to reduce over-utilized tests and treatments in bronchiolitis. Pediatrics. 2018;141(6):e20170485. https://doi.org/10.1542/peds.2017-0485.
28. Rosen R, Vandenplas Y, Singendonk M, et al. Pediatric gastroesophageal reflux clinical practice guidelines: joint recommendations of the North American Society for Pediatric Gastroenterology, Hepatology, and Nutrition and the European Society for Pediatric Gastroenterology, Hepatology, and Nutrition. J Pediatr Gastroenterol Nutr. 2018;66(3):516-554. https://doi.org/10.1097/MPG.0b013e3181b7f563.
29. Eichenwald EC, COMMITTEE ON FETUS AND NEWBORN. Diagnosis and management of gastroesophageal reflux in preterm infants. Pediatrics. 2018;142(1):e20181061. https://doi.org/10.1542/peds.2018-1061
30. van der Pol RJ, Smits MJ, van Wijk MP, Omari TI, Tabbers MM, Benninga MA. Efficacy of proton-pump inhibitors in children with gastroesophageal reflux disease: a systematic review. Pediatrics. 2011;127(5):925-935. https://doi.org/10.1542/peds.2010-2719.
31. IOM Report: Estimated $750B Wasted Annually In Health Care System. Kaiser Health News Web site. https://khn.org/morning-breakout/iom-report/. Updated 2012. Accessed May 2019.
32. Leyenaar JK, Ralston SL, Shieh M, Pekow PS, Mangione‐Smith R, Lindenauer PK. Epidemiology of pediatric hospitalizations at general hospitals and freestanding children’s hospitals in the United States. J Hosp Med. 2016;11(11):743-749. https://doi.org/10.1002/jhm.2624.
33. Bernal JL, Cummins S, Gasparrini A. Interrupted time series regression for the evaluation of public health interventions: a tutorial. Int J Epidemiol. 2017;46(1):348-355. https://doi.org/10.1093/ije/dyw098.
© 2019 Society of Hospital Medicine