User login
Deimplementation of Routine Chest X-rays in Adult Intensive Care Units
Despite increased awareness of Choosing Wisely (CW)® recommendations to reduce low-value care,1 there is limited published data about strategies to implement these guidelines or evidence that they have influenced ordering patterns or reduced healthcare spending
In ICUs, CXR ordering strategies may be routine (daily) or on-demand (with clinical indication). The former strategy’s principal advantage is the potential to detect life-threatening situations that may otherwise escape diagnosis.11 Disadvantages include cost, radiation exposure, patient inconvenience, false-positive workups, and low diagnostic and therapeutic value.12,13 On-demand strategies may safely reduce CXR ordering by 32% to 45%.11-17 Based on this evidence, the Critical Care Societies Collaborative and the American College of Radiology have recommended on-demand CXR ordering.18,19 Here, we describe the effectiveness of an intervention to reduce CXR ordering in two ICUs while evaluating the deimplementation strategies using a validated framework.
METHODS
Setting and Design
Vanderbilt University Medical Center (VUMC) is an academic referral center in Nashville, Tennessee. The cardiovascular ICU (CVICU) has 27 beds and the medical ICU (MICU) has 34 beds. Acute care nurse practitioners (ACNPs) and two critical care physicians staff the CVICU; cardiology fellows, anesthesia critical care fellows, and transplant and cardiac surgeons are also active in patient care. The MICU is staffed by two critical care physicians who supervise one team of ACNPs and two teams of medical residents who rotate through the unit every two weeks. Each MICU team is assigned a fellow in pulmonary and critical care.
We conducted a prospective, nonrandomized study in these units from October 2015 to June 2016. The VUMC Institutional Review Board approved the intervention as a quality improvement (QI) activity, waiving the requirement for informed consent.
Intervention
Following the top CW recommendation of the Critical Care Societies Collaborative—“Don’t order diagnostic tests at regular intervals (such as every day), but rather in response to specific clinical questions.”19—the VUMC resident-led CW Steering Committee designed a multifaceted approach to reduce ordering of routine CXRs in ICUs. The intervention included a didactic session on CW and proper CXR ordering practices, peer champions, data audits, and feedback to providers through weekly e-mails (see Supplemental Materials, 1 – Resident Presentation and 2 – CXR Flyer). 20
In September 2015, CVICU and MICU teams received a didactic session highlighting CW, current CXR ordering rates, and the plan for reducing CXR ordering. On October 5, 2015, teams began receiving weekly e-mails with ordering rates defined as CXRs ordered per patient per day and a brief rationale for reducing unnecessary CXRs. To encourage friendly competition, we provided weekly rates to the MICU teams, allowing for transparent benchmarking against one another. A similar competition strategy was not used in the CVICU due to the lack of multiple teams.
In the CVICU, two ACNPs volunteered as peer champions. These champions coordinated data feedback and advocated for the intervention among their colleagues. In the MICU, three internal medicine residents volunteered as peer champions and fulfilled similar roles.
To facilitate deimplementation, we conducted two Plan-Do-Study-Act (PDSA) cycles, the first from November to mid-December 2015 and the second from mid-December 2015 to mid-January 2016. During these cycles, we tailored our deimplementation strategy based on barriers identified by the peer champions and ICU leaders (described in the Qualitative Results section). Peer champions and the CW Steering Committee generated potential solutions by conversing with stakeholders and using the Expert Recommendations for Implementing Change (ERIC).20 Interventions included disseminating promotional flyers, holding meetings with stakeholders, and providing monthly CXR ordering rates. After the PDSA cycles, we continued reexamining the deimplementation efforts by reviewing ordering rates and soliciting feedback from ICU leaders and peer champions. However, no significant changes to the intervention were made during this time.
Quantitative Evaluation
We extracted data from VUMC’s Enterprise Data Warehouse during the intervention period (October 5, 2015 to May 24, 2016) and a historical control period (October 1, 2014 to October 4, 2015). Within each ICU, descriptive statistics were used to compare patient cohorts in the baseline and intervention periods by age, sex, and race.
The primary outcome was CXRs ordered per patient per day by hospital unit (CVICU or MICU). The baseline period included all data between October 1, 2014 and September 15, 2015. To account for priming of providers from didactic education, we allowed a washout period from September 16, 2015 to October 4, 2015. As a preliminary analysis, we compared CXR rates in the baseline and intervention periods using Wilcoxon rank-sum tests. We then conducted interrupted time-series analyses with segmented linear regression to assess differences in linear trends in CXR rates over the two periods. To account for different staffing models in the MICU, we stratified the impact of the intervention by team—medical resident (physician) or ACNP. R version 3.4.0 was used for statistical analysis.21
Qualitative Evaluation
Our qualitative evaluation consisted of embedded observation and semistructured interviews with stakeholders. The qualitative portion was guided by the Consolidated Framework for Implementation Research (CFIR), a widely used framework for design and evaluation of improvement initiatives that helped us to determine major facilitators and barriers to implementation.22,23
Embedded Observation
From November 2015 to January 2016, we observed morning rounds in the CVICU and MICU one to two times weekly to understand factors facilitating and inhibiting uptake of the intervention. Observations were recorded and organized using a CFIR-based template and directed toward understanding interactions among team members (eg, the decision-making process hierarchy), team workflows and decision-making processes, process of ordering CXRs, and providers’ knowledge and perceptions of the CXR intervention (see Supplemental Material, 3 – CFIR Table).22,23 After rounds, ICU team members were invited to share suggestions for improving the intervention. All observations occurred during and shortly following morning rounds when the vast majority of routine CXRs are ordered; we did not evaluate night or evening workflows. In the spirit of continuous improvement, we evaluated data in real-time.
Semistructured Interviews
Based on the direct observations, we developed semistructured interview questions to further evaluate provider perspectives (eg, “Do you believe ICU patients need a daily CXR?”) and constructs aligning with CFIR (eg, “intervention source—internally vs externally developed;” see Supplemental Material, 4 – Interview Questions).
Stakeholders from both ICUs were recruited through e-mail and in-person requests to participate in semistructured interviews. In the CVICU, we interviewed critical care physicians, anesthesia critical care fellows, and ACNPs. In the MICU, we interviewed medical students, interns, residents, critical care fellows, attending intensivist physicians, and ACNPs. We also interviewed X-ray technologists who routinely perform portable films in the units.
RESULTS
Quantitative Results
Cardiovascular Intensive Care Unit
The median baseline CXR ordering rate in the CVICU was 1.16 CXRs per patient per day, with interquartile range (IQR) 1.06-1.28. During the intervention period, the rate dropped to 1.07 (IQR 0.94-1.21; P < .001; Table 2). The time-series analysis suggested an essentially flat trend during the baseline period, followed by a small but significant drop in ordering rates during the intervention period (P < .001; Table 3 and Figure 1). Ordering rates appeared to increase slightly over the course of the intervention period, but this slight upward trend was not significantly different from the flat trend seen during the baseline period.
Medical Intensive Care Unit
For both physician and ACPN teams, the median baseline CXR ordering rates in the MICU were much lower than the baseline rate in the CVICU (Table 2). For the MICU physician care team, the baseline CXR ordering rate was 0.60 CXRs per patient per day (IQR 0.48-0.73). For the ACNP team, the median rate was 0.39 CXRs per patient per day (IQR 0.21-0.57). Both rates stayed approximately the same during the intervention period (Table 2). The time-series analysis suggested a statistically significant but very slight downward trend in CXR ordering rates during the baseline period, in the physician (P = .011) and ACNP (P = .022) teams (Table 3, Figure 2). Under this model, a small increase in CXR ordering initially occurred during the intervention period for both physician and ACNP teams (P = .010 and P = .055, respectively), after which the rates declined slightly. Trends in ordering rates during the intervention period were not significantly different from the slight downward trends seen during the baseline period.
Qualitative Results
We identified 25 of 39 CFIR constructs as relevant to the initiative (see Supplemental Materials, 3 – CFIR Table.) We determined the major facilitators of deimplementation to be peer champion discussions about CXR ordering on rounds and weekly data feedback, particularly if accompanied by in-person follow-up.
We also noted differences in CXR ordering rationales and decisions between the units. Generally, residents in the MICU and ACNPs in the CVICU made decisions to order CXRs. However, decisions were influenced by the expectations of attending physicians. While CVICU providers tended to order CXRs reflexively as part of morning labs, MICU providers—in particular, ACNPs who had been trained on indications for proper CXR ordering—generally ordered CXRs for specific indications (eg, worsening respiratory status). Of note, MICU ACNPs reported the use of bedside ultrasound as an alternate imaging modality and a reason for their higher threshold to order CXRs.
Deimplementation barriers in both units included the need to identify goal CXR ordering rates and the intervention’s limited visibility. To address these barriers, we conducted PDSA cycles and used the CFIR and ERIC to generate potential solutions.24 We established a goal of a 20% absolute reduction in the CVICU, added monthly CXR rates to weekly e-mail reports to better account for variations in patient populations and ordering practices, and circulated materials to promote on-demand CXR ordering. Promotional materials contained guidelines on CXR ordering and five “Frequently Held Misconceptions” about ordering practices with succinct, evidence-based explanations (see Supplemental Material, 2 – CXR Flyer).
Approximately four months after the start of intervention, some CVICU physicians became concerned that on-demand CXR ordering might be inappropriate for high-risk surgical patients, including those who are undergoing or have undergone heart transplants, lung transplants, or left-ventricular assist device placement. This concern arose following two adverse outcomes, which were not attributed to the CXR initiative, but which heightened concerns about patient safety. A rise in CXR ordering then occurred, and CVICU providers requested that we perform an analysis of these high-risk groups. While segmented linear regression in this subgroup suggested that average daily CXR ordering rates did decrease among the high-risk group at the start of the intervention period (P = .001), the difference between the rates in the two periods was not significant using the Wilcoxon rank-sum test. Exclusion of these patients from the main analysis did not alter the interpretation of the findings reported above for the CVICU.
DISCUSSION
A deimplementation intervention using provider education, peer champions, and data feedback was associated with fewer CXRs in the CVICU (P < .001) but not in the MICU. The CFIR-guided qualitative analysis was valuable for evaluating our deimplementation strategy and for identifying differences between the two ICUs.
Relatively few studies have demonstrated effective interventions that address CW recommendations.25-28 However, three population-level analyses of insurance claims show mixed results.3,4,29 Experts have thus proposed using implementation science to improve uptake of CW recommendations.2,3,7,8 Our study demonstrates the effectiveness of this approach. As expected, providers largely endorsed an on-demand CXR ordering strategy. Using the CFIR, however, we discovered barriers (eg, concern that data feedback did not reflect variations in patients’ needs). Using methods from implementation science allowed us to diagnose and tailor our approaches.
Our qualitative evaluation suggested that the intervention was ineffective mostly due to CFIR’s “inner setting” constructs, including resident fatigue, competing priorities, and decreased investment in QI projects because of the rotating nature of providers in training. Baseline CXR ordering rates in the MICU were also considerably lower than in the CVICU. We observed that CVICU providers ordered many CXRs following the placement of lines or tubes and that ACNPs in the MICU had received education on appropriate CXR ordering practices and had access to an alternate imaging modality in ultrasound. These factors may partially explain the difference in baseline rates.
As noted in a study of cardiac stress testing guidelines, the existence of high-value care recommendations does not mean overuse.30 Indeed, the lack of significant CXR over-ordering in the MICU highlights the importance of baseline measurement and partnering with information technology departments to create the best possible data feedback systems.30-32 Our experience shows that these systems should provide sufficient pre-implementation data (ideally >1 year), such that teams selecting QI projects can ensure that a project is a good use of institutional resources and change capital.
To inform future work, we informally assessed program costs and savings. We estimate the initiative cost $1,600, including $1,000 for curriculum development and teaching time, $300 for educational materials, and $300 for CXR tracking dashboard development. Hospital charges and reimbursements for CXR vary widely.33 We calculated savings using a range of rates, from a conservative $23 (the Medicare reimbursement rate for single-view CXR, CPT code 71010, global fee) to $50 (an approximate blended reimbursement rate across payers).34,35 In the CVICU, we estimate that 51 CXRs were avoided each month, saving $1,173-$2,550 per month or $9,384-$20,400 over eight months of follow-up. Annualizing these figures, we estimate net savings of $12,476-$29,000 in the first year in a 27-bed ICU. Costs to continue the program include education of new employees, booster training, and dashboard maintenance for an estimated annual cost of $1,000. It is difficult to estimate effectiveness over time, but if we conservatively assume that 30 CXRs were avoided each month, then the projected savings would be $8,280-$18,000 per year or an annual net savings of $7,280-$17,000 in the ICU. Although these amounts are modest, providing trainees with experiential learning opportunities in high-value care is valuable in its own right, meets curricular goals, may result in spill-over effects to other diagnostic and therapeutic decisions, and may influence long-term practice patterns. Institutional decisions to pursue projects such as this should take into account these potential benefits.
This evaluation is not without limitations. First, the study was conducted in a single tertiary-care hospital, potentially limiting its generalizability.36 Second, the study design lacked a concurrent control group, and observed outcomes may have been influenced by broader CXR utilization trends, increased awareness of low-value care generally or from previous CW projects at VUMC, seasonal effects, or the Hawthorne effect. Third, the study outcome was all CXRs ordered, rather than CXRs that were unnecessary or not clinically indicated. We chose all CXRs because it was more pragmatic, did not require clinical case review, and could be incorporated promptly into dashboards, enabling timely performance feedback. Other performance measures have taken a similar tack (eg, tracking all-cause readmissions rather than preventable readmissions). Given this approach, we did not track clinical indications for CXRs (eg, central line placement). Fourth, although we compared resident and APRN orders, we did not collect data on other provider characteristics such as years in/out of training or board certification status. These considerations should be addressed in future research.
Finally, the increase in CVICU CXR ordering at the end of the intervention period, which occurred following two adverse events, raises concerns about sustainability. While unrelated to CXR orders, the events resulted in increased ordering of diagnostic tests and showed the difficulty of deimplementation in ICUs. Indeed, some CVICU providers argued that on-demand CXR ordering represented minimal potential cost savings and had not been studied among heart and lung transplant patients. Subsequently, Tonna et al. have shown that on-demand CXR ordering can be safely implemented among such patients.37 Also similar to our study, Tonna et al. observed an initial decrease in CXR ordering, followed by a gradual increase toward baseline ordering rates. These findings highlight the need for sustained awareness and interventions and for the careful selection of high-value projects.
In conclusion, our study shows that a deimplementation intervention based on CW recommendations can reduce CXR ordering and that ongoing evaluation of contextual factors provides insights for both real-time modifications of current interventions and the design of future interventions. We found that messaging about reducing unnecessary tests works well when discussions are framed at the unit level but may be counterproductive if used to question individual ordering decisions.38 Additional lessons learned include the value of participation on rounds to build trust among stakeholders, the utility of monthly rather than weekly statistics for feedback, stakeholder input and peer champions, and differences in approach with physician and ACNP audiences.
Acknowledgments
The authors thank the VUMC Choosing Wisely committee; Mr. Bill Harrell in Advanced Data Analytics for developing the Tableau platform used in our data feedback strategy; Emily Feld, MD, Jerry Zifodya, MD, and Ryan Kindle, MD for assisting with data feedback to providers in the MICU; Todd Rice, MD, Director of the MICU, for his support of the initiative; and Beth Prusaczyk, PhD, MSW and David Stevenson, PhD for providing feedback on earlier drafts of this manuscript.
Disclosures
Dr. Kripalani reports personal fees from Verustat, personal fees from SAI Interactive, and equity from Bioscape Digital, outside the submitted work. All other authors have nothing to disclose.
Funding
This work was supported by an Innovation Grant from the Alliance for Academic Internal Medicine (AAIM, 2016) and by the Departments of Internal Medicine and Graduate Medical Education at Vanderbilt University Medical Center. The AAIM did not have a role in the study design, data collection, data analysis, data interpretation, or manuscript writing.
1. Cassel CK, Guest JA. Choosing Wisely: Helping physicians and patients make smart decisions about their care. JAMA. 2012;307(17):1801-1802.
2. Gonzales R, Cattamanchi A. Changing clinician behavior when less is more. JAMA Intern Med. 2015;175(12):1921-1922.
3. Rosenberg A, Agiro A, Gottlieb M, et al. Early trends among seven recommendations from the Choosing Wisely campaign. JAMA Intern Med. 2015;175(12):1913-1920.
4. Hong AS, Ross-Degnan D, Zhang F, Wharam JF. Small decline in low-value back imaging associated with the ‘Choosing Wisely’ campaign, 2012-14. Health Aff (Millwood). 2017;36(4):671-679. doi: 10.1377/hlthaff.2016.1263.
5. Parks AL, O’Malley PG. From choosing wisely to practicing value—more to the story. JAMA Intern Med. 2016;176(10):1571-1572.
6. Johnson PT, Pahwa AK, Feldman LS, Ziegelstein RC, Hellmann DB. Advancing high-value health care: a new AJM column dedicated to cost-conscious care quality improvement. Am J Med. 2017;130(6):619-621. doi: 10.1016/j.amjmed.2016.12.018.
7. Eccles MP, Mittman BS. Welcome to implementation science. Implement Sci. 2006;1:1. doi: 10.1186/1748-5908-1-1.
8. Bhatia RS, Levinson W, Shortt S, et al. Measuring the effect of Choosing Wisely: an integrated framework to assess campaign impact on low-value care. BMJ Qual Saf. 2015;24:523-531. doi: 10.1136/bmjqs-2015-004070.
9. Selby K, Barnes GD. Learning to de-adopt ineffective healthcare practices. Am J Med. 2018;131(7):721-722. doi: 10.1016/j.amjmed.2018.03.014.
10. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs. Med Care. 2012;50(3):217-226. doi: 10.1097/MLR.0b013e3182408812.
11. Ganapathy A, Adhikari NK, Spiegelman J, Scales DC. Routine chest x-rays in intensive care units: a systematic review and meta-analysis. Crit Care. 2012;16(2):R68. doi: 10.1186/cc11321.
12. Graat ME, Kröner A, Spronk PE, et al. Elimination of daily routine chest radiographs in a mixed medical-surgical intensive care unit. Intensive Care Med. 2007;33(4):639-644. doi: 10.1007/s00134-007-0542-1.
13. Hendrikse KA, Gratama JWC, Ten Hove W, Rommes JH, Schultz MJ, Spronk PE. Low value of routine chest radiographs in a mixed medical-surgical ICU. Chest. 2007;132(3):823-828. doi: 10.1378/chest.07-1162.
14. Clec’h C, Simon P, Hamdi A, et al. Are daily routine chest radiographs useful in critically ill, mechanically ventilated patients? A randomized study. Intensive Care Med. 2008;34(2):264-270. doi: 10.1007/s00134-007-0919-1.
15. Oba Y, Zaza T. Abandoning daily routine chest radiography in the intensive care unit: meta-analysis. Radiology. 2010;255(2):386-395. doi: 10.1148/radiol.10090946.
16. Mets O, Spronk PE, Binnekade J, Stoker J, de Mol BAJM, Schultz MJ. Elimination of daily routine chest radiographs does not change on-demand radiography practice in post-cardiothoracic surgery patients. J Thorac Cardiovasc Surg. 2007;134(1):139-144. doi: 10.1016/j.jtcvs.2007.02.029.
17. Hejblum G, Chalumeau-Lemoine L, Ioos V, et al. Comparison of routine and on-demand prescription of chest radiographs in mechanically ventilated adults: a multicentre, cluster-randomised, two-period crossover study. Lancet. 2009;374(9702):1687-1693. doi: 10.1016/S0140-6736(09)61459-8.
18. McComb BL, Chung JH, Crabtree TD, et al. ACR appropriateness criteria® routine chest radiography. J Thorac Imaging. 2016;31(2):W13-W15. doi: 10.1097/RTI.0000000000000200.
19. Halpern SD, Becker D, Curtis JR, et al. An official American Thoracic Society/American Association of Critical-Care Nurses/American College of Chest Physicians/Society of Critical Care Medicine policy statement: the Choosing Wisely® top 5 list in critical care medicine. Am J Respir Crit Care Med. 2014;190(7):818-826. doi: 10.1164/rccm.201407-1317ST.
20. Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of implementation strategies: Results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:21. doi: 10.1186/s13012-015-0209-1.
21. R [computer program]. Version 3.4.0. Vienna, Austria: R Foundation for Statistical Computing; 2013.
22. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50. doi: 10.1186/1748-5908-4-50.
23. Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the Consolidated Framework for Implementation Research. Implement Sci. 2016;11:72. doi: 10.1186/s13012-016-0437-z.
24. Speroff T, James BC, Nelson EC, Headrick LA, Brommels M, Reed JE. Guidelines for appraisal and publication of PDSA quality improvement. Qual Manag Health Care. 2014;13(1):33-39. doi: 10.1097/00019514-200401000-00003.
25. Corson AH, Fan VS, White T, et al. A multifaceted hospitalist quality improvement intervention: Decreased frequency of common labs. J Hosp Med. 2015;10(6):390-395. doi: 10.1002/jhm.2354.
26. Ferrari R. Evaluation of the Canadian Rheumatology Association Choosing Wisely recommendation concerning anti-nuclear antibody (ANA) testing. Clin Rheumatol. 2015;34(9):1551-1556. doi: 10.1007/s10067-015-2985-z.
27. Ferrari R, Prosser C. Testing vitamin D levels and choosing wisely. JAMA Intern Med. 2016;176(7):1019-1020.
28. Iams W, Heck J, Kapp M, et al. A multidisciplinary housestaff-led initiative to safely reduce daily laboratory testing. Acad Med. 2016;91(6):813-820. doi: 10.1097/ACM.0000000000001149.
29. Kost A, Genao I, Lee JW, Smith SR. Clinical decisions made in primary care clinics before and after Choosing WiselyTM. J Am Board Fam Med. 2015;28(4):471-474. doi: 10.3122/jabfm.2015.05.140332.
30. Kerr EA, Chen J, Sussman JB, Klamerus ML, Nallamothu BK. Stress testing before low-risk surgery: so many recommendations, so little overuse. JAMA Intern Med. 2015;175(4):645-647. doi: 10.1001/jamainternmed.2014.7877.
31. Colla CH, Morden NE, Sequist TD, Schpero WL, Rosenthal MB. Choosing wisely: prevalence and correlates of low-value health care services in the United States. J Gen Intern Med. 2014;30(2):221-228. doi: 10.1007/s11606-014-3070-z.
32. Shetty KD, Meeker D, Schneider EC, Hussey PS, Damberg CL. Evaluating the feasibility and utility of translating Choosing Wisely recommendations into e-Measures. Healthcare. 2015;3(1):24-37. doi: 10.1016/j.hjdsi.2014.12.002.
33. Woodland DC, Cooper CR, Rashid MF, et al. Routine chest X-ray is unnecessary after ultrasound-guided central venous line placement in the operating room. J Crit Care. 2018;46:13-16. doi: 10.1016/j.jcrc.2018.03.027.
34. Krause TM, Ukhanova M, Revere FL. Private carriers’ physician payment rates compared with Medicare and Medicaid. Tex Med. 2016;112(6):e1.
35. American College of Radiology. Medicare physician fee schedule. Available at https://www.acr.org/Advocacy-and-Economics/Radiology-Economics/Medicare-Medicaid/MPFS. Accessed October 15, 2018.
36. Siegel MD, Rubinowitz AN. Routine daily vs on-demand chest radiographs in intensive care. Lancet. 2009;374(9702):1656-1658. doi: 10.1016/S0140-6736(09)61632-9.
37. Tonna JE, Kawamoto K, Presson AP, et al. Single intervention for a reduction in portable chest radiography (pCXR) in cardiovascular and surgical/trauma ICUs and associated outcomes. J Crit Care. 2018;44:18-23. doi: 10.1016/j.jcrc.2017.10.003.
38. Wolfson D, Santa J, Slass L. Engaging physicians and consumers in conversations about treatment overuse and waste: a short history of the choosing wisely campaign. Acad Med. 2014;89(7):990-995. doi: 10.1097/ACM.0000000000000270.
Despite increased awareness of Choosing Wisely (CW)® recommendations to reduce low-value care,1 there is limited published data about strategies to implement these guidelines or evidence that they have influenced ordering patterns or reduced healthcare spending
In ICUs, CXR ordering strategies may be routine (daily) or on-demand (with clinical indication). The former strategy’s principal advantage is the potential to detect life-threatening situations that may otherwise escape diagnosis.11 Disadvantages include cost, radiation exposure, patient inconvenience, false-positive workups, and low diagnostic and therapeutic value.12,13 On-demand strategies may safely reduce CXR ordering by 32% to 45%.11-17 Based on this evidence, the Critical Care Societies Collaborative and the American College of Radiology have recommended on-demand CXR ordering.18,19 Here, we describe the effectiveness of an intervention to reduce CXR ordering in two ICUs while evaluating the deimplementation strategies using a validated framework.
METHODS
Setting and Design
Vanderbilt University Medical Center (VUMC) is an academic referral center in Nashville, Tennessee. The cardiovascular ICU (CVICU) has 27 beds and the medical ICU (MICU) has 34 beds. Acute care nurse practitioners (ACNPs) and two critical care physicians staff the CVICU; cardiology fellows, anesthesia critical care fellows, and transplant and cardiac surgeons are also active in patient care. The MICU is staffed by two critical care physicians who supervise one team of ACNPs and two teams of medical residents who rotate through the unit every two weeks. Each MICU team is assigned a fellow in pulmonary and critical care.
We conducted a prospective, nonrandomized study in these units from October 2015 to June 2016. The VUMC Institutional Review Board approved the intervention as a quality improvement (QI) activity, waiving the requirement for informed consent.
Intervention
Following the top CW recommendation of the Critical Care Societies Collaborative—“Don’t order diagnostic tests at regular intervals (such as every day), but rather in response to specific clinical questions.”19—the VUMC resident-led CW Steering Committee designed a multifaceted approach to reduce ordering of routine CXRs in ICUs. The intervention included a didactic session on CW and proper CXR ordering practices, peer champions, data audits, and feedback to providers through weekly e-mails (see Supplemental Materials, 1 – Resident Presentation and 2 – CXR Flyer). 20
In September 2015, CVICU and MICU teams received a didactic session highlighting CW, current CXR ordering rates, and the plan for reducing CXR ordering. On October 5, 2015, teams began receiving weekly e-mails with ordering rates defined as CXRs ordered per patient per day and a brief rationale for reducing unnecessary CXRs. To encourage friendly competition, we provided weekly rates to the MICU teams, allowing for transparent benchmarking against one another. A similar competition strategy was not used in the CVICU due to the lack of multiple teams.
In the CVICU, two ACNPs volunteered as peer champions. These champions coordinated data feedback and advocated for the intervention among their colleagues. In the MICU, three internal medicine residents volunteered as peer champions and fulfilled similar roles.
To facilitate deimplementation, we conducted two Plan-Do-Study-Act (PDSA) cycles, the first from November to mid-December 2015 and the second from mid-December 2015 to mid-January 2016. During these cycles, we tailored our deimplementation strategy based on barriers identified by the peer champions and ICU leaders (described in the Qualitative Results section). Peer champions and the CW Steering Committee generated potential solutions by conversing with stakeholders and using the Expert Recommendations for Implementing Change (ERIC).20 Interventions included disseminating promotional flyers, holding meetings with stakeholders, and providing monthly CXR ordering rates. After the PDSA cycles, we continued reexamining the deimplementation efforts by reviewing ordering rates and soliciting feedback from ICU leaders and peer champions. However, no significant changes to the intervention were made during this time.
Quantitative Evaluation
We extracted data from VUMC’s Enterprise Data Warehouse during the intervention period (October 5, 2015 to May 24, 2016) and a historical control period (October 1, 2014 to October 4, 2015). Within each ICU, descriptive statistics were used to compare patient cohorts in the baseline and intervention periods by age, sex, and race.
The primary outcome was CXRs ordered per patient per day by hospital unit (CVICU or MICU). The baseline period included all data between October 1, 2014 and September 15, 2015. To account for priming of providers from didactic education, we allowed a washout period from September 16, 2015 to October 4, 2015. As a preliminary analysis, we compared CXR rates in the baseline and intervention periods using Wilcoxon rank-sum tests. We then conducted interrupted time-series analyses with segmented linear regression to assess differences in linear trends in CXR rates over the two periods. To account for different staffing models in the MICU, we stratified the impact of the intervention by team—medical resident (physician) or ACNP. R version 3.4.0 was used for statistical analysis.21
Qualitative Evaluation
Our qualitative evaluation consisted of embedded observation and semistructured interviews with stakeholders. The qualitative portion was guided by the Consolidated Framework for Implementation Research (CFIR), a widely used framework for design and evaluation of improvement initiatives that helped us to determine major facilitators and barriers to implementation.22,23
Embedded Observation
From November 2015 to January 2016, we observed morning rounds in the CVICU and MICU one to two times weekly to understand factors facilitating and inhibiting uptake of the intervention. Observations were recorded and organized using a CFIR-based template and directed toward understanding interactions among team members (eg, the decision-making process hierarchy), team workflows and decision-making processes, process of ordering CXRs, and providers’ knowledge and perceptions of the CXR intervention (see Supplemental Material, 3 – CFIR Table).22,23 After rounds, ICU team members were invited to share suggestions for improving the intervention. All observations occurred during and shortly following morning rounds when the vast majority of routine CXRs are ordered; we did not evaluate night or evening workflows. In the spirit of continuous improvement, we evaluated data in real-time.
Semistructured Interviews
Based on the direct observations, we developed semistructured interview questions to further evaluate provider perspectives (eg, “Do you believe ICU patients need a daily CXR?”) and constructs aligning with CFIR (eg, “intervention source—internally vs externally developed;” see Supplemental Material, 4 – Interview Questions).
Stakeholders from both ICUs were recruited through e-mail and in-person requests to participate in semistructured interviews. In the CVICU, we interviewed critical care physicians, anesthesia critical care fellows, and ACNPs. In the MICU, we interviewed medical students, interns, residents, critical care fellows, attending intensivist physicians, and ACNPs. We also interviewed X-ray technologists who routinely perform portable films in the units.
RESULTS
Quantitative Results
Cardiovascular Intensive Care Unit
The median baseline CXR ordering rate in the CVICU was 1.16 CXRs per patient per day, with interquartile range (IQR) 1.06-1.28. During the intervention period, the rate dropped to 1.07 (IQR 0.94-1.21; P < .001; Table 2). The time-series analysis suggested an essentially flat trend during the baseline period, followed by a small but significant drop in ordering rates during the intervention period (P < .001; Table 3 and Figure 1). Ordering rates appeared to increase slightly over the course of the intervention period, but this slight upward trend was not significantly different from the flat trend seen during the baseline period.
Medical Intensive Care Unit
For both physician and ACPN teams, the median baseline CXR ordering rates in the MICU were much lower than the baseline rate in the CVICU (Table 2). For the MICU physician care team, the baseline CXR ordering rate was 0.60 CXRs per patient per day (IQR 0.48-0.73). For the ACNP team, the median rate was 0.39 CXRs per patient per day (IQR 0.21-0.57). Both rates stayed approximately the same during the intervention period (Table 2). The time-series analysis suggested a statistically significant but very slight downward trend in CXR ordering rates during the baseline period, in the physician (P = .011) and ACNP (P = .022) teams (Table 3, Figure 2). Under this model, a small increase in CXR ordering initially occurred during the intervention period for both physician and ACNP teams (P = .010 and P = .055, respectively), after which the rates declined slightly. Trends in ordering rates during the intervention period were not significantly different from the slight downward trends seen during the baseline period.
Qualitative Results
We identified 25 of 39 CFIR constructs as relevant to the initiative (see Supplemental Materials, 3 – CFIR Table.) We determined the major facilitators of deimplementation to be peer champion discussions about CXR ordering on rounds and weekly data feedback, particularly if accompanied by in-person follow-up.
We also noted differences in CXR ordering rationales and decisions between the units. Generally, residents in the MICU and ACNPs in the CVICU made decisions to order CXRs. However, decisions were influenced by the expectations of attending physicians. While CVICU providers tended to order CXRs reflexively as part of morning labs, MICU providers—in particular, ACNPs who had been trained on indications for proper CXR ordering—generally ordered CXRs for specific indications (eg, worsening respiratory status). Of note, MICU ACNPs reported the use of bedside ultrasound as an alternate imaging modality and a reason for their higher threshold to order CXRs.
Deimplementation barriers in both units included the need to identify goal CXR ordering rates and the intervention’s limited visibility. To address these barriers, we conducted PDSA cycles and used the CFIR and ERIC to generate potential solutions.24 We established a goal of a 20% absolute reduction in the CVICU, added monthly CXR rates to weekly e-mail reports to better account for variations in patient populations and ordering practices, and circulated materials to promote on-demand CXR ordering. Promotional materials contained guidelines on CXR ordering and five “Frequently Held Misconceptions” about ordering practices with succinct, evidence-based explanations (see Supplemental Material, 2 – CXR Flyer).
Approximately four months after the start of intervention, some CVICU physicians became concerned that on-demand CXR ordering might be inappropriate for high-risk surgical patients, including those who are undergoing or have undergone heart transplants, lung transplants, or left-ventricular assist device placement. This concern arose following two adverse outcomes, which were not attributed to the CXR initiative, but which heightened concerns about patient safety. A rise in CXR ordering then occurred, and CVICU providers requested that we perform an analysis of these high-risk groups. While segmented linear regression in this subgroup suggested that average daily CXR ordering rates did decrease among the high-risk group at the start of the intervention period (P = .001), the difference between the rates in the two periods was not significant using the Wilcoxon rank-sum test. Exclusion of these patients from the main analysis did not alter the interpretation of the findings reported above for the CVICU.
DISCUSSION
A deimplementation intervention using provider education, peer champions, and data feedback was associated with fewer CXRs in the CVICU (P < .001) but not in the MICU. The CFIR-guided qualitative analysis was valuable for evaluating our deimplementation strategy and for identifying differences between the two ICUs.
Relatively few studies have demonstrated effective interventions that address CW recommendations.25-28 However, three population-level analyses of insurance claims show mixed results.3,4,29 Experts have thus proposed using implementation science to improve uptake of CW recommendations.2,3,7,8 Our study demonstrates the effectiveness of this approach. As expected, providers largely endorsed an on-demand CXR ordering strategy. Using the CFIR, however, we discovered barriers (eg, concern that data feedback did not reflect variations in patients’ needs). Using methods from implementation science allowed us to diagnose and tailor our approaches.
Our qualitative evaluation suggested that the intervention was ineffective mostly due to CFIR’s “inner setting” constructs, including resident fatigue, competing priorities, and decreased investment in QI projects because of the rotating nature of providers in training. Baseline CXR ordering rates in the MICU were also considerably lower than in the CVICU. We observed that CVICU providers ordered many CXRs following the placement of lines or tubes and that ACNPs in the MICU had received education on appropriate CXR ordering practices and had access to an alternate imaging modality in ultrasound. These factors may partially explain the difference in baseline rates.
As noted in a study of cardiac stress testing guidelines, the existence of high-value care recommendations does not mean overuse.30 Indeed, the lack of significant CXR over-ordering in the MICU highlights the importance of baseline measurement and partnering with information technology departments to create the best possible data feedback systems.30-32 Our experience shows that these systems should provide sufficient pre-implementation data (ideally >1 year), such that teams selecting QI projects can ensure that a project is a good use of institutional resources and change capital.
To inform future work, we informally assessed program costs and savings. We estimate the initiative cost $1,600, including $1,000 for curriculum development and teaching time, $300 for educational materials, and $300 for CXR tracking dashboard development. Hospital charges and reimbursements for CXR vary widely.33 We calculated savings using a range of rates, from a conservative $23 (the Medicare reimbursement rate for single-view CXR, CPT code 71010, global fee) to $50 (an approximate blended reimbursement rate across payers).34,35 In the CVICU, we estimate that 51 CXRs were avoided each month, saving $1,173-$2,550 per month or $9,384-$20,400 over eight months of follow-up. Annualizing these figures, we estimate net savings of $12,476-$29,000 in the first year in a 27-bed ICU. Costs to continue the program include education of new employees, booster training, and dashboard maintenance for an estimated annual cost of $1,000. It is difficult to estimate effectiveness over time, but if we conservatively assume that 30 CXRs were avoided each month, then the projected savings would be $8,280-$18,000 per year or an annual net savings of $7,280-$17,000 in the ICU. Although these amounts are modest, providing trainees with experiential learning opportunities in high-value care is valuable in its own right, meets curricular goals, may result in spill-over effects to other diagnostic and therapeutic decisions, and may influence long-term practice patterns. Institutional decisions to pursue projects such as this should take into account these potential benefits.
This evaluation is not without limitations. First, the study was conducted in a single tertiary-care hospital, potentially limiting its generalizability.36 Second, the study design lacked a concurrent control group, and observed outcomes may have been influenced by broader CXR utilization trends, increased awareness of low-value care generally or from previous CW projects at VUMC, seasonal effects, or the Hawthorne effect. Third, the study outcome was all CXRs ordered, rather than CXRs that were unnecessary or not clinically indicated. We chose all CXRs because it was more pragmatic, did not require clinical case review, and could be incorporated promptly into dashboards, enabling timely performance feedback. Other performance measures have taken a similar tack (eg, tracking all-cause readmissions rather than preventable readmissions). Given this approach, we did not track clinical indications for CXRs (eg, central line placement). Fourth, although we compared resident and APRN orders, we did not collect data on other provider characteristics such as years in/out of training or board certification status. These considerations should be addressed in future research.
Finally, the increase in CVICU CXR ordering at the end of the intervention period, which occurred following two adverse events, raises concerns about sustainability. While unrelated to CXR orders, the events resulted in increased ordering of diagnostic tests and showed the difficulty of deimplementation in ICUs. Indeed, some CVICU providers argued that on-demand CXR ordering represented minimal potential cost savings and had not been studied among heart and lung transplant patients. Subsequently, Tonna et al. have shown that on-demand CXR ordering can be safely implemented among such patients.37 Also similar to our study, Tonna et al. observed an initial decrease in CXR ordering, followed by a gradual increase toward baseline ordering rates. These findings highlight the need for sustained awareness and interventions and for the careful selection of high-value projects.
In conclusion, our study shows that a deimplementation intervention based on CW recommendations can reduce CXR ordering and that ongoing evaluation of contextual factors provides insights for both real-time modifications of current interventions and the design of future interventions. We found that messaging about reducing unnecessary tests works well when discussions are framed at the unit level but may be counterproductive if used to question individual ordering decisions.38 Additional lessons learned include the value of participation on rounds to build trust among stakeholders, the utility of monthly rather than weekly statistics for feedback, stakeholder input and peer champions, and differences in approach with physician and ACNP audiences.
Acknowledgments
The authors thank the VUMC Choosing Wisely committee; Mr. Bill Harrell in Advanced Data Analytics for developing the Tableau platform used in our data feedback strategy; Emily Feld, MD, Jerry Zifodya, MD, and Ryan Kindle, MD for assisting with data feedback to providers in the MICU; Todd Rice, MD, Director of the MICU, for his support of the initiative; and Beth Prusaczyk, PhD, MSW and David Stevenson, PhD for providing feedback on earlier drafts of this manuscript.
Disclosures
Dr. Kripalani reports personal fees from Verustat, personal fees from SAI Interactive, and equity from Bioscape Digital, outside the submitted work. All other authors have nothing to disclose.
Funding
This work was supported by an Innovation Grant from the Alliance for Academic Internal Medicine (AAIM, 2016) and by the Departments of Internal Medicine and Graduate Medical Education at Vanderbilt University Medical Center. The AAIM did not have a role in the study design, data collection, data analysis, data interpretation, or manuscript writing.
Despite increased awareness of Choosing Wisely (CW)® recommendations to reduce low-value care,1 there is limited published data about strategies to implement these guidelines or evidence that they have influenced ordering patterns or reduced healthcare spending
In ICUs, CXR ordering strategies may be routine (daily) or on-demand (with clinical indication). The former strategy’s principal advantage is the potential to detect life-threatening situations that may otherwise escape diagnosis.11 Disadvantages include cost, radiation exposure, patient inconvenience, false-positive workups, and low diagnostic and therapeutic value.12,13 On-demand strategies may safely reduce CXR ordering by 32% to 45%.11-17 Based on this evidence, the Critical Care Societies Collaborative and the American College of Radiology have recommended on-demand CXR ordering.18,19 Here, we describe the effectiveness of an intervention to reduce CXR ordering in two ICUs while evaluating the deimplementation strategies using a validated framework.
METHODS
Setting and Design
Vanderbilt University Medical Center (VUMC) is an academic referral center in Nashville, Tennessee. The cardiovascular ICU (CVICU) has 27 beds and the medical ICU (MICU) has 34 beds. Acute care nurse practitioners (ACNPs) and two critical care physicians staff the CVICU; cardiology fellows, anesthesia critical care fellows, and transplant and cardiac surgeons are also active in patient care. The MICU is staffed by two critical care physicians who supervise one team of ACNPs and two teams of medical residents who rotate through the unit every two weeks. Each MICU team is assigned a fellow in pulmonary and critical care.
We conducted a prospective, nonrandomized study in these units from October 2015 to June 2016. The VUMC Institutional Review Board approved the intervention as a quality improvement (QI) activity, waiving the requirement for informed consent.
Intervention
Following the top CW recommendation of the Critical Care Societies Collaborative—“Don’t order diagnostic tests at regular intervals (such as every day), but rather in response to specific clinical questions.”19—the VUMC resident-led CW Steering Committee designed a multifaceted approach to reduce ordering of routine CXRs in ICUs. The intervention included a didactic session on CW and proper CXR ordering practices, peer champions, data audits, and feedback to providers through weekly e-mails (see Supplemental Materials, 1 – Resident Presentation and 2 – CXR Flyer). 20
In September 2015, CVICU and MICU teams received a didactic session highlighting CW, current CXR ordering rates, and the plan for reducing CXR ordering. On October 5, 2015, teams began receiving weekly e-mails with ordering rates defined as CXRs ordered per patient per day and a brief rationale for reducing unnecessary CXRs. To encourage friendly competition, we provided weekly rates to the MICU teams, allowing for transparent benchmarking against one another. A similar competition strategy was not used in the CVICU due to the lack of multiple teams.
In the CVICU, two ACNPs volunteered as peer champions. These champions coordinated data feedback and advocated for the intervention among their colleagues. In the MICU, three internal medicine residents volunteered as peer champions and fulfilled similar roles.
To facilitate deimplementation, we conducted two Plan-Do-Study-Act (PDSA) cycles, the first from November to mid-December 2015 and the second from mid-December 2015 to mid-January 2016. During these cycles, we tailored our deimplementation strategy based on barriers identified by the peer champions and ICU leaders (described in the Qualitative Results section). Peer champions and the CW Steering Committee generated potential solutions by conversing with stakeholders and using the Expert Recommendations for Implementing Change (ERIC).20 Interventions included disseminating promotional flyers, holding meetings with stakeholders, and providing monthly CXR ordering rates. After the PDSA cycles, we continued reexamining the deimplementation efforts by reviewing ordering rates and soliciting feedback from ICU leaders and peer champions. However, no significant changes to the intervention were made during this time.
Quantitative Evaluation
We extracted data from VUMC’s Enterprise Data Warehouse during the intervention period (October 5, 2015 to May 24, 2016) and a historical control period (October 1, 2014 to October 4, 2015). Within each ICU, descriptive statistics were used to compare patient cohorts in the baseline and intervention periods by age, sex, and race.
The primary outcome was CXRs ordered per patient per day by hospital unit (CVICU or MICU). The baseline period included all data between October 1, 2014 and September 15, 2015. To account for priming of providers from didactic education, we allowed a washout period from September 16, 2015 to October 4, 2015. As a preliminary analysis, we compared CXR rates in the baseline and intervention periods using Wilcoxon rank-sum tests. We then conducted interrupted time-series analyses with segmented linear regression to assess differences in linear trends in CXR rates over the two periods. To account for different staffing models in the MICU, we stratified the impact of the intervention by team—medical resident (physician) or ACNP. R version 3.4.0 was used for statistical analysis.21
Qualitative Evaluation
Our qualitative evaluation consisted of embedded observation and semistructured interviews with stakeholders. The qualitative portion was guided by the Consolidated Framework for Implementation Research (CFIR), a widely used framework for design and evaluation of improvement initiatives that helped us to determine major facilitators and barriers to implementation.22,23
Embedded Observation
From November 2015 to January 2016, we observed morning rounds in the CVICU and MICU one to two times weekly to understand factors facilitating and inhibiting uptake of the intervention. Observations were recorded and organized using a CFIR-based template and directed toward understanding interactions among team members (eg, the decision-making process hierarchy), team workflows and decision-making processes, process of ordering CXRs, and providers’ knowledge and perceptions of the CXR intervention (see Supplemental Material, 3 – CFIR Table).22,23 After rounds, ICU team members were invited to share suggestions for improving the intervention. All observations occurred during and shortly following morning rounds when the vast majority of routine CXRs are ordered; we did not evaluate night or evening workflows. In the spirit of continuous improvement, we evaluated data in real-time.
Semistructured Interviews
Based on the direct observations, we developed semistructured interview questions to further evaluate provider perspectives (eg, “Do you believe ICU patients need a daily CXR?”) and constructs aligning with CFIR (eg, “intervention source—internally vs externally developed;” see Supplemental Material, 4 – Interview Questions).
Stakeholders from both ICUs were recruited through e-mail and in-person requests to participate in semistructured interviews. In the CVICU, we interviewed critical care physicians, anesthesia critical care fellows, and ACNPs. In the MICU, we interviewed medical students, interns, residents, critical care fellows, attending intensivist physicians, and ACNPs. We also interviewed X-ray technologists who routinely perform portable films in the units.
RESULTS
Quantitative Results
Cardiovascular Intensive Care Unit
The median baseline CXR ordering rate in the CVICU was 1.16 CXRs per patient per day, with interquartile range (IQR) 1.06-1.28. During the intervention period, the rate dropped to 1.07 (IQR 0.94-1.21; P < .001; Table 2). The time-series analysis suggested an essentially flat trend during the baseline period, followed by a small but significant drop in ordering rates during the intervention period (P < .001; Table 3 and Figure 1). Ordering rates appeared to increase slightly over the course of the intervention period, but this slight upward trend was not significantly different from the flat trend seen during the baseline period.
Medical Intensive Care Unit
For both physician and ACPN teams, the median baseline CXR ordering rates in the MICU were much lower than the baseline rate in the CVICU (Table 2). For the MICU physician care team, the baseline CXR ordering rate was 0.60 CXRs per patient per day (IQR 0.48-0.73). For the ACNP team, the median rate was 0.39 CXRs per patient per day (IQR 0.21-0.57). Both rates stayed approximately the same during the intervention period (Table 2). The time-series analysis suggested a statistically significant but very slight downward trend in CXR ordering rates during the baseline period, in the physician (P = .011) and ACNP (P = .022) teams (Table 3, Figure 2). Under this model, a small increase in CXR ordering initially occurred during the intervention period for both physician and ACNP teams (P = .010 and P = .055, respectively), after which the rates declined slightly. Trends in ordering rates during the intervention period were not significantly different from the slight downward trends seen during the baseline period.
Qualitative Results
We identified 25 of 39 CFIR constructs as relevant to the initiative (see Supplemental Materials, 3 – CFIR Table.) We determined the major facilitators of deimplementation to be peer champion discussions about CXR ordering on rounds and weekly data feedback, particularly if accompanied by in-person follow-up.
We also noted differences in CXR ordering rationales and decisions between the units. Generally, residents in the MICU and ACNPs in the CVICU made decisions to order CXRs. However, decisions were influenced by the expectations of attending physicians. While CVICU providers tended to order CXRs reflexively as part of morning labs, MICU providers—in particular, ACNPs who had been trained on indications for proper CXR ordering—generally ordered CXRs for specific indications (eg, worsening respiratory status). Of note, MICU ACNPs reported the use of bedside ultrasound as an alternate imaging modality and a reason for their higher threshold to order CXRs.
Deimplementation barriers in both units included the need to identify goal CXR ordering rates and the intervention’s limited visibility. To address these barriers, we conducted PDSA cycles and used the CFIR and ERIC to generate potential solutions.24 We established a goal of a 20% absolute reduction in the CVICU, added monthly CXR rates to weekly e-mail reports to better account for variations in patient populations and ordering practices, and circulated materials to promote on-demand CXR ordering. Promotional materials contained guidelines on CXR ordering and five “Frequently Held Misconceptions” about ordering practices with succinct, evidence-based explanations (see Supplemental Material, 2 – CXR Flyer).
Approximately four months after the start of intervention, some CVICU physicians became concerned that on-demand CXR ordering might be inappropriate for high-risk surgical patients, including those who are undergoing or have undergone heart transplants, lung transplants, or left-ventricular assist device placement. This concern arose following two adverse outcomes, which were not attributed to the CXR initiative, but which heightened concerns about patient safety. A rise in CXR ordering then occurred, and CVICU providers requested that we perform an analysis of these high-risk groups. While segmented linear regression in this subgroup suggested that average daily CXR ordering rates did decrease among the high-risk group at the start of the intervention period (P = .001), the difference between the rates in the two periods was not significant using the Wilcoxon rank-sum test. Exclusion of these patients from the main analysis did not alter the interpretation of the findings reported above for the CVICU.
DISCUSSION
A deimplementation intervention using provider education, peer champions, and data feedback was associated with fewer CXRs in the CVICU (P < .001) but not in the MICU. The CFIR-guided qualitative analysis was valuable for evaluating our deimplementation strategy and for identifying differences between the two ICUs.
Relatively few studies have demonstrated effective interventions that address CW recommendations.25-28 However, three population-level analyses of insurance claims show mixed results.3,4,29 Experts have thus proposed using implementation science to improve uptake of CW recommendations.2,3,7,8 Our study demonstrates the effectiveness of this approach. As expected, providers largely endorsed an on-demand CXR ordering strategy. Using the CFIR, however, we discovered barriers (eg, concern that data feedback did not reflect variations in patients’ needs). Using methods from implementation science allowed us to diagnose and tailor our approaches.
Our qualitative evaluation suggested that the intervention was ineffective mostly due to CFIR’s “inner setting” constructs, including resident fatigue, competing priorities, and decreased investment in QI projects because of the rotating nature of providers in training. Baseline CXR ordering rates in the MICU were also considerably lower than in the CVICU. We observed that CVICU providers ordered many CXRs following the placement of lines or tubes and that ACNPs in the MICU had received education on appropriate CXR ordering practices and had access to an alternate imaging modality in ultrasound. These factors may partially explain the difference in baseline rates.
As noted in a study of cardiac stress testing guidelines, the existence of high-value care recommendations does not mean overuse.30 Indeed, the lack of significant CXR over-ordering in the MICU highlights the importance of baseline measurement and partnering with information technology departments to create the best possible data feedback systems.30-32 Our experience shows that these systems should provide sufficient pre-implementation data (ideally >1 year), such that teams selecting QI projects can ensure that a project is a good use of institutional resources and change capital.
To inform future work, we informally assessed program costs and savings. We estimate the initiative cost $1,600, including $1,000 for curriculum development and teaching time, $300 for educational materials, and $300 for CXR tracking dashboard development. Hospital charges and reimbursements for CXR vary widely.33 We calculated savings using a range of rates, from a conservative $23 (the Medicare reimbursement rate for single-view CXR, CPT code 71010, global fee) to $50 (an approximate blended reimbursement rate across payers).34,35 In the CVICU, we estimate that 51 CXRs were avoided each month, saving $1,173-$2,550 per month or $9,384-$20,400 over eight months of follow-up. Annualizing these figures, we estimate net savings of $12,476-$29,000 in the first year in a 27-bed ICU. Costs to continue the program include education of new employees, booster training, and dashboard maintenance for an estimated annual cost of $1,000. It is difficult to estimate effectiveness over time, but if we conservatively assume that 30 CXRs were avoided each month, then the projected savings would be $8,280-$18,000 per year or an annual net savings of $7,280-$17,000 in the ICU. Although these amounts are modest, providing trainees with experiential learning opportunities in high-value care is valuable in its own right, meets curricular goals, may result in spill-over effects to other diagnostic and therapeutic decisions, and may influence long-term practice patterns. Institutional decisions to pursue projects such as this should take into account these potential benefits.
This evaluation is not without limitations. First, the study was conducted in a single tertiary-care hospital, potentially limiting its generalizability.36 Second, the study design lacked a concurrent control group, and observed outcomes may have been influenced by broader CXR utilization trends, increased awareness of low-value care generally or from previous CW projects at VUMC, seasonal effects, or the Hawthorne effect. Third, the study outcome was all CXRs ordered, rather than CXRs that were unnecessary or not clinically indicated. We chose all CXRs because it was more pragmatic, did not require clinical case review, and could be incorporated promptly into dashboards, enabling timely performance feedback. Other performance measures have taken a similar tack (eg, tracking all-cause readmissions rather than preventable readmissions). Given this approach, we did not track clinical indications for CXRs (eg, central line placement). Fourth, although we compared resident and APRN orders, we did not collect data on other provider characteristics such as years in/out of training or board certification status. These considerations should be addressed in future research.
Finally, the increase in CVICU CXR ordering at the end of the intervention period, which occurred following two adverse events, raises concerns about sustainability. While unrelated to CXR orders, the events resulted in increased ordering of diagnostic tests and showed the difficulty of deimplementation in ICUs. Indeed, some CVICU providers argued that on-demand CXR ordering represented minimal potential cost savings and had not been studied among heart and lung transplant patients. Subsequently, Tonna et al. have shown that on-demand CXR ordering can be safely implemented among such patients.37 Also similar to our study, Tonna et al. observed an initial decrease in CXR ordering, followed by a gradual increase toward baseline ordering rates. These findings highlight the need for sustained awareness and interventions and for the careful selection of high-value projects.
In conclusion, our study shows that a deimplementation intervention based on CW recommendations can reduce CXR ordering and that ongoing evaluation of contextual factors provides insights for both real-time modifications of current interventions and the design of future interventions. We found that messaging about reducing unnecessary tests works well when discussions are framed at the unit level but may be counterproductive if used to question individual ordering decisions.38 Additional lessons learned include the value of participation on rounds to build trust among stakeholders, the utility of monthly rather than weekly statistics for feedback, stakeholder input and peer champions, and differences in approach with physician and ACNP audiences.
Acknowledgments
The authors thank the VUMC Choosing Wisely committee; Mr. Bill Harrell in Advanced Data Analytics for developing the Tableau platform used in our data feedback strategy; Emily Feld, MD, Jerry Zifodya, MD, and Ryan Kindle, MD for assisting with data feedback to providers in the MICU; Todd Rice, MD, Director of the MICU, for his support of the initiative; and Beth Prusaczyk, PhD, MSW and David Stevenson, PhD for providing feedback on earlier drafts of this manuscript.
Disclosures
Dr. Kripalani reports personal fees from Verustat, personal fees from SAI Interactive, and equity from Bioscape Digital, outside the submitted work. All other authors have nothing to disclose.
Funding
This work was supported by an Innovation Grant from the Alliance for Academic Internal Medicine (AAIM, 2016) and by the Departments of Internal Medicine and Graduate Medical Education at Vanderbilt University Medical Center. The AAIM did not have a role in the study design, data collection, data analysis, data interpretation, or manuscript writing.
1. Cassel CK, Guest JA. Choosing Wisely: Helping physicians and patients make smart decisions about their care. JAMA. 2012;307(17):1801-1802.
2. Gonzales R, Cattamanchi A. Changing clinician behavior when less is more. JAMA Intern Med. 2015;175(12):1921-1922.
3. Rosenberg A, Agiro A, Gottlieb M, et al. Early trends among seven recommendations from the Choosing Wisely campaign. JAMA Intern Med. 2015;175(12):1913-1920.
4. Hong AS, Ross-Degnan D, Zhang F, Wharam JF. Small decline in low-value back imaging associated with the ‘Choosing Wisely’ campaign, 2012-14. Health Aff (Millwood). 2017;36(4):671-679. doi: 10.1377/hlthaff.2016.1263.
5. Parks AL, O’Malley PG. From choosing wisely to practicing value—more to the story. JAMA Intern Med. 2016;176(10):1571-1572.
6. Johnson PT, Pahwa AK, Feldman LS, Ziegelstein RC, Hellmann DB. Advancing high-value health care: a new AJM column dedicated to cost-conscious care quality improvement. Am J Med. 2017;130(6):619-621. doi: 10.1016/j.amjmed.2016.12.018.
7. Eccles MP, Mittman BS. Welcome to implementation science. Implement Sci. 2006;1:1. doi: 10.1186/1748-5908-1-1.
8. Bhatia RS, Levinson W, Shortt S, et al. Measuring the effect of Choosing Wisely: an integrated framework to assess campaign impact on low-value care. BMJ Qual Saf. 2015;24:523-531. doi: 10.1136/bmjqs-2015-004070.
9. Selby K, Barnes GD. Learning to de-adopt ineffective healthcare practices. Am J Med. 2018;131(7):721-722. doi: 10.1016/j.amjmed.2018.03.014.
10. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs. Med Care. 2012;50(3):217-226. doi: 10.1097/MLR.0b013e3182408812.
11. Ganapathy A, Adhikari NK, Spiegelman J, Scales DC. Routine chest x-rays in intensive care units: a systematic review and meta-analysis. Crit Care. 2012;16(2):R68. doi: 10.1186/cc11321.
12. Graat ME, Kröner A, Spronk PE, et al. Elimination of daily routine chest radiographs in a mixed medical-surgical intensive care unit. Intensive Care Med. 2007;33(4):639-644. doi: 10.1007/s00134-007-0542-1.
13. Hendrikse KA, Gratama JWC, Ten Hove W, Rommes JH, Schultz MJ, Spronk PE. Low value of routine chest radiographs in a mixed medical-surgical ICU. Chest. 2007;132(3):823-828. doi: 10.1378/chest.07-1162.
14. Clec’h C, Simon P, Hamdi A, et al. Are daily routine chest radiographs useful in critically ill, mechanically ventilated patients? A randomized study. Intensive Care Med. 2008;34(2):264-270. doi: 10.1007/s00134-007-0919-1.
15. Oba Y, Zaza T. Abandoning daily routine chest radiography in the intensive care unit: meta-analysis. Radiology. 2010;255(2):386-395. doi: 10.1148/radiol.10090946.
16. Mets O, Spronk PE, Binnekade J, Stoker J, de Mol BAJM, Schultz MJ. Elimination of daily routine chest radiographs does not change on-demand radiography practice in post-cardiothoracic surgery patients. J Thorac Cardiovasc Surg. 2007;134(1):139-144. doi: 10.1016/j.jtcvs.2007.02.029.
17. Hejblum G, Chalumeau-Lemoine L, Ioos V, et al. Comparison of routine and on-demand prescription of chest radiographs in mechanically ventilated adults: a multicentre, cluster-randomised, two-period crossover study. Lancet. 2009;374(9702):1687-1693. doi: 10.1016/S0140-6736(09)61459-8.
18. McComb BL, Chung JH, Crabtree TD, et al. ACR appropriateness criteria® routine chest radiography. J Thorac Imaging. 2016;31(2):W13-W15. doi: 10.1097/RTI.0000000000000200.
19. Halpern SD, Becker D, Curtis JR, et al. An official American Thoracic Society/American Association of Critical-Care Nurses/American College of Chest Physicians/Society of Critical Care Medicine policy statement: the Choosing Wisely® top 5 list in critical care medicine. Am J Respir Crit Care Med. 2014;190(7):818-826. doi: 10.1164/rccm.201407-1317ST.
20. Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of implementation strategies: Results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:21. doi: 10.1186/s13012-015-0209-1.
21. R [computer program]. Version 3.4.0. Vienna, Austria: R Foundation for Statistical Computing; 2013.
22. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50. doi: 10.1186/1748-5908-4-50.
23. Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the Consolidated Framework for Implementation Research. Implement Sci. 2016;11:72. doi: 10.1186/s13012-016-0437-z.
24. Speroff T, James BC, Nelson EC, Headrick LA, Brommels M, Reed JE. Guidelines for appraisal and publication of PDSA quality improvement. Qual Manag Health Care. 2014;13(1):33-39. doi: 10.1097/00019514-200401000-00003.
25. Corson AH, Fan VS, White T, et al. A multifaceted hospitalist quality improvement intervention: Decreased frequency of common labs. J Hosp Med. 2015;10(6):390-395. doi: 10.1002/jhm.2354.
26. Ferrari R. Evaluation of the Canadian Rheumatology Association Choosing Wisely recommendation concerning anti-nuclear antibody (ANA) testing. Clin Rheumatol. 2015;34(9):1551-1556. doi: 10.1007/s10067-015-2985-z.
27. Ferrari R, Prosser C. Testing vitamin D levels and choosing wisely. JAMA Intern Med. 2016;176(7):1019-1020.
28. Iams W, Heck J, Kapp M, et al. A multidisciplinary housestaff-led initiative to safely reduce daily laboratory testing. Acad Med. 2016;91(6):813-820. doi: 10.1097/ACM.0000000000001149.
29. Kost A, Genao I, Lee JW, Smith SR. Clinical decisions made in primary care clinics before and after Choosing WiselyTM. J Am Board Fam Med. 2015;28(4):471-474. doi: 10.3122/jabfm.2015.05.140332.
30. Kerr EA, Chen J, Sussman JB, Klamerus ML, Nallamothu BK. Stress testing before low-risk surgery: so many recommendations, so little overuse. JAMA Intern Med. 2015;175(4):645-647. doi: 10.1001/jamainternmed.2014.7877.
31. Colla CH, Morden NE, Sequist TD, Schpero WL, Rosenthal MB. Choosing wisely: prevalence and correlates of low-value health care services in the United States. J Gen Intern Med. 2014;30(2):221-228. doi: 10.1007/s11606-014-3070-z.
32. Shetty KD, Meeker D, Schneider EC, Hussey PS, Damberg CL. Evaluating the feasibility and utility of translating Choosing Wisely recommendations into e-Measures. Healthcare. 2015;3(1):24-37. doi: 10.1016/j.hjdsi.2014.12.002.
33. Woodland DC, Cooper CR, Rashid MF, et al. Routine chest X-ray is unnecessary after ultrasound-guided central venous line placement in the operating room. J Crit Care. 2018;46:13-16. doi: 10.1016/j.jcrc.2018.03.027.
34. Krause TM, Ukhanova M, Revere FL. Private carriers’ physician payment rates compared with Medicare and Medicaid. Tex Med. 2016;112(6):e1.
35. American College of Radiology. Medicare physician fee schedule. Available at https://www.acr.org/Advocacy-and-Economics/Radiology-Economics/Medicare-Medicaid/MPFS. Accessed October 15, 2018.
36. Siegel MD, Rubinowitz AN. Routine daily vs on-demand chest radiographs in intensive care. Lancet. 2009;374(9702):1656-1658. doi: 10.1016/S0140-6736(09)61632-9.
37. Tonna JE, Kawamoto K, Presson AP, et al. Single intervention for a reduction in portable chest radiography (pCXR) in cardiovascular and surgical/trauma ICUs and associated outcomes. J Crit Care. 2018;44:18-23. doi: 10.1016/j.jcrc.2017.10.003.
38. Wolfson D, Santa J, Slass L. Engaging physicians and consumers in conversations about treatment overuse and waste: a short history of the choosing wisely campaign. Acad Med. 2014;89(7):990-995. doi: 10.1097/ACM.0000000000000270.
1. Cassel CK, Guest JA. Choosing Wisely: Helping physicians and patients make smart decisions about their care. JAMA. 2012;307(17):1801-1802.
2. Gonzales R, Cattamanchi A. Changing clinician behavior when less is more. JAMA Intern Med. 2015;175(12):1921-1922.
3. Rosenberg A, Agiro A, Gottlieb M, et al. Early trends among seven recommendations from the Choosing Wisely campaign. JAMA Intern Med. 2015;175(12):1913-1920.
4. Hong AS, Ross-Degnan D, Zhang F, Wharam JF. Small decline in low-value back imaging associated with the ‘Choosing Wisely’ campaign, 2012-14. Health Aff (Millwood). 2017;36(4):671-679. doi: 10.1377/hlthaff.2016.1263.
5. Parks AL, O’Malley PG. From choosing wisely to practicing value—more to the story. JAMA Intern Med. 2016;176(10):1571-1572.
6. Johnson PT, Pahwa AK, Feldman LS, Ziegelstein RC, Hellmann DB. Advancing high-value health care: a new AJM column dedicated to cost-conscious care quality improvement. Am J Med. 2017;130(6):619-621. doi: 10.1016/j.amjmed.2016.12.018.
7. Eccles MP, Mittman BS. Welcome to implementation science. Implement Sci. 2006;1:1. doi: 10.1186/1748-5908-1-1.
8. Bhatia RS, Levinson W, Shortt S, et al. Measuring the effect of Choosing Wisely: an integrated framework to assess campaign impact on low-value care. BMJ Qual Saf. 2015;24:523-531. doi: 10.1136/bmjqs-2015-004070.
9. Selby K, Barnes GD. Learning to de-adopt ineffective healthcare practices. Am J Med. 2018;131(7):721-722. doi: 10.1016/j.amjmed.2018.03.014.
10. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs. Med Care. 2012;50(3):217-226. doi: 10.1097/MLR.0b013e3182408812.
11. Ganapathy A, Adhikari NK, Spiegelman J, Scales DC. Routine chest x-rays in intensive care units: a systematic review and meta-analysis. Crit Care. 2012;16(2):R68. doi: 10.1186/cc11321.
12. Graat ME, Kröner A, Spronk PE, et al. Elimination of daily routine chest radiographs in a mixed medical-surgical intensive care unit. Intensive Care Med. 2007;33(4):639-644. doi: 10.1007/s00134-007-0542-1.
13. Hendrikse KA, Gratama JWC, Ten Hove W, Rommes JH, Schultz MJ, Spronk PE. Low value of routine chest radiographs in a mixed medical-surgical ICU. Chest. 2007;132(3):823-828. doi: 10.1378/chest.07-1162.
14. Clec’h C, Simon P, Hamdi A, et al. Are daily routine chest radiographs useful in critically ill, mechanically ventilated patients? A randomized study. Intensive Care Med. 2008;34(2):264-270. doi: 10.1007/s00134-007-0919-1.
15. Oba Y, Zaza T. Abandoning daily routine chest radiography in the intensive care unit: meta-analysis. Radiology. 2010;255(2):386-395. doi: 10.1148/radiol.10090946.
16. Mets O, Spronk PE, Binnekade J, Stoker J, de Mol BAJM, Schultz MJ. Elimination of daily routine chest radiographs does not change on-demand radiography practice in post-cardiothoracic surgery patients. J Thorac Cardiovasc Surg. 2007;134(1):139-144. doi: 10.1016/j.jtcvs.2007.02.029.
17. Hejblum G, Chalumeau-Lemoine L, Ioos V, et al. Comparison of routine and on-demand prescription of chest radiographs in mechanically ventilated adults: a multicentre, cluster-randomised, two-period crossover study. Lancet. 2009;374(9702):1687-1693. doi: 10.1016/S0140-6736(09)61459-8.
18. McComb BL, Chung JH, Crabtree TD, et al. ACR appropriateness criteria® routine chest radiography. J Thorac Imaging. 2016;31(2):W13-W15. doi: 10.1097/RTI.0000000000000200.
19. Halpern SD, Becker D, Curtis JR, et al. An official American Thoracic Society/American Association of Critical-Care Nurses/American College of Chest Physicians/Society of Critical Care Medicine policy statement: the Choosing Wisely® top 5 list in critical care medicine. Am J Respir Crit Care Med. 2014;190(7):818-826. doi: 10.1164/rccm.201407-1317ST.
20. Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of implementation strategies: Results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:21. doi: 10.1186/s13012-015-0209-1.
21. R [computer program]. Version 3.4.0. Vienna, Austria: R Foundation for Statistical Computing; 2013.
22. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50. doi: 10.1186/1748-5908-4-50.
23. Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the Consolidated Framework for Implementation Research. Implement Sci. 2016;11:72. doi: 10.1186/s13012-016-0437-z.
24. Speroff T, James BC, Nelson EC, Headrick LA, Brommels M, Reed JE. Guidelines for appraisal and publication of PDSA quality improvement. Qual Manag Health Care. 2014;13(1):33-39. doi: 10.1097/00019514-200401000-00003.
25. Corson AH, Fan VS, White T, et al. A multifaceted hospitalist quality improvement intervention: Decreased frequency of common labs. J Hosp Med. 2015;10(6):390-395. doi: 10.1002/jhm.2354.
26. Ferrari R. Evaluation of the Canadian Rheumatology Association Choosing Wisely recommendation concerning anti-nuclear antibody (ANA) testing. Clin Rheumatol. 2015;34(9):1551-1556. doi: 10.1007/s10067-015-2985-z.
27. Ferrari R, Prosser C. Testing vitamin D levels and choosing wisely. JAMA Intern Med. 2016;176(7):1019-1020.
28. Iams W, Heck J, Kapp M, et al. A multidisciplinary housestaff-led initiative to safely reduce daily laboratory testing. Acad Med. 2016;91(6):813-820. doi: 10.1097/ACM.0000000000001149.
29. Kost A, Genao I, Lee JW, Smith SR. Clinical decisions made in primary care clinics before and after Choosing WiselyTM. J Am Board Fam Med. 2015;28(4):471-474. doi: 10.3122/jabfm.2015.05.140332.
30. Kerr EA, Chen J, Sussman JB, Klamerus ML, Nallamothu BK. Stress testing before low-risk surgery: so many recommendations, so little overuse. JAMA Intern Med. 2015;175(4):645-647. doi: 10.1001/jamainternmed.2014.7877.
31. Colla CH, Morden NE, Sequist TD, Schpero WL, Rosenthal MB. Choosing wisely: prevalence and correlates of low-value health care services in the United States. J Gen Intern Med. 2014;30(2):221-228. doi: 10.1007/s11606-014-3070-z.
32. Shetty KD, Meeker D, Schneider EC, Hussey PS, Damberg CL. Evaluating the feasibility and utility of translating Choosing Wisely recommendations into e-Measures. Healthcare. 2015;3(1):24-37. doi: 10.1016/j.hjdsi.2014.12.002.
33. Woodland DC, Cooper CR, Rashid MF, et al. Routine chest X-ray is unnecessary after ultrasound-guided central venous line placement in the operating room. J Crit Care. 2018;46:13-16. doi: 10.1016/j.jcrc.2018.03.027.
34. Krause TM, Ukhanova M, Revere FL. Private carriers’ physician payment rates compared with Medicare and Medicaid. Tex Med. 2016;112(6):e1.
35. American College of Radiology. Medicare physician fee schedule. Available at https://www.acr.org/Advocacy-and-Economics/Radiology-Economics/Medicare-Medicaid/MPFS. Accessed October 15, 2018.
36. Siegel MD, Rubinowitz AN. Routine daily vs on-demand chest radiographs in intensive care. Lancet. 2009;374(9702):1656-1658. doi: 10.1016/S0140-6736(09)61632-9.
37. Tonna JE, Kawamoto K, Presson AP, et al. Single intervention for a reduction in portable chest radiography (pCXR) in cardiovascular and surgical/trauma ICUs and associated outcomes. J Crit Care. 2018;44:18-23. doi: 10.1016/j.jcrc.2017.10.003.
38. Wolfson D, Santa J, Slass L. Engaging physicians and consumers in conversations about treatment overuse and waste: a short history of the choosing wisely campaign. Acad Med. 2014;89(7):990-995. doi: 10.1097/ACM.0000000000000270.
© 2019 Society of Hospital Medicine