Eric G. Poon, MD, MPH

The adoption of electronic health records (EHRs) in US hospitals continues to rise steeply, with nearly 60% of all hospitals having at least a basic EHR as of 2014.[1] EHRs bring with them the ability to inform and guide clinicians as they make decisions. In theory, this form of clinical decision support (CDS) ensures quality of care, reduces adverse events, and improves efficiency; in practice, experience in the field paints a mixed picture.[2, 3] This issue of the Journal of Hospital Medicine presents 3 examples of CDS that illustrate the distance between what we see as CDS' full potential and current limitations.

In the study by Herzig et al.[4] investigators took on the challenge of implementing stress ulcer prophylaxis guidelines developed by the Society of Hospital Medicine. The investigators first demonstrated that targeted electronic prompts captured patients' indications for acid suppressive therapy, and could be used to prohibit prescribers from ordering acid suppressive therapy among patients outside the intensive care unit (ICU) setting. Through an elegant interrupted time series study design deployed across 2 hospital campuses, the investigators were able to demonstrate immediate and clinically significant reduction in acid suppressive therapy outside the ICU. They further found that the impact of this reduction was augmented over time, suggesting that the electronic prompts had a sustained impact on provider ordering behavior. However, below the headlineand relevant to the limitations of CDSthe investigators noted that much of the reduction in the use of acid suppressive therapy for stress ulcer prophylaxis could be accounted for by providers' choice of another acceptable indication (eg, continuing preadmission medication). The authors speculated that the CDS intervention prompted providers to more accurately record the indication for acid suppressive therapy. It is also possible that providers simply chose an alternate indication to circumvent the decision‐support step. Perhaps as a result of these 2 offsetting factors, the actual use of acid suppressive therapy, regardless of indication, only decreased in a modest and statistically nonsignificant way, casting the true effectiveness of this CDS intervention into question.

Two other studies in this issue of the Journal of Hospital Medicine[5, 6] provide valuable insights into interactions between social and technical factors[7, 8, 9, 10] that determine the success or failure in the use of technology such as CDS to drive organizational performance. At the technical end of this sociotechnical spectrum, the study by Knight et al.[5] illustrated that a minimally configured and visually unintuitive medication decision‐support system resulted in a high number of alerts (approximately 17% of studied orders), leading to the well‐reported phenomena of alert fatigue and substantially lower response rate compared to those reported in the literature.[11, 12, 13] Moreover, the analysis suggested that response to these alerts were particularly muted among situations that were particularly high risk, including the patient being older, patient having a greater length of stay, care being delivered in the internal medicine service, resident physician being the prescriber, and the medication being on the Institute for Safe Medication Practices list of high‐alert medications. The investigators concluded that a redesign of the medication decision‐support system was needed.

The study by Chen et al.[6] illuminated how social factors pose challenges in implementing CDS. Investigators in this study were previously successful in using a combination of an education campaign and interruptive decision‐support prompts to reduce the inappropriate ordering of blood transfusions. However, even with a successful intervention, up to 30% of transfusions occurred outside of recommended guidelines. This finding prompted the investigators to analyze the free‐text reasons offered by providers for overriding the recommended guidelines. Two key patterns emerged from their structured analysis. First, many of the apparently inappropriate transfusions occurred under officially sanctioned protocols (such as stem cell transplant) that the computer system was not able to take into account in generating alerts. Second, many orders that reflected questionable practices were being entered by resident physicians, physician assistants, nurse practitioners, and nurses who were least empowered to challenge requests from senior staff.

Several practical and actionable lessons can be drawn from the 3 sets of investigators featured in this issue of the Journal of Hospital Medicine. First, all investigators defined metrics that should be tracked over time to demonstrate progress and to make iterative improvements; this discipline is needed in both academic and community settings to prioritize limited CDS resources in an objective and data‐driven way. Second, as the Herzig et al.[4] article illustrated, when it comes to evaluating the impact of CDS, we cannot be satisfied merely with process measures (eg, change in clinical documentation) at the expense of outcome measures (eg, decrease in inappropriate use of therapies). Third, as Chen et al.[6] recognized, CDS is but a component of an educational program to guide and alter clinical behavior, and must be deployed in conjunction with other educational tools such as newsletters, traditional lectures, or academic detailing. Fourth, clinicians with a stake in improving quality and safety should be on guard against the well‐documented phenomena of alert fatigue by ensuring their organization selects an appropriate framework for deciding which CDS alerts are activated andwhere possibledisplay the highest‐priority alerts in the most prominent and interruptive manner. Fifth, CDS must be maintained over time as clinical guidelines and clinicians' receptivity to each CDS evolve. Alerts that are not changing clinical behavior should either be modified or simply turned off. Sixth, free text entered as part of structured data entry (eg, while placing orders) or as reasons for overriding CDS (as in Chen et al.[6]) offer significant insights on how to optimize CDS, and should be monitored systematically on an ongoing basis to ensure the EMR addresses users' changing needs and mental models.

So what is the clinician with an interest in improving healthcare outcomes and organizational efficiency to do given CDS' limitations? One option is to wait for the science of CDS to further mature and have those advances embedded in the EMR at your organization. Another option might be to rely on the information technology and clinical informatics professionals at your organization to decide how CDS should be used locally. In 2014, these may be untenable choices for the following reasons. First, given the universal pressures to improve healthcare outcomes and contain costs,[14] healthcare organizations must use all available tools to achieve challenging performance goals. Second, as EMRs with CDS become commonplace, and as the 3 articles in this issue of the Journal of Hospital Medicine and others have illustrated, there are many opportunities to misuse or poorly implement CDS, with potentially dire consequences.[15] Third, design and deployment of effective CDS require information technology and informatics professionals to collaborate with clinicians to gauge the quality of EMR data used to drive CDS and clinicians' receptivity to CDS, illuminate the sociotechnical context in which to deploy the CDS, and champion the CDS intervention among their colleagues. Clinicians' input is therefore an essential ingredient to success. Fourth, organizational trust, a key aspect of a healthy safety culture, is hard to build and easy to erode.[9, 16] If clinicians at an organization lose trust in CDS because of poor design and deployment strategies, they are likely to ignore CDS in the future.[17]

Like tools introduced into medicine such as magnetic resonance imaging and highly active antiretroviral therapy, CDS will need to evolve as the clinical community grapples with its potential and limitations. As EMRs move toward ubiquity in the hospital setting, CDS will become part of the fabric of hospital‐based practice, and the Journal of Hospital Medicine readership would do well to learn about this new tool of the trade.

The adoption of electronic health records (EHRs) in US hospitals continues to rise steeply, with nearly 60% of all hospitals having at least a basic EHR as of 2014.[1] EHRs bring with them the ability to inform and guide clinicians as they make decisions. In theory, this form of clinical decision support (CDS) ensures quality of care, reduces adverse events, and improves efficiency; in practice, experience in the field paints a mixed picture.[2, 3] This issue of the Journal of Hospital Medicine presents 3 examples of CDS that illustrate the distance between what we see as CDS' full potential and current limitations.

In the study by Herzig et al.[4] investigators took on the challenge of implementing stress ulcer prophylaxis guidelines developed by the Society of Hospital Medicine. The investigators first demonstrated that targeted electronic prompts captured patients' indications for acid suppressive therapy, and could be used to prohibit prescribers from ordering acid suppressive therapy among patients outside the intensive care unit (ICU) setting. Through an elegant interrupted time series study design deployed across 2 hospital campuses, the investigators were able to demonstrate immediate and clinically significant reduction in acid suppressive therapy outside the ICU. They further found that the impact of this reduction was augmented over time, suggesting that the electronic prompts had a sustained impact on provider ordering behavior. However, below the headlineand relevant to the limitations of CDSthe investigators noted that much of the reduction in the use of acid suppressive therapy for stress ulcer prophylaxis could be accounted for by providers' choice of another acceptable indication (eg, continuing preadmission medication). The authors speculated that the CDS intervention prompted providers to more accurately record the indication for acid suppressive therapy. It is also possible that providers simply chose an alternate indication to circumvent the decision‐support step. Perhaps as a result of these 2 offsetting factors, the actual use of acid suppressive therapy, regardless of indication, only decreased in a modest and statistically nonsignificant way, casting the true effectiveness of this CDS intervention into question.

Two other studies in this issue of the Journal of Hospital Medicine[5, 6] provide valuable insights into interactions between social and technical factors[7, 8, 9, 10] that determine the success or failure in the use of technology such as CDS to drive organizational performance. At the technical end of this sociotechnical spectrum, the study by Knight et al.[5] illustrated that a minimally configured and visually unintuitive medication decision‐support system resulted in a high number of alerts (approximately 17% of studied orders), leading to the well‐reported phenomena of alert fatigue and substantially lower response rate compared to those reported in the literature.[11, 12, 13] Moreover, the analysis suggested that response to these alerts were particularly muted among situations that were particularly high risk, including the patient being older, patient having a greater length of stay, care being delivered in the internal medicine service, resident physician being the prescriber, and the medication being on the Institute for Safe Medication Practices list of high‐alert medications. The investigators concluded that a redesign of the medication decision‐support system was needed.

The study by Chen et al.[6] illuminated how social factors pose challenges in implementing CDS. Investigators in this study were previously successful in using a combination of an education campaign and interruptive decision‐support prompts to reduce the inappropriate ordering of blood transfusions. However, even with a successful intervention, up to 30% of transfusions occurred outside of recommended guidelines. This finding prompted the investigators to analyze the free‐text reasons offered by providers for overriding the recommended guidelines. Two key patterns emerged from their structured analysis. First, many of the apparently inappropriate transfusions occurred under officially sanctioned protocols (such as stem cell transplant) that the computer system was not able to take into account in generating alerts. Second, many orders that reflected questionable practices were being entered by resident physicians, physician assistants, nurse practitioners, and nurses who were least empowered to challenge requests from senior staff.

Several practical and actionable lessons can be drawn from the 3 sets of investigators featured in this issue of the Journal of Hospital Medicine. First, all investigators defined metrics that should be tracked over time to demonstrate progress and to make iterative improvements; this discipline is needed in both academic and community settings to prioritize limited CDS resources in an objective and data‐driven way. Second, as the Herzig et al.[4] article illustrated, when it comes to evaluating the impact of CDS, we cannot be satisfied merely with process measures (eg, change in clinical documentation) at the expense of outcome measures (eg, decrease in inappropriate use of therapies). Third, as Chen et al.[6] recognized, CDS is but a component of an educational program to guide and alter clinical behavior, and must be deployed in conjunction with other educational tools such as newsletters, traditional lectures, or academic detailing. Fourth, clinicians with a stake in improving quality and safety should be on guard against the well‐documented phenomena of alert fatigue by ensuring their organization selects an appropriate framework for deciding which CDS alerts are activated andwhere possibledisplay the highest‐priority alerts in the most prominent and interruptive manner. Fifth, CDS must be maintained over time as clinical guidelines and clinicians' receptivity to each CDS evolve. Alerts that are not changing clinical behavior should either be modified or simply turned off. Sixth, free text entered as part of structured data entry (eg, while placing orders) or as reasons for overriding CDS (as in Chen et al.[6]) offer significant insights on how to optimize CDS, and should be monitored systematically on an ongoing basis to ensure the EMR addresses users' changing needs and mental models.

So what is the clinician with an interest in improving healthcare outcomes and organizational efficiency to do given CDS' limitations? One option is to wait for the science of CDS to further mature and have those advances embedded in the EMR at your organization. Another option might be to rely on the information technology and clinical informatics professionals at your organization to decide how CDS should be used locally. In 2014, these may be untenable choices for the following reasons. First, given the universal pressures to improve healthcare outcomes and contain costs,[14] healthcare organizations must use all available tools to achieve challenging performance goals. Second, as EMRs with CDS become commonplace, and as the 3 articles in this issue of the Journal of Hospital Medicine and others have illustrated, there are many opportunities to misuse or poorly implement CDS, with potentially dire consequences.[15] Third, design and deployment of effective CDS require information technology and informatics professionals to collaborate with clinicians to gauge the quality of EMR data used to drive CDS and clinicians' receptivity to CDS, illuminate the sociotechnical context in which to deploy the CDS, and champion the CDS intervention among their colleagues. Clinicians' input is therefore an essential ingredient to success. Fourth, organizational trust, a key aspect of a healthy safety culture, is hard to build and easy to erode.[9, 16] If clinicians at an organization lose trust in CDS because of poor design and deployment strategies, they are likely to ignore CDS in the future.[17]

Like tools introduced into medicine such as magnetic resonance imaging and highly active antiretroviral therapy, CDS will need to evolve as the clinical community grapples with its potential and limitations. As EMRs move toward ubiquity in the hospital setting, CDS will become part of the fabric of hospital‐based practice, and the Journal of Hospital Medicine readership would do well to learn about this new tool of the trade.

The adoption of electronic health records (EHRs) in US hospitals continues to rise steeply, with nearly 60% of all hospitals having at least a basic EHR as of 2014.[1] EHRs bring with them the ability to inform and guide clinicians as they make decisions. In theory, this form of clinical decision support (CDS) ensures quality of care, reduces adverse events, and improves efficiency; in practice, experience in the field paints a mixed picture.[2, 3] This issue of the Journal of Hospital Medicine presents 3 examples of CDS that illustrate the distance between what we see as CDS' full potential and current limitations.

In the study by Herzig et al.[4] investigators took on the challenge of implementing stress ulcer prophylaxis guidelines developed by the Society of Hospital Medicine. The investigators first demonstrated that targeted electronic prompts captured patients' indications for acid suppressive therapy, and could be used to prohibit prescribers from ordering acid suppressive therapy among patients outside the intensive care unit (ICU) setting. Through an elegant interrupted time series study design deployed across 2 hospital campuses, the investigators were able to demonstrate immediate and clinically significant reduction in acid suppressive therapy outside the ICU. They further found that the impact of this reduction was augmented over time, suggesting that the electronic prompts had a sustained impact on provider ordering behavior. However, below the headlineand relevant to the limitations of CDSthe investigators noted that much of the reduction in the use of acid suppressive therapy for stress ulcer prophylaxis could be accounted for by providers' choice of another acceptable indication (eg, continuing preadmission medication). The authors speculated that the CDS intervention prompted providers to more accurately record the indication for acid suppressive therapy. It is also possible that providers simply chose an alternate indication to circumvent the decision‐support step. Perhaps as a result of these 2 offsetting factors, the actual use of acid suppressive therapy, regardless of indication, only decreased in a modest and statistically nonsignificant way, casting the true effectiveness of this CDS intervention into question.

Two other studies in this issue of the Journal of Hospital Medicine[5, 6] provide valuable insights into interactions between social and technical factors[7, 8, 9, 10] that determine the success or failure in the use of technology such as CDS to drive organizational performance. At the technical end of this sociotechnical spectrum, the study by Knight et al.[5] illustrated that a minimally configured and visually unintuitive medication decision‐support system resulted in a high number of alerts (approximately 17% of studied orders), leading to the well‐reported phenomena of alert fatigue and substantially lower response rate compared to those reported in the literature.[11, 12, 13] Moreover, the analysis suggested that response to these alerts were particularly muted among situations that were particularly high risk, including the patient being older, patient having a greater length of stay, care being delivered in the internal medicine service, resident physician being the prescriber, and the medication being on the Institute for Safe Medication Practices list of high‐alert medications. The investigators concluded that a redesign of the medication decision‐support system was needed.

The study by Chen et al.[6] illuminated how social factors pose challenges in implementing CDS. Investigators in this study were previously successful in using a combination of an education campaign and interruptive decision‐support prompts to reduce the inappropriate ordering of blood transfusions. However, even with a successful intervention, up to 30% of transfusions occurred outside of recommended guidelines. This finding prompted the investigators to analyze the free‐text reasons offered by providers for overriding the recommended guidelines. Two key patterns emerged from their structured analysis. First, many of the apparently inappropriate transfusions occurred under officially sanctioned protocols (such as stem cell transplant) that the computer system was not able to take into account in generating alerts. Second, many orders that reflected questionable practices were being entered by resident physicians, physician assistants, nurse practitioners, and nurses who were least empowered to challenge requests from senior staff.

Several practical and actionable lessons can be drawn from the 3 sets of investigators featured in this issue of the Journal of Hospital Medicine. First, all investigators defined metrics that should be tracked over time to demonstrate progress and to make iterative improvements; this discipline is needed in both academic and community settings to prioritize limited CDS resources in an objective and data‐driven way. Second, as the Herzig et al.[4] article illustrated, when it comes to evaluating the impact of CDS, we cannot be satisfied merely with process measures (eg, change in clinical documentation) at the expense of outcome measures (eg, decrease in inappropriate use of therapies). Third, as Chen et al.[6] recognized, CDS is but a component of an educational program to guide and alter clinical behavior, and must be deployed in conjunction with other educational tools such as newsletters, traditional lectures, or academic detailing. Fourth, clinicians with a stake in improving quality and safety should be on guard against the well‐documented phenomena of alert fatigue by ensuring their organization selects an appropriate framework for deciding which CDS alerts are activated andwhere possibledisplay the highest‐priority alerts in the most prominent and interruptive manner. Fifth, CDS must be maintained over time as clinical guidelines and clinicians' receptivity to each CDS evolve. Alerts that are not changing clinical behavior should either be modified or simply turned off. Sixth, free text entered as part of structured data entry (eg, while placing orders) or as reasons for overriding CDS (as in Chen et al.[6]) offer significant insights on how to optimize CDS, and should be monitored systematically on an ongoing basis to ensure the EMR addresses users' changing needs and mental models.

So what is the clinician with an interest in improving healthcare outcomes and organizational efficiency to do given CDS' limitations? One option is to wait for the science of CDS to further mature and have those advances embedded in the EMR at your organization. Another option might be to rely on the information technology and clinical informatics professionals at your organization to decide how CDS should be used locally. In 2014, these may be untenable choices for the following reasons. First, given the universal pressures to improve healthcare outcomes and contain costs,[14] healthcare organizations must use all available tools to achieve challenging performance goals. Second, as EMRs with CDS become commonplace, and as the 3 articles in this issue of the Journal of Hospital Medicine and others have illustrated, there are many opportunities to misuse or poorly implement CDS, with potentially dire consequences.[15] Third, design and deployment of effective CDS require information technology and informatics professionals to collaborate with clinicians to gauge the quality of EMR data used to drive CDS and clinicians' receptivity to CDS, illuminate the sociotechnical context in which to deploy the CDS, and champion the CDS intervention among their colleagues. Clinicians' input is therefore an essential ingredient to success. Fourth, organizational trust, a key aspect of a healthy safety culture, is hard to build and easy to erode.[9, 16] If clinicians at an organization lose trust in CDS because of poor design and deployment strategies, they are likely to ignore CDS in the future.[17]

Like tools introduced into medicine such as magnetic resonance imaging and highly active antiretroviral therapy, CDS will need to evolve as the clinical community grapples with its potential and limitations. As EMRs move toward ubiquity in the hospital setting, CDS will become part of the fabric of hospital‐based practice, and the Journal of Hospital Medicine readership would do well to learn about this new tool of the trade.

If you wish to receive credit for this activity, please refer to the website: www.wileyblackwellcme.com.

Accreditation and Designation Statement

Blackwell Futura Media Services designates this journal‐based CME activity for a maximum of 1 AMA PRA Category 1 Credit.. Physicians should only claim credit commensurate with the extent of their participation in the activity.

Blackwell Futura Media Services is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians.

Educational Objectives

Upon completion of this educational activity, participants will be better able to:

• Illustrate the elements of a systematic approach to successful hospital smoking cessation programs.

• Describe the efficacy of a coordinated real world” hospital smoking cessation program in a U.S. hospital.

• Evaluate the barriers to successful hospital smoking cessation programs.

This manuscript underwent peer review in line with the standards of editorial integrity and publication ethics maintained by Journal of Hospital Medicine. The peer reviewers have no relevant financial relationships. The peer review process for Journal of Hospital Medicine is blinded. As such, the identities of the reviewers are not disclosed in line with the standard accepted practices of medical journal peer review.

Conflicts of interest have been identified and resolved in accordance with Blackwell Futura Media Services's Policy on Activity Disclosure and Conflict of Interest. The primary resolution method used was peer review and review by a non‐conflicted expert.

Instructions on Receiving Credit

For information on applicability and acceptance of CME credit for this activity, please consult your professional licensing board.

This activity is designed to be completed within an hour; physicians should claim only those credits that reflect the time actually spent in the activity. To successfully earn credit, participants must complete the activity during the valid credit period, which is up to two years from initial publication.

Follow these steps to earn credit:

• Log on to www.wileyblackwellcme.com

• Read the target audience, learning objectives, and author disclosures.

• Read the article in print or online format.

• Reflect on the article.

• Access the CME Exam, and choose the best answer to each question.

• Complete the required evaluation component of the activity.

This activity will be available for CME credit for twelve months following its publication date. At that time, it will be reviewed and potentially updated and extended for an additional twelve months.

If you wish to receive credit for this activity, please refer to the website: www.wileyblackwellcme.com.

Accreditation and Designation Statement

Blackwell Futura Media Services designates this journal‐based CME activity for a maximum of 1 AMA PRA Category 1 Credit.. Physicians should only claim credit commensurate with the extent of their participation in the activity.

Blackwell Futura Media Services is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians.

Educational Objectives

Upon completion of this educational activity, participants will be better able to:

• Illustrate the elements of a systematic approach to successful hospital smoking cessation programs.

• Describe the efficacy of a coordinated real world” hospital smoking cessation program in a U.S. hospital.

• Evaluate the barriers to successful hospital smoking cessation programs.

This manuscript underwent peer review in line with the standards of editorial integrity and publication ethics maintained by Journal of Hospital Medicine. The peer reviewers have no relevant financial relationships. The peer review process for Journal of Hospital Medicine is blinded. As such, the identities of the reviewers are not disclosed in line with the standard accepted practices of medical journal peer review.

Conflicts of interest have been identified and resolved in accordance with Blackwell Futura Media Services's Policy on Activity Disclosure and Conflict of Interest. The primary resolution method used was peer review and review by a non‐conflicted expert.

Instructions on Receiving Credit

For information on applicability and acceptance of CME credit for this activity, please consult your professional licensing board.

This activity is designed to be completed within an hour; physicians should claim only those credits that reflect the time actually spent in the activity. To successfully earn credit, participants must complete the activity during the valid credit period, which is up to two years from initial publication.

Follow these steps to earn credit:

• Log on to www.wileyblackwellcme.com

• Read the target audience, learning objectives, and author disclosures.

• Read the article in print or online format.

• Reflect on the article.

• Access the CME Exam, and choose the best answer to each question.

• Complete the required evaluation component of the activity.

This activity will be available for CME credit for twelve months following its publication date. At that time, it will be reviewed and potentially updated and extended for an additional twelve months.

If you wish to receive credit for this activity, please refer to the website: www.wileyblackwellcme.com.

Accreditation and Designation Statement

Blackwell Futura Media Services designates this journal‐based CME activity for a maximum of 1 AMA PRA Category 1 Credit.. Physicians should only claim credit commensurate with the extent of their participation in the activity.

Blackwell Futura Media Services is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians.

Educational Objectives

Upon completion of this educational activity, participants will be better able to:

• Illustrate the elements of a systematic approach to successful hospital smoking cessation programs.

• Describe the efficacy of a coordinated real world” hospital smoking cessation program in a U.S. hospital.

• Evaluate the barriers to successful hospital smoking cessation programs.

This manuscript underwent peer review in line with the standards of editorial integrity and publication ethics maintained by Journal of Hospital Medicine. The peer reviewers have no relevant financial relationships. The peer review process for Journal of Hospital Medicine is blinded. As such, the identities of the reviewers are not disclosed in line with the standard accepted practices of medical journal peer review.

Conflicts of interest have been identified and resolved in accordance with Blackwell Futura Media Services's Policy on Activity Disclosure and Conflict of Interest. The primary resolution method used was peer review and review by a non‐conflicted expert.

Instructions on Receiving Credit

For information on applicability and acceptance of CME credit for this activity, please consult your professional licensing board.

This activity is designed to be completed within an hour; physicians should claim only those credits that reflect the time actually spent in the activity. To successfully earn credit, participants must complete the activity during the valid credit period, which is up to two years from initial publication.

Follow these steps to earn credit:

• Log on to www.wileyblackwellcme.com

• Read the target audience, learning objectives, and author disclosures.

• Read the article in print or online format.

• Reflect on the article.

• Access the CME Exam, and choose the best answer to each question.

• Complete the required evaluation component of the activity.

This activity will be available for CME credit for twelve months following its publication date. At that time, it will be reviewed and potentially updated and extended for an additional twelve months.

Many hospitalized patients have microbiology test results pending at the time of discharge.1, 2 Failure to follow up on these results in a timely fashion can lead to delays in diagnosis and adequate treatment of important infections. Prompt communication of the results of these pending tests to the responsible providers is crucial to minimize these delays.36 As hospitalized patients are increasingly cared for by clinicians other than their primary care providers,7 important information may be lost during the discharge process.8 This increasing fragmentation makes reliable communication of pending tests even more crucial.9, 10

Studies to date have primarily investigated tests from general medical services. In that setting, there is clearly room for improvement in test result communication. Discharge summaries often do not reach the outpatient providers at the time of the patients' follow‐up visits after hospitalization.11 When the discharge summaries are present, the majority of pending tests are not mentioned in them,2, 12, 13 and both inpatient and outpatient physicians are unaware of most of these results when they return.1 However, the specific characteristics of postdischarge microbiology results and the extent to which these results represent potential follow‐up errors in specialties other than general medicine have not been adequately studied.

We aimed to describe the issue of microbiology tests pending at the time of discharge from a hospital‐wide perspective. Specifically, we sought to determine: (1) frequency and characteristics of these results across all admitting services; and (2) how often these results potentially require a change in antimicrobial therapy.

Methods

Manual Review

We manually reviewed a random sample of 94 of the clinically important and untreated postdischarge results to determine if the results potentially required a change in therapy and therefore required follow‐up. For each case, the electronic patient chart was reviewed by two internal medicine‐trained physicians on the study staff. Each reviewer was blinded to events that occurred after the cultures returned, and determined whether the results necessitated a potential change in antibiotic. The reviewer then indicated the level of certainty of that determination on a 6‐point Likert scale that had been previously used in reviews to identify adverse medical events15, 16: 1 = little or no evidence, 2 = slight evidence, 3 = not quite likely (<50:50 but close call), 4 = more likely than not (>50:50 but close call), 5 = strong evidence, and 6 = virtually certain evidence. To standardize the assignment of certainty for potential need for antibiotic change, we used a set of review guidelines developed by our study staff (Figure 1). A microbiology result was defined as potentially necessitating antibiotic change if both reviewers indicated as such and recorded a certainty with a score 4. Differences in assessments were resolved through discussion of the case between the reviewers.

Figure 1

Results

Of the 111,331 blood, urine, sputum, and CSF cultures analyzed, 77,349 (69%) were collected from hospitalized patients. The majority (63%) of the inpatient results were for blood cultures and one quarter (24%) were for urine cultures. Table 1 shows the distribution of the microbiology results across primary services responsible for the patients at the time of discharge. Half (49%) of the patients from whom the specimens were collected were female. The mean age of patients was 55 years. Most (68%) were white and most (86%) had either commercial insurance or Medicare (Table 1).

Of the 77,349 microbiology tests obtained during hospital stays, 8668 (11%) of the inpatient microbiology results were reported after the patients were discharged from the hospital. Most (54%) of these postdischarge results were for blood cultures. The distribution of results across primary hospital service, patient sex, race, insurance, and mean patient age were similar to those for all inpatient results (Table 1). Of the 8668 postdischarge results, 385 (4%) met our screening criteria of being both clinically important and not treated by an antibiotic to which the organism was found susceptible at the time of discharge from the hospital. After manual review of a random subset of 94 of these screen‐positive cases, 50 (53%) required follow‐up (Figure 2). The interrater reliability for the reviewers was found to be kappa = 0.58 (P < 0.001). From our results, we estimated that 2.4% of the postdischarge microbiology results required follow‐up and potential change in therapy.

Figure 2

Potential need for antibiotic change was present in 30 of 45 (67%) urine cultures, 12 of 32 (38%) blood cultures, 8 of 16 (50%) sputum cultures, and 0 of 1 (0%) CSF cultures. By primary service, reviewers identified a potential need for antibiotic change in 19 of 25 (76%) of results from surgical services, 17 of 29 (59%) from general medicine, 6 of 16 (38%) from oncology, and 8 of 23 (35%) from medical subspecialties. Examples of cases that potentially required antibiotic change are shown in Table 2.

In our logistic regression model, both the type of culture and the primary hospital service were found to be significant predictors of a potential need for antibiotic change in the manually reviewed cases. Urine cultures were more likely than non‐urine cultures to potentially require antibiotic change (P = 0.03; OR 2.8, 95% CI 1.1‐7.2). Results from surgical services were most likely to potentially require antibiotic change, followed by general medicine, oncology, and medical subspecialties (Table 3).

Discussion

We performed a retrospective analysis of all blood, urine, sputum, and CSF cultures finalized at our institution in 2007 and found that many returned after patients were discharged. Overall, we estimated that 2.4% of these postdischarge results potentially required a change in antibiotic. This proportion varied by culture type and by primary hospital service at the time of discharge, with urine cultures and cultures from surgical services being most likely to potentially need change in antibiotic.

We speculate that postdischarge urine cultures may have been more likely to require antibiotic change in part due to different urgency that clinicians assign to different culture types. Urinary tract infections may present with more vague, transient, or minor complaints compared with bacteremia, pneumonia, and cerebrospinal fluid infections. For that reason, clinicians may be more likely to forego empiric antibiotics for pending urine cultures in favor of watchful waiting. Therefore, the postdischarge urine cultures with growth may include a higher proportion of untreated true infections compared with other culture types.

A similar difference in prescription of empiric antibiotics may help explain the differences seen across primary hospital specialties. For example, if patients on surgical services were less likely to receive empiric antibiotics, then the pool of postdischarge results would be more likely to include true infections that require antibiotic change. Furthermore, it is possible that surgical services may tend to order cultures for patients only if they already have convincing evidence of infections. It may be that selecting a group with higher likelihood of infection led to a higher proportion of true infections in surgical patients with cultures with growth.

Prior studies led by Roy and Were illustrated that pending microbiology results from general medicine services were often not communicated and followed up adequately.1, 2 For patients discharged with pending test results, between 47% and 89% of discharge summaries did not mention the pending tests.2, 12, 13, 17 These deficiencies in discharge summaries likely have a substantial impact on the proportion of tests followed up by outpatient clinicians. By extending the analysis hospital‐wide, our study suggests that pending microbiology results occur for a wide range of hospital services. While our study was not designed to determine whether these results were followed up appropriately, opportunities for miscommunication and missed follow‐up likely exist for all specialties.

The potential harms associated with inadequate test follow‐up have gained the attention of the patient safety community. In 2005, the Joint Commission underscored the importance of proper communication of critical lab results.3, 5, 18 Their recommendations included the development of systems to ensure adequate follow‐up of critical results in high‐risk scenarios including the postdischarge period.5 While many of the microbiology results do not fall into the criticalcategory, we feel that these results should be considered for inclusion in hospital efforts to track postdischarge results. These efforts should also address issues specific to microbiology results, such as preliminary status before antibiotic sensitivities are known.

Developing a comprehensive strategy for test result communication is challenging, and more so for results that return after transitions of care. Even defining the proper target of communication interventions can involve complex organizational and cultural issues. As these results span the inpatient and outpatient domains, there may be some ambiguity as to which provider is responsible when the results return. The inpatient clinicians ordering the microbiology cultures are in the best position to put the results into the patient's clinical context. However, these clinicians may no longer be on clinical duty when the results return, or they may not have a system to ensure that they are notified about these results. While the outpatient providers may be available, they have often not seen the patient in follow‐up at the time the results return and would need to repeat a clinical assessment to determine whether a change in antibiotics is required. While many feel that the ordering provider is a logical choice to perform the follow‐up of the result, not all agree and few institutions have developed clear policies on this issue. To avoid this ambiguity, future work will require institutions to clearly outline which party is responsible for test result follow‐up during transitions of care.

Potential solutions to improve communication of these results must be tailored to the local infrastructure of the institution. In hospitals that do not have extensive electronic systems, a solution might involve a registered nurse, nurse practitioner, or lab technician whose responsibilities include identifying postdischarge results and communicating them to the ordering clinician, primary care provider, and patient. In settings with more advanced electronic infrastructure, solutions could be designed to automatically notify the responsible providers electronically, as well as post the results to a patient portal. Regardless of the level of technical sophistication, it is vital to create a system that has is highly reliable to prevent these important results from falling through the cracks.

Our study did have some limitations. First, we evaluated results from only one institution. It is unclear how substantially differences in practice patterns or patient populations would affect the number of postdischarge microbiology results in other settings. Second, we did not assess whether these results were actually followed up or whether treatment regimens were altered. As this study was retrospective in nature, we could not expect clinicians to recall the clinical scenarios surrounding each result and decided that documentation in clinical notes would be an unreliable indicator of whether any follow‐up action had been taken. Even without this information, however, we would submit that our findings represent a substantial near‐miss rate and threat to patient safety (approximately one potentially actionable, postdischarge microbiology result every other day for our hospital), and call for a fail‐safe system to ensure appropriate actions are taken.

In conclusion, microbiology results are often pending at the time patients are discharged from the hospital and roughly 2.4% of these results potentially require a change in therapy. This proportion was highest for urine cultures and cultures drawn from surgical patients. Our results suggest that a hospital‐wide system is warranted to ensure adequate communication of postdischarge microbiology results. Further research is required to evaluate the impact of such a system on the follow‐up rates of pending microbiology tests.

Many hospitalized patients have microbiology test results pending at the time of discharge.1, 2 Failure to follow up on these results in a timely fashion can lead to delays in diagnosis and adequate treatment of important infections. Prompt communication of the results of these pending tests to the responsible providers is crucial to minimize these delays.36 As hospitalized patients are increasingly cared for by clinicians other than their primary care providers,7 important information may be lost during the discharge process.8 This increasing fragmentation makes reliable communication of pending tests even more crucial.9, 10

Studies to date have primarily investigated tests from general medical services. In that setting, there is clearly room for improvement in test result communication. Discharge summaries often do not reach the outpatient providers at the time of the patients' follow‐up visits after hospitalization.11 When the discharge summaries are present, the majority of pending tests are not mentioned in them,2, 12, 13 and both inpatient and outpatient physicians are unaware of most of these results when they return.1 However, the specific characteristics of postdischarge microbiology results and the extent to which these results represent potential follow‐up errors in specialties other than general medicine have not been adequately studied.

We aimed to describe the issue of microbiology tests pending at the time of discharge from a hospital‐wide perspective. Specifically, we sought to determine: (1) frequency and characteristics of these results across all admitting services; and (2) how often these results potentially require a change in antimicrobial therapy.

Methods

Manual Review

We manually reviewed a random sample of 94 of the clinically important and untreated postdischarge results to determine if the results potentially required a change in therapy and therefore required follow‐up. For each case, the electronic patient chart was reviewed by two internal medicine‐trained physicians on the study staff. Each reviewer was blinded to events that occurred after the cultures returned, and determined whether the results necessitated a potential change in antibiotic. The reviewer then indicated the level of certainty of that determination on a 6‐point Likert scale that had been previously used in reviews to identify adverse medical events15, 16: 1 = little or no evidence, 2 = slight evidence, 3 = not quite likely (<50:50 but close call), 4 = more likely than not (>50:50 but close call), 5 = strong evidence, and 6 = virtually certain evidence. To standardize the assignment of certainty for potential need for antibiotic change, we used a set of review guidelines developed by our study staff (Figure 1). A microbiology result was defined as potentially necessitating antibiotic change if both reviewers indicated as such and recorded a certainty with a score 4. Differences in assessments were resolved through discussion of the case between the reviewers.

Figure 1

Results

Of the 111,331 blood, urine, sputum, and CSF cultures analyzed, 77,349 (69%) were collected from hospitalized patients. The majority (63%) of the inpatient results were for blood cultures and one quarter (24%) were for urine cultures. Table 1 shows the distribution of the microbiology results across primary services responsible for the patients at the time of discharge. Half (49%) of the patients from whom the specimens were collected were female. The mean age of patients was 55 years. Most (68%) were white and most (86%) had either commercial insurance or Medicare (Table 1).

Of the 77,349 microbiology tests obtained during hospital stays, 8668 (11%) of the inpatient microbiology results were reported after the patients were discharged from the hospital. Most (54%) of these postdischarge results were for blood cultures. The distribution of results across primary hospital service, patient sex, race, insurance, and mean patient age were similar to those for all inpatient results (Table 1). Of the 8668 postdischarge results, 385 (4%) met our screening criteria of being both clinically important and not treated by an antibiotic to which the organism was found susceptible at the time of discharge from the hospital. After manual review of a random subset of 94 of these screen‐positive cases, 50 (53%) required follow‐up (Figure 2). The interrater reliability for the reviewers was found to be kappa = 0.58 (P < 0.001). From our results, we estimated that 2.4% of the postdischarge microbiology results required follow‐up and potential change in therapy.

Figure 2

Potential need for antibiotic change was present in 30 of 45 (67%) urine cultures, 12 of 32 (38%) blood cultures, 8 of 16 (50%) sputum cultures, and 0 of 1 (0%) CSF cultures. By primary service, reviewers identified a potential need for antibiotic change in 19 of 25 (76%) of results from surgical services, 17 of 29 (59%) from general medicine, 6 of 16 (38%) from oncology, and 8 of 23 (35%) from medical subspecialties. Examples of cases that potentially required antibiotic change are shown in Table 2.

In our logistic regression model, both the type of culture and the primary hospital service were found to be significant predictors of a potential need for antibiotic change in the manually reviewed cases. Urine cultures were more likely than non‐urine cultures to potentially require antibiotic change (P = 0.03; OR 2.8, 95% CI 1.1‐7.2). Results from surgical services were most likely to potentially require antibiotic change, followed by general medicine, oncology, and medical subspecialties (Table 3).

Discussion

We performed a retrospective analysis of all blood, urine, sputum, and CSF cultures finalized at our institution in 2007 and found that many returned after patients were discharged. Overall, we estimated that 2.4% of these postdischarge results potentially required a change in antibiotic. This proportion varied by culture type and by primary hospital service at the time of discharge, with urine cultures and cultures from surgical services being most likely to potentially need change in antibiotic.

We speculate that postdischarge urine cultures may have been more likely to require antibiotic change in part due to different urgency that clinicians assign to different culture types. Urinary tract infections may present with more vague, transient, or minor complaints compared with bacteremia, pneumonia, and cerebrospinal fluid infections. For that reason, clinicians may be more likely to forego empiric antibiotics for pending urine cultures in favor of watchful waiting. Therefore, the postdischarge urine cultures with growth may include a higher proportion of untreated true infections compared with other culture types.

A similar difference in prescription of empiric antibiotics may help explain the differences seen across primary hospital specialties. For example, if patients on surgical services were less likely to receive empiric antibiotics, then the pool of postdischarge results would be more likely to include true infections that require antibiotic change. Furthermore, it is possible that surgical services may tend to order cultures for patients only if they already have convincing evidence of infections. It may be that selecting a group with higher likelihood of infection led to a higher proportion of true infections in surgical patients with cultures with growth.

Prior studies led by Roy and Were illustrated that pending microbiology results from general medicine services were often not communicated and followed up adequately.1, 2 For patients discharged with pending test results, between 47% and 89% of discharge summaries did not mention the pending tests.2, 12, 13, 17 These deficiencies in discharge summaries likely have a substantial impact on the proportion of tests followed up by outpatient clinicians. By extending the analysis hospital‐wide, our study suggests that pending microbiology results occur for a wide range of hospital services. While our study was not designed to determine whether these results were followed up appropriately, opportunities for miscommunication and missed follow‐up likely exist for all specialties.

The potential harms associated with inadequate test follow‐up have gained the attention of the patient safety community. In 2005, the Joint Commission underscored the importance of proper communication of critical lab results.3, 5, 18 Their recommendations included the development of systems to ensure adequate follow‐up of critical results in high‐risk scenarios including the postdischarge period.5 While many of the microbiology results do not fall into the criticalcategory, we feel that these results should be considered for inclusion in hospital efforts to track postdischarge results. These efforts should also address issues specific to microbiology results, such as preliminary status before antibiotic sensitivities are known.

Developing a comprehensive strategy for test result communication is challenging, and more so for results that return after transitions of care. Even defining the proper target of communication interventions can involve complex organizational and cultural issues. As these results span the inpatient and outpatient domains, there may be some ambiguity as to which provider is responsible when the results return. The inpatient clinicians ordering the microbiology cultures are in the best position to put the results into the patient's clinical context. However, these clinicians may no longer be on clinical duty when the results return, or they may not have a system to ensure that they are notified about these results. While the outpatient providers may be available, they have often not seen the patient in follow‐up at the time the results return and would need to repeat a clinical assessment to determine whether a change in antibiotics is required. While many feel that the ordering provider is a logical choice to perform the follow‐up of the result, not all agree and few institutions have developed clear policies on this issue. To avoid this ambiguity, future work will require institutions to clearly outline which party is responsible for test result follow‐up during transitions of care.

Potential solutions to improve communication of these results must be tailored to the local infrastructure of the institution. In hospitals that do not have extensive electronic systems, a solution might involve a registered nurse, nurse practitioner, or lab technician whose responsibilities include identifying postdischarge results and communicating them to the ordering clinician, primary care provider, and patient. In settings with more advanced electronic infrastructure, solutions could be designed to automatically notify the responsible providers electronically, as well as post the results to a patient portal. Regardless of the level of technical sophistication, it is vital to create a system that has is highly reliable to prevent these important results from falling through the cracks.

Our study did have some limitations. First, we evaluated results from only one institution. It is unclear how substantially differences in practice patterns or patient populations would affect the number of postdischarge microbiology results in other settings. Second, we did not assess whether these results were actually followed up or whether treatment regimens were altered. As this study was retrospective in nature, we could not expect clinicians to recall the clinical scenarios surrounding each result and decided that documentation in clinical notes would be an unreliable indicator of whether any follow‐up action had been taken. Even without this information, however, we would submit that our findings represent a substantial near‐miss rate and threat to patient safety (approximately one potentially actionable, postdischarge microbiology result every other day for our hospital), and call for a fail‐safe system to ensure appropriate actions are taken.

In conclusion, microbiology results are often pending at the time patients are discharged from the hospital and roughly 2.4% of these results potentially require a change in therapy. This proportion was highest for urine cultures and cultures drawn from surgical patients. Our results suggest that a hospital‐wide system is warranted to ensure adequate communication of postdischarge microbiology results. Further research is required to evaluate the impact of such a system on the follow‐up rates of pending microbiology tests.

Many hospitalized patients have microbiology test results pending at the time of discharge.1, 2 Failure to follow up on these results in a timely fashion can lead to delays in diagnosis and adequate treatment of important infections. Prompt communication of the results of these pending tests to the responsible providers is crucial to minimize these delays.36 As hospitalized patients are increasingly cared for by clinicians other than their primary care providers,7 important information may be lost during the discharge process.8 This increasing fragmentation makes reliable communication of pending tests even more crucial.9, 10

Studies to date have primarily investigated tests from general medical services. In that setting, there is clearly room for improvement in test result communication. Discharge summaries often do not reach the outpatient providers at the time of the patients' follow‐up visits after hospitalization.11 When the discharge summaries are present, the majority of pending tests are not mentioned in them,2, 12, 13 and both inpatient and outpatient physicians are unaware of most of these results when they return.1 However, the specific characteristics of postdischarge microbiology results and the extent to which these results represent potential follow‐up errors in specialties other than general medicine have not been adequately studied.

We aimed to describe the issue of microbiology tests pending at the time of discharge from a hospital‐wide perspective. Specifically, we sought to determine: (1) frequency and characteristics of these results across all admitting services; and (2) how often these results potentially require a change in antimicrobial therapy.

Methods

Manual Review

We manually reviewed a random sample of 94 of the clinically important and untreated postdischarge results to determine if the results potentially required a change in therapy and therefore required follow‐up. For each case, the electronic patient chart was reviewed by two internal medicine‐trained physicians on the study staff. Each reviewer was blinded to events that occurred after the cultures returned, and determined whether the results necessitated a potential change in antibiotic. The reviewer then indicated the level of certainty of that determination on a 6‐point Likert scale that had been previously used in reviews to identify adverse medical events15, 16: 1 = little or no evidence, 2 = slight evidence, 3 = not quite likely (<50:50 but close call), 4 = more likely than not (>50:50 but close call), 5 = strong evidence, and 6 = virtually certain evidence. To standardize the assignment of certainty for potential need for antibiotic change, we used a set of review guidelines developed by our study staff (Figure 1). A microbiology result was defined as potentially necessitating antibiotic change if both reviewers indicated as such and recorded a certainty with a score 4. Differences in assessments were resolved through discussion of the case between the reviewers.

Figure 1

Results

Of the 111,331 blood, urine, sputum, and CSF cultures analyzed, 77,349 (69%) were collected from hospitalized patients. The majority (63%) of the inpatient results were for blood cultures and one quarter (24%) were for urine cultures. Table 1 shows the distribution of the microbiology results across primary services responsible for the patients at the time of discharge. Half (49%) of the patients from whom the specimens were collected were female. The mean age of patients was 55 years. Most (68%) were white and most (86%) had either commercial insurance or Medicare (Table 1).

Of the 77,349 microbiology tests obtained during hospital stays, 8668 (11%) of the inpatient microbiology results were reported after the patients were discharged from the hospital. Most (54%) of these postdischarge results were for blood cultures. The distribution of results across primary hospital service, patient sex, race, insurance, and mean patient age were similar to those for all inpatient results (Table 1). Of the 8668 postdischarge results, 385 (4%) met our screening criteria of being both clinically important and not treated by an antibiotic to which the organism was found susceptible at the time of discharge from the hospital. After manual review of a random subset of 94 of these screen‐positive cases, 50 (53%) required follow‐up (Figure 2). The interrater reliability for the reviewers was found to be kappa = 0.58 (P < 0.001). From our results, we estimated that 2.4% of the postdischarge microbiology results required follow‐up and potential change in therapy.

Figure 2

Potential need for antibiotic change was present in 30 of 45 (67%) urine cultures, 12 of 32 (38%) blood cultures, 8 of 16 (50%) sputum cultures, and 0 of 1 (0%) CSF cultures. By primary service, reviewers identified a potential need for antibiotic change in 19 of 25 (76%) of results from surgical services, 17 of 29 (59%) from general medicine, 6 of 16 (38%) from oncology, and 8 of 23 (35%) from medical subspecialties. Examples of cases that potentially required antibiotic change are shown in Table 2.

In our logistic regression model, both the type of culture and the primary hospital service were found to be significant predictors of a potential need for antibiotic change in the manually reviewed cases. Urine cultures were more likely than non‐urine cultures to potentially require antibiotic change (P = 0.03; OR 2.8, 95% CI 1.1‐7.2). Results from surgical services were most likely to potentially require antibiotic change, followed by general medicine, oncology, and medical subspecialties (Table 3).

Discussion

We performed a retrospective analysis of all blood, urine, sputum, and CSF cultures finalized at our institution in 2007 and found that many returned after patients were discharged. Overall, we estimated that 2.4% of these postdischarge results potentially required a change in antibiotic. This proportion varied by culture type and by primary hospital service at the time of discharge, with urine cultures and cultures from surgical services being most likely to potentially need change in antibiotic.

We speculate that postdischarge urine cultures may have been more likely to require antibiotic change in part due to different urgency that clinicians assign to different culture types. Urinary tract infections may present with more vague, transient, or minor complaints compared with bacteremia, pneumonia, and cerebrospinal fluid infections. For that reason, clinicians may be more likely to forego empiric antibiotics for pending urine cultures in favor of watchful waiting. Therefore, the postdischarge urine cultures with growth may include a higher proportion of untreated true infections compared with other culture types.

A similar difference in prescription of empiric antibiotics may help explain the differences seen across primary hospital specialties. For example, if patients on surgical services were less likely to receive empiric antibiotics, then the pool of postdischarge results would be more likely to include true infections that require antibiotic change. Furthermore, it is possible that surgical services may tend to order cultures for patients only if they already have convincing evidence of infections. It may be that selecting a group with higher likelihood of infection led to a higher proportion of true infections in surgical patients with cultures with growth.

Prior studies led by Roy and Were illustrated that pending microbiology results from general medicine services were often not communicated and followed up adequately.1, 2 For patients discharged with pending test results, between 47% and 89% of discharge summaries did not mention the pending tests.2, 12, 13, 17 These deficiencies in discharge summaries likely have a substantial impact on the proportion of tests followed up by outpatient clinicians. By extending the analysis hospital‐wide, our study suggests that pending microbiology results occur for a wide range of hospital services. While our study was not designed to determine whether these results were followed up appropriately, opportunities for miscommunication and missed follow‐up likely exist for all specialties.

The potential harms associated with inadequate test follow‐up have gained the attention of the patient safety community. In 2005, the Joint Commission underscored the importance of proper communication of critical lab results.3, 5, 18 Their recommendations included the development of systems to ensure adequate follow‐up of critical results in high‐risk scenarios including the postdischarge period.5 While many of the microbiology results do not fall into the criticalcategory, we feel that these results should be considered for inclusion in hospital efforts to track postdischarge results. These efforts should also address issues specific to microbiology results, such as preliminary status before antibiotic sensitivities are known.

Developing a comprehensive strategy for test result communication is challenging, and more so for results that return after transitions of care. Even defining the proper target of communication interventions can involve complex organizational and cultural issues. As these results span the inpatient and outpatient domains, there may be some ambiguity as to which provider is responsible when the results return. The inpatient clinicians ordering the microbiology cultures are in the best position to put the results into the patient's clinical context. However, these clinicians may no longer be on clinical duty when the results return, or they may not have a system to ensure that they are notified about these results. While the outpatient providers may be available, they have often not seen the patient in follow‐up at the time the results return and would need to repeat a clinical assessment to determine whether a change in antibiotics is required. While many feel that the ordering provider is a logical choice to perform the follow‐up of the result, not all agree and few institutions have developed clear policies on this issue. To avoid this ambiguity, future work will require institutions to clearly outline which party is responsible for test result follow‐up during transitions of care.

Potential solutions to improve communication of these results must be tailored to the local infrastructure of the institution. In hospitals that do not have extensive electronic systems, a solution might involve a registered nurse, nurse practitioner, or lab technician whose responsibilities include identifying postdischarge results and communicating them to the ordering clinician, primary care provider, and patient. In settings with more advanced electronic infrastructure, solutions could be designed to automatically notify the responsible providers electronically, as well as post the results to a patient portal. Regardless of the level of technical sophistication, it is vital to create a system that has is highly reliable to prevent these important results from falling through the cracks.

Our study did have some limitations. First, we evaluated results from only one institution. It is unclear how substantially differences in practice patterns or patient populations would affect the number of postdischarge microbiology results in other settings. Second, we did not assess whether these results were actually followed up or whether treatment regimens were altered. As this study was retrospective in nature, we could not expect clinicians to recall the clinical scenarios surrounding each result and decided that documentation in clinical notes would be an unreliable indicator of whether any follow‐up action had been taken. Even without this information, however, we would submit that our findings represent a substantial near‐miss rate and threat to patient safety (approximately one potentially actionable, postdischarge microbiology result every other day for our hospital), and call for a fail‐safe system to ensure appropriate actions are taken.

In conclusion, microbiology results are often pending at the time patients are discharged from the hospital and roughly 2.4% of these results potentially require a change in therapy. This proportion was highest for urine cultures and cultures drawn from surgical patients. Our results suggest that a hospital‐wide system is warranted to ensure adequate communication of postdischarge microbiology results. Further research is required to evaluate the impact of such a system on the follow‐up rates of pending microbiology tests.

If you wish to receive credit for this activity, which beginson the next page, please refer to the website: www.blackwellpublishing.com/cme.

Accreditation and Designation Statement

Blackwell Futura Media Services designates this educational activity for a 1 AMA PRA Category 1 Credit. Physicians should only claim credit commensurate with the extent of their participation in the activity.

Blackwell Futura Media Services is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians.

Educational Objectives

Continuous participation in the Journal of Hospital Medicine CME program will enable learners to be better able to:

• Interpret clinical guidelines and their applications for higher quality and more efficient care for all hospitalized patients.

• Describe the standard of care for common illnesses and conditions treated in the hospital; such as pneumonia, COPD exacerbation, acute coronary syndrome, HF exacerbation, glycemic control, venous thromboembolic disease, stroke, etc.

• Discuss evidence‐based recommendations involving transitions of care, including the hospital discharge process.

• Gain insights into the roles of hospitalists as medical educators, researchers, medical ethicists, palliative care providers, and hospital‐based geriatricians.

• Incorporate best practices for hospitalist administration, including quality improvement, patient safety, practice management, leadership, and demonstrating hospitalist value.

• Identify evidence‐based best practices and trends for both adult and pediatric hospital medicine.

Instructions on Receiving Credit

For information on applicability and acceptance of continuing medical education credit for this activity, please consult your professional licensing board.

This activity is designed to be completed within the time designated on the title page; physicians should claim only those credits that reflect the time actually spent in the activity. To successfully earn credit, participants must complete the activity during the valid credit period that is noted on the title page.

Follow these steps to earn credit:

• Log on to www.blackwellpublishing.com/cme.

• Read the target audience, learning objectives, and author disclosures.

• Read the article in print or online format.

• Reflect on the article.

• Access the CME Exam, and choose the best answer to each question.

• Complete the required evaluation component of the activity.

If you wish to receive credit for this activity, which beginson the next page, please refer to the website: www.blackwellpublishing.com/cme.

Accreditation and Designation Statement

Blackwell Futura Media Services designates this educational activity for a 1 AMA PRA Category 1 Credit. Physicians should only claim credit commensurate with the extent of their participation in the activity.

Blackwell Futura Media Services is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians.

Educational Objectives

Continuous participation in the Journal of Hospital Medicine CME program will enable learners to be better able to:

• Interpret clinical guidelines and their applications for higher quality and more efficient care for all hospitalized patients.

• Describe the standard of care for common illnesses and conditions treated in the hospital; such as pneumonia, COPD exacerbation, acute coronary syndrome, HF exacerbation, glycemic control, venous thromboembolic disease, stroke, etc.

• Discuss evidence‐based recommendations involving transitions of care, including the hospital discharge process.

• Gain insights into the roles of hospitalists as medical educators, researchers, medical ethicists, palliative care providers, and hospital‐based geriatricians.

• Incorporate best practices for hospitalist administration, including quality improvement, patient safety, practice management, leadership, and demonstrating hospitalist value.

• Identify evidence‐based best practices and trends for both adult and pediatric hospital medicine.

Instructions on Receiving Credit

For information on applicability and acceptance of continuing medical education credit for this activity, please consult your professional licensing board.

This activity is designed to be completed within the time designated on the title page; physicians should claim only those credits that reflect the time actually spent in the activity. To successfully earn credit, participants must complete the activity during the valid credit period that is noted on the title page.

Follow these steps to earn credit:

• Log on to www.blackwellpublishing.com/cme.

• Read the target audience, learning objectives, and author disclosures.

• Read the article in print or online format.

• Reflect on the article.

• Access the CME Exam, and choose the best answer to each question.

• Complete the required evaluation component of the activity.

If you wish to receive credit for this activity, which beginson the next page, please refer to the website: www.blackwellpublishing.com/cme.

Accreditation and Designation Statement

Blackwell Futura Media Services designates this educational activity for a 1 AMA PRA Category 1 Credit. Physicians should only claim credit commensurate with the extent of their participation in the activity.

Blackwell Futura Media Services is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians.

Educational Objectives

Continuous participation in the Journal of Hospital Medicine CME program will enable learners to be better able to:

• Interpret clinical guidelines and their applications for higher quality and more efficient care for all hospitalized patients.

• Describe the standard of care for common illnesses and conditions treated in the hospital; such as pneumonia, COPD exacerbation, acute coronary syndrome, HF exacerbation, glycemic control, venous thromboembolic disease, stroke, etc.

• Discuss evidence‐based recommendations involving transitions of care, including the hospital discharge process.

• Gain insights into the roles of hospitalists as medical educators, researchers, medical ethicists, palliative care providers, and hospital‐based geriatricians.

• Incorporate best practices for hospitalist administration, including quality improvement, patient safety, practice management, leadership, and demonstrating hospitalist value.

• Identify evidence‐based best practices and trends for both adult and pediatric hospital medicine.

Instructions on Receiving Credit

For information on applicability and acceptance of continuing medical education credit for this activity, please consult your professional licensing board.

This activity is designed to be completed within the time designated on the title page; physicians should claim only those credits that reflect the time actually spent in the activity. To successfully earn credit, participants must complete the activity during the valid credit period that is noted on the title page.

Follow these steps to earn credit:

• Log on to www.blackwellpublishing.com/cme.

• Read the target audience, learning objectives, and author disclosures.

• Read the article in print or online format.

• Reflect on the article.

• Access the CME Exam, and choose the best answer to each question.

• Complete the required evaluation component of the activity.

The period following discharge is a vulnerable time for patientsthe prevalence of medical errors related to this transition is high and has important patient safety and medico‐legal ramifications.13 Factors contributing to this vulnerability include complexity of hospitalized patients, shorter lengths of stay, and increased discontinuity of care. Hospitalists have recognized this threat to patient safety and have worked toward improving information exchange between inpatient and outpatient providers at hospital discharge.46 Nonetheless, the evidence suggests that more work is necessary. A recent study found that discharge summaries are often incomplete, and do not contain important information requiring follow‐up, such as pending tests.7 Additionally, a review by Kripalani et al. characterizing information deficits at hospital discharge found few interventions which specifically improve communication of pending tests at hospital discharge.8

In a prior study we determined that 41% of patients left the hospital before all laboratory and radiology test results were finalized. Of these results, 9.4% were potentially actionable and could have altered management. Physicians were aware of only 38% of post‐discharge test results.9 This awareness gap is a consequence of several factors including the lack of systems to track and alert providers of test results finalized post discharge. Also, it is unclear who is responsible for pending tests at discharge, since these tests are ordered by the inpatient physicians but often reported in the time period between hospital discharge and the patient's first follow‐up appointment with the primary care physician (PCP). Because responsibility is not explicitly made in the final communication between physicians at discharge, such test results may not be reviewed in a timely manner, potentially resulting in delays in treatment, a need for readmission, or other unfavorable outcomes.

Even in integrated health systems with advanced electronic health records, missed test results which result in treatment delays remain prevalent.10, 11 Test result management applications aid clinicians in reviewing and acting upon results as they become available and such systems may provide solutions to this problem. At Partners Healthcare in Boston, the Results Manager (RM) application was developed to help clinicians in the ambulatory setting safely, reliably, and efficiently review and act upon test results. The application enables clinicians to prioritize test results, utilize guidelines, and generate letters to patients. This system also prompts physicians to set reminders for future testing.12 In a 2.5‐year study evaluating the impact of this intervention, PCPs at 26 adult primary care practices were able to expedite communication of outpatient laboratory and imaging test results to patients with the help of RM. Patients of physicians who participated in the project reported greater satisfaction with test result communication and with information provided about their condition than did a control group of similar patients.13 RM has not yet been studied in the inpatient setting or at care transitions. We describe an attempt at modifying the Partners RM application to help inpatient physicians manage pending tests at hospital discharge.

Methods

Test Result Management Application

RM was originally developed by Partners Healthcare to improve timely review and appropriate management of test results in the ambulatory setting. RM was developed for and vetted by primarily ambulatory physicians. The application is browser‐based, provider‐centric, and embedded in the LMR to help ambulatory clinicians review and act upon test results in a safe, reliable, and efficient manner. Although RM has access to all inpatient and outpatient data in the Partners Clinical Data Repository (CDR), given the volume of inpatient tests ordered, hospital‐based results are suppressed by default to limit inundating ambulatory clinicians' queues. Therefore, users of RM only receive results of laboratory and radiology tests ordered in the ambulatory setting. They can track these tests for specific patients for a designated period of time by placing the patient on a watch list. Finally, RM incorporates extensive decision support features to classify the degree of abnormality for each result, presents guidelines to help clinicians manage abnormal results, allows clinicians to generate result letters to patients using predefined, context‐sensitive templates, and prompts physicians to set reminders for future testing. Because RM was developed from the ambulatory perspective, there was limited input from hospitalist physicians with regard to inpatient workflow in the original design of the module.12 See Figure 1 for a screen shot of RM and a description of its features.

Figure 1

For purposes of this pilot, we modified RM to allow results of tests ordered in the inpatient setting to be available for viewing (Hospitalist Results Manager, HRM). This feature was turned on only for inpatient physicians as previously defined. Inpatient tests, including pending tests at discharge, continued to be suppressed from PCP's RM queue (however, any physician could access a patient's test result(s) directly from the Partners CDR). Inpatient physicians could track laboratory and radiology results finalized after discharge by keeping discharged patients on their HRM watch list for a designated period of time. The finalized results would become available for review in their HRM queue and abnormal results were displayed prominently at the top of this queue. Inpatient physicians were trained to use HRM in a series of meetings and demonstrations. Although HRM could be accessed from inpatient clinical workstations, it was not part of the inpatient clinical information system.

Results

A total of 35 inpatient physicians participated in the pilot. Among 649 patients discharged during the study period, there were 1075 tests pending of which 555 were subsequently flagged as abnormal in HRM. Study surveys were sent to the 35 inpatient physician participants and 29 were completed, including partial responses (83% survey response rate). The 35 inpatient physician participants had the following characteristics: 22 were male, 13 were female; 21 were trainees and 14 were nontrainees/faculty. All 21 trainees were PGY2s. Nontrainees and faculty varied in experience level (PGY 15: 5, PGY 610: 7, PGY 1120: 1, PGY 21+: 1). Of 29 survey respondents, 7 were from hospital A and 22 were from hospital B; 19 were trainees and 10 were nontrainees/faculty. Of the 6 nonrespondents, 2 were from hospital A and 4 were from hospital B; 2 were trainees and 4 were nontrainees/faculty.

Table 1 shows the results of our survey of inpatient physicians regarding usage of HRM. Of 29 survey respondents, 14 (48%) reported never using HRM. Thirteen (45%) reported using HRM 1 to 2 times per week. None of the respondents used it more than 4 times per week. The frequency of usage was similar for hospitals A and B. Table 2 details barriers to using HRM. Twenty‐three inpatient physicians (79%) reported barriers. Seventeen (59%) thought that results in their HRM queue were not clinically relevant, 16 (55%) felt that HRM did not fit into their daily workflow, 14 (48%) had limited time to use HRM, and 12 (41%) noted that too many results in their HRM queue were on other physician's patients. Seven (24%) reported operational issues and 3 (10%) reported technical issues prohibiting use of HRM. With regard to preferred elements of an ideal results manager system, 21 (72%) inpatient physician respondents wanted to receive notification of abnormal and clinician‐designated pending test results. Four (14%) wanted to receive only abnormal results and 1 (3%) wanted to receive all results. Twenty‐seven (93%) physicians agreed that an ideally designed computerized test result management application would be valuable for managing pending tests at discharge.

Table 3 provides comments from inpatient physician respondents regarding obstacles prohibiting use of HRM and suggestions for future systems.

Discussion

We describe a pilot implementation of a computerized application for the management of pending tests at hospital discharge. From responses to post‐implementation surveys, we were able to identify multiple factors prohibiting successful implementation of the application. These observations may help inform future interventions and evaluations.

Almost half of inpatient physicians reported never using HRM despite training and reminders. The feedback provided by physicians in our study suggested that HRM was not ideally designed from an inpatient physician perspective. We discovered several barriers to its use: (1) HRM overburdened physicians with clinically irrelevant test results, suggesting that more robust filtering of abnormal but low importance test results may be required (eg, a borderline electrolyte abnormality or low but stable hematocrit); (2) HRM did not integrate well into inpatient workflowthe system was not integrated into the inpatient results viewing and computerized physician order entry (CPOE) applications, and therefore required an extra step to access; (3) there was no mechanism of alerting inpatient physicians that finalized test results were available for viewing in their HRM queues (eg, by email or by an alert in the inpatient computer system); (4) because responsibility for these results was unclear, most inpatient physicians had no formal method of managing them, and for many, using HRM represented an additional task; and finally (5) several physicians commented on finding results in their HRM queue that belonged to other physician's patients, implying that the hospital databases were inaccurate in identifying the discharging physician or that rotation schedules, and therefore patient responsibility, had changed in the intervening period. Table 4 summarizes the advantages and respective limitations of features of HRM available to inpatient physicians.

In the literature, there is little information regarding optimal features of a test result management system for transitions from the inpatient to ambulatory care setting. Prior studies outline important functions for results management systems developed for noninpatient sites of care, including the ambulatory and emergency room setting.12, 14, 15 These include a method of prioritizing by degree of abnormality, the ability to reliably and efficiently act upon results, and an automated alerting system for abnormal results. Findings from our study provide insight in defining core functions for result management systems which focus on transitions from the inpatient to ambulatory care setting. These functions include tight integration with applications used by inpatient physicians, clear assignment of responsibility for test results finalized after hospital discharge (as well as a mechanism to reassign responsibility), automated alerts to responsible providers of test results finalized post‐discharge, and ways to automatically filter test results to avoid over‐burdening physicians with clinically irrelevant results.

Almost all surveyed inpatient physicians agreed that an ideally designed electronic post‐discharge results management system would be valuable. For such systems to be successfully adopted, we offer several principles to help guide future work. These include: (1) clarifying responsibility at the time a test is ordered and again at discharge, (2) understanding workflow and communication patterns among inpatient and outpatient clinicians, and (3) integrating technological solutions into existing systems to minimize workflow disruptions. For example, if the primary responsibility for post‐discharge result follow‐up lies with the ordering physician, the system should be integrated within the EMR most often used by inpatient physicians and become part of inpatient physician workflow. If the system depends on administrative databases to identify the responsible providers, these must be accurate. Alternatively, in organizations with computerized provider order entry, responsibility for the result could be assigned when the test is ordered and confirmed at discharge (ie, the results management system would be integrated into the discharge order such that pending tests are reviewed at the time of discharge). The discharging physician should have the ability to assign responsibility for each pending test and select preferred mode(s) of notification once its result is finalized (eg, e‐mail, alphanumeric page, etc.). The system should have the ability to generate an automatic notification to the inpatient and PCP (and perhaps other designated providers involved in the patient's inpatient care), but it should not burden busy clinicians with unnecessary alerts and warnings. Finally, the rules by which results are prioritized must be robust enough to filter out less urgent results, and should be modified to reflect the severity of illness of recently discharged patients. In essence, in consideration of the time constraints of busy clinicians, an ideal results management system should achieve automated notification of test results while minimizing the risk of alert fatigue from the potentially large volume of alerts generated.

Our study has several important limitations. First, although our survey response rate was high, the sample size of actual participants was small. Second, because the study was conducted in 2 similar, tertiary care academic centers, it may not generalize to other settings (we note that hospital B included a nonteaching service similar to those in nonacademic medical centers). This may be particularly true in assessing the importance of specific barriers to use of results management systems, which may vary at different institutions. Third, the representation of survey respondents were skeweda majority of the responses were from trainees (all post‐graduate level [PGY] level 2) and from hospital B. Fourth, we did not actively monitor physician interaction with the test result management application, and therefore, we depended heavily on physician recollection of use of the system when responding to surveys. Finally, we did not convene focus groups of key individuals with regard to the factors facilitating or prohibiting adoption of the system. Use of semi‐structured, key informant interviews (ie, focus groups) before and after implementation of an electronic results management application, have been shown to be effective in evaluating potential barriers and facilitators of adoption.16 Focus groups of and/or interviews with inpatient and PCPs, physician extenders, and housestaff could have been useful to better characterize the potential barriers and facilitators of adoption noted by survey respondents in our study.

In summary, we offer several lessons from our attempt to implement a system to manage pending tests at hospital discharge. The success of implementing future systems to address this patient safety concern will rely on accurately assigning responsibility for these test results, integrating the system within clinical information systems commonly used by the inpatient physician, addressing workflow issues and time constraints, maximizing appropriateness of alerting, and minimizing alert fatigue.

The period following discharge is a vulnerable time for patientsthe prevalence of medical errors related to this transition is high and has important patient safety and medico‐legal ramifications.13 Factors contributing to this vulnerability include complexity of hospitalized patients, shorter lengths of stay, and increased discontinuity of care. Hospitalists have recognized this threat to patient safety and have worked toward improving information exchange between inpatient and outpatient providers at hospital discharge.46 Nonetheless, the evidence suggests that more work is necessary. A recent study found that discharge summaries are often incomplete, and do not contain important information requiring follow‐up, such as pending tests.7 Additionally, a review by Kripalani et al. characterizing information deficits at hospital discharge found few interventions which specifically improve communication of pending tests at hospital discharge.8

In a prior study we determined that 41% of patients left the hospital before all laboratory and radiology test results were finalized. Of these results, 9.4% were potentially actionable and could have altered management. Physicians were aware of only 38% of post‐discharge test results.9 This awareness gap is a consequence of several factors including the lack of systems to track and alert providers of test results finalized post discharge. Also, it is unclear who is responsible for pending tests at discharge, since these tests are ordered by the inpatient physicians but often reported in the time period between hospital discharge and the patient's first follow‐up appointment with the primary care physician (PCP). Because responsibility is not explicitly made in the final communication between physicians at discharge, such test results may not be reviewed in a timely manner, potentially resulting in delays in treatment, a need for readmission, or other unfavorable outcomes.

Even in integrated health systems with advanced electronic health records, missed test results which result in treatment delays remain prevalent.10, 11 Test result management applications aid clinicians in reviewing and acting upon results as they become available and such systems may provide solutions to this problem. At Partners Healthcare in Boston, the Results Manager (RM) application was developed to help clinicians in the ambulatory setting safely, reliably, and efficiently review and act upon test results. The application enables clinicians to prioritize test results, utilize guidelines, and generate letters to patients. This system also prompts physicians to set reminders for future testing.12 In a 2.5‐year study evaluating the impact of this intervention, PCPs at 26 adult primary care practices were able to expedite communication of outpatient laboratory and imaging test results to patients with the help of RM. Patients of physicians who participated in the project reported greater satisfaction with test result communication and with information provided about their condition than did a control group of similar patients.13 RM has not yet been studied in the inpatient setting or at care transitions. We describe an attempt at modifying the Partners RM application to help inpatient physicians manage pending tests at hospital discharge.

Methods

Test Result Management Application

RM was originally developed by Partners Healthcare to improve timely review and appropriate management of test results in the ambulatory setting. RM was developed for and vetted by primarily ambulatory physicians. The application is browser‐based, provider‐centric, and embedded in the LMR to help ambulatory clinicians review and act upon test results in a safe, reliable, and efficient manner. Although RM has access to all inpatient and outpatient data in the Partners Clinical Data Repository (CDR), given the volume of inpatient tests ordered, hospital‐based results are suppressed by default to limit inundating ambulatory clinicians' queues. Therefore, users of RM only receive results of laboratory and radiology tests ordered in the ambulatory setting. They can track these tests for specific patients for a designated period of time by placing the patient on a watch list. Finally, RM incorporates extensive decision support features to classify the degree of abnormality for each result, presents guidelines to help clinicians manage abnormal results, allows clinicians to generate result letters to patients using predefined, context‐sensitive templates, and prompts physicians to set reminders for future testing. Because RM was developed from the ambulatory perspective, there was limited input from hospitalist physicians with regard to inpatient workflow in the original design of the module.12 See Figure 1 for a screen shot of RM and a description of its features.

Figure 1

For purposes of this pilot, we modified RM to allow results of tests ordered in the inpatient setting to be available for viewing (Hospitalist Results Manager, HRM). This feature was turned on only for inpatient physicians as previously defined. Inpatient tests, including pending tests at discharge, continued to be suppressed from PCP's RM queue (however, any physician could access a patient's test result(s) directly from the Partners CDR). Inpatient physicians could track laboratory and radiology results finalized after discharge by keeping discharged patients on their HRM watch list for a designated period of time. The finalized results would become available for review in their HRM queue and abnormal results were displayed prominently at the top of this queue. Inpatient physicians were trained to use HRM in a series of meetings and demonstrations. Although HRM could be accessed from inpatient clinical workstations, it was not part of the inpatient clinical information system.

Results

A total of 35 inpatient physicians participated in the pilot. Among 649 patients discharged during the study period, there were 1075 tests pending of which 555 were subsequently flagged as abnormal in HRM. Study surveys were sent to the 35 inpatient physician participants and 29 were completed, including partial responses (83% survey response rate). The 35 inpatient physician participants had the following characteristics: 22 were male, 13 were female; 21 were trainees and 14 were nontrainees/faculty. All 21 trainees were PGY2s. Nontrainees and faculty varied in experience level (PGY 15: 5, PGY 610: 7, PGY 1120: 1, PGY 21+: 1). Of 29 survey respondents, 7 were from hospital A and 22 were from hospital B; 19 were trainees and 10 were nontrainees/faculty. Of the 6 nonrespondents, 2 were from hospital A and 4 were from hospital B; 2 were trainees and 4 were nontrainees/faculty.

Table 1 shows the results of our survey of inpatient physicians regarding usage of HRM. Of 29 survey respondents, 14 (48%) reported never using HRM. Thirteen (45%) reported using HRM 1 to 2 times per week. None of the respondents used it more than 4 times per week. The frequency of usage was similar for hospitals A and B. Table 2 details barriers to using HRM. Twenty‐three inpatient physicians (79%) reported barriers. Seventeen (59%) thought that results in their HRM queue were not clinically relevant, 16 (55%) felt that HRM did not fit into their daily workflow, 14 (48%) had limited time to use HRM, and 12 (41%) noted that too many results in their HRM queue were on other physician's patients. Seven (24%) reported operational issues and 3 (10%) reported technical issues prohibiting use of HRM. With regard to preferred elements of an ideal results manager system, 21 (72%) inpatient physician respondents wanted to receive notification of abnormal and clinician‐designated pending test results. Four (14%) wanted to receive only abnormal results and 1 (3%) wanted to receive all results. Twenty‐seven (93%) physicians agreed that an ideally designed computerized test result management application would be valuable for managing pending tests at discharge.

Table 3 provides comments from inpatient physician respondents regarding obstacles prohibiting use of HRM and suggestions for future systems.

Discussion

We describe a pilot implementation of a computerized application for the management of pending tests at hospital discharge. From responses to post‐implementation surveys, we were able to identify multiple factors prohibiting successful implementation of the application. These observations may help inform future interventions and evaluations.

Almost half of inpatient physicians reported never using HRM despite training and reminders. The feedback provided by physicians in our study suggested that HRM was not ideally designed from an inpatient physician perspective. We discovered several barriers to its use: (1) HRM overburdened physicians with clinically irrelevant test results, suggesting that more robust filtering of abnormal but low importance test results may be required (eg, a borderline electrolyte abnormality or low but stable hematocrit); (2) HRM did not integrate well into inpatient workflowthe system was not integrated into the inpatient results viewing and computerized physician order entry (CPOE) applications, and therefore required an extra step to access; (3) there was no mechanism of alerting inpatient physicians that finalized test results were available for viewing in their HRM queues (eg, by email or by an alert in the inpatient computer system); (4) because responsibility for these results was unclear, most inpatient physicians had no formal method of managing them, and for many, using HRM represented an additional task; and finally (5) several physicians commented on finding results in their HRM queue that belonged to other physician's patients, implying that the hospital databases were inaccurate in identifying the discharging physician or that rotation schedules, and therefore patient responsibility, had changed in the intervening period. Table 4 summarizes the advantages and respective limitations of features of HRM available to inpatient physicians.

In the literature, there is little information regarding optimal features of a test result management system for transitions from the inpatient to ambulatory care setting. Prior studies outline important functions for results management systems developed for noninpatient sites of care, including the ambulatory and emergency room setting.12, 14, 15 These include a method of prioritizing by degree of abnormality, the ability to reliably and efficiently act upon results, and an automated alerting system for abnormal results. Findings from our study provide insight in defining core functions for result management systems which focus on transitions from the inpatient to ambulatory care setting. These functions include tight integration with applications used by inpatient physicians, clear assignment of responsibility for test results finalized after hospital discharge (as well as a mechanism to reassign responsibility), automated alerts to responsible providers of test results finalized post‐discharge, and ways to automatically filter test results to avoid over‐burdening physicians with clinically irrelevant results.

Almost all surveyed inpatient physicians agreed that an ideally designed electronic post‐discharge results management system would be valuable. For such systems to be successfully adopted, we offer several principles to help guide future work. These include: (1) clarifying responsibility at the time a test is ordered and again at discharge, (2) understanding workflow and communication patterns among inpatient and outpatient clinicians, and (3) integrating technological solutions into existing systems to minimize workflow disruptions. For example, if the primary responsibility for post‐discharge result follow‐up lies with the ordering physician, the system should be integrated within the EMR most often used by inpatient physicians and become part of inpatient physician workflow. If the system depends on administrative databases to identify the responsible providers, these must be accurate. Alternatively, in organizations with computerized provider order entry, responsibility for the result could be assigned when the test is ordered and confirmed at discharge (ie, the results management system would be integrated into the discharge order such that pending tests are reviewed at the time of discharge). The discharging physician should have the ability to assign responsibility for each pending test and select preferred mode(s) of notification once its result is finalized (eg, e‐mail, alphanumeric page, etc.). The system should have the ability to generate an automatic notification to the inpatient and PCP (and perhaps other designated providers involved in the patient's inpatient care), but it should not burden busy clinicians with unnecessary alerts and warnings. Finally, the rules by which results are prioritized must be robust enough to filter out less urgent results, and should be modified to reflect the severity of illness of recently discharged patients. In essence, in consideration of the time constraints of busy clinicians, an ideal results management system should achieve automated notification of test results while minimizing the risk of alert fatigue from the potentially large volume of alerts generated.

Our study has several important limitations. First, although our survey response rate was high, the sample size of actual participants was small. Second, because the study was conducted in 2 similar, tertiary care academic centers, it may not generalize to other settings (we note that hospital B included a nonteaching service similar to those in nonacademic medical centers). This may be particularly true in assessing the importance of specific barriers to use of results management systems, which may vary at different institutions. Third, the representation of survey respondents were skeweda majority of the responses were from trainees (all post‐graduate level [PGY] level 2) and from hospital B. Fourth, we did not actively monitor physician interaction with the test result management application, and therefore, we depended heavily on physician recollection of use of the system when responding to surveys. Finally, we did not convene focus groups of key individuals with regard to the factors facilitating or prohibiting adoption of the system. Use of semi‐structured, key informant interviews (ie, focus groups) before and after implementation of an electronic results management application, have been shown to be effective in evaluating potential barriers and facilitators of adoption.16 Focus groups of and/or interviews with inpatient and PCPs, physician extenders, and housestaff could have been useful to better characterize the potential barriers and facilitators of adoption noted by survey respondents in our study.

In summary, we offer several lessons from our attempt to implement a system to manage pending tests at hospital discharge. The success of implementing future systems to address this patient safety concern will rely on accurately assigning responsibility for these test results, integrating the system within clinical information systems commonly used by the inpatient physician, addressing workflow issues and time constraints, maximizing appropriateness of alerting, and minimizing alert fatigue.

The period following discharge is a vulnerable time for patientsthe prevalence of medical errors related to this transition is high and has important patient safety and medico‐legal ramifications.13 Factors contributing to this vulnerability include complexity of hospitalized patients, shorter lengths of stay, and increased discontinuity of care. Hospitalists have recognized this threat to patient safety and have worked toward improving information exchange between inpatient and outpatient providers at hospital discharge.46 Nonetheless, the evidence suggests that more work is necessary. A recent study found that discharge summaries are often incomplete, and do not contain important information requiring follow‐up, such as pending tests.7 Additionally, a review by Kripalani et al. characterizing information deficits at hospital discharge found few interventions which specifically improve communication of pending tests at hospital discharge.8

In a prior study we determined that 41% of patients left the hospital before all laboratory and radiology test results were finalized. Of these results, 9.4% were potentially actionable and could have altered management. Physicians were aware of only 38% of post‐discharge test results.9 This awareness gap is a consequence of several factors including the lack of systems to track and alert providers of test results finalized post discharge. Also, it is unclear who is responsible for pending tests at discharge, since these tests are ordered by the inpatient physicians but often reported in the time period between hospital discharge and the patient's first follow‐up appointment with the primary care physician (PCP). Because responsibility is not explicitly made in the final communication between physicians at discharge, such test results may not be reviewed in a timely manner, potentially resulting in delays in treatment, a need for readmission, or other unfavorable outcomes.

Even in integrated health systems with advanced electronic health records, missed test results which result in treatment delays remain prevalent.10, 11 Test result management applications aid clinicians in reviewing and acting upon results as they become available and such systems may provide solutions to this problem. At Partners Healthcare in Boston, the Results Manager (RM) application was developed to help clinicians in the ambulatory setting safely, reliably, and efficiently review and act upon test results. The application enables clinicians to prioritize test results, utilize guidelines, and generate letters to patients. This system also prompts physicians to set reminders for future testing.12 In a 2.5‐year study evaluating the impact of this intervention, PCPs at 26 adult primary care practices were able to expedite communication of outpatient laboratory and imaging test results to patients with the help of RM. Patients of physicians who participated in the project reported greater satisfaction with test result communication and with information provided about their condition than did a control group of similar patients.13 RM has not yet been studied in the inpatient setting or at care transitions. We describe an attempt at modifying the Partners RM application to help inpatient physicians manage pending tests at hospital discharge.

Methods

Test Result Management Application

RM was originally developed by Partners Healthcare to improve timely review and appropriate management of test results in the ambulatory setting. RM was developed for and vetted by primarily ambulatory physicians. The application is browser‐based, provider‐centric, and embedded in the LMR to help ambulatory clinicians review and act upon test results in a safe, reliable, and efficient manner. Although RM has access to all inpatient and outpatient data in the Partners Clinical Data Repository (CDR), given the volume of inpatient tests ordered, hospital‐based results are suppressed by default to limit inundating ambulatory clinicians' queues. Therefore, users of RM only receive results of laboratory and radiology tests ordered in the ambulatory setting. They can track these tests for specific patients for a designated period of time by placing the patient on a watch list. Finally, RM incorporates extensive decision support features to classify the degree of abnormality for each result, presents guidelines to help clinicians manage abnormal results, allows clinicians to generate result letters to patients using predefined, context‐sensitive templates, and prompts physicians to set reminders for future testing. Because RM was developed from the ambulatory perspective, there was limited input from hospitalist physicians with regard to inpatient workflow in the original design of the module.12 See Figure 1 for a screen shot of RM and a description of its features.

Figure 1

For purposes of this pilot, we modified RM to allow results of tests ordered in the inpatient setting to be available for viewing (Hospitalist Results Manager, HRM). This feature was turned on only for inpatient physicians as previously defined. Inpatient tests, including pending tests at discharge, continued to be suppressed from PCP's RM queue (however, any physician could access a patient's test result(s) directly from the Partners CDR). Inpatient physicians could track laboratory and radiology results finalized after discharge by keeping discharged patients on their HRM watch list for a designated period of time. The finalized results would become available for review in their HRM queue and abnormal results were displayed prominently at the top of this queue. Inpatient physicians were trained to use HRM in a series of meetings and demonstrations. Although HRM could be accessed from inpatient clinical workstations, it was not part of the inpatient clinical information system.

Results

A total of 35 inpatient physicians participated in the pilot. Among 649 patients discharged during the study period, there were 1075 tests pending of which 555 were subsequently flagged as abnormal in HRM. Study surveys were sent to the 35 inpatient physician participants and 29 were completed, including partial responses (83% survey response rate). The 35 inpatient physician participants had the following characteristics: 22 were male, 13 were female; 21 were trainees and 14 were nontrainees/faculty. All 21 trainees were PGY2s. Nontrainees and faculty varied in experience level (PGY 15: 5, PGY 610: 7, PGY 1120: 1, PGY 21+: 1). Of 29 survey respondents, 7 were from hospital A and 22 were from hospital B; 19 were trainees and 10 were nontrainees/faculty. Of the 6 nonrespondents, 2 were from hospital A and 4 were from hospital B; 2 were trainees and 4 were nontrainees/faculty.

Table 1 shows the results of our survey of inpatient physicians regarding usage of HRM. Of 29 survey respondents, 14 (48%) reported never using HRM. Thirteen (45%) reported using HRM 1 to 2 times per week. None of the respondents used it more than 4 times per week. The frequency of usage was similar for hospitals A and B. Table 2 details barriers to using HRM. Twenty‐three inpatient physicians (79%) reported barriers. Seventeen (59%) thought that results in their HRM queue were not clinically relevant, 16 (55%) felt that HRM did not fit into their daily workflow, 14 (48%) had limited time to use HRM, and 12 (41%) noted that too many results in their HRM queue were on other physician's patients. Seven (24%) reported operational issues and 3 (10%) reported technical issues prohibiting use of HRM. With regard to preferred elements of an ideal results manager system, 21 (72%) inpatient physician respondents wanted to receive notification of abnormal and clinician‐designated pending test results. Four (14%) wanted to receive only abnormal results and 1 (3%) wanted to receive all results. Twenty‐seven (93%) physicians agreed that an ideally designed computerized test result management application would be valuable for managing pending tests at discharge.

Table 3 provides comments from inpatient physician respondents regarding obstacles prohibiting use of HRM and suggestions for future systems.

Discussion

We describe a pilot implementation of a computerized application for the management of pending tests at hospital discharge. From responses to post‐implementation surveys, we were able to identify multiple factors prohibiting successful implementation of the application. These observations may help inform future interventions and evaluations.

Almost half of inpatient physicians reported never using HRM despite training and reminders. The feedback provided by physicians in our study suggested that HRM was not ideally designed from an inpatient physician perspective. We discovered several barriers to its use: (1) HRM overburdened physicians with clinically irrelevant test results, suggesting that more robust filtering of abnormal but low importance test results may be required (eg, a borderline electrolyte abnormality or low but stable hematocrit); (2) HRM did not integrate well into inpatient workflowthe system was not integrated into the inpatient results viewing and computerized physician order entry (CPOE) applications, and therefore required an extra step to access; (3) there was no mechanism of alerting inpatient physicians that finalized test results were available for viewing in their HRM queues (eg, by email or by an alert in the inpatient computer system); (4) because responsibility for these results was unclear, most inpatient physicians had no formal method of managing them, and for many, using HRM represented an additional task; and finally (5) several physicians commented on finding results in their HRM queue that belonged to other physician's patients, implying that the hospital databases were inaccurate in identifying the discharging physician or that rotation schedules, and therefore patient responsibility, had changed in the intervening period. Table 4 summarizes the advantages and respective limitations of features of HRM available to inpatient physicians.

In the literature, there is little information regarding optimal features of a test result management system for transitions from the inpatient to ambulatory care setting. Prior studies outline important functions for results management systems developed for noninpatient sites of care, including the ambulatory and emergency room setting.12, 14, 15 These include a method of prioritizing by degree of abnormality, the ability to reliably and efficiently act upon results, and an automated alerting system for abnormal results. Findings from our study provide insight in defining core functions for result management systems which focus on transitions from the inpatient to ambulatory care setting. These functions include tight integration with applications used by inpatient physicians, clear assignment of responsibility for test results finalized after hospital discharge (as well as a mechanism to reassign responsibility), automated alerts to responsible providers of test results finalized post‐discharge, and ways to automatically filter test results to avoid over‐burdening physicians with clinically irrelevant results.

Almost all surveyed inpatient physicians agreed that an ideally designed electronic post‐discharge results management system would be valuable. For such systems to be successfully adopted, we offer several principles to help guide future work. These include: (1) clarifying responsibility at the time a test is ordered and again at discharge, (2) understanding workflow and communication patterns among inpatient and outpatient clinicians, and (3) integrating technological solutions into existing systems to minimize workflow disruptions. For example, if the primary responsibility for post‐discharge result follow‐up lies with the ordering physician, the system should be integrated within the EMR most often used by inpatient physicians and become part of inpatient physician workflow. If the system depends on administrative databases to identify the responsible providers, these must be accurate. Alternatively, in organizations with computerized provider order entry, responsibility for the result could be assigned when the test is ordered and confirmed at discharge (ie, the results management system would be integrated into the discharge order such that pending tests are reviewed at the time of discharge). The discharging physician should have the ability to assign responsibility for each pending test and select preferred mode(s) of notification once its result is finalized (eg, e‐mail, alphanumeric page, etc.). The system should have the ability to generate an automatic notification to the inpatient and PCP (and perhaps other designated providers involved in the patient's inpatient care), but it should not burden busy clinicians with unnecessary alerts and warnings. Finally, the rules by which results are prioritized must be robust enough to filter out less urgent results, and should be modified to reflect the severity of illness of recently discharged patients. In essence, in consideration of the time constraints of busy clinicians, an ideal results management system should achieve automated notification of test results while minimizing the risk of alert fatigue from the potentially large volume of alerts generated.

Our study has several important limitations. First, although our survey response rate was high, the sample size of actual participants was small. Second, because the study was conducted in 2 similar, tertiary care academic centers, it may not generalize to other settings (we note that hospital B included a nonteaching service similar to those in nonacademic medical centers). This may be particularly true in assessing the importance of specific barriers to use of results management systems, which may vary at different institutions. Third, the representation of survey respondents were skeweda majority of the responses were from trainees (all post‐graduate level [PGY] level 2) and from hospital B. Fourth, we did not actively monitor physician interaction with the test result management application, and therefore, we depended heavily on physician recollection of use of the system when responding to surveys. Finally, we did not convene focus groups of key individuals with regard to the factors facilitating or prohibiting adoption of the system. Use of semi‐structured, key informant interviews (ie, focus groups) before and after implementation of an electronic results management application, have been shown to be effective in evaluating potential barriers and facilitators of adoption.16 Focus groups of and/or interviews with inpatient and PCPs, physician extenders, and housestaff could have been useful to better characterize the potential barriers and facilitators of adoption noted by survey respondents in our study.

In summary, we offer several lessons from our attempt to implement a system to manage pending tests at hospital discharge. The success of implementing future systems to address this patient safety concern will rely on accurately assigning responsibility for these test results, integrating the system within clinical information systems commonly used by the inpatient physician, addressing workflow issues and time constraints, maximizing appropriateness of alerting, and minimizing alert fatigue.

Self‐check‐in kiosks started to appear in airports in the late 1990s, and within a few years, they seem to have become ubiquitous in the airline industry. Today, almost 70% of business travelers use them, and other sectors of the travel industry are beginning to experiment with the technology.1 Compared to this innovation in the airline industry, adoption of computerized physician order entry (CPOE) in U.S. hospitals, first pioneered in the early 1970s,2, 3 has taken a much more leisurely pace. Despite numerous studies documenting its benefits,47 promotion by prominent national patient safety advocacy groups such as LeapFrog,8 and numerous guides on best adoption practices.912 fewer than 10% of U.S. hospitals have fully adopted this technology.13 Moreover, as Lindenauer et al.14 pointed out, most hospitals that have successfully implemented CPOE are academic medical centers that rely on house staff to enter orders. With notable exceptions,3 adoption of CPOE in community hospitals where attending physicians write most orders remains anemic.

Although an increasing number of scholarly articles has documented the reasons for this slow rate of adoption even in hospitals that have the resources to invest in this technology, much of that research is based on expert opinion and case studies.11, 1519 In this context, Lindenauer et al.14 should be commended for using empirical evidence to delineate the predictors of adoption. Lindenauer et al. found that physicians who trained in hospitals with CPOE were more likely to be frequent users of CPOE in their new environment. Although the analysis did not account for possible confounding such as employment status of the physician, this result does confirm the conventional wisdom that physicians‐in‐training are more malleable and that residency is an important opportunity to expose physicians to safety technologies. If this finding is borne out by further research, it would bode well for the adoption of CPOE, as many physicians are trained in academic institutions, which are more likely to have CPOE,20 and almost all physicians spend part of their training in a VA hospital, which has uniformly adopted CPOE. Similarly, Lindenauer et al. found that physicians who use computers for personal purposes are more likely to be frequent users of CPOE. Given the increasingly ubiquitous use of computers in all spheres of life, time is on the side of increasing acceptance of CPOE.

However, a closer examination of the data presented by Lindenauer et al. raises several concerns. First, the substantial number of infrequent users across all demographic subgroups and clinical disciplines, even among users who were exposed to CPOE during training or those who used computers regularly for personal purposes, highlights the absence of shortcuts to the universal acceptance of CPOE. Second, whereas 63% of surveyed physicians believed that CPOE would reduce the incidence of medication errors and 71% believed that CPOE would prevent aspects of care from slipping through the cracks, only 42% of the surveyed physicians were frequent users of CPOE. This implies that even when physicians believe in the safety and quality benefits of CPOE, that belief alone may not be sufficient to convince all of them to adopt this technology wholeheartedly; other factors such as speed, ease of use, and training are likely important prerequisites. Third, although 66% of orders placed in person at the 2 study hospitals were entered through CPOE, acceptance of this technology, as measured by Lindenauer et al, was moderate at both institutions. This suggests that even when organizations have reached the 70% threshold set by Leapfrog as the proportion of orders placed in CPOE that qualifies as full implementation, they may continue to face resistance to full acceptance of the technology.

Compared to their academic counterparts, community hospitals face additional hurdles as they implement CPOE. Not only does their smaller size make it difficult to achieve economies of scale, they are also at a disadvantage because of the relationship the community hospital has with its physicians. Unlike physicians‐in‐training in academic medical centers, physicians in community hospitals function as largely autonomous agents over whom the hospital administration has little control. Although these physicians and their hospitals share the common goals of patient safety and quality, the financial incentives for the adoption of CPOE are often misaligned. For example, a recent cost benefit analysis21 showed the enormous potential for hospitals to cut costs if physicians fully adopt a CPOE system with rich decision support features. However, those savings typically accrue to the hospital, not to the physicians who use the system. Assuming the typical learning curve that accompanies the use of any new technology, physicians in community hospitals may have little incentive to invest the time to learn to use the system efficiently.

So what can be done to overcome these seemingly formidable barriers to full adoption of CPOE? Emerging research, which has so far largely focused on CPOE implementation at academic hospitals, suggests there is no silver bullet. Instead, it has taught us how the complex interplay among vendor capability, organizational behavior, clinician work flow, and implementation strategy determines the success or failure of adoption.11, 17, 18, 22 Although physician characteristics will play a role in determining whether an individual adopts this technology, local factors such as the presence of champions, governance model for the project, support for staff throughout the process, and relationship between administration and physicians are likely important determinants of success at both academic and community hospitals. In addition, organizations that embark on CPOE implementation need to understand the enormity of the task at hand and must devote not only sufficient financial but also human capital over time.11, 18 In the words of a chief medical information officer, Implementing CPOE should not be thought of as an event, but a long‐term commitment.

Beyond following proposed best practices for the implementation of CPOE, community hospitals may need to adopt additional strategies to address their unique challenges. Given the misalignment of incentives for physicians' use of CPOE, leadership in community hospitals must be particularly skilled at articulating the benefits of CPOE to physicians. These benefits include not only decreased professional liability from improved patient safety and better quality of care, but also fewer pharmacy callbacks, remote access, and rapid ordering through order sets. Hospitals may also want to elicit support from physicians early by empowering them to create order sets for their disciplines. Mechanisms for hospitals and physicians to engage in mutual cost‐sharing arrangements may provide addition opportunities for hospitals to entice physicians to adopt the technology. Finally, and of particular interest to the readership of this journal, as hospitalists become more prevalent and take care of an increasing proportion of hospitalized patients,23 they are often ideal candidates to lead the implementation of CPOE in community hospitals. Because hospitalists spend most of their time in the hospital, they are often in the best position to get fully trained on CPOE, to define their own order sets, and to redesign care processes in order to take full advantage of CPOE capabilities. In addition, as many hospitalists are directly employed or supported by the hospital, their goals for quality, safety, and efficiency are usually better aligned with those of the hospital.

The stakes involved in implementing CPOE are high. Hospitals invest enormous sums of money in these systems, and many will not have the financial or political capital to attempt a second implementation after an initial failure. In addition, as recent research has pointed out,24 inappropriate implementation strategies may lead to delays in essential care and direct patient harm. In many ways, the complex task of implementing CPOE is not unlike other endeavors in patient care, where optimal outcomes require sound knowledge and reliable processes and where disaster can strike for lack of attention to detail or common sense. If Hippocrates were alive today, he might have this to say about CPOE implementation: Life is short, the art long, opportunity fleeting, experience treacherous, judgment difficult.

Self‐check‐in kiosks started to appear in airports in the late 1990s, and within a few years, they seem to have become ubiquitous in the airline industry. Today, almost 70% of business travelers use them, and other sectors of the travel industry are beginning to experiment with the technology.1 Compared to this innovation in the airline industry, adoption of computerized physician order entry (CPOE) in U.S. hospitals, first pioneered in the early 1970s,2, 3 has taken a much more leisurely pace. Despite numerous studies documenting its benefits,47 promotion by prominent national patient safety advocacy groups such as LeapFrog,8 and numerous guides on best adoption practices.912 fewer than 10% of U.S. hospitals have fully adopted this technology.13 Moreover, as Lindenauer et al.14 pointed out, most hospitals that have successfully implemented CPOE are academic medical centers that rely on house staff to enter orders. With notable exceptions,3 adoption of CPOE in community hospitals where attending physicians write most orders remains anemic.

Although an increasing number of scholarly articles has documented the reasons for this slow rate of adoption even in hospitals that have the resources to invest in this technology, much of that research is based on expert opinion and case studies.11, 1519 In this context, Lindenauer et al.14 should be commended for using empirical evidence to delineate the predictors of adoption. Lindenauer et al. found that physicians who trained in hospitals with CPOE were more likely to be frequent users of CPOE in their new environment. Although the analysis did not account for possible confounding such as employment status of the physician, this result does confirm the conventional wisdom that physicians‐in‐training are more malleable and that residency is an important opportunity to expose physicians to safety technologies. If this finding is borne out by further research, it would bode well for the adoption of CPOE, as many physicians are trained in academic institutions, which are more likely to have CPOE,20 and almost all physicians spend part of their training in a VA hospital, which has uniformly adopted CPOE. Similarly, Lindenauer et al. found that physicians who use computers for personal purposes are more likely to be frequent users of CPOE. Given the increasingly ubiquitous use of computers in all spheres of life, time is on the side of increasing acceptance of CPOE.

However, a closer examination of the data presented by Lindenauer et al. raises several concerns. First, the substantial number of infrequent users across all demographic subgroups and clinical disciplines, even among users who were exposed to CPOE during training or those who used computers regularly for personal purposes, highlights the absence of shortcuts to the universal acceptance of CPOE. Second, whereas 63% of surveyed physicians believed that CPOE would reduce the incidence of medication errors and 71% believed that CPOE would prevent aspects of care from slipping through the cracks, only 42% of the surveyed physicians were frequent users of CPOE. This implies that even when physicians believe in the safety and quality benefits of CPOE, that belief alone may not be sufficient to convince all of them to adopt this technology wholeheartedly; other factors such as speed, ease of use, and training are likely important prerequisites. Third, although 66% of orders placed in person at the 2 study hospitals were entered through CPOE, acceptance of this technology, as measured by Lindenauer et al, was moderate at both institutions. This suggests that even when organizations have reached the 70% threshold set by Leapfrog as the proportion of orders placed in CPOE that qualifies as full implementation, they may continue to face resistance to full acceptance of the technology.

Compared to their academic counterparts, community hospitals face additional hurdles as they implement CPOE. Not only does their smaller size make it difficult to achieve economies of scale, they are also at a disadvantage because of the relationship the community hospital has with its physicians. Unlike physicians‐in‐training in academic medical centers, physicians in community hospitals function as largely autonomous agents over whom the hospital administration has little control. Although these physicians and their hospitals share the common goals of patient safety and quality, the financial incentives for the adoption of CPOE are often misaligned. For example, a recent cost benefit analysis21 showed the enormous potential for hospitals to cut costs if physicians fully adopt a CPOE system with rich decision support features. However, those savings typically accrue to the hospital, not to the physicians who use the system. Assuming the typical learning curve that accompanies the use of any new technology, physicians in community hospitals may have little incentive to invest the time to learn to use the system efficiently.

So what can be done to overcome these seemingly formidable barriers to full adoption of CPOE? Emerging research, which has so far largely focused on CPOE implementation at academic hospitals, suggests there is no silver bullet. Instead, it has taught us how the complex interplay among vendor capability, organizational behavior, clinician work flow, and implementation strategy determines the success or failure of adoption.11, 17, 18, 22 Although physician characteristics will play a role in determining whether an individual adopts this technology, local factors such as the presence of champions, governance model for the project, support for staff throughout the process, and relationship between administration and physicians are likely important determinants of success at both academic and community hospitals. In addition, organizations that embark on CPOE implementation need to understand the enormity of the task at hand and must devote not only sufficient financial but also human capital over time.11, 18 In the words of a chief medical information officer, Implementing CPOE should not be thought of as an event, but a long‐term commitment.

Beyond following proposed best practices for the implementation of CPOE, community hospitals may need to adopt additional strategies to address their unique challenges. Given the misalignment of incentives for physicians' use of CPOE, leadership in community hospitals must be particularly skilled at articulating the benefits of CPOE to physicians. These benefits include not only decreased professional liability from improved patient safety and better quality of care, but also fewer pharmacy callbacks, remote access, and rapid ordering through order sets. Hospitals may also want to elicit support from physicians early by empowering them to create order sets for their disciplines. Mechanisms for hospitals and physicians to engage in mutual cost‐sharing arrangements may provide addition opportunities for hospitals to entice physicians to adopt the technology. Finally, and of particular interest to the readership of this journal, as hospitalists become more prevalent and take care of an increasing proportion of hospitalized patients,23 they are often ideal candidates to lead the implementation of CPOE in community hospitals. Because hospitalists spend most of their time in the hospital, they are often in the best position to get fully trained on CPOE, to define their own order sets, and to redesign care processes in order to take full advantage of CPOE capabilities. In addition, as many hospitalists are directly employed or supported by the hospital, their goals for quality, safety, and efficiency are usually better aligned with those of the hospital.

The stakes involved in implementing CPOE are high. Hospitals invest enormous sums of money in these systems, and many will not have the financial or political capital to attempt a second implementation after an initial failure. In addition, as recent research has pointed out,24 inappropriate implementation strategies may lead to delays in essential care and direct patient harm. In many ways, the complex task of implementing CPOE is not unlike other endeavors in patient care, where optimal outcomes require sound knowledge and reliable processes and where disaster can strike for lack of attention to detail or common sense. If Hippocrates were alive today, he might have this to say about CPOE implementation: Life is short, the art long, opportunity fleeting, experience treacherous, judgment difficult.

Self‐check‐in kiosks started to appear in airports in the late 1990s, and within a few years, they seem to have become ubiquitous in the airline industry. Today, almost 70% of business travelers use them, and other sectors of the travel industry are beginning to experiment with the technology.1 Compared to this innovation in the airline industry, adoption of computerized physician order entry (CPOE) in U.S. hospitals, first pioneered in the early 1970s,2, 3 has taken a much more leisurely pace. Despite numerous studies documenting its benefits,47 promotion by prominent national patient safety advocacy groups such as LeapFrog,8 and numerous guides on best adoption practices.912 fewer than 10% of U.S. hospitals have fully adopted this technology.13 Moreover, as Lindenauer et al.14 pointed out, most hospitals that have successfully implemented CPOE are academic medical centers that rely on house staff to enter orders. With notable exceptions,3 adoption of CPOE in community hospitals where attending physicians write most orders remains anemic.

Although an increasing number of scholarly articles has documented the reasons for this slow rate of adoption even in hospitals that have the resources to invest in this technology, much of that research is based on expert opinion and case studies.11, 1519 In this context, Lindenauer et al.14 should be commended for using empirical evidence to delineate the predictors of adoption. Lindenauer et al. found that physicians who trained in hospitals with CPOE were more likely to be frequent users of CPOE in their new environment. Although the analysis did not account for possible confounding such as employment status of the physician, this result does confirm the conventional wisdom that physicians‐in‐training are more malleable and that residency is an important opportunity to expose physicians to safety technologies. If this finding is borne out by further research, it would bode well for the adoption of CPOE, as many physicians are trained in academic institutions, which are more likely to have CPOE,20 and almost all physicians spend part of their training in a VA hospital, which has uniformly adopted CPOE. Similarly, Lindenauer et al. found that physicians who use computers for personal purposes are more likely to be frequent users of CPOE. Given the increasingly ubiquitous use of computers in all spheres of life, time is on the side of increasing acceptance of CPOE.

However, a closer examination of the data presented by Lindenauer et al. raises several concerns. First, the substantial number of infrequent users across all demographic subgroups and clinical disciplines, even among users who were exposed to CPOE during training or those who used computers regularly for personal purposes, highlights the absence of shortcuts to the universal acceptance of CPOE. Second, whereas 63% of surveyed physicians believed that CPOE would reduce the incidence of medication errors and 71% believed that CPOE would prevent aspects of care from slipping through the cracks, only 42% of the surveyed physicians were frequent users of CPOE. This implies that even when physicians believe in the safety and quality benefits of CPOE, that belief alone may not be sufficient to convince all of them to adopt this technology wholeheartedly; other factors such as speed, ease of use, and training are likely important prerequisites. Third, although 66% of orders placed in person at the 2 study hospitals were entered through CPOE, acceptance of this technology, as measured by Lindenauer et al, was moderate at both institutions. This suggests that even when organizations have reached the 70% threshold set by Leapfrog as the proportion of orders placed in CPOE that qualifies as full implementation, they may continue to face resistance to full acceptance of the technology.

Compared to their academic counterparts, community hospitals face additional hurdles as they implement CPOE. Not only does their smaller size make it difficult to achieve economies of scale, they are also at a disadvantage because of the relationship the community hospital has with its physicians. Unlike physicians‐in‐training in academic medical centers, physicians in community hospitals function as largely autonomous agents over whom the hospital administration has little control. Although these physicians and their hospitals share the common goals of patient safety and quality, the financial incentives for the adoption of CPOE are often misaligned. For example, a recent cost benefit analysis21 showed the enormous potential for hospitals to cut costs if physicians fully adopt a CPOE system with rich decision support features. However, those savings typically accrue to the hospital, not to the physicians who use the system. Assuming the typical learning curve that accompanies the use of any new technology, physicians in community hospitals may have little incentive to invest the time to learn to use the system efficiently.

So what can be done to overcome these seemingly formidable barriers to full adoption of CPOE? Emerging research, which has so far largely focused on CPOE implementation at academic hospitals, suggests there is no silver bullet. Instead, it has taught us how the complex interplay among vendor capability, organizational behavior, clinician work flow, and implementation strategy determines the success or failure of adoption.11, 17, 18, 22 Although physician characteristics will play a role in determining whether an individual adopts this technology, local factors such as the presence of champions, governance model for the project, support for staff throughout the process, and relationship between administration and physicians are likely important determinants of success at both academic and community hospitals. In addition, organizations that embark on CPOE implementation need to understand the enormity of the task at hand and must devote not only sufficient financial but also human capital over time.11, 18 In the words of a chief medical information officer, Implementing CPOE should not be thought of as an event, but a long‐term commitment.

Beyond following proposed best practices for the implementation of CPOE, community hospitals may need to adopt additional strategies to address their unique challenges. Given the misalignment of incentives for physicians' use of CPOE, leadership in community hospitals must be particularly skilled at articulating the benefits of CPOE to physicians. These benefits include not only decreased professional liability from improved patient safety and better quality of care, but also fewer pharmacy callbacks, remote access, and rapid ordering through order sets. Hospitals may also want to elicit support from physicians early by empowering them to create order sets for their disciplines. Mechanisms for hospitals and physicians to engage in mutual cost‐sharing arrangements may provide addition opportunities for hospitals to entice physicians to adopt the technology. Finally, and of particular interest to the readership of this journal, as hospitalists become more prevalent and take care of an increasing proportion of hospitalized patients,23 they are often ideal candidates to lead the implementation of CPOE in community hospitals. Because hospitalists spend most of their time in the hospital, they are often in the best position to get fully trained on CPOE, to define their own order sets, and to redesign care processes in order to take full advantage of CPOE capabilities. In addition, as many hospitalists are directly employed or supported by the hospital, their goals for quality, safety, and efficiency are usually better aligned with those of the hospital.

The stakes involved in implementing CPOE are high. Hospitals invest enormous sums of money in these systems, and many will not have the financial or political capital to attempt a second implementation after an initial failure. In addition, as recent research has pointed out,24 inappropriate implementation strategies may lead to delays in essential care and direct patient harm. In many ways, the complex task of implementing CPOE is not unlike other endeavors in patient care, where optimal outcomes require sound knowledge and reliable processes and where disaster can strike for lack of attention to detail or common sense. If Hippocrates were alive today, he might have this to say about CPOE implementation: Life is short, the art long, opportunity fleeting, experience treacherous, judgment difficult.