David T. Liss

Design

We conducted a pre‐post evaluation of the quality and timeliness of discharge summaries. In the initial phase of the study, we convened an advisory board comprised of 16 Department of Medicine physicians. The advisory board gave input on needs assessment and helped to create a survey to be administered to all medicine faculty with an outpatient practice. All respondents who had at least 1 patient admitted to the hospital within the 6 months prior to the survey were eligible. The results of the initial survey were reviewed with the advisory board and an electronic discharge summary was created with their input. To evaluate its impact, we conducted a repeat survey of all medicine faculty with an outpatient practice approximately 1 year after implementation of the electronic discharge summary.

To complement data received from the outpatient physician survey, a randomly selected sample of discharge summaries from general medical services during the same 3 month period before and after implementation of the electronic discharge summary were rated by 1 of 3 board‐certified internists (D.B.E., N.K., or M.P.L.).

Setting and Participants

The study was conducted at Northwestern Memorial Hospital, a 753‐bed hospital in Chicago, IL. The study was approved by the Institutional Review Board of the Northwestern University Feinberg School of Medicine. General medical patients were admitted to 1 of 2 primary physician services during the study period: a teaching service or a nonteaching hospitalist service. Discharge summaries had traditionally been dictated by inpatient physicians and delivered to outpatient physicians by both mail and facsimile via the medical record department. A recommended template for dictated discharge summaries was provided in the paper paging directory distributed yearly to inpatient physicians.

The hospital implemented an EMR and computerized physician order entry (CPOE) system (PowerChart Millennium; Cerner Corporation, Kansas City, MO) in August 2004. Although all history and physicals and progress notes were documented in the EMR, the system did not provide a method for delivering discharge summaries performed within the EMR to outpatient physician offices. Because of this, inpatient physicians were instructed to continue to dictate discharge summaries during the initial phase of the study.

Approximately 65% of outpatient physicians at the study site used an EMR in their offices during the study. Approximately 10% of outpatient physicians used the same EMR the hospital uses, while approximately 55% used a different EMR (EPIC Hyperspace; EPIC Systems Corporation, Verona, WI). The remaining physicians did not use an EMR in their offices.

Intervention: The Electronic Discharge Summary

A draft electronic discharge summary template was created by including elements ranked as highly important by outpatient physicians in our initial survey8 and elements required by The Joint Commission.9 The draft electronic discharge summary template was reviewed by the advisory board and modifications were made with their input. We automated the insertion of specific patient data elements, such as listed allergies and home medications, into the discharge summary template. We also created an electronic reminder system to inpatient physicians for summaries not completed 24 hours after discharge.

Because the majority of physicians in our initial survey preferred discharge summaries to be delivered either by facsimile or via an EMR, we concentrated our efforts on creating reliable systems for delivery by those routes. We created logic that queried the primary care physician field within the EMR at the time the discharge summary was electronically signed. An automated process then sent the discharge summary via electronic fax to the physician listed in the primary care physician field. Because a large number of outpatient physicians used an EMR different from the hospital's, we also created a process that sent discharge summaries from the hospital EMR into patient charts within this separate EMR.

The draft electronic discharge summary template was available for use in the EMR beginning in July 2005. The final electronic discharge summary, including automated content, physician reminder for incomplete summaries, and delivery systems as described above was implemented in June 2006. Upon implementation, inpatient physicians were instructed via email announcements and group meetings to begin completing electronic discharge summaries using the EMR. Beyond these announcements, inpatient physicians did not receive any specific training with regard to the new discharge summary process. An example of the final electronic discharge summary product is available in the Appendix.

Outpatient Physician Survey

Satisfaction with timeliness and quality of discharge summaries was assessed using a 5‐point Likert scale, where 5 represented very satisfied and 1 represented very dissatisfied. We also asked respondents to estimate the number of their patients who had sustained a preventable adverse event or near miss related to suboptimal transfer of information at discharge. We defined a preventable adverse event as a preventable medical problem or worsening of an existing problem and near miss as an error that did not result in patient harm but easily could have.

The preimplementation survey, accompanied by a cover letter signed by the hospital's chief of staff, was sent out in March 2005. A postcard reminder was sent approximately 2 weeks after the initial mail survey. A second survey was sent to nonresponders 6 weeks after the initial survey. Simultaneously, the survey was also sent in web‐based format to nonresponders via email. The postimplementation survey was sent out in February 2007 using a similar survey process.

Discharge Summary Review

A random sample of discharge summaries completed before and after the implementation of the electronic discharge summary was selected for review. The sample universe consisted of all general medicine service discharges between August and November 2005, before the electronic discharge summary was implemented, and August to November 2006, after implementation. To provide a balanced comparison, the sample was further limited to only the first chronological (index) discharge of a unique patient to home self‐care or home health nursing, with length of stay between 3 and 14 days. A total of 2232 discharges in 2005 and 2570 discharges in 2006 met these criteria. The discharge summary review sample was designed to randomly select approximately 100 discharge summaries meeting the criteria above within each study year, to produce an approximate 200‐record analysis sample. Each of the 3 physician reviewers was assigned to complete an approximately equal number of the 200 primary reviews.

Physician reviewers recorded whether the discharge summary was dictated versus done electronically, the length of the discharge summary (in words), the number of days from discharge to discharge summary completion, the type of service the patient was discharged from, and the author type (medical student, intern, resident, or attending). Physicians reviewers also assessed the overall clarity of discharge summaries using a 5‐point ordinal scale (1 = unintelligible; 2 = hard to read; 3 = neutral; 4 = understandable; and 5 = lucid).

Prior studies have evaluated the quality of discharge summaries using scoring tools created by the investigators.10, 11 We created our own discharge summary scoring tool based on these prior studies, recommendations from the literature,12 and the findings from our initial survey.8 We pilot‐tested the scoring tool and made minor revisions prior to the study. The final scoring tool consisted of 16 essential elements. Reviewers assessed whether each of the 16 essential elements was present, absent, or not applicable. A Discharge Summary Completeness Score was calculated by the number of the 16 essential elements that were rated as present divided by the number of applicable elements for each discharge summary, multiplied by 100 to produce a completeness percentage.

To assess interrater reliability, reviewers were assigned to independently complete second, duplicate reviews of approximately 90 summaries (30 per reviewer). The duplicate review sample was designed to produce approximately 45 paired re‐reviews in each year for reliability assessment. A final sample of 196 available summaries was completed for the main analysis and 174 primary and duplicate reviews were used to establish interrater reliability across 87 reviewer pairs.

Data Analysis

Physician characteristics, including specialty, faculty appointment type, and year of medical school graduation were provided by the hospital's medical staff office. Physician characteristics from before and after the implementation of the electronic discharge summary were compared using chi‐square tests. Likert scale ratings of physician satisfaction with the timeliness and quality of discharge summaries were compared using t‐tests. The proportion of physicians reporting 1 or more preventable adverse event or near miss before the implementation of the electronic discharge summary was compared to postimplementation proportions using chi‐square tests. In addition, we performed multivariate logistic regression to examine the likelihood of physicians reporting any preventable adverse event or near miss related to suboptimal information transfer. The regression models tested the likelihood of 1 or more preventable adverse event or near miss before versus after the implementation of the electronic discharge summary, controlling for physician characteristics and their number of hospitalized patients in the previous 6 months.

The proportions of discharge summary elements found to be present, the proportion of discharge summaries completed within 3 days, and discharge summary readability ratings before and after the implementation of the electronic discharge summary were compared using chi‐square tests; length in words was compared using t‐tests. Preimplementation and postimplementation Discharge Summary Completeness Scores were compared using the Mann‐Whitney U test. Discharge summary score interrater reliability was assessed using the Brennan‐Prediger Kappa for individual elements.13

Outpatient Physician Survey

Discharge Summary Review

Design

We conducted a pre‐post evaluation of the quality and timeliness of discharge summaries. In the initial phase of the study, we convened an advisory board comprised of 16 Department of Medicine physicians. The advisory board gave input on needs assessment and helped to create a survey to be administered to all medicine faculty with an outpatient practice. All respondents who had at least 1 patient admitted to the hospital within the 6 months prior to the survey were eligible. The results of the initial survey were reviewed with the advisory board and an electronic discharge summary was created with their input. To evaluate its impact, we conducted a repeat survey of all medicine faculty with an outpatient practice approximately 1 year after implementation of the electronic discharge summary.

To complement data received from the outpatient physician survey, a randomly selected sample of discharge summaries from general medical services during the same 3 month period before and after implementation of the electronic discharge summary were rated by 1 of 3 board‐certified internists (D.B.E., N.K., or M.P.L.).

Setting and Participants

The study was conducted at Northwestern Memorial Hospital, a 753‐bed hospital in Chicago, IL. The study was approved by the Institutional Review Board of the Northwestern University Feinberg School of Medicine. General medical patients were admitted to 1 of 2 primary physician services during the study period: a teaching service or a nonteaching hospitalist service. Discharge summaries had traditionally been dictated by inpatient physicians and delivered to outpatient physicians by both mail and facsimile via the medical record department. A recommended template for dictated discharge summaries was provided in the paper paging directory distributed yearly to inpatient physicians.

The hospital implemented an EMR and computerized physician order entry (CPOE) system (PowerChart Millennium; Cerner Corporation, Kansas City, MO) in August 2004. Although all history and physicals and progress notes were documented in the EMR, the system did not provide a method for delivering discharge summaries performed within the EMR to outpatient physician offices. Because of this, inpatient physicians were instructed to continue to dictate discharge summaries during the initial phase of the study.

Approximately 65% of outpatient physicians at the study site used an EMR in their offices during the study. Approximately 10% of outpatient physicians used the same EMR the hospital uses, while approximately 55% used a different EMR (EPIC Hyperspace; EPIC Systems Corporation, Verona, WI). The remaining physicians did not use an EMR in their offices.

Intervention: The Electronic Discharge Summary

A draft electronic discharge summary template was created by including elements ranked as highly important by outpatient physicians in our initial survey8 and elements required by The Joint Commission.9 The draft electronic discharge summary template was reviewed by the advisory board and modifications were made with their input. We automated the insertion of specific patient data elements, such as listed allergies and home medications, into the discharge summary template. We also created an electronic reminder system to inpatient physicians for summaries not completed 24 hours after discharge.

Because the majority of physicians in our initial survey preferred discharge summaries to be delivered either by facsimile or via an EMR, we concentrated our efforts on creating reliable systems for delivery by those routes. We created logic that queried the primary care physician field within the EMR at the time the discharge summary was electronically signed. An automated process then sent the discharge summary via electronic fax to the physician listed in the primary care physician field. Because a large number of outpatient physicians used an EMR different from the hospital's, we also created a process that sent discharge summaries from the hospital EMR into patient charts within this separate EMR.

The draft electronic discharge summary template was available for use in the EMR beginning in July 2005. The final electronic discharge summary, including automated content, physician reminder for incomplete summaries, and delivery systems as described above was implemented in June 2006. Upon implementation, inpatient physicians were instructed via email announcements and group meetings to begin completing electronic discharge summaries using the EMR. Beyond these announcements, inpatient physicians did not receive any specific training with regard to the new discharge summary process. An example of the final electronic discharge summary product is available in the Appendix.

Outpatient Physician Survey

Satisfaction with timeliness and quality of discharge summaries was assessed using a 5‐point Likert scale, where 5 represented very satisfied and 1 represented very dissatisfied. We also asked respondents to estimate the number of their patients who had sustained a preventable adverse event or near miss related to suboptimal transfer of information at discharge. We defined a preventable adverse event as a preventable medical problem or worsening of an existing problem and near miss as an error that did not result in patient harm but easily could have.

The preimplementation survey, accompanied by a cover letter signed by the hospital's chief of staff, was sent out in March 2005. A postcard reminder was sent approximately 2 weeks after the initial mail survey. A second survey was sent to nonresponders 6 weeks after the initial survey. Simultaneously, the survey was also sent in web‐based format to nonresponders via email. The postimplementation survey was sent out in February 2007 using a similar survey process.

Discharge Summary Review

A random sample of discharge summaries completed before and after the implementation of the electronic discharge summary was selected for review. The sample universe consisted of all general medicine service discharges between August and November 2005, before the electronic discharge summary was implemented, and August to November 2006, after implementation. To provide a balanced comparison, the sample was further limited to only the first chronological (index) discharge of a unique patient to home self‐care or home health nursing, with length of stay between 3 and 14 days. A total of 2232 discharges in 2005 and 2570 discharges in 2006 met these criteria. The discharge summary review sample was designed to randomly select approximately 100 discharge summaries meeting the criteria above within each study year, to produce an approximate 200‐record analysis sample. Each of the 3 physician reviewers was assigned to complete an approximately equal number of the 200 primary reviews.

Physician reviewers recorded whether the discharge summary was dictated versus done electronically, the length of the discharge summary (in words), the number of days from discharge to discharge summary completion, the type of service the patient was discharged from, and the author type (medical student, intern, resident, or attending). Physicians reviewers also assessed the overall clarity of discharge summaries using a 5‐point ordinal scale (1 = unintelligible; 2 = hard to read; 3 = neutral; 4 = understandable; and 5 = lucid).

Prior studies have evaluated the quality of discharge summaries using scoring tools created by the investigators.10, 11 We created our own discharge summary scoring tool based on these prior studies, recommendations from the literature,12 and the findings from our initial survey.8 We pilot‐tested the scoring tool and made minor revisions prior to the study. The final scoring tool consisted of 16 essential elements. Reviewers assessed whether each of the 16 essential elements was present, absent, or not applicable. A Discharge Summary Completeness Score was calculated by the number of the 16 essential elements that were rated as present divided by the number of applicable elements for each discharge summary, multiplied by 100 to produce a completeness percentage.

To assess interrater reliability, reviewers were assigned to independently complete second, duplicate reviews of approximately 90 summaries (30 per reviewer). The duplicate review sample was designed to produce approximately 45 paired re‐reviews in each year for reliability assessment. A final sample of 196 available summaries was completed for the main analysis and 174 primary and duplicate reviews were used to establish interrater reliability across 87 reviewer pairs.

Data Analysis

Physician characteristics, including specialty, faculty appointment type, and year of medical school graduation were provided by the hospital's medical staff office. Physician characteristics from before and after the implementation of the electronic discharge summary were compared using chi‐square tests. Likert scale ratings of physician satisfaction with the timeliness and quality of discharge summaries were compared using t‐tests. The proportion of physicians reporting 1 or more preventable adverse event or near miss before the implementation of the electronic discharge summary was compared to postimplementation proportions using chi‐square tests. In addition, we performed multivariate logistic regression to examine the likelihood of physicians reporting any preventable adverse event or near miss related to suboptimal information transfer. The regression models tested the likelihood of 1 or more preventable adverse event or near miss before versus after the implementation of the electronic discharge summary, controlling for physician characteristics and their number of hospitalized patients in the previous 6 months.

The proportions of discharge summary elements found to be present, the proportion of discharge summaries completed within 3 days, and discharge summary readability ratings before and after the implementation of the electronic discharge summary were compared using chi‐square tests; length in words was compared using t‐tests. Preimplementation and postimplementation Discharge Summary Completeness Scores were compared using the Mann‐Whitney U test. Discharge summary score interrater reliability was assessed using the Brennan‐Prediger Kappa for individual elements.13

Outpatient Physician Survey

Discharge Summary Review

Design

We conducted a pre‐post evaluation of the quality and timeliness of discharge summaries. In the initial phase of the study, we convened an advisory board comprised of 16 Department of Medicine physicians. The advisory board gave input on needs assessment and helped to create a survey to be administered to all medicine faculty with an outpatient practice. All respondents who had at least 1 patient admitted to the hospital within the 6 months prior to the survey were eligible. The results of the initial survey were reviewed with the advisory board and an electronic discharge summary was created with their input. To evaluate its impact, we conducted a repeat survey of all medicine faculty with an outpatient practice approximately 1 year after implementation of the electronic discharge summary.

To complement data received from the outpatient physician survey, a randomly selected sample of discharge summaries from general medical services during the same 3 month period before and after implementation of the electronic discharge summary were rated by 1 of 3 board‐certified internists (D.B.E., N.K., or M.P.L.).

Setting and Participants

The study was conducted at Northwestern Memorial Hospital, a 753‐bed hospital in Chicago, IL. The study was approved by the Institutional Review Board of the Northwestern University Feinberg School of Medicine. General medical patients were admitted to 1 of 2 primary physician services during the study period: a teaching service or a nonteaching hospitalist service. Discharge summaries had traditionally been dictated by inpatient physicians and delivered to outpatient physicians by both mail and facsimile via the medical record department. A recommended template for dictated discharge summaries was provided in the paper paging directory distributed yearly to inpatient physicians.

The hospital implemented an EMR and computerized physician order entry (CPOE) system (PowerChart Millennium; Cerner Corporation, Kansas City, MO) in August 2004. Although all history and physicals and progress notes were documented in the EMR, the system did not provide a method for delivering discharge summaries performed within the EMR to outpatient physician offices. Because of this, inpatient physicians were instructed to continue to dictate discharge summaries during the initial phase of the study.

Approximately 65% of outpatient physicians at the study site used an EMR in their offices during the study. Approximately 10% of outpatient physicians used the same EMR the hospital uses, while approximately 55% used a different EMR (EPIC Hyperspace; EPIC Systems Corporation, Verona, WI). The remaining physicians did not use an EMR in their offices.

Intervention: The Electronic Discharge Summary

A draft electronic discharge summary template was created by including elements ranked as highly important by outpatient physicians in our initial survey8 and elements required by The Joint Commission.9 The draft electronic discharge summary template was reviewed by the advisory board and modifications were made with their input. We automated the insertion of specific patient data elements, such as listed allergies and home medications, into the discharge summary template. We also created an electronic reminder system to inpatient physicians for summaries not completed 24 hours after discharge.

Because the majority of physicians in our initial survey preferred discharge summaries to be delivered either by facsimile or via an EMR, we concentrated our efforts on creating reliable systems for delivery by those routes. We created logic that queried the primary care physician field within the EMR at the time the discharge summary was electronically signed. An automated process then sent the discharge summary via electronic fax to the physician listed in the primary care physician field. Because a large number of outpatient physicians used an EMR different from the hospital's, we also created a process that sent discharge summaries from the hospital EMR into patient charts within this separate EMR.

The draft electronic discharge summary template was available for use in the EMR beginning in July 2005. The final electronic discharge summary, including automated content, physician reminder for incomplete summaries, and delivery systems as described above was implemented in June 2006. Upon implementation, inpatient physicians were instructed via email announcements and group meetings to begin completing electronic discharge summaries using the EMR. Beyond these announcements, inpatient physicians did not receive any specific training with regard to the new discharge summary process. An example of the final electronic discharge summary product is available in the Appendix.

Outpatient Physician Survey

Satisfaction with timeliness and quality of discharge summaries was assessed using a 5‐point Likert scale, where 5 represented very satisfied and 1 represented very dissatisfied. We also asked respondents to estimate the number of their patients who had sustained a preventable adverse event or near miss related to suboptimal transfer of information at discharge. We defined a preventable adverse event as a preventable medical problem or worsening of an existing problem and near miss as an error that did not result in patient harm but easily could have.

The preimplementation survey, accompanied by a cover letter signed by the hospital's chief of staff, was sent out in March 2005. A postcard reminder was sent approximately 2 weeks after the initial mail survey. A second survey was sent to nonresponders 6 weeks after the initial survey. Simultaneously, the survey was also sent in web‐based format to nonresponders via email. The postimplementation survey was sent out in February 2007 using a similar survey process.

Discharge Summary Review

A random sample of discharge summaries completed before and after the implementation of the electronic discharge summary was selected for review. The sample universe consisted of all general medicine service discharges between August and November 2005, before the electronic discharge summary was implemented, and August to November 2006, after implementation. To provide a balanced comparison, the sample was further limited to only the first chronological (index) discharge of a unique patient to home self‐care or home health nursing, with length of stay between 3 and 14 days. A total of 2232 discharges in 2005 and 2570 discharges in 2006 met these criteria. The discharge summary review sample was designed to randomly select approximately 100 discharge summaries meeting the criteria above within each study year, to produce an approximate 200‐record analysis sample. Each of the 3 physician reviewers was assigned to complete an approximately equal number of the 200 primary reviews.

Physician reviewers recorded whether the discharge summary was dictated versus done electronically, the length of the discharge summary (in words), the number of days from discharge to discharge summary completion, the type of service the patient was discharged from, and the author type (medical student, intern, resident, or attending). Physicians reviewers also assessed the overall clarity of discharge summaries using a 5‐point ordinal scale (1 = unintelligible; 2 = hard to read; 3 = neutral; 4 = understandable; and 5 = lucid).

Prior studies have evaluated the quality of discharge summaries using scoring tools created by the investigators.10, 11 We created our own discharge summary scoring tool based on these prior studies, recommendations from the literature,12 and the findings from our initial survey.8 We pilot‐tested the scoring tool and made minor revisions prior to the study. The final scoring tool consisted of 16 essential elements. Reviewers assessed whether each of the 16 essential elements was present, absent, or not applicable. A Discharge Summary Completeness Score was calculated by the number of the 16 essential elements that were rated as present divided by the number of applicable elements for each discharge summary, multiplied by 100 to produce a completeness percentage.

To assess interrater reliability, reviewers were assigned to independently complete second, duplicate reviews of approximately 90 summaries (30 per reviewer). The duplicate review sample was designed to produce approximately 45 paired re‐reviews in each year for reliability assessment. A final sample of 196 available summaries was completed for the main analysis and 174 primary and duplicate reviews were used to establish interrater reliability across 87 reviewer pairs.

Data Analysis

Physician characteristics, including specialty, faculty appointment type, and year of medical school graduation were provided by the hospital's medical staff office. Physician characteristics from before and after the implementation of the electronic discharge summary were compared using chi‐square tests. Likert scale ratings of physician satisfaction with the timeliness and quality of discharge summaries were compared using t‐tests. The proportion of physicians reporting 1 or more preventable adverse event or near miss before the implementation of the electronic discharge summary was compared to postimplementation proportions using chi‐square tests. In addition, we performed multivariate logistic regression to examine the likelihood of physicians reporting any preventable adverse event or near miss related to suboptimal information transfer. The regression models tested the likelihood of 1 or more preventable adverse event or near miss before versus after the implementation of the electronic discharge summary, controlling for physician characteristics and their number of hospitalized patients in the previous 6 months.

The proportions of discharge summary elements found to be present, the proportion of discharge summaries completed within 3 days, and discharge summary readability ratings before and after the implementation of the electronic discharge summary were compared using chi‐square tests; length in words was compared using t‐tests. Preimplementation and postimplementation Discharge Summary Completeness Scores were compared using the Mann‐Whitney U test. Discharge summary score interrater reliability was assessed using the Brennan‐Prediger Kappa for individual elements.13

Outpatient Physician Survey

Discharge Summary Review

Study Site

The study was conducted at a 753‐bed academic hospital in Chicago, Illinois. Discharge summaries have traditionally been dictated by inpatient physicians and delivered to outpatient physicians by both mail and facsimile via the medical records department. The hospital has used an electronic medical record and computerized physician order entry system (PowerChart Millennium from Cerner Corporation) since August 2004. Although all history and physicals and progress notes were documented in the electronic medical record, the system did not provide a method for delivering the discharge summaries contained in the electronic medical record to outpatient physician offices. Because of this, inpatient physicians continued to dictate discharge summaries during this study.

Participants

An advisory board consisting of 16 physicians from the Department of Medicine was convened. The advisory board gave input on needs assessment and helped to create a survey to be administered to all 425 medicine faculty who have an outpatient practice. All respondents who had at least 1 patient admitted to the hospital within the 6 months prior to the survey were eligible.

Survey Content

Our survey consisted of 2 parts. In the first part, we asked respondents to estimate how many of their patients had been discharged from the hospital in the past 6 months and to reflect on these patients as they completed the survey. Satisfaction with the timeliness and quality of discharge summaries was assessed using a 5‐point Likert scale, from 5, very satisfied, to 1, very unsatisfied. The frequency of hospital follow‐up of a patient occurring prior to arrival of the discharge summary was assessed as the percentage of times this occurred in 20% increments (0%‐19%, 20%‐39%, 40%‐59%, 60%‐79%, and 80%‐100%). The number of discharge summaries missing critical information and the number of summaries containing unnecessary information were similarly assessed using 20% increments. We then asked respondents to estimate the number of patients who had sustained a preventable adverse event related to suboptimal transfer of information at discharge. We defined a preventable adverse event as a preventable medical problem or worsening of an existing problem.

In the second part of the survey, we elicited preferences for discharge summary content and method of delivery. We assessed preferences for discharge summary content by asking respondents to rank items on a scale from 1 to 10, from 10, most important, to 1, least important. Preferences for delivery of discharge summaries were assessed by asking respondents to indicate one or more delivery methods, including facsimile, mail, the electronic medical record, and E‐mail.

Survey Process

The survey was sent out in March 2005. A postcard reminder was sent out approximately 2 weeks after the initial survey was mailed. A second survey was sent to nonresponders 6 weeks after the initial survey. Simultaneously, the survey was also sent in Web‐based format to nonresponders via email.

Data Analysis

Physician characteristics, including practice type, faculty appointment type, and year of medical school graduation, were provided by the hospital's medical staff office. Physician respondents and nonrespondents were compared using the chi‐square test and logistic regression to determine potential response biases. We calculated means and standard deviations and percentages for categorical variables. Logistic regression was used to examine the likelihood of participants reporting any preventable adverse event related to suboptimal transfer of information. The regression model tested the likelihood of one or more preventable adverse events reported with the frequency of seeing patients for follow‐up prior to the arrival of discharge summaries, controlling for participant characteristics and the number of hospitalized patients each physician had in the previous 6 months.

Physician Characteristics

The survey was sent to 425 physicians, 9 of whom were excluded because they had had no patients admitted within the past 6 months. Of the 416 eligible respondents, 2 returned a survey that was incomplete and not usable, and 226 returned a completed survey (response rate of 54%). The characteristics of responders and nonresponders are shown in Table 1. General medicine physicians completed the survey more often than specialist physicians (56% vs. 44%, P < .001). Affiliated faculty were also more likely to complete the survey than full‐time faculty; multivariate logistic regression revealed this was purely a function of the larger number of specialists among the full‐time faculty.

Timeliness and Content

Only 19% of the participants were satisfied or very satisfied with the timeliness of discharge summaries. Among all participants, 33% indicated that 60% or more of their patients were seen for their follow‐up outpatient visit prior to the arrival of the discharge summary, and 22% indicated that for 60% or more of their patients they never received a discharge summary at all.

Only 32% of the participants were satisfied or very satisfied with the quality of discharge summaries. Among all participants, 17% believed that 60% or more of discharge summaries missed critical information. Unnecessary information in the discharge summary was less problematic; only 9% of participants indicated that 60% or more of discharge summaries contained unnecessary information.

Preventable Adverse Events

Overall, 41% of participants believed that in the previous 6 months at least one of their patients had sustained a preventable adverse event related to poor transfer of information at hospital discharge. Reporting one or more preventable adverse events was positively associated with physicians' reports of how often they saw patients for a first postdischarge follow‐up without having a discharge summary available. After adjusting for participant characteristics and the number of patients hospitalized by each physician, logistic regression results indicated that each 20% increase in the frequency of discharge summaries not arriving prior to patient follow‐up appointments was associated with a 28% increase in the odds of a reported preventable adverse event (adjusted OR = 1.28, P = .04).

Preferences for Content and Delivery

The mean rating for importance of discharge summary elements is shown in Table 2. No discharge summary element had a mean rating of less than 5. Participants preferred discharge summaries be delivered via the following methods: facsimile, 48%; mail, 30%; electronic medical record, 41%; and E‐mail, 30%.

Study Site

The study was conducted at a 753‐bed academic hospital in Chicago, Illinois. Discharge summaries have traditionally been dictated by inpatient physicians and delivered to outpatient physicians by both mail and facsimile via the medical records department. The hospital has used an electronic medical record and computerized physician order entry system (PowerChart Millennium from Cerner Corporation) since August 2004. Although all history and physicals and progress notes were documented in the electronic medical record, the system did not provide a method for delivering the discharge summaries contained in the electronic medical record to outpatient physician offices. Because of this, inpatient physicians continued to dictate discharge summaries during this study.

Participants

An advisory board consisting of 16 physicians from the Department of Medicine was convened. The advisory board gave input on needs assessment and helped to create a survey to be administered to all 425 medicine faculty who have an outpatient practice. All respondents who had at least 1 patient admitted to the hospital within the 6 months prior to the survey were eligible.

Survey Content

Our survey consisted of 2 parts. In the first part, we asked respondents to estimate how many of their patients had been discharged from the hospital in the past 6 months and to reflect on these patients as they completed the survey. Satisfaction with the timeliness and quality of discharge summaries was assessed using a 5‐point Likert scale, from 5, very satisfied, to 1, very unsatisfied. The frequency of hospital follow‐up of a patient occurring prior to arrival of the discharge summary was assessed as the percentage of times this occurred in 20% increments (0%‐19%, 20%‐39%, 40%‐59%, 60%‐79%, and 80%‐100%). The number of discharge summaries missing critical information and the number of summaries containing unnecessary information were similarly assessed using 20% increments. We then asked respondents to estimate the number of patients who had sustained a preventable adverse event related to suboptimal transfer of information at discharge. We defined a preventable adverse event as a preventable medical problem or worsening of an existing problem.

In the second part of the survey, we elicited preferences for discharge summary content and method of delivery. We assessed preferences for discharge summary content by asking respondents to rank items on a scale from 1 to 10, from 10, most important, to 1, least important. Preferences for delivery of discharge summaries were assessed by asking respondents to indicate one or more delivery methods, including facsimile, mail, the electronic medical record, and E‐mail.

Survey Process

The survey was sent out in March 2005. A postcard reminder was sent out approximately 2 weeks after the initial survey was mailed. A second survey was sent to nonresponders 6 weeks after the initial survey. Simultaneously, the survey was also sent in Web‐based format to nonresponders via email.

Data Analysis

Physician characteristics, including practice type, faculty appointment type, and year of medical school graduation, were provided by the hospital's medical staff office. Physician respondents and nonrespondents were compared using the chi‐square test and logistic regression to determine potential response biases. We calculated means and standard deviations and percentages for categorical variables. Logistic regression was used to examine the likelihood of participants reporting any preventable adverse event related to suboptimal transfer of information. The regression model tested the likelihood of one or more preventable adverse events reported with the frequency of seeing patients for follow‐up prior to the arrival of discharge summaries, controlling for participant characteristics and the number of hospitalized patients each physician had in the previous 6 months.

Physician Characteristics

The survey was sent to 425 physicians, 9 of whom were excluded because they had had no patients admitted within the past 6 months. Of the 416 eligible respondents, 2 returned a survey that was incomplete and not usable, and 226 returned a completed survey (response rate of 54%). The characteristics of responders and nonresponders are shown in Table 1. General medicine physicians completed the survey more often than specialist physicians (56% vs. 44%, P < .001). Affiliated faculty were also more likely to complete the survey than full‐time faculty; multivariate logistic regression revealed this was purely a function of the larger number of specialists among the full‐time faculty.

Timeliness and Content

Only 19% of the participants were satisfied or very satisfied with the timeliness of discharge summaries. Among all participants, 33% indicated that 60% or more of their patients were seen for their follow‐up outpatient visit prior to the arrival of the discharge summary, and 22% indicated that for 60% or more of their patients they never received a discharge summary at all.

Only 32% of the participants were satisfied or very satisfied with the quality of discharge summaries. Among all participants, 17% believed that 60% or more of discharge summaries missed critical information. Unnecessary information in the discharge summary was less problematic; only 9% of participants indicated that 60% or more of discharge summaries contained unnecessary information.

Preventable Adverse Events

Overall, 41% of participants believed that in the previous 6 months at least one of their patients had sustained a preventable adverse event related to poor transfer of information at hospital discharge. Reporting one or more preventable adverse events was positively associated with physicians' reports of how often they saw patients for a first postdischarge follow‐up without having a discharge summary available. After adjusting for participant characteristics and the number of patients hospitalized by each physician, logistic regression results indicated that each 20% increase in the frequency of discharge summaries not arriving prior to patient follow‐up appointments was associated with a 28% increase in the odds of a reported preventable adverse event (adjusted OR = 1.28, P = .04).

Preferences for Content and Delivery

The mean rating for importance of discharge summary elements is shown in Table 2. No discharge summary element had a mean rating of less than 5. Participants preferred discharge summaries be delivered via the following methods: facsimile, 48%; mail, 30%; electronic medical record, 41%; and E‐mail, 30%.

Study Site

The study was conducted at a 753‐bed academic hospital in Chicago, Illinois. Discharge summaries have traditionally been dictated by inpatient physicians and delivered to outpatient physicians by both mail and facsimile via the medical records department. The hospital has used an electronic medical record and computerized physician order entry system (PowerChart Millennium from Cerner Corporation) since August 2004. Although all history and physicals and progress notes were documented in the electronic medical record, the system did not provide a method for delivering the discharge summaries contained in the electronic medical record to outpatient physician offices. Because of this, inpatient physicians continued to dictate discharge summaries during this study.

Participants

An advisory board consisting of 16 physicians from the Department of Medicine was convened. The advisory board gave input on needs assessment and helped to create a survey to be administered to all 425 medicine faculty who have an outpatient practice. All respondents who had at least 1 patient admitted to the hospital within the 6 months prior to the survey were eligible.

Survey Content

Our survey consisted of 2 parts. In the first part, we asked respondents to estimate how many of their patients had been discharged from the hospital in the past 6 months and to reflect on these patients as they completed the survey. Satisfaction with the timeliness and quality of discharge summaries was assessed using a 5‐point Likert scale, from 5, very satisfied, to 1, very unsatisfied. The frequency of hospital follow‐up of a patient occurring prior to arrival of the discharge summary was assessed as the percentage of times this occurred in 20% increments (0%‐19%, 20%‐39%, 40%‐59%, 60%‐79%, and 80%‐100%). The number of discharge summaries missing critical information and the number of summaries containing unnecessary information were similarly assessed using 20% increments. We then asked respondents to estimate the number of patients who had sustained a preventable adverse event related to suboptimal transfer of information at discharge. We defined a preventable adverse event as a preventable medical problem or worsening of an existing problem.

In the second part of the survey, we elicited preferences for discharge summary content and method of delivery. We assessed preferences for discharge summary content by asking respondents to rank items on a scale from 1 to 10, from 10, most important, to 1, least important. Preferences for delivery of discharge summaries were assessed by asking respondents to indicate one or more delivery methods, including facsimile, mail, the electronic medical record, and E‐mail.

Survey Process

The survey was sent out in March 2005. A postcard reminder was sent out approximately 2 weeks after the initial survey was mailed. A second survey was sent to nonresponders 6 weeks after the initial survey. Simultaneously, the survey was also sent in Web‐based format to nonresponders via email.

Data Analysis

Physician characteristics, including practice type, faculty appointment type, and year of medical school graduation, were provided by the hospital's medical staff office. Physician respondents and nonrespondents were compared using the chi‐square test and logistic regression to determine potential response biases. We calculated means and standard deviations and percentages for categorical variables. Logistic regression was used to examine the likelihood of participants reporting any preventable adverse event related to suboptimal transfer of information. The regression model tested the likelihood of one or more preventable adverse events reported with the frequency of seeing patients for follow‐up prior to the arrival of discharge summaries, controlling for participant characteristics and the number of hospitalized patients each physician had in the previous 6 months.

Physician Characteristics

The survey was sent to 425 physicians, 9 of whom were excluded because they had had no patients admitted within the past 6 months. Of the 416 eligible respondents, 2 returned a survey that was incomplete and not usable, and 226 returned a completed survey (response rate of 54%). The characteristics of responders and nonresponders are shown in Table 1. General medicine physicians completed the survey more often than specialist physicians (56% vs. 44%, P < .001). Affiliated faculty were also more likely to complete the survey than full‐time faculty; multivariate logistic regression revealed this was purely a function of the larger number of specialists among the full‐time faculty.

Timeliness and Content

Only 19% of the participants were satisfied or very satisfied with the timeliness of discharge summaries. Among all participants, 33% indicated that 60% or more of their patients were seen for their follow‐up outpatient visit prior to the arrival of the discharge summary, and 22% indicated that for 60% or more of their patients they never received a discharge summary at all.

Only 32% of the participants were satisfied or very satisfied with the quality of discharge summaries. Among all participants, 17% believed that 60% or more of discharge summaries missed critical information. Unnecessary information in the discharge summary was less problematic; only 9% of participants indicated that 60% or more of discharge summaries contained unnecessary information.

Preventable Adverse Events

Overall, 41% of participants believed that in the previous 6 months at least one of their patients had sustained a preventable adverse event related to poor transfer of information at hospital discharge. Reporting one or more preventable adverse events was positively associated with physicians' reports of how often they saw patients for a first postdischarge follow‐up without having a discharge summary available. After adjusting for participant characteristics and the number of patients hospitalized by each physician, logistic regression results indicated that each 20% increase in the frequency of discharge summaries not arriving prior to patient follow‐up appointments was associated with a 28% increase in the odds of a reported preventable adverse event (adjusted OR = 1.28, P = .04).

Preferences for Content and Delivery

The mean rating for importance of discharge summary elements is shown in Table 2. No discharge summary element had a mean rating of less than 5. Participants preferred discharge summaries be delivered via the following methods: facsimile, 48%; mail, 30%; electronic medical record, 41%; and E‐mail, 30%.