Ndidi I. Unaka, MD, MEd

The transition from hospital to home can be overwhelming for caregivers.¹ Stress of hospitalization coupled with the expectation of families to execute postdischarge care plans make understandable discharge communication critical. Communication failures, inadequate education, absence of caregiver confidence, and lack of clarity regarding care plans may prohibit smooth transitions and lead to adverse postdischarge outcomes.^2-4

Health literacy plays a pivotal role in caregivers’ capacity to navigate the healthcare system, comprehend, and execute care plans. An estimated 90 million Americans have limited health literacy that may negatively impact the provision of safe and quality care^5,6 and be a risk factor for poor outcomes, including increased emergency department (ED) utilization and readmission rates.^7-9 Readability strongly influences the effectiveness of written materials.¹⁰ However, written medical information for patients and families are frequently between the 10^th and 12^th grade reading levels; more than 75% of all pediatric health information is written at or above 10^th grade reading level.¹¹ Government agencies recommend between a 6^th and 8^th grade reading level, for written material;^5,12,13 written discharge instructions have been identified as an important quality metric for hospital-to-home transitions.^14-16

At our center, we found that discharge instructions were commonly written at high reading levels and often incomplete.¹⁷ Poor discharge instructions may contribute to increased readmission rates and unnecessary ED visits.^9,18 Our global aim targeted improved health-literate written information, including understandability and completeness.

Our specific aim was to increase the percentage of discharge instructions written at or below the 7th grade level for hospital medicine (HM) patients on a community hospital pediatric unit from 13% to 80% in 6 months.

Context

The improvement work took place at a 42-bed inpatient pediatric unit at a community satellite of our large, urban, academic hospital. The unit is staffed by medical providers including attendings, fellows, nurse practitioners (NPs), and senior pediatric residents, and had more than 1000 HM discharges in fiscal year 2016. Children with common general pediatric diagnoses are admitted to this service; postsurgical patients are not admitted primarily to the HM service. In Cincinnati, the neighborhood-level high school drop-out rates are as high as 64%.¹⁹ Discharge instructions are written by medical providers in the electronic health record (EHR). A printed copy is given to families and verbally reviewed by a bedside nurse prior to discharge. Quality improvement (QI) efforts focused on discharge instructions were ignited by a prior review of 200 discharge instructions that showed they were difficult to read (median reading level of 10^th grade), poorly understandable (36% of instructions met the threshold of understandability as measured by the Patient Education Materials Assessment Tool²⁰) and were missing key elements of information.¹⁷

Improvement Team

The improvement team consisted of 4 pediatric hospitalists, 2 NPs, 1 nurse educator with health literacy expertise, 1 pediatric resident, 1 fourth-year medical student, 1 QI consultant, and 2 parents who had first-hand experience on the HM service. The improvement team observed the discharge process, including roles of the provider, nurse and family, outlined a process map, and created a modified failure mode and effect analysis.²¹ Prior to our work, discharge instructions written by providers often occurred as a last step, and the content was created as free text or from nonstandardized templates. Key drivers that informed interventions were determined and revised over time (Figure 1). The study was reviewed by our institutional review board and deemed not human subjects research.

Improvement Activities

Key drivers were identified, and interventions were executed using Plan-Do Study-Act cycles.²² The key drivers thought to be critical for the success of the QI efforts were family engagement; standardization of discharge instructions; medical staff engagement; and audit and feedback of data. The corresponding interventions were as follows:

Family Engagement

Understanding the discharge information families desired. Prior to testing, 10 families admitted to the HM service were asked about the discharge experience. We asked families about information they wanted in written discharge instructions: 1) reasons to call your primary doctor or return to the hospital; 2) when to see your primary doctor for a follow-up visit; 3) the phone number to reach your child’s doctor; 4) more information about why your child was admitted; 5) information about new medications; and 6) what to do to help your child continue to recover at home.

Development of templates. We engaged families throughout the process of creating general and disease-specific discharge templates. After a specific template was created and reviewed by the parents on our team, it was sent to members of the institutional Patient Education Committee, which includes parents and local health literacy experts, to review and critique. Feedback from the reviewers was incorporated into the templates prior to use in the EHR.

Postdischarge phone calls.A convenience sample of families discharged from the satellite campus was called 24 to 48 hours after discharge over a 2-week period in January, 2016. A member of our improvement team solicited feedback from families about the quality of the discharge instructions. Families were asked if discharge instructions were reviewed with them prior to going home, if they were given a copy of the instructions, how they would rate the ability to read and use the information, and if there were additional pieces of information that would have improved the instructions.

Standardization of Instructions

Education. A presentation was created and shared with medical providers; it was re-disseminated monthly to new residents rotating onto the service and to the attendings, fellows, and NPs scheduled for shifts during the month. This education continued for the duration of the study. The presentation included the definition of health literacy, scope of the problem, examples of poorly written discharge instructions, and tips on how to write readable and understandable instructions. Laminated cards that included tips on how to write instructions were also placed on work stations.

While the general template was an important first step, the content relied heavily on free text by providers, which could still lead to instructions written at a high reading level. Thus, disease-specific discharge instruction templates were created with prepopulated information that was written at a reading level at or below 7^th grade level (Figure 2). The diseases were prioritized based on the most common diagnoses on our HM service. Each template included information under each of the subheadings noted in the general template. Twelve disease-specific templates were tested and ultimately embedded in the EHR; the general template remained for use when the discharge diagnosis was not covered by a disease-specific template.

Medical Staff Engagement

Previously described tests of change also aimed to enhance staff engagement. These included frequent e-mails, discussion of the QI efforts at specific team meetings, and the creation of visual cues posted at computer work stations, which prompted staff to begin to work on discharge instructions soon after admission.

Audit and Feedback of Data

Weekly phone calls. One team updated clinicians through a regularly scheduled bi-weekly phone conference. The phone conference was established prior to our work and was designed to relay pertinent information to attendings and NPs who work at the satellite hospital. During the phone conferences, clinicians were notified of current performance on discharge instruction readability and specific tests of change for the week. Additionally, providers gave feedback about the improvement efforts. These updates continued for the first 6 months of the project until sustained improvements were observed.

E-mails. Weekly e-mails were sent to all providers scheduled for clinical time at the satellite campus. The e-mail contained information on current tests of change, a list of discharge instruction templates that were available in the EHR, and the annotated run chart illustrating readability levels over time.

Additionally, individual e-mails were sent to each provider after review of the written discharge instructions for the week. Providers were given information on the number of discharge instructions they personally composed, the percentage of those instructions that were written at or below 7^th grade level, and specific feedback on how their written instructions could be improved. We also encouraged feedback from each provider to better identify barriers to achieving our goal.

Study of the Interventions

Baseline data included a review of all instructions for patients discharged from the satellite campus from the end of April 2015 through mid-September 2015. The time period for testing of interventions during the fall and winter months allowed for rapid cycle learning due to higher patient census and predictability of admissions for specific diagnosis (ie, asthma and bronchiolitis). An automated report was generated from the EHR weekly with specific demographics and identifiers for patient discharged over the past 7 days, including patient age, gender, length of stay, discharge diagnosis, and insurance classification. Data was collected during the intervention period via structured review of the discharge instructions in the EHR by the principal investigator or a trained research coordinator. Discharge instructions for medically cleared mental health patients admitted to hospital medicine while awaiting psychiatric bed availability and patients and parents who were non-English speaking were excluded from review. All other instructions for patients discharged from the HM service at our Liberty Campus were included for review.

Measures

Readability, our primary measure of interest, was calculated using the mean score from the following formulas: Flesch Kincaid Grade Level,²³ Simple Measure of Gobbledygook Index,²⁴ Coleman-Liau Index,²⁵ Gunning-Fog Index,²⁶ and Automated Readability Index²⁷ by means of an online platform (https://readability-score.com).²⁸ This platform was chosen because it incorporated a variety of formulas, was user-friendly, and required minimal data cleaning. Each of the readability formulas have been used to assesses readability of health information given to patients and families.^29,30 The threshold of 7th grade is in alignment with our institutional policy for educational materials and with recommendations from several government agencies.^5,12

Analysis

A statistical process control p-chart was used to analyze our primary measure of readability, dichotomized as percent discharge instructions written at or below 7th grade level. Run charts were used to follow mean reading level of discharge instructions and our process measure of percent of discharge instruction written with a general or disease-specific standardized template. Run chart and control chart rules for identifying special cause were used for midline shifts.³¹

The Table includes the demographic and clinical information of patients included in our analyses. Through sequential interventions, the percentage of discharge instructions written at or below 7th grade readability level increased from a mean of 13% to more than 80% in 3 months (Figure 3). Furthermore, the mean was sustained above 90% for 10 months and at 98% for the last 4 months. The use of 1 of the 13 EHR templates increased from 0% to 96% and was associated with the largest impact on the overall improvements (Supplemental Figure 1). Additionally, the average reading level of the discharge instructions decreased from 10th grade to 6th grade level (Supplemental Figure 2).

Qualitative comments from providers about the discharge instructions included:

“Are these [discharge instructions] available at base??  Great resource for interns.”
“These [discharge] instructions make the [discharge] process so easy!!! Love these...”
“Also feel like they have helped my discharge teaching in the room!”

Qualitative comments from families postdischarge included:
“I thought the instructions were very clear and easy to read. I especially thought that highlighting the important areas really helped.”
“I think this form looks great, and I really like the idea of having your child’s name on it.”

Through sequential Plan-Do Study-Act cycles, we increased the percentage of discharge instructions written at or below 7^th grade reading level from 13% to 98%. Our most impactful intervention was the creation and dissemination of standardized disease-specific discharge instruction templates. Our findings complement evidence in the adult and pediatric literature that the use of standardized, disease-specific discharge instruction templates may improve readability of instructions.^32,33 And, while quality improvement efforts have been employed to improve the discharge process for patients,^34-36 this is the first study in the inpatient setting that, to our knowledge, specifically addresses discharge instructions using quality improvement methods.

Our work targeted the critical intersection between individual health literacy, an individual’s capacity to acquire, interpret, and use health information, and the necessary changes needed within our healthcare system to ensure that appropriately written instructions are given to patients and families.^17,37 Our efforts focused on improving discharge instructions answer the call to consider health literacy a modifiable clinical risk factor.³⁷ Furthermore, we address the 6 aims for quality healthcare delivery: 1) safe, timely, efficient and equitable delivery of care through the creation and dissemination of standardized instructions that are written at the appropriate reading level for families to ease hospital-to-home transitions and streamline the workflow of medical providers; 2) effective education of medical providers on health literacy concepts; and 3) family-centeredness through the involvement of families in our QI efforts. While previous QI efforts to improve hospital-to-home transitions have focused on medication reconciliation, communication with primary care physicians, follow-up appointments, and timely discharges of patients, none have specifically focused on the quality of discharge instructions.^34-36

Most physicians do not receive education about how to write information that is readable and understandable; more than half of providers desired more education in this area.³⁸ Furthermore, pediatric providers may overestimate parental health literacy levels,³⁹ which may contribute to variability in the readability of written health materials. While education alone can contribute to a provider’s ability to create readable instructions, we note the improvement after the introduction of disease templates to demonstrate the importance of workflow-integrated higher reliability interventions to sustain improvements.

Our baseline poor readability rates were due to limited knowledge by frontline providers composing the instructions and a system in which an important element for successful hospital-to-home transitions was not tackled until patients were ready for discharge. Streamlining of the discharge process, including the creation of discharge instructions, may lead to improved efficiency, fewer discrepancies, more effective communication, and an enhanced family experience. Moreover, the success of our improvement work was due to key stakeholders, including parents, being a part of the team and the notable buy-in from providers.

Our work was not without limitations. We excluded non-English speaking families from the study. We were unable to measure reading level of our population directly and instead based our goals on national estimates. Our primary measure was readability, which is only 1 piece contributing to quality discharge instructions. Understandability and actionability are also important considerations; ^17,20,29,40 however, improvements in these areas were limited by our design options within the EHR. Our efforts focused on children with common general pediatric diagnoses, and it is unclear how our interventions would generalize to medically complex patients with more volume of information to communicate at discharge and with uncommon diagnoses that are less readily incorporated into standardized templates. Relatedly, our work occurred at the satellite campus of our tertiary care center and may not represent generalizable material or methods to implement templates at our main campus location or at other hospitals. To begin to better understand this, we have spread to HM patients at our main campus, including medically complex patients with technology dependence and/or neurological impairments. Standardized, disease-specific templates most relevant to this population as well as several patient specific templates, for those with frequent readmissions due to medical complexity, have been created and are actively being tested.

In conclusion, in using interventions targeted at standardization of discharge instructions and timely feedback to staff, we saw rapid, dramatic, and sustained improvement in the readability of discharge instructions. Next steps include adaptation and spread to other patient populations and care teams, collaborations with other centers, and assessing the impact of effectively written discharge instructions on patient outcomes, such as adverse drug events, readmission rates, and family experience.

Disclosure

No external funding was secured for this study. Dr. Brady is supported by a Patient-Centered Outcomes Research Mentored Clinical Investigator Award from the Agency for Healthcare Research and Quality, Award Number K08HS023827. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding organizations. The funding organization had no role in the design, preparation, review, or approval of this paper; nor the decision to submit the manuscript for publication. The authors have no financial relationships relevant to this article to disclose.

The transition from hospital to home can be overwhelming for caregivers.¹ Stress of hospitalization coupled with the expectation of families to execute postdischarge care plans make understandable discharge communication critical. Communication failures, inadequate education, absence of caregiver confidence, and lack of clarity regarding care plans may prohibit smooth transitions and lead to adverse postdischarge outcomes.^2-4

Health literacy plays a pivotal role in caregivers’ capacity to navigate the healthcare system, comprehend, and execute care plans. An estimated 90 million Americans have limited health literacy that may negatively impact the provision of safe and quality care^5,6 and be a risk factor for poor outcomes, including increased emergency department (ED) utilization and readmission rates.^7-9 Readability strongly influences the effectiveness of written materials.¹⁰ However, written medical information for patients and families are frequently between the 10^th and 12^th grade reading levels; more than 75% of all pediatric health information is written at or above 10^th grade reading level.¹¹ Government agencies recommend between a 6^th and 8^th grade reading level, for written material;^5,12,13 written discharge instructions have been identified as an important quality metric for hospital-to-home transitions.^14-16

At our center, we found that discharge instructions were commonly written at high reading levels and often incomplete.¹⁷ Poor discharge instructions may contribute to increased readmission rates and unnecessary ED visits.^9,18 Our global aim targeted improved health-literate written information, including understandability and completeness.

Our specific aim was to increase the percentage of discharge instructions written at or below the 7th grade level for hospital medicine (HM) patients on a community hospital pediatric unit from 13% to 80% in 6 months.

Context

The improvement work took place at a 42-bed inpatient pediatric unit at a community satellite of our large, urban, academic hospital. The unit is staffed by medical providers including attendings, fellows, nurse practitioners (NPs), and senior pediatric residents, and had more than 1000 HM discharges in fiscal year 2016. Children with common general pediatric diagnoses are admitted to this service; postsurgical patients are not admitted primarily to the HM service. In Cincinnati, the neighborhood-level high school drop-out rates are as high as 64%.¹⁹ Discharge instructions are written by medical providers in the electronic health record (EHR). A printed copy is given to families and verbally reviewed by a bedside nurse prior to discharge. Quality improvement (QI) efforts focused on discharge instructions were ignited by a prior review of 200 discharge instructions that showed they were difficult to read (median reading level of 10^th grade), poorly understandable (36% of instructions met the threshold of understandability as measured by the Patient Education Materials Assessment Tool²⁰) and were missing key elements of information.¹⁷

Improvement Team

The improvement team consisted of 4 pediatric hospitalists, 2 NPs, 1 nurse educator with health literacy expertise, 1 pediatric resident, 1 fourth-year medical student, 1 QI consultant, and 2 parents who had first-hand experience on the HM service. The improvement team observed the discharge process, including roles of the provider, nurse and family, outlined a process map, and created a modified failure mode and effect analysis.²¹ Prior to our work, discharge instructions written by providers often occurred as a last step, and the content was created as free text or from nonstandardized templates. Key drivers that informed interventions were determined and revised over time (Figure 1). The study was reviewed by our institutional review board and deemed not human subjects research.

Improvement Activities

Key drivers were identified, and interventions were executed using Plan-Do Study-Act cycles.²² The key drivers thought to be critical for the success of the QI efforts were family engagement; standardization of discharge instructions; medical staff engagement; and audit and feedback of data. The corresponding interventions were as follows:

Family Engagement

Understanding the discharge information families desired. Prior to testing, 10 families admitted to the HM service were asked about the discharge experience. We asked families about information they wanted in written discharge instructions: 1) reasons to call your primary doctor or return to the hospital; 2) when to see your primary doctor for a follow-up visit; 3) the phone number to reach your child’s doctor; 4) more information about why your child was admitted; 5) information about new medications; and 6) what to do to help your child continue to recover at home.

Development of templates. We engaged families throughout the process of creating general and disease-specific discharge templates. After a specific template was created and reviewed by the parents on our team, it was sent to members of the institutional Patient Education Committee, which includes parents and local health literacy experts, to review and critique. Feedback from the reviewers was incorporated into the templates prior to use in the EHR.

Postdischarge phone calls.A convenience sample of families discharged from the satellite campus was called 24 to 48 hours after discharge over a 2-week period in January, 2016. A member of our improvement team solicited feedback from families about the quality of the discharge instructions. Families were asked if discharge instructions were reviewed with them prior to going home, if they were given a copy of the instructions, how they would rate the ability to read and use the information, and if there were additional pieces of information that would have improved the instructions.

Standardization of Instructions

Education. A presentation was created and shared with medical providers; it was re-disseminated monthly to new residents rotating onto the service and to the attendings, fellows, and NPs scheduled for shifts during the month. This education continued for the duration of the study. The presentation included the definition of health literacy, scope of the problem, examples of poorly written discharge instructions, and tips on how to write readable and understandable instructions. Laminated cards that included tips on how to write instructions were also placed on work stations.

While the general template was an important first step, the content relied heavily on free text by providers, which could still lead to instructions written at a high reading level. Thus, disease-specific discharge instruction templates were created with prepopulated information that was written at a reading level at or below 7^th grade level (Figure 2). The diseases were prioritized based on the most common diagnoses on our HM service. Each template included information under each of the subheadings noted in the general template. Twelve disease-specific templates were tested and ultimately embedded in the EHR; the general template remained for use when the discharge diagnosis was not covered by a disease-specific template.

Medical Staff Engagement

Previously described tests of change also aimed to enhance staff engagement. These included frequent e-mails, discussion of the QI efforts at specific team meetings, and the creation of visual cues posted at computer work stations, which prompted staff to begin to work on discharge instructions soon after admission.

Audit and Feedback of Data

Weekly phone calls. One team updated clinicians through a regularly scheduled bi-weekly phone conference. The phone conference was established prior to our work and was designed to relay pertinent information to attendings and NPs who work at the satellite hospital. During the phone conferences, clinicians were notified of current performance on discharge instruction readability and specific tests of change for the week. Additionally, providers gave feedback about the improvement efforts. These updates continued for the first 6 months of the project until sustained improvements were observed.

E-mails. Weekly e-mails were sent to all providers scheduled for clinical time at the satellite campus. The e-mail contained information on current tests of change, a list of discharge instruction templates that were available in the EHR, and the annotated run chart illustrating readability levels over time.

Additionally, individual e-mails were sent to each provider after review of the written discharge instructions for the week. Providers were given information on the number of discharge instructions they personally composed, the percentage of those instructions that were written at or below 7^th grade level, and specific feedback on how their written instructions could be improved. We also encouraged feedback from each provider to better identify barriers to achieving our goal.

Study of the Interventions

Baseline data included a review of all instructions for patients discharged from the satellite campus from the end of April 2015 through mid-September 2015. The time period for testing of interventions during the fall and winter months allowed for rapid cycle learning due to higher patient census and predictability of admissions for specific diagnosis (ie, asthma and bronchiolitis). An automated report was generated from the EHR weekly with specific demographics and identifiers for patient discharged over the past 7 days, including patient age, gender, length of stay, discharge diagnosis, and insurance classification. Data was collected during the intervention period via structured review of the discharge instructions in the EHR by the principal investigator or a trained research coordinator. Discharge instructions for medically cleared mental health patients admitted to hospital medicine while awaiting psychiatric bed availability and patients and parents who were non-English speaking were excluded from review. All other instructions for patients discharged from the HM service at our Liberty Campus were included for review.

Measures

Readability, our primary measure of interest, was calculated using the mean score from the following formulas: Flesch Kincaid Grade Level,²³ Simple Measure of Gobbledygook Index,²⁴ Coleman-Liau Index,²⁵ Gunning-Fog Index,²⁶ and Automated Readability Index²⁷ by means of an online platform (https://readability-score.com).²⁸ This platform was chosen because it incorporated a variety of formulas, was user-friendly, and required minimal data cleaning. Each of the readability formulas have been used to assesses readability of health information given to patients and families.^29,30 The threshold of 7th grade is in alignment with our institutional policy for educational materials and with recommendations from several government agencies.^5,12

Analysis

A statistical process control p-chart was used to analyze our primary measure of readability, dichotomized as percent discharge instructions written at or below 7th grade level. Run charts were used to follow mean reading level of discharge instructions and our process measure of percent of discharge instruction written with a general or disease-specific standardized template. Run chart and control chart rules for identifying special cause were used for midline shifts.³¹

The Table includes the demographic and clinical information of patients included in our analyses. Through sequential interventions, the percentage of discharge instructions written at or below 7th grade readability level increased from a mean of 13% to more than 80% in 3 months (Figure 3). Furthermore, the mean was sustained above 90% for 10 months and at 98% for the last 4 months. The use of 1 of the 13 EHR templates increased from 0% to 96% and was associated with the largest impact on the overall improvements (Supplemental Figure 1). Additionally, the average reading level of the discharge instructions decreased from 10th grade to 6th grade level (Supplemental Figure 2).

Qualitative comments from providers about the discharge instructions included:

“Are these [discharge instructions] available at base??  Great resource for interns.”
“These [discharge] instructions make the [discharge] process so easy!!! Love these...”
“Also feel like they have helped my discharge teaching in the room!”

Qualitative comments from families postdischarge included:
“I thought the instructions were very clear and easy to read. I especially thought that highlighting the important areas really helped.”
“I think this form looks great, and I really like the idea of having your child’s name on it.”

Through sequential Plan-Do Study-Act cycles, we increased the percentage of discharge instructions written at or below 7^th grade reading level from 13% to 98%. Our most impactful intervention was the creation and dissemination of standardized disease-specific discharge instruction templates. Our findings complement evidence in the adult and pediatric literature that the use of standardized, disease-specific discharge instruction templates may improve readability of instructions.^32,33 And, while quality improvement efforts have been employed to improve the discharge process for patients,^34-36 this is the first study in the inpatient setting that, to our knowledge, specifically addresses discharge instructions using quality improvement methods.

Our work targeted the critical intersection between individual health literacy, an individual’s capacity to acquire, interpret, and use health information, and the necessary changes needed within our healthcare system to ensure that appropriately written instructions are given to patients and families.^17,37 Our efforts focused on improving discharge instructions answer the call to consider health literacy a modifiable clinical risk factor.³⁷ Furthermore, we address the 6 aims for quality healthcare delivery: 1) safe, timely, efficient and equitable delivery of care through the creation and dissemination of standardized instructions that are written at the appropriate reading level for families to ease hospital-to-home transitions and streamline the workflow of medical providers; 2) effective education of medical providers on health literacy concepts; and 3) family-centeredness through the involvement of families in our QI efforts. While previous QI efforts to improve hospital-to-home transitions have focused on medication reconciliation, communication with primary care physicians, follow-up appointments, and timely discharges of patients, none have specifically focused on the quality of discharge instructions.^34-36

Most physicians do not receive education about how to write information that is readable and understandable; more than half of providers desired more education in this area.³⁸ Furthermore, pediatric providers may overestimate parental health literacy levels,³⁹ which may contribute to variability in the readability of written health materials. While education alone can contribute to a provider’s ability to create readable instructions, we note the improvement after the introduction of disease templates to demonstrate the importance of workflow-integrated higher reliability interventions to sustain improvements.

Our baseline poor readability rates were due to limited knowledge by frontline providers composing the instructions and a system in which an important element for successful hospital-to-home transitions was not tackled until patients were ready for discharge. Streamlining of the discharge process, including the creation of discharge instructions, may lead to improved efficiency, fewer discrepancies, more effective communication, and an enhanced family experience. Moreover, the success of our improvement work was due to key stakeholders, including parents, being a part of the team and the notable buy-in from providers.

Our work was not without limitations. We excluded non-English speaking families from the study. We were unable to measure reading level of our population directly and instead based our goals on national estimates. Our primary measure was readability, which is only 1 piece contributing to quality discharge instructions. Understandability and actionability are also important considerations; ^17,20,29,40 however, improvements in these areas were limited by our design options within the EHR. Our efforts focused on children with common general pediatric diagnoses, and it is unclear how our interventions would generalize to medically complex patients with more volume of information to communicate at discharge and with uncommon diagnoses that are less readily incorporated into standardized templates. Relatedly, our work occurred at the satellite campus of our tertiary care center and may not represent generalizable material or methods to implement templates at our main campus location or at other hospitals. To begin to better understand this, we have spread to HM patients at our main campus, including medically complex patients with technology dependence and/or neurological impairments. Standardized, disease-specific templates most relevant to this population as well as several patient specific templates, for those with frequent readmissions due to medical complexity, have been created and are actively being tested.

In conclusion, in using interventions targeted at standardization of discharge instructions and timely feedback to staff, we saw rapid, dramatic, and sustained improvement in the readability of discharge instructions. Next steps include adaptation and spread to other patient populations and care teams, collaborations with other centers, and assessing the impact of effectively written discharge instructions on patient outcomes, such as adverse drug events, readmission rates, and family experience.

Disclosure

No external funding was secured for this study. Dr. Brady is supported by a Patient-Centered Outcomes Research Mentored Clinical Investigator Award from the Agency for Healthcare Research and Quality, Award Number K08HS023827. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding organizations. The funding organization had no role in the design, preparation, review, or approval of this paper; nor the decision to submit the manuscript for publication. The authors have no financial relationships relevant to this article to disclose.

The transition from hospital to home can be overwhelming for caregivers.¹ Stress of hospitalization coupled with the expectation of families to execute postdischarge care plans make understandable discharge communication critical. Communication failures, inadequate education, absence of caregiver confidence, and lack of clarity regarding care plans may prohibit smooth transitions and lead to adverse postdischarge outcomes.^2-4

Health literacy plays a pivotal role in caregivers’ capacity to navigate the healthcare system, comprehend, and execute care plans. An estimated 90 million Americans have limited health literacy that may negatively impact the provision of safe and quality care^5,6 and be a risk factor for poor outcomes, including increased emergency department (ED) utilization and readmission rates.^7-9 Readability strongly influences the effectiveness of written materials.¹⁰ However, written medical information for patients and families are frequently between the 10^th and 12^th grade reading levels; more than 75% of all pediatric health information is written at or above 10^th grade reading level.¹¹ Government agencies recommend between a 6^th and 8^th grade reading level, for written material;^5,12,13 written discharge instructions have been identified as an important quality metric for hospital-to-home transitions.^14-16

At our center, we found that discharge instructions were commonly written at high reading levels and often incomplete.¹⁷ Poor discharge instructions may contribute to increased readmission rates and unnecessary ED visits.^9,18 Our global aim targeted improved health-literate written information, including understandability and completeness.

Our specific aim was to increase the percentage of discharge instructions written at or below the 7th grade level for hospital medicine (HM) patients on a community hospital pediatric unit from 13% to 80% in 6 months.

Context

The improvement work took place at a 42-bed inpatient pediatric unit at a community satellite of our large, urban, academic hospital. The unit is staffed by medical providers including attendings, fellows, nurse practitioners (NPs), and senior pediatric residents, and had more than 1000 HM discharges in fiscal year 2016. Children with common general pediatric diagnoses are admitted to this service; postsurgical patients are not admitted primarily to the HM service. In Cincinnati, the neighborhood-level high school drop-out rates are as high as 64%.¹⁹ Discharge instructions are written by medical providers in the electronic health record (EHR). A printed copy is given to families and verbally reviewed by a bedside nurse prior to discharge. Quality improvement (QI) efforts focused on discharge instructions were ignited by a prior review of 200 discharge instructions that showed they were difficult to read (median reading level of 10^th grade), poorly understandable (36% of instructions met the threshold of understandability as measured by the Patient Education Materials Assessment Tool²⁰) and were missing key elements of information.¹⁷

Improvement Team

The improvement team consisted of 4 pediatric hospitalists, 2 NPs, 1 nurse educator with health literacy expertise, 1 pediatric resident, 1 fourth-year medical student, 1 QI consultant, and 2 parents who had first-hand experience on the HM service. The improvement team observed the discharge process, including roles of the provider, nurse and family, outlined a process map, and created a modified failure mode and effect analysis.²¹ Prior to our work, discharge instructions written by providers often occurred as a last step, and the content was created as free text or from nonstandardized templates. Key drivers that informed interventions were determined and revised over time (Figure 1). The study was reviewed by our institutional review board and deemed not human subjects research.

Improvement Activities

Key drivers were identified, and interventions were executed using Plan-Do Study-Act cycles.²² The key drivers thought to be critical for the success of the QI efforts were family engagement; standardization of discharge instructions; medical staff engagement; and audit and feedback of data. The corresponding interventions were as follows:

Family Engagement

Understanding the discharge information families desired. Prior to testing, 10 families admitted to the HM service were asked about the discharge experience. We asked families about information they wanted in written discharge instructions: 1) reasons to call your primary doctor or return to the hospital; 2) when to see your primary doctor for a follow-up visit; 3) the phone number to reach your child’s doctor; 4) more information about why your child was admitted; 5) information about new medications; and 6) what to do to help your child continue to recover at home.

Development of templates. We engaged families throughout the process of creating general and disease-specific discharge templates. After a specific template was created and reviewed by the parents on our team, it was sent to members of the institutional Patient Education Committee, which includes parents and local health literacy experts, to review and critique. Feedback from the reviewers was incorporated into the templates prior to use in the EHR.

Postdischarge phone calls.A convenience sample of families discharged from the satellite campus was called 24 to 48 hours after discharge over a 2-week period in January, 2016. A member of our improvement team solicited feedback from families about the quality of the discharge instructions. Families were asked if discharge instructions were reviewed with them prior to going home, if they were given a copy of the instructions, how they would rate the ability to read and use the information, and if there were additional pieces of information that would have improved the instructions.

Standardization of Instructions

Education. A presentation was created and shared with medical providers; it was re-disseminated monthly to new residents rotating onto the service and to the attendings, fellows, and NPs scheduled for shifts during the month. This education continued for the duration of the study. The presentation included the definition of health literacy, scope of the problem, examples of poorly written discharge instructions, and tips on how to write readable and understandable instructions. Laminated cards that included tips on how to write instructions were also placed on work stations.

While the general template was an important first step, the content relied heavily on free text by providers, which could still lead to instructions written at a high reading level. Thus, disease-specific discharge instruction templates were created with prepopulated information that was written at a reading level at or below 7^th grade level (Figure 2). The diseases were prioritized based on the most common diagnoses on our HM service. Each template included information under each of the subheadings noted in the general template. Twelve disease-specific templates were tested and ultimately embedded in the EHR; the general template remained for use when the discharge diagnosis was not covered by a disease-specific template.

Medical Staff Engagement

Previously described tests of change also aimed to enhance staff engagement. These included frequent e-mails, discussion of the QI efforts at specific team meetings, and the creation of visual cues posted at computer work stations, which prompted staff to begin to work on discharge instructions soon after admission.

Audit and Feedback of Data

Weekly phone calls. One team updated clinicians through a regularly scheduled bi-weekly phone conference. The phone conference was established prior to our work and was designed to relay pertinent information to attendings and NPs who work at the satellite hospital. During the phone conferences, clinicians were notified of current performance on discharge instruction readability and specific tests of change for the week. Additionally, providers gave feedback about the improvement efforts. These updates continued for the first 6 months of the project until sustained improvements were observed.

E-mails. Weekly e-mails were sent to all providers scheduled for clinical time at the satellite campus. The e-mail contained information on current tests of change, a list of discharge instruction templates that were available in the EHR, and the annotated run chart illustrating readability levels over time.

Additionally, individual e-mails were sent to each provider after review of the written discharge instructions for the week. Providers were given information on the number of discharge instructions they personally composed, the percentage of those instructions that were written at or below 7^th grade level, and specific feedback on how their written instructions could be improved. We also encouraged feedback from each provider to better identify barriers to achieving our goal.

Study of the Interventions

Baseline data included a review of all instructions for patients discharged from the satellite campus from the end of April 2015 through mid-September 2015. The time period for testing of interventions during the fall and winter months allowed for rapid cycle learning due to higher patient census and predictability of admissions for specific diagnosis (ie, asthma and bronchiolitis). An automated report was generated from the EHR weekly with specific demographics and identifiers for patient discharged over the past 7 days, including patient age, gender, length of stay, discharge diagnosis, and insurance classification. Data was collected during the intervention period via structured review of the discharge instructions in the EHR by the principal investigator or a trained research coordinator. Discharge instructions for medically cleared mental health patients admitted to hospital medicine while awaiting psychiatric bed availability and patients and parents who were non-English speaking were excluded from review. All other instructions for patients discharged from the HM service at our Liberty Campus were included for review.

Measures

Readability, our primary measure of interest, was calculated using the mean score from the following formulas: Flesch Kincaid Grade Level,²³ Simple Measure of Gobbledygook Index,²⁴ Coleman-Liau Index,²⁵ Gunning-Fog Index,²⁶ and Automated Readability Index²⁷ by means of an online platform (https://readability-score.com).²⁸ This platform was chosen because it incorporated a variety of formulas, was user-friendly, and required minimal data cleaning. Each of the readability formulas have been used to assesses readability of health information given to patients and families.^29,30 The threshold of 7th grade is in alignment with our institutional policy for educational materials and with recommendations from several government agencies.^5,12

Analysis

A statistical process control p-chart was used to analyze our primary measure of readability, dichotomized as percent discharge instructions written at or below 7th grade level. Run charts were used to follow mean reading level of discharge instructions and our process measure of percent of discharge instruction written with a general or disease-specific standardized template. Run chart and control chart rules for identifying special cause were used for midline shifts.³¹

The Table includes the demographic and clinical information of patients included in our analyses. Through sequential interventions, the percentage of discharge instructions written at or below 7th grade readability level increased from a mean of 13% to more than 80% in 3 months (Figure 3). Furthermore, the mean was sustained above 90% for 10 months and at 98% for the last 4 months. The use of 1 of the 13 EHR templates increased from 0% to 96% and was associated with the largest impact on the overall improvements (Supplemental Figure 1). Additionally, the average reading level of the discharge instructions decreased from 10th grade to 6th grade level (Supplemental Figure 2).

Qualitative comments from providers about the discharge instructions included:

“Are these [discharge instructions] available at base??  Great resource for interns.”
“These [discharge] instructions make the [discharge] process so easy!!! Love these...”
“Also feel like they have helped my discharge teaching in the room!”

Qualitative comments from families postdischarge included:
“I thought the instructions were very clear and easy to read. I especially thought that highlighting the important areas really helped.”
“I think this form looks great, and I really like the idea of having your child’s name on it.”

Through sequential Plan-Do Study-Act cycles, we increased the percentage of discharge instructions written at or below 7^th grade reading level from 13% to 98%. Our most impactful intervention was the creation and dissemination of standardized disease-specific discharge instruction templates. Our findings complement evidence in the adult and pediatric literature that the use of standardized, disease-specific discharge instruction templates may improve readability of instructions.^32,33 And, while quality improvement efforts have been employed to improve the discharge process for patients,^34-36 this is the first study in the inpatient setting that, to our knowledge, specifically addresses discharge instructions using quality improvement methods.

Our work targeted the critical intersection between individual health literacy, an individual’s capacity to acquire, interpret, and use health information, and the necessary changes needed within our healthcare system to ensure that appropriately written instructions are given to patients and families.^17,37 Our efforts focused on improving discharge instructions answer the call to consider health literacy a modifiable clinical risk factor.³⁷ Furthermore, we address the 6 aims for quality healthcare delivery: 1) safe, timely, efficient and equitable delivery of care through the creation and dissemination of standardized instructions that are written at the appropriate reading level for families to ease hospital-to-home transitions and streamline the workflow of medical providers; 2) effective education of medical providers on health literacy concepts; and 3) family-centeredness through the involvement of families in our QI efforts. While previous QI efforts to improve hospital-to-home transitions have focused on medication reconciliation, communication with primary care physicians, follow-up appointments, and timely discharges of patients, none have specifically focused on the quality of discharge instructions.^34-36

Most physicians do not receive education about how to write information that is readable and understandable; more than half of providers desired more education in this area.³⁸ Furthermore, pediatric providers may overestimate parental health literacy levels,³⁹ which may contribute to variability in the readability of written health materials. While education alone can contribute to a provider’s ability to create readable instructions, we note the improvement after the introduction of disease templates to demonstrate the importance of workflow-integrated higher reliability interventions to sustain improvements.

Our baseline poor readability rates were due to limited knowledge by frontline providers composing the instructions and a system in which an important element for successful hospital-to-home transitions was not tackled until patients were ready for discharge. Streamlining of the discharge process, including the creation of discharge instructions, may lead to improved efficiency, fewer discrepancies, more effective communication, and an enhanced family experience. Moreover, the success of our improvement work was due to key stakeholders, including parents, being a part of the team and the notable buy-in from providers.

Our work was not without limitations. We excluded non-English speaking families from the study. We were unable to measure reading level of our population directly and instead based our goals on national estimates. Our primary measure was readability, which is only 1 piece contributing to quality discharge instructions. Understandability and actionability are also important considerations; ^17,20,29,40 however, improvements in these areas were limited by our design options within the EHR. Our efforts focused on children with common general pediatric diagnoses, and it is unclear how our interventions would generalize to medically complex patients with more volume of information to communicate at discharge and with uncommon diagnoses that are less readily incorporated into standardized templates. Relatedly, our work occurred at the satellite campus of our tertiary care center and may not represent generalizable material or methods to implement templates at our main campus location or at other hospitals. To begin to better understand this, we have spread to HM patients at our main campus, including medically complex patients with technology dependence and/or neurological impairments. Standardized, disease-specific templates most relevant to this population as well as several patient specific templates, for those with frequent readmissions due to medical complexity, have been created and are actively being tested.

In conclusion, in using interventions targeted at standardization of discharge instructions and timely feedback to staff, we saw rapid, dramatic, and sustained improvement in the readability of discharge instructions. Next steps include adaptation and spread to other patient populations and care teams, collaborations with other centers, and assessing the impact of effectively written discharge instructions on patient outcomes, such as adverse drug events, readmission rates, and family experience.

Disclosure

No external funding was secured for this study. Dr. Brady is supported by a Patient-Centered Outcomes Research Mentored Clinical Investigator Award from the Agency for Healthcare Research and Quality, Award Number K08HS023827. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding organizations. The funding organization had no role in the design, preparation, review, or approval of this paper; nor the decision to submit the manuscript for publication. The authors have no financial relationships relevant to this article to disclose.

The average American adult reads at an 8th-grade level.¹ Limited general literacy can affect health literacy, which is defined as the “degree to which individuals have the capacity to obtain, process and understand basic health information and services needed to make appropriate health decisions.”^2,3 Adults with limited health literacy are at risk for poorer outcomes, including overuse of the emergency department and lower adherence to preventive care recommendations.⁴

Children transitioning from hospital to home depend on their adult caregivers (and their caregivers’ health literacy) to carry out discharge instructions. During the immediate postdischarge period, complex care needs can involve new or changed medications, follow-up instructions, home care instructions, and suggestions regarding when and why to seek additional care.

The discharge education provided to patients in the hospital is often subpar because of lack of standardization and divided responsibility among providers.⁵ Communication of vital information to patients with low health literacy has been noted to be particularly poor,⁶ as many patient education materials are written at 10th-, 11th-, and 12th-grade reading levels.⁴ Evidence supports providing materials written at 6th-grade level or lower to increase comprehension.⁷ Several studies have evaluated the quality and readability of discharge instructions for hospitalized adults,^8,9 and one study found a link between poorly written instructions for adult patients and readmission risk.¹⁰ Less is known about readability in pediatrics, in which education may be more important for families of children most commonly hospitalized for acute illness.

We conducted a study to describe readability levels, understandability scores, and completeness of written instructions given to families at hospital discharge.

Study Design and Setting

In this study, we performed a cross-sectional review of discharge instructions within electronic health records at Cincinnati Children’s Hospital Medical Center (CCHMC). The study was reviewed and approved by CCHMC’s Institutional Review Board. Charts were randomly selected from all hospital medicine service discharges during two 3-month periods of high patient volume: January-March 2014 and January-March 2015.

CCHMC is a large urban academic referral center that is the sole provider of general, subspecialty, and critical pediatric inpatient care for a large geographical area. CCHMC, which has 600 beds, provides cares for many children who live in impoverished settings. Its hospital medicine service consists of 4 teams that care for approximately 7000 children hospitalized with general pediatric illnesses each year. Each team consists of 5 or 6 pediatric residents supervised by a hospital medicine attending.

Providers, most commonly pediatric interns, generate discharge instructions in electronic health records. In this nonautomated process, they use free-text or nonstandardized templates to create content. At discharge, instructions are printed as part of the postvisit summary, which includes updates on medications and scheduled follow-up appointments. Bedside nurses verbally review the instructions with families and provide printed copies for home use.

Data Collection and Analysis

A random sequence generator was used to select charts for review. Instructions written in a language other than English were excluded. Written discharge instructions and clinical information, including age, sex, primary diagnosis, insurance type, number of discharge medications, number of scheduled appointments at discharge, and hospital length of stay, were abstracted from electronic health records and anonymized before analysis. The primary outcomes assessed were discharge instruction readability, understandability, and completeness. Readability was calculated with Fry Readability Scale (FRS) scores,¹¹ which range from 1 to 17 and correspond to reading levels (score 1 = 1st-grade reading level). Health literacy experts have used the FRS to assess readability in health care environments.¹²

Understandability was measured with the Patient Education Materials Assessment Tool (PEMAT), a validated scoring system provided by the Agency for Healthcare Research and Quality.¹³ The PEMAT measures the understandability of print materials on a scale ranging from 0% to 100%. Higher scores indicate increased understandability, and scores under 70% indicate instructions are difficult to understand.

Although recent efforts have focused on the development of quality metrics for hospital-to-home transitions of pediatric patients,¹⁴ during our study there were no standard items to include in pediatric discharge instructions. Five criteria for completeness were determined by consensus of 3 pediatric hospital medicine faculty and were informed by qualitative results of work performed at our institution—work in which families noted challenges with information overload and a desire for pertinent and usable information that would enhance caregiver confidence and discharge preparedness.¹⁵ The criteria included statement of diagnosis, description of diagnosis, signs and symptoms indicative of the need for escalation of care (warning signs), the person caregivers should call if worried, and contact information for the primary care provider, subspecialist, and/or emergency department. Each set of discharge instructions was manually evaluated for completeness (presence of each individual component, number of components present, presence of all components). All charts were scored by the same investigator. A convenience sample of 20 charts was evaluated by a different investigator to ensure rating parameters were clear and classification was consistent (defined as perfect agreement). If the primary rater was undecided on a discharge instruction score, the secondary rater rated the instruction, and consensus was reached.

Means, medians, and ranges were calculated to enumerate the distribution of readability levels, understandability scores, and completeness of discharge instructions. Instructions were classified as readable if the FRS score was 6 or under, as understandable if the PEMAT score was under 70%, and as complete if all 5 criteria were satisfied. Descriptive statistics were generated for all demographic and clinical variables.

Of the study period’s 3819 discharges, 200 were randomly selected for review. Table 1 lists the demographic and clinical information of patients included in the analyses. Median FRS score was 10, indicating a 10th-grade reading level (interquartile range, 8-12; range, 1-13) (Table 2). Only 14 (7%) of 200 discharge instructions had a score of 6 or under. Median PEMAT understandability score was 73% (interquartile range, 64%-82%), and 36% of instructions had a PEMAT score under 70%. No instruction satisfied all 5 of the defined characteristics of complete discharge instructions (Table 2).

To our knowledge, this is the first study of the readability, understandability, and completeness of discharge instructions in a pediatric population. We found that the majority of discharge instruction readability levels were 10th grade or higher, that many instructions were difficult to understand, and that important information was missing from many instructions.

Discharge instruction readability levels were higher than the literacy level of many families in surrounding communities. The high school dropout rates in Cincinnati are staggering; they range from 22% to 64% in the 10 neighborhoods with the largest proportion of residents not completing high school.¹⁶ However, such findings are not unique to Cincinnati; low literacy is prevalent throughout the United States. Caregivers with limited literacy skills may struggle to navigate complex health systems, understand medical instructions and anticipatory guidance, perform child care and self-care tasks, and understand issues related to consent, medical authorization, and risk communication.¹⁷

Although readability is important, other factors also correlate with comprehension and execution of discharge tasks.¹⁸ Information must be understandable, or presented in a way that makes sense and can inform appropriate action. In many cases in our study, instructions were incomplete, despite previous investigators’ emphasizing caregivers’ desire and need for written instructions that are complete, informative, and inclusive of clearly outlined contingency plans.^15,19 In addition, families may differ in the level of support needed after discharge; standardizing elements and including families in the development of discharge instructions may improve communication.⁸

This study had several limitations. First, the discharge instructions randomly selected for review were all written during the winter months. As the census on the hospital medicine teams is particularly high during that time, authors with competing responsibilities may not have had enough time to write effective discharge instructions then. We selected the winter period in order to capture real-world instructions written during a busy clinical time, when providers care for a high volume of patients. Second, caregiver health literacy and English-language proficiency were not assessed, and information regarding caregivers’ race/ethnicity, educational attainment, and socioeconomic status was unavailable. Third, interrater agreement was not formally evaluated. Fourth, this was a single-center study with results that may not be generalizable.

In conclusion, discharge instructions for pediatric patients are often difficult to read and understand, and incomplete. Efforts to address these communication gaps—including educational initiatives for physician trainees focused on health literacy, and quality improvement work directed at standardization and creation of readable, understandable, and complete discharge instructions—are crucial in providing safe, high-value care. Researchers need to evaluate the relationship between discharge instruction quality and outcomes, including unplanned office visits, emergency department visits, and readmissions.

Disclosure

Nothing to report.

The average American adult reads at an 8th-grade level.¹ Limited general literacy can affect health literacy, which is defined as the “degree to which individuals have the capacity to obtain, process and understand basic health information and services needed to make appropriate health decisions.”^2,3 Adults with limited health literacy are at risk for poorer outcomes, including overuse of the emergency department and lower adherence to preventive care recommendations.⁴

Children transitioning from hospital to home depend on their adult caregivers (and their caregivers’ health literacy) to carry out discharge instructions. During the immediate postdischarge period, complex care needs can involve new or changed medications, follow-up instructions, home care instructions, and suggestions regarding when and why to seek additional care.

The discharge education provided to patients in the hospital is often subpar because of lack of standardization and divided responsibility among providers.⁵ Communication of vital information to patients with low health literacy has been noted to be particularly poor,⁶ as many patient education materials are written at 10th-, 11th-, and 12th-grade reading levels.⁴ Evidence supports providing materials written at 6th-grade level or lower to increase comprehension.⁷ Several studies have evaluated the quality and readability of discharge instructions for hospitalized adults,^8,9 and one study found a link between poorly written instructions for adult patients and readmission risk.¹⁰ Less is known about readability in pediatrics, in which education may be more important for families of children most commonly hospitalized for acute illness.

We conducted a study to describe readability levels, understandability scores, and completeness of written instructions given to families at hospital discharge.

Study Design and Setting

In this study, we performed a cross-sectional review of discharge instructions within electronic health records at Cincinnati Children’s Hospital Medical Center (CCHMC). The study was reviewed and approved by CCHMC’s Institutional Review Board. Charts were randomly selected from all hospital medicine service discharges during two 3-month periods of high patient volume: January-March 2014 and January-March 2015.

CCHMC is a large urban academic referral center that is the sole provider of general, subspecialty, and critical pediatric inpatient care for a large geographical area. CCHMC, which has 600 beds, provides cares for many children who live in impoverished settings. Its hospital medicine service consists of 4 teams that care for approximately 7000 children hospitalized with general pediatric illnesses each year. Each team consists of 5 or 6 pediatric residents supervised by a hospital medicine attending.

Providers, most commonly pediatric interns, generate discharge instructions in electronic health records. In this nonautomated process, they use free-text or nonstandardized templates to create content. At discharge, instructions are printed as part of the postvisit summary, which includes updates on medications and scheduled follow-up appointments. Bedside nurses verbally review the instructions with families and provide printed copies for home use.

Data Collection and Analysis

A random sequence generator was used to select charts for review. Instructions written in a language other than English were excluded. Written discharge instructions and clinical information, including age, sex, primary diagnosis, insurance type, number of discharge medications, number of scheduled appointments at discharge, and hospital length of stay, were abstracted from electronic health records and anonymized before analysis. The primary outcomes assessed were discharge instruction readability, understandability, and completeness. Readability was calculated with Fry Readability Scale (FRS) scores,¹¹ which range from 1 to 17 and correspond to reading levels (score 1 = 1st-grade reading level). Health literacy experts have used the FRS to assess readability in health care environments.¹²

Understandability was measured with the Patient Education Materials Assessment Tool (PEMAT), a validated scoring system provided by the Agency for Healthcare Research and Quality.¹³ The PEMAT measures the understandability of print materials on a scale ranging from 0% to 100%. Higher scores indicate increased understandability, and scores under 70% indicate instructions are difficult to understand.

Although recent efforts have focused on the development of quality metrics for hospital-to-home transitions of pediatric patients,¹⁴ during our study there were no standard items to include in pediatric discharge instructions. Five criteria for completeness were determined by consensus of 3 pediatric hospital medicine faculty and were informed by qualitative results of work performed at our institution—work in which families noted challenges with information overload and a desire for pertinent and usable information that would enhance caregiver confidence and discharge preparedness.¹⁵ The criteria included statement of diagnosis, description of diagnosis, signs and symptoms indicative of the need for escalation of care (warning signs), the person caregivers should call if worried, and contact information for the primary care provider, subspecialist, and/or emergency department. Each set of discharge instructions was manually evaluated for completeness (presence of each individual component, number of components present, presence of all components). All charts were scored by the same investigator. A convenience sample of 20 charts was evaluated by a different investigator to ensure rating parameters were clear and classification was consistent (defined as perfect agreement). If the primary rater was undecided on a discharge instruction score, the secondary rater rated the instruction, and consensus was reached.

Means, medians, and ranges were calculated to enumerate the distribution of readability levels, understandability scores, and completeness of discharge instructions. Instructions were classified as readable if the FRS score was 6 or under, as understandable if the PEMAT score was under 70%, and as complete if all 5 criteria were satisfied. Descriptive statistics were generated for all demographic and clinical variables.

Of the study period’s 3819 discharges, 200 were randomly selected for review. Table 1 lists the demographic and clinical information of patients included in the analyses. Median FRS score was 10, indicating a 10th-grade reading level (interquartile range, 8-12; range, 1-13) (Table 2). Only 14 (7%) of 200 discharge instructions had a score of 6 or under. Median PEMAT understandability score was 73% (interquartile range, 64%-82%), and 36% of instructions had a PEMAT score under 70%. No instruction satisfied all 5 of the defined characteristics of complete discharge instructions (Table 2).

To our knowledge, this is the first study of the readability, understandability, and completeness of discharge instructions in a pediatric population. We found that the majority of discharge instruction readability levels were 10th grade or higher, that many instructions were difficult to understand, and that important information was missing from many instructions.

Discharge instruction readability levels were higher than the literacy level of many families in surrounding communities. The high school dropout rates in Cincinnati are staggering; they range from 22% to 64% in the 10 neighborhoods with the largest proportion of residents not completing high school.¹⁶ However, such findings are not unique to Cincinnati; low literacy is prevalent throughout the United States. Caregivers with limited literacy skills may struggle to navigate complex health systems, understand medical instructions and anticipatory guidance, perform child care and self-care tasks, and understand issues related to consent, medical authorization, and risk communication.¹⁷

Although readability is important, other factors also correlate with comprehension and execution of discharge tasks.¹⁸ Information must be understandable, or presented in a way that makes sense and can inform appropriate action. In many cases in our study, instructions were incomplete, despite previous investigators’ emphasizing caregivers’ desire and need for written instructions that are complete, informative, and inclusive of clearly outlined contingency plans.^15,19 In addition, families may differ in the level of support needed after discharge; standardizing elements and including families in the development of discharge instructions may improve communication.⁸

This study had several limitations. First, the discharge instructions randomly selected for review were all written during the winter months. As the census on the hospital medicine teams is particularly high during that time, authors with competing responsibilities may not have had enough time to write effective discharge instructions then. We selected the winter period in order to capture real-world instructions written during a busy clinical time, when providers care for a high volume of patients. Second, caregiver health literacy and English-language proficiency were not assessed, and information regarding caregivers’ race/ethnicity, educational attainment, and socioeconomic status was unavailable. Third, interrater agreement was not formally evaluated. Fourth, this was a single-center study with results that may not be generalizable.

In conclusion, discharge instructions for pediatric patients are often difficult to read and understand, and incomplete. Efforts to address these communication gaps—including educational initiatives for physician trainees focused on health literacy, and quality improvement work directed at standardization and creation of readable, understandable, and complete discharge instructions—are crucial in providing safe, high-value care. Researchers need to evaluate the relationship between discharge instruction quality and outcomes, including unplanned office visits, emergency department visits, and readmissions.

Disclosure

Nothing to report.

The average American adult reads at an 8th-grade level.¹ Limited general literacy can affect health literacy, which is defined as the “degree to which individuals have the capacity to obtain, process and understand basic health information and services needed to make appropriate health decisions.”^2,3 Adults with limited health literacy are at risk for poorer outcomes, including overuse of the emergency department and lower adherence to preventive care recommendations.⁴

Children transitioning from hospital to home depend on their adult caregivers (and their caregivers’ health literacy) to carry out discharge instructions. During the immediate postdischarge period, complex care needs can involve new or changed medications, follow-up instructions, home care instructions, and suggestions regarding when and why to seek additional care.

The discharge education provided to patients in the hospital is often subpar because of lack of standardization and divided responsibility among providers.⁵ Communication of vital information to patients with low health literacy has been noted to be particularly poor,⁶ as many patient education materials are written at 10th-, 11th-, and 12th-grade reading levels.⁴ Evidence supports providing materials written at 6th-grade level or lower to increase comprehension.⁷ Several studies have evaluated the quality and readability of discharge instructions for hospitalized adults,^8,9 and one study found a link between poorly written instructions for adult patients and readmission risk.¹⁰ Less is known about readability in pediatrics, in which education may be more important for families of children most commonly hospitalized for acute illness.

We conducted a study to describe readability levels, understandability scores, and completeness of written instructions given to families at hospital discharge.

Study Design and Setting

In this study, we performed a cross-sectional review of discharge instructions within electronic health records at Cincinnati Children’s Hospital Medical Center (CCHMC). The study was reviewed and approved by CCHMC’s Institutional Review Board. Charts were randomly selected from all hospital medicine service discharges during two 3-month periods of high patient volume: January-March 2014 and January-March 2015.

CCHMC is a large urban academic referral center that is the sole provider of general, subspecialty, and critical pediatric inpatient care for a large geographical area. CCHMC, which has 600 beds, provides cares for many children who live in impoverished settings. Its hospital medicine service consists of 4 teams that care for approximately 7000 children hospitalized with general pediatric illnesses each year. Each team consists of 5 or 6 pediatric residents supervised by a hospital medicine attending.

Providers, most commonly pediatric interns, generate discharge instructions in electronic health records. In this nonautomated process, they use free-text or nonstandardized templates to create content. At discharge, instructions are printed as part of the postvisit summary, which includes updates on medications and scheduled follow-up appointments. Bedside nurses verbally review the instructions with families and provide printed copies for home use.

Data Collection and Analysis

A random sequence generator was used to select charts for review. Instructions written in a language other than English were excluded. Written discharge instructions and clinical information, including age, sex, primary diagnosis, insurance type, number of discharge medications, number of scheduled appointments at discharge, and hospital length of stay, were abstracted from electronic health records and anonymized before analysis. The primary outcomes assessed were discharge instruction readability, understandability, and completeness. Readability was calculated with Fry Readability Scale (FRS) scores,¹¹ which range from 1 to 17 and correspond to reading levels (score 1 = 1st-grade reading level). Health literacy experts have used the FRS to assess readability in health care environments.¹²

Understandability was measured with the Patient Education Materials Assessment Tool (PEMAT), a validated scoring system provided by the Agency for Healthcare Research and Quality.¹³ The PEMAT measures the understandability of print materials on a scale ranging from 0% to 100%. Higher scores indicate increased understandability, and scores under 70% indicate instructions are difficult to understand.

Although recent efforts have focused on the development of quality metrics for hospital-to-home transitions of pediatric patients,¹⁴ during our study there were no standard items to include in pediatric discharge instructions. Five criteria for completeness were determined by consensus of 3 pediatric hospital medicine faculty and were informed by qualitative results of work performed at our institution—work in which families noted challenges with information overload and a desire for pertinent and usable information that would enhance caregiver confidence and discharge preparedness.¹⁵ The criteria included statement of diagnosis, description of diagnosis, signs and symptoms indicative of the need for escalation of care (warning signs), the person caregivers should call if worried, and contact information for the primary care provider, subspecialist, and/or emergency department. Each set of discharge instructions was manually evaluated for completeness (presence of each individual component, number of components present, presence of all components). All charts were scored by the same investigator. A convenience sample of 20 charts was evaluated by a different investigator to ensure rating parameters were clear and classification was consistent (defined as perfect agreement). If the primary rater was undecided on a discharge instruction score, the secondary rater rated the instruction, and consensus was reached.

Means, medians, and ranges were calculated to enumerate the distribution of readability levels, understandability scores, and completeness of discharge instructions. Instructions were classified as readable if the FRS score was 6 or under, as understandable if the PEMAT score was under 70%, and as complete if all 5 criteria were satisfied. Descriptive statistics were generated for all demographic and clinical variables.

Of the study period’s 3819 discharges, 200 were randomly selected for review. Table 1 lists the demographic and clinical information of patients included in the analyses. Median FRS score was 10, indicating a 10th-grade reading level (interquartile range, 8-12; range, 1-13) (Table 2). Only 14 (7%) of 200 discharge instructions had a score of 6 or under. Median PEMAT understandability score was 73% (interquartile range, 64%-82%), and 36% of instructions had a PEMAT score under 70%. No instruction satisfied all 5 of the defined characteristics of complete discharge instructions (Table 2).

To our knowledge, this is the first study of the readability, understandability, and completeness of discharge instructions in a pediatric population. We found that the majority of discharge instruction readability levels were 10th grade or higher, that many instructions were difficult to understand, and that important information was missing from many instructions.

Discharge instruction readability levels were higher than the literacy level of many families in surrounding communities. The high school dropout rates in Cincinnati are staggering; they range from 22% to 64% in the 10 neighborhoods with the largest proportion of residents not completing high school.¹⁶ However, such findings are not unique to Cincinnati; low literacy is prevalent throughout the United States. Caregivers with limited literacy skills may struggle to navigate complex health systems, understand medical instructions and anticipatory guidance, perform child care and self-care tasks, and understand issues related to consent, medical authorization, and risk communication.¹⁷

Although readability is important, other factors also correlate with comprehension and execution of discharge tasks.¹⁸ Information must be understandable, or presented in a way that makes sense and can inform appropriate action. In many cases in our study, instructions were incomplete, despite previous investigators’ emphasizing caregivers’ desire and need for written instructions that are complete, informative, and inclusive of clearly outlined contingency plans.^15,19 In addition, families may differ in the level of support needed after discharge; standardizing elements and including families in the development of discharge instructions may improve communication.⁸

This study had several limitations. First, the discharge instructions randomly selected for review were all written during the winter months. As the census on the hospital medicine teams is particularly high during that time, authors with competing responsibilities may not have had enough time to write effective discharge instructions then. We selected the winter period in order to capture real-world instructions written during a busy clinical time, when providers care for a high volume of patients. Second, caregiver health literacy and English-language proficiency were not assessed, and information regarding caregivers’ race/ethnicity, educational attainment, and socioeconomic status was unavailable. Third, interrater agreement was not formally evaluated. Fourth, this was a single-center study with results that may not be generalizable.

In conclusion, discharge instructions for pediatric patients are often difficult to read and understand, and incomplete. Efforts to address these communication gaps—including educational initiatives for physician trainees focused on health literacy, and quality improvement work directed at standardization and creation of readable, understandable, and complete discharge instructions—are crucial in providing safe, high-value care. Researchers need to evaluate the relationship between discharge instruction quality and outcomes, including unplanned office visits, emergency department visits, and readmissions.

Disclosure

Nothing to report.