Daniel D. Dressler, MD

In the United States, hospitalized patients are at risk of acquiring health careassociated infections that increase morbidity, mortality, length of hospital stay, and cost of care.1 If a health careassociated infection is caused by an antimicrobial‐resistant pathogen, treatment efforts may be further complicated.2, 3 With the decreasing effectiveness of antimicrobials and suboptimal adherence to certain infection control measures, new and multifaceted prevention strategies are necessary to address the problem of health careassociated infections and antimicrobial resistance.410

One strategy that hospitals can use to reduce the incidence of health careassociated infections and antimicrobial resistance is implementation of quality improvement programs. These programs require clinicians to employ techniques, such as root cause analysis (RCA), which investigates contributing factors to an event to prevent reoccurrence, and healthcare failure mode effects analysis (HFMEA), which applies a systematic method of identifying and preventing problems before they occur.1113 Programs and strategies such as these require leadership and adoption within the hospital. Because of their availability and specialized role in the hospital setting, hospitalists are in a unique position to promote and uphold quality improvement efforts.1417 Professional societies, health care organizations, and governmental agencies can play a role in engaging this group of physicians in improving the quality of patient care in hospitals by providing educational programs and materials.18

In 2004, the Society of Hospital Medicine (SHM) collaborated with the Centers for Disease Control and Prevention (CDC) to develop a quality improvement tool kit to reduce antimicrobial resistance and health careassociated infections. The tool kit was based on the CDC's Campaign to Prevent Antimicrobial Resistance in Healthcare Settings (Campaign), an educational program targeted at clinicians.19 The SHM/CDC tool kit contained campaign materials, a set of slides about quality improvement, worksheets, and additional materials such as infection control policies and guidelines to supplement a 90‐minute workshop consisting of didactic lectures about antimicrobial resistance, quality improvement initiatives, RCA, and HFMEA; a lecture and case study about intravascular catheter‐related infections; and small‐group activity and discussion. The complete toolkit is now available online via the SHM Antimicrobial Resistance Resource Room at http://www.hospitalmedicine.org/AM/Template.cfm?Section=Home&Template=/CM/HTMLDisplay.cfm&ContentID=7542.

The purpose of the workshop was to present the tool kit and increase hospitalists' knowledge and awareness about antimicrobial resistance, health careassociated infections, and quality improvement programs. We assessed the workshop participants' familiarity with the Campaign prior to the workshop, perceptions of antimicrobial resistance, knowledge gained as a result of the workshop, and opinions about the usefulness of the workshop.

METHODS

Data were collected from pretests and posttests administered to participants of one of the SHM workshops in May, June, or July 2005 in Denver, Colorado; Boston, Massachusetts; or Portland, Oregon. One SHM physician leader (D.D.D., coauthor of this article) presented all 3 workshops. The workshops were advertised by SHM using E‐mail to local chapter members. Individual sites used a variety of methods to encourage their hospitalists to attend, and participants were provided a complimentary dinner.

Prior to each workshop, participants completed a 10‐question pretest that had been pilot‐tested by hospitalists in other cities. The pretest assessed demographics; perceptions of the problem of antimicrobial resistance using a Likert scale; familiarity with the Campaign; and knowledge of common infection sites, RCA, HFMEA, and antimicrobial resistance prevention measures.

Immediately following each workshop, a 13‐question posttest was administered to participants. This posttest evaluated the workshop and materials using Likert scales, asked for suggestions for future programming using open‐ended questions, and repeated pretest questions to assess changes in perceptions and knowledge.

Data were entered into an Excel spreadsheet and analyzed using descriptive statistics and t tests to compare pre‐ and posttest changes in knowledge. Likert data assessing perceptions were dichotomized into strongly agree versus all other scale responses. Qualitative open‐ended responses were categorized by theme.

RESULTS

A total of 69 SHM members attended the workshops. Of the 69 participants, 65 completed the pretest, 53 completed the posttest, and 50 completed both the pre‐ and the posttests. Only participants who completed both the pretest and the posttest were included in the analyses (n = 21, Denver; n = 11, Boston; n = 18, Portland). Of the 50 participants who completed both the pre‐ and posttests, 44 (88%) classified themselves as hospitalists in practices ranging from 2 to more than 25 physicians. Participants averaged 9.2 years (range = 1‐27 years) in practice and 4.9 years (range = 1‐10 years) as practicing hospitalists, with no significant differences between the 3 groups. Only 17 participants (34%) were familiar with the Campaign prior to the workshop, and there was no significant variation between the 3 workshops. Those familiar with the Campaign had heard about or received the educational materials from colleagues (n = 5), their facilities (n = 4), professional journals (n = 4), medical conferences (n = 4), or the CDC or SHM websites (n = 4).

Overall, most participants strongly agreed with the statement that antimicrobial resistance was a problem nationally, institutionally, and within their individual practices (Table 1). These perceptions did not significantly differ between the pretest and the posttest. However, statistically significant differences were found when comparing perceptions of the problem of antimicrobial resistance at the national, institutional, and practice levels; more participants strongly agreed that antimicrobial resistance was a problem nationally than within their institutions (pretest, P = .01; posttest, P = .04) or within their practices (pretest, P < .0001; posttest, P = .01).

On the knowledge‐based questions, the overall average test score was 48% on the pretest and 63% on the posttest (P < .0001), with scores varying by question (Table 2). For example, knowledge of quality improvement initiatives/HFMEA was low (an average of 10% correct on the pretest, 48% on the posttest) compared with knowledge about the key prevention strategies from the Campaign to Prevent Antimicrobial Resistance (average of 94% correct on the pretest, 98% on the posttest). Furthermore, scores also varied by workshop location. On the pretest, participants in Boston and Portland scored higher (both 53%) than Denver participants (40%). On the posttest, Portland participants scored the highest (78%) followed by Boston participants (64%) and then Denver participants (50%). Boston and Denver participants differed significantly on pretest knowledge score (P = .04) and Portland and Denver participants differed significantly on posttest knowledge score (P < .0001).

Overall, 43 participants (85%) rated the workshop as either very good or excellent. All but 1 participant (n = 49, 98%) would encourage a colleague to attend the workshop, giving reasons such as that the workshop outlined a major program in delivering good and safe care, offered great information on antimicrobial resistance and methods of quality improvement systems implementation, assisted in find[ing] new tools for improving hospital practice, and addressed a significant factor in hospitals related to morbidity [and] mortality. When asked for general comments about the workshop and suggestions for future improvements, participants requested more direction, more detail, more discussion, specific examples of antimicrobial resistance, and protocols and processes for implementing quality improvement programs. On a scale from 1 (not useful) to 5 (essential), participants rated the usefulness of each workshop segment: intravascular catheter‐related infections lecture and case study (x̄ = 4.3, range = 3‐5), quality improvement initiatives lecture (x̄ = 4.1, range = 2‐5), background on antimicrobial resistance (x̄ = 3.9, range = 2‐5), RCA lecture (x̄ = 3.9, range = 2‐5), HFMEA lecture (x̄ = 3.8, range = 2‐5), and small‐group discussion (x̄ = 3.4, range = 2‐5). These ratings did not vary significantly between the 3 groups.

CONCLUSIONS

To address antimicrobial resistance and health careassociated infections in the hospital setting, the SHM and CDC developed a tool kit and presented a quality improvement workshop to hospitalists in 3 U.S. cities. Overall, the participants scored significantly higher on the knowledge‐based questions on the posttest than on the pretest, indicating that knowledge improved as a result of the workshop. By providing a format that combined didactic lectures with case‐based education, small‐group activities, and discussion, the SHM workshop may have optimized its ability to increase knowledge, similar to the findings in previous research.2021

There were no significant differences between the 3 groups in years of practice, perceptions of the problem, and overall evaluation of the workshop. However, differences were found in knowledge gained as a result of the workshop. For example, the Denver group scored lower on the knowledge‐based questions than did the Boston group on the pretest and the Portland group on the posttest, indicating that knowledge and learning styles may differ by location. These differences may be attributed to variations in hospital environments, hospital‐based educational programs, or medical school and residency training. Differences like these may impact the effectiveness of a program and should be a consideration in the program development process, especially when a program is national in scope, like the CDC's Campaign to Prevent Antimicrobial Resistance in Healthcare Settings. In addition, more than 90% of participants correctly identified key prevention strategies of the Campaign, whereas only 34% were familiar with the Campaign itself prior to the workshop. This result may be a result of the key prevention strategies of the Campaign being derived from well‐established and ‐recognized evidence‐based best practices for patient safety and care.

Although knowledge changed as a result of the workshop, overall perceptions of the problem of antimicrobial resistance did not change significantly from pretest to posttest. It is possible this is because changes in perception require a different or more intensive educational approach. This result also may reflect the initial levels of agreement on the pretest, the measurement instrument itself, and/or the inability to detect differences because of the small number of participants.

Difference did exist in perceptions of the problem of antimicrobial resistance at the national, institutional, and practice levels. Antimicrobial resistance was perceived to be a greater problem on the national level than on the institutional and practice levels. Other studies also have found that clinicians more strongly agree that antimicrobial resistance is a problem nationally than within their institutions and practices.2224 When antimicrobial resistance is not perceived as a problem within institutions and practices, physicians may be less likely to overcome the barriers to following recommended infection prevention guidelines or to implementing quality improvement projects.4 Therefore, educational and intervention efforts like this workshop should address hospitalists' perceptions of the problem of antimicrobial resistance on the individual level as a first step in motivating them to engage in quality improvement.

Although participants' knowledge scores increased from pretest to posttest, gaps in knowledge remained, as indicated by the significantly improved but low overall posttest scores related to RCA and HFMEA. As hospitalists are in a unique position to promote quality improvement programs, these topic areas should be given more attention in future workshops and in training. Furthermore, by adding more specific questions related to each section of the workshop, associations among presentation style, knowledge gained, and perceived usefulness of each section could be evaluated. For example, the participants significantly increased their scores from pretest to posttest on the catheter‐related knowledge‐based question and rated the lecture and case study on intravascular catheter‐related infections as the most useful sections. Future research may explore these possible relationships to better guide selection of presentation styles and topics to ensure that participants gain knowledge and perceive the sections as useful. In addition, by addressing the feedback from participants, such as offering more detail, examples, and discussion, future workshops may have greater perceived usefulness and be better able to increase the knowledge and awareness of quality improvement programs for the prevention of health careassociated infections and antimicrobial resistance.

Although there were 3 workshops conducted in 3 areas across the United States, the sample size at each site was small, and results may not be representative of hospitalists at large. In addition, power calculations should be considered in future studies to increase the ability to better detect differences between and within groups. Another limitation of this study was that the limited data available and participant anonymity meant it was not possible to follow‐up with participants after the workshop to evaluate whether the knowledge they gained was sustained and/or whether they reported changes in practice. However, possession of knowledge and skills to inform practice does not mean that practice will change; therefore, follow‐up is necessary to determine if this workshop was effective in changing behaviors in the long term.25 Although the SHM workshop improved knowledge, more intensive educational strategies may be necessary to affect perceptions and improve the leadership skills required for implementation of quality improvement programs at an institutional level.

Overall, the SHM workshop was found to be a useful tool for increasing knowledge and outlining methods by which hospitalists can lead, coordinate, or participate in measures to prevent infections and improve patient safety. In addition, through the workshop, the SHM and the CDC have provided an example of how professional societies and government agencies can collaborate to address emerging issues in the health care setting.

In the United States, hospitalized patients are at risk of acquiring health careassociated infections that increase morbidity, mortality, length of hospital stay, and cost of care.1 If a health careassociated infection is caused by an antimicrobial‐resistant pathogen, treatment efforts may be further complicated.2, 3 With the decreasing effectiveness of antimicrobials and suboptimal adherence to certain infection control measures, new and multifaceted prevention strategies are necessary to address the problem of health careassociated infections and antimicrobial resistance.410

One strategy that hospitals can use to reduce the incidence of health careassociated infections and antimicrobial resistance is implementation of quality improvement programs. These programs require clinicians to employ techniques, such as root cause analysis (RCA), which investigates contributing factors to an event to prevent reoccurrence, and healthcare failure mode effects analysis (HFMEA), which applies a systematic method of identifying and preventing problems before they occur.1113 Programs and strategies such as these require leadership and adoption within the hospital. Because of their availability and specialized role in the hospital setting, hospitalists are in a unique position to promote and uphold quality improvement efforts.1417 Professional societies, health care organizations, and governmental agencies can play a role in engaging this group of physicians in improving the quality of patient care in hospitals by providing educational programs and materials.18

In 2004, the Society of Hospital Medicine (SHM) collaborated with the Centers for Disease Control and Prevention (CDC) to develop a quality improvement tool kit to reduce antimicrobial resistance and health careassociated infections. The tool kit was based on the CDC's Campaign to Prevent Antimicrobial Resistance in Healthcare Settings (Campaign), an educational program targeted at clinicians.19 The SHM/CDC tool kit contained campaign materials, a set of slides about quality improvement, worksheets, and additional materials such as infection control policies and guidelines to supplement a 90‐minute workshop consisting of didactic lectures about antimicrobial resistance, quality improvement initiatives, RCA, and HFMEA; a lecture and case study about intravascular catheter‐related infections; and small‐group activity and discussion. The complete toolkit is now available online via the SHM Antimicrobial Resistance Resource Room at http://www.hospitalmedicine.org/AM/Template.cfm?Section=Home&Template=/CM/HTMLDisplay.cfm&ContentID=7542.

The purpose of the workshop was to present the tool kit and increase hospitalists' knowledge and awareness about antimicrobial resistance, health careassociated infections, and quality improvement programs. We assessed the workshop participants' familiarity with the Campaign prior to the workshop, perceptions of antimicrobial resistance, knowledge gained as a result of the workshop, and opinions about the usefulness of the workshop.

METHODS

Data were collected from pretests and posttests administered to participants of one of the SHM workshops in May, June, or July 2005 in Denver, Colorado; Boston, Massachusetts; or Portland, Oregon. One SHM physician leader (D.D.D., coauthor of this article) presented all 3 workshops. The workshops were advertised by SHM using E‐mail to local chapter members. Individual sites used a variety of methods to encourage their hospitalists to attend, and participants were provided a complimentary dinner.

Prior to each workshop, participants completed a 10‐question pretest that had been pilot‐tested by hospitalists in other cities. The pretest assessed demographics; perceptions of the problem of antimicrobial resistance using a Likert scale; familiarity with the Campaign; and knowledge of common infection sites, RCA, HFMEA, and antimicrobial resistance prevention measures.

Immediately following each workshop, a 13‐question posttest was administered to participants. This posttest evaluated the workshop and materials using Likert scales, asked for suggestions for future programming using open‐ended questions, and repeated pretest questions to assess changes in perceptions and knowledge.

Data were entered into an Excel spreadsheet and analyzed using descriptive statistics and t tests to compare pre‐ and posttest changes in knowledge. Likert data assessing perceptions were dichotomized into strongly agree versus all other scale responses. Qualitative open‐ended responses were categorized by theme.

RESULTS

A total of 69 SHM members attended the workshops. Of the 69 participants, 65 completed the pretest, 53 completed the posttest, and 50 completed both the pre‐ and the posttests. Only participants who completed both the pretest and the posttest were included in the analyses (n = 21, Denver; n = 11, Boston; n = 18, Portland). Of the 50 participants who completed both the pre‐ and posttests, 44 (88%) classified themselves as hospitalists in practices ranging from 2 to more than 25 physicians. Participants averaged 9.2 years (range = 1‐27 years) in practice and 4.9 years (range = 1‐10 years) as practicing hospitalists, with no significant differences between the 3 groups. Only 17 participants (34%) were familiar with the Campaign prior to the workshop, and there was no significant variation between the 3 workshops. Those familiar with the Campaign had heard about or received the educational materials from colleagues (n = 5), their facilities (n = 4), professional journals (n = 4), medical conferences (n = 4), or the CDC or SHM websites (n = 4).

Overall, most participants strongly agreed with the statement that antimicrobial resistance was a problem nationally, institutionally, and within their individual practices (Table 1). These perceptions did not significantly differ between the pretest and the posttest. However, statistically significant differences were found when comparing perceptions of the problem of antimicrobial resistance at the national, institutional, and practice levels; more participants strongly agreed that antimicrobial resistance was a problem nationally than within their institutions (pretest, P = .01; posttest, P = .04) or within their practices (pretest, P < .0001; posttest, P = .01).

On the knowledge‐based questions, the overall average test score was 48% on the pretest and 63% on the posttest (P < .0001), with scores varying by question (Table 2). For example, knowledge of quality improvement initiatives/HFMEA was low (an average of 10% correct on the pretest, 48% on the posttest) compared with knowledge about the key prevention strategies from the Campaign to Prevent Antimicrobial Resistance (average of 94% correct on the pretest, 98% on the posttest). Furthermore, scores also varied by workshop location. On the pretest, participants in Boston and Portland scored higher (both 53%) than Denver participants (40%). On the posttest, Portland participants scored the highest (78%) followed by Boston participants (64%) and then Denver participants (50%). Boston and Denver participants differed significantly on pretest knowledge score (P = .04) and Portland and Denver participants differed significantly on posttest knowledge score (P < .0001).

Overall, 43 participants (85%) rated the workshop as either very good or excellent. All but 1 participant (n = 49, 98%) would encourage a colleague to attend the workshop, giving reasons such as that the workshop outlined a major program in delivering good and safe care, offered great information on antimicrobial resistance and methods of quality improvement systems implementation, assisted in find[ing] new tools for improving hospital practice, and addressed a significant factor in hospitals related to morbidity [and] mortality. When asked for general comments about the workshop and suggestions for future improvements, participants requested more direction, more detail, more discussion, specific examples of antimicrobial resistance, and protocols and processes for implementing quality improvement programs. On a scale from 1 (not useful) to 5 (essential), participants rated the usefulness of each workshop segment: intravascular catheter‐related infections lecture and case study (x̄ = 4.3, range = 3‐5), quality improvement initiatives lecture (x̄ = 4.1, range = 2‐5), background on antimicrobial resistance (x̄ = 3.9, range = 2‐5), RCA lecture (x̄ = 3.9, range = 2‐5), HFMEA lecture (x̄ = 3.8, range = 2‐5), and small‐group discussion (x̄ = 3.4, range = 2‐5). These ratings did not vary significantly between the 3 groups.

CONCLUSIONS

To address antimicrobial resistance and health careassociated infections in the hospital setting, the SHM and CDC developed a tool kit and presented a quality improvement workshop to hospitalists in 3 U.S. cities. Overall, the participants scored significantly higher on the knowledge‐based questions on the posttest than on the pretest, indicating that knowledge improved as a result of the workshop. By providing a format that combined didactic lectures with case‐based education, small‐group activities, and discussion, the SHM workshop may have optimized its ability to increase knowledge, similar to the findings in previous research.2021

There were no significant differences between the 3 groups in years of practice, perceptions of the problem, and overall evaluation of the workshop. However, differences were found in knowledge gained as a result of the workshop. For example, the Denver group scored lower on the knowledge‐based questions than did the Boston group on the pretest and the Portland group on the posttest, indicating that knowledge and learning styles may differ by location. These differences may be attributed to variations in hospital environments, hospital‐based educational programs, or medical school and residency training. Differences like these may impact the effectiveness of a program and should be a consideration in the program development process, especially when a program is national in scope, like the CDC's Campaign to Prevent Antimicrobial Resistance in Healthcare Settings. In addition, more than 90% of participants correctly identified key prevention strategies of the Campaign, whereas only 34% were familiar with the Campaign itself prior to the workshop. This result may be a result of the key prevention strategies of the Campaign being derived from well‐established and ‐recognized evidence‐based best practices for patient safety and care.

Although knowledge changed as a result of the workshop, overall perceptions of the problem of antimicrobial resistance did not change significantly from pretest to posttest. It is possible this is because changes in perception require a different or more intensive educational approach. This result also may reflect the initial levels of agreement on the pretest, the measurement instrument itself, and/or the inability to detect differences because of the small number of participants.

Difference did exist in perceptions of the problem of antimicrobial resistance at the national, institutional, and practice levels. Antimicrobial resistance was perceived to be a greater problem on the national level than on the institutional and practice levels. Other studies also have found that clinicians more strongly agree that antimicrobial resistance is a problem nationally than within their institutions and practices.2224 When antimicrobial resistance is not perceived as a problem within institutions and practices, physicians may be less likely to overcome the barriers to following recommended infection prevention guidelines or to implementing quality improvement projects.4 Therefore, educational and intervention efforts like this workshop should address hospitalists' perceptions of the problem of antimicrobial resistance on the individual level as a first step in motivating them to engage in quality improvement.

Although participants' knowledge scores increased from pretest to posttest, gaps in knowledge remained, as indicated by the significantly improved but low overall posttest scores related to RCA and HFMEA. As hospitalists are in a unique position to promote quality improvement programs, these topic areas should be given more attention in future workshops and in training. Furthermore, by adding more specific questions related to each section of the workshop, associations among presentation style, knowledge gained, and perceived usefulness of each section could be evaluated. For example, the participants significantly increased their scores from pretest to posttest on the catheter‐related knowledge‐based question and rated the lecture and case study on intravascular catheter‐related infections as the most useful sections. Future research may explore these possible relationships to better guide selection of presentation styles and topics to ensure that participants gain knowledge and perceive the sections as useful. In addition, by addressing the feedback from participants, such as offering more detail, examples, and discussion, future workshops may have greater perceived usefulness and be better able to increase the knowledge and awareness of quality improvement programs for the prevention of health careassociated infections and antimicrobial resistance.

Although there were 3 workshops conducted in 3 areas across the United States, the sample size at each site was small, and results may not be representative of hospitalists at large. In addition, power calculations should be considered in future studies to increase the ability to better detect differences between and within groups. Another limitation of this study was that the limited data available and participant anonymity meant it was not possible to follow‐up with participants after the workshop to evaluate whether the knowledge they gained was sustained and/or whether they reported changes in practice. However, possession of knowledge and skills to inform practice does not mean that practice will change; therefore, follow‐up is necessary to determine if this workshop was effective in changing behaviors in the long term.25 Although the SHM workshop improved knowledge, more intensive educational strategies may be necessary to affect perceptions and improve the leadership skills required for implementation of quality improvement programs at an institutional level.

Overall, the SHM workshop was found to be a useful tool for increasing knowledge and outlining methods by which hospitalists can lead, coordinate, or participate in measures to prevent infections and improve patient safety. In addition, through the workshop, the SHM and the CDC have provided an example of how professional societies and government agencies can collaborate to address emerging issues in the health care setting.

In the United States, hospitalized patients are at risk of acquiring health careassociated infections that increase morbidity, mortality, length of hospital stay, and cost of care.1 If a health careassociated infection is caused by an antimicrobial‐resistant pathogen, treatment efforts may be further complicated.2, 3 With the decreasing effectiveness of antimicrobials and suboptimal adherence to certain infection control measures, new and multifaceted prevention strategies are necessary to address the problem of health careassociated infections and antimicrobial resistance.410

One strategy that hospitals can use to reduce the incidence of health careassociated infections and antimicrobial resistance is implementation of quality improvement programs. These programs require clinicians to employ techniques, such as root cause analysis (RCA), which investigates contributing factors to an event to prevent reoccurrence, and healthcare failure mode effects analysis (HFMEA), which applies a systematic method of identifying and preventing problems before they occur.1113 Programs and strategies such as these require leadership and adoption within the hospital. Because of their availability and specialized role in the hospital setting, hospitalists are in a unique position to promote and uphold quality improvement efforts.1417 Professional societies, health care organizations, and governmental agencies can play a role in engaging this group of physicians in improving the quality of patient care in hospitals by providing educational programs and materials.18

In 2004, the Society of Hospital Medicine (SHM) collaborated with the Centers for Disease Control and Prevention (CDC) to develop a quality improvement tool kit to reduce antimicrobial resistance and health careassociated infections. The tool kit was based on the CDC's Campaign to Prevent Antimicrobial Resistance in Healthcare Settings (Campaign), an educational program targeted at clinicians.19 The SHM/CDC tool kit contained campaign materials, a set of slides about quality improvement, worksheets, and additional materials such as infection control policies and guidelines to supplement a 90‐minute workshop consisting of didactic lectures about antimicrobial resistance, quality improvement initiatives, RCA, and HFMEA; a lecture and case study about intravascular catheter‐related infections; and small‐group activity and discussion. The complete toolkit is now available online via the SHM Antimicrobial Resistance Resource Room at http://www.hospitalmedicine.org/AM/Template.cfm?Section=Home&Template=/CM/HTMLDisplay.cfm&ContentID=7542.

The purpose of the workshop was to present the tool kit and increase hospitalists' knowledge and awareness about antimicrobial resistance, health careassociated infections, and quality improvement programs. We assessed the workshop participants' familiarity with the Campaign prior to the workshop, perceptions of antimicrobial resistance, knowledge gained as a result of the workshop, and opinions about the usefulness of the workshop.

METHODS

Data were collected from pretests and posttests administered to participants of one of the SHM workshops in May, June, or July 2005 in Denver, Colorado; Boston, Massachusetts; or Portland, Oregon. One SHM physician leader (D.D.D., coauthor of this article) presented all 3 workshops. The workshops were advertised by SHM using E‐mail to local chapter members. Individual sites used a variety of methods to encourage their hospitalists to attend, and participants were provided a complimentary dinner.

Prior to each workshop, participants completed a 10‐question pretest that had been pilot‐tested by hospitalists in other cities. The pretest assessed demographics; perceptions of the problem of antimicrobial resistance using a Likert scale; familiarity with the Campaign; and knowledge of common infection sites, RCA, HFMEA, and antimicrobial resistance prevention measures.

Immediately following each workshop, a 13‐question posttest was administered to participants. This posttest evaluated the workshop and materials using Likert scales, asked for suggestions for future programming using open‐ended questions, and repeated pretest questions to assess changes in perceptions and knowledge.

Data were entered into an Excel spreadsheet and analyzed using descriptive statistics and t tests to compare pre‐ and posttest changes in knowledge. Likert data assessing perceptions were dichotomized into strongly agree versus all other scale responses. Qualitative open‐ended responses were categorized by theme.

RESULTS

A total of 69 SHM members attended the workshops. Of the 69 participants, 65 completed the pretest, 53 completed the posttest, and 50 completed both the pre‐ and the posttests. Only participants who completed both the pretest and the posttest were included in the analyses (n = 21, Denver; n = 11, Boston; n = 18, Portland). Of the 50 participants who completed both the pre‐ and posttests, 44 (88%) classified themselves as hospitalists in practices ranging from 2 to more than 25 physicians. Participants averaged 9.2 years (range = 1‐27 years) in practice and 4.9 years (range = 1‐10 years) as practicing hospitalists, with no significant differences between the 3 groups. Only 17 participants (34%) were familiar with the Campaign prior to the workshop, and there was no significant variation between the 3 workshops. Those familiar with the Campaign had heard about or received the educational materials from colleagues (n = 5), their facilities (n = 4), professional journals (n = 4), medical conferences (n = 4), or the CDC or SHM websites (n = 4).

Overall, most participants strongly agreed with the statement that antimicrobial resistance was a problem nationally, institutionally, and within their individual practices (Table 1). These perceptions did not significantly differ between the pretest and the posttest. However, statistically significant differences were found when comparing perceptions of the problem of antimicrobial resistance at the national, institutional, and practice levels; more participants strongly agreed that antimicrobial resistance was a problem nationally than within their institutions (pretest, P = .01; posttest, P = .04) or within their practices (pretest, P < .0001; posttest, P = .01).

On the knowledge‐based questions, the overall average test score was 48% on the pretest and 63% on the posttest (P < .0001), with scores varying by question (Table 2). For example, knowledge of quality improvement initiatives/HFMEA was low (an average of 10% correct on the pretest, 48% on the posttest) compared with knowledge about the key prevention strategies from the Campaign to Prevent Antimicrobial Resistance (average of 94% correct on the pretest, 98% on the posttest). Furthermore, scores also varied by workshop location. On the pretest, participants in Boston and Portland scored higher (both 53%) than Denver participants (40%). On the posttest, Portland participants scored the highest (78%) followed by Boston participants (64%) and then Denver participants (50%). Boston and Denver participants differed significantly on pretest knowledge score (P = .04) and Portland and Denver participants differed significantly on posttest knowledge score (P < .0001).

Overall, 43 participants (85%) rated the workshop as either very good or excellent. All but 1 participant (n = 49, 98%) would encourage a colleague to attend the workshop, giving reasons such as that the workshop outlined a major program in delivering good and safe care, offered great information on antimicrobial resistance and methods of quality improvement systems implementation, assisted in find[ing] new tools for improving hospital practice, and addressed a significant factor in hospitals related to morbidity [and] mortality. When asked for general comments about the workshop and suggestions for future improvements, participants requested more direction, more detail, more discussion, specific examples of antimicrobial resistance, and protocols and processes for implementing quality improvement programs. On a scale from 1 (not useful) to 5 (essential), participants rated the usefulness of each workshop segment: intravascular catheter‐related infections lecture and case study (x̄ = 4.3, range = 3‐5), quality improvement initiatives lecture (x̄ = 4.1, range = 2‐5), background on antimicrobial resistance (x̄ = 3.9, range = 2‐5), RCA lecture (x̄ = 3.9, range = 2‐5), HFMEA lecture (x̄ = 3.8, range = 2‐5), and small‐group discussion (x̄ = 3.4, range = 2‐5). These ratings did not vary significantly between the 3 groups.

CONCLUSIONS

To address antimicrobial resistance and health careassociated infections in the hospital setting, the SHM and CDC developed a tool kit and presented a quality improvement workshop to hospitalists in 3 U.S. cities. Overall, the participants scored significantly higher on the knowledge‐based questions on the posttest than on the pretest, indicating that knowledge improved as a result of the workshop. By providing a format that combined didactic lectures with case‐based education, small‐group activities, and discussion, the SHM workshop may have optimized its ability to increase knowledge, similar to the findings in previous research.2021

There were no significant differences between the 3 groups in years of practice, perceptions of the problem, and overall evaluation of the workshop. However, differences were found in knowledge gained as a result of the workshop. For example, the Denver group scored lower on the knowledge‐based questions than did the Boston group on the pretest and the Portland group on the posttest, indicating that knowledge and learning styles may differ by location. These differences may be attributed to variations in hospital environments, hospital‐based educational programs, or medical school and residency training. Differences like these may impact the effectiveness of a program and should be a consideration in the program development process, especially when a program is national in scope, like the CDC's Campaign to Prevent Antimicrobial Resistance in Healthcare Settings. In addition, more than 90% of participants correctly identified key prevention strategies of the Campaign, whereas only 34% were familiar with the Campaign itself prior to the workshop. This result may be a result of the key prevention strategies of the Campaign being derived from well‐established and ‐recognized evidence‐based best practices for patient safety and care.

Although knowledge changed as a result of the workshop, overall perceptions of the problem of antimicrobial resistance did not change significantly from pretest to posttest. It is possible this is because changes in perception require a different or more intensive educational approach. This result also may reflect the initial levels of agreement on the pretest, the measurement instrument itself, and/or the inability to detect differences because of the small number of participants.

Difference did exist in perceptions of the problem of antimicrobial resistance at the national, institutional, and practice levels. Antimicrobial resistance was perceived to be a greater problem on the national level than on the institutional and practice levels. Other studies also have found that clinicians more strongly agree that antimicrobial resistance is a problem nationally than within their institutions and practices.2224 When antimicrobial resistance is not perceived as a problem within institutions and practices, physicians may be less likely to overcome the barriers to following recommended infection prevention guidelines or to implementing quality improvement projects.4 Therefore, educational and intervention efforts like this workshop should address hospitalists' perceptions of the problem of antimicrobial resistance on the individual level as a first step in motivating them to engage in quality improvement.

Although participants' knowledge scores increased from pretest to posttest, gaps in knowledge remained, as indicated by the significantly improved but low overall posttest scores related to RCA and HFMEA. As hospitalists are in a unique position to promote quality improvement programs, these topic areas should be given more attention in future workshops and in training. Furthermore, by adding more specific questions related to each section of the workshop, associations among presentation style, knowledge gained, and perceived usefulness of each section could be evaluated. For example, the participants significantly increased their scores from pretest to posttest on the catheter‐related knowledge‐based question and rated the lecture and case study on intravascular catheter‐related infections as the most useful sections. Future research may explore these possible relationships to better guide selection of presentation styles and topics to ensure that participants gain knowledge and perceive the sections as useful. In addition, by addressing the feedback from participants, such as offering more detail, examples, and discussion, future workshops may have greater perceived usefulness and be better able to increase the knowledge and awareness of quality improvement programs for the prevention of health careassociated infections and antimicrobial resistance.

Although there were 3 workshops conducted in 3 areas across the United States, the sample size at each site was small, and results may not be representative of hospitalists at large. In addition, power calculations should be considered in future studies to increase the ability to better detect differences between and within groups. Another limitation of this study was that the limited data available and participant anonymity meant it was not possible to follow‐up with participants after the workshop to evaluate whether the knowledge they gained was sustained and/or whether they reported changes in practice. However, possession of knowledge and skills to inform practice does not mean that practice will change; therefore, follow‐up is necessary to determine if this workshop was effective in changing behaviors in the long term.25 Although the SHM workshop improved knowledge, more intensive educational strategies may be necessary to affect perceptions and improve the leadership skills required for implementation of quality improvement programs at an institutional level.

Overall, the SHM workshop was found to be a useful tool for increasing knowledge and outlining methods by which hospitalists can lead, coordinate, or participate in measures to prevent infections and improve patient safety. In addition, through the workshop, the SHM and the CDC have provided an example of how professional societies and government agencies can collaborate to address emerging issues in the health care setting.