Affiliations
Department of Medicine, Massachusetts General Hospital
Given name(s)
Zaven
Family name
Sargsyan
Degrees
MD

Things We Do for No Reason™: Supplemental Oxygen for Patients without Hypoxemia

Article Type
Changed
Tue, 09/21/2021 - 11:03

Inspired by the ABIM Foundation’s Choosing Wisely® campaign, the “Things We Do for No Reason” (TWDFNR) series reviews practices that have become common parts of hospital care but may provide little value to our patients. Practices reviewed in the TWDFNR series do not represent clear-cut conclusions or clinical practice standards but are meant as a starting place for research and active discussions among hospitalists and patients. We invite you to be part of that discussion.

CLINICAL SCENARIO

A 65-year-old woman with hypertension presents to the emergency department with three days of dyspnea, malaise, and pleuritic chest pain. Her temperature is 100.1°F, heart rate 110 beats per minute, and blood pressure 110/60 mm Hg. She is breathing 24 times per minute and has an oxygen saturation (SpO2) of 94% on room air. Her exam is remarkable for dry mucous membranes and right lower lung crackles. Her nurse places her on 3 L of oxygen per minute via nasal cannula, and her SpO2 rises to 99%.

WHY YOU MIGHT THINK SUPPLEMENTAL OXYGEN FOR NORMOXEMIC PATIENTS IS HELPFUL

Shortly after the discovery of oxygen in the late 18th century, physicians began using it to treat a variety of conditions including tuberculosis, pneumonia, respiratory failure, and angina. By the 1970s, most medical texts recommended oxygen use in suspected myocardial infarction (MI) because of the theoretical appeal of increasing delivery of oxygen to the heart and other vital organs.1 Additionally, there is a tendency to believe that supplemental oxygen alleviates dyspnea regardless of etiology or oxygen saturation. Recent studies have shown widespread use of oxygen in scenarios without clear indications and without oxygen saturation goals. A 2010 survey of clinicians managing acute MI found that 98% “always or usually” used oxygen and 55% believed that oxygen “definitely or probably reduces the risk of death.”2 In a Danish prehospital study, supplemental oxygen was used in 34% of ambulance patients even though only 17% of these patients had an SpO2 less than 94%.3 A study of critically ill patients found that most of the time, SpO2 exceeded 98%. Even when the fraction of inspired oxygen (FiO2) was between 0.3 and 0.4, no one adjusted the oxygen dose.4

WHY IT IS NOT HELPFUL TO PROVIDE SUPPLEMENTAL OXYGEN TO NORMOXEMIC PATIENTS

The reflexive use of oxygen in patients with acute respiratory or cardiovascular illness is problematic for several reasons. First, when oxygen saturation is near-normal, the potential benefit from supplemental oxygen lacks physiologic plausibility. More compellingly, evidence exists that hyperoxemia may cause significant harm. Finally, the unnecessary use of supplemental oxygen incurs practical inconveniences and expenses.

To understand why the physiologic basis for reflexive oxygen use is weak, it is important to distinguish hypoxemia (low arterial oxygen tension and hemoglobin oxygen saturation), tissue hypoxia (which can occur from hypoxemia or focal abnormalities in perfusion), and dyspnea (a subjective experience of breathing discomfort). A variety of mechanisms cause dyspnea, most of which do not involve hypoxemia. A patient with acute heart failure may experience severe dyspnea caused by activation of pressure-sensitive J-receptors in the lung, even if oxygen saturation and tissue perfusion are intact. This process will be relieved by reducing pulmonary capillary pressures, but it is unaffected by supplemental oxygen. Coronary occlusion causes hypoxia of the heart muscle, but restoring perfusion is the most effective treatment. The instinct to maximize the oxygen-carrying capacity of the remaining blood flow is understandable. However, in a normoxemic patient, increasing the inspired fraction of oxygen has a marginal effect on oxygen-carrying capacity, since hemoglobin saturation and concentration rather than arterial oxygen tension (PaO2) predominantly determine oxygen-carrying capacity. On the other hand, supraphysiologic levels of dissolved oxygen may lead to toxicity.5

For over a century, we have known the potential harms of hyperoxia. Original studies in animal models showed that hyperoxia led to lung injury, altered hemodynamics, endothelial cell dysfunction, and inflammatory activation.5 Many of these detrimental effects involve the generation of reactive oxygen species and oxidative stress.5 High levels of inspired oxygen can also cause increased pulmonary shunting through inhibition of physiologic hypoxic vasoconstriction and due to absorption atelectasis.6 Oxygen negatively affects cardiovascular function by reducing coronary blood flow, increasing systemic vascular resistance, and reducing cardiac output.1

Chronic obstructive pulmonary disease (COPD) is the clinical setting in which risks of supplemental oxygen are most well-recognized historically. In patients with COPD at risk for hypercarbia, oxygen titrated to a goal SpO2 outside 88%-92% is associated with a two-fold risk of mortality.7 Worsening ventilation-perfusion matching and the Haldane effect (decreased affinity of hemoglobin for carbon dioxide as the PaO2 rises), rather than the previously theorized decrease in hypoxic drive, are now believed to contribute most to hyperoxia-induced hypercarbia. These unintended consequences may also occur in patients with other forms of acute and chronic lung disease.

The British Medical Journal published the first randomized controlled trial of oxygen use in suspected MI in 1976.1 Patients who received oxygen at 6 L per minute for 24 hours had more episodes of sinus tachycardia without any improvement in mortality, analgesic use, or infarct size.1 More recent and robust trials comparing outcomes in normoxemic patients randomized to supplemental oxygen versus room air have had similar findings: no difference in mortality, infarct size, or pain ratings.8,9 One found a significantly increased rate of MI recurrence with the use of oxygen.8 These data have led the latest guidelines for the management of ST-elevation MI from the European Society of Cardiology to discourage the use of supplemental oxygen unless SpO2 is <90%.10

Two recent trials investigated the effects of hyperoxia in critically ill patients.11,12 Girardis and colleagues randomized 480 critically ill patients in an Italian medical-surgical intensive care unit to conservative (SpO2 between 94% and 98% or PaO2 between 70 and 100 mm Hg) versus conventional oxygenation targets (SpO2 between 97% and 100% and PaO2 up to 150 mm Hg). Compared with conventional oxygen targets, conservative oxygen use was associated with an absolute risk reduction in mortality of 8.6% (11.6% vs 20.2%; P =.01).11 Another trial from 22 centers in France compared outcomes in mechanically ventilated patients with septic shock who received FiO2 at 1.0 compared with those with oxygen titration to SpO2 between 88% and 95%. The trial was stopped early for safety concerns. Those in the hyperoxemia group had a higher incidence of serious adverse events (85% vs 76%; P =.02), including pneumothorax, clinically relevant bleeding, myocardial infarction, and arrhythmias, as well as a trend toward increased mortality.12

Trials of liberal oxygen use in other settings of acute illness,13 including ischemic stroke,14 traumatic brain injury,15 and postcardiac arrest,16 have also linked liberal oxygen use with increased risk of mortality and other adverse events. “Liberal” use in these trials ranged from an FiO2 of 0.28 (equivalent to 2 L of nasal cannula) to 1.0. Significant secondary outcomes included fewer hospital-free and ventilator-free days in patients with liberal oxygen use. Furthermore, a meta-analysis of 25 trials including over 16,000 patients found dose-dependent toxicity: for every 1% increase in SpO2 above 94%-96% (the median SpO2 in the liberal oxygen groups), there was a 25% relative increase in in-hospital mortality.13

In addition to the data above, there are practical reasons to avoid unnecessary use of supplemental oxygen. Providing supplemental oxygen to a patient who is not hypoxemic may delay the recognition of cardiopulmonary decompensation by delaying detection of hypoxemia.6 Beyond the effects of oxygen itself, oxygen delivery methods carry their own potential adverse effects. These include epistaxis (with nasal cannula), claustrophobia (with face masks), decreased mobility, falls, and delirium.17 Finally, oxygen administration has direct and indirect financial costs, including those of supplies, care coordination, and monitoring.

 

 

WHEN SUPPLEMENTAL OXYGEN MIGHT BE HELPFUL

Importantly, the above discussion pertains to normoxemic patients receiving supplemental oxygen. There is no dispute that significantly hypoxemic patients should receive supplemental oxygen. There are also instances where the use of supplemental oxygen in normoxemic patients may be beneficial, such as in carbon monoxide poisoning, decompression injury, gas embolism, cluster headaches, sickle cell crisis, and pneumothorax.17

WHAT YOU SHOULD DO INSTEAD

Like any other drug, oxygen should be administered after assessment of its indications, intended benefits, and possible harms. Both significant hypoxemia and hyperoxemia should be avoided. In patients with neither hypoxemia nor the indications above, clinicians should not administer supplemental oxygen. Recent society guidelines can be applied in various clinical contexts. In patients with suspected MI, oxygen should be administered if SpO2 is <90%.10 For most other acutely ill patients, clinicians should administer supplemental oxygen if SpO2 <90%-92% and target an SpO2 of no higher than 94%-96%,18-19 as meta-analyses found evidence of harm above this level.13 Results of randomized trials currently underway should add supporting evidence for more specific oxygenation targets in different patient populations. With respect to implementation, it must be noted that factors beyond physician decision influence the use of supplemental oxygen. Appropriate institutional policies, standards of care, and educational efforts to all hospital providers must be enacted in order to reduce the unnecessary use of supplemental oxygen.

RECOMMENDATIONS

  • For most acutely ill patients, do not administer supplemental oxygen when SpO2 >92%. If supplemental oxygen is used, the SpO2 should not exceed 94%-96%.
  • For patients with suspected MI, only start supplemental oxygen for SpO2 <90%.
  • For patients at risk for hypercapnic respiratory failure (eg, COPD patients), target SpO2 of 88%-92%.
  • Provide supplemental oxygen to normoxemic patients with carbon monoxide poisoning, decompression injury, gas embolism, cluster headache, sickle cell crisis, and pneumothorax.
  • Review and revise institutional practices and policies that contribute to unnecessary use of supplemental oxygen.

CONCLUSIONS

In the opening case, the patient is acutely ill and requires further workup. Her current SpO2 of 99% puts her at risk for adverse events and death, and supplemental oxygen should be titrated down or stopped to avoid SpO2 greater than 94%-96%. For years, clinicians have erred on the side of using supplemental oxygen, without recognizing its dangers. However, over a century of evidence from pathophysiologic experiments and randomized trials across multiple clinical settings have associated hyperoxemia with adverse outcomes and increased mortality. Professional societies are adopting this evidence into their guideline recommendations, and clinicians should use supplemental oxygen judiciously in their daily practice.

Do you think this is a low-value practice? Is this truly a “Thing We Do for No Reason”? Share what you do in your practice and join in the conversation online by retweeting it on Twitter (#TWDFNR) and liking it on Facebook. We invite you to propose ideas for other “Things We Do for No Reason” topics by emailing TWDFNR@hospitalmedicine.org.

 

 

References

1. Rawles JM, Kenmure AC. Controlled trial of oxygen in uncomplicated myocardial infarction. Br Med J. 1976;1(6018):1121-1123. https://doi.org/10.1136/bmj.1.6018.1121.
2. Burls A, Emparanza JI, Quinn T, Cabello J. Oxygen use in acute myocardial infarction: an online survey of health professionals’ practice and beliefs. Emerg Med J. 2010;27(4):283-286. https://doi.org/10.1136/emj.2009.077370.
3. Hale KE, Gavin C, O’Driscoll BR. Audit of oxygen use in emergency ambulances and in a hospital emergency department. Emerg Med J. 2008;25(11):773-776. https://doi.org/10.1136/emj.2008.059287.
4. Suzuki S, Eastwood G, Peck L, Glassford N, Bellomo R. Oxygen management in mechanically ventilated patients: a prospective observational cohort study. Aust Crit Care. 2014;27(1):50-51. https://doi.org/10.1016/j.aucc.2013.10.025.
5. Helmerhorst HJ, Schultz MJ, van der Voort PH, de Jonge E, van Wasterloo DJ. Bench-to-bedside review: the effects of hyperoxia during critical illness. Crit Care. 2015;19(1):284. https://doi.org/10.1186/s13054-015-0996-4.
6. Downs JB. Has oxygen administration delayed appropriate respiratory care? Fallacies regarding oxygen therapy. Respir Care. 2003;48(6):611-620.
7. Austin MA, Willis KE, Blizzard L, Walters EH, Wood-Baker R. Effect of high flow oxygen on mortality in chronic obstructive pulmonary disease patients in prehospital setting: randomised controlled trial. BMJ. 2010;341:c5462. https://doi.org/10.2307/20800296.
8. Stub D, Smith K, Bernard S, et al. Air versus oxygen in ST-segment-elevation myocardial infarction. Circulation. 2015;131(24):2143-2150. https://doi.org/10.1161/CIRCULATIONAHA.114.014494.
9. Hofman R. Witt N, Lagergvist B, et al. Oxygen therapy in ST-elevation myocardial infarction. Eur Heart J. 2018;39(29):2730-2739. https://doi.org/10.1093/eurheartj/ehy326.
10. Ibanez B, James S, Agewall S, et al. 2017 ESC Guidelines for the management of acute myocardial infarction in patients presenting with ST-segment elevation: the Task Force for the management of acute myocardial infarction in patients presenting with ST-segment elevation of the European Society of Cardiology (ESC). Eur Heart J. 2018:39(2):119-177. https://doi.org/10.1093/eurheartj/ehx393.
11. Girardis M, Busani S, Damiani E, et al. Effect of conservative vs conventional oxygen therapy on mortality among patients in an intensive care unit. JAMA. 2016;316(15):1583-1589. https://doi.org/10.1001/jama.2016.11993.
12. Asfar P, Schortgen F, Boisramé-Helms J, et al. Hyperoxia and hypertonic saline in patients with septic shock (HYPERS2S): a two-by-two factorial, multicentre, randomised, clinical trial. Lancet Respir Med. 2017:5(3):180-190. https://doi.org/10.1016/S2213-2600(17)30046-2.
13. Chu DK, Kim LH, Young PJ, et al. Mortality and morbidity in acutely ill adults treated with liberal versus conservative oxygen therapy (IOTA): a systematic review and meta-analysis. Lancet. 2018;391(10131):1693-1705. https://doi.org/10.1016/S0140-6736(18)30479-3.
14. Rincon F, Kang J, Maltenfort M, et al. Association between hyperoxia and mortality after stroke: a multicenter cohort study. Crit Care Med. 2014;42(2):387-396. https://doi.org/10.1097/CCM.0b013e3182a27732.
15. Brenner M, Stein D, Hu P, Kufera J, Woodford M, Scalea T. Association between early hyperoxia and worse outcomes after traumatic brain injury. Arch Surg. 2012;147(11):1042-1046. https://doi.org/10.1001/archsurg.2012.1560.
16. Kilgannon JH, Jones AE, Shapiro NI, et al. Association between arterial hyperoxia following resuscitation from cardiac arrest and in-hospital mortality. JAMA. 2010;303(21):2165-2171. https://doi.org/10.1
001/jama.2010.707.
17. Siemieniuk RA, Chu DK, Kim L, et al. Oxygen therapy for acutely ill medical patients: a clinical practice guideline. BMJ. 2018;363:k4169. https://doi.org/10.1136/bmj.k4169.
18. O’Driscoll BR, Howard LS, Earis J, et al. BTS guideline for oxygen use in adults in healthcare and emergency settings. Thorax. 2017;72(1):ii1-ii90. https://doi.org/10.1136/thoraxjnl-2016-209729.
19. Beasley R, Chien J, Douglas J, et al. Thoracic Society of Australia and New Zealand oxygen guidelines for acute oxygen use in adults: ‘Swimming between the flags’. Respirology. 2015;20(8):1182-1191. https://doi.org/10.1111/resp.12620.

Article PDF
Author and Disclosure Information

Department of Internal Medicine, Baylor College of Medicine, Houston, Texas.

Disclosures

The authors have nothing to disclose.

Issue
Journal of Hospital Medicine 15(4)
Publications
Topics
Page Number
242-244. Published Online First October 23, 2019.
Sections
Author and Disclosure Information

Department of Internal Medicine, Baylor College of Medicine, Houston, Texas.

Disclosures

The authors have nothing to disclose.

Author and Disclosure Information

Department of Internal Medicine, Baylor College of Medicine, Houston, Texas.

Disclosures

The authors have nothing to disclose.

Article PDF
Article PDF

Inspired by the ABIM Foundation’s Choosing Wisely® campaign, the “Things We Do for No Reason” (TWDFNR) series reviews practices that have become common parts of hospital care but may provide little value to our patients. Practices reviewed in the TWDFNR series do not represent clear-cut conclusions or clinical practice standards but are meant as a starting place for research and active discussions among hospitalists and patients. We invite you to be part of that discussion.

CLINICAL SCENARIO

A 65-year-old woman with hypertension presents to the emergency department with three days of dyspnea, malaise, and pleuritic chest pain. Her temperature is 100.1°F, heart rate 110 beats per minute, and blood pressure 110/60 mm Hg. She is breathing 24 times per minute and has an oxygen saturation (SpO2) of 94% on room air. Her exam is remarkable for dry mucous membranes and right lower lung crackles. Her nurse places her on 3 L of oxygen per minute via nasal cannula, and her SpO2 rises to 99%.

WHY YOU MIGHT THINK SUPPLEMENTAL OXYGEN FOR NORMOXEMIC PATIENTS IS HELPFUL

Shortly after the discovery of oxygen in the late 18th century, physicians began using it to treat a variety of conditions including tuberculosis, pneumonia, respiratory failure, and angina. By the 1970s, most medical texts recommended oxygen use in suspected myocardial infarction (MI) because of the theoretical appeal of increasing delivery of oxygen to the heart and other vital organs.1 Additionally, there is a tendency to believe that supplemental oxygen alleviates dyspnea regardless of etiology or oxygen saturation. Recent studies have shown widespread use of oxygen in scenarios without clear indications and without oxygen saturation goals. A 2010 survey of clinicians managing acute MI found that 98% “always or usually” used oxygen and 55% believed that oxygen “definitely or probably reduces the risk of death.”2 In a Danish prehospital study, supplemental oxygen was used in 34% of ambulance patients even though only 17% of these patients had an SpO2 less than 94%.3 A study of critically ill patients found that most of the time, SpO2 exceeded 98%. Even when the fraction of inspired oxygen (FiO2) was between 0.3 and 0.4, no one adjusted the oxygen dose.4

WHY IT IS NOT HELPFUL TO PROVIDE SUPPLEMENTAL OXYGEN TO NORMOXEMIC PATIENTS

The reflexive use of oxygen in patients with acute respiratory or cardiovascular illness is problematic for several reasons. First, when oxygen saturation is near-normal, the potential benefit from supplemental oxygen lacks physiologic plausibility. More compellingly, evidence exists that hyperoxemia may cause significant harm. Finally, the unnecessary use of supplemental oxygen incurs practical inconveniences and expenses.

To understand why the physiologic basis for reflexive oxygen use is weak, it is important to distinguish hypoxemia (low arterial oxygen tension and hemoglobin oxygen saturation), tissue hypoxia (which can occur from hypoxemia or focal abnormalities in perfusion), and dyspnea (a subjective experience of breathing discomfort). A variety of mechanisms cause dyspnea, most of which do not involve hypoxemia. A patient with acute heart failure may experience severe dyspnea caused by activation of pressure-sensitive J-receptors in the lung, even if oxygen saturation and tissue perfusion are intact. This process will be relieved by reducing pulmonary capillary pressures, but it is unaffected by supplemental oxygen. Coronary occlusion causes hypoxia of the heart muscle, but restoring perfusion is the most effective treatment. The instinct to maximize the oxygen-carrying capacity of the remaining blood flow is understandable. However, in a normoxemic patient, increasing the inspired fraction of oxygen has a marginal effect on oxygen-carrying capacity, since hemoglobin saturation and concentration rather than arterial oxygen tension (PaO2) predominantly determine oxygen-carrying capacity. On the other hand, supraphysiologic levels of dissolved oxygen may lead to toxicity.5

For over a century, we have known the potential harms of hyperoxia. Original studies in animal models showed that hyperoxia led to lung injury, altered hemodynamics, endothelial cell dysfunction, and inflammatory activation.5 Many of these detrimental effects involve the generation of reactive oxygen species and oxidative stress.5 High levels of inspired oxygen can also cause increased pulmonary shunting through inhibition of physiologic hypoxic vasoconstriction and due to absorption atelectasis.6 Oxygen negatively affects cardiovascular function by reducing coronary blood flow, increasing systemic vascular resistance, and reducing cardiac output.1

Chronic obstructive pulmonary disease (COPD) is the clinical setting in which risks of supplemental oxygen are most well-recognized historically. In patients with COPD at risk for hypercarbia, oxygen titrated to a goal SpO2 outside 88%-92% is associated with a two-fold risk of mortality.7 Worsening ventilation-perfusion matching and the Haldane effect (decreased affinity of hemoglobin for carbon dioxide as the PaO2 rises), rather than the previously theorized decrease in hypoxic drive, are now believed to contribute most to hyperoxia-induced hypercarbia. These unintended consequences may also occur in patients with other forms of acute and chronic lung disease.

The British Medical Journal published the first randomized controlled trial of oxygen use in suspected MI in 1976.1 Patients who received oxygen at 6 L per minute for 24 hours had more episodes of sinus tachycardia without any improvement in mortality, analgesic use, or infarct size.1 More recent and robust trials comparing outcomes in normoxemic patients randomized to supplemental oxygen versus room air have had similar findings: no difference in mortality, infarct size, or pain ratings.8,9 One found a significantly increased rate of MI recurrence with the use of oxygen.8 These data have led the latest guidelines for the management of ST-elevation MI from the European Society of Cardiology to discourage the use of supplemental oxygen unless SpO2 is <90%.10

Two recent trials investigated the effects of hyperoxia in critically ill patients.11,12 Girardis and colleagues randomized 480 critically ill patients in an Italian medical-surgical intensive care unit to conservative (SpO2 between 94% and 98% or PaO2 between 70 and 100 mm Hg) versus conventional oxygenation targets (SpO2 between 97% and 100% and PaO2 up to 150 mm Hg). Compared with conventional oxygen targets, conservative oxygen use was associated with an absolute risk reduction in mortality of 8.6% (11.6% vs 20.2%; P =.01).11 Another trial from 22 centers in France compared outcomes in mechanically ventilated patients with septic shock who received FiO2 at 1.0 compared with those with oxygen titration to SpO2 between 88% and 95%. The trial was stopped early for safety concerns. Those in the hyperoxemia group had a higher incidence of serious adverse events (85% vs 76%; P =.02), including pneumothorax, clinically relevant bleeding, myocardial infarction, and arrhythmias, as well as a trend toward increased mortality.12

Trials of liberal oxygen use in other settings of acute illness,13 including ischemic stroke,14 traumatic brain injury,15 and postcardiac arrest,16 have also linked liberal oxygen use with increased risk of mortality and other adverse events. “Liberal” use in these trials ranged from an FiO2 of 0.28 (equivalent to 2 L of nasal cannula) to 1.0. Significant secondary outcomes included fewer hospital-free and ventilator-free days in patients with liberal oxygen use. Furthermore, a meta-analysis of 25 trials including over 16,000 patients found dose-dependent toxicity: for every 1% increase in SpO2 above 94%-96% (the median SpO2 in the liberal oxygen groups), there was a 25% relative increase in in-hospital mortality.13

In addition to the data above, there are practical reasons to avoid unnecessary use of supplemental oxygen. Providing supplemental oxygen to a patient who is not hypoxemic may delay the recognition of cardiopulmonary decompensation by delaying detection of hypoxemia.6 Beyond the effects of oxygen itself, oxygen delivery methods carry their own potential adverse effects. These include epistaxis (with nasal cannula), claustrophobia (with face masks), decreased mobility, falls, and delirium.17 Finally, oxygen administration has direct and indirect financial costs, including those of supplies, care coordination, and monitoring.

 

 

WHEN SUPPLEMENTAL OXYGEN MIGHT BE HELPFUL

Importantly, the above discussion pertains to normoxemic patients receiving supplemental oxygen. There is no dispute that significantly hypoxemic patients should receive supplemental oxygen. There are also instances where the use of supplemental oxygen in normoxemic patients may be beneficial, such as in carbon monoxide poisoning, decompression injury, gas embolism, cluster headaches, sickle cell crisis, and pneumothorax.17

WHAT YOU SHOULD DO INSTEAD

Like any other drug, oxygen should be administered after assessment of its indications, intended benefits, and possible harms. Both significant hypoxemia and hyperoxemia should be avoided. In patients with neither hypoxemia nor the indications above, clinicians should not administer supplemental oxygen. Recent society guidelines can be applied in various clinical contexts. In patients with suspected MI, oxygen should be administered if SpO2 is <90%.10 For most other acutely ill patients, clinicians should administer supplemental oxygen if SpO2 <90%-92% and target an SpO2 of no higher than 94%-96%,18-19 as meta-analyses found evidence of harm above this level.13 Results of randomized trials currently underway should add supporting evidence for more specific oxygenation targets in different patient populations. With respect to implementation, it must be noted that factors beyond physician decision influence the use of supplemental oxygen. Appropriate institutional policies, standards of care, and educational efforts to all hospital providers must be enacted in order to reduce the unnecessary use of supplemental oxygen.

RECOMMENDATIONS

  • For most acutely ill patients, do not administer supplemental oxygen when SpO2 >92%. If supplemental oxygen is used, the SpO2 should not exceed 94%-96%.
  • For patients with suspected MI, only start supplemental oxygen for SpO2 <90%.
  • For patients at risk for hypercapnic respiratory failure (eg, COPD patients), target SpO2 of 88%-92%.
  • Provide supplemental oxygen to normoxemic patients with carbon monoxide poisoning, decompression injury, gas embolism, cluster headache, sickle cell crisis, and pneumothorax.
  • Review and revise institutional practices and policies that contribute to unnecessary use of supplemental oxygen.

CONCLUSIONS

In the opening case, the patient is acutely ill and requires further workup. Her current SpO2 of 99% puts her at risk for adverse events and death, and supplemental oxygen should be titrated down or stopped to avoid SpO2 greater than 94%-96%. For years, clinicians have erred on the side of using supplemental oxygen, without recognizing its dangers. However, over a century of evidence from pathophysiologic experiments and randomized trials across multiple clinical settings have associated hyperoxemia with adverse outcomes and increased mortality. Professional societies are adopting this evidence into their guideline recommendations, and clinicians should use supplemental oxygen judiciously in their daily practice.

Do you think this is a low-value practice? Is this truly a “Thing We Do for No Reason”? Share what you do in your practice and join in the conversation online by retweeting it on Twitter (#TWDFNR) and liking it on Facebook. We invite you to propose ideas for other “Things We Do for No Reason” topics by emailing TWDFNR@hospitalmedicine.org.

 

 

Inspired by the ABIM Foundation’s Choosing Wisely® campaign, the “Things We Do for No Reason” (TWDFNR) series reviews practices that have become common parts of hospital care but may provide little value to our patients. Practices reviewed in the TWDFNR series do not represent clear-cut conclusions or clinical practice standards but are meant as a starting place for research and active discussions among hospitalists and patients. We invite you to be part of that discussion.

CLINICAL SCENARIO

A 65-year-old woman with hypertension presents to the emergency department with three days of dyspnea, malaise, and pleuritic chest pain. Her temperature is 100.1°F, heart rate 110 beats per minute, and blood pressure 110/60 mm Hg. She is breathing 24 times per minute and has an oxygen saturation (SpO2) of 94% on room air. Her exam is remarkable for dry mucous membranes and right lower lung crackles. Her nurse places her on 3 L of oxygen per minute via nasal cannula, and her SpO2 rises to 99%.

WHY YOU MIGHT THINK SUPPLEMENTAL OXYGEN FOR NORMOXEMIC PATIENTS IS HELPFUL

Shortly after the discovery of oxygen in the late 18th century, physicians began using it to treat a variety of conditions including tuberculosis, pneumonia, respiratory failure, and angina. By the 1970s, most medical texts recommended oxygen use in suspected myocardial infarction (MI) because of the theoretical appeal of increasing delivery of oxygen to the heart and other vital organs.1 Additionally, there is a tendency to believe that supplemental oxygen alleviates dyspnea regardless of etiology or oxygen saturation. Recent studies have shown widespread use of oxygen in scenarios without clear indications and without oxygen saturation goals. A 2010 survey of clinicians managing acute MI found that 98% “always or usually” used oxygen and 55% believed that oxygen “definitely or probably reduces the risk of death.”2 In a Danish prehospital study, supplemental oxygen was used in 34% of ambulance patients even though only 17% of these patients had an SpO2 less than 94%.3 A study of critically ill patients found that most of the time, SpO2 exceeded 98%. Even when the fraction of inspired oxygen (FiO2) was between 0.3 and 0.4, no one adjusted the oxygen dose.4

WHY IT IS NOT HELPFUL TO PROVIDE SUPPLEMENTAL OXYGEN TO NORMOXEMIC PATIENTS

The reflexive use of oxygen in patients with acute respiratory or cardiovascular illness is problematic for several reasons. First, when oxygen saturation is near-normal, the potential benefit from supplemental oxygen lacks physiologic plausibility. More compellingly, evidence exists that hyperoxemia may cause significant harm. Finally, the unnecessary use of supplemental oxygen incurs practical inconveniences and expenses.

To understand why the physiologic basis for reflexive oxygen use is weak, it is important to distinguish hypoxemia (low arterial oxygen tension and hemoglobin oxygen saturation), tissue hypoxia (which can occur from hypoxemia or focal abnormalities in perfusion), and dyspnea (a subjective experience of breathing discomfort). A variety of mechanisms cause dyspnea, most of which do not involve hypoxemia. A patient with acute heart failure may experience severe dyspnea caused by activation of pressure-sensitive J-receptors in the lung, even if oxygen saturation and tissue perfusion are intact. This process will be relieved by reducing pulmonary capillary pressures, but it is unaffected by supplemental oxygen. Coronary occlusion causes hypoxia of the heart muscle, but restoring perfusion is the most effective treatment. The instinct to maximize the oxygen-carrying capacity of the remaining blood flow is understandable. However, in a normoxemic patient, increasing the inspired fraction of oxygen has a marginal effect on oxygen-carrying capacity, since hemoglobin saturation and concentration rather than arterial oxygen tension (PaO2) predominantly determine oxygen-carrying capacity. On the other hand, supraphysiologic levels of dissolved oxygen may lead to toxicity.5

For over a century, we have known the potential harms of hyperoxia. Original studies in animal models showed that hyperoxia led to lung injury, altered hemodynamics, endothelial cell dysfunction, and inflammatory activation.5 Many of these detrimental effects involve the generation of reactive oxygen species and oxidative stress.5 High levels of inspired oxygen can also cause increased pulmonary shunting through inhibition of physiologic hypoxic vasoconstriction and due to absorption atelectasis.6 Oxygen negatively affects cardiovascular function by reducing coronary blood flow, increasing systemic vascular resistance, and reducing cardiac output.1

Chronic obstructive pulmonary disease (COPD) is the clinical setting in which risks of supplemental oxygen are most well-recognized historically. In patients with COPD at risk for hypercarbia, oxygen titrated to a goal SpO2 outside 88%-92% is associated with a two-fold risk of mortality.7 Worsening ventilation-perfusion matching and the Haldane effect (decreased affinity of hemoglobin for carbon dioxide as the PaO2 rises), rather than the previously theorized decrease in hypoxic drive, are now believed to contribute most to hyperoxia-induced hypercarbia. These unintended consequences may also occur in patients with other forms of acute and chronic lung disease.

The British Medical Journal published the first randomized controlled trial of oxygen use in suspected MI in 1976.1 Patients who received oxygen at 6 L per minute for 24 hours had more episodes of sinus tachycardia without any improvement in mortality, analgesic use, or infarct size.1 More recent and robust trials comparing outcomes in normoxemic patients randomized to supplemental oxygen versus room air have had similar findings: no difference in mortality, infarct size, or pain ratings.8,9 One found a significantly increased rate of MI recurrence with the use of oxygen.8 These data have led the latest guidelines for the management of ST-elevation MI from the European Society of Cardiology to discourage the use of supplemental oxygen unless SpO2 is <90%.10

Two recent trials investigated the effects of hyperoxia in critically ill patients.11,12 Girardis and colleagues randomized 480 critically ill patients in an Italian medical-surgical intensive care unit to conservative (SpO2 between 94% and 98% or PaO2 between 70 and 100 mm Hg) versus conventional oxygenation targets (SpO2 between 97% and 100% and PaO2 up to 150 mm Hg). Compared with conventional oxygen targets, conservative oxygen use was associated with an absolute risk reduction in mortality of 8.6% (11.6% vs 20.2%; P =.01).11 Another trial from 22 centers in France compared outcomes in mechanically ventilated patients with septic shock who received FiO2 at 1.0 compared with those with oxygen titration to SpO2 between 88% and 95%. The trial was stopped early for safety concerns. Those in the hyperoxemia group had a higher incidence of serious adverse events (85% vs 76%; P =.02), including pneumothorax, clinically relevant bleeding, myocardial infarction, and arrhythmias, as well as a trend toward increased mortality.12

Trials of liberal oxygen use in other settings of acute illness,13 including ischemic stroke,14 traumatic brain injury,15 and postcardiac arrest,16 have also linked liberal oxygen use with increased risk of mortality and other adverse events. “Liberal” use in these trials ranged from an FiO2 of 0.28 (equivalent to 2 L of nasal cannula) to 1.0. Significant secondary outcomes included fewer hospital-free and ventilator-free days in patients with liberal oxygen use. Furthermore, a meta-analysis of 25 trials including over 16,000 patients found dose-dependent toxicity: for every 1% increase in SpO2 above 94%-96% (the median SpO2 in the liberal oxygen groups), there was a 25% relative increase in in-hospital mortality.13

In addition to the data above, there are practical reasons to avoid unnecessary use of supplemental oxygen. Providing supplemental oxygen to a patient who is not hypoxemic may delay the recognition of cardiopulmonary decompensation by delaying detection of hypoxemia.6 Beyond the effects of oxygen itself, oxygen delivery methods carry their own potential adverse effects. These include epistaxis (with nasal cannula), claustrophobia (with face masks), decreased mobility, falls, and delirium.17 Finally, oxygen administration has direct and indirect financial costs, including those of supplies, care coordination, and monitoring.

 

 

WHEN SUPPLEMENTAL OXYGEN MIGHT BE HELPFUL

Importantly, the above discussion pertains to normoxemic patients receiving supplemental oxygen. There is no dispute that significantly hypoxemic patients should receive supplemental oxygen. There are also instances where the use of supplemental oxygen in normoxemic patients may be beneficial, such as in carbon monoxide poisoning, decompression injury, gas embolism, cluster headaches, sickle cell crisis, and pneumothorax.17

WHAT YOU SHOULD DO INSTEAD

Like any other drug, oxygen should be administered after assessment of its indications, intended benefits, and possible harms. Both significant hypoxemia and hyperoxemia should be avoided. In patients with neither hypoxemia nor the indications above, clinicians should not administer supplemental oxygen. Recent society guidelines can be applied in various clinical contexts. In patients with suspected MI, oxygen should be administered if SpO2 is <90%.10 For most other acutely ill patients, clinicians should administer supplemental oxygen if SpO2 <90%-92% and target an SpO2 of no higher than 94%-96%,18-19 as meta-analyses found evidence of harm above this level.13 Results of randomized trials currently underway should add supporting evidence for more specific oxygenation targets in different patient populations. With respect to implementation, it must be noted that factors beyond physician decision influence the use of supplemental oxygen. Appropriate institutional policies, standards of care, and educational efforts to all hospital providers must be enacted in order to reduce the unnecessary use of supplemental oxygen.

RECOMMENDATIONS

  • For most acutely ill patients, do not administer supplemental oxygen when SpO2 >92%. If supplemental oxygen is used, the SpO2 should not exceed 94%-96%.
  • For patients with suspected MI, only start supplemental oxygen for SpO2 <90%.
  • For patients at risk for hypercapnic respiratory failure (eg, COPD patients), target SpO2 of 88%-92%.
  • Provide supplemental oxygen to normoxemic patients with carbon monoxide poisoning, decompression injury, gas embolism, cluster headache, sickle cell crisis, and pneumothorax.
  • Review and revise institutional practices and policies that contribute to unnecessary use of supplemental oxygen.

CONCLUSIONS

In the opening case, the patient is acutely ill and requires further workup. Her current SpO2 of 99% puts her at risk for adverse events and death, and supplemental oxygen should be titrated down or stopped to avoid SpO2 greater than 94%-96%. For years, clinicians have erred on the side of using supplemental oxygen, without recognizing its dangers. However, over a century of evidence from pathophysiologic experiments and randomized trials across multiple clinical settings have associated hyperoxemia with adverse outcomes and increased mortality. Professional societies are adopting this evidence into their guideline recommendations, and clinicians should use supplemental oxygen judiciously in their daily practice.

Do you think this is a low-value practice? Is this truly a “Thing We Do for No Reason”? Share what you do in your practice and join in the conversation online by retweeting it on Twitter (#TWDFNR) and liking it on Facebook. We invite you to propose ideas for other “Things We Do for No Reason” topics by emailing TWDFNR@hospitalmedicine.org.

 

 

References

1. Rawles JM, Kenmure AC. Controlled trial of oxygen in uncomplicated myocardial infarction. Br Med J. 1976;1(6018):1121-1123. https://doi.org/10.1136/bmj.1.6018.1121.
2. Burls A, Emparanza JI, Quinn T, Cabello J. Oxygen use in acute myocardial infarction: an online survey of health professionals’ practice and beliefs. Emerg Med J. 2010;27(4):283-286. https://doi.org/10.1136/emj.2009.077370.
3. Hale KE, Gavin C, O’Driscoll BR. Audit of oxygen use in emergency ambulances and in a hospital emergency department. Emerg Med J. 2008;25(11):773-776. https://doi.org/10.1136/emj.2008.059287.
4. Suzuki S, Eastwood G, Peck L, Glassford N, Bellomo R. Oxygen management in mechanically ventilated patients: a prospective observational cohort study. Aust Crit Care. 2014;27(1):50-51. https://doi.org/10.1016/j.aucc.2013.10.025.
5. Helmerhorst HJ, Schultz MJ, van der Voort PH, de Jonge E, van Wasterloo DJ. Bench-to-bedside review: the effects of hyperoxia during critical illness. Crit Care. 2015;19(1):284. https://doi.org/10.1186/s13054-015-0996-4.
6. Downs JB. Has oxygen administration delayed appropriate respiratory care? Fallacies regarding oxygen therapy. Respir Care. 2003;48(6):611-620.
7. Austin MA, Willis KE, Blizzard L, Walters EH, Wood-Baker R. Effect of high flow oxygen on mortality in chronic obstructive pulmonary disease patients in prehospital setting: randomised controlled trial. BMJ. 2010;341:c5462. https://doi.org/10.2307/20800296.
8. Stub D, Smith K, Bernard S, et al. Air versus oxygen in ST-segment-elevation myocardial infarction. Circulation. 2015;131(24):2143-2150. https://doi.org/10.1161/CIRCULATIONAHA.114.014494.
9. Hofman R. Witt N, Lagergvist B, et al. Oxygen therapy in ST-elevation myocardial infarction. Eur Heart J. 2018;39(29):2730-2739. https://doi.org/10.1093/eurheartj/ehy326.
10. Ibanez B, James S, Agewall S, et al. 2017 ESC Guidelines for the management of acute myocardial infarction in patients presenting with ST-segment elevation: the Task Force for the management of acute myocardial infarction in patients presenting with ST-segment elevation of the European Society of Cardiology (ESC). Eur Heart J. 2018:39(2):119-177. https://doi.org/10.1093/eurheartj/ehx393.
11. Girardis M, Busani S, Damiani E, et al. Effect of conservative vs conventional oxygen therapy on mortality among patients in an intensive care unit. JAMA. 2016;316(15):1583-1589. https://doi.org/10.1001/jama.2016.11993.
12. Asfar P, Schortgen F, Boisramé-Helms J, et al. Hyperoxia and hypertonic saline in patients with septic shock (HYPERS2S): a two-by-two factorial, multicentre, randomised, clinical trial. Lancet Respir Med. 2017:5(3):180-190. https://doi.org/10.1016/S2213-2600(17)30046-2.
13. Chu DK, Kim LH, Young PJ, et al. Mortality and morbidity in acutely ill adults treated with liberal versus conservative oxygen therapy (IOTA): a systematic review and meta-analysis. Lancet. 2018;391(10131):1693-1705. https://doi.org/10.1016/S0140-6736(18)30479-3.
14. Rincon F, Kang J, Maltenfort M, et al. Association between hyperoxia and mortality after stroke: a multicenter cohort study. Crit Care Med. 2014;42(2):387-396. https://doi.org/10.1097/CCM.0b013e3182a27732.
15. Brenner M, Stein D, Hu P, Kufera J, Woodford M, Scalea T. Association between early hyperoxia and worse outcomes after traumatic brain injury. Arch Surg. 2012;147(11):1042-1046. https://doi.org/10.1001/archsurg.2012.1560.
16. Kilgannon JH, Jones AE, Shapiro NI, et al. Association between arterial hyperoxia following resuscitation from cardiac arrest and in-hospital mortality. JAMA. 2010;303(21):2165-2171. https://doi.org/10.1
001/jama.2010.707.
17. Siemieniuk RA, Chu DK, Kim L, et al. Oxygen therapy for acutely ill medical patients: a clinical practice guideline. BMJ. 2018;363:k4169. https://doi.org/10.1136/bmj.k4169.
18. O’Driscoll BR, Howard LS, Earis J, et al. BTS guideline for oxygen use in adults in healthcare and emergency settings. Thorax. 2017;72(1):ii1-ii90. https://doi.org/10.1136/thoraxjnl-2016-209729.
19. Beasley R, Chien J, Douglas J, et al. Thoracic Society of Australia and New Zealand oxygen guidelines for acute oxygen use in adults: ‘Swimming between the flags’. Respirology. 2015;20(8):1182-1191. https://doi.org/10.1111/resp.12620.

References

1. Rawles JM, Kenmure AC. Controlled trial of oxygen in uncomplicated myocardial infarction. Br Med J. 1976;1(6018):1121-1123. https://doi.org/10.1136/bmj.1.6018.1121.
2. Burls A, Emparanza JI, Quinn T, Cabello J. Oxygen use in acute myocardial infarction: an online survey of health professionals’ practice and beliefs. Emerg Med J. 2010;27(4):283-286. https://doi.org/10.1136/emj.2009.077370.
3. Hale KE, Gavin C, O’Driscoll BR. Audit of oxygen use in emergency ambulances and in a hospital emergency department. Emerg Med J. 2008;25(11):773-776. https://doi.org/10.1136/emj.2008.059287.
4. Suzuki S, Eastwood G, Peck L, Glassford N, Bellomo R. Oxygen management in mechanically ventilated patients: a prospective observational cohort study. Aust Crit Care. 2014;27(1):50-51. https://doi.org/10.1016/j.aucc.2013.10.025.
5. Helmerhorst HJ, Schultz MJ, van der Voort PH, de Jonge E, van Wasterloo DJ. Bench-to-bedside review: the effects of hyperoxia during critical illness. Crit Care. 2015;19(1):284. https://doi.org/10.1186/s13054-015-0996-4.
6. Downs JB. Has oxygen administration delayed appropriate respiratory care? Fallacies regarding oxygen therapy. Respir Care. 2003;48(6):611-620.
7. Austin MA, Willis KE, Blizzard L, Walters EH, Wood-Baker R. Effect of high flow oxygen on mortality in chronic obstructive pulmonary disease patients in prehospital setting: randomised controlled trial. BMJ. 2010;341:c5462. https://doi.org/10.2307/20800296.
8. Stub D, Smith K, Bernard S, et al. Air versus oxygen in ST-segment-elevation myocardial infarction. Circulation. 2015;131(24):2143-2150. https://doi.org/10.1161/CIRCULATIONAHA.114.014494.
9. Hofman R. Witt N, Lagergvist B, et al. Oxygen therapy in ST-elevation myocardial infarction. Eur Heart J. 2018;39(29):2730-2739. https://doi.org/10.1093/eurheartj/ehy326.
10. Ibanez B, James S, Agewall S, et al. 2017 ESC Guidelines for the management of acute myocardial infarction in patients presenting with ST-segment elevation: the Task Force for the management of acute myocardial infarction in patients presenting with ST-segment elevation of the European Society of Cardiology (ESC). Eur Heart J. 2018:39(2):119-177. https://doi.org/10.1093/eurheartj/ehx393.
11. Girardis M, Busani S, Damiani E, et al. Effect of conservative vs conventional oxygen therapy on mortality among patients in an intensive care unit. JAMA. 2016;316(15):1583-1589. https://doi.org/10.1001/jama.2016.11993.
12. Asfar P, Schortgen F, Boisramé-Helms J, et al. Hyperoxia and hypertonic saline in patients with septic shock (HYPERS2S): a two-by-two factorial, multicentre, randomised, clinical trial. Lancet Respir Med. 2017:5(3):180-190. https://doi.org/10.1016/S2213-2600(17)30046-2.
13. Chu DK, Kim LH, Young PJ, et al. Mortality and morbidity in acutely ill adults treated with liberal versus conservative oxygen therapy (IOTA): a systematic review and meta-analysis. Lancet. 2018;391(10131):1693-1705. https://doi.org/10.1016/S0140-6736(18)30479-3.
14. Rincon F, Kang J, Maltenfort M, et al. Association between hyperoxia and mortality after stroke: a multicenter cohort study. Crit Care Med. 2014;42(2):387-396. https://doi.org/10.1097/CCM.0b013e3182a27732.
15. Brenner M, Stein D, Hu P, Kufera J, Woodford M, Scalea T. Association between early hyperoxia and worse outcomes after traumatic brain injury. Arch Surg. 2012;147(11):1042-1046. https://doi.org/10.1001/archsurg.2012.1560.
16. Kilgannon JH, Jones AE, Shapiro NI, et al. Association between arterial hyperoxia following resuscitation from cardiac arrest and in-hospital mortality. JAMA. 2010;303(21):2165-2171. https://doi.org/10.1
001/jama.2010.707.
17. Siemieniuk RA, Chu DK, Kim L, et al. Oxygen therapy for acutely ill medical patients: a clinical practice guideline. BMJ. 2018;363:k4169. https://doi.org/10.1136/bmj.k4169.
18. O’Driscoll BR, Howard LS, Earis J, et al. BTS guideline for oxygen use in adults in healthcare and emergency settings. Thorax. 2017;72(1):ii1-ii90. https://doi.org/10.1136/thoraxjnl-2016-209729.
19. Beasley R, Chien J, Douglas J, et al. Thoracic Society of Australia and New Zealand oxygen guidelines for acute oxygen use in adults: ‘Swimming between the flags’. Respirology. 2015;20(8):1182-1191. https://doi.org/10.1111/resp.12620.

Issue
Journal of Hospital Medicine 15(4)
Issue
Journal of Hospital Medicine 15(4)
Page Number
242-244. Published Online First October 23, 2019.
Page Number
242-244. Published Online First October 23, 2019.
Publications
Publications
Topics
Article Type
Sections
Article Source

© 2019 Society of Hospital Medicine

Disallow All Ads
Correspondence Location
Zaven Sargsyan, MD; E-mail: zaven.sargsyan@bcm.edu; Telephone: 713-873-3560; Twitter: @sargsyanz.
Content Gating
Open Access (article Unlocked/Open Access)
Alternative CME
Disqus Comments
Default
Use ProPublica
Hide sidebar & use full width
render the right sidebar.
Attach Teaching Materials
Conference Recap Checkbox
Not Conference Recap
Clinical Edge
Display the Slideshow in this Article
Medscape Article
Display survey writer
Reuters content
Disable Inline Native ads
WebMD Article
Article PDF Media
Attach Teaching Materials

Simulation Resident‐as‐Teacher Program

Article Type
Changed
Tue, 05/16/2017 - 22:49
Display Headline
A simulation‐based resident‐as‐teacher program: The impact on teachers and learners

Residency training, in addition to developing clinical competence among trainees, is charged with improving resident teaching skills. The Liaison Committee on Medical Education and the Accreditation Council for Graduate Medical Education require that residents be provided with training or resources to develop their teaching skills.[1, 2] A variety of resident‐as‐teacher (RaT) programs have been described; however, the optimal format of such programs remains in question.[3] High‐fidelity medical simulation using mannequins has been shown to be an effective teaching tool in various medical specialties[4, 5, 6, 7] and may prove to be useful in teacher training.[8] Teaching in a simulation‐based environment can give participants the opportunity to apply their teaching skills in a clinical environment, as they would on the wards, but in a more controlled, predictable setting and without compromising patient safety. In addition, simulation offers the opportunity to engage in deliberate practice by allowing teachers to facilitate the same case on multiple occasions with different learners. Deliberate practice, which involves task repetition with feedback aimed at improving performance, has been shown to be important in developing expertise.[9]

We previously described the first use of a high‐fidelity simulation curriculum for internal medicine (IM) interns focused on clinical decision‐making skills, in which second‐ and third‐year residents served as facilitators.[10, 11] Herein, we describe a RaT program in which residents participated in a workshop, then served as facilitators in the intern curriculum and received feedback from faculty. We hypothesized that such a program would improve residents' teaching and feedback skills, both in the simulation environment and on the wards.

METHODS

We conducted a single‐group study evaluating teaching and feedback skills among upper‐level resident facilitators before and after participation in the RaT program. We measured residents' teaching skills using pre‐ and post‐program self‐assessments as well as evaluations completed by the intern learners after each session and at the completion of the curriculum.

Setting and Participants

We embedded the RaT program within a simulation curriculum administered July to October of 2013 for all IM interns at Massachusetts General Hospital (interns in the preliminary program who planned to pursue another field after the completion of the intern year were excluded) (n = 52). We invited postgraduate year (PGY) II and III residents (n = 102) to participate in the IM simulation program as facilitators via email. The curriculum consisted of 8 cases focusing on acute clinical scenarios encountered on the general medicine wards. The cases were administered during 1‐hour sessions 4 mornings per week from 7 AM to 8 AM prior to clinical duties. Interns completed the curriculum over 4 sessions during their outpatient rotation. The case topics were (1) hypertensive emergency, (2) post‐procedure bleed, (3) congestive heart failure, (4) atrial fibrillation with rapid ventricular response, (5) altered mental status/alcohol withdrawal, (6) nonsustained ventricular tachycardia heralding acute coronary syndrome, (7) cardiac tamponade, and (8) anaphylaxis. During each session, groups of 2 to 3 interns worked through 2 cases using a high‐fidelity mannequin (Laerdal 3G, Wappingers Falls, NY) with 2 resident facilitators. One facilitator operated the mannequin, while the other served as a nurse. Each case was followed by a structured debriefing led by 1 of the resident facilitators (facilitators switched roles for the second case). The number of sessions facilitated varied for each resident based on individual schedules and preferences.

Four senior residents who were appointed as simulation leaders (G.A.A., J.K.H., R.K., Z.S.) and 2 faculty advisors (P.F.C., E.M.M.) administered the program. Simulation resident leaders scheduled facilitators and interns and participated in a portion of simulation sessions as facilitators, but they were not analyzed as participants for the purposes of this study. The curriculum was administered without interfering with clinical duties, and no additional time was protected for interns or residents participating in the curriculum.

Resident‐as‐Teacher Program Structure

We invited participating resident facilitators to attend a 1‐hour interactive workshop prior to serving as facilitators. The workshop focused on building learner‐centered and small‐group teaching skills, as well as introducing residents to a 5‐stage debriefing framework developed by the authors and based on simulation debriefing best practices (Table 1).[12, 13, 14]

Stages of Debriefing
Stage of DebriefingActionRationale
  • NOTE: *To standardize the learner experience, all interns received an e‐mail after each session describing the key learning objectives and takeaway points with references from the medical literature for each case.

Emotional responseElicit learners' emotions about the caseIt is important to acknowledge and address both positive and negative emotions that arise during the case before debriefing the specific medical and communications aspects of the case. Unaddressed emotional responses may hinder subsequent debriefing.
Objectives*Elicit learners' objectives and combine them with the stated learning objectives of the case to determine debriefing objectivesThe limited amount of time allocated for debriefing (1520 minutes) does not allow the facilitator to cover all aspects of medical management and communication skills in a particular case. Focusing on the most salient objectives, including those identified by the learners, allows the facilitator to engage in learner‐centered debriefing.
AnalysisAnalyze the learners' approach to the caseAnalyzing the learners' approach to the case using the advocacy‐inquiry method[11] seeks to uncover the learner's assumptions/frameworks behind the decision made during the case. This approach allows the facilitator to understand the learners' thought process and target teaching points to more precisely address the learners' needs.
TeachingAddress knowledge gaps and incorrect assumptionsLearner‐centered debriefing within a limited timeframe requires teaching to be brief and targeted toward the defined objectives. It should also address the knowledge gaps and incorrect assumptions uncovered during the analysis phase.
SummarySummarize key takeawaysSummarizing highlights the key points of the debriefing and can be used to suggest further exploration of topics through self‐study (if necessary).

Resident facilitators were observed by simulation faculty and simulation resident leaders throughout the intern curriculum and given structured feedback either in‐person immediately after completion of the simulation session or via a detailed same‐day e‐mail if the time allotted for feedback was not sufficient. Feedback was structured by the 5 stages of debriefing described in Table 1, and included soliciting residents' observations on the teaching experience and specific behaviors observed by faculty during the scenarios. E‐mail feedback (also structured by stages of debriefing and including observed behaviors) was typically followed by verbal feedback during the next simulation session.

The RaT program was composed of 3 elements: the workshop, case facilitation, and direct observation with feedback. Because we felt that the opportunity for directly observed teaching and feedback in a ward‐like controlled environment was a unique advantage offered by the simulation setting, we included all residents who served as facilitators in the analysis, regardless of whether or not they had attended the workshop.

Evaluation Instruments

Survey instruments were developed by the investigators, reviewed by several experts in simulation, pilot tested among residents not participating in the simulation program, and revised by the investigators.

Pre‐program Facilitator Survey

Prior to the RaT workshop, resident facilitators completed a baseline survey evaluating their preparedness to teach and give feedback on the wards and in a simulation‐based setting on a 5‐point scale (see Supporting Information, Appendix I, in the online version of this article).

Post‐program Facilitator Survey

Approximately 3 weeks after completion of the intern simulation curriculum, resident facilitators were asked to complete an online post‐program survey, which remained open for 1 month (residents completed this survey anywhere from 3 weeks to 4 months after their participation in the RaT program depending on the timing of their facilitation). The survey asked residents to evaluate their comfort with their current post‐program teaching skills as well as their pre‐program skills in retrospect, as previous research demonstrated that learners may overestimate their skills prior to training programs.[15] Resident facilitators could complete the surveys nonanonymously to allow for matched‐pairs analysis of the change in teaching skills over the course of the program (see Supporting Information, Appendix II, in the online version of this article).

Intern Evaluation of Facilitator Debriefing Skills

After each case, intern learners were asked to anonymously evaluate the teaching effectiveness of the lead resident facilitator using the adapted Debriefing Assessment for Simulation in Healthcare (DASH) instrument.[16] The DASH instrument evaluated the following domains: (1) instructor maintained an engaging context for learning, (2) instructor structured the debriefing in an organized way, (3) instructor provoked in‐depth discussions that led me to reflect on my performance, (4) instructor identified what I did well or poorly and why, (5) instructor helped me see how to improve or how to sustain good performance, (6) overall effectiveness of the simulation session (see Supporting Information, Appendix III, in the online version of this article).

Post‐program Intern Survey

Two months following the completion of the simulation curriculum, intern learners received an anonymous online post‐program evaluation assessing program efficacy and resident facilitator teaching (see Supporting Information, Appendix IV, in the online version of this article).

Statistical Analysis

Teaching skills and learners' DASH ratings were compared using the Student t test, Pearson 2 test, and Fisher exact test as appropriate. Pre‐ and post‐program rating of teaching skills was undertaken in aggregate and as a matched‐pairs analysis.

The study was approved by the Partners Institutional Review Board.

RESULTS

Forty‐one resident facilitators participated in 118 individual simulation sessions encompassing 236 case scenarios. Thirty‐four residents completed the post‐program facilitator survey and were included in the analysis. Of these, 26 (76%) participated in the workshop and completed the pre‐program survey. Twenty‐three of the 34 residents (68%) completed the post‐program evaluation nonanonymously (13 PGY‐II, 10 PGY‐III). Of these, 16 completed the pre‐program survey nonanonymously. The average number of sessions facilitated by each resident was 3.9 (range, 112).

Pre‐ and Post‐program Self‐Assessment of Residents' Teaching Skills

Participation in the simulation RaT program led to improvements in resident facilitators' self‐reported teaching skills across multiple domains (Table 2). These results were consistent when using the retrospective pre‐program assessment in matched‐pairs analysis (n=34) and when performing the analysis using the true pre‐program preparedness compared to post‐program comfort with teaching skills in a non‐matched‐pairs fashion (n = 26) and matched‐pairs fashion (n = 16). We report P values for the more conservative estimates using the retrospective pre‐program assessment matched‐pairs analysis. The most significant improvements occurred in residents' ability to teach in a simulated environment (2.81 to 4.16, P < 0.001 [5‐point scale]) and give feedback (3.35 to 3.77, P < 0.001).

Pre‐ and Post‐program Self‐Assessment of Resident Facilitators Teaching Skills*
 Pre‐program Rating (n = 34)Post‐program Rating (n = 34)P Value
  • NOTE: *Survey data were collected before participation in the workshop and 3 weeks after completion of the 4‐month curriculum. Five‐point Likert scale: very uncomfortable (1) to very comfortable (5).

Teaching on rounds3.754.030.005
Teaching on wards outside rounds3.834.070.007
Teaching in simulation2.814.16<0.001
Giving feedback3.353.77<0.001

Resident facilitators reported that participation in the RaT program had a significant impact on their teaching skills both within and outside of the simulation environment (Table 3). However, the greatest gains were seen in the domain of teaching in simulation. It was also noted that participation in the program improved resident facilitators' medical knowledge.

Resident Facilitators' Perceived Improvement in Skills Due to Resident‐as‐Teacher Program
CategoryNot at AllSlightly ImprovedModerately ImprovedGreatly ImprovedNot Sure
Teaching on rounds, n = 344 (12%)12 (35%)13 (38%)4 (12%)1 (3%)
Teaching on wards outside rounds, n = 343 (9%)13 (38%)12 (35%)5 (15%)1 (3%)
Teaching in simulation, n = 340 (0%)4 (12%)7 (21%)23 (68%)0 (0%)
Giving feedback, n = 344 (12%)10 (29%)12 (35%)6 (18%)2 (6%)
Medical knowledge, n = 342 (6%)11 (32%)18 (53%)3 (9%)0 (0%)

Subgroup analyses were performed comparing the perceived improvement in teaching and feedback skills among those who did or did not attend the facilitator workshop, those who facilitated 5 or more versus less than 5 sessions, and those who received or did not receive direct observation and feedback from faculty. Although numerically greater gains were seen across all 4 domains among those who attended the workshop, facilitated 5 or more sessions, or received feedback from faculty, only teaching on rounds and on the wards outside rounds reached statistical significance (Table 4). It should be noted that all residents who facilitated 5 or more sessions also attended the workshop and received feedback from faculty. We also compared perceived improvement among PGY‐II and PGY‐III residents. In contrast to PGY‐II residents, who demonstrated an improvement in all 4 domains, PGY‐III residents only demonstrated improvement in simulation‐based teaching.

Pre‐ and Post‐program Self‐Assessment of Resident Facilitators Teaching Skills According to Number of Sessions Facilitated, Workshop Attendance, Receipt of Feedback, and PGY Year
 Pre‐programPost‐programP ValuePre‐programPost‐programP Value
 Facilitated Less Than 5 Sessions (n = 18)Facilitated 5 or More Sessions (n = 11)
 Did Not Attend Workshop (n = 10)Attended Workshop (n = 22)
 Received Feedback From Resident Leaders Only (n = 11)Received Faculty Feedback (n = 21)
 PGY‐II (n = 13)PGY‐III (n = 9)
  • NOTE: Abbreviations: PGY, postgraduate year.

Teaching on rounds3.683.790.163.854.380.01
Teaching on wards outside rounds3.8240.083.854.150.04
Teaching in simulation2.894.06<0.012.694.31<0.01
Giving feedback3.333.670.013.383.920.01
Teaching on rounds44.10.343.644<0.01
Teaching on wards outside rounds441.003.764.1<0.01
Teaching in simulation2.894.11<0.012.774.18<0.01
Giving feedback3.563.780.173.273.77<0.01
Teaching on rounds3.553.820.193.864.140.01
Teaching on wards outside rounds441.003.754.1<0.01
Teaching in simulation2.73.8<0.012.864.33<0.01
Giving feedback3.23.60.043.433.86<0.01
Teaching on rounds3.383.850.034.224.221
Teaching on wards outside rounds3.543.850.044.144.141
Teaching in simulation2.464.15<0.013.134.13<0.01
Giving feedback3.233.620.023.53.880.08

Intern Learners' Assessment of Resident Facilitators and the Program Overall

During the course of the program, intern learners completed 166 DASH ratings evaluating 34 resident facilitators (see Supporting Information, Appendix V, in the online version of this article). Ratings for the 6 DASH items ranged from 6.49 to 6.73 (7‐point scale), demonstrating a high level of facilitator efficacy across multiple domains. No differences in DASH scores were noted among subgroups of resident facilitators described in the previous paragraph.

Thirty‐eight of 52 intern learners (73%) completed the post‐program survey.

Resident Facilitators' Use of Specific Teaching Skills During Debriefing as Rated by Intern Learners
Facilitator BehaviorsVery Often, >75%Often, >50%Sometimes, 25%50%Rarely, <25%Never
Elicited emotional reactions, n = 3818 (47%)16 (42%)4 (11%)0 (0%)0 (0%)
Elicited objectives from learner, n = 3726 (69%)8 (22%)2 (6%)1 (3%)0 (0%)
Asked to share clinical reasoning, n = 3821 (56%)13 (33%)4 (11%)0 (0%)0 (0%)
Summarized learning points, n = 3831 (81%)7 (19%)0 (0%)0 (0%)0 (0%)
Spoke for less than half of the session, n = 388 (22%)17 (44%)11 (28%)2 (6%)0 (0%)

All intern learners rated the overall simulation experience as either excellent (81%) or good (19%) on the post‐program evaluation (4 or 5 on a 5‐point Likert scale, respectively). All interns strongly agreed (72%) or agreed (28%) that the simulation sessions improved their ability to manage acute clinical scenarios. Interns reported that resident facilitators frequently utilized specific debriefing techniques covered in the RaT curriculum during the debriefing sessions (Table 5).

DISCUSSION

We describe a unique RaT program embedded within a high‐fidelity medical simulation curriculum for IM interns. Our study demonstrates that resident facilitators noted an improvement in their teaching and feedback skills, both in the simulation setting and on the wards. Intern learners rated residents' teaching skills and the overall simulation curriculum highly, suggesting that residents were effective teachers.

The use of simulation in trainee‐as‐teacher curricula holds promise because it can provide an opportunity to teach in an environment closely approximating the wards, where trainees have the most opportunities to teach. However, in contrast to true ward‐based teaching, simulation can provide predictable scenarios in a controlled environment, which eliminates the distractions and unpredictability that exist on the wards, without compromising patient safety. Recently, Tofil et al. described the first use of simulation in a trainee‐as‐teacher program.[17] The investigators utilized a 1‐time simulation‐based teaching session, during which pediatric fellows completed a teacher‐training workshop, developed and served as facilitators in a simulated case, and received feedback. The use of simulation allowed fellows an opportunity to apply newly acquired skills in a controlled environment and receive feedback, which has been shown to improve teaching skills.[18]

The experience from our program expands on that of Tofil et al., as well as previously described trainee‐as‐teacher curricula, by introducing a component of deliberate practice that is unique to the simulation setting and has been absent from most previously described RaT programs.[3] Most residents had the opportunity to facilitate the same case on multiple occasions, allowing them to receive feedback and make adjustments. Residents who facilitated 5 or more sessions demonstrated more improvement, particularly in teaching outside of simulation, than residents who facilitated fewer sessions. It is notable that PGY‐II resident facilitators reported an improvement in their teaching skills on the wards, though less pronounced as compared to teaching in the simulation‐based environment, suggesting that benefits of the program may extend to nonsimulation‐based settings. Additional studies focusing on objective evaluation of ward‐based teaching are needed to further explore this phenomenon. Finally, the self‐reported improvements in medical knowledge by resident facilitators may serve as another benefit of our program.

Analysis of learner‐level data collected in the postcurriculum intern survey and DASH ratings provides additional support for the effectiveness of the RaT program. The majority of intern learners reported that resident facilitators used the techniques covered in our program frequently during debriefings. In addition, DASH scores clustered around maximum efficacy for all facilitators, suggesting that residents were effective teachers. Although we cannot directly assess whether the differences demonstrated in resident facilitators' self‐assessments translated to their teaching or were significant from the learners' perspective, these results support the hypothesis that self‐assessed improvements in teaching and feedback skills were significant.

In addition to improving resident teaching skills, our program had a positive impact on intern learners as evidenced by intern evaluations of the simulation curriculum. While utilizing relatively few faculty resources, our program was able to deliver an extensive and well‐received simulation curriculum to over 50 interns. The fact that 40% of second‐ and third‐year residents volunteered to teach in the program despite the early morning timing of the sessions speaks to the interest that trainees have in teaching in this setting. This model can serve as an important and efficient learning platform in residency training programs. It may be particularly salient to IM training programs where implementation of simulation curricula is challenging due to large numbers of residents and limited faculty resources. The barriers to and lessons learned from our experience with implementing the simulation curriculum have been previously described.[10, 11]

Our study has several limitations. Changes in residents' teaching skills were self‐assessed, which may be inaccurate as learners may overestimate their abilities.[19] Although we collected data on the experiences of intern learners that supported residents' self‐assessment, further studies using more objective measures (such as the Objective Structured Teaching Exercise[20]) should be undertaken. We did not objectively assess improvement of residents' teaching skills on the wards, with the exception of the residents' self assessment. Due to the timing of survey administration, some residents had as little as 1 month between completion of the curriculum and responding to the post‐curriculum survey, limiting their ability to evaluate their teaching skills on the wards. The transferability of the skills gained in simulation‐based teaching to teaching on the wards deserves further study. We cannot definitively attribute perceived improvement of teaching skills to the RaT program without a control group. However, the frequent use of recommended techniques during debriefing, which are not typically taught in other settings, supports the efficacy of the RaT program.

Our study did not allow us to determine which of the 3 components of the RaT program (workshop, facilitation practice, or direct observation and feedback) had the greatest impact on teaching skills or DASH ratings, as those who facilitated more sessions also completed the other components of the program. Furthermore, there may have been a selection bias among facilitators who facilitated more sessions. Because only 16 of 34 participants completed both the pre‐program and post‐program self‐assessments in a non‐anonymous fashion, we were not able to analyze the effect of pre‐program factors, such as prior teaching experience, on program outcomes. It should also be noted that allowing resident facilitators the option to complete the survey non‐anonymously could have biased our results. The simulation curriculum was conducted in a single center, and resident facilitators were self‐selecting; therefore, our results may not be generalizable. Finally, the DASH instrument was only administered after the RaT workshop and was likely limited further by the ceiling effect created by the learners' high satisfaction with the simulation program overall.

In summary, our simulation‐based RaT program improved resident facilitators' self‐reported teaching and feedback skills. Simulation‐based training provided an opportunity for deliberate practice of teaching skills in a controlled environment, which was a unique component of the program. The impact of deliberate practice on resident teaching skills and optimal methods to incorporate deliberate practice in RaT programs deserves further study. Our curriculum design may serve as a model for the development of simulation programs that can be employed to improve both intern learning and resident teaching skills.

Acknowledgements

The authors acknowledge Deborah Navedo, PhD, Assistant Professor, Massachusetts General Hospital Institute of Health Professions, and Emily M. Hayden, MD, Assistant Professor, Department of Emergency Medicine, Massachusetts General Hospital and Harvard Medical School, for their assistance with development of the RaT curriculum. The authors thank Dr. Jenny Rudolph, Senior Director, Institute for Medical Simulation at the Center for Medical Simulation, for her help in teaching us to use the DASH instrument. The authors also thank Dr. Daniel Hunt, MD, Associate Professor, Department of Medicine, Massachusetts General Hospital and Harvard Medical School, for his thoughtful review of this manuscript.

Disclosure: Nothing to report.

Files
References
  1. Liaison Committee on Medical Education. Functions and structure of a medical school: standards for accreditation of medical education programs leading to the M.D. degree. Washington, DC, and Chicago, IL: Association of American Medical Colleges and American Medical Association; 2000.
  2. Accreditation Council for Graduate Medical Education. ACGME program requirements for graduate medical education in pediatrics. Available at: http://www.acgme.org/acgmeweb/Portals/0/PFAssets/2013-PR-FAQ-PIF/320_pediatrics_07012013.pdf. Accessed June 18, 2014.
  3. Hill AG, Yu TC, Barrow M, Hattie J. A systematic review of resident‐as‐teacher programmes. Med Educ. 2009;43(12):11291140.
  4. Okuda Y, Bryson EO, DeMaria S, et al. The utility of simulation in medical education: what is the evidence? Mt Sinai J Med. 2009;76:330343.
  5. Okuda Y, Bond WF, Bonfante G, et al. National growth in simulation training within emergency medicine residency programs, 2003–2008. Acad Emerg Med. 2008;15:11131116.
  6. Fernandez R, Wang E, Vozenilek JA, et al. Simulation center accreditation and programmatic benchmarks: a review for emergency medicine. Acad Emerg Med. 2010;17(10):10931103.
  7. Cook DA. How much evidence does it take? A cumulative meta‐analysis of outcomes of simulation‐based education. Med Educ. 2014;48(8):750760.
  8. Farrell SE, Pacella C, Egan D, et al.. Resident‐as‐teacher: a suggested curriculum for emergency medicine. Acad Emerg Med. 2006;13(6):677679.
  9. Ericsson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med. 2004;79(10 suppl):S70S81.
  10. Miloslavsky EM, Hayden EM, Currier PF, Mathai SK, Contreras‐Valdes F, Gordon JA. Pilot program utilizing medical simulation in clinical decision making training for internal medicine interns. J Grad Med Educ. 2012;4:490495.
  11. Mathai SK, Miloslavsky EM, Contreras‐Valdes FM, et al. How we implemented a resident‐led medical simulation curriculum in a large internal medicine residency program. Med Teach. 2014;36(4):279283.
  12. Rudolph JW, Simon R, Dufresne RL, Raemer DB. There's no such thing as “nonjudgmental” debriefing: a theory and method for debriefing with good judgment. Simul Healthc. 2006;1:4955.
  13. Minehart RD, Rudolph JW, Pian‐Smith MCM, Raemer DB. Improving faculty feedback to resident trainees during a simulated case: a randomized, controlled trial of an educational intervention. Anesthesiology. 2014;120(1):160171.
  14. Gardner R. Introduction to debriefing. Semin Perinatol. 2013:37(3)166174.
  15. Howard G, Dailey PR. Response‐shift bias: a source of contamination of self‐report measures. J Appl Psychol. 1979;4:93106.
  16. Center for Medical Simulation. Debriefing assessment for simulation in healthcare. Available at: http://www.harvardmedsim.org/debriefing‐assesment‐simulation‐healthcare.php. Accessed June 18, 2014.
  17. Tofil NM, Peterson DT, Harrington KF, et al. A novel iterative‐learner simulation model: fellows as teachers. J Grad Med Educ. 2014;6(1):127132.
  18. Regan‐Smith M, Hirschmann K, Iobst W. Direct observation of faculty with feedback: an effective means of improving patient‐centered and learner‐centered teaching skills. Teach Learn Med. 2007;19(3):278286.
  19. Kruger J, Dunning D. Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self assessments. J Pers Soc Psychol. 1999;77:11211134.
  20. Morrison EH, Boker JR, Hollingshead J, et al. Reliability and validity of an objective structured teaching examination for generalist resident teachers. Acad Med. 2002;77(10 suppl):S29S32.
Article PDF
Issue
Journal of Hospital Medicine - 10(12)
Publications
Page Number
767-772
Sections
Files
Files
Article PDF
Article PDF

Residency training, in addition to developing clinical competence among trainees, is charged with improving resident teaching skills. The Liaison Committee on Medical Education and the Accreditation Council for Graduate Medical Education require that residents be provided with training or resources to develop their teaching skills.[1, 2] A variety of resident‐as‐teacher (RaT) programs have been described; however, the optimal format of such programs remains in question.[3] High‐fidelity medical simulation using mannequins has been shown to be an effective teaching tool in various medical specialties[4, 5, 6, 7] and may prove to be useful in teacher training.[8] Teaching in a simulation‐based environment can give participants the opportunity to apply their teaching skills in a clinical environment, as they would on the wards, but in a more controlled, predictable setting and without compromising patient safety. In addition, simulation offers the opportunity to engage in deliberate practice by allowing teachers to facilitate the same case on multiple occasions with different learners. Deliberate practice, which involves task repetition with feedback aimed at improving performance, has been shown to be important in developing expertise.[9]

We previously described the first use of a high‐fidelity simulation curriculum for internal medicine (IM) interns focused on clinical decision‐making skills, in which second‐ and third‐year residents served as facilitators.[10, 11] Herein, we describe a RaT program in which residents participated in a workshop, then served as facilitators in the intern curriculum and received feedback from faculty. We hypothesized that such a program would improve residents' teaching and feedback skills, both in the simulation environment and on the wards.

METHODS

We conducted a single‐group study evaluating teaching and feedback skills among upper‐level resident facilitators before and after participation in the RaT program. We measured residents' teaching skills using pre‐ and post‐program self‐assessments as well as evaluations completed by the intern learners after each session and at the completion of the curriculum.

Setting and Participants

We embedded the RaT program within a simulation curriculum administered July to October of 2013 for all IM interns at Massachusetts General Hospital (interns in the preliminary program who planned to pursue another field after the completion of the intern year were excluded) (n = 52). We invited postgraduate year (PGY) II and III residents (n = 102) to participate in the IM simulation program as facilitators via email. The curriculum consisted of 8 cases focusing on acute clinical scenarios encountered on the general medicine wards. The cases were administered during 1‐hour sessions 4 mornings per week from 7 AM to 8 AM prior to clinical duties. Interns completed the curriculum over 4 sessions during their outpatient rotation. The case topics were (1) hypertensive emergency, (2) post‐procedure bleed, (3) congestive heart failure, (4) atrial fibrillation with rapid ventricular response, (5) altered mental status/alcohol withdrawal, (6) nonsustained ventricular tachycardia heralding acute coronary syndrome, (7) cardiac tamponade, and (8) anaphylaxis. During each session, groups of 2 to 3 interns worked through 2 cases using a high‐fidelity mannequin (Laerdal 3G, Wappingers Falls, NY) with 2 resident facilitators. One facilitator operated the mannequin, while the other served as a nurse. Each case was followed by a structured debriefing led by 1 of the resident facilitators (facilitators switched roles for the second case). The number of sessions facilitated varied for each resident based on individual schedules and preferences.

Four senior residents who were appointed as simulation leaders (G.A.A., J.K.H., R.K., Z.S.) and 2 faculty advisors (P.F.C., E.M.M.) administered the program. Simulation resident leaders scheduled facilitators and interns and participated in a portion of simulation sessions as facilitators, but they were not analyzed as participants for the purposes of this study. The curriculum was administered without interfering with clinical duties, and no additional time was protected for interns or residents participating in the curriculum.

Resident‐as‐Teacher Program Structure

We invited participating resident facilitators to attend a 1‐hour interactive workshop prior to serving as facilitators. The workshop focused on building learner‐centered and small‐group teaching skills, as well as introducing residents to a 5‐stage debriefing framework developed by the authors and based on simulation debriefing best practices (Table 1).[12, 13, 14]

Stages of Debriefing
Stage of DebriefingActionRationale
  • NOTE: *To standardize the learner experience, all interns received an e‐mail after each session describing the key learning objectives and takeaway points with references from the medical literature for each case.

Emotional responseElicit learners' emotions about the caseIt is important to acknowledge and address both positive and negative emotions that arise during the case before debriefing the specific medical and communications aspects of the case. Unaddressed emotional responses may hinder subsequent debriefing.
Objectives*Elicit learners' objectives and combine them with the stated learning objectives of the case to determine debriefing objectivesThe limited amount of time allocated for debriefing (1520 minutes) does not allow the facilitator to cover all aspects of medical management and communication skills in a particular case. Focusing on the most salient objectives, including those identified by the learners, allows the facilitator to engage in learner‐centered debriefing.
AnalysisAnalyze the learners' approach to the caseAnalyzing the learners' approach to the case using the advocacy‐inquiry method[11] seeks to uncover the learner's assumptions/frameworks behind the decision made during the case. This approach allows the facilitator to understand the learners' thought process and target teaching points to more precisely address the learners' needs.
TeachingAddress knowledge gaps and incorrect assumptionsLearner‐centered debriefing within a limited timeframe requires teaching to be brief and targeted toward the defined objectives. It should also address the knowledge gaps and incorrect assumptions uncovered during the analysis phase.
SummarySummarize key takeawaysSummarizing highlights the key points of the debriefing and can be used to suggest further exploration of topics through self‐study (if necessary).

Resident facilitators were observed by simulation faculty and simulation resident leaders throughout the intern curriculum and given structured feedback either in‐person immediately after completion of the simulation session or via a detailed same‐day e‐mail if the time allotted for feedback was not sufficient. Feedback was structured by the 5 stages of debriefing described in Table 1, and included soliciting residents' observations on the teaching experience and specific behaviors observed by faculty during the scenarios. E‐mail feedback (also structured by stages of debriefing and including observed behaviors) was typically followed by verbal feedback during the next simulation session.

The RaT program was composed of 3 elements: the workshop, case facilitation, and direct observation with feedback. Because we felt that the opportunity for directly observed teaching and feedback in a ward‐like controlled environment was a unique advantage offered by the simulation setting, we included all residents who served as facilitators in the analysis, regardless of whether or not they had attended the workshop.

Evaluation Instruments

Survey instruments were developed by the investigators, reviewed by several experts in simulation, pilot tested among residents not participating in the simulation program, and revised by the investigators.

Pre‐program Facilitator Survey

Prior to the RaT workshop, resident facilitators completed a baseline survey evaluating their preparedness to teach and give feedback on the wards and in a simulation‐based setting on a 5‐point scale (see Supporting Information, Appendix I, in the online version of this article).

Post‐program Facilitator Survey

Approximately 3 weeks after completion of the intern simulation curriculum, resident facilitators were asked to complete an online post‐program survey, which remained open for 1 month (residents completed this survey anywhere from 3 weeks to 4 months after their participation in the RaT program depending on the timing of their facilitation). The survey asked residents to evaluate their comfort with their current post‐program teaching skills as well as their pre‐program skills in retrospect, as previous research demonstrated that learners may overestimate their skills prior to training programs.[15] Resident facilitators could complete the surveys nonanonymously to allow for matched‐pairs analysis of the change in teaching skills over the course of the program (see Supporting Information, Appendix II, in the online version of this article).

Intern Evaluation of Facilitator Debriefing Skills

After each case, intern learners were asked to anonymously evaluate the teaching effectiveness of the lead resident facilitator using the adapted Debriefing Assessment for Simulation in Healthcare (DASH) instrument.[16] The DASH instrument evaluated the following domains: (1) instructor maintained an engaging context for learning, (2) instructor structured the debriefing in an organized way, (3) instructor provoked in‐depth discussions that led me to reflect on my performance, (4) instructor identified what I did well or poorly and why, (5) instructor helped me see how to improve or how to sustain good performance, (6) overall effectiveness of the simulation session (see Supporting Information, Appendix III, in the online version of this article).

Post‐program Intern Survey

Two months following the completion of the simulation curriculum, intern learners received an anonymous online post‐program evaluation assessing program efficacy and resident facilitator teaching (see Supporting Information, Appendix IV, in the online version of this article).

Statistical Analysis

Teaching skills and learners' DASH ratings were compared using the Student t test, Pearson 2 test, and Fisher exact test as appropriate. Pre‐ and post‐program rating of teaching skills was undertaken in aggregate and as a matched‐pairs analysis.

The study was approved by the Partners Institutional Review Board.

RESULTS

Forty‐one resident facilitators participated in 118 individual simulation sessions encompassing 236 case scenarios. Thirty‐four residents completed the post‐program facilitator survey and were included in the analysis. Of these, 26 (76%) participated in the workshop and completed the pre‐program survey. Twenty‐three of the 34 residents (68%) completed the post‐program evaluation nonanonymously (13 PGY‐II, 10 PGY‐III). Of these, 16 completed the pre‐program survey nonanonymously. The average number of sessions facilitated by each resident was 3.9 (range, 112).

Pre‐ and Post‐program Self‐Assessment of Residents' Teaching Skills

Participation in the simulation RaT program led to improvements in resident facilitators' self‐reported teaching skills across multiple domains (Table 2). These results were consistent when using the retrospective pre‐program assessment in matched‐pairs analysis (n=34) and when performing the analysis using the true pre‐program preparedness compared to post‐program comfort with teaching skills in a non‐matched‐pairs fashion (n = 26) and matched‐pairs fashion (n = 16). We report P values for the more conservative estimates using the retrospective pre‐program assessment matched‐pairs analysis. The most significant improvements occurred in residents' ability to teach in a simulated environment (2.81 to 4.16, P < 0.001 [5‐point scale]) and give feedback (3.35 to 3.77, P < 0.001).

Pre‐ and Post‐program Self‐Assessment of Resident Facilitators Teaching Skills*
 Pre‐program Rating (n = 34)Post‐program Rating (n = 34)P Value
  • NOTE: *Survey data were collected before participation in the workshop and 3 weeks after completion of the 4‐month curriculum. Five‐point Likert scale: very uncomfortable (1) to very comfortable (5).

Teaching on rounds3.754.030.005
Teaching on wards outside rounds3.834.070.007
Teaching in simulation2.814.16<0.001
Giving feedback3.353.77<0.001

Resident facilitators reported that participation in the RaT program had a significant impact on their teaching skills both within and outside of the simulation environment (Table 3). However, the greatest gains were seen in the domain of teaching in simulation. It was also noted that participation in the program improved resident facilitators' medical knowledge.

Resident Facilitators' Perceived Improvement in Skills Due to Resident‐as‐Teacher Program
CategoryNot at AllSlightly ImprovedModerately ImprovedGreatly ImprovedNot Sure
Teaching on rounds, n = 344 (12%)12 (35%)13 (38%)4 (12%)1 (3%)
Teaching on wards outside rounds, n = 343 (9%)13 (38%)12 (35%)5 (15%)1 (3%)
Teaching in simulation, n = 340 (0%)4 (12%)7 (21%)23 (68%)0 (0%)
Giving feedback, n = 344 (12%)10 (29%)12 (35%)6 (18%)2 (6%)
Medical knowledge, n = 342 (6%)11 (32%)18 (53%)3 (9%)0 (0%)

Subgroup analyses were performed comparing the perceived improvement in teaching and feedback skills among those who did or did not attend the facilitator workshop, those who facilitated 5 or more versus less than 5 sessions, and those who received or did not receive direct observation and feedback from faculty. Although numerically greater gains were seen across all 4 domains among those who attended the workshop, facilitated 5 or more sessions, or received feedback from faculty, only teaching on rounds and on the wards outside rounds reached statistical significance (Table 4). It should be noted that all residents who facilitated 5 or more sessions also attended the workshop and received feedback from faculty. We also compared perceived improvement among PGY‐II and PGY‐III residents. In contrast to PGY‐II residents, who demonstrated an improvement in all 4 domains, PGY‐III residents only demonstrated improvement in simulation‐based teaching.

Pre‐ and Post‐program Self‐Assessment of Resident Facilitators Teaching Skills According to Number of Sessions Facilitated, Workshop Attendance, Receipt of Feedback, and PGY Year
 Pre‐programPost‐programP ValuePre‐programPost‐programP Value
 Facilitated Less Than 5 Sessions (n = 18)Facilitated 5 or More Sessions (n = 11)
 Did Not Attend Workshop (n = 10)Attended Workshop (n = 22)
 Received Feedback From Resident Leaders Only (n = 11)Received Faculty Feedback (n = 21)
 PGY‐II (n = 13)PGY‐III (n = 9)
  • NOTE: Abbreviations: PGY, postgraduate year.

Teaching on rounds3.683.790.163.854.380.01
Teaching on wards outside rounds3.8240.083.854.150.04
Teaching in simulation2.894.06<0.012.694.31<0.01
Giving feedback3.333.670.013.383.920.01
Teaching on rounds44.10.343.644<0.01
Teaching on wards outside rounds441.003.764.1<0.01
Teaching in simulation2.894.11<0.012.774.18<0.01
Giving feedback3.563.780.173.273.77<0.01
Teaching on rounds3.553.820.193.864.140.01
Teaching on wards outside rounds441.003.754.1<0.01
Teaching in simulation2.73.8<0.012.864.33<0.01
Giving feedback3.23.60.043.433.86<0.01
Teaching on rounds3.383.850.034.224.221
Teaching on wards outside rounds3.543.850.044.144.141
Teaching in simulation2.464.15<0.013.134.13<0.01
Giving feedback3.233.620.023.53.880.08

Intern Learners' Assessment of Resident Facilitators and the Program Overall

During the course of the program, intern learners completed 166 DASH ratings evaluating 34 resident facilitators (see Supporting Information, Appendix V, in the online version of this article). Ratings for the 6 DASH items ranged from 6.49 to 6.73 (7‐point scale), demonstrating a high level of facilitator efficacy across multiple domains. No differences in DASH scores were noted among subgroups of resident facilitators described in the previous paragraph.

Thirty‐eight of 52 intern learners (73%) completed the post‐program survey.

Resident Facilitators' Use of Specific Teaching Skills During Debriefing as Rated by Intern Learners
Facilitator BehaviorsVery Often, >75%Often, >50%Sometimes, 25%50%Rarely, <25%Never
Elicited emotional reactions, n = 3818 (47%)16 (42%)4 (11%)0 (0%)0 (0%)
Elicited objectives from learner, n = 3726 (69%)8 (22%)2 (6%)1 (3%)0 (0%)
Asked to share clinical reasoning, n = 3821 (56%)13 (33%)4 (11%)0 (0%)0 (0%)
Summarized learning points, n = 3831 (81%)7 (19%)0 (0%)0 (0%)0 (0%)
Spoke for less than half of the session, n = 388 (22%)17 (44%)11 (28%)2 (6%)0 (0%)

All intern learners rated the overall simulation experience as either excellent (81%) or good (19%) on the post‐program evaluation (4 or 5 on a 5‐point Likert scale, respectively). All interns strongly agreed (72%) or agreed (28%) that the simulation sessions improved their ability to manage acute clinical scenarios. Interns reported that resident facilitators frequently utilized specific debriefing techniques covered in the RaT curriculum during the debriefing sessions (Table 5).

DISCUSSION

We describe a unique RaT program embedded within a high‐fidelity medical simulation curriculum for IM interns. Our study demonstrates that resident facilitators noted an improvement in their teaching and feedback skills, both in the simulation setting and on the wards. Intern learners rated residents' teaching skills and the overall simulation curriculum highly, suggesting that residents were effective teachers.

The use of simulation in trainee‐as‐teacher curricula holds promise because it can provide an opportunity to teach in an environment closely approximating the wards, where trainees have the most opportunities to teach. However, in contrast to true ward‐based teaching, simulation can provide predictable scenarios in a controlled environment, which eliminates the distractions and unpredictability that exist on the wards, without compromising patient safety. Recently, Tofil et al. described the first use of simulation in a trainee‐as‐teacher program.[17] The investigators utilized a 1‐time simulation‐based teaching session, during which pediatric fellows completed a teacher‐training workshop, developed and served as facilitators in a simulated case, and received feedback. The use of simulation allowed fellows an opportunity to apply newly acquired skills in a controlled environment and receive feedback, which has been shown to improve teaching skills.[18]

The experience from our program expands on that of Tofil et al., as well as previously described trainee‐as‐teacher curricula, by introducing a component of deliberate practice that is unique to the simulation setting and has been absent from most previously described RaT programs.[3] Most residents had the opportunity to facilitate the same case on multiple occasions, allowing them to receive feedback and make adjustments. Residents who facilitated 5 or more sessions demonstrated more improvement, particularly in teaching outside of simulation, than residents who facilitated fewer sessions. It is notable that PGY‐II resident facilitators reported an improvement in their teaching skills on the wards, though less pronounced as compared to teaching in the simulation‐based environment, suggesting that benefits of the program may extend to nonsimulation‐based settings. Additional studies focusing on objective evaluation of ward‐based teaching are needed to further explore this phenomenon. Finally, the self‐reported improvements in medical knowledge by resident facilitators may serve as another benefit of our program.

Analysis of learner‐level data collected in the postcurriculum intern survey and DASH ratings provides additional support for the effectiveness of the RaT program. The majority of intern learners reported that resident facilitators used the techniques covered in our program frequently during debriefings. In addition, DASH scores clustered around maximum efficacy for all facilitators, suggesting that residents were effective teachers. Although we cannot directly assess whether the differences demonstrated in resident facilitators' self‐assessments translated to their teaching or were significant from the learners' perspective, these results support the hypothesis that self‐assessed improvements in teaching and feedback skills were significant.

In addition to improving resident teaching skills, our program had a positive impact on intern learners as evidenced by intern evaluations of the simulation curriculum. While utilizing relatively few faculty resources, our program was able to deliver an extensive and well‐received simulation curriculum to over 50 interns. The fact that 40% of second‐ and third‐year residents volunteered to teach in the program despite the early morning timing of the sessions speaks to the interest that trainees have in teaching in this setting. This model can serve as an important and efficient learning platform in residency training programs. It may be particularly salient to IM training programs where implementation of simulation curricula is challenging due to large numbers of residents and limited faculty resources. The barriers to and lessons learned from our experience with implementing the simulation curriculum have been previously described.[10, 11]

Our study has several limitations. Changes in residents' teaching skills were self‐assessed, which may be inaccurate as learners may overestimate their abilities.[19] Although we collected data on the experiences of intern learners that supported residents' self‐assessment, further studies using more objective measures (such as the Objective Structured Teaching Exercise[20]) should be undertaken. We did not objectively assess improvement of residents' teaching skills on the wards, with the exception of the residents' self assessment. Due to the timing of survey administration, some residents had as little as 1 month between completion of the curriculum and responding to the post‐curriculum survey, limiting their ability to evaluate their teaching skills on the wards. The transferability of the skills gained in simulation‐based teaching to teaching on the wards deserves further study. We cannot definitively attribute perceived improvement of teaching skills to the RaT program without a control group. However, the frequent use of recommended techniques during debriefing, which are not typically taught in other settings, supports the efficacy of the RaT program.

Our study did not allow us to determine which of the 3 components of the RaT program (workshop, facilitation practice, or direct observation and feedback) had the greatest impact on teaching skills or DASH ratings, as those who facilitated more sessions also completed the other components of the program. Furthermore, there may have been a selection bias among facilitators who facilitated more sessions. Because only 16 of 34 participants completed both the pre‐program and post‐program self‐assessments in a non‐anonymous fashion, we were not able to analyze the effect of pre‐program factors, such as prior teaching experience, on program outcomes. It should also be noted that allowing resident facilitators the option to complete the survey non‐anonymously could have biased our results. The simulation curriculum was conducted in a single center, and resident facilitators were self‐selecting; therefore, our results may not be generalizable. Finally, the DASH instrument was only administered after the RaT workshop and was likely limited further by the ceiling effect created by the learners' high satisfaction with the simulation program overall.

In summary, our simulation‐based RaT program improved resident facilitators' self‐reported teaching and feedback skills. Simulation‐based training provided an opportunity for deliberate practice of teaching skills in a controlled environment, which was a unique component of the program. The impact of deliberate practice on resident teaching skills and optimal methods to incorporate deliberate practice in RaT programs deserves further study. Our curriculum design may serve as a model for the development of simulation programs that can be employed to improve both intern learning and resident teaching skills.

Acknowledgements

The authors acknowledge Deborah Navedo, PhD, Assistant Professor, Massachusetts General Hospital Institute of Health Professions, and Emily M. Hayden, MD, Assistant Professor, Department of Emergency Medicine, Massachusetts General Hospital and Harvard Medical School, for their assistance with development of the RaT curriculum. The authors thank Dr. Jenny Rudolph, Senior Director, Institute for Medical Simulation at the Center for Medical Simulation, for her help in teaching us to use the DASH instrument. The authors also thank Dr. Daniel Hunt, MD, Associate Professor, Department of Medicine, Massachusetts General Hospital and Harvard Medical School, for his thoughtful review of this manuscript.

Disclosure: Nothing to report.

Residency training, in addition to developing clinical competence among trainees, is charged with improving resident teaching skills. The Liaison Committee on Medical Education and the Accreditation Council for Graduate Medical Education require that residents be provided with training or resources to develop their teaching skills.[1, 2] A variety of resident‐as‐teacher (RaT) programs have been described; however, the optimal format of such programs remains in question.[3] High‐fidelity medical simulation using mannequins has been shown to be an effective teaching tool in various medical specialties[4, 5, 6, 7] and may prove to be useful in teacher training.[8] Teaching in a simulation‐based environment can give participants the opportunity to apply their teaching skills in a clinical environment, as they would on the wards, but in a more controlled, predictable setting and without compromising patient safety. In addition, simulation offers the opportunity to engage in deliberate practice by allowing teachers to facilitate the same case on multiple occasions with different learners. Deliberate practice, which involves task repetition with feedback aimed at improving performance, has been shown to be important in developing expertise.[9]

We previously described the first use of a high‐fidelity simulation curriculum for internal medicine (IM) interns focused on clinical decision‐making skills, in which second‐ and third‐year residents served as facilitators.[10, 11] Herein, we describe a RaT program in which residents participated in a workshop, then served as facilitators in the intern curriculum and received feedback from faculty. We hypothesized that such a program would improve residents' teaching and feedback skills, both in the simulation environment and on the wards.

METHODS

We conducted a single‐group study evaluating teaching and feedback skills among upper‐level resident facilitators before and after participation in the RaT program. We measured residents' teaching skills using pre‐ and post‐program self‐assessments as well as evaluations completed by the intern learners after each session and at the completion of the curriculum.

Setting and Participants

We embedded the RaT program within a simulation curriculum administered July to October of 2013 for all IM interns at Massachusetts General Hospital (interns in the preliminary program who planned to pursue another field after the completion of the intern year were excluded) (n = 52). We invited postgraduate year (PGY) II and III residents (n = 102) to participate in the IM simulation program as facilitators via email. The curriculum consisted of 8 cases focusing on acute clinical scenarios encountered on the general medicine wards. The cases were administered during 1‐hour sessions 4 mornings per week from 7 AM to 8 AM prior to clinical duties. Interns completed the curriculum over 4 sessions during their outpatient rotation. The case topics were (1) hypertensive emergency, (2) post‐procedure bleed, (3) congestive heart failure, (4) atrial fibrillation with rapid ventricular response, (5) altered mental status/alcohol withdrawal, (6) nonsustained ventricular tachycardia heralding acute coronary syndrome, (7) cardiac tamponade, and (8) anaphylaxis. During each session, groups of 2 to 3 interns worked through 2 cases using a high‐fidelity mannequin (Laerdal 3G, Wappingers Falls, NY) with 2 resident facilitators. One facilitator operated the mannequin, while the other served as a nurse. Each case was followed by a structured debriefing led by 1 of the resident facilitators (facilitators switched roles for the second case). The number of sessions facilitated varied for each resident based on individual schedules and preferences.

Four senior residents who were appointed as simulation leaders (G.A.A., J.K.H., R.K., Z.S.) and 2 faculty advisors (P.F.C., E.M.M.) administered the program. Simulation resident leaders scheduled facilitators and interns and participated in a portion of simulation sessions as facilitators, but they were not analyzed as participants for the purposes of this study. The curriculum was administered without interfering with clinical duties, and no additional time was protected for interns or residents participating in the curriculum.

Resident‐as‐Teacher Program Structure

We invited participating resident facilitators to attend a 1‐hour interactive workshop prior to serving as facilitators. The workshop focused on building learner‐centered and small‐group teaching skills, as well as introducing residents to a 5‐stage debriefing framework developed by the authors and based on simulation debriefing best practices (Table 1).[12, 13, 14]

Stages of Debriefing
Stage of DebriefingActionRationale
  • NOTE: *To standardize the learner experience, all interns received an e‐mail after each session describing the key learning objectives and takeaway points with references from the medical literature for each case.

Emotional responseElicit learners' emotions about the caseIt is important to acknowledge and address both positive and negative emotions that arise during the case before debriefing the specific medical and communications aspects of the case. Unaddressed emotional responses may hinder subsequent debriefing.
Objectives*Elicit learners' objectives and combine them with the stated learning objectives of the case to determine debriefing objectivesThe limited amount of time allocated for debriefing (1520 minutes) does not allow the facilitator to cover all aspects of medical management and communication skills in a particular case. Focusing on the most salient objectives, including those identified by the learners, allows the facilitator to engage in learner‐centered debriefing.
AnalysisAnalyze the learners' approach to the caseAnalyzing the learners' approach to the case using the advocacy‐inquiry method[11] seeks to uncover the learner's assumptions/frameworks behind the decision made during the case. This approach allows the facilitator to understand the learners' thought process and target teaching points to more precisely address the learners' needs.
TeachingAddress knowledge gaps and incorrect assumptionsLearner‐centered debriefing within a limited timeframe requires teaching to be brief and targeted toward the defined objectives. It should also address the knowledge gaps and incorrect assumptions uncovered during the analysis phase.
SummarySummarize key takeawaysSummarizing highlights the key points of the debriefing and can be used to suggest further exploration of topics through self‐study (if necessary).

Resident facilitators were observed by simulation faculty and simulation resident leaders throughout the intern curriculum and given structured feedback either in‐person immediately after completion of the simulation session or via a detailed same‐day e‐mail if the time allotted for feedback was not sufficient. Feedback was structured by the 5 stages of debriefing described in Table 1, and included soliciting residents' observations on the teaching experience and specific behaviors observed by faculty during the scenarios. E‐mail feedback (also structured by stages of debriefing and including observed behaviors) was typically followed by verbal feedback during the next simulation session.

The RaT program was composed of 3 elements: the workshop, case facilitation, and direct observation with feedback. Because we felt that the opportunity for directly observed teaching and feedback in a ward‐like controlled environment was a unique advantage offered by the simulation setting, we included all residents who served as facilitators in the analysis, regardless of whether or not they had attended the workshop.

Evaluation Instruments

Survey instruments were developed by the investigators, reviewed by several experts in simulation, pilot tested among residents not participating in the simulation program, and revised by the investigators.

Pre‐program Facilitator Survey

Prior to the RaT workshop, resident facilitators completed a baseline survey evaluating their preparedness to teach and give feedback on the wards and in a simulation‐based setting on a 5‐point scale (see Supporting Information, Appendix I, in the online version of this article).

Post‐program Facilitator Survey

Approximately 3 weeks after completion of the intern simulation curriculum, resident facilitators were asked to complete an online post‐program survey, which remained open for 1 month (residents completed this survey anywhere from 3 weeks to 4 months after their participation in the RaT program depending on the timing of their facilitation). The survey asked residents to evaluate their comfort with their current post‐program teaching skills as well as their pre‐program skills in retrospect, as previous research demonstrated that learners may overestimate their skills prior to training programs.[15] Resident facilitators could complete the surveys nonanonymously to allow for matched‐pairs analysis of the change in teaching skills over the course of the program (see Supporting Information, Appendix II, in the online version of this article).

Intern Evaluation of Facilitator Debriefing Skills

After each case, intern learners were asked to anonymously evaluate the teaching effectiveness of the lead resident facilitator using the adapted Debriefing Assessment for Simulation in Healthcare (DASH) instrument.[16] The DASH instrument evaluated the following domains: (1) instructor maintained an engaging context for learning, (2) instructor structured the debriefing in an organized way, (3) instructor provoked in‐depth discussions that led me to reflect on my performance, (4) instructor identified what I did well or poorly and why, (5) instructor helped me see how to improve or how to sustain good performance, (6) overall effectiveness of the simulation session (see Supporting Information, Appendix III, in the online version of this article).

Post‐program Intern Survey

Two months following the completion of the simulation curriculum, intern learners received an anonymous online post‐program evaluation assessing program efficacy and resident facilitator teaching (see Supporting Information, Appendix IV, in the online version of this article).

Statistical Analysis

Teaching skills and learners' DASH ratings were compared using the Student t test, Pearson 2 test, and Fisher exact test as appropriate. Pre‐ and post‐program rating of teaching skills was undertaken in aggregate and as a matched‐pairs analysis.

The study was approved by the Partners Institutional Review Board.

RESULTS

Forty‐one resident facilitators participated in 118 individual simulation sessions encompassing 236 case scenarios. Thirty‐four residents completed the post‐program facilitator survey and were included in the analysis. Of these, 26 (76%) participated in the workshop and completed the pre‐program survey. Twenty‐three of the 34 residents (68%) completed the post‐program evaluation nonanonymously (13 PGY‐II, 10 PGY‐III). Of these, 16 completed the pre‐program survey nonanonymously. The average number of sessions facilitated by each resident was 3.9 (range, 112).

Pre‐ and Post‐program Self‐Assessment of Residents' Teaching Skills

Participation in the simulation RaT program led to improvements in resident facilitators' self‐reported teaching skills across multiple domains (Table 2). These results were consistent when using the retrospective pre‐program assessment in matched‐pairs analysis (n=34) and when performing the analysis using the true pre‐program preparedness compared to post‐program comfort with teaching skills in a non‐matched‐pairs fashion (n = 26) and matched‐pairs fashion (n = 16). We report P values for the more conservative estimates using the retrospective pre‐program assessment matched‐pairs analysis. The most significant improvements occurred in residents' ability to teach in a simulated environment (2.81 to 4.16, P < 0.001 [5‐point scale]) and give feedback (3.35 to 3.77, P < 0.001).

Pre‐ and Post‐program Self‐Assessment of Resident Facilitators Teaching Skills*
 Pre‐program Rating (n = 34)Post‐program Rating (n = 34)P Value
  • NOTE: *Survey data were collected before participation in the workshop and 3 weeks after completion of the 4‐month curriculum. Five‐point Likert scale: very uncomfortable (1) to very comfortable (5).

Teaching on rounds3.754.030.005
Teaching on wards outside rounds3.834.070.007
Teaching in simulation2.814.16<0.001
Giving feedback3.353.77<0.001

Resident facilitators reported that participation in the RaT program had a significant impact on their teaching skills both within and outside of the simulation environment (Table 3). However, the greatest gains were seen in the domain of teaching in simulation. It was also noted that participation in the program improved resident facilitators' medical knowledge.

Resident Facilitators' Perceived Improvement in Skills Due to Resident‐as‐Teacher Program
CategoryNot at AllSlightly ImprovedModerately ImprovedGreatly ImprovedNot Sure
Teaching on rounds, n = 344 (12%)12 (35%)13 (38%)4 (12%)1 (3%)
Teaching on wards outside rounds, n = 343 (9%)13 (38%)12 (35%)5 (15%)1 (3%)
Teaching in simulation, n = 340 (0%)4 (12%)7 (21%)23 (68%)0 (0%)
Giving feedback, n = 344 (12%)10 (29%)12 (35%)6 (18%)2 (6%)
Medical knowledge, n = 342 (6%)11 (32%)18 (53%)3 (9%)0 (0%)

Subgroup analyses were performed comparing the perceived improvement in teaching and feedback skills among those who did or did not attend the facilitator workshop, those who facilitated 5 or more versus less than 5 sessions, and those who received or did not receive direct observation and feedback from faculty. Although numerically greater gains were seen across all 4 domains among those who attended the workshop, facilitated 5 or more sessions, or received feedback from faculty, only teaching on rounds and on the wards outside rounds reached statistical significance (Table 4). It should be noted that all residents who facilitated 5 or more sessions also attended the workshop and received feedback from faculty. We also compared perceived improvement among PGY‐II and PGY‐III residents. In contrast to PGY‐II residents, who demonstrated an improvement in all 4 domains, PGY‐III residents only demonstrated improvement in simulation‐based teaching.

Pre‐ and Post‐program Self‐Assessment of Resident Facilitators Teaching Skills According to Number of Sessions Facilitated, Workshop Attendance, Receipt of Feedback, and PGY Year
 Pre‐programPost‐programP ValuePre‐programPost‐programP Value
 Facilitated Less Than 5 Sessions (n = 18)Facilitated 5 or More Sessions (n = 11)
 Did Not Attend Workshop (n = 10)Attended Workshop (n = 22)
 Received Feedback From Resident Leaders Only (n = 11)Received Faculty Feedback (n = 21)
 PGY‐II (n = 13)PGY‐III (n = 9)
  • NOTE: Abbreviations: PGY, postgraduate year.

Teaching on rounds3.683.790.163.854.380.01
Teaching on wards outside rounds3.8240.083.854.150.04
Teaching in simulation2.894.06<0.012.694.31<0.01
Giving feedback3.333.670.013.383.920.01
Teaching on rounds44.10.343.644<0.01
Teaching on wards outside rounds441.003.764.1<0.01
Teaching in simulation2.894.11<0.012.774.18<0.01
Giving feedback3.563.780.173.273.77<0.01
Teaching on rounds3.553.820.193.864.140.01
Teaching on wards outside rounds441.003.754.1<0.01
Teaching in simulation2.73.8<0.012.864.33<0.01
Giving feedback3.23.60.043.433.86<0.01
Teaching on rounds3.383.850.034.224.221
Teaching on wards outside rounds3.543.850.044.144.141
Teaching in simulation2.464.15<0.013.134.13<0.01
Giving feedback3.233.620.023.53.880.08

Intern Learners' Assessment of Resident Facilitators and the Program Overall

During the course of the program, intern learners completed 166 DASH ratings evaluating 34 resident facilitators (see Supporting Information, Appendix V, in the online version of this article). Ratings for the 6 DASH items ranged from 6.49 to 6.73 (7‐point scale), demonstrating a high level of facilitator efficacy across multiple domains. No differences in DASH scores were noted among subgroups of resident facilitators described in the previous paragraph.

Thirty‐eight of 52 intern learners (73%) completed the post‐program survey.

Resident Facilitators' Use of Specific Teaching Skills During Debriefing as Rated by Intern Learners
Facilitator BehaviorsVery Often, >75%Often, >50%Sometimes, 25%50%Rarely, <25%Never
Elicited emotional reactions, n = 3818 (47%)16 (42%)4 (11%)0 (0%)0 (0%)
Elicited objectives from learner, n = 3726 (69%)8 (22%)2 (6%)1 (3%)0 (0%)
Asked to share clinical reasoning, n = 3821 (56%)13 (33%)4 (11%)0 (0%)0 (0%)
Summarized learning points, n = 3831 (81%)7 (19%)0 (0%)0 (0%)0 (0%)
Spoke for less than half of the session, n = 388 (22%)17 (44%)11 (28%)2 (6%)0 (0%)

All intern learners rated the overall simulation experience as either excellent (81%) or good (19%) on the post‐program evaluation (4 or 5 on a 5‐point Likert scale, respectively). All interns strongly agreed (72%) or agreed (28%) that the simulation sessions improved their ability to manage acute clinical scenarios. Interns reported that resident facilitators frequently utilized specific debriefing techniques covered in the RaT curriculum during the debriefing sessions (Table 5).

DISCUSSION

We describe a unique RaT program embedded within a high‐fidelity medical simulation curriculum for IM interns. Our study demonstrates that resident facilitators noted an improvement in their teaching and feedback skills, both in the simulation setting and on the wards. Intern learners rated residents' teaching skills and the overall simulation curriculum highly, suggesting that residents were effective teachers.

The use of simulation in trainee‐as‐teacher curricula holds promise because it can provide an opportunity to teach in an environment closely approximating the wards, where trainees have the most opportunities to teach. However, in contrast to true ward‐based teaching, simulation can provide predictable scenarios in a controlled environment, which eliminates the distractions and unpredictability that exist on the wards, without compromising patient safety. Recently, Tofil et al. described the first use of simulation in a trainee‐as‐teacher program.[17] The investigators utilized a 1‐time simulation‐based teaching session, during which pediatric fellows completed a teacher‐training workshop, developed and served as facilitators in a simulated case, and received feedback. The use of simulation allowed fellows an opportunity to apply newly acquired skills in a controlled environment and receive feedback, which has been shown to improve teaching skills.[18]

The experience from our program expands on that of Tofil et al., as well as previously described trainee‐as‐teacher curricula, by introducing a component of deliberate practice that is unique to the simulation setting and has been absent from most previously described RaT programs.[3] Most residents had the opportunity to facilitate the same case on multiple occasions, allowing them to receive feedback and make adjustments. Residents who facilitated 5 or more sessions demonstrated more improvement, particularly in teaching outside of simulation, than residents who facilitated fewer sessions. It is notable that PGY‐II resident facilitators reported an improvement in their teaching skills on the wards, though less pronounced as compared to teaching in the simulation‐based environment, suggesting that benefits of the program may extend to nonsimulation‐based settings. Additional studies focusing on objective evaluation of ward‐based teaching are needed to further explore this phenomenon. Finally, the self‐reported improvements in medical knowledge by resident facilitators may serve as another benefit of our program.

Analysis of learner‐level data collected in the postcurriculum intern survey and DASH ratings provides additional support for the effectiveness of the RaT program. The majority of intern learners reported that resident facilitators used the techniques covered in our program frequently during debriefings. In addition, DASH scores clustered around maximum efficacy for all facilitators, suggesting that residents were effective teachers. Although we cannot directly assess whether the differences demonstrated in resident facilitators' self‐assessments translated to their teaching or were significant from the learners' perspective, these results support the hypothesis that self‐assessed improvements in teaching and feedback skills were significant.

In addition to improving resident teaching skills, our program had a positive impact on intern learners as evidenced by intern evaluations of the simulation curriculum. While utilizing relatively few faculty resources, our program was able to deliver an extensive and well‐received simulation curriculum to over 50 interns. The fact that 40% of second‐ and third‐year residents volunteered to teach in the program despite the early morning timing of the sessions speaks to the interest that trainees have in teaching in this setting. This model can serve as an important and efficient learning platform in residency training programs. It may be particularly salient to IM training programs where implementation of simulation curricula is challenging due to large numbers of residents and limited faculty resources. The barriers to and lessons learned from our experience with implementing the simulation curriculum have been previously described.[10, 11]

Our study has several limitations. Changes in residents' teaching skills were self‐assessed, which may be inaccurate as learners may overestimate their abilities.[19] Although we collected data on the experiences of intern learners that supported residents' self‐assessment, further studies using more objective measures (such as the Objective Structured Teaching Exercise[20]) should be undertaken. We did not objectively assess improvement of residents' teaching skills on the wards, with the exception of the residents' self assessment. Due to the timing of survey administration, some residents had as little as 1 month between completion of the curriculum and responding to the post‐curriculum survey, limiting their ability to evaluate their teaching skills on the wards. The transferability of the skills gained in simulation‐based teaching to teaching on the wards deserves further study. We cannot definitively attribute perceived improvement of teaching skills to the RaT program without a control group. However, the frequent use of recommended techniques during debriefing, which are not typically taught in other settings, supports the efficacy of the RaT program.

Our study did not allow us to determine which of the 3 components of the RaT program (workshop, facilitation practice, or direct observation and feedback) had the greatest impact on teaching skills or DASH ratings, as those who facilitated more sessions also completed the other components of the program. Furthermore, there may have been a selection bias among facilitators who facilitated more sessions. Because only 16 of 34 participants completed both the pre‐program and post‐program self‐assessments in a non‐anonymous fashion, we were not able to analyze the effect of pre‐program factors, such as prior teaching experience, on program outcomes. It should also be noted that allowing resident facilitators the option to complete the survey non‐anonymously could have biased our results. The simulation curriculum was conducted in a single center, and resident facilitators were self‐selecting; therefore, our results may not be generalizable. Finally, the DASH instrument was only administered after the RaT workshop and was likely limited further by the ceiling effect created by the learners' high satisfaction with the simulation program overall.

In summary, our simulation‐based RaT program improved resident facilitators' self‐reported teaching and feedback skills. Simulation‐based training provided an opportunity for deliberate practice of teaching skills in a controlled environment, which was a unique component of the program. The impact of deliberate practice on resident teaching skills and optimal methods to incorporate deliberate practice in RaT programs deserves further study. Our curriculum design may serve as a model for the development of simulation programs that can be employed to improve both intern learning and resident teaching skills.

Acknowledgements

The authors acknowledge Deborah Navedo, PhD, Assistant Professor, Massachusetts General Hospital Institute of Health Professions, and Emily M. Hayden, MD, Assistant Professor, Department of Emergency Medicine, Massachusetts General Hospital and Harvard Medical School, for their assistance with development of the RaT curriculum. The authors thank Dr. Jenny Rudolph, Senior Director, Institute for Medical Simulation at the Center for Medical Simulation, for her help in teaching us to use the DASH instrument. The authors also thank Dr. Daniel Hunt, MD, Associate Professor, Department of Medicine, Massachusetts General Hospital and Harvard Medical School, for his thoughtful review of this manuscript.

Disclosure: Nothing to report.

References
  1. Liaison Committee on Medical Education. Functions and structure of a medical school: standards for accreditation of medical education programs leading to the M.D. degree. Washington, DC, and Chicago, IL: Association of American Medical Colleges and American Medical Association; 2000.
  2. Accreditation Council for Graduate Medical Education. ACGME program requirements for graduate medical education in pediatrics. Available at: http://www.acgme.org/acgmeweb/Portals/0/PFAssets/2013-PR-FAQ-PIF/320_pediatrics_07012013.pdf. Accessed June 18, 2014.
  3. Hill AG, Yu TC, Barrow M, Hattie J. A systematic review of resident‐as‐teacher programmes. Med Educ. 2009;43(12):11291140.
  4. Okuda Y, Bryson EO, DeMaria S, et al. The utility of simulation in medical education: what is the evidence? Mt Sinai J Med. 2009;76:330343.
  5. Okuda Y, Bond WF, Bonfante G, et al. National growth in simulation training within emergency medicine residency programs, 2003–2008. Acad Emerg Med. 2008;15:11131116.
  6. Fernandez R, Wang E, Vozenilek JA, et al. Simulation center accreditation and programmatic benchmarks: a review for emergency medicine. Acad Emerg Med. 2010;17(10):10931103.
  7. Cook DA. How much evidence does it take? A cumulative meta‐analysis of outcomes of simulation‐based education. Med Educ. 2014;48(8):750760.
  8. Farrell SE, Pacella C, Egan D, et al.. Resident‐as‐teacher: a suggested curriculum for emergency medicine. Acad Emerg Med. 2006;13(6):677679.
  9. Ericsson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med. 2004;79(10 suppl):S70S81.
  10. Miloslavsky EM, Hayden EM, Currier PF, Mathai SK, Contreras‐Valdes F, Gordon JA. Pilot program utilizing medical simulation in clinical decision making training for internal medicine interns. J Grad Med Educ. 2012;4:490495.
  11. Mathai SK, Miloslavsky EM, Contreras‐Valdes FM, et al. How we implemented a resident‐led medical simulation curriculum in a large internal medicine residency program. Med Teach. 2014;36(4):279283.
  12. Rudolph JW, Simon R, Dufresne RL, Raemer DB. There's no such thing as “nonjudgmental” debriefing: a theory and method for debriefing with good judgment. Simul Healthc. 2006;1:4955.
  13. Minehart RD, Rudolph JW, Pian‐Smith MCM, Raemer DB. Improving faculty feedback to resident trainees during a simulated case: a randomized, controlled trial of an educational intervention. Anesthesiology. 2014;120(1):160171.
  14. Gardner R. Introduction to debriefing. Semin Perinatol. 2013:37(3)166174.
  15. Howard G, Dailey PR. Response‐shift bias: a source of contamination of self‐report measures. J Appl Psychol. 1979;4:93106.
  16. Center for Medical Simulation. Debriefing assessment for simulation in healthcare. Available at: http://www.harvardmedsim.org/debriefing‐assesment‐simulation‐healthcare.php. Accessed June 18, 2014.
  17. Tofil NM, Peterson DT, Harrington KF, et al. A novel iterative‐learner simulation model: fellows as teachers. J Grad Med Educ. 2014;6(1):127132.
  18. Regan‐Smith M, Hirschmann K, Iobst W. Direct observation of faculty with feedback: an effective means of improving patient‐centered and learner‐centered teaching skills. Teach Learn Med. 2007;19(3):278286.
  19. Kruger J, Dunning D. Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self assessments. J Pers Soc Psychol. 1999;77:11211134.
  20. Morrison EH, Boker JR, Hollingshead J, et al. Reliability and validity of an objective structured teaching examination for generalist resident teachers. Acad Med. 2002;77(10 suppl):S29S32.
References
  1. Liaison Committee on Medical Education. Functions and structure of a medical school: standards for accreditation of medical education programs leading to the M.D. degree. Washington, DC, and Chicago, IL: Association of American Medical Colleges and American Medical Association; 2000.
  2. Accreditation Council for Graduate Medical Education. ACGME program requirements for graduate medical education in pediatrics. Available at: http://www.acgme.org/acgmeweb/Portals/0/PFAssets/2013-PR-FAQ-PIF/320_pediatrics_07012013.pdf. Accessed June 18, 2014.
  3. Hill AG, Yu TC, Barrow M, Hattie J. A systematic review of resident‐as‐teacher programmes. Med Educ. 2009;43(12):11291140.
  4. Okuda Y, Bryson EO, DeMaria S, et al. The utility of simulation in medical education: what is the evidence? Mt Sinai J Med. 2009;76:330343.
  5. Okuda Y, Bond WF, Bonfante G, et al. National growth in simulation training within emergency medicine residency programs, 2003–2008. Acad Emerg Med. 2008;15:11131116.
  6. Fernandez R, Wang E, Vozenilek JA, et al. Simulation center accreditation and programmatic benchmarks: a review for emergency medicine. Acad Emerg Med. 2010;17(10):10931103.
  7. Cook DA. How much evidence does it take? A cumulative meta‐analysis of outcomes of simulation‐based education. Med Educ. 2014;48(8):750760.
  8. Farrell SE, Pacella C, Egan D, et al.. Resident‐as‐teacher: a suggested curriculum for emergency medicine. Acad Emerg Med. 2006;13(6):677679.
  9. Ericsson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med. 2004;79(10 suppl):S70S81.
  10. Miloslavsky EM, Hayden EM, Currier PF, Mathai SK, Contreras‐Valdes F, Gordon JA. Pilot program utilizing medical simulation in clinical decision making training for internal medicine interns. J Grad Med Educ. 2012;4:490495.
  11. Mathai SK, Miloslavsky EM, Contreras‐Valdes FM, et al. How we implemented a resident‐led medical simulation curriculum in a large internal medicine residency program. Med Teach. 2014;36(4):279283.
  12. Rudolph JW, Simon R, Dufresne RL, Raemer DB. There's no such thing as “nonjudgmental” debriefing: a theory and method for debriefing with good judgment. Simul Healthc. 2006;1:4955.
  13. Minehart RD, Rudolph JW, Pian‐Smith MCM, Raemer DB. Improving faculty feedback to resident trainees during a simulated case: a randomized, controlled trial of an educational intervention. Anesthesiology. 2014;120(1):160171.
  14. Gardner R. Introduction to debriefing. Semin Perinatol. 2013:37(3)166174.
  15. Howard G, Dailey PR. Response‐shift bias: a source of contamination of self‐report measures. J Appl Psychol. 1979;4:93106.
  16. Center for Medical Simulation. Debriefing assessment for simulation in healthcare. Available at: http://www.harvardmedsim.org/debriefing‐assesment‐simulation‐healthcare.php. Accessed June 18, 2014.
  17. Tofil NM, Peterson DT, Harrington KF, et al. A novel iterative‐learner simulation model: fellows as teachers. J Grad Med Educ. 2014;6(1):127132.
  18. Regan‐Smith M, Hirschmann K, Iobst W. Direct observation of faculty with feedback: an effective means of improving patient‐centered and learner‐centered teaching skills. Teach Learn Med. 2007;19(3):278286.
  19. Kruger J, Dunning D. Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self assessments. J Pers Soc Psychol. 1999;77:11211134.
  20. Morrison EH, Boker JR, Hollingshead J, et al. Reliability and validity of an objective structured teaching examination for generalist resident teachers. Acad Med. 2002;77(10 suppl):S29S32.
Issue
Journal of Hospital Medicine - 10(12)
Issue
Journal of Hospital Medicine - 10(12)
Page Number
767-772
Page Number
767-772
Publications
Publications
Article Type
Display Headline
A simulation‐based resident‐as‐teacher program: The impact on teachers and learners
Display Headline
A simulation‐based resident‐as‐teacher program: The impact on teachers and learners
Sections
Article Source

© 2015 Society of Hospital Medicine

Disallow All Ads
Correspondence Location
Address for correspondence and reprint requests: Eli M. Miloslavsky, MD, Massachusetts General Hospital, 55 Fruit St., Suite 2C, Boston, MA 02114; Telephone: 617‐726‐7938; Fax: 617‐643‐1274; E‐mail: emiloslavsky@mgh.harvard.edu
Content Gating
No Gating (article Unlocked/Free)
Alternative CME
Article PDF Media
Media Files