User login
In internal medicine residency training, the most commonly used metric for measuring workload of physicians is the number of patients being followed or the number being admitted. There are data to support the importance of these census numbers. One study conducted at an academic medical center demonstrated that for patients admitted to medical services, the number of patients admitted on a call night was positively associated with mortality, even after adjustment in multivariable models.1
The problem with a census is that it is only a rough indicator of the amount of work that a given intern or resident will have. In a focus group study that our group conducted with internal medicine residents, several contributors to patient care errors were identified. Workload was identified as a major factor contributing to patient care mistakes.2 In describing workload, residents noted not only census but the complexity of the patient as contributing factors to workload.
A more comprehensive method than relying on census data has been used in anesthesia.3, 4 In 2 studies, anesthesiologists were asked to rate the effort or intensity associated with the tasks that they performed in the operating room.4, 5 In subsequent studies, this group used a trained observer to record the tasks anesthesiologists performed during a case.6, 7 Work density was calculated by multiplying the duration of each task by the previously developed task intensity score. In this way, work per unit of time can be calculated as can a cumulative workload score for a certain duration of time.
These methods provide the background for the work that we conducted in this study. The purpose of this study was to assign a task effort score to the tasks performed during periods that include admitting patients to the hospital.
METHODS
Study Site
A single 500‐bed Midwest academic institution. Residents rotate through 3 hospitals (a private community hospital, a Veterans hospital, and an academic medical center) during a typical 3‐year internal medicine residency program.
Study Design and Subjects
A cross‐sectional survey was conducted. Subjects recruited for the survey included internal medicine interns and residents, internal medicine ward attending physicians and hospitalists. Attending physicians had to have been on the wards in the past year. The survey was conducted in November, when all eligible house staff should have had at least 1 ward month. Nearly every hospitalist recruited had spent time on both teaching and nonteaching services.
Task List Compilation and Survey Development
An expert panel was convened consisting of 10 physicians representing 3 hospitals, including residents and faculty, some of which were hospitalists. During the session, the participants developed a task list and discussed the work intensity associated with some of the tasks. The task list was reviewed by the study team and organized into categories. The final list included 99 tasks divided into 6 categories: (1) direct patient care, (2) indirect patient care, (3) search for/finding things, (4) educational/academic activities, (5) personal/downtime activities, and (6) other. Table 1 gives examples of items found in each category. We used the terminology that the study participants used to describe their work (eg, they used the term eyeballing a patient to describe the process of making an initial assessment of the patient's status). This list of 99 items was formatted into a survey to allow study participants to rate each task across 3 domains: physical effort, mental effort, and psychological effort, based on previous studies in anesthesia4 (see Supporting Information). The term mental refers to cognitive effort, whereas psychological refers to emotional effort. We used the same scales with the same anchors as described in the anesthesia literature,4 but substituted the internal medicine specific tasks. Each item was rated on a 7‐point Likert‐type scale (1 = almost no stress or effort; 7 = most effort). The survey also included demographic information regarding the respondent and instructions. The instructions directed respondents to rate each item based on their average experience in performing each task. They were further instructed not to rate tasks they had never performed.
| Categories of Tasks | Examples |
|---|---|
| |
| Direct patient care | Conducting the physical examination, hand washing, putting on isolation gear |
| Indirect patient care | Writing H&P, writing orders, ordering additional labs or tests |
| Searching for/finding things | Finding a computer, finding materials for procedures, finding the patient |
| Personal/downtime activities | Eating dinner, sleep, socializing, calling family members |
| Educational/academic activities | Literature search, teaching medical students, preparing a talk |
| Other | Transporting patients, traveling from place to place, billing |
Survey Process
The potential survey participants were notified via e‐mail that they would be asked to complete the survey during a regularly scheduled meeting. The interns, residents, and faculty met during separate time slots. Data from residents and interns were obtained from teaching sessions they were required to attend (as long as their schedule permitted them to). Survey data for attending physicians were obtained from a general internal medicine meeting and a hospitalist meeting. Because of the type of meeting, subspecialists were less likely to have been included. The objectives of the study and its voluntary nature were presented to the groups, and the survey was given to all attendees at the meetings. Due to the anonymous nature of the survey, a waiver of written informed consent was granted. Time was reserved during the course of the meeting to complete the survey. Before distributing the survey, we counted the total number of people in the room so that a participation rate could be calculated. Respondents were instructed to place the survey in a designated envelope after completing it or to return a blank survey if they did not wish to complete it. There was no time limit for completion of the survey. At all of these sessions, this survey was one part of the meeting agenda.
Data Analysis
Surveys were entered into a Microsoft Excel (Redmond, WA) spreadsheet and then transferred into Stata version 8.0 (College Station, TX), which was used for analysis. Our analysis focused on (1) the description of the effort associated with individual tasks, (2) the description of the effort associated with task categories and comparisons across key categories, and (3) a comparison of effort across the task categories' physical, mental, and psychological domains.
Each task had 3 individual domain scores associated with it: physical, mental (ie, cognitive work), and psychological (ie, emotional work). A composite task effort score was calculated for each task by determining the mean of the 3 domain scores for that task.
An overall effort score was calculated for each of the 6 task categories by determining the mean of the composite task effort scores within each category. We used the composite effort score for each task to calculate the Cronbach's value for each category except other. We compared the overall category effort scores for direct versus indirect patient care using 2‐tailed paired t tests with a significance level of P < 0.05. We further evaluated differences in overall category effort scores for direct patient care between physicians of different genders and between house staff and faculty, using 2‐tailed unpaired t tests, with a significance level of P < 0.05.
Finally, we compared the physical, mental, and psychological domain scores for direct versus indirect patient care categories, using paired t tests.
Ethics
This study was approved by the Institutional Review Board at the Medical College of Wisconsin.
RESULTS
The study participation rate was 69% (59/85). The sample consisted of 31 (52%) women and 40 (68%) house staff (see Table 2). The mean age was 34 years. This participation rate represents approximately 1/3 of the internal medicine house staff and a smaller percentage of the faculty that would have been eligible.
| Demographic | Value |
|---|---|
| |
| Age, y, mean (SD) | 34 (8.8) |
| Female gender, no. (%) | 31 (52) |
| Physician description, no. (%) | |
| Intern | 7 (12) |
| Resident | 33 (56) |
| Hospitalist | 4 (7) |
| Nonhospitalist faculty | 15 (25) |
Individual Task Effort
The mean composite effort score of all 99 tasks is provided in the Supporting Information Table. Overall, the most difficult task was going to codes (in the direct patient care category), with a mean composite rating of 5.37 (standard deviation [SD] 1.5); this was also the most difficult psychological task (5.78 [SD 1.65]). The most difficult mental task was transferring an unstable patient to the intensive care unit (5.47 [SD 1.53]). The most difficult physical task was placing a central line (5.02 [SD 1.63]). The easiest task was using the Internet (in the personal/downtime activities category), with a mean composite rating of 1.41 (SD 0.74); this was also the easiest mental (1.52 [SD 1.01]), psychological (1.3 [SD 0.68]), and physical (1.42 [SD 0.76]) task.
Analysis of Task Categories
The overall and domain characteristics of each task category are given in Table 3. Categories contained between 5 and 41 tasks. The Cronbach's ranged from 0.83 for the personal/downtime activities category to 0.98 for the direct patient care category. The mean overall effort ranged from least difficult for the personal/downtime category (1.72 [SD 0.76]) to most difficult for the education category (3.61 [SD 1.06]).
| Category | No. of Items | Cronbach's | Effort Score, Mean (SD)* | |||
|---|---|---|---|---|---|---|
| Composite Effort | Physical Effort | Mental Effort | Psychological Effort | |||
| ||||||
| Direct patient care | 32 | 0.97 | 3.55 (0.91) | 3.22 (1.06) | 3.89 (0.99) | 3.52 (1.04) |
| Indirect patient care | 41 | 0.98 | 3.21 (0.92) | 2.71 (1.09) | 3.80 (1.02) | 3.20 (1.08) |
| Education | 8 | 0.92 | 3.61 (1.06) | 3.12 (1.26) | 4.27 (1.17) | 3.43 (1.30) |
| Finding things | 5 | 0.85 | 2.94 (0.91) | 3.59 (1.23) | 2.43 (1.05) | 2.79 (1.13) |
| Personal | 7 | 0.83 | 1.72 (0.76) | 1.86 (0.92) | 1.69 (0.85) | 1.63 (0.72) |
| Other | 6 | NC | NC | NC | NC | NC |
Using paired t tests, we determined that the direct patient care category was more difficult than the indirect patient care category overall (3.58 versus 3.21, P < 0.001). Direct patient care was statistically significantly more challenging than indirect patient care on the physical (3.23 vs 2.71; P < 0.001), mental (3.90 vs 3.84; P < 0.05), and psychological domains (3.57 vs 3.20; P < 0.001) as well. There were no significant differences between men and women or between house staff and faculty on the difficulty of direct patient care. We found a trend toward increased difficulty of indirect patient care for house staff versus faculty (3.36 vs 2.92; P 0.10), but no differences by gender.
DISCUSSION
In this study, we used a comprehensive list of tasks performed by internal medicine doctors while admitting patients and produced a numeric assessment of the effort associated with each. The list was generated by an expert panel and comprised 6 categories and 99 items. Residents and attending physicians then rated each task based on level of difficulty, specifically looking at the mental, psychological, and physical effort required by each.
Indirect patient care was the task category in our study that had the most tasks associated with it (41 out of 99). Direct patient care included 32 items, but 10 of these were procedures (eg, lumbar puncture), some of which are uncommonly performed. Several time‐motion studies have been performed to document the work done by residents815 and hospitalists.16, 17 Although our study did not assess the time spent on each task, the distribution of tasks across categories is consistent with these time‐motion studies, which show that the amount of time spent in direct patient care is a small fraction of the amount of time spent in the hospital,12 and that work such as interprofessional communication10 and documentation16 consume the majority of time.
This project allowed us to consider the effort required for inpatient internal medicine work on a more granular level than has been described previously. Although the difficulty of tasks associated with anesthesia and surgical work has been described,3, 4, 7, 1820 our study is a unique contribution to the internal medicine literature. Understanding the difficulty of tasks performed by inpatient physicians is an important step toward better management of workload. With concerns about burnout in hospitalists21, 22 and residents,2325 it seems wise to take the difficulty of the work they do into consideration in a more proactive manner. In addition, understanding workload may have patient safety applications. In one study of mistakes made by house staff, 51% of the survey respondents identified workload as a contributing factor.26
We assessed effort for inpatient work by generating a task list and then measuring 3 domains of each task: physical, mental, and psychological. As a result, we were able to further quantify the difficulty of work completed by physicians. Recent work from outside of medicine suggests that individuals have a finite capacity for mental workload, and when this is breached, decision‐making quality is impaired.27 This suggests that it is important to take work intensity into account when assigning work to individuals. For example, a detailed assessment of workload at the task level combined with the amount of time spent on each task would allow us to know how much effort is typically involved with admitting a new patient. This information would allow for more equal distribution of workload across admitting teams. In addition, these methods could be expanded to understand how much effort is involved in the discharge process. This could be taken into account at the beginning of a day when allocating work such as admissions and discharges between members of a team.
This methodology has the potential to be used in other ways to help quantify the effort required for the work that physicians do. Many departments are struggling to develop a system for giving credit to faculty for the time they spend on nonpatient care activities. Perhaps these methods could be used to develop effort scores associated with administrative tasks, and administrative relative value units could be calculated accordingly. Similar techniques have been used with educational relative value units.28
We know from the nursing literature that workload is related to both burnout and patient safety. Burnout is a process related to the emotional work of providing care to people.29 Our methods clearly incorporate the psychological stress of work into the workload assessment. Evaluating the amount of time spent on tasks with high psychological scores may be helpful in identifying work patterns that are more likely to produce burnout in physicians and nurses.
With respect to patient safety, higher patient‐to‐nurse ratios are associated with failure to rescue30 and nosocomial infections.31 Furthermore, researchers have demonstrated that systems issues can add substantially to nursing workload.32 Methods such as those described in our study take into account both patient‐related and systems‐related tasks, and therefore could result in more detailed workload assessments. With more detailed information about contributors to workload, better predictions about optimal staffing could be made, which would ultimately lead to fewer adverse patient events.
Our study has limitations. First, the initial task list was based on the compilation efforts from only 10 physicians. However, this group of physicians represented 3 hospitals and included both resident and attending physicians. Second, the survey data were gathered from a single institution. Although we included trainees and faculty, more participants would be needed to answer questions about how experience and setting/environmental factors affect these assessments. However, participants were instructed to reflect on their whole experience with each task, which presumably includes multiple institutions and training levels. Third, the sample size is fairly small, with more house staff than faculty (hospitalists and nonhospitalists) represented. Regardless, this study is the first attempt to define and quantify workload for internal medicine physicians using these methods. In future studies, we will expand the number of institutions and levels of experience to validate our current data. Finally, the difficulty of the tasks is clearly a subjective assessment. Although this methodology has face validity, further work needs to be done to validate these findings against other measurements of workload, such as census, or more general subjective workload assessments, such as the NASA task load index.33
In conclusion, we have described the tasks performed by inpatient physicians and the difficulty associated with them. Moreover, we have described a methodology that could be replicated at other centers for the purpose of validating our findings or quantifying workload of other types of tasks. We believe that this is the first step toward a more comprehensive understanding of the workload encountered by inpatient physicians. Because workload has implications for physician burnout and patient safety, it is essential that we fully understand the contributors to workload, including the innate difficulty of the tasks that comprise it.
Acknowledgements
The authors Alexis Visotcky, MS, and Sergey Tarima, PhD, for their assistance with statistics.
This work was presented in poster form at the Society of Hospital Medicine Annual Meeting in April 2010, the Society of General Internal Medicine Annual Meeting in May 2010, and the Society of General Internal Medicine regional meeting in September 2010.
Funding Source: The study team was supported by the following funds during this work: VA grants PPO 0925901 (Marilyn M. Schapira and Kathlyn E. Fletcher) and IIR 07201 (Marilyn M. Schapira, Siddhartha Singh, and Kathlyn E. Fletcher).
- ,,,,.House staff team workload and organization effects on patient outcomes in an academic general internal medicine inpatient service.Arch Intern Med.2007;167:47–52.
- ,,, et al.The work hour rules and contributors to patient care mistakes: A focus group study with internal medicine residentsJ Hosp Med.2008;3:228–237.
- ,,.Multiple measures of anesthesia workload during teaching and nonteaching cases.Anesth Analg.2004;98:1419–1425.
- ,,,,.Developing a technique to measure anesthesiologists' real‐time workload.Proceedings of the Human Factors and Ergonomics Society Annual Meeting.2000;44:241–244.
- ,,, et al.Quantitative description of the workload associated with airway management procedures.J Clin Anesth.2000;12:273–282.
- ,,,,,.An objective methodology for task analysis and workload assessment in anesthesia providers.Anesthesiology.1994;80:77–92.
- ,.Effects of intraoperative reading on vigilance and workload during anesthesia care in an academic medical center.Anesthesiology.2009;110:275–283.
- ,,.Resident work hours: what they are really doing.Arch Surg.2004;139:490–493; discussion, 493–494.
- ,,,,,.Analyzing the time and value of housestaff inpatient work.J Gen Intern Med.1998;13:534–540.
- ,,,.All in a day's work: an observational study to quantify how and with whom doctors on hospital wards spend their time.[see comment].Med J Aust.2008;188:506–509.
- ,,,,.How do house officers spend their nights? A time study of internal medicine house staff on call.N Engl J Med.1989;320:1673–1677.
- ,,,,.Systematic review of time studies evaluating physicians in the hospital setting.J Hosp Med.2010;5:353–359.
- ,,.Time analysis of a general medicine service: results from a random work sampling study.J Gen Intern Med.1994;9:272–277.
- ,,,.A phenomenology of scut.Ann Intern Med.1991;115:372–376.
- ,,, et al.The on‐call experience of interns in internal medicine. Medical Education Task Force of Henry Ford Hospital.Arch Intern Med.1990;150:2294–2297.
- ,,, et al.Where did the day go? A time‐motion study of hospitalists.J Hosp Med.2010;5:323–328.
- ,,.How hospitalists spend their time: insights on efficiency and safety.J Hosp Med.2006;1:88–93.
- ,,, et al.Differences in day and night shift clinical performance in anesthesiology.Hum Factors.2008;50:276–290.
- ,,,,.Assessment of the intrarater and interrater reliability of an established clinical task analysis methodology.Anesthesiology.2002;96:1129–1139.
- ,,.The effect of electronic record keeping and transesophageal echocardiography on task distribution, workload, and vigilance during cardiac anesthesia.Anesthesiology.1997;87:144–155.
- .Fight burnout while fostering experience: investing in hospitalist programs now can fight burnout later.ACP Hospitalist. July2008.
- .Hospitalist burnout: recognize it in yourself and others, and avoid or eliminate it.The Hospitalist. March2006.
- ,,,.Burnout and internal medicine resident work‐hour restrictions.Arch Intern Med.2005;165:2595–2600.
- ,,,,.The effects of work‐hour limitations on resident well‐being, patient care, and education in an internal medicine residency program.Arch Intern Med.2005;165:2601–2606.
- ,,,.Burnout and self‐reported patient care in an internal medicine residency program.Ann Intern Med.2002;136:358–367.
- ,,,.Do house officers learn from their mistakes?Qual Saf Health Care.2003;12:221–226; discussion, 227–228.
- ,,.Extraneous factors in judicial decisions.Proc Natl Acad Sci U S A.2011;108:6889–6892.
- ,.Quantifying physician teaching productivity using clinical relative value units.J Gen Intern Med.1999;14:617–621.
- .Maslach Burnout Inventory Manual.3rd ed.Palo Alto, CA:Consulting Psychology Press;1986.
- ,,,,.Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction.JAMA.2002;288:1987–1993.
- ,,,,.Patient density, nurse‐to‐patient ratio and nosocomial infection risk in a pediatric cardiac intensive care unit.Pediatr Infect Dis J.1997;16:1045–1048.
- ,.Operational failures and interruptions in hospital nursing.Health Serv Res.2006;41:643–662.
- ,.Development of NASA‐TLX (Task Load Index): results of empirical and theoretical research. In: Hancock PA, Meshkati N, eds.Human Mental Workload.Amsterdam, Netherlands:North Holland Press;1988:239–250.
In internal medicine residency training, the most commonly used metric for measuring workload of physicians is the number of patients being followed or the number being admitted. There are data to support the importance of these census numbers. One study conducted at an academic medical center demonstrated that for patients admitted to medical services, the number of patients admitted on a call night was positively associated with mortality, even after adjustment in multivariable models.1
The problem with a census is that it is only a rough indicator of the amount of work that a given intern or resident will have. In a focus group study that our group conducted with internal medicine residents, several contributors to patient care errors were identified. Workload was identified as a major factor contributing to patient care mistakes.2 In describing workload, residents noted not only census but the complexity of the patient as contributing factors to workload.
A more comprehensive method than relying on census data has been used in anesthesia.3, 4 In 2 studies, anesthesiologists were asked to rate the effort or intensity associated with the tasks that they performed in the operating room.4, 5 In subsequent studies, this group used a trained observer to record the tasks anesthesiologists performed during a case.6, 7 Work density was calculated by multiplying the duration of each task by the previously developed task intensity score. In this way, work per unit of time can be calculated as can a cumulative workload score for a certain duration of time.
These methods provide the background for the work that we conducted in this study. The purpose of this study was to assign a task effort score to the tasks performed during periods that include admitting patients to the hospital.
METHODS
Study Site
A single 500‐bed Midwest academic institution. Residents rotate through 3 hospitals (a private community hospital, a Veterans hospital, and an academic medical center) during a typical 3‐year internal medicine residency program.
Study Design and Subjects
A cross‐sectional survey was conducted. Subjects recruited for the survey included internal medicine interns and residents, internal medicine ward attending physicians and hospitalists. Attending physicians had to have been on the wards in the past year. The survey was conducted in November, when all eligible house staff should have had at least 1 ward month. Nearly every hospitalist recruited had spent time on both teaching and nonteaching services.
Task List Compilation and Survey Development
An expert panel was convened consisting of 10 physicians representing 3 hospitals, including residents and faculty, some of which were hospitalists. During the session, the participants developed a task list and discussed the work intensity associated with some of the tasks. The task list was reviewed by the study team and organized into categories. The final list included 99 tasks divided into 6 categories: (1) direct patient care, (2) indirect patient care, (3) search for/finding things, (4) educational/academic activities, (5) personal/downtime activities, and (6) other. Table 1 gives examples of items found in each category. We used the terminology that the study participants used to describe their work (eg, they used the term eyeballing a patient to describe the process of making an initial assessment of the patient's status). This list of 99 items was formatted into a survey to allow study participants to rate each task across 3 domains: physical effort, mental effort, and psychological effort, based on previous studies in anesthesia4 (see Supporting Information). The term mental refers to cognitive effort, whereas psychological refers to emotional effort. We used the same scales with the same anchors as described in the anesthesia literature,4 but substituted the internal medicine specific tasks. Each item was rated on a 7‐point Likert‐type scale (1 = almost no stress or effort; 7 = most effort). The survey also included demographic information regarding the respondent and instructions. The instructions directed respondents to rate each item based on their average experience in performing each task. They were further instructed not to rate tasks they had never performed.
| Categories of Tasks | Examples |
|---|---|
| |
| Direct patient care | Conducting the physical examination, hand washing, putting on isolation gear |
| Indirect patient care | Writing H&P, writing orders, ordering additional labs or tests |
| Searching for/finding things | Finding a computer, finding materials for procedures, finding the patient |
| Personal/downtime activities | Eating dinner, sleep, socializing, calling family members |
| Educational/academic activities | Literature search, teaching medical students, preparing a talk |
| Other | Transporting patients, traveling from place to place, billing |
Survey Process
The potential survey participants were notified via e‐mail that they would be asked to complete the survey during a regularly scheduled meeting. The interns, residents, and faculty met during separate time slots. Data from residents and interns were obtained from teaching sessions they were required to attend (as long as their schedule permitted them to). Survey data for attending physicians were obtained from a general internal medicine meeting and a hospitalist meeting. Because of the type of meeting, subspecialists were less likely to have been included. The objectives of the study and its voluntary nature were presented to the groups, and the survey was given to all attendees at the meetings. Due to the anonymous nature of the survey, a waiver of written informed consent was granted. Time was reserved during the course of the meeting to complete the survey. Before distributing the survey, we counted the total number of people in the room so that a participation rate could be calculated. Respondents were instructed to place the survey in a designated envelope after completing it or to return a blank survey if they did not wish to complete it. There was no time limit for completion of the survey. At all of these sessions, this survey was one part of the meeting agenda.
Data Analysis
Surveys were entered into a Microsoft Excel (Redmond, WA) spreadsheet and then transferred into Stata version 8.0 (College Station, TX), which was used for analysis. Our analysis focused on (1) the description of the effort associated with individual tasks, (2) the description of the effort associated with task categories and comparisons across key categories, and (3) a comparison of effort across the task categories' physical, mental, and psychological domains.
Each task had 3 individual domain scores associated with it: physical, mental (ie, cognitive work), and psychological (ie, emotional work). A composite task effort score was calculated for each task by determining the mean of the 3 domain scores for that task.
An overall effort score was calculated for each of the 6 task categories by determining the mean of the composite task effort scores within each category. We used the composite effort score for each task to calculate the Cronbach's value for each category except other. We compared the overall category effort scores for direct versus indirect patient care using 2‐tailed paired t tests with a significance level of P < 0.05. We further evaluated differences in overall category effort scores for direct patient care between physicians of different genders and between house staff and faculty, using 2‐tailed unpaired t tests, with a significance level of P < 0.05.
Finally, we compared the physical, mental, and psychological domain scores for direct versus indirect patient care categories, using paired t tests.
Ethics
This study was approved by the Institutional Review Board at the Medical College of Wisconsin.
RESULTS
The study participation rate was 69% (59/85). The sample consisted of 31 (52%) women and 40 (68%) house staff (see Table 2). The mean age was 34 years. This participation rate represents approximately 1/3 of the internal medicine house staff and a smaller percentage of the faculty that would have been eligible.
| Demographic | Value |
|---|---|
| |
| Age, y, mean (SD) | 34 (8.8) |
| Female gender, no. (%) | 31 (52) |
| Physician description, no. (%) | |
| Intern | 7 (12) |
| Resident | 33 (56) |
| Hospitalist | 4 (7) |
| Nonhospitalist faculty | 15 (25) |
Individual Task Effort
The mean composite effort score of all 99 tasks is provided in the Supporting Information Table. Overall, the most difficult task was going to codes (in the direct patient care category), with a mean composite rating of 5.37 (standard deviation [SD] 1.5); this was also the most difficult psychological task (5.78 [SD 1.65]). The most difficult mental task was transferring an unstable patient to the intensive care unit (5.47 [SD 1.53]). The most difficult physical task was placing a central line (5.02 [SD 1.63]). The easiest task was using the Internet (in the personal/downtime activities category), with a mean composite rating of 1.41 (SD 0.74); this was also the easiest mental (1.52 [SD 1.01]), psychological (1.3 [SD 0.68]), and physical (1.42 [SD 0.76]) task.
Analysis of Task Categories
The overall and domain characteristics of each task category are given in Table 3. Categories contained between 5 and 41 tasks. The Cronbach's ranged from 0.83 for the personal/downtime activities category to 0.98 for the direct patient care category. The mean overall effort ranged from least difficult for the personal/downtime category (1.72 [SD 0.76]) to most difficult for the education category (3.61 [SD 1.06]).
| Category | No. of Items | Cronbach's | Effort Score, Mean (SD)* | |||
|---|---|---|---|---|---|---|
| Composite Effort | Physical Effort | Mental Effort | Psychological Effort | |||
| ||||||
| Direct patient care | 32 | 0.97 | 3.55 (0.91) | 3.22 (1.06) | 3.89 (0.99) | 3.52 (1.04) |
| Indirect patient care | 41 | 0.98 | 3.21 (0.92) | 2.71 (1.09) | 3.80 (1.02) | 3.20 (1.08) |
| Education | 8 | 0.92 | 3.61 (1.06) | 3.12 (1.26) | 4.27 (1.17) | 3.43 (1.30) |
| Finding things | 5 | 0.85 | 2.94 (0.91) | 3.59 (1.23) | 2.43 (1.05) | 2.79 (1.13) |
| Personal | 7 | 0.83 | 1.72 (0.76) | 1.86 (0.92) | 1.69 (0.85) | 1.63 (0.72) |
| Other | 6 | NC | NC | NC | NC | NC |
Using paired t tests, we determined that the direct patient care category was more difficult than the indirect patient care category overall (3.58 versus 3.21, P < 0.001). Direct patient care was statistically significantly more challenging than indirect patient care on the physical (3.23 vs 2.71; P < 0.001), mental (3.90 vs 3.84; P < 0.05), and psychological domains (3.57 vs 3.20; P < 0.001) as well. There were no significant differences between men and women or between house staff and faculty on the difficulty of direct patient care. We found a trend toward increased difficulty of indirect patient care for house staff versus faculty (3.36 vs 2.92; P 0.10), but no differences by gender.
DISCUSSION
In this study, we used a comprehensive list of tasks performed by internal medicine doctors while admitting patients and produced a numeric assessment of the effort associated with each. The list was generated by an expert panel and comprised 6 categories and 99 items. Residents and attending physicians then rated each task based on level of difficulty, specifically looking at the mental, psychological, and physical effort required by each.
Indirect patient care was the task category in our study that had the most tasks associated with it (41 out of 99). Direct patient care included 32 items, but 10 of these were procedures (eg, lumbar puncture), some of which are uncommonly performed. Several time‐motion studies have been performed to document the work done by residents815 and hospitalists.16, 17 Although our study did not assess the time spent on each task, the distribution of tasks across categories is consistent with these time‐motion studies, which show that the amount of time spent in direct patient care is a small fraction of the amount of time spent in the hospital,12 and that work such as interprofessional communication10 and documentation16 consume the majority of time.
This project allowed us to consider the effort required for inpatient internal medicine work on a more granular level than has been described previously. Although the difficulty of tasks associated with anesthesia and surgical work has been described,3, 4, 7, 1820 our study is a unique contribution to the internal medicine literature. Understanding the difficulty of tasks performed by inpatient physicians is an important step toward better management of workload. With concerns about burnout in hospitalists21, 22 and residents,2325 it seems wise to take the difficulty of the work they do into consideration in a more proactive manner. In addition, understanding workload may have patient safety applications. In one study of mistakes made by house staff, 51% of the survey respondents identified workload as a contributing factor.26
We assessed effort for inpatient work by generating a task list and then measuring 3 domains of each task: physical, mental, and psychological. As a result, we were able to further quantify the difficulty of work completed by physicians. Recent work from outside of medicine suggests that individuals have a finite capacity for mental workload, and when this is breached, decision‐making quality is impaired.27 This suggests that it is important to take work intensity into account when assigning work to individuals. For example, a detailed assessment of workload at the task level combined with the amount of time spent on each task would allow us to know how much effort is typically involved with admitting a new patient. This information would allow for more equal distribution of workload across admitting teams. In addition, these methods could be expanded to understand how much effort is involved in the discharge process. This could be taken into account at the beginning of a day when allocating work such as admissions and discharges between members of a team.
This methodology has the potential to be used in other ways to help quantify the effort required for the work that physicians do. Many departments are struggling to develop a system for giving credit to faculty for the time they spend on nonpatient care activities. Perhaps these methods could be used to develop effort scores associated with administrative tasks, and administrative relative value units could be calculated accordingly. Similar techniques have been used with educational relative value units.28
We know from the nursing literature that workload is related to both burnout and patient safety. Burnout is a process related to the emotional work of providing care to people.29 Our methods clearly incorporate the psychological stress of work into the workload assessment. Evaluating the amount of time spent on tasks with high psychological scores may be helpful in identifying work patterns that are more likely to produce burnout in physicians and nurses.
With respect to patient safety, higher patient‐to‐nurse ratios are associated with failure to rescue30 and nosocomial infections.31 Furthermore, researchers have demonstrated that systems issues can add substantially to nursing workload.32 Methods such as those described in our study take into account both patient‐related and systems‐related tasks, and therefore could result in more detailed workload assessments. With more detailed information about contributors to workload, better predictions about optimal staffing could be made, which would ultimately lead to fewer adverse patient events.
Our study has limitations. First, the initial task list was based on the compilation efforts from only 10 physicians. However, this group of physicians represented 3 hospitals and included both resident and attending physicians. Second, the survey data were gathered from a single institution. Although we included trainees and faculty, more participants would be needed to answer questions about how experience and setting/environmental factors affect these assessments. However, participants were instructed to reflect on their whole experience with each task, which presumably includes multiple institutions and training levels. Third, the sample size is fairly small, with more house staff than faculty (hospitalists and nonhospitalists) represented. Regardless, this study is the first attempt to define and quantify workload for internal medicine physicians using these methods. In future studies, we will expand the number of institutions and levels of experience to validate our current data. Finally, the difficulty of the tasks is clearly a subjective assessment. Although this methodology has face validity, further work needs to be done to validate these findings against other measurements of workload, such as census, or more general subjective workload assessments, such as the NASA task load index.33
In conclusion, we have described the tasks performed by inpatient physicians and the difficulty associated with them. Moreover, we have described a methodology that could be replicated at other centers for the purpose of validating our findings or quantifying workload of other types of tasks. We believe that this is the first step toward a more comprehensive understanding of the workload encountered by inpatient physicians. Because workload has implications for physician burnout and patient safety, it is essential that we fully understand the contributors to workload, including the innate difficulty of the tasks that comprise it.
Acknowledgements
The authors Alexis Visotcky, MS, and Sergey Tarima, PhD, for their assistance with statistics.
This work was presented in poster form at the Society of Hospital Medicine Annual Meeting in April 2010, the Society of General Internal Medicine Annual Meeting in May 2010, and the Society of General Internal Medicine regional meeting in September 2010.
Funding Source: The study team was supported by the following funds during this work: VA grants PPO 0925901 (Marilyn M. Schapira and Kathlyn E. Fletcher) and IIR 07201 (Marilyn M. Schapira, Siddhartha Singh, and Kathlyn E. Fletcher).
In internal medicine residency training, the most commonly used metric for measuring workload of physicians is the number of patients being followed or the number being admitted. There are data to support the importance of these census numbers. One study conducted at an academic medical center demonstrated that for patients admitted to medical services, the number of patients admitted on a call night was positively associated with mortality, even after adjustment in multivariable models.1
The problem with a census is that it is only a rough indicator of the amount of work that a given intern or resident will have. In a focus group study that our group conducted with internal medicine residents, several contributors to patient care errors were identified. Workload was identified as a major factor contributing to patient care mistakes.2 In describing workload, residents noted not only census but the complexity of the patient as contributing factors to workload.
A more comprehensive method than relying on census data has been used in anesthesia.3, 4 In 2 studies, anesthesiologists were asked to rate the effort or intensity associated with the tasks that they performed in the operating room.4, 5 In subsequent studies, this group used a trained observer to record the tasks anesthesiologists performed during a case.6, 7 Work density was calculated by multiplying the duration of each task by the previously developed task intensity score. In this way, work per unit of time can be calculated as can a cumulative workload score for a certain duration of time.
These methods provide the background for the work that we conducted in this study. The purpose of this study was to assign a task effort score to the tasks performed during periods that include admitting patients to the hospital.
METHODS
Study Site
A single 500‐bed Midwest academic institution. Residents rotate through 3 hospitals (a private community hospital, a Veterans hospital, and an academic medical center) during a typical 3‐year internal medicine residency program.
Study Design and Subjects
A cross‐sectional survey was conducted. Subjects recruited for the survey included internal medicine interns and residents, internal medicine ward attending physicians and hospitalists. Attending physicians had to have been on the wards in the past year. The survey was conducted in November, when all eligible house staff should have had at least 1 ward month. Nearly every hospitalist recruited had spent time on both teaching and nonteaching services.
Task List Compilation and Survey Development
An expert panel was convened consisting of 10 physicians representing 3 hospitals, including residents and faculty, some of which were hospitalists. During the session, the participants developed a task list and discussed the work intensity associated with some of the tasks. The task list was reviewed by the study team and organized into categories. The final list included 99 tasks divided into 6 categories: (1) direct patient care, (2) indirect patient care, (3) search for/finding things, (4) educational/academic activities, (5) personal/downtime activities, and (6) other. Table 1 gives examples of items found in each category. We used the terminology that the study participants used to describe their work (eg, they used the term eyeballing a patient to describe the process of making an initial assessment of the patient's status). This list of 99 items was formatted into a survey to allow study participants to rate each task across 3 domains: physical effort, mental effort, and psychological effort, based on previous studies in anesthesia4 (see Supporting Information). The term mental refers to cognitive effort, whereas psychological refers to emotional effort. We used the same scales with the same anchors as described in the anesthesia literature,4 but substituted the internal medicine specific tasks. Each item was rated on a 7‐point Likert‐type scale (1 = almost no stress or effort; 7 = most effort). The survey also included demographic information regarding the respondent and instructions. The instructions directed respondents to rate each item based on their average experience in performing each task. They were further instructed not to rate tasks they had never performed.
| Categories of Tasks | Examples |
|---|---|
| |
| Direct patient care | Conducting the physical examination, hand washing, putting on isolation gear |
| Indirect patient care | Writing H&P, writing orders, ordering additional labs or tests |
| Searching for/finding things | Finding a computer, finding materials for procedures, finding the patient |
| Personal/downtime activities | Eating dinner, sleep, socializing, calling family members |
| Educational/academic activities | Literature search, teaching medical students, preparing a talk |
| Other | Transporting patients, traveling from place to place, billing |
Survey Process
The potential survey participants were notified via e‐mail that they would be asked to complete the survey during a regularly scheduled meeting. The interns, residents, and faculty met during separate time slots. Data from residents and interns were obtained from teaching sessions they were required to attend (as long as their schedule permitted them to). Survey data for attending physicians were obtained from a general internal medicine meeting and a hospitalist meeting. Because of the type of meeting, subspecialists were less likely to have been included. The objectives of the study and its voluntary nature were presented to the groups, and the survey was given to all attendees at the meetings. Due to the anonymous nature of the survey, a waiver of written informed consent was granted. Time was reserved during the course of the meeting to complete the survey. Before distributing the survey, we counted the total number of people in the room so that a participation rate could be calculated. Respondents were instructed to place the survey in a designated envelope after completing it or to return a blank survey if they did not wish to complete it. There was no time limit for completion of the survey. At all of these sessions, this survey was one part of the meeting agenda.
Data Analysis
Surveys were entered into a Microsoft Excel (Redmond, WA) spreadsheet and then transferred into Stata version 8.0 (College Station, TX), which was used for analysis. Our analysis focused on (1) the description of the effort associated with individual tasks, (2) the description of the effort associated with task categories and comparisons across key categories, and (3) a comparison of effort across the task categories' physical, mental, and psychological domains.
Each task had 3 individual domain scores associated with it: physical, mental (ie, cognitive work), and psychological (ie, emotional work). A composite task effort score was calculated for each task by determining the mean of the 3 domain scores for that task.
An overall effort score was calculated for each of the 6 task categories by determining the mean of the composite task effort scores within each category. We used the composite effort score for each task to calculate the Cronbach's value for each category except other. We compared the overall category effort scores for direct versus indirect patient care using 2‐tailed paired t tests with a significance level of P < 0.05. We further evaluated differences in overall category effort scores for direct patient care between physicians of different genders and between house staff and faculty, using 2‐tailed unpaired t tests, with a significance level of P < 0.05.
Finally, we compared the physical, mental, and psychological domain scores for direct versus indirect patient care categories, using paired t tests.
Ethics
This study was approved by the Institutional Review Board at the Medical College of Wisconsin.
RESULTS
The study participation rate was 69% (59/85). The sample consisted of 31 (52%) women and 40 (68%) house staff (see Table 2). The mean age was 34 years. This participation rate represents approximately 1/3 of the internal medicine house staff and a smaller percentage of the faculty that would have been eligible.
| Demographic | Value |
|---|---|
| |
| Age, y, mean (SD) | 34 (8.8) |
| Female gender, no. (%) | 31 (52) |
| Physician description, no. (%) | |
| Intern | 7 (12) |
| Resident | 33 (56) |
| Hospitalist | 4 (7) |
| Nonhospitalist faculty | 15 (25) |
Individual Task Effort
The mean composite effort score of all 99 tasks is provided in the Supporting Information Table. Overall, the most difficult task was going to codes (in the direct patient care category), with a mean composite rating of 5.37 (standard deviation [SD] 1.5); this was also the most difficult psychological task (5.78 [SD 1.65]). The most difficult mental task was transferring an unstable patient to the intensive care unit (5.47 [SD 1.53]). The most difficult physical task was placing a central line (5.02 [SD 1.63]). The easiest task was using the Internet (in the personal/downtime activities category), with a mean composite rating of 1.41 (SD 0.74); this was also the easiest mental (1.52 [SD 1.01]), psychological (1.3 [SD 0.68]), and physical (1.42 [SD 0.76]) task.
Analysis of Task Categories
The overall and domain characteristics of each task category are given in Table 3. Categories contained between 5 and 41 tasks. The Cronbach's ranged from 0.83 for the personal/downtime activities category to 0.98 for the direct patient care category. The mean overall effort ranged from least difficult for the personal/downtime category (1.72 [SD 0.76]) to most difficult for the education category (3.61 [SD 1.06]).
| Category | No. of Items | Cronbach's | Effort Score, Mean (SD)* | |||
|---|---|---|---|---|---|---|
| Composite Effort | Physical Effort | Mental Effort | Psychological Effort | |||
| ||||||
| Direct patient care | 32 | 0.97 | 3.55 (0.91) | 3.22 (1.06) | 3.89 (0.99) | 3.52 (1.04) |
| Indirect patient care | 41 | 0.98 | 3.21 (0.92) | 2.71 (1.09) | 3.80 (1.02) | 3.20 (1.08) |
| Education | 8 | 0.92 | 3.61 (1.06) | 3.12 (1.26) | 4.27 (1.17) | 3.43 (1.30) |
| Finding things | 5 | 0.85 | 2.94 (0.91) | 3.59 (1.23) | 2.43 (1.05) | 2.79 (1.13) |
| Personal | 7 | 0.83 | 1.72 (0.76) | 1.86 (0.92) | 1.69 (0.85) | 1.63 (0.72) |
| Other | 6 | NC | NC | NC | NC | NC |
Using paired t tests, we determined that the direct patient care category was more difficult than the indirect patient care category overall (3.58 versus 3.21, P < 0.001). Direct patient care was statistically significantly more challenging than indirect patient care on the physical (3.23 vs 2.71; P < 0.001), mental (3.90 vs 3.84; P < 0.05), and psychological domains (3.57 vs 3.20; P < 0.001) as well. There were no significant differences between men and women or between house staff and faculty on the difficulty of direct patient care. We found a trend toward increased difficulty of indirect patient care for house staff versus faculty (3.36 vs 2.92; P 0.10), but no differences by gender.
DISCUSSION
In this study, we used a comprehensive list of tasks performed by internal medicine doctors while admitting patients and produced a numeric assessment of the effort associated with each. The list was generated by an expert panel and comprised 6 categories and 99 items. Residents and attending physicians then rated each task based on level of difficulty, specifically looking at the mental, psychological, and physical effort required by each.
Indirect patient care was the task category in our study that had the most tasks associated with it (41 out of 99). Direct patient care included 32 items, but 10 of these were procedures (eg, lumbar puncture), some of which are uncommonly performed. Several time‐motion studies have been performed to document the work done by residents815 and hospitalists.16, 17 Although our study did not assess the time spent on each task, the distribution of tasks across categories is consistent with these time‐motion studies, which show that the amount of time spent in direct patient care is a small fraction of the amount of time spent in the hospital,12 and that work such as interprofessional communication10 and documentation16 consume the majority of time.
This project allowed us to consider the effort required for inpatient internal medicine work on a more granular level than has been described previously. Although the difficulty of tasks associated with anesthesia and surgical work has been described,3, 4, 7, 1820 our study is a unique contribution to the internal medicine literature. Understanding the difficulty of tasks performed by inpatient physicians is an important step toward better management of workload. With concerns about burnout in hospitalists21, 22 and residents,2325 it seems wise to take the difficulty of the work they do into consideration in a more proactive manner. In addition, understanding workload may have patient safety applications. In one study of mistakes made by house staff, 51% of the survey respondents identified workload as a contributing factor.26
We assessed effort for inpatient work by generating a task list and then measuring 3 domains of each task: physical, mental, and psychological. As a result, we were able to further quantify the difficulty of work completed by physicians. Recent work from outside of medicine suggests that individuals have a finite capacity for mental workload, and when this is breached, decision‐making quality is impaired.27 This suggests that it is important to take work intensity into account when assigning work to individuals. For example, a detailed assessment of workload at the task level combined with the amount of time spent on each task would allow us to know how much effort is typically involved with admitting a new patient. This information would allow for more equal distribution of workload across admitting teams. In addition, these methods could be expanded to understand how much effort is involved in the discharge process. This could be taken into account at the beginning of a day when allocating work such as admissions and discharges between members of a team.
This methodology has the potential to be used in other ways to help quantify the effort required for the work that physicians do. Many departments are struggling to develop a system for giving credit to faculty for the time they spend on nonpatient care activities. Perhaps these methods could be used to develop effort scores associated with administrative tasks, and administrative relative value units could be calculated accordingly. Similar techniques have been used with educational relative value units.28
We know from the nursing literature that workload is related to both burnout and patient safety. Burnout is a process related to the emotional work of providing care to people.29 Our methods clearly incorporate the psychological stress of work into the workload assessment. Evaluating the amount of time spent on tasks with high psychological scores may be helpful in identifying work patterns that are more likely to produce burnout in physicians and nurses.
With respect to patient safety, higher patient‐to‐nurse ratios are associated with failure to rescue30 and nosocomial infections.31 Furthermore, researchers have demonstrated that systems issues can add substantially to nursing workload.32 Methods such as those described in our study take into account both patient‐related and systems‐related tasks, and therefore could result in more detailed workload assessments. With more detailed information about contributors to workload, better predictions about optimal staffing could be made, which would ultimately lead to fewer adverse patient events.
Our study has limitations. First, the initial task list was based on the compilation efforts from only 10 physicians. However, this group of physicians represented 3 hospitals and included both resident and attending physicians. Second, the survey data were gathered from a single institution. Although we included trainees and faculty, more participants would be needed to answer questions about how experience and setting/environmental factors affect these assessments. However, participants were instructed to reflect on their whole experience with each task, which presumably includes multiple institutions and training levels. Third, the sample size is fairly small, with more house staff than faculty (hospitalists and nonhospitalists) represented. Regardless, this study is the first attempt to define and quantify workload for internal medicine physicians using these methods. In future studies, we will expand the number of institutions and levels of experience to validate our current data. Finally, the difficulty of the tasks is clearly a subjective assessment. Although this methodology has face validity, further work needs to be done to validate these findings against other measurements of workload, such as census, or more general subjective workload assessments, such as the NASA task load index.33
In conclusion, we have described the tasks performed by inpatient physicians and the difficulty associated with them. Moreover, we have described a methodology that could be replicated at other centers for the purpose of validating our findings or quantifying workload of other types of tasks. We believe that this is the first step toward a more comprehensive understanding of the workload encountered by inpatient physicians. Because workload has implications for physician burnout and patient safety, it is essential that we fully understand the contributors to workload, including the innate difficulty of the tasks that comprise it.
Acknowledgements
The authors Alexis Visotcky, MS, and Sergey Tarima, PhD, for their assistance with statistics.
This work was presented in poster form at the Society of Hospital Medicine Annual Meeting in April 2010, the Society of General Internal Medicine Annual Meeting in May 2010, and the Society of General Internal Medicine regional meeting in September 2010.
Funding Source: The study team was supported by the following funds during this work: VA grants PPO 0925901 (Marilyn M. Schapira and Kathlyn E. Fletcher) and IIR 07201 (Marilyn M. Schapira, Siddhartha Singh, and Kathlyn E. Fletcher).
- ,,,,.House staff team workload and organization effects on patient outcomes in an academic general internal medicine inpatient service.Arch Intern Med.2007;167:47–52.
- ,,, et al.The work hour rules and contributors to patient care mistakes: A focus group study with internal medicine residentsJ Hosp Med.2008;3:228–237.
- ,,.Multiple measures of anesthesia workload during teaching and nonteaching cases.Anesth Analg.2004;98:1419–1425.
- ,,,,.Developing a technique to measure anesthesiologists' real‐time workload.Proceedings of the Human Factors and Ergonomics Society Annual Meeting.2000;44:241–244.
- ,,, et al.Quantitative description of the workload associated with airway management procedures.J Clin Anesth.2000;12:273–282.
- ,,,,,.An objective methodology for task analysis and workload assessment in anesthesia providers.Anesthesiology.1994;80:77–92.
- ,.Effects of intraoperative reading on vigilance and workload during anesthesia care in an academic medical center.Anesthesiology.2009;110:275–283.
- ,,.Resident work hours: what they are really doing.Arch Surg.2004;139:490–493; discussion, 493–494.
- ,,,,,.Analyzing the time and value of housestaff inpatient work.J Gen Intern Med.1998;13:534–540.
- ,,,.All in a day's work: an observational study to quantify how and with whom doctors on hospital wards spend their time.[see comment].Med J Aust.2008;188:506–509.
- ,,,,.How do house officers spend their nights? A time study of internal medicine house staff on call.N Engl J Med.1989;320:1673–1677.
- ,,,,.Systematic review of time studies evaluating physicians in the hospital setting.J Hosp Med.2010;5:353–359.
- ,,.Time analysis of a general medicine service: results from a random work sampling study.J Gen Intern Med.1994;9:272–277.
- ,,,.A phenomenology of scut.Ann Intern Med.1991;115:372–376.
- ,,, et al.The on‐call experience of interns in internal medicine. Medical Education Task Force of Henry Ford Hospital.Arch Intern Med.1990;150:2294–2297.
- ,,, et al.Where did the day go? A time‐motion study of hospitalists.J Hosp Med.2010;5:323–328.
- ,,.How hospitalists spend their time: insights on efficiency and safety.J Hosp Med.2006;1:88–93.
- ,,, et al.Differences in day and night shift clinical performance in anesthesiology.Hum Factors.2008;50:276–290.
- ,,,,.Assessment of the intrarater and interrater reliability of an established clinical task analysis methodology.Anesthesiology.2002;96:1129–1139.
- ,,.The effect of electronic record keeping and transesophageal echocardiography on task distribution, workload, and vigilance during cardiac anesthesia.Anesthesiology.1997;87:144–155.
- .Fight burnout while fostering experience: investing in hospitalist programs now can fight burnout later.ACP Hospitalist. July2008.
- .Hospitalist burnout: recognize it in yourself and others, and avoid or eliminate it.The Hospitalist. March2006.
- ,,,.Burnout and internal medicine resident work‐hour restrictions.Arch Intern Med.2005;165:2595–2600.
- ,,,,.The effects of work‐hour limitations on resident well‐being, patient care, and education in an internal medicine residency program.Arch Intern Med.2005;165:2601–2606.
- ,,,.Burnout and self‐reported patient care in an internal medicine residency program.Ann Intern Med.2002;136:358–367.
- ,,,.Do house officers learn from their mistakes?Qual Saf Health Care.2003;12:221–226; discussion, 227–228.
- ,,.Extraneous factors in judicial decisions.Proc Natl Acad Sci U S A.2011;108:6889–6892.
- ,.Quantifying physician teaching productivity using clinical relative value units.J Gen Intern Med.1999;14:617–621.
- .Maslach Burnout Inventory Manual.3rd ed.Palo Alto, CA:Consulting Psychology Press;1986.
- ,,,,.Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction.JAMA.2002;288:1987–1993.
- ,,,,.Patient density, nurse‐to‐patient ratio and nosocomial infection risk in a pediatric cardiac intensive care unit.Pediatr Infect Dis J.1997;16:1045–1048.
- ,.Operational failures and interruptions in hospital nursing.Health Serv Res.2006;41:643–662.
- ,.Development of NASA‐TLX (Task Load Index): results of empirical and theoretical research. In: Hancock PA, Meshkati N, eds.Human Mental Workload.Amsterdam, Netherlands:North Holland Press;1988:239–250.
- ,,,,.House staff team workload and organization effects on patient outcomes in an academic general internal medicine inpatient service.Arch Intern Med.2007;167:47–52.
- ,,, et al.The work hour rules and contributors to patient care mistakes: A focus group study with internal medicine residentsJ Hosp Med.2008;3:228–237.
- ,,.Multiple measures of anesthesia workload during teaching and nonteaching cases.Anesth Analg.2004;98:1419–1425.
- ,,,,.Developing a technique to measure anesthesiologists' real‐time workload.Proceedings of the Human Factors and Ergonomics Society Annual Meeting.2000;44:241–244.
- ,,, et al.Quantitative description of the workload associated with airway management procedures.J Clin Anesth.2000;12:273–282.
- ,,,,,.An objective methodology for task analysis and workload assessment in anesthesia providers.Anesthesiology.1994;80:77–92.
- ,.Effects of intraoperative reading on vigilance and workload during anesthesia care in an academic medical center.Anesthesiology.2009;110:275–283.
- ,,.Resident work hours: what they are really doing.Arch Surg.2004;139:490–493; discussion, 493–494.
- ,,,,,.Analyzing the time and value of housestaff inpatient work.J Gen Intern Med.1998;13:534–540.
- ,,,.All in a day's work: an observational study to quantify how and with whom doctors on hospital wards spend their time.[see comment].Med J Aust.2008;188:506–509.
- ,,,,.How do house officers spend their nights? A time study of internal medicine house staff on call.N Engl J Med.1989;320:1673–1677.
- ,,,,.Systematic review of time studies evaluating physicians in the hospital setting.J Hosp Med.2010;5:353–359.
- ,,.Time analysis of a general medicine service: results from a random work sampling study.J Gen Intern Med.1994;9:272–277.
- ,,,.A phenomenology of scut.Ann Intern Med.1991;115:372–376.
- ,,, et al.The on‐call experience of interns in internal medicine. Medical Education Task Force of Henry Ford Hospital.Arch Intern Med.1990;150:2294–2297.
- ,,, et al.Where did the day go? A time‐motion study of hospitalists.J Hosp Med.2010;5:323–328.
- ,,.How hospitalists spend their time: insights on efficiency and safety.J Hosp Med.2006;1:88–93.
- ,,, et al.Differences in day and night shift clinical performance in anesthesiology.Hum Factors.2008;50:276–290.
- ,,,,.Assessment of the intrarater and interrater reliability of an established clinical task analysis methodology.Anesthesiology.2002;96:1129–1139.
- ,,.The effect of electronic record keeping and transesophageal echocardiography on task distribution, workload, and vigilance during cardiac anesthesia.Anesthesiology.1997;87:144–155.
- .Fight burnout while fostering experience: investing in hospitalist programs now can fight burnout later.ACP Hospitalist. July2008.
- .Hospitalist burnout: recognize it in yourself and others, and avoid or eliminate it.The Hospitalist. March2006.
- ,,,.Burnout and internal medicine resident work‐hour restrictions.Arch Intern Med.2005;165:2595–2600.
- ,,,,.The effects of work‐hour limitations on resident well‐being, patient care, and education in an internal medicine residency program.Arch Intern Med.2005;165:2601–2606.
- ,,,.Burnout and self‐reported patient care in an internal medicine residency program.Ann Intern Med.2002;136:358–367.
- ,,,.Do house officers learn from their mistakes?Qual Saf Health Care.2003;12:221–226; discussion, 227–228.
- ,,.Extraneous factors in judicial decisions.Proc Natl Acad Sci U S A.2011;108:6889–6892.
- ,.Quantifying physician teaching productivity using clinical relative value units.J Gen Intern Med.1999;14:617–621.
- .Maslach Burnout Inventory Manual.3rd ed.Palo Alto, CA:Consulting Psychology Press;1986.
- ,,,,.Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction.JAMA.2002;288:1987–1993.
- ,,,,.Patient density, nurse‐to‐patient ratio and nosocomial infection risk in a pediatric cardiac intensive care unit.Pediatr Infect Dis J.1997;16:1045–1048.
- ,.Operational failures and interruptions in hospital nursing.Health Serv Res.2006;41:643–662.
- ,.Development of NASA‐TLX (Task Load Index): results of empirical and theoretical research. In: Hancock PA, Meshkati N, eds.Human Mental Workload.Amsterdam, Netherlands:North Holland Press;1988:239–250.
Copyright © 2012 Society of Hospital Medicine