Original Research

Assessment of teamwork during structured interdisciplinary rounds on medical units

Abstract

Abstract

BACKGROUND:

Interdisciplinary rounds (IDR) provide a means to assemble hospital team members and improve collaboration. Little is known about teamwork during IDR.

OBJECTIVE:

To evaluate and characterize teamwork during IDR.

DESIGN:

Cross‐sectional observational study.

SETTING:

Six medical units which had implemented structured interdisciplinary rounds (SIDR).

MEASUREMENTS:

We adapted the Observational Teamwork Assessment for Surgery (OTAS) tool, a behaviorally anchored rating scale shown to be reliable and valid in surgical settings. OTAS provides scores ranging from 0 to 6 (0 = problematic behavior; 6 = exemplary behavior) across 5 domains (communication, coordination, cooperation/backup behavior, leadership, and monitoring/situational awareness) and for prespecified subteams. Two researchers conducted direct observations using the adapted OTAS tool.

RESULTS:

We conducted 7–8 independent observations for each unit (total = 44) and 20 joint observations. Inter‐rater reliability was excellent at the unit level (Spearman's rho = 0.75), and good across domains (rho = 0.53–0.68) and subteams (rho = 0.53–0.76) with the exception of the physician subteam, for which it was poor (rho = 0.35). Though teamwork scores were generally high, we found differences across units, with a median (interquartile range [IQR]) 4.5 (3.9–4.9) for the lowest and 5.4 (5.3–5.5) for the highest performing unit (P < 0.01). Domain scores differed, with leadership receiving the lowest (median [IQR] = 5.0 [4.6–5.3]), and cooperation/backup behavior and monitoring/situational awareness receiving highest scores (median [IQR] = 5.4 [5.0–5.5] and 5.4 [5.0–5.7]). Differences across subteams were of borderline significance (P = 0.05).

CONCLUSIONS:

The adapted OTAS instrument demonstrated acceptable reliability for assessing teamwork during SIDR across units, domains, and most subteams. Variation in performance suggests a need to improve consistency of teamwork and emphasizes the importance of leadership. Journal of Hospital Medicine 2012. © 2012 Society of Hospital Medicine

Copyright © 2012 Society of Hospital Medicine

Teamwork is essential to delivering safe and effective hospital care,15 yet the fluidity and geographic dispersion of team members in the hospital setting presents a significant barrier to teamwork.6 Physicians, nurses, and other hospital professionals frequently lack convenient and reliable opportunities to interact, and may struggle in efforts to discuss the care of their patients in person. Research studies show that nurses and physicians on patient care units do not communicate consistently and frequently do not agree on key aspects of their patients' plans of care.7, 8

Interdisciplinary rounds (IDR), also known as multidisciplinary rounds, provide a means to assemble hospital care team members and improve collaboration.913 Prior research on the use of IDR has demonstrated improved ratings of collaboration,11, 12 but inconsistent effects on length of stay and cost.10, 12, 13 Notably, the format, frequency, and duration of IDR in prior studies has been variable and no studies, to our knowledge, have evaluated teamwork performance during IDR. Lamb and colleagues conducted observations of cancer teams during multidisciplinary meetings.14 Trained observers used a validated observation tool to rate teamwork and found significant variation in performance by subteams. However, the study focused mainly on discussion among physician team members during meetings to plan longitudinal care for oncology patients.

We recently reported on the use of structured interdisciplinary rounds (SIDR) on 2 medical units in our hospital.15, 16 SIDR combines a structured format for communication, similar to a goals‐of‐care form,17, 18 with a forum for daily interdisciplinary meetings. Though no effect was seen on length of stay or cost, SIDR was associated with significantly higher ratings of the quality of collaboration and teamwork climate, and a reduction in the rate of adverse events.19 In March 2010, we implemented SIDR across all medical units in our hospital. We subjectively noted variation in teamwork performance during SIDR after a modification of nurse manager roles. We sought to evaluate teamwork during SIDR and to determine whether variation in performance existed and, if present, to characterize it.

METHODS

Setting and Study Design

The study was conducted at Northwestern Memorial Hospital (NMH), a 920‐bed tertiary care teaching hospital in Chicago, IL, and was deemed exempt by the Institutional Review Board of Northwestern University. General medical patients were admitted to 1 of 6 units based on bed availability. Five of the medical units consisted of 30 beds, and 1 unit consisted of 23. Each unit was equipped with continuous cardiac telemetry monitoring. Three units were staffed by teaching service physician teams consisting of 1 attending, 1 resident, and 1 or 2 interns. The other 3 units were staffed by hospitalists without the assistance of resident or intern physicians. As a result of a prior intervention, physicians' patients were localized to specific units in an effort to improve communication practices among nurses and physicians.20

Beginning in March 2010, all general medical units held SIDR each weekday morning. SIDR took place in the unit conference room, was expected to last approximately 3040 minutes, and was co‐led by the unit nurse manager and a medical director. Unit nurse managers and medical directors received specific training for their roles, including 3 hours of simulation‐based exercises designed to enhance their skills in facilitating discussion during SIDR. All nurses and physicians caring for patients on the unit, as well as the pharmacist, social worker, and case manager assigned to the unit, attended SIDR. Attendees used a structured communication tool to review patients admitted in the previous 24 hours. The plan of care for other patients was also discussed in SIDR, but without the use of the structured communication tool.

Importantly, nurse management underwent restructuring in the summer of 2011. Nurse managers, who had previously been responsible for overseeing all nursing activities on a single unit, were redeployed to be responsible for specific activities across 34 units. This restructuring made it very difficult for nurse managers to colead SIDR. As a result, the unit nurse clinical coordinator assumed coleadership of SIDR with the unit medical director. Nurse clinical coordinators worked every weekday and did not have patient care responsibilities while on duty. In addition to their role in coleading SIDR, nurse clinical coordinators addressed daily staffing and scheduling challenges and other short‐term patient care needs.

Teamwork Assessment

We adapted the Observational Teamwork Assessment for Surgery (OTAS) tool, a behaviorally anchored rating scale shown to be reliable and valid in surgical settings.2123 The OTAS tool provides scores ranging from 0 to 6 (0 = problematic behavior; 3 = team function neither hindered nor enhanced by behavior; 6 = exemplary behavior) across 5 domains (communication, coordination, cooperation/backup behavior, leadership, and monitoring/situational awareness) and for prespecified subteams. We defined domains as described by the researchers who developed OTAS. Communication referred to the quality and the quantity of information exchanged by team members. Coordination referred to management and timing of activities and tasks. Cooperation and backup behavior referred to assistance provided among members of the team, supporting others and correcting errors. Leadership referred to provision of directions, assertiveness, and support among team members. Monitoring and situational awareness referred to team observation and awareness of ongoing processes. We defined subteams for each group of professionals expected to attend SIDR. Specifically, subteams included physicians, nurses, social work‐case management (SW‐CM), pharmacy, and coleaders. We combined social work and case management because these professionals have similar patient care activities. Similarly, we combined unit medical directors and nurse clinical coordinators as coleaders. By providing data on teamwork performance within specific domains and for specific subteams, the OTAS instrument helps identify factors influencing overall teamwork performance. We modified OTAS anchors to reflect behaviors during SIDR. Anchors assisted observers in their rating of teamwork behaviors during SIDR. For example, an anchor for exemplary physician communication behavior was listens actively to other team members (looks at other team members, nods, etc). An anchor for exemplary physician leadership was assigns responsibility for task completion when appropriate.

Two researchers conducted unannounced direct observations of SIDRs. One researcher (Y.N.B) was a medical librarian with previous experience conducting observational research. The other researcher (A.J.C.) had observed 170 prior SIDRs as part of a related study. Both researchers observed 10 SIDRs to practice data collection and to inform minor revisions of the anchors. We aimed to conduct 78 independent observations for each unit, and 20 joint observations to assess inter‐rater reliability. All subteams were scored for each domain. For example, all subteams received leadership domain scores because all team members exhibit leadership behaviors, depending on the situation. In addition to teamwork scores, observers recorded the number of patients on the unit, the number of patients discussed during SIDR, attendance by subteam members, and the duration of SIDR. For the SW‐CM and coleader subteams, we documented presence if one of the subteam members was present for each patients' discussion. For example, we recorded present for SW‐CM if the social worker was in attendance but the case manager was not.

Data Analysis

We calculated descriptive statistics to characterize SIDRs. We used Spearman's rank correlation coefficients to assess inter‐rater reliability for joint observations. Spearman's rank correlation is a nonparametric test of association and appropriate for assessing agreement between observers when using data that is not normally distributed. Spearman rho values range from 1 to 1, with 1 signifying perfect inverse correlation, 0 signifying no correlation, and 1 signifying perfect correlation. We used the MannWhitney U test to assess for differences in overall team scores between services (teaching vs nonteaching hospitalist service) and KruskalWallis tests to assess for differences across units, domains, and subteams. The Kruskal‐Wallis test is a nonparametric test appropriate for comparing three or more independent samples in which the outcome is not normally distributed. We used a t test to assess for difference in duration by service, and Spearman rank correlation to assess for correlation between time spent in discussion per patient and overall team score. All analyses were conducted using Stata version 11.0 (College Station, TX).

RESULTS

SIDR Characteristics

We performed 7 direct observations of SIDR for 4 units, and 8 observations for 2 units (44 total observations). Units were at 99% capacity, and SIDR attendees discussed 98% of patients on the unit. Attendance exceeded 98% for each subteam (physicians, nurses, SW‐CM, pharmacy, and coleaders). SIDR lasted a mean 41.4 11.1 minutes, with a mean 1.5 0.4 minutes spent in discussion per patient. SIDR was significantly longer in duration on teaching service units compared to the nonteaching hospitalist service units (1.7 0.3 vs 1.3 0.4 minutes per patient; P < 0.001).

Inter‐Rater Reliability

Inter‐rater reliability across unit level scores was excellent (rho = 0.75). As shown in Table 1, inter‐rater reliability across domains was good (rho = 0.530.68). Inter‐rater reliability across subteams was good to excellent (rho = 0.530.76) with the exception of the physician subteam, for which it was poor (rho = 0.35).

Inter‐Rater Reliability Across Domain and Subteams
Spearman's rhoP Value
  • Abbreviations: SW‐CM, social work‐case management.

Domain (n = 20)
Communication0.62<0.01
Coordination0.60<0.01
Cooperation/backup behavior0.66<0.01
Leadership0.68<0.01
Monitoring/situational awareness0.530.02
Subteam (n = 20)
Physicians0.350.14
Nurses0.530.02
SW‐CM0.60<0.01
Pharmacy0.76<0.01
Coleaders0.68<0.01

Assessment of Teamwork by Unit, Domain, and Subteams

Teaching and nonteaching hospitalist units had similar team scores (median [interquartile range (IQR)] = 5.2 [1.0] vs 5.2 [0.4]; P = 0.55). We found significant differences in teamwork scores across units and domains, and found differences of borderline statistical significance across subteams (see Table 2). For unit teamwork scores, the median (IQR) was 4.4 (3.94.9) for the lowest and 5.4 (5.35.5) for the highest performing unit (P = 0.008). Across domain scores, leadership received the lowest score (median [IQR] = 5.0 [4.65.3]), and cooperation/backup behavior and monitoring/situational awareness received the highest scores (median [IQR]) = 5.4 [5.05.5] and 5.4 [5.05.7]; P = 0.02). Subteam scores ranged from a median (IQR) 5.0 (4.45.8) for coleaders to 5.5 (5.05.8) for SW‐CM (P = 0.05). We found no relationship between unit teamwork score and time spent in discussion per patient (rho = 0.04; P = 0.79).

Teamwork Scores Across Units, Domains, and Subteams
Median (IQR)P Value
  • NOTE: Scores ranged from 0 to 6 (0 = problematic behavior; 3 = team function neither hindered nor enhanced by behavior; 6 = exemplary behavior). Abbreviations: IQR, interquartile range; SW‐CM, social work‐case management.

  • Units A, B, D, and F had 7 observations each; units C and E had 8 observations each.

Unit (n = 44)*
A5.3 (5.15.4)0.008
B5.4 (5.35.5)
C5.1 (4.95.2)
D5.4 (5.25.6)
E4.4 (3.94.9)
F5.3 (5.15.5)
Domain (n = 44)
Communication5.2 (4.95.4)0.02
Coordination5.2 (4.75.4)
Cooperation/backup behavior5.4 (5.05.5)
Leadership5.0 (4.65.3)
Monitoring/situational awareness5.4 (5.05.7)
Subteam (n = 44)
Physicians5.2 (4.95.4)0.05
Nurses5.2 (5.05.4)
SW‐CM5.5 (5.05.8)
Pharmacy5.3 (4.85.8)
Coleaders5.0 (4.45.8)

DISCUSSION

We found that the adapted OTAS instrument demonstrated acceptable reliability for assessing teamwork during SIDR across units, domains, and most subteams. Although teamwork scores during SIDR were generally high, we found variation in performance across units, domains, and subteams. Variation in performance is notable in light of our efforts to implement a consistent format for SIDR across units. Specifically, all units have similar timing, duration, frequency, and location of SIDR, use a structured communication tool for new patients, expect the same professions to be represented, and use coleaders to facilitate discussion. We believe teamwork within IDR likely varies across units in other hospitals, and perhaps to a larger degree, given the emphasis on purposeful design and implementation of SIDR in our hospital.

Our findings are important for several reasons. First, though commonly used in hospital settings, the effectiveness of IDR is seldom assessed. Hospitalists and other professionals may not be able to identify or characterize deficiencies in teamwork during IDR without objective assessment. The adapted OTAS instrument provides a useful tool to evaluate team performance during IDR. Second, professionals may conclude that the mere implementation of an intervention such as SIDR will improve teamwork ratings and improve patient safety. Importantly, published studies evaluating the benefits of SIDR reflected a pilot study occurring on 2 units.15, 16, 19 The current study emphasizes the need to ensure that interventions proven to be effective on a small scale are implemented consistently when put into place on a larger scale.

Despite good reliability for assessing teamwork during SIDR across units, domains, and most subteams, we found poor inter‐rater reliability for the physician subteam. The explanation for this finding is not entirely clear. We reviewed the anchors for the physician subteam behaviors and were unable to identify ambiguity in anchor definitions. An analysis of domain scores within the physician subteam did not reveal any specific pattern to explain the poor correlation.

We found that the leadership domain and coleader subteam received particularly low scores. The explanation for this finding likely relates to changes in the nurse management structure shortly before our study, which reduced attendance by nurse managers and created a need for clinical coordinators to take on a leadership role during SIDR. Although we provided simulation‐based training to unit medical directors and nurse managers prior to implementing SIDR in March 2010, clinical coordinators were not part of the initial training. Our study suggests a need to provide additional training to coleaders, including clinical coordinators, to enhance their ability to facilitate discussion in SIDR.

We found no difference in overall teamwork scores when comparing teaching service units to nonteaching hospitalist service units. Duration of SIDR was significantly longer on teaching service units, but there was no association between duration of discussion and overall team score. The difference in duration of SIDR is likely explained by less succinct discussions on the part of housestaff physicians compared to more experienced hospitalists. Importantly, the quality of input, and its impact on teamwork during SIDR, does not appear to suffer when physician discussion is less efficient.

Our study has several limitations. First, we evaluated IDR in a single, urban, academic institution, which may limit generalizability. Our version of IDR (ie, SIDR) was designed to improve teamwork and incorporate a structured communication tool with regularly held interdisciplinary meetings. Features of IDR may differ in other hospitals. Second, the high teamwork scores seen in our study may not be generalizable to hospitals which have used a less rigorous, less standardized approach to IDR. Third, SIDR did not include patients or caregivers. Research is needed to test strategies to include patients and caregivers as active team members and participants in clinical decisions during hospitalization. Finally, we used the term interdisciplinary rounds to be consistent with prior published research. The term interprofessional may be more appropriate, as it specifically describes interactions among members of different professions (eg, physicians, nurses, social workers) rather than among different disciplines within a profession (eg, cardiologists, hospitalists, surgeons).

In summary, we found that teamwork during IDR could be reliably assessed using an adapted OTAS instrument. Although scores were generally high, we found variation in performance across units and domains suggesting a need to improve consistency of teamwork performance across units, domains, and subteams. Our study fills an important gap in the literature. Although IDR is commonly used in hospitals, and research shows improvements in ratings of collaboration,11, 12 little if any research has evaluated teamwork during IDR. Beyond the mere implementation of IDR, our study suggests the need to confirm that teamwork is optimal and consistent. Furthermore, hospital leaders should consider specific training for clinicians leading discussion during IDR.

Acknowledgements

The authors express their gratitude to Nick Sevdalis, BSc, MSc, PhD for providing the OTAS instrument and detailed instructions on its use.

Disclosures: Dr O'Leary, Ms Creden, and Dr Williams received salary support from the Agency for Healthcare Research and Quality, grant R18 HS019630. All authors disclose no other relevant or financial conflicts of interest.

References

Online-Only Materials

AttachmentSize
File Supporting Information (1)38.69 KB
   Comments ()