Brief Reports

A Randomized Cohort Controlled Trial to Compare Intern Sign-Out Training Interventions

Abstract

BACKGROUND: Although previous studies have investigated the efficacy of specific sign-out protocols (such as the illness severity, patient summary, action list, situation awareness and contingency planning, and synthesis by reviewer [I-PASS] bundle), the implementation of a bundle can be time consuming and costly. We compared 4 sign-out training pedagogies on sign-out quality.

OBJECTIVE: To evaluate training interventions that best enhance multidimensional sign-out quality measured by information exchange, task accountability, and personal responsibility.

INTERVENTION: Four general internal medicine firms were randomly assigned into 1 of the following 4 training interventions: didactics (control), I-PASS, policy mandate on task accountability, and Plan-Do-Study-Act (PDSA).

SETTING: First-year interns at a large, Mid-Atlantic internal medicine residency program.

MEASUREMENTS: Eight trained observers examined 10 days each in the pre- and postintervention periods for each firm using a standardized sign-out checklist.

RESULTS: Pre- and postintervention differences showed significant improvements in the transfer of patient information, task accountability, and personal responsibility for the I-PASS, policy mandate, and PDSA groups, respectively, in line with their respective training foci. Compared to the control, I-PASS reported the best improvements in sign-out quality, although there was room to improve in task accountability and responsibility.

CONCLUSIONS: Different training emphases improved different dimensions of sign-out quality. A combination of training pedagogies is likely to yield optimal results.

© 2017 Society of Hospital Medicine

Patient sign-outs are defined as the transition of patient care that includes the transfer of information, task accountability, and personal responsibility between providers.1-3 The adoption of mnemonics as a memory aid has been used to improve the transfer of patient information between providers.4 In the transfer of task accountability, providers transfer follow-up tasks to on-call or coverage providers and ensure that directives are understood. Joint task accountability is enhanced through collaborative giving and cross-checking of information received through assertive questioning to detect errors, and it also enables the receiver to codevelop an understanding of a patient’s condition.5-8 In the transfer of personal responsibility for the primary team’s patients, the provision of anticipatory guidance enables the coverage provider to have prospective information about potential, upcoming issues to facilitate care plans.6 Enabling coverage providers to anticipate overnight events helps them exercise responsibility for patients who are under their temporary care.2

The Accreditation Council for Graduate Medical Education requires residency programs to provide formal instruction on sign-outs.9 Yet, variability across training programs exists,8,10 with training emphasis on the transfer of information over accountability or responsibility.11 Previous studies have demonstrated the efficacy of sign-out training, such as the illness severity, patient summary, action list, situation awareness and contingency planning, and synthesis by reviewer (I-PASS) bundle.3 Yet, participation is far from 100% because the I-PASS bundle requires in-person workshops, e-learning platforms, organizational change campaigns, and faculty participation,12 involving resource and time commitments that few programs can afford. To address this issue, we seek to compare resource-efficient, knowledge-based, skill-based, compliance-based, and learner-initiated sign-out training pedagogies. We focused on the evening sign-out because it is a high-risk period when care for inpatients is transferred to smaller coverage intern teams.

METHODS

Setting and Study Design

A prospective, randomized cohort trial of 4 training interventions was conducted at an internal medicine residency program at a Mid-Atlantic, academic, tertiary-care hospital with 1192 inpatient beds. The 52 interns admitted to the program were randomly assigned to 4 firms caring for up to 25 inpatients on each floor of the hospital. The case mix faced by each firm was similar because patients were randomly assigned to firms based on bed availability. Teams of 5 interns in each firm worked in 5-day duty cycles, during which each intern rotated as a night cover for his or her firm. Interns remain in their firm throughout their residency. Sign-outs were conducted face to face with a computer. Receivers printed sign-out sheets populated with patient information and took notes when senders communicated information from the computer. The hospital’s institutional review board approved this study.

Interventions

The firms were randomly assigned to 1 of 4 one-hour quality-improvement training interventions delivered at the same time and day in November 2014 at each firm’s office, located on different floors of the hospital. There was virtually no cross-talk among the firms in the first year, which ensured the integrity of the cohort randomization and interventions. Faculty from an affiliated business school of the academic center worked with attending physicians to train the firms.

All interventions took 1 hour at noontime. Firm 1 (the control) received a didactic lecture on sign-out, which participants heard during orientation. Repeating that lecture reinforced their knowledge of sign-outs. Firm 2 was trained on the I-PASS mnemonic with a predictable progression of information elements to transfer.3,12 Interns role-played 3 scenarios to practice sign-out.3 They received skills feedback and a debriefing to link I-PASS with information elements to transfer. Firm 3 was dealt a policy mandate by the interns’ attending physician to perform specific tasks at sign-out. Senders were to provide the night cover with to-do tasks, and receivers were to actively discuss and verify these tasks to ensure task accountability.13 Firm 4 was trained on a Plan-Do-Study-Act (PDSA) protocol to identify and solve perceived barriers to sign-outs. Firm 4 agreed to solve the problem of the lack of care plans by the day team to the night cover. An ad hoc team in Firm 4 refined, pilot tested, and rolled out the solution within a month. Its protocol emphasized information on anticipated changes in patient status, providing contingency plans and their rationale as well as discussions to clarify care plans. Details of the 4 interventions are shown in the Table.

Data Collection Process

Eight trained senior residents, recruited by the last author (S.V.D.), volunteered to observe 10 evening sign-outs in each firm 1 month prior to the intervention and another 10 nights 4 months after training. Observations were standardized with a sign-out checklist developed from the literature review and the Joint Commission’s 2006 National Patient Safety Goal 2E that followed the Situation, Background, Assessment, and Recommendation communication structure with opportunities for questioning and information verification.14,15 Observers indicated “1” for each of the 17 sign-out elements in the checklist they observed, as shown in the supporting Table. Observers did not have supervisory relationships with the interns. Occasionally, the pairs of observers were different depending on their availability.

Outcomes

We measured improvements in sign-out quality by the mean percentage differences for each of the 3 dimensions of sign-out, as well as a multidimensional measure of sign-out comprising the 3 dimensions for each firm in 2 ways: (1) pre- and postintervention, and (2) vis-à-vis the control group postintervention.

Statistical Analysis

We factor analyzed the 17 sign-out elements using principal components analysis with varimax rotation to confirm their groupings within the 3 dimensions of sign-out using Statistical Package for the Social Sciences (SPSS) version 24 (IBM, North Castle, NY). We calculated the mean percentage differences and used Student t tests to evaluate statistical differences at P < 0.05.

RESULTS

Five hundred and sixty-three patient sign-outs were observed prior to the training interventions (κ = 0.646), and 620 patient sign-outs were observed after the interventions (κ = 0.648). Kappa values derived from SPSS were within acceptable interrater agreement ranges. Factor analysis of the 17 sign-out elements yielded 3 factors that we named patient information, task accountability, and responsibility, as shown in the supporting Table.

The supporting Figure reports 2 sets of results. The line graphs show the pre- and postintervention differences for each firm while the bar charts show the postintervention differences between each firm vis-à-vis the control group on sign-out dimensions. The line graphs indicate the greatest improvements in patient information, task accountability, and responsibility for the I-PASS, policy mandate, and PDSA groups, respectively. Mandate and PDSA groups reported low relative scores on sign-out dimensions that were not the foci of their training while the didactics group scored around 0 pre- and postintervention. I-PASS had the highest improvement on the multidimensional measure of sign-out quality but was not significantly different from the PDSA group at P < 0.05 (see supporting Figure for the calculations). The bar charts indicate that all groups vis-à-vis the control had higher improvements in task accountability, responsibility, and the multidimensional measure of sign-out quality. I-PASS vis-à-vis the control had the highest improvement but was not statistically different from the PDSA at P < 0.05. No sentinel events were reported during the entire study period.

DISCUSSION

The results indicated that after only 1 hour of training, skill-based, compliance-based, and learner-initiated sign-out training improved sign-out quality beyond knowledge-based didactics even though the number of sign-out elements taught in the latter 2 was lower than in the didactics group. Different training emphases influenced different dimensions of sign-out quality so that training interns to focus on task accountability or responsibility led to improvements in those dimensions only. The lower scores in other dimensions suggest potential risks in sign-out quality from focusing attention on 1 dimension at the expense of other dimensions. I-PASS, which covered the most sign-out elements and utilized 5 facilitators, led to the best overall improvement in sign-out quality, which is consistent with previous studies.3,12 We demonstrated that only 1 hour of training on the I-PASS mnemonics using video, role-playing, and feedback led to significant improvements. This approach is portable and easily applied to any program. Potential improvements in I-PASS training could be obtained by emphasizing task accountability and responsibility because the mandate and PDSA groups obtained higher scores than the I-PASS group in these dimensions.

Limitations

We measured sign-out quality in the evening at this site because it was at greatest risk for errors. Future studies should consider daytime sign-outs, interunit handoffs, and other hospital settings, such as community or rural hospitals and nonacute patient settings, to ascertain generalizability. Data were collected from observations, so Hawthorne effects may introduce bias. However, we believe that using a standardized checklist, a control group, and assessing relative changes minimized this risk. Although we observed almost 1200 patient sign-outs over 80 shift changes, we were not able to observe every intern in every firm. Finally, no sentinel events were reported during the study period, and we did not include other measures of clinical outcomes, which represent an opportunity for future researchers to test which specific sign-out elements or dimensions are related to clinical outcomes or are relevant to specific patient types.

CONCLUSION

The results of this study indicate that 1 hour of formal training can improve sign-out quality. Program directors should consider including I-PASS with additional focus on task accountability and personal responsibility in their sign-out training plans.

Disclosure

The authors have nothing to disclose.

References

   Comments ()

Related Articles from PubMed