Original Research

Using observed structured teaching exercises (OSTE) to enhance hospitalist teaching during family centered rounds

Abstract

Abstract

BACKGROUND:

“Rounds” are the organizing structure for academic hospitalist‐trainee teams. Family centered rounds (FCR) are endorsed by the Institute of Medicine and by the American Academy of Pediatrics, however, rounds are often conducted in the privacy of the conference room where patients and families are not privy to the decision‐making process used to determine their diagnosis and management. Less than half of pediatric hospitalists reported conducting FCR, citing concerns about inefficiency and diminished teaching.

OBJECTIVE:

The aim of this study was to design and implement a faculty development program to address the need of hospitalists to efficiently teach during FCR.

DESIGN:

Scoring templates were developed to measure evidence‐based teaching behaviors to optimize orientation, feedback, clinical reasoning, assessing physical exam findings, and promoting resident leadership during FCR. Hospitalists were scored by Standardized Learners and Standardized Parents during 4 Observed Structured Teaching Exercises (OSTEs) before and after focused workshops.

RESULTS:

Fourteen of 14 hospitalist participants had 17 months ± 14 months of experience; 71% were female; none had previous training in the areas proposed in the study. The differences between pre‐ and post‐OSTEs for the 4 stations were statistically significant (P < .0001). Particular improvements were noted in the correction of incorrect clinical reasoning (new patient diagnosis) (56% pre, 86% post) and orientation (65% pre, 95% post).

CONCLUSIONS:

We found incorporating OSTEs into a FCR faculty development program to be an effective strategy for improving faculty teaching behavior. Additional study is needed to determine if this strategy results in sustained improvements in conducting FCRs in real inpatient settings. Journal of Hospital Medicine 2011. © 2011 Society of Hospital Medicine

Copyright © 2011 Society of Hospital Medicine

Providing family centered care has been identified as a goal in the Institute of Medicine's report Crossing the Quality Chiasm1 and endorsed by the American Academy of Pediatrics.2 Traditionally, rounds are the central organizing structure for clinical work, decision making, and teaching in the inpatient setting. Patient care and educational goals emanate from rounds. Over the past several decades rounds have migrated from the patient's bedside to the privacy of the conference room. In our experience, although conference room rounds offer some advantages, patients and families are not privy to the data or decision‐making process used to determine their diagnosis and plan of care. The ritual that frequently occurs after conference‐room rounds is that the team members (medical students, residents, nurses, attending) visit the patient and family independently throughout the course of the day, communicating their understanding of medical and affective issues in a manner that families often view as providing confusing, if not contradictory information.

Conducting rounds entirely at the bedside can bypass this systemic flaw, allowing parents and patients to correct inaccurate data, and enable them to make their values and concerns known to the team. This model can help to connect the caregivers and receivers of care, and represents a collaborative communication process, the foundation for effective family‐centered rounds (FCR). When team members discuss how they interpret clinical data in the presence of the family it helps them to understand how and why a management plan is conceived. The care team develops an alliance of trust with the family through this transparent communication and joint decision‐making.

Despite the potential for enhancing patient/family satisfaction and endorsements by public and professional organizations, in a recent study less than half of pediatric hospitalists reported conducting FCR.3 Trainees and attending physicians raised concerns about the potential for FCR to waste time and diminish teaching.4 Trainees' perceptions of the educational value of FCR has not been well studied, but a recent qualitative study of pediatric residents reported that if conducted well, FCRs enhance education and clinical skills by increasing the number of patients seen by each team member, and by offering opportunities to improve physical examination skills. Trainees appreciated role‐modeling and realtime feedback by attending physicians. Senior residents reported enhanced leadership and teaching opportunities.5

The aim of this study was to design and implement a faculty development program to address the need of our junior hospitalist faculty members to enhance teaching during FCR.

Methods

We determined, based upon direct observation, a focus group and survey feedback from our pediatric residents, that for inpatient teaching during FCR to be successful, our faculty needed training in the following areas: orienting learners, providing feedback, teaching assessment of key physical exam findings, correcting errors in clinical reasoning, and promoting the role of the senior resident as team leader. We developed the Observed Structured Teaching Exercises (OSTE)6 and related workshops to promote key behaviors identified from the literature for each of the areas.

All of the Children's National Medical Center (CNMC) Pediatric hospitalists (N = 14) who were not investigators in the study were asked to participate. They were informed of the study design and the overall goal of making inpatient rounds more effective and efficient through better teaching skills. The study was approved by the CNMC institutional review board and was conducted from August to September 2007 in the CLASS (Clinical Learning and Simulation Skills) at The George Washington University School of Medicine and Health Sciences.

To assess faculty and fellow baseline knowledge and skills, the authors conducted a preintervention OSTE consisting of 4 stations: 1) physical exam interpretation and promoting PL‐3 autonomy (Established Patient), 2) stimulating clinical reasoning (New Patient), 3) feedback, and 4) facilitating an orientation. This exercise was followed within 2 weeks by four 90‐minute interactive workshops that focused on the topic areas as evaluated in the OSTEs. Each workshop consisted of a brief evidence‐based didactic component, interactive discussion, and skill building exercises to practice desired teaching behaviors. Two weeks following the workshops, the group participated in postintervention OSTEs similar to the preintervention scenarios, with minor changes, such as presenting diagnoses, to avoid pattern recognition.

Development of the Evaluation Process

The authors reviewed the literature on providing effective feedback7 and orientation8; in teaching a skill9; promoting senior resident autonomy10 and clinical reasoning.11 We also reviewed the faculty development literature12, 13 to determine which behaviors were found to be effective specifically for promoting teaching during FCR, but no studies specifically addressed evaluation of teaching skills during FCR. Checklists were created based on the evidence in the literature and supplemented by the consensus of the investigators when there was no evidence available (see Supp. Appendix S1, which is available online).

Two stations simulating FCR (physical exam interpretation and promoting PL‐3 autonomy [Established Patient]; and stimulating clinical reasoning [New Patient]), each used 2 Standardized Learners (SL) and 1 Standardized Parent (SP). The patient was portrayed using a poster or simulator. The stations simulating feedback and orientation used 1 SL. To conduct 14 pre‐ and post‐OSTEs, we used a total of 5 SPs and 20 SLs. The SPs were recruited from a cohort of individuals that regularly participate in OSCE teaching and evaluation scenarios in the CLASS Center. The SLs were 4th year medical students enrolled in the TALKS (Teaching and Learning Communication Skills) elective and trained how to portray SLs.14

Training consisted of advanced distribution of specific scripts to SPs/SLs and practice through role playing the scenarios with study investigators acting as the attending hospitalist. The SP/SLs and investigators tried to anticipate several possible ways participants might react to the scenarios so that SPs and SLs could standardize their responses and interrater reliability for rating checklists of desired teaching behaviors. SLs rated faculty according to the teaching behavior template during a 5‐minute interval immediately after each OSTE. Different SLs were used for pre‐ versus postintervention OSTEs and were unaware of the intervention itself or whether faculty participants were pre‐ or postintervention.

Each of the 4 OSTE stations began with the hospitalist reading a brief paragraph describing the scenario and the overall goals for the OSTE. SLs/SPs acted out scripts designed by investigators to provide opportunities for hospitalists to demonstrate desired teaching behaviors. Each OSTE was designed to be completed within 10 minutes.

Development of the Intervention Workshops

Five Hospitalist faculty members with extensive training in faculty development facilitated four 90‐minute workshops, each focused on the goals of a particular pretest OSTE session. The learning objectives for each workshop are listed in Table 1. Each interactive workshop included a brief, evidence‐based didactic portion followed by a presentation of the evaluation checklists and an aggregate summary of hospitalist pretest ratings on the corresponding OSTE.

Workshop Objectives
  • Abbreviations: OSTE, Observed Structured Teaching Exercises; PE, physical examination.

Established Patient Workshop: Promoting the Senior Resident Leadership Role and Physical Exam Assessment
1.Identify barriers to teaching PE skills/emnterpretation at bedside
2.Identify barriers to promoting the role of the senior resident as leader
3.Discuss strategies for overcoming 1 & 2
4.State what is meant by Deliberate Practice
5.State the key aspects of Activated Demonstration
6.Practice Activated Demonstration through deliberate practice using the OSTE scoring template in role plays
Feedback Workshop
1.State the value of feedback to learners
2.Identify barriers to giving feedback, especially corrective feedback
3.Discuss strategies for promoting reflective self‐assessment
4.Describe examples that represent effective strategies for reinforcing behaviors
5.Describe examples that represent effective strategies for correcting behaviors
6.Practice through role play (using the OSTE scoring template):
a.Developing a learner‐centered action plan
b.Eliciting learner's feelings about feedback and action plan
c.Exploring the learner's readiness to implement plan
Workshop Promoting Clinical Reasoning‐Correcting Wrong New Patient Diagnosis
1.Identify barriers to trainees giving focused oral presentations
2.Identify barriers to teaching clinical reasoning
3.Identify barriers inherent in discussing diagnostic uncertainty and misdiagnosis in front of families
4.Discuss strategies for overcoming 1‐3
5.Describe the theoretical framework behind Problem Representation
6.Describe the key behaviors that comprise the OMP model
7.Practice using abstractions of the key features to represent the problem
8.Practice identifying knowledge/synthesis gaps and correcting learner mistakes using the OSTE scoring template in role‐plays
Orientation Workshop
1.State the value of orientation to learners
2.State the key elements for an effective orientation
3.Identify barriers to providing an orientation
4.Discuss strategies for effectively orienting learners
5.Practice orienting a learner through role play using the OSTE scoring template

After facilitators explained the theory behind determining the checklist behaviors, participants discussed the checklists and agreed on the validity of the rating instruments. The participants determined strategies to consistently remember to incorporate the desired behaviors, such as using mnemonics on pocket‐sized laminated cards and then practiced desired behaviors using roleplay.

Analysis

The percentage of total points possible on each of the pretest and posttest OSTE scoring templates was compared using a paired Student t test for each of the 14 participants.

Results

All 14 eligible hospitalists voluntarily participated. Their mean year postcompletion of residency training for the faculty was 17 months 14 months; 71% were female. None of the participants experienced previous training in the areas proposed in the study.

Participants assigned high scores to the quality of the workshops, the OSTE experience and their learning from the participating in the faculty development exercise. The differences between pre‐ and post‐OSTE scenario as well as overall scores for the 4 stations were statistically significant (P < .0001). Particular improvements were noted in the correction of incorrect new admission diagnoses (56% pre, 86% post) and orientation (65% pre, 95% post; see Table 2).

Pre‐ and Post‐OSTE t Test Results
OSTE stationPrePostDFt value
  • P < .0001.

  • Abbreviations: OSTE, Observed Structured Teaching Exercises; PE, physical examination.

PE skill/ leadership70%91%129.07*
Feedback71%94%127.40*
Clinical reasoning56%86%1312.40*
Orientation65%95%137.56*
Overall64%90%1317.58*

Discussion

If FCR are to be universally adopted in the academic pediatric inpatient setting, faculty must successfully balance the educational needs of trainees as well as efficiently negotiating a plan of management with patients and families. The ability of faculty to consistently orchestrate rounds so that they meet educational needs of varied levels of learners, while ensuring that patient management is correct and well communicated to families is a very complex task.

We found using OSTEs to frame desired behavior, supplemented with background information to validate the desired behaviors followed by deliberate practice opportunities during the workshops to be an effective faculty development strategy. We not only provided participants with feedback on the group's performance according to the rating scale, but also gave them the opportunity to practice rating each other using the scale so that they could reflect on the elements of their performance that merited a specific rating.

This strategy for training faculty to perform well in the complex environment within the patient's room during FCR is similar in some respects with training military personnel for complex battle situations.15 Desired behaviors are broken down and packaged within a framework to be implemented in a specific context. For example, we combined aspects of the One Minute Preceptor model (OMP)12 with Bordage's Problem Representation model to create a framework of behaviors to promote and correct errors in clinical reasoning.16, 17 Another framework was created and practiced to promote assessment of the physical exam at the bedside. Orientation and feedback, although not frequently used components of actual FCR, are necessary to set expectations and calibrate learner's performance during FCR.

The OSTE is an observed examination that has been validated for evaluating the teaching skills of faculty and residents.18 We planned to use learner‐centered, interactive workshops as the key component of the training intervention with the pre‐ and posttest OSTE as a measure of their effectiveness. However, we found in faculty feedback that the OSTEs were actually a key adjuvant, to the workshop training in that they provided a major source of feedback and learning opportunities in addition to their inherent evaluative qualities.

Each of 4 workshops was designed to teach participants the behaviors assessed in the 4 OSTE stations. The pretest OSTE provided a baseline for participant performance and served to activate the participants to focus on key teaching behaviors during the workshop. During the workshop following the pretest OSTE, participants were given copies of the rating scales and feedback on the performance of the group as a whole on each rated behavior. The evidence used to create the rating instruments was presented and participants had the opportunity to debate and agree on the instrument's construct validity. They then had the opportunity to engage in deliberate practice during role plays depicting challenges to orienting a learner, providing feedback, and to family centered rounds. The posttest OSTE served as summative evaluation of the participants' ability to perform the practiced behaviors effectively in a simulated teaching environment.

We chose to focus the FCR scenarios on correcting mistakes in clinical reasoning for a new patient and on teaching key parts of the physical exam during rounds for an established patient. Errors in clinical reasoning lead to misdirected patient management and are the number 1 cause of medical errors.19, 20 Bedside rounds are a perfect venue for reinforcing and fine‐tuning diagnostic reasoning because all the crucial sources of data are present: the patient, the parent, the nurse, and the computer with lab and imaging results. Faculty members and trainees have both expressed discomfort at correcting errors in clinical reasoning in front of families, leading to missed learning opportunities.21

During the workshop on clinical reasoning, we taught faculty how to use the Problem Representation method to analyze and correct errors in clinical reasoning. The method, studied by Bordage and associates22 forces learners to identify the key features of a presentation and relate their interpretation of the findings by using semantic qualifiers. We trained faculty to deliberately listen for the learner's interpretation of the key features to determine how a misdiagnosis occurred. They were also trained to walk trainees back through their thought process in an objective way, correcting the misinterpretation of data, so that the trainee's competence is not compromised in the eyes of the team or the parents. Teaching the trainee to think correctly about a clinical problem benefits the other members of the team, as well as providing the parents with a better understanding of the rationale for the management plan.

Correct interpretation of the physical exam findings is crucial to making the correct diagnosis. However, there have been several articles chronicling the lost art of eliciting and interpreting physical exam findings, ranging from the cardiac exam to neurological exam.23 A minority of physical exam teaching occurs at the patient's bedside, partly attributed to faculty members discomfort with this type of teaching.24, 25 To enhance the comfort of our faculty members, we included behaviors referenced in articles on teaching a skill,26 activated demonstration,10 and effective bedside teaching27 to guide faculty to incorporate eliciting and interpreting focused aspects of the physical exam during rounds.

Our FCR evaluation templates awarded the highest scores if the hospitalist encouraged senior residents to model clinical reasoning or physical exam skills for junior learners. Hospitalists' presence, especially on work rounds, can diminish the senior resident's opportunity to gain experience and confidence in leading the team.28 We therefore explicitly directed hospitalists to promote the role of the senior resident as the team leader during workshops, while priming faculty to assume the role of educational coach.29

We hypothesize that several factors contributed to the success of the OSTE workshopOSTE intervention. First, faculty members willingly volunteered to participate because they recognized gaps in their own knowledge and skill at leading FCR. They found the ability to deliberately practice the desired behaviors in the OSTE exercises to be the most useful part of the exercise, because the scenarios were authentic and SLs were real trainees.

Although we included all the junior faculty members of our large Pediatric Hospitalist Division in our study, our sample size is still small; limiting our ability to generalize our findings. We found the scheduling of 14 hospitalists to attend 4 different events in close succession to be problematic. Conducting the OSTE sessions at the GW CLASS center 5 miles away from our hospital was also logistically challenging.

We plan to simplify the logistics so that we can incorporate this model in the training of new hospitalists in the division. We still plan to use preintervention FCR OSTEs, but instead of workshops, will provide background information by means of self‐directed Web‐based modules. We will also videotape the OSTEs and provide faculty with a template to rate their own performance and then compare it with ratings from SLs. This individualized feedback and self‐reflection could result in better performance30 than the summary group feedback we gave during the workshops.

Another limitation of this study is the lack of data regarding the consistency of our faculty participants' performance in real FCR. Finally, we did not study the impact of the desired behaviors on patient, trainee, or nursing satisfaction, learning, or efficiency.

Conclusion

In conclusion, we found incorporating OSTEs into a faculty development program to improve FCR to be an effective strategy for changing faculty behavior in leading FCR. Additional study is needed to determine if replacing the workshops with Web‐based tutorials is equally effective and to determine if this faculty development strategy results in long‐term consistent practice in conducting rounds in real inpatient settings.

References

Online-Only Materials

   Comments ()