Original Research

Effect of a CD‐ROM‐based educational intervention on resident knowledge and adherence to deep venous thrombosis prophylaxis guidelines

Abstract

Abstract

BACKGROUND

CD‐ROM‐based educational methods are not new to residency training, yet little is known about how they affect resident knowledge and patient care practices.

OBJECTIVE

We evaluated the effects of a CD‐ROM‐based educational tool on residents' knowledge of anticoagulation and their adherence to anticoagulation guidelines.

DESIGN, SETTING AND PARTICIPANTS

Residents in the departments of cardiothoracic surgery, emergency medicine, otolaryngology, internal medicine, neurosurgery, dental medicine, neurology, obstetrics and gynecology, orthopedics, surgery, and urology at a university hospital participated in the study.

INTERVENTION

Residents were provided with CD‐ROM‐based training on the proper use of anticoagulation based on the sixth ACCP guidelines for antithrombotic therapy. Multiple choice testing was carried out before and after the CD‐ROM intervention to assess resident knowledge, and resident compliance with venous thromboembolism prophylaxis guidelines was assessed via inpatient chart review by an independent committee.

MAIN OUTCOME MEASURES

Changes in knowledge were measured via test scores and the rate of compliance with anticoagulation guidelines.

RESULTS

Multiple choice test scores of 117 residents increased significantly after reviewing the CD‐ROM (from 46.7% ± 15.1% to 77.8% ± 15.1%, P < .005). As a control, we administered the same test at the same 2 times at a comparable institution but without the CD‐ROM intervention and found that test scores did not significantly increase. Chart review revealed a rate of compliance with anticoagulation guidelines for prophylaxis of venous thromboembolism of 75% before the CD‐ROM intervention, which increased to 95% after the intervention, an increase that was sustained for at least 7 months.

CONCLUSIONS

These findings suggest that CD‐ROM‐based interventions might be useful not only in enhancing resident knowledge but also in improving the quality of care by favorably affecting clinical practice. Journal of Hospital Medicine 2008;3:42–47. © 2008 Society of Hospital Medicine.

Copyright © 2008 Society of Hospital Medicine

Given recent changes in the goals and objectives of residency training as well as changes in the functioning of teaching hospitals, traditional educational formats may need to be supplemented or replaced.1 The Accreditation Council for Graduate Medical Education (ACGME) is promoting changes in resident education with the goal of not only enhancing trainee competency using innovative methods but also of demonstrating that these educational innovations result in enhanced quality of patient care and improved patient safety.1 A challenging aspect of these initiatives is that programs are working to implement them at a time when there are greater nonteaching demands on faculty time, mandated resident work‐hour limitations have been instituted, in some states by law, and resident patient care and educational activities are prone to disruptions inherent in caring for patients in a complex health system. Various solutions have been proposed including increased incorporation of self‐directed learning as a means of meeting modern resident educational challenges, yet the ideal tools with which to accomplish this are unknown.

Computer‐based instruction in medicine has been available since the 1960s, and although its use had initially been more widespread in medical student, nursing, allied health professional, and patient education,2, 3 it is being increasingly incorporated into resident education as well. Some studies have shown that for medical students, computer‐based teaching is at least as effective in improving knowledge as conventional lectures4 and that learners' satisfaction with computer‐based formats appears comparable with that of traditional didactic lectures.5 In recent years computer‐based teaching has been applied to resident education in various fields including surgery and surgical subspecialties, pediatrics, and obstetrics and gynecology.612 Little is known, however, about how computer‐based educational methods affect resident knowledge and especially how these methods might affect clinical practice.

Venous thromboembolism (VTE) is a common and hazardous complication of acute inpatient hospitalization.13 Recognizing that errors in proper prescribing and monitoring of anticoagulants are a major cause of acute inpatient morbidity and mortality,14 we began an initiative to educate our residents and improve their patient care practices regarding the proper use of anticoagulants. To accomplish this, we developed a CD‐ROM‐based learning module with the aim of increasing resident knowledge of anticoagulation as well as compliance with national standards for VTE prevention. In this study we assessed the impact of the CD‐ROM intervention on resident knowledge and their appropriate use of VTE prophylaxis.

METHODS

The study was approved by the institutional review board. With the participation of faculty educators in the departments of medicine, surgery, and neurology, one of the authors (H.K.) coordinated the development of a CD‐ROM containing concise modules on core topics in anticoagulation (Table 1). The presenters for these topics included the director of clinical hematology, 2 cardiologists including the director of the coronary care unit, the director of the medical intensive care unit, the director of cerebrovascular diseases, and 2 vascular surgeons, one of whom serves as vice chair of surgery. These modules, each lasting about 1 hour, had audio and slide components detailing the proper indications, monitoring, and efficacy of anticoagulants in atrial fibrillation, acute ischemic stroke, acute coronary syndromes, and VTE prevention in acutely ill hospitalized patients. The guidelines presented were based on the sixth (2000) ACCP guidelines for antithrombotic therapy for the prevention and treatment of thrombosis.15 The content of the CD‐ROM was reviewed for accuracy by the authors, though none of them were speakers. We asked that before all current residents in the departments of cardiothoracic surgery, emergency medicine, otolaryngology, internal medicine, neurosurgery, dental medicine, neurology, obstetrics and gynecology, orthopedics, surgery, and urology viewed the CD‐ROM, they complete a pretest to determine their baseline knowledge of this subject. After completing the pretest, the residents were required to view the CD‐ROM and retake the same test. We then compared pre‐ and posttest scores.

Anticoagulation Topics Covered in CD‐ROM
Overview of anticoagulation
Venous thromboembolism
Atrial fibrillation
Unfractionated heparin in acute coronary syndrome
Treatment of thromboembolic events with intravenous heparin
Anticoagulants in the management of patients with acute ischemic stroke
Deep venous thrombosis prophylaxis

To determine whether an increase in knowledge was secondary to the CD‐ROM intervention or simply a consequence of acquired clinical experience during training, we compared test scores of residents who did and did not receive the CD‐ROM intervention. In the academic year following our initial testing, we asked the incoming categorical medical PGY‐1 classes at our hospital and at a comparable local tertiary‐care hospital in our health system 2 miles away to take an anticoagulation pretest (different from the examination given for the initial testing) during their PGY‐1 orientations. The 2 institutions are comparable in many ways including in patient demographics and size and most residents come from the same medical schools, have a similar rotation structure, and use a comparable curriculum under a unified graduate medical education office. The CD‐ROMs were only given to categorical PGY‐1 residents at our institution. Both groups then retook the same test (posttest) 3 months into their clinical training, a time chosen because it is when all PGY‐1s would be expected to have gained significant clinical experience on the medical wards and or in the intensive care units. The exam questions were generated by one of the authors (B.M.) and covered all the topics in the CD‐ROM.

An Anticoagulation Steering Committee was formed to assess whether the CD‐ROM intervention affected our residents' patient care practices. None of the members of this committee were authors of this work. Members of this committee reviewed inpatient charts and documented resident compliance with VTE prevention standards during periods before and after they had viewed the CD‐ROMs. We chose this particular portion of the CD‐ROM because at both the test and control hospitals, initiatives were underway using order sets to improve anticoagulation in cardiac, neurological, and surgical patients but not in VTE prophylaxis. Charts from the same 2 nursing units on the medical service were reviewed in each period and included patients with a discharge diagnosis of congestive heart failure, any oncologic diagnosis, or sepsis. The chart review tool was developed by the anticoagulation committee and included a thrombosis risk factor assessment section as well as a list of contraindications to anticoagulation to determine if anticoagulation was appropriately implemented. Charts were reviewed for compliance with VTE prophylaxis after the CD‐ROM intervention (given in July 2004) in August 2004. To have a comparable pre‐CD‐ROM comparison, charts of patients with the diagnoses stated above were reviewed from August of the preceding year. The same month was chosen in the previous year to minimize any impact of resident experience, which would likely be a confounding factor if charts from May or June of the academic year were used as a control, for example. To determine whether an improvement in adherence to VTE prophylaxis standards was sustained, an additional chart review was carried out 7 months after the initial CD‐ROM viewing. The same group of observers, none of whom were authors, did all the chart assessments.

Statistics

Continuous variables are reported as means SDs. Comparisons of test scores before and after the CD‐ROM intervention were carried out using paired t testing. Comparisons of pre‐ and posttest scores between both institutions were carried out using analysis of variance with Tukey‐Kramer multiple‐comparisons testing (GraphPad InStat Statistical Software, version 3.01, GraphPad Software, Inc.). We calculated that 13 residents would need to be tested in order to have a statistical power of 80% to detect a 25% increase in test scores with a type I error of 0.05. Comparisons of the proportions of patients who received appropriate VTE prophylaxis were carried out using chi‐square testing. Statistical significance was defined as a 2‐tailed P value less than 0.05.

RESULTS

Overall and Departmental Resident Test Results

One hundred and seventeen residents from all departments participated in the project including taking the pre‐ and posttests. The response rate was 44% overall and ranged from 10% to 100% for individual departments. For all residents combined, there was a statistically significant increase in scores (pretest 46.7% 15.1%, posttest 77.8% 15.1%, P < .005). Overall scores and those for individual departments are summarized in Table 2. As can be seen, there was a significant increase in test results for each department. The only exception was a department that already had a high baseline score and that had only 4 residents, limiting the power of statistical analysis. These findings suggest that the CD‐ROM intervention favorably affected resident knowledge of anticoagulation across all medical specialties tested.

Resident Pre‐ and Posttest Scores by Department
DepartmentnPrescorePostscoreP value*
  • NA, not applicable.

  • Data are reported as means SDs. Comparisons of test scores before and after the CD‐ROM intervention were carried out using paired t testing.

Cardiothoracic surgery17283NA
Dentistry2234.9 10.372.3 12.4< .0001
Surgery1952.6 14.577.1 14.5< .0001
Medicine2154.3 11.684.0 8.9< .0001
Emergency medicine461.3 4.594.3 8.0< .05
Otolaryngology548.8 5.080.0 11.6< .01
Urology466.5 23.684.5 15.80.15
Neurology1042.1 11.568.8 18.9< .01
Orthopedics1243.4 16.070.4 24.1< .01
Obstetrics/gynecology1941.4 13.181.8 11.0< .0001
ALL11746.7 15.177.8 15.1< .005

Assessment of Independent Effect of CD‐ROM Intervention

To determine what independent effect the CD‐ROM intervention might have, given that scores may improve with the acquisition of clinical experience alone, in July 2004 we tested internal medicine categorical PGY‐1s at our institution and at another tertiary‐care hospital, as described in the Methods section. The results of testing both groups are shown in Figure 1. Nineteen medical PGY‐1s at our hospital (hospital A) completed the anticoagulation pretest, and 16 completed the posttest. Twenty‐two medical PGY‐1s completed the pretest, and 17 completed the posttest at our neighboring hospital (hospital B). Although posttest scores were higher at both institutions, the increase in scores at our institution, which received the CD‐ROM intervention, was statistically significant, whereas the increase for the group not receiving the intervention was not significant. These findings suggest that the CD‐ROM intervention may have had an independent effect on resident knowledge of anticoagulation.

Figure 1

Pre‐ and posttest scores for incoming medical PGY‐1 residents at our institution (hospital A), who received the CD‐ROM intervention, and at a neighboring institution (hospital B), who did not receive the CD‐ROM intervention. Data are expressed as mean ± SD. Comparisons of pre‐ and posttest scores were carried out using analysis of variance with Tukey‐Kramer multiple‐comparisons testing (GraphPad InStat® statistical software, version 3.01, GraphPad Software, Inc., San Diego, CA).

Effect of CD‐ROM Intervention on Resident Use of VTE Prophylaxis

Appropriate use of VTE prophylaxis by residents was assessed at 3 points, as detailed in the Methods section: 1 year before the CD‐ROM intervention (baseline), immediately after the CD‐ROM intervention, and 7 months after the CD‐ROM intervention. VTE prophylaxis, one element of the CD‐ROM, was chosen as a surrogate marker for the impact of the CD‐ROM initiative. A review of 40 charts of patients with the specified diagnoses (100% of the patients with the specified diagnoses, which represented about one third of admissions to the unit) before the CD‐ROM intervention revealed that 30 patients (75%) received appropriate VTE prophylaxis. A review of 38 charts after the CD‐ROM intervention showed that 36 patients (95%) received appropriate prophylaxis; similar findings were obtained 7 months after the CD‐ROM intervention (33 of 35 patients, 94%, P = .0107). These findings, which are shown in Figure 2, suggest that the CD‐ROM intervention enhanced resident compliance with VTE prophylaxis guidelines and that this effect was sustained for at least 7 months.

Figure 2

Percentage of patients who received appropriate venous thromboembolism prophylaxis prior to (baseline, pre‐CD‐ROM), soon after (CD‐ROM), and 7 months after the CD‐ROM intervention. Comparisons of the proportions of patients who received appropriate VTE prophylaxis were carried out using the chi‐square test.

DISCUSSION

Residency training is facing challenges on several fronts. In addition to substantially changed educational requirements, strict limits on the amount of time that trainees can spend in the hospital have resulted from ACGME requirements and several state laws. Residents who are on night rotations or were on call the night before often miss educational conferences or must choose between attending patient carerelated activities and educational sessions. Time constraints on faculty have compounded this problem, and for residents to effectively learn, the focus of graduate medical education may need to shift somewhat from teaching medical information toward teaching the practice of self‐directed learning, with CD‐ROMs one such mode by which this can take place. Accomplishing this will require novel teaching approaches, and residency programs will need to document their effectiveness.

In this study we demonstrated that our residents increased their knowledge and improved their patient care practices using a CD‐ROM‐based educational tool. Residents frequently make use of computer‐based educational resources in the form of journals, textbooks, informational databases such as comprehensive drug listings, and personal digital assistantbased tools. Advantages of the computer‐based learning format include increased accessibility and flexibility in viewing the material. Residents have the option of repeated screening as desired and of viewing the CD‐ROM in segments if necessary. Although residents often must choose to attend a scheduled traditional lecture or engage in a patient carerelated activity, the CD‐ROM format allows the resident to choose the ideal time and setting to engage in structured educational activities. Other advantages of the CD‐ROM format would be ease of monitoring for accuracy, applicability, and comprehensiveness as well as more flexibility in faculty time commitments. It should be noted that we have no information about how much time residents devoted to the CD‐ROM program and how often they may have returned to the module for review. It should also be noted that although there have been some reports suggesting that CD‐ROM‐based education may play a useful role in student and perhaps resident education,1618 there is no evidence to date demonstrating that widespread use of CD‐ROMs in residency training can differentially affect resident behavior compared with the use of traditional methods.

A number of variables could have affected our results. For overall test scores, the response rate was less than 50%, with variability between departments suggesting that perhaps it was more motivated residents who participated and were therefore more likely to demonstrate improvement. Although our data comparing institutions with and without the CD‐ROM intervention suggested that the CD‐ROM intervention had a discernable effect on resident knowledge, we must also consider the possibility that the 2 groups might not have been comparable, as attitudes, expectations, and other variables might have differed. All the residents were categorical trainees, and given the similarities in many aspects of the training programs in these 2 tertiary‐care hospitals, as described in the Methods section, it is hoped that any such differences were minor. Nevertheless, this must be considered a limitation of our study. Also of note, the number of trainees was small, as was the patient population studied with VTE prophylaxis; hence, we recognize that our work can best be regarded as a pilot study using an alternative learning method. We also realize that giving a group of residents a test followed by distribution of a CD‐ROM might have suggested that we were directing them toward a goal, and this may have affected the results. Heightened awareness of the importance of anticoagulation from the introduction of new guidelines and other variables also could have affected our findings. The taking of an examination itself might also have had an impact on knowledge that could affect subsequent test scores. An additional point to consider is that if knowledge and patient care did improve, we do not know whether this affects residents acquiring other knowledge or whether this will translate into improved patient care in other areas.

Although CD‐ROM‐based learning could serve a useful function in the increasingly complex environment of residency training, this learning method also has disadvantages, including not providing personal contact or having the capability of question‐and‐answer sessions between teacher and resident. This could be overcome by providing time for faculty‐precepted question‐and‐answer sessions or perhaps creating a Web‐based venue for questions to be submitted and answered. In addition, the CD‐ROMs themselves can be designed in an interactive format in which residents can provide answers to clinical questions with feedback based on their selections provided as part of the CD‐ROM program.

In summary, the CD‐ROM‐based program in this study appears to have had an effect on not only knowledge but also patient care practice and suggests that this type of format could serve a useful role in residency training. Studies of additional interventions such as this one might allow for more extensive evaluation of the utility of CD‐ROM‐based learning as a residency training tool.

References

   Comments ()