Original Research

Creating a better discharge summary: Improvement in quality and timeliness using an electronic discharge summary

Abstract

Abstract

BACKGROUND:

Deficits in information transfer between inpatient and outpatient physicians are common and potentially dangerous.

OBJECTIVE:

To evaluate the effect of a newly‐created electronic discharge summary.

DESIGN AND PARTICIPANTS:

Pre‐post evaluation of discharge summaries using a survey of outpatient physicians and a medical records review.

MEASUREMENTS:

Outpatient physicians' ratings of satisfaction with discharge summaries before and after implementation of an electronic discharge summary using a 5‐point Likert scale (1 = very dissatisfied; 5 = very satisfied). Additionally, 196 randomly selected discharge summaries before and after implementation were rated for timeliness and presence of 16 key content areas by 3 internists.

RESULTS:

Two hundred and twenty‐six of 416 (54%) and 256 of 397 (64%) outpatient physicians completed the baseline and postimplementation surveys. Satisfaction with quality and timeliness of discharge summaries improved with the use of the electronic discharge summary (mean quality rating 3.04 versus 3.64; P < 0.001, mean timeliness rating 2.59 versus 3.34; P < 0.001). A higher percentage of electronic discharge summaries were completed within 3 days of discharge as compared with dictated discharge summaries (44.8% versus 74.1%; P < 0.001). Several elements of the discharge summary were present more often with the electronic discharge summary, including discussion of follow‐up issues (52.0% versus 75.8%; P = 0.001), pending test results (13.9% versus 46.3%; P < 0.001), and information provided to the patient and/or family (85.1% versus 95.8%; P = 0.01).

CONCLUSIONS:

The use of an electronic discharge summary significantly improved the quality and timeliness of discharge summaries. Journal of Hospital Medicine 2009;4:219–225. © 2009 Society of Hospital Medicine.

Copyright © 2009 Society of Hospital Medicine

Preventable or ameliorable adverse events have been reported to occur in 12% of patients in the period immediately following hospital discharge.1, 2 A potential contributor to this is the inadequate transfer of clinical information at hospital discharge. The discharge summary comprises a vital component of the information transfer between the inpatient and outpatient settings. Unfortunately, discharge summaries are often unavailable at the time of follow‐up care and often lack important content.37

A growing number of hospitals are implementing electronic medical records (EMR). This creates the opportunity to standardize the content of clinical documentation and creates the potential to assemble, immediately at the time of hospital discharge, major components of a discharge summary. With enhanced communication systems, this information can be delivered in a variety of ways with minimal delay. Previously, we reported the results of a survey of medicine faculty at an urban academic medical center evaluating the timeliness and quality of discharge summaries, the perceived incidence of preventable adverse events related to suboptimal information transfer at discharge, and a needs assessment for an electronically generated discharge summary that we planned to design.8 We now report the results of the follow‐up survey of outpatient physicians and an evaluation of the quality and timeliness of the electronic discharge summary we created.

Materials and Methods

Design

We conducted a pre‐post evaluation of the quality and timeliness of discharge summaries. In the initial phase of the study, we convened an advisory board comprised of 16 Department of Medicine physicians. The advisory board gave input on needs assessment and helped to create a survey to be administered to all medicine faculty with an outpatient practice. All respondents who had at least 1 patient admitted to the hospital within the 6 months prior to the survey were eligible. The results of the initial survey were reviewed with the advisory board and an electronic discharge summary was created with their input. To evaluate its impact, we conducted a repeat survey of all medicine faculty with an outpatient practice approximately 1 year after implementation of the electronic discharge summary.

To complement data received from the outpatient physician survey, a randomly selected sample of discharge summaries from general medical services during the same 3 month period before and after implementation of the electronic discharge summary were rated by 1 of 3 board‐certified internists (D.B.E., N.K., or M.P.L.).

Setting and Participants

The study was conducted at Northwestern Memorial Hospital, a 753‐bed hospital in Chicago, IL. The study was approved by the Institutional Review Board of the Northwestern University Feinberg School of Medicine. General medical patients were admitted to 1 of 2 primary physician services during the study period: a teaching service or a nonteaching hospitalist service. Discharge summaries had traditionally been dictated by inpatient physicians and delivered to outpatient physicians by both mail and facsimile via the medical record department. A recommended template for dictated discharge summaries was provided in the paper paging directory distributed yearly to inpatient physicians.

The hospital implemented an EMR and computerized physician order entry (CPOE) system (PowerChart Millennium; Cerner Corporation, Kansas City, MO) in August 2004. Although all history and physicals and progress notes were documented in the EMR, the system did not provide a method for delivering discharge summaries performed within the EMR to outpatient physician offices. Because of this, inpatient physicians were instructed to continue to dictate discharge summaries during the initial phase of the study.

Approximately 65% of outpatient physicians at the study site used an EMR in their offices during the study. Approximately 10% of outpatient physicians used the same EMR the hospital uses, while approximately 55% used a different EMR (EPIC Hyperspace; EPIC Systems Corporation, Verona, WI). The remaining physicians did not use an EMR in their offices.

Intervention: The Electronic Discharge Summary

A draft electronic discharge summary template was created by including elements ranked as highly important by outpatient physicians in our initial survey8 and elements required by The Joint Commission.9 The draft electronic discharge summary template was reviewed by the advisory board and modifications were made with their input. We automated the insertion of specific patient data elements, such as listed allergies and home medications, into the discharge summary template. We also created an electronic reminder system to inpatient physicians for summaries not completed 24 hours after discharge.

Because the majority of physicians in our initial survey preferred discharge summaries to be delivered either by facsimile or via an EMR, we concentrated our efforts on creating reliable systems for delivery by those routes. We created logic that queried the primary care physician field within the EMR at the time the discharge summary was electronically signed. An automated process then sent the discharge summary via electronic fax to the physician listed in the primary care physician field. Because a large number of outpatient physicians used an EMR different from the hospital's, we also created a process that sent discharge summaries from the hospital EMR into patient charts within this separate EMR.

The draft electronic discharge summary template was available for use in the EMR beginning in July 2005. The final electronic discharge summary, including automated content, physician reminder for incomplete summaries, and delivery systems as described above was implemented in June 2006. Upon implementation, inpatient physicians were instructed via email announcements and group meetings to begin completing electronic discharge summaries using the EMR. Beyond these announcements, inpatient physicians did not receive any specific training with regard to the new discharge summary process. An example of the final electronic discharge summary product is available in the Appendix.

Outpatient Physician Survey

Satisfaction with timeliness and quality of discharge summaries was assessed using a 5‐point Likert scale, where 5 represented very satisfied and 1 represented very dissatisfied. We also asked respondents to estimate the number of their patients who had sustained a preventable adverse event or near miss related to suboptimal transfer of information at discharge. We defined a preventable adverse event as a preventable medical problem or worsening of an existing problem and near miss as an error that did not result in patient harm but easily could have.

The preimplementation survey, accompanied by a cover letter signed by the hospital's chief of staff, was sent out in March 2005. A postcard reminder was sent approximately 2 weeks after the initial mail survey. A second survey was sent to nonresponders 6 weeks after the initial survey. Simultaneously, the survey was also sent in web‐based format to nonresponders via email. The postimplementation survey was sent out in February 2007 using a similar survey process.

Discharge Summary Review

A random sample of discharge summaries completed before and after the implementation of the electronic discharge summary was selected for review. The sample universe consisted of all general medicine service discharges between August and November 2005, before the electronic discharge summary was implemented, and August to November 2006, after implementation. To provide a balanced comparison, the sample was further limited to only the first chronological (index) discharge of a unique patient to home self‐care or home health nursing, with length of stay between 3 and 14 days. A total of 2232 discharges in 2005 and 2570 discharges in 2006 met these criteria. The discharge summary review sample was designed to randomly select approximately 100 discharge summaries meeting the criteria above within each study year, to produce an approximate 200‐record analysis sample. Each of the 3 physician reviewers was assigned to complete an approximately equal number of the 200 primary reviews.

Physician reviewers recorded whether the discharge summary was dictated versus done electronically, the length of the discharge summary (in words), the number of days from discharge to discharge summary completion, the type of service the patient was discharged from, and the author type (medical student, intern, resident, or attending). Physicians reviewers also assessed the overall clarity of discharge summaries using a 5‐point ordinal scale (1 = unintelligible; 2 = hard to read; 3 = neutral; 4 = understandable; and 5 = lucid).

Prior studies have evaluated the quality of discharge summaries using scoring tools created by the investigators.10, 11 We created our own discharge summary scoring tool based on these prior studies, recommendations from the literature,12 and the findings from our initial survey.8 We pilot‐tested the scoring tool and made minor revisions prior to the study. The final scoring tool consisted of 16 essential elements. Reviewers assessed whether each of the 16 essential elements was present, absent, or not applicable. A Discharge Summary Completeness Score was calculated by the number of the 16 essential elements that were rated as present divided by the number of applicable elements for each discharge summary, multiplied by 100 to produce a completeness percentage.

To assess interrater reliability, reviewers were assigned to independently complete second, duplicate reviews of approximately 90 summaries (30 per reviewer). The duplicate review sample was designed to produce approximately 45 paired re‐reviews in each year for reliability assessment. A final sample of 196 available summaries was completed for the main analysis and 174 primary and duplicate reviews were used to establish interrater reliability across 87 reviewer pairs.

Data Analysis

Physician characteristics, including specialty, faculty appointment type, and year of medical school graduation were provided by the hospital's medical staff office. Physician characteristics from before and after the implementation of the electronic discharge summary were compared using chi‐square tests. Likert scale ratings of physician satisfaction with the timeliness and quality of discharge summaries were compared using t‐tests. The proportion of physicians reporting 1 or more preventable adverse event or near miss before the implementation of the electronic discharge summary was compared to postimplementation proportions using chi‐square tests. In addition, we performed multivariate logistic regression to examine the likelihood of physicians reporting any preventable adverse event or near miss related to suboptimal information transfer. The regression models tested the likelihood of 1 or more preventable adverse event or near miss before versus after the implementation of the electronic discharge summary, controlling for physician characteristics and their number of hospitalized patients in the previous 6 months.

The proportions of discharge summary elements found to be present, the proportion of discharge summaries completed within 3 days, and discharge summary readability ratings before and after the implementation of the electronic discharge summary were compared using chi‐square tests; length in words was compared using t‐tests. Preimplementation and postimplementation Discharge Summary Completeness Scores were compared using the Mann‐Whitney U test. Discharge summary score interrater reliability was assessed using the Brennan‐Prediger Kappa for individual elements.13

Results

Outpatient Physician Survey

Physician Characteristics

Two hundred and twenty‐six of 416 (54%) eligible outpatient physicians completed the baseline survey and 256 of 397 (64%) completed the postimplementation survey. As shown in Table 1, there were no significant differences in specialty, faculty appointment type, or number of patients hospitalized between respondents to the survey before compared to respondents after the implementation of the electronic discharge summary. The number of respondents graduating medical school in 1990 or later was higher after implementation of the electronic discharge summary; however, this result was of borderline statistical significance.

Characteristics of Respondents to Outpatient Physician Discharge Summary Satisfaction Surveys
Preelectronic Discharge Summary (n = 226)Postelectronic Discharge Summary (n = 256)P Value
  • Excludes 5 respondents with missing information on graduation year.

  • Excludes 91 respondents with missing data about the number of their hospitalized patients.

Practice Type 0.23
Generalist, n (%)127 (56.2)130 (50.8)
Specialist, n (%)99 (43.8)126 (49.2)
Faculty Appointment 0.38
Full‐time, n (%)104 (46.0)128 (50.0)
Affiliated, n (%)122 (54.0)128 (50.0)
Year of medical school graduation* 0.06
Before 1990, n (%)128 (57.4)124 (48.8)
1990 or later, n (%)95 (42.6)130 (51.2)
Number of patients hospitalized (last 6 months) 0.56
1‐4, n (%)15 (7.9)24 (12.0)
5‐10, n (%)62 (32.5)66 (33.0)
11‐19, n (%)35 (18.3)33 (16.5)
20 or more, n (%)79 (41.4)77 (38.5)

Timeliness and Content

Changes in outpatient physician satisfaction with the timeliness and quality of discharge summaries are summarized in Table 2. Satisfaction with the timeliness and quality of discharge summarizes improved significantly after the implementation of the electronic discharge summary (mean standard deviation [SD] timeliness rating, 2.59 1.02 versus 3.34 1.09; P < 0.001, mean quality rating 3.04 0.93 versus 3.64 0.99; P < 0.001).

Outpatient Physician Satisfaction with Timeliness, Quality and Communication
Likert Scale Mean Score (SD)*
Preelectronic Discharge SummaryPostelectronic Discharge SummaryP Value
  • Outpatient physicians rated items using a 5‐point scale (1 = very dissatisfied; 2 = dissatisfied; 3 = somewhat satisfied; 4 = satisfied; and 5 = very satisfied). There were 14 and 19 missing values for each item, respectively.

Timeliness of the discharge summary2.59 (1.02)3.34 (1.09)<0.001
Quality of the discharge summary3.04 (0.93)3.64 (0.99)<0.001

Medical Error

The effect of the electronic discharge summary on perceived near misses and preventable adverse events is summarized in Table 3. Fewer outpatient physicians felt that 1 or more of their patients hospitalized in the preceding 6 months sustained a near miss due to suboptimal transfer of information after the implementation of the electronic discharge summary (65.7% vs. 52.9%, P = 0.008). Similarly, fewer outpatient physicians felt that 1 or more of their patients hospitalized in the preceding 6 months sustained a preventable adverse event due to suboptimal transfer of information after the implementation of the electronic discharge summary (40.7% vs. 30.2%, P = 0.02). In multivariate logistic regression analyses controlling for physician characteristics and their number of hospitalized patients in the previous 6 months, there was a statistically significant 40% reduction in the odds of a reported near miss (adjusted odds ratio [OR] = 0.60, P = 0.02). Although not quite statistically significant, there was a 33% reduction in the odds of a reported preventable adverse event (OR = 0.67, P = 0.08) after the implementation of the electronic discharge summary.

Reduction in Outpatient Physician Perception of Errors Related to Suboptimal Transfer of Information at Hospital Discharge
Preelectronic Discharge SummaryPostelectronic Discharge SummaryP Value
  • Defined as an error that did not result in patient harm but easily could have. There were 23 missing responses.

  • Defined as a preventable medical problem or worsening of an existing problem. There were 22 missing responses.

Near miss*
Number (%) reporting 1142 (65.7)108 (52.9)
Crude odds ratioRef.0.570.008
Adjusted odds ratioRef.0.600.02
Preventable adverse event
Number (%) reporting 188 (40.7)62 (30.2)
Crude odds ratioRef.0.630.03
Adjusted odds ratioRef.0.670.08

Discharge Summary Review

Discharge Summary Characteristics

One hundred and one discharge summaries before implementation of the electronic discharge summary were compared to 95 discharge summaries produced the following year. Characteristics of discharge summaries before and after the implementation of the electronic discharge summary are summarized in Table 4. A large number of discharge summaries (52.5%) were already being typed into the EMR in 2005, prior to the implementation of our final electronic discharge summary product. The number of dictated discharge summaries decreased from 47.5% to 10.5% after implementation of the final electronic discharge summary product (P < 0.001). Discharge summaries were similar in length before and after the implementation of the electronic discharge summary. A higher percentage of discharge summaries were completed within 3 days of discharge after implementation of the electronic discharge summary; however, this result was of borderline statistical significance (59.4% vs. 72.6%; P = 0.05). The type of service from which patients were discharged and the distribution of author types were similar after the implementation of the electronic discharge summary.

Characteristics of Discharge Summaries
Number (%) or MeanSDP Value
Preelectronic Discharge Summary (n = 101)Postelectronic Discharge Summary (n = 95)
Dictated, n (%)48 (47.5)10 (10.5)<0.001
Length in words, mean SD785 407830 3890.43
Completed within 3 days, n (%)60 (59.4)69 (72.6)0.05
Type of service, n (%) 0.29
Teaching service63 (62.4)66 (69.5)
Nonteaching hospitalist service38 (37.6)29 (30.5)
Author type, n (%) 0.62
Fourth year medical student13 (12.9)13 (13.7)
Intern31 (30.7)37 (38.9)
Resident19 (18.8)15 (15.8)
Attending38 (37.6)30 (31.6)

Because a large percentage of discharge summaries were already being done electronically in 2005, we evaluated the timeliness of dictated discharge summaries compared to electronic discharge summaries across both periods combined (preimplementation and postimplementation of the electronic discharge summary). A higher percentage of electronic discharge summaries were completed within 3 days of discharge as compared to dictated discharge summaries (44.8% versus 74.1%; P < 0.001).

Discharge Summary Completeness Score

The presence or absence of discharge summary elements before and after the implementation of the electronic discharge summary is shown in Table 5. Several elements of the discharge summary were present more often after the implementation of the electronic discharge summary. Specific improvements included discussion of follow‐up issues (52.0% versus 75.8%; P = 0.001, = 0.78), pending test results (13.9% vs. 46.3%; P < 0.001, = 0.92), and information provided to the patient and/or family (85.1% vs. 95.8%; P = 0.01, = 0.91). Significant laboratory findings were present less often after implementation of the electronic discharge summary (66.0% versus 51.1%; P = 0.04, = 0.84). The Discharge Summary Completeness Score was higher after the implementation of the electronic discharge summary (mean 74.1 versus 80.3, P = 0.007). Dictated discharge summaries had a significantly lower Discharge Summary Completeness Score compared to discharge summaries done electronically (71.3 vs. 79.6, P = 0.002) across both periods combined.

Improved Likelihood of Pertinent Content Items Present in Discharge Summary
Number (%) of Content Items Present*P ValueBrennan‐Prediger Kappa
Preelectronic Discharge Summary (n = 101)Postelectronic Discharge Summary (n = 95)
  • n is less for certain elements as information was not applicable.

Dates of admission and discharge96 (95.0)94 (98.9)0.111.0
Reason for hospitalization100 (99.0)94 (100)0.331.0
Significant findings from history and exam78 (77.2)65 (68.4)0.160.26
Significant laboratory findings64 (66.0)47 (51.1)0.040.84
Significant radiological findings67 (75.3)71 (81.6)0.310.89
Significant findings from other tests41 (63.1)40 (71.4)0.330.88
List of procedures performed45 (81.8)35 (77.8)0.770.99
Procedure report findings49 (80.3)43 (78.2)0.610.92
Stress test report findings7 (100)3 (100)N/A1.0
Pathology report findings11 (39.3)3 (30.0)0.600.91
Discharge diagnosis89 (88.1)86 (93.5)0.200.86
Condition at discharge81 (81.0)80 (85.1)0.450.76
Discharge medications88 (87.1)88 (93.6)0.130.79
Follow‐up issues52 (52.0)72 (75.8)0.0010.78
Pending test results14 (13.9)44 (46.3)<0.0010.92
Information provided to patient and/or family, as appropriate86 (85.1)91 (95.8)0.010.91
Discharge Summary Completeness Score (percent present all applicable items)74.180.30.007

Significantly more discharge summaries were rated as understandable or lucid after the implementation of the electronic discharge summary (41.6% vs. 59.0%; P = 0.02). In both periods combined, dictated discharge summaries were rated as understandable or lucid less often than electronic discharge summaries (34.5% vs. 56.5%; P < 0.001).

Discussion

Our study found that an electronic discharge summary was well accepted by inpatient physicians and significantly improved the quality and timeliness of discharge summaries. Prior studies have shown that the use of electronically entered discharge summaries improved the timeliness of discharge summaries.1416 However, the discharge summaries used in these studies required manual input of data into a computer system separate from the patient's medical record. To our knowledge, this is the first study to report the impact of discharge summaries generated from an EMR. Leveraging the EMR, we were able to automate the insertion of specific patient data elements, streamline delivery, and create an electronic reminder system to inpatient physicians for summaries not completed 24 hours after discharge.

Prior research has shown that the quality of discharges summaries is improved with the use of standardized content.10, 17 Using a standardized template for the electronic discharge summary, we likewise demonstrated improved quality of discharge summaries. Key discharge summary elements, specifically discussion of follow‐up issues, pending test results, and information provided to the patient and/or family, were present more reliably after the implementation of the electronic discharge summary. The importance of identifying pending test results is underscored by a recent study showing that many patients are discharged from hospitals with test results still pending, and that physicians are often unaware when results are abnormal.18 One discharge summary element, significant laboratory findings, was present less often after the implementation of the electronic discharge summary. Our template did not designate significant laboratory findings under a separate heading. Instead, we used a heading entitled Key Results (labs, imaging, pathology). Physicians completing the discharge summaries may have prioritized the report of imaging and pathology results in this section. A simple revision of our discharge summary template to include a separate heading for significant laboratory findings may result in improvement in this regard.

Timeliness of discharge summaries was improved in our study, but remained less than optimal. Although nearly three‐quarters of electronic discharge summaries were completed within 3 days of discharge, our ultimate goal is to have 100% of discharge summaries completed within 3 days. This is especially important for complicated patients requiring outpatient follow‐up soon after discharge. We are currently in the process of designing further modifications to the electronic discharge summary completion process. One modification that may be beneficial is the automation of additional patient specific data elements into the discharge summary. We also plan to link performance of medication reconciliation, completion of patient discharge instructions, and completion of the discharge summary into an integrated set of activities performed in the EMR prior to patient discharge.

We found that fewer outpatient physicians reported 1 or more of their patients having a preventable adverse event or near miss as a result of suboptimal transfer of information at discharge after the implementation of the electronic discharge summary. Although we did not measure preventable adverse events directly in our study, this is an important finding in light of the large number of patients who sustain preventable adverse events after hospital discharge1, 2 and prior research showing that the absence of discharge summaries at postdischarge follow‐up visits increased the risk for hospital readmission.19

We had wondered what effect the electronic discharge summary would have on the length and clarity of discharge summaries. A published commentary suggested that notes performed in EMRs were inordinately long and often difficult to read.20 We were pleased to discover that electronic discharge summaries were similar in length to previous discharge summaries and were rated higher with regard to clarity.

Our study has several limitations. First, many inpatient physicians began to use electronic discharge summaries prior to our creation of the final electronic discharge summary product. We had explicitly instructed physicians to continue to dictate discharge summaries in the first phase of our study. The fact that physicians quickly adopted the practice of completing discharge summaries electronically suggests that they preferred this method for completion and may help to explain the improvement in timeliness. A second limitation, as previously mentioned, is that our study did not measure adverse events directly. Instead, we asked outpatient physicians to estimate the number of their patients discharged in the last 6 months who had sustained a preventable adverse event or near miss related to suboptimal information transfer at discharge. We had limited space in the survey to define the meaning of a preventable adverse event; therefore, the description in the survey does not exactly match previous definitions.1, 2 Finally, the ordinal scale used to assess clarity of discharge summaries has not been previously validated.

In conclusion, the use of an electronic discharge summary significantly improved the quality and timeliness of discharge summaries. The discharge summary comprises a vital component of the information transfer between the inpatient and outpatient settings during the vulnerable period following hospital discharge. As hospitals expand their use of EMRs, they should take advantage of opportunities to leverage functionality to improve quality and timeliness of discharge summaries.

References

Online-Only Materials

AttachmentSize
PDF icon Final Discharge Summary Product (1)94.96 KB
   Comments ()