Original Research

Lessons learned from implementation of a computerized application for pending tests at hospital discharge

Abstract

Abstract

BACKGROUND:

Patients are often discharged from the hospital before test results are finalized. Awareness of these results is poor and therefore an important patient safety concern. Few computerized systems have been deployed at care transitions to address this problem. We describe an attempt to implement a computerized application to help inpatient physicians manage these test results.

METHODS:

We modified an ambulatory electronic medical record (EMR)‐based results management application to track pending tests at hospital discharge (Hospitalist Results Manager, HRM). We trained inpatient physicians at 2 academic medical centers to track these tests using this application. We surveyed inpatient physicians regarding usage of and satisfaction with the application, barriers to use, and the characteristics of an ideal system to track pending tests at discharge.

RESULTS:

Of 29 survey respondents, 14 (48%) reported never using HRM, and 13 (45%) used it 1 to 2 times per week. A total of 23 (79%) reported barriers prohibiting use, including being inundated with clinically “irrelevant” results, not having sufficient time, and a lack of integration of post‐discharge test result management into usual workflow. Twenty‐one (72%) wanted to receive notification of abnormal and clinician‐designated pending test results. Twenty‐seven physicians (93%) agreed that an ideally designed computerized application would be valuable for managing pending tests at discharge.

CONCLUSIONS:

Although inpatient physicians would highly value a computerized application to manage pending tests at discharge, the characteristics of an ideal system are unclear and there are important barriers prohibiting adoption and optimal usage of such systems. We outline suggestions for future electronic systems to manage pending tests at discharge. Journal of Hospital Medicine 2011. © 2010 Society of Hospital Medicine.

Copyright © 2010 Society of Hospital Medicine

The period following discharge is a vulnerable time for patientsthe prevalence of medical errors related to this transition is high and has important patient safety and medico‐legal ramifications.13 Factors contributing to this vulnerability include complexity of hospitalized patients, shorter lengths of stay, and increased discontinuity of care. Hospitalists have recognized this threat to patient safety and have worked toward improving information exchange between inpatient and outpatient providers at hospital discharge.46 Nonetheless, the evidence suggests that more work is necessary. A recent study found that discharge summaries are often incomplete, and do not contain important information requiring follow‐up, such as pending tests.7 Additionally, a review by Kripalani et al. characterizing information deficits at hospital discharge found few interventions which specifically improve communication of pending tests at hospital discharge.8

In a prior study we determined that 41% of patients left the hospital before all laboratory and radiology test results were finalized. Of these results, 9.4% were potentially actionable and could have altered management. Physicians were aware of only 38% of post‐discharge test results.9 This awareness gap is a consequence of several factors including the lack of systems to track and alert providers of test results finalized post discharge. Also, it is unclear who is responsible for pending tests at discharge, since these tests are ordered by the inpatient physicians but often reported in the time period between hospital discharge and the patient's first follow‐up appointment with the primary care physician (PCP). Because responsibility is not explicitly made in the final communication between physicians at discharge, such test results may not be reviewed in a timely manner, potentially resulting in delays in treatment, a need for readmission, or other unfavorable outcomes.

Even in integrated health systems with advanced electronic health records, missed test results which result in treatment delays remain prevalent.10, 11 Test result management applications aid clinicians in reviewing and acting upon results as they become available and such systems may provide solutions to this problem. At Partners Healthcare in Boston, the Results Manager (RM) application was developed to help clinicians in the ambulatory setting safely, reliably, and efficiently review and act upon test results. The application enables clinicians to prioritize test results, utilize guidelines, and generate letters to patients. This system also prompts physicians to set reminders for future testing.12 In a 2.5‐year study evaluating the impact of this intervention, PCPs at 26 adult primary care practices were able to expedite communication of outpatient laboratory and imaging test results to patients with the help of RM. Patients of physicians who participated in the project reported greater satisfaction with test result communication and with information provided about their condition than did a control group of similar patients.13 RM has not yet been studied in the inpatient setting or at care transitions. We describe an attempt at modifying the Partners RM application to help inpatient physicians manage pending tests at hospital discharge.

Methods

Study Setting and Participants

We piloted our application at 2 major academic medical centers (hospitals A and B) associated with Partners Healthcare, an integrated regional health delivery network in eastern Massachusetts, from October 2004 to March 2005. Both centers use the longitudinal medical record (LMR), the electronic medical record (EMR), for nearly all ambulatory practices. The LMR is an internally developed full‐featured EMR, including a repository of laboratory and radiology reports, discharge summaries, ambulatory care notes, medication lists, problem lists, coded allergies, and other patient data. Both centers also have their own inpatient results viewing and order entry systems which provide clinicians caring for patients in the hospital the ability to review results and write orders. Although possible, clinicians caring for patients in the inpatient setting do not routinely access LMR to view test results. Inpatient physician use of the LMR is generally limited to review of the outpatient record, medication lists, and ambulatory notes at admission.

At hospital A, the hospitalist attending physician is typically responsible for all communication to outpatient physicians at discharge, as well as for follow‐up on all test results that return after discharge. Hospital B has 2 types of hospitalist services. One is staffed only by hospitalist and nonhospitalist attending physicians. Nonhospitalist attending physicians were excluded because they care for their own patients in the inpatient and ambulatory setting and typically use RM to manage test results. The other hospitalist service at hospital B is a teaching service consisting of an attending physician, resident, and interns. For this service, the resident is responsible for communication at discharge and follow‐up on all pending tests. For purposes of this study inpatient physicians refers to those physicians responsible for communication with PCPs and follow‐up on pending tests. All inpatient physicians were eligible to participate during the study period.

Test Result Management Application

RM was originally developed by Partners Healthcare to improve timely review and appropriate management of test results in the ambulatory setting. RM was developed for and vetted by primarily ambulatory physicians. The application is browser‐based, provider‐centric, and embedded in the LMR to help ambulatory clinicians review and act upon test results in a safe, reliable, and efficient manner. Although RM has access to all inpatient and outpatient data in the Partners Clinical Data Repository (CDR), given the volume of inpatient tests ordered, hospital‐based results are suppressed by default to limit inundating ambulatory clinicians' queues. Therefore, users of RM only receive results of laboratory and radiology tests ordered in the ambulatory setting. They can track these tests for specific patients for a designated period of time by placing the patient on a watch list. Finally, RM incorporates extensive decision support features to classify the degree of abnormality for each result, presents guidelines to help clinicians manage abnormal results, allows clinicians to generate result letters to patients using predefined, context‐sensitive templates, and prompts physicians to set reminders for future testing. Because RM was developed from the ambulatory perspective, there was limited input from hospitalist physicians with regard to inpatient workflow in the original design of the module.12 See Figure 1 for a screen shot of RM and a description of its features.

Figure 1

Partners Results Manager (RM). 1Within LMR, the web‐based EMR, clinicians can access RM by selecting the appropriate tab in the navigation bar. 2Clinicians can add patients to a watch list to track test results. 3Clinicians can access chemistry (C), hematology (H), and radiology (R) test results available in the Partners Clinical Data Repository (CDR). 4Abnormal test results are automatically flagged and filtered in RM by severity: !!! = critical result, !! = urgent result, ! = routine result. 5Clinicians can acknowledge any result listed within their RM queue. 6Clinicians can access visit notes associated with the ordered test. 7Clinicians can generate a pre‐populated patient results letter to enhance and expedite communication of test results. Abbreviations: LMR, longitudinal medical record; EMR, electronic medical record. [Color
figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

For purposes of this pilot, we modified RM to allow results of tests ordered in the inpatient setting to be available for viewing (Hospitalist Results Manager, HRM). This feature was turned on only for inpatient physicians as previously defined. Inpatient tests, including pending tests at discharge, continued to be suppressed from PCP's RM queue (however, any physician could access a patient's test result(s) directly from the Partners CDR). Inpatient physicians could track laboratory and radiology results finalized after discharge by keeping discharged patients on their HRM watch list for a designated period of time. The finalized results would become available for review in their HRM queue and abnormal results were displayed prominently at the top of this queue. Inpatient physicians were trained to use HRM in a series of meetings and demonstrations. Although HRM could be accessed from inpatient clinical workstations, it was not part of the inpatient clinical information system.

Surveys

Study surveys were developed and refined through an iterative process and pilot tested among inpatient physicians at both centers for clarity. We surveyed inpatient physicians five months after HRM implementation. Inpatient physicians were asked how often they used HRM, what barriers they faced (respondents asked to quantify agreement to statements on a 5‐point Likert scale), and which elements of an ideal system they would prefer. Finally, we solicited comments regarding perceived obstacles and suggestions for improvement. Because HRM was targeted to inpatient physicians, and because RM has been evaluated from the ambulatory perspective in a prior study,13 PCPs were not surveyed. See Supporting Information Appendix for the survey instrument used in the study.

Results

A total of 35 inpatient physicians participated in the pilot. Among 649 patients discharged during the study period, there were 1075 tests pending of which 555 were subsequently flagged as abnormal in HRM. Study surveys were sent to the 35 inpatient physician participants and 29 were completed, including partial responses (83% survey response rate). The 35 inpatient physician participants had the following characteristics: 22 were male, 13 were female; 21 were trainees and 14 were nontrainees/faculty. All 21 trainees were PGY2s. Nontrainees and faculty varied in experience level (PGY 15: 5, PGY 610: 7, PGY 1120: 1, PGY 21+: 1). Of 29 survey respondents, 7 were from hospital A and 22 were from hospital B; 19 were trainees and 10 were nontrainees/faculty. Of the 6 nonrespondents, 2 were from hospital A and 4 were from hospital B; 2 were trainees and 4 were nontrainees/faculty.

Table 1 shows the results of our survey of inpatient physicians regarding usage of HRM. Of 29 survey respondents, 14 (48%) reported never using HRM. Thirteen (45%) reported using HRM 1 to 2 times per week. None of the respondents used it more than 4 times per week. The frequency of usage was similar for hospitals A and B. Table 2 details barriers to using HRM. Twenty‐three inpatient physicians (79%) reported barriers. Seventeen (59%) thought that results in their HRM queue were not clinically relevant, 16 (55%) felt that HRM did not fit into their daily workflow, 14 (48%) had limited time to use HRM, and 12 (41%) noted that too many results in their HRM queue were on other physician's patients. Seven (24%) reported operational issues and 3 (10%) reported technical issues prohibiting use of HRM. With regard to preferred elements of an ideal results manager system, 21 (72%) inpatient physician respondents wanted to receive notification of abnormal and clinician‐designated pending test results. Four (14%) wanted to receive only abnormal results and 1 (3%) wanted to receive all results. Twenty‐seven (93%) physicians agreed that an ideally designed computerized test result management application would be valuable for managing pending tests at discharge.

Usage of HRM by Inpatient Physicians
FrequencyNumber of Inpatient Physicians Using HRM, n (%)
OverallHospital AHospital B
  • Abbreviation: HRM, hospitalist results manager.

Never14 (48)3 (43)11 (50)
12 times per week13 (45)3 (43)10 (45.5)
34 times per week2 (7)1 (14)1 (4.5)
57 times per week000
>7 times per week000
Barriers Prohibiting Use of HRM by Inpatient Physicians
BarrierOverall, n (%)Hospital A, n (%)Hospital B, n (%)
  • Abbreviation: HRM, hospitalist results manager.

  • A total of 29 surveys were fully or partially completed. Percent of responses reflects number of respondents who agreed or strongly agreed to each barrier question on a 5‐point Likert scale.

  • A total of 7 and 22 survey respondents were from hospital A and B, respectively. All survey respondents who were trainees were from hospital B.

Forgot to use HRM23 (79)7 (100)16 (73)
Results not clinically relevant17 (59)7 (100)10 (45)
Did not fit daily workflow16 (55)7 (100)9 (41)
Too little time to use HRM14 (48)6 (86)8 (46)
Results on others' patients12 (41)6 (86)6 (27)
HRM was difficult to use7 (24)2 (29)5 (23)
Had technical difficulties3 (10)0 (0)3 (14)

Table 3 provides comments from inpatient physician respondents regarding obstacles prohibiting use of HRM and suggestions for future systems.

Comments From Inpatient Physician Respondents Describing Suggestions for and Obstacles to Developing an Ideal Test Result Management System
  • Abbreviation: PCP, primary care physician.

Suggestions
Would be more useful if accessible from (the inpatient clinical information system).
Email notification (would have been useful).
At time of discharge, if there is a way to find pending labs at discharge, this would be of great utility.
Linking responsibility for follow‐up to test ordering (would have been useful).
Smarter system for filtering results so less important results are filtered out (is desirable).
Can the system be tied into PCP's email somehow?
Obstacles
Blood cultures, abnormal films can be difficult and time‐consuming to look up.
A big problem is results that automatically trigger even though they're not clinically relevant.
Keeping a record of patients that left with tests pending (is often difficult to do).
Addressing pending results is very time consuming.

Discussion

We describe a pilot implementation of a computerized application for the management of pending tests at hospital discharge. From responses to post‐implementation surveys, we were able to identify multiple factors prohibiting successful implementation of the application. These observations may help inform future interventions and evaluations.

Almost half of inpatient physicians reported never using HRM despite training and reminders. The feedback provided by physicians in our study suggested that HRM was not ideally designed from an inpatient physician perspective. We discovered several barriers to its use: (1) HRM overburdened physicians with clinically irrelevant test results, suggesting that more robust filtering of abnormal but low importance test results may be required (eg, a borderline electrolyte abnormality or low but stable hematocrit); (2) HRM did not integrate well into inpatient workflowthe system was not integrated into the inpatient results viewing and computerized physician order entry (CPOE) applications, and therefore required an extra step to access; (3) there was no mechanism of alerting inpatient physicians that finalized test results were available for viewing in their HRM queues (eg, by email or by an alert in the inpatient computer system); (4) because responsibility for these results was unclear, most inpatient physicians had no formal method of managing them, and for many, using HRM represented an additional task; and finally (5) several physicians commented on finding results in their HRM queue that belonged to other physician's patients, implying that the hospital databases were inaccurate in identifying the discharging physician or that rotation schedules, and therefore patient responsibility, had changed in the intervening period. Table 4 summarizes the advantages and respective limitations of features of HRM available to inpatient physicians.

Advantages and Limitations of Features in Hospitalist Results Manager
AdvantagesLimitations
  • Abbreviations: EMR, electronic medical record; LMR, longitudinal medical record; PCP, primary care physician.

Creates a physician‐managed queue of pending test results by patientDoes not provide alert or push notification when new results available for patients
Filters test results by severity with most critical results appearing at the top of the queueSeverity filter set for outpatients; not restrictive enough for post‐discharge period, resulting in excessive alerting
Independent, voluntary acknowledgement of results by userActive acknowledgment not required; no audit trail, feedback, or escalation if result not acknowledged
Embedded within LMR (the ambulatory EMR)LMR not routinely used by many inpatient physicians
Offers patient communication tools (eg, pre‐populated patient results letter)Tools not optimized for post‐discharge test result communication by inpatient physicians (eg, a tool for PCP result notification and acknowledgment)

In the literature, there is little information regarding optimal features of a test result management system for transitions from the inpatient to ambulatory care setting. Prior studies outline important functions for results management systems developed for noninpatient sites of care, including the ambulatory and emergency room setting.12, 14, 15 These include a method of prioritizing by degree of abnormality, the ability to reliably and efficiently act upon results, and an automated alerting system for abnormal results. Findings from our study provide insight in defining core functions for result management systems which focus on transitions from the inpatient to ambulatory care setting. These functions include tight integration with applications used by inpatient physicians, clear assignment of responsibility for test results finalized after hospital discharge (as well as a mechanism to reassign responsibility), automated alerts to responsible providers of test results finalized post‐discharge, and ways to automatically filter test results to avoid over‐burdening physicians with clinically irrelevant results.

Almost all surveyed inpatient physicians agreed that an ideally designed electronic post‐discharge results management system would be valuable. For such systems to be successfully adopted, we offer several principles to help guide future work. These include: (1) clarifying responsibility at the time a test is ordered and again at discharge, (2) understanding workflow and communication patterns among inpatient and outpatient clinicians, and (3) integrating technological solutions into existing systems to minimize workflow disruptions. For example, if the primary responsibility for post‐discharge result follow‐up lies with the ordering physician, the system should be integrated within the EMR most often used by inpatient physicians and become part of inpatient physician workflow. If the system depends on administrative databases to identify the responsible providers, these must be accurate. Alternatively, in organizations with computerized provider order entry, responsibility for the result could be assigned when the test is ordered and confirmed at discharge (ie, the results management system would be integrated into the discharge order such that pending tests are reviewed at the time of discharge). The discharging physician should have the ability to assign responsibility for each pending test and select preferred mode(s) of notification once its result is finalized (eg, e‐mail, alphanumeric page, etc.). The system should have the ability to generate an automatic notification to the inpatient and PCP (and perhaps other designated providers involved in the patient's inpatient care), but it should not burden busy clinicians with unnecessary alerts and warnings. Finally, the rules by which results are prioritized must be robust enough to filter out less urgent results, and should be modified to reflect the severity of illness of recently discharged patients. In essence, in consideration of the time constraints of busy clinicians, an ideal results management system should achieve automated notification of test results while minimizing the risk of alert fatigue from the potentially large volume of alerts generated.

Our study has several important limitations. First, although our survey response rate was high, the sample size of actual participants was small. Second, because the study was conducted in 2 similar, tertiary care academic centers, it may not generalize to other settings (we note that hospital B included a nonteaching service similar to those in nonacademic medical centers). This may be particularly true in assessing the importance of specific barriers to use of results management systems, which may vary at different institutions. Third, the representation of survey respondents were skeweda majority of the responses were from trainees (all post‐graduate level [PGY] level 2) and from hospital B. Fourth, we did not actively monitor physician interaction with the test result management application, and therefore, we depended heavily on physician recollection of use of the system when responding to surveys. Finally, we did not convene focus groups of key individuals with regard to the factors facilitating or prohibiting adoption of the system. Use of semi‐structured, key informant interviews (ie, focus groups) before and after implementation of an electronic results management application, have been shown to be effective in evaluating potential barriers and facilitators of adoption.16 Focus groups of and/or interviews with inpatient and PCPs, physician extenders, and housestaff could have been useful to better characterize the potential barriers and facilitators of adoption noted by survey respondents in our study.

In summary, we offer several lessons from our attempt to implement a system to manage pending tests at hospital discharge. The success of implementing future systems to address this patient safety concern will rely on accurately assigning responsibility for these test results, integrating the system within clinical information systems commonly used by the inpatient physician, addressing workflow issues and time constraints, maximizing appropriateness of alerting, and minimizing alert fatigue.

References

Online-Only Materials

   Comments ()