Original Research

Monitoring the pulse of hospital activity: Electronic health record utilization as a measure of care intensity

Abstract

BACKGROUND

Hospital care on weekends has been associated with reduced quality and poor clinical outcomes, suggesting that decreases in overall intensity of care may have important clinical effects. We describe a new measure of hospital intensity of care based on utilization of the electronic health record (EHR).

METHODS

We measured global intensity of care at our academic medical center by monitoring the use of the EHR in 2011. Our primary measure, termed EHR interactions, was the number of accessions of a patient's electronic record by a clinician, adjusted for hospital census, per unit of time. Our secondary measure was percent of total available central processing unit (CPU) power used to access EHR servers at a given time.

RESULTS

EHR interactions were lower on weekend days as compared to weekdays at every hour ( P  < 0.0001), and the daytime peak in intensity noted each weekday was blunted on weekends. The relative rate and 95% confidence interval (CI) of census‐adjusted record accessions per patient on weekdays compared with weekends were: 1.76 (95% CI: 1.74‐1.77), 1.52 (95% CI: 1.50‐1.55), and 1.14 (95% CI: 1.12‐1.17) for day, morning/evening, and night hours, respectively. Percent CPU usage correlated closely with EHR interactions ( r  = 0.90).

CONCLUSIONS

EHR usage is a valid and easily reproducible measure of intensity of care in the hospital. Using this measure we identified large, hour‐specific differences between weekend and weekday intensity. EHR interactions may serve as a useful measure for tracking and improving temporal variations in care that are common, and potentially deleterious, in hospital systems. Journal of Hospital Medicine 2013;8:513–518. © 2013 Society of Hospital Medicine

Copyright © 2013 Society of Hospital Medicine

Hospitals provide acute inpatient care all day and every day. Nonetheless, a number of studies have shown that weekend care may have lower quality than care delivered on weekdays. [1, 2, 3, 4, 5, 6, 7] Weekend care has been associated with increased mortality among patients suffering from a number of conditions [4, 5, 6, 7, 8, 9, 10] and in a number of clinical settings. [11, 12, 13, 14] Findings of poor outcomes on weekends are not universal, though exceptions occur typically in settings where services and clinical staffing are fairly constant throughout the week, such as intensive care units with on‐site intensivists. [15, 16, 17]

These differences in quality and outcomes suggest that there is a difference in intensity of care over the course of the week. Initiatives to address this important safety issue would be strengthened if a shared metric existed to describe the intensity of care provided at a given time of day or day of the week. However, few measures of global hospital intensity of care exist. The metrics that have been developed are limited by measuring only 1 aspect of care delivery, such as weekend nurse staffing, [18] by requiring extensive chart abstraction, [19] or through reliance on hospital expenses in an older payment model. [20] These measures notably predate contemporary hospital care delivery, which is increasingly dependent on the electronic health record (EHR). To our knowledge, no prior measure of hospital care intensity has been described using the EHR.

Recently, our medical center began an initiative to increase weekend services and staffing to create more uniform availability of care throughout the week. The purpose of this study was to develop a global measure of intensity of care using data derived from the EHR as part of the evaluation of this initiative. [21]

METHODS

We conducted a retrospective analysis of EHR activity for hospital inpatients between January 1, 2011 and December 31, 2011 at New York University Langone Medical Center (NYULMC). During that time period, nearly all inpatient clinical activities at NYULMC were performed and documented through the EHR, Sunrise Clinical Manager (Allscripts, Chicago, IL). These activities include clinical documentation, orders, medication administration, and results of diagnostic tests. Access to the EHR is through Citrix Xenapp Servers (Citrix Systems, Santa Clara, CA), which securely deliver applications from central servers to providers' local terminals.

The primary measure of EHR activity, which we refer to as EHR interactions, was defined as the accessing of a patient's electronic record by a clinician. A provider initiates an interaction by opening a patient's electronic record and concludes it by either opening the record of a different patient or logging out of the EHR. An EHR interaction thus captures a unit of direct patient care, such as documenting a clinical encounter, recording medication administration, or reviewing patient data. Most EHR systems routinely log such interactions to enable compliance departments to audit which users have accessed a patient chart.

The secondary measure of EHR activity was percent central processing unit (CPU), which represents the percent of total available server processing power being used by the Citrix servers at a given time. CPU utilization was averaged over an hour to determine the mean percentage of use for any given hour. As CPU data are not retained for more than 30 days at our institution, we examined CPU usage for the period July 1, 2012 to August 31, 2012.

We evaluated subgroups of EHR interactions by provider type: nurses, resident physicians, attending physicians, pharmacy staff, and all others. We also examined EHR interactions for 2 subgroups of patients: those admitted to the medicine service from the emergency room (ER) and those electively admitted to the surgical service. An elective hospitalization was defined as 1 in which the patient was neither admitted from the ER nor transferred from another hospital. These 2 subgroups were further refined to evaluate EHR interactions among groups of patients admitted during the day, night, weekday, and weeknight. Finally, we examined specific EHR orders that we considered reflective of advancing or altering care, including orders to increase the level of mobilization, to insert or remove a urinary catheter, or to initiate or discontinue antibiotics. As antibiotics are frequently initiated on the day of admission and stopped on the day of discharge, we excluded antibiotic orders that occurred on days of admission or discharge.

Statistical Analysis

EHR interactions were presented as hourly rates by dividing the number of patient chart accessions by the inpatient census for each hour. Hourly census was determined by summing the number of patients who were admitted prior to and discharged after the hour of interest. We calculated the arithmetic means of EHR interactions for each hour and day of the week in 2011. Analysis of variance was used to test differences in EHR interactions among days of the week, and t tests were used to test differences in EHR interactions between Saturday and Sunday, Tuesday and Wednesday, and weekend and weekdays. As a result of multiple comparisons in these analyses, we applied a Bonferroni correction.

EHR interaction rates were assigned to 1 of 3 periods based on a priori suspected peak and trough intensity: day (9:00 am to 4:59 pm ), morning/evening (7:00 am to 8:59 am and 5:00 pm to 7:59 pm ), and night (8:00 pm to 6:59 am ). Negative binomial regression models were used to determine the relative rate of weekday to weekend EHR interactions per patient for the 3 daily time periods. Similar models were developed for EHR interactions by provider types and for specific orders.

We calculated the correlation coefficient between total EHR interactions and occurrence of specific orders, EHR interactions by provider type, EHR interactions by patient subgroups, and CPU. A sensitivity analysis was performed in which EHR interactions were calculated as the number of patient chart accessions per number of daily discharges; this analysis considered the fact that discharges were likely to have high associated intensity but may be less common on weekends as compared to weekdays.

All analyses were performed using Stata 12 (StataCorp, College Station, TX). This study was approved by the institutional review board at NYU School of Medicine.

RESULTS

During the study period, the mean (standard deviation) number of EHR interactions per patient per hour was 2.49 (1.30). EHR interactions differed by hour and day of the week; the lowest number occurred in early morning on weekends, and the highest occurred around 11:00 am on weekdays (Figure 1 ). EHR interactions differed among days of the week at all times ( P <0.001), whereas EHR interactions were similar on most hours of Saturday and Sunday as well as Tuesday and Wednesday. At every hour, weekends had a lower number of EHR interactions in comparison to weekdays ( P <0.001). Weekdays showed a substantial increase in the number of EHR interactions between 8:00 am and 12:00 pm , followed by a slight decrease in activity between 12:00 pm and 2:00 pm , and another increase in activity from 2:00 pm to 5:00 pm (Figure 1 ). Weekend days demonstrated marked blunting of this midday peak in intensity. The tracking of an entire month of hospital EHR interactions produced a detailed graphic picture of hospital activity, clearly demarcating rounding hours, lunch hours, weekend days, hospital holidays, and other landmarks (Figure 2 ). The relative rates of census‐adjusted EHR interactions on weekdays versus weekends were 1.76 (95% confidence interval [CI]: 1.74‐1.77) for day, 1.52 (95% CI: 1.50‐1.55) for morning/evening, and 1.14 (95% CI: 1.12‐1.17) for night hours.

Figure 1

Mean electronic health record interactions per patient per hour for calendar year 2011.

Figure 2

Electronic heath record (EHR) interactions per patient by hour, January 2011. January 1, 2, 8, 9, 15, 16, 22, 23, 29, 30 were weekend days, whereas January 17 (*) was a federal holiday. An episode of EHR downtime for regular maintenance occurred in the early morning of January 16.

Nurses performed the largest number of EHR interactions (39.7%), followed by resident physicians (15.2%), attending physicians (10.2%), and pharmacists (7.6%). The remainder of EHR interactions were performed by other providers (27.4%), whose role in the majority of cases was undefined in the EHR (see Supporting Table 1 in the online version of this article). Daily variation in EHR interactions differed by provider type (Table 1 ). Nurses and resident physicians showed smaller differences in their EHR interactions between weekend and weekdays and among times of day when compared to attending physicians, pharmacy staff, and other staff. EHR interactions showed similar variations to the overall cohort for both medicine patients admitted from the ER and elective surgery patients (Table 1 ). Specific clinical orders designed to sample particularly meaningful interactions, including urinary catheter insertion and removal, patient mobilization orders, and new antibiotic orders, were moderately correlated with total EHR interactions (Table 2 ). In a sensitivity analysis, the comparison of weekday to weekend EHR interactions per daily discharge was similar to the primary analysis, with relative rates of 1.75 (95% CI: 1.73‐1.77), 1.54 (95% CI: 1.50‐1.57), and 1.15 (95% CI: 1.11‐1.18) for day, morning/evening, and night hours, respectively.

Rate Ratio of Electronic Health Record Interactions per Patient for Weekdays Versus Weekends, 2011
Hour Group a
Daytime HoursMorning/Evening HoursNighttime HoursCorrelation Coefficient to
Rate Ratio95% CIRate Ratio95% CIRate Ratio95% CIEHR Interactions
  • NOTE: Abbreviations: CI, confidence interval; EHR, electronic health record.

  • Hour groups represent the following times of day: day (9:00 am to 4:59 pm ); morning/evening (7:00 am to 8:59 am and 5:00 pm to 7:59 pm ); night (8:00 pm to 6:59 am ).

All EHR interactions1.761.74‐1.771.521.50‐1.551.141.12‐1.171.00
By provider subgroup
Nurses1.351.34‐1.371.211.20‐1.221.101.08‐1.120.92
Residents1.381.36‐1.401.461.43‐1.491.141.11‐1.180.90
Attending physicians1.701.67‐1.732.041.97‐2.101.321.26‐1.390.96
Pharmacy staff1.641.61‐1.671.681.61‐1.751.191.15‐1.230.90
All others2.752.70‐2.792.082.002.171.201.15‐1.250.97
By admission subgroup
Medicine ER admissions
All1.641.62‐1.671.411.38‐1.441.071.04‐1.090.92
Day admissions1.641.62‐1.671.411.38‐1.441.081.05‐1.110.92
Night admissions1.651.62‐1.681.411.37‐1.451.031.001.050.88
Weekday admissions1.761.73‐1.791.561.52‐1.591.281.23‐1.300.93
Weekend admissions1.401.38‐1.421.141.11‐1.170.700.68‐0.720.88
Surgery elective admissions
All1.631.61‐1.651.531.50‐1.561.251.21‐1.280.97
Day admissions1.621.60‐1.641.541.51‐1.571.261.23‐1.300.96
Night admissions1.651.62‐1.671.501.46‐1.541.221.18‐1.250.94
Weekday admissions1.641.62‐1.661.551.52‐1.591.271.24‐1.310.96
Weekend admissions1.601.56‐1.651.261.21‐1.310.890.85‐0.930.73
Rate Ratio of Specific Orders per Patient for Weekdays Versus Weekends, 2011
Hour Group a
Daytime HoursMorning/Evening HoursNighttime HoursCorrelation Coefficient to
Rate Ratio95% CIRate Ratio95% CIRate Ratio95% CIEHR Interactions
  • NOTE: Abbreviations: CI, confidence interval; EHR, electronic health record.

  • Hour groups represent the following times of day: day (9:00 am to 4:59 pm ); morning/evening (7:00 am to 8:59 am and 5:00 pm to 7:59 pm ); night (8:00 pm to 6:59 am ).

  • Excludes those orders from day of admission.

  • Excludes those orders from day of discharge.

Urinary catheter2.812.61‐3.023.172.85‐3.541.421.27‐1.580.66
Mobilization2.512.39‐2.632.912.71‐3.121.421.34‐1.520.70
Antibiotic initiation b1.331.24‐1.421.651.47‐1.851.131.01‐1.250.41
Antibiotic discontinue c1.741.63‐1.851.681.50‐1.871.281.13‐1.450.61

As seen in Figure 3 , CPU usage was well correlated with EHR interactions ( r =0.90) and both metrics were lower on weekends as compared to weekdays. A few outliers of increased CPU usage on weekends were observed; these outliers all occurred on Sundays from 12:00 am to 2:00 am . The information technology (IT) department at our institution confirmed that these outliers coincided with a weekly virus scan performed at that time.

Figure 3

Correlation of central processing unit (CPU) usage to electronic health record (EHR) interactions per patient. Each point represents the percent CPU usage (y‐axis) and number of EHR interactions per patient (x‐axis) for a given hour and day of July 1, 2012 to August 31, 2012. The weekend points primarily reside in the left lower quadrant, reflecting reduced EHR interactions and CPU activity on weekends. The points in the upper left corner correspond to the hours of Sunday 12:00
am
and 1:00
am
, which were determined to be outliers related to a weekly virus scan. The correlation coefficient is 0.90.

DISCUSSION

EHR interactions represents a new and accessible measure of hospital care intensity that may be used to track temporal variations in care that are likely to exist in all hospital systems. As the number of hospitals that process and document all clinical activities through the EHR continues to increase, [22, 23] such a measure has the potential to serve as a useful metric in efforts to raise nighttime and weekend care to the same quality as that during weekdays.

We found that EHR interactions differed markedly between days and nights and between weekdays and weekends. A number of prior studies have found temporal variations in clinical outcomes and care. [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14] Our study adds to this literature by defining a new measure of the overall intensity of the process of inpatient care. Using this measure, we demonstrate that the increase in midday clinical activity seen during the week is substantially blunted on weekends. This finding adds validity to the designation of both nights and weekends as off hours when examining temporal variations in clinical care. [2, 7]

Other subtle patterns of care delivery emerged throughout the day. For instance, we found a decrease in EHR interactions between 12:00 pm and 2:00 pm , likely representing a small lull in clinical activity during the lunchtime hour, whereas EHR interactions were higher in the night and evening hours than in the early morning. The clinical significance of such fluctuations remains undetermined.

Variation throughout the week differed by provider type. Nurses and resident physicians, who are scheduled around the clock, demonstrated the least fluctuation in EHR interaction intensity over time. Attending physicians and other staff showed the greatest variation in weekday versus weekend care.

We defined a second measure of care intensity based on EHR utilization, which we named CPU usage. The advantage of using CPU as a metric is that it easy to measure and track as it is routinely monitored by hospital IT departments as part of regular maintenance. CPU usage can therefore be easily implemented in any institution with an EHR, whereas measuring EHR interactions requires EHR data abstraction and manipulation. The disadvantage of CPU usage as a measure is that it includes nonclinical reporting and other functions and therefore represents a less specific measure of clinical activity. Nonetheless, at our institution, nearly all clinical activities are processed or documented through the EHR, which is maintained on central servers, and CPU usage was well correlated with EHR interactions. We therefore believe that CPU usage represents a useful indicator of hospital clinical activity at a given time.

Several limitations deserve mention. First, EHR interactions may not always reflect actual patient care or clinical documentation. Nonetheless, we also examined specific patient orders and found that variations in these process measures were similar to those in the EHR interaction measure. Second, our methods were developed and evaluated at a single institution and may not be generalizable. Third, increased intensity of care does not necessarily increase quality of care. For instance, laboratory tests are commonly obtained on a daily basis in the hospital, a practice that is costly and often unnecessary. [24] Weekends have been associated with fewer laboratory tests, which may be appropriate as compared to the more frequent testing observed on weekdays. [25] Finally, we do not ascertain whether variations in EHR interactions correlate with clinical outcomes, and believe this to be an area for further research.

In conclusion, EHR interactions represent a new global process measure of care intensity, which was demonstrated to vary over the course of a week. We intend to use this measure at our institution to track progress of an initiative to ensure a high standard of care throughout the course of the week. The extent to which temporal variations in EHR interactions or reductions in these variations are correlated with clinical outcomes deserves further study. We believe this measure, which can be adapted to other institutions, may have a valuable role to play in hospitals' efforts to eliminate excess morbidity and mortality associated with care delivered during nights and weekends.

References

Online-Only Materials

   Comments ()