Original Research

Early recognition of acutely deteriorating patients in non‐intensive care units: Assessment of an innovative monitoring technology

Abstract

Abstract

BACKGROUND:

Continuous vital sign monitoring has the potential to detect early clinical deterioration. While commonly employed in the intensive care unit (ICU), accurate and noninvasive monitoring technology suitable for floor patients has yet to be used reliably.

OBJECTIVE:

To establish the accuracy of the Earlysense continuous monitoring system in predicting clinical deterioration.

DESIGN:

Noninterventional prospective study with retrospective data analysis.

SETTING:

Two medical wards in 2 academic medical centers.

PATIENTS:

Patients admitted to a medical ward with a diagnosis of an acute respiratory condition.

INTERVENTION:

Enrolled patients were monitored for heart rate (HR) and respiration rate (RR) by the Earlysense monitor with the alerts turned off.

MEASUREMENTS:

Retrospective analysis of vital sign data was performed on a derivation cohort to identify optimal cutoffs for threshold and 24‐hour trend alerts. This was internally validated through correlation with clinical events recognized through chart review.

RESULTS:

Of 113 patients included in the study, 9 suffered major clinical deterioration. Alerts were found to be infrequent (2.7 and 0.2 alerts per patient‐day for threshold and trend alert, respectively). For the threshold alerts, sensitivity and specificity in predicting deterioration was found to be 82% and 67%, respectively, for HR and 64% and 81%, respectively, for RR. For trend alerts, sensitivity and specificity were 78% and 90% for HR, and 100% and 64% for RR, respectively.

CONCLUSIONS:

The Earlysense monitor was able to continuously measure RR and HR, providing low alert frequency. The current study provides data supporting the ability of this system to accurately predict patient deterioration. Journal of Hospital Medicine 2012; © 2012 Society of Hospital Medicine

Copyright © 2012 Society of Hospital Medicine

Treatment of hospitalized patients is becoming more and more complex as a result of rising patient age, multiple comorbidities, and more complex procedures, yet most are hospitalized in a non‐intensive care unit (non‐ICU) setting. With this rising complexity, a large proportion of hospital patients experience serious adverse events during their hospital stay, including cardiac arrest, respiratory failure, and shock, leading to unplanned admissions to the ICU, and death.1 For patients with unexpected clinical deterioration, delayed or suboptimal intervention is associated with increased morbidity and mortality.2, 3 This has prompted many hospitals to implement some form of a rapid response system (RRS) for early detection of clinical deterioration, and adequate response through rapid response teams (RRT).2, 4, 5 Still, evidence suggests that many of these systems are deficient in the detection and/or the response phase, and do not lead to the expected outcomes in terms of preventing critical events.6, 7

Researchers have established that patients frequently demonstrate clinical signs of deterioration hours before cardiac arrests or urgent transfers to ICUs.8, 9 The importance of timely interventions in many acute clinical conditions like sepsis, acute myocardial infarction, and stroke is also well established.1012 Taking these facts together with the observed shortcomings of most current rapid response systems has led experts to call for a shift of focus from the efferent limb of the RRS (the response team) to the afferent limb (the means of detecting patients at risk and obtaining help).13 A consensus conference held in 2008 emphasized the importance of accurate monitoring of vital signs for all hospitalized patients, and at the same time recommended, if practical and affordable, that all patients be monitored continuously by monitoring technology, which improves patient comfort, is easy to use, and reduces the number of false alerts.13

Technology applications that allow for continuous vital sign monitoring designed for non‐ICU settings may help hospitals achieve meaningful results by providing earlier detection of clinical deterioration, either when implemented as part of an RRS or as a stand‐alone system. To fit this description, monitors would have to be easy to use by the staff, impose little limitation on the patient, and be capable of trend analyses so that they minimize the incidence of false alerts and alert fatigue. We set out to investigate the capability of such a system to predict patient deterioration in a medical floor setting.

METHODS

Patients and Settings

We conducted a feasibility noninterventional study in two 36‐bed medicine units of two tertiary academic medical centers: Sheba Medical Center, Tel Hashomer, and Sapir Medical Center, Kfar Sababoth in Israel. The aim of the study was to define optimal cutoff values for vital signs alerts and define the accuracy of these alerts in predicting clinical deterioration. The study was approved beforehand by the independent Institutional Review Boards of both institutions.

The Earlysense system is a contactless continuous‐measurement monitor for heart rate (HR), respiration rate (RR), and bed motion. It is based on a piezoelectric sensor, sensitive to applied mechanical strain, which is placed under the patient's mattress and functions without the need for contact between patient and device. The Earlysense system was previously shown to measure accurately both HR and RR in sleep laboratory and general ICU settings.14

We chose to evaluate patients who were at increased risk for respiratory failure, which would enable us to capture more events in this study. We thus included, in the study population, patients hospitalized with an admitting diagnosis due to an acute respiratory condition including pneumonia, chronic obstructive pulmonary disease or asthma exacerbation, congestive heart failure with pulmonary edema or congestion, and patients who needed supplemental oxygen on admission. Patients were enrolled only if within the initial 24 hours of hospitalization in the study units. Dementia and inability to sign informed consent were exclusion criteria for enrollment in this study. Patients with a do not resuscitate order were not excluded from enrollment. Patient enrollment took place during the period of January to December of 2008 in Sheba Medical Center and July to November of 2008 in Sapir Medical Center. Since the study was noninterventional, and the Earlysense monitor did not produce alerts for the duration of this study, patients were still monitored by other monitoring devices, such as telemetry and pulse oximeter, as per clinical decision.

Following enrollment and informed consent, the Earlysense sensor was placed under the mattress and connected to the Earlysense bedside monitor that recorded the signals and the interpreted vital signs. The Earlysense system automatically started measuring respiratory rate, heart rate, and motion rate with no need for patient, nurse, or technician involvement, as long as the patient remained in bed. Since this was a noninterventional study, no alerts were produced from the Earlysense monitor, and floor staff was not trained to make clinical decisions based on vital sign data from the device. Patients were monitored for the full extent of their stay in the 2 medical departments. Since we had only 2 monitors at each site, in an attempt to enroll more patients, in cases where patients stayed for more than 2 weeks, a decision was made by the principal investigator whether to continue monitoring or disconnect the sensor and enroll another patient. This was based on an assessment of clinical stability and on how much longer patients were expected to stay in the unit, in an attempt to avoid a situation where a patient was utilizing 1 monitor for a very long period of time. Following patient discharge or discontinuation of the monitor, all data were downloaded to a central server for analysis. A full description of the contactless sensor and monitor was published previously.14, 15 The signal analysis determining the HR and RR from the motion waves is proprietary.

Clinical and Alert Definitions

Patients were followed for major clinical events during their hospitalization. A major clinical event was defined as any one of the following: 1) patient was transferred to an ICU (medical or cardiac); 2) patient was intubated and mechanically ventilated on the floor; or 3) patient had a cardiac arrest while in the unit.

Respiratory rate or heart rate alerts were based on vital sign readings recorded and analyzed retrospectively for this study. The Earlysense system performs signal processing in order to isolate the respiratory and the heart pulse patterns from the signal obtained by the piezoelectric sensor. Each 0.5 seconds, an updated HR reading is established based on analysis of the heart pulse pattern for the last 8 seconds, and an updated RR reading based on analysis of the last 1 minute of the respiration pattern. These specific time periods were intended to provide up‐to‐date HR and RR values, and still reduce possible false HR and RR readings caused by signal artifacts. For defining optimal cutoffs for alerts in an internal validation process, alerts were considered true‐positive if they were followed by a major clinical event within 24 hours. An alert was considered false‐positive (false alert) if no major clinical event occurred in the following 24 hours.

As a secondary outcome, we also analyzed 24‐hour trends to look for correlations with major clinical events. For trend analysis, we grouped together HR and RR readings for 6‐hour periods throughout the day in a running window fashion (data for the last 6 hours was clustered every 3 minutes). We then compared the median of the readings for each period with the corresponding period of the previous day. Post‐hoc alerts were generated when the difference (delta) between medians passed the threshold set. Only 6‐hour time windows with at least 420 valid RR or HR results were included in the analysis.

Data Analysis

In the data analysis, we excluded patients with less than 30 hours of monitoring, allowing us to evaluate the trend analysis for at least one comparable time window. We examined increasing possible cutoff values for the differences in median values that would best predict the clinical events. To find the best possible models, we performed a receiver operating characteristic (ROC) curve analysis. We looked for the optimal cutoff to yield the maximal sum of sensitivity plus specificity, and computed associated statistics (area under the curve [AUC]) and associated confidence interval; if the lower bound of interval was found to be above 0.5, the AUC was significantly different from chance. We determined a P value that tests the null hypothesis that the AUC really equals 0.5. A value below 0.05 was considered significant.

RESULTS

Of 149 patients monitored by the Earlysense system for this study, 113 had at least 30 hours of monitoring. The characteristics of these patients are presented in Table 1. Of these 113 patients, 9 had a major clinical event (8.0%), including 2 patients who were transferred to an ICU, 1 patient who was intubated and ventilated in the study unit and later had a cardiac arrest and died, and 6 more patients who developed cardiac arrest and died in the study units (overall, 10 major clinical events). Comparing patients with a major clinical event to patients without, we found no significant difference in age, gender, body mass index, or comorbidity (Charlson score), however, patients who had a clinical event stayed significantly longer than those without a clinical event (Table 1). On average, time from admission to the major clinical event was 11.6 days (range 228).

Demographics and Description of Hospitalized Patients Participating in the Study
 All PatientsPatients Without a Major Clinical EventPatients With a Major Clinical EventP (t Test)
  • Abbreviations: CHF, congestive heart failure; COPD, chronic obstructive pulmonary disease; SD, standard deviation.

  • Including multiple diagnoses and other conditions such as primary or secondary lung malignancy, pulmonary emboli, and interstitial lung disease.

Patients1131049 
Gender (% female)45.146.233.30.46
Age, average SD69 1869 18.569 16.40.99
Body mass index, average (range)27 (1752.7)27.1 (1752.7)25.2 (17.932)0.41
Charlson comorbidity score1.581.551.990.64
Length of stay (days), average SD7.3 5.96.9 6.011.9 7.80.02
Primary diagnosis    
Pneumonia36324 
CHF33312 
COPD17161 
Other respiratory*27252 

Overall, this study included a total of 8628 recording hours for the 113 patients we analyzed. Using the vital signs thresholds found to provide the highest sensitivity and specificity, we recorded 898 RR alerts (average frequency of 2.4 alerts per patient‐day) and 107 HR alerts (average frequency of 0.3 alerts per patient‐day). For the entire study, we have found 63 trend alerts (46 RR and 17 HR) when using thresholds for maximal sensitivity and specificity, at a frequency of 0.2 alerts per patient‐day.

In Table 2, we summarize the sensitivity, specificity, positive predictive value, and negative predictive value of alerts in predicting a major clinical deterioration per patient for both the threshold alerts and the trend alerts. The cutoff values are those of optimal performance (maximal sensitivity plus specificity) along the ROC curves presented in Figure 1. The optimal cutoffs for the threshold alerts were HRs below 40 or above 115 beats/min, and RRs below 8 or above 40 breaths/min. For the trend alerts, when comparing between time periods, we found that a cutoff of a rise of 20 or more beats/min and 5 or more breaths/min corresponded with a maximal sensitivity and specificity, and we used these as the thresholds for the trending alerts.

Figure 1
Receiver operator characteristic (ROC) curves for (A) heart rate (HR) and respiration rate (RR) threshold alerts, and (B) for 24‐hour trend HR and RR alerts. Optimal cutoffs were used when comparing time periods (20/min for HR and 5/min for RR) for maximal sum of sensitivity and specificity. Abbreviations: AUC, area under the curve.
Sensitivity, Specificity, Positive Predictive Value, and Negative Predictive Value of Alerts in Predicting Major Clinical Events per Patient, Optimized for Maximum Sensitivity Plus Specificity
 Sensitivity (%)Specificity (%)Positive Predictive Value (%)Negative Predictive Value (%)
  • Abbreviations: HR, heart rate; RR, respiration rate.

  • Heart rate thresholds set to below 40 or above 115 beats/min based on maximal sensitivity plus specificity achieved.

  • Respiration rate thresholds set to below 8 or above 40 breaths/min based on maximal sensitivity plus specificity achieved.

Threshold alerts
HR*82672197
RR64812695
HR and RR55945095
Trend alerts
HR 2078904197
RR 51006420100
HR and RR78945498

For the determined alerts, out of the 10 clinical events, in 7 cases the alerts would have indicated the deterioration within more than 1 hour from the actual event and within 24 hours prior to the clinical event. In one case, the alert would have indicated the risk within less than 1 hour; in another case, within the 2448 hours window; and in the last case, the alerts would have done so more than 48 hours prior to the event (these last 2 cases were considered false‐negative for our post hoc analysis).

The ROC analysis for both the threshold alerts and the trend alerts for both HR and RR correlated well with the clinical events (P < 0.05 for all models) (Figure 1). When comparing the threshold alerts with the trend alerts, trend alerts had a higher AUC, suggesting that these would be more accurate in alerting for major clinical events. Specifically, the model for HR 6‐hour trend alerts demonstrated a larger AUC (0.9 and 0.85, respectively), while the combination of HR and RR had the largest AUC (0.93).

DISCUSSION

The development of monitoring systems designed for hospitalized patients on non‐ICU floors derives from the need to minimize preventable in‐hospital mortality. This study has explored one such possible solution. Through retrospective definition of optimal alert settings and internal validation within a derivation cohort, we have found the Earlysense monitor to be accurate in predicting major clinical deterioration by using both simple thresholds and a vital signs trend algorithm. We found that the post hoc determined alerts were relatively infrequent. We also found that trend alerts showed greater ability to accurately capture these deteriorations compared with simple alert thresholds, and that using a combination alert based on both HR and RR trends provided the highest accuracy and confidence for predicting clinical deterioration. These results suggest the advantages that smart alerts can bringincreasing sensitivity while lowering the alert burden and reducing the number of false alerts.

An effective and efficient patient monitoring system must be able to quickly detect acute life‐threatening events or subacute patient deterioration that will lead to a life‐threatening event; in addition, this system should have a low false‐positive rate (few false alerts). While a high sensitivity is desirable,16 this usually comes at the cost of lower specificity and higher false‐positive rates. Studies have found alerts from monitoring systems in an ICU and emergency department environment to be tremendously frequent and with an extremely high false‐positive rate.1721 Unsurprisingly, it has been reported that only about 10% of alerts are responded to by the clinical staff.22 In an attempt to improve alert relevancy and accuracy, newly designed integrated monitoring systems for ICUs use alert algorithms and trend analysis performed on data received from multiple vital signs monitors.2327 With the evident need to continuously monitor patients on hospital floors,13, 28 there is a clear gap to‐date in such experience and a need to study the implications for patients and staff as we try to optimize the monitoring tools for this environment.

As more emphasis is placed on the afferent limb of rapid response systems, we can expect to see a greater need for systems able to detect changes in clinical status that might lead to patient deterioration. The intermittent vital sign measurement performed by nurses and support staff on the floors is now regarded as inadequate by itself, because it may not be performed predictably, accurately, or completely.28 Continuous monitoring systems should complement manual intermittent vital signs monitoring and play an important role in bringing the nurse to the patient's side at the right time so that a complete assessment can be done.29, 30 There remains no substitution for a nurse evaluating the patient and making critical decisions based on his or her knowledge of the patient's condition.

While today, both electrocardiographic monitoring (telemetry) and continuous pulse oximetry provide some solution for the need for continuous monitoring on the floors, these carry disadvantages that prevent them from becoming the optimal solution. While telemetry is invaluable for higher risk cardiac patients, significant overuse of this application does occur,31 and most monitored patients gain little cardiac arrest survival benefits.32 Pulse oximetry, although regarded as one of the most important technological advancements in monitoring patients, carries important limitations that might prevent early detection of respiratory failure.33, 34 Furthermore, these 2 modalities require continuous contact of leads/sensors to the patient, a disadvantage when regarding monitoring for low/average risk patients on non‐ICU hospital floors. Related to these disadvantages, the results presented here might position the implementation of the Earlysense monitor as a preferred alternative.

This study has several important limitations. This was a retrospective noninterventional study limited to internal medicine patients. Our patients served as a derivation cohort, alert thresholds were set post hoc, and sensitivity and specificity were internally validated. It remains to be shown that the results shown here can be reproduced in a real‐time interventional study. We only included patients who were considered above average risk within medical wards, as we were hoping to capture more clinical deteriorations with a relatively small cohort. We cannot rule out that our results might be biased and may not similarly apply to all non‐ICU medical patients. Finally, the retrospective nature of this study did not allow us to assess other outcomes of appropriateness, such as staff objective assessment of alert appropriateness or change in clinical management as a result of an alert. When using these devices in the alert mode, alert thresholds may be changed by the clinical staff based on clinical judgment to further lower false alerts and alert frequency. We also did not collect information that could have allowed us to assess clinical usefulness of the alerts compared to routine clinical assessment. Documenting whether the clinical events were suspected beforehand by the clinical teams could have allowed us to assess clinical usefulness. Future interventional studies are needed to validate the Earlysense system prospectively in both medical and surgical floor settings, first using a common alert threshold alerting system and secondly using the smart trend alerts described here.

In conclusion, we found that the Earlysense monitor is able to continuously measure RR and HR, providing low alert frequency. The current study demonstrates the potential of this system to provide timely prediction of patient deterioration. Utilizing a smart trend algorithm has been shown to improve the device's accuracy and reduce associated alert burden and false‐positive alerts.

Acknowledgements

Disclosures: The study was funded by an industry grant provided by Earlysense LTD. Eyal Zimlichman, MD, MSc Ronen Rozenblum, PhD, MPH, and Jeffrey M. Rothschild, MD, MPH, have each received a research grant supported by Earlysense LTD. Martine Szyper‐Kravitz, MD, Avraham Unterman, MD, Howard Amital, MD, MHA, and Yehuda Shoenfeld, MD, report no conflicts of interest. Zvika Shinar, PhD, Tal Klap, and Shiraz Levkovich are employed by Earlysense LTD and are holding equity with the company.

References

Online-Only Materials

AttachmentSize
Microsoft Office document icon Supporting Information (1)66.5 KB