Original Research

An electronic strategy to identify hospitalized heart failure patients

Abstract

Abstract

BACKGROUND

A common challenge in improving performance measures regarding heart failure (HF) is identifying patients early in the course of their hospitalization so that multidisciplinary education and clinical interventions can be implemented. We describe the accuracy of using an electronic pharmacy‐based strategy to identify hospitalized patients likely to have a principal diagnosis of HF at discharge.

METHODS

We evaluated 2 strategies. The first used the receipt of an intravenous loop diuretic as a single predictor; the second incorporated additional lab, pharmacy, and demographic information in a multivariable general estimating equation.

RESULTS

Receipt of an intravenous diuretic predicted a discharge diagnosis of heart failure with a sensitivity of 0.89 and a specificity of 0.87. Adding age, B‐type natriuretic peptide level, previous hospitalizations, attending physician specialty, and receipt of spironolactone into the predictor improved the sensitivity to 0.91 and the specificity to 0.89.

CONCLUSIONS

The receipt of intravenous loop diuretics is a reasonable and easily implemented screening test to identify patients likely to have a principal diagnosis of heart failure at discharge. The accuracy is improved by incorporating other electronically available variables. Journal of Hospital Medicine 2007;2:409–414. © 2007 Society of Hospital Medicine.

Copyright © 2007 Society of Hospital Medicine

There has been increasing emphasis on the development and successful execution of disease management strategies to improve the delivery of evidence‐based care for hospitalized patients with heart failure.14 Current care is woefully suboptimal for heart failure patients. Fonorow et al. describe the significant gap in performance on the Joint Commission for the Accreditation of Healthcare Organizations (JCAHO) heart failure core measures, with the median rate of conformity with all measures at only 24% nationally.5 For a variety of clinical and external factors, such as publicly reported quality measures and pay‐for‐performance incentives, institutions are increasingly motivated to identify patients who will make up the denominator of the heart failure metrics. At first glance, system‐level identification of heart failure patients may not seem critical to the delivery of evidence‐based care, but given that the management of patients with heart failure is multidisciplinary, it is critical that patients who have heart failure be clearly identified for all members of the care team. The capability of prospectively identifying inpatients with the principal diagnosis of heart failure is an essential first step in the implementation of performance improvement programs.

Successful interventions have included a multidisciplinary intervention with postdischarge follow‐up.6 However, the interventions described do not fully indicate how patients with heart failure are identified while in the hospital, so those interventions may be difficult to replicate in other settings. It has not been easy to identify these patients in a timely fashion given that a chief complaint of shortness of breath can indicate other clinical conditions in addition to heart failure. Previous studies have used an admission diagnosis of heart failure or suggestive chest X‐ray findings to trigger a clinical evaluation.7, 8 However, the sensitivity and specificity of these case‐finding methods have not been reported. Furthermore, patients presenting with shortness of breath may not have a diagnosis established until a series of diagnostic and therapeutic maneuvers have been performed. The challenge of promoting physician provision and documentation of evidence‐based care is compounded by these patients usually not being housed in a single geographical unit, possibly being attended by any number of medical specialties, and often having short lengths of stay. Given the multiple factors contributing to the complexity of identifying patients hospitalized with heart failure, it is important to delineate case‐finding strategies that efficiently and effectively identify heart failure patients so that clinical care and self‐management interventions are optimized.

With this goal in mind, we hypothesized that the receipt of intravenous loop diuretics may be an important trigger for identifying patients with heart failure. Receipt of intravenous loop diuretics is ubiquitous in the management of decompensated systolic and diastolic heart failure. We compare 2 electronic pharmacy‐based strategies in a tertiary‐care community teaching hospital to identify hospitalized patients early in their stay who were likely to be discharged with a principal diagnosis of heart failure (HF).

METHODS

Study Setting

The study was conducted in a 487‐bed not‐for‐profit community hospital in southeastern Michigan. The organization's institutional review board for all studies involving human subjects approved the study. In this hospital, heart failure patients are geographically dispersed throughout the institution, but all patient care orders are entered in a computerized provider order entry system. Approximately 70% of heart failure patients are admitted to the general medicine service, where care is directed by 3 types of attending physicians (academic hospitalists, private‐practice hospitalists, and community physicians, as previously described),9 14% are on the cardiology service, and the remainder are distributed among the surgical and intensive care unit services. The accuracy of 2 case‐finding strategies was tested using data from a 2‐year period. The institution had 28,005 adult hospitalizations during the prediction development period, July 1, 2003, to June 30, 2004, and 28,297 adult hospitalizations during the prediction testing period, July 1, 2004, to June 30, 2005. Receipt of intravenous loop diuretics had been used previously as a marker by the hospital's disease management program, but the accuracy of this strategy had not been tested.

Development of Prediction Algorithms

The outcome of interest was a principal diagnosis of HF, as assigned by medical records personnel after hospital discharge. This population corresponds to the denominator used to construct various performance measures. We evaluated 2 strategies for identifying targeted patients using information available prior to discharge. The first was the receipt of an intravenous loop diuretic at any point during the hospitalization (yes or no) as a single indicator. The second strategy used additional information to construct a multivariable predictor. Explanatory variables were considered for inclusion if they were available electronically, did not require additional manual retrieval or data entry, and had a clinical relationship with a diagnosis of HF. The variables selected were patient age, sex, receipt of intravenous loop diuretic, spironolactone, B‐type natriuretic peptide (BNP) level, serum creatinine, serum sodium, number of previous hospitalizations in the last 180 days with a principal discharge diagnosis of heart failure, and attending physician specialty. Cardiac ejection fraction was not included because the data were not available electronically.

Statistical Analysis

All analyses were performed using SAS version 9 (SAS Institute Inc., Cary NC). The data set was split chronologically into 2 sets each covering a 1‐year period in order to test the stability of the case‐finding strategies from one year to the next.

Initial model building for the multivariable strategy was done through logistic regression. Individual variables associated with heart failure (P < .05) were entered into a multivariable derivation model and retained if the main effect had a P value < .05. Sex and serum sodium were not included in the final model because of their high P values. To account for circumstances in which patients could have combinations of risk factors, interaction terms were also considered and were retained in the multivariable model at the same level of significance. The final parameter estimates for the derivation model were obtained from a generalized estimating equation (GEE) with an exchangeable working correlation structure to account for the possibility of multiple hospitalizations per year for a given patient. The z scores for the variables in the model provided insight into the relative importance of the factors associated with a heart failure diagnosis.

Laboratory values for the potential prediction variables were not available for every patient in our study; for example, a BNP level was obtained for only 7.6% of the study population. A simple strategy for addressing missing laboratory information was needed in order to derive a multivariable prediction model that could be used on a daily basis in a real‐world setting. We found that patients who had a BNP test drawn, regardless of the result, had a 27% chance of heart failure compared with those for whom BNP results were not available, whose chance of HF was 1%. Therefore, we could not simply impute the average BNP level for patients missing data on this parameter. Instead, we assumed the BNP levels of those not tested would be very low, and so gave these patients a BNP level of 1. Serum creatinine was not included in the multivariable model, despite its having a significant bivariate relationship with HF diagnosis, because valid imputation strategies for creatinine would be too complicated to implement in daily clinical practice.

The sensitivity and specificity of the single loop diuretic indicator was determined from a 2‐by‐2 table using data from the second study year. For both the multivariable and single loop diuretic approaches, test discrimination was evaluated by the c statistic from logistic regression.10 The calibration and overall performance of the multivariable derivation GEE model was assessed by a second GEE model run with the second‐year data set. For the testing model, the sole explanatory variable was a linear predictor derived from the covariate values of year 2 patients with the corresponding parameter estimates from the year 1 GEE. A well‐calibrated model with this configuration would be expected to have an intercept of 0 and a beta coefficient of the linear predictor of 1.11 Sensitivity, specificity, and positive predictive value were determined for 2 thresholds of predicted probability of heart failure, as derived from the linear predictor. If a subject's predicted probability at least equaled the threshold, then he or she would be considered to have tested positive for heart failure.

RESULTS

Salient features of the study population in the first and second study years are shown in Table 1. Mean age was approximately 59 years, and women made up 60% of the patients. The percentage of patients with a principal diagnosis of heart failure was 3% each year. Serum creatinine levels were available for 78% of patients in year 1 and 80% of patients in year 2. Serum BNP levels were available for 7.6% of patients in year 1 and 9% of patients in year 2.

Characteristics of Study Population
VariableYear 1 (n = 28,005)Year 2 (n = 28, 297)
No. with informationMean or percentNo. with informationMean or percentP value
  • Chi‐square test;

  • Wilcoxon rank sum test; HF, heart failure.

Age (years)28,00558.728,29758.9.36
HF principal diagnosis (%)28,0053.0%28,2973.1%.41*
Female (%)28,00560.6%28,29760.3%.48*
First BNP level obtained (pg/mL)2132813.72578766.5.83
First serum creatinine level obtained (mg/dL)21,8391.422,5961.4.54
Patient received IV loop diuretic (%)28,00516.3%28,29715.8%.07*
Patient received spironolactone (%)28,0052.8%28,2973.0%.08*
Number of previous hospitalizations with HF in preceding 180 days28,0052.428,2972.9.09

The parameter estimates and 95% confidence intervals of the main effects of the final prediction model are shown in Table 2, with interaction terms noted in the footnote. Examination of the z scores (available from the authors) indicated that by far the most influential risk factor in the multivariable model was receipt of intravenous diuretics, with receipt of spironolactone a very distant second. The probability that a given patient had heart failure increased with the number of risk factors present and the magnitude of their parameter estimates. For example, an older patient who had been hospitalized with heart failure previously and who was currently receiving intravenous diuretics and spironolactone would be more likely to have a principal diagnosis of heart failure than would an older patient receiving intravenous diuretics who had no other risk factors. However, the interaction terms with negative values (see the footnote in Table 2) indicate that certain combinations of risk factors convey a level of risk somewhat less than the sum of their parts.

Multivariable Generalized Estimating Equation from Year 1 Data (All Main Effects P < .0001)
EstimateStandard errorLower 95% CLUpper 95% CL
  • Interaction terms and their estimates: centered logged BNP squared, 0.03; squared count of prior hospitalizations, 1.15; cubed count of prior hospitalizations, 0.22; interaction of age and logged BNP, 0.06; interaction of logged BNP and IV loop diuretic, 0.14; interaction of age and count of prior hospitalizations, 0.16; interaction of logged BNP and spironolactone, 0.12; interaction of logged BNP and physician specialty, 0.29; interaction of logged BNP and count of previous hospitalizations, 0.09. CL, confidence limit.

Intercept8.280.409.067.50
Centered age in 10‐year increments0.310.050.220.40
Receipt of IV loop diuretic2.720.152.423.01
Receipt of spironolactone1.530.191.161.90
Centered logged BNP0.680.110.470.89
Attending physician specialty2.460.411.663.26
Count of hospitalizations for heart failure in previous 180 days2.430.481.483.37

The identification strategies performed well from one year to the next, as summarized in Table 3. Receipt of intravenous loop diuretics had a strong association with diagnosis of heart failure (OR 51.6, 95% CI 41.7, 63.7, P < .0001), with a c statistic of 0.88, a sensitivity of 0.89, and a specificity of 0.87.

Performance Characteristics of Case‐Finding Strategies Applied in Year 2 (28,297 Hospitalizations, of Which 890 had Principal Discharge Diagnosis of Heart Failure)
StrategyTPs of possible 890 HF cases (n)Sensitivity (# TPs/890)FPs (n)TNs of possible 27,407 hospitalizations without HF principal diagnosis (n)Specificity (# TNs/27,40)Positive predictive value (# TPs/all positives)Likelihood ratio (TP/FP)
  • HF, heart failure; TP, true positive; FP, false positive; TN, true negative.

Use receipt of IV loop diuretic7910.89367623,7310.870.186.6
Use predicted probability of heart failure (per multivariable model) 0.028330.94385923,5480.860.186.6
Use predicted probability of heart failure (per multivariable model) 0.048080.91304524,3620.890.218.2

The linear predictor of the multivariable prediction model as described in the Methods section also performed well in year 2 with excellent discrimination (c statistic of 0.96). Calibration was also excellent, as demonstrated by an intercept of 0.03 (standard error 0.05) and a beta coefficient of 1.02 (SE 0.03). If the threshold for identifying potential heart failure cases was defined as a predicted probability of at least 0.02, then the sensitivity of the multivariable predictor was 0.94 and the specificity was 0.86. If the positivity threshold was raised to 0.04, then the predictor's sensitivity dropped slightly, to 0.91, but specificity increased to 0.89.

The principal diagnoses of the 3045 patients in year 2 who were incorrectly predicted as having a principal diagnosis of heart failure (ie, false positives) were cardiac related (1026 of 3045; 34%), pulmonary related (685 of 3045; 22%), and renal‐ or fluid electrolyte related (117 of 3045; 4%), as determined using the multivariable approach with a 0.04 positivity threshold.

DISCUSSION

Identification of patients with heart failure early in their hospitalization is critical for successfully implementing disease management programs targeted at optimizing evidenced‐based care. Furthermore, public reporting of performance measures has increased the scrutiny of care delivered to those having this principal diagnosis. We developed a strategy that used the receipt of intravenous diuretics as a trigger of further clinical evaluation. We subsequently tested the value of other electronically available indicators to improve the sensitivity and specificity of the case‐finding strategy.

The receipt of an intravenous loop diuretic alone had a sensitivity of .89 and a specificity of .87. Incorporation of the additional information available to us electronically improved the sensitivity to .91 and the specificity to .89 (using a positivity threshold of 0.04), although these might be slightly different if BNP levels had been available for more patients. As with all diagnostic testing, there is a trade‐off between improved sensitivity and decreased specificity. At first glance, the resulting number of false positives generated by either prediction strategy may appear problematic. Although a formal cost‐effectiveness analysis of our case‐finding strategies is beyond the scope of this article, the cost of a false positive in this scenario is likely to be small.

For example in our hospital, clinical pharmacists place a reminder on the charts of patients who appear to have heart failure in order to prompt the clinical team to provide the recommended care processes. A list of inpatients treated with an IV diuretic is generated daily. A clinical pharmacist then reviews identified patient medical records to determine whether the diuretic was ordered for heart failure management or for some other purpose. This review consists of reading the completed history and physical and/or progress notes. On average, each medical record review takes 60 seconds to complete, with a range of 30‐90 seconds. At this speed, roughly 3000 minutes per year (or approximately 1 hour per week) would be spent reviewing the medical records of patients who would not have a principal diagnosis of heart failure. Nevertheless, we found that at least one‐third of the nominal false positives (multivariable rule, threshold of 0.04) still had a cardiac‐related diagnosis. Many of these had heart failure as a secondary diagnosis, but other diagnoses such as acute myocardial infarction took precedence in coding algorithms that assigned the principal diagnosis at discharge. Such patients might still benefit from the interventions and so are not truly false positives.

Patients with heart failure missed by this strategy included patients admitted for placement of pacemakers and/or defibrillators. Patients in this specialized population always had a single team managing their care, so the clinical and educational interventions were integrated into that team's daily work flow. Patients on dialysis with volume overload were not identified using this algorithm and constituted a very small number of patients in our heart failure population. Patients with stable heart failure on oral diuretics were not the focus of this case‐finding strategy and became a target for further intervention only if their heart failure worsened and required intravenous diuretics while they were hospitalized.

The identification of predictors for heart failure has allowed us not only to more effectively identify and risk‐stratify patients with heart failure but also to integrate the case‐finding strategy into clinical care delivery. This approach may also be relevant in hospitals that do not have computerized provider order entry (CPOE) systems but may be able to implement this case‐finding strategy by simply requesting a daily report of patients prescribed intravenous diuretics. As more institutions move to adopt CPOE platforms and clinical information such as ejection fractions become available, the predictors studied here may be augmented to form more sophisticated clinical rules and alerts.

Our study had several limitations. Although we validated our predictors in a separate cohort of patients, this is a single‐site study and may not be representative of the diverse institutions that care for patients with heart failure. There may also be interinstitutional differences in how a principal diagnosis of heart failure is assigned. We have demonstrated the stability of locally derived predictors from one year to the next but cannot make claims about how well our parameter estimates would perform in other settings. Finally, the complexity of the multivariable predictor requires an integrated database and computer application of a formula that may not be commonly available elsewhere at this time.

If disease management strategies are to be successful, early identification of at‐risk patients is crucial for both clinical care delivery and patient education regarding self‐management. The strategies tested here may be useful for other community‐based institutions whose care of heart failure patients is decentralized and involves multiple clinicians.

References

   Comments ()