Original Research

Development, validation, and results of a risk‐standardized measure of hospital 30‐day mortality for patients with exacerbation of chronic obstructive pulmonary disease

Abstract

BACKGROUND

Despite its large clinical and economic significance, measuring and improving the outcomes of patients hospitalized for chronic obstructive pulmonary disease (COPD) is only beginning to emerge as a national priority for policy makers and payers.

OBJECTIVE

To facilitate the public reporting of hospital outcomes, we developed a risk‐standardized measure of hospital 30‐day mortality for patients admitted with exacerbation of COPD.

DESIGN

Hierarchical logistic regression model.

SETTING/PATIENTS

Medicare Part A and Part B claims in a random sample of half of all admissions for patients admitted to acute care hospitals in 2008 (development cohort) and remaining 2008 admissions (validation cohort). We also assessed model performance and predictive ability in 2007 and 2009 data.

MEASUREMENTS

Hospital risk‐standardized 30‐day mortality rates.

RESULTS

The model development sample consisted of 150,035 admissions at 4537 nonfederal acute care US hospitals, with a mean unadjusted hospital 30‐day mortality rate of 8.6%. The mean risk‐standardized mortality rate was 8.6% and ranged from 5.9% to 13.5%. The development and validation models had good discrimination (areas under the receiver operating characteristic curve 0.72 and 0.72, respectively) and predictive ability (predicted mortality at the 1st and 10th deciles 1.5%, 23.7%, and 1.6%, 23.8%, respectively) and showed no evidence of over‐fitting.

CONCLUSIONS

A 30‐day mortality model based on administrative claims had similar discrimination to other public reporting models and can be used to compare risk‐adjusted outcomes for patients with exacerbations of COPD and to track changes in outcomes over time. The high mortality and variation in rates across institutions suggest opportunities to improve quality of care. Journal of Hospital Medicine 2013;8:428–435. © 2013 Society of Hospital Medicine

Copyright © 2013 Society of Hospital Medicine

Chronic obstructive pulmonary disease (COPD) affects as many as 24 million individuals in the United States, is responsible for more than 700,000 annual hospital admissions, and is currently the nation's third leading cause of death, accounting for nearly $49.9 billion in medical spending in 2010.[1, 2] Reported in‐hospital mortality rates for patients hospitalized for exacerbations of COPD range from 2% to 5%.[3, 4, 5, 6, 7] Information about 30‐day mortality rates following hospitalization for COPD is more limited; however, international studies suggest that rates range from 3% to 9%,[8, 9] and 90‐day mortality rates exceed 15%.[10]

Despite this significant clinical and economic impact, there have been no large‐scale, sustained efforts to measure the quality or outcomes of hospital care for patients with COPD in the United States. What little is known about the treatment of patients with COPD suggests widespread opportunities to increase adherence to guideline‐recommended therapies, to reduce the use of ineffective treatments and tests, and to address variation in care across institutions.[5, 11, 12]

Public reporting of hospital performance is a key strategy for improving the quality and safety of hospital care, both in the United States and internationally.[13] Since 2007, the Centers for Medicare and Medicaid Services (CMS) has reported hospital mortality rates on the Hospital Compare Web site, and COPD is 1 of the conditions highlighted in the Affordable Care Act for future consideration.[14] Such initiatives rely on validated, risk‐adjusted performance measures for comparisons across institutions and to enable outcomes to be tracked over time. We present the development, validation, and results of a model intended for public reporting of risk‐standardized mortality rates for patients hospitalized with exacerbations of COPD that has been endorsed by the National Quality Forum.[15]

METHODS

Approach to Measure Development

We developed this measure in accordance with guidelines described by the National Quality Forum,[16] CMS' Measure Management System,[17] and the American Heart Association scientific statement, Standards for Statistical Models Used for Public Reporting of Health Outcomes.[18] Throughout the process we obtained expert clinical and stakeholder input through meetings with a clinical advisory group and a national technical expert panel (see Acknowledgments). Last, we presented the proposed measure specifications and a summary of the technical expert panel discussions online and made a widely distributed call for public comments. We took the comments into consideration during the final stages of measure development (available at https://www.cms.gov/MMS/17_CallforPublicComment.asp).

Data Sources

We used claims data from Medicare inpatient, outpatient, and carrier (physician) Standard Analytic Files from 2008 to develop and validate the model, and examined model reliability using data from 2007 and 2009. The Medicare enrollment database was used to determine Medicare Fee‐for‐Service enrollment and mortality.

Study Cohort

Admissions were considered eligible for inclusion if the patient was 65 years or older, was admitted to a nonfederal acute care hospital in the United States, and had a principal diagnosis of COPD or a principal diagnosis of acute respiratory failure or respiratory arrest when paired with a secondary diagnosis of COPD with exacerbation (Table 1).

ICD‐9‐CM Codes Used to Define the Measure Cohort
ICD‐9‐CMDescription
  • NOTE: Abbreviations: COPD, chronic obstructive pulmonary disease; ICD‐9‐CM, International Classification of Diseases, 9th Revision, Clinical Modification; NOS, not otherwise specified.

  • Principal diagnosis when combined with a secondary diagnosis of acute exacerbation of COPD (491.21, 491.22, 493.21, or 493.22)

491.21Obstructive chronic bronchitis; with (acute) exacerbation; acute exacerbation of COPD, decompensated COPD, decompensated COPD with exacerbation
491.22Obstructive chronic bronchitis; with acute bronchitis
491.8Other chronic bronchitis; chronic: tracheitis, tracheobronchitis
491.9Unspecified chronic bronchitis
492.8Other emphysema; emphysema (lung or pulmonary): NOS, centriacinar, centrilobular, obstructive, panacinar, panlobular, unilateral, vesicular; MacLeod's syndrome; Swyer‐James syndrome; unilateral hyperlucent lung
493.20Chronic obstructive asthma; asthma with COPD, chronic asthmatic bronchitis, unspecified
493.21Chronic obstructive asthma; asthma with COPD, chronic asthmatic bronchitis, with status asthmaticus
493.22Chronic obstructive asthma; asthma with COPD, chronic asthmatic bronchitis, with (acute) exacerbation
496Chronic: nonspecific lung disease, obstructive lung disease, obstructive pulmonary disease (COPD) NOS. (Note: This code is not to be used with any code from categories 491493.)
518.81aOther diseases of lung; acute respiratory failure; respiratory failure NOS
518.82aOther diseases of lung; acute respiratory failure; other pulmonary insufficiency, acute respiratory distress
518.84aOther diseases of lung; acute respiratory failure; acute and chronic respiratory failure
799.1aOther ill‐defined and unknown causes of morbidity and mortality; respiratory arrest, cardiorespiratory failure

If a patient was discharged and readmitted to a second hospital on the same or the next day, we combined the 2 acute care admissions into a single episode of care and assigned the mortality outcome to the first admitting hospital. We excluded admissions for patients who were enrolled in Medicare Hospice in the 12 months prior to or on the first day of the index hospitalization. An index admission was any eligible admission assessed in the measure for the outcome. We also excluded admissions for patients who were discharged against medical advice, those for whom vital status at 30 days was unknown or recorded inconsistently, and patients with unreliable data (eg, age >115 years). For patients with multiple hospitalizations during a single year, we randomly selected 1 admission per patient to avoid survival bias. Finally, to assure adequate risk adjustment we limited the analysis to patients who had continuous enrollment in Medicare Fee‐for‐Service Parts A and B for the 12 months prior to their index admission so that we could identify comorbid conditions coded during all prior encounters.

Outcomes

The outcome of 30‐day mortality was defined as death from any cause within 30 days of the admission date for the index hospitalization. Mortality was assessed at 30 days to standardize the period of outcome ascertainment,[19] and because 30 days is a clinically meaningful time frame, during which differences in the quality of hospital care may be revealed.

Risk‐Adjustment Variables

We randomly selected half of all COPD admissions in 2008 that met the inclusion and exclusion criteria to create a model development sample. Candidate variables for inclusion in the risk‐standardized model were selected by a clinician team from diagnostic groups included in the Hierarchical Condition Category clinical classification system[20] and included age and comorbid conditions. Sleep apnea (International Classification of Diseases, 9th Revision, Clinical Modification [ICD‐9‐CM] condition codes 327.20, 327.21, 327.23, 327.27, 327.29, 780.51, 780.53, and 780.57) and mechanical ventilation (ICD‐9‐CM procedure codes 93.90, 96.70, 96.71, and 96.72) were also included as candidate variables.

We defined a condition as present for a given patient if it was coded in the inpatient, outpatient, or physician claims data sources in the preceding 12 months, including the index admission. Because a subset of the condition category variables can represent a complication of care, we did not consider them to be risk factors if they appeared only as secondary diagnosis codes for the index admission and not in claims submitted during the prior year.

We selected final variables for inclusion in the risk‐standardized model based on clinical considerations and a modified approach to stepwise logistic regression. The final patient‐level risk‐adjustment model included 42 variables (Table 2).

Adjusted OR for Model Risk Factors and Mortality in Development Sample (Hierarchical Logistic Regression Model)
VariableDevelopment Sample (150,035 Admissions at 4537 Hospitals)Validation Sample (149,646 Admissions at 4535 Hospitals)
Frequency, %OR95% CIFrequency, %OR95% CI
  • NOTE: Abbreviations: CI, confidence interval; DM, diabetes mellitus; ICD‐9‐CM, International Classification of Diseases, 9th Revision, Clinical Modification; OR, odds ratio; CC, condition category.

  • Indicates variable forced into the model.

Demographics
Age 65 years (continuous) 1.031.03‐1.04 1.031.03‐1.04
Cardiovascular/respiratory
Sleep apnea (ICD‐9‐CM: 327.20, 327.21, 327.23, 327.27, 327.29, 780.51, 780.53, 780.57)a9.570.870.81‐0.949.720.840.78‐0.90
History of mechanical ventilation (ICD‐9‐CM: 93.90, 96.70, 96.71, 96.72)a6.001.191.11‐1.276.001.151.08‐1.24
Respirator dependence/respiratory failure (CC 7778)a1.150.890.77‐1.021.200.780.68‐0.91
Cardiorespiratory failure and shock (CC 79)26.351.601.53‐1.6826.341.591.52‐1.66
Congestive heart failure (CC 80)41.501.341.28‐1.3941.391.311.25‐1.36
Chronic atherosclerosis (CC 8384)a50.440.870.83‐0.9050.120.910.87‐0.94
Arrhythmias (CC 9293)37.151.171.12‐1.2237.061.151.10‐1.20
Vascular or circulatory disease (CC 104106)38.201.091.05‐1.1438.091.020.98‐1.06
Fibrosis of lung and other chronic lung disorder (CC 109)16.961.081.03‐1.1317.081.111.06‐1.17
Asthma (CC 110)17.050.670.63‐0.7016.900.670.63‐0.70
Pneumonia (CC 111113)49.461.291.24‐1.3549.411.271.22‐1.33
Pleural effusion/pneumothorax (CC 114)11.781.171.11‐1.2311.541.181.12‐1.25
Other lung disorders (CC 115)53.070.800.77‐0.8353.170.830.80‐0.87
Other comorbid conditions
Metastatic cancer and acute leukemia (CC 7)2.762.342.14‐2.562.792.151.97‐2.35
Lung, upper digestive tract, and other severe cancers (CC 8)a5.981.801.68‐1.926.021.981.85‐2.11
Lymphatic, head and neck, brain, and other major cancers; breast, prostate, colorectal and other cancers and tumors; other respiratory and heart neoplasms (CC 911)14.131.030.97‐1.0814.191.010.95‐1.06
Other digestive and urinary neoplasms (CC 12)6.910.910.84‐0.987.050.850.79‐0.92
Diabetes and DM complications (CC 1520, 119120)38.310.910.87‐0.9438.290.910.87‐0.94
Protein‐calorie malnutrition (CC 21)7.402.182.07‐2.307.442.091.98‐2.20
Disorders of fluid/electrolyte/acid‐base (CC 2223)32.051.131.08‐1.1832.161.241.19‐1.30
Other endocrine/metabolic/nutritional disorders (CC 24)67.990.750.72‐0.7867.880.760.73‐0.79
Other gastrointestinal disorders (CC 36)56.210.810.78‐0.8456.180.780.75‐0.81
Osteoarthritis of hip or knee (CC 40)9.320.740.69‐0.799.330.800.74‐0.85
Other musculoskeletal and connective tissue disorders (CC 43)64.140.830.80‐0.8664.200.830.80‐0.87
Iron deficiency and other/unspecified anemias and blood disease (CC 47)40.801.081.04‐1.1240.721.081.04‐1.13
Dementia and senility (CC 4950)17.061.091.04‐1.1416.971.091.04‐1.15
Drug/alcohol abuse, without dependence (CC 53)a23.510.780.75‐0.8223.380.760.72‐0.80
Other psychiatric disorders (CC 60)a16.491.121.07‐1.1816.431.121.06‐1.17
Quadriplegia, paraplegia, functional disability (CC 6769, 100102, 177178)4.921.030.95‐1.124.921.080.99‐1.17
Mononeuropathy, other neurological conditions/emnjuries (CC 76)11.350.850.80‐0.9111.280.880.83‐0.93
Hypertension and hypertensive disease (CC 9091)80.400.780.75‐0.8280.350.790.75‐0.83
Stroke (CC 9596)a6.771.000.93‐1.086.730.980.91‐1.05
Retinal disorders, except detachment and vascular retinopathies (CC 121)10.790.870.82‐0.9310.690.900.85‐0.96
Other eye disorders (CC 124)a19.050.900.86‐0.9519.130.980.85‐0.93
Other ear, nose, throat, and mouth disorders (CC 127)35.210.830.80‐0.8735.020.800.77‐0.83
Renal failure (CC 131)a17.921.121.07‐1.1818.161.131.08‐1.19
Decubitus ulcer or chronic skin ulcer (CC 148149)7.421.271.19‐1.357.421.331.25‐1.42
Other dermatological disorders (CC 153)28.460.900.87‐0.9428.320.890.86‐0.93
Trauma (CC 154156, 158161)9.041.091.03‐1.168.991.151.08‐1.22
Vertebral fractures (CC 157)5.011.331.24‐1.444.971.291.20‐1.39
Major complications of medical care and trauma (CC 164)5.470.810.75‐0.885.550.820.76‐0.89

Model Derivation

We used hierarchical logistic regression models to model the log‐odds of mortality as a function of patient‐level clinical characteristics and a random hospital‐level intercept. At the patient level, each model adjusts the log‐odds of mortality for age and the selected clinical covariates. The second level models the hospital‐specific intercepts as arising from a normal distribution. The hospital intercept represents the underlying risk of mortality, after accounting for patient risk. If there were no differences among hospitals, then after adjusting for patient risk, the hospital intercepts should be identical across all hospitals.

Estimation of Hospital Risk‐Standardized Mortality Rate

We calculated a risk‐standardized mortality rate, defined as the ratio of predicted to expected deaths (similar to observed‐to‐expected), multiplied by the national unadjusted mortality rate.[21] The expected number of deaths for each hospital was estimated by applying the estimated regression coefficients to the characteristics of each hospital's patients, adding the average of the hospital‐specific intercepts, transforming the data by using an inverse logit function, and summing the data from all patients in the hospital to obtain the count. The predicted number of deaths was calculated in the same way, substituting the hospital‐specific intercept for the average hospital‐specific intercept.

Model Performance, Validation, and Reliability Testing

We used the remaining admissions in 2008 as the model validation sample. We computed several summary statistics to assess the patient‐level model performance in both the development and validation samples,[22] including over‐fitting indices, predictive ability, area under the receiver operating characteristic (ROC) curve, distribution of residuals, and model 2. In addition, we assessed face validity through a survey of members of the technical expert panel. To assess reliability of the model across data years, we repeated the modeling process using qualifying COPD admissions in both 2007 and 2009. Finally, to assess generalizability we evaluated the model's performance in an all‐payer sample of data from patients admitted to California hospitals in 2006.

Analyses were conducted using SAS version 9.1.3 (SAS Institute Inc., Cary, NC). We estimated the hierarchical models using the GLIMMIX procedure in SAS.

The Human Investigation Committee at the Yale University School of Medicine/Yale New Haven Hospital approved an exemption (HIC#0903004927) for the authors to use CMS claims and enrollment data for research analyses and publication.

RESULTS

Model Derivation

After exclusions were applied, the development sample included 150,035 admissions in 2008 at 4537 US hospitals (Figure 1). Factors that were most strongly associated with the risk of mortality included metastatic cancer (odds ratio [OR] 2.34), protein calorie malnutrition (OR 2.18), nonmetastatic cancers of the lung and upper digestive tract, (OR 1.80) cardiorespiratory failure and shock (OR 1.60), and congestive heart failure (OR 1.34) (Table 2).

Figure 1

Model development and validation samples. Abbreviations: COPD, chronic obstructive pulmonary disease; FFS, Fee‐for‐Service. Exclusion categories are not mutually exclusive.

Model Performance, Validation, and Reliability

The model had a C statistic of 0.72, indicating good discrimination, and predicted mortality in the development sample ranged from 1.52% in the lowest decile to 23.74% in the highest. The model validation sample, using the remaining cases from 2008, included 149,646 admissions from 4535 hospitals. Variable frequencies and ORs were similar in both samples (Table 2). Model performance was also similar in the validation samples, with good model discrimination and fit (Table 3). Ten of 12 technical expert panel members responded to the survey, of whom 90% at least somewhat agreed with the statement, the COPD mortality measure provides an accurate reflection of quality. When the model was applied to patients age 18 years and older in the 2006 California Patient Discharge Data, overall discrimination was good (C statistic, 0.74), including in those age 18 to 64 years (C statistic, 0.75; 65 and above C statistic, 0.70).

Model Performance in Development and Validation Samples
DevelopmentValidationData Years
IndicesSample, 2008Sample, 200820072009
  • NOTE: Abbreviations: ROC, receiver operating characteristic; SD, standard deviation. Over‐fitting indices (0, 1) provide evidence of over‐fitting and require several steps to calculate. Let b denote the estimated vector of regression coefficients. Predicted probabilities (p^)=1/(1+exp{Xb}), and Z=Xb (eg, the linear predictor that is a scalar value for everyone). A new logistic regression model that includes only an intercept and a slope by regressing the logits on Z is fitted in the validation sample (eg, Logit(P(Y=1|Z))=0+1Z. Estimated values of 0 far from 0 and estimated values of 1 far from 1 provide evidence of over‐fitting.

Number of admissions150,035149,646259,911279,377
Number of hospitals4537453546364571
Mean risk‐standardized mortality rate, % (SD)8.62 (0.94)8.64 (1.07)8.97 (1.12)8.08 (1.09)
Calibration, 0, 10.034, 0.9850.009, 1.0040.095, 1.0220.120, 0.981
Discriminationpredictive ability, lowest decile %highest decile %1.5223.741.6023.781.5424.641.4222.36
Discriminationarea under the ROC curve, C statistic0.7200.7230.7280.722
Residuals lack of fit, Pearson residual fall %
20000
2, 091.1491.491.0891.93
0, 21.661.71.961.42
2+6.936.916.966.65
Model Wald 2 (number of covariates)6982.11 (42)7051.50 (42)13042.35 (42)12542.15 (42)
P value<0.0001<0.0001<0.0001<0.0001
Between‐hospital variance, (standard error)0.067 (0.008)0.078 (0.009)0.067 (0.006)0.072 (0.006)

Reliability testing demonstrated consistent performance over several years. The frequency and ORs of the variables included in the model showed only minor changes over time. The area under the ROC curve (C statistic) was 0.73 for the model in the 2007 sample and 0.72 for the model using 2009 data (Table 3).

Hospital Risk‐Standardized Mortality Rates

The mean unadjusted hospital 30‐day mortality rate was 8.6% and ranged from 0% to 100% (Figure 2a). Risk‐standardized mortality rates varied across hospitals (Figure 2b). The mean risk‐standardized mortality rate was 8.6% and ranged from 5.9% to 13.5%. The odds of mortality at a hospital 1 standard deviation above average was 1.20 times that of a hospital 1 standard deviation below average.

Figure 2

(a) Distribution of hospital‐level 30‐day mortality rates and (b) hospital‐level 30‐day risk‐standardized mortality rates (2008 development sample; n = 150,035 admissions from 4537 hospitals). Abbreviations: COPD, chronic obstructive pulmonary disease.

DISCUSSION

We present a hospital‐level risk‐standardized mortality measure for patients admitted with COPD based on administrative claims data that are intended for public reporting and that have achieved endorsement by the National Quality Forum, a voluntary consensus standards‐setting organization. Across more than 4500 US hospitals, the mean 30‐day risk‐standardized mortality rate in 2008 was 8.6%, and we observed considerable variation across institutions, despite adjustment for case mix, suggesting that improvement by lower‐performing institutions may be an achievable goal.

Although improving the delivery of evidence‐based care processes and outcomes of patients with acute myocardial infarction, heart failure, and pneumonia has been the focus of national quality improvement efforts for more than a decade, COPD has largely been overlooked.[23] Within this context, this analysis represents the first attempt to systematically measure, at the hospital level, 30‐day all‐cause mortality for patients admitted to US hospitals for exacerbation of COPD. The model we have developed and validated is intended to be used to compare the performance of hospitals while controlling for differences in the pretreatment risk of mortality of patients and accounting for the clustering of patients within hospitals, and will facilitate surveillance of hospital‐level risk‐adjusted outcomes over time.

In contrast to process‐based measures of quality, such as the percentage of patients with pneumonia who receive appropriate antibiotic therapy, performance measures based on patient outcomes provide a more comprehensive view of care and are more consistent with patients' goals.[24] Additionally, it is well established that hospital performance on individual and composite process measures explains only a small amount of the observed variation in patient outcomes between institutions.[25] In this regard, outcome measures incorporate important, but difficult to measure aspects of care, such as diagnostic accuracy and timing, communication and teamwork, the recognition and response to complications, care coordination at the time of transfers between levels of care, and care settings. Nevertheless, when used for making inferences about the quality of hospital care, individual measures such as the risk‐standardized hospital mortality rate should be interpreted in the context of other performance measures, including readmission, patient experience, and costs of care.

A number of prior investigators have described the outcomes of care for patients hospitalized with exacerbations of COPD, including identifying risk factors for mortality. Patil et al. carried out an analysis of the 1996 Nationwide Inpatient Sample and described an overall in‐hospital mortality rate of 2.5% among patients with COPD, and reported that a multivariable model containing sociodemographic characteristics about the patient and comorbidities had an area under the ROC curve of 0.70.[3] In contrast, this hospital‐level measure includes patients with a principal diagnosis of respiratory failure and focuses on 30‐day rather than inpatient mortality, accounting for the nearly 3‐fold higher mortality rate we observed. In a more recent study that used clinical from a large multistate database, Tabak et al. developed a prediction model for inpatient mortality for patients with COPD that contained only 4 factors: age, blood urea nitrogen, mental status, and pulse, and achieved an area under the ROC curve of 0.72.[4] The simplicity of such a model and its reliance on clinical measurements makes it particularly well suited for bedside application by clinicians, but less valuable for large‐scale public reporting programs that rely on administrative data. In the only other study identified that focused on the assessment of hospital mortality rates, Agabiti et al. analyzed the outcomes of 12,756 patients hospitalized for exacerbations of COPD, using similar ICD‐9‐CM diagnostic criteria as in this study, at 21 hospitals in Rome, Italy.[26] They reported an average crude 30‐day mortality rate of 3.8% among a group of 5 benchmark hospitals and an average mortality of 7.5% (range, 5.2%17.2%) among the remaining institutions.

To put the variation we observed in mortality rates into a broader context, the relative difference in the risk‐standardized hospital mortality rates across the 10th to 90th percentiles of hospital performance was 25% for acute myocardial infarction and 39% for heart failure, whereas rates varied 30% for COPD, from 7.6% to 9.9%.[27] Model discrimination in COPD (C statistic, 0.72) was also similar to that reported for models used for public reporting of hospital mortality in acute myocardial infarction (C statistic, 0.71) and pneumonia (C statistic, 0.72).

This study has a number of important strengths. First, the model was developed from a large sample of recent Medicare claims, achieved good discrimination, and was validated in samples not limited to Medicare beneficiaries. Second, by including patients with a principal diagnosis of COPD, as well as those with a principal diagnosis of acute respiratory failure when accompanied by a secondary diagnosis of COPD with acute exacerbation, this model can be used to assess hospital performance across the full spectrum of disease severity. This broad set of ICD‐9‐CM codes used to define the cohort also ensures that efforts to measure hospital performance will be less influenced by differences in documentation and coding practices across hospitals relating to the diagnosis or sequencing of acute respiratory failure diagnoses. Moreover, the inclusion of patients with respiratory failure is important because these patients have the greatest risk of mortality, and are those in whom efforts to improve the quality and safety of care may have the greatest impact. Third, rather than relying solely on information documented during the index admission, we used ambulatory and inpatient claims from the full year prior to the index admission to identify comorbidities and to distinguish them from potential complications of care. Finally, we did not include factors such as hospital characteristics (eg, number of beds, teaching status) in the model. Although they might have improved overall predictive ability, the goal of the hospital mortality measure is to enable comparisons of mortality rates among hospitals while controlling for differences in patient characteristics. To the extent that factors such as size or teaching status might be independently associated with hospital outcomes, it would be inappropriate to adjust away their effects, because mortality risk should not be influenced by hospital characteristics other than through their effects on quality.

These results should be viewed in light of several limitations. First, we used ICD‐9‐CM codes derived from claims files to define the patient populations included in the measure rather than collecting clinical or physiologic information prospectively or through manual review of medical records, such as the forced expiratory volume in 1 second or whether the patient required long‐term oxygen therapy. Nevertheless, we included a broad set of potential diagnosis codes to capture the full spectrum of COPD exacerbations and to minimize differences in coding across hospitals. Second, because the risk‐adjustment included diagnoses coded in the year prior to the index admission, it is potentially subject to bias due to regional differences in medical care utilization that are not driven by underlying differences in patient illness.[28] Third, using administrative claims data, we observed some paradoxical associations in the model that are difficult to explain on clinical grounds, such as a protective effect of substance and alcohol abuse or prior episodes of respiratory failure. Fourth, although we excluded patients from the analysis who were enrolled in hospice prior to, or on the day of, the index admission, we did not exclude those who choose to withdraw support, transition to comfort measures only, or enrolled in hospice care during a hospitalization. We do not seek to penalize hospitals for being sensitive to the preferences of patients at the end of life. At the same time, it is equally important that the measure is capable of detecting the outcomes of suboptimal care that may in some instances lead a patient or their family to withdraw support or choose hospice. Finally, we did not have the opportunity to validate the model against a clinical registry of patients with COPD, because such data do not currently exist. Nevertheless, the use of claims as a surrogate for chart data for risk adjustment has been validated for several conditions, including acute myocardial infarction, heart failure, and pneumonia.[29, 30]

CONCLUSIONS

Risk‐standardized 30‐day mortality rates for Medicare beneficiaries with COPD vary across hospitals in the US. Calculating and reporting hospital outcomes using validated performance measures may catalyze quality improvement activities and lead to better outcomes. Additional research would be helpful to confirm that hospitals with lower mortality rates achieve care that meets the goals of patients and their families better than at hospitals with higher mortality rates.

Acknowledgment

The authors thank the following members of the technical expert panel: Darlene Bainbridge, RN, MS, NHA, CPHQ, CPHRM, President/CEO, Darlene D. Bainbridge & Associates, Inc.; Robert A. Balk, MD, Director of Pulmonary and Critical Care Medicine, Rush University Medical Center; Dale Bratzler, DO, MPH, President and CEO, Oklahoma Foundation for Medical Quality; Scott Cerreta, RRT, Director of Education, COPD Foundation; Gerard J. Criner, MD, Director of Temple Lung Center and Divisions of Pulmonary and Critical Care Medicine, Temple University; Guy D'Andrea, MBA, President, Discern Consulting; Jonathan Fine, MD, Director of Pulmonary Fellowship, Research and Medical Education, Norwalk Hospital; David Hopkins, MS, PhD, Senior Advisor, Pacific Business Group on Health; Fred Martin Jacobs, MD, JD, FACP, FCCP, FCLM, Executive Vice President and Director, Saint Barnabas Quality Institute; Natalie Napolitano, MPH, RRT‐NPS, Respiratory Therapist, Inova Fairfax Hospital; Russell Robbins, MD, MBA, Principal and Senior Clinical Consultant, Mercer. In addition, the authors acknowledge and thank Angela Merrill, Sandi Nelson, Marian Wrobel, and Eric Schone from Mathematica Policy Research, Inc., Sharon‐Lise T. Normand from Harvard Medical School, and Lein Han and Michael Rapp at The Centers for Medicare & Medicaid Services for their contributions to this work.

Disclosures

Peter K. Lindenauer, MD, MSc, is the guarantor of this article, taking responsibility for the integrity of the work as a whole, from inception to published article, and takes responsibility for the content of the manuscript, including the data and data analysis. All authors have made substantial contributions to the conception and design, or acquisition of data, or analysis and interpretation of data; have drafted the submitted article or revised it critically for important intellectual content; and have provided final approval of the version to be published. Preparation of this manuscript was completed under Contract Number: HHSM‐5002008‐0025I/HHSM‐500‐T0001, Modification No. 000007, Option Year 2 Measure Instrument Development and Support (MIDS). Sponsors did not contribute to the development of the research or manuscript. Dr. Au reports being an unpaid research consultant for Bosch Inc. He receives research funding from the NIH, Department of Veterans Affairs, AHRQ, and Gilead Sciences. The views of the this manuscript represent the authors and do not necessarily represent those of the Department of Veterans Affairs. Drs. Drye and Bernheim report receiving contract funding from CMS to develop and maintain quality measures.

References

Online-Only Materials

AttachmentSize
PDF icon Supplementary Information (1)88.91 KB
   Comments ()