Original Research

When do patient‐reported outcome measures inform readmission risk?



To characterize changes in patient‐reported outcome measures from hospital discharge to assess when they best inform risk of utilization as defined by readmissions or emergency department use.


Patients discharged from an urban safety‐net hospital.


Longitudinal cohort study.


We serially administered the Memorial Symptom Assessment Scale (MSAS) and the PROMIS Global Health short form assessing General Self‐Rated Health (GSRH), Global Physical (GPH), and Mental (GMH) Health at 0, 30, 90, and 180 days from hospital discharge. Time to first utilization from each survey was plotted by dichotomizing our sample on each patient‐reported measure, and equivalence of the time‐to‐event curves was assessed using the log‐rank test. Cox proportional hazard models were used to control for available covariates including prior utilization during the study, Charlson score, age, gender, and race/ethnicity. We assessed each measure's effect on the fit of the predictive models using the likelihood ratio test.


We recruited 196 patients, of whom 100%, 98%, 90%, and 88% completed each respective survey wave. Participants’ mean age was 52 years, 51% were women, 60% were non‐Hispanic black, and 21% completed the questionnaires in Spanish. In‐hospital assessments revealed high symptom burden and poor health status. In‐hospital assessments of GMH and GSRH predicted 14‐day reutilization, whereas posthospitalization assessments of MSAS and GPH predicted subsequent utilizations. Each measure selectively improved predictive model fit.


Routine measurement of patient‐reported outcomes can help identify patients at higher risk for utilizations. At different time points, MSAS, GPH, GMH, and GSRH all informed utilization risk. Journal of Hospital Medicine 2015;10:294–300. © 2015 Society of Hospital Medicine

© 2015 Society of Hospital Medicine

Despite widespread efforts to predict 30‐day rehospitalizations among discharged general medical patients,[1, 2, 3] not many strategies have incorporated patient‐reported outcome (PRO) measures in predictive models.[4] This despite the many longitudinal studies of the ambulatory population that demonstrate the higher likelihood of hospitalizations among those who score poorly on General Self‐Rated Health (GSRH),[5, 6, 7] baseline or declining Health‐Related Quality of Life,[8, 9, 10, 11, 12] psychological symptoms,[13, 14] and physical symptoms assessments.[15] One of the few existing studies that included PRO measures in 30‐day readmission models showed the predictive value of the 12‐item short form (SF12) Physical Component Score.[16] Others showed that persistent symptoms were associated with readmissions in patients with heart disease.[17, 18]

The paucity of efforts to connect PRO measures to utilization may be due to the limited availability of these measures in routine clinical records and the incomplete knowledge about how various PRO measures may fluctuate during episodes of acute illnesses and their treatments during hospitalizations. Health perception measures reflect both enduring features like self‐concept as well as dynamic features like a person's immediate health status.[19] As such, GSRH reflects the presence of chronic illnesses but is also responsive to acute events.[20, 21] Similarly, Health‐Related Quality of Life measures are dynamic as they decline around episodes of acute illness but are stable over a longer time window in their tendency to recover.[22] We do not know how fluctuations in measures of symptom burden, perceived health, and quality of life around the hospital‐to‐home transition may differentially inform readmission risk. Using a longitudinal cohort study, we addressed 2 questions: (1) How do PRO measures change when measured serially during the hospital‐to‐home transition? (2) How does the relative timing of each PRO measure variably inform the risk of subsequent utilization events including hospital readmissions?


We conducted a longitudinal cohort study using data originally collected for a trial (ClinicalTrials.gov Identifier NCT01391026) of an intervention that was shown to have no associations with variables evaluated in this study. Patients were recruited from the John H. Stroger Hospital of Cook County, an urban safety‐net hospital that serves 128 municipalities in northeastern Illinois including the City of Chicago. Patients were eligible if they (1) were admitted to the general medical wards, the medical intensive care unit, or the cardiac care unit between May 2011 and February 2012; (2) had a clinic appointment in the Hospital's general medicine clinic (GMC) in the prior 12 months to facilitate follow‐up; and (3) were able to communicate independently in English or Spanish. Randomly selected patients were approached during their hospitalization and consenting subjects completed an in‐person questionnaire on the day of discharge. Subjects were contacted by telephone around 30, 90, and 180 days thereafter to complete follow‐up questionnaires; we began calling patients around 2 weeks prior to the target day anticipating noncontact on the first attempts. All telephone interviews were conducted by research assistants who had no clinical training and who did not give care‐related advice to patients based on their survey response. A few patients whose follow‐up survey window straddled the date of a scheduled clinic appointment were invited to complete the questionnaire in the GMC's waiting area using computer kiosks enabled with audio computer‐assisted self‐interview technology described elsewhere.[23] The Charlson Comorbidity Index was calculated inclusive of diagnostic codes assigned over 3 months preceding the index hospitalization.[24]

The following instruments were administered at each interview. The physical symptom severity portion of the Memorial Symptom Assessment Scale (MSAS) solicited the severity rank (none/a little bit/somewhat/quite a bit/very much) of 17 physical symptoms in the last week; the score was calculated by averaging the severity rank of the 12 most common symptom in the sample.[25, 26] The Patient Reported Outcomes Measurement Information System (PROMIS) Global Health Short Form is an instrument assessing GSRH (1 item), Social Activities (1 item), Global Physical Health (4 items), and Global Mental Health (4 items including a single‐item quality‐of‐life measure). Fatigue and pain for Global Physical Health, and emotional health for Global Mental Health were assessed over the past 7 days. Each of the 2 Global Health scores was standardized to a national mean of 50 and standard deviation of 10.[27]

The rate of survey completion at each follow‐up was calculated. Characteristics of participants were tabulated. Characteristics of patients censored prior to study completion were compared with patients with complete data. Box plots for MSAS physical symptom severity, and Global Physical and Mental Health scores were constructed to illustrate the comparisons of the mean scores between each consecutive survey period using t tests assuming unequal variance. A similar box plot of GSRH illustrated the comparison of the median score between consecutive surveys using the rank sum test. Hospital‐based utilization events were defined as either an emergency department visit or hospitalization at 1 of the 2 hospitals of the Cook County Health & Hospitals System (CCHHS). After accounting for patient data censored due to death (date reported by family) or withdrawal from study, Kaplan‐Meier curves showing time to first hospital‐based utilization event during each interval between surveys were drawn separately for above‐ and below‐median MSAS, Global Physical and Mental Health scores, and for poor or fair versus good, very good, or excellent GSRH assessment. The null hypothesis that the survivor functions were equal between the better and worse median quantiles or GSRH categories was tested using the log‐rank test at 14 and 30 days from survey completion. Hazard ratios for time to first utilization event within 14 days of each survey were calculated for the MSAS score, Global Physical and Mental Health as continuous variables, and GSRH response categories relative to poor using bivariate and multivariate Cox proportional hazard equations. Multivariate models incorporated the following 5 covariates: at least 1 utilization event during the study period prior to the survey, Charlson score, age, gender, and race/ethnicity category. Likelihood ratio statistics were calculated to test the hypothesis that the model including the PRO measure and covariates predicted the outcome equally well compared to the nested model with only covariates. We used the traditional threshold of .05 when reporting significance. All analyses were performed in Stata 13 (StataCorp, College Station, TX). The methods for patient consent, data collection, analyses, and reporting were reviewed and approved by the CCHHS institutional review board.


A total of 196 patients completed the initial survey. The completion rates were 98%, 90%, and 88% for the 30‐, 90‐, and 180‐day follow‐up surveys, respectively. As shown in Table 1, participants average age was 52 years, and about half were women. The majority was non‐Hispanic black, and 21% preferred to complete the survey in Spanish. Diabetes, congestive heart failure, cancer, and chronic pulmonary disease were each prevalent in at least one‐fifth of our patient cohort. Demographic characteristics were similar between the 160 patients who completed all 3 follow‐up surveys and the 36 who missed at least 1 follow‐up survey. Among the latter group, 1 withdrew at 30 days, 1 withdrew and 4 had died at 90 days, and 1 withdrew and 9 had died at 180 days.

Participating Patient Characteristics (N=196)
  • NOTE: Abbreviations: SD, standard deviation.

Age, y, mean (SD)52 (10)
Female, n (%)100 (51)
Race/ethnicity category, n (%)
Non‐Hispanic black117 (60)
Hispanic52 (27)
Non‐Hispanic white20 (10)
Other6 (3)
Language, n (%)
English155 (79)
Spanish41 (21)
Charlson Comorbidity Index, median (range)1 (09)
Charlson comorbidities, n (%)
Diabetes71 (36)
Congestive heart failure52 (27)
Cancer (with and without metastases)43 (22)
Chronic pulmonary disease40 (20)
Myocardial infarction17 (9)
Renal disease11 (6)

Figure 1 shows a timeline of the follow‐up surveys and utilization events in the form of overlapping histograms. The majority of 30‐day follow‐up questionnaires were completed earlier than targeted, at a median of 17 (interquartile range [IQR] 16, 20) days after discharge. Similarly, questionnaires targeted for 90 and 180 days were completed at medians of 78 (IQR 7684) and 167 (IQR 166169) days from discharge. Fifty‐four (28%) patients experienced a first utilization event in the first 30 days following discharge. During the 60‐, 90‐, and 30‐day intervals after the first, second, and third follow‐up surveys, respectively, 63 (33%), 54 (31%), and 16 (9%) patients experienced a first utilization event.

Figure 1

Overlapping histogram showing the timeline of the study's follow‐up survey completion and first hospital‐based utilization events following each survey wave. All participants were surveyed in the hospital at time zero.

A significant improvement in MSAS physical symptom severity was detected between the hospitalization and the 30‐day follow‐up (Figure 2A). Although the mean Global Physical Health score was below the national mean of 50 at every survey period, a similar improvement in the measure was noted between the hospitalization and the 30‐day follow‐up (Figure 2B). The mean Global Mental Health score was also below the national mean but remained stable throughout the study (Figure 2C). The median GSRH was stable at 2 (IQR 23) at every survey wave (Figure 2D). Of note, compared to patients who completed all 3 follow‐up surveys, patients who missed at least 1 follow‐up reported higher MSAS score (1.5 vs 1.8, P=0.03), lower Global Physical Health (36.1 vs 33.5, P=0.09), and lower Global Mental Health (44.7 vs 41.0, P=0.03) during their hospitalization. In addition, patients with complete data experienced an average of 1.2 utilization events during the study, whereas those with missing data experienced an average of 2.1 utilization events (P=0.03).

Figure 2

Box plots summarizing the physical symptom severity score of the Memorial Symptom Assessment Scale, PROMIS Global Physical and Mental Health, and General Self‐Rated Health at each survey wave. Brackets indicate P values from the comparisons of the score distribution between each consecutive survey wave using the t test assuming unequal variance (A, B, C) or rank sum test (D).

The MSAS physical symptom severity and Global Physical Health scores from the index hospitalizations did not identify patients with a first utilization event within 30 days. However, patients with poor Global Mental Health and GSRH in the hospital were more likely to experience a utilization event within 14 days of discharge (Figure 3). During the postdischarge period, patients scoring poorly on each of the PRO measures trended toward a greater risk of an early utilization event, but the association between utilization and MSAS was most consistently significant (Figure 3A). In general, the associations with MSAS, Global Physical Health, and GSRH were stronger with the risk of utilization events within 14 days than within 30 days (Figure 3A,B,D). The Global Mental Health score was not associated with a subsequent utilization when measured during the 180‐day postdischarge period.

Figure 3

Kaplan‐Meier plots of time to first hospital‐based utilization by the better (dark line) versus poorer (faint line) median quantiles of each patient‐reported outcomes measure (A, B, C) and “excellent,” “very good,” or “good” versus “poor” or “fair” General Self‐Rated Health (D) categories obtained at hospital discharge and around 30, 90, and 180 days thereafter. The P values test the equality of the “survivor” functions at 14 and 30 days from measurement using the log‐rank test.

As shown in Table 2, Cox proportional hazard models incorporating covariates preserved most of the significant associations seen in the unadjusted analyses. Global Mental Health and good relative to poor GSRH obtained during the hospitalization remained significant. MSAS obtained at each postdischarge follow‐up trended positively with utilization and was statistically significant at 90 and 180 days. Global Physical Health obtained at each postdischarge follow‐up similarly trended negatively with utilization and was significant at 180 days. Each multivariate model incorporating a PRO measure with a significant coefficient contributed to better fit of the predictive model compared to the nested model without the PRO measure.

Hazard Ratios Associated With Patient‐Reported Outcome Measures for Time to First Utilization Event Within 14 Days of Each Survey Wave
Unadjusted Hazard RatioPAdjusted Hazard Ratio*PLikelihood RatioP
  • NOTE: The likelihood ratio statistic tests the hypothesis that the Cox proportional hazard model, including the patient‐reported outcome measure and covariates, predicts the outcome equally well compared to the model with only covariates. Abbreviations: GSRH, General Self‐Rated Health; MSAS, Memorial Symptom Assessment Scale physical symptoms score; NC, not computed due to inadequate response; NS, not statistically significant. *Covariates for the adjusted models are at least 1 utilization event during the study period prior to the survey, Charlson score, age, gender, and race/ethnicity category. Referent on poor GSRH rating.

Hospital discharge
Global Physical Health0.960.100.960.132.290.13
Global Mental Health0.960.050.960.054.050.04
Very good1.090.901.400.63
30 days
Global Physical Health0.960.080.970.241.420.23
Global Mental Health0.980.420.990.620.250.62
90 days
Global Physical Health0.940.070.950.112.750.10
Global Mental Health0.960.200.950.152.110.15
180 days
Global Physical Health0.930.060.930.034.610.03
Global Mental Health0.970.380.960.330.950.33


In this longitudinal cohort study, patients, on average, reported relatively severe symptoms, low PROMIS Global Physical and Mental Health scores, and poor GSRH during the inpatient stay in an urban safety‐net hospital. Symptom severity and Global Physical Health improved, on average, by 30 days before stabilizing, but their poor levels in the hospital did not predict 30‐day hospital‐based utilization events. On the other hand, Global Mental Health and GSRH were stable through hospitalizations, and patients scoring poorly on these measures were at greater risk of utilization events within 14 days of discharge. PRO measures obtained during the 180‐day postdischarge period trended toward distinguishing populations with greater baseline risk of proximate utilization events. However, MSAS physical symptom severity and Global Physical Health were more consistently predictive of these events at statistically significant levels compared to Global Mental Health and GSRH in our relatively small sample of patients. Each of these measures selectively improved the fit‐of‐risk prediction models for hospital‐based utilization.

Some of the heterogeneity in readmission risk is explained by differences in PRO measures. Although the MSAS score and Global Physical Health assessment were reliable predictors of utilization when measured in ambulatory settings, they were less discriminating during acute hospitalizations when everyone, on average, reported severe symptoms and poor function. Our results were consistent with other studies that demonstrated the fairly rapid recovery in symptoms that follow hospitalizations,[28, 29] and these measures may become informative of utilization risk as early as 2 weeks postdischarge. GSRH and Global Mental Health (a measure of health‐related quality of life) only predicted utilizations immediately at hospital discharge. As multidimensional measures that reflect physical, social, and emotional capacity, these measures may indicate vulnerabilities in patients least able to handle the stresses of the early postdischarge period.

There is growing momentum around collecting PRO measures in routine clinical care as quality indicators that capture patient‐centered concerns.[30] Our study explored a novel application of these measures whose routine collection will likely proliferate, not solely for the purpose of helping healthcare systems identify patients at risk of unplanned resource utilization. Although multidimensional PRO measures seldom reflect conditions directly modifiable by simple interventions, we believe that the association between physical symptom burden and utilization in our data reveals a possible target for practice improvement. Hospitalists have contributed enormously to shorter lengths of stay that risk sicker and quicker discharges.[31] To mitigate its potential side effects on symptom management, a discharge plan that acknowledges physical symptoms that sometimes persist or recur beyond the hospitalization may be appropriate. This may be accomplished by ensuring that acute symptoms are resolving, giving clear instructions for symptom management at home, as now the standard of care for conditions like asthma,[32] and explicitly communicating the presence of residual symptoms to providers entrusted with continuity care. As an effective feedback measure that can drive continuous quality improvement, we believe that a technology‐based surveillance strategy that spans both the inpatient and outpatient domains is necessary.[23]

There are some notable similarities and differences between the results of our study and a recent hospital‐based study of PRO measures that used data from the Multi‐Center Hospitalist Project.[16] The Physical Component Score of the SF12 is similar to the PROMIS Global Physical Health score in that both incorporate measures of physical function, perceived health, pain, and energy level. Curiously, the SF12 Physical Component Score, but not the PROMIS Global Physical Health score, was associated with 30‐day rehospitalizations. An important difference between the measures is where the SF12 asks about limitations during the past 4 weeks the PROMIS instrument inquires about physical function in general and levels of fatigue and pain in the past 7 days. Considering most hospitalizations last <7 days, the PROMIS instrument may better reflect the declines associated with the acute illness related to the hospitalization than the SF12 score. Additionally, the discrepancy between the association between hospital‐based GSRH and utilization in our study and the absence, thereof, in Hasan et al. is noteworthy. The difference here may be explained by their use of a 0‐ to 100‐point response scale in contrast to our study's verbally labeled 5‐point scale in the PROMIS instrument. The range of rating scales for survey questions is traditionally governed by the tension between the difficulty with mapping respondents judgment on an excessively large scale on one hand, and the failure of insufficient response options to discriminate between respondents with different underlying judgment on the other.[33] We suspect the former to be a drawback of the unlabeled 100‐point response scale, and conjecture that an association might be found in the Multi‐Center Hospitalist Study data if the responses were grouped into summative categories.

We recognize several limitations in our study. The first is the generalizability of our patient population to others, not insignificantly because of the high proportion of the uninsured (around 70% during the study period) and racial/ethnic minorities among them. Although utilization patterns are clearly affected by socioeconomic status,[34] there may also be differences in the way validated PRO measures are calibrated between patients of public and private healthcare systems.[35] Another limitation is our inability to count utilization events at institutions outside of the CCHHS during our study. However, because the study was conducted prior to Cook County's Medicaid expansion demonstration program as part of the Affordable Care Act,[36] many patients established in our system faced barriers to receiving nonemergency care outside of the CCHHS supporting our assumption that few of our patients were discharged from other hospitals. Causality cannot be established in observational studies. Consequently, high prior‐symptom burden may be associated with utilizations through unmeasured variables. Measures of symptom burden are vulnerable to overendorsement and amplification.[37, 38] Inferences based on statistical significance are affected by sample size, and our conclusions may change if conducted with a larger number of participants. Our response rates were excellent through the survey waves, but we did not achieve perfect follow‐up. Worse levels of PRO responses and higher levels of utilization among censored patients biased our results toward the null. Finally, although we did not find any predominant comorbidities associated with hospital‐based utilizations in our sample, our analyses may be vulnerable to inadequate control for illness severity, which may also have biased our results.

PRO measures are likely to be useful in clinical medicine.[39] But to fully apply the powers of PROs in informing clinically and operationally relevant outcomes, we must actively develop a system for obtaining these measures in routine clinical care. The availability of patient downtime makes hospitalizations conducive to gathering patient‐generated data, and may further enhance patient‐provider communication if survey output was readily available in electronic medical records. Exploring innovative strategies for collecting PROs in the hospital and beyond remains our future work.


Funded by the Agency for Healthcare Research and Quality: R24 HS19481‐01 to support technology implementation. The authors report no conflicts of interest, relevant financial interests, activities, relationships, and affiliations that influenced this work. The first and senior authors had full access to all data and take responsibility for their integrity and the accuracy of the data analysis.


Online-Only Materials

Microsoft Office document icon Supplementary Information (1)61 KB
   Comments ()