Original Research

Hospital-level factors associated with pediatric emergency department return visits



Return visits (RVs) and RVs with admission (RVAs) are commonly used emergency department quality measures. Visit- and patient-level factors, including several social determinants of health, have been associated with RV rates, but hospital-specific factors have not been studied.


To identify what hospital-level factors correspond with high RV and RVA rates.


Multicenter mixed-methods study of hospital characteristics associated with RV and RVA rates.


Pediatric Health Information System with survey of emergency department directors.


Adjusted return rates were calculated with generalized linear mixed-effects models. Hospitals were categorized by adjusted RV and RVA rates for analysis.


Twenty-four hospitals accounted for 1,456,377 patient visits with an overall adjusted RV rate of 3.7% and RVA rate of 0.7%. Hospitals with the highest RV rates served populations that were more likely to have government insurance and lower median household incomes and less likely to carry commercial insurance. Hospitals in the highest RV rate outlier group had lower pediatric emergency medicine specialist staffing, calculated as full-time equivalents per 10,000 patient visits: median (interquartile range) of 1.9 (1.5-2.1) versus 2.9 (2.2-3.6). There were no differences in hospital population characteristics or staffing by RVA groups.


RV rates were associated with population social determinants of health and inversely related to staffing. Hospital-level variation may indicate population-level economic factors outside the control of the hospital and unrelated to quality of care. Journal of Hospital Medicine 2017;12:536-543. © 2017 Society of Hospital Medicine

© 2017 Society of Hospital Medicine

Return visit (RV) rate is a quality measure commonly used in the emergency department (ED) setting. This metric may represent suboptimal care at the index ED visit.1-5 Although patient- and visit-level factors affecting ED RVs have been evaluated,1,3,4,6-9 hospital-level factors and factors of a hospital’s patient population that may play roles in ED RV rates have not been examined. Identifying the factors associated with increased RVs may allow resources to be designated to areas that improve emergent care for children.10

Hospital readmission rates are a closely followed quality measure and are linked to reimbursement by the federal government, but a recent study found the influence a hospital can have on this marker may be mitigated by the impact of the social determinates of health (SDHs) of the hospital’s patient population.11 That study and others have prompted an ongoing debate about adjusting quality measures for SDHs.12,13 A clearer understanding of these interactions may permit us to focus on factors that can truly lead to improvement in care instead of penalizing practitioners or hospitals that provide care to those most in need.

Prior work has identified several SDHs associated with higher ED RV rates in patient- or visit-level analyses.3,11,14 We conducted a study of hospital-level characteristics and characteristics of a hospital’s patient population to identify potentially mutable factors associated with increased ED RV rates that, once recognized, may allow for improvement in this quality measure.


This study was not considered human subjects research in accordance with Common Rule 45 CFR§46.104(f) and was evaluated by the Ann and Robert H. Lurie Children’s Hospital and Northwestern University Feinberg School of Medicine Institutional Review Boards and deemed exempt from review.

Study Population and Protocol

Our study had 2 data sources (to be described in detail): the Pediatric Health Information System (PHIS) and a survey of ED medical directors of the hospitals represented within PHIS. Hospitals were eligible for inclusion in the study if their data (1) met PHIS quality control standards for ED patient visits as determined by internal data assurance processes incorporated in PHIS,3,14,15 (2) included data only from an identifiable single main ED, and (3) completed the ED medical director’s survey.

PHIS Database

PHIS, an administrative database managed by Truven Health Analytics, includes data from ED, ambulatory surgery, observation, and inpatient encounters across Children’s Hospital Association member children’s hospitals in North America. Data are subjected to validity checks before being included in the database.16 PHIS assigns unique patient identifiers to track individual patient visits within participating institutions over time.

Hospitals were described by percentages of ED patients in several groups: age (<1, 1-4, 5-9, 10-14, and 15-18 years)17; sex; race/ethnicity; insurance type (commercial, government, other); ED International Classification of Diseases, Ninth Edition (ICD-9) diagnosis code–based severity classification system score (1-2, low severity; 3-5, high severity)18; complex chronic condition presence at ED visits in prior year14,19-21; home postal (Zip) code median household income from 2010 US Census data compared with Federal Poverty Level (<1.5, 1.5-2, 2-3, and >3 × FPL)17; and primary care physician (PCP) density in Federal Health Service Area of patient’s home address as reported by Dartmouth Atlas of Health Care modeled by quartiles.22 Density of PCPs—general pediatricians, family practitioners, general practitioners, and general internists—is calculated as number of PCPs per 100,000 residents. We used PCP density to account for potential care provided by any of the PCPs mentioned. We also assessed, at hospital level, index visit arrival time (8:01 am to 4:00 pm; 4:01 pm to 12:00 am; 12:01 am to 8:00 am) and index visit season.23

ED Medical Director Survey

A web-based survey was constructed in an iterative process based on literature review and expert opinion to assess hospital-level factors that may impact ED RV rates.3,7,24-26 The survey was piloted at 3 institutions to refine its structure and content.

The survey included 15 close-ended or multiple-choice questions on ED environment and operations and 2 open-ended questions, “What is the largest barrier to reducing the number of return visits within 72 hours of discharge from a previous ED visit?” and “In your opinion, what is the best way of reducing the number of the return visits within 72 hours of previous ED visit ?” (questionnaire in Supplemental material). Hospital characteristics from the survey included total clinical time allotment, or full-time equivalent (FTE), among all physicians, pediatric emergency medicine (PEM) fellowship-trained physicians, and all other (non-PEM) physicians. The data were standardized across sites by calculating FTE-per-10,000-visits values for each hospital; median duration of ED visit for admitted and discharged patients; median time from arrival to ED physician evaluation; rate of leaving without being seen; discharge educational material authorship and age specificity; follow-up visit scheduling procedure; and percentage of ED patients for whom English was a second language.

Responses to the 2 open-ended questions were independently categorized by Drs. Pittsenbarger and Alpern. Responses could be placed in more than 1 category if multiple answers to the question were included in the response. Categorizations were compared for consistency, and any inconsistencies were resolved by the consensus of the study investigators.

Outcome Measures From PHIS Database

All ED visits within a 12-month period (July 1, 2013–June 30, 2014) by patients younger than 18 years at time of index ED visit were eligible for inclusion in the study. An index visit was defined as any ED visit without another ED visit within the preceding 72 hours. The 72-hour time frame was used because it is the most widely studied time frame for ED RVs.5 Index ED visits that led to admission, observation status, death, or transfer were excluded.

The 2 primary outcomes of interest were (1) RVs within 72 hours of index ED visit discharge and (2) RVs within 72 hours that resulted in hospital admission or observation status at the next ED visit (RVA).7,9,27-30 For patients with multiple ED revisits within 72 hours, only the first was assessed. There was a 72-hour minimum between index visits for the same patient.

Statistical Analyses

To determine hospital groups based on RV and RVA rates, we adjusted RV and RVA rates using generalized linear mixed-effects models, controlling for clustering and allowing for correlated data (within hospitals), nonconstant variability (across hospitals), and non-normally distributed data, as we did in a study of patient-level factors associated with ED RV and RVA.3 For each calculated rate (RV, RVA), the hospitals were then classified into 3 groups based on whether the hospital’s adjusted RV and RVA rates were outside 2 SDs from the mean, below the 5th or above the 95th percentile, or within that range. These groups were labeled lowest outliers, highest outliers, and average-performing hospitals.

After the groups of hospitals were determined, we returned to using unadjusted data to statistically analyze them. We summarized continuous variables using minimum and maximum values, medians, and interquartile ranges (IQRs). We present categorical variables using counts and percentages. To identify hospital characteristics with the most potential to gain from improvement, we also analyzed associations using 2 collapsed groups: hospitals with RV (or RVA) rates included in the average-performing and lowest outlier groups and hospitals within the highest outlier group. Hospital characteristics and hospital’s patient population characteristics from the surveys are summarized based on RV and RVA rate groups. Differences in distributions among continuous variables were assessed by Kruskal-Wallis 1-way analysis of variance. Chi-square tests were used to evaluate differences in proportions among categorical variables. All statistical analyses were performed with SAS Version 9.4 (SAS Institute); 2-sided P < 0.05 was considered statistically significant.


Return Visit Rates and Hospital ED Site Population Characteristics

Twenty-four of 35 (68%) eligible hospitals that met PHIS quality control standards for ED patient visits responded to the ED medical director survey. The included hospitals that both met quality control standards and completed the survey had a total of 1,456,377 patient visits during the study period. Individual sites had annual volumes ranging from 26,627 to 96,637 ED encounters. The mean RV rate across the institutions was 3.7% (range, 3.0%-4.8%), and the mean RVA rate across the hospitals was 0.7% (range, 0.5%-1.1%) (Figure).

Adjusted 72-hour revisit rates at 24 children’s hospitals.


There were 5 hospitals with RV rates less than 2 SDs of the mean rate, placing them in the lowest outlier group for RV; 13 hospitals with RV rates within 2 SDs of the mean RV rate, placing them in the average-performing group; and 6 hospitals with RV rates above 2 SDs of the mean, placing them in the highest outlier group. Table 1 lists the hospital ED site population characteristics among the 3 RV rate groups. Hospitals in the highest outlier group served populations with higher proportions of patients with insurance from a government payer, lower proportions of patients covered by a commercial insurance plan, and higher proportion of patients with lower median household incomes.

Unadjusted Hospital Emergency Department Site Population Characteristics Among Return Visit Rate Groups

Table 1

In the RVA analysis, there were 6 hospitals with RVA rates less than 2 SDs of the mean RVA rate (lowest outliers); 14 hospitals with RVA rates within 2 SDs of the mean RVA rate (average performers); and 4 hospitals with RVA rates above 2 SDs of the mean RVA rate (highest outliers). When using these groups based on RVA rate, there were no statistically significant differences in hospital ED site population characteristics (Supplemental Table 1).

RV Rates and Hospital-Level Factors Survey Characteristics

Table 2 lists the ED medical director survey hospital-level data among the 3 RV rate groups. There were fewer FTEs by PEM fellowship-trained physicians per 10,000 patient visits at sites with higher RV rates (Table 2). Hospital-level characteristics assessed by the survey were not associated with RVA rates (Supplemental Table 2).

Hospital-Level Factors (From Medical Director Survey Responses) and Return Visit Rates

Table 2

Evaluating characteristics of hospitals with the most potential to gain from improvement, hospitals with the highest RV rates (highest outlier group), compared with hospitals in the lowest outlier and average-performing groups collapsed together, persisted in having fewer PEM fellowship-trained physician FTEs per patient visit (Table 3). A similar collapsed analysis of RVA rates demonstrated that hospitals in the highest outlier group had longer-wait-to-physician time (81 minutes; IQR, 51-105 minutes) compared with hospitals in the other 2 groups (30 minutes; IQR, 19-42.5 minutes) (Table 3).

Hospital-Level Factors and Return Visit Rates in Collapsed Groups

Table 3

In response to the first qualitative question on the ED medial director survey, “In your opinion, what is the largest barrier to reducing the number of return visits within 72 hours of discharge from a previous ED visit?”, 15 directors (62.5%) reported limited access to primary care, 4 (16.6%) reported inadequate discharge instructions and/or education provided, and 3 (12.5%) reported lack of access to specialist care. To the second question, “In your opinion, what is the best way of reducing the number of the return visits within 72 hours of previous ED visit for the same condition?”, they responded that RVs could be reduced by innovations in scheduling primary care or specialty follow-up visits (19, 79%), improving discharge education and instructions (6, 25%), and identifying more case management or care coordination (4, 16.6%).


Other studies have identified patient- and visit-level characteristics associated with higher ED RV and RVA rates.3,8,9,31 However, as our goal was to identify possible modifiable institutional features, our study examined factors at hospital and population-served levels (instead of patient or visit level) that may impact ED RV and RVA rates. Interestingly, our sample of tertiary-care pediatric center EDs provided evidence of variability in RV and RVA rates. We identified factors associated with RV rates related to the SDHs of the populations served by the ED, which suggests these factors are not modifiable at an institution level. In addition, we found that the increased availability of PEM providers per patient visit correlated with fewer ED RVs.

Hospitals serving ED populations with more government-insured and fewer commercially insured patients had higher rates of return to the ED. Similarly, hospitals with larger proportions of patients from areas with lower median household incomes had higher RV rates. These factors may indicate that patients with limited resources may have more frequent ED RVs,3,6,32,33 and hospitals that serve them have higher ED RV rates. Our findings complement those of a recent study by Sills et al.,11 who evaluated hospital readmissions and proposed risk adjustment for performance reimbursement. This study found that hospital population-level race, ethnicity, insurance status, and household income were predictors of hospital readmission after discharge.

Of note, our data did not identify similar site-level attributes related to the population served that correlated with RVA rates. We postulate that the need for admission on RV may indicate an inherent clinical urgency or medical need associated with the return to the ED, whereas RV without admission may be related more to patient- or population-level sociodemographic factors than to quality of care and clinical course, which influence ED utilization.1,3,30 EDs treating higher proportions of patients of minority race or ethnicity, those with fewer financial resources, and those in more need of government health insurance are at higher risk for ED revisits.

We observed that increased PEM fellowship-trained physician staffing was associated with decreased RV rates. The availability of specialty-trained physicians in PEM may allow a larger proportion of patients treated by physicians with honed clinical skills for the patient population. Data from a single pediatric center showed PEM fellowship-trained physicians had admission rates lower than those of their counterparts without subspecialty fellowship training.34 The lower RV rate for this group in our study is especially interesting in light of previously reported lower admission rates at index visit in PEM trained physicians. With lower index admission rates, it may have been assumed that visits associated with PEM trained physician care would have an increased (rather than decreased) chance of RV. In addition, we noted the increased RVA rates were associated with longer waits to see a physician. These measures may indicate the effect of institutional access to robust resources (the ability to hire and support more specialty-trained physicians). These novel findings warrant further evaluation, particularly as our sample included only pediatric centers.

Our survey data demonstrated the impact that access to care has on ED RV rates. The ED medical directors indicated that limited access to outpatient appointments with PCPs and specialists was an important factor increasing ED RVs and a potential avenue for interventions. As the 2 open-ended questions addressed barriers and potential solutions, it is interesting that the respondents cited access to care and discharge instructions as the largest barriers and identified innovations in access to care and discharge education as important potential remedies.

This study demonstrated that, at the hospital level, ED RV quality measures are influenced by complex and varied SDHs that primarily reflect the characteristics of the patient populations served. Prior work has similarly highlighted the importance of gaining a rigorous understanding of other quality measures before widespread use, reporting, and dissemination of results.11,35-38 With this in mind, as quality measures are developed and implemented, care should be taken to ensure they accurately and appropriately reflect the quality of care provided to the patient and are not more representative of other factors not directly within institutional control. These findings call into question the usefulness of ED RVs as a quality measure for comparing institutions.

Study Limitations

This study had several limitations. The PHIS dataset tracks only patients within each institution and does not include RVs to other EDs, which may account for a proportion of RVs.39 Our survey response rate was 68% among medical directors, excluding 11 hospitals from analysis, which decreased the study’s power to detect differences that may be present between groups. In addition, the generalizability of our findings may be limited to tertiary-care children’s hospitals, as the PHIS dataset included only these types of healthcare facilities. We also included data only from the sites’ main EDs, and therefore cannot know if our results are applicable to satellite EDs. ED staffing of PEM physicians was analyzed using FTEs. However, number of clinical hours in 1 FTE may vary among sites, leading to imprecision in this hospital characteristic.


Hospitals with the highest RV rates served populations with a larger proportion of patients with government insurance and lower household income, and these hospitals had fewer PEM trained physicians. Variation in RV rates among hospitals may be indicative of the SDHs of their unique patient populations. ED revisit rates should be used cautiously in determining the quality of care of hospitals providing care to differing populations.


Nothing to report.


Online-Only Materials

   Comments ()

Related Articles from PubMed

Comput Methods Programs Biomed. 2017 Jun;144:105-112. doi: 10.1016/j.cmpb.2017.03.022. Epub 2017 Mar 25.
Pediatr Emerg Care. 2004 Mar;20(3):166-71.