Reviews

A STEEEP Hill to Climb: A Scoping Review of Assessments of Individual Hospitalist Performance

Abstract

BACKGROUND: Although ensuring high-quality care requires assessment of individual hospitalist performance, current assessment approaches lack consistency and coherence. The Institute of Medicine’s STEEEP framework for quality healthcare conceptualizes quality through domains of “Safe,” “Timely,” “Effective,” “Efficient,” “Equitable,” and “Patient Centered.” This framework may be applicable to assessing individual hospitalists.
OBJECTIVE: This scoping review sought to identify studies that describe variation in individual hospitalist performance and to code this data to the domains of the STEEEP framework.
METHODS: Via a systematic search of peer-reviewed literature that assessed the performance of individual hospitalists in the Medline database, we identified studies that described measurement of individual hospitalist performance. Forty-two studies were included in the final review and coded into one or more domains of the STEEEP framework.
RESULTS: Studies in the Safe domain focused on transitions of care, both at discharge and within the hospital. Many studies were coded to more than one domain, especially Timely, Effective, and Efficient. Examples include adherence to evidence-based guidelines or Choosing Wisely recommendations. The Patient Centered domain was most frequently coded, but approaches were heterogeneous. No included studies addressed the domain Equitable.
CONCLUSIONS: Applying the STEEEP framework to the published literature on assessment of individual hospitalist performance revealed strengths and weaknesses. Areas of strength were assessments of transitions of care and application of consensus guidelines. Other areas, such as equity and some components of safe practice, need development. All domains would benefit from more practical approaches. These findings should stimulate future work on feasibility of multidimensional assessment approaches.

© 2020 Society of Hospital Medicine

Healthcare quality is defined as the extent to which healthcare services result in desired outcomes.1 Quality of care depends on how the healthcare system’s various components, including healthcare practitioners, interact to meet each patient’s needs.2 These components can be shaped to achieve desired outcomes through rules, incentives, and other approaches, but influencing the behaviors of each component, such as the performance of hospitalists, requires defining goals for performance and implementing measurement approaches to assess progress toward these goals.

One set of principles to define goals for quality and guide assessment of desired behaviors is the multidimensional STEEEP framework. This framework, created by the Institute of Medicine, identifies six domains of quality: Safe, Timely, Effective, Efficient, Equitable, and Patient Centered.2 Briefly, “Safe” means avoiding injuries to patients, “Timely” means reducing waits and delays in care, “Effective” means providing care based on evidence, “Efficient” means avoiding waste, “Equitable” means ensuring quality does not vary based on personal characteristics such as race and gender, and “Patient Centered” means providing care that is responsive to patients’ values and preferences. The STEEEP domains are not coequal; rather, they ensure that quality is considered broadly, while avoiding errors such as measuring only an intervention’s impact on effectiveness but not assessing its impact on multiple domains of quality, such as how patient centered, efficient (cost effective), or equitable the resulting care is.

Based on our review of the literature, a multidimensional framework like STEEEP has not been used in defining and assessing the quality of individual hospitalists’ performance. Some quality metrics at the hospital level impact several dimensions simultaneously, such as door to balloon time for acute myocardial infarction, which measures effectiveness and timeliness of care. Programs like pay-for-performance, Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS), and the Merit-Based Incentive Payment System (MIPS) have tied reimbursement to assessments aligned with several STEEEP domains at both individual and institutional levels but lack a holistic approach to quality.3-6 The every-­other-year State of Hospital Medicine Report, the most widely used description of individual hospitalist performance, reports group-level performance including relative value units and whether groups are accountable for measures of quality such as performance on core measures, timely documentation, and “citizenship” (eg, committee participation or academic work).7 While these are useful benchmarks, the report focuses on performance at the group level. Concurrently, several academic groups have described more complete dashboards or scorecards to assess individual hospitalist performance, primarily designed to facilitate comparison across hospitalist groups or to incentivize overall group performance.8-10 However, these efforts are not guided by an overarching framework and are structured after traditional academic models with components related to teaching and scholarship, which may not translate to nonacademic environments. Finally, the Core Competencies for Hospital Medicine outlines some goals for hospitalist performance but does not speak to specific measurement approaches.11

Overall, assessing individual hospitalist performance is hindered by lack of consensus on important concepts to measure, a limited number of valid measures, and challenges in data collection such as resource limitations and feasibility. Developing and refining measures grounded in the STEEEP framework may provide a more comprehensive assessment of hospitalist quality and identify approaches to improve overall health outcomes. Comparative data could help individual hospitalists improve performance; leaders of hospitalist groups could use this data to guide faculty development and advancement as they ensure quality care at the individual, group, and system levels.

To better inform quality measurement of individual hospitalists, we sought to identify existing publications on individual hospitalist quality. Our goal was to define the published literature about quality measurement at the individual hospitalist level, relate these publications to domains of quality defined by the STEEEP framework, and identify directions for assessment or further research that could affect the overall quality of care.

METHODS

We conducted a scoping review following methods outlined by Arksey and O’Malley12 and Tricco.13 The goal of a scoping review is to map the extent of research within a specific field. This methodology is well suited to characterizing the existing research related to the quality of hospitalist care at the individual level. A protocol for the scoping review was not registered.

Evidence Search

A systematic search for published, English-language literature on hospitalist care was conducted in Medline (Ovid; 1946 - June 4, 2019) on June 5, 2019. The search used a combination of keywords and controlled vocabulary for the concept of hospitalists or hospital medicine. The search strategy used in this review is described in the Appendix. In addition, a hand search of reference lists of articles was used to discover publications not identified in the database searches.

Study Selection

All references were uploaded to Covidence systematic review software (www.covidence.org; Covidence), and duplicates were removed. Four reviewers (A.D., B.C., L.H., R.Q.) conducted title and abstract, as well as full-text, review to identify studies that measured differences in the performance of hospitalists at the individual level. Any disagreements among reviewers were resolved by consensus. Articles included both adult and pediatric populations. Articles that focused on group-level outcomes could be included if nonpooled data at the individual level was also reported. Studies were excluded if they did not focus on individual quality of care indicators or were not published in English.

Data Charting and Synthesis

We extracted the following information using a standardized data collection form: author, title, year of publication, study design, intervention, and outcome measures. Original manuscripts were accessed as needed to supplement analysis. Critical appraisal of individual studies was not conducted in this review because the goal of this review was to analyze which quality indicators have been studied and how they were measured. Articles were then coded for their alignment to the STEEEP framework by two reviewers (AD and BC). After initial coding was conducted, the reviewers met to consolidate codes and resolve any disagreement by consensus. The results of the analysis were summarized in both text and tabular format with studies grouped by focus of assessment with each one’s methods of assessment listed.

RESULTS

Results of the search strategy are shown in the Figure. The search retrieved a total of 2,363 references of which 113 were duplicates, leaving 2,250 to be screened. After title and abstract and full-text screening, 42 studies were included in the review. The final 42 studies were coded for alignment with the STEEEP framework. The Table displays the focus of assessment and methods of assessment within each STEEEP domain.

Flow Diagram of Studies in the Selection Process

Eighteen studies were coded into a single domain while the rest were coded into at least two domains. The domain Patient Centered was coded as having the most studies (n = 23), followed by the domain of Safe (n = 15). Timely, Effective, and Efficient domains had 11, 9, and 12 studies, respectively. No studies were coded into the domain of Equitable.

Foci and Methods of Assessment Categorized by STEEEP Domaina

Safe

Nearly all studies coded into the Safe domain focused on transitions of care. These included transfers into a hospital from other hospitals,14 transitions of care to cross-covering providers15,16 and new primary providers,17 and transition out from the acute care setting.18-28 Measures of hospital discharge included measures of both processes18-22 and outcomes.23-27 Methods of assessment varied from use of trained observers or scorers to surveys of individuals and colleagues about performance. Though a few leveraged informatics,22,27 all approaches relied on human interaction, and none were automated.

Foci and Methods of Assessment Categorized by STEEEP Domaina

Timely

All studies coded into the Timely domain were coded into at least one other domain. For example, Anderson et al looked at how hospitalists communicated about potential life-limiting illness at the time of hospital admission and the subsequent effects on plans of care29; this was coded as both Timely and Patient Centered. Likewise, another group of studies centered on application of evidence-based guidelines, such as giving antibiotics within a certain time interval for sepsis and were coded as both Timely and Effective. Another set of authors described dashboards or scorecards that captured a number of group-level metrics of processes of care that span STEEEP domains and may be applicable to individuals, including Fox et al for pediatrics8 and Hwa et al for an adult academic hospitalist group.9 Methods of assessment varied widely across studies and included observations in the clinical environment,28,30,31 performance in simulations,32 and surveys about performance.22-26 A handful of approaches were more automated and made use of informatics8,9,22 or data collected for other health system purposes.8,9

Effective

Effectiveness was most often assessed through adherence to consensus and evidence-based guidelines. Examples included processes of care related to sepsis, venous thromboembolism prophylaxis, COPD, heart failure, pediatric asthma, and antibiotic appropriateness.8,9,23,32-36 Through the review, multiple other studies that included group-level measures of effectiveness for a variety of health conditions were excluded because data on individual-level variation were not reported. Methods of assessment included expert review of cases or discharge summaries, compliance with core measures, performance in simulation, and self-assessment on practice behaviors. Other than those efforts aligned with institutional data collection, most approaches were resource intensive.

Efficient

As with those in the Timely domain, most studies coded into the Efficient domain were coded into at least one other domain. One exception measured unnecessary daily lab work and both showed provider-level variation and demonstrated improvement in quality based on an intervention.37 Another paper coded into the Effective domain evaluated adherence to components of the Choosing Wisely® recommendations.34 In addition to these two studies focusing on cost efficacy, other studies coded to this domain assessed concepts such as ensuring more efficient care from other providers by optimizing transitions of care15-17 and clarifying patients’ goals for care.38 Although integrating insurer information into care plans is emphasized in the Core Competencies of Hospital Medicine,11 this concept was not represented in any of the identified articles. Methods of assessment varied and mostly relied on observation of behaviors or survey of providers. Several approaches were more automated or used Medicare claims data to assess the efficiency of individual providers relative to peers.34,37,39

Equitable

Among the studies reviewed, none were coded into the Equitable domain despite care of vulnerable populations being identified as a core competency of hospital medicine.40

Patient Centered

Studies coded to the Patient Centered domain assessed hospitalist performance through ratings of patient satisfaction,8,9,41-44 rating of communication between hospitalists and patients,19-21,29,45-51 identification of patient preferences,38,52 outcomes of patient-centered care activities,27,28 and peer ratings.53,54 Authors applied several theoretical constructs to these assessments including shared decision-making,50 etiquette-based medicine,47,48 empathetic responsiveness,45 agreement about the goals of care between the patient and healthcare team members,52 and lapses in professionalism.53 Studies often crossed STEEEP domains, such as those assessing quality of discharge information provided to patients, which were coded as both Safe and Patient Centered.19-21 In addition to coded or observed performance in the clinical setting, studies in this domain also used patient ratings as a method of assessment.8,9,28,41-44,49,50 Only a few of these approaches aligned with existing performance measures of health systems and were more automated.8,9

DISCUSSION

This scoping review of performance data for individual hospitalists coded to the STEEEP framework identified robust areas in the published literature, as well as opportunities to develop new approaches or refine existing measures. Transitions of care, both intrahospital and at discharge, and adherence to evidence-based guidelines are areas for which current research has created a foundation for care that is Safe, Timely, Effective, and Efficient. The Patient Centered domain also has several measures described, though the conceptual underpinnings are heterogeneous, and consensus appears necessary to compare performance across groups. No studies were coded to the Equitable domain. Across domains, approaches to measurement varied in resource intensity from simple ones, like integrating existing data collected by hospitals, to more complex ones, like shadowing physicians or coding interactions.

Methods of assessment coded into the Safe domain focused on communication and, less so, patient outcomes around transitions of care. Transitions of care that were evaluated included transfer of patients into a new facility, sign-out to new physicians for both cross-cover responsibilities and for newly assuming the role of primary attending, and discharge from the hospital. Most measures rated the quality of communication, although several23-27 examined patient outcomes. Approaches that survey individuals downstream from a transition of care15,17,24-26 may be the simplest and most feasible approach to implement in the future but, as described to date, do not include all transitions of care and may miss patient outcomes. Important core competencies for hospital medicine under the Safe domain that were not identified in this review include areas such as diagnostic error, hospital-acquired infections, error reporting, and medication safety.11 These are potential areas for future measure development.

The assessments in many studies were coded across more than one domain; for example, measures of the application of evidence-based guidelines were coded into domains of Effective, Timely, Efficient, and others. Applying the six domains of the STEEEP framework revealed the multidimensional outcomes of hospitalist work and could guide more meaningful quality assessments of individual hospitalist performance. For example, assessing adherence to evidence-based guidelines, as well as consideration of the Core Competencies of Hospital Medicine and recommendations of the Choosing Wisely® campaign, are promising areas for measurement and may align with existing hospital metrics. Notably, several reviewed studies measured group-level adherence to guidelines but were excluded because they did not examine variation at the individual level. Future measures based on evidence-based guidelines could center on the Effective domain while also integrating assessment of domains such as Efficient, Timely, and Patient Centered and, in so doing, provide a richer assessment of the diverse aspects of quality.

Several other approaches in the domains of Timely, Effective, and Efficient were described only in a few studies yet deserve consideration for further development. Two time-­motion studies30,31 were coded into the domains of Timely and Efficient and would be cumbersome in regular practice but, with advances in wearable technology and electronic health records, could become more feasible in the future. Another approach used Medicare payment data to detect provider-level variation.39 Potentially, “big data” could be analyzed in other ways to compare the performance of individual hospitalists.

The lack of studies coded into the Equitable domain may seem surprising, but the Institute for Healthcare Improvement identifies Equitable as the “forgotten aim” of the STEEEP framework. This organization has developed a guide for health care organizations to promote equitable care.55 While this guide focuses mostly on organizational-level actions, some are focused on individual providers, such as training in implicit bias. Future research should seek to identify disparities in care by individual providers and develop interventions to address any discovered gaps.

The “Patient Centered” domain was the most frequently coded and had the most heterogeneous underpinnings for assessment. Studies varied widely in terminology and conceptual foundations. The field would benefit from future work to identify how “Patient Centered” care might be more clearly conceptualized, guided by comparative studies among different assessment approaches to define those most valid and feasible.

The overarching goal for measuring individual hospitalist quality should be to improve the delivery of patient care in a supportive and formative way. To further this goal, adding or expanding on metrics identified in this article may provide a more complete description of performance. As a future direction, groups should consider partnering with one another to define measurement approaches, collaborate with existing data sources, and even share deidentified individual data to establish performance benchmarks at the individual and group levels.

While this study used broad search terms to support completeness, the search process could have missed important studies. Grey literature, non–English language studies, and industry reports were not included in this review. Groups may also be using other assessments of individual hospitalist performance that are not published in the peer-reviewed literature. Coding of study assessments was achieved through consensus reconciliation; other coders might have classified studies differently.

CONCLUSION

This scoping review describes the peer-reviewed literature of individual hospitalist performance and is the first to link it to the STEEEP quality framework. Assessments of transitions of care, evidence-based care, and cost-effective care are exemplars in the published literature. Patient-centered care is well studied but assessed in a heterogeneous fashion. Assessments of equity in care are notably absent. The STEEEP framework provides a model to structure assessment of individual performance. Future research should build on this framework to define meaningful assessment approaches that are actionable and improve the welfare of our patients and our system.

Disclosures

The authors have nothing to disclose.

References

Online-Only Materials

AttachmentSize
PDF icon dow_appendix_1.pdf8.7 KB
   Comments ()