Perspectives in Hospital Medicine

Grade pending: Lessons for hospital quality reporting from the New York City restaurant sanitation inspection program

Abstract

Public quality reporting programs have been widely implemented in hospitals in an effort to improve quality and safety. One such program is Hospital Compare, Medicare's national quality reporting program for US hospitals. The New York City sanitary grade inspection program is a parallel effort for restaurants. The aims of Hospital Compare and the New York City sanitary inspection program are fundamentally similar: to address a common market failure resulting from consumers' lack of information on quality and safety. However, by displaying easily understandable information at the point of service, the New York City sanitary inspection program is better designed to encourage informed consumer decision making. We argue that this program holds important lessons for public quality reporting of US hospitals. Journal of Hospital Medicine 2015;10:116–119. © 2014 Society of Hospital Medicine

© 2014 Society of Hospital Medicine

Few consumers would choose to dine at a restaurant if they knew the kitchen was infested with cockroaches. Few patients would choose to undergo a liver transplant in a hospital that was performing the procedure for the first time. In most sectors, consumers gather information about quality (and price) from the marketplace, where economic theory predicts that rational behavior and competition will lead to continuous improvement over time. However, for some goods and services, information is sparse and asymmetric between consumers and suppliers. In sectors where consumer health is at risk, society has often intervened to assure minimum standards. Yet sometimes these efforts have fallen short. In healthcare, physician licensure and hospital accreditation (eg, through the Joint Commission), although providing an important foundation to assure safety, have not come close to solving the widespread quality problems.[1] Basic regulatory requirements for restaurants have also proven inadequate to prevent food‐borne illness. Consumer trust, without information, can be a recipe (or prescription) for trouble.

In response, high‐profile efforts have been introduced to publicize the quality and safety of service providers. One example is Hospital Compare, Medicare's national quality reporting program for US hospitals.[2] The New York City sanitary grade inspection program is a parallel effort for restaurants. Although customers can judge how much they like the food from a restaurantor look up reviews at Yelp.comthey face greater difficulty identifying whether a restaurant was responsible for making them sick. By publicizing restaurants' sanitation conditions, the New York City inspection program seeks to use market forces to decrease food‐borne illness by deterring consumers from eating at restaurants with poor sanitation grades.

The aims of Hospital Compare and the New York City sanitary inspection program are fundamentally similar. Both initiatives seek to address a common market failure resulting in the consumer's lack of information on quality and safety. By infusing the market with information, these programs enable consumers to make better choices and encourage service providers to improve quality and safety.[3] Despite the promise of these programs, a copious literature about the effects of public quality reporting in healthcare has found mixed results.[4, 5] Although the performance measures in any public reporting program must be valid and reliable, good measures are not sufficient to achieve the goals of public reporting. To engage patients, reported results must also be accessible, understandable, and meaningful. Both patients' lack of knowledge about the reports[6] and patients' inability to effectively use these data to make better decisions[7] are some reasons why public quality reporting has fallen short of its expectations. This article argues that the New York City program is much better structured to positively affect patient choice, and holds important lessons for public quality reporting in US hospitals.

CONTRASTS BETWEEN HOSPITAL COMPARE AND THE NEW YORK CITY RESTAURANT SANITARY INSPECTION PROGRAM

Hospital Compare reports performance for 108 separate quality indicators related to quality and patient safety for US hospitals (Table 1). These are a combination of structure measures (eg, hospital participation in a systematic database for cardiac surgery), process of care measures (eg, acute myocardial infarction patients receiving fibrinolytic therapy within 30 minutes of hospital arrival), outcomes (eg, 30‐day mortality and readmission), and patient experience measures (eg, how you would rate your communication with your physician). Hospital Compare data, frequently based on hospital quality performance 1 to 3 years prior to publication, are displayed on a website. Hospitals do not receive a summary measure of quality or safety.[8] Hospitals face financial incentives that are tied to measure reporting[9] and performance for some of the measures on Hospital Compare.[10, 11] Hospital accreditation is only loosely related to performance on these measures.

Contrasts Between Hospital Compare and the New York City Sanitary Inspection Program
Attribute Hospital Compare New York City Sanitary Inspection Program
Display of information On a website (http://www.medicare.gov/hospitalcompare/search.html). On the front of the restaurant, with additional information also available on a website (http://www.nyc.gov/html/doh/html/services/restaurant‐inspection.shtml).
Frequency of information update Quarterly; data often lag by between 1 and 3 years. Unannounced inspections occur at least annually. Grades are posted immediately after inspection.
Quality measures Mix of measures pertaining to quality improvement activities (eg, hospital participation in a cardiac surgery registry or a quality improvement initiative), rates of adherence with evidence‐based medicine (eg, heart failure patients receiving discharge instructions, acute myocardial infarction patients receiving ‐blocker at arrival), and patient outcomes (eg, 30‐day mortality and 30‐day readmission for acute myocardial infarction, heart failure, and pneumonia). Mix of measures pertaining to conditions of the facility (eg, improper sewage disposal system, improper food contact surface, evidence of live rats in the facility) and the treatment and handling of food (eg, food is unwrapped, appropriate thermometer not used to measure temperature of potentially hazardous foods, food not prepared to sufficiently high temperature).
Clarity and simplicity of information 108 individual measures. No summary measure. Single summary letter grade displayed on front of restaurant. Detailed data on individual violations (ie, measures) available on website.
Consequences of poor performance and mechanisms for enforcement Hospitals are subject to financial penalties for not reporting certain measures and face financial incentives for performance on a subset of measures. Restaurants are fined for violations, are subject to repeated inspections for poor performance, and are subject to closure for severe violations.
Consumer awareness Limited Widespread

The New York City sanitation program regularly inspects restaurants and scores them on a standard set of indicators that correspond to critical violations (eg, food is contaminated by mouse droppings) or general violations (eg, garbage is not adequately covered).[12] Points are assigned to each type and severity of violation, and the sum of the points are converted into a summary grade of A, B, or C. Restaurants can dispute the grades, receiving a grade pending designation until the dispute is adjudicated. After inspection, sanitation grades are immediately posted on restaurants' front door or window, providing current information that is clearly visible to consumers before entering. More detailed information on sanitation violations is also available on a website. If restaurants receive an A grade, they face no additional inspections for 1 year, but poorly graded restaurants may receive monthly inspections. Restaurants face fines from violations and are subject to closure from severe violations. Recently proposed changes would decrease fines and give restaurants greater opportunities to appeal grades, but leave the program otherwise intact.[13]

IMPLICATIONS FOR PUBLIC QUALITY REPORTING IN HOSPITALS

Along with value‐based payment reforms, public quality reporting is one of the few major system‐level approaches that is being implemented in the US to improve quality and safety in healthcare. However, without a simple and understandable display of information that is available when a patient needs it, quality and safety information will likely go unused.[14] Hospital Compare leaves it up the patient to find the quality and safety information and does little to help patients understand and use the information effectively. Hospital Compare asks patients to do far more work, which is perhaps why it has been largely ignored by patients.[2, 15] The New York City sanitation inspection program evaluates restaurants, prominently displays an understandable summary result, and puts the scoring details in the background. Although peer‐reviewed evaluations of the New York City sanitation inspection program have not yet been published, internal data show that the program has decreased customer concern about getting sick, improved sanitary practices, and decreased salmonella.[16] Evidence from a similar program in Los Angeles County found that hygiene grades steered consumers toward restaurants with better sanitary conditions and decreased food‐borne illness.[17]

The nature of choice in healthcare, particularly the choice of hospital, is much different than it is for restaurants. In some areas, a single hospital may serve a large geographical area, severely limiting choice. Even when patients have the ability to receive care at different hospitals, choice may be limited because patients are referred to a specific hospital by their outpatient physician or are brought to a hospital during an emergency.[18] In these cases, quality grades on the front doors of hospitals would not affect patient decisions, at least for that admission. Nonetheless, if quality grades were posted on the front doors of hospitals, patients receiving both inpatient and outpatient care would see the grades, and could use the information to make future decisions. Posted grades may also lead patients to review more in‐depth quality information related to their condition on the Hospital Compare website. Posted quality grades would also increase the visibility of the grades for other stakeholdersincluding the media and boards of directorsmagnifying their salience and impact.

How quality information is displayed and summarized can make or break public reporting programs. The New York City sanitation inspection program displays summarized, composite measures in the form of widely understood letter grades. Hospital Compare, however, displays myriad, unrelated performance measures that are not summarized into a global quality or safety measure. This information display is at odds with best practice. Patients find it difficult to synthesize data from multiple performance indicators to determine the relative quality of healthcare providers or insurance plans.7 In many cases, more information can lead to worse decision making.[19] Patients' difficulty making optimal choices has been noted in numerous healthcare settings, including purchasing Medicare Part D plans[20] and choosing health plans.[21] Recent evidence suggests that Nursing Home Compare's shift from an unsummarized collection of disparate performance measures to a 5‐star rating system has led patients to choose higher‐ranked facilities.[22] The fact that commercial providers of product quality information, such as Consumer Reports[23] and US News and World Report,[24] publish global summary scores, in addition to component scores, is a hint that this style of reporting is more appealing to consumers. Reports suggest that Medicare is moving toward a 5‐star quality rating system for hospitals,[8] which is a welcome development.

Different types of patients may demand different types of quality information, and a single summary measure for Hospital Compare may not meet the needs of a diverse set of patients. Nonetheless, the benefits from an actionable, understandable, comprehensive, and appropriate summary measure likely outweigh the costs of a potential mismatch for certain types of patients. Many of the performance measures on Hospital Compare already apply broadly to diverse sets of patients (eg, the structure measures, patient experience, and surgical safety) and are not specific to certain disease areas. Global summary measures could be complemented by separate component scores (eg, by disease area or domain of quality) for patients who wanted information on different aspects of care.

The inspection regime that underlies the New York City sanitary inspection program has parallels in healthcare that could be extended to Hospital Compare. For instance, the Joint Commission performs surprise inspections of hospitals as part of its accreditation process. The publicly reported 5‐star ratings for nursing homes are also based, in part, on inspection results.[25] Results from these types of inspections can capture up‐to‐date information on important dimensions of quality and safety that are not available in standard administrative data sources. Incorporating inspection results into Hospital Compare could increase both the timeliness and validity of the reporting.

The New York City sanitation inspection program is not a panacea: the indicators may not capture all relevant aspects of restaurant sanitation, some research suggests that past sanitary grades do not predict future grades,[26] and sanitary grade inflation over time has the potential to mask meaningful differences in sanitary conditions that are related to food‐borne illness.[16, 26] However, by providing understandable and meaningful reports at the point of service, the New York City program is well designed to encourage sanitation improvement through both consumer and supplier behavior.

Where the New York City sanitation inspection program succeeds, Hospital Compare fails. Hospital Compare is not patient centered, and it is not working for patients. Medicare can learn from the New York City restaurant sanitation inspection program to enhance the effects of public reporting by presenting information to consumers that is relevant, easy to access and interpret, and up to date. The greater complexity of hospital product lines should not deter these efforts. Patients' lives, not just the health of their gastrointestinal tracts, are at stake.

ACKNOWLEDGEMENTS

The authors thank Kaveh G. Shojania, MD, and Edward E. Etchells, MD, MSc, University of Toronto, and Martin Roland, DM, University of Oxford and RAND Europe for their comments on an earlier draft of the manuscript. None were compensated for their contributions.

Disclosures: Nothing to report.

References

Online-Only Materials

AttachmentSize
Microsoft Office document icon Supplementary Information (1)39 KB
   Comments ()