Featured Article:

by Sarah S. Stith and Richard A. Hirth

Performance standards are designed to ensure a basic level of quality, and through public reporting of firm performance, encourage firms to compete on quality thus allowing the market to determine the optimal level of quality. In markets with substantial excess demand, however, demand effects may be insufficient to induce any change in firm behavior and enforcement may be required to ensure high quality. Even with enforcement, quality still may not improve at underperforming firms if gaming the system is less costly than improving quality. We test whether information alone or with regulatory enforcement improves outcomes or elicits gaming behavior in our study of 266 kidney transplant centers between 2001 and 2012. In a context of excess demand induced by price controls, we show that information alone has no impact and enforcement may actually increase market inefficiencies; firms respond to costly quality requirements, not by improving quality, but by reducing supply, which exacerbates the disequilibrium between supply and demand, and by cream-skimming, which reduces access to transplantation among sicker patients.

The following section is a Q&A with Professors Stith and Hirth

1. Please briefly describe the main results of your paper.

Performance standards are used to ensure a minimum level of quality. Such standards are increasingly used in health care in order to maintain quality and potentially reduce costs through the delivery of higher quality care. Our study shows that in markets with substantial excess demand, e.g., many healthcare markets, such standards may not be enough to maintain quality. Public reporting of information alone had no effect on quality in organ transplantation. Enforcement in addition to public reporting does affect producer behavior, but primarily leads to gaming behaviors.

2. How does your focus differ from previous explorations of the effect of performance standards on healthcare provider behavior?

The prior literature in health economics focuses primarily on public reporting of healthcare provider performance. Our research analyzes not only the effect of information provision, but the effect of enforced quality standards as well. Perhaps clarifying the mixed effects of public reporting found in the literature, we find no effect from information alone. Gaming behaviors appear only with enforcement of performance standards.

3. What are the most important managerial and public policy implications of this work? 

As shown by our study and the existing literature, the simple provision of information to consumers is unlikely to improve performance. In contexts with sufficient excess demand, such standards must to be enforced to elicit any response from producers. Even enforcement alone may not lead to the desired policy outcome. In order to produce a higher quality product, a producer can either improve the production process or increase the quality of the raw materials. Policymakers typically seek the former, but unless safeguards are in place, firms may find it cheaper to simply use higher quality raw materials. In so far as managers or policy makers seek to ensure the quality of the production process, they must identify measures of the quality of the production process that can be distinguished from simply the use of better inputs.

4. Why does evidence from kidney transplantation serve as a particularly apt source of data for this topic?

We are aware of no other complex medical procedure with as rich a history in terms of policy variability and data availability. Survival-based performance has been tracked by regulatory authorities since 1991. Beginning in 2001, oversight authorities began publishing performance metrics online. In 2007, the federal government began enforcing specific standards with the threat of audit and potential closure. Detailed data, including hospital names, are available for all patients receiving transplants in the U.S. since October 1987 (subject to a confidentiality agreement). With 266 kidney transplant centers that met our inclusion criteria, our sample size was large enough to allow us to perform the type of rigorous statistical analysis needed to support our conclusions.

Apart from the high quality of the data, the study of kidney transplantation alone is important. Kidney disease is a leading cause of death worldwide, and particularly in the U.S. In 2009, Medicare spent 6.7% of its total expenditures paying for the treatment of kidney failure. Absent a kidney transplant, the only treatment for kidney failure is dialysis. After five years, a kidney transplant actually costs less than ongoing dialysis, while rendering a significantly higher quality of life.

5. In your paper, you state that “the issues we raise (topped out measures, intrinsic competition on quality/performance, gaming of measures) have been shown to arise both theoretically and empirically in healthcare and beyond.” To what fields other than healthcare might your findings be particularly relevant? What directions might you suggest for future research?

These three issues could arise in almost any market. At some point, there is usually little room for measurable improvement. In most markets, firms maximize profits by competing, at least to some extent, on the basis of quality rather than price.  Economic theory tells us that if it is cheaper for firms to game a performance measure rather than actually improving performance, they will.

Two very interesting areas of future research are the dynamic effects of performance standards and the creation of quality measures that better target the desired behavioral changes. We are currently investigating whether performance standards decrease innovation or lead to reversion to prior practice patterns. The creation of better quality standards is certainly an open area for further investigation, given that most of the literature finds little benefit from the types of performance standards currently in use.

Sarah Stith

Richard Hirth