Please analyze the seminal article “Identifying achievable benchmarks of care: concepts and methodology” published in the International Journal for Quality in Health Care in November 1998. This paper, authored by Catarina I. Kiefe, Norman W. Weissman, Jeroan J. Allison, Robert Farmer, Michael Weaver, and O. Dale Williams, is critical for understanding a data-driven approach to quality improvement in healthcare.
The authors begin by highlighting that while benchmarking is prevalent in healthcare quality improvement, existing benchmarks are often based on subjective assessments rather than objective data, potentially failing to represent genuinely achievable levels of excellence. To address this, they introduce and describe the evolution of their Achievable Benchmark of Care (ABCTM) methodology, which is founded on process-of-care indicators.
A core contribution of this article is the postulation of five premises for sound benchmarks:
- Benchmarks should signify a level of excellence.
- They must be demonstrably attainable and clinically realistic.
- High-performing providers should be selected from all providers in a predefined, data-driven way using reliable data.
- All high-performing providers should contribute to the benchmark level.
- Providers with high performance but small numbers of cases should not unduly influence the benchmark level.
The ABCTM utilizes a “pared-mean” method to identify top performance, defining it as the mean of the best care achieved for at least 10% of the population. This method specifically addresses the challenge of avoiding undue weight being assigned to providers with few cases, which could otherwise inflate the benchmark. An example of its application in the Cooperative Cardiovascular Project (CCP) is provided, illustrating its computation for indicators like smoking cessation counseling rates. For instance, in the Alabama pilot CCP, the overall pooled mean counseling rate was 14%, while the ABCTM identified a benchmark performance level of 49% by selecting the top-performing hospitals that accounted for at least 10% of eligible patients.
The article further discusses refinements to the methodology, particularly concerning small denominators or low numbers of patients for a given measure. It introduces the Bayesian Estimator technique to calculate an Adjusted Performance Fraction (APF), effectively reducing the impact of providers with small case numbers and allowing all data to be used. Another refinement involves identifying a “minimum sufficient denominator” (MSD) to prevent distortion when aggregating performance across multiple indicators, especially for providers with very small denominators.
Kiefe et al. emphasize that the ABCTM method is deliberately applied to process-of-care indicators because they represent the most “actionable” aspects of quality measurement. This focus avoids the complex issue of risk-adjustment that typically arises when comparing outcomes and benefits from the higher sensitivity of process measures in detecting differences between providers.
Ultimately, this methodology provides a data-driven, objective, and reproducible approach to establishing achievable benchmarks, moving beyond arbitrarily defined tools. The authors suggest that this approach enhances the perceived validity of provider profiles used for feedback and continuous quality improvement.
APA Reference:
Kiefe, C. I., Weissman, N. W., Allison, J. J., Farmer, R., Weaver, M., & Williams, O. D. (1998). Identifying achievable benchmarks of care: concepts and methodology. International Journal for Quality in Health Care, 10(5), 443–447.
