In an era increasingly focused on pay-for-performance programs and improved patient outcomes, the accuracy of healthcare quality measures is paramount. A pivotal study by Paul C. Tang and colleagues, published in the Journal of the American Medical Informatics Association, highlights a critical flaw in current quality measurement methodologies and proposes a transformative solution: shifting from administrative claims data to richer, more accurate clinical data from Electronic Health Record (EHR) systems.
The Problem: Inaccurate Patient Identification with Claims Data
The study reveals a significant and concerning discrepancy: most commonly used quality metrics, traditionally derived from administrative claims data, fail to accurately identify the target patient population. These claims-based measures often require a patient to have at least two visits with a specific encounter diagnosis during the measurement period. However, this approach has severe limitations:
- Only 75% of diabetic patients were identified when using standard administrative definitions (requiring two visits with a diabetes diagnosis), compared to the “gold standard” of manual EHR review. This means a quarter of actual diabetic patients were missed.
- This two-visit requirement can disqualify a significant number of patients who genuinely have the disease, potentially because diabetes was not the primary focus of every visit or not all diagnoses were entered for billing.
- Prior studies have also documented significant disparities between claims data and clinical databases, with agreement rates as low as 0.09 in some cases, leading to misleading results.
The Solution: Harnessing the Power of EHR Clinical Data
In stark contrast, the research demonstrates the superior accuracy of using coded clinical information from EHRs:
- 97% of diabetic patients were correctly identified using coded data within the EHR system, reflecting a much more comprehensive and reliable picture of the patient population.
- Diabetes was most often identified through a specific diagnosis on the EHR’s problem list (93%), followed by anti-diabetic medications or lab results.
- This method effectively leverages the clinical content of EHRs without adding burden to the care process.
Significant Biases in Quality Reporting
These differences in patient identification have profound implications for reported quality measures. The study found statistically significant differences in critical quality measures for diabetic patients when comparing those identified by administrative data versus clinical data:
- Frequency of HbA1c testing (97% for those with two or more visits for diabetes vs. 68% for those with fewer than two visits, p < 0.001) [2, 23, 29, Table 5].
- Control of blood pressure (61% vs. 45%, p < 0.05) [2, 23, Table 5].
- Frequency of testing for urine protein (85% vs. 55%, p < 0.001) [2, 23, Table 5].
- Frequency of eye exams (62% vs. 41%, p < 0.03) [2, 23, Table 5].
This evidence suggests that using administrative data for performance measures can significantly bias quality reports, potentially overestimating the quality of care delivered by inadvertently selecting patients already actively receiving care. This “self-fulfilling prophecy” can lead to organizational attention being misdirected, rather than focusing on areas where true improvement is needed, such as identifying untreated or unrecognized conditions.
Call to Action: A New Standard for Quality Measurement
The authors, affiliated with the Palo Alto Medical Foundation and Lumetra, and funded by the Centers for Medicare & Medicaid Services (CMS), emphasize the urgent need for change:
- New standardized quality measures must shift from claims-based to clinically-based measures that can be derived from coded information in EHRs.
- Policymakers should design measurement systems and incentive programs that capitalize on computer-based information systems, which are becoming the new standard of care.
- To encourage this transition and account for the inherent bias of claims-based measures, a temporary adjustment or premium should be built into incentives for reporting quality measures based on actual clinical data from EHR systems. This would also incentivize faster adoption of EHRs.
- There is a clear need for standardizing relevant codes and requiring EHR products to adopt data standards that support quality measurement.
Conclusion:
Accurate and reliable performance measures are critical for improving patient care and effective resource allocation. By embracing and leveraging the rich, physician-entered clinical data generated as a byproduct of care within EHRs, the healthcare system can achieve a more truthful, efficient, and impactful approach to measuring quality, ultimately benefiting patients and providers alike.
Reference:
Tang, P. C., Ralston, M., Fernandez Arrigotti, M., Qureshi, L., & Graham, J. (2007). Comparison of methodologies for calculating quality measures based on administrative data versus clinical data from an electronic health record system: Implications for performance measures. Journal of the American Medical Informatics Association, 14(1), 10–15. https://doi.org/10.1197/jamia.M2198
