“Prepare to discuss the article titled “Development of the Leapfrog methodology for evaluating hospital implemented inpatient computerized physician order entry systems” authored by P.M. Kilbridge, E.M. Welebob, and D.C. Classen, and published in Quality and Safety in Health Care in 2006.
This paper addresses a critical issue in healthcare quality and safety: adverse drug events (ADEs), which are a leading cause of patient injury, accounting for 19% of all adverse events in the Harvard Medical Practice Study. More recent studies estimate that between 6.5% and 20% of hospitalized patients in the United States experience an ADE, with attributable costs ranging from $2000–$2500 per event, largely due to increased hospital stays. The 1999 Institute of Medicine report, “To Err is Human,” significantly raised public awareness regarding the frequency and cost of such adverse events.
In response to this pressing concern, the Leapfrog Group was formed in November 2000 by a coalition of healthcare purchasers, dedicated to advancing healthcare safety and quality in America. Their initial focus was on Computerized Physician Order Entry (CPOE) systems, chosen specifically for their potential to reduce patient harm from medications. The Leapfrog inpatient CPOE standard mandated that organizations operating CPOE systems must demonstrate, through a test, their ability to alert physicians to at least 50% of common serious prescribing errors.
This article outlines the development of a robust methodology to evaluate implemented CPOE systems’ effectiveness in preventing medication errors likely to lead to ADEs. Key aspects of this methodology include:
- A comprehensive framework encompassing 12 distinct categories of CPOE-based decision support designed to prevent prescribing errors. These categories cover areas such as therapeutic duplication, single and cumulative dose limits, allergies, contraindicated routes of administration, drug-drug and drug-food interactions, and contraindications based on patient diagnosis, age, weight, or laboratory/radiology studies. The methodology also evaluates for efficiency functions like nuisance alerts and cost of care.
- A sophisticated scoring system that interprets raw test results, reflecting the relative importance of each decision support type for patient harm prevention. This system assigns higher scores based on the severity of a potential ADE (life-threatening, severe, significant) and its likely frequency (most frequent, less frequent, least frequent), drawing on expert opinion and large-scale ADE surveillance studies.
- The use of simulated test patients and accompanying simulated test medication orders to assess the CPOE system’s ability to intercept prescribing errors across all 12 decision support categories.
- A web-based application allowing hospitals to self-administer the evaluation, with specific time constraints for programming test patients, downloading orders, entering orders, and reporting results, along with measures to prevent “gaming” the system, such as random selection of orders from a master set and periodic revisions.
- The methodology was validated at multiple inpatient sites utilizing both commercially available and custom-developed CPOE systems. This testing revealed a range of CPOE capabilities across organizations, noting that some categories, like drug-diagnosis interactions, did not generate alerts in any tested system, highlighting areas for future development.
The authors emphasize that this is the first systematic, evidence-based methodology for evaluating CPOE systems as they are actually implemented and operating in hospitals. It serves not only as a certification method but also as a quality improvement tool, providing hospitals with feedback on their system’s decision support capabilities and performance, including issues like “alert fatigue”.
This methodology is designed to complement the Leapfrog Group’s existing hospital questionnaire and is anticipated to become publicly available and required for hospitals and outpatient clinics demonstrating compliance with Leapfrog CPOE standards. Furthermore, it sets a precedent for the certification of other electronic health record (EHR) applications and products, aiming to ensure system benefits, define capabilities, promote interoperability, and reduce investment risk for providers.”
APA Reference:
Kilbridge, P. M., Welebob, E. M., & Classen, D. C. (2006). Development of the Leapfrog methodology for evaluating hospital implemented inpatient computerized physician order entry systems. Qual Saf Health Care, 15(2), 81–84.