This document is an excerpt from a seminal paper titled “Evaluating the Quality of Medical Care” by Avedis Donabedian, originally published in 1966 in The Milbank Memorial Fund Quarterly and reprinted in 2005 in The Milbank Quarterly, Vol. 83, No. 4.
Purpose and Scope of the Paper The paper’s main purpose is to describe and evaluate current methods for assessing the quality of medical care and to suggest future directions for study. It focuses on the methodologies of assessment rather than specific findings, offering a general evaluation of how quality is appraised. Donabedian’s work primarily examines the evaluation of the medical care process at the level of physician-patient interaction. Therefore, it does not cover processes related to community-level delivery of care or administrative aspects of quality control. The paper also generally avoids discussing types of health care beyond physician-provided care and explicitly excludes the problem of economic efficiency as a measurable dimension of quality. It is not an exhaustive review but includes key studies as illustrative examples.
Key Areas of Discussion The author systematically explores various critical aspects of evaluating medical care quality:
Definition of Quality: The paper highlights the inherent difficulty in defining “quality of medical care,” noting that it often reflects values and goals within the medical system and society. It suggests that quality is not a unitary concept and likely has no single comprehensive criterion.
Approaches to Assessment: Donabedian introduces three fundamental approaches to assessing quality:
◦Outcome: This uses indicators such as recovery, restoration of function, and survival. While outcomes are often considered valid and concrete, their use is limited by factors beyond medical care, the time required for outcomes to manifest, and the challenge of measuring subjective outcomes like patient satisfaction. Despite these limitations, outcomes are considered the ultimate validators of effectiveness and quality.
◦Process: This examines the actual delivery of care, focusing on aspects like appropriateness, completeness of information, justification of diagnosis and therapy, and technical competence. This approach emphasizes whether “good” medical care has been applied and requires significant attention to specifying dimensions, values, and standards. Donabedian also mentions “procedural end points” as a related concept under process assessment.
◦Structure: This assesses the settings and instrumentalities through which care is provided, including facilities, equipment, staff qualifications, and administrative structures. The assumption is that proper settings lead to good care, though the relationship between structure and process or outcome is often not well-established.
Sources and Methods of Obtaining Information: The paper delves into how information is collected for quality appraisal:
◦Clinical Records: A common source for process studies, but their limitations include being sketchy in private practice, issues of veracity (accuracy), and completeness. A key challenge is separating the quality of recording from the quality of care provided. Supplementing records with physician interviews is discussed, though its effectiveness and validity are questioned.
◦Direct Observation: An alternative for general practice where records are inadequate, but it faces limitations such as potential changes in physician behavior due to observation and the difficulty of observing all dimensions of care.
◦Indirect Methods: These include sociometric approaches (e.g., physicians choosing care for themselves/family) and autoreputational approaches (e.g., assessing quality based on opinions of those involved with or connected to hospitals). The validity of judgments derived from these methods, however, often lacks external criteria.
Sampling and Selection: This section discusses specifying the universe to be sampled (e.g., care by providers, care received by people, or provider capacity) and the importance of homogeneity in care levels. Empirical evidence on homogeneity is presented, indicating correlations in performance across different components or diagnostic categories.
Measurement Standards: Standards are derived from two sources:
◦Empirical Standards: Based on actual practice and used for comparison, resting on attainable levels of care.
◦Normative Standards: Derived from legitimate sources of knowledge and practice, such as textbooks, expert panels, or highly qualified practitioners, representing “best” or “acceptable” care. The paper notes the debate around applying standards developed by specialists to general practice. Standards can vary in specificity and directiveness.
Measurement Scales: This discusses the use of qualitative categories (e.g., “excellent,” “good,” “fair,” “poor”) versus numerical scores for classifying care. A significant unsolved problem in numerical scoring is the arbitrary weighting of different components of care.
Reliability: The consistency of assessments, especially agreement between judges, is a major consideration. The paper notes that detailed specification of criteria and standards is the main mechanism for achieving higher reliability. However, it raises the critical question of whether increased reliability gained through rigid frameworks might come at the cost of reduced validity, as clinical situations have infinite variations.
Bias: Acknowledges that bias among observers or judges is common, with some consistently applying stricter standards than others. Precautions like randomization and masking the identity of physicians or hospitals under review are suggested to mitigate bias.
Validity: The ultimate validator of quality is the effectiveness of care in achieving health and satisfaction. The paper explains that the validity of other indicators depends on their relationship to health outcomes, with clinical sciences playing a key role in establishing these links. It also reviews studies on the relationship between structural properties and the process/outcome of care, highlighting their complexity and ambiguity.
Indices of Medical Care: This discusses the ongoing search for simple, readily measurable data (indices) to infer quality, acknowledging their convenience but questioning their validity.
Problems of Assessing Ambulatory Care: Mentions specific challenges such as poor record-keeping, prior physician knowledge of patients, and defining the appropriate unit of care (e.g., single visit vs. sequence of visits over time).
Donabedian concludes that despite the often-lacking rigor of past methods, current techniques are adequate for making broad judgments about quality, revealing a range from outstanding to deplorable. He proposes further refinements, emphasizing the need to: Better understand the impact of observers on observed practice and the reliability/validity of observation itself. Empirically test the presumed gains in reliability from explicit standardization against potential losses in validity. Study the reliability and bias when multiple judges are involved, as well as the reliability of repeated judgments by the same assessor. Broaden the definition of quality beyond technical management to include prevention, rehabilitation, coordination, continuity of care, and the patient-physician relationship. Engage in conceptual and empirical exploration of quality, including aspects like logical and economic efficiency and understanding how physicians define quality.
Shift focus from merely evaluating quality to concentrating on understanding the medical care process itself—how patients and physicians interact, and how physicians function. This deeper understanding is seen as foundational to attaching value judgments to elements of care and creating a more complex profile of quality rather than a single summary judgment.
The paper also briefly touches on the application of decision-making theory and the ongoing question of whether medical care performance is a homogeneous or heterogeneous phenomenon. Ultimately, Donabedian advocates for greater neutrality and detachment in quality studies, distinguishing values from the elements of structure, process, or outcome, and subjecting both to critical study.
