The article “Methodology for Measuring Health-State Preferences—I: Measurement Strategies,” published in the Journal of Clinical Epidemiology in 1989 by Debra G. Froberg and Robert L. Kane, serves as the foundational first part of a four-paper series dedicated to analyzing and critiquing the state-of-the-art in measuring preferences, particularly those related to health states. This seminal work addresses the crucial role that values play in decision-making within the healthcare system, impacting individuals (patients, consumers, providers) and policymakers alike.
The authors highlight the inherent complexity of determining values, noting that choices often involve trade-offs between desirable or undesirable outcomes, and values themselves can evolve over time or with experience. When moving beyond individual choices to policy-making, additional complexities arise regarding how to elicit and aggregate values from various constituencies in a defensible and just manner. Recognizing the importance of measuring individual preferences, the paper specifically focuses on health-state preferences, defining “preferences” or “utilities” as levels of subjective satisfaction, distress, or desirability associated with a particular health state.
The general process for obtaining health-state preferences typically involves three steps:
- Defining a set of health states of interest.
- Identifying a judge or group of judges to provide desirability judgments for each health state.
- Aggregating judgments to determine scale values for each health state.
Within this framework, researchers face several unresolved questions that this series of papers scrutinizes, including the selection of relevant health dimensions, how health states should be presented to respondents, which preference scaling method to use, how population groups might differ in their preferences, and how to control situational variables for more consistent and accurate values.
Selecting Relevant Health Dimensions and Measurement Strategy
A key initial decision is the selection of relevant attributes to describe health states. The research purpose dictates the scope of attributes, with a general rule of thumb suggesting no more than nine, and preferably fewer, as humans can simultaneously process only five to nine pieces of information. Examples of health attributes include physical function, social function, emotional well-being, pain, and cognitive ability. These attributes are then defined by multiple levels representing stepwise increments of functioning, often focusing on function rather than clinical diagnosis. Health states are formed by combining one level from each attribute, leading to numerous potential combinations, each with an associated “cardinal value” or “index value”. The primary challenge is obtaining these values, which leads to the choice of a measurement strategy.
The paper clearly distinguishes between measurement strategy and scaling method. The measurement strategy refers to the overall structure of posing questions to respondents (e.g., rating multiattribute states vs. rating attributes separately) and the corresponding data analysis method (e.g., regression, ANOVA). The scaling method, conversely, is the specific task required of the respondent to achieve scale values (e.g., standard gamble, rating scale).
Overview of Measurement Strategies
The authors discuss three main measurement strategies:
- Holistic Designs:
- This approach requires judges to assign scale values to each possible multiattribute health state.
- It was common in early pioneering work. Examples include Patrick and colleagues who sampled 400 case descriptions from thousands of possible combinations, and Sackett and Torrance who developed detailed scenarios for well-understood disorders.
- Limitations: Holistic approaches often assume equal-interval properties for scale values rather than empirically testing them, making it impossible to validate this assumption within the strategy. They also do not provide information on how different attributes are weighted and combined by judges, and the burden on judges to rate a large number of states is significant.
- Decomposed Designs:
- These designs greatly reduce the number of subjective judgments needed to assign scale values to a complete set of health states. They express the overall value of a health state as a decomposed function of its attributes.
- Explicitly Decomposed Models (Multiattribute Utility Theory – MAU):
- These procedures ask respondents to evaluate each level of a particular attribute independently, assuming other attributes are constant, thus requiring fewer multiattribute judgments.
- The conditional utility function method is a prominent example, involving three subtasks: checking independence assumptions among attributes, assessing utility functions for single attributes, and measuring multiattribute states to determine scaling constants. Independence conditions (utility, mutual utility, additive utility) dictate the appropriate model form (multilinear, quasi-additive, or additive).
- Practical difficulties: Checking independence assumptions is “tedious, exacting, and time-consuming,” often modified in practice due to the extensive interviewer-subject interaction required. Like holistic designs, MAU approaches do not provide a way to validate the weights, utilities, or the model itself, nor the scale properties beyond definition. Torrance et al. provide an example of a modified MAU method application.
- Statistically Inferred Decomposed Models:
- These models aim to develop an algebraic model of the decision-maker’s preferences from a set of multiattribute judgments. They require fewer subjective judgments than holistic models and allow for the analysis of individual attribute effects.
- Functional Measurement:
- This approach, central to the paper’s recommendation, simultaneously tests theories of information processing and measures scale values. It posits that subjective constructs can only be measured within a valid theory.
- It employs factorial designs and analysis-of-variance (ANOVA) procedures. If the data support the model’s predictions (e.g., parallel lines in interaction plots for an additive model), the subjective stimulus and response scale values can be derived, and the interval property of the scale validated.
- Key advantage: Functional measurement is the only approach that simultaneously validates the process by which judges combine attributes, the scale values they assign to health states, and the interval property of the scale.
- Shortcomings: Logistical challenges arise with many attributes and levels, potentially requiring complex “fractional” designs. It also demands technical expertise in experimental design and ANOVA, and interpreting interactions can be difficult. Reliability of multiattribute judgments may decline with a larger number of attributes. Despite these, it has been successfully applied in studies of health-state preferences, showing its practical utility.
- Multiple Regression:
- Used to infer parameters of statistical models, often estimating subjective weights and scale values of an additive utility model.
- Major limitation: It does not test the validity of the scale values themselves, assuming their validity from direct scaling procedures. The multiple correlation coefficient (R²) is deemed an inadequate test of scale values, as a high R² can still coexist with significant and systematic deviations from model predictions.
Conclusion and Recommendation
The paper concludes by emphasizing that while attribute selection is purpose-driven, limiting attributes to nine or fewer is prudent for multiattribute judgments. Among the discussed strategies, the functional measurement approach is technically superior. It stands out as the only method that concurrently validates the attribute combination process, the derived scale values, and the interval property of the scale. While its practicality with a large number of attributes remains a consideration, fractional designs offer solutions for reducing respondent burden.
Finally, the authors briefly touch upon the implicit assumption that individual preferences can be aggregated to form social preferences by calculating the arithmetic mean. They caution that this area involves significant debate within social choice theory, particularly regarding Arrow’s Impossibility Theorem, the appropriateness of interpersonal comparisons of utility, and equity considerations, which are beyond the scope of this initial paper but crucial for policy applications.
This article thus lays critical groundwork for understanding the complexities and technical requirements for accurately and reliably measuring health-state preferences, strongly advocating for the functional measurement approach as the most scientifically robust methodology.
Reference for the article:
Froberg, D. G., & Kane, R. L. (1989). Methodology for measuring health-state preferences—I: Measurement strategies. Journal of Clinical Epidemiology, 42(4), 345–354.