This paper, “Discrete choice experiments in health economics: A review of the literature” by Esther W. de Bekker‐Grob, Mandy Ryan, and Karen Gerard, provides a comprehensive update to prior reviews of Discrete Choice Experiments (DCEs) in the field of health economics, specifically covering the period from 2001 to 2008. DCEs have become a commonly used instrument in health economics, employed to address a wide range of policy questions.
The methodology of DCEs is an attribute-based measure of benefit, predicated on the idea that healthcare interventions, services, or policies can be described by their various attributes, and an individual’s valuation of these depends on the levels of those attributes. Respondents are asked to choose between two or more alternatives, and their choices reveal an underlying (latent) utility function. This approach integrates several theoretical components:
- Random utility theory (RUT).
- Consumer theory.
- Experimental design theory.
- Econometric analysis.
The review aims to systematically examine current DCE practice, compare it with previous findings (from 1990 to 2000), and delve into specific key methodological issues such as experimental design, estimation procedures, and validity of responses.
Methodology of the Review: A systematic review was conducted using Medline to identify English-language DCE studies published between 2001 and 2008. The search terms included “discrete choice experiment(s),” “stated preference,” “conjoint analysis,” and other related terms. Studies were included if they were choice-based, published as full-text articles, and applied to health care. The search identified 114 original DCE studies for the review period.
Key Trends and Developments (2001–2008 vs. 1990–2000):
- Increased Applications: There has been a significant increase in the number of DCE applications in health care, rising from a mean of 3 per year in the 1990-2000 period to a mean of 14 per year from 2001-2008.
- Geographical Spread: While the UK remained the primary user, the US, Australia, and Canada were also major contributors, and the technique was applied in 26 studies (23%) in countries where it had not been previously used, including high-, middle-, and low-income nations.
- Study Design Characteristics:
- The mean number of attributes incorporated decreased from 7 to 5, potentially reflecting concerns about task complexity and non-compensatory decision rules.
- Conversely, the mean number of choice sets per respondent increased from 12 to 14.
- There was a notable increase in interviewer-administered surveys, rising from 9% to 19%. Computerized interviews also saw a rise from 9% to 11%.
- Experimental Design and Choice Set Construction:
- Fractional factorial designs continued to dominate, being used in 100% of current studies.
- There has been a shift towards statistically more efficient designs. The use of SAS software to generate choice sets has increased (used in 14 studies or 12%), alongside other software like SPEED (19%) and SPSS (12%) for creating orthogonal main arrays.
- Foldover methods (11 studies) and D-efficient designs (14 studies, mostly using SAS) are increasingly being used.
- However, the review found no application of designs incorporating a priori assumptions for parameter estimates.
- A significant portion of studies did not report the source of the experimental design (37%) or sufficient detail on choice set creation (28%).
- Econometric Models for Estimation:
- There has been a clear shift towards more flexible econometric models that relax the restrictive assumptions of the Multinomial Logit (MNL) model, such as the independence of irrelevant alternatives (IIA) and taste homogeneity.
- The Nested Logit (NL) model was used in 5 studies in the current period (compared to none at baseline).
- Mixed Logit (MXL) models were employed in 6 studies (up from 1 at baseline), with all 6 finding evidence of preference heterogeneity and 4 reporting improved goodness of fit. MXL models allow for random taste variation, which is crucial for policy implications, though they require assumptions about parameters and distributions.
- The Latent Class Model (LCM) was applied in one study, demonstrating significant improvement in fit and allowing for easier interpretation of class probabilities compared to MXL.
- Despite these advancements, more advanced models accounting for scale heterogeneity, like Generalized-MNL, have not yet been widely explored in health economics.
- Reporting of Key Findings/Outputs:
- The reporting of monetary values (Willingness to Pay – WTP) continues to be popular, being used in 44 studies (54% at baseline, 54% current). WTP can be used in cost-benefit analyses.
- The use of utility scores has not gained widespread popularity, reflecting skepticism about their ordinal nature and difficulty in cardinal interpretation or aggregation across people.
- There has been an increasing use of odds ratios (e.g., in 9 studies) and probabilities (e.g., in 15 studies) as outputs. These are considered valuable at the policy level for investigating the likely take-up and acceptability of new interventions.
Validity and Qualitative Methods:
- External Validity: Limited progress has been made in testing for external validity, likely due to the challenges of investigating this in publicly provided health-care systems where consumers have limited choice and typically don’t pay at the point of consumption. Only one study (Mark and Swait, 2004) was identified that used stated preference (DCE) and revealed preference data to evaluate doctor’s prescribing decisions, concluding similar preference functions.
- Internal Validity: Tests of theoretical validity (whether parameters move in the expected direction) remain common (56% of studies) and encouraging. Non-satiation tests (49%) also continue to be applied, despite concerns. However, transitivity tests (a fundamental test of rationality) remain unpopular (4% of studies), possibly due to application difficulties. The debate continues regarding the deletion of “irrational” responses, with some arguing against it as these responses may be valid or modelable within the random utility framework.
- Qualitative Methods: There has been an increasing application of qualitative work to enhance the face validity of DCEs. Focus groups and interviews are commonly used to aid attribute and level selection, as well as questionnaire layout and comprehension (used in 69% of studies for attribute selection, 33% for level selection, and 32% for pre-testing). Some studies also employed qualitative approaches (like “think aloud” exercises or debriefing choices) to better understand respondent preferences. The review encourages the move towards mixed methods.
Areas of Application: DCEs have broadened significantly beyond their initial introduction in health economics to value patient experiences. Current applications include:
- Valuing patient/consumer experience factors (40 studies).
- Valuing health outcomes (8 studies).
- Investigating trade-offs between health outcomes and patient experience factors (38 studies).
- Estimating utility weights within the Quality Adjusted Life Year (QALY) framework (2 studies).
- Investigating labour-market choices among health professionals (5 studies).
- Developing priority setting frameworks (6 studies).
- Clinical decision-making and health professional’s preferences for treatment/screening options (17 studies).
Important Areas for Future Research: The paper highlights several crucial directions for future research to enhance the robustness and applicability of DCEs:
- Incorporation of interaction terms in the design and analysis of DCEs.
- Improved explanations of risk attributes to respondents, as understanding of risk remains a challenge.
- More rigorous and imaginative tests of external validity of responses, especially in publicly provided health-care systems where direct market comparisons are difficult. Exploration of laboratory, framed field, and natural field experiments is suggested.
- The incorporation of DCE results into a decision-making framework, such as economic evaluation models (e.g., cost-benefit analysis), to complement traditional valuation methods like QALYs.
- Further exploration of mixed methods (combining qualitative and quantitative approaches) to enhance face validity and deepen the understanding of responses.
- Research into models that account for scale heterogeneity (e.g., Generalized-MNL).
- Investigation of the optimal number of choice sets to reduce cognitive burden on respondents.
- Consideration of DCE transfers for generalisability of results across studies.
In summary, the review underscores the rapid growth and broadening application of DCEs in health economics, noting shifts towards more efficient designs and flexible econometric models. Despite these advancements, significant methodological challenges and crucial areas for future research remain, particularly concerning external validity and the integration of DCE results into real-world decision-making frameworks.
Reference: de Bekker-Grob, E. W., Ryan, M., & Gerard, K. (2012). Discrete choice experiments in health economics: A review of the literature. Health Economics, 21(2), 145–172. https://doi.org/10.1002/hec.1697
Concept Map: https://claude.ai/public/artifacts/af8187f0-7775-4031-a6f9-f7722282e72a