Developing Discrete Choice Experiments for Digital Health Uptake

The core objective of the paper is to enhance methodological awareness of stated preference methods, particularly DCEs, within the burgeoning field of digital health. It provides a practical guide for researchers who may lack prior experience with this quantitative methodology.

What is a Discrete Choice Experiment (DCE)? A DCE is a stated preference method that helps elicit individuals’ preferences, especially when observational data is unavailable or difficult to collect. It operates on the principle that individuals make trade-offs when making decisions, ultimately choosing an alternative that offers the greatest utility or benefit. This survey-based experimental design presents participants with hypothetical scenarios, known as choice tasks, where they must select a product or service from two or more alternatives. Each alternative is defined by a set of characteristics, called attributes, which have various attribute levels. By observing these choices, researchers can quantify the relative strength of preferences for different attributes.

Relevance to Digital Health: Understanding user preferences for digital health products, such as smoking cessation or physical activity apps, is crucial for improving their uptake (selection, download, and installation). Digital health tools often suffer from low uptake, and DCEs offer a powerful way to understand what aspects of a product (e.g., presentation, design, features) drive user choice. While widely used in health economics, DCEs are currently underused in digital health. This tutorial aims to address this gap by demonstrating its application using a case study focused on smoking cessation apps.

Key Stages in Developing a DCE, as outlined in the tutorial: The paper meticulously describes the stages involved in a DCE, drawing on established recommendations:

1. Establishing Attributes and Levels: This critical step involves identifying relevant characteristics (attributes) for the product or service, typically through literature reviews and qualitative research. Attributes can be quantitative (e.g., cost) or qualitative (e.g., design). The tutorial suggests limiting attributes to less than 10, ideally 5-7, to manage cognitive load. Attribute levels must be defined with a wide enough range to ensure participants perceive meaningful trade-offs. In their case study, the authors identified five attributes for smoking cessation apps: monthly price, developer, star ratings, app description, and images shown.
2. Constructing Choice Tasks: Researchers decide between full-profile (displaying all attributes) or partial-profile tasks. For five attributes, a full-profile approach is often feasible. The inclusion of an opt-out alternative (e.g., “Neither of these”) is recommended to make choices more realistic, simulating real-world scenarios where users might choose not to engage with any offered option.
3. Developing the Experimental Design: This systematic process generates the choice sets presented to respondents, aiming for a high-quality dataset.
   • Analytical Model Specification: This involves defining the statistical model (e.g., multinomial logit (MNL) model) that will be used for data analysis, based on the random utility framework and Lancaster’s theory of consumer demand.
   • Main Effects or Interaction Effects: Researchers typically focus on main effects in novel research, examining the impact of each attribute level on the choice variable.
   • Labeled or Unlabeled Experiment: The case study used an unlabeled approach, comparing different presentations of the same app rather than distinct alternatives.
   • Number of Choice Tasks and Blocking: To minimize respondent burden, the number of choice tasks is limited (e.g., 18 per participant maximum). Blocking is a technique to partition large experimental designs into smaller subsets, reducing the number of tasks any single respondent completes. The case study administered 12 choice tasks per participant across four survey versions.
   • Type of Choice Matrix Design: Due to the high number of combinations in a full-factorial design, a fractional-factorial design is typically used to create a representative sample of possible alternatives.
   • Efficient Design: The tutorial highlights a shift towards efficient designs (non-orthogonal) over orthogonal designs, especially for studies with five or more attributes, to increase the precision of parameter estimates. It details three types: Dz-efficient (zero priors), Dp-efficient (fixed priors), and Db-efficient (Bayesian), which uses probability distributions for parameters. The case study utilized a Dp-efficient design for piloting and then generated a Db-efficient design for the final DCE, as it generally offers superior performance with reliable priors.
4. Piloting the DCE: A crucial step to ensure clarity, realism, and manageable cognitive load for participants. Pilot data can provide valuable “priors” for generating the final efficient design. The authors’ pilot study with 49 individuals led to reordering attributes to improve engagement with text-heavy elements.
5. Internal Validity: Assessing internal validity, for example, through stability validity tests (repeating a choice task), helps understand the consistency of participants’ decisions.

Benefits of DCEs: DCEs offer several benefits in digital health, including the ability to quantify preferences for complex decisions, investigate the uptake of new interventions in hypothetical scenarios, and capture preferences under controlled experimental conditions. They are particularly valuable for situations where empirical measurement of uptake predictors is challenging.

Limitations of DCEs: Despite their advantages, DCEs face challenges, primarily concerning external validity. The hypothetical nature of choice sets can lead to an “intention-behavior gap,” where stated preferences may not perfectly align with real-world decisions. Testing external validity remains difficult, with limited studies addressing it in healthcare and none, to the authors’ knowledge, in digital health. Furthermore, a single choice set may not present all relevant attributes, necessitating extensive preparatory research.

Conclusion: In conclusion, the paper asserts that DCEs hold significant potential for digital health research, serving as an important decision-making tool in an area often characterized by a lack of observational data. It aims to equip researchers with the knowledge needed to develop and apply these experiments effectively.

Reference for this article:

Szinay, D., Cameron, R., Naughton, F., Whitty, J. A., Brown, J., & Jones, A. (2021). Understanding Uptake of Digital Health Products: Methodology Tutorial for a Discrete Choice Experiment Using the Bayesian Efficient Design. Journal of Medical Internet Research, 23(10), e32365. https://www.jmir.org/2021/10/e32365