Electronic health (eHealth) and mobile health (mHealth) interventions present a powerful, cost-effective avenue for delivering comprehensive, ongoing, and tailored support to enhance public health. Despite their inherent advantages, a persistent challenge in these digital health initiatives is frequently reported low levels of adherence and high rates of attrition. Acknowledging this, researchers and developers have increasingly called for the design and implementation of more engaging interventions. However, a clear, unified understanding of what constitutes engagement and how it contributes to intervention effectiveness has remained elusive in the literature.
Bridging Conceptual Gaps for Deeper Understanding The insightful viewpoint article, “Measuring Engagement in eHealth and mHealth Behavior Change Interventions: Viewpoint of Methodologies,” by Short et al. (2018), provides a critical framework to address these ambiguities. The authors highlight that engagement has often been conceptualized either narrowly as a psychological process (user perceptions and experience) or purely as a behavioral construct (intervention usage), leading to confusion with concepts like adherence. To foster a more coherent foundation for future research, the article draws upon two pivotal conceptual models—by Yardley et al. and Perski et al.. These models establish engagement as a multifaceted construct encompassing both the extent of intervention usage and a subjective user experience related specifically to attention, interest, and affect. Key distinctions are made between:
- Microlevel engagement: This refers to the moment-to-moment interaction with the intervention, including the actual extent of use (e.g., number of activities completed) and the user’s experience (e.g., level of interest and attention during activities).
- Macrolevel engagement: This denotes the depth of involvement with the overarching behavior change process (e.g., the extent of motivation for changing behavior), directly linking to the intervention’s behavioral goals.
A crucial takeaway from these models, emphasized by Short et al., is that intervention usage, while valuable, is not considered a valid indicator of engagement in the behavior change process on its own. This underscores the need for more nuanced measurement approaches.
A Comprehensive Toolkit for Enhanced Measurement The core contribution of this influential viewpoint is its comprehensive overview and critical appraisal of diverse methodologies available for measuring engagement in eHealth and mHealth behavior change interventions. Moving beyond the common but limited reliance on system usage data, the authors present a wide spectrum of methods, advocating for a multi-method approach to achieve a more complete and accurate understanding of user interaction:
- Qualitative Measures: Methods such as focus groups, observations, interviews, and “think-aloud” activities enable an in-depth exploration of both micro- and macrolevel engagement. They provide rich, detailed accounts of user experiences and perceptions of how interventions facilitate behavior change, making them excellent for hypothesis generation.
- Self-Report Questionnaires: These instruments are useful for assessing both experiential (subjective experience) and behavioral (intervention usage) aspects of microlevel engagement, as well as changes in psychological mechanisms (e.g., self-efficacy) related to macrolevel engagement. While offering a systematic and standardized approach, the article highlights a notable lack of validated, eHealth/mHealth-specific scales, with many studies relying on purpose-built questionnaires.
- Ecological Momentary Assessments (EMAs): EMAs involve delivering brief, real-time surveys to users via smartphones or wearable devices, either on-demand, at set intervals, or triggered by specific events. This method is particularly well-suited for capturing moment-to-moment or microlevel engagement and providing crucial contextual data, thereby reducing recall bias and enhancing ecological validity.
- System Usage Data: The most frequently collected measure, system usage data quantitatively tracks how an intervention is physically utilized by participants. To move beyond simple frequency counts, the article suggests applying the FITT acronym (Frequency, Intensity, Time, and Type), commonly used in physical activity research. This framework allows for a multidimensional analysis of usage patterns, offering deeper insights into engagement by considering how often (frequency), how deeply (intensity), how long (time), and what kind of activities (type) users engage in. This sophisticated approach can reveal meaningful differences in engagement profiles even among users with similar overall time on site.
- Sensor Data: Unobtrusive sensors, including GPS, cameras (for eye-tracking), microphones, and accelerometers, embedded in smartphones or external trackers, can monitor user behavior and physical context. This data can enrich usage information, provide insights into macrolevel engagement (e.g., tracking real-life activity levels), and indicate intervention fidelity.
- Social Media Data: Analyzing users’ social media patterns offers another unobtrusive, low-burden method to gauge engagement. Social media message threads can yield valuable information on user experience (microlevel) and behavioral achievements (macrolevel). The use of unique, intervention-specific “markers” can significantly facilitate the identification and analysis of relevant social media conversations.
- Psychophysiological Measures: Techniques such as Electroencephalography (EEG) and eye-tracking are employed to examine the intricate relationship between physiological responses and cognitive or affective processes. These methods provide valuable insights into the experiential aspects of microlevel engagement, particularly attention and arousal, and are increasingly applicable in field settings.
Advancing the Science of User Engagement Short et al. strongly recommend the validation and widespread adoption of a broader array of engagement measurements. They underscore that relying solely on system usage data is insufficient, and a multi-method approach is essential to fully capture the complex, multidimensional nature of engagement. This comprehensive viewpoint serves as an invaluable resource for researchers, guiding the thoughtful application and validation of diverse engagement measures, thereby significantly advancing the field of eHealth and mHealth behavioral science.
Reference: Short, C. E., DeSmet, A., Woods, C., Williams, S. L., Maher, C., Middelweerd, A., Müller, A. M., Wark, P. A., Vandelanotte, C., Poppe, L., Hingle, M. D., & Crutzen, R. (2018). Measuring Engagement in eHealth and mHealth Behavior Change Interventions: Viewpoint of Methodologies. Journal of Medical Internet Research, 20(11), e292. https://www.jmir.org/2018/11/e292/