Generative Participatory Design for eHealth Interventions: A Review

The background to this review highlights the increasing importance of stakeholder participation in the development of electronic health (eHealth) interventions. Despite its recognized value, persistent challenges such as gaining stakeholder trust, managing multiple stakeholders, and effectively involving end-users continue to plague eHealth development. In this context, generative participatory design (PD) emerges as a particularly promising approach. PD distinguishes itself from traditional user-centered design by actively involving all stakeholders in creative activities, rather than just limited user input. From a research-through-design perspective, understanding the rationale behind choosing specific forms of generative PD is crucial for advancing its methodology. However, the existing scientific literature is notably ambiguous regarding the exact forms of PD employed in eHealth development and, more critically, the specific methodological arguments used to justify these choices. As PD evolves into a recognized research methodology, a clear understanding of its core methodological elements—including the recruitment and management of stakeholders, the strategic use of tools, and the application of outcome measures—becomes indispensable. The authors point out a general lack of robust methodological explanations in the literature, which in turn impedes the establishment of a more rigorous science of PD. Therefore, making methodological decisions more explicit and providing clear justifications could significantly enhance the scientific rigor of PD as a research methodology.

The primary objective of this systematic review was to conduct a comprehensive exploration of how generative PD methodologies are reported and substantiated within empirical eHealth studies published in scientific journals. The ultimate aim was to contribute to the further development and refinement of PD methodology within the rapidly evolving field of eHealth. This study was specifically designed as a foundational step to assess the current state of reporting PD research methodology in peer-reviewed journals, with the potential to guide researchers and practitioners toward areas requiring greater methodological substantiation.

To achieve this, a systematic literature review with qualitative synthesis was meticulously conducted, adhering strictly to Cochrane guidelines, considered the gold standard in the medical field. The review also incorporated a Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) reporting statement to ensure completeness and transparency. An experienced medical information specialist, Wichor M. Bramer, developed the search queries, which included terms like “participatory design,” “co-design,” “cocreation,” and “collaborative design” in the field of telehealth, along with descriptive terms like “human centeredness” and “patient involvement”. Major databases, including EMBASE, MEDLINE Ovid, Web of Science Core Collection, and CINAHL EBSCOhost, were systematically searched from their inception up until November 12, 2019. After removing duplicates, an initial 3131 studies were identified.

A specific working definition of PD was employed for study selection: PD refers to the collective creative design process involving designers and non-designers, where users are explicitly considered partners throughout the design process. Studies were included if they described PD-related activities, even if using other terms like co-design or cocreation, provided that PD tools were explicitly described as part of the methodology. Key inclusion criteria included English language full-text empirical studies (including full conference papers) describing the use of PD to develop eHealth, with PD activities taking place in a group setting. Exclusion criteria covered non-empirical studies (e.g., reviews, editorials, dissertations), studies where users were treated as subjects (user-centered design), those only using qualitative research tools without PD activities, or studies where all PD tools were used by individuals only. After screening, 69 studies were ultimately selected for qualitative synthesis. The primary reasons for full-text exclusion included studies not being empirical or peer-reviewed documents, or mentioning PD-related activities but lacking descriptions of specific PD tools.

Data extraction focused on three key methodological elements: the recruitment and management of stakeholders, the specific use of PD tools, and the outcome measures employed. Studies were categorized into different design phases—predesign, early design, or post-first prototype—based on the stage at which the study began. An assessment of the sufficiency of reporting was also conducted using an 8-item checklist, covering aspects like setting, stakeholders, facilitators, procedure, materials, intensity, schedule, and clarity of the PD process description.

The results of the qualitative synthesis revealed significant insights. Overall Findings: The selected 69 studies covered a broad range of 65 unique eHealth technology products and services, primarily developed between 2006 and 2019. A large diversity was observed in the health domains addressed, with mental health being the most frequent, followed by disease-specific interventions and self-management tools. This prevalence of self-management aims was considered aligned with PD’s democratic principle, which emphasizes user involvement and may facilitate later uptake of eHealth solutions. Nearly all studies (65 out of 69) explicitly referenced a theory of PD, with prominent references to the work of Clemensen et al., Sanders and Stappers, and Simonsen et al.. Reporting Variability: The reporting of PD methods was highly variable, largely dependent on whether the study explicitly stated that reporting on the PD process was a major aim. Studies that emphasized reporting the PD process generally scored highest on the reporting scale. Stakeholders:

• Types of Stakeholders: All 63 studies that reported on stakeholders involved the main intended users (patients, care professionals, or both), including young adults and children in some cases. Other stakeholders like informal caregivers, designers, software developers, and researchers were also involved, though less frequently reported. Some studies even included advisory groups or specialized professionals like dieticians, psychologists, business analysts, or government representatives.
• Recruitment: Reporting on recruitment primarily focused on patients or content experts, with limited explanation for recruiting designers or software developers, who might have been part of the project team already. The most common strategies were purposive or convenience sampling, followed by snowball or in-person recruitment. Only one study used representative sampling, and five aimed for diversity. Recruitment criteria often focused on age and health care exposure, with some also including internet access, basic computer skills, and, notably, personal traits such as social skills, communicative abilities, creativity, motivation, and active engagement capabilities. Financial incentives were also frequently used. A significant finding was the general lack of methodological arguments provided for recruitment choices, making it unclear why certain stakeholders were included or excluded, or how the criteria served PD aims. For example, methodological arguments referring to principles like mutual learning to justify selecting communicative and motivated participants were rarely explicit.
• Stakeholder Management: Various approaches were adopted to manage stakeholders, often implicitly considering PD principles like democracy and creativity. Techniques included creating safe environments (e.g., small groups, reassurance, flat communication structures), shortening sessions for chronically ill patients, using icebreakers or games, and providing moderation or training. Some studies actively managed group mixing, such as separating health professionals and patients to address power imbalances, or mixing them to foster cross-fertilization of perspectives. Measures to stimulate creativity included providing blank cards for intuitive representations, active facilitation, and “world cafés” to increase perspective diversity. While arguments related to PD principles were sometimes provided for stimulating creativity, further substantiation linking these to design goals was often missing. Tools:
• A wide variety of PD tools (categorized as make, tell, and enact) were utilized across different design phases. The predesign phase showed the greatest combination of tools, often adopting a generative approach to produce new ideas (e.g., 2D mapping, brainstorms, prototyping, personas, cards, storyboarding, scenarios, service blueprints, role-play, design journals). Specific techniques like “thinking aloud” for participatory prototyping and “card sorting” (CARD) for tell tools were mentioned, along with “Design studio,” “Scaffold,” “good enough model,” and “future workshops”.
• The substantiation for tool selection typically revolved around four types of methodological arguments related to knowledge development:
  1. Gathering information or developing ideas to improve product/service design (Type 1).
  2. Arguments based on analogy to other PD literature where similar tools were used for similar design aims (Type 2).
  3. Explaining the aim of tools to capture specific types of knowledge (e.g., tacit, latent, emotional experiences, visualizing actionable information, merging knowledge domains) (Type 3).
  4. Relating the knowledge advantage of tools to the specific stakeholders involved (Type 4), such as using design experts to select tools or involving clinicians to enhance their views. This focus on knowledge arguments was expected given its implicit role in PD, though its explicit categorization in terms of argumentation levels was a novel finding. Outcome Measures:
• A notable lack of reported outcome measures was identified for evaluating both the eHealth product development and the PD process itself. Of the 50 studies with positive or effective outputs, only 22 reported outcome measures, predominantly related to usability and user feedback for eHealth quality. Some studies measured the number or quality of ideas generated, or clinical parameters as user outcomes.
• For the few studies that did report outcome measures, methodological arguments for their selection were generally missing. While some studies implicitly aimed to evaluate the knowledge development process through measures like idea grouping or understanding of new technology, explicit links to PD principles like mutual learning or creativity were rare.
• Only 3 studies out of 55 that considered the PD method successful reported outcome measures for evaluating the PD process itself, focusing on the quality of knowledge development (e.g., unique ideas) and stakeholder management (e.g., “voices heard”). This limited reporting is consistent with findings from other systematic reviews.

In conclusion, this systematic literature review highlights a critical need for greater transparency and methodological substantiation in the reporting of PD within eHealth development. While the choice of PD tools is generally well-substantiated, the selection and management of stakeholders and the chosen outcome measures receive considerably less methodological justification. The authors propose that the mixed origins of PD (from social science to design sciences) and a prevailing emphasis on “design reporting” over “scientific reporting” likely contribute to this reporting gap, where design products are prioritized over methodological explanations. The study advocates for improving education on scientific documentation for designers and eHealth developers, urging them to adopt a more scientific attitude towards PD projects by explicitly documenting methodological choices. Furthermore, it calls for the development of a more rigorous methodological framework for PD, with a particular focus on the knowledge development process, to enhance the rigor and accountability of PD science and ultimately lead to the creation of more effective eHealth interventions. This approach could provide clearer guidelines for selecting appropriate PD forms for specific research designs and help translate design terminology into scientific terminology for better reporting.

APA Reference:

Vandekerckhove, P., de Mul, M., Bramer, W. M., & de Bont, A. A. (2020). Generative Participatory Design Methodology to Develop Electronic Health Interventions: Systematic Literature Review. Journal of Medical Internet Research, 22(4), e13780. https://doi.org/10.2196/13780