This article, titled “Structured methodology review identified seven (RETREAT) criteria for selecting qualitative evidence synthesis approaches,” was authored by Andrew Booth, Jane Noyes, Kate Flemming, Ansgar Gehardus, Philip Wahlster, Gert Jan van der Wilt, Kati Mozygemba, Pietro Refolo, Dario Sacchini, Marcia Tummers, and Eva Rehfuess, and was published in the Journal of Clinical Epidemiology in 2018. The article addresses a critical challenge in qualitative evidence synthesis (QES): the existing deficit in guidance for selecting the most appropriate QES method, which can lead to a mismatch between review aims and optimal methods. The authors note that the field of QES has experienced rapid development and change, often with promotion of specific approaches preceding empirical testing of their utility.
The primary objective of this structured methodology review was to compare and contrast different methods of qualitative evidence synthesis (QES) against criteria identified from the literature. The aim was to map the attributes of these methods to inform the selection of the most appropriate QES method for various qualitative research questions. The ultimate goal was to develop a framework that would assist both reviewers and those commissioning reviews in choosing the most suitable QES method.
To achieve this, the researchers employed a comprehensive methodology.
- They used electronic databases, citation searching, and a study register (specifically the Cochrane Qualitative and Implementation Methods Group (CQIMG) Methodology Register) to identify relevant studies.
- Attributes were compiled from 26 methodological papers published between 2001 and 2014, forming a framework for data extraction.
- The synthesis and analysis phase utilized a variant of the “best fit framework synthesis approach”. This involved a three-stage process:
- Mapping and analysis of domains from key methodological texts against an initial, experience-based framework called TREAD (Time, Resources, Expertise, Audience, Data).
- Expanding this preliminary framework to incorporate additional data, which led to the creation of the new RETREAT framework.
- Reviewing a wider body of methodological literature against the newly established RETREAT framework.
- The study primarily focused on QES methods that are predominantly qualitative, such as Thematic Synthesis, Meta-Ethnography, and Meta-Study, while acknowledging but excluding mixed-methods approaches for the scope of this particular paper.
The key finding of the review was the identification of seven fundamental considerations that influence the choice of QES methods, which are succinctly summarized by the mnemonic RETREAT. These criteria are:
- Review question: This is a critical consideration that determines the type of data needed. Questions can be fixed (“anchor”) or emergent (“compass”), impacting the choice between aggregative (e.g., Meta-Aggregation) and interpretive (e.g., Meta-Ethnography) QES methods.
- Epistemology: This criterion emphasizes the importance of aligning the synthesis method with the philosophical foundations of the primary studies. QES approaches can be positioned along an idealist-realist continuum, affecting the review process and interpretation.
- Time/Timescale: While not the sole determinant, time and duration moderate method selection. Factors like methodological complexity, the volume of literature, and conceptual richness influence the time required. Some methods (e.g., Meta-aggregation, Best fit framework synthesis, Thematic synthesis) facilitate quicker approaches.
- Resources: This includes the collective skills and effort of the review team, as well as financial and physical support. Certain methods may benefit from or require specialist software.
- Expertise: All QES methods require generic synthesis skills and topic expertise. However, some methods, such as Meta-Ethnography, demand a high degree of qualitative research skill and familiarity with the specific method.
- Audience and purpose: Understanding the intended audience (e.g., policymakers, practitioners, patients) and the purpose of the review (e.g., informing practice, enhancing understanding) is crucial. This influences the choice of method and the presentation of findings, determining whether outputs are more descriptive or interpretive.
- Type of Data: This refers to the conceptual richness (potential for theoretical development) and contextual thickness (situational context) of the included studies, as well as their quality and quantity. “Thin” data, for example, may limit choices to methods like Meta-Aggregation or Thematic Synthesis, while “rich” data might suit more interpretive methods like Meta-Ethnography.
The authors then mapped 15 different published QES methods against these seven RETREAT criteria. They highlight that the Review Question and Audience and Purpose are often privileged considerations, with Resources, Time, and Expertise acting as secondary moderating factors. The underlying Epistemology also plays a crucial role in shaping the review.
This study is significant as it represents the first known criterion-based approach to inform the selection of QES methods. The findings provide a “contemporary perspective” that establishes a conceptual foundation for future empirical research into the advantages and disadvantages of different QES methods. The authors express their hope that this framework will lead to more appropriate selections of QES approaches and serve as a catalyst for ongoing empirical research and debate within the methodological community. They also suggest that the RETREAT framework could potentially be applied to document the characteristics of other knowledge synthesis approaches beyond qualitative research. In situations where the RETREAT framework does not offer a clear path, the authors propose a “risk-averse” strategy, recommending thematic synthesis due to its accessibility and its potential to serve as an initial stage for meta-ethnography if the source data proves sufficiently rich.
Reference: Booth, A., Noyes, J., Flemming, K., Gehardus, A., Wahlster, P., van der Wilt, G. J., Mozygemba, K., Refolo, P., Sacchini, D., Tummers, M., & Rehfuess, E. (2018). Structured methodology review identified seven (RETREAT) criteria for selecting qualitative evidence synthesis approaches. Journal of Clinical Epidemiology. https://doi.org/10.1016/j.jclinepi.2018.03.003