The article, “Grand rounds in methodology: when are realist reviews useful, and what does a ‘good’ realist review look like?”, authored by Claire Duddy and Geoff Wong and published in BMJ Quality & Safety in December 2022, serves as an indispensable resource for researchers contemplating the use of a realist review approach. It not only introduces the fundamental principles and concepts involved in planning and executing a realist review but also elucidates its unique value, addresses prevalent challenges, and offers concrete solutions for producing high-quality work.
The growing significance and application of Realist Reviews Realist reviews are highlighted as an increasingly popular form of evidence synthesis that offers a theory-driven, interpretive approach to secondary research. This methodology is particularly pertinent in complex fields such as healthcare quality and safety, where a deep understanding of intricate phenomena or the multifaceted workings of interventions is crucial. Unlike aggregative reviews that primarily assess intervention effectiveness, realist reviews aim to construct explanatory theories about how, when, and for whom interventions ‘work’ or outcomes occur. For instance, it can help hospital management understand the complex causes of staff shortages or evaluate potential solutions for integrating new professionals, recognizing that outcomes are often variable and context-dependent. They are designed to answer questions that demand explanations taking into account complexity.
Three Pillars of Distinction: What sets Realist Reviews apart? The article meticulously outlines three important distinctive features that differentiate realist reviews from other evidence synthesis methods, particularly qualitative evidence synthesis:
- Explicitly Theory-Driven Approach: At its core, a realist review is designed to develop and refine “programme theory”. This theory explains what an intervention is expected to achieve and how it is supposed to work. The causal explanations generated are centred on Context-Mechanism-Outcome Configurations (CMOCs). For example, Friedemann Smith et al.’s work on “safety-netting” in primary care developed a programme theory demonstrating the contexts, mechanisms, and outcomes involved in successful patient safety. The ultimate aim is to identify solutions to complex problems by developing theory-informed responses, such as recommendations or new intervention designs. A realist review must start with an “initial programme theory” which is then refined and confirmed or refuted by evidence.
- Inclusion of Diverse Data Sources: Realist reviews exhibit significant flexibility in the types of evidence they incorporate. They can draw from any data source that contains relevant information for theory building, including qualitative, quantitative, mixed-methods studies, and grey literature. This broad scope is invaluable for comprehending complex issues, especially when traditional research literature is scarce. An example is Abrams et al.’s review on delegating GP home visits, which utilized evaluations, policy documents, commentary, and news stories due to the novelty of the intervention.
- Specific Realist Philosophy of Science (Ontology): This methodology is founded on a distinct ontological position that emphasizes generative causation. It postulates that outcomes result from “hidden, context-sensitive causal forces” called “mechanisms” which are activated by specific “contexts”. The helpful heuristic Context + Mechanism = Outcome (C+M=O) encapsulates this form of causal explanation, leading to the development of CMOCs. Mechanisms are described as underlying processes, often relating to the reasoning and behaviour of agents, and are not the same as interventions or variables. This philosophy also warrants the transferability of realist causal explanations, meaning that insights derived from one dataset may be applicable to other situations where similar mechanisms are believed to be at play. Outcomes in realist research are broadly understood, extending beyond final intended outcomes to include proximal, unintended, and desirable outcomes. Similarly, “context” is not limited to fixed demographic characteristics but is understood as anything that triggers or modifies a mechanism’s behaviour. Realist reviews view interventions as attempts to manipulate contexts to trigger desired mechanisms and outcomes.
Common Challenges and Misconceptions Despite their increasing popularity, researchers often face challenges when undertaking realist reviews. The article outlines several common pitfalls:
- Misconception as Qualitative Evidence Synthesis: Many researchers mistakenly treat realist reviews like other forms of qualitative synthesis, failing to fully appreciate generative causation, the purpose of CMOCs, and key realist concepts like mechanisms, context, and programme theory. This can lead to thematic analysis where contexts, mechanisms, and outcomes are reported as disconnected lists, missing the causal relationships that CMOCs are designed to explain.
- Misunderstanding Mechanisms: A frequent error is confusing a mechanism with an intervention or an interventional component. This misinterpretation can undermine the transferability of learning from the literature. For instance, Friedemann Smith et al.’s review clearly shows mechanisms as hidden processes (e.g., “patient has a sense of ownership”) that lead to an outcome (e.g., “safety-netting advice is adhered to”) when a specific context is present (e.g., “information is personally relevant and tailored”).
- Narrow Conceptualization of Context: Some reviewers define context too narrowly (e.g., only as physical settings or demographics), limiting the scope of their analysis and potentially overlooking crucial contexts that could be manipulated to alter outcomes.
- Absence of a Coherent Realist Programme Theory: While developing individual CMOCs is vital, their full explanatory value is realized when they are organized into a coherent realist programme theory. Without this overarching theory, CMOCs can appear disconnected and undermine the review’s explanatory power.
Recommendations for High-Quality Realist Reviews To overcome these challenges and ensure high-quality work, the authors provide several recommendations:
- Undertake Specific Training: Researchers new to realist reviews should undergo training to grasp the underlying philosophy of science, generative causation, and key concepts like contexts, mechanisms, outcomes, and programme theory.
- Seek Methodological Support: Ongoing guidance from an experienced realist methodologist or reviewer is crucial for translating abstract understanding into practical application and ensuring adherence to accepted practices.
- Embrace Flexibility (with Transparency): Realist reviews are iterative, allowing for protocol changes as long as they are transparently reported and respond to emerging data or knowledge gaps. This flexibility allows for a deeper focus on specific aspects of a problem.
- Adhere to RAMESES Standards: Researchers should utilize and cite the appropriate RAMESES (Realist And Meta-narrative Evidence Syntheses—Evolving Standards) quality and publication standards. These open-access standards guide conduct and reporting, clarifying that quality standards (for execution) and publication standards (for reporting) serve different purposes.
- Appropriate Assessment of Rigour: The article cautions against judging realist reviews with standards developed for other evidence syntheses, particularly regarding searching and quality appraisal. Realist searching focuses on identifying data for theory development, not exhaustive identification. Rigour is primarily assessed at the level of the programme theory, prioritizing its explanatory plausibility, coherence, consilience, simplicity, and analogy with existing knowledge, rather than the systematic application of critical appraisal tools to individual included documents. Tips for peer reviewers are also provided to help evaluate quality, such as checking for the presence of a programme theory and detailed, causally sound CMOCs.
In conclusion, this paper is an essential guide for researchers, practitioners, and policymakers navigating complex issues in healthcare quality and safety. It offers a robust methodological framework for understanding why and how things happen, ultimately facilitating the development of theory-informed, practical solutions to complex problems.
Reference for the Article:
Duddy, C., & Wong, G. (2022). Grand rounds in methodology: when are realist reviews useful, and what does a ‘good’ realist review look like? BMJ Quality & Safety, 32(12), 859–865. Advance online publication. https://doi.org/10.1136/bmjqs-2022-015236