Literature reviews are fundamental to scientific research, serving as crucial tools for advancing knowledge by systematically collecting, describing, analyzing, and integrating vast amounts of information and data. They are indispensable for informing critical aspects such as policymaking, curriculum development in medical education, accreditation standards, and ultimately, patient care. However, the full potential of these knowledge syntheses can be compromised when the underlying methodology, philosophical foundations, and markers of rigor for specific review types are not clearly understood. This article delves into a groundbreaking six-step methodology for conducting State-of-the-Art (SotA) reviews, a distinct and powerful form of knowledge synthesis whose methods have, until recently, remained largely undefined.
The original research, published in Perspectives in Medical Education, addresses a significant void in the existing literature. While numerous types of literature reviews exist, the precise methodology for SotA reviews has been conspicuously absent. Despite their widespread use across fields such as biomedical sciences, medicine, and engineering, SotA reviews are rarely explicitly labeled or fully appreciated for their unique contributions within medical education. This lack of a clear, articulated methodology is problematic because it conceals the specific purpose and the extensive work involved in a SotA synthesis, often leading to a perception of lacking rigor. The authors’ study directly tackles this foundational problem by inductively developing a comprehensive and clearly articulated methodology for SotA reviews. Their analysis of 940 articles labeled as SotA reviews between 2014 and 2021 revealed that an overwhelming 98% (923 articles) lacked citations or references explaining how to conduct such a review, underscoring the critical need for this new methodology.
The Distinctive Nature and Purpose of State-of-the-Art Reviews
SotA reviews offer a unique approach to knowledge synthesis, distinguishable from other review types like systematic or scoping reviews. While systematic reviews focus on narrow, specific research questions and scoping reviews map the current state of knowledge, SotA reviews provide a broader, chronologically rooted narrative synthesis. Their core purpose is to answer three fundamental questions: “This is where we are now. This is how we got here. This is where we could be going.”.
Unlike other reviews that might simply report on existing data, SotA reviews delve deeply into the historical development of a body of knowledge. They explore:
- Why knowledge evolved in a particular way.
- What other directions investigations might have taken.
- What turning points in our thinking should be revisited to gain new insights.
This interpretive synthesis allows researchers to critically analyze the evolution of understanding, question the reasons behind its unfolding, and propose novel directions for future research. For example, a SotA review could examine how medical education’s understanding of learner assessment shifted from a measurement problem, to one embracing human judgment and expertise, and now to a whole-systems problem, or alternatively, how it was historically constrained by available media and is now poised for an AI-driven transformation.
The Philosophical Foundations: Relativism and Subjectivism
A pivotal finding of the study is the explicit identification of the philosophical underpinnings that govern SotA reviews: relativist ontology and subjectivist epistemology.
- Relativist Ontology: This perspective is crucial for understanding the “reality” that SotA reviews explore. It acknowledges that the literature on any phenomenon presents multiple perspectives and interpretations. Reality itself, from a relativist standpoint, cannot be fully perceived objectively because it is a subjective and inter-subjective construction. This construction is heavily influenced by researchers’ individual experiences, expectations, specific research purposes, social orientations, and disciplinary contexts. Consequently, no single objective truth is believed to exist; instead, reality is shaped by social and experiential factors. A researcher’s interpretation in a SotA review thus reflects their unique conceptualization of the literature, which can evolve over time and even conflict with others’ understandings. For instance, a review of gender identity literature would likely be synthesized differently by gender studies scholars compared to those in sex reassignment surgery.
- Subjectivist Epistemology: SotA reviews fundamentally embrace subjectivism, meaning the knowledge generated through the review is value-dependent. It arises directly from the subjective interpretations of the researcher(s) conducting the synthesis. The generated knowledge is deeply informed by the researchers’ expertise, experiences, social contexts, and the specific historical point in time when the review is conducted. This reflects the perspective that knowledge is influenced by individuals and their communities, is not static, and is subject to change over time.
The Six-Stage State-of-the-Art Review Methodology
Based on their inductive analysis, the authors constructed a six-stage SotA review methodology meticulously designed to align with its relativist and subjectivist underpinnings. This robust framework provides clear steps for researchers to follow:
- Stage 1: Determine initial research question and field of inquiry.
- This initial stage involves establishing the topic to be summarized and defining the specific field of knowledge or practice the review will address. The generated knowledge is inherently shaped by its context, making the domain identification a foundational task. For example, a researcher might ask: “How has thinking about Interprofessional Education (IPE) evolved?”.
- Stage 2: Determine timeframe.
- Researchers conduct a broad-scope overview of the literature to gain insights into the historical development of knowledge on the topic, identifying key turning points that define current thinking. This deep understanding is crucial for justifying the “now” in the SotA’s “where we are now” argument. This stage is complete when the researcher can explicitly justify the starting point for “state-of-the-art” thinking, such as the World Health Organization defining IPE in 2010.
- Stage 3: Finalize research question(s) to reflect timeframe.
- Drawing on the insights gained in Stage 2, the initial research question(s) are refined and formalized. The revised topic description, final research question(s), and justification for the chosen timeline’s start year must be reported as prerequisites for rigor. An example question could be: “What is the state-of-the-art way of conceptualizing and realizing IPE?”.
- Stage 4: Develop search strategy to find relevant articles.
- This stage involves crafting a search strategy to identify literature for inclusion. Crucially, because SotA reviews describe “how we got here,” the strategy must include literature that predates the defined “state-of-the-art” timeframe to offer the necessary historical perspective. This is an iterative process, and consulting a librarian is highly recommended to expedite searches and ensure comprehensive resource identification. The search strategy must be explicitly reported (e.g., in the manuscript or supplementary file) to allow for replication, serving as another marker of rigor.
- Stage 5: Analyses.
- The literature analysis reflects the researcher’s subjective insights but is guided by inductive research principles. It begins with familiarization, noting similarities, observing evolving ways of thinking, identifying assumptions, decision points, and knowledge gaps.
- Researchers then generate premises about the historical development, current understanding, and future directions. They must document articles that support or contradict their premises, noting dominant authors or schools of thought, and actively searching for marginalized viewpoints. Theory can be incorporated to shape insights. Importantly, not all articles from the corpus will be used; instead, researchers will sample across the corpus to construct a timeline of seminal moments.
- To ensure thoroughness, researchers verify their interpretations by examining if different corpus articles reflect their premises or by seeking out contradictory interpretations. The goal is not objective triangulation, but rather to ensure the interpretations accurately represent the synthesized articles and respond to other interpretations. This consideration and reporting of differing interpretations is a key marker of rigor for SotA reviews.
- Stage 6: Reflexivity.
- Given the relativist and subjectivist nature of SotA reviews, a robust reflexivity description is essential. This description articulates how the researcher’s own subjectivity informed their interpretations of the data. These reflections also shape the suggested future directions presented in the “where we could go next” part of the argument, constituting the final marker of rigor for SotA reviews: researcher reflexivity must be considered and reported.
Conclusion
This explicit methodology description is vital, particularly as many academic journals—such as Educational Research Review, the American Academy of Pediatrics, and Thorax—already list SotA reviews as an accepted type of literature review. While valued by academia, the absence of clear guidelines has hindered their rigorous application. By providing a detailed account of the ontology, epistemology, and a six-stage methodological process, this article empowers researchers, especially in medical education, to rigorously conduct SotA reviews and generate unique insights that advance the field’s research and scholarship. SotA reviews provide invaluable information on the historical progression of understanding, pivotal turning points, and potential new directions for future research, offering a powerful lens through which to question past decisions and forge new investigative paths.
Reference for the Article:
Barry, E. S., Merkebu, J., & Varpio, L. (2022). State-of-the-art literature review methodology: A six-step approach for knowledge synthesis. Perspectives in Medical Education. https://doi.org/10.1007/s40037-022-00725-9