This editorial, “From the Editors: Advantages of Starting with Theory” by Jason D. Shaw, published in The Academy of Management Journal, critically examines the pervasive practice of “HARKing” (hypothesizing after the results are known) within deductive hypothesis-driven quantitative research in the field of management. Shaw, as the editor-in-chief of The Academy of Management Journal (AMJ), aims to discuss the likely outcomes of post-result theorizing from the perspective of the journal’s review process, rather than focusing on ethical concerns or inductive theory building.
The author describes HARKing as a process where a researcher obtains a dataset, often with only a general or no specific research question, and then scours the data for unanticipated significant associations. This can involve looking for findings that deviate from conventional wisdom or running numerous models to identify signs of moderation, mediation, or both. Once “novel” and “significant” findings are identified, the researcher then constructs a narrative by searching for applicable theory to frame these findings as if they were predicted a priori using a hypothetico-deductive approach. This practice is common, with surveys indicating that about a third of psychology authors admit to it, and Shaw’s experience suggests it is prevalent in the management community as well.
While some authors might perceive HARKing as a “success-facilitating practice”, Shaw argues from his extensive editorial experience that it is generally a “rejection-creating” practice. He identifies four “telltale signs” of post-result theorizing that often lead to unfavorable commentary and rejection during the review process for quantitative papers:
- Contorted Theory: This occurs when the theory itself is unfit to support the predictions being made, or authors fail to adequately use the logic, assumptions, and central tenets of the evoked theoretical perspective. It also includes invoking multiple disparate theoretical perspectives to justify various predictions, a phenomenon sometimes referred to as the “too-convenient qualifier”.
- Poorly Defined Constructs: Retrofitting results to theory can lead to conceptual sloppiness and a lack of precision in defining constructs, which can be easily identified by reviewers with strong theoretical knowledge.
- Construct-Measurement Mismatch: There is often a disconnect between the constructs in the theoretical model and their actual operationalizations in the study, as variables might have been added, deleted, or modified during the search for a good-fitting empirical model.
- Theory-Design Mismatch: The study design or available measurements may not adequately capture the mechanisms suggested by the evoked theoretical framework, leading reviewers to suspect that the authors opted for a model that “worked” rather than one logically derived from theory. This often results in a Discussion section that merely restates findings instead of delving into theoretical implications.
Conversely, Shaw advocates for “starting with theory”, highlighting its significant advantages for authors in the hypothetico-deductive genre. This approach leads to less contorted and more coherent predictions, allows for more refined and accurate construct definitions, ensures measurements align with theoretical constructs, and results in a study design capable of properly testing underlying theoretical mechanisms.
Beyond these direct benefits, Shaw identifies two subtler advantages:
- Clarifying Theoretical Contribution: A strong theoretical frame from the outset helps authors identify and articulate their unique theoretical contribution, enabling them to produce novel and interesting work that challenges, changes, or advances existing theoretical understanding.
- Improving the Discussion Section: With a clear theoretical foundation, authors can more effectively offer thoughtful reflections on their study’s contribution to theory, articulating how it fundamentally refines understanding of extant theory and its core concepts.
Shaw acknowledges that there are specific situations where post-result theorizing can have a place at AMJ. Unexpected findings can serve as a starting point for further specific deductive theory development and follow-up quantitative studies. Additionally, authors can engage in “tharking”, which involves transparently discussing alternative results discovered in exploratory analyses within a dedicated section of additional findings in the Discussion, potentially enriching the paper and inspiring future theory innovations.
Ultimately, Shaw encourages authors of quantitative papers to carefully consider the advantages of a theory-first approach as an alternative to results-driven retrospective theorizing. He concludes that strong conceptual framing, proper study execution, conceptual clarity, construct-measure matching, and appropriate design features for testing theory will reduce rejection-creating commentary and contribute to higher quality theorizing and minimized biases in the literature.
Reference: Shaw, J. D. (2017). From the editors: Advantages of starting with theory. The Academy of Management Journal, 60(3), 819–822. https://doi.org/10.5465/amj.2017.4003