This extensive editorial, “From Scarcity to Abundance: Scholars and Scholarship in an Age of Generative Artificial Intelligence,” published in the Academy of Management Journal in 2023, comprehensively addresses the transformative implications of generative artificial intelligence (AI) for academic scholarship. Authored by Matthew Grimes, Georg von Krogh, Stefan Feuerriegel, Floor Rink, and Marc Gruber, the piece serves as a vital exploration of both the profound opportunities and the significant perils these emerging technologies present to the academic landscape.
The authors precisely define generative AI as a class of machine learning technologies capable of producing new content—including images, text, audio, and videos—that closely mimics human-created output. They specifically highlight the considerable attention garnered by large language models (LLMs), such as ChatGPT and Microsoft Copilot, owing to their advanced ability to generate coherent and contextually relevant text based on user-defined prompts. As editors of a journal dedicated to advancing knowledge in management and organizations, they acknowledge the necessity of recognizing generative AI’s potential to accelerate knowledge production.
The editorial systematically identifies four critical stages of knowledge production where generative AI holds considerable promise for enhancing both efficiency and rigor:
- Knowledge Synthesis: The scholarly process frequently commences with the identification of theoretically interesting and practically significant research questions. Traditionally, developing expertise in synthesizing broad literatures to identify frontiers and opportunities for advancing knowledge takes years. Generative AI tools are now increasingly offering capabilities to significantly increase these efficiencies and accelerate the pace at which scholars can achieve them. Furthermore, while discerning interesting research questions is often considered a domain of human creativity, recent studies indicate that models like GPT-4 can outperform a high percentage of people (e.g., 90% and 99% in different creative tests) on various creative tasks, suggesting a potential for generative AI to augment or even replace academics in some creative aspects of scholarship.
- Knowledge Development: This stage involves collecting, modeling, and analyzing data, followed by crafting parsimonious arguments. Generative AI is poised to profoundly impact the methodological process underlying knowledge development by increasing both efficiency and rigor.
- Efficiency: Generative AI is expected to lower barriers to methodological competence and expedite single research tasks. For instance, scholars can leverage it to assist with research design and instrument design (e.g., survey questionnaires, interview guides). In data analysis, simple prompts could enable sophisticated inductive exploration of large quantitative or qualitative datasets. Combined with knowledge synthesis, prompts could quickly surface credible hypotheses and facilitate systematic testing, potentially leading to an “instant research paradigm” in management research.
- Rigor: Generative AI can enhance reliability and validity in several ways. For experimental research, it can produce augmented data to test hypotheses across various scenarios, ensuring consistent and reliable findings. It can identify inconsistencies in data, thereby improving data quality and result reliability. It can also create complex simulation models or scenarios that are challenging to examine in real-world conditions, potentially enhancing external validity. Moreover, it can serve as a source of researcher feedback, identifying potential biases or errors in methods that could undermine reliability and validity. Beyond data, these tools can improve argumentative clarity through outline generation, language enhancement, and proactively identifying and responding to counterarguments, thus increasing the persuasiveness of scholars’ work.
- Knowledge Evaluation: Just as generative AI can help authors identify deficiencies in their analysis and argumentation, it can also assist journal editors and reviewers. It has the potential to provide additional feedback on manuscript quality and review quality, helping to mitigate peer-review concerns such as reviewer hostility, bias, and dissensus. Additionally, generative AI could be used to match reviewers based on expertise, thereby reducing bias stemming from a lack of topical or methodological familiarity.
- Knowledge Translation: This involves efforts to ensure the broad accessibility and impact of scholarly knowledge. While traditional translation often focuses on converting empirical findings into theoretical contributions, there’s an increasing call to close the theory-practice divide and ensure the responsible impact of scholarship. Generative AI promises to significantly increase the efficiency of such translation efforts through simple prompts, helping to overcome language barriers and bridge the gap between academic theory and practical application.
Despite these considerable opportunities, the authors meticulously outline manifold risks associated with the usage of generative AI:
- Lack of Transparency and Reproducibility: Generative AI, especially large language models, is trained on vast datasets, and its responses are formulated based on patterns rather than specific sources. The actual data and algorithms are often “black boxed,” making it challenging to understand how conclusions are reached. This lack of transparency impedes the assessment of reliability and validity of generated results and makes it difficult for other researchers to replicate findings or identify methodological flaws, particularly when proprietary systems are involved.
- Contextual Understanding and Accuracy: Current generative AI versions rarely offer and source evidence to substantiate arguments. Moreover, they do not yet truly understand the nuanced use of language in specific academic contexts in the way humans do. This limitation means generative AI would struggle to reliably provide or source the most current or relevant evidence, which is a significant concern where precision and accuracy are paramount.
- AI Hallucinations: Generative AI models are probabilistic and have a tendency to “hallucinate,” producing persuasive but factually inaccurate and misrepresented information without applying logical or contextual reasoning. If unchecked in academic contexts, such hallucinations could have serious repercussions, including an increase in retractions, damage to author, journal, and professional reputation, and the misleading of future research.
- Deep Research Fakes: The ability of generative AI to produce human-like text, images, and videos opens the door to creating “deep fakes” within academic scholarship. This includes the fabrication of both quantitative and qualitative data sets (e.g., fake experimental results, surveys, observational data), the production of fake citations, and the manipulation of images and graphs—all in support of false claims.
- Challenges in Reviewing and Evaluation: Should reviewers or editors increasingly use generative AI to summarize, critique, and evaluate manuscripts, journals face additional challenges in avoiding “false negatives” (rejecting potentially groundbreaking work) and “false positives” (accepting inaccurate, biased, or even fabricated studies).
Given the profundity of these risks, the authors argue that the assumption of good faith from authors, reviewers, and editors is inadequate. They call for additional and evolving governance designed to deter bad practice, potentially including AI training, specialized review protocols for AI-assisted papers, AI-assisted verification systems, and periodic audits.
The article then poses critical questions about the future of the academic profession itself. Currently, the profession operates on the assumption of scarcity of rigorous scholarly knowledge production, with journals gaining status from exclusivity and faculty promotions based on the perceived quality and quantity of scholarship. Generative AI, however, promises to lower the barriers for individuals to engage more efficiently in rigorous research, potentially leading to an “abundance” of knowledge. This shift necessitates addressing two fundamental questions:
- “What does it mean to be a ‘scholar’ when the ‘know-what’ and ‘know-how’ barriers to becoming one are minimized?”.
- “What does it mean to be a journal that publishes ‘scholarship’ when the field is flooded with manuscripts that meet the highest possible human-mediated standards for practical importance, theoretical intrigue, and methodological rigor?”.
To explore these questions, the authors engage in scenario planning, identifying two critical uncertainties affecting generative AI’s impact on the academic profession:
- Systems Transparency: Defined as the clarity and comprehensibility of the data and methods used in generative AI systems, enabling researchers to evaluate and replicate processes. The authors note that many privately owned models fall short of the conventional assumption requiring access to all data and model details.
- Societal Regulation: Refers to the extent of governmental legislation restricting the development or usage of generative AI, potentially through domain restrictions, mandated transparency, audits, data privacy controls, or watermarking outputs.
These two dimensions combine to form four distinct scenarios for the future of management academia:
- Scenario 1: Low Systems Transparency – Low Societal Regulation: In this scenario, generative AI systems are proprietary and opaque, and there are minimal legislative restrictions on AI usage. This leads to potential over-reliance on AI without full understanding, raising concerns about bias and misinformation. The credibility of AI-generated knowledge is questionable, but this may reinforce the need for human academic experts, thus keeping the academic profession’s exclusivity relatively stable.
- Scenario 2: High Systems Transparency – Low Societal Regulation: Here, AI developers prioritize transparency (e.g., open-source licensing), allowing comprehensive understanding of AI systems, but societal regulation remains limited. The rapid advancement of AI outpaces regulation, leading to widespread use and an exponential increase in knowledge production. This scenario could significantly lower barriers to knowledge creation and dissemination, potentially decreasing the exclusivity of the academic profession.
- Scenario 3: Low Systems Transparency – High Societal Regulation: This involves strong societal regulation to mitigate risks, but the AI systems themselves remain opaque. Academics are hesitant to rely on AI-generated knowledge without deeper understanding, limiting AI integration into research. Human academics retain a central role, with a focus on AI-human augmentation and preserving the profession’s exclusivity by emphasizing high standards and ethics.
- Scenario 4: High Systems Transparency – High Societal Regulation: This ideal scenario involves novel AI models developed with strong transparency (inspectable, shared, well-documented, open-sourced algorithms) and high societal regulation. This ensures biases and errors can be identified and corrected, and AI-generated research undergoes thorough evaluation. While initial adoption may be slow, it becomes increasingly exponential. The role of human academics shifts from traditional knowledge production to knowledge verification, oversight, and translation, with a focus on ensuring impact. The exclusivity of the academic profession might decrease as journals become more likely to accept and trust knowledge produced and synthesized by accessible AI tools.
In conclusion, the authors emphasize that this investigation is not a defense of the profession’s current boundaries, but rather a call to prepare scholars—including current and future PhD students—with the necessary knowledge to ethically and transparently use and, more critically, evaluate algorithmic knowledge production. This includes training in the risks, ethics of transparent usage, and methodological competencies for ensuring scholarly integrity with powerful, yet currently opaque, tools. In the long term, the editorial calls for a fundamental rethinking of the distinctive value of the academic profession in a world of abundant management scholarship, urging scholars to consider: “To what problems in society is management scholarship the (unique) solution?”.
Reference: Grimes, M., Von Krogh, G., Feuerriegel, S., Rink, F., & Gruber, M. (2023). From Scarcity to Abundance: Scholars and Scholarship in an Age of Generative Artificial Intelligence. Academy of Management Journal, 66(6), 1617–1624. https://doi.org/10.5465/amj.2023.4006