This article by Lisa Wallander, titled “25 years of factorial surveys in sociology: A review,” offers an easy-to-follow overview of how the factorial survey approach (FSA) has been applied within sociology between 1982 and 2006. The review is based on an analysis of 106 articles published in prominent sociology journals, categorizing their findings across three main sections: factorial survey applications, factorial survey designs, and factorial survey analyses.
The factorial survey approach, first developed by Peter Henry Rossi and systematically presented in “Measuring Social Judgments” (1982), is designed to uncover the social and individual structures underlying human judgments of social objects. It is characterized as a “hybrid” technique that skillfully integrates the rigorous principles of experimental design with the rich, detailed data collection capabilities of social surveys. At its core, the method involves presenting respondents with standardized vignettes, which are fictive, detailed descriptions of social objects or situations where specific characteristics (dimensions) are systematically manipulated.
A key advantage of the factorial survey approach over traditional survey research lies in its ability to:
- Study the contexts and conditions that influence judgments by presenting concrete scenarios with systematically varied factors.
- Reduce social desirability bias, as respondents are less likely to be fully aware of the manipulated elements within the vignettes.
- Uncover the actual determinants of human judgments, especially when individuals may not consciously recognize what influences their decisions.
- Disentangle the unique effects of highly correlated variables, a common challenge in real-world social phenomena.
Wallander’s review highlights that the FSA has been most frequently applied in the subdiscipline of crime, law, and deviance, accounting for nearly half of the reviewed publications, though it has also been used across various other sociological fields such as family and social welfare, and social differentiation. The approach has been used to investigate a broad range of judgments, including normative judgments (e.g., recommendations, responsibility, fairness), positive beliefs (e.g., definitions, outcome estimations), and individuals’ estimations of their own intended actions, feelings, and thoughts. While the general population often serves as respondents, there’s also a significant body of research focusing on professional judgments (e.g., from nurses, judges, teachers). The majority of these studies originate from the United States, indicating a concentrated use within a specific research community.
In terms of design, the review notes the widespread use of ordered categories for rating tasks (e.g., scales), though other formats like number matching and open-ended questions are also present. Vignette designs vary considerably in the number of dimensions and levels included, demonstrating the method’s flexibility. Simple random sampling with replacement is the most common strategy for drawing vignettes, where each respondent evaluates a unique subset of scenarios.
For data analysis, regression analysis is the most frequently employed statistical method, often used to model both the “social component” (effects of vignette dimensions) and the “individual component” (subgroup variations in judgments). The article particularly emphasizes the increasing adoption of multilevel analysis (e.g., hierarchical regression, random effects models) for its ability to correct for intrarater correlation and to effectively explore consensus and variation in human judgments by decomposing unexplained variance.
Despite its advantages, the review suggests that the factorial survey approach remains underutilized and less known among sociologists, partly due to its infrequent inclusion in research methods textbooks. However, Wallander concludes by highlighting the approach’s significant potential, especially for comparative studies across professions or countries, and its future application in longitudinal research to link judgments with actual behaviors and outcomes.
Reference: Wallander, L. (2009). 25 years of factorial surveys in sociology: A review. Social Science Research, 38(2), 505–520.
