Structural model robustness checks in PLS-SEM

The article, “Structural model robustness checks in PLS-SEM” by Sarstedt, Ringle, Cheah, Ting, Moisescu, and Radomir (2019), is an invited commentary specifically aimed at enhancing methodological rigor in research fields that widely employ Partial Least Squares Structural Equation Modeling (PLS-SEM). PLS-SEM is a powerful analytical tool used across numerous scientific disciplines, from tourism and marketing to engineering, medicine, and psychology, to analyze complex relationships between observed and latent variables. Its popularity stems from its ability to handle complex models with relatively small datasets, estimate formatively specified measurement models, and produce determinate latent variable scores. Crucially, PLS-SEM excels in its prediction focus, bridging the gap between academic explanation and practical managerial implications, making it particularly appealing for applied science.

Despite the method’s widespread use and continuous advancements in PLS-SEM, the authors observe a slow adoption of certain complementary methods, particularly robustness checks, in fields like tourism research. This article aims to rectify this by illustrating the nature and application of these improvements, focusing on three key areas: nonlinear effects, endogeneity, and unobserved heterogeneity. The core idea behind robustness checks is to ensure that a model’s findings are reliable and valid, even when confronted with underlying complexities or alternative assumptions. Latan (2018) emphasized the urgent need for these issues to be addressed and reported in PLS-SEM results.

Delving into Robustness Checks in PLS-SEM

For someone new to these concepts, imagine you’ve built a sophisticated model to understand various factors influencing, for example, customer satisfaction. You’ve found some relationships, but how sure are you that these relationships hold true under different conditions or if there are hidden factors at play? Robustness checks are essentially these “quality assurance” steps for your model.

1. Nonlinear Effects

When analyzing relationships in path models, researchers typically assume they are linear, meaning a constant change in one variable leads to a constant, proportional change in another. However, this isn’t always the case in reality. If a relationship is mistakenly assumed to be linear when it’s actually nonlinear, the true relationship might be underestimated, or its effects could appear weak or non-significant. The size of an effect between two constructs in a nonlinear relationship depends not only on the magnitude of the change in the predictor but also on its current value.

The article outlines two main approaches to check for nonlinear effects:

• Ramsey’s (1969) Regression Equation Specification Error Test (RESET): This is a general test for nonlinearities.
  • How it works: Researchers first estimate their original PLS-SEM model. The resulting latent variable scores are then used as input for the RESET, which can be performed using standard statistical software like Stata or SPSS. The test typically considers quadratic and cubic effects by default.
  • Interpretation: A non-significant result from the RESET indicates that the model is not subject to general nonlinearities, thus supporting the robustness of the linear assumption. In the article’s empirical example, neither the partial regression of “Attitude” (ATT) on “Trust” (TRUST) and “Communicability” (COMM), nor “Intention to Purchase Travels Online” (IPTO) on ATT, COMM, and TRUST showed significant nonlinearities (F(3,494) = 0.58, p = 0.625 and F(3,493) = 1.54, p = 0.204, respectively).
• Including Interaction Terms (Quadratic Effects): To specifically test for quadratic effects, which are a common type of nonlinearity, researchers can expand their polynomial model by adding a quadratic term.
  • How it works: This quadratic term is conceptually similar to an interaction term where an exogenous construct interacts with itself (e.g., TRUST * TRUST).
  • Interpretation: If the effect of this interaction term is statistically significant, it indicates a nonlinear relationship. A positive coefficient suggests the exogenous construct’s effect strengthens at higher values, while a negative coefficient suggests it weakens. Conversely, a nonsignificant interaction term provides evidence for the robustness of the linear effect. The empirical example further supported the linear model’s robustness, as none of the tested nonlinear effects (e.g., TRUSTTRUST -> ATT, COMMCOMM -> ATT, ATT*ATT -> IPTO, etc.) were significant (p-values ranged from 0.130 to 0.789).

It’s important to note that a review of tourism and hospitality journals revealed that only a very small percentage (2.4% of 375 studies) actually estimated nonlinear effects, with most doing so through interaction terms or second-stage analyses using various techniques. This highlights the need for increased adoption of these checks.

2. Endogeneity

Endogeneity is a critical concern in regression-based methods like PLS-SEM, particularly for studies focused on explanation. It occurs when a predictor construct is correlated with the error term of the dependent construct it is meant to explain. This means the predictor isn’t just explaining the outcome; it’s also explaining part of the unexplained variance (the error). The most common source of endogeneity is omitted constructs – important variables that are left out of the model but correlate with both a predictor and the dependent construct.

• Why it’s a problem: If endogeneity is present and not accounted for, the coefficient estimates from the analysis will be biased, meaning they don’t accurately reflect the true causal effect. This can lead to incorrect conclusions, including Type I (false positive) or Type II (false negative) errors. For instance, researching visitor satisfaction on spending without considering all salient aspects could lead to endogeneity if an omitted aspect affects both satisfaction and spending.
• Systematic Procedure (Hult et al., 2018): The article advocates for a systematic procedure to identify and treat endogeneity in PLS-SEM.
  • Initial Step: Check for Non-normality: The procedure begins by verifying if the variables potentially exhibiting endogeneity are non-normally distributed, often using tests like the Kolmogorov–Smirnov test with Lilliefors correction. This is a prerequisite for the subsequent Gaussian copula approach. In the empirical example, the latent variable scores for ATT, COMM, and TRUST were found to be non-normally distributed, allowing the procedure to continue.
  • Gaussian Copula Approach (Park and Gupta, 2012): This method controls for endogeneity by directly modeling the correlation between the endogenous variable and the error term. The latent variable scores from the original model estimation serve as input for this approach.
  • Interpretation: If the Gaussian copula for a given predictor is non-significant, it indicates the absence of endogeneity for that relationship, supporting the robustness of the structural model results. In the example, none of the Gaussian copulas (for ATT, TRUST, COMM, or any combination of them as potentially endogenous predictors of IPTO) were significant (all p-values > 0.05), leading to the conclusion that endogeneity was not present.
  • Treatment (if endogeneity is found): If endogeneity is detected, researchers should implement instrumental variables. These are variables that are highly correlated with the independent (predictor) constructs but are uncorrelated with the dependent construct’s error term, helping to explain the sources of endogeneity.

Despite its relevance for explanatory studies, tourism researchers have largely overlooked endogeneity in PLS-SEM, with only one study in a comprehensive review explicitly addressing it.

3. Unobserved Heterogeneity

Unobserved heterogeneity refers to the presence of hidden subgroups within the data that have substantially different model estimates. If these subgroups exist, analyzing the entire dataset as a single, homogenous group can lead to misleading results, as positive and negative effects from different groups might cancel each other out. For example, the drivers of “intention to purchase travels online” might vary greatly between different consumer segments (e.g., based on age, tech-savviness).

The article recommends a multi-step systematic procedure (Sarstedt et al., 2017b) to identify and treat unobserved heterogeneity:

• Step 1: Finite Mixture PLS (FIMIX-PLS): This latent class technique is used to determine if unobserved heterogeneity is an issue in the data and to suggest the optimal number of distinct segments.
  • How it works: FIMIX-PLS is run, typically starting with a one-segment solution and then incrementally increasing the number of segments (e.g., up to a maximum number determined by minimum sample size requirements for each segment).
  • Model Selection Criteria: FIMIX-PLS generates various criteria to guide the decision on how many segments to retain. The article specifically highlights the importance of jointly considering modified Akaike’s information criterion with factor 3 (AIC3) and consistent AIC (CAIC). Other criteria include AIC4, Bayesian Information Criterion (BIC), and Minimum Description Length with factor 5 (MDL5). Additionally, researchers should consider the segments’ “fuzziness” (indicated by high values in the entropy statistic, EN) and ensure segment sizes meet minimum requirements.
  • Interpretation: If the metrics point to a one-segment solution or produce divergent results (e.g., AIC3 and CAIC suggest different numbers of segments), researchers can conclude that unobserved heterogeneity does not significantly affect the data, supporting the robustness of the overall dataset’s results. In the empirical example, the fit indices for one- to five-segment solutions were ambiguous: AIC3 suggested five segments, while CAIC, AIC4, and BIC pointed to two segments. MDL5 also suggested two segments, but this criterion is known to underestimate the number of segments. Given these divergent results, the authors concluded that unobserved heterogeneity was not at a critical level, supporting the robustness of the one-segment solution.
• Step 2 (if heterogeneity is found): PLS Prediction-Oriented Segmentation (PLS-POS): If FIMIX-PLS indicates a significant degree of heterogeneity (i.e., multiple segments), PLS-POS is used to uncover the specific segment structure and estimate group-specific parameters.
• Step 3 (explaining and testing differences): After identifying segments, researchers can then aim to explain the uncovered heterogeneity using other variables (e.g., age, income, sex) through methods like contingency table analyses and Chi-squared Automatic Interaction Detector (CHAID) analysis. A subsequent multigroup analysis, combined with a measurement invariance assessment, allows for testing significant differences in parameter estimates across these identified groups.

Despite the importance of latent class techniques for addressing heterogeneity, their application in PLS-SEM contexts within tourism research remains scarce. This article strongly encourages their routine use to gain a more nuanced understanding of complex relationships.

In conclusion, Sarstedt et al. (2019) emphasize that the continuous methodological advancements in PLS-SEM, particularly regarding robustness checks for nonlinear effects, endogeneity, and unobserved heterogeneity, are crucial for increasing the methodological rigor of research. By systematically applying these checks, researchers can ensure their findings are more robust, valid, and reliable, thereby strengthening the quality and impact of their contributions across various fields.

Reference: Sarstedt, M., Ringle, C. M., Cheah, J. H., Ting, H., Moisescu, O. I., & Radomir, L. (2019). Structural model robustness checks in PLS-SEM. Tourism Economics, 25(4), 1–24.