PROMIS Psychometric Summit: Advancing Measurement Methodologies

Discover the cutting-edge discussions and recommendations from the Third Patient-Reported Outcomes Measurement Information System (PROMIS®) Psychometric Summit in the special report, “Advancing PROMIS’s methodology: results of the Third Patient-Reported Outcomes Measurement Information System (PROMIS®) Psychometric Summit”. This essential article details the collaborative efforts of leading investigators to address the complex psychometric challenges in modern clinical research, driven by the National Institutes of Health (NIH) Roadmap for Medical Research.

PROMIS aims to create the next generation of patient-reported outcome (PRO) measures using Item Response Theory (IRT) and Computerized Adaptive Testing (CAT) to provide precise, meaningful measurements with a minimum number of questions. This report captures the pivotal discussions from the 2010 summit, focusing on the second wave of PROMIS studies (PROMIS II), which introduced new challenges with longitudinal data, sociodemographically diverse samples, and an increased focus on pediatric populations.

This article provides an in-depth review of five critical themes and the innovative solutions proposed:

1. Linking Scores Across Scales: Learn about advanced statistical methods for transforming scores from different measures onto a common metric. The summit highlighted calibrated projection, a full-information factor analytic approach that offers greater flexibility than previous methods by modeling two distinct but correlated constructs, which is crucial for linking pediatric and adult scales.
2. Differential Item Functioning (DIF): Explore the critical issue of measurement bias, where individuals with the same health level respond differently to questions based on variables like age or race. The article presents a comparative analysis of three sophisticated DIF detection methods—MIMIC models, a general latent variable approach (MG-MIMIC), and a hybrid ordinal logistic regression (OLR)/IRT framework—demonstrating how different models can lead to different conclusions and emphasizing the need for multiple analytical approaches.
3. Dimensionality: Understand the profound impact of multidimensionality (when data measures more than one construct) on IRT models. The report argues against simply forcing multidimensional data into unidimensional models and instead advocates for using bifactor models to evaluate and account for the influence of multiple factors, thereby preventing distorted IRT parameters. It also introduces the concept of a multidimensional CAT (M-CAT), which promises greater efficiency and validity by embracing complex data structures.
4. IRT Models for Longitudinal Applications: Delve into new models designed for data collected over time. The article describes two powerful approaches: hierarchical Bayesian models, which account for uncertainty at multiple levels (e.g., responses within individuals over time), and the two-tier full-information factor analysis model, which can effectively capture the multidimensional structure inherent in longitudinal data and model change over time.
5. New IRT Software: Get a preview of next-generation software like EQSIRT and IRTPRO, designed to make advanced IRT modeling more accessible and user-friendly. These tools integrate features for handling multidimensionality, longitudinal data, and DIF testing, empowering researchers to apply these sophisticated methods more easily.

This report is more than a summary; it’s a guide to the future of psychometrics in health outcomes research. It concludes with key recommendations for analysts, stressing the importance of evaluating model assumptions, using appropriate models for linking and dimensionality, and employing multiple methods to ensure measurement validity across diverse groups. The advancements discussed are set to improve the quality and application of PROs in clinical trials and practice, making this article indispensable for researchers, psychometricians, and clinicians in the field.

Reference:

Carle, A. C., Cella, D., Cai, L., Choi, S. W., Crane, P. K., Curtis, S. M., Gruhl, J., Lai, J.-S., Mukherjee, S., Reise, S. P., Teresi, J. A., Thissen, D., Wu, E. J., & Hays, R. D. (2011). Advancing PROMIS’s methodology: Results of the Third Patient-Reported Outcomes Measurement Information System (PROMIS®) Psychometric Summit. Expert Review of Pharmacoeconomics & Outcomes Research, 11(6), 677–684.