Introduction:
“Today, we will delve into a crucial paper published in Health Economics in 2004 by Gerald Richardson and Andrea Manca, titled ‘Calculation of quality adjusted life years in the published literature: A review of methodology and transparency.’ This article is highly relevant to anyone interested in economic evaluations in healthcare, particularly the methodology behind estimating health outcomes. The authors highlight that while the poor quality of costing methodology in economic evaluations alongside clinical trials has been acknowledged, the estimation of health outcomes, specifically Quality Adjusted Life Years (QALYs), requires equally appropriate and transparent methods.
Key Objectives and Methodology: The primary objective of Richardson and Manca’s paper was to review how preference-based health status (utility) data are utilized to generate QALYs in published cost-utility analyses (CUAs) conducted alongside Randomized Controlled Trials (RCTs). They aimed to assess the quality of the methodology and the transparency of reporting effectiveness, focusing on the measurement of mean differential QALYs. To achieve this, they conducted a literature review, identifying 23 published CUAs that met their inclusion criteria: studies conducted alongside RCTs and utilizing patient-level preference-based health status measures to value patient utility. A custom data extraction form was employed to systematically determine the specific methodology and the level of transparency in QALY estimation across these studies.
Core Findings and Identified Issues: The review unearthed significant concerns regarding QALY estimation practices in the published literature:
- Inconsistent and Poor Reporting: A central finding was that the methodology employed to calculate QALYs was not always consistent and was often poorly reported. This lack of transparency makes reproducibility difficult.
- Impact on Evaluation Results: The authors emphasize that the use of different methodologies in QALY calculation can significantly influence both the magnitude and direction of the evaluation’s results.
- Variability in Utility Measures and Preferences: The review found that the EQ-5D was the most commonly used measure of utility (in 16 studies), followed by the Health Utilities Index (HUI) (in 6 studies). However, QALY values are likely to differ depending on the utility instrument used, whose preferences were elicited (e.g., patients, general population, clinicians), and the technique used to measure those preferences (e.g., Time Trade-Off, Standard Gamble).
- Reporting Deficiencies: Many studies failed to report essential data, such as utility scores at baseline and follow-up for each trial arm (9 studies did not). Furthermore, reporting of variability around utility estimates (only 8 studies) and QALY estimates (only 10 studies) was notably poor.
- Assumptions on Utility Changes Over Time: While most studies implicitly used a linear assumption for utility changes over time (17 studies), this was often deduced by the reviewers rather than explicitly stated. Some studies lacked clarity or justification for their chosen assumption.
- QALY Calculation Methods: The Area Under the Curve (AUC) approach was most common (13 studies), but a significant number (5 studies) used “change from baseline”. The authors caution that the “change from baseline” approach, while potentially valid for some outcomes, may not be appropriate for QALY calculation as it is a function of time and utility, and can lead to regression to the mean.
- Missing Data: A prevalent issue was the inadequate handling of missing data. Most papers either did not acknowledge the problem or dealt with it insufficiently; only 5 papers used any form of imputation, and only one reported the impact of that imputation on results. This oversight can introduce substantial biases and lead to erroneous conclusions.
Recommendations and Conclusion: Richardson and Manca conclude with strong recommendations for future economic evaluations:
- Transparency and Consistency: Analysts must be consistent and fully transparent in their chosen methodology for QALY calculation.
- Advanced QALY Estimation: They recommend that a simple AUC calculation is inadequate. Instead, baseline differences in utility scores should be accounted for, preferably using a multiple regression approach.
- Handling Missing Data: Missing data must be acknowledged and addressed using an appropriate technique. Sensitivity analyses to assess the impact of imputation are crucial but were rarely performed in the reviewed literature.
- Data Reporting: For true transparency and reproducibility, baseline and follow-up utilities should be presented separately for each arm of the trial.
- Caution in Comparisons: The paper advises caution when comparing cost-per-QALY results across studies that employ different utility measures or methodologies.
In essence, this paper serves as a vital call for improved methodological rigor and explicit reporting in QALY estimation, reinforcing the need for robust and transparent practices in health economic evaluations.”
APA Reference: Richardson, G., & Manca, A. (2004). Calculation of quality adjusted life years in the published literature: A review of methodology and transparency. Health Economics, 13(12), 1203–1210. https://doi.org/10.1002/hec.901
