Constructive Technology Assessment in Health Care

The article, titled “Methodology of constructive technology assessment in health care,” was authored by Kirsten F. L. Douma, Kim Karsenberg, Marjan J. M. Hummel, Jolien M. Bueno-de-Mesquita, and Wim H. van Harten. It was published in the International Journal of Technology Assessment in Health Care, 23(2) in February 2007, pages 162–168. The publication addresses a critical gap in the traditional approach to evaluating medical innovations, particularly within the rapidly evolving fields of biotechnological and genetic research.

Core Problem with Traditional Health Technology Assessment (HTA) Since the 1970s, Health Technology Assessment (HTA) has been employed to inform decisions about implementing new technologies into clinical practice. While HTA is formally intended to be a broad assessment including organizational, social, economic, and ethical considerations, its focus often narrows to clinical efficacy and cost-effectiveness analysis, influenced by policy pressures and national adaptations. The authors highlight several significant limitations of conventional HTA:

• Limited Impact: International evidence confirms that HTA research often has limited impact on practical implementation and coverage decisions.
• Unrealistic Paradigm of Stability: The primary reason for this limited impact is HTA’s common reliance on an unrealistic paradigm of stability for both the technology and its environment, often referred to as the ceteris paribus principle. This assumption that relevant domains like practice organization, financing, patient reactions, and juridical/ethical aspects remain constant is frequently not the case.
• Outdated Evaluations: As a consequence, HTA studies that assume stability are likely to produce outdated evaluations by the time results are published. Study periods can easily span 6 to 7 years from design submission to results presentation, making the findings obsolete by the time guidelines can be developed. This delay leads practitioners to develop their own guidelines or adapt technology use based on more recent studies.
• Insufficient for Evolving Technologies: The focus on effectiveness in Cost-Effectiveness Analysis (CEA) is often insufficient to address the scope of questions related to the implementation of evolving technology.

Introduction of Constructive Technology Assessment (CTA) To overcome these shortcomings, the article proposes Constructive Technology Assessment (CTA) as a complementary approach to HTA. CTA was first described in the 1980s, originating outside the healthcare sector. It has evolved from assessing the exact impact of a new technology to a broader approach that includes the analysis of its design, development, and implementation.

• Dynamic Nature of Technology: Unlike traditional HTA, CTA explicitly takes into account the dynamic nature of technology development by emphasizing sociodynamic processes.
• Combining Assessment with Influence: Its core aim is to combine comprehensive assessment with an intentional influence in a favorable direction to guide technological design and implementation. This ultimately leads to a more effective technology by improving its quality and effectiveness in clinical practice from an early stage.
• Shift in Focus: The methodology involves shifting the focus from merely studying a technology’s quality to actively optimizing its quality and effectiveness under dynamic circumstances.

Objectives and Scope of the Study The primary objective of Douma et al.’s study is to explain the methodological aspects of CTA, with a particular focus on its diagnostic use in health care. Diagnostic methods of CTA utilize traditional social science techniques and sociotechnical mapping to identify past and potential future scenarios of technological dynamics. The paper specifically notes that intervention methods (action techniques like awareness initiatives, controlled experimentation, and dialogue workshops) are outside the scope of this particular study.

Methodology of CTA in Health Care

1. Phases of Technology Development and Relevant Research Aspects
   • Early Engagement: To properly study dynamics, CTA must commence before a new technology is introduced into clinical practice. It tracks technology development through a “development chain” from invention to marketing, interacting with the scientific body at all stages.
   • Rogers’ Diffusion Theory: The methodology integrates Rogers’ technology diffusion theory to relate evaluation parameters to different phases characterized by user groups: innovators, early adopters, early majority, late majority, and laggards.
   • Dynamic Assessment: The assessment approach must differ for a new technology used by innovators versus one used by the early majority. For instance, CEAs are typically performed only when sufficient patient numbers can be included, usually in later phases.
   • Focus on Technology and Environment: CTA focuses not only on the technology itself but also on the environment in which it is introduced, recognizing that aspects to be studied can vary across different phases. The continuous change of interaction between technology and environment can lead to necessary adjustments in monitored aspects per phase, with each transition ideally marked by an evaluation.
   • End Point: The CTA measurements conclude when the technology’s quality appears optimized, its use stabilizes, or its implementation is halted due to quality concerns.
   • Quality Definition: The article uses the term “quality” in a broad sense, referring to the ultimate impact of the technology in medical practice, beyond just the Institute of Medicine definition.
2. Aspects Studied in CTA
   • CTA requires a comprehensive approach, where all relevant aspects are considered, although in actual design, only those relevant to the particular technology and environmental interaction are included.
   • The article presents four key parameters, each with various aspects, drawing from Poulsen (12) and the Institute of Medicine (5):
     • Clinical: Efficacy, safety, effectiveness, outcomes, and population effect.
     • Economic: Cost-effectiveness.
     • Patient-related: Social and environmental impact, ethics, acceptability, psychological reactions, patient centeredness, and other patient-related aspects.
     • Organizational: Diffusion, dissemination, organizational implementation, accessibility/equity, skills/routines, education/training, and other organizational aspects.
3. Research Methods
   • The choice of method depends on the technology’s nature (e.g., hard technology like a drug vs. soft technology like a management system), its development and diffusion stage, and the aspects included.
   • Combination of Methods: CTA typically employs a combination of several additional or concurrent methods, which are generally accepted in social sciences or health services research. Examples include:
     • Process analysis
     • Patient satisfaction measurement
     • Impact measurement
     • Various forms of cost-effectiveness analysis
     • Action research: Particularly appropriate in innovator and early adopter phases.
     • Scenario methods: Often used to study dynamic processes, these can monitor implementation, identify needs for intermediate evaluation, or even facilitate intervention through decision-making or other “action research”-related techniques, thereby intentionally influencing the introduction process.
   • Feedback Mechanisms: In early adopter phases, when user numbers are small and technology use unstable, feedback on collected findings (e.g., logistic aspects, effects on guidelines) from researchers, patients, or professionals can enhance more effective implementation.

Case Study: Controlled Introduction of Microarray Analysis in Breast Cancer Treatment To illustrate CTA’s methodology, the article describes a case study involving the controlled introduction of microarray analysis into clinical practice for breast cancer treatment.

• Technology: Microarray analysis (specifically, a 70-gene expression profile) was discovered to predict survival chances in node-negative breast cancer patients more accurately than existing clinical and pathological factors. This technology allows for a more precise assessment of distant recurrence risk within 5–10 years.
• Context for CTA:
  • At the study’s start, the 70-gene profile was newly developed and validated in retrospective series, with firm evidence expected to be confirmed.
  • Traditional prospective clinical trials would likely take at least 10 years for implementation, making a controlled introduction appropriate for this promising innovation in an early stage.
  • The Dutch Health Care Insurance Board initiated a program to stimulate early, controlled introduction of promising innovations.
  • The CTA in this case aimed to ensure and improve the quality of implementation of microarrays in clinical practice during its introduction into different health organizations.
• Operationalized Aspects: Based on sociodynamics theory, the case study specified seven aspects to measure quality and factors involved in design, development, and implementation:
  1. Patient centeredness
  2. User friendliness
  3. Timing
  4. Efficiency/efficacy
  5. Juridical and ethical aspects
  6. Safety
  7. Cost-effectiveness These aspects cover clinical, economic, social, and organizational dimensions, with special attention paid to changes developing over time.
• Operationalization of CTA in Phases:
  • Preparation Phase (Zero Base Measurement): Information is gathered on microarray analysis and organizational settings through literature research, documentation analysis, observations, and semi-structured interviews with professionals. This informs proposed changes, and a process description of breast cancer care (before and after introduction) is made. These findings are then used to develop a guideline for effective implementation.
  • Second Phase (Monitoring and Feedback): Each participating hospital is monitored for actual application of the 70-gene profile, comparing findings with earlier analyses and the guideline. Significant deviations trigger feedback. Data on various aspects are collected via repeated measures of documentation research, observations, tape recordings, and semi-structured interviews with professionals and patients. These results, combined with prospective validation, can inform a cost-effectiveness study, and a theoretical scenario for implementation and diffusion is created.
  • Progression and End Point: Evaluation points are decided at each transition between diffusion phases. As more knowledge is gained, more hospitals are invited to participate. A cost-effectiveness study begins when patient numbers are sufficient. The end point is the optimal implementation or the decision to stop implementation.

Discussion and Conclusion The authors conclude that CTA is an appropriate method for evaluating the introduction of new technology, particularly for promising technologies in early stages with developmental uncertainty or dynamic interaction with their environment. Its use is especially relevant for innovations qualifying for early or conditional coverage decisions.

However, the article also highlights several issues and considerations:

• When to Choose CTA: While CTA addresses dynamics, other methods like Bayesian approaches also deal with dynamic aspects but often require an existing data basis for cost-effectiveness, making them less suitable for innovation/early adopter phases.
• Clinician Commitment: Clinicians often favor classic HTA designs focused on effectiveness, making obtaining their commitment a major issue for CTA. The case study’s early-stage focus and inherent uncertainties helped involve the clinical team.
• Suitable Technologies: It needs further study to determine which technologies are most suitable for CTA. Drug-related research might still lean towards classic designs, while technologies involving various domains or intense environmental interaction are strong candidates.
• Objectivity: The close interaction between researchers and clinical teams, necessary for studying changes in daily routines, can influence both clinical practice and the researcher’s interpretation. Objectivity must be guaranteed through standardized measurements and verification by other researchers, especially in earlier introduction phases lacking traditional designs.
• Phases and Continuation: A change of methodology or focus is logical when a new diffusion phase begins, particularly with the prospect of stabilization or sufficient patient numbers for powerful studies.

Ultimately, CTA combines speed with carefulness, providing an impetus for a more direct influence on policy-making, especially as agencies explore early coverage decisions. It serves as a complementary approach to HTA for guiding the controlled introduction of technologies.

Reference for the article:

Douma, K. F. L., Karsenberg, K., Hummel, M. J. M., Bueno-de-Mesquita, J. M., & van Harten, W. H. (2007). Methodology of constructive technology assessment in health care. International Journal of Technology Assessment in Health Care, 23(2), 162–168. doi:10.1017/S0266462307070262