In the rapidly evolving landscape of healthcare, the promise of Health Information Technology (IT) to deliver efficient, quality care is often hampered by a critical, yet frequently overlooked, factor: usability. While health IT offers immense potential, a lack of attention to its evaluation can lead to frustrating user experiences, decreased efficiency, increased costs, workflow disruptions, and even increases in healthcare errors.
Addressing this pressing need, the seminal paper, “Review of health information technology usability study methodologies” by Po-Yin Yen and Suzanne Bakken, provides an invaluable resource for anyone involved in the design, development, and implementation of health IT systems. Published in the Journal of the American Medical Informatics Association, this work offers a timely and in-depth review of existing usability study methods, alongside practical guidance for their effective application.
Key Contributions and Insights:
- Integrated Usability Specification and Evaluation Framework: The paper introduces a novel framework that bridges the “when” of evaluation (System Development Life Cycle – SDLC stages) with the “what” to evaluate (user, tool, task, and environment components). This framework categorizes studies into five distinct stages, from initial task understanding to routine use in a real-world setting, providing a holistic view of the evaluation process.
- A Stratified View of Usability Evaluation: To overcome the complexity of identifying the root cause of usability problems, the authors propose a three-level stratified view of health IT usability evaluation. This innovative approach helps differentiate between system-related issues and confounding factors such as task/expectation complexity, user variances, and organizational support.
- Level 1 focuses on system specification and understanding user-task interaction.
- Level 2 examines task performance to assess system validation and human-computer interaction (HCI).
- Level 3 integrates environmental factors to identify work processes and system impact in real-world settings.
- Identification of Methodological Gaps: The review meticulously highlights several recurring issues in existing health IT usability studies, including:
- A prevalent lack of explicit theoretical frameworks or models.
- Insufficient detail in qualitative study approaches, hindering rigor and reproducibility.
- A tendency towards a single evaluation focus, failing to capture the comprehensive picture of usability.
- The delayed evaluation of environmental factors in early development stages.
- Over-reliance on guideline adherence as a primary outcome for clinical decision support systems, which may not accurately reflect true usability.
- Practical Guidance for Researchers and Developers: The paper offers tangible guidance for selecting appropriate theoretical frameworks, outcome measures, and evaluation methods based on the specific type of interaction being studied and the SDLC stage. This empowers researchers and system implementers to design usability studies that are precisely aligned with their system and evaluation goals.
By providing a structured approach to understanding, categorizing, and conducting usability evaluations, Yen and Bakken’s work is indispensable. It serves as a critical tool for researchers and professionals seeking to enhance the effectiveness, efficiency, and satisfaction derived from health IT, ultimately contributing to its successful adoption and the improvement of healthcare delivery.
Reference:
Yen, P.-Y., & Bakken, S. (2011). Review of health information technology usability study methodologies. Journal of the American Medical Informatics Association, 19(3), 413–422.