This prompt introduces the article titled “A systematic review of the methodologies used to evaluate telemedicine service initiatives in hospital facilities”. This systematic review was authored by Sharifah AlDossary, Melinda G. Martin-Khan, Natalie K. Bradford, and Anthony C. Smith. It was accepted for publication on October 12, 2016, and appeared in the International Journal of Medical Informatics.
Background and Purpose of the Study: The authors highlight that the adoption of telemedicine into mainstream health services has been slower than expected, with many projects failing to progress beyond the trial phase. This issue has been noted since 1999 and continues to be a concern in contemporary literature. Despite challenges like funding, technology, and planning, some telemedicine services have been successful and integrated into routine healthcare delivery. The authors argue that reporting and evaluating these successful services is crucial for improving future uptake and sustainability. The primary aim of this systematic literature review was to identify peer-reviewed publications of deployed telemedicine services in hospital facilities and to report and appraise the methodologies used to evaluate these services. Additionally, the review aimed to reflect on the planning phase of telemedicine service implementation, particularly based on need assessment.
Methodology: The researchers conducted computerized literature searches of bibliographic databases, including PubMed, CINAHL, Medline, and Health Source: Nursing/Academic Edition, using MeSH terms like “Telemedicine” and “Hospital Services” or “Hospital”. Papers published up to May 2016, written in English, and with an available abstract were included. The inclusion criteria focused on established healthcare providers with recurrent funding (minimum 12 months duration) utilizing telemedicine for care delivery, excluding technical, educational, or reliability studies, as well as pilot projects not converted to ongoing services. Data extraction included author, year, country, study methodology, telemedicine modality, specialty, outcome measures, and telemedicine sites. Study quality was assessed independently by three reviewers using a modified tool, ranking papers on a four-point scale (A-D) based on service reporting and study evaluation methodology.
Key Findings: The review identified 164 papers describing 137 distinct telemedicine services.
- Geographic Distribution and Modality: The majority of services were based in the United States (44.5%), followed by Australia and the UK. Almost two-thirds of the services (62.7%) were delivered using real-time telemedicine methods, such as videoconferencing, while 25.5% used store-and-forward methods.
- Specialties: Most services focused on a single clinical specialty, with Neurology (13.1%) and Emergency/Trauma medicine (12.4%) being the top reported areas.
- Evaluation Perspectives:
- Comprehensive Evaluation: Almost half of the reviewed studies (49.3%) assessed their services from three different evaluation perspectives: clinical outcomes, economics, and satisfaction.
- Descriptive Studies: The remaining half (50.6%) described their service activities without reporting any evaluation measures. These studies focused on aspects like the number of consultations or system utilization.
- Clinical Outcome Evaluation: 27 studies evaluated clinical outcomes, predominantly using measures like length of stay (LOS), morbidity, and mortality. Neurology was the most common specialty for this type of evaluation. However, none of the reviewed studies conducting clinical outcome assessments were randomized controlled trials (RCTs); only two were non-randomized open intervention studies, with others being retrospective or prospective observational designs.
- Economic Evaluation: 29 studies assessed economic perspectives. The most common methods were simple cost analysis (14 studies) and cost minimization analysis (15 studies). No studies reported using more rigorous economic evaluations such as cost-effectiveness analysis, cost-utility analysis, or cost-benefit analysis.
- Satisfaction Evaluation: 35 studies examined satisfaction among healthcare providers and/or consumers. Measures included comfort, perceived privacy, ease of use, and technical functionality. Psychiatry, Emergency/Trauma medicine, and Pharmacy had the most reported studies assessing satisfaction. A significant limitation was the poor reporting on the validity of questionnaires used, with 78.7% of studies providing no details.
- Planning Strategies: The review found limited information on structured planning strategies. Only 30 studies (18.2%) indicated a two-step implementation and evaluation process. Some implementations were based on the success of pilot studies, replacing existing outreach clinics, or responding to identified clinical needs like long waiting lists.
Conclusions and Recommendations: The authors conclude that the low number of identified telemedicine services (137) suggests either that telemedicine is not yet part of mainstream clinical services or that its implementation is not adequately reported in peer-reviewed literature. They emphasize that the depth and quality of reported information varied significantly, reducing generalizability.
- The review strongly recommends encouraging the reporting of service implementation and planning strategies to enable others to learn and implement sustainable services.
- It highlights that planning, particularly based on needs assessment, was a key component that was underreported.
- The authors suggest that more rigorous methodology is needed for evaluating telemedicine services, and that applying a combination of evaluation methods (clinical outcome, economics, and satisfaction) would be more effective to overcome individual method limitations and provide a better understanding of the service’s impact on patient care.
- Finally, the creation of a framework for health data utilization that integrates planning, implementation, and evaluation in a standardized way is recommended to facilitate the sharing of lessons learn
Reference: AlDossary, S., Martin-Khan, M. G., Bradford, N. K., & Smith, A. C. (2016). A systematic review of the methodologies used to evaluate telemedicine service initiatives in hospital facilities. International Journal of Medical Informatics. Retrieved from http://dx.doi.org/10.1016/j.ijmedinf.2016.10.012
