(This protocol was used as a general guide in conducting site visits to ensure that discussions covered a broad range of issues.)
On what area of evaluation do you think evaluators of telemedicine should be focusing? (Please rank in order of most to least important.)
Quality of care and health outcomes
- What types of health outcomes measures are appropriate for telemedicine evaluation?
- What questions can evaluators ask to determine the effects of telemedicine on health (clinical) outcomes compared to the alternative(s)?
Access to care
- What types of access measures are appropriate for telemedicine evaluation?
- How can evaluators determine the effect of telemedicine on the use of services (utilization) or the level or appropriateness of care compared to the alternative(s)?
- How can evaluators determine if and how telemedicine affects the timeliness of care or the burden of obtaining care compared to the alternative(s)?
- What questions can evaluators ask to determine the barriers to utilization of telemedicine?
Costs and Cost-effectiveness
- What questions can evaluators ask to determine the costs of a telemedicine application for participating health care providers or health plans compared to the alternative(s)?
- What questions can evaluators ask to determine the costs of telemedicine for patients and families compared to the alternative(s)?
- What questions can evaluators ask to determine the costs for society overall compared to the alternative(s)?
- What questions can evaluators ask to determine how the costs of telemedicine relate to its benefits, compared to the alternative(s)?
- What types of cost evaluation measures are appropriate for telemedicine evaluation (e.g., cost benefit, cost-effectiveness, and cost minimization)?
- How can evaluators determine if patients are satisfied with the telemedicine service compared to the alternative(s)?
- How can evaluators determine if attending and consulting clinicians are satisfied with the telemedicine application compared to the alternative(s)?
- Below is a list of questions that we have collected from the IOM framework. Which of these questions is more important? Less important? Irrelevant?
- What is the perspective of the evaluation? Society? Network? Site? Physician?
- What is the (1) setting of care (e.g., hospital or physician's office); (2) condition being treated (e.g., dermatology or psychiatry); (3) technology used (e.g., store-and-forward)?
- What is the alternative to which telemedicine is being compared? That is, what would happen in the absence of telemedicine, or similarly, what is the control group in the evaluation?
- How is success measured?
- How is the telemedicine implemented? That is, is it well-integrated into patient care? Does it effectively meet population needs?
- How is the program dealing with issues such as confidentiality, privacy, equipment and protocols, if at all?
- At what point in a program's development (i.e., its life cycle) should an evaluation begin?
- What types of research design/evaluation methods do you feel are most appropriate for telemedicine?
- large randomized controlled trial (RCT);
- small RCT;
- nonrandomized trial with contemporaneous controls;
- nonrandomized trial with historical controls;
- cohort study;
- case-control study;
- cross-sectional study;
- surveillance (e.g., using registers or surveys);
- series of consecutive cases;
- single case report.
- Based on your experience/knowledge of telemedicine evaluation, do you believe that sufficient scientific rigor has been applied to current and past evaluation of telemedicine?
- Please comment on how telemedicine evaluation is best conducted with respect to the following methodological factors.
- perspective of analysis (e.g., society, payer, provider patient)
- accounting of direct costs (medical and non-medical)
- accounting of indirect costs (e.g., loss of productivity, patient time)
- use of charges or prices versus actual costs
- choice of time horizon for analysis (i.e., short-term versus long-term)
- use of sensitivity analysis
- To what degree do you feel reimbursement drives and/or directs use of telemedicine services, and subsequent evaluations of such programs?
- What do you see as the emerging issues (in terms of policy and evaluation) within the field of telemedicine?