In the first two expert panels, participants were asked to estimate the number of hours for various types of personnel (which they specified) to install an EHR system in various settings (which they specified) using various architectures (which they specified). In the first expert panel, they also were asked to provide similar estimates for HII in a community using two different architectures. A Delphi procedure was used, whereby the participants submitted their results anonymously and were then able to see all the results for the group and modify their initial estimates.
The estimates had a huge range – typically an order of magnitude, but sometimes two or even more orders of magnitude. Also, the substantial changes in individual estimates resulting from “reasonableness” discussions indicated that most panel participants were really “guessing” rather than sharing meaningful data. This was further reinforced by the extremely wide ranges in the data. From all this, it was concluded that the work hours estimates could not be reliably extracted “on the fly” from experts during group meetings.
The discussions about activities needed to complete the NHIN, architectures for the activities, and the personnel types needed were very helpful. While very little time was spent soliciting views on the calendar time needed for each type of implementation project, the information obtained on this topic was also helpful.
The third and fourth expert panels provided additional focused feedback and guidance about the study, omitting any further data collection. The participants confirmed the overall strategy and direction of the research. In addition, the experts agreed with the planned “off-line” data collection and provided suggestions of sources to contact for this purpose.