In reviewing the options presented in this section, it is necessary to weigh the importance of the questions to be asked, the feasibility of the approach, the scientific rigor of the evaluation design, and the cost of implementing the design. Ultimately, the decision about whether to implement an evaluation and which one is a question of value for money -- does the value provided by the evaluation merit the expenditure of the funds?
In terms of feasibility, scientific rigor, and cost, the RTI International/Urban Institute team believes that some of the options presented above seem especially weak at this time. First, because case studies have recently been conducted of five of the largest LTC RAPs as part of this project, there is little to be gained in conducting additional case studies at this time. Most of the remaining programs are relatively small. If the evaluation is not conducted until several years from now, then case studies may be worthwhile. Second, while randomized controlled trials are the gold standard of research, the LTC RAP is already an ongoing intervention. A randomized controlled trial would be difficult to implement on a scale large enough to yield statistically significant results and would be expensive. While an attractive option in many ways, it does not seem to meet the mandate of evaluating an ongoing program. It could only be considered seriously in the context of a major demonstration program and the likely need for significant government or foundation funding. Third, pre-post designs without comparison groups seem particularly problematic for job training programs where merely continuing to work at the job would provide individuals with increased experience and expertise in caregiving even without any formal additional training. Moreover, this approach cannot control for events external to the training, such as inflation, recessions, and changes in management, which may affect the performance of direct care workers. Fourth, although a strong research design, new data collection that involves following a cohort of apprentices and a comparison group from the beginning of their training through the end of the apprenticeship and perhaps some time later would be expensive because of the long time it would take to gather data on a sufficient number of apprentices and comparison workers and the long training period. Data collection on a flow basis is expensive.