Evaluation Design Options for the Long-Term Care Registered Apprenticeship Program. 5. Assessment of a Broad Range of Evaluation Options


There are many possible research designs for an evaluation of the LTC RAP, with varying costs and degrees of scientific rigor. Most evaluations of job training programs focus solely on the program participants, mainly their level of participation, knowledge and skill improvements and gains in earnings. However, since many of the policy motivations for the LTC RAP have to do with improving the performance of and quality of care of long-term care providers, this section explores both possible studies of apprentices and their employers. The overarching approach for most designs is to compare the apprentice and employer performance to what it would have been in the absence of the LTC RAP. There are many possible ways to conduct that comparison. Exhibit 1 provides a broad overview of the range of methods, types of data collection, and their relationship to costs and ability to generalize the findings to the total population of LTC RAPs. In some cases, the studies assess associations between the independent and dependent variables, while other studies make a stronger claim that the LTC RAP caused changes in the dependent variables.

EXHIBIT 1. Overview of Methods, Data Collection, and Potential Feasibility
Analysis Methods Qualitative Analysis Quantitative Analysis
Descriptive analysis (single point in time AND no comparison group) Multivariate analysis (two points in time OR with a comparison group OR both)
Data Collection Focus groups of workers

Case studies of employers

In-depth ethnographic studies and implementation evaluation

Survey of LTC RAP workers or employers across occupation/ organization types

LTC RAP administrative data

Linking Medicare/ Medicaid claims/ OSCAR data

Survey only

Survey and existing secondary data (National Nursing Assistant Survey; National Home and Hospice Care Survey)

Feasibility Lower cost
Lower generalizability
<––––––––––> Higher cost
Higher generalizability

View full report


"LTCRAPedo.pdf" (pdf, 870.86Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®