Evaluation Design Options for the Long-Term Care Registered Apprenticeship Program. 6.4. Cost-Benefit Analysis


Employers in long-term care are highly constrained in what they pay their workers, face government training requirements even if they do not offer apprenticeships, and often pay low-wage rates even to highly qualified workers. As in any service industry, productivity in long-term care is hard to measure. Indeed, in some cases, increased “efficiency” is an indicator of poorer quality (e.g., reduced feeding time for severely disabled nursing home residents). For any voluntary program, like the LTC RAP, that provides more extensive training than that required by federal and state law and regulation, long-term care providers must believe that the additional training is worthwhile for the organization. In other words, the benefits must exceed the costs. This component of the evaluation will quantify those factors for employers participating in the LTC RAP.

The primary resource costs are the salaries of instructors, mentors, office staff, and others for the periods they deliver the training and the dollar costs of materials, rent, and other material inputs. Another cost is the lost productivity of participants while they are taking part in the training, a cost which is minimized in apprenticeship programs. The primary benefits to the employers is reduced turnover and improved productivity/quality of care provided by the trainees, usually measured in evaluations of job training programs as the present value of the increases in earnings over time. Improved quality may reduce accidents and increase the number of consumers who use the service.

Research Questions

Given these considerations, the key questions in the cost-benefit analysis are:

  • What is the impact of LTC RAPs on employer costs?

  • What is the value of benefits of long-term care apprenticeships to employers?

  • What is the ratio of benefits to costs for employers?

  • What costs do apprentices incur by participating in a long-term apprenticeship?

  • What is the impact of participating in an apprenticeship on subsequent employment and earnings?

  • What is the ratio of benefits to costs to workers of participating in LTC RAPs?

Overview of Design

This evaluation option will conduct a cost-benefit analysis from the perspective of LTC RAP employer sponsors and from the perspective of apprentices. Since little direct public money is spent on LTC RAP, the analysis will not be from the societal government perspective. Data on the employers will be collected through a web, mail and telephone follow-up survey and structured discussions with the management of LTC RAP employer sponsors. The option would not include a comparison group. Data generated from the LEHD data analysis will be used to conduct the cost-benefit analysis for apprentices.

1. Employers

Through in-depth interviews and surveys, the evaluator can work with employers to determine cost elements, cost savings, and other benefits to LTC RAP sponsors. The survey questions would be explained carefully to the employer and the evaluator would provide technical support to clarify any questions the employer might have. The evaluation option will collect data on the costs and benefits of the LTC RAP from 50 employer/sponsors. Forty of the employers/sponsors will complete a web survey. In addition, ten of the largest LTC RAP employer sponsors would receive a more detailed telephone survey to gain a deeper understanding of their costs and benefits. The LEHD administrative record analysis will provide estimates of the extent to which LTC RAP apprentices leave their employers at rates similar to or different from other workers with similar wages in the firm. This information will strengthen estimates of the potential benefits employers derive from reduced turnover.

LTC RAP employers will be asked to identify the time costs of trainers, the costs of related instruction, thewage costs of the apprentices while in class training, and the cost of replacing apprentices while they are in class training. They will be asked to specify whether the apprentices are paid for the time they invest in related instruction. In some cases, other organizations, government agencies, and foundations reimburse some employer costs associated with apprenticeships. Nonetheless, these costs are true costs regardless of who pays for the use of resources and should be included.

On the benefit side, employers will be asked to report turnover for various classes of workers. LTC RAP employers will be asked to supply data on turnover for those who took apprenticeships and those who did not, and turnover rates before and after the adoption of apprenticeship training, if available. If apprenticeships do lead to fewer quits or discharges, the monetary benefit of reductions in turnover will be calculated. To do so, employers will be asked to quantify recruitment and initial training costs for regular workers not in an apprenticeship program. In addition, the apprenticeship program might lower the likelihood of errors or accidents. Employers of long-term care workers would be asked to make informed guesses based on their experience about the potential cost savings from reductions in errors or accidents. These estimates will have a high-level of measurement error and will be used only to develop gross order of magnitude estimates.

2. Apprentices

In order to conduct the cost-benefit analysis from the perspective of the apprentice, data from the LEHD will be used. We propose that the evaluator follow the procedure of Hollenbeck (2011) in estimating foregone earnings and post-apprenticeship net earnings. In the case of long-term care, wages have little variation, resulting in modest benefits from the standpoint of wage gains. However, the advantages of apprenticeship for the workers may be significant in terms of retaining employment and the stability of employment within the firm and within the industry and in being assigned to work more hours as a more highly valued staff person. Thus, earnings of workers may rise as a result of apprenticeship programs even if hourly wage rates do not increase very much. The empirical evidence drawn from the LEHD-based impact study will yield estimates of earnings differentials between apprentices and matched comparison groups in each calendar quarter from the point of entry into the apprenticeship program through the latest post-program period.

The analysis will analyze earnings during the apprenticeship period to determine whether apprentices have foregone earnings by participating in the program. Based on the site visits, apprentices will not incur any foregone earnings within the long-term care firm for which they work, but they might forgo earnings that might have been achieved by leaving for a new job at another provider or leaving the long-term care field altogether. In some cases, apprentices incur out-of-pocket costs and they do forgo some leisure time to participate in classes and to study. Next, the analysis will calculate estimates of earnings gains achieved after completing the program. Projecting gains far beyond the period for which data are available involves uncertainties, but evaluators can provide sensitivity analyses to determine how the size of net benefits for apprentices varies with what one assumes about whether any observed gains in earnings erode over time after the last follow-up period. Once each quarter’s earnings impact is determined, the evaluators will calculate the present value of earnings gains.

Domains on Which Information Will Be Gathered and Questionnaire Development

The employer survey will gather information on the costs and benefits to employers. The primary costs to employers are:

  • Salary costs per hour of the mentor/teacher times the number of lost production hours
  • Wages of the apprentice lost to production
  • Employer spending on classroom instruction
  • Management salary and other costs for administering the program
  • Miscellaneous administrative costs (e.g., reporting to DOL, record keeping, award certificates, etc.)

Weighed against these costs are the benefits to the employer. The primary benefits are:

  • The value of production generated by apprentices during the apprenticeship period.

  • The ability for employers to obtain post-program benefits because the productivity of workers completing their apprenticeships exceeds their wage. This result can occur because of imperfect information (the training firm knows the worker better than other firms) or other factors (Wolter and Ryan, 2011).

  • Savings in recruitment and other training costs associated with lower turnover.

  • Increases in the quality of goods and/or services resulting from a more highly trained workforce; may include quality improvements that reduce risks of high cost mistakes.

  • Savings in worker compensation and insurance premiums.

  • Improved reputation for quality as high-level training will be perceived by consumers as a proxy for higher quality services.

Evaluations of some job training programs and particularly apprenticeship programs ask employers to examine the effect of the training on productivity. Exhibit 9 provides an example of a typical question for assessing productivity gains. This question and other questions form part of the existing self-assessment tools used in other countries by which employers can examine the benefits and costs of their apprenticeship program.

EXHIBIT 9. Example of Question Used to Assess Productivity Gains

How does the productivity of apprentices compare to the productivity of a worker who has achieved mastery in the occupation?

Please take time to answer this question and consider the average values of the past years.

Handling productive and challenging issues on the job is considered to lead to a better development of professional competence. At the beginning of their apprenticeship, trainees can accomplish only some tasks normally undertaken by master employees. Please indicate the productivity of apprentices in percentage terms compared to the productivity of skilled workers.

Example: 50% is equivalent to “apprentice is half as productive as a master employee”

100% is equivalent to “apprentice is as productive as a master employee in this occupation”

    At the end of each semi-annual interval  
Apprentice 1 2 3 4
In percent of a master employee ___% ___% ___% ___%
SOURCE: Adapted from the Quality Returns and Costs (Form 6 Cost-Benefit Analysis) an on-line tool for company self-assessment developed by Ursel Hauschildt and Felix Rauner at the TVET Research Group, Institute for Technology and Education, University of Bremen.

Time Frame to Collect and Analyze Data

The main tasks are: (1) developing the survey instrument, creating an on-line version and pretesting the survey; (2) obtaining OMB clearance; (3) conducting data collecting and providing technical assistance to employers as they complete the survey with 40 employers/sponsors; (4) 1 hour in-depth telephone interviews with ten of the employers/sponsors; (5) analysis of the data; and, (6) write up of the cost-benefit analysis. It is estimated that, including 6 months for OMB clearance, that the cost-benefit analysis will take 14 months.

Cost Estimate

The estimated cost for conducting the cost-benefit analysis is $100,000.

Main Statistical Methods for Analyzing Data

The standard method for estimating the value of production is to ask employers about the productivity of apprentices relative to the productivity of workers who have fully mastered the occupation. On the assumption that master workers contribute to production what they earn as wages, the production benefits equal:

VPa = Wq * (Pa/Pq) * Ha,

where VPa is the value of the added production generated by the apprentice, Wq is the wage rate of master workers, Pa/Pq is the productivity of the apprentice relative to the productivity of master workers, and Ha is the hours the apprentice devotes to production.

Additional Analyses

Based on the value of production and other benefits (e.g., reduced hiring and training costs) relative to costs, it is possible to calculate the present value of benefits and costs during the apprenticeship period and during the apprenticeship and 3-4 years of post-apprenticeship for each employer. The report will then display the distribution of net benefits (benefits - costs) and the cost-benefit ratios across employers. For example, it will show what share of firms reaped net benefits of say, $4,000 and over, $3,000-$3,999, $2,000-$2,999, … negative 1,000-negative 1,999, less than negative $2,000, etc.

The cost-benefit analysis can examine the relationship between the characteristics of employers and their programs and the net benefits they accrue from a LTC RAP. Other studies have documented high-levels of employer satisfaction from their participation in registered apprenticeship programs (Lerman, Eyster, and Chambers, 2009 ). To the extent that the results indicate high net benefits in this apprenticeship program, the findings can be used to help market the program to the long-term care industry as a whole.

View full report


"LTCRAPedo.pdf" (pdf, 870.86Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®