Characteristics of Long-Term Care Registered Apprenticeship Programs: Implications for Evaluation Design

Executive Summary

Daniel Kuehn, M.P.P., Robert Lerman, Ph.D., and Lauren Eyster, M.P.P.
Urban Institute

Wayne Anderson, Ph.D., Galina Khatutsky, M.S., and Joshua Wiener, Ph.D.
RTI International Institute

September 13, 2011

Improving the skills of the workforce is often proposed as a strategy for improving the quality of care in long-term care settings (Institute of Medicine, 2008). The long-term care system currently lacks a formal training and career development system that is applicable across different states and long-term care settings. A federal standard requires a minimum of 75 hours of training for certified nursing assistants (CNAs) in skilled nursing facilities and home health aides (HHAs) providing home health care services but the content of this training greatly varies. There are no federal (or sometimes even state) training requirements for personal assistant service workers, personal care attendants, aides in assisted living facilities, or direct care workers serving people with developmental disabilities (Institute of Medicine, 2008).

One strategy for improving the skills of workers in the long-term care system is to expand registered apprenticeship. It combines classroom and experiential learning, wage growth, and job ladders. The goals of the apprenticeship strategy are to raise skill levels, productivity, and organizational efficiency and thereby permit employers to improve the quality of care and pay higher wages. The Long-Term Care Registered Apprenticeship Programs (LTC RAPs) are registered by the U.S. Department of Labor’s (DOL) Office of Apprenticeship (OA) and developed by employers, employer associations, and labor-management organizations. They provide formal training and work experience for direct care workers in long-term care settings. Although few employers currently operate LTC RAPs, the Federal Government considers this training approach as a potential way of improving job quality and quality of care in the industry as a whole.

As an initial step in testing whether the registered apprenticeship model is an effective approach for training long-term care workers, the U.S. Department of Health and Human Services (HHS) and DOL have funded a study to examine design options for evaluating LTC RAPs. This report, developed by the Urban Institute and RTI International, examines the characteristics of selected LTC RAPs as part of the effort to determine the feasibility of evaluating the programs. The site descriptions do not constitute full case studies, but they capture key program features relevant to structuring a rigorous evaluation. A final report in this study, to follow this report, will assess and present evaluation design options for the LTC RAPs.

The sites selected for this study are among the largest programs in number of apprentices and represent a range of occupations, apprenticeship models (time-based or competency-based), and geographic regions. Apprenticeship programs are on average small with 3-5 active apprentices during a year. But, this study selected large LTC RAPs because of the advantages of scale in developing rigorous evaluation designs. The LTC RAP sponsors selected are Developmental Services, Inc. (Indiana), the Evangelical Lutheran Good Samaritan Society (Idaho), Home Care Associates (Pennsylvania), Air Force Villages, Inc. (Texas), and Agape Senior (South Carolina). Employer sponsors offered a LTC RAP in one of the four occupations -- CNA, developmental support specialist (DSS), health support specialist (HSS), and HHA. Additional details on these programs are available in the site summaries provided in Appendix B.

The study team conducted semi-structured interviews with key sponsor staff involved in the LTC RAPs, apprentices, and partners to obtain information on the goals and structure of the LTC RAPs, the settings in which they operate, recruitment and selection of apprentices, on-the-job training (OJT) and related technical instruction components, wage progression, program data, perspectives on benefits and challenges, and sustainability and replicability of the programs. The following section provides a summary of the findings from the five sites visited.

Implementation of Long-Term Care Registered Apprenticeship Programs in Five Sites

The five sites designed and implement their LTC RAPs in different settings. Each LTC RAP sponsor usually implemented their LTC RAP for only one of the four occupations in this study. Four of the five sponsors are paid principally through Medicaid reimbursements, which all sponsors considered as being too low to support their LTC RAP costs. As a result, each sought supplementary external funding.

The LTC RAP intervention is generally consistent across the sites visited. Most sites (Good Samaritan, Home Care Associates, Air Force Villages, and Agape) used the LTC RAP for advanced training and mentoring of employees who had already received basic training and had leadership or personal qualities for which they were selected into the apprenticeship. Alternatively, one site (Developmental Services) used its LTC RAP for entry-level training of all new employees. The sites visited were among the largest LTC RAPs. Still, the size of even these sites varied widely, ranging from 26 to 1,150 in total number of apprentices registered between January 2005 and May 2011.

Four sites (Good Samaritan, Home Care Associates, Air Force Villages, and Agape) recruit their apprentices from current employees who must apply or be recommended and be selected for an apprenticeship. The fifth site, Developmental Services, places all new hires into its LTC RAP as apprentices are recruited through employment advertising and employment screening processes. Recruitment challenges, particularly associated with finding motivated and qualified apprentices, were problematic for Developmental Services, the one site that required all direct care workers to participate in the apprenticeship program.

The goals of the LTC RAPs, which are roughly consistent across sites, are to improve the long-term care workforce in order to improve quality of care delivered and to create jobs that are more attractive for apprentices who perform caregiving. Sites also noted that reduction in medical errors, receipt of state certification for workers, and improvement of the self-sufficiency of workers were additional goals of the LTC RAP.

One site (Air Force Villages) uses the LTC RAP as part of a formal occupational career ladder, while four additional sites use the program as a way of introducing a career ladder or seniority system within the workforce. All sites use the LTC RAP to improve the quality of care and increase the stature of the job in the eyes of the apprentice. Quality of care could be inferred from data kept by most sponsors on clinical proficiency checks or client satisfaction, or from survey inspection results in facilities, but none of the LTC RAP sponsors currently analyze these data to determine the effectiveness of their programs. Instead, they rely on anecdotal evidence which they believe strongly indicates improvements in worker skills and quality of care. Apprentices in these sites are usually middle-age females with high school degrees or general equivalency degrees (GEDs), which differs somewhat from the national profile of LTC RAP apprentices in each of the occupations of training. Rates of employees actively participating in or having completed an apprenticeship varied greatly across sites (43%-96%).

The programs varied considerably in the total length of the apprenticeships offered. They ranged from 1,680 hours of training with Good Samaritan to 3,232 hours with Air Force Villages. The difference is largely in the length of OJT.

The LTC RAP covers many of the same core competencies provided in basic training to all long-term care workers. However, the LTC RAPs go beyond the basics to provide a more extended period of “advanced” and “specialty” training to reinforce learning and improve techniques covered in basic training, as well as to teach more advanced material. The LTC RAPs also widen the scope of training by including some soft skills components such as person-centered care delivery, communication skills, and receipt of mentoring.

Sponsor budgets for patient services and staff development programs such as LTC RAPs are greatly influenced by public payer long-term care reimbursement policies. In the United States, long-term care services are paid out-of-pocket, or seldom, through privately held long-term care insurance policies. Facilities often charge higher rates for private paying patients, who are only a minority of patients receiving care (approximately 22% of patients in nursing homes (AHCA, 2011)). Medicare covers only short-term acute care in nursing homes facilities if the patient had a preceding recent hospital stay and for home health services. Once a person completely exhausts their private assets for long-term care services, state Medicaid programs become the payer of last resort. In fact, Medicaid pays for the services for almost two-thirds (63% (AHCA, 2011)) of all nursing home patients. Because of Medicaid’s dominant financing role, state Medicaid programs are the key rate setter. Since state governments have competing demands for their resources, Medicaid rates are usually lower than any other payer source.

Under these circumstances, wage progression for apprentices is greatly constrained by public payer reimbursement rates that are widely perceived by the industry to be too low. Wage increases in the sites visited total approximately $1.25 per hour for apprentices by the time they complete the program, although one large sponsor with multiple locations noted that a wage increase does not occur at all of its locations. Wages start off low, around $7-$9 an hour. Apart from wage increases, all sites noted that the pride and distinction of apprentices relative to their co-workers was a major incentive for participating in the program.

The LTC RAPs visited collect a limited amount of outcome data, often because of resource and time constraints, and do not formally monitor the performance of the program. Most of the sites interviewed have data on wages and benefits paid, tenure, and turnover, but these data are not collected in the same form across sites or even regularly.

The primary benefit of the LTC RAPs noted by all sponsors is that they produce a better skilled workforce. Even those sponsors who do not explicitly track apprentice performance or client satisfaction described what they perceive to be an impact on work quality. Many of the apprentices interviewed said that they recognize the improvement in the skills they learned during the LTC RAPs. These workers said that they appreciate having the qualifications to serve their clients effectively. In fact, their perception of performing at a higher quality than non-apprentices was generally viewed as a more important benefit of the program than the wage increases earned in the apprenticeship. Notwithstanding, the emphasis by sponsors and workers on job quality, sponsors were not particularly concerned with tracking or monitoring the outcomes of the LTC RAPs relative to other workers.

Two key issues for the LTC RAP model are the sustainability of the programs and the potential for replication by other sponsors. Regarding sustainability, some site managers highlighted the costs of their LTC RAPs -- namely the cost of wage increases, and the financial and time resources required to prepare training resources and implement the apprenticeship -- as the major barrier to operating the program. These high costs almost always were reported in interviews without detailed documentation. One site, which reported costs of $8,000-$10,000 per apprentice for only the related technical instruction component, has ended its LTC RAP ostensibly because of unsustainable costs. Another site suggested outside funding similar to South Carolina’s $1,000 apprenticeship tax credit would help defray costs.

Lack of qualified candidates among staff to participate in apprenticeship was a common theme. Developmental Services noted the availability of qualified recruits as a challenge, although this registered apprenticeship program was unique in training all of its several hundred employees using the registered apprenticeship program. Still, other sites selected only better qualified staff for apprenticeships.

Some sites also noted that “buy-in” from leadership and a “champion” for implementing the LTC RAP were important to sustainability. Some sites are still testing whether apprenticeship is the right training approach on a subset of their staff. Some sponsor leadership has not supported full implementation. For example, Good Samaritan’s program is voluntary for its facilities to offer and for employees to participate.

One concern of apprentices -- the lack of a recognized and portable credential -- may hinder the sustainability of LTC RAPs. For example, staff and apprentices at Home Care Associates noted that while the apprentices are proud of earning their apprenticeship credential, they also find it challenging to communicate the meaning and the value of the credential with other individuals in their field. However, the use of a community or technical college as a training provider may lend credibility to the apprenticeship credential.

The sites visited rarely reported partnerships with the workforce investment system, the educational system, or the long-term care industry, all of which often help sustain training programs. For example, staff seldom identified partnerships with One Stop Career Centers, community colleges, or long-term care industry groups. The lack of such partnerships may greatly hinder sustainability. The initiative for developing such partnerships, if they are to occur, may need to come from the partnering organizations as opposed to from sponsors.

Some sites indicated that their LTC RAPs could easily be replicated, suggesting that the most significant obstacle to replication was that other long-term care providers were not aware of the apprenticeship model as a training option. The staff at Good Samaritan and Air Force Villages suggested that a successful program may be difficult to replicate without a champion. Agape administrators and apprentices credited their “local champion” with the success of the registered apprenticeship program and suggested that it would not have been implemented without the champion’s efforts.

Replicability also seems to depend on a variety of local conditions, including the degree to which sites are dependent on Medicaid and Medicare reimbursement for revenue. Sites that were able to identify other funding sources besides Medicaid and Medicare did not cite as many financial constraints to their work. Therefore, while Medicaid and Medicare reimbursement provided a budget constraint on many of the LTC RAPs, it did not appear to be a deciding factor in the success of the programs.

Implications for Policy

The goals of the LTC RAP are to increase the skills and productivity of the workers, raise quality of care, prolong job tenure and reduce turnover, and improve job satisfaction and employee wages. The sponsors interviewed for this study saw these advantages as reasons to operate their LTC RAP. However, other findings in this report present a challenging picture for apprenticeship, or any other advanced training for that matter, in the long-term care industry. A central problem in this field is the inability of employers to achieve and document sufficient cost savings or extract sufficient revenue increases from gains in productivity and quality of care that might result from a more highly skilled workforce. Another problem is a third-party reimbursement system that does not provide higher payments to higher quality providers, which limits the ability of employers to pay wage increases for progressing through and completing an apprenticeship. Other challenges, such as limited knowledge of the approach and limited recognition of credentials, reflect the fact that LTC RAPs are still in their early stages in the industry.

The registered apprenticeship model is intended to be self-sustaining because the sponsor uses its training budget for the LTC RAP in place of the training approach they would use otherwise. Usually, new long-term care workers receive only minimal basic training before they start work. In four of the five sites visited, the LTC RAP provides training that lasts much longer than is typical, far exceeding the usual minimal requirements. As such, apprenticeship does indeed generate more training, but presumably at additional cost to employer sponsors. Therefore, in order for the program to be sustained, the benefits of the LTC RAP need to outweigh the costs.

While these challenges do exist, expanding the number of these programs in the long-term care setting may offer an innovative approach to addressing the workforce shortage. While the number of LTC RAPs is small compared to the universe of registered apprenticeship programs, some ground has been gained through efforts by OA to promote LTC RAPs in the industry and new initiatives such as the Health Professions Opportunity Grants program, which provides health care occupational training to welfare recipients and other low-income individuals, requires partnerships with registered apprenticeship. If evaluated, these combined efforts at expanding LTC RAPs may offer some evidence of their effectiveness and document ways for addressing the challenges. Thus, policymakers potentially could consider registered apprenticeship as a partial means to address the long-term care workforce shortage and the need for better skills, but the need to understand and address the challenges to its success are formidable.

Implications for Evaluation Design

Although the main discussion of evaluation options will appear in a subsequent report, this report provides an overview of the implications for any potential future evaluation drawn from the site visits. Several aspects of the LTC RAPs visited are relevant to a better understanding of how these programs may be evaluated in the future:

• A key issue is whether the sites’ program goals are the same and the intervention is generally uniform across sites. The goals across programs need to be relatively similar in order to evaluate the programs as a whole. Although the programs are registered by DOL, sponsors have considerable latitude in deciding their goals and activities. That said, the goals of the LTC RAPs visited are roughly consistent across the programs.

• The duration of the LTC RAPs is an important issue for any evaluation. An evaluation that involves longitudinal analysis would need to consider how much time is needed to implement an intervention in order to be able to assess its full effect. The programs in this study vary widely in time for completion, ranging from 1,680 hours to 3,232 hours (approximately 1.5 years). Longer interventions can be more expensive than shorter ones, particularly if they involve multiple waves of data collection.

• The size of the LTC RAPs visited ranged from eight to 183 active apprentices as of May 2011. These sites were the largest sites with active programs, but they were still relatively small for using conventional experimental or quasi-experimental evaluation options. One option would be to pool samples of apprentices across multiple programs but such an approach might complicate efforts to assure that comparison groups are appropriate.

• Across these sites, limited data on important outcomes are collected. Most sites did obtain data on wages, benefits, tenure, and turnover, but not in a common form across sites. Most sites collect annual turnover but one tracks only monthly turnover. Any future evaluation would involve collecting additional data beyond what sites currently collect.

• Designing valid comparison groups for those entering apprenticeships may be difficult because of the selection process for entrants into the program. Almost all sites have selection criteria for apprenticeships; employees must typically apply or be recommended and subsequently be assessed and selected for an apprenticeship from a subset of all employees. As a result, regular workers not selected to enter apprenticeships would not be a valid comparison group, since unmeasured differences between them and apprentices would likely bias estimates of the program impact. A randomized control trial effectively addresses such selection issues. However, without random assignment, evaluators must seek other options to distinguish between program effects and effects linked to unmeasured individual differences by identifying natural experiments or quasi-experimental evaluation methods.

• A final potential complicating factor is that most programs have apprentices who complete the apprenticeship serve as mentors to the remaining non-apprentice staff. This intentional spillover of the intervention to non-intervention employees makes the comparison of apprentice outcomes to non-apprentice outcomes within a site extremely difficult. An evaluation might require a comparison group outside of the sponsor’s organization, or at least another of the sponsor’s facilities not subject to the intervention, to address this issue.

• Evaluating the gains and losses for employers using the LTC RAP model is another option. There are research tools for assessing the employer perspective, but usually not in an experimental or comparison group context.