Evaluation Design Options for the Long-Term Care Registered Apprenticeship Program. 6.3. Focus Groups


Primary data collection using focus groups may provide a low-cost research design option that would obtain information to inform policy. Focus groups would provide a means for apprentices to voice opinions on a range of topics about how the apprenticeship program works from their perspective, its strengths and weaknesses, and how it could be improved. Focus groups of employer sponsors could provide information on why they chose an apprenticeship program and important program elements, and what they perceive to be the benefits to employers and apprentices. In both cases, focus groups allow for a more extensive, open-ended data gathering than is possible in a survey. The focus groups would also allow gathering detailed information about the LTC RAP professions for which there are two few sites to conduct statistical analyses. Because the data is qualitative and does not include a large enough sample to conduct statistical analyses, it cannot answer such quantitative questions as whether LTC RAP increases earnings or reduces turnover.

Research Questions

The research questions to be addressed by the focus groups center on the motivation for entering/starting the LTC RAP and the views of direct care workers and employers about the operation of the program.

1. Apprentices

Potential research questions to be addressed by focus groups of apprentices include:

  • How and to what extent does apprenticeship affect caregiving abilities? What types of new knowledge and skills were attained? How and how well does the apprenticeship teach hard and soft skills and problem-solving skills?

  • How does the apprenticeship affect working with supervisors and working in teams? How do the apprentices relate to comparable workers who do not participate in the LTC RAP?

  • How does apprenticeship differ from other training previously received?

  • What personal funds and time did apprentices use in order to participate?

  • Were they able to complete the apprenticeship and why or why not?

  • What did they like about the apprenticeship experience? What did they dislike?

  • What changes would they suggest making in the apprenticeship program?

  • Can LTC RAP play a major role in solving long-term care workforce problems, and if so, under what circumstances?

2. Employers

Research questions to be addressed by focus groups of employer sponsors would include:

  • Why did the organization choose an apprenticeship program to train and develop direct care workers?

  • What relationships with outside organizations were beneficial in developing your program?

  • What financial, training and other resources would help the organization to best operate the program?

  • What is the value added of apprenticeship over traditional training?

  • What criteria does the organization use to select workers into the program?

  • What do employers like about the LTC RAP? What did they dislike?

  • What changes would employers suggest making in the apprenticeship program?

  • Can LTC RAP play a major role in solving long-term care workforce problems, and if so, under what circumstances?

Brief Overview of Research Design

1. Apprentice Focus Groups

This evaluation design would conduct multiple apprentice and employer/sponsor focus groups. First, eight apprentice focus groups in eight sites would be conducted. The evaluator would recruit apprentices from selected employer sponsors who would provide the names and contact information for all of their employed apprentices. Employers would not be allowed to select the participants. Two focus groups would be conducted for each of the four LTC RAP occupations. Conducting more than one focus group for each occupation and conducting multiple focus groups across employers minimizes the possibility that results are purely idiosyncratic or reflect the experience of just a few participants who are the most verbal in an individual group. Final site selection of employer sponsors would occur after discussions with ASPE and DOL. Possible selection criteria for identifying employer sponsors from whom apprentices would be selected would be total number of apprentices employed, occupation type, ownership type and chain status.

Each focus group will consist of approximately 8-9 worker participants, for a total of approximately 70 participants. Budget assumptions are based on recruiting English speaking participants in order to avoid the added costs associated with translating the focus group protocol into the additional language and translating the focus group discussion into English. Even though 70 participants would be included, OMB clearance would probably not be needed because each focus group would contain a maximum of nine individuals, and question scripts would necessarily differ across occupations and by employer so that no set of focus group questions would be the same.

The evaluator would work with each employer to help arrange for local logistics for conducting each focus group. In order to encourage workers to be candid, the focus group will be held at a location away from the employer’s work site, but convenient for the apprentices. A small token of appreciation would be provided to the employer for their efforts in providing the names and contact information of the apprentices and suggesting focus group locations. A $75 payment would be provided to focus group members as an incentive to participate.

No comparison group of non-apprentices would be involved. With such small numbers of participants, no statistical analysis would be performed.

2. Employer Focus Groups

Because of the difficulty and expense of bringing employer sponsors around the country together for a focus group, only two focus groups of LTC RAP employer sponsors is proposed. For convenience, this focus group would occur as a side meeting of long-term care national association annual meetings. For example, the American Health Care Association and LeadingAge are two of the largest national associations of nursing homes and residential care facilities. Currently, many LTC RAP employer sponsors are members of these organizations. Both organizations have annual meetings at which a focus group potentially could be conducted.

In addition to invitations to employer sponsors from the evaluator, letters from officials of the national associations and high ranking HHS and DOL officials encouraging organizational members to participate in the focus groups would be included to promote the importance and legitimacy of the endeavor. The evaluator would select up to nine employer sponsors from each association who would attend separate focus groups. Once contacted, the evaluator would work to identify the best day and time across all selected employer sponsor personnel attending their respective national association meetings. Employer sponsors would not incur additional costs for attending the focus group session beyond their time to prepare and attend. Therefore, only some nominal incentive payment to participants is anticipated.

Sample Frame Construction

For the apprentice focus groups, the entire universe of employers and currently employed apprentices, all identified from the most recent RAPIDS data extract, would serve as the sample frame. Employers would be stratified by their LTC RAP occupations and number of employed apprentices, and on selected apprenticeship program characteristics thought to be instrumental in conducting apprenticeships such as ownership status. Once employers are identified, all currently employed past and current apprentices would be eligible for recruitment. For the employer sponsor focus groups, the evaluator would work with association management to identify a convenience sample of LTC RAPs in RAPIDS data who are members of the respective associations.

Domains on Which Information Will Be Gathered

To guide discussion, the evaluator would develop a focus group protocol and structured discussion guides, including specific questions with suggested probes. The protocol would include discussion of approximately 7-8 topics and last about 2 hours. Questions for apprentices would be prepared recognizing the limited education of most apprentices. Topic domains would include those addressed in the research questions previously listed.

Data Collection Process

As part of the enlistment process, participants will receive letters and phone calls asking them to participate. Provisions for informed consent and confidentiality of responses would be provided as part of the process. Any special needs of enlisted participants would be identified. Details of the logistics (e.g., date, time, and location) of the focus group would be provided to participants.

The day before the focus group, two staff members from the evaluator would travel to each site to preview the site, make any final arrangements, and setup. Upon arrival, focus group participants would be greeted, registered, introduced to other participants, and invited into the room. Light refreshments and food would be provided. The focus group moderator would provide a brief overview of the process, establish rapport with participants, and conduct the focus group. The second staff person would take notes on a laptop and provide assistance for issues that arise during the group so the moderator could continue with group moderation. The focus group will be recorded; only apprentices who agree to have their comments recorded will be included in the focus group. Generally, it is assumed that given the physically demanding job of apprentices, none would need extensive accommodations during the meeting. For the focus group for employer sponsors, any special accommodations most likely would be provided as part of their association meeting.

Timeframe to Collect and Analyze Data

Approximately 12-14 months would be needed to identify employers and sites, develop the necessary materials, recruit apprentices and leadership staff from employer sponsors, conduct the focus groups, analyze the data, and develop a research report of findings. If OMB clearance is required, then an additional 6 months would be required to complete the project.

Ballpark Estimated Cost

The entire focus group evaluation option would cost approximately $150,000, which includes labor and travel costs to develop the protocol, conduct the focus groups, analyze the focus group data, and write a single summary report of findings. The subset of costs for actual conduct of each focus group would be approximately $10,000 for recruitment from a list, logistics, facilitation and written transcripts.

Main Methods for Analyzing Data

The transcripts of the focus groups would be analyzed for themes either by hand or through a qualitative analysis software package such as QSR NVivo. A single thematic report would be developed along the lines of the topic domains contained in the discussion guide. One section of the report would discuss apprentice findings, and a second section would discuss employer sponsor findings.

View full report


"LTCRAPedo.pdf" (pdf, 870.86Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®