Characteristics of Long-Term Care Registered Apprenticeship Programs: Implications for Evaluation Design. Appendix C. Semi-structured Interview Guide for LTC RAP Site Visits

09/13/2011

Development of Potential Research Designs to Evaluate the Long-Term Care Registered Apprenticeship Program

Site Visit Interview Guide

Introduction

I am (we are) researchers from [RTI International and/or the Urban Institute]. We are conducting a research study of Long-Term Care Registered Apprenticeship Programs (LTC RAPs) for the U.S. Department of Health and Human Services and U.S. Department of Labor (DOL). The goal of the study is to provide the Federal Government with feedback regarding how a future evaluation of these programs could be designed.

Our visit here today is a part of the study to learn about how these programs are implemented. We are conducting a total of six 3-day site visits to selected programs to talk with [apprentices, LTC RAP directors and staff; employer staff (if different from the LTC RAP director and staff), related instruction providers (either onsite or offsite); state workforce agency liaisons; state and local workforce investment board (WIB) staff; and DOL Office of Apprenticeship (OA) state and regional directors]. Our aim is to learn from your experiences, not audit or judge your programs.

Confidentiality Statement: Before beginning the interview, I (we) want to thank you for agreeing to participate in the study. I (we) know that you are busy and we will try to be as focused as possible. We have many questions and are going to talk to many different people, so please do not feel as though we expect you to be able to answer every question. Your participation in this discussion is voluntary and you may choose to not answer questions you do not wish to.

In addition, before we start, I want to let you know that though we take notes at these interviews, information provided will not be linked with the name of the person who provided the information in our report about the site visits. When we write about our findings, information from all interviews is organized and presented so that no one person can be identified.

With your permission, we would like to tape record the interview so that as we review our notes, if anything is unclear, we would review that part of the tape to clarify what was said. After the interview, we will send you our written notes regarding the interview to receive any comments you may have for us. Do we have your permission to tape record the interview?
     Yes     No

Do you have any questions before we begin?

Will you participate?

Background Information and Program Context

  1. Before we begin, we’d like to get some general information on you and verify some information about your Registered Apprenticeship Program. [Ask for a card.]
    1. Name and Title
    2. Organization name
    3. Contact information (address, telephone, fax, e-mail)
    4. Web site address
    5. Role in Registered Apprenticeship Program and how long involved in program
  2. We learned a little about your organization from public sources. (Tell them 2-3 things). Would you provide some additional background about your organization and its workforce needs? How does the local labor market affect your hiring and retaining of long-term care staff? [Look up information on company/organizational information, occupations and skills needed for organization, and local labor market conditions prior to visit.]
  3. What are your reasons for participating in a Registered Apprenticeship Program? How was it decided to use registered apprenticeship rather than other training options?
  4. What were the main goals/purposes of yourRegistered Apprenticeship Program?

Design and Early Implementation

  1. Please describe how your Registered Apprenticeship Program is designed and how these features compare to other training options.
    1. Occupation(s) of focus [CNA, DSS, HSS, and/or HHA]
    2. Skills/competencies to be learned [consult work process documents from DOL]
    3. Length of the program
    4. Number of apprentices active at this time and over the course of the year.
    5. Selection process for apprentices
    6. Time spent in structured on-the-job training (OJT) per week and over the course of the program
    7. Time spent in classroom or on-line receiving related instruction per week and over the course of the program
    8. Milestones (such as interim credentialing, advanced training) and testing
    9. Scheduled salary increases through the program (wage progression)
    10. Mentoring or informal peer-to-peer support
    11. Intended outcomes such as licensure, retention, or promotion
  2. How did the start-up and early implementation of your Registered Apprenticeship Program proceed (e.g., on-time, slow, etc.)? What financial resources or materials, or additional personnel or organizational relationships, did you need to begin the program? What factors facilitated project start-up? What factors hindered project start-up? How were they resolved?

Recruitment of Apprentices

  1. How do you recruit participants? What incentives (if any) are used to encourage participation? How does recruitment into your Registered Apprenticeship Program differ from recruitment activities for other training programs you offer?
  2. Is there an application process or test for employees or individuals to be accepted into your Registered Apprenticeship Program?
  3. Has it been easy or difficult to recruit participants into your Registered Apprenticeship Program? If there are difficulties in recruiting enough apprentices, what factors make recruitment difficult?

Training

  1. What does the training curriculum contain? May we view/have copies of any materials used?
  2. How is the time spent in OJT structured (e.g., duration, frequency, content)?
    1. Who is providing instruction, supervision, and mentorship? What qualifications do they have?
    2. What other resources are needed for the OJT (e.g., equipment)?
    3. What skills are apprentices learning and how are they learning them?
  3. How is the time spent in related instruction structured (e.g., duration, frequency, content)?
    1. Where is the “related instruction” provided -- onsite or at a separate training provider (e.g., community college, vocational/tech school, WIBs, in-house, other)?
    2. What are the qualifications of the instructors (e.g., nursing degree)? If in-house, what challenges, if any, have there been in hiring and retaining instructors?
    3. Who pays for the related instruction?
    4. Do apprentices receive any funds from the local One-Stop or WIB for related instruction? Does your state provide tuition subsidies for apprentices?
    5. Is the time spent in related instruction paid by the employer (i.e., is this time spent in the classroom during work hours)?
  4. What costs does an apprentice incur for participating in your Registered Apprenticeship Program (e.g., tuition, books, equipment)?
  5. How is the apprentice’s overall progress monitored? Who is responsible for the apprentice? How does coordination occur between the onsite mentor or supervisor and classroom instructors?
  6. How does your training program compare to other training programs for this occupation?

Partnering (where relevant, questions are also asked of employers and other types of co-sponsoring organizations including unions, associations, and government agencies)

  1. Please describe the key partnership(s) with other organizations for your Apprenticeship Program, including the role and responsibilities of each partner:
    1. Federal/state/regional apprenticeship organizations
    2. Education/training organizations (e.g., community colleges, universities/colleges, technical schools, state boards of nursing)
    3. One-Stop Career Centers/WIBs
    4. Employer or industry associations
    5. Labor-management organizations
    6. Others?
  2. How are employers recruited?
  3. Do one or more partners operate in multiple states?
  4. How were partnerships formed? Evolved over time?
  5. How do partners work together and communicate?
  6. How do partners ensure quality of training?
  7. Were there any organizations that you wanted to be more involved but were not? What were the reasons for their lack of involvement?

Staffing and Resources

  1. Please describe your Registered Apprenticeship Program’s management and staffing structure:
    1. Type and number of project staff
    2. Their roles in the program (e.g., clinical experience, mentors, classroom/related instruction)
    3. Employed versus contracted staff and oversight
    4. Location of staff
    5. Experience and/or credentials
    6. Frequency of turnover and recruitment process for new staff
  2. What are the major ongoing costs/expenditures for your Registered Apprenticeship Program? [If available, obtain budget document.]

Apprentice Participation and Characteristics

  1. How many apprentices are currently enrolled in your Registered Apprenticeship Program? How many apprentices have been enrolled over the life of the program regardless of whether they completed their apprenticeship? How many apprentices are enrolled over the course of a year?
  2. What are the important characteristics of your apprentices? [Review data in RAPIDS prior to the site visit.] Are there challenges with particular groups of apprentices (e.g., English as 2nd language, literacy, cultural/ethnic differences)?
  3. How many apprentices have completed your Registered Apprenticeship Program? Of those that completed the program, how many are still working at your company/organization? Are they still working in the same role, or have they changed jobs/been promoted in the organization? What challenges have apprentices experienced in completing the program? Have you been able to take any action to help apprentices complete the program?
  4. Have any apprentices dropped out of your Registered Apprenticeship Program? What have been the reasons for leaving the program?
  5. How far did they make it in the program before leaving? Do you know what any of them are doing now?

Apprenticeship Outcomes

  1. What measures do you use to assess the benefits of apprenticeship to participating apprentices (e.g., length of employment or promotions or increased wages)?
  2. How do the apprentices’ outcomes compare to the outcomes of employees or similar groups of individuals who do not participate in the program?
  3. Is your company tracking the changes in retention, productivity, quality of work, skill level of employees, or your overall return on investment related to costs? What have been the outcomes to date?
  4. How do you collect this information and how often? Do you have a database that you use?
  5. What are the advantages and disadvantages of the program to the employer? What have been the advantages and disadvantages of the program to the apprentices? Overall, what were the greatest impacts of your Registered Apprenticeship Program?
  6. Within the Registered Apprenticeship Program, are there other approaches, strategies, or services that you believe would contribute to better outcomes for the apprentices or the company/organization?

Sustainability, Replicability and Lessons Learned

  1. Has your Registered Apprenticeship Program become a regular activity in your company/organization? Are you seeing a return on investment? What resources are needed to better implement and sustain the program?
  2. To what extent would you recommend the Registered Apprenticeship Program to employers or organizations thinking about sponsoring a Registered Apprenticeship Program?
  3. What features of your Registered Apprenticeship Program would you recommend to other employers or organizations in designing their program?
  4. What features of the project would be difficult for other employers or organizations to use in designing their program?
  5. Looking back, what do you consider your most important accomplishments for your Registered Apprenticeship Program?
  6. What do you believe to be the main lessons learned from your Registered Apprenticeship Program?
  7. Is there anything that we have not asked but you will like to tell us about your program?

Supplemental Questions for Other LTC RAP Respondents

Trainers/Supervisors at Apprentices’ Employer (if not Program Sponsor) and Related Instruction Providers

  1. What is your role in designing and oversight of apprentice training activities?
  2. What do you teach them?
  3. How has the training for apprentices differed from training provided to other employees/students in this occupation?
  4. How do you test their mastery of a particular skill or subject area?
  5. What resources would you need as a supervisor/trainer/classroom instructor to bring more apprentices into the program?

LTC Apprentices (Note: The team will attempt to talk to apprentices in different phases of the program -- new enrollee, middle of training, completed training, and dropouts if available)

  1. How did you become an apprentice? How did you find out about the program? Why did you decide to become an apprentice? How is the program different from the regular training provided by the employer to other employees?
  2. How does the apprenticeship program differ from other training opportunities you may have had for this or a similar job?
  3. What does your typical week as an apprentice look like (e.g., general schedule during the week; time in OJT versus related instruction)? What kinds of skills are you learning?
  4. What do you think of the overall apprenticeship program? What did you think of the training you receive while on-the-job? In the classroom? What parts of the program worked best in your opinion (e.g., mentoring, skills learned, wage increases)? What parts of the program need improvement in your opinion? How does the apprenticeship program compare to other training or education experiences you have had? How has this training helped you do your job?
  5. Have there been any costs to you in participating in the apprenticeship program? Is your time spent in the classroom or in on-line training “on the clock,” or do you have to spend your own unpaid time to complete classroom or on-line training?
  6. What credential(s) will you have when you complete the program? Do you earn any credentials as you progress through the program? What position or title will you have when you complete the program?
  7. What have been the advantages to being in the apprenticeship program? What have been the disadvantages? Would you become an apprentice again, or encourage other people to become apprentices here? What are your career goals and does this apprenticeship move you toward achieving these goals (e.g., career advancement/career lattice, intent to stay in field, commitment to job/field)?

DOL OA State or Regional Directors/State Workforce Agency Liaisons/State and Local WIB or One-Stop Career Center Staff

  1. What has been your involvement with the LTC RAP in your area generally? What involvement specifically have you had with this LTC RAP we are interviewing here?
  2. How difficult or easy is it to market the Registered Apprenticeship model to long-term care providers? What have been some of the successes and challenges in marketing Registered Apprenticeship to long-term care employers?
  3. How did this LTC -RAP get started?
  4. What have been this LTC -RAPs accomplishments to date? What are any ongoing challenges?
  5. In your opinion, does the LTC -RAP have the capacity to increase the number of apprentices they have? What resources do you think the program would need to do so?

Checklist of Items to Collect/Review Ahead of or During Site Visit

  1. Background information about the local economic conditions, local labor market data, and information on industry in the area.
  2. Background information about the organization.
  3. Additional documentation/reports detailing major program components.
  4. Evaluation reports that may have been prepared on the program.
  5. Blank set of forms used in the apprentice file.
  6. Diagram showing how apprentices flow through the program (if they have it).
  7. Organizational chart for the company/organization (if they have it).

View full report

Preview
Download

"LTCRAPch.pdf" (pdf, 1.19Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®