5.2.1 Survey Questions
Many of the important decisions required in designing a survey of employers are related to the interview's content. The experience of Holzer and others suggests that a telephone survey of employers should last no longer than 15-20 minutes. As a result, both the topics covered and the specific questions included in the survey would have to be limited. However, there are many high-priority subjects for the survey to address. Exhibit 5.1 presents a list of potential topics for the survey as well as examples of questions that might be asked for each of the topics. The selection of specific questions for the core survey would depend on a number of factors, including whether supplementary interviews were conducted in addition to the core survey (see discussion below).(23)
|Survey Topics||Illustrative Questions|
|Employer Characteristics and Circumstances|
|Business characteristics||Does this company operate at more than one site?
Is this a minority-owned company?
What percentage of your customer/clients are African American? Asian? Hispanic?
|Workforce characteristics||How many employees currently work for the company at this location?
How many of your employees are in jobs that do not require any particular skills, education, previous training or experience when they are hired?
|Business and employment conditions||Has the company's total revenue during the last year increased compared to its revenue in the previous year?
Is the number of current employees greater than it was a year ago?
Approximately how many entry-level vacancies are you currently trying to fill?
|Employer Attitudes and Perceptions|
|Interest in TANF recipients||Are you ready or reluctant to hire welfare recipients as employees?
Do you use the services of a public agency, private company, or community organization that focuses on working with welfare recipients?
Would you be more likely to hire welfare recipients, especially without a high school diploma or work experience, if an agency provided a 50 percent tax credit against their wages for one year?
|Perceptions of recipients||In general, how does the job performance of welfare recipients you have hired compare to the performance of your other entry-level employees?|
|Government supports||Do you know that tax credits for hiring welfare recipients are available from the federal government? Have you claimed such credits for employees you have hired?|
|Recruitment and hiring||Do you use the services of a public agency, private company, or community organization in recruiting and hiring employees? Would you rely on one or more organizations as a partner in finding and hiring good people?
Have you participated in a job fair or some other event sponsored by a public agency or a community organization?
During the past year, have you used the Internet when trying to fill job vacancies?
Do you have job applicants take any kind of tests? Which kinds of tests do you administer?
Roughly what percentage of your job applications is from females? African Americans? Asians? Hispanics?
|Compensation||Do you use the services of a public agency, private company, or community organization in determining the wages or fringe benefits paid to employees?
What is the range of starting wages for entry-level positions? Is this figure hourly, weekly, monthly or yearly?
Do you offer health insurance to entry-level employees? Is there a waiting period for health insurance eligibility? What percentage of your employees is fully enrolled?
Do you offer a retirement plan?
|Employee support and assistance||Does your company offer employees assistance in finding child care, or with other types of assistance (e.g., housing, counseling, or transportation)?
Do you use the services of a public agency, private company, or community organization in providing assistance to employees in obtaining child care, housing, counseling, or transportation?
|Training||Do you use the services of a public agency, private company, or community organization in training any of your employees?
What types of training does the company provide to employees?
|Employee performance assessment||How do you assess the performance of your employees?
Do you use the services of a public agency, private company, or community organization in assessing the performance of your employees?
Another key decision concerns the frame of reference for the survey questions. Many of the questions in Holzer's surveys asked the respondent about the last employee hired. For example, the four-city survey asked about the type of work the last employee performed, whether the position required a high school diploma, and whether the person was first hired through a temporary employment agency. Alternatively, a survey could ask about multiple individuals hired by the employer over a longer time period. For example, the respondent could be asked about the last five to 10 employees hired by the company or about all employees hired during the last 12 months.
Holzer's approach asking about the employer's most recent hire has two important advantages. First, it minimizes recall error and maximizes respondent understanding by focusing unambiguously on recent experience. Second, it can be posed, using identical language, to all respondents whose companies have hired at least one person. The key disadvantage is that, particularly for larger establishments, the most recent hire may not be representative.
5.2.2 Sample Design
As for the survey's content, important decisions about the survey's sample, data collection method, survey testing and survey implementation would need to be made. In this section we review some of the issues to be considered in developing a survey with employers. Survey sampling issues-identifying the study's target population, finding or assembling a list of target population members, and actually selecting the sample-are examined first. This is followed by a discussion of survey testing and implementation issues.
Target Population. The study's target population should be defined as precisely as possible in terms of the larger population to which the survey's findings would be generalized. This requires the development of inclusion and exclusion criteria that describe the characteristics of the business organizations to be included in the survey's sample.
For a survey of employers, there are a number of possible criteria that might be used. These include the following:
- Number of Employees. Given the survey's focus on employment practices, businesses targeted for the survey would need to have at least one employee. There are a number of ways to define "employee." The definition, for example, might include full-time, part-time, or seasonal laborers, contractors and/or independent consultants. An additional consideration is whether the minimum number of employees should be set at a number greater than one that is, should enterprises operating on a very small scale be excluded from the survey.
- Business Organization. As previously discussed, it is important to distinguish which business units should be targeted by the survey. For example, in the case of multi-unit businesses, does the portrayal of employment practices offered by "headquarters" accurately describe what occurs at lower levels of the organization's hierarchy? Past surveys such as Holzer's MCTES and four-city surveys have targeted the establishment level of multi-unit businesses.
- Industry Sector. The survey's target population may include private for-profit, non-profit, and public-sector (including or excluding military) employers. The MCTES survey included all for-profit, non-profit and public-sector employers, but not military employers. Additionally, depending on the list of business organizations that is used, it is often possible to distinguish between employers within specific industry sectors, such as manufacturing, service, and transportation.
- Geography. The geographical coverage of a national survey has to be specified. Does "national" include Hawaii and Alaska as well as the 48 continental states? Are Puerto Rico and the other U.S. territories included?
Sampling Frame. As discussed above, identifying a sampling frame that provides adequate coverage of business organizations and employers poses a number of challenges. Several factors and tradeoffs should be taken into account in developing the sampling frame. For example:
- To what extent is national coverage essential? Is a frame that is limited to specific states, regions, cities, or industries satisfactory?
- To what extent should the sample frame allow stratification? Should the stratification include number of employees, geographic location, industry sector, or other factors?
- Is one concerned about certain types of systematic coverage errors such as exclusion of certain types of employers? If so, are some types of errors acceptable?
- Particularly given the dynamic nature of the private sector, at what frequency should the database or list be updated?
- How important is it for the sample list to include a contact name in the organization (such as the name of an employee in the organization's human resources unit)?
Exhibit 5.2 contrasts the strengths and weaknesses of four possible sample frames: the Longitudinal Employer-Household Dynamics (LEHD) database; InfoUSA; Dun & Bradstreet; and business or trade organization lists. Applying the above criteria to these lists illustrates some of the choices, or tradeoffs, that would need to be made. For example, the LEHD includes comprehensive Unemployment Insurance data that has been linked to a number of other sources including the Survey of Income and Program Participation, but only covers 20 states. Alternatively, the InfoUSA list provides slightly less comprehensive coverage of employers, but is available nationwide. It also includes fields that identify establishments (versus companies or enterprises) and provides contact names for human resources personnel within the organization.
|Dun & Bradstreet||
|Chamber of Commerce, National Association of Manufacturers, or Other List Maintained by Trade Association or Business Group||
Sample Selection. Once a sampling frame is identified, several choices need to be made about how to actually draw the sample. First, investigators must decide what, if any, subgroups (or strata) within the target population are of particular interest. For example, to what extent is one interested in making distinctions among the following types of employers:
- Employers with different numbers of employees (e.g., small, medium and large businesses);
- Employers from different geographic locations (e.g., regions or urban and rural areas); and
- Employers with different business types (e.g., manufacturing, retail, service, as noted by Standard Industrial, or SIC, codes)?
Additionally, investigators need to decide whether two-way strata are of interest. For example, is there interest in making comparisons among small businesses that are located in rural versus urban areas?
Based on these decisions, an appropriate sample size can be determined. The survey sample size depends on:
- The amount of sampling error that is acceptable;
- The size of the target population;
- How varied the population is with respect to the characteristics of interest (such as the percentage of employers estimated to employ TANF recipients); and
- The smallest subgroup within the overall sample for which separate estimates will be made.
5.2.3 Data Collection Method
Surveys of employers and labor market intermediaries could be either self- or interviewer-administered. Additionally, a number of different data collection modes-mail, telephone, Internet, or in-person-could be used to collect responses. To some extent, the data collection method is affected by the sampling frame and design. For example, it is difficult to conduct a mail survey without up-to-date mailing addresses or a telephone survey without current telephone numbers.
Some survey methods or combinations of methods typically produce better survey response from specific populations than other methods. Mail surveys with businesses generally result in lower response rates than telephone surveys or mixed-method approaches using both a mail and telephone contact. Mail surveys, however, are generally less expensive to administer than those conducted by telephone. Holzer's MTCES and four-city surveys were administered by telephone. In contrast, Abt Associates survey with intermediary organizations utilized a mixed-method approach, with an initial mail contact followed by telephone contact with non-responders.
5.2.4 Resources for Questionnaire Development
Development of the employer and intermediary survey instruments probably would draw heavily on two types of resources. One includes existing instruments, notably Holzer's employer surveys and Abt Associates' survey of labor market intermediaries. The other important resource includes researchers and other individuals with pertinent expertise, including employers themselves. Several members of the expert advisory panel would be valuable advisors in the questionnaire development process.
5.2.5 Survey Testing
Pre-testing a survey instrument is an essential component of good survey design. Pre-testing may involve focus groups that provide additional input regarding the survey's content and format, or pilot tests where the instrument is administered to individuals who resemble or are members of the survey's target population. In the latter case, feedback may be obtained through written comments or debriefing interviews.
"report.pdf" (pdf, 212.99Kb)