Private Employers and TANF Recipients. 5.2  Design of a Core Survey of Employers

05/25/2004

5.2.1 Survey Questions

Many of the important decisions required in designing a survey of employers are related to the interview's content. The experience of Holzer and others suggests that a telephone survey of employers should last no longer than 15-20 minutes. As a result, both the topics covered and the specific questions included in the survey would have to be limited. However, there are many high-priority subjects for the survey to address. Exhibit 5.1 presents a list of potential topics for the survey as well as examples of questions that might be asked for each of the topics. The selection of specific questions for the core survey would depend on a number of factors, including whether supplementary interviews were conducted in addition to the core survey (see discussion below).(23)

Exhibit 5.1
Potential Employer Survey Topics and Illustrative Questions
Survey Topics Illustrative Questions
Employer Characteristics and Circumstances
Business characteristics Does this company operate at more than one site?

Is this a minority-owned company?

What percentage of your customer/clients are African American? Asian? Hispanic?

Workforce characteristics How many employees currently work for the company at this location?

How many of your employees are in jobs that do not require any particular skills, education, previous training or experience when they are hired?

Business and employment conditions Has the company's total revenue during the last year increased compared to its revenue in the previous year?

Is the number of current employees greater than it was a year ago?

Approximately how many entry-level vacancies are you currently trying to fill?

Employer Attitudes and Perceptions
Interest in TANF recipients Are you ready or reluctant to hire welfare recipients as employees?

Do you use the services of a public agency, private company, or community organization that focuses on working with welfare recipients?

Would you be more likely to hire welfare recipients, especially without a high school diploma or work experience, if an agency provided a 50 percent tax credit against their wages for one year?

Perceptions of recipients In general, how does the job performance of welfare recipients you have hired compare to the performance of your other entry-level employees?
Government supports Do you know that tax credits for hiring welfare recipients are available from the federal government? Have you claimed such credits for employees you have hired?
Employer Practices
Recruitment and hiring Do you use the services of a public agency, private company, or community organization in recruiting and hiring employees? Would you rely on one or more organizations as a partner in finding and hiring good people?

Have you participated in a job fair or some other event sponsored by a public agency or a community organization?

During the past year, have you used the Internet when trying to fill job vacancies?

Do you have job applicants take any kind of tests? Which kinds of tests do you administer?

Roughly what percentage of your job applications is from females? African Americans? Asians? Hispanics?

Compensation Do you use the services of a public agency, private company, or community organization in determining the wages or fringe benefits paid to employees?

What is the range of starting wages for entry-level positions? Is this figure hourly, weekly, monthly or yearly?

Do you offer health insurance to entry-level employees? Is there a waiting period for health insurance eligibility? What percentage of your employees is fully enrolled?

Do you offer a retirement plan?

Employee support and assistance Does your company offer employees assistance in finding child care, or with other types of assistance (e.g., housing, counseling, or transportation)?

Do you use the services of a public agency, private company, or community organization in providing assistance to employees in obtaining child care, housing, counseling, or transportation?

Training Do you use the services of a public agency, private company, or community organization in training any of your employees?

What types of training does the company provide to employees?

Employee performance assessment How do you assess the performance of your employees?

Do you use the services of a public agency, private company, or community organization in assessing the performance of your employees?

Another key decision concerns the frame of reference for the survey questions. Many of the questions in Holzer's surveys asked the respondent about the last employee hired. For example, the four-city survey asked about the type of work the last employee performed, whether the position required a high school diploma, and whether the person was first hired through a temporary employment agency. Alternatively, a survey could ask about multiple individuals hired by the employer over a longer time period. For example, the respondent could be asked about the last five to 10 employees hired by the company or about all employees hired during the last 12 months.

Holzer's approach  asking about the employer's most recent hire  has two important advantages. First, it minimizes recall error and maximizes respondent understanding by focusing unambiguously on recent experience. Second, it can be posed, using identical language, to all respondents whose companies have hired at least one person. The key disadvantage is that, particularly for larger establishments, the most recent hire may not be representative.

5.2.2 Sample Design

As for the survey's content, important decisions about the survey's sample, data collection method, survey testing and survey implementation would need to be made. In this section we review some of the issues to be considered in developing a survey with employers. Survey sampling issues-identifying the study's target population, finding or assembling a list of target population members, and actually selecting the sample-are examined first. This is followed by a discussion of survey testing and implementation issues.

Target Population. The study's target population should be defined as precisely as possible in terms of the larger population to which the survey's findings would be generalized. This requires the development of inclusion and exclusion criteria that describe the characteristics of the business organizations to be included in the survey's sample.

For a survey of employers, there are a number of possible criteria that might be used. These include the following:

  1. Number of Employees. Given the survey's focus on employment practices, businesses targeted for the survey would need to have at least one employee. There are a number of ways to define "employee." The definition, for example, might include full-time, part-time, or seasonal laborers, contractors and/or independent consultants. An additional consideration is whether the minimum number of employees should be set at a number greater than one  that is, should enterprises operating on a very small scale be excluded from the survey.
  2. Business Organization. As previously discussed, it is important to distinguish which business units should be targeted by the survey. For example, in the case of multi-unit businesses, does the portrayal of employment practices offered by "headquarters" accurately describe what occurs at lower levels of the organization's hierarchy? Past surveys such as Holzer's MCTES and four-city surveys have targeted the establishment level of multi-unit businesses.
  3. Industry Sector. The survey's target population may include private for-profit, non-profit, and public-sector (including or excluding military) employers. The MCTES survey included all for-profit, non-profit and public-sector employers, but not military employers. Additionally, depending on the list of business organizations that is used, it is often possible to distinguish between employers within specific industry sectors, such as manufacturing, service, and transportation.
  4. Geography. The geographical coverage of a national survey has to be specified. Does "national" include Hawaii and Alaska as well as the 48 continental states? Are Puerto Rico and the other U.S. territories included?

Sampling Frame. As discussed above, identifying a sampling frame that provides adequate coverage of business organizations and employers poses a number of challenges. Several factors and tradeoffs should be taken into account in developing the sampling frame. For example:

  • To what extent is national coverage essential? Is a frame that is limited to specific states, regions, cities, or industries satisfactory?
  • To what extent should the sample frame allow stratification? Should the stratification include number of employees, geographic location, industry sector, or other factors?
  • Is one concerned about certain types of systematic coverage errors  such as exclusion of certain types of employers? If so, are some types of errors acceptable?
  • Particularly given the dynamic nature of the private sector, at what frequency should the database or list be updated?
  • How important is it for the sample list to include a contact name in the organization (such as the name of an employee in the organization's human resources unit)?

Exhibit 5.2 contrasts the strengths and weaknesses of four possible sample frames: the Longitudinal Employer-Household Dynamics (LEHD) database; InfoUSA; Dun & Bradstreet; and business or trade organization lists. Applying the above criteria to these lists illustrates some of the choices, or tradeoffs, that would need to be made. For example, the LEHD includes comprehensive Unemployment Insurance data that has been linked to a number of other sources including the Survey of Income and Program Participation, but only covers 20 states. Alternatively, the InfoUSA list provides slightly less comprehensive coverage of employers, but is available nationwide. It also includes fields that identify establishments (versus companies or enterprises) and provides contact names for human resources personnel within the organization.

Exhibit 5.2
Strengths and Weaknesses of Possible Employer Sample Frames
  Description Strengths Weaknesses
Longitudinal
Employer-Household
Dynamics (LEHD)
  • This data source is comprised of unemployment wage records and other employer-level data taken from tax and wage reporting records in more than 20 states. States report 27 indicators for each county, industry and quarter.
  • Anderson et al. (2002) use this data source in their research on interactions of workers and firms in the low-wage labor market.
  • Includes all employers who are required to file unemployment and other employer-level data taken from tax and wage reporting records. Employer coverage has been documented as being as high as 98% in LEHD covered states.
  • List of employers is updated at least quarterly.
  • Data will support stratification by employer size and type, as well as by some geography.
  • Cannot be used to develop a national sample - data are currently available for more than 20 states (that comprise about 50% of the total U.S. population).
  • National studies show that about 90% of the workers in the US economy are in jobs covered by the UI system, with systematic exclusions in the low-wage labor market (e.g., agricultural workers, household help, and small businesses), as well as some public sector employers such as the federal government and railroads.
  • Data do not include contact information for individuals at firms.
  • Unclear to what extent data could be used to "profile" business operations so that establishments could be sampled.
InfoUSA
  • InfoUSA is a national employment directory that is based on telephone directory listings that have been supplemented with industry directories and other sources to increase coverage.
  • Businesses listed in this database can be targeted by SIC code, sales volume, employee size and title code (e.g., human resources contact).
  • Geography can be targeted down to the Census block-group level.
  • The business listings are continuously updated as new information becomes available.
  • The database is available from Genesys Survey Sampling for $0.11 per record, with a $100 set up fee.
  • A similar database (produced by Survey Sampling International) was used to develop the MCTES survey sample and the sample used by Holzer in his 1998-99 study of employers.
  • Could be used to draw a national sample of employers, including both for-profit, government, and non-profit employers.
  • Data will support stratification by employer size and type, as well as by geography (i.e., Census block-group).
  • Data are readily accessible from Genesys Survey Sampling for a small fee.
  • Data files can include address and telephone information and, in some cases, the name of a human resource personnel contact, SIC codes, geographic markers at the Census block-group level, and employer size.
  • Coverage is described as comprehensive, but is not 100%. It is unknown to what extent the coverage error is systematic (i.e., whether firms with specific characteristics are systematically excluded from the list).
Dun & Bradstreet
  • The Dun & Bradstreet database is a national employment directory that is compiled using credit rating information for businesses.
  • Businesses listed in this database can be targeted by SIC code, sales volume, employee size and title code (e.g., human resources contact). The title code options are more comprehensive than those offered by InfoUSA.
  • Geography can be targeted down to the Census block-group level.
  • The business listings are continuously updated as new information becomes available.
  • The database is available from Genesys Survey Sampling for $0.16 per record, with a $100 set up fee.
  • Could be used to draw a national sample.
  • Data will support stratification by employer size and type, as well as by geography (i.e. Census block-group).
  • Data are readily accessible from Genesys Survey Sampling for a small fee.
  • Data files can include address and telephone information and, in some cases, the name of a human resource personnel contact, SIC codes, geographic markers at the Census block-group level, and employer size.
  • D&B includes more human resources personnel contact information than does InfoUSA.
  • Coverage is described as comprehensive, but is not 100%.
  • There may be systematic coverage errors since the list of businesses is comprised of those who have a credit history (e.g., small and new employers may not have a credit history).
Chamber of Commerce, National Association of Manufacturers, or Other List Maintained by Trade Association or Business Group
  • Lists available from the Chamber of Commerce, National Association of Manufacturers, or other trade associations generally include contact information for members and their affiliates. Generally speaking, the lists include address information and, in some cases, a contact person within the organization.
  • Coverage is consistent with membership (e.g., national organizations = national coverage; industry organizations = sector-specific coverage).
  • SIC codes and other information required for establishment profiling is list dependent, but frequently unavailable.
  • Geography may be targeted by zip code.
  • Lists are generally updated concurrent with membership renewal (e.g., annually).
  • May be obtained with minimal cost, but requires cooperation from organization.
  • Partnerships with membership organizations may improve survey legitimacy and corresponding survey response.
  • It may be possible to create a national sample.
  • Stand-alone lists systematically exclude certain employers (e.g., non-manufacturers). However, coverage could be improved by combining lists from multiple organizations.
  • Data elements that could be used to assist with sampling (e.g., SIC codes, Census geographic designations) are frequently unavailable.
  • Trade association lists generally do not include the information necessary to "profile" the sample for multi-unit businesses.

Sample Selection. Once a sampling frame is identified, several choices need to be made about how to actually draw the sample. First, investigators must decide what, if any, subgroups (or strata) within the target population are of particular interest. For example, to what extent is one interested in making distinctions among the following types of employers:

  • Employers with different numbers of employees (e.g., small, medium and large businesses);
  • Employers from different geographic locations (e.g., regions or urban and rural areas); and
  • Employers with different business types (e.g., manufacturing, retail, service, as noted by Standard Industrial, or SIC, codes)?

Additionally, investigators need to decide whether two-way strata are of interest. For example, is there interest in making comparisons among small businesses that are located in rural versus urban areas?

Based on these decisions, an appropriate sample size can be determined. The survey sample size depends on:

  • The amount of sampling error that is acceptable;
  • The size of the target population;
  • How varied the population is with respect to the characteristics of interest (such as the percentage of employers estimated to employ TANF recipients); and
  • The smallest subgroup within the overall sample for which separate estimates will be made.

5.2.3 Data Collection Method

Surveys of employers and labor market intermediaries could be either self- or interviewer-administered. Additionally, a number of different data collection modes-mail, telephone, Internet, or in-person-could be used to collect responses. To some extent, the data collection method is affected by the sampling frame and design. For example, it is difficult to conduct a mail survey without up-to-date mailing addresses or a telephone survey without current telephone numbers.

Some survey methods or combinations of methods typically produce better survey response from specific populations than other methods. Mail surveys with businesses generally result in lower response rates than telephone surveys or mixed-method approaches using both a mail and telephone contact. Mail surveys, however, are generally less expensive to administer than those conducted by telephone. Holzer's MTCES and four-city surveys were administered by telephone. In contrast, Abt Associates survey with intermediary organizations utilized a mixed-method approach, with an initial mail contact followed by telephone contact with non-responders.

5.2.4 Resources for Questionnaire Development

Development of the employer and intermediary survey instruments probably would draw heavily on two types of resources. One includes existing instruments, notably Holzer's employer surveys and Abt Associates' survey of labor market intermediaries. The other important resource includes researchers and other individuals with pertinent expertise, including employers themselves. Several members of the expert advisory panel would be valuable advisors in the questionnaire development process.

5.2.5 Survey Testing

Pre-testing a survey instrument is an essential component of good survey design. Pre-testing may involve focus groups that provide additional input regarding the survey's content and format, or pilot tests where the instrument is administered to individuals who resemble or are members of the survey's target population. In the latter case, feedback may be obtained through written comments or debriefing interviews.

View full report

Preview
Download

"report.pdf" (pdf, 212.99Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"appendix.pdf" (pdf, 433.16Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®