[ Main Page of Report | Contents of Report ]
Appendix A: Data Sources and Definitions
Appendix B: Methodology for SIPP Analysis
In this appendix, we describe the three data sources used for our study of alternative work arrangements. First, we discuss the Current Population Survey, which is used to address how alternative and non-alternative arrangements differ in their job and worker characteristics. Next, we discuss the Survey of Income and Program Participation, which is used to analyze the effects of working for a temporary agency on subsequent employment and income. Finally, we discuss the Current Employment Statistics, which is used for the limited purpose of examining aggregate growth in employment in the temporary employment industry over time.
The CPS is a monthly survey of about 50,000 households conducted by the Bureau of the Census for the Bureau of Labor Statistics. The CPS is the primary source of data on labor force characteristics of the US civilian noninstitutional population. Monthly supplements provide estimates on a variety of topics of interest, including an annual March demographic supplement and a biannual February supplement on alternative work arrangements.(61) This latter supplement is used extensively by papers discussed in the literature review in Chapter 2.
The CPS data appropriate to analyze the characteristics of alternative work arrangement jobs and workers for low-income households come entirely from the February and March Basic Surveys and the supplements to these surveys. Beginning in 1995 and continuing in 1997 and 1999, the February Contingent Worker and Alternative Employment Supplement provides data on the types of work arrangements held by working respondents and on benefits provided by employers. All questions refer to the week prior to the survey date. The March Demographic supplement, administered each year, provides detailed demographic data and information on income levels and receipt of public assistance in the calendar year preceding the survey.
In our analysis of the CPS, we focus on comparing alternative and non-alternative arrangements regardless of income level, and for persons who are at risk of welfare receipt and/or low income, as defined in the next section.
As discussed in Chapter 2, the CPS February supplement asks a series of questions that allow us to categorize workers into a variety of alternative work arrangements. The questions ascertain whether the worker is employed on a temporary basis, and if so, the reason for that temporary status. Follow-up questions ask whether the worker's salary is paid by a temporary agency or a contract company, whether the worker is employed on-call, as a day laborer, or is self-employed. Responses to these questions are used to categorize workers as temporary agency and on-call workers.
Although the CPS data allow us to focus on many alternative work arrangements, we focus our analysis on temporary agency and on-call workers. We choose not to focus on the other work arrangements identified in the CPS because the distinctions between regular employment and these types of alternative work arrangements can be narrow. Also, since the core notion of alternative work arrangements describes the relationship between employer and employee (a relationship that differs from standard work arrangements), the nature of temporary agency and on-call work more closely fit our definition of alternative work arrangements for the purposes of this analysis.
In addition to examining the temporary and on-call population as a whole, we also use the CPS to examine the subset of workers who may be at risk of welfare recipiency. The definition of at risk of welfare recipiency is conceptually difficult to pin down. There are different types of public assistance--Aid to Families with Dependent Children/Temporary Assistance for Needy Families, Medicaid, and Food Stamps--and eligibility measures vary by state and family background. Thus, workers with identical earnings, but in different states and in different environments, might well be at different levels of risk of welfare recipiency. Therefore, we use two measures of at-risk workers: those workers who live in households with incomes below 150 percent of the federal poverty level and those who have received public assistance in the previous year (who may not be current welfare recipients).(62)
In the CPS, respondents are included in the survey for four consecutive months, left off for the following eight months, then included again in the survey for four months. This pattern permits us to match observations across different months of the survey and gather information on work arrangements and income at several points in time. Approximately three-fourths of the cases interviewed in the February supplement are also interviewed in March. The actual sample sizes for temporary workers, on-call workers, and regular workers, as well as the subset of public assistance recipients and workers with income below 150 percent of the poverty line are presented in Table 3.1 (see Chapter 3).
The matched data from the February and March surveys from 1995, 1997, and 1999 provide information on current work arrangements and benefits provided by the job held in February as well as receipt of public assistance and income from the previous (even-numbered) year. Almost three-fourths of the observations in each February supplement can be matched to March of the same year. In addition to matching within year, we also considered matching those cases interviewed in February with March of the following year. This would have allowed us to examine impacts of alternative work arrangements on subsequent outcomes. About three-eighths (37.5 percent) of those cases interviewed in the February supplement are also interviewed in February or March of the following year.(63) Matching observations to data from February and/or March of the following year reduces the number of observations by at least half, but allows examination of several outcome measures. Specifically, the supplements for February of 1996, 1998, and 2000 (a year earlier) provide data on whether persons employed in various work arrangements are still employed, and whether they are still employed at the same job. The basic survey for March 1996, 1998, and 2000 provides information on current employment, hours worked, and wages earned from the primary job. However, matching across years resulted in sample sizes that were not large enough to make strong statements about the effects of alternative work arrangements in the at risk population.
[Go to Contents]
The SIPP is a large-scale survey sponsored by the Census Bureau. For the years 1990 to 1993, fresh national samples were drawn annually. Each fresh annual sample constitutes a panel. Each panel is interviewed 8-10 times; each household is interviewed every fourth month, with successive quarters of the sample interviewed in each month. A 1996 panel with interview data through March 2000 has recently been released, although the longitudinal file is not yet available. Given the resource limitations on this project, we analyzed the 1990, 1991, 1992, and 1993 SIPP panels.
In each wave, the basic questionnaire provides data on the primary jobs held during the previous four months. Questions focus on the two jobs held for the largest number of hours. For these jobs, we know earnings or wage rate, months the job was held, usual hours worked per week, industry, and receipt of health insurance coverage. These variables, together with measures of income and public assistance receipt, are the primary outcome measures for our analysis of the subsequent effects of temporary work. The SIPP supplements the basic survey in each wave with detailed topical modules that provide information including employment and welfare recipiency history. These modules are used to select comparison groups with relatively similar work histories.
Our ability to determine alternative work arrangements in the SIPP is limited to identifying workers likely to be employed by temporary help agencies. The industry categorization of the two principal jobs, which is based on a 3-digit SIC code, can be used to learn whether the worker is employed in the temporary help services industry. We categorize a person as being employed in the temporary help services industry if he/she reports that either of the two reported jobs for a given wave is in SIC 736. SIC 736 includes those working for temporary help agencies and employment agencies. The four-digit category for temporary work (which includes some leasing companies) accounts for 89 percent of employment in SIC 736; the category for employment agencies accounts for the remaining 11 percent. Because individuals self-report the SIC code, we expect relatively few persons who work at companies managed by leasing companies will report in SIC 736; they would seem more likely to report the industry in which they work.
One question that arises when using SIC 736 to define temporary help workers is how well SIC 736 identifies agency temporary workers. We examine this using the February Contingent Workers Supplement to the CPS. Using CPS data for 1995 through 1999, we find that 57-70 percent of those in SIC 736 are classified as agency temporaries based on the supplement. In addition, roughly half of those who are identified as agency temporaries in the supplement reported SIC 736 as their industry. A large share of the latter mismatch appears to result from respondents reporting the industry of the place where the temporary agency assigned them to work rather than the industry of the temporary agency. Although these findings suggest caution in using the industry code to define temporary agency workers, we believe the share of agency workers within SIC 736 is high enough that strong differentials associated with agency work will be picked up by our analysis.
Similar to the CPS, in addition to examining temporary workers as a whole, we also use the SIPP to examine the subset of workers who may be at risk of welfare recipiency. We considered definitions of at risk of welfare receipt based on public assistance receipt as well as income relative to the federal poverty level.(64) In the end, we use a definition that balances the need to have enough cases for our analysis with a low enough income cutoff such that the sample members are at significant risk of welfare receipt.
Table A.1 reports the number of spells of temporary work in the 1990-1993 panels of the SIPP. The start of each spell is sampled and used as the base period in the analysis of the effects of temporary work. These numbers include only those temporary workers with data available one year later, a necessary requirement to measure outcomes. In addition to showing sample sizes for the full sample, Table A.1 shows various possible definitions of at risk or low income, including public assistance receipt, income below 150 percent of the poverty line, and income below 200 percent of the poverty line. Given the small sample sizes, we use the broadest of the three definitions--under 200 percent of poverty--for at risk or low income. This serves to balance the need for adequately-sized samples for analysis with the need to focus on a group with sufficiently low income to be at risk of receipt of public assistance.
| Previously Employed | Previously Unemployed | |
|---|---|---|
| Received Public Assistance in Prior Month | 65 | 152 |
| Individuals below 150% of Federal Poverty Level | 143 | 345 |
| Individuals below 200% of Federal Poverty Level | 234 | 425 |
| All Individuals | 648 | 738 |
| Source: SIPP 1990-1993 panels, calculations
by the Urban Institute. Note: Sample sizes include all cases that are observed a year after their first month in SIC 736. Poverty figures are based on income in the month prior to employment in SIC 736. |
||
The establishment payroll survey, known as the Current Employment Statistics (CES) survey, is administered to a monthly sample of nearly 400,000 business establishments nationwide. Employment is the total number of persons on establishment payrolls employed full or part time who received pay for any part of the pay period which includes the 12th day of the month. Temporary and intermittent employees are included, as are any workers who are on paid sick leave, on paid holiday, or who work during only part of the specified pay period. Data exclude proprietors, self-employed, unpaid family or volunteer workers, farm workers, and domestic workers.(65)
Data from the CES survey are used for only limited analysis in this research because, although they have the advantage of a long time series of data, they lack occupational and demographic detail. More specifically, the CES data are derived from establishment-level surveys and are based on aggregate earnings and hours for workers at each establishment. The only breakout by occupation is for production/non-production workers and the only demographic breakout is for males and females. Consequently, there is no information about workers at risk of welfare recipiency. Also, these series classify workers by establishment--so the definition of a temporary worker, which is based on an industry definition, is conceptually different from that in the CPS. Here, employees themselves may be temporarily assigned to outside customers, but any individual employee may be long term and hold many temporary assignments.
[Go to Contents]
The goal of the SIPP analysis is an improved understanding of how temporary work affects subsequent labor market outcomes. Broadly put, we want to estimate how outcomes for persons who begin temporary agency jobs would differ if they had not taken such jobs. The basic approach is to compare the subsequent outcomes for those employed in temporary work with persons who have similar demographic and human capital characteristics, but who did not work for temporary agencies.
To undertake this analysis, we need to define employment in temporary work and identify plausible comparison groups to serve as counterfactuals. Our sample of temporary agency workers includes all instances in which persons begin work for an agency (as either their primary or secondary job) as measured by the SIC code. The decision to focus on the start of spells of temporary work simplifies the modeling of comparison groups and fits naturally with our interest in the effect of decisions to take temporary work rather than other options.
We measure the effect of temporary work relative to both regular employment and nonemployment.(66) To operationalize this, we compare our sample of temporary agency workers to two comparison groups: one matched sample of persons employed in nontemporary work and another of persons not employed. By using two comparison groups, we hope to learn the subsequent effects of obtaining a temporary agency job as compared with being employed at a "regular" job and with not being employed.
The samples are matched based on propensity scores. Roughly put, we estimate a regression model that describes the probability of starting a job with a temporary agency. The predicted probability from such a model is known as a propensity score. We follow recent research (Dehejia & Wahba (1998), Berk and Newton (1985), and Rosenbaum & Rubin (1984)) in choosing comparison group members who match members of our sample of temporary agency workers in their likelihood of becoming a temporary worker, as measured by their propensity score. An alternative would be to match on many characteristics of the individuals (e.g., those included in the regression model). Rosenbaum and Rubin (1984), however, argue that matching on the propensity score, which is a single variable, is nearly as effective as matching on all of the many variables used in the regression model to predict propensity score.
Using the matched comparison groups, we estimate the effect of entering temporary work on several outcomes measured a year later. The outcome measures include employment status, wages, hours worked, health insurance coverage, and receipt of public assistance. These subsequent outcomes are then compared to those for each of the comparison groups, with the differences in outcomes interpreted as the effects of entering temporary work relative to the counterfactual; that is, working at a nontemporary job or not working.
When considering this type of approach, a natural question arises: Why bother matching the temporary workers and nontemporary workers? Why not simply estimate a regression model that controls for the variables used in matching?
There are several answers to this question. First, use of the matched comparison group brings us (incrementally) closer to a random assignment design, by trying to limit the comparison group to those who match actual temporary workers in their likelihood of taking a temporary job. Second, including persons in a regression analysis with characteristics that indicate that they are very unlikely to be temporary workers adds no additional information to our estimate of the effect of temporary work. Finally, a regression model typically assumes that the relationship between the independent variables and the dependent variable is structurally similar for all members of the sample. Thus, inclusion of many regular workers and non-workers who are dissimilar to temporary workers in the regression could produce spurious results if the relationship between their background characteristics and subsequent outcomes differs from that of those who are similar to temporary workers. Including people dissimilar from the temporary workers in the regression thus may decrease the ability of the regression to accurately estimate how the choice of temporary work affects future employment, for those people for whom this is a reasonable choice.
A good summary of this argument is provided by Dehejia and Wahba (1998) who make the following comments about the methods under consideration:
"[Propensity score methods] reduce the task of controlling for differences in pre-intervention variables between the treatment and the non-experimental comparison groups to controlling for differences in the estimated propensity score (the probability of assignment to treatment, conditional on covariates). It is difficult to control for differences in pre-intervention variables when they are numerous and when the treatment and comparison groups are dissimilar, whereas controlling for the estimated propensity score, a single variable on the unit interval, is a straightforward task. We apply several methods, such as stratification on the propensity score and matching on the propensity score, and show that they result in accurate estimates of the treatment impact." (p.1)
[Go to Contents]
It is worth noting one caveat to this approach. Any conclusions will be based on comparing temporary workers with those most similar to them in their human capital and demographic characteristics. This will be interpreted as an estimate of the impact of temporary work for those who worked for a temporary agency. It can also be reasonably taken as an estimate of the effect of temporary work for those with human capital and demographic characteristics quite similar to those who worked for an agency. However, our estimates cannot provide a measure of the likely effect of temporary work for persons with characteristics quite different from those of the temporary workers or for a broader group, such as all persons on welfare.(67) The conclusions drawn from this analysis, although narrower in scope, are expected to be more reliable and in keeping with the primary research question.
In this section, we first discuss the definitions of the temporary worker groups (referred to as treatment groups) and the nontemporary worker comparison groups (referred to as comparison groups). We then discuss the details of the propensity score regression analysis used to construct the comparison groups. The factors used in the propensity score regression analysis are those thought to affect both labor market outcomes and decisions to work for temporary agencies (e.g., demographic characteristics, work history, family structure). Using the constructed comparison groups, we estimate the effects of temporary employment on outcome measures one year after the start of a spell of temporary employment.
To obtain a sample of persons beginning temporary work, we select all workers in temporary work (SIC 736) from each month who were not in temporary work in the previous month. The sample is limited to workers between ages 18 and 45. Only those temporary workers whose employment begins at least 12 months before the last month of a panel are included in the analysis, as this allows us to observe outcomes one year later.(68) We include all spells of temporary employment, including multiple spells from the same individual, in our analysis and adjust our model standard errors for correlations among the observations.
This group of temporary workers is further divided into two groups. Prior to entering a temporary job, a person is either employed in nontemporary work or not employed. Since we believe that these two groups may be entering temporary jobs for different reasons, we divide the temporary worker groups into two groups. Treatment Group 1 includes temporary workers who were working in nontemporary employment in the month prior to taking a temporary job. Treatment Group 2 includes temporary workers who were not working in the month prior to taking a temporary job (either unemployed or out of the labor force).
The comparison group contains data for all persons who are not observed in temporary work in any wave of the SIPP.(69) As with the treatment groups, this broad comparison group is divided into two groups. Comparison Group 1 includes persons who were working in the month prior. Comparison Group 2 includes persons who were not working in the month prior (either unemployed or out of the labor force). Thus, based on work status in the prior month, we have two comparison groups.
We further divide Comparison Groups 1 and 2 into two groups based on employment status in the current month, which allows us to estimate the effect of temporary work as compared with both nontemporary employment and nonemployment. Comparison Group 1 is divided into Comparison Group 1A--those employed in the current month--and Comparison Group 1B--those not employed in the current month. Likewise, Comparison Group 2 is divided into Comparison Group 2A--those employed in the current month--and Comparison Group 2B--those not employed in the current month.
In sum, we have the following six treatment and comparison groups:
| Prior Month | Current Month | |
|---|---|---|
| Treatment Group1 | Employed in Nontemporary Work | Employed in Temporary Work |
| Treatment Group 2 | Not Employed | |
| Comparison Group 1A | Employed in Nontemporary Work | Employed in Nontemporary Work |
| Comparison Group 1B | Not Employed | |
| Comparison Group 2A | Not Employed | Employed in Nontemporary Work |
| Comparison Group 2B | Not Employed |
After defining our treatment and comparison groups, the next step in our methodology is to construct matched comparison groups. That is, we select persons from the comparison group who most closely resemble members of the treatment group on a number of key factors (e.g., demographic characteristics, work and welfare history, family structure). We also control for the timing of the survey interviews, so that the labor market conditions faced by temporary agency workers and the comparison groups will be roughly similar. Samples are matched separately for those who start temporary work following employment and nonemployment, since the relationships in the model are likely to vary with work status.(70)
The basic approach is to use a non-linear regression model to describe who becomes a temporary worker, and then use the predicted probabilities of temporary work from that model as the basis for matching samples. Separate models of the probability of starting a temporary agency job are estimated for those with and without employment in the previous month, allowing the factors affecting the probability to differ for these groups. A multinomial logit model is used for the estimation, to allow for joint estimation of temporary work as compared with the two alternatives: employment and nonemployment.
We estimate two multinomial logit models. The first multinomial logit compares temporary workers who were employed in nontemporary work in the prior month (Treatment Group 1) to nontemporary workers (Comparison Group 1A) and non-workers (Comparison Group1B) who were employed in nontemporary work in the prior month. The second multinomial logit compares temporary workers who were not employed in the prior month (Treatment Group 2) to nontemporary workers (Comparison Group 2A) and non-workers (Comparison Group 2B) not employed in the prior month.
Independent variables for the logit models include:
The specific measures used are somewhat different for those employed and not employed in the month prior to when we measure temporary work. The complete list of variables used is reported in Table B.1.
Human Capital Variables:
Indicators of need/ability to work flexible work schedule:
Other demographic factors:
Indicators of previous employment
Indicators of low-income status:
Measures of wave and panel:
Indicators of missing data
Human Capital Variables:
Indicators of need/ability to work flexible work schedule:
Other demographic factors:
Indicators of low-income status:
Measures of wave and panel:
Indicators of missing data
|
We then use a two-step matching procedure. First, using the first multinomial logit model estimated above, for each person in the sample, we predict a propensity score--the probability of employment by a temporary agency (Treatment Group 1) as compared with being employed in a nontemporary job (Comparison Group 1A) or not being employed (Comparison Group 1B).
To assess whether the propensity score from the model adequately controls for differences between temporary workers and each of the comparison groups, we compare the mean characteristics of temporary workers (Treatment Group 1), employed (Comparison Group 1A), and nonemployed (Comparison Group 1B) persons with comparable probabilities of temporary work. To do this, we sort the temporary agency cases (Treatment Group 1) by their predicted probability of being a temporary agency worker and find the probabilities associated with each quintile of the distribution. For example, let p80 be the probability associated with the 80th percentile and pmax be the maximum probability for temporary agency cases.
Second, we then compare the mean characteristics of temporary workers (Treatment Group 1) with probabilities in each quintile to those employed/not temporary (Comparison Group 1A) and nonemployed (Comparison Group 1B) persons with probabilities in the same ranges. For instance, we compare the means of variables used in the logit model for those Treatment Group 1, Comparison Group 1A, and Comparison Group 1B cases with probabilities between p80 and pmax. If the model is appropriate for building matched comparison groups, the mean characteristics of these three groups' cases should be similar for cases with probabilities within each chosen range. If, as occurs in our analysis, some characteristics are not similar, we re-estimate the regression model, including higher order functions of the variables that are not similar across the groupings.
After attempting to make the characteristics of the temporary agency workers and the two comparison groups similar within each range of predicted probabilities (e.g., between p80 and pmax), we use the predicted probabilities to create a matched sample. The goal is to choose cases from the Comparison Groups 1A and 1B with the same distribution of propensities as those who start temporary work. The propensity score literature suggests several approaches. The easiest approach is to reweight the data for the comparison group so that a weighted one-fifth of the comparison group members have propensity scores between the cutoffs for each quintile of scores for the temporary agency workers. That is, we weight so that one-fifth of the cases have propensity scores between pmin and p20; a fifth between p20 and p40; etc….(71)
One remaining issue is how to treat data from multiple months for a given case. Multiple observations for the same case are likely correlated and thus need to be accounted for in calculating the standard errors. Among the temporary worker cases, approximately 15 percent have multiple spells. However, because we have relatively few observations of temporary work, we plan to include all of them in our analysis and adjust the standard errors for correlations among the observations.(72)
The comparison groups allow more flexibility as to whether to include multiple observations from a case. Each comparison group must represent all months of the data for which our temporary agency workers are included to avoid misattributing the effects of different labor market conditions to temporary work. However, we expect the data for the comparison group persons to be highly correlated over time and as a consequence, little gain from including multiple observations for the same person in the analysis. Furthermore, because we are aiming to obtain comparison groups roughly similar in size to our sample of temporary workers, we do not anticipate needing multiple observations per case.
Our solution is to randomly include one month of data for each person in the comparison groups. Each observation is assigned to comparison groups according to its employment status in the sampled and previous month. By randomly choosing the selected months, we maintain the representativeness of our sample while ensuring that individuals do not show up more than once.
[Go to Contents]
A commonly accepted method of evaluating the quality of the match is to examine whether or not the comparison and treatment groups differ in their observable characteristics.(73) We therefore perform a series of t-tests that compare the characteristics of the two treatment groups to the characteristics of each of their potential comparison groups. Table B.2 reports the t-statistics derived from comparing the mean of each characteristic of the treatment group with that of the comparison groups for both the full and the low-income samples.
| Comparison Group 2b: Not Employed to Not Employed | Comparison Group 1b: Employed to Not Employed | Comparison Group 2a: Not Employed to Employed | Comparison Group 1a: Employed to Employed | |||||
|---|---|---|---|---|---|---|---|---|
| Full Sample | At-Risk | Full Sample | At-Risk | Full Sample | At-Risk | Full Sample | At-Risk | |
| Demographic Characteristics | ||||||||
| Age | 0.18 | -0.41 | 2.67* | 0.72 | 2.59* | 1.23 | -0.48 | -0.34 |
| White | 2.01* | 1.68 | 1.59 | 1.44 | -0.93 | -0.68 | -0.11 | -0.01 |
| Edlv11 | 0.98 | 1.05 | 5.60* | 3.07* | 1.9 | 0.38 | 0.38 | 0.86 |
| High school | -0.15 | 0.06 | -0.48 | -1.09 | 0.21 | 1.64 | -0.29 | -0.85 |
| College | 0.74 | 0.86 | 4.10* | 3.06* | 1.61 | -0.11 | 0.26 | 0.78 |
| Job training | -0.18 | -0.42 | 2.50* | -0.01 | 1.31 | 1.21 | -0.06 | -0.36 |
| Household Composition | ||||||||
| Married | 0.15 | 0.16 | 0.9 | -0.42 | 0.53 | -0.27 | -0.03 | -0.03 |
| Married and female | -0.26 | -0.07 | 0.57 | -0.04 | 1.89 | 0.14 | 0.54 | 0.11 |
| Change in marital status | -0.86 | -1.18 | 1.49 | 0.28 | -0.31 | -1 | 0.49 | 0.49 |
| Number of children | -1.51 | -1.16 | -2.67* | -1.55 | -0.8 | -0.15 | -0.14 | 0.21 |
| Decrease in number of children | -0.64 | 0.07 | -1.37 | -1.2 | -1.23 | -0.78 | -0.85 | -0.81 |
| Child under one | 0.12 | 0.13 | -1.22 | -0.19 | 0.48 | 0.84 | -0.06 | 0.11 |
| Child under three | -0.66 | -0.42 | -1.53 | -0.12 | 0.44 | 1.05 | -0.35 | 0.25 |
| Child under five | -1.13 | -0.77 | -1.83 | -0.35 | 0.32 | 1.02 | -0.6 | 0.1 |
| Number of adults | 0.65 | -0.64 | -1.52 | -0.92 | -1.69 | -1.85 | -0.42 | -0.15 |
| Increase in number of adults | -0.12 | -0.43 | -3.55* | -1.93 | 1.09 | 0.33 | -0.07 | -0.32 |
| Decrease in number of adults | 0.49 | 0.41 | -1.01 | -0.5 | -4.54* | -4.27* | 0.12 | 0.36 |
| Poverty History | ||||||||
| 100 to 200% of poverty | 0.01 | 0.42 | 0.75 | 3.67* | -1.05 | -0.24 | 0.39 | -0.35 |
| 200% of poverty | 0.79 | 2.47* | 1.64 | -0.77 | ||||
| Work History | ||||||||
| Short term work history | 2.66* | 2.50* | 2.87* | 1.42 | -2.22* | -3.01* | 0.15 | 1.2 |
| Percent of last 10years working | -2.12* | 1.79 | 5.46* | 2.85* | -0.7 | -1.06 | -0.21 | -0.6 |
| Percent of time in welfare | -1.13 | -1.03 | 1.34 | 1.75 | ||||
| Duration unemployment | -3.42* | -2.8 | 1.57 | 1.56 | ||||
| Duration of current job | 1.57 | 1.05 | -2.94* | -1.35 | ||||
| Duration between jobs | -2.48* | -1.08 | -0.34 | -0.41 | ||||
| One job | -2.16* | -0.5 | -0.2 | 0.39 | ||||
| Previous wage | 3.22* | 1.12 | -1.4 | -0.5 | ||||
| Low wage occupation | -2.41* | -1.44 | -0.54 | -0.35 | ||||
| Low wage industry | -2.48* | -0.01 | -0.51 | -0.07 | ||||
| Sample Size | 19,613 | 9,867 | 1,620 | 600 | 1,769 | 1005 | 49,449 | 9,820 |
| Number of Temp Workers | 738 | 425 | 648 | 234 | 738 | 425 | 648 | 234 |
| Source: SIPP 1990-1993 panels,
calculations by the Urban Institute. Note: At risk defined as below 200% of family poverty level in month prior to reference month. The comparison group mean is the average of the mean within each of the five quintiles. * Significance of the t-statistics at the 0.05 level. |
||||||||
An analysis of this table reveals that the matching procedure generally worked well in grouping like individuals based on demographic characteristics. There is little significant difference between either set of treatment and comparison groups on the basis of age, sex, race, and education. There is also little difference between the two groups in terms of household structure--marital status, number of children--or changes in the household structure. An exception is in matching temporary workers who were previously employed to those who moved to non-work. For that comparison, many demographic characteristics show significant differences between the comparison and treatment groups.
The only set of characteristics in which the matching procedure consistently performed poorly was on the work history variables: particularly the measures of long- and short-term work history and unemployment duration. This suggests that the models that we use fail to capture the full process by which individuals select into each group, and hence that our estimates are likely to be biased by the degree to which this failure occurs. This is not surprising; it would be difficult to argue that individuals take temporary jobs without the existence of work history factors that affect that choice. The construction of more detailed work histories might well be a solution to controlling for the differences we observe, but this is not possible with the current SIPP dataset.(74) These results, however, do reinforce our earlier suspicion that datasets that are unable to control for such work history measures (such as the CPS) would not be appropriate for use in such an analysis.
61. Bureau of Labor Statistics. "CPS Overview." U.S. DOL/BLS. http://www.bls.census.gov/cps/cpsmain.htm.
62. We define at risk as 150 percent of the federal poverty level rather than 200 percent (our definition of at risk for the SIPP analysis, described below). The lower cutoff is used here because the sample size is adequate for analysis and the lower cutoff provides a sample at greater risk of receipt than is the case with the higher cutoff.
63. However, because the CPS is an addressed-based household survey, the actual number of matched cases is lower, due primarily to individuals changing residences from month to month.
64. Income is based on family income from the month prior to either the start of temporary work or a randomly chosen month (for members of the comparison group), multiplied by 12 to get an annual equivalent. This annualized income is then compared to the federal poverty level.
65. Source: http://www.bls.gov/ces/cesprog.htm.
66. This is particularly important for our analysis of persons at risk of welfare, for whom nonemployment may be at least as likely of an alternative to temporary work as nontemporary work.
67. Our inability to describe the impact of temporary work for welfare recipients results from our relatively small number of temporary agency workers. With a large enough sample of temporary workers, we could sub-sample cases to obtain a distribution of propensity scores similar to that of all welfare recipients. Then we would be more comfortable claiming that we had estimated the effect of temporary work on the full sample of nontemporary workers.
68. We include in our logit analysis of temporary work observations that are missing data a year later in an attempt to include as many cases as possible in predicting who is likely to be employed in temporary work. These cases are excluded from the matching procedure and from the analysis of the effects of temporary work because they lack the outcome information from a year later.
69. To make the sample sizes manageable (and to ensure that they reflect the distribution of survey months), we include data for only one month chosen at random per household in the comparison group. The month is chosen from all months that occur at least 12 months before the end of the panel to ensure a sufficient follow-up period.
70. Separate analyses by previous employment status are also expected to make the experiences of those categorized as temporary workers more homogeneous within a grouping.
71. The quintiles procedure was suggested by Rosenbaum and Rubin (1984). A second approach would be to choose for each temporary agency person, the comparison group person with the most similar propensity score. Both approaches will lead to similar distributions of propensity scores for the two comparison groups and the treatment group of temporary workers. For relatively rare transitions, such as those from employment to nonemployment, the reweighting approach is more feasible for our analysis, since it requires fewer observations than a one-to-one match.
72. As of June 2001, standard errors have not yet been adjusted for non-independence of the cases using STATA's cluster option.
73. While it is possible, and even likely, that the groups differ in their unobservable characteristics--and that this may systematically bias the evaluation of the impact--this problem is endemic to evaluation studies (see, e.g., Heckman et al., 2000), and currently unsolved.
74. A variant of this that was suggested by Rosenbaum and Rubin (1984) is to include in the model interaction terms that capture the variation across sample groups in the effects of work history. For example, work history variables may have different effects on the likelihood of temporary work for older women with no children than for young men. To date, experimentation with such interactions--such as separate models for low- and high-income cases or for men and women did not yield an appreciable improvement in the quality of our match.
Main Page of Report | Contents of Report
Home Pages:
Human Services Policy
(HSP)
Assistant Secretary for Planning and Evaluation (ASPE)
U.S. Department of Health and Human Services
(HHS)