Approaches to Evaluating Welfare Reform: Lessons from Five State Demonstrations. C. SELECTING SITES

10/01/1996

The selection of sites is one aspect of sample design that is rarely addressed formally. Only one of the states reviewed selected sites for its welfare reform waiver evaluations even partially through a formal sampling process. None of the states analyzed the precision of its sample estimates as estimates of the state population as a whole and, thus, none attempted to take into account the increased variance due to clustering of its samples in selected sites rather than in the entire state. The federal government has not required states to do this, mainly because of the considerable political and administrative realities that limit site selection. In addition, the state may have other goals for the demonstration (such as assessing whether a program will work under the most favorable conditions). However, the lack of representativeness (also referred to as external validity) can limit the usefulness of the information from a demonstration. Here, we discuss different possible approaches for selecting sites.