Outcome-based performance measures are part of a broader system of program information. As such, they should not be viewed as the way to answer all of our questions about the effects of a program. Neither should they be expected to substitute for other kinds of studies that provide different kinds of information.
For example, policymakers and researchers are greatly interested in whether individuals leaving welfare are succeeding in retaining employment. This question can best be answered with long-term data, covering periods of a year or more. However, such a time frame may be inappropriate for a performance measurement system, which must provide timely feedback to program operators. Similarly, cost constraints may prohibit the inclusion of performance measures based on data that can be collected only through detailed surveys. Therefore, it is usually necessary to supplement the data reported through a performance measurement system with more intensive studies to address questions that are outside the range of such a system. However, such surveys are generally conducted only intermittently and in limited geographic areas. Performance measurement, on the other hand, provides ongoing information about programs which cannot be obtained from such surveys.
In the welfare-to-work arena, this report has shown that it is difficult to identify performance measures that are reliable measures of the "value-added" benefits of a program - that is, measures that reflect outcomes beyond those that would have occurred without the program in place. This difficulty is compounded for a program such as TANF, where state policies and strategies vary widely, as do state capacities, economic conditions and caseload characteristics. To truly understand program effectiveness, it is important to supplement performance measures with other evaluation activities. Together these efforts will provide a more thorough and accurate assessment of program performance.
Used properly, performance measurement systems and rigorous evaluations complement each other. Performance measures can be a valuable tool for generating hypotheses about the relationships between interventions and outcomes. These hypotheses can then be tested through in-depth evaluations of program effectiveness in a limited number of research sites. Similarly, the findings from research and evaluation can be used to refine performance measurement systems and improve their value for monitoring and motivating performance.