Promote evaluation as a tool for program improvement. Pattons (1997) utilization-focused evaluation approach stresses the importance of engaging the primary users of evaluation in every step of the process in order to build support for its use. Stakeholders include not only representatives of funding agencies and program coordinators, but front-line staff who implement the program. Focusing the evaluation on the questions they consider critical will improve both its relevance and implementation. Program staff can play a vital role in identifying questions that will complement standard designs and increase the relevance of evaluation to practice.
|Strategic funding and evaluation design can reduce barriers and create incentives for evaluation.|
Structure evaluation processes so that they are useful to programs and families. A related recommendation is to ensure that evaluation processes provide useful feedback to participants. A major barrier to evaluation among PAS program staff was the belief that families were being asked to spend time completing instruments without receiving any direct benefit in return. The choice of instruments should favor those that can provide useful feedback to program staff and families. This will also help mitigate the sense among program staff that evaluations compete with program activities for scarce resources. As noted in the case study, many adoptive families have difficulty obtaining assessment services and interpretation of clinical data. Although evaluation instruments would not substitute for a comprehensive assessment, feedback on the information collected is likely to be perceived as valuable information by many families.
Earmark funds for evaluation. PAS programs need funding that is specifically designated for evaluation and related activities. Without separate evaluation funds, many program leaders will choose to use all, or nearly all, of their resources for services to families and children. Earmarking funds for evaluation will convey the fact that funding agencies (at both the federal and state levels) view evaluation as essential. Designating funds will also help mitigate concerns by program coordinators that evaluation takes resources away from needed services. Program leaders would then be held accountable for allocating those resources for evaluation.
Fund programs for multiple years. Short funding cycles make it difficult to plan, implement, and evaluate programs in the time allotted, so that managers are unlikely to invest in evaluation staff and activities. Funding programs for four years or longer ensures that they have sufficient time to develop, implement, learn from their evaluations, and incorporate those lessons into ongoing practice. Extended funding also provides opportunities for PAS programs to conduct follow-up activities, producing more substantive evaluations and facilitating assessment of outcomes.
Provide evaluation technical assistance. Accessible, culturally appropriate technical assistance can be used to supplement PAS programs evaluation skills, or to build long-term evaluation capacity within the organization. Depending on the programs needs, technical assistance may emphasize support (where the provider conducts some of the evaluation activities with input from the program) or capacity building (where the provider trains and coaches program staff who carry out the evaluation). Technical assistance should be tailored to the particular needs and interests of the program, and may include evaluation design, development or selection of data collection tools, data management and analysis, and application of findings to program development.