Assessing the Field of Post-Adoption Services: Family Needs, Program Models, and Evaluation Issues. Summary Report. 4.3.1 Fundamental Evaluation Tools

11/01/2002

Develop consistent service classifications. The spectrum of PAS has now been identified in several descriptive efforts (e.g., Smith and Howard, 1997). Yet these have not been carefully described so that different raters would consistently categorize the kind of service received  that is, the difference between classifying a service as therapy, advocacy, or case management may not be readily distinguished. If the field is going to describe post-adoption activities  and eventually link these to case characteristics, consumer satisfaction, and client outcomes  then a more precise nomenclature is needed. In addition, we need basic research to determine the overlap between a variety of services to understand whether these interventions can be separated out (and monitored or tested separately) or combined into clusters of services.

Basic evaluation tools can reduce evaluation burden and increase data comparability.

Identify best practice models with recommended evaluation strategies. The field of PAS has been dramatically strengthened by the high level of innovation evidenced during the past decade. As discussed earlier, the fast pace of program development has to some extent come at the cost of evaluation. With the emerging recognition of promising models, it should now be possible to propose a core set of interventions with associated evaluation strategies. Such an effort could be led by an expert panel, working in consultation with program coordinators in the field. For each intervention component, the panel would characterize intended participants, objectives, program activities, process and outcome measures and recommended instruments. While not all programs would follow the strategies, identification of recommended measures and instruments could considerably reduce evaluation design costs and facilitate cross-site comparisons.

Develop a model data set. Program data are rarely used to evaluate PAS, although most programs collect information on family characteristics and services provided. A model data set, offered to PAS providers in a basic database format, would facilitate consistent data collection across programs and jurisdictions, and speed the development of a broad understanding of who uses PAS and how. A contractor and Technical Work Group should determine whether this basic data collection format would also include measures of services provided, child or family functioning, or satisfaction with services.

Improve data on adoption subsidies. Improved administrative data about subsidies could provide a variety of insights that would help shape the future of post-adoption services and supports. Given such data elements as subsidy amounts, their basis, and reasons for changes; duration of subsidy; basis for subsidy amounts at the time of adoption and later; prior foster care payments to the family; linkages to vendor payment files, and reasons for subsidy termination, it would be possible to determine the duration of subsidies; the total amount of a childs subsidy; and the reasons that subsidies stop, start, or change. In addition, subsidy data should be stored in ways that ensure confidentiality but allow for retrieval for purposes of managing the program, with linkages to the childs foster care record and detailed information on vendor payment. Information should be stored in a format that maintains historical data to support longitudinal analysis.

Develop programmed child and family assessments. Use of child and family assessments that have been programmed into computers, which can be linked to a computerized case management system, could benefit both case workers and evaluators. Such a system could notify the case manager when a follow-up assessment is needed, allowing assessments during home visits (using a laptop). Data could be transmitted into the database system at the office, where it would be aggregated for evaluation purposes.

Conduct rigorous evaluations. Enhanced information resources should expand the possibilities for a multisite experimental design. Rigorous evaluation would start the long process of determining whether PAS are effectively helping families. Several possible approaches should be considered:

  • applying well-tested family-based interventions (e.g., multisystemic therapy) that have shown promise with other difficult populations;
  • finding clinical interventions with enough similarities to group them into a set of smaller intervention studies; and/or
  • locating some larger jurisdictions that can support a single experimental study that may be of interest.

View full report

Preview
Download

"report.pdf" (pdf, 727.57Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®