Assessing the Field of Post-Adoption Services: Family Needs, Program Models, and Evaluation Issues. Evaluation Issues. 4.1 Fundamental Evaluation Tools

11/01/2002

Develop consistent service classifications. The spectrum of PAS has now been identified in several descriptive efforts (e.g., Smith and Howard, 1997). Yet these have not been carefully described so that different raters would consistently categorize the kind of service received chap151; that is, the difference between classifying a service as therapy, advocacy, and case management may not be readily distinguished. If the field is going to describe post-adoption activities chap151; and eventually link these to case characteristics, consumer satisfaction, and client outcomes chap151; then a more precise nomenclature is needed. In addition, we need basic research to determine the overlap between a variety of services to understand whether these interventions can be separated out (and monitored or tested separately) or combined into clusters of services.

Identify "best practice" models. The field of PAS has been strengthened dramatically by the high level of innovation evidenced during the past decade. As discussed earlier, the fast pace of program development has come, to some extent, at the cost of evaluation. With the emerging recognition of promising models, it should now be possible to propose a core set of interventions with associated evaluation strategies. Such an effort could be led by an expert panel, working in consultation with program coordinators in the field. For each intervention component, the panel would characterize the following:

  • The intended participants
  • Short- and (depending on the intensity of the intervention) long-term objectives
  • Typical inputs and processes
  • Recommended process measures (describing the characteristics of persons served and types of services delivered)
  • Standardized measures for outcomes
  • Recommended instruments, with information on mode of administration, age and literacy requirements, administration time, availability and cost, and scoring

Some PAS programs might choose to adapt the models, or not use them at all. For those that do choose to use them, the identification of recommended measures and instruments could considerably reduce evaluation design costs, thus addressing a major barrier to evaluation. In addition, the use of common measures across programs would facilitate cross-site comparisons, so that the influence of variations in population served and delivery approaches could be assessed and a body of knowledge built.

Develop a basic data set. We found that program data are not consistently collected in PAS programs. This is consistent with the findings of other investigators looking at family support programs. Yet data about the family characteristics and services provided are being collected for many PAS. A significant evaluation opportunity exists in the development of a minimum data set that would clarify the characteristics of families that receive PAS. Such data would help to capture basic descriptive information about the delivery of PAS, including the type of adoption, the time since the child entered the family, whether or not adoption subsidies are provided and the level of those subsidies, and the family situation. This data set would be offered to PAS providers in a basic database format. Consistent data collection across agencies, counties, and states would be an important asset to the development of a broad understanding of which clients are served and for how long.

A contractor and Technical Work Group should determine whether this basic data collection format would include any measures of (1) services provided, (2) child or family functioning, or (3) family or child satisfaction with services. Although each of these measures is ultimately necessary, the optimal timing for the addition of such modules is not a certainty.

Improve data on adoption subsidies. Better administrative data about subsidies could provide a variety of insights that would help shape the future of post-adoption services and supports. Subsidy data compiled by states could provide information about subsidy amounts, their basis, and reasons for changes; duration of subsidy; basis for subsidy amounts at the time of adoption and later; prior foster care payments to the family; linkages to vendor payment files; and reasons for subsidy termination. From these reconfigured data we would be able to determine the duration of subsidies; the total amount of a child's subsidy; and the reasons that subsidies stop, start, increase, and decrease. With federal support, prototype information system modules might be developed to link with Statewide Automated Child Welfare Information System (SACWIS) data.

In addition, subsidy data should be stored in ways that ensure confidentiality but allow for retrieval for purposes of managing the program. Data storage criteria would include linkages to the child's foster care record so relationships can be understood between a child's foster care histories and post-adoption subsidy and vendor payment use. Vendor payment information should be uniquely identified by organization or individual service providers. Information should be stored in a longitudinal format with each entry having a unique storage space and date. That is, new data should not replace old data chap151; the old data should be maintained in order to support longitudinal analysis.

Develop programmed child and family assessments. Child and family assessments that have been programmed into computers, and linked to a computerized case management system, could benefit both caseworkers and evaluators. Although we are not aware of instruments distributed in programmed versions, they have been incorporated into computer-assisted survey instruments and could be integrated into case management systems. The system could notify the case manager when a follow-up assessment is needed on a periodic basis or at the time the case is closed. Greater use of computerized case management systems would expand the opportunity to increase the number of outcome evaluations by using these assessments. The advantages of a programmed assessment for caseworkers are that the system could calculate the scores instantly, and caseworkers could conduct the assessments during home visits using a laptop. Data could then be transmitted to a web-based system or downloaded into a desktop system. The advantage for evaluators is that a linked system that includes assessment data allows for analysis of aggregate scores for evaluation purposes.

Conduct rigorous evaluations. Assessment of the possibilities for a multisite experimental design should be coincident with the development of an information infrastructure. Rigorous evaluation would start the long process of determining whether PAS are effectively helping families. Several possible approaches should be considered:

  • applying well-tested family-based interventions (e.g., multisystemic therapy) that have shown promise with other difficult populations;
  • finding clinical interventions with enough similarities to group them into a set of smaller intervention studies; and/or
  • locating some larger jurisdictions that can support a single experimental study that may be of interest.

View full report

Preview
Download

"report.pdf" (pdf, 399.11Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®