Assessing the Field of Post-Adoption Services: Family Needs, Program Models, and Evaluation Issues. Evaluation Issues

Submitted by:
Deborah Gibbs and Kristin Siebenaler
RTI International

Richard P. Barth
University of North Carolina School of Social Work
Contract No. 100-99-0006

Submitted to:
Laura Feig Radel
U.S. Department of Health and Human Services
Office of the Assistant Secretary for Planning and Evaluation

Funded by:
U.S. Department of Health and Human Services,
Administration for Children and Families
Administration on Children, Youth and Families, Childrenchap146;s Bureau

Executive Summary

Although most adoptions have positive outcomes for the children and their families, many families need supportive services during some part of their child's development. In response to these needs, many states have developed post-adoption service (PAS) programs and other supports for adoptive families. The U.S. Department of Health and Human Services contracted with RTI International to examine these rapidly growing and evolving programs. Research questions covered the need for PAS, characteristics of existing programs, and strategies used to assess program effectiveness. RTI, in collaboration with the University of North Carolina at Chapel Hill School of Social Work, conducted a literature review, case studies of five PAS programs, analysis of secondary data, and an assessment of evaluation issues affecting PAS.

Current and Recent Evaluations of PAS Programs

Little research has been completed on PAS programs to date. The study team reviewed five published evaluations and ongoing evaluation activities within the case study sites (Georgia, Massachusetts, Oregon, Texas, and Virginia).

Types of Evaluations Conducted

Needs assessments document the extent of demand for PAS and support service planning, sometimes including information on anticipated needs and severity of need. Most commonly conducted by state agencies via mail surveys of families receiving adoption subsidies, they are therefore unable to determine the exact number of adoptive families needing services. Their findings may not represent the needs of the larger population of adoptive families, especially those who adopted privately or internationally.

Data on the characteristics of children and families served, which many PAS programs collect, can provide useful information for program delivery. The information may include demographics, adoption history, and descriptions of child and family strengths. Data on the amount and types of services delivered are used for both evaluations and program planning. These data may serve as mediating variables in outcome evaluations. Case-study states used a variety of approaches for documenting service delivery, ranging from case records to a web-based case management system.

Client satisfaction assessments collect feedback from adoptive parents to help PAS programs improve service delivery. Case-study states used mail or telephone surveys to assess satisfaction. The validity of client satisfaction data is often limited by poor response rates, but the information is seen as useful by program staff.

Outcome evaluations are not common among PAS programs, due to the difficulty of demonstrating effects, the lack of an identified point at which to measure outcomes, and other challenges. They are essential in documenting program effectiveness and comparing alternative service delivery approaches. There is no consensus yet on which measures or clinical assessment instruments are best suited to the needs of adoptive families and most appropriate for different program models.

Services and Programs Evaluated

Crisis intervention and counseling services were more likely than other interventions to include outcome evaluations, using either subjective ratings by parents or workers, events such as clinical placements, or clinical assessments. Evaluations of information and referral services typically report on characteristics of families and children served, services delivered, and client satisfaction, using telephone or mail questionnaires and event-tracking databases. Given the brief nature of these interactions, evaluations requiring collection of additional information are not common. Evaluations of comprehensive PAS programs tend to be similar to evaluations of crisis intervention programs in terms of data collected and methods used. These evaluations are limited in their ability to identify which services are effective for which families.

Data Collection Methods

Several factors influence the choice of data collection method: the type of evaluation, the PAS offered, the respondent (e.g., program staff or adoptive family), and the case management system (e.g., paper or computerized); as well as the evaluation goals and level of evaluation funding. A web-based system used in one case-study state combined case management functions with compilation of data for evaluation.

EVALUATION CONSTRAINTS AND INCENTIVES

Common Barriers

Among evaluation constraints identified in the case-study interviews, several were common to other service delivery arenas. Funding was the most commonly identified constraint, limiting programs' ability to conduct evaluation activities and to contract with external evaluators. With limited funding, program coordinators frequently put service needs ahead of evaluations. Evaluation expertise is related not only to funding but also to the availability of staff with qualifications in evaluation design, data collection, and analysis. Program staff may need to invest time in orienting evaluators to issues that affect the choice of outcome measures, instrument, and timing of data collection.

PAS staff expressed concerns about the impact of evaluation on families. The staff viewed time spent completing evaluation instruments as encroaching on the therapeutic interactions without benefiting the family, and introducing a more clinical tone to the process. Finally, evaluation was seen to be of limited value to programs. Case-study interviews revealed few instances in which PAS coordinators or providers identified ways in which evaluation findings had been, or were expected to be, useful to them.

Barriers Specific to PAS

Some evaluation barriers were specific to PAS programs. Because adoptive families make up a small part of the population and may not need or use PAS, small population sizes may limit evaluations. Many programs will have difficulty describing subgroup patterns or demonstrating statistically significant differences in service use or outcomes. Compounding these problems is the fact that even effective PAS programs are likely to achieve only modest outcomes. Program goals may be stated in terms of prevention of adoption disruption or dissolution. Because such outcomes are rare chap151; and may be inevitable in some instances even with optimal post-adoption support chap151; impact on their prevention is difficult to demonstrate.

Flexible, family-centered service delivery may further limit evaluation by making it difficult to identify consistent points for data collection, to link outcomes to services received, and to choose respondents for outcome measures. Finally, PAS programs generally have limited administrative and program data. States may be unable to track services provided by contractors and the relationship between services provided by PAS programs and those funded with subsidies. In addition, the lack of consistent service classifications makes it difficult to compare across programs.

The ways in which PAS programs typically are developed and administered also affect their evaluation. The field of PAS is young, and program models continue to evolve, so that evaluation must focus on a moving target. Because PAS programs serve entire families, whose members have diverse needs, they generally comprise a package of coordinated interventions rather than a single, more readily evaluated intervention. Like adoption, PAS programs may be administered at the state, county, or community level, creating problems of adequate sample size to detect outcomes. Finally, there is a notable lack of demand for evaluation of PAS programs among the agencies that fund them.

Evaluation Facilitators

Although the list of barriers is long, some characteristics of PAS programs may actually facilitate evaluation. Those involved with the programs have a genuine curiosity about how programs can be improved. Because of their investment in adoption issues, many adoptive parents are quite open to participating in evaluation. Finally, the field can draw on the experience of other areas of child and family services for evaluation instruments that can be adapted to the needs of adoptive families.

FUTURE DIRECTIONS IN EVALUATING POST-ADOPTION SERVICES

Two general strategies are suggested in this report. First is the development of fundamental evaluation tools that can reduce the start-up costs of evaluation to programs and increase comparability of evaluations across programs. Specific measures include the following:

• Develop consistent service classifications to facilitate program description and analysis.
• Identify a core set of "best practice" models with recommended evaluation strategies.
• Develop a basic program data set to describe child and family characteristics and functioning.
• Improve data on adoption subsidies by capturing significant events and facilitating retrieval and linkages to other data.
• Develop programmed child and family assessments.
• Conduct rigorous evaluations that provide conclusive data on PAS effectiveness.

Second, funding agencies at both the federal and state level can consider strategies to reduce barriers to evaluation. Among the measures they could take are the following:

• Promote evaluation as a tool for program improvement rather than an arbitrary chap151; or nonexistent chap151; requirement.
• Structure evaluation processes so that they are useful to programs and families.
• Earmark funds for evaluation so that is not perceived as impinging on service delivery.
• Fund programs for multiple years to allow adequate time for planning, full implementation, and identification of effects.
• Provide evaluation technical assistance to PAS programs to assist with design, data collection, and analysis.

Introduction

Most adoptions have positive outcomes both for children and their families. Many families need supportive services, however, during some part of their child's development. In response to these needs, most states have developed post-adoption service (PAS) programs designed to prevent adoption disruptions or dissolutions and support child and family well-being.

This report is part of a project that examines these rapidly growing and evolving PAS programs, using a literature review (Barth, Gibbs, and Siebenaler, 2001), case studies of well-regarded programs (Gibbs, Siebenaler, Harris, and Barth, 2002), and secondary data analysis (Barth, Wildfire, Lee, and Gibbs, 2002). To examine PAS evaluation issues, the study team drew on the literature review and data from the case studies to address the following questions:

• What kinds of evaluations are currently being done in PAS programs?
• How do the characteristics and context of PAS programs affect their evaluation?
• What goals and strategies should guide evaluation practice for PAS?

This project was funded by the U.S. Department of Health and Human Services (DHHS), Administration for Children and Families (ACF), under contract to the Office of the Assistant Secretary for Planning and Evaluation (ASPE). Research was conducted by RTI and the School of Social Work, University of North Carolina at Chapel Hill. Staff involved in the PAS programs participating in the case studies, as well as Susan Smith of the Center for Adoption Studies at Illinois State University, gave generously of their time and insights.

Current and Recent Evaluations of PAS Programs

As reported in the literature review, available research on post-adoption services and supports is largely descriptive and based on few projects. Although these projects can be considered front-runners in a relatively unexplored field, they are not without methodological problems. The small sample sizes and nonrandom sampling for several of these projects suggest that the results should not be taken as generalizable. Other concerns related to measures and methods are readily identifiable. For example, the length of the service period was rarely clearly specified, even when it was used as a measure of success for disruption rates (e.g., chap147;percentage of families remaining together at the close of the service periodchap148;). Based on currently available evaluation findings, it is difficult to determine confidently whether post-adoption services and programs have succeeded.

This section describes current and recent PAS evaluations in terms of the types of services evaluated, which aspects of the programs were measured, and data collection methods used. Two groups of evaluations are discussed: those identified by our literature review (Barth, Gibbs, and Siebenaler, 2001) and ongoing evaluations described in the case study report (Gibbs, Siebenaler, Harris, and Barth, 2002). Although the former group is generally more fully developed, the latter represents ongoing and recent efforts by well-regarded programs.

Exhibit 2-1 provides background information on the five evaluations identified in the literature review. The Illinois program was known originally as the Adoption Preservation Project. As the population served changed, the name of the program became the Adoption/Guardianship Preservation Program, which is the name we use throughout.

Evaluation efforts in each of the five case-study states are discussed here, but more detail is available for Georgia, Massachusetts, and Oregon, because those states have produced written evaluation reports on at least some components of their programs. Exhibit 2-2 summarizes background information on all of the case-study states.

Several types of formal evaluations are possible in assessing PAS interventions or programs, including descriptions of children and families served, descriptions of services received, assessments of adoptive family satisfaction with services, and outcome evaluations. In addition, needs assessments can provide states or other funding organizations with information about adoptive families to facilitate the creation of PAS programs or to provide current programs with findings that facilitate a mid-course correction in allocation of services. Based on the program evaluations described above, this section describes in greater detail the types of evaluation most commonly conducted, the specific types of data gathered, and the data collection methods used.

2.1 Types of Evaluations Conducted

2.1.1 Needs Assessment

Needs assessments are designed to describe the kinds of services most needed by families. The pool of adopted children has grown dramatically in recent years, increasing the importance of accurate needs assessments for establishing and adapting PAS programs. These studies can be used to document the need for a PAS program and to support planning of services to be provided. State-sponsored needs assessments in the field of adoption have generally gathered information from surveys of families who adopted from a state's public welfare system and were receiving an adoption subsidy. However, a PAS program could also conduct a needs assessment of its existing client population. Contacting these families is facilitated by the fact that states keep contact information for subsidy payment purposes.

Needs assessments can provide useful information, including the following types of data:

• services needed,
• services received,
• anticipated needs,
• level of importance of the need by type of service, and
• frequency of use of services

One limitation of needs assessments, as they are generally conducted, is that they lack the ability to determine the absolute number of adoptive families needing services, and the needs expressed by respondents may not represent those of the larger population of adoptive families. Accurate estimates of need would require follow-up efforts to maximize response rates and measures of nonresponse bias. States that offer PAS to families who adopted privately or internationally lack a sampling frame from which they can survey these families, whose needs may be substantially different from those of families adopting from the public child welfare system.

Despite these limitations, needs assessments are fairly commonly done. They are generally not published for circulation outside the sponsoring state, and the absence of standardized measures and categories would make synthesis challenging. Two examples of needs assessments are those conducted by Oregon (Fine, 2000) and Illinois (Howard and Smith, 2001), which are summarized in Exhibit 2-3 .

2.1.2 Characteristics of Children and Families Served

It is common practice for case managers providing health and social services, often at intake and assessment, to regularly collect data on the characteristics of children and families served. In the case of PAS programs, evaluators have used the data gathered by case managers and program staff as part of a process evaluation, yielding a range of potentially useful information to guide PAS program direction and service delivery.

Data on families and children served include basic demographic information, history prior to adoption, risk to adoption, family problems and strengths, and family functioning. Clinical instruments are sometimes used to describe child and family functioning, as well as to provide a baseline for outcome evaluations. These instruments are discussed in Section 2.1.5.

Exhibit 2-4 shows the types of data collected to describe characteristics of children and families served in three PAS programs: the Casey Family Services Post-Adoption Program, the Illinois Adoption/Guardianship Preservation Program, and Massachusetts's Adoption Crossroads program. The Casey and Illinois evaluations go beyond the demographic, risk to adoption, and initial contact and assessment data collected in the Massachusetts Adoption Crossroads evaluation to collect more detailed information on the circumstances of the adoption, service history, family functioning, and history of maltreatment.

Data collection strategies vary among these programs. In the Illinois Adoption/Guardianship Preservation Program, the program's evaluators drew from intake forms and closing summaries to collect information on the characteristics of children and families served and the range of problems identified at intake/time of referral to PAS. For the Massachusetts Adoption Crossroads program, evaluators used data taken from entries in the client database made by intake workers and regional response team members to create a client profile. The profile described the magnitude of the family's problem, family's demographic profile, risk to adoption, and a case assessment (most salient or pressing needs). Casey Family Services workers collected data from families at the time of case opening.

Although Massachusetts is the only case-study state to have reported on characteristics of children and families served, other case-study states report having collected this information and using it to prepare summaries in monthly or annual program reports.

2.1.3 Services Delivered

The collection of data on service delivery and usage is also fairly common for PAS programs; these data have been used in several PAS evaluations and are also critically important to planning PAS programs and funding. The programs summarized in Exhibit 2-5 demonstrate three approaches to compiling data on services delivered.

Data on services delivered may serve as mediating variables in outcome evaluations, establishing the effect of specific types of services or a threshold service level necessary for effect. If used as part of an outcome evaluation, services must be documented as they are delivered rather than summarized at case closing. PAS program records are also unlikely to capture services that a family may have received from private providers or other sources not affiliated with the PAS program, which may influence outcomes (National Adoption Information Clearinghouse, 2002).

These examples demonstrate the range of data collected and data collection strategies for this form of evaluation. The client database used in the Massachusetts Adoption Crossroads program evaluation provided service data on a range of factors related to the delivery and receipt of services. The database is a web-based case management system that can be accessed by any of the program's regional providers and can produce either case-level or aggregate data in a variety of formats. In Illinois, evaluators of the Adoption/Guardianship Preservation Program drew on a range of data from case summary forms completed by workers at the close of services (with family consent). In Texas, regional PAS providers used service plans to record information at intake and case closing. Because the Texas PAS program is based on cost-reimbursement from the state, providers also fill out a service authorization form prior to service provision. Although this level of detail on services provided could be extremely useful for evaluation purposes, the data collected in these forms are currently not aggregated or analyzed at the regional or state level.

2.1.4 Client Satisfaction

An assessment of family satisfaction with services received is a common evaluation approach to improving PAS program staffing and programmatic planning. Most client satisfaction surveys involve the adoptive parent. In several of the case-study states, program staff and evaluators used client satisfaction surveys to guide service delivery. Exhibit 2-6 illustrates the type of data collected in client satisfaction surveys for PAS provided by statewide programs in Texas, Massachusetts, and Oregon.

The Oregon survey focuses on information and referral services, providing the opportunity for parents to comment on the services in an open-ended fashion. The Massachusetts survey has tailored questions for each type of service provided by the program and also provides the opportunity for parents to comment by service. The Texas survey also collects information on the services parents received and additional information about the provision of services: how the family heard about the program, how services were coordinated, and whether the location providing the services was safe. It also allows parents to comment in an open-ended manner.

As with needs assessments, the validity of client satisfaction survey data is often limited by poor response rates. Nevertheless, these efforts serve as a useful barometer for program staff, as well as an opportunity to maintain communication with adoptive families regarding their needs and preferences.

2.1.5 Outcomes

Outcome evaluations are the least common of all evaluation types. Among challenges inherent in outcome evaluations are the difficulty of demonstrating effects, particularly for less intensive interventions, and the lack of a clear point at which outcomes are to be measured. Although outcome evaluations are not necessarily appropriate for all interventions, there is likely to be increasing pressure on PAS programs to document their effectiveness, and increasing interest within the field in comparing alternative service delivery approaches.

A variety of measures have been used for outcome assessment, including clinical assessments, events such as adoption disruption or out-of-home placements, goal attainment, or subjective assessments by workers or parents. Exhibit 2-7 summarizes several approaches to outcome assessments.

Child and family clinical assessments offer detailed measures of child and family outcomes for more intensive interventions, with the opportunity for pre/post comparison. There is no consensus yet on which measures are best suited to the needs of adoptive families, as shown in Exhibit 2-8.

The use of child self-reporting assessments, though not reported in any of the evaluation efforts, is a potential evaluation tool. The National Survey of Child and Adolescent Well-Being, sponsored by the Administration for Children, Youth and Families, has developed a module for adopted children (and those in guardianship placements). Although the instrument was not developed for clinical use and is not standardized, it has been used to collect self-reported data on topics including the following:

• Child's involvement in adoption decision-making process
• Child's feelings about being adopted (for children aged 6 and older)
• Contact with biological family members

2.2 Services and Programs Evaluated

The establishment of several new statewide PAS programs in recent years has increased the potential for evaluations of PAS interventions and programs to be statewide, rather than smaller scale or geographically limited projects. This section examines two broad categories of evaluations:

• Specific post-adoption services or interventions, in which clearly defined services are delivered to families with similar needs; or
• PAS programs, which bring together an array of interventions with different objectives and activities to serve a broad range of adoptive families.

2.2.1 Crisis Intervention/Counseling

The evaluations that have assessed crisis intervention and counseling services include two relatively new statewide PAS programs in Georgia (L. Liphart, personal communication, December 10, 2001) and Massachusetts (Hudson et al., 2002) and three more locally based PAS projects (Barth, Gibbs, and Siebenaler, 2001) in Illinois, Oregon, and Washington that were established in the early 1990s. Two of the latter programs are still operating: the Illinois Adoption/Guardianship Preservation Program that assists families who were referred to adoption preservation services, now on a statewide basis, and the Oregon Post-Adoption Family Therapy Project that provides counseling and case management services to families who have adopted from the state and live in the metro-Portland area. Although families may have received other services available under the programs, these evaluations pertain only to specific services and interventions. Exhibit 2-9 summarizes key features of the evaluations.

Crisis intervention and counseling services are more likely than other interventions to include outcome evaluations, using either subjective ratings by parents or workers, events such as out-of-home placement, or clinical assessments. If services are offered on a flexible rather than time-limited schedule, evaluators must grapple with the problem of defining an endpoint at which outcomes are to be measured. The evaluations summarized here had widely varying study populations, ranging from 22 children and families (Medina) to 1,162 children and families (Illinois). Evaluations with very small populations will lack the statistical power needed to demonstrate significant differences in outcomes.

2.2.2 Information and Referral Services

The evaluations of post adoption information and referral services include evaluations by statewide PAS programs established in the late 1990s in Massachusetts (Hudson et al., 2002) and Oregon (ORPARC, 2001). The evaluations looked at adoptive parent satisfaction with information and referral services, including the content provided and manner in which the information or referrals were received. In both states, an event-tracking database recorded calls to the toll-free number that served as the basis for the sample of parents. Exhibit 2-10 summarizes key features of the evaluations.

Because of the low intensity of these services, evaluations are generally limited to descriptions of the families and children served and service utilization. Measures of client satisfaction may be the most appropriate outcome measure. Given the brief nature of the interaction, evaluations that require collecting additional information from the client (other than that collected within the information and referral request) are unlikely to be feasible. The Massachusetts Adoption Crossroads Program, however, has used data from its web-based case management system (described in Section 2.3.2) to document the extent to which problems identified in the information and referral service were eventually resolved.

2.2.3 Evaluations of Comprehensive Post-Adoption Service Programs

Several PAS programs have endeavored to evaluate the program as a whole, in place of or in addition to evaluation of specific PAS. Two more formal evaluations (presented in the literature review) include (1) the Iowa PARTNERS project from the early 1990s that provided sustained adoption counseling, intensive services, and support groups; and (2) the Casey Family Services project, in operation since 1992, that provides case advocacy; family, child, and group counseling; parent education; and support groups to families in New England.

Evaluations of comprehensive PAS programs tend to follow the pattern of evaluations of counseling and crisis intervention by compiling data on child and family characteristics, clinical assessments, risks to adoption, service usage, client satisfaction, and case outcomes. Evaluation methods are also similar and include clinical assessments, case records, and parent feedback forms. The Casey Family Services evaluation also examined case outcomes by assessing parental feelings about their child's progress at case closing using focus groups. Exhibit 2-11 summarizes key features of the evaluation.

Evaluations that attempt to assess the entire program rather than specific components will inevitably be limited in their ability to link services to outcomes. The nearly limitless combinations of amount and type of services, compounded by the diversity of adoptive families, make it difficult to unravel the threads of what services are effective for which families. Program staff may maintain, however, that the comprehensiveness of the service package in itself is a vital element in the effectiveness of each component.

2.3 Data Collection

2.3.1 Data Collection Methods

Evaluations of PAS interventions and PAS programs have employed a variety of data collection methods. Choice of method is influenced by several factors, including the type of evaluation, type of PAS offered, type of respondent (e.g., program staff or adoptive family), type of case management system (e.g., paper or computerized), evaluation goals, and level of evaluation funding. For example, an evaluation of a support group for adoptive parents might use focus groups with members of the group. A linked computerized case management system could allow for analysis of aggregate data on child and family characteristics, service usage, and case outcomes entered by caseworkers. If ensuring high quality service delivery is a primary goal, conducting a client satisfaction survey might be an appropriate method. If funding allows, an evaluator could conduct an outcome evaluation using clinical assessments and even a comparison group. Exhibit 2-12 illustrates the data collection methods used for the evaluations described in Sections 2.1 and 2.2.

2.3.2 Web-Based Case Management or Program Information Systems

The advent of web-based database systems, along with continued increases in computerized case management systems, has greatly increased the prospect of using technology to support program management, case management, and evaluation. Through technological advances, these systems can be designed to facilitate case management tasks and programmatic reporting while providing a wealth of easily accessible data for evaluators. Program staff may already use computers for client intake and initial assessments, service plans, service usage, client characteristics, and client progress. Evaluators, in turn, can aggregate and analyze the data entered across the client population. Even given the potential efficiencies in evaluation that technology brings, program staff, program leadership, and evaluators must work together closely during the design phase so that the system does not create unnecessary hardships for program staff and make them less inclined to use it.

The use of web-based technology has been a unique aspect of Massachusetts's Adoption Crossroads program. The program used a web-based database system, built by a local social services provider, as a case management and evaluation tool (initially, the system was not linked). Eventually it became a web-based system that caseworkers in each region could access through the Internet to record case opening and closing information, service usage, and contacts. The database also informed program operations on an ongoing basis through a reporting function that produced monthly tallies of case statistics, number of calls and reasons for calling, services provided, and third-party billing and fundraising activities. The program evaluators used the database because it allowed for analysis of information and referral services and service utilization data, and it provided a list of clients to survey regarding satisfaction with services received. Additionally, the evaluators felt it supported their analysis of whether the goals identified in the information and referral process were attained chap151; a basic outcome measure.

Evaluation Barriers and Facilitators

Observations on the factors that influence evaluation of PAS programs are drawn largely from case-study interviews. There is no assumption that the states participating in the case studies were typical; in fact, they were selected by experts in the field to represent well-regarded programs. The barriers to evaluation that these programs have encountered are therefore likely to be present equally, if not more so, in other states.

3.1 Common Barriers to Evaluation

When discussing evaluation, program coordinators identified several difficulties common to other service delivery arenas. These included lack of adequate funding for evaluation, lack of access to evaluation expertise inside or outside the program, concerns about the ethics of experimental designs, and general staff resistance to implementing evaluations.

Funding was the barrier mentioned most frequently by state adoption program managers and PAS coordinators and providers. Evaluation requires substantial resources, whether it is contracted out to an external provider or performed in-house by program staff. Given limited funding, program coordinators frequently place higher priority on meeting service needs than on evaluation. Their belief that "we don't have enough funding for evaluation," might more accurately be stated as, "we don't have enough funding to provide the services that we know are really needed and are convinced to be effective and perform an evaluation." Funding agencies contribute to this situation if they require evaluation without specifying the level at which it is to be done or do not allocate adequate resources for both service delivery and evaluation.

Among the case-study states, the one with the most sophisticated evaluation (and the only one with a specific budget line item for evaluation) allocated approximately 5 percent of its budget to evaluation. This is a rather modest chap151; and almost certainly inadequate chap151; amount, particularly in a new program area for which service delivery models and evaluation methods are not well established.

Evaluation expertise is in part related to funding. Contracting with an external evaluator requires a greater commitment of program funds but provides access to a higher level of expertise. Although program coordinators may have some experience and training in evaluation, it is unlikely to be at the same level as someone whose primary role is evaluation. Program staff with the skills needed to serve adoptive families may have limited qualifications in evaluation design, data collection, or analysis. In addition, in-house program staff may be less likely than external evaluators to implement more rigorous designs because of the tension that they might create among skeptical staff.

Even if a PAS program is willing to commit the resources to contracting with an external evaluator, finding an evaluator with adequate understanding of adoption issues may be difficult. The field of PAS is young, with neither a large base of published research nor an extensive network of experienced researchers. Program staff may need to invest considerable time in orienting their evaluators to issues that affect the choice of outcome measures, instruments, and timing of data collection. "We learn from them," said one program coordinator, "and have to make sure they learn from us."

PAS program staff identified several concerns about the impact of evaluation on their interactions with families. Some were concerned that the time required to collect evaluation data not only added to their workload but also impinged on their interactions with families. Time spent completing evaluation instruments was seen as encroaching on their opportunities for therapeutic interaction, without necessarily providing any direct benefit to the family. Some staff indicated concern that this would keep families from coming to, or remaining in, PAS.

Program staff were also concerned that evaluation activities introduced a clinical tone to their interaction that was at odds with their efforts to normalize the adoption experience, especially when the instruments used focused on child and family problems. Adding "strength-based" instruments was a commonly suggested strategy, but these are not well developed and can make interviews unacceptably long.

Case-study interviews revealed few instances in which PAS coordinators or providers identified ways in which evaluation findings had been useful to them or were expected to be. Some limited applications were noted. For example, data were cited to document the volume of services delivered or families' satisfaction with the program. In one program, staff reported having adjusted their training topics and schedule in response to client satisfaction surveys. But there were no reports of evaluation as a source of new and useful input on substantive questions of program design. If evaluation data are not useful to the program's own staff, they are unlikely to be seen as offering much to the larger field. Yet information from evaluations often accrues slowly into a focused message that may not be disclosed until years, or even decades, after the first rigorous evaluations are begun.

3.2 Barriers Specific to PAS

Beyond these common challenges, some of the fundamental characteristics of PAS programs seem to conspire against assessment of the programs and their outcomes. These include the relatively small number of families served, the family-centered and client-driven nature of service delivery, and the long-term and formidable challenges of adoption.

Adoptive families are relatively few in number, estimated at less than 3 percent of the population (Chandra, Aloma, Maza, and Bachrach, 1999). Among adoptive families, an even smaller proportion has service needs that are adoption specific or are not met by existing community resources. Thus, the total number of families served may be fairly small. This is particularly true for PAS programs that are not delivered statewide, statewide programs in less populous states, or programs that are restricted to specific types of adoptions. The modest scale of these programs does not in any way argue against their importance. However, programs with relatively small populations served will be limited in the extent to which they can describe patterns of needs and services for subgroups, such as families with preschool children or those whose children have a history of multiple preadoptive placements. They will also have difficulty demonstrating statistically significant differences in service use or outcomes.

Compounding the problem of small numbers is the fact that outcomes achieved may be relatively modest and diverse. While program goals may focus on the prevention of adoption disruption or dissolution ⁽²⁾ or out-of-home placement, these outcomes occur in no more than 15 percent of the special needs child population (National Adoption Information Clearinghouse, 2002). Further, the pervasive effect of early trauma suggests that these outcomes will occur in some families no matter what supportive services are provided. Improvements in problem behavior and family relationships may also be confounded by developmental changes as children move toward adolescence and its typical disturbances. Programs need to identify mediating measures that can detect meaningful benefits to the family, such as connectedness that is sustained even when the child cannot live in the family, or parental confidence in their ability to respond to the challenges presented by the child. Yet improvements in parental efficacy or parent-child closeness do not have the compelling impact of disruption or dissolution prevention.

PAS programs are typically flexible in their delivery. The length of service and mix of services provided are tailored to the needs of the family, in accordance with the guiding principles articulated by the National Consortium for Post Legal Adoption Services (NCPLAS) (Howard and Smith, 1997). While this client-driven approach enables programs to respond to families' specific needs, it creates several limitations to evaluation, particularly with respect to outcome evaluation. Because families use the service on an "as needed" basis, discontinuing and reentering as their concerns change, it is difficult to identify points at which pre- and post-measures should be administered. Designs that assess change at a case closure may be flawed because many families do not formally exit, and then may often reappear months or years later. Yet designs that rely on fixed-length measurement chap151; e.g., one year after case opening chap151; may capture families in the midst of treatment, so the changes that are measured may not reflect the eventual apex of improvement. An additional concern is that if families are not in touch with the program at that point, follow-up will require considerable effort and data are likely to be incomplete.

The needs and concerns of adoptive families are diverse, and PAS programs typically tailor services to meet family needs. This creates two challenges to evaluation. First, variations in services received make data on satisfaction or other outcomes more difficult to interpret. Second, the outcomes of interest will vary according to family needs. Evaluators must choose between tailoring outcome measures to the specific issues of the family (that is, having greater specificity but smaller groups) and measuring outcomes more broadly (increasing statistical power but with less informative measures).

PAS programs are typically family focused, also in accordance with the NCPLAS principles. Recognizing that all family members are affected by adoption, services are designed to meet the needs of adoptive parents and siblings, as well as adopted children. This strength of PAS programs again creates difficult choices around data collection. Collecting data from all family members increases respondent burden, and it may obscure outcomes by including the experience of family members without significant concerns or substantial involvement in the program. On the other hand, limiting measurement to those family members with the most acute needs may downplay the systemic nature of adoptive family dynamics and raise concerns about stigmatizing "problem children."

Administrative data have been used for sampling and for examining program outcomes in family preservation (Schuerman, Rzepnicki, and Littell, 1994), child welfare (Wulczyn and Zeidman, 1997), and welfare evaluation projects (Barth, Brown, Cuccaro-Alamin, and Needell, 2002), but have been of little help in evaluating PAS programs. Many of these services are contracted out and not covered by the Statewide Automated Child Welfare Information System (SACWIS). Some jurisdictions with relatively comprehensive SACWIS have been late in developing their adoption modules because of concerns about confidentiality. When data about services are gathered, they are often vague, as there is yet no standard nomenclature for characterizing PAS components.

Administrative data systems are further handicapped by the lack of data on adoption subsidies. Subsidies are a key aspect of service provision because many states expect families to request and use subsidies to purchase the particular services that they need. Yet subsidy data are often not linked with case records and are not always captured in a longitudinal format. In some states, changes in subsidy amounts are not saved and dated; in such cases the history of the subsidy and its uses cannot be captured.

3.3 Program Context Issues

Several characteristics of the way in which PAS programs are developed and administered also affect their evaluation. The field of PAS is young, and program models continue to evolve, so that evaluation must focus on a moving target. Because PAS programs serve entire families, whose members have diverse needs, they typically consist of a package of coordinated interventions rather than a single, more readily evaluated intervention. PAS programs, like adoption, may be administered at the state, county, or community level, creating problems of adequate sample size to detect outcomes. Finally, there is a notable lack of demand for evaluation of PAS programs from the agencies that fund them.

The rapid and recent growth of PAS programs means that there have been limited opportunities for program maturation and stability. Among the PAS programs described in this project's literature review (Barth, Gibbs, and Siebenaler, 2001), few were more than 10 years old. The field has developed rapidly in that time, spurred by both the recent acceleration in adoptions and the influx of federal funds. This relatively fast-paced environment has allowed few opportunities for service delivery models to be refined, outcomes to be tracked, or findings to be shared across sites. Evolving program models, while enriched by new information, can wreak havoc on evaluation if program objectives, participants, or interventions are redefined in midcourse. The dearth of evaluations to date means that newer programs have little shared knowledge to build on, forcing their staff to reinvent the evaluation wheel.

In considering evaluation, a distinction must be made between PAS interventions chap151; a clearly defined set of services delivered to families with similar needs chap151; and PAS programs, which bring together an array of interventions with different objectives and activities to serve a broad range of adoptive families. PAS interventions may operate in the context of a broader PAS program (as does Illinois's Adoption/Guardianship Preservation Program) or independently, as in the case of Portland, Oregon's, PAFT program. Examples of PAS programs with multiple interventions include the statewide programs in Georgia, Massachusetts, and Virginia.

The distinction between intervention and program has important ramifications for evaluation. Interventions with specific populations, activities, and outcomes are far more amenable to systematic evaluation. In fact, the majority of published evaluations, such as Illinois's Adoption/Guardianship Preservation Program, PAFT, and PARTNERS, are of specific interventions. PAS programs that do not structure data collection so that families can be grouped by services received will have difficulty identifying outcomes from their work. An example is Casey Family Services, in which data collected on a comprehensive PAS program could not support links between outcomes and services received. The comprehensive nature of PAS programs in many cases is integral to their design, and there is much to be learned, of course, in examining these programs. Yet this does make evaluation more challenging. Descriptive evaluations that monitor which kinds of families use different mixes of services, and how families move among different services over time, offer valuable lessons for ongoing program development. Outcome evaluations of such programs (rather than their component interventions) may not, however, be sufficiently informative to justify the resources they require.

There is, nevertheless, a middle ground between intervention-specific and comprehensive evaluations that the evaluation of PAS programs can pursue. Programs like Multi-Systemic Therapy (MST) and Assertive Community Treatment (ACT) are flexible and open-ended but have enough structure to be evaluable. Indeed, these programs also work with high-risk populations of children and young adults and have shown their efficacy under very demanding experimental conditions.

Like other child welfare services, adoption and post-adoption services are administered at either the state or local level. PAS programs that are developed at the local or regional level will encounter several barriers to evaluation. Among states responding to the Illinois State University (ILSU) survey, approximately one-half reported that PAS were delivered statewide. As noted earlier, programs that serve relatively small numbers of families will have difficulty discerning patterns in service utilization and demonstrating outcomes. Evaluation expertise is also less likely to be available to local programs than at the statewide level. In addition, the start-up cost of evaluation design will be proportionally more burdensome for a small program. Statewide models, in which a single program model is delivered statewide (as in Oregon) or regionally (as in Massachusetts and Texas), are far more amenable to evaluation.

A final barrier to evaluation among PAS programs is the apparent lack of demand from funding agencies. Among the case-study states, evaluation was generally included in the request for proposals. However, there was little indication that program sponsors are setting clear standards for evaluation or actively advocating for stronger evaluations. Given the natural focus on service delivery among program coordinators, it is unlikely that they will go beyond what is required of them in evaluation. Among state adoption managers interviewed, none cited any pressure to document the activities or effectiveness of the PAS programs they fund.

More basically, funders are not requiring that programs be evidence-based, building on rigorously evaluated work with troubled children and families. Although families routinely indicate that PAS must be adoption sensitive, they need not be developed entirely anew for this population. Bringing science to PAS calls for reconsidering the development of rigorous PAS methods, as well as evaluations.

It may be that PAS programs are currently being funded based on the high visibility of foster care adoptions and the common sense appeal of supporting adoptive families. The testimony of one adoptive parent may be far more persuasive in a legislative committee than the best possible evaluation data. However, higher standards on accountability for requested funding are likely at some point in the future, particularly as many states face budget shortages. The field need look no farther than the relatively meteoric rise and fall of intensive family preservation services to understand that family testimony and anecdote do not help a field reach its potential. Strong theory- and evidence-based interventions that are adapted to adoption and rigorously tested are the best strategy for ensuring the future of PAS.

3.4 Evaluation Facilitators

Given this discouragingly long list of barriers to evaluation, it is reasonable to ask whether any characteristics of PAS programs might facilitate evaluation. Based on case-study interviews, several are apparent. The relative youth of the PAS field, and its recent surge in federal funding, has given rise to a ferment of new approaches. First, adoption program managers, together with PAS coordinators and providers, have a genuine curiosity about "what works?" and "is this an improvement on other approaches?" Evaluations that respond to this appetite for program improvement could garner substantial cooperation in spite of the evaluation barriers described above. Second, adoptive parents (a major source of data for such evaluations) have an enormous investment in adoption-related topics and are often ready participants in evaluation. PAS program staff are appropriately protective of parents' time and goodwill, and supportive of their desires to normalize family life. However, experience in this study and others suggests that many adoptive parents are open to participating in efforts that will support and improve PAS programs. Finally, although PAS programs are relatively new, they can draw upon evaluation experiences in other areas of child and family services, and on the existence of psychometrically tested instruments for both children and families. Use of these instruments allows considerable streamlining of evaluation design, as well as the opportunity for comparability across evaluations.

Discussion

A substantial boost in our knowledge of PAS will require many steps and stages. Two general strategies are suggested here: first, the development of fundamental evaluation tools, including service classifications, basic administrative data sets, and procedures for the collection and storage of subsidy data. These steps would both reduce the start-up costs of evaluation and increase comparability of evaluations across programs. Second, funding agencies at both the federal and state level can consider strategies to reduce barriers to evaluation by making them more useful to program staff, ensuring that evaluation activities do not impinge upon service delivery, and providing adequate resources in the form of both funds and expertise.

4.1 Fundamental Evaluation Tools

Develop consistent service classifications. The spectrum of PAS has now been identified in several descriptive efforts (e.g., Smith and Howard, 1997). Yet these have not been carefully described so that different raters would consistently categorize the kind of service received chap151; that is, the difference between classifying a service as therapy, advocacy, and case management may not be readily distinguished. If the field is going to describe post-adoption activities chap151; and eventually link these to case characteristics, consumer satisfaction, and client outcomes chap151; then a more precise nomenclature is needed. In addition, we need basic research to determine the overlap between a variety of services to understand whether these interventions can be separated out (and monitored or tested separately) or combined into clusters of services.

Identify "best practice" models. The field of PAS has been strengthened dramatically by the high level of innovation evidenced during the past decade. As discussed earlier, the fast pace of program development has come, to some extent, at the cost of evaluation. With the emerging recognition of promising models, it should now be possible to propose a core set of interventions with associated evaluation strategies. Such an effort could be led by an expert panel, working in consultation with program coordinators in the field. For each intervention component, the panel would characterize the following:

• The intended participants
• Short- and (depending on the intensity of the intervention) long-term objectives
• Typical inputs and processes
• Recommended process measures (describing the characteristics of persons served and types of services delivered)
• Standardized measures for outcomes
• Recommended instruments, with information on mode of administration, age and literacy requirements, administration time, availability and cost, and scoring

Some PAS programs might choose to adapt the models, or not use them at all. For those that do choose to use them, the identification of recommended measures and instruments could considerably reduce evaluation design costs, thus addressing a major barrier to evaluation. In addition, the use of common measures across programs would facilitate cross-site comparisons, so that the influence of variations in population served and delivery approaches could be assessed and a body of knowledge built.

Develop a basic data set. We found that program data are not consistently collected in PAS programs. This is consistent with the findings of other investigators looking at family support programs. Yet data about the family characteristics and services provided are being collected for many PAS. A significant evaluation opportunity exists in the development of a minimum data set that would clarify the characteristics of families that receive PAS. Such data would help to capture basic descriptive information about the delivery of PAS, including the type of adoption, the time since the child entered the family, whether or not adoption subsidies are provided and the level of those subsidies, and the family situation. This data set would be offered to PAS providers in a basic database format. Consistent data collection across agencies, counties, and states would be an important asset to the development of a broad understanding of which clients are served and for how long.

A contractor and Technical Work Group should determine whether this basic data collection format would include any measures of (1) services provided, (2) child or family functioning, or (3) family or child satisfaction with services. Although each of these measures is ultimately necessary, the optimal timing for the addition of such modules is not a certainty.

Improve data on adoption subsidies. Better administrative data about subsidies could provide a variety of insights that would help shape the future of post-adoption services and supports. Subsidy data compiled by states could provide information about subsidy amounts, their basis, and reasons for changes; duration of subsidy; basis for subsidy amounts at the time of adoption and later; prior foster care payments to the family; linkages to vendor payment files; and reasons for subsidy termination. From these reconfigured data we would be able to determine the duration of subsidies; the total amount of a child's subsidy; and the reasons that subsidies stop, start, increase, and decrease. With federal support, prototype information system modules might be developed to link with Statewide Automated Child Welfare Information System (SACWIS) data.

In addition, subsidy data should be stored in ways that ensure confidentiality but allow for retrieval for purposes of managing the program. Data storage criteria would include linkages to the child's foster care record so relationships can be understood between a child's foster care histories and post-adoption subsidy and vendor payment use. Vendor payment information should be uniquely identified by organization or individual service providers. Information should be stored in a longitudinal format with each entry having a unique storage space and date. That is, new data should not replace old data chap151; the old data should be maintained in order to support longitudinal analysis.

Develop programmed child and family assessments. Child and family assessments that have been programmed into computers, and linked to a computerized case management system, could benefit both caseworkers and evaluators. Although we are not aware of instruments distributed in programmed versions, they have been incorporated into computer-assisted survey instruments and could be integrated into case management systems. The system could notify the case manager when a follow-up assessment is needed on a periodic basis or at the time the case is closed. Greater use of computerized case management systems would expand the opportunity to increase the number of outcome evaluations by using these assessments. The advantages of a programmed assessment for caseworkers are that the system could calculate the scores instantly, and caseworkers could conduct the assessments during home visits using a laptop. Data could then be transmitted to a web-based system or downloaded into a desktop system. The advantage for evaluators is that a linked system that includes assessment data allows for analysis of aggregate scores for evaluation purposes.

Conduct rigorous evaluations. Assessment of the possibilities for a multisite experimental design should be coincident with the development of an information infrastructure. Rigorous evaluation would start the long process of determining whether PAS are effectively helping families. Several possible approaches should be considered:

• applying well-tested family-based interventions (e.g., multisystemic therapy) that have shown promise with other difficult populations;
• finding clinical interventions with enough similarities to group them into a set of smaller intervention studies; and/or
• locating some larger jurisdictions that can support a single experimental study that may be of interest.

4.2 Strategies To Facilitate Evaluation

Promote evaluation as a tool for program improvement. Program staff are more likely to support and use evaluation if they believe that it is likely to inform their practice (Patton, 1997; Gibbs, Napp, Jolly, Westover, and Uhl, 2002). Patton's utilization-focused evaluation approach stresses the importance of engaging the primary users of evaluation in every step of the process. These stakeholders include not only representatives of funding agencies and program coordinators, but frontline staff who implement the program. Focusing the evaluation on the questions they consider critical will improve both its relevance and implementation.

This approach may suggest strategies that elaborate on the more standardized evaluation approaches provided by the expert panel above. Program staff may identify questions on how to tailor interventions to specific types of clients, for example, that can be addressed with qualitative studies. These can complement, rather than conflict with, the use of standardized measures and clinical instruments.

Structure evaluation processes so that they are useful to programs. A related recommendation is to ensure that evaluation processes provide useful feedback to participants. A major barrier to evaluation among PAS program staff was the belief that families were being asked to spend time completing instruments without receiving any direct benefit in return. The choice of instruments should favor those that can provide useful feedback to program staff and families. This will also help mitigate the sense among program staff that evaluations compete with program activities for scarce resources.

The Case Study Report from this project (Gibbs, Siebenaler, Harris, and Barth, 2002) reported that families were frustrated by the difficulty of obtaining assessments for their children and desired detailed discussion of their children's needs and strengths with someone who could interpret clinical data for them. Although evaluation instruments would not substitute for a comprehensive assessment, feedback on the information collected is likely to be perceived as valuable information by many families. Susan Smith of ILSU's Center for Adoption Studies reports that this aspect of the Adoption/Guardianship Preservation Program evaluation has received a favorable response from participating families.

Earmark funds for evaluation. PAS programs need funding that is specifically designated for evaluation and related activities. Without separate evaluation funds, many program leaders will choose to use all, or nearly all, of their resources for services to families and children. Earmarking funds for evaluation will convey the fact that funding agencies (at both the federal and state levels) view evaluation as essential. Designating funds will also help mitigate concerns by program coordinators that evaluation takes resources away from needed services. Program leaders would then be held accountable for allocating those resources for evaluation.

Fund programs for multiple years. Short funding cycles make it difficult to plan, implement, and evaluate programs in the time allotted, so that managers are unlikely to invest in evaluation staff and activities. Funding programs for four years or longer ensures that they have sufficient time to develop, implement, learn from their evaluations, and incorporate those lessons into ongoing practice. Extended funding also provides opportunities for PAS programs to conduct follow-up activities, producing more substantive evaluations and facilitating assessment of outcomes.

Provide evaluation technical assistance. Accessible, culturally appropriate technical assistance can be used to supplement PAS programs' evaluation skills, or to build long-term evaluation capacity within the organization. Depending on the program's needs, technical assistance may emphasize support (where the provider conducts some of the evaluation activities with input from the program) or capacity building (where the provider trains and coaches program staff who carry out the evaluation). Technical assistance should be tailored to the particular needs and interests of the program, and it may include evaluation design, development or selection of data collection tools, data management and analysis, and application of findings to program development.

References

Barth, R. P. (1991). Adoption preservation services. In E. M. Tracy, D. A. Haapala, J. Kinney, and P. J. Pecora (Eds.), Intensive family preservation services: An instructional sourcebook (pp. 237-249). Cleveland, OH: Mandel School of Applied Social Sciences.

Barth, R. P., Brown, E. L., Cuccaro-Alamin, S., and Needell, B. (2002). Administrative data on the well-being of children on and off welfare." In R. Maynard (Ed.), Administrative data in the evaluation of welfare reform (pp. 316-353). Washington, DC: National Academy of Science.

Barth, R. P., Gibbs, D. A., and Siebenaler, K. (2001). Literature review: Assessing the field of post-adoption service: Family needs, program models and evaluation issues. Research Triangle Park, NC: Research Triangle Institute.

Barth, R. P., Wildfire, J., Lee, C. K., and Gibbs, D. A. (In press). Secondary data report: Assessing the field of post-adoption services. Research Triangle Park, NC: RTI International.

Chandra, A., Abma, J., Maza, P., and Bachrach, C. (1999). Adoption, adoption seeking, and relinquishment for adoption in the United States. Vital and Health Statistics Advance Data, Number 306.

Fine, D. N. (2000). Adoptive family needs assessment Final report. Portland, OR: Oregon Post Adoption Resource Center.

Fine, D. N. (2002). Oregon Post Adoption Resource Center Client Satisfaction Survey. Portland, OR: Oregon Post Adoption Resource Center.

Gibbs, D., Barth, R. P., and Lenerz, K. (2000). Casey Family Services post-adoption services program: Results of a baseline evaluation. Unpublished report, available from the authors.

Gibbs, D., Napp, D., Jolly, D., Westover, B., and Uhl. G. (2002). Increasing evaluation capacity within community-based HIV prevention programs. Evaluation and Program Planning, 25, 261-269.

Gibbs, D., Siebenaler, K., Harris, S., and Barth, R. P. (In press). Case study report: Assessing the field of post-adoption services: Family needs, program models and evaluation issues. Research Triangle Park, NC: RTI International.

Groze, V., Young, J., and Corcran-Rumppe, K. (1991). Post adoption resources for training, networking and evaluation services (PARTNERS): Working with special needs adoptive families in stress . Washington, DC: Prepared with Four Oaks, Inc., Cedar Rapids, Iowa, for the Department of Health and Human Services, Adoption Opportunities.

Hartman, A., and Laird, J. (1983). Family-centered social work practice. New York: Free Press.

Howard, J., and Smith, S. (1997). Strengthening adoptive families: A synthesis of post-legal adoption opportunities grants. Normal, IL: Illinois State University.

Howard, J., and Smith, S. (2001). The Illinois Adoption/ Guardianship Preservation Program: The first ten years. Center for Adoption Studies, Illinois State University.

Hudson, C. G., Cedeno-Zamor, P., Rosenthal, M., Springer, C., Hudson, B., Ford, D. A., Kowal, L. W., and Riley, D. 2002. Adoption Crossroads: The Fourth Year Evaluation. Salem, MA.: Salem State College.

National Adoption Information Clearinghouse (2002). Disruption and dissolution statistics summarized at http://www.calib.com/naic/pubs/s_disrup.htm

Oregon Post Adoption Resource Center (ORPARC). 2001. Oregon Post Adoption Resource Center, A Program of Northwest Resource Associates: Semi-Annual Report. Portland, OR: ORPARC.

Patton, M. Q. (1997). Utilization-focused evaluation: The new century text, Edition 3. Thousand Oaks, CA: Sage Publications.

Prew, C. (1990). Therapy with adoptive families: An innovative approach. The Prevention Report, Fall, 8.

Prew, C., Suter, S., and Carrington, J. (1990). Post adoption family therapy: A practice manual. Salem, OR: Children's Services Division.

Schuerman, J. R., Rzepnicki, T. L., and Littell, J. H. (1994). Putting families first: An experiment in family preservation. Hawthorne, NY: Aldine de Gruyter.

Smith, S. L., and Howard, J. A. (1994). The adoption preservation project. Normal, IL: Illinois State University.

Wulczyn, F. H., and Zeidman, D. (with Svirsky, A.) (1997). Home Rebuilders: A family reunification demonstration project. In J. D. Berrick, R. P. Barth, and N. Gilbert (Eds.), Child Welfare Research Review (Vol. II). New York: Columbia University Press, pp. 252-271.

Endnotes

1. * RTI International is a trade name of Research Triangle Institute.

2.  Disruption refers to the breakup of an adoption between the time the child is placed with a family and the time the adoption is finalized. Dissolution refers to the legal abolishment of an adoption.