Performance Improvement 1997. Administration for Children and Families

02/01/1997

MISSION: To promote the economic and social well-being of families, children, individuals, and communities.

Evaluation Program

The Administration for Children and Families (ACF) administers a broad range of entitlement and discretionary programs, including programs designed to move families from welfare to work (Temporary Assistance for Needy Families); child support; children and family services (Head Start, Child Welfare, Family Preservation and Support, youth programs); four block grants; and special programs for targeted populations, such as the developmentally disabled, immigrants, and Native Americans.

The objectives of ACF's evaluations are to furnish information on designing and operating effective programs; to test new service delivery approaches, building on the success of completed demonstrations; to apply evaluation data to policy development, legislative planning, budget decisions, program management, and strategic planning and performance measures development; and to disseminate findings of completed studies and promote application of results by State and local governments.

ACF actively engages with other Federal agencies, State and local policy and program officials, national organizations, foundations, professional groups and practitioners, and consumers to maintain currency on the emerging issues affecting its programs and to identify questions for evaluation studies. Systems changes and how they affect vulnerable populations, particularly the well-being of children, are of primary concern. The movement toward devolving responsibility for health and human services to State and local organizations--in particular, the Personal Responsibility and Work Opportunity Reconciliation Act of 1996--offer both tremendous opportunities and unprecedented challenges in redefining and implementing services to families.

Evaluation study designs are carefully negotiated with the States and other interest groups. Studies are frequently funded as joint ventures with ASPE and other Federal agencies and foundations. Such collaborations enable efforts that are better informed, more representative of varying perspectives, and larger in scale. Proposals are reviewed by multidisciplinary experts. Work groups of various kinds are used throughout the projects to monitor progress and to advise on refinements in designing and presenting the findings.

Summary of Fiscal Year 1996 Evaluations

ACF's major evaluations are linked closely to its two strategic goals:

  • Measurably improve the economic independence and productivity of families by reforming the welfare system and by stimulating the changes in attitude and behavior necessary to achieve results.
  • Increase the number of children, youth, and families who have improved health, development, and well-being and who live in stable communities.

Described below are several major evaluations completed by ACF in FY 1996 that relate to these two strategic goals.

Economic Independence and Productivity of Families

Two studies addressed child support enforcement issues. First, Evaluation of Child Support Guidelines, highlighted in chapter II, evaluated the impact of shifting from voluntary to presumptive guidelines using the Current Population Survey Child Support and Alimony Supplement of 1992. Little impact on child support orders was found when States treated guidelines as presumptive rather than voluntary. Review of State studies of deviations from guidelines revealed that while deviation ranges from 3 percent to 81 percent, most States deviate in 25 percent of cases or less.

Second is an evaluation also highlighted in chapter II, Evaluation of Child Access Demonstration Projects: Report to Congress. This ACF Office of Child Support Enforcement report evaluates different forms of interventions to bring noncustodial parents closer to their children after divorce and separation. The interventions included mediation, parenting training, counseling, enforcement of visitation, and monitoring of visitation. The report found that mediation where both parties attended resulted in parenting plans in 65 to 70 percent of the cases. These parenting plans stimulated more visitation by noncustodial parents and better compliance with child support. A majority of both parents were satisfied with mediation. Other forms of intervention for more longer term and problematic cases did not register impact.

Healthy Development of Children and Families

Two evaluations completed during FY 1996 focused on ACF's Head Start program. First, Survey of Head Start Family Self-Sufficiency Initiatives explored ways programs could work with families to improve employability and literacy and deal with substance abuse. The study found that Head Start programs use formal surveys and assessments to identify a family's need for literacy and employability services but assess the need for substance abuse services more informally. Also, most program directors perceive the greatest need for employability services and the least need for substance abuse services.

Second, Survey of Head Start Family Self-Sufficiency Initiatives: Case Studies in Six Communities found that although most program directors were concerned about substance abuse and its effect on families, assessing and meeting families' service needs was difficult. According to the case studies, program directors believe their Head Start families need literacy services, although the underlying causes of illiteracy vary across sites. The six programs recognized and addressed employability in varying ways. For example, some programs offered information and referral, but only when parents requested assistance.

Several evaluation projects were targeted to programs for family members at risk, National Evaluation of Home-Based Services Programs for Runaway Youth reports on five demonstration programs--Baltimore, Maryland; Nashville, Tennessee; Kauai, Hawaii; San Diego, California; and Tucson, Arizona--targeting at-risk youth in dysfunctional families. In each, short-term interventions were applied to link the families to existing community resources. The projects reported difficulties in dealing with multiproblem families, lack of community resources, lack of support from other community agencies, and staff turnover. Effective service practices included bilingual and culturally sensitive staff, round-the-clock staff availability, and using a cotherapist approach.

Second, Gang Families in a Public Housing Project studied families having more than one gang member in a low-income Mexican-American community in Los Angeles. The project studied relationships among macrostructural and economic forces and household organization, family childrearing practices, sibling and relational influences, socialization of street children, culture and traditions, and levels of acculturation, especially in the colonization or marginalization process. The study identified family patterns and processes leading to gang membership and provided a better understanding of the dynamics of families with more than one gang member, how childrearing practices and street culture are transmitted to children, and how gang habits and values are transmitted among family members.

Length of Service and Cost-Effectiveness in Four Family-Based Placement Prevention Programs used an experimental design to study the impact of length of service on outcomes in family preservation programs in Portland and Pendleton, Oregon, and Baltimore, Maryland. The study found that, overall, participants (a total of 460 families) experienced low out-of-home placement rates, low rates of maltreatment, and significant improvement in family, parent, and child functioning. Six-month periods of in-home family treatment provided to families with older children and significant histories of prior services were the most effective.

Last, Final Evaluation Report for the Case Management Enhancement Project at the East Orange District Office of the New Jersey Division of Youth and Family Services (DYFS) documents a 4-year test of the effectiveness of a personal computer-based system to record child protective services case flow information. The overall impact has been to facilitate communication between caseworkers and supervisors, office staff, and outside agencies. Supervisors can now examine case records directly from their PC's, identify problems, assess risks, consult with caseworkers, and redirect case management in a timely manner; the network fax is used to transmit information directly from the computer. The Deputy Attorney General's office in Essex County is able to transmit affidavits, court documents, and complaints by computer sharply reducing the time needed to edit and finalize documents. East Orange office staff can also notify the prosecutor about child abuse and neglect cases on a timely basis. The project positively affected staff knowledge and skill levels and strengthened their motivation to tap the potential of computer technology. The DYFS client-tracking system has already proven its replication potential, both within the State of New Jersey and beyond. The experiment was expanded to all 39 district offices in New Jersey; Connecticut and Wisconsin have inquired about the East Orange system.

Evaluations in Progress

Described below are some of ACF's evaluations in progress that relate to the strategic goals.

Economic Independence and Productivity of Families

One study is looking at child support enforcement issues. Arkansas Prenatal and Postnatal Paternity Project tests the success of in-hospital and postnatal establishment of paternity. Early findings indicate the following: (1) Paternity is established in about half the cases; (2)38 percent of established paternities are for children in the IV-D caseload (Title IV, Part D of the Social Security Act authorizes Federal matching funds to be used for support obligations by locating nonresident parents, establishing paternity, establishing child support awards, and collecting money.); (3) 30 percent of child support cases receive some financial support; (4) 27 percent of child support paternity cases had closed; (5) 95 percent of paternities were established within 30 days of birth; and (6) 44 percent of parents who acknowledged paternity were cohabiting at the time of birth. Mothers at prenatal clinics were generally Medicaid-eligible and poor. Seventy-five percent of them said they wanted paternity established when their child was born and predicted that 71 percent of fathers would acknowledge their paternity; however, only 32 percent of the fathers actually signed paternity acknowledgments. Twenty-five percent of mothers at prenatal clinics did not want to establish paternity for reasons that included the following: (1) not wanting the father involved; (2) the father already gives money to the mother; (3) the mother does not know the location of the father; (4) the mother fears the father's reaction; (5) the mother does not know the identity of the father; (6) the mother sees no reason to establish paternity; or (7) the mother does not want to lose benefits. Programs to establish interstate paternities at the borders and programs to provide job and social services for unemployed fathers both succeeded. Models for deemphasizing child support and the use of child support staff were unsuccessful.

Two ACF studies under way address parenthood in welfare families. First, Responsible Fatherhood: Theoretical and Empirical Foundations for Policy and Program Development, a joint project with ASPE, focuses on a wide range of fathers, including disadvantaged, never-married, noncustodial fathers; separated or divorced noncustodial fathers; and fathers living with their children. By developing a theoretical underpinning to guide empirical research, program development, and program evaluation, this project intends to help inform policymakers about what is necessary to enable fathers to support and nurture a child.

A second study, Home Visitor Services Demonstration: Home Visiting for Teen Parents Required to Participate in JOBS, is testing the effectiveness of combining the Job Opportunities and Basic Skills Training (JOBS) Program with weekly home visits by paraprofessionals. The first-time teen parent welfare recipients are required to participate in education, training, and employment-related activities through the JOBS program, including home visiting. This controlled experiment is designed to evaluate whether home visiting helps teen parents to increase their participation in JOBS activities; improve parenting; experience fewer repeat pregnancies and births; and increase the use of preventive health care (including immunizations) both for themselves and their children. The evaluation is funded by ACF and the Henry J. Kaiser Family Foundation.

Eleven ACF evaluations projects relate to the Empowerment Zone/Enterprise Community Initiative. The Office of Community Services supports evaluations of 10 projects funded under the Job Opportunities for Low-Income Individuals (JOLI) Program. ACF also provides training and technical assistance to JOLI grantees. The services will help the grantees develop project designs and finalize evaluation plans.

Bridgeport Artisan Center, a project of Action for Bridgeport Community Development and its collaborative network of public and private agencies, is creating jobs and enterprise opportunities for low-income artisans in the inner city and surrounding neighborhoods of Bridgeport, Connecticut, an Enterprise Community.

Through its Harlem Railyards Transportation (HRT) Program, the South Bronx Overall Economic Development Corporation provides well-paying employment and business development opportunities for at least 102 recipients of Aid to Families with Dependent Children (AFDC) and other low-income individuals in the New York City Empowerment Zone. The HRT program, a not-for-profit trucking venture, operates a cost-effective transportation system that transfers loaded rail cars to trucks for delivery to New York City area businesses.

Venture, a highway construction program of the Rural Advancement Fund, trains 100 AFDC recipients for highway construction careers in three North Carolina counties. Venture capitalizes on Federal and State highway construction mandates aimed at employing AFDC recipients and other low-income people, particularly women and minorities.

JOLI Project of the Women's Self-Employment Project (WSEP) grows microenterprises, some incubated under a previous JOLI grant, to provide employment for AFDC recipients in Chicago, Illinois. The project identifies up to 30 employers to receive technical assistance aimed at expanding their businesses and creating 100 new jobs. WSEP increases access to financial services--such as loan capital, savings, and investment vehicles--by providing loans through its Revolving Loan Fund.

JOLI Project is enabling Bethel New Life to expand its 10-year-old Chicago home care service and to create 40 new upwardly mobile jobs. Bethel New Life plans to increase business by 25 percent by broadening its service population and diversifying its payer mix to include private-pay individuals and managed care provider contracts. It also plans to establish a career ladder to advance people from welfare to minimum-wage jobs as homemakers or home health aides and to living-wage jobs as certified nurse assistants, allied health workers, licensed practical nurses, and registered nurses.

Avenues (Avenidas), a project of the Mi Casa Resource Center for Women in Denver, Colorado, trains low-income persons, primarily women, for jobs and apprenticeships in highway construction and maintenance. Recruiters target the unemployed, public housing residents, and homeless persons. Keys to the project's success are a steering committee of collaborators to guide program implementation and facilitate job placements; motivational marketing and outreach to develop interest in nontraditional employment; comprehensive assessment, case management, and linkage to supportive services; a rigorous 6-week training program; and peer support and mentors.

Green Institute's JOLI Project is creating new job opportunities for 100 low-income residents in the Phillips neighborhood of Minneapolis, Minnesota, through the incubation of green businesses. The project focuses on five areas: (1) creating jobs that build on the success of the Green Institute's ReUse Center; (2) an incubator program for businesses that develop new products from industrial waste products; (3) businesses offering products and services to promote energy efficiency and conservation; (4) a program for youth entrepreneurs to develop environmental businesses and services through the Science Magnet Program at South High School; and (5) businesses focused on environmental technology and alternative energy that also provide better paying unskilled and semiskilled jobs.

Through Project RISE, Yakima Valley Opportunities Industrialization Center is expanding its housing development company into the Yakima Valley Rural Enterprise Community by creating 16 permanent, year-round jobs and safe, affordable housing for low- and moderate-income families. The project targets at-risk youth, public assistance recipients, displaced workers, and individuals enrolled in the JOBS program or in a program funded by the Job Training Partnership Administration. In 3 years, 56 homes will be constructed, weatherized, or rehabilitated on 8 acres developed by the project.

JOLI Project of Black Dollars Days Task Force in Seattle, Washington, is creating 71 jobs through three businesses: (1) NW ServiceMaster, a cleaning franchise expansion of the Handyman Connection, which provides JOLI participants opportunity for self-employment; (2) a new home health services business; and (3) a new driving and delivery service cooperative. The project is based on the Task Force's Multifaceted Business Development program, bringing together public-private partnerships, streamlined resources, and effective skills training to help people make the transition from welfare to self-sufficiency.

HOMECARE Co-op is a JOLI project of the San Jose Development Corporation. This self-employment house-cleaning services, business training, and business cooperative for AFDC recipients in Santa Clara County, California, is creating 40 new business enterprises which are, in turn, creating 60 new full-time jobs.

National Study of Protective, Preventive Reunification Services Delivered to Children and Their Families will examine the number and percentages of children and families in the child welfare system receiving protective, preventive reunification, out-of-home care, or aftercare services. The study will also obtain national data on the number, types, and dynamics of the services provided. Researchers will abstract case records on a nationally representative sample of 3,000 children and their families served by public child welfare agencies.

Foster Youth Mentors project is examining factors characteristic of successful relationships between foster youth and older citizen mentors by comparing 250 successful matches with 250 unsuccessful ones. Data is being collected on the characteristics of mentors, foster youth, and the mentoring program itself. Findings to be disseminated to independent living programs throughout the United States are expected to facilitate the use of mentors in older youths' transition from foster care.

Last, Evaluation of the Impact of Homelessness on ACYF Programs will identify service demands on ACYF programs serving homeless families, children, and youth; key strategies for increasing the effectiveness of ACYF programs; and methods to help reduce the risk of homelessness. Data will be collected from 40 communities in which a small-scale longitudinal study was done of homeless people and from case studies in five local programs.

Healthy Development of Children and Families

ACF has several studies under way that address Head Start issues. First, Descriptive Study of the Head Start Health Component describes health-screening, examination, referral, treatment, and follow-up procedures across four domains of the Head Start health component: medical, dental, nutrition, and mental health. Case records of 1,200 4-year-old children enrolled in a national sample of 40 randomly selected Head Start programs were reviewed, and parents were interviewed concerning their child's health status and health services utilization patterns.

Next, through a consortium of local evaluators, the Evaluation of the Head Start Family Service Center Demonstrations project, conducted by Abt Associates, Inc., is evaluating 41 Family Service Center Demonstrations. The project focuses on how Head Start can collaborate with community programs to help meet the needs of Head Start families that must deal with problems like illiteracy, substance abuse, and unemployment.

The Study of the Characteristics of Families Served by Head Start Migrant Programs is profiling Head Start migrant families in the main migratory streams; identifying unique services issues; documenting the availability and coordination of services for Head Start families during migration; and providing a national estimate of the number of eligible migrant children. Findings will inform policy decisions on both Head Start migrant programs and the new Early Head Start program for infants and toddlers.

The Early Head Start Research and Evaluation project is evaluating the effectiveness of the Early Head Start (EHS) program in 15 diverse communities. The study examines child, family, staff, and community outcomes in a sample of 3,400 children and their families, randomly selected into project and comparison groups when the mothers are pregnant or before the children reach the age of 12 months. Children, families, and child care environments will be assessed when children are 14, 24, and 34 months of age. Service use interviews will be conducted semiannually and programs visited annually. The study will produce the following reports: (1) Descriptive Study of EHS Programs; (2) Study of Program Variations; (3) Pathways to Early Head Start Quality; (4) Interim Study of Outcomes; (5) Longitudinal Study of EHS Outcomes; and (6) Selected Policy Papers.

Descriptive Study of Families Served by Head Start is examining policy-relevant issues with a nationally representative sample of families served by Head Start in 40 programs across the country. Employing survey and case study methods, the project is charting families' demographics, strengths, needs, expectations, and experiences in Head Start and programmatic efforts to join in partnership with families.

The National Child Welfare Research Center in the School of Social Welfare at the University of California, Berkeley, will serve as a knowledge-building and information-disseminating resource for improved child welfare services. The Center will give special attention to (1) child abuse and child welfare; (2) family preservation and maintenance; (3) foster care and adoption; (4) drug- and AIDS-affected children; and (5) organizing, financing, and evaluating child welfare services.

One ACF project focuses on family protective services. Evaluation of Nine Comprehensive Community-Based Child Abuse and Neglect Prevention Programs, a contract with CSR, Incorporated, is (1) designing and implementing a process and impact evaluation of nine comprehensive community-based child abuse and neglect prevention projects funded by the National Center for Child Abuse and Neglect; (2) providing technical assistance to the projects in meeting evaluation requirements; and (3) helping the programs design and implement their own internal program evaluations. Because each project has up to 10 service components, many differing across projects, the contractor has developed a series of experimental designs for each service component.

Two ACF studies are looking at family services from an international perspective. First, Transfer of International Innovations--Development of a Clinical Monitoring System to Support Foster Care in Michigan features a computerized system based on a model combining structured and systematic monitoring of each individual child with the aggregation of this information across the whole agency. Developed and now mandated in Israel, and modified for the U.S. foster care system, the system provides an integrated response to the needs of all partners in the agency, including administrators, managers, and policymakers.

A second project, Social and Educational Development of Tribal-Based Communities of the Sonoran and Neger Deserts, is a knowledge-transfer project assessing the application of a preteen Bedouin Arab dropout prevention program in a tribal community in the Negev Desert of Southern Israel to the Pascua Yaqui Indian Tribe in the Sonoran Desert of Southern Arizona. The assessment will examine the effectiveness of a cooperative community-university model for empowering economically disadvantaged minority communities to respond to the unique needs of their at-risk children.

Two ACF projects address cross-cutting issues of substance abuse treatment and gang membership. The first project, Women and Infant Nurturing Services (WINGS), is designed to counteract the upward spiral of female incarceration and substance abuse. This demonstration project at the Rose M. Singer Correction Facility on Riker's Island, New York City, targets pregnant, substance-abusing inmates and uses incarceration as a point of treatment intervention. The evaluation tests the effectiveness of a comprehensive service program, including substance abuse treatment, prenatal health and nutritional care, HIV education, parenting classes, mental health services, and assistance with entitlement preparation.

The second project, Factors Related to Gang Membership Resistance, is gathering data on gangs from two contrasting Los Angeles communities, one with higher-than-average Hispanic and African-American gang activity, the other with lower-than-average gang activity. The project is designed to increase understanding of how youth in urban areas with high levels of street gang activity avoid gang involvement. Expected products include replicable interview protocol, data tapes for other researchers, a final report including implications for prevention programming, and plans for extended validation and replication.

New Directions for Evaluation

ACF's evaluation activities will focus on questions of family economic independence and child well-being that arise from States exercising their new authority to determine public assistance policy and program design under the Personal Responsibility and Work Opportunity Reconciliation Act of 1996. Key questions will concern not only issues of cash assistance and employment policy but also a range of other issues, including interactions among public assistance systems and child and family welfare programs. Assessing the impact of welfare reform on low-income families calls for continuing efforts to develop measures of child well-being; to explore administrative data bases as informational sources for evaluation; to identify methods for measuring outcomes and performance; to stimulate the field to benchmark, measure performance, and track results; and to provide information rapidly to local and State decisionmakers and to national associations.

Initiating and completing evaluations of programs begun as State welfare reform demonstrations is necessary to provide timely information about public assistance strategies and to add to the knowledge base for Federal, State, and local policymakers. Principal descriptive questions include the following: How are Temporary Assistance for Needy Families (TANF) programs implemented at the State and local levels? What is the extent of devolution of decision making to local government? How are child care supply and quality affected? Impact analyses of key policies and interventions (e.g., income disregard strategies and time limits) must follow descriptive studies to examine the effectiveness of these changes and to help States modify policies and approaches.

The effects of interaction and coordination between and among TANF, child development, child welfare, child support, and communitywide interventions constitute another key element of ACF's evaluation agenda. The economic security and overall well-being of disadvantaged children and families may be improved not only by how States design their TANF programs but, more likely, by how States integrate public assistance systems with programs and resources to address child development, family stability, child welfare, and parenting development. Efforts to create a complete picture of public assistance and family and child well-being outcomes will include mechanisms to collect accurate data from States. ACF will work with the Census Bureau and other organizations to ensure that national surveys address issues of concern and will work with States to improve quality and linkages of administrative data.

This evaluation agenda--describing State systems changes, assessing impacts on affected populations, and monitoring State and local program interaction--can only be realized through partnerships with State and local governments, professional organizations, service providers, and others in both the public and private sectors.