Performance Improvement 1998

Foreword

Performance Improvement 1998: Evaluation Activities of the U.S. Department of Health and Human Servicesis the third annual report summarizing evaluation efforts by the Department's agencies and offices during the past fiscal year. This report is intended to help those interested in evaluations of health or social services to locate information on study results, recommendations, or applications of program improvement or policy development. Generally, the audience for this report includes decisionmakers who need information on program results; program managers who want to know how other service programs are operating and how their own performance can be improved; and the community of researchers, advocates, and practitioners who use the program information and evaluation tools.

The mission of the Department of Health and Human Services (HHS) is to enhance the health and well-being of Americans by providing for effective health and human services and by fostering strong, sustained advances in the sciences underlying medicine, public health, and social services. To accomplish this mission, the Department manages an array of programs in basic and applied science, public health, and child and adolescent development; programs that foster economic self-sufficiency and support working families; and programs that finance or regulate health, mental health, and social services.

Through departmentwide strategic planning and performance management, HHS programs are continuously improving and redirecting to meet new needs of customers, beneficiaries, and other constituencies. The 1990's trend toward results-oriented management of HHS programs makes evaluation an important resource for producing knowledge to measure program performance, develop and improve programs, and analyze and develop policy.

For example, last year the Substance Abuse and Mental Health Services Administration in HHS completed one of the largest and most rigorous studies of drug treatment programs (Chapter II, page 12). Among the key findings were that clients of HHS-supported drug treatment significantly reduced their alcohol and other drug use; that treatment has had lasting benefits for clients; and that clients reported increases in employment and income, as well as decreases in criminal behavior, risk for human immunodeficiency virus/acquired immuno-deficiency syndrome (HIV/AIDS), and homelessness. This evaluation gives us new knowledge about what works in substance abuse treatment and the cost, which can be applied to improving current programs and designing new programs to advance future treatment.

Evaluations also are useful for implementing or operating programs. One such evaluation completed last year by the Centers for Disease Control and Prevention on syphilis control programs is helping local health departments in the Southern States to expand syphilis prevention efforts, increase access to services, and improve program operations and methods of contact tracing and notification (Chapter II, page 19). The evaluation findings offered new information on population groups at risk, barriers to reaching those at greatest risk, how best to reach those at risk through community institutions, and innovations in sexually-transmitted disease control and prevention programs. This study illustrates how HHS officials can effectively use evaluation resources to produce information useful for working with our partners in State and local communities to improve public health systems.

Assessing the environmental or external factors that affect HHS program or policy development is another major application of HHS evaluation activity. This past year, the Office of the Assistant Secretary for Planning and Evaluation completed a policy study on informal and formal kinship care of children and its importance for the child welfare system (Chapter II, page 26). Children living with relatives but away from parents, either informally or through formal foster care placements, is a growing phenomenon. This study provides new knowledge that will help shape future policy and program development in foster care services.

Performance Improvement 1998 contains information on HHS agency and office evaluation projects completed and in progress during fiscal year (FY) 1997. The report is organized into three Chapters. Chapter I describes the organization of HHS evaluations--activities, resources, and planning and management. Chapter II highlights the results from nine FY 1997 evaluations, selected by an outside review panel for their potential use by the larger health and human services community. Chapter III presents the evaluation activities of the 11 HHS agencies and the Office of the Secretary, including information on their evaluation programs, evaluations completed in FY 1997, and evaluations in progress. A complete inventory of the 155 HHS evaluation projects completed in FY 1997 is provided in Appendix A, and the 335 HHS agency projects currently in progress are listed in Appendix B.

We hope that you will find this report useful and informative.

Donna E. Shalala
Secretary
U.S. Department of Health and Human Services

Margaret A. Hamburg, M.D.
Assistant Secretary for Planning and Evaluation

Acknowledgments

Performance Improvement 1998: Evaluation Activities of the U.S. Department of Health and Human Services describes the continuous efforts of the various HHS agencies to examine service and research programs for the efficiency of their operations and their effectiveness in achieving objectives. The planning, development, and coordination of those evaluations is largely the responsibility of the following HHS planning and evaluation offices:

Administration for Children and Families
Office of Planning, Research and Evaluation
    Howard Rolston, Director

Administration on Aging
Office of Program Operations and Development
    Edwin Walker, Director

Agency for Health Care Policy and Research
Office of the Administrator
    Lisa Simpson, Deputy Administrator

Agency for Toxic Substances and Disease Registry
Office of Policy and External Affairs
    Georgi A. Jones, Director

Centers for Disease Control and Prevention
Office of Program Planning and Evaluation
    Kathy Cahill, Director

Food and Drug Administration
Office of Planning and Evaluation
    Paul L. Coppinger, Associate Commissioner

Health Care Financing Administration
Office of Strategic Planning
    Barbara S. Cooper, Director

Health Resources and Services Administration
Office of Planning, Evaluation and Legislation
    Paul W. Nannis, Director

Indian Health Service
Office of Public Health
    Leo Nolan, Director

National Institutes of Health
Office of Science Policy and Technology Transfer
    Lana Skirboll, Associate Director

Office of the Assistant Secretary for Planning and Evaluation
Office of Program Systems
    Susanne A. Stoiber, Deputy Assistant Secretary for Program Systems

Office of Public Health and Science
    Carol Roddy, Senior Program Advisor

Substance Abuse and Mental Health Services Administration
Office of Policy and Program Coordination
    Mary Knipmeyer, Director

Preparation of this report and the noted appendices was coordinated by staff in the Office of the Assistant Secretary for Planning and Evaluation, under the supervision of Susanne A. Stoiber, Deputy Assistant Secretary for Program Systems. Responsible staff include Mike Herrell, Paul Johnson, Colleen Monaghan, Carolyn Solomon, and Joan Lee Turek. Development and production of the report was supported, under contract, by the services of Camille Collett, Sacha Sanger, and Margaret Long of Sanad Support Technologies, Inc.; and C. Brannon Underwood, Reid Jackson, Wayne Hall, and Lydia Paddock of the South Carolina State University Policy Analysis Consortium.

The following persons from HHS agencies contributed to writing, collecting information on evaluation activities, and reviewing the report:

Administration for Children and Families
    James V. Dolson Helen Howerton

Administration on Aging
    Saadia Greenberg

Agency for Health Care Policy and Research
    Wendy Perry
    Harvey Schwartz

Agency for Toxic Substances and Disease Registry
    Woodrow Garrett
    Donna Jones

Centers for Disease Control and Prevention
    Nancy Cheal

Food and Drug Administration
    Kathleen McEvoy Mary Gamunev

Health Care Financing Administration
    William Saunders
    Sydney Galloway

Health Resources and Services Administration
    Anabel Crane

Indian Health Service
    Frank Marion
    Sarah Crazythunder

National Institutes of Health
    John Uzzell
    Joan Bailey

Office of Public Health and Science
    Carol Roddy

Substance Abuse and Mental Health Services Administration
    Anna Marsh

 TOP OF DOCUMENT

Evaluation Review Panel

The following individuals served on the evaluation review panel that made recommendations of the reports highlighted in Chapter II of Performance Improvement 1998: Evaluation Activities of the U.S. Department of Health and Human Services.

Heather Becker, Ph.D.
Professor, School of Nursing
University of Texas
FACE="Times New Roman">Austin, Texas

Frank Caro, Ph.D.
University of Massachusetts at Boston
Gerontology Institute and Center
FACE="Times New Roman">Boston, Massachusetts

Tresmaine R. Grimes, Ph.D.
South Carolina State University
Orangeburg, South Carolina

Holly Korda, Ph.D.
Evaluation Consultant
Boston, Massachusetts

John Kralewski, Ph.D.
Director, Institute for Health Services Research
School of Public Health
University of Minnesota
Minneapolis, Minnesota

Kenneth McLeroy, Ph.D.
Health Promotion Sciences Department
University of Oklahoma
Oklahoma City, Oklahoma

Donna Mertens, Ph.D.
Professor
Gallaudet University
Washington, D.C.

Ira Raskin, Ph.D.
Health Policy Consultant
Bethesda, Maryland

Felix A. Okojie, Ph.D.
Jackson State University
Jackson, Mississippi

 TOP OF DOCUMENT

Executive Summary

Performance Improvement 1998 is the third annual report of the U.S. Department of Health and Human Services (HHS) on its evaluation activities. As a report to Congress, it summarizes the findings of HHS evaluations completed in fiscal year (FY) 1997. In that year, HHS agencies produced 155 evaluation reports and supported more than 335 evaluation projects in progress.

In general, the report is intended for three audiences: decisionmakers, who need information on program results across the broad spectrum of health and human services; program managers, who need information on how they can make improvements in program operations and outcomes; and the health and human services community, which can benefit by applying the knowledge and lessons learned from HHS evaluations.

In the Foreword, Secretary Donna Shalala and Dr. Margaret A. Hamburg, Assistant Secretary for Planning and Evaluation, stress that evaluation is an important tool for producing the knowledge necessary to develop and improve the performance of HHS programs and services to meet the needs of the 21st century. HHS program managers are continuously engaged in efforts to determine whether programs and services reach the intended populations or communities, perform efficiently, and achieve desired results.

CHAPTER I

Chapter I provides an overview of evaluation at HHS. The overview contains information about the types of evaluation activities; evaluation resources; and evaluation management, including planning and coordination, project management, quality assurance, and dissemination. The last section of the Chapter discusses four ways in which HHS makes effective use of evaluation results for strategic planning and program and policy development. These are performance measurement and data systems to monitor progress in achieving departmental goals and strategies as well as program outcomes; evaluations to assess the effectiveness of HHS strategies and programs used in achieving goals and objectives; environmental assessments to study changes in the larger society and their impact on departmental programs and strategies; and evaluation resources to improve program management and support evaluation efforts.

CHAPTER II

Chapter II highlights nine HHS evaluations completed during FY 1997 and selected by the Evaluation Review Panel as potentially useful to the larger health and human services research and practice community. For the selection criteria, see Appendix C. Summaries of each report are presented according to three general purposes of evaluation: performance measurement or assessment, program management and development, and policy analysis and development.

Performance Measurement or Assessment

LEAP: Final Report on Ohio's Welfare Initiative to Improve School Attendance Among Teenage Parents

The goal of Ohio's Learning, Earning, and Parenting (LEAP) Program is to use financial incentives to increase the proportion of pregnant teenagers and teen parents on welfare who graduate from high school or receive a General Equivalency Diploma (GED), find jobs, and ultimately achieve self-sufficiency. The immediate goals were to induce dropouts to return to high school or to enroll in General Educational Development programs, and to promote better attendance among those already enrolled in school. These short-term goals were accomplished--statistically significant increases in school enrollment and attendance were observed. Longer-term goals were to increase the rate of high school graduation, GED attainment, employment, and to reduce welfare payments. These long-term goals were only partly successful. The evaluation of LEAP is timely in light of new changes to national welfare laws which increases emphasis on school attendance and on more employment-oriented activities for teens and other welfare recipients.

National Treatment Improvement Evaluation Study: Final Report

The congressionally mandated 5-year National Treatment Improvement Evaluation Study (NTIES) examined the impact of three federally-sponsored drug and alcohol treatment demonstration programs. The study found that the demonstration programs significantly reduced drug and alcohol use, and that treatment had lasting benefits. Clients reported increases in employment and income, improvements in mental and physical health, decreases in criminal activities and homelessness, and modification of sexually risky behavior.

Program Management and Development

Self-Sufficiency Project Implementation Manual: Lessons Learned from Eight Years of Office of Community Services Demonstration Partnership Programs

This manual summarizes lessons learned during eight years of the Demonstration Partnership Program (DPP), which supports innovative approaches toward increasing the self-sufficiency of the poor, tests and evaluates the approaches, and encourages their replication through dissemination of project results and findings. These projects were divided into five areas: (1) case management, (2) micro-enterprise development, (3) minority males, (4) homelessness, and (5) youth at risk. The manual was designed as a step-by-step guide for designing and implementing Self-Sufficiency Projects by community action agencies, community-based organizations, and local community program planners.

The Effects of Informatics Tools and Decision Aids to Help Patient Decision-making about Medical Screening and Treatment

This study identified, evaluated, and synthesized research on our knowledge base on the effectiveness of tools intended to help inform patients about their medical choices, their treatment alternatives, and the risks and benefits of those alternatives. These included interactive computer disks, videotapes, audiotapes, brochures, and computer-generated fact sheets to help patients make decisions about medical screening and treatment. The research found that although some studies do point to the promise these tools hold for helping patients and consumers, few controlled and comparative studies have been done.

Syphilis in the South: A Case Study Assessment in Eight Southern Communities

This project provided a comparative case study of local-level syphilis prevention efforts in eight communities in Alabama, Mississippi, South Carolina, and Tennessee. The project focused on groups perceived to be at high risk of becoming infected with syphilis, the extent to which public health activities target such groups, and identification of factors that affect the reach of services to these groups. African Americans were found to be at greatest risk of syphilis infection. The report highlights the cultural, programmatic, and political barriers that restrict the prevention and control of syphilis and other sexually transmitted diseases for those at greatest risk. Cultural barriers include local norms that restrict public discussion of human sexuality, distrust of the public health system among African Americans, and a low priority of health relative to other issues of poverty in the community.

Stakeholders Revisit Healthy People 2000 to Maximize the Impact for 2010

This report presents an assessment by Healthy People 2000 stakeholders on how to restructure the goals and objectives of Healthy People 2000 for the next decade. Healthy People 2000 is a set of national objectives for improving the health of Americans through effective prevention strategies. These objectives form a model framework for results-oriented performance measurements that are relevant to both population-based and individual-based health care initiatives. The objectives also serve as benchmarks against which performance can be measured over time. The report explores new approaches for the development of Healthy People 2010, taking into consideration the many major changes in health care since 1990, and past experience with Healthy People 2000 and the health objectives for 1990.

Policy Analysis and Development

Impact of the Medicare Fee Schedule on Teaching Physicians

This study assessed the impact of a new Medicare Fee Schedule (MFS) on the revenues of teaching physicians. The new schedule was based on the Resource-Based Relative Value Scale and altered the relative prices paid to physicians. Relative payment levels were greatly increased for visits (less intensive relative value units, RVUs), and reduced for most types of diagnostic tests and surgical procedures (more intensive RVUs). Relative payment levels were raised in rural areas and lowered in large urban areas. It was hypothesized that teaching physicians would be adversely affected by the MFS because they perform more high-technology procedures and provide comparatively fewer primary care services. Teaching physicians also are located in large urban areas and experience customary charges that may be higher than the area-wide historical payments calculated for fee schedule transition. The study analyzed all Medicare Part B claims associated with discharges during 1991-1993 from a sample of teaching and non-teaching hospitals. Part B revenues and selected price variables were assessed for different categories of service. No evidence was found to suggest that teaching physicians had been adversely affected by the MFS.

Informal and Formal Kinship Care

This study used existing national data sources to describe the characteristics of children in kinship living arrangements, or living with relatives in the absence of a parent, and to define recent trends in the pattern of kinship caregiving. Comparisons were made between formal kinship arrangements in which foster care was provided under State auspices, and informal kinship arrangements in which foster care was provided without State intervention. Over 2 million, or 3 percent, of all children in the United States are in kinship care (i.e., a child living in a relative's household without a parent present). Kinship caregiving and the children living in single mother families both appear to be adaptations to family disruption and the decline of the traditional two-parent family.

Market Barriers to the Development of Pharmacotherapies for the Treatment of Cocaine Abuse and Addiction

This study analyzed market barriers to the development of drug therapies for substance abuse and addiction, and for cocaine use and addiction in particular. Using market analysis, quantitative modeling, case studies, and industry interviews, the study concluded that the development of a new cocaine pharmacotherapy was not economically viable for the pharmaceutical industry under current conditions. The industry faces three critical market barriers: a small and uncertain market for cocaine addiction and abuse drug therapy, a substance abuse treatment system that limits access to this market, and limited and uncertain payment possibilities.

CHAPTER III

Chapter III provides an overview of HHS agency evaluation activities, including information on the evaluation program, summaries of evaluations completed during FY 1997, and evaluations in progress.

Administration for Children and Families

The Administration for Children and Families (ACF) administers a broad range of programs to support children, families, and other targeted populations. ACF evaluation objectives include providing information on program design and operations, testing services delivery approaches, conducting policy analyses, and disseminating and applying evaluation results. In FY 1997, evaluations were completed on welfare-to-work strategies; responsible fatherhood programs; a paternity acknowledgment in child support program; the Head Start Program; protective, preventative, and reunification services for children and their families; special needs adoptions; child maltreatment; and the Child Abuse and Neglect Prevention Programs. Evaluations in progress during FY 1997 include State welfare reform programs, employment training programs for welfare recipients, child impact studies related to welfare reform, community services demonstrations, child support programs, various aspects of the Head Start Program, child care and welfare reform, homeless youth programs, and quality of foster care.

Administration on Aging

The Administration on Aging (AoA) supports studies that provide information on the implementation of the Older Americans Act of 1992. During FY 1997, AoA initiated a FACE="Times New Roman">n evaluation of the Supportive Services and Senior Centers provisions of the Act (Title III-B), which provides home- and community-based services for all elderly persons, particularly those in great economic or social need. Most home- and community-based services fall under three broad categories: access services, in-home services, and other community-based services. Many State and Area Agencies on Aging also are conducting evaluations of their Older Americans Act service programs.

Agency for Health Care Policy and Research

The goals of the evaluation program at the Agency for Health Care Policy and Research (AHCPR) are to assess the Agency's effectiveness in meeting major and long-term priorities, to obtain information to respond to critical agency and departmental concerns, and to conduct internal evaluations to improve the efficacy of key program areas. Evaluation projects completed in FY 1997 included assessments of private sector outcomes and effectiveness research, informatics as a tool for patient diagnosis and treatment decisions, quality of care measurement, data on managed care organizations for health services research, and customer satisfaction with AHCPR's research dissemination efforts. Ongoing evaluations during FY 1997 include projects to examine AHCPR's grant application and review processes and survey data collections procedures.

Agency for Toxic Substances and Disease Registry

The Agency for Toxic Substances and Disease Registry (ATSDR) was created as a Federal agency by the Comprehensive Environmental Response, Compensation, and Liability Act (CERCLA), more commonly known as Superfund legislation. The evaluation program is coordinated with the Agency's strategic planning process and implementation of the Government Performance and Results Act (GPRA). During FY 1997, ATSDR developed an agencywide inventory of evaluation activities, conducted an assessment of its site-specific evaluation needs, and developed a guidance document for State cooperative agreement evaluation activities. ATSDR also initiated an evaluation of its Hazardous Substances Emergency Events Surveillance (HSEES) program to measure the sensitivity and reliability of the system.

Centers for Disease Control and Prevention

The Centers for Disease Control and Prevention (CDC) places high priority on evaluations seeking to answer policy, program, and strategic planning questions related to the Agency's mission. In FY 1997, CDC's completed projects include major program evaluations, such as the National Laboratory Training Network, the Business Responds to AIDS Program, and suicide interventions for Native American communities. Several data policy/surveillance studies were completed, such as one on the medical certification process for death certificates. CDC also finished a major project on performance measurement to support GPRA implementation. CDC's evaluations in progress include program assessments in such areas as diabetes control, field epidemiology training, teen pregnancy prevention, and sexually-transmitted disease partner notification. Evaluation of data policy and surveillance systems are important at CDC. There are four projects currently underway to develop performance measurement systems, several of which are in the area of human immunodeficiency virus (HIV) prevention programs. CDC also invests it evaluation resources in more specialized assessments, such as evaluating the activities of tuberculosis outreach workers and making recommendations on bicycle helmet use.

Food and Drug Administration

The evaluation program at the Food and Drug Administration (FDA) continues to be aligned with the agency's policy, program and strategic planning initiatives as well as with the implementation of GPRA. FACE="Times New Roman">FDA managers also are identifying further opportunities to involve their customers in the design and testing of alternative ways of doing business. In FY 1997, FDA continued to identify performance measures jointly with industry customers as part of the drug and biologic review processes necessary for implementation of the Prescription Drug User Fee Act. In addition, FDA completed evaluation projects on mammography quality standards, diagnostic medical devices, and food safety issues. FDA's evaluation projects in progress include continuing studies on the Prescription Drug User Fee Act, mammography quality standards, and the assistance program for small businesses. New studies were initiated on food safety standards and the President's tobacco initiative.

Health Care Financing Administration

The Office of Strategic Planning, the research arm of the Health Care Financing Administration (HCFA), performs, coordinates, and supports research and demonstration projects (through intramural studies, contracts, grants, and waivers) to develop and implement new health care financing policies and to provide information on the impact of HCFA's programs. The scope of HCFA's research and demonstration activities embraces all areas of health care: costs, access, quality, service delivery models, and financing approaches. In FY 1997, HCFA completed 20 evaluations on the Medicare Fee Schedule, Medicaid quality of care, effectiveness of the Operation Restore Trust demonstrations, inpatient hospital services, alternative payment of Medicare cataract surgery, ambulatory patient groups, Medicare case management demonstrations, nursing home quality indicators, and health plan report cards. HCFA has over 22 projects in progress, with 13 of them scheduled for completion in FY 1998. Some of the demonstrations involve the Medicaid managed care programs or waivers, Medicaid uninsured demonstrations (particularly the expansion of insurance to children), rural health clinics, Medicare Alzheimer's disease demonstration, use of telemedicine, and long-term care for persons with developmental disabilities.

Health Resources and Services Administration

The purposes of the Health Resources and Services Administration (HRSA) evaluation program are to enhance strategic and performance planning, to strengthen budget and legislative development, and to improve program management. Evaluation priorities include developing and supporting performance measurements, assessing program implementation, and conducting crosscutting policy analysis and research. During FY 1997, HRSA completed evaluations related to its mission to eliminate barriers to care, which included assessments of rural applications of telemedicine and living-related kidney transplant operations. Related to HRSA's mission to assure quality of care, HRSA completed a study on human immunodeficiency virus/acquired immunodeficiency syndrome (HIV/AIDS) treatment services and tuberculosis treatment programs in major cities. Another major study assessed the availability of primary care services under Medicaid managed care. Ongoing evaluations in FY 1997 include effectiveness of community health centers, development and refinement of tools for monitoring cultural competence in managed care; the Healthy Start Program to reduce infant mortality; Ryan White CARE Act programs; and development of a comprehensive performance monitoring system for the Bureau of Health Professions.

Indian Health Service

The evaluation program of the Indian Health Service (IHS) is managed by the Office of Public Health's Staff Office of Planning, Evaluation, and Research. This Office provides national leadership and consultation for IHS and Area Offices on strategic and tactical planning, program evaluation and assessment, public health and medical services, research agendas, and special public health initiatives for the agency. One IHS evaluation completed in FY 1997 focused on the impact of the Alaska Tribal Health Compact on programs and services in Anchorage, Alaska, and another project looked at the prior trauma care of intoxicated patients as a predictor of subsequently fatal injury. Major projects in progress include an examination of the IHS Resource Requirement Methodology, a process for determining needs to deliver adequate care; an assessment of aftercare services provided by IHS regional substance abuse treatment centers; and a study of the IHS capacity for epidemiologic surveillance.

National Institutes of Health

The National Institutes of Health (NIH) Evaluation Program provides information to assist the NIH Director and the various Institute, Center, and Division Directors in determining whether NIH goals and objectives are being achieved and to help guide policy development and program direction. In FY 1997, NIH evaluated the intramural research program at the National Institute of Mental Health; the Bridges to the Future Program, a minority student initiative; and the NIH Consensus Conference on physician prescribing patterns for treatment of peptic ulcer disease. Other major evaluations looked at the status of biomedical research facilities; the National Research Service Award Research Training Program; article citation and patent references at the National Heart, Lung and Blood Institute; the rare disease clinical research database; laboratory animal use, facilities, and resources; safe and effective genetic testing; the Physician Data Query database at the National Cancer Institute; research instrumentation needs; toxicology and environmental health information resources; the international cooperative biodiversity groups; and the Federal Interagency Forum on Aging-Related Statistics. Projects in progress include the Surgeon General's Report on Oral Health, an evaluation of the NIH Human Resources Management System, completion of the Study of National Needs for Biomedical Research Personnel, and the biennial Survey of Scientific and Engineering Research Facilities at Colleges and Universities.

Office of the Assistant Secretary for Planning and Evaluation

The Assistant Secretary for Planning and Evaluation (ASPE) functions as principal advisor to the Secretary on policy development and conducts evaluations and policy research studies. It is also responsible for HHS-wide coordination of legislative, planning, and evaluation activities. During FY 1997, ASPE's major evaluation projects looked at pharmacotherapies for treating cocaine abuse and addiction; public health laboratories; performance measures for public health, substance abuse, and mental health; insurer response to health care market changes; privacy and health research; State-assisted living policies; HIV/AIDS home- and community-based services; consumer-directed personal assistance services; welfare-to-work program approaches; welfare reform and teenage parents; informal and formal kinship care; child welfare; and domestic violence. Major evaluations in progress include understanding private sector changes; improving data and analytic capability; disability, aging, and long-term care trends; promoting active lifestyles and maximum independence; managed care and people with disabilities; assisted living; moving welfare recipients to work; measuring implementation and outcomes of welfare reform, child support, child and youth development, and teen pregnancy programs.

Office of Public Health and Science

The Office of Public Health and Science (OPHS) conducts evaluations to support the Surgeon General and the Assistant Secretary for Health in his dual role as the Nation's top doctor and senior advisor to the Secretary on public health and science matters. OPHS evaluations completed in FY 1997 focused on developing a framework of national health goals and disease prevention objectives in anticipation of the development of the Healthy People 2010 plan; public health expenditures of nine State and local public health, mental health, substance abuse and environmental agencies; and the Office of Minority Health-sponsored program for Historically Black Colleges and Universities. Evaluations and policy studies in progress include dietary supplement labels, school health programs, public health infrastructure, a new model of coordinating and integrating HIV-prevention and primary care services, the Minority Health Resource Center, the Bilingual/Bicultural Service Demonstration Grant Program, and linguistically and culturally appropriate community health promotion programs in local health departments.

Substance Abuse and Mental Health Services Administration

The Substance Abuse and Mental Health Services Administration (SAMHSA) is committed to evaluating its programs and grant projects to assess the effectiveness of prevention, treatment, and rehabilitation approaches and system of care. In FY 1997, SAMHSA completed the National Treatment Improvement Evaluation Study, a comprehensive assessment of the results of 157 three-year substance abuse treatment demonstrations, and an assessment of the Job Corps program, which provides enriched substance abuse treatment for adolescents. Also completed were studies on substance abuse prevention services for pregnant and postpartum women and their infants; the National Training System for planning and delivering substance abuse prevention services; the Faculty Development Program to enhance research, teaching, and clinical practices on issues of alcohol, tobacco, and other drugs; and a program for mental health care provider education in HIV/AIDS programs. Evaluations in progress include a services integration experiment for the chronically mentally ill homeless persons; the impact of managed care on access, cost, and quality of substance abuse treatment; an outcome evaluation of the Community Partnership Program; evaluation of substance abuse treatment improvement protocols; the long-term effects of drug abuse treatment; and an evaluation of opioid treatment program accreditation.

APPENDICES

A complete inventory of HHS evaluations completed in FY 1997 is provided in Appendix A, and HHS Agency projects currently in progress are listed in Appendix B.

The criteria used by the Evaluation Review Panel to select the reports highlighted in Chapter II are located in Appendix C.

Chapter I. Evaluation in the Department of Health and HumanServices

The mission of the U.S. Department of Health and Human Services (HHS) is to enhance the health and well-being of Americans by providing for effective health and human services and by fostering strong, sustained advances in the sciences underlying medicine, public health, and social services. The following HHS agencies and offices accomplish this goal through their program activities and performance evaluations.

• Administration for Children and Families (ACF)
• Administration on Aging (AoA)
• Agency for Health Care Policy and Research (AHCPR)
• Agency for Toxic Substances and Disease Registry (ATSDR)
• Centers for Disease Control and Prevention (CDC)
• Food and Drug Administration (FDA)
• Health Care Financing Administration (HCFA)
• Health Resources and Services Administration (HRSA)
• Indian Health Service (IHS)
• National Institutes of Health (NIH)
• Office of the Secretary
• Substance Abuse and Mental Health Services Administration (SAMHSA)

The Assistant Secretary for Planning and Evaluation (ASPE), located in the Office of the Secretary, coordinates evaluation activities throughout HHS.

Evaluation plays an integral role in carrying out the HHS mission. Assessing various aspects of agency program performance allows staff to identify means of improving that performance. The HHS evaluation function has three goals: (1) to provide information on HHS programs that helps government officials and members of the Congress make decisions related to programs, policies, budgets, and strategic planning; (2) to help HHS managers improve program operations and performance; and (3) to disseminate evaluation results and methodological tools useful to the larger health and human services community of State and local officials, researchers, advocates, and practitioners for improving the performance of their programs. This last goal is very important to HHS. Its agencies have an important obligation to foster the development of new knowledge about the effectiveness of health and human services programs and interventions and evaluation tools for use by the larger health and human services community. Although the findings and recommendations of HHS evaluations are usually first used by the Administration and the Congress, they can also be applied by others in the research and practice communities to improve program performance at the State and community levels. The purpose of this report is to disseminate information about recent HHS evaluations and to make sure that the potential for wider application is realized.

This Chapter describes the organization and operation of evaluation at HHS. It first provides an overview of the kinds of evaluation activities supported by HHS agencies and then describes the resources and funding mechanisms used to support them. The Chapter details HHS evaluation management, including planning procedures, project management, quality assurance, and dissemination of results. The Chapter concludes with a discussion of effective uses of HHS evaluations, with illustrations from projects completed in the past year.

HHS Evaluation Activities

HHS defines evaluation as the assessment of program performance (efficiency, effectiveness, and responsiveness) through the analysis of data or information collected systematically and ethically, and the effective use of resulting information in program or policy decisionmaking and program management. This definition encompasses a range of evaluation activities, such as those listed below.

• Outcome evaluations measure the immediate or intermediate effects of a program with respect to the stated goals or objectives
• Impact evaluations look at the broader and long-term results, intended or unintended, of a program on populations or institutions involved.
• Implementation or process evaluations assess the nature of program inputs and outputs and their relationship to the stated goals and objectives.
• Policy assessments examine health policies with respect to their development, implementation, or their impact on public health or program activity.
• Cost-benefit or cost-effectiveness analyses develop methodology and its application to assess the relationship of program results to program costs (direct and indirect), often in comparison with alternative programs.
• Survey data analyses evaluate the results of HHS programs or policies by analyzing data obtained from surveys.
• Performance measurement and data systems identify and test the validity and reliability of process, output, and outcome indicators to measure the performance of programs and develop data systems supporting implementation of the Government Performance and Results Act (GPRA) of 1993.
• Simulations and models use computer simulations and modeling techniques to analyze the impact of policy changes on service delivery systems and beneficiaries.
• Management studies examine the effectiveness or efficiency of the administration or operation of HHS programs and offices.
• Evaluation syntheses integrate the results from multiple independent evaluation studies within a defined program or policy area in a fashion that improves the accessibility and application of those results.
• Evaluation feasibility studies assess the clarity and importance of program goals and objectives, the consensus of program stakeholders on the potential utility of evaluation information, and the availability of relevant performance data before committing to a full-scale program evaluation.
• Evaluation design projects procure assistance developing of an evaluation design, measurement tools, or analytic models in preparation for fully implementing an evaluation.
• Instrument development projects develop evaluation instruments (design, measurement, or analytic) for a specific HHS program or for general use by the health and human services community.
• Evaluation technical assistance helps HHS officials and grantees with any aspect of evaluation planning, project design implementation analysis, or use of results.
• Evaluation dissemination identifies target audiences and mechanisms to inform program constituencies and evaluation stakeholders about evaluation results.
• Evaluation training/conferences maintain the professional skills and expertise of evaluation staff through training opportunities, and promote the dissemination of HHS evaluations through conferences.

TOP OF DOCUMENT

Evaluation Resources

Evaluation activities of the various HHS agencies are largely supported through two funding mechanisms: direct use of program funds and use of special legislative set-aside authorities for evaluation. The first is a common mechanism by which programs managers have discretionary authority to use appropriated program funds to support contracts that will design, implement, and analyze evaluation data. In some cases, a program's legislative authority calls for a specially mandated evaluation, and program funds are used directly to support the evaluation.

The second mechanism for evaluation funding is the legislative set-aside authority which permits the Secretary of HHS to use a proportion of overall program funds for evaluation purposes. The largest of such set-aside authorities is one established for evaluations conducted by several agencies of the U.S. Public Health Service (AHCPR, CDC, HRSA, NIH, and SAMHSA), ASPE, and the Office of Public Health and Science in the Office of the Secretary. The mechanism is called the 1-percent evaluation set-aside legislative authority, which is provided for in Section 241 of the Public Health Service (PHS) Act. This authority was established in 1970, when the Congress amended the Act to permit the HHS Secretary to use up to 1 percent of appropriated funds to evaluate authorized programs. Section 241 limits the base from which 1 percent of appropriated funds can be reserved for evaluations of programs authorized by the PHS Act. Excluded are funds appropriated for FDA, IHS⁽¹⁾, and certain other programs that are managed by PHS agencies but not authorized by the Act (e.g., HRSA's Maternal and Child Health Block Grant and CDC's National Institute for Occupational Safety and Health). In FY 1997, HHS invested more than $35 million in 1-percent evaluation funds to carry out evaluation activities related to relevant public health programs. The FY 1997 Labor, HHS, and Education Appropriations Act provided that an additional $96 million in 1-percent evaluation funds be used to support data collection and analysis activities in AHCPR and CDC's National Center for Health Statistics.

Evaluation Management

The management of HHS evaluations, which are carried out on a regular basis by the HHS agencies and offices and coordinated by ASPE, involves four basic functions:

1. Evaluation planning and coordination

2. Project management

3. Quality assurance

4. Dissemination of evaluation reports.

A description of each function in general terms follows. Additional information on the individual HHS agencies, ASPE, and OPHS evaluation functions is found in Chapter III.

Evaluation Planning and Coordination

The HHS agencies, ASPE, the Office of the Inspector General (OIG), and OPHS develop evaluation plans annually in concert with HHS's program planning, legislative development, and budgeting cycles. Plan development is coordinated by ASPE. Each agency or office plan generally states the evaluation priorities, or projects under consideration for implementation. Typically, HHS evaluation priorities include congressionally mandated program evaluations, evaluations of Secretarial program or policy initiatives, and assessments of new programs, programs that are candidates for reauthorization, or programs for which key budget decisions are anticipated.

More specifically, HHS evaluation planning activities involve preparing two reports to the Congress. First, those agencies and offices that use the PHS 1-percent evaluation set-aside authority--AHCPR, CDC, HRSA, NIH, ASPE, OPHS, SAMHSA--submit a formal plan to ASPE, which coordinates and develops the individual plans into the HHS report to the Congress on the use of the PHS 1-percent authority. This report must be submitted to the Congress before HHS can implement the plan.

Second, the Congress requests that HHS coordinate all of its research, demonstration, and evaluation programs to ensure that the results of these projects address HHS program goals and objectives. ASPE and the Assistant Secretary for Management and Budget work together with HHS agencies to provide the Congress with a special annual research, demonstration, and evaluation budget plan that coincides with the preparation of the President's fiscal year budget. The plan outlines each HHS agency's research, demonstration, and evaluation priorities as related to its strategic goals and objectives.

Project Management

The design and management of evaluation projects at HHS is principally decentralized--the HHS agencies, OIG, and ASPE all are responsible for executing annual evaluation plans, developing evaluation contracts, and disseminating and applying evaluation results. Even within agencies-- while there is some oversight responsibility and execution capability in the Office of the Director or Administrator--the various subunits (centers, institutes, and bureaus) conduct much of the day-to-day evaluation activity.

The OIG performs independent evaluations through its Office of Evaluations and Inspections (OEI). The OEI's mission is to improve HHS programs by conducting inspections that provide timely, useful, and reliable information and advice to decisionmakers. This information (findings of deficiencies or vulnerabilities and recommendations for corrective action) is usually disseminated through inspection reports issued by the Inspector General. A summary of individual inspection reports and other OIG reports can be viewed on the Internet at http://www.sbaonline.sba.gov/ignet. OEI also provides technical assistance to HHS agencies in conducting their evaluations.

Quality Assurance

Most evaluation projects are developed at the program level. The initial quality review is generally conducted by a committee of agency- or office-level policy and planning staff members. Before a project is approved, however, it is also reviewed for technical quality by a second committee with expertise in evaluation methodology. Technical review committees follow a set of criteria for quality evaluation practice established by each agency. Some HHS agencies also have external evaluation review committees composed of evaluation researchers and policy experts from universities and research centers. More details on the quality assurance procedures for the various HHS agencies, ASPE, and OPHS are presented in Chapter III.

Dissemination of Evaluation Reports

Maintaining report libraries and distributing information on evaluation results is an important component of HHS evaluation management. Project information and reports are continuously submitted to the HHS Policy Information Center (PIC)--the departmental evaluation data base and library maintained by ASPE. As an information data base and library resource, the PIC contains nearly 7,000 completed and in-progress evaluation and policy research studies conducted by the Department, as well as key studies completed outside of HHS by the U.S. General Accounting Office (GAO) and private foundations.

Typically, the results of HHS evaluations are disseminated through targeted distribution of final reports, articles in refereed journals, and presentations at professional meetings and conferences. Although individual HHS agencies have primary responsibility for disseminating results, there is a departmentwide effort under way to expand dissemination to the larger research and practice communities through centralized computer communications and publications. Abstracts of all studies maintained in the PIC evaluation data base are now accessible on the Internet at http://www.os.HHS.gov. Information is available on completed projects, including the name and telephone number of an HHS official responsible for the project.

In addition, HHS widely distributes copies of the annual evaluation report series, beginning with Performance Improvement 1995 and continuing to this report, through the HHS website. For convenience, these reports can be accessed on the Internet at: http://aspe.os.HHS.gov/PIC/gate2pic.htm.

Ensuring Effective Use of Evaluation Results

HHS is committed to ensuring that evaluations yield a high return on the investment of available program funds. Today, the Department's evaluation resources are used in several ways related to strategic planning, program, and policy development. Performance measurement and data systems are the primary mechanisms used to monitor progress in achieving departmental goals and objectives as well as specific program outcomes. Effectiveness of programs and strategies are in-depth evaluation studies to understand how HHS strategies and programs are linked to performance goals and objectives. Environmental assessments are prospective evaluation studies that assess how changes in the larger society affect the Department's programs and strategies. Program management and support evaluations are used to improve the management of health and social service programs and the quality of the Department's evaluation efforts. Each of these four uses, with examples of current HHS evaluations, is described below.

Performance Measurement and Data Systems

Implementation of the HHS strategic plan and the performance plans of the PHS agencies, pursuant to the Government Performance and Results Act (GPRA) of 1993, requires the Department to focus a portion of its evaluation activities on the development of performance objectives and measures and information systems necessary to produce the data needed to assess progress toward achieving its goals. During FY 1997, evaluation priorities of the individual HHS agencies included a number of projects serving this purpose. For example, ACF completed a project for Head Start Programs to develop outcome performance measures (e.g., how these children benefit from the program), in addition to process measures (e.g., how many teachers have the appropriate credentials?) (6693). FDA completed its fifth and final report on performance data for the Prescription Drug User Fee Act (PDUFA), documenting the improvements obtained in the last few years in speeding up the process for drug reviews and approvals (6079.2). ASPE produced a report synthesizing the state-of-the-art performance measurement for public health and developed recommendations for measures that could be used to monitor Federal-State performance partnership grants being considered for the future (6177). An example of a major survey providing nationally representative data is one completed last year by HRSA on the users of community health centers (CHC) and the services they were provided (5737). It produced, for the first time, nationally representative data on CHC users that is comparable to similar data on health care use from the National Health Interview Survey. The data are now being analyzed to address questions about users' race and ethnicity, health risk behaviors, most prevalent diagnoses, and services used.

Effectiveness of Programs and Strategies

Implementation of the Department's strategic and performance plans also requires evaluative information on how well the programs and strategies are working. Most of the HHS agency evaluations completed in FY 1997 provide this information. For example, HCFA submitted a report to the Congress on the quality of care in the Medicaid program (6302). It assessed the variations in the rate of performance of selected treatments and procedures on Medicaid beneficiaries for small areas within and between the States. Medical records in a sample of hospitals were also examined for three conditions: pediatric asthma, hysterectomy, and complicated delivery. Overall, the report concluded that care was considered adequate and comparable to that received by privately insured patients. Another example of a focused program effectiveness study is one conducted by HRSA on rural applications of telemedicine (5749). The project included a mail survey of all non-Federal hospitals located outside metropolitan areas in the summer of 1995, plus a short survey for hospitals doing only teleradiology and a longer survey for hospitals applying telemedicine to purposes beyond radiology. The study demonstrated that although rural telemedicine is in the earliest stages of development, it is expanding rapidly. HRSA is using the results to design a common telemedicine evaluation instrument for HRSA grantees and for consideration by other Federal sponsors, as well as in shaping the Rural Telemedicine Grant Program. In terms of evaluating research programs, NIH conducted an evaluation of the NIH Consensus Conference as an effective mechanism for translating research into practice (6284).

Environmental Assessment

The Department's strategic plan acknowledges that achievement of its goals and objectives is contingent on many external factors that are beyond the Department's control. For example, the managed care revolution is having a significant impact on access to services and health outcomes for those populations traditionally served by the public health system. Understanding the impact of these environmental forces on public health programs and the customers they serve is essential for assessing and adjusting the Department's goals and strategies in the future. A number of evaluations in FY 1997 addressed these environmental issues.

For example, AHCPR conducted a study to help clarify its future priorities in outcomes and effectiveness research and to provide insights regarding future opportunities for public-private partnerships in this area (6385). The study found that many private sector organizations conduct effectiveness research, often to monitor health care performance against accreditation standards. AHCPR will also use the findings to help guide its support to developing and validating effectiveness and outcome research tools used by the private sector. In another example, ASPE last year analyzed market barriers to the development of pharmacotherapies for substance abuse and addiction, particularly for abuse of and addiction to cocaine (6694). This report, featured in Chapter II, found several critical market barriers that must be taken into account in future HHS program and policy development. Since the substance abuse treatment market relies heavily upon State and Federal reimbursement, most substance abuse treatment services are subsumed under the mental health benefits of entitlement programs. Drug companies are reluctant to rely upon this kind of reimbursement in an age of shrinking budgets for mental health services.

Program Management and Support

Effective management of programs that achieve departmental goals and objectives is essential to the success of those programs. In FY 1997, the evaluations completed and in progress include a number of priorities devoted to assessing and improving the management of health and social service programs. For example, HCFA last year looked at the implementation of Ambulatory Patient Groups (APGs)--the Medicaid outpatient prospective payment system that groups patients for payment purposes rather than paying on a cost basis (6320). In this case study of Iowa's implementation of the APG system and an analysis of the reimbursement methodology, success was reported in reducing outpatient costs, where that was the immediate goal, and the system encouraged higher-cost facilities to reduce costs and reward lower-cost facilities. In another example of studies to improve program operations, OPHS conducted a Healthy People 2000 stakeholder study to help plan and prepare for the Healthy People 2010 goals and objectives (6491), also featured in Chapter II. This evaluation reviewed the successes and failures of the Healthy People 2000 benchmarks, with the hope of making this national framework of performance indicators on health status more results-oriented. The focus group participants recommended new communication avenues to make Healthy People 2010 available to more professionals and community leaders.

Chapter II. Highlights of Selected Evaluations Completed During Fiscal Year 1997

In this Chapter, the U.S. Department of Health and Human Services (HHS) highlights evaluations of general interest to the public health, health care services, and human services community and illustrates the diversity of HHS evaluations completed in fiscal year (FY) 1997. Included are summaries of 9 evaluation projects selected by the Evaluation Review Panel--on the basis of criteria identified in Appendix C--and applied to 28 reports nominated by HHS agencies. These criteria are as follows:

• Is the report important? Does it address a significant issue or problem for which evaluation would help confirm or change program direction, or measure program impact? Are the findings likely to be useful and generalizable?
• Is the report methodologically sound? Are its concepts, designs, data collection, and analyses conducted and reported in a competent manner?
• Is the report faithful to the data? Do the conclusions and recommendations logically follow from the data and analyses, and are they relevant to the questions asked?

The nine studies are organized under three headings: performance measurement or assessment, program management and development, and policy analysis and development. These headings represent the three most common uses of HHS evaluation resources. Performance measurement or assessment is a high priority for HHS agencies as the development, implementation, and refinement of programs are more results oriented in the 1990's and are required under the Government Performance and Results Act (GPRA) of 1993. Program management and development reflects the kind of evaluation projects that program managers initiate to obtain information or data that will help them manage a program efficiently and ensure successful results. Policy analysis and development includes the evaluation projects conducted by HHS agencies to examine the impact of alternative policies, either in the past or in the future, on the future direction of HHS programs or services.

Each summary includes a brief abstract; a description of the study, including its purpose, background, methods, findings, and use of results; the names of any publications that resulted; and the name and phone number of the HHS official to contact for additional information.

 

Performance Measurement and Assessment

LEAP: Final Report on Ohio's Welfare Initiative to Improve School Attendance Among Teenage Parents--Ohio's Learning,Earning, and Parenting Program

Highlights

The goal of Ohio's Learning, Earning, and Parenting (LEAP) Program was to use financial incentives to increase the proportion of pregnant teenagers and teen parents on welfare who graduate from high school or receive a General Educational Development (GED) certificate, find jobs, and ultimately achieve self-sufficiency. The teens are recipients of cash assistance, formerly under the Aid to Families with Dependent Children (AFDC) program and more recently under the Temporary Assistance for Needy Families program. This report is the final product in a multi-year evaluation of LEAP. Immediate goals were to induce dropouts to return to high school or to enroll in GED programs, and to promote better attendance among those already enrolled when they were called into LEAP. These short-term goals were accomplished--statistically significant increases in school enrollment and attendance were observed. Longer-term goals were to increase the rate of high school graduation, GED attainment, and teen employment, and to reduce welfare payments. These long-term goals were only partly successful. The LEAP program did not increase high school completion rates except for teens who were initially enrolled in school in the city of Cleveland. LEAP increased employment among those initially enrolled in school but not among those not initially enrolled. There were initial impacts on earnings for the initially enrolled group, but earnings were matched by the control group after two years. Welfare payments fell by more than the cost of the LEAP Program. The evaluation of LEAP is timely in light of new changes to national welfare law, which increases emphasis on school attendance by those who have not completed high school (or its equivalent) and on more employment-oriented activities for teens and other welfare recipients.

Purpose

This study concluded a multi-year evaluation of LEAP's effect on school enrollment, attendance, employment, earnings, and reduction of dependency on welfare among pregnant teenagers and teen parents who receive AFDC. This report is based on a 4-year followup that traced the cumulative impact of LEAP on these school- and work-related outcomes in Ohio.

Background

Families started by women who first gave birth as teenagers account for approximately 50 percent of all long-term AFDC recipients. The enactment of the Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA) of 1996 replaced the entitlement to AFDC with block grants to States. Under the law, unmarried custodial minor parents who have not graduated from high school or received a GED certificate may not receive federally funded welfare assistance unless they attend high school or a program that prepares them to earn an alternative education or training credential. Prior to its enactment, however, many States had already implemented some form of school requirement for teen parents on welfare. The LEAP Program in Ohio, initiated in 1989, was one of the first to mandate school attendance for the entire teen parent welfare caseload and to include a rigorous program evaluation. It was hoped that LEAP's direct effects on enrollment and attendance would translate into improvement in other outcomes, such as greater progress in school, more receipt of high school diplomas and GEDs, increased employment, and reduced welfare dependence.

Methods

The evaluation of LEAP uses a random assignment research design and multiple data sources. For this report, 12 Ohio counties were sampled to provide data on 4,151 teens, of whom 3,479 were randomly assigned as program group members, and 672 were assigned as control group members. Teens in the program group who met LEAP's requirements had their welfare checks increased by $62 for school enrollment and an additional $62 for each month of acceptable school attendance. Teens who did not comply had $62 deducted from their welfare grant every month until they complied with program rules. During 1992, a teen living on her own with one child was eligible for a monthly AFDC grant of $274--the bonus or sanction of $62 was a significant amount.

A survey was administered to 1,188 teens 1 year after entry to assess school enrollment status and experience with LEAP. Selected county case files were reviewed to study LEAP's implementation and the bonus/sanction process. A survey of 913 teens in 7 counties was made 3 years after random assignment to measure education, employment, and welfare outcomes. Data on employment earnings and welfare dependency were drawn from unemployment and AFDC records, respectively.

Findings

A statistically significant increase in teen parent enrollment and attendance in school or in a GED program was achieved. Statistically significant increases were also observed in the completion of the 9^th, 10^th, and 11^th grades. However, the longer-term goal of high school completion or its equivalent was not achieved for the full sample. The program increased GED certificate receipt, but not high school graduation rates, among the subgroup of teens who were enrolled in school when they entered LEAP. No statistically significant differences were found in GED certificate receipt or high school graduation among teens who were not enrolled in school when they entered LEAP. High school graduation rates increased only in Cleveland, which may be attributable to an earlier and enhanced LEAP pilot program in the city Cleveland.

Both the program and control groups of teens experienced substantial growth in employment rates and earnings during the 4-year followup period, with quarterly employment rates increasing from about 15 percent to about 40 percent. The LEAP program increased employment for initially enrolled teens, but not for their control group counterparts during the followup years. Earnings of the initially-enrolled students also increased during the first 2 years of followup, but were ultimately matched by earnings in the control group of teens. Rates of AFDC receipt remained high throughout the followup period (more than 60 percent of all teens were on welfare), but declined from 100 percent at the time of random assignment to 60.9 percent in the last quarter of the fourth year. Program group members were somewhat less likely to receive AFDC than were control group members in at least 1 month in most quarters. LEAP reduced both the number of teens receiving AFDC and the amount of AFDC received, with more of the reduction found among those who were initially enrolled in school. Although LEAP was deemed cost-effective, the size and duration of the effects on employment and AFDC receipts may have been suppressed by LEAP's disappointing effect on school completion.

Recommendations for improved policy and program design focused on ensuring case management and well-designed management information systems, improving teens' understanding of the rules and expectations, intervening early to prevent teens from dropping out of school, addressing teens' reservations about school attendance, rewarding academic progress and school completion, and integrating more work opportunities into the program.

Use of Results

This study provides a useful State model for operating new national welfare rules. It suggests that such programs may need to be complemented by incentives to reward academic progress and school completion in addition to school enrollment and attendance. The education message, however, may need to be combined with more employment-oriented activities in order to meet the more work-oriented expectations and time-limited benefit rules that were mandated in 1996. Programs may be more effective if they take into account the specific needs of distinct groups of teen parents. What works for a 17-year-old who is still in school and can complete within 1 year may not work for a 19-year-old who has been out of school for 3 years and has two children. Providing more tailored services and incentives that are meaningful to specific groups may be necessary to improve upon the outcomes observed under LEAP and to attain improvements in the longer term outcomes for these young families.

AGENCY SPONSOR: Administration for Children and Families, Office of Planning, Research, and Evaluation

FEDERAL CONTACT: Nancy Campbell

PHONE NUMBER: (202) 401-5760

PIC ID: 6668

PERFORMER ORGANIZATION: Manpower Demonstration Research Corporation, New York, NY

NTIES: National Treatment Improvement Evaluation Study

Highlights

The congressionally mandated 5-year study National Treatment Improvement Evaluation Study (NTIES) examined the impact of three federally-sponsored drug and alcohol treatment demonstration programs. The study found that the demonstration programs significantly reduced drug and alcohol use, and that treatment had lasting benefits. Clients reported increases in employment and income, improvements in mental and physical health, decreases in criminal activities and homelessness, and modification of sexually-risky behavior.

Purpose

In addition to profiling the services, programs, and clients involved in three demonstration programs funded by the Center for Substance Abuse Treatment (CSAT), the study was designed to address the effects of comprehensive treatment on client access to services and on patient outcomes. Lessons learned about the implementation and cost of delivering comprehensive drug treatment services were to be documented.
 

Background

A comprehensive treatment model assumes that positive and well-sustained outcomes occur when substance abuse treatment systems draw from a full array of medical and rehabilitative services. These services can include primary medical care, psychiatric services, health education, HIV/AIDS counseling, psychological counseling, legal assistance, social and welfare services, spiritual counseling, job training, educational counseling, social activities, and continuing care services. To demonstrate the value of such a comprehensive substance abuse treatment model, CSAT initiated three major demonstration projects to enhance treatment services by building on an existing infrastructure for substance abuse treatment. The three demonstration programs were Cooperative Agreement for Drug Abuse Treatment in Large Cities, Model Comprehensive Treatment Programs for Critical Populations (with special components for adolescent/juvenile justice and public housing), and Model Drug Abuse Treatment Programs for Criminal Justice Populations (for nonincarcerated offenders and those in correctional settings). Recipients of demonstration funding were required to initiate some form of local self-study and to participate in a multisite evaluation (the NTIES) to capture and disseminate lessons learned from these programs.

Methods

NTIES employed a two-level study design to assess the effectiveness of the demonstration programs and a comprehensive treatment approach. The first level was a multiphase survey that addressed treatment unit administration, including treatment orientation, size, budget, and staffing distribution for each of the demonstration programs. The second level involved collecting data on clinical outcomes from clients enrolled in drug treatment at a subsample of the funded sites. This level included three waves of longitudinal data collection. Clients were interviewed at admission to treatment, when they left treatment, and at a followup 1 year after treatment. A total of 6,593 clients at 78 service delivery units were enrolled in the clinical outcomes study. More than 82 percent of the recruited sample completed the followup interview. The outcomes analyses focused on the 4,411 clients for whom followup data were available, along with either an interview at treatment exit or a patient records abstract form.

NTIES, like many other major research surveys, relied primarily on self-reported data. To validate self-reports regarding illicit drug use and arrests, the study also collected data on urine sample drug testing and arrest records. A pre- and post-panel design was used. This method compares behaviors or other individual characteristics in the same research subjects, measured in similar ways before and after intervention. The findings are expressed as percentage of the occurrence of a behavior or circumstances in the NTIES population during clearly defined intervals.

Findings

The study generally found significant reductions in substance abuse during and immediately following treatment. Clients served by the demonstration programs were able to reduce their drug use by about 50 percent for as long as 1 year after leaving treatment. The NTIES data indicate that substance abuse treatment can play a major role in crime reduction. Respondents reported significant decreases in multiple indicators of criminal involvement. The study found that self-reported outcome data on arrests were similar to what might be seen if official records were used, especially for clients interviewed in the community rather than in jails or prisons.

NTIES also explored whether specific patient or treatment unit characteristics could explain variations in pre- and post-treatment outcomes. It found that drug and alcohol use, criminal activity, and employment outcomes were measurably better among individuals who completed their treatment, received more intensive treatment, and were treated longer. The results showed that the length of stay varied across gender, race and ethnicity, and age groups within each type of treatment.

Nearly all clients were involved in the development of their own treatment plans. About 38 percent of the clients said that they received HIV tests, 57 percent received TB tests, and 72 percent received AIDS prevention classes or counseling. Although almost half of the clients needed legal or housing services, only 10 percent were receiving these services. The multivariate analysis showed a significant relationship between the duration of services provided and client satisfaction with the treatment program.

Treatment was determined to be less costly than incarceration. Treatment costs ranged from $1,800 to almost $6,800 per client, while the American Correctional Association estimates the annual cost of incarceration at $18,330.

Use of Results

The study documents that comprehensive and focused substance abuse treatment can make a difference in patient outcomes. The evidence is compelling that the nature and duration of treatment has a positive impact on clinical and employment outcomes and on rates of crime and incarceration. The lower cost of substance abuse treatment (compared to that of incarceration) could have significant implications for the allocation of resources among competing public program priorities.

AGENCY SPONSOR: Substance Abuse and Mental Health Services Administration, Center for Substance Abuse Treatment

FEDERAL CONTACT: Ron Smith

PHONE NUMBER: (301) 443-7730

PIC ID: 5346.1

PERFORMER ORGANIZATION: National Opinion Research Center (NORC), Chicago, IL; and Research Triangle Park, Chapel Hill, NC

Program Management and Development

Self-Sufficiency Project Implementation Manual: Lessons Learned from Eight Years of Office of Community Services Demonstration Partnership Programs

Highlights

This manual summarizes lessons learned during eight years of the Demonstration Partnership Program (DPP) projects. DPP has been developing innovative approaches toward increasing the self-sufficiency of the poor, testing and evaluating these approaches, and encouraging their replication through dissemination of project results and findings. The DPP projects were designed to strengthen the ability of grantees to integrate, coordinate, and redirect activities through community partnerships that promote maximum self-sufficiency among low-income individuals and families who rely on or who are at risk of relying on public assistance. These projects were divided into five program areas and project types: (1) case management, (2) micro-enterprise development, (3) minority males, (4) homelessness, and (5) youth at risk . Generic models for establishing effective community-based programs are presented, and materials for program evaluation are elaborated. For each of the five areas, general and specific lessons learned are presented. A project implementation manual was designed as a step-by-step guide to the successful design and implementation of Self-Sufficiency Projects by Community Action Agencies (CAAs), community-based organizations, and local community program planners.

Purpose

The DPP was designed to permit CAAs to implement and assess innovative approaches toward increasing the self-sufficiency of the poor, including individuals and families who rely on, or are at risk of relying on, public assistance. The purpose of the Self-Sufficiency Project Implementation Manual was to disseminate evaluation findings and information about best practices in the DPP in order to guide and facilitate the future development of each of the five types of self-sufficiency projects supported by the program.

Background

CAAs have been the principal field organizations in the war on poverty for three decades. Since 1981, they have been the primary recipients of Community Services Block Grant funds, dealing with the problems of poverty and attempting to bring poor people up to decent standards of living in economically healthy communities. With the advent of funding for DPP in 1986, CAAs were encouraged by the Congress to add a new dimension to their activities by forming partnerships with public and private entities in their communities and seeking innovative approaches to community revitalization and the problems of poverty. Avoidance of dependency, development of new ways to improve the capabilities of the poor, and overcoming the barriers to workforce entry were central values of the DPP projects. The CAAs have typically served for many years as successful advocates for the poor, forging special bonds with poor people in their communities. As a result of these bonds, CAAs serve as important generators of innovative ideas. The linkage of CAAs with the DPP is intended to build on these special bonds and innovative ideas, and to join them with community partners to develop and experiment with new ways of increasing the self-sufficiency of the poor.

Methods

Development of the implementation manual involved three steps. First, "best practices" were identified by Project Directors and Project Evaluators at a 1994 DPP Reporting Out Conference. These individuals, who were involved in projects funded in 1991 and concluded in 1993, were asked to reflect on their experiences. The evaluators then conducted a more in-depth assessment of what had been learned by the Office of Community Services (OCS) from these projects. In the second step, five team leaders reviewed DPP monographs for each of the five program areas and project types (case management, micro-enterprise development, minority males, homelessness, and youth at risk). The team leaders used quantitative and qualitative information to identify lessons learned and to articulate logic models based on project results and evaluation findings for each program area. In the third step, intensive inputs were obtained from small panels made up of Project Directors and Project Evaluators. These individuals were then convened in focus groups to review and discuss concepts. Team leaders then modified the lessons learned and the logic model based on the discussion. A draft implementation manual was reviewed by experts, project staff, and OCS/DPP personnel.

The results of the study are presented as "lessons learned" in each of the five program and project areas, as well as general lessons that cut across different projects. The Implementation Manual is intended as a brief, step-by-step guide to successful design and implementation of DPP projects. The manual is one of a series of publications developed by OCS to provide technical assistance to CAAs and other grantees. References are provided to other documents in the series for more detailed discussion of evaluation methods, as well as specific project descriptions and evaluation monographs from successful programs.

Findings

The findings indicate that each project is unique because of different environmental factors that affect implementation. Some lessons, however, are generalizable. The findings demonstrate that it is prudent for project directors and managers to learn from prior DPP projects, from literature on promoting self-sufficiency, and from the CAAs' project experiences.

In each of the five project types, the complexities and difficulties of project management are elaborated. This includes basic recommendations, such as the importance of focusing on clients, partner relationships, and staff support. Client focus is essential to ensure that project elements are appropriate for their needs, status, and stage of development toward self-sufficiency. Relationships with partners require particular attention to establish and maintain operational communications and coordinated actions within a noncompetitive environment of cooperation. Staff members are also crucial to success--the selection and professional development of staff should be a well-planned aspect of project management, and communication among staff at all levels cannot be neglected.

Findings also show that each stage of a project requires special attention to ensure success. At startup, emphasis needs to be placed on the physical environment for the safety and convenience of clients and staff. The political environment and issues of competition will require attention, as does the exploration of relationships with other agencies and programs serving the same client group. Although many project managers are tempted to think of evaluation in a post hoc manner, successful evaluation is based, in large part, on careful planning and preparation during the startup phase of a project. Key outcomes need to be framed from the outset in measurable terms.

During the initial operations stage of the project, the design assumptions should be tested and may require adjustment of the model. Client and program operator expectations should be reviewed and tested early in implementation. Organizational management may require special attention as the program structure and staff management issues are being worked out. Findings also point to the importance of beginning to collect evaluation data at this stage.

During the ongoing operations stage, program operators are reminded to be flexible, as past experiences do not necessarily dictate present needs. Staff nurturing becomes increasingly important as the excitement of startup wears off, staff burnout becomes a problem, and additional time is needed to recruit, train, and support volunteers. Evaluation begins to play a greater role as data become available for monitoring and feedback.

At all stages, readers of the Implementation Manual are cautioned not to overlook the complexity of self-sufficiency programs. Logic models are presented for each of the five project types, providing a graphic illustration of the linkages among project assumptions, implementation strategies, and project outcomes, and demonstrating the importance of considering these linkages when managing projects.

Use of Results

Experience-based and empirically supported lessons learned from demonstration projects can be valuable tools for helping to ensure the success of future programs seeking to help low-income families move toward self-sufficiency. The DPP projects offer well grounded insights about the challenges of effective partnerships, about the importance of understanding the logic of the causal relationships between program interventions and expected outcomes and program goals, and about the many complex interactions among program components and program staff. The implementation manual explores the administrative requirements and challenges of successive stages of a project and the critical nature of staff and volunteer training and support, and the profiles of successful self-sufficiency projects attest to the powerful role of evaluation in establishing proper monitoring and feedback loops and quantifiable outcome measures. The manual has already proved to be a valuable tool for staff training in a variety of social agencies, and is being used as a text in at least one university graduate school. It can provide a critical reality check for legislators and policymakers as they strive to design laws and programs that will use scarce resources more effectively to foster self-sufficiency and build sustaining capacity within low-income families and communities.

PUBLICATION: The full report is available from the U.S. Superintendent of Documents, Stock Number 017-090-000-84-4

AGENCY SPONSOR: Administration for Children and Families, Office of Community Services

FEDERAL CONTACT: Richard Saul

PHONE NUMBER: (202) 401-9341

PIC ID: 4336.4

PERFORMER ORGANIZATION: BHM International, Inc., Silver Spring, MD

Effects of Information Tools and Decision Aids to Help Patient Decision-making About Medical Screening and Treatment

Highlights

This study identified, evaluated, and synthesized research on the effectiveness of tools intended to inform patients about their medical choices, their treatment alternatives, and the risks and benefits of those alternatives. These tools include interactive computer disks, videotapes, audiotapes, brochures, and computer-generated fact sheets to help patients make decisions about medical screening and treatment. The research found that although some studies do point to the promise these tools hold for helping patients and consumers make informed decisions, few controlled and comparative studies have been done. Specifically, only a few treatment choices have been examined, and most studies have looked more at the effects of the tools on patient knowledge/satisfaction and less at the effects of the tools on patient-clinician communication and health behavior/outcomes. Future studies need to be better grounded in theory and draw upon the empirical evidence of other relevant disciplines. Four areas have been identified as priorities for future research. These include an assessment of the effects of informatics tools on a wider range of outcomes, identification of factors that influence patient use of these tools, assessment of the effects of tools on patient/clinician communication, and analysis of the cost effectiveness of different types of patient informatics tools.

Purpose

The purpose of this study was to identify, evaluate, and synthesize research on the effects of informatics tools and decision aids on patient decisions about medical screening and treatment. The study assessed the findings, the quality of the research, recommendations for research design, and priorities for future investigation.

Background

Health information and medical care organizations are investing in the development and dissemination of health informatics tools to help patients and consumers make decisions about screening and treatment. These tools provide treatment- and disease-specific health information to patients, especially when they are facing choices about the ways to treat and manage their health conditions. Information about the risks and benefits of making alternative choices is often included in these tools. An example might be the choice between "watchful waiting" and surgery as responses to low back pain or for benign prostatic hyperplasia. Although patient health informatics tools have the potential to empower patients to make more informed choices, there is limited empirical evidence of their value. Many studies have been small and exploratory, and their research designs have not always employed rigorous controls, adequate sample size, or standardized measurements. Further, it is difficult to ascertain what the tools' effects have been on patient outcomes or health care costs. This study attempted to analyze the literature to see if there were any answers to these questions.

Methods

The study reviews both computerized and noncomputerized informatics tools and decision aids, including such media as interactive computer disks, videotapes, audiotapes, brochures, and computer-generated fact sheets. Experts and researchers in the United States, Canada, and the United Kingdom were consulted to make sure that the study captured as much of the relevant existing research as possible, and that the analysis of it was systematic and comprehensive. Both the published and unpublished literature about the effects of patient informatics tools and decision aids were reviewed, with particular attention paid to findings and the research quality.

Findings

Only 30 controlled studies were identified for evaluation and synthesis in this area; the study presents detailed comparative information on these studies. Some of the more substantive findings suggest that the use of informatics tools can increase patient knowledge of treatment alternatives; further patients report that they like the tools. Some studies report effects on treatment choice, but the number of underlying health conditions and range of tools studied are quite limited. Little is known about the operational and background factors that may influence patient use of informatics tools, such as the requirement for a second visit to use them.

Although patients may tend to ask their physicians more questions as a result of using the tools, their effects on broader patient/clinician communication patterns or on time burdens imposed on clinical staff have not been studied. Some studies have shown that information tools can increase patient adherence to prescribed medical regimen and influence patient preference for nonsurgical interventions. The body of literature would improve if future studies assess a wider range of treatment choices, describe the context of implementation, and document the effects of information tools on patient attitudes, treatment selection, health behaviors and outcomes, and costs.

Much work remains to be done to better understand patient health informatics and to ensure that future tools will be developed and supported on the basis of sound health services research. There are many issues to be addressed. Only a few treatment choices have been examined, and few studies have investigated the effects of tools on patient/clinician communications and health behavior/outcomes. There are no comparative studies of the cost and effectiveness of different types of tools, particularly in the case of computerized versus noncomputerized tools. The studies are of varying methodological quality and most have been exploratory. The cost and clinical significance of informatics tools need to be clearly established by more rigorous research designs, including use of randomized control groups, adequate sample sizes, a wider range of measurable effects, and standard outcome measures.

Future research priorities include 1) identifying factors that promote the use of information tools, 2) assessing the effects of tools on the nature and content of patient/clinician communications and related clinician workload; 3) assessing the effects of tools on health outcomes and health behavior, including quality of life; and 4) comparing and examining the cost effectiveness of different types of patient informatics tools for specified objectives.

Use of Results

This study points to the need to conduct systematic and rigorous research on the use of informatics tools at a time when investments in these tools are being made without sufficient knowledge about their effect on patient decisionmaking, health outcomes, and related costs. Suggestions are made for upgrading the quality of future studies to methodological standards found in health services and clinical research.

By organizing information about the state of our knowledge base, the report is intended to help decisionmakers better understand what is known about consumer/patient health informatics tools. Researchers from all fields can use the report to develop future research projects, and especially to identify specific variables that need further investigation. Practical information on how to reach the developers of some of the existing tools is included as well.

AGENCY SPONSOR: Agency for Health Care Policy and Research, Center for Organization and Delivery Studies

FEDERAL CONTACT: Denise Dougherty

PHONE NUMBER: (301) 594-1321

PIC ID: 6376

PERFORMER ORGANIZATION: Research Triangle Institute, Research Triangle Park, NC

Syphilis in the South: A Case Study Assessment in Eight Southern Communities

Highlights

This project provided a comparative case study of local-level syphilis prevention efforts in eight communities in Alabama, Mississippi, South Carolina, and Tennessee. The project focused on groups perceived to be at high risk of becoming infected with syphilis, the extent to which public health activities target such groups, and identification of factors that affect the reach of services to these groups. African Americans were found to be at greatest risk of syphilis infection. The report highlights the cultural, programmatic, and political barriers that restrict the prevention and control of syphilis and other sexually transmitted diseases (STDs) for those at greatest risk. Cultural barriers include restrictive local norms about public discussion of human sexuality, distrust of the public health system among African Americans, and a low priority of health relative to other issues of poverty in the community. The region also suffers from a shortage of providers and facilities, with inconvenient hours of operation, lack of transportation, and few trained minority staff. Women's access to services is especially limited by the need for child care and transportation, and is complicated by comorbidities (domestic violence and substance abuse). Several innovations in STD control and prevention are described. Recommendations include technical assistance to improve community prevention efforts, gender-sensitive programs that deal with the special problems of women, training/recruitment activities, and improvements in electronic data exchange.

Purpose

The purpose of this study was to compare syphilis prevention efforts in southern communities during an epidemic in the Southern States between 1990 and 1992. Case studies of factors affecting the natural history, epidemiology, and management of syphilis in the South were conducted. Specific objectives included development of an improved understanding of service delivery to persons at high risk of infection with syphilis, discovery of innovative syphilis prevention and control measures planned or implemented in Southern States, and production of recommendations for improving community-level prevention strategies.

Background

The syphilis epidemic in the South during 1990-1992 involved the reemergence of an easily diagnosed and readily treatable STD. Syphilis is a systemic disease with an initial acute stage followed by a long period of latency. Transmission occurs through lesions, normally during sexual contact. These lesions increase the likelihood that HIV transmission will occur as well. Syphilis is treated with penicillin, and it is controlled through activities that involve identification, testing, and treatment of exposed sex partners. Historically, African Americans in the South have suffered a disproportionate burden of early syphilis and congenital syphilis. During 1985-1990, rates of syphilis rose 165 percent in this population, while significantly decreasing for all other races and ethnic groups in this region. African Americans represent 90 percent of all reported cases. There is little empirical evidence to explain why syphilis rates are highest in the South and why the disease primarily affects African Americans in that region.

Methods

Case studies were conducted in eight communities in four States (Alabama, Mississippi, South Carolina, and Tennessee). Sites eligible for the study were communities in ten Southern States identified on the basis of high syphilis morbidity during 1990. Criteria for selecting States and communities included consistently high syphilis rates since 1990, or rates that show a decrease suggestive of successful control activities, and known demographic indicators of high syphilis risk (significant number of African-American residents and proportion of households with incomes below the poverty line). Selected metropolitan areas each were paired with a rural counterpart to allow urban/rural comparison of social contexts and public health activities. Background information about each community was gathered. Week-long site visits were held, with interviews of between 40 and 60 public health providers, other providers, and community representatives at each site. Open-ended interviews focused on who is perceived to be at greatest risk for syphilis transmission/

infection, what institutions are best able to reach these individuals, what barriers stand in the way of reaching at-risk individuals, and innovative ideas or activities that STD prevention programs in other locations might find useful.

Multiple interviewers were used, providing an ongoing check for validity and reliability of data collection. A descriptive case study was prepared for each study site using interview data and information from a background document review. The case studies were sent to leaders in the State and local STD programs to supply any missing information, correct misunderstandings, or add comments.

Findings

This study emphasized that a set of dynamic interactions between distinct social scenarios, institutional situations, and persons in the communities placed certain groups and individuals at greater risk for syphilis infection. Of particular importance in these communities is the overarching issue of poverty. African Americans are the demographic group in the South at greatest risk of syphilis infection. The exchange of sex for drugs, especially when related to crack cocaine use, was considered an important risk behavior. Other risk groups include homeless persons, incarcerated individuals, adolescents, and male homosexuals.

Local health departments are the only community organizations that focus directly on syphilis and STD control and prevention. Other organizations that offer STD diagnosis/treatment provide limited partner notification and contact tracing. Public health agencies tended to assign priority to disease control. Schools were a consistent source of STD prevention messages but the content of these messages was limited by local restrictions on sexually explicit material in health education curricula. Churches were effective in delivering prevention programs but faced significant barriers in any discussion that impinged on issues of human sexuality. Community-based organizations have improved the accessibility of clinical services but have done little to reduce the risk of transmission by sponsoring prevention programs.

Barriers to reaching those at greatest risk include distrust of the public health system among African Americans and a low priority of health relative to other issues of poverty in the community. A long history of mistreatment has led to a general distrust of government in many communities. In the South, prenatal and neonatal care are seen as more pressing health problems. Also, cardiovascular disease, teen pregnancy, diabetes, and cancer are given a higher priority by the African-American community. The region suffers from a shortage of health care providers and facilities, a lack of transportation, and a lack of trained minority staff. Access to services for women are particularly constrained by the need for child care and transportation, inconvenient hours of operation, and the complications caused by comorbidities, such as domestic violence and substance abuse.

Some innovative measures to improve syphilis prevention and control programs have evolved. These include training in screening and referral for agencies that are in contact with persons in high-risk situations, involvement of parents and churches in supporting school health prevention messages, and use of key local community members to serve as local experts and resources on health promotion issues (following the "natural helper" model).

Recommendations include the provision of technical assistance to State and local health departments seeking to improve community involvement in syphilis and other STD prevention efforts. The issue of distrust of the public health system among African Americans needs to be confronted directly, with open dialogue as a necessary condition for improvement. Creative adjustments are needed to overcome barriers to the utilization of clinical facilities, such as more flexible hours of operation and the provision of on-site child care services. STD prevention programs need to take into account the special problems of women. This includes the need to develop child care arrangements, new protocols for handling domestic violence, and strategies for incorporating STD services into the broader array of services offered in women's health clinics. Training in STD prevention/control and cultural competency is encouraged for workers in public health centers, corrections facilities, substance abuse treatment centers, and professional schools. Minority health staff need to be recruited. Finally, enhancements in electronic data exchange and telecommunications support for State and local public health agencies are necessary to improve contact tracing and partner notification.

Use of Results

This study provides a comprehensive overview of problems involved in the prevention and control of syphilis and other STDs in the South. This study has important implications for public health policy development and innovative intervention designs by using an approach that highlights scenarios or conditions for risk of syphilis infection, rather than focusing on the individuals who contracted the disease. The actual and perceived barriers to the use of public health services encompass a wide and complex spectrum of cultural, political, and health system factors. High rates of syphilis in a community are indicative of more than just insufficient public health budgets and point to the need for the integration of disease prevention approaches in the comprehensive planning, policy, and budgeting processes of all organizations providing services to local communities. These results indicate the need to strengthen public health infrastructure and to involve communities and community organizations in the effort to reduce the conditions that place individuals at risk. An action agenda is provided that can help strengthen STD prevention programs and make them more accessible to various at-risk population groups, especially African Americans and women in the southern States. The case studies also provide an overview of innovative ideas that can improve the availability and use of STD-related information and services.

AGENCY SPONSOR: Centers for Disease Control and Prevention, National Center for HIV, STD and TB Prevention

FEDERAL CONTACT: Janelle Dixon

PHONE NUMBER: (404) 639-8344

PIC ID: 6579

PERFORMER ORGANIZATION: Battelle Corporation, Arlington, VA

Stakeholders Revisit Healthy People 2000 to Maximize the Impact for 2010

Highlights

This report presents an assessment by Healthy People 2000 stakeholders on how to restructure the goals and objectives of Healthy People 2000 for the next decade. Healthy People 2000 is a set of national objectives for improving the health of Americans through effective prevention strategies. These objectives form a model framework for results-oriented performance measurements that are relevant to both population- and individual-based health care initiatives. They also serve as benchmarks against which performance can be measured over time. The development of Healthy People 2010 provides an opportunity to reflect on the successes and shortcomings of Healthy People 2000 in achieving these important functions. Healthy People 2000 stakeholders comprise a Steering Committee and Consortium. The Steering Committee consists of representatives from all agencies within the Department of Health and Human Services (HHS). The Consortium consists of representatives from a broad array of 350 organizations, including professional societies, volunteer groups, and trade associations. In addition, State public health, mental health, substance abuse, and environmental agencies are members of the Consortium. Formed in 1987 by the Institute of Medicine, the Consortium is engaged in developing and implementing national prevention objectives.

The report explores new approaches for the development of Healthy People 2010, taking into consideration the many major changes in health care since 1990 and past experience with Healthy People 2000 and the health objectives for 1990. Both the Federal and private sector stakeholders who participated in the focus group review process felt that Healthy People 2010 would benefit from an updated presentation, taking advantage of new information and communication technology and creating a flexible electronic data base on CD-ROM and the Internet, in addition to the reference document. Healthy People 2010 needs to be targeted at an audience broader than the previously targeted public health community.

Purpose

The purpose of this study was to determine whether various stakeholders believed that the Healthy People 2000 document was useful as a framework for results-oriented performance measurement. A review of the successes and shortcomings of Healthy People 2000 in achieving its goals and objectives can facilitate improvement and new approaches in the content, format, and dissemination of its next iteration, Healthy People 2010.

Background

Healthy People 2000 serves as a vision for improving the health of Americans through effective prevention. Its multiple goals, objectives, and priorities were designed to guide health-related decisions and actions by organizations and individuals in the public and private sectors. The development of a similar document for the next decade, Healthy People 2010, provides an opportunity to learn from the experiences of various stakeholders about the use of the Healthy People 2000 document as a standard of comparison and as a framework for results-oriented performance measurement. The creative review of Healthy People 2000 is especially important in light of increased national emphasis on performance and accountability, and various changes in the health care system, such as the growth in managed care arrangements, limits on access-to-care and insurance coverage, and the intersection of public health and personal health care functions. High quality input from the people and organizations who have the commitment and resources to endorse the Nation's health goals, objectives, and priorities, and from those who provide health care services is essential before a new framework for Healthy People 2010 can be developed. Consequently, creative ideas for revision of Healthy People 2000 and thoughtful responses to these ideas were sought from the Healthy People 2000 Steering Committee and Consortium.

Methods

Two major activities were undertaken: (1) a Steering Committee retreat and (2) 7 focus groups, involving a total of 58 participants from various member organizations in the Healthy People 2000 Consortium. Various themes were identified in the process of gathering data, review, and revision. Questions and options were revised based on Steering Committee input and a pilot focus group of Consortium members. The final set of questions for the focus groups dealt with issues involving the extent of a required overhaul of the Healthy People 2000 document; what works and does not work about the functioning of the document; and the implications of changing the document, its priority areas, and the criteria for selecting objectives. Each successive focus group allowed for further review of issues raised in the preceding focus groups. For example, the last focus group included representatives from employers/purchasers and managed care providers in order to address the recurrent theme of the growing importance of business, employers, and managed care.

Findings

Major findings were gleaned from the comments of the Steering Committee and the Consortium focus groups, both collectively and individually. The primary finding was that the Healthy People 2010 document should serve as a voice for public health. Other recommendations were to include the goals of establishing a healthy environment, increasing years of quality life, and making the Healthy People 2010 document relevant to purchasers of health care services. The most prevalent theme in the Steering Committee retreat was the size of the Healthy People 2000 document and the number of objectives and priority areas. Some Steering Committee members thought that the new document should be reduced in size. However, most members felt that local policymakers would want more options to specifically address their circumstances. Others believed that continuity would be ensured if the document were not substantially changed.

Steering Committee members were more willing than Consortium members to make major changes in the organization and content of the document. "Life stages" were considered as an alternative way to conceptually organize the document because it would make Healthy People 2000 more about people and less about health departments. However, many Consortium members believed that most groups who use Healthy People 2000 at State and community levels would find it difficult to change their plans given organizational structure and funding streams. Most Consortium members emphasized other creative changes in the presentation, including taking advantage of the Internet and related technologies and allowing for more flexibility in how the document is organized and accessed. Important themes discussed in all settings were the presentation, comprehensibility, and marketability of the document. Consortium members were concerned about the potential usefulness of the document in communicating with different audiences, including community coalitions, business partners, and the public.

Use of Results

This report provided the opportunity for wide-ranging discussion of Healthy People 2000's impact on the health of the Nation. This discussion helped to identify areas for improvement and modification in the next iteration of national public health goals and objectives, Healthy People 2010. The suggestions for change in content, format, and dissemination of Healthy People 2000 are based on the experience of organizations and people on the front line of public health, health care service delivery, and resource utilization. As such, Healthy People 2010 will benefit from the insight and commitment of those who have not only endorsed national health goals but who have the responsibility to achieve them. The full report is available on the Internet at http://web.health.gov/healthypeople.

AGENCY SPONSOR: Office of Disease Prevention and Health Promotion

FEDERAL CONTACT: Deborah R. Maiese

PHONE NUMBER: (202) 401-5809

PIC ID: 6491

PERFORMER ORGANIZATION: Academy for State and Local Government, Washington, DC

Policy Analysis and Development

Impact of the Medicare Fee Schedule on Teaching Physicians

Highlights

This study assessed the impact of a new Medicare Fee Schedule (MFS) on the revenues of teaching physicians. The new MFS, initiated in January 1992, is based on the Resource-Based Relative Value Scale; it altered the relative prices paid to physicians. Relative payment levels were greatly increased for visits (less intensive relative value units, [RVUs]), and reduced for most types of diagnostic tests and surgical procedures (more intensive RVUs). Relative payment levels were raised in rural areas and lowered in large urban areas. It was hypothesized that teaching physicians would be adversely affected by the MFS because they perform more high-technology procedures and provide comparatively fewer primary care services. In addition, they are located in large urban areas, and often experience customary charges that may be higher than the area-wide historical payments calculated for fee schedule transition. Further, some academic medical practices had not signed Medicare participation agreements and would be affected by MFS limits on balanced billing. The study analyzed all Medicare Part B claims associated with discharges during 1991-1993 from a sample of teaching and non-teaching hospitals. Part B revenues and selected price variables were assessed for different categories of service. No evidence was found to suggest that teaching physicians had been adversely affected by the MFS. A reduction in prices and the quantity of services per admission led to a similar percentage decline for inpatient revenues per admission in all types of hospitals. An increased shift of diagnostic tests to outpatient settings, a secular decline in some surgical procedures, and MFS-related coding changes led to a general decline in inpatient physician service intensity. Teaching physicians altered the mix of services provided, substituting less RVU-intensive visits for high-tech diagnostic tests.

Purpose

The purpose of this study was to evaluate whether and how the MFS had a disproportionate impact on teaching physicians compared to non-teaching physicians. The study also addressed whether physicians in large teaching hospitals, especially academic medical centers, were more affected than physicians in small teaching hospitals.

Background

Methods used by Medicare to reimburse physician services were fundamentally changed in January 1992. A new MFS was initiated, which was based on the Resource-Based Relative Value Scale and which dramatically altered the relative prices paid to physicians. Relative payment levels were greatly increased for visits (less intensive RVUs) and were reduced for most types of diagnostic tests and surgical procedures (more intensive RVUs). In addition, the MFS adjusted payments by a Geographic Practice Cost Index, which took into account the actual costs of physician practice in different parts of the country. This adjustment raised relative physician payment levels in rural areas and lowered them in large urban areas.

It was expected that the redistributive effects of the MFS would be particularly adverse for teaching physicians, who perform relatively more high-technology procedures and less primary care and who are located in large urban areas. Customary charges for teaching physicians also could be higher than the area-wide historical payments calculated for fee schedule transition. In addition, an unknown number of academic medical practices had not signed Medicare participation agreements, and the MFS limits on balance billing could further constrain revenues for some teaching physicians. Prior research had shown that physician responses to Medicare fee reductions will depend on the relative magnitude of income effects, such as an increase in service volume to offset a fall in revenue; and substitution effects, such as an increase in services to non-Medicare patients. Relatively little is known about physician response to Medicare fee increases. It was hoped that higher visit reimbursement levels under the MFS would lead physicians to encourage more contact with patients.

Methods

A stratified random sample of 701 hospitals was selected from Medicare's 1991 Provider of Service file. The strata were defined by teaching status of the hospital, ranging from academic medical centers to non-teaching hospitals. MedPAR records for all discharges from the sample hospitals during 1991-1993 were identified. Using the Health Insurance Claim number of these records, all Part B claims associated with these discharges were extracted.

Key revenue and price variables were constructed for each hospital-year: Part B revenues per admission, RVUs per admission, a price per service index, and a price per RVU. Each of these variables was created for overall inpatient services, and for five separate categories of service: evaluation and management services; high-tech surgical procedures (RVUs of 40 or more); general (all other) surgical procedures; high-tech tests (RVUs of 1.3 or more); and routine (all other) tests.

Findings

There was no evidence that teaching physicians were adversely affected by the MFS. Inpatient revenues per admission fell by the same percentage in all types of hospitals. Revenues fell because of a reduction in prices and the quantity of services per admission. The contributions of price per RVU and RVUs per admission to the declines in revenue were similar across all hospitals. A general decrease of inpatient physician service intensity resulted from a shift of diagnostic tests to outpatient settings, a secular decline in some surgical procedures, and MFS-related coding changes. Teaching physicians appear to have altered the mix of services provided (and/or billed for) by substituting less RVU-intensive visits for high-tech diagnostic tests. As a result, the actual decline in price per RVU was similar for teaching and non-teaching physicians.

Use of Results

The study documents the successful impact of Medicare's new RVU-based physician payment system on overall revenues and the substitution of less costly, more visit-oriented services for high-tech diagnostic services. The methods used will help to assess substitution and income effects associated with physician responses to future Medicare price changes and should result in more accurate estimates of Medicare program savings. The results suggest that price decreases will be associated with increasing volume but decreasing intensity per service and, thus, are important for monitoring future volume changes and determining the appropriate level of sustainable growth in physician services.

AGENCY SPONSOR: Health Care Financing Administration, Office of Strategic Planning

FEDERAL CONTACT: William Buczko

PHONE NUMBER: (410) 786-6593

PIC ID: 6666

PERFORMER ORGANIZATION: Center for Health Economics Research, Waltham, MA

Informal and Formal Kinship Care

Highlights

This study used existing national data sources to describe the characteristics of children in kinship living arrangements and to define recent trends in the pattern of kinship caregiving. Comparisons were made between formal kinship arrangements, in which foster care were provided under State auspices, and informal kinship arrangements, in which care was provided by relatives in the absence of a parent--but without State intervention. Over 2 million, or 3 percent, of all children in the United State are in kinship care (defined for study purposes as a child living in a relative's household without a parent present). Kinship caregiving and the children living in single-mother families both appear to be adaptations to family disruption and the decline of the traditional two-parent family. The majority of children in kinship care are white. However, African-American children are four to five times more likely to live in kinship care settings than white non-Hispanic children. Kinship care providers tend to be older than parental caregivers (about two-thirds of care providers are grandparents), and more likely to be poorly educated, out of the labor force, below or near the poverty level, and recipients of public assistance programs. Persons caring for their related children tend to be an economically and socially marginal class. Formal kinship care tends to be a creation of local policy, agency practice, and public administration. Children in formal kinship care tend to be under 5 years of age, African American, and highly urban. They are likely to draw more public attention than the generally white, older, and non-metropolitan children in informal kinship care. Several data issues are addressed in the report, as is the potential influence of changes in the welfare system on the future role of kinship care.

Purpose

This study used existing national data sources to describe the characteristics of children in kinship living arrangements in the United States, to identify recent trends in the pattern of kinship caregiving, and to compare formal and informal kinship care arrangements.

Background

The foster care population has grown by over 60 percent during the past decade. This increase has been attributed to increased reporting of abuse and neglect, the spread of crack-cocaine addiction, increased levels of poverty, reduced caseworker staffing, increased duration of stay, and court-ordered reform. Kinship foster care has also grown during this period, rising to 54 percent of all foster care in Illinois, 45 percent in California, and 36 percent in New York in 1993. In 1994, 3.1 percent of all children were estimated to be in kinship care. Children in informal kinship settings (provided by relatives in the absence of a parent, but not as foster care under auspices of the State) are of particular importance for the child welfare system as an alternative model of caregiving and as a potential "feeder" population for more formal arrangements provided by the State. Recent welfare changes may further accelerate the growth in kinship care arrangements. More information is required, however, about trends in the demographic and socioeconomic characteristics of national and State patterns of formal and informal kinship care.

Methods

Data on national patterns and trends in kinship care are drawn from 12 years of the Current Population Survey (CPS). State data on living arrangement patterns come from the 1990 Census of Population. Administrative foster care records in California, Illinois, New York, and Missouri provide a breakdown of children living with relatives into separate categories of formal and informal kinship care. These data allow for aggregate State analysis and for analysis of "primary urban places" (such as Chicago and Los Angeles County) in comparison to the rest of the State. In the State of Illinois, individual-level records were examined for all recent (1990-1995) child AFDC grant recipients and all foster children in the State. These data were accessed from the Illinois Child Multi-Service Data Base and allowed analysis of transition in foster care arrangements. Study variables included race/ethnicity, age, gender, poverty status, employment status, marital status, geographic area, family structure, characteristics of caregiver, and type of kinship care.

Findings

National trend analysis indicates that children in kinship care are disproportionately made up of minority children and cared for by relatives who are much older, less educated, poorer, and generally with fewer advantages than parents who live with their own children. Living arrangement patterns by State indicate that kinship care levels across States tend to vary strongly and inversely with the proportion of children living within a traditional two-parent family structure--kinship care appears to be a direct product of higher levels of social disruption and family disorganization. Analysis of administrative foster care records shows that African-American children are about eight times as likely as all others to be in formal kinship placements. This racial effect holds across regions and age groups. Informal kinship care is much more common than formal kinship care, which tends to be concentrated in urban places. In the four States studied, 15.5 percent of all children in kinship living arrangements were in kinship foster care (the range among States was 2.7 to 22.5 percent). The remainder were in informal arrangements.

Analysis of AFDC grant recipients and all foster children in the State of Illinois indicates that formal kinship group is younger, disproportionately African American, and living in Chicago. Most children remain in their current status over time. AFDC children living with relatives are about twice as likely to move into formal kinship care as are AFDC children living with their parents, although the likelihood of such a change is less than 2 percent per year. Children who move between different AFDC and kinship settings tend to be younger.

Several data issues were discussed, including the importance of individual-level data in order to control for race/ethnicity and poverty status; the difficulty of clearly defining family relationships, such as the presence or absence of a child's parent, in survey data; and the need for large population surveys in order to obtain a sufficient number of kinship care cases. Recommendations included the maintenance of baseline information on kinship care, more detailed information on the living arrangements of children, conducting similar data analysis for additional States, and creating linked and integrated information resources describing the full range of children's contacts with social services and other public systems.

Use of Results

The descriptive analysis of kinship care arrangements can inform national and State policy on welfare reform and family support. These analyses informed debate on sections of the Adoption and Safe Families Act (Public Law 105-89), which includes provisions on kinship care. They also will be used to prepare a report to the Congress on kinship care and in the work of a Federal Advisory Committee on Kinship Care ordered by that law. This study and related work also will be used to better understand caseload shifts in safety-net programs following welfare reform and their effects on children and families. Variations in the size and prevalence of the kinship care population are related to stresses in the social conditions of families. The number of children in kinship care might serve as a barometer of the impact of social and economic problems on family structure. Welfare changes are likely to affect the transition of children between various living arrangements, both formal and informal. Information is needed to track the family status of children at risk and the types, timing, and reason for changes in kinship care arrangements.

AGENCY SPONSOR: Office for the Assistant Secretary for Planning and Evaluation, Office of Human Services Policy

FEDERAL CONTACT: Laura Feig

PHONE NUMBER: (202) 690-5938

PIC ID: 6016

PERFORMER ORGANIZATION: University of Chicago, Chicago, IL; and Urban Institute, Washington, DC

Market Barriers to the Development of Pharmacotherapies for the Treatment of Cocaine Abuse and Addiction: Final Report

Highlights

This study analyzed the market barriers to the development of drug therapies for substance abuse and addiction, and for cocaine use and addiction in particular. Using market analysis, quantitative modeling, case studies, and industry interviews the study concluded that the development of a new cocaine pharmacotherapy was not economically viable for the pharmaceutical industry under current market conditions. Three critical market barriers faced by the industry included the small and uncertain market for cocaine addiction and abuse drug therapy, a substance abuse treatment system that limits access to this market, and limited and uncertain payment. Public strategies to lower these barriers include government funding of a considerable portion of new drug development; expansion and enhancement of the substance abuse treatment systems; creation of a guaranteed market, such as through purchase orders for minimum volumes of a medication; and extended market exclusivity, which would tend to give the therapies an "orphan drug" status.

Purpose

The purpose of this project was to analyze market barriers to the development of medications for substance abuse and addiction, with emphasis on cocaine abuse and addiction. Information was to be provided on the characteristics of the market for substance abuse pharmacotherapies, real and perceived market barriers, case studies of pharmaceutical companies that have developed and marketed substance abuse medications such as LAAM (levo-alpha-acetylmethadol hydrochloride) and naltrexone, and the industry's perception of the readiness of the science base for the development of substance abuse pharmacotherapies.

Background

Pharmacotherapy offers a means for improving the treatment of drug addiction. However, only a handful of medications have been developed and approved by FDA for treatment of opiate addiction during the past 30 years. A 1995 report by the Institute of Medicine on the development of anti-opiate and anti-cocaine medications found a reluctance on the part of the pharmaceutical industry to enter the field of anti-addiction products because of considerable market disincentives. These included an inadequate science base on addiction and the prevention of relapse, uncertain treatment financing, lack of specialists for treatment of drug addiction, Federal and State regulations, market size, pricing issues, societal stigma, liability issues, and difficulties in conducting clinical research.

Methods

A market analysis was performed for a prospective cocaine medication from the perspective of a pharmaceutical company. Data were compiled on the prevalence/incidence of opiate and cocaine addiction, current rates of treatment, and existing patterns of service delivery/financing for drug treatment. A market analysis model was designed to determine the expected profitability of a particular product. This analysis incorporated development cost, product sales, manufacturing/distribution costs, and the product/sales life. Financial indicators such as product net present value and peak annual revenue of the product were computed. Quantitative modeling was done based on hypothetical scenarios of company decision-making.

Case studies were conducted for the selected pharmacotherapies of LAAM, naltrexone, clozapine, and Nicorette. Literature reviews involved direct search of computerized data bases on clinical trials, product development, and marketing. Five private firms were interviewed to explore and characterize their views on market barriers and the readiness of the science base in developing medications for substance abuse and addiction. A detailed discussion guide served as an interview protocol.

Findings

Market analysis indicated that a combination of factors minimizes the attractiveness of the cocaine abuse market to the pharmaceutical industry. These include a relatively modest potential market (2.1 million heavy users), a low proportion of users currently in treatment (250,000 enrollees on any given day), concerns about compliance in this population, and apparent market expectation of a low price point. Anticipated low market penetration is reinforced by a substance abuse treatment model that generally emphasizes psychosocial rather than medical intervention. The treatment system involves little or no physician time in the treatment of patients. High price sensitivity of treatment programs subject to annual government appropriations has contributed to the disappointing market experience of two substance abuse medications, LAAM and naltrexone. Because the average treatment cost is $9.00 per day for non-intensive patients, which would be consistent with the great majority of all cocaine abuse patients, a cocaine medication price at a daily dose of a few dollars would represent a significant proportionate cost increase.

Quantitative analysis of various scenarios of corporate decisionmaking and market conditions indicated that the prospects for developing a new medication for cocaine abuse and taking it through a full product development cycle do not appear favorable given a moderate wholesale price and the expected market penetration. Financial returns are not sufficient to justify the participation of larger pharmaceutical companies. Case studies of selected medications documented the significant market barriers to pharmacotherapies for substance abuse and addiction, and the important role played by favorable government interventions--FDA fast-track approval (LAAM, clozapine, and Nicorette), modified phase IV clinical trial requirements (ReVia), market exclusivity (orphan drug status or other market protection for all four drugs), and mandated Medicaid coverage (clozapine).

Three market barriers would have to be reduced or eliminated in order to make new drug development attractive to pharmaceutical companies. One barrier is the small and uncertain market for cocaine addiction and abuse medication, given patient compliance problems, limited access to patients, and non-physician treatment managers who oppose the use of drugs to treat substance abuse. A second barrier is that the substance abuse treatment system limits access to the market, partly through the "anti-medication" climate among publicly-funded treatment staff. The third barrier is limited and uncertain payment for pharmacotherapy, given that substance abuse services continue to be subsumed under mental health benefits of shrinking entitlement programs, and given that high price sensitivity/resistance may limit market size. Pharmaceutical company interviews also indicated disagreement about the readiness of the science base for cocaine pharmacotherapies.

Possible government actions to overcome market barriers and to improve opportunities for developing cocaine addiction medications would include funding of a considerable portion of new drug development, expansion and enhancement of the substance abuse treatment system, guaranteed markets or purchase orders for minimum volumes of a medication, and extended market exclusivity via orphan drug status.

Uses of Results

The study complements the findings of the Institute of Medicine's earlier report. It tests various market assumptions to confirm the lack of economic viability for the development of substance abuse treatment medications and the importance of government intervention to lower critical market barriers if development and availability of such medication is deemed a desirable public policy goal. The results provide public decisionmakers with valuable information to weigh the potential advantages and disad-vantages of alternative policy actions in this area.

AGENCY SPONSOR: Office of the Assistant Secretary for Planning and Evaluation, Office of Health Policy

FEDERAL CONTACT: Kevin Hennessy

PHONE NUMBER: (202) 690-7272

PIC ID: 6694

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

Chapter III. HHS Agency Evaluation Activities

Each of the 11 agencies of the Department of Health and Human Services (HHS), the Office of the Assistant Secretary for Planning and Evaluation (OASPE), and the Office of Public Health and Science (OPHS) in the Office of the Secretary maintains its own evaluation program--including the functions of evaluation planning and policy review, quality assurance through technical review, project coordination and management, dissemination of reports, and utilization of results. This Chapter provides an agency-by-agency summary of the evaluation activities. Each summary begins with an overview of the relevant evaluation program, describing its philosophy, policies, and procedures. Next is an overview of the major evaluations completed in fiscal year (FY) 1997 and the evaluations that are currently in progress. Abstracts and contact persons for all evaluations completed in FY 1997, organized by agency, are presented in Appendix A. Listings of all evaluations in progress (by agency) are located in Appendix B.

Administration for Children and Families

MISSION: To promote the economic and social well-being of families, children, individuals, and communities.

Evaluation Program

The Administration for Children and Families (ACF) administers a broad range of entitlement and discretionary programs, including income maintenance (Temporary Assistance for Needy Families [TANF]); child support; children and family services (Head Start, Child Welfare, Family Preservation and Support, and youth programs); four block grants; and special programs for targeted populations, such as the developmentally disabled, immigrants, and Native Americans.

The objectives of ACF's evaluations are to furnish information on designing and operating effective programs; to test new service delivery approaches capitalizing on the success of completed demonstrations; to apply evaluation data to policy development, legislative planning, budget decisions, program management, and strategic planning and performance measures development; and to disseminate findings of completed studies and promote application of results by State and local governments.

ACF actively engages with other Federal agencies, State and local policy and program officials, national organizations, foundations, professional groups and practitioners, and consumers to stay current on emerging issues affecting its programs and to identify questions for evaluation studies. Systems changes and how they affect vulnerable populations, particularly children, are of primary concern. The movement toward devolving responsibility for health and human services to State and local organizations--in particular, the Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA) of 1996--offer both tremendous opportunities and unprecedented challenges in redefining and implementing services for families. Evaluation study designs are negotiated carefully with the States and other interest groups. Studies often are funded as joint ventures with Office of the Assistant Secretary for Planning and Evaluation and other Federal agencies and foundations. Such collaborations permit large-scale efforts that are better informed and more representative of varying perspectives. Proposals are reviewed by multidisciplinary experts. Work groups of various kinds are used to monitor the progress of projects and to advise on design refinements and the presentation of findings.

Summary of Fiscal Year 1997 Evaluations

ACF's evaluations are linked closely to its two strategic goals:

• To measurably improve the economic independence and productivity of families by reforming the welfare system and by stimulating the changes in attitude and behavior necessary to achieve results; and
• To increase the number of children, youth, and families who have improved health, development, and well-being and live in stable communities.

During FY 1997, ACF completed 19 evaluations--4 pertaining to the economic independence and productivity of families, and 15 related to the healthy development of children and families.

Economic Independence and Productivity of Families

LEAP: Final Report on Ohio's Welfare Initiative to Improve School Attendance Among Teenage Parents--Ohio's Learning, Earning, and Parenting Program (6668), highlighted in Chapter II, found that (1) even the initial implementation of LEAP was a considerable achievement as an unprecedented statewide effort requiring new linkages between county welfare and education systems; (2) the program achieved smooth operations after implementing a sophisticated statewide public assistance computer system; and (3) to a large extent, LEAP met its initial goals by significantly increasing school enrollment and attendance for in-school teens and dropouts (these outcomes were directly linked to the incentives and sanctions provided); (4) in general, the impacts on school completion and employment were more positive for teens who were still in school when they were identified as eligible for the program (initially-enrolled participants) than for dropouts who returned to school; and (6) LEAP increased the employment rates for initially-enrolled teens and had positive impacts on their earnings during the first 2 years of the 4-year followup.

Responsible Fatherhood: An Overview and Conceptual Framework (5981) found that fathering, even more than mothering, is influenced by contextual forces in the family and the community. A father in an adversarial relationship with the mother is at risk of becoming an irresponsible father, as is a father who lacks adequate employment and income. Responsible fathering can be fostered by positive changes in cultural, economic, institutional, and interpersonal influences. The report contends that fathering programs should involve a wide range of interventions, including (1) involving the mother where feasible and, for unmarried fathers, the families of origin; (2) promoting collaborative co-parenting; (3) emphasizing critical transitions, such as the birth of a child and divorce of the parents; (4) dealing with employment, economic issues, and community systems; (5) promoting father-to-father learning; and (6) promoting the visibility of committed and collaborative marriage. By developing a theoretical underpinning to guide empirical research, program development, and program evaluation, this project is intended to help inform policymakers about what is necessary to enable a father to support and nurture a child.

The Self-Sufficiency Project Implementation Manual: Lessons Learned from Eight Years of Office of Community Services Demonstration Partnership Programs (4336.4), featured in Chapter II, summarizes lessons learned during eight years of Demonstration Partnership Program (DPP) projects. DPP's purposes have been to stimulate the development of innovative approaches to increasing the self-sufficiency of the poor, to test and evaluate these new approaches, and to encourage their replication through dissemination of project results and findings. These projects were designed to strengthen the ability of grantees to integrate, coordinate, and redirect activities through community partnerships that promote maximum self-sufficiency among low-income individuals and families who rely on or are at risk of relying on public assistance. The report divides the projects into five program areas: (1) case management and family development, (2) micro-enterprise and self-employment, (3) minority males, (4) homelessness, and (5) youth at risk. Generic models for establishing effective community-based programs are presented, and materials for program evaluation are elaborated. For each of the five program areas, general lessons learned are presented, followed by specific lessons learned pertaining to project design, project start-up, project operations, and project evaluation. The project implementation manual was designed as a step-by-step guide to the successful design, implementation, and evaluation of self-sufficiency projects by community action agencies, community-based organizations, and local community program planners. It is also meant to serve as a critical reality check for legislators and policymakers as they strive to design laws and programs that will use scarce resources more effectively to foster self-sufficiency and build sustaining capacity within low-income families and communities.

The Arkansas Prenatal and Postnatal Paternity Acknowledgment Project (6221) used five basic models: (1) basic in-hospital paternity acknowledgment with a genetic testing option; (2) prenatal and multiple postnatal opportunities to acknowledge paternity; (3) interstate paternity acknowledgment transfer project (i.e., parents from one State traveled to another State, gave birth, and signed paternity acknowledgments in that State); (4) home-visiting nurse program; and (5) alternative parental support program, which abates or reduces child support payments for young fathers who participate in opportunities to increase parental responsibility. First, the study found that mothers, identified by child support staff to be uncooperative in establishing paternity, stated that they did not want to establish paternity because either the child's father was not involved and that the father gave the mother money already, or that the mother did not know where the father was living. By the end of the project, 13,688 paternity acknowledgments had been received, and the ratio of paternity acknowledgments to unmarried births ranged from 48 percent in 1995 to 56 percent for the first 3 months of 1996. The study also found that 38 percent of the acknowledgments were matched to children in the Arkansas child support data base, and 27 percent of these had closed by September 1996. Payments had been received on only 16 percent of matched cases.

Healthy Development of Children and Families

The Descriptive Study of the Head Start Health Component (5849) describes health screening, examination, referral, treatment, and followup procedures across the four health domains of medical, dental, nutrition, and mental health. The researchers reviewed the case records of 1,200 4-year-old children enrolled in a national sample of 40 randomly selected Head Start programs and interviewed parents regarding their child's health status and health service utilization patterns. Through interviews with appropriate Head Start personnel, descriptions were obtained on Head Start program staffing patterns and staff training, the use of community health resources, and barriers to providing health services.

The First Progress Report on the Head Start Program Performance Measures (6693) is the first program assessment using performance measures to promote accountability through the assessment of program quality and outcomes. It describes the development of the performance measures, noting their focus on outcomes rather than on processes. The outcomes that will be measured concern how well the children in the program do, rather than on such things as teacher credentials. The Head Start program also has in place a series of Program Performance Standards-- which define program activities--while the Program Performance Measures define program results. The report describes the background of the Program Performance Measures development, including the conceptual model and data sources. The report includes an account of how the Head Start program will gather data for the measures and establish a "feedback loop" for policy and resource decisionmaking. The report concludes that the performance measures will provide a snapshot of the Head Start's program performance at a given point in time and will allow the program to compare its performance to the past. At the national level, it will allow the program to determine how well it is doing through the production of periodic national Head Start progress reports. This first report will serve as a benchmark against which future performance can be measured and furnishes program accountability information to be used in appropriate decisions.

The National Study of Protective, Preventive, and Reunification Services Delivered to Children and Their Families (3874) assessed the numbers of children and families in the child welfare system who are receiving protective, preventive reunification, out-of-home care, and/or aftercare services. It also obtained national data on the number, types, and dynamics of the services provided. Case record abstracts were completed on a nationally representative sample of 3,000 children and their families served by public child welfare agencies. A subsample was followed for a 9-month period. These findings were compared with findings from a previous study, the 1977 National Study of Social Services to Children and Their Families. The study found that (1) between 1977 and 1994, there was a dramatic decline in the number of children receiving child welfare services, reflecting the evolution of the child welfare system from a broad-based social services system to one primarily serving abused and neglected children and their families; (2) the child welfare system has not evolved into an in-home family-based system from a foster care system, as had been envisioned by the Adoption Assistance and Child Welfare Act of 1980; (3) foster care drift remains a problem, with more than one-third of the children in foster care remaining there for more than 18 months; (4) minority children, particularly African-American children, are more likely to be in foster care placement than to receive in-home services, even when they have the same characteristics and problems as white children; and (5) kinship care does not explain the dramatically longer stays in foster care for African-American and Hispanic children compared to white children.

The Examination of Special Needs Adoption in New York State--Phase III Report: Subsidized Adoptions in New York State, 1989-1993 (4380.1) was initiated in order to pilot a statewide data collection system for special needs adoptions. The study analyzed information on the characteristics of adopted children and their adoptive families, looking at procedural issues and the nature and amount of subsidy payments provided to the adopted child. This was done for the entire population of children being placed in New York by public and private agencies. The Phase III report, one of five phase reports, contains a descriptive analysis of statistics on the number of children placed between 1989 and 1993 in New York State, the level of this support, and the nature of the children's special needs. The report found that (1) 12,858 new adoption subsidies were approved in New York State during the study period; (2) 62 percent of children receiving an adoption subsidy in the State qualified because they were hard to place; (3) personality or behavioral problems were the most prevalent reason for a child's classification as handicapped (about one-fourth of children qualified for a subsidy had physical handicaps or severe medical conditions); (4) more male than female children were adopted with a subsidy in New York State; and (5) about one-half of all special needs children are placed in adoptive homes headed by a single mother.

Eleven evaluations addressed the issue of child maltreatment. First, Child Maltreatment 1995: Reports from the States to the National Child Abuse and Neglect Data System (5387.4) found that (1) in 1995, more than 1 million children were identified as victims of abuse or neglect (a rate of 15 per 1,000 children); (2) about 80 percent of the perpetrators of child maltreatment were the parents of the victims, while another 10 percent were other relatives and about 2 percent were people in other caretaking roles; (3) 52 percent of victims suffered from neglect, about twice as many as were subjected to physical abuse, the next most frequent kind of maltreatment,(25 percent); (3) about 13 percent of victims were sexually abused; (4) more than half of all victims were 7 years of age or younger, while about 26 percent younger than 4 years old (about 21 percent of victims were teenagers); (5) the majority of victims of neglect and medical neglect were younger than 8 years old, while most victims of other forms of maltreatment were older than 8 years old; and (6) 45 States reported that 996 children were known by the child protection services (CPS) agency to have died as a result of abuse or neglect--most of these deaths were children 3 years of age or younger. The report also found that (1) CPS agencies investigated nearly 2 million reports alleging maltreatment of about 3 million children (a rate of 43 per 1,000 children); (2) reports were received from professionals (53 percent), family members (19 percent), friends and neighbors (9 percent), and anonymous sources (19 percent); and (3) nationally, about 36 percent of investigations resulted in a disposition of either substantiated or indicated maltreatment, while 58 percent of allegations were not substantiated.

Evaluation of Nine Comprehensive Community-Based Child Abuse and Neglect Prevention Programs: Cross-Site Evaluation Report (5851) is a synthesis of the site evaluations that were part of ACF's comprehensive community-based child abuse and neglect prevention programs. The purposes of the study were (1) to design and implement a process and impact evaluation of nine comprehensive community-based child abuse and neglect prevention projects; (2) to provide technical assistance to the nine projects in meeting the requirements of the evaluation; and (3) to aid the programs in designing and implementing their own internal program evaluations. The evaluation was conducted in three phases over a 3-year period. The nine projects had up to ten service components, many of which differed across the programs. Therefore, individual experimental designs were developed for each service component. Process and impact data were collected across programs and through a series of studies conducted in each site. All grantees were aided in refining their evaluation and research plans. The report provides a context for understanding the experiences of the nine projects. A literature review, an examination of study methodology, the projects' implementation experiences, an accounting of the study findings, and policy recommendations are offered. The report found several program elements vital to the success of the projects, including (1) an emphasis on community involvement and ownerships, (2) employing a positive approach, (3) starting on a small scale, and (4) implementing a strong evaluation and using it as a program management tool. The report recommends that the National Center on Child Abuse and Neglect (NCCAN) implement several strategies to enhance the success of these kinds of projects, including (1) focusing future grant programs on more narrowly defined target populations, (2) stressing the importance of community involvement, and (3) providing grantees with a research framework and priorities delineating key research questions on child maltreatment and requiring grantees to implement appropriate process and outcome evaluation designs.

Described below are the results of the individual evaluations of the nine comprehensive community-based child abuse and neglect prevention programs. Preventing Child Abuse and Neglect: A Case Study of Family Care Connection (5851.3) describes a replication in several high-risk communities of a preexisting program in Allegheny County, Pennsylvania, using community-based parent education to improve parent-child relationships. Family Care Connection (FCC) expanded services at an existing drop-in center and established four centers in three more at-risk communities. The centers provided such services as respite care, child development, recreation, education, and counseling. An evaluation found that (1) 279 parents had graduated from the parenting classes from September 1989 through June 1991, and that these graduates reported that they would be less likely to use physical punishment with their children; (2) the rate of low birthweight babies decreased after program implementation in two communities; and (3) FCC was successfully established as an institution in the communities through aggressive pursuit of funding and by using staff that were hired and paid by "partner agencies." The report concludes that FCC achieved notable success in institutionalizing its services in the communities in which it established drop-in centers under the NCCAN grant. FCC carefully selected community agencies as partners, and established personal contacts and networking connections with agencies and organizations in its target communities.

The report of Preventing Child Abuse and Neglect: A Case Study of Families First in Fairfax (5851.1) found that (1) the program being operated in Fairfax County, Virginia, at the end of the grant period differed substantially from the one originally proposed: a number of child abuse prevention strategies attempted during the first two years were ultimately discontinued; (2) the greatest barrier to program implementation during the first two years was a lack of specificity on how to achieve the program's mission; (3) during the last two years, program development was dynamic and yielded many effective strategies that were made permanent at the end of the demonstration period; (4) the demonstration program fostered a growth in parenting programs, family and early infant health care programs, neighborhood resource centers, and directories of services made available to ethnic minority populations; (5) the program also encouraged increased collaboration and information-sharing between the agencies and organizations working in the area of child abuse and neglect prevention; and (6) the biggest problems during the 5-year demonstration period were frequent leadership changes and staff turnover.

The report, Preventing Child Abuse and Neglect: A Case Study of Community Coalition Acting for Positive Parenting (5851.2), developed by Temple University's Center for Social Policy and Community Development, found that the project (1) affected three different groups: the community, child abuse professionals, and the University, allowing members of each group to learn from members of other groups and fostering an expanded community cooperation; (2) sparked the institutionalization of community-based efforts to support families and inform the community about its role in preventing child abuse; (3) created a community-based planning and development council or community board through which key leaders can both receive and disseminate information; (4) shifted its attention away from child abuse and neglect to a more positive, family-support approach by fostering Family Life Festivals, library programs, and parent-child activities, which reduced parental isolation; (5) using community-based providers as agents for prevention activities enabled the program to attract more of the target population; and (6) universities that have experience working with social issues facilitate program implementation and lend legitimacy to community-based efforts.

Preventing Child Abuse and Neglect: A Case Study of Community Lifelines Program (5851.4) describes a program developed by the Family Life Development Center of Cornell University in collaboration with the government and human service agencies of Chemung County, New York. The program was designed to alleviate community conditions that lead to isolation, poor self-image, and economic stress, thereby reducing some underlying causes of child abuse and neglect. Community Lifelines Programs (CLPs) were responsible for various program initiatives in the city of Elmira and the rural Van Etten/Spencer school district. The initiatives included Parent Partner Programs (PPPs), parent support groups, activities encouraging better parent-child communication, and family support programs. The report describes the community, the grantee agency, the program design and changes over time, barriers to program implementation, and strategies used to overcome these barriers. It also describes the program effects and efforts to institutionalize CLP components. The report finds that the Elmira PPP was very successful, as was the CLP in Van Etten/Spencer. In Elmira, the PPP fostered better relationships between schools and parents, between parents, and between parents and children. In Van Etten/Spencer, several initiatives succeeded, including a program for parents of teenagers (Parent Lifelines), an after-school program, and a playgroup. However, several other initiatives had little impact or started too late in the demonstration period to be measured. These included the Family Portraits component, the Family Connections Room, and Food Stamps Outreach. The public awareness campaign launched in conjunction with CLP was successful and useful, according to a reader survey.

The report of the project Preventing Child Abuse and Neglect: A Case Study of PARE (Physical Abuse and Neglect Reduction Effort) (5851.5), developed by the Exchange Club Center for the Prevention of Child Abuse in Carolina, Puerto Rico, discusses the environmental factors and indicators of social disruption associated with increased risk for child abuse and neglect in Puerto Rico. It provides a comprehensive overview of all aspects of the PARE program design and operations, as well as evaluation findings. PARE provided intervention strategies in an effort to stop future child abuse while preserving the family. PARE was designed to be family-oriented and to emphasize self-help and volunteer action. The demonstration model emphasized interventions directed towards the individual, the family, and the community, as well as cultural factors. The report describes several interventions, including (1) an interagency task force, (2) a public awareness campaign, (3) respite centers, (4) a life skills curriculum, (5) an educational curriculum for prenatal clinics, (6) a parent aid program, and (7) a parent laboratory. The report finds that (1) the development and implementation of three respite centers pioneered new concepts and approaches to providing secondary prevention; (2) 702 volunteers provided support services and served as role models; (3) the program gained the support of the community, government officials, public and private agencies, the business community, and the media; and (4) the public awareness campaign was broadcast throughout Puerto Rico and reached a large segment of the population. The report concludes that PARE increased child abuse awareness in Puerto Rico, but that more work needs to be done before child abuse prevention becomes a priority for leaders and politicians on the island.

Preventing Child Abuse and Neglect: A Case Study of North Lawndale Family Support Initiative (5851.6) describes a project developed by the Greater Chicago Council of the National Committee to Prevent Child Abuse to test the efficacy of a model that advocates comprehensive prevention strategies aimed at both the community and its residents. The model includes community education, parent education, parent support groups, school-based prevention services for children, and therapeutic services for abused children and their families. The evaluation found that (1) the most successful components of the North Lawndale Family Support Initiative (NLFSI) were the advisory council, the community needs assessment, public awareness, and life skills training; (2) the least effective components were the parent education program, the parent support groups, and therapeutic care--this ineffectiveness was due to inadequate program design, local implementation difficulties, and the grantee's perception that NCCAN required more direct services; (3) the NLFSI benefitted from stable and credible staff that had already established themselves in the community before the program's implementation; (4) although staff possessed strong outreach skills which served them well in the public awareness campaign, but lacked the skills and background needed to implement direct services; and (5) the NLFSI lacked a clearly defined or effectively developed relationship with its grantee Agency and had difficulty in being acknowledged as a full-fledged member of the community, due to its situation on the campus of a community college located at the edge of the target community. The report also discusses barriers to program implementation and efforts to institutionalize the program, and includes several recommendations in key areas.

The report, Preventing Child Abuse and Neglect: A Case Study of Dorchester CARES (5851.7), describes a collaborative project of the Massachusetts Committee for Children and Youth, Inc., Federated Dorchester Neighborhood House, and, by the end of the demonstration period, several other collaborators. The model was designed to demonstrate that through collaboration, advocacy, resource development, education, and services, a community can create opportunities for families to increase their social networks and become more independent and self-sustaining. The report found that (1) Dorchester CARES staff stressed that it was not an agency or a program, but rather a collaborative comprised of member agencies and staff who thought of themselves as a process; (2) the project reflected an ecological approach that combined a psychiatric model and a sociological model (the first was directed toward a "sick parent" and stressed therapy and education, while the second was aimed at the "sick society" and stressed educational and political strategies); (3) over the demonstration period, the program improved family functioning to provide a more nurturing environment for children's development; (4) the program created opportunities for family members to become more independent and self-sustaining and for community members to work together to achieve common goals; (5) the CARES family support program worked by proactively encouraging social networking, increasing nurturing values and skills, and empowering parents to carry out their caregiving roles; and (6) the program was successfully duplicated in several other neighborhoods and was institutionalized in Dorchester.

Preventing Child Abuse and Neglect: A Case Study of Maine Families (5851.8) describes a collaboration between the urban Cumberland County Child Abuse and Neglect Council (CCCANC) and the rural Franklin County Children's Task Force (FCCTF). Other local community, school, and service organizations joined the two county organizations to identify and address the needs of children and their families. CCCANC implemented a school family center, a media program, a public library discussion series, a teenage parent day care and support center, a drop-in laundry program, a parents' speaker and support group, and a parent education program in a low-income housing development. FCCTF implemented a summer reading program, a resource directory, a parent education program for Head Start-eligible families, parent cooperative support and education groups, and a parent mentoring and home visitation program. In both counties, the project supported community events to reduce the isolation of families and to encourage a sense of support and togetherness. The Maine Families Project evaluation was primarily a qualitative assessment of the reactions of program participants and leaders. Major findings were that (1) the project was guided by an effort to structure interventions based upon what parents said they needed, rather than on what project staff or other agency staff believed they needed; (2) focus groups revealed that few parents believed that they were doing an effective parenting job, and most felt isolated from support and in need of information and assistance; (3) the project emphasized involving business and industry in designing and implementing community programming; and (4) collaboration is enhanced by using a businesslike approach.

Preventing Child Abuse and Neglect: A Case Study of I CARE (5851.9) describes a project developed by the Ohio Research Institute on Child Abuse and Prevention to model a prevention program in direct response to the needs and existing resources of the target community in Columbus, Ohio. The conclusions and recommendations were that (1) a prevention model containing all the components specified for this demonstration would be difficult, if not impossible, notwithstanding an annual budget of $200,000; (2) although community-based programs cannot correct all the societal problems leading to child abuse and neglect, it should be aware of their effects; (3) an interdisciplinary task force from the community should be considered mandatory; and (4) the uncertainty of future funding is a major obstacle for demonstration or pilot programs.

Evaluations in Progress

ACF has 71 evaluations in progress, 43 of which relate to the economic independence and productivity of families and 28 to the healthy development of children and families.

Economic Independence and Productivity of Families

Policymakers and program operators recognize that job loss among newly employed welfare recipients is a major barrier to the more general goal of increasing the rate at which recipients obtain employment and leave welfare. A high proportion of welfare recipients who become employed lose their jobs and return to public assistance, often within a few months. The four-site Post-Employment Services Demonstration (5974) addresses this important aspect of a comprehensive welfare-to-work strategy. Demonstrations in Riverside, California; Chicago, Illinois; Portland, Oregon; and San Antonio, Texas; are providing job retention and re-employment services to recently employed Job Opportunities and Basic Skills Training (JOBS) program participants randomly assigned to receive additional services, which will be provided regardless of continued AFDC receipt. Each site will enroll approximately 500 participants.

The Home Visiting Services Demonstration (5980) evaluation is assessing the effectiveness of the Home Visiting Services Demonstration in three sites--Chicago, Illinois; Dayton, Ohio; and Portland, Oregon--in which first-time teenaged parents on AFDC are required to participate in the JOBS program. Those randomly assigned to receive the treatment are assigned a paraprofessional home visitor. The demonstrations are testing whether adding weekly home visitor services to mandatory JOBS programs will result in a substantial strengthening of the effectiveness of JOBS programs in helping young mothers better support themselves and their children while promoting positive parenting and reductions in repeat childbearing.

The Prenatal and Early Childhood Nurse Home Visitation Program--Replication/Dissemination (6757) project allows for data collection and evaluation of an initiative to replicate and disseminate the nurse home visiting model. The model has been tested through randomized trials and found to be effective in improving outcomes for mothers and their children on a number of important outcome measures, including (1) educational attainment, (2) employment, (3) welfare dependency, (4) parenting attitudes, and (5) subsequent pregnancies. The Department of Justice's Office of Juvenile Justice and Delinquency Prevention funded a grant to support technical assistance and training to replicate the model in selected Weed and Seed sites. This project supports the evaluation component of the initiative to test the effectiveness of the replication process within normal operating environments.

The New Hope Project (5982) is assessing the effect of subsidizing work for individuals and families that are currently poor. The project offers participants assistance in finding a job, or finding a community service job if they are unable to obtain employment in the private sector after 8 weeks. Wage subsidies are provided to assure an income above the poverty level. Participants are also offered access to affordable health insurance and child care. The offer is designed to be flexible according to individual circumstances. Benefits are phased out as earnings increase to make a smooth transition.

Capitalizing the Bridge from Welfare to Independence (CBWI) (6755) is designed to expand the employment and training services provided by Goodwill Industries by constructing new centers in four counties in southwest Florida and six parishes in southeast Louisiana. Once constructed, the centers are expected to be self-supporting (i.e., no need for continuing public support) by utilizing the business activity and payroll generated by the donated goods business.

Analyzing the Employment and Wage Patterns of Welfare Recipients (6756) will describe the employment and wage patterns of welfare recipients using data from the National Longitudinal Survey of Youth and from the evaluation of the Post-Employment Services Demonstration, which operated in California, Illinois, Oregon, and Texas. While many studies have looked at the dynamics of welfare receipt and the reasons people leave welfare, few studies have focused on employment patterns among welfare recipients or wage profiles of welfare recipients who obtain employment. The project will address the circumstances of welfare recipients who find jobs, the employment and wage patterns of welfare recipients who find jobs, and what accounts for welfare recipients' employment and wage patterns.

The Ohio Works First Evaluation (6758) is an assessment of the Ohio Works First Program, which is designed to help families obtain employment and become self-sufficient. It includes provisions for completing a self-sufficiency contract, more generous earned income disregards, a 36-out-of-any-60-month time limit, and whole family sanctions. The evaluation focuses on the following topics: implementation of the core set of services; variations across counties in program design and service delivery; policy changes affecting recipients' attitudes and behaviors toward work participation; the response and reaction of local officials to the new policies and governance structure under welfare reform; the effects on recipients' participation in work activities; the effectiveness of work participation activities in increasing employment levels and self-sufficiency; the most effective components of work participation activities; differences among population groups affected by the program; the groups most at risk of exceeding the time limit; and the impact of specific policy provisions (e.g., time limits and sanctions).

The Examination of State Diversion Programs (6759) project is gathering information from all States on the policies and practices that constitute diversion programs and/or activities designed to divert Temporary Assistance for Needy Families (TANF) applications. More detailed case studies will be conducted in five to seven States in order to more fully document actual implementation and operation of the diversion policies and activities. A major focus of the study will be on examining linkages between Medicaid enrollment and diversion programs/activities. The study also will examine whether and how local communities and institutions, particularly traditional safety net providers, are affected by diversion programs or activities. The project will address the following issues: (1) how State diversion programs or activities are being conceived, structured, and implemented; (2) the effects of these programs/activities on participants, particularly with respect to Medicaid enrollment; (3) how local community institutions are affected, particularly by changes in Medicaid enrollment rates; and (4) whether strategies for monitoring changes in Medicaid enrollment rates can be developed based on existing data.

The Evaluation of Community-Based Job Retention Programs (6760) is divided into two phases. The first phase consists of a detailed implementation analysis and short-term outcome findings for approximately 700 participants (employed TANF recipients) receiving various mixes of job retention and post-employment services through five community-based neighborhood service organizations in Pittsburgh that received funding from the Pittsburgh Foundation (referred to as the GAPS program). A common set of services will be provided, but within the context of somewhat different existing service delivery systems among the community-based organizations. The second phase, in the second and third years of the project, will include State-funded job retention programs in addition to the GAPS projects. Issues addressed will include the post-employment services most instrumental in helping newly employed TANF recipients maintain employment, the organizational features important to successful delivery of job retention services, and the appropriate duration of services.

The Family Investment Program (FIP) (6761) project continues the evaluation of a welfare reform demonstration combining program changes designed to ease a family's transition from welfare to work with strict requirements that recipients participate in developing and executing a social contract, the Family Investment Agreement (FIA), under Iowa's employment program. The FIA details the steps a family will take to become self-sufficient and sets a time frame for doing so. Families opting not to develop an FIA or not following through with the self-sufficiency plan outlined in the agreement are placed on a 6-month Limited Benefit Plan, which leads to the complete loss of cash assistance for the following 6-month period.

The State Welfare Reform Evaluation Track 2 (6762), complementing a separately funded evaluation being completed on Iowa's Family Investment Program (FIP), consists of a study of repeat Limited Benefit Plan (LBP) assignments and a study of post-employment services. The study of repeat LBP assignments will gather information on why cases are reassigned, how reassigned cases are affected by the immediate cessation of cash assistance in a second LBP, and the time pattern of their return to cash assistance. The study of post-employment services will describe and compare standard and enhanced services and assess their contribution to a client's progress toward self-sufficiency and off cash assistance. It will also document processes for developing, implementing, and delivering post-employment services.

Neighborhood variation in the availability of public and private social services throughout Los Angeles County early in the process of welfare reform will be examined in the Neighbors, Service Providers, and Welfare Reform in Los Angeles County (6763) project. The project will also study how agencies are adapting to the current and anticipated changes in demand for their services as a result of welfare reform. The focus of the inquiry will be specifically on services directed toward children and families.

The Welfare Restructuring Project (WRP) (6764) continues the evaluation of a welfare reform demonstration in Vermont. A key feature of the demonstration is a time limit that requires some adult recipients to participate in community work experience after 15 months (for two-parent families) or 30 months (for single-parent families) of receiving cash assistance. Other policies of the project include asset and disregard changes, which require minor parents with temporary disabilities to participate in rehabilitation and training programs.

Achieving Change for Texans (6765) is the evaluation of Texas' original welfare reform demonstration. It consists of a policy on recipient responsibilities for immunization of children, school attendance, and adhering to a personal responsibility agreement; a policy on differential benefit time limits based on work experience and the need for education; several policy options for individual development accounts and fill-the-gap budgeting; and a one-county pilot offering a check for $1,000 in lieu of regular TANF with no reapplication for benefits for one year. The major study questions are as follows: How does the differential time limit work, taking into consideration work experience and education for setting the limit's duration? Is TANF-One-Time an effective way to help people with temporary needs without making them recipients? Will personal responsibility measures help TANF recipients become self-sufficient faster? Can policies designed to promote and reward work help the process of self-sufficiency?

The Evaluation of Arizona Employing and Moving People Off Welfare and Encouraging Responsibility Program (EMPOWER) (6818) project will address whether there are significant differences between experimental and control groups with respect to: (1) income and employment, (2) program participation, (3) program duration, (4) family structure (including fertility) and stability, (5) children's well-being, and (6) employer health benefit provisions.

The State Welfare Reform Evaluation Project (Jobs First) (6819), initiated in Connecticut, will look at the following program outcomes: work and earnings; welfare receipt and costs; job-retention and recidivism; family composition and stability; the well-being of children; child support and other sources of income; family income and poverty status; the use and perceived quality of program services and child care; material hardship; and attitudes toward work, welfare, and the Jobs First program. The implementation and operation of the program will be explored to determine how, and how well, it was integrated with other agencies, including staff training, changes in office culture, and changes in case management.

The Florida Family Transition Program (FTP) Evaluation Project (6820) will address the following questions: Is the Family Transition Program (FTP) program being implemented as intended? How do program staff across all components of FTP achieve the multiple goals implicit in the program model? How do welfare recipients respond to the new package of rules and services? Who reaches the time limits? What happens prior to and at the point when time limits are reached, and after benefits are terminated? Does FTP increase the rate at which welfare recipients participate in and complete various education and employment activities? Does FTP lead to increases in employment and earnings? Does FTP reduce welfare receipt? Is FTP cost-effective?

In Illinois, the Youth Employment and Training Initiative (YETI) (6821) is studying whether interventions that focus on inner-city youth from welfare families (1,000 students in 3 high schools randomly assigned to a treatment or a control group) will help participants complete high school, obtain and hold regular jobs, avoid the pitfalls of substance abuse or early parenthood, and achieve economic self-sufficiency. The process study centers on whether such a program can be effectively implemented and operated in inner-city schools. A cost-benefit analysis will determine the cost-effectiveness of YETI for government, participants, and the general public.

The Indiana Welfare Reform Evaluation Project (6869) continues the evaluation of the Indiana Manpower Placement and Comprehensive Training Program waiver demonstration now operating under TANF. The study will examine the following questions: Was the reform successfully implemented? In what ways does the new program differ from its predecessor? What organizational characteristics, service features, and aspects of the demonstration environment determine its impacts? How do these reforms impact benefit payments, income, self-sufficiency, children's well-being, and family stability? What are the consequences of the changes on the size and composition of the welfare caseload under TANF? What trends should the State expect in levels of benefit payments and services needed? What happens to individuals after they leave welfare, especially those who reach the time limit? What are the impacts of time limits? How has welfare reform affected welfare entry?

The Maryland Family Investment Program Evaluation (6823) is a statewide process study to examine and document front-line assessment and allocation practices in the State's 24 local jurisdictions. The research questions to be addressed are as follows: How much discretion do county-level workers have in assessing clients and directing them to services that are likely to address their needs? How does worker discretion vary from one county to another? How do county management differences influence assessment practices? How do county front-line assessment practices and differences affect welfare service delivery? Do assessment practices appear to be appropriate to clients' situations and clearly directed to desired TANF outcomes (e.g., successful employment placement before a time limit is reached) and other measures of family well-being?

An eight-county (four urban and four rural), 17,000-case (randomly assigned to treatment or control group) evaluation of the Minnesota Family Investment Program (MFIP) (6824) is attempting to answer the following major research questions: What are the effects of financial incentives in employment and welfare receipt? What is the effect of mandatory case management in combination with increased financial incentives? What is the effect of case management services timing? What is the extent of client understanding and participation in MFIP? What is the nature of services being provided and the interactions between clients and staff? What are the net costs of the program from a variety of perspectives?

The Minnesota WorkFIRST Program (Track 2) (6825) project is a comparative evaluation of Minnesota's WorkFIRST with the Minnesota Family Investment Program and STRIDE services (MFIP-S). WorkFIRST uses a rapid labor force attachment strategy versus MFIP-S (the State's TANF program), which represents a progressive labor force attachment model. The project will address the following questions: Will the WorkFIRST program have a greater impact on moving first-time applicants into employment and promoting long-term self-sufficiency or will the concept of "make work pay" strategy of the MFIP-S programs prove more effective in moving people off welfare? Is WorkFIRST more effective in preventing first-time applicants from becoming dependent on public assistance? Do tough sanctions and vendor payment reinforce the WorkFIRST message of personal responsibility?

The Evaluation of the Employment First Program (6826) project in Nebraska includes an impact study featuring a random assignment experiment in Omaha of 4,800 recipient and 2,250 applicant cases, and a subsample of 2,000 randomly assigned cases that will be interviewed in a survey. A process study will be conducted in four sites, including Omaha and Lincoln and will include interviews with a subsample of 180 cases 3 months from the date of initial assignment to determine understanding of the program. Another subsample of 350 cases will be used to collect data not available from the State's data system. The case manager study will begin with a case manager background information questionnaire supported by client outcomes data. High-performing case managers will be identified and interviewed to determine the components of their success.

The New Hampshire Employment and Training Program Process and Outcome Study (6827) will examine the planning, funding, and implementation of the State's welfare reform program and its different components at the State and local levels. This study will examine the following questions: What are the allocated funding levels for planning and ongoing program operation? What are the reactions to, and effects of, the various sorts of interagency collaborations around which the program is designed? What activities has the State conducted to promote culture change among the staff, clients, and community partners? What has been the reaction to the up-front diversion and employment aspects of the program? How are staff implementing assessment and program assignment procedures? Overall, what do the assessment procedures help discover about client barriers and needs? What are the outcomes of measures such as employment rates, length of employment, total earned income, child support collection, total family income, and savings?

The Impact Study of the New Hampshire Employment Program (6828) will involve a pre-post non-experimental design consisting of comparisons between baseline and post-implementation cohorts, caseload modeling techniques, and a pre-post time series analysis of trends on selected welfare-related aggregate measures. Answers will be sought to the following questions: Does New Hampshire's welfare reform program have an impact on State welfare case dynamics, including exit rates, length of benefit receipt, and recidivism? Does the program have an impact on welfare caseloads compared to forecasted caseload values such new welfare case openings, case closings, denials and withdrawals, "child only" cases, earned income cases, and benefit payments? Does the program affect welfare-related measures such as foster care placements, child abuse and neglect, homelessness, paternity establishments, and child support collections?

Data gathered in eight counties in the Evaluation of the North Carolina Work First Program (6829) will be used to address several questions: To what extent can the observed differences in client outcomes between counties be attributable to variations in local economic conditions and unemployment rates? To what extent are the intercounty differences attributable to other contextual factors such as variations in health coverage or demographics of the local population? What political and organizational factors may have affected the results for each county? What are the outcomes of Work First in terms of indicators for self-sufficiency, welfare costs, levels of work activity participation, family structure and stability, and child health and well-being?

In North Dakota, the Training, Education, Employment, and Management (TEEM) Project (6830) will continue the evaluation of the TEEM project, which consolidates TANF and the Low-Income Home Energy Assistance Program (LIHEAP) into a single cash assistance program. The TEEM project includes social contracts, increased work incentives, sanctions, raised asset limits, and incentives for family stability and marriage. The study will look at the following questions: How are policies, services, and program rules implemented? How are families and county staff affected by the new policies and programs, and does this vary by subpopulation (e.g., ethnicity) within and between the counties? What are the components of the TEEM screening system? What barriers to employment are addressed? How do client participation rates in welfare programs, including employment/training programs and the Child Support Enforcement program, change with the implementation of TEEM? How well does TEEM serve clients with special needs (such as disabled clients, clients with disabled children, and clients with substance abuse or domestic violence problems)?

The Virginia Independence Program (6831) project, an evaluation of TANF implementation, will study the following policy topics: diversionary assistance, family caps, compulsory school attendance, paternity establishment, minor parent residency, required immunizations, savings accounts, 2-year time limits, personal responsibility agreements, expanded earned income disregards, transitional Medicaid, and child care benefits. The major research questions are as follows: What happens to families after they reach the time limit? What are the effects of post-employment and job-retention services offered by the State? What are the implications of the growing number of child-only cases the State is finding on its welfare rolls? What are the issues around players and localities where unique innovations have been introduced in the State? How does diversionary assistance work in a limited number of localities? What are the dynamics of the Virginia caseload over time, under the TANF regime? Are there social health indicators for TANF recipients other than those required by law that would help the State keep abreast of developments as they affect the caseload?

Welfare-to-Work: Monitoring the Impact of Welfare Reform on American Indian Families with Children (6832) is being conducted in Arizona. The State was selected because it has the largest population of American Indians living on reservations and because the 21 reservations differ on a variety of dimensions. The research questions that will be addressed are as follows: What are the demographic, social, and economic characteristics of Indian families with children on welfare? What are the reservation-based or individual barriers to raising the skills and employment potential of Indian parents on welfare? How have the tribal councils prepared themselves to deal with the consequences of welfare reform? How are tribes who plan to administer TANF independently positioning themselves to undertake this task? Will the benefits and outcomes to families vary significantly depending on whether TANF is administered by the State or the tribe? What strategies are time-limited parents using to attain independence? What proportion of those losing eligibility shift to Indian general assistance programs?

The Assessing Effective Welfare-to-Work Strategies for Domestic Violence Victims and Survivors in the Options/Opciones Project (6833) will document the needs of battered girls and women on welfare and will identify those strategies successfully employed to eliminate violence and to provide alternatives to welfare. The research will focus on services provided through Options/Opciones in partnership with the Illinois Department of Public Aid in one community of North Lawndale. Questions to be addressed are as follows. What are the successful strategies that battered women on welfare use to become economically self-sufficient? What are the service needs of battered girls and women on welfare? Which problems serve as the major welfare-to-work barriers for battered girls and women? What combination of services is needed to address the domestic violence and welfare-to-work issues, and in what order? Did these services lead to labor market participation, and for what percentage of women and under what time frame? What are the characteristics of girls and women who successfully remove themselves from situations containing domestic violence? What are the characteristics of the girls and women who successfully enter the labor force versus those who do not?

The purpose of the Welfare Reform Studies and Analyses (Rural TANF) (6834) project is to conduct an in-depth process evaluation of family response to the implementation of welfare reform in three remote rural counties of northeastern Washington State: Stevens, Ferry, and Pend Oreille. Eighty families (40 experimental and 40 comparison) will be interviewed 5 times within 16 months. The families will be selected from the 340 households currently participating in the Comprehensive Child Development Program and who are also enrolled in TANF and eligible to participate in Washington's Work First program. This study will look at several questions: How do families formulate and act upon strategies to attain economic self-sufficiency in an environment of scarcity? What is the relationship of a family's social network to its effectiveness in formulating strategies and successfully carrying out a plan to implement them? How do the actions of community agencies and organizations carrying out the tasks of welfare reform respond to family strategies?

The Jobs-Plus: Community Revitalization Initiative for Public Housing Families (6835), a 7½-year demonstration program, is aimed at dramatically increasing employment, earnings, and job retention among working-age residents, a large percentage of whom are on public welfare or are at risk of dependency. Aside from Housing and Urban Development (HUD) and ACF, the demonstration is supported by the Rockefeller Foundation and three other foundations. The program supports the planning, development, implementation and evaluation of locally-based approaches to providing saturation-level employment opportunities and supportive services for working-age residents of family housing developments in seven cities. It incorporates a combination of state-of-the-art employment, training and supportive services; financial incentives (occasioned by the welfare reform and new public housing policies); and rigorous efforts to rebuild and strengthen the community in support of work.

The purpose of the Management-Focused Welfare Reform Evaluation Data Access and Policy Manual Development (6836) is to develop (1) a software package that organizes public assistance evaluation data into readily accessible formats that can be continually accessed and used for timely and responsive decisionmaking; (2) a policy analysis manual that assists users in understanding how to read and apply continually accessed outcome or impact data to program or policy decisions; and (3) technical documentation for systems personnel and regular users. The manual will assist the user in bridging the gaps between what outcomes and impacts may appear in the sample, what this information suggests, caveats in interpreting the results, and ultimately, what program or policy responses should be considered.

The Welfare Policy Typology Project (6837) is laying the groundwork for a data base-- containing key information about State welfare policies (including TANF), State-funded maintenance-of-effort programs, and federally-funded child care assistance programs--that would be available to the public, probably through the Internet. With the enactment of the Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA) of 1996, there are few, if any, strict Federal requirements for welfare programs, and States are not required to report their policies in any detail, let alone in a uniform format. This will make research on the effects of welfare reform somewhat problematic, because researchers will not have a ready source of information on State policies. Without a standard compilation and classification of State welfare policies, different researchers are likely to classify policies in different ways, making cross-study analysis virtually impossible.

The Wisconsin Pay for Performance/Self-Sufficiency First Evaluation (6838) project will complete an impact evaluation of a statewide work-focused welfare reform demonstration. Within a research site, current public assistance recipients and new applicants were randomly assigned to either an experimental group subject to the Pay for Performance/Self-Sufficiency First (PFP/SSF) provisions, or a control group subject to the regular program rules according to the State's approved AFDC/Medicaid State Plans and approved Food Stamp Plan of Operations. Several questions will be addressed: Do the program policies improve employment rates, length of employment, amount of earned income, hours worked per month, child support collections, total family income, and accumulated savings? Do these policies affect public assistance participation and program costs? Does the program affect participation in employment and training activities? Do these program rules and policies affect marriage and separation rates and use of foster care? Does the demonstration affect the incidence of reported child abuse and neglect and health insurance status of children?

The Partner and Father Involvement in the Lives of Low-Income First-Time Mothers and Their Children (6839) project is investigating the role that fathers and partners play in improving the material, emotional, and developmental well-being of low-income women and children. The study consists of a set of intensive secondary analyses using data from three longitudinal randomized experiments of a prenatal and infancy home visitation program serving first-time mothers from various ethnic and racial groups. Research questions to be addressed are as follows: What is the natural history of father and partner involvement in the lives of low-income women and children? What is the impact of a prenatal and early childhood home visitation program on partner and father involvement? What is the impact of partner and father involvement on women's life course development (e.g., welfare, dependence, subsequent pregnancies, and mental health)? What is the impact of father and partner involvement on children's health and development (e.g., child abuse and neglect, intellectual development, and antisocial behavior)?

The Evaluation of Los Angeles Jobs-First GAIN (6840) project will examine the results of changing Los Angeles GAIN, a human resource-focused welfare-to-work program, into Los Angeles Jobs-First GAIN, a labor force attachment strategy. Data collection will be primarily from administrative records, with a 2-year survey of approximately 250 experimental and control group cases. The following research hypotheses are expected to be confirmed. It will be possible to convert a large, diverse, mature, education-focused welfare-to-work program to a Work First program. The new program will produce earnings and recipient changes of greater magnitude than the original program. The new program will produce earnings increases and welfare reductions with much smaller expenditures per enrollee than its predecessor. Having profited by experience, Jobs-First GAIN will produce equal or better results than previously evaluated Work First programs. Jobs-First GAIN can effectively operate on a large scale, serving a diverse welfare population. The program will quickly move many recipients into jobs, thereby minimizing the number who reach the TANF time limit unemployed. Finally, the programs' economies of scale will achieve savings for government budgets.

The principal objective of the Front-Line Management and Practice Study (6841) is to evaluate whether front-line workers are implementing the welfare goals and policies established by the States. Using the local TANF office as the unit of analysis, researchers will execute an in-depth process/implementation study relying on observable variables such as assessment processes, allocation of client services, allocation of paid and unpaid work, enforcement of work-related policies, delivery of information, orientation of individual worker practice, office level practices, and implicit messages or signals.

The ACF Office of Community Services (OCS) is supporting evaluations of grantees funded under the Job Opportunities for Low-Income Individuals (JOLI) Program (6430). Assistance is provided to grantees for developing project designs and evaluation plans. For example, the Bridgeport Artisan Center is creating jobs and enterprise opportunities for low-income artisans in the inner-city and surrounding neighborhoods of Bridgeport, Connecticut, an Enterprise Community. Another example is Avenues (Avenidas), a project of the Mi Casa Resource Center for Women in Denver, Colorado, which trains low-income persons--primarily women--for jobs and apprenticeships in highway construction and maintenance. OCS will evaluate the impact of the demonstrations and disseminate the project results to Congress and other interested parties.

During FY 1997, OCS initiated several reports on evaluation results of projects funded under the Demonstration Partnership Program (6883). One report will contain summaries of eight Demonstration Partnership Program (DPP) projects focused on various facets of case management in the delivery of social services. A second report will focus on youth at risk and minority males. A third report will summarize projects in the area of micro-enterprise and self-employment, and homeless individuals and families. In addition, OCS will be examining 21 DPP projects, funded in FY 1994, that are in the final stages of their evaluations.

ACF also has three projects addressing child support enforcement. The first project, the Study of the Impact of the Child Support Enforcement (CSE) Program on Avoiding Costs to Public Programs (6842) will include (1) a literature review on CSE and cost avoidance and an annotated bibliography; (2) a small conference on cost avoidance consisting of experts from State and Federal government, academia, and the private sector; (3) an assessment of national micro-simulation models to study the impacts of the CSE program; and (4) an assessment of State administrative data capacity for measuring cost avoidance.

The second project, Evaluation and Reporting on State Access and Visitation Grant Programs (6843), will help define data elements to report on and evaluate State Access and Visitation Grant Programs. The project will also assist States in implementing data elements and reporting as part of the monitoring requirement.

The third project, Partners for Fragile Families (6844), is an assessment of a program to assist low-income young fathers to work with the mothers of their children to share the legal, financial, and emotional responsibilities of parenthood and improve the child support system's interaction with the fathers of fragile families. The National Center for Strategic Non-Profit Planning and Community Leadership in Washington, D.C., will offer technical assistance to service providers in public agencies and community-based organizations, facilitate dialogue and information exchange among public and private partnerships; and identify models and promote public and private partnerships.

Healthy Development of Children and Families

ACF has eight studies under way that address Head Start issues. The Early Head Start Research and Evaluation National Study (3570) is evaluating the effectiveness of the Early Head Start program in 15 diverse communities in 13 States and the District of Columbia. The study is examining child, family, staff, and community outcomes in a sample of 3,400 children and their families who were randomly selected for program and comparison groups while the mothers are pregnant or until the children are 12 months old. Assessments of the children, families, and child-care environments will made when children are 14, 24 and 36 months of age.

The Profiles of Head Start Families: Development of a Head Start Family Information System (3869) project will (1) provide a national description of Head Start family and child characteristics using family-level data, (2) identify Head Start child and family service needs and services received within a given program year, and (3) provide for the regular reporting of data over time so as to permit analyses of changes in Head Start family and child characteristics.

The Evaluation of the Head Start/Public School Early Childhood Transition Demonstration (4393) project is assessing the effectiveness of providing comprehensive, continuous, and coordinated services to Head Start families and children from the time of Head Start enrollment through the third grade in public school. Working in concert with local evaluators, the contractor is developing a set of common data collection instruments to be used across all sites. The project is providing data regarding the effectiveness of the Transition Project models in maintaining gains by children and families while in Head Start. The evaluation and demonstrations were mandated by the Head Start reauthorization legislation. Other studies completed during the ten years prior to 1992--notably the Head Start Transition Study and Developmental Continuity--did not provide information on services extended to children after they left Head Start.

All 41 Family Service Center Demonstrations are being evaluated through a consortium of local evaluators under the Evaluation of the Head Start Family Service Center Demonstrations (4394) project. The contractor is providing coordination, technical assistance, and analysis on common data elements across sites to yield a coordinated and integrated summary of process and impact evaluations conducted by local evaluators. This consortium approach is ensuring a consistent and technically sound method for evaluating these demonstrations of how Head Start can collaborate with community programs to meet the needs of Head Start families dealing with problems such as illiteracy, substance abuse, and unemployment.

The Study of the Characteristics of Families Served by Head Start Migrant Programs (4974) is documenting and describing the implementation of 26 Head Start Migrant programs, providing a profile of Head Start migrant families in the main migratory streams and generating information on unique issues related to serving migrant families through Head Start programs. It is also documenting the availability and coordination of services for Head Start families during their migration. Finally, it will provide a national estimate of the number of eligible migrant children versus the number of those children being served by the Migrant Head Start programs. Findings from the study will be used to inform future policy decisions on Head Start migrant programs, as well as the new Early Head Start program for infants and toddlers.

The Evaluation of Head Start Family Child Care Homes (4975) project is assessing the effectiveness of the 18 Head Start Family Child Care (FCC) Homes demonstration projects funded by the Administration on Children, Youth and Families (ACYF) in FY 1992. The project is assessing the quality of Head Start services provided in FCC homes with regard to Head Start Program Performance Standards. The project is comparing services delivered in FCC homes to those delivered in Head Start centers. Children are randomly assigned to the experimental and control groups. Data is being collected on the cognitive, socio-emotional, and physical development of the two cohorts of children participating in the study.

The Descriptive Study of Head Start Bilingual/Multicultural Program Services (5845) aims to identify the number, geographic distribution, and sociodemographic characteristics of the Head Start-eligible population for different cultural and linguistic groups by region and nationally. It will describe the bilingual and multicultural children currently served by the Head Start program. The range of bilingual and multicultural services provided by the Head Start program nationally will be compiled using existing data and a survey of all Head Start programs. Finally, the project will assess the service models, staff training approaches, community partnerships, and administrative plans and processes of 30 Head Start programs that have developed innovative methods for addressing the unique program needs of one or more of the diverse cultural and linguistic groups.

A 3-year Descriptive Study of Families Served by Head Start (6331) will provide crucial policy-relevant information on a nationally representative sample of families served by Head Start in 40 programs across the country. Through a survey and more intensive case study methods, the project will chart family demographics, strengths, needs, expectations, and experiences within Head Start programs, as well as programmatic efforts to join in partnership with families.

The Evaluation of the Comprehensive Child Development Programs (3868) is an assessment of the effectiveness of the first group of 21 Comprehensive Child Development Program grantees. In particular, the study will examine (1) the longitudinal impact of each program model on the development of the participating children and their parents, (2) the effectiveness of the programs in achieving their stated objectives, and (3) the impact of related programs on the delivery of services.

The National Study of Low-Income Child Care (6845) is studying the low-income child care market in 25 communities in 5-7 States with a substudy to examine the license-exempt family care market in 5-7 neighborhoods drawn from these communities. It will provide essential information to help inform the issues surrounding subsidized child care and its implementation by the States, with particular reference to the provisions in the Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA) of 1996 over time. It will further examine how significant shifts in welfare policy and programs affect the child care market for welfare recipients and the working poor at the community level. It will study the impact of subsidies on the types, amounts, and quality of child care available; children's child care placements; and family decisionmaking. Additionally, it will report on the largely unknown license-exempt family care market for both preschool and after-school cohorts.

In the Role of Child Care in Low-Income Families' Labor Market Participation (6846), optional research designs to identify and address child care services needed by parents to succeed at work will be developed. A series of stand-alone working papers will be prepared that critically evaluate relevant research related to child care and labor force attachment and that develop the rationale for the factors included in the research designs. Priority outcomes to be incorporated in the research designs include employment, welfare participation, children's development, family well-being, and child-care quality.

The Child Impact Studies (6847) project will augment welfare reform demonstration evaluations in five States (Connecticut, Iowa, Minnesota, Indiana, and Florida) to assess the effects of different welfare approaches on children's well-being. The project adds detailed data on children to these evaluations. At each site, treatment groups subject to welfare reform policies will be compared with control groups subject to former AFDC policies. Child outcome data, focusing on ages 5-12, will be collected through surveys and administrative records. The study will address two questions. What are the effects of alternative approaches to welfare reform on child well-being, including school achievement, behavioral problems, and health status? What intervening mechanisms--such as quality and regularity of the home environment, child care arrangements, and parental employment and income-- will affect these outcomes?

Three projects address the issue of homelessness. The first, Evaluation of the Impact of Homelessness on ACYF Programs (4396), will conduct site visits and collect information from service providers for the homeless, administrators, and representatives of local service delivery networks in 40 communities. The project will examine programs supported by ACYF, but will also review non-ACYF-sponsored programs. In addition, the project will conduct a small-scale longitudinal study of homeless people and the impact of services upon them. Five local programs serve as case studies. The results of the review will focus on key strategies for increasing the effectiveness of ACYF programs and on measures to help reduce people's risk of homelessness.

The second, Evaluation of the Transitional Living Program for Homeless Youth (4397), will examine the number and characteristics of homeless youth served by this program. The project will examine the effectiveness of the transitional living program in alleviating the immediate problems of homeless youth and its success in preparing homeless youth for self-sufficiency and in helping them decide upon future education, employment, and independent living. The ability of the program to strengthen family relationships and to encourage intrafamilial problem solving through counseling and developing self-sufficient skills will be assessed.

The third, Evaluation of Runaway and Homeless Youth Programs--A Follow-Up Study (4361), will examine the impact and effect of the Runaway and Homeless Youth (RHY) shelter program on youth and review the policy, program, and service delivery issues that impede or facilitate RHY shelter goals. The study will obtain information from 23 sites on runaway and homeless youth shelters and services and assess the impact and effect of the RHY program from the perspective of program administrators and service providers. It will identify the policy, program, and service delivery issues that influence whether or not RHY shelter goals are met.

Factors Related to Gang Membership Resistance (5042.1) is studying gangs from two contrasting Los Angeles communities--one with higher-than-average Hispanic and African-American gang activity, and the other with lower-than-average gang activity. The project is designed to increase understanding of how youth in urban areas with high levels of street gang activity avoid gang involvement. The project will compare gang members with youth who do not belong to a gang in different urban settings, such as a community with high gang activity versus one with low gang activity. Individuals and community variables and characteristics in the contrasting sites will be analyzed.

In Prevention of Foster Care Placement of Children at Risk for Domestic Violence (4378), the Family Advocate Project (FAP) is building on existing programs operated by the Family Intervention Center of the Children's Hospital of Pittsburgh, which provide medical and psychosocial evaluations to battered women and their children. The project is studying 150 mothers assigned to either a study or a control group. Study-group mothers receive intensive services in the areas of counseling, housing assistance, and legal, health care, and social services. The project is looking at ways to alleviate the emotional trauma of battered women and explore whether the provision of support services to increase self-esteem and coping skills can prevent out-of-home placement or reduce the length of time children spend in foster care.

The National Child Welfare Research Center: University of Chicago (4390.1), located in the Chapin Hall Center for Children, is conducting research on the following issues: (1) research and management uses of computerized child welfare data; (2) service and policy issues concerning in-home and community-based care; (3) the uses, consequences, and costs of in-home care; and (4) ways to apply knowledge and research about child development to children's policies. The Center holds conferences and sponsors a child development fellowship in conjunction with the Society for Research on Adolescence. Publications will include research syntheses, a training manual, and a report on national data issues.

The National Child Welfare Research Center: University of California at Berkeley (4390.2), located in the School of Social Welfare, will conduct projects to examine (1) child abuse and child welfare; (2) family preservation and maintenance; (3) foster care and adoption; (4) drug and AIDS-affected children; and (5) the organization, financing, and evaluation of child welfare services. The center serves as a knowledge-building and information-disseminating resource for improved child welfare services.

The National Study of Outcomes for Children Placed in Foster Care with Relatives (5846) is examining the outcomes for and associated costs of children and families in the various configurations of relative foster care compared to the configurations of non-relative foster care. Information obtained in an initial nine-State survey is being used to design a national study on the outcomes of children placed in relative foster care compared to the outcomes for those placed in non-relative foster care.

The aim of How Decisions to Change the Case Plan Goal Are Initiated (6848) is to conduct a 3-year study of children entering State custody as infants that will identify factors that facilitate or delay changes in the case plan goal leading to permanency. Data will be collected through in-depth interviews with caseworkers responsible for selected cases. Products of this study will include three major reports and policy and practice recommendations for facilitating permanence for children entering State custody as infants.

The Factors Related to the Quality of Family Foster Care (6849) study seeks to determine the quality of the foster care experience and the factors that influence it, based on data collected on a sample of 500 families whose children spent time in foster care in Wayne County (Detroit) Michigan during 1993. The study examines the relationship between agency and service characteristics (e.g., private versus public agency, caseload size, staff turnover, continuity of case services, provision of family preservation services, and use of kinship care) and the quality of foster care. Five aspects of foster care will be assessed: the health and well-being of the child while in care, incidence of maltreatment in care, family continuity, number of placements by type, and duration of care and recidivism.

The Assessing the Quality of Foster Family Care: An Initiative for the Integration of Research and Practice (6850) project aims to fill the gap in the literature on foster family care by examining the individual characteristics of foster parents across domains, as well as the environmental characteristics of the home. Participants (240 families) will be drawn from the pool of foster parents and children in Maryland's Prince George's and Montgomery counties. The project will examine the following questions: (1) Does the current pool of foster parents possess the characteristics to meet the needs of the foster child population? (2) What is the appropriate role of relative foster homes? (3) How does ethnicity relate to other characteristics of foster family homes?

Project REFRESH: Research and Evaluation of Foster Children's Reception into Environmentally Supportive Homes (6851) seeks to discover the effect of the everyday occurrences in foster family homes. Research indicates that foster children aged 9-18 may perceive themselves as outsiders to their foster families. Some birth children in foster families perceive a foster child as an intruder. Many foster caregivers have difficulties balancing parenting and caregiving roles, which impacts quality, duration, and level of birth and foster children's care. Assessment tools and evaluation protocols are needed to examine factors affecting quality and satisfaction with care in kinship/non-kinship placements. The major research goal of this 3-year study by Oregon State University is to develop a model to assess a child's integration into a foster family.

The purpose of the Evaluating Quality of Out-of-Home Care in Kinship Foster Families (6852) project is to identify criteria for assessing the quality of out-of-home care provided to children in kinship care homes and to develop objective items that use these criteria to effectively measure the quality of kinship care. Instruments to be developed by the University of Illinois-Urbana will measure the quality of the kinship family home, contextual factors related to kinship family functioning, and indicators of child functioning in the home.

Two studies address issues of family preservation and family support. The first, the National Evaluation of Family Support Programs (5848), is a comprehensive review of what is currently known about family support programs and their effects. The project will include several studies to fill gaps in understanding about family support programs and their effects, and to synthesize the results with the results of the earlier review. The review conducted in the first year will guide the formulation of a research strategy for the remaining four years of the project. Up to ten program evaluations will be designed and implemented.

The second, the Family Preservation and Family Support Services (FP/FS) Implementation Study (5975), examines the FP/FS implementation process across States and communities and among different stakeholders over time. It examines the process broadly over all 50 States, and in depth in 10 States and 20 communities. The first component of the study analyzes and synthesizes first-year applications and examines State planning and use of FP/FS funds, e.g., the nature of the planning process, the involvement of various stakeholders, and decisions about fund allocation. The study will assist ACYF in its joint planning efforts with States by providing feedback on States' progress in implementing FP/FS legislation.

The purpose of the National Longitudinal Study of Children and Families in the Child Welfare System (6748) is to describe the outcomes experienced by children and families who come to the attention of the child welfare system, and to gain an understanding of the factors, including system-level and service factors as well as child and family characteristics, that contribute to those outcomes. A nationally representative sample of 6,000 children will be sampled upon entry into the child welfare system. Data will be collected during the first year after entry and at three annual followup interviews. Information will be obtained from the children and their caregivers, caseworkers, and other child welfare agency personnel and service providers.

 TOP OF DOCUMENT

Administration on Aging

MISSION: To foster the development of services to help older persons maintain their independence.

Evaluation Program

The Administration on Aging (AoA) is the Federal focal point and advocate agency for older persons and their concerns. The AoA administers key Federal programs mandated under various titles of the Older Americans Act. These programs help vulnerable older persons remain in their own homes by providing supportive services. Other programs offer opportunities for older Americans to enhance their health and to be active contributors to their families, communities, and the Nation through employment and volunteer programs. The AoA works closely with its nationwide network of regional offices and State and Area Agencies on Aging to plan, coordinate, and develop community-level systems of services that meet the unique needs of individual older persons and their caregivers. The AoA collaborates with Federal agencies, national organizations, and representatives of business to ensure that, whenever possible, their programs and resources are targeted to the elderly and coordinated with those of the network on aging.

As the responsibilities of this nationwide network of State and Area Agencies on Aging continue to grow, it is essential that they have the necessary information to meet these responsibilities.

The overall evaluation priorities of the AoA are to support studies that provide information on the following:

• The success of existing program implementation in meeting the goals of the Older Americans Act;
• The design and operation of effective programs; and
• Issues relevant to policy development, legislative planning, and program management.

Summary of Fiscal Year 1997 Evaluations

Although there are no major evaluations to report during FY 1997, AoA initiated an evaluation of the Supportive Services and Senior Centers provisions of the Older Americans Act (Title III-B). These provisions provide home- and community-based services (HCBS) for which all elderly persons are eligible, with particular attention to older persons in greatest economic or social need. Most home- and community-based services fall under three broad categories: access services, in-home services, and other community-based services. Access services include care coordination, transportation, outreach, and information and assistance. In-home services include personal caregivers, homemakers, chores, and telephone reassurance. States and local communities decide on the mix of services these funds provide through State and area plans for which public participation is required. In FY 1995, the last year for which complete data are available, the Title III-B Program ($306,711,000) provided services to approximately 6,050,000 persons.

A basic operating principle of the Older Americans Act is to provide funds through States to local governments and community organizations with Federal assistance and leadership, but which is flexible enough to address local community needs. This approach is consistent with current legislative trends in establishing intergovernmental funding relationships. There is substantial anecdotal evidence that this flexible decentralized approach has worked very well in fostering the development of HCBS appropriate to diverse States and communities. However, the diversity of the States, communities, service programs, and resource inputs, the long time frame for development, and the general nature of the outcomes sought (comprehensive coordinated HCBS systems) make it difficult to evaluate progress related to Title III-B. State Program Report (SPR) data are a necessary starting point in describing results from Title III-B. But the SPR shows only the output of the services funded and does not illuminate the developmental effects of the aging network funded through Title III-B. Under provisions of the Government Performance and Results Act (GPRA) of 1993, it is appropriate for AoA to evaluate the outcome results from Title III-B despite the conceptual and methodological difficulties.

Evaluations in Progress

Given the evolving roles of AoA and State and Area Agencies on Aging and the continued (and projected) growth of the Nation's elderly population, AoA's evaluation efforts will continue to focus on effective program planning and service delivery, as well as on continued monitoring of the program's effectiveness in addressing the goals of the Older Americans Act. During FY 1997, AoA initiated a study to plan an evaluation of the Supportive Services and Senior Centers provisions of the Older Americans Act (Title III-B) (6855). Through this project AoA will identify the conceptual and methodological issues in the evaluation and will develop an approach to this evaluation. The evaluation is expected to develop outcome measures with broad applicability, which can be used repeatedly over time. It is anticipated that the actual evaluation will begin early in FY 1999.

Evaluation is an important part of the AoA program, and a number of evaluation activities are currently under way, although not on a national level. Many State and Area Agencies on Aging are conducting evaluations of their Older Americans Act service programs. These studies generally focus on such issues as the needs of the target population, the quantity and quality of services delivered, and the impact of the services on the older person. Results are used to better design and target Older Americans Act services. Regional offices of the AoA also conduct assessments of the States' needs for training and technical assistance. Finally, demonstration projects funded by the AoA have evaluation components to enable them to assess their progress in meeting the objectives of their demonstrations.

Agency for Health Care Policy and Research

MISSION: To generate and disseminate information that improves the health care system.

Evaluation Program

The Evaluation Program within the Agency for Health Care Policy and Research (AHCPR) is used to respond to three types of information needs:

• The need to determine the Agency's effectiveness in meeting its long-term goals and priorities
• The need to improve the efficiency of the Agency's program performance through internal evaluations; and
• The need to respond to critical Agency and departmental concerns via "fast-track" or quick-turnaround assessments.

To address these needs, evaluation components are built into virtually all major AHCPR programmatic activities and a wide variety of "freestanding" projects are undertaken as needs are identified. Among the evaluation mechanisms used by the Agency are targeted evaluation studies undertaken through contracts or grants; efforts to obtain feedback from "customers" on the usefulness of AHCPR research efforts, including such mechanisms as focus groups and surveys; and feedback from AHCPR's User Liaison program (which provides information and technical assistance to State policymakers, health departments, and officials). Information gathered from AHCPR's evaluation-related activities will be used in the annual performance plans developed in response to the Government Performance and Results Act starting in FY 1999.

All "freestanding" evaluation activities undergo two levels of review. First, brief evaluation proposals are developed by staff and undergo review by affected senior managers. Those proposals approved by senior managers are submitted for review by the administrator. The administrator evaluates proposals for policy relevance, priority, cost effectiveness, and timeliness. Those that receive the administrator's approval are then methodologically developed and centrally reviewed for technical merit, including technical feasibility, costs, and relation to ongoing evaluation activities.

Summary of Fiscal Year 1997 Evaluations

Outcomes and Effectiveness Research in the Private Sector

To help clarify AHCPR's future priorities in outcomes and effectiveness research (OER) and to provide insights regarding future opportunities for public-private partnerships in this area, AHCPR undertook an analysis and evaluation of private-sector OER activities (6385). The study found that a wide variety of private-sector organizations report performing OER, often for the purposes of monitoring health care performance against certain standards, e.g., accreditation standards. In addition, private-sector organizations conduct OER primarily to improve their strategic and financial positions. More specifically, health product companies usually focus their OER efforts on new drugs and devices, while less attention is given to other health care interventions, particularly those that are not new or those that are used primarily for traditionally underserved or vulnerable populations. AHCPR has supported the development and validation of many of the research tools used in private-sector OER. The report found little evidence of a redundant effort between publicly and privately supported OER and reinforced the need for public support for continued methodologic development in OER.

Informatics

Drawing upon research previously performed in the U.S., Canada, and the United Kingdom, a recent study reported that informatics tools and decision aids affect patient decisionmaking about medical screening and treatment (6376). More specifically, researchers found that patients who use informatics tools are more satisfied with the decisionmaking process and have more knowledge of treatment alternatives than patients who do not use the tools. The researchers recommend that the effects of informatics tools on a full range of outcomes be assessed (e.g., patient-provider communication, treatment selection, health behavior, clinical and health status outcomes, and health care costs). It is important that factors influencing patient use of informatics tools be identified as well as the cost-effectiveness of different types of patient informatics tools.

Quality Measurement

As part of its effort to measure and improve quality of care, AHCPR completed two related evaluations of CONQUEST--the Computerized Needs-Oriented Quality Measurement System (5961.1 and 5961.2). CONQUEST is a prototype software system for collecting and evaluating clinical performance measures, and organizing them into a classification scheme that presents essential information in a standard format. The two evaluations identified short-term, intermediate, and long-range strategies for CONQUEST's continued development. While finding CONQUEST 1.0 to be a good starting point, the evaluations recommended that future versions of CONQUEST be expanded to include more measures, more conditions, and more detail on measure construction/specifications. In addition, the evaluations indicated that CONQUEST should be modified to make it more intuitive and user-friendly. Many of the recommendations were incorporated into CONQUEST 2.0.

Managed Care Organizations

To help determine the extent to which AHCPR and other organizations will be able to analyze data from managed care organizations (MCOs), AHCPR completed work on a project to determine the feasibility of creating a managed care encounter-level data base (6374). The rapid growth of managed care is changing the ability of health services researchers and policy analysts to answer fundamental questions about access, utilization, cost, and quality of care. Data that were previously produced as a "by-product" of the fee-for-service system are either no longer available or vary in comprehensiveness and accuracy. One major finding of this study was that MCO data bases vary substantially in the availability and comprehensiveness of encounter-level data and in the ability to link these data with administrative and financial data. MCOs increasingly view their information systems, and the data they store, as proprietary, and there are a number of disincentives for MCOs to share their data. In addition, there are no industrywide definitions and reporting standards. Despite these barriers, MCOs are interested in exploring ways to work with Federal agencies to facilitate the availability of data for the purposes of research.

AHCPR has used the study findings to 1) identify future opportunities for public-private partnerships; 2) solicit feedback from multiple constituents on models that the Agency could employ to facilitate improvements in, and accessibility of, encounter-level data; 3) assess current AHCPR supply-side data initiatives (e.g., Healthcare Cost and Utilization Project); 4) formulate a longer-term encounter-level data strategy; and 5) inform and support related activities in the Secretary's Quality Initiative and the President's Commission on Consumer Protection and Quality.

Dissemination Evaluation

AHCPR conducted a customer survey to determine satisfaction with its publications and their dissemination (6375). The effort found that AHCPR's monthly newsletter, Research Activities, is well-received as it is currently written, presented, and distributed. A large percentage of readers keep the publication as a reference source and would prefer to continue to receive the publication in hard copy format, rather than via the World Wide Web (where it is also currently available). Overall, the survey results showed that the publication has a relatively loyal following. The study also found that AHCPR's patient/consumer guides are most frequently requested by phone and delivered in a timely manner. In addition, recipients are pleased with the services of AHCPR's publications clearinghouse and had very positive reactions to many of the guides' different features. This information will be used as performance data in AHCPR's Government Performance and Results Act Plan and will serve as a baseline for any future surveys fielded to determine satisfaction with changes made to Research Activities. During FY 1997, AHCPR also completed projects on evaluating minority health services research training activities (6384), identifying private sector organizations as potential research partners (6386), developing a workplan for preparing an inventory of performance measures currently being used or considered by HHS agencies (6691), and evaluating AHCPR's small conference grant program (6692).

Evaluations in Progress

Grant Application and Review Processes

AHCPR is conducting an evaluation of its grants (large and small) application receipt and review processes (6696). The evaluation includes a combination of qualitative and quantitative techniques that will document the extent to which changes made to the grants application submission and peer review processes have resulted in improved efficiency and effectiveness from the perspectives of applicants, reviewers, and Agency staff.

Data Collection Evaluation

AHCPR is undertaking a project to assess the usefulness to policymakers and researchers of AHCPR-supported major data collection efforts, such as the Medical Expenditure Panel Survey, the Health Care Cost and Utilization project, and the HIV Cost and Services Utilization Survey. More specifically, it will examine methods to improve survey operations and timely data release. The study will examine how other Federal agencies, as well as private foundations, conduct various aspects of their large-scale surveys (6383).

Agency for Toxic Substances and Disease Registry

MISSION: To prevent exposure and adverse human health effects and diminished quality of life associated with exposure to hazardous substances from waste sites, unplanned releases, and other sources of pollution present in the environment.

Evaluation Program

The Agency for Toxic Substances and Disease Registry (ATSDR) was created as a Federal agency by the Comprehensive Environmental Response, Compensation, and Liability Act (CERCLA), more commonly known as Superfund legislation. ATSDR was created to implement the health-related sections of CERCLA and other laws that protect the public from hazardous waste and environmental spills of hazardous substances. ATSDR relies heavily on strong working relationships with tribal, State, and local organizations; private, nonprofit, and community-based organizations; and other Federal agencies to implement critical environmental health programs. As part of the public health assessment process, ATSDR recommends actions that can be taken to reduce or mitigate the risks of adverse health effects from exposures to toxic substances to the Environmental Protection Agency (EPA) and State and local governments.

The ATSDR evaluation program is coordinated with the agencywide strategic planning process, which was started in March 1996 to implement requirements of the Government Performance and Results Act (GPRA). ATSDR's strategic goals and its annual performance plan are the result of an interactive process that reflects a long-term commitment by Agency staff to develop stronger relationships among external clients and stakeholders, to assess products and services using relevant data, and to improve our processes and systems for more efficient accomplishment of ATSDR's mission. The following strategic questions are important to the Agency:

FACE="WP MathA">‚ How should ATSDR evaluate the public health needs of its stakeholders and involve those stakeholders in the Agency's program planning, implementation, and evaluation?

• How should ATSDR's roles in assessment, science, health promotion, and service be implemented to meet the needs of its stakeholders and to achieve its mission?
• How does ATSDR ensure that it has the authorities and resources necessary to address the public health needs of its stakeholders?
• How should ATSDR evaluate the public health impact of its products and services?

Effectively focusing ATSDR's programs on these issues enables the Agency to understand stakeholders' needs, address its mandates, and evaluate its programs so that ATSDR operations, management, products, and services can be improved.

ATSDR receives its funds from the Hazardous Substance Trust Fund/Superfund appropriations rather than Public Health Service appropriations; therefore, ATSDR does not receive a 1-percent evaluation set-aside. Nevertheless, the new planning system provides the basis for measuring ATSDR performance and for making systematic improvements a part of its internal evaluation activities.

An example of such "in-house" evaluation activities through ATSDR resources is the Evaluation Working Group. This group was formed to develop a long-term strategy for site-specific evaluations to better assess the outcomes and impact of ATSDR activities at hazardous waste sites. The Methyl Parathion Evaluation Workshop was established to allow coordination between and continue work being accomplished by ATSDR, the EPA, and State and local agencies when responding to the illegal spraying of homes and businesses with methyl parathion, an agricultural pesticide. The workshop's intended purpose is to discuss approaches and reach a consensus on the best way to document the activities in the methyl parathion program.

Summary of Fiscal Year 1997 Evaluations

In FY 1997, ATSDR (1) developed an agencywide inventory of evaluation activities, (2) conducted an assessment of its site-specific evaluation needs, and (3) developed a guidance document for State cooperative agreement evaluation activities. These activities and information enabled ATSDR to assess why certain public health actions or decisions may or may not be working effectively. These activities also allowed for the identification of program strengths and weaknesses and suggestions to improve the efficiency and effectiveness of public health actions.

A number of States collaborated with ATSDR to develop a guideline for assessing the impacts and outcomes of site-specific health education activities. The documents described in the following paragraphs are the results of this collaborative effort. These tools will allow cooperative agreement participants to assess the overall improvement in public health as a result of site-specific health education activities and demonstrate the cooperative agreement program's effectiveness.

Site-Specific Outcome Measurement Guidelines for ATSDR Cooperative Agreement States (6853.1) provides guidelines for assessing the impacts and outcomes of site-specific health education activities. Evaluation of State Capacity Building--Program Announcement #607, published in the Federal Register, was also developed collaboratively by ATSDR staff members and State partners to assist in the overall evaluation of the effectiveness of the capacity building effort occurring in participating States. The evaluation also includes an assessment of ATSDR activities by State cooperative agreement personnel.

Next, ATSDR developed the document, Did We Make a Difference? ATSDR's Framework for Community-Focused Evaluation (6853) to provide guidance and information on how to conduct evaluations of ATSDR's site-specific programs and activities.

Evaluations in Progress

ATSDR has a number of ongoing evaluation activities. In addition to conducting traditional evaluation studies, ATSDR's commitment to viewing evaluation as an integral part of program planning is exemplified in the following activity now in progress.

Hazardous Substances Emergency Events Surveillance (6854). This evaluation of ATSDR's Hazardous Substances Emergency Events Surveillance (HSEES) program is based on the ability to measure the sensitivity and reliability of the system. Sensitivity is measured by the ability of the system to receive notification of events involving hazardous substances meeting the HSEES case definition. Reliability is measured by the ability of the system to receive uniformly recorded data from the 13 State health department partners.

To address the sensitivity of HSEES, ATSDR provides to each participating State health department a list of events involving hazardous substance releases that occurred in their State during the previous quarter and that were reported to one of two national data bases--the Department of Transportation's Hazardous Materials Information System or the EPA's Emergency Response Notification System. This activity provides the State partners with a mechanism to detect whether there were (1) incidents reported to one of the national data bases that were not reported to HSEES, (2) incidents reported to HSEES but not reported to one of the national data bases, or (3) incidents reported to both HSEES and a national data base.

To address the reliability of HSEES, ATSDR mails to participating State health departments each quarter a report describing a case study involving the release or threatened release of one or more hazardous substances. State health department partners review the case study and complete a hard-copy data collection form, which is returned to ATSDR. Then ATSDR compares the responses on these forms to each other and to ATSDR's preferred responses.

Centers for Disease Control and Prevention

MISSION: To promote health and quality of life by preventing and controlling disease, injury, and disability.

Evaluation Program

The Centers for Disease Control and Prevention (CDC) conducts evaluation studies designed to provide information essential for CDC's 11 strategies to achieve its mission:

• Conduct public health research, including epidemiology, laboratory, behavior, and social sciences.
• Develop and implement ongoing evaluation of scientific research.
• Assure that scientific information is communicated effectively to the general public.
• Develop and implement public health information systems for monitoring and promoting the health of the Nation.
• Deploy multidisciplinary teams to detect and investigate health threats.
• Develop and implement a system for establishing CDC scientific and programmatic priorities.
• Routinely evaluate the effectiveness and cost of CDC programs.
• Demonstrate the value of investment in health prevention.
• Collaborate with diverse partners.
• Design, implement, and evaluate prevention programs based on community needs.
• Prepare the public health leaders (present and future) through training in management and public health science.

CDC places high priority on evaluations to answer policy, program, and strategic planning questions related to the goals and objectives of Healthy People 2000. Performance improvement studies, such as those focusing upon the development of indicators consistent with the Government Performance and Results Act (GPRA), are of particular interest and import to the Agency. With the support of 1-percent evaluation funds, the GPRA planning process began at CDC in FY 1995 and is continuing. CDC's commitment to performance measurement is exemplified by this year's requirement that each project proposal be linked to one of CDC's strategic goals. These goals were identified through the GPRA strategic planning process.

On an annual basis, the Director of CDC provides guidance to the various Center, Institute, and Office Directors on 1-percent evaluation activities. This memorandum generally includes information about (1) types of studies to be carried out with 1-percent evaluation funds, (2) the format for submitting proposals, and (3) a timeline for selection. Each proposal undergoes multiple levels of review. Initial review is conducted by staff within the Office of Program Planning and Evaluation. Subsequent reviews are completed by the Office of the Assistant Secretary for Planning and Evaluation. Study authors are provided with comments, questions, and recommendations made by reviewers. In addition to providing their responses, authors are given the opportunity to revise their proposals at this time.

A panel of CDC evaluators, scientists, and program managers is convened to review and rank proposals. Review criteria include (1) relevance to prevention effectiveness, (2) relative importance of the public health problem being addressed, (3) probability that the proposed project will accomplish its objectives, and (4) extent to which other CDC programs will benefit from this project. Results from the review panel are presented to the Director of CDC for final funding decisions.

Finally, staff within the Office of Program Planning and Evaluation work closely with program staff to ensure development of a clear statement of work for selected projects. Prior to initiation of procurements, a final ad hoc review of the project's statement of work is completed.

Summary of Fiscal Year 1997 Evaluations

A total of 22 evaluation projects were completed in FY 1997. These studies are of four types: program evaluations, data policy and surveillance studies, development of performance measures, and assessments of specialized aspects of proposed or existing programs.

Program Evaluations

Six program evaluations were completed in FY 1997. These included an evaluation of the programmatic impact of CDC's 1-Percent Evaluation Program (6581 and 6581.1), an evaluation of suicide interventions in three Native American communities (5504), an evaluation design for the Business Responds to AIDS (BRTA) Program (6277), and an evaluation of the National Laboratory Training Network (NLTN) (5507). These studies represented three types of evaluations: informational, process, and outcome. For example, one of the studies looked at the design of BRTA, which was initially undertaken to help businesses create and implement comprehensive HIV/AIDS workplace programs, but also included questions that could help track trends in worksite policies, screening programs, and health promotion activities. In addition, one evaluability assessment that focused on CDC's Traumatic Brain Injury/Spinal Cord Injury program (6338) was completed.

Data Policy/Surveillance Studies

Four data policy/surveillance studies were completed in FY 1997. Two of the studies addressed reporting issues. For example, one study was conducted to ascertain whether the codes for International Classification of Disease (ICD)-10 should be modified to allow for greater specificity for morbidity use (5503). A second study in this reporting category evaluated data collected on birth records completed by multiracial and Hispanic women to see how they interpret the race question on the birth certificate (5918). Finally, two other studies addressed the area of surveillance, looking at the effectiveness of CDC surveillance for drug-resistant Streptococcus pneumoniae and the feasibility of developing a public health surveillance plan to assess current community public health. (6580, 6580.1)

Development of Performance Measures

In keeping with the congressional intent of the Government Performance and Results Act (GPRA) of 1993, one study was conducted in FY 1997 to support development of strategic planning at CDC (5928). This report offers a record of CDC's experience in its early implementation of GPRA by documenting the process, summarizing key findings, and making recommendations for strategies and approaches for the ongoing implementation of GPRA at CDC.

Assessments of Specialized Aspects of Proposed or Existing Programs

Eleven studies relating to discrete aspects of current or proposed programs were completed in FY 1997. Two of these projects were assessments of the Sexually Transmitted Disease (STD) prevention programs supported by the CDC (6579, 5348). One project was a comparative case study of local-level syphilis prevention efforts, while the other assessed the feasibility of parallel STD surveillance systems currently operated by CDC. Additionally, two studies were conducted to assess the effects of HIV prevention activities: (1) through HIV Community Planning to learn how community planning is confirming, enhancing, and changing HIV prevention programs (5927); and (2) through the Prevention Marketing Initiative, which is aimed to influence behaviors that contribute to the sexual transmission of HIV among young people below 25 years of age (6337). Another study looked at CDC's efforts to link health promotion and disease prevention research to public health practice (6535).

Three studies focused on issues related to women. The first of these studies identified and investigated factors that place women at risk for an abusive relationship and, in turn, how women can protect themselves from further violence during and after the relationship (6515). The second study focused on presenting successful strategies used by grantees of the National Breast and Cervical Cancer Early Detection Program (NBCCEDP) to reach women in need of mammography screening, especially low-income, underserved, and uninsured women (6273, 6273.1). The information will be shared with grantees in order to describe how various programs and interventions have reached marginalized populations. This information will also be useful for private sector and other organizations that are working to provide prevention services to hard-to-reach groups.

Two additional studies measuring the effectiveness of particular programs and activities were completed during FY 1997. These studies involved an evaluation of the National Immunization Survey (6477) and the Fatality Assessment and Control Evaluation (FACE) Program (5506.1). Finally, another activity focused on the first phase of a national study designed to profile the relationship between youths' access to tobacco and the contextual variables that define communities' policies, attitudes, and activities regarding youth and tobacco (6474).

Evaluations in Progress

CDC's evaluation studies in progress during FY 1997 consisted of four types: program evaluations, data policy and surveillance studies, development of performance measures, and assessments of specialized aspects of proposed or existing programs.

Program Evaluations

Nine program evaluations are currently in progress. Included are evaluations of the C. Everett Koop Community Health Center (CHIC) (6709), diabetes control programs (6584), the Field Epidemiology Training Program (FETP) (5923, 5923.1), evaluation of teen pregnancy interventions (6274), lead poisoning prevention training (6702), sexually transmitted disease (STD) partner notification (6711), impact of the Clinical Laboratory Improvement Amendments (CLIA) (6335), and CDC's Prevention Centers Program (5919). The study of the C. Everett Koop Community Health Center is a process evaluation that will look at the effectiveness of the center to serve as a national physician-based model. This will be accomplished by assessing (1) the effectiveness of the CHIC products and services from the users' perspective; (2) the use of the products and services, as well as barriers to their use; (3) the effectiveness of collaborative arrangements with nonprofit organizations, corporations, libraries, government agencies, and voluntary health agencies; and (4) the effectiveness of the products and services, such as quality of resources.

By contrast, the remaining studies in this category are outcome evaluations. The evaluation of CDC-supported, State-based diabetes control programs seeks to develop and implement an evaluation strategy to improve capacity for uniform assessment of States' progress in reducing the burden of diabetes on an evolving health system.

Similarly, the evaluation of CDC's Field Epidemiology Training Program (FETP) will ascertain whether the program has achieved its objectives, which are to train public health professionals in applied epidemiological skills, to promote the sustainability of autonomous FETPs, and to develop a global network of national programs.

The evaluation of teen pregnancy interventions will assess and test the adolescent pregnancy prevention initiatives in 13 cities within the United States. This project will involve the development and testing of efficient, systematic, and durable approaches to translating evaluation findings and program experiences of the 13 communities into changes in program interventions in demonstration and replication communities.

The last of the outcome evaluations involves the development of a systematic evaluation of notification to sexual partners of individuals who are diagnosed with sexually transmitted diseases. This project seeks to ascertain whether the substantial investment of State, Federal, and local health care dollars is decreasing disease prevalence or incidence when partners are notified.

Evaluation of Data Policy and Surveillance Systems

Ten studies are under way that relate to data management or reporting systems. Seven studies include evaluations of the U.S. Standard Certificates (6699), medical certification process for death certificates (4230), pregnancy-related violence (6712), Assisted Reproductive Technology (ART) embryo lab procedures (6716), use of data from Immunization Information Systems (IIS) (6713), evaluation of data uses of the National Electronic Telecommunications System for Surveillance (NETSS) and the Public Health Laboratory Information System (PHLIS) (6332), and the Urban Health Systems Sentinel Network (6276). In addition, three surveys are being conducted to evaluate guidelines for preventing perinatal HIV infection (6279), assess immunization knowledge and practice of primary care providers in the U.S. (5930.1), and inform the public about skin cancer (6710). The Skin Cancer National Survey examines the knowledge, attitudes, and behaviors of parents regarding skin cancer and the protection of their children from the sun. Results from this survey will enable CDC to design and distribute pertinent health messages about sun exposure.

The overall objective of the U.S. Standard Certificates evaluation is to examine the birth, death, and fetal-death information currently obtained from State vital registration programs to determine if relevant and high-quality data are being collected for State and national decisionmaking. The assessment will also consider what changes should be made to the standard certificates to improve their utility.

The other two evaluations are examining the use of survey instruments in data collection. In the area of pregnancy-related violence, a national population-based survey will be conducted of administrators and clinical providers in family planning programs. Information gained in the survey will be used to develop recommendations for the development of clinical guidelines. Similarly, since CDC is responsible for developing a model certification program for ART embryo laboratories in the United States, a survey will be conducted on the laboratories currently using ART to provide an enumeration of those procedures and practices.

Finally, at this point in the development cycle of the Immunization Information Systems (IIS), an assessment is being conducted on how the IIS data are used. The objectives of this project are to (1) assess how IIS data are used to assess public clinic and private provider performance and to identify structural and functional barriers to using the data; (2) assess how IIS data are used to assess community-based immunization coverage; (3) assess how IIS data are used to monitor the impact of changing recommendations in immunization schedule and the health care delivery system; and (4) promote the development of immunization registries.

Development of Performance Measures

Four ongoing projects involve the development of performance measurement systems. One of these projects, Development and Implementation of Strategic Planning at CDC, seeks to gather and analyze information that supports a comprehensive, unified, and data-driven strategic and performance planning framework for the CDC (6275). Three other projects focus on the development of indicators that will be used to assess HIV prevention efforts in the community, management and operational aspects of HIV community planning, and the coordination of multiple HIV prevention programs (5518, 6278, 6707).

Assessment of Specialized Aspects of Proposed or Existing Programs

Fifteen specialized studies relating to discrete aspects of current or proposed programs are in progress. These wide-ranging studies include evaluations of the development of a public health surveillance plan (5925), a feasibility study of Pap screening in STD clinics (6705), dental treatment and health status differences attributable to water fluoridation (6334), and the effectiveness of Group B streptococcal disease prevention guidelines (6333), still other studies assess the public health impact of the National Institute for Occupational Safety and Health (NIOSH) health hazard evaluation program (5922), tuberculosis outreach worker activities (5502), elder care groups (6708), small business interventions (6336), the Morbidity and Mortality Weekly Report (MMWR) recommendations for the use of bike helmets (6700), coordinated tobacco control efforts (6714), CDC's technical assistance to local health departments on violence prevention (5929), the efficacy of different types of vancomycin-resistant enterococci (6703), and HIV-related projects (6701, 6704, 6715, and 6706).

With the goal of complete eradication of tuberculosis by the year 2000, one of the studies focuses on an evaluation of tuberculosis outreach worker activities. Specifically, the project will (1) identify how these workers accomplish their tasks, (2) assess the quality of outreach workers' interactions with clients and supervisors, and (3) evaluate the amount of time spent on various research activities.

Many of the studies being conducted hope to provide the public with better information for all age groups. For instance, the elder care focus group study was undertaken in order to obtain data that will be used to produce an elder care health and safety handbook in the United States. One study focuses on the safety of our youth, as well as adults, with a desire to develop updated recommendations and guidelines for the use of bicycle helmets. In addition, an evaluation of tobacco control efforts will continue by looking at the impact and the relative cost effectiveness of tobacco control programs.

Several HIV prevention-related projects are in progress. Projects are being conducted to develop community indicators for HIV, to administer PCP prophylaxis for women and persons in underserved communities, to evaluate the current HIV technical assistance (TA) network, and to determine how the next iteration of TA could be improved. Finally, a study to determine the feasibility of developing a demonstration project to assess identification and screening of women at high risk of HIV infection is in progress. The community indicators study will identify (1) cultural and social-structural characteristics of communities that are associated with HIV/AIDS risk behavior and prevention efforts; (2) theories, assumptions, or hypotheses that explain the causal role of community characteristics; and (3) measures or indicators that could be used to assess changes in those characteristics that support sustained individual behavior change and community prevention efforts. This information will be used to support community-level research on HIV/AIDS prevention.

The goal of evaluating the CDC HIV Technical Assistance Network is to provide CDC staff who are responsible for planning the next iteration of a TA network and stakeholders of the current CDC-supported TA network with valuable information that can be used to develop and strengthen the existing TA model.

Food and Drug Administration

MISSION: To protect and promote public health through food, drug, medical device, and cosmetic regulation.

Evaluation Program

The evaluation program of the Food and Drug Administration (FDA) continues to be aligned with the Agency's policy, its program and strategic planning initiatives, and the goals of the Government Performance and Results Act (GPRA). Since GPRA and the reorientation of all government managers toward performance management, the responsibility for program evaluation has shifted from traditional specialized staff offices and contractor studies to managers of all FDA programs. In addition, FDA managers are identifying further opportunities to involve their customers in the design and testing of alternative ways of doing business. Both performance-based goals and customer participation also impact FDA's response to the needs of small businesses and enhance the participation and performance of small businesses.

One element of FDA's strategic framework is called external leveraging. The goal here is to augment the ability of external stakeholders to manage FDA-regulated risk. FDA's evaluation program illustrates the implementation of this goal with a number of studies under way or completed during FY 1997 that illustrate the strategies that FDA is using to accomplish this goal. One strategy is the fostering of an industry quality assurance program, where FDA will establish protocols to guide firms, certify their approaches, and audit industry practices. A current example of this quality assurance is the Mammography Quality and Standards Act (MQSA), which monitors the administration of mammography practices. The following reports are based on evaluations of mammography activities: the Mammography Quality Standards Act: Second Annual Government Entity Declaration Program Audit, and the Economic Impact Analysis of Regulations Under the Mammography Quality Standards Act of 1992. In addition, the Center for Safety and Applied Nutrition (CFSAN) has implemented the Survey of Current Hazard Assessment Quality Control Points (HAACP) Practices of the Food Industry. While excluding seafood, this survey originated as a result of the Seafood HAACP Program, another FDA-fostered quality assurance program.

Other CFSAN evaluations, both completed and in progress, are clearly in line with FDA's objective of communicating risk associated with FDA-regulated products. These studies are directly linked to the President's National Food Safety Initiative, which will involve a collaboration between FDA, the U.S. Department of Agriculture (USDA), the Environmental Protection Agency (EPA), and the Centers for Disease Control and Prevention (CDC) in order to assure safety and quality of our food supply. Special emphasis will be placed on microbial hazards, which account for 90 percent of the confirmed food-borne outbreaks, 6.5 to 33 million illnesses, and 9,000 deaths in the U.S. annually. Since food-borne chemical hazards often present chronic rather than acute health threats, specific estimates of their impact on health and the economy are not readily available. Finally, data collected from the FDA's Safe Alert Survey will demonstrate how well the Agency is communicating risk associated with medical devices to the medical community.

Summary of Fiscal Year 1997 Evaluations

Prescription Drug User Fee Act

The Prescription Drug User Fee Act (PDUFA) has demonstrated that the Agency and industry, by combining their resources and working together, can achieve the levels of performance the American people deserve. The fifth and final report completed in November 1997 demonstrates that the successes reported in FY 1996 have been confirmed and surpassed. PDUFA has resulted in more and better applications filed which can be accepted immediately, reviewed, and approved more quickly (6079.2). Nearly all applications now go directly into the review process, with "applications refused" approaching the vanishing point. There are substantially more approvals on first review, which demonstrates that submission quality is a key factor in achieving timely approvals. Another indication of improved submission quality is the dramatic increase in the percentage of submissions that are ultimately approved faster. By working to meet the PDUFA's stringent review goals, new products are getting on the market at an unprecedented pace without compromising safety.

Product Review Performance Report

In FY 1997, FDA completed its first annual report analyzing performance in reviewing applications for human drugs, biological products, medical devices, food and color additives, and veterinary drugs (6731). Responding to legislation (such as PDUFA) as well as to requirements under the Federal Food, Drug, and Cosmetic Act, FDA has developed clear performance goals for reviewing new product applications. Setting these goals has provided a valuable management tool for identifying performance expectations, assessing achievements, and meeting the requirements of GPRA. The report finds that (1) for most categories of products, FDA performance has improved since FY 1993; (2) the greatest improvements in performance have occurred for product reviews covered by PDUFA; (3) significant progress has been made in reviewing applications for medical devices; and (4) on-time performance for reviews of non-PDUFA applications should be improved.

Assessment of the Mammography Quality Standards Act of 1992 (MQSA)

Under this statute, FDA is charged with ensuring that accessibility to quality mammography services is maintained for all facilities performing mammographies. This program is FDA's second major user-fee, performance-oriented, GPRA-style initiative. During FY 1997, Eastern Research Group, Inc. completed an economic analysis of FDA's final rule under MQSA working under contract to the Office of Planning and Evaluation (OPE), Economics Staff. The purpose of the cost-benefit analysis was to predict the impact of the proposed standards on the costs of providing mammography. The results of their analysis are presented in the report entitled, Economic Impact Analysis of Regulations Under the Mammography Quality Standards Act of 1992 (6080). The results of this analysis will be used by FDA's managers to develop a performance-oriented regulatory policy that will be coordinated with existing customers. OPE's Economics Staff developed a model for estimating health outcomes to a patient population at various levels of mammography quality.

Quality Standards Act: Second Annual Government Entity Declaration Program Audit and the Economic Impact Analysis of Regulations Under the Mammography Quality Standards Act of 1992

FDA's Division of Mammography Quality and Radiation Programs requested an audit of the MQSA Government Entity (GE) Declaration Program (6730). The audit was undertaken by the Division of Planning, Analysis, and Finance. The goals of the audit were to notify facilities that they will be held accountable for their self-certification of fee exemption and to determine the rate of compliance with the program. Based on the results, the auditors concluded that the GE Declaration Program appears to be quite effective. Several recommendations were made based on the findings of the audit. First, the results should be published to allow the impact of the audit to reach most of the facilities. This would discourage any future misrepresentations and demonstrate that FDA will exercise its authority. Second, the sample size of future audits should be smaller and should contain a larger portion of those facilities claiming fee exemption based solely on funding from the CDC. Finally, the MQSA contractor should continue to examine each GE Declaration form to ensure that it identifies the body operating the facility.

Evaluation of Device Performance in the Determination of Substantial Equivalence

Many diagnostic medical devices that reach the market are based on diagnostic agreement with some device already on the market that targets the same attribute. Agreement is often assessed by side-by-side testing of the same subjects or specimens, even if the true disease status is unknown. If each new test is compared with the most recent predicate, what would be the effect on device performance over time? Would it deteriorate, remain the same, or improve? FDA produced an analysis that addressed these questions using statistical measures of sensitivity, specificity, and predictive values (6734). When a new test was found to be equivalent to the current predicate, it became the new predicate. After 50 new predicates were found, a forecast procedure was applied to the sequence of 50 performance parameters in order to evaluate the long-term trend.

Center for Food Safety and Applied Nutrition (CFSAN) Evaluation

During FY 1997, three evaluations relating to food issues were completed by the Economics Branch in CFSAN's Division of Market Studies. The first, Preliminary Investigation into the Morbidity and Mortality Associated with the Consumption of Fruit and Vegetable Juices (6858), assessed the population risk from the consumption of fruit and vegetable juices. The main risks studied were microbial pathogens. Two studies evaluated regulation of the seafood industry: Cost of Restrictions on Gulf Oyster Harvesting for Control of Vibrio vulnificus-Caused Disease (6856), an economic impact analysis of alternative remedies for controlling Vibrio vulnificus-related illnesses in the Gulf States oyster industry, and Health Transfers: An Application of Health-Health Analysis to Assess Food Safety Regulation (6857), an evaluation tool based on the first study's methodology that can be used to measure economic versus health costs of public health decisions.

Evaluations in Progress

FDA currently has a broad range of evaluations in progress which, as in previous years, are based upon the two driving forces relating to GPRA goals: performance management and customer participation (with particular focus on small businesses). The following are examples of FDA evaluations in progress during FY 1997.

Although PDUFA ends in FY 1997, FDA and the pharmaceutical and biological prescription drug industry have negotiated re-enactment beyond FY 1997. Reauthorization of the collection of user fees under the Food and Drug Administration Modernization Act (enacted in November 1997) will continue to enhance the review process of new human drugs and biological products (6079.3). The PDUFA program serves as a model for reinventing government, with the Congress, FDA, the industry, and consumer groups working together to provide the necessary resources, set performance goals, and hold the Agency accountable. Revenues generated from the fees paid by the pharmaceutical and biological prescription drug industries will continue to be dedicated to improving and expediting the prescription and application review and approval process. The legislation reauthorizing PDUFA for FY 1998 was critical in ensuring uninterrupted service to U.S. customers.

Evaluation of Influenza Virus Vaccines in High-Risk Patient Categories (4355) is intended to ensure public health by investigating the reactogenicity and immunogenicity of influenza virus vaccines. The information will be critical to making accurate recommendations for the selection of new strains of the influenza virus vaccine to be used in the United States.

Nationwide Evaluation of X-Ray Trends (NEXT) (4984) is a collaborative program conducted jointly between FDA and State radiation control agencies. The study estimates the radiation dosage from diagnostic x-ray examinations using data collected from previous NEXT surveys of different types of x-ray examinations. The data are used by public health and professional organizations in setting policy.

Food Labeling and Package Survey (FLAPS) (5711) was implemented by the FDA in 1977 to produce a data base containing label and package information obtained from a sample of food products from the processed packaged food industry. FLAPS data enable FDA personnel to keep abreast of market responses to food labeling rules via changes in product package labels.

FDA Penalty Reduction Program for Small Businesses (6461) reviews pre-existing and revised policies related to monetary penalties for noncompliance with FDA regulations, evaluates the scope of small businesses that qualified (or failed to qualify) for the program, and assesses the total amount of penalty reductions or waivers. The Small Business Regulatory Enforcement Fairness Act (SBREFA) required FDA to create a program or policy that specifies the circumstances under which the Agency may reduce or waive the penalties levied against small business entities for noncompliance. The final report for this project is anticipated in March 1998.

FDA Inquiries Assistance Program for Small Businesses (6462) is FDA's response to the SBREFA requirement to establish a program to respond to inquiries and to help small businesses and other small entities apply FDA rules. The report on this project is anticipated in March 1998.

During FY 1998, FDA will continue to implement the requirements of the Mammography Quality Standards Act of 1992 (MQSA) (6080.1). FDA requested an increase in MQSA authorized inspection fees over the FY 1997 level. Continuing implementation of the MQSA requirements is essential to ensure that women receive quality mammography from facilities that maintain a high standard of safety and accuracy. The primary emphasis of this program for FY 1998 will be to continue to ensure that all facilities meet the quality standards and that deficiencies previously identified through annual inspections are corrected. In FY 1998, Federal and State personnel will continue to conduct annual inspections of 10,000 facilities and certifications of 6,000 facilities, and to provide training for new inspectors. The fees will be used to pay for the costs of the inspections.

Review of Therapeutic Agents as an Antimicrobial Adjunct in Treatment of Sepsis (6736) is a technique used in the drug approval process. No therapeutic agent has yet been identified that shows either a clinical benefit or an alternative that is superior to standard care. FDA faces many challenges in designing clinical trials that demonstrate efficacy. These are due to the heterogeneity of patient populations, poor understanding of the pathophysiology of acute sepsis, and no uniformly-accepted standard of care in these international trials. Intensive efforts have been undertaken by FDA to propose the optimal trial design and prospective analysis plans. This analysis of failed sepsis trials is being conducted by FDA to achieve better clinical trial results for the evaluation of adjunctive sepsis therapy in the future.

The Real-Time PMA Supplement Program Evaluation (6732), implemented by the Office of Device Evaluation (ODE), has evolved as a result of the Medical Device Review of the Premarket Approval (PMA) Applications program. FDA has reduced review times and backlogs for medical device applications. But, in order to achieve mandated goals, continued improvement is needed. FDA requested an increase in funds for FY 1998 in order to improve the quality and timeliness of its review process for Class III PMA applications and PMA supplements. These applications involve the highest-risk devices that require full clinical investigations of safety and efficacy. An increase in funding will facilitate FDA in allocating more resources to the PMA process without sacrificing performance gains made in the 510(k) program. It is projected that FY 1998 will be the first year of this enhancement effort, and FDA expects a 15 percent increase in PMA workload during FY 1997 and FY 1998 due to requests for reclassifications and submissions of required preamendment PMAs. Given these additional resources, FDA will attempt to limit the increase in pending PMAs at the end of FY 1998 to 5 percent. FDA has defined specific review performance goals for Class III applications that should be achieved by the end of FY 1998 with the additional resources.

The Safe Alert Satisfaction Survey (6733) will determine the impact of the safety alert and public health advisory on the behavior and knowledge of the public. The collection of these data is an important step in determining how effectively FDA is communicating risk to the public.

Survey of Current Hazard Analysis and Critical Control Point (HACCP) Practices of the Food Industry (6866) is a two-part survey that will yield information that will be used to estimate the HACCP-related practices of the food industry (except seafood, meat, and poultry), and a model that will be used to estimate the marginal costs of firms to comply with HACCP.

Economic Characterization of the Dietary Supplement Industry (6867) will provide a detailed analysis of the dietary supplement industry, including a description of the types of firms producing and marketing dietary supplements, and will create a relational data base that will link firms to products.

Estimating the Benefits of Various Kinds of Food Standards (6865) will categorize food standards by type of food and function and to estimate consumers' valuations of those functions.

Harmonizing Nutritional Goals and Food Processing (6864) is a paper to be given at the "Towards Convergence on the Dietary Guidelines, Research and Policy Needs in the Nutrition, Health, and Agricultural Sectors" conference on April 30, 1998.

The Impact of Tobacco Initiative on Youth (6890) is a project in which Office of Planning and Evaluation staff are currently designing a sample questionnaire to facilitate data collection on tobacco use. The survey responder will determine whether the tobacco initiative has had any positive impact on the propensity of youth to use these products. In August 1996, President Clinton approved FDA's final rule for regulating tobacco products. The President's Tobacco Initiative demonstrates a collaborative effort to thwart major public health risk with the objective of decreasing tobacco use over the next 7 years. FDA is working with the States to conduct unannounced compliance checks of retail establishments that sell tobacco products. FDA will rely on Federal and State agencies to monitor retailer compliance with the new regulations and track the degree of success in decreasing tobacco use by youth.

Health Care Financing Administration

MISSION: To promote the timely delivery of appropriate, quality health care to the Nation's aged, disabled, and poor through administration of the Medicare and Medicaid programs.

Evaluation Program

The research arm of the Health Care Financing Administration (HCFA)--the Office of Strategic Planning (OSP)--performs, coordinates, and supports research and demonstration projects (through intramural studies, contracts, grants, and waivers) to develop and implement new health care financing policies and to provide information on the impact of HCFA's programs. The scope of HCFA's research and demonstration activities embraces all areas of health care: cost, access, quality, service delivery models, and financing approaches. OSP's research responsibilities include evaluations both of the ongoing Medicare and Medicaid programs and of demonstration projects testing new health care financing and delivery approaches. These projects address four major themes.

• Monitoring and Evaluating Health System Performance: Access, Quality, Program Efficiency and Costs--HCFA's research program produces information and descriptive statistics on the infrastructure of the health system, on populations of health care users, and on service and expenditure patterns; examines differences in costs, quality, and access to care; and assesses the effects of HCFA programs on beneficiary health status.
• Improving Health Care Financing and Delivery Mechanisms: Current Programs and New Models--HCFA performs research and demonstrations to develop and test new payment and delivery models intended to increase the efficiency and effectiveness of Medicare. OSP's evaluations of these demonstration projects provide policymakers with information about their impact.
• Meeting the Needs of Vulnerable Populations--HCFA's research and evaluation program includes projects that develop new approaches to improve access to cost-efficient and appropriate health services for vulnerable populations.
• Information to Improve Consumer Choice and Health Status--Improving beneficiaries' knowledge and ability to make informed health care choices, both in the health plans they select and in the services they use, is part of HCFA's commitment to improving communication of information to beneficiaries. OSP examines variations in the use of services and treatments and the impact of new information tools and technologies in making health care decisions and improving beneficiaries' health status.

Summary of Fiscal Year 1997 Evaluations

During FY 1997, HCFA completed 21 evaluations, which are briefly described below.

In, Access to Physicians' Services for Vulnerable Beneficiaries: Impact of the Medicare Fee Schedule (6301), HCFA evaluated the effect of the physician fee schedule on a beneficiary's access to care. Six different strata of beneficiaries were selected based on the relative size of the payment change under the fee schedule compared to the earlier payment scheme. The project examined the use of services, outcomes of services, and the change in the beneficiary's financial liability.

The Impact of the Medicare Fee Schedule on Teaching Physicians (6666) was started because of four concerns: (1) teaching physicians perform more high-technology procedures and less primary care, which could result in disproportionate net loss for these practices; (2) teaching hospitals and medical schools tend to be located in large urban areas, where the Geographic Practice Cost Index portion of the payment is lower; (3) customary charges for teaching physicians may be higher than the areawide historical payments calculated for fee schedule transition; and (4) a number of academic medical practices had not signed Medicare participation agreements. For this study, 720 hospitals were selected, including all U.S. academic medical centers. The results indicated that there is no evidence that teaching physicians have been adversely affected by the Medicare Fee Schedule.

1995 Influenza Immunizations Paid for by Medicare: State and County Rates (6615) is a data book tied to the program begun in 1993 to stimulate the provision of flu vaccine to Medicare beneficiaries. It provides information about National, State, and county rates of vaccination in fee-for-service settings (managed care beneficiaries could not be counted because their encounter data are not reported).

The Department was legislatively required to report to the Congress on the relative quality of the care in the Medicaid program. HCFA's A Study of the Appropriateness, Process of Care, and Outcomes of Medical Care for Medicaid Patients (6302) examined the necessity, appropriateness, and effectiveness of selected medical treatments and surgical procedures for Medicaid patients. It assessed the variation that exists in the rate of performance of selected treatments and procedures on Medicaid beneficiaries for small areas within and among States. It determined that there are underutilized, medically necessary treatments and procedures for which failure to furnish could have an adverse effect on health status. The study had two phases. The first used extant data in HCFA to examine patterns of utilization for over 160 different conditions and procedures. It found that, for all conditions combined, utilization was higher in the Medicaid population and that the differences were higher for medical than for surgical conditions. Within Medicaid, medical conditions showed somewhat more small-area geographic variation than did surgical conditions. For all conditions combined, the Medicaid population tended to exhibit more small-area geographic variation than did the non-Medicaid population, but the pattern was not consistent across States. In the second phase, data were abstracted from medical records at 118 hospitals in California, Georgia, and Michigan for a selected series of index diagnoses: pediatric asthma, hysterectomy, and complicated labor and delivery. Five indicators were developed to assess the level of quality. In general, Medicaid women and children were treated in public hospitals, teaching hospitals, and hospitals with large bed capacities. The appropriateness of the decision to hospitalize was comparable between Medicaid and the privately-insured. And, once hospitalized, they received a similar level of care. Most importantly, the outcomes of care were essentially similar between the two payer groups, although the Medicaid patients were more severely ill. As a result, there were differences in outcomes that were not related to the inpatient quality of care provided. The inpatient care for the Medicaid patients was considered adequate and comparable with care received by privately-insured patients.

Evaluation of the Medicaid Uninsured Demonstrations (6298) assesses three demonstrations conducted in the States of Maine, South Carolina, and Washington. These demonstrations, implemented in response to a Congressional mandate under section 4745 of the Omnibus Budget Reconciliation Act of 1990, tested the effect of allowing States to extend Medicaid coverage to low-income families. The project examined the ability of the Medicaid programs to enroll significant numbers of eligible persons, looking particularly at the conditions under which eligible persons and their families are willing to participate in such programs, given their scarce financial resources. The evaluation also looked at the ability of the programs to induce adequate levels and the willingness of employers to participate in the programs.

The Inspector General of the Department conducted a demonstration of improved methods for the investigation and prosecution of fraud and abuse. The Evaluation of the Effectiveness of the Operation Restore Trust Demonstrations (6311) examined whether this more concentrated effort, rendered through the demonstration's partnership model, had a relatively greater impact on health industry fraudulent behavior than the same efforts conducted separately. Operation Restore Trust (ORT) involved a targeted set of initiatives that focused on a few States, a few areas of the Medicare benefit, and a few standard operating procedures (especially, failures to coordinate among the most central agencies and to exploit available data resources). The evaluation concluded that ORT turned around certain troubling trends of the early 1990's and has brought new vigor to government efforts to combat health care fraud and abuse. ORT had important effects across the full spectrum of organizations it sought to affect, and even had some important effects on organizations over the boundaries of the demonstration (e.g., Medicare contractors) that were not so directly orchestrated by the ORT initiative.

Assessing the Viability of All-Payer Systems for Inpatient Hospital Services (6594) explored the feasibility and possible structure of alternative approaches to developing such a system. It looked at the practicality of adapting or extending Medicare's Prospective Payment System to other payers. Two analyses were conducted to determine (1) whether diagnosis-related group-based payment rates for Medicare and non-Medicare patients should be adjusted based on hospital characteristics, and (2) alternative approaches in defining the Medicare standardized prospective payment rate. Current Medicare standardized prospective payment rates are based on the mean adjusted cost for a given category of hospital; high-cost hospitals are penalized under the assumption that their costs reflect presumed inefficiency. It developed hospital-specific measures of inefficiency and then excluded the costs of inefficient hospitals from the calculation of standardized payment rates. The conclusion was that a frontier cost function model approach is superior to the current system and can result in substantial program savings.

Monitoring and Evaluation of the Medicare Cataract Surgery Alternate Payment Demonstration: Final Evaluation Report (6295) assisted HCFA in tracking the 3-year demonstration that tested the concept of a single global Medicare payment for outpatient cataract procedures. The payment amount represented the cost for physician, facility, and intraocular lenses. The evaluation found that the project was reasonably successful in encouraging provider flexibility and in managing the bundle of services. There was ample anecdotal evidence that the cost-effectiveness incentives were working; e.g., cost reduction efforts by surgeons in the techniques and time required to perform the surgery and actions to standardize the intraocular lenses and other supplies and materials. Once operations began, government activities related to pricing were greatly reduced and limited to simple verification and payment of the global fee. The evaluator was not able to measure appropriateness, but in terms of quality control and utilization review, the demonstration was successful in that the outcomes were unchanged. The project offered some insights into the strengths and weaknesses of particular incentives in encouraging providers to participate. However, it provided only limited new information on the factors that influenced beneficiaries to participate.

HCFA completed an evaluation on the implementation of ambulatory patient groups (APGs) under the title, Toward a Prospective Payment System for Outpatient Services: Implementation for Outpatient Services-- Implementation of APGs by State Medicaid Agencies and Private Payers. (6320) The focus of this study was the Medicaid outpatient prospective payment system that groups patients for payment purposes rather than paying on a cost basis. These groups are similar in concept to diagnosis-related groups, which form the heart of Medicare's prospective payment system for inpatient care. It involved a case study of Iowa's implementation of the APG system and an analysis of the reimbursement methodology. All six payers that had operational experience tailored their APG systems to their own priorities and markets. They all implemented the system without major incident. The payers reported success in reducing outpatient costs where that was the immediate goal, and they believe the system encourages higher-cost facilities to reduce costs and rewards lower-cost facilities. The views of providers were more mixed. Though they reported generally adequate overall payment, they also viewed the system as complex, could not calculate expected payment, and often let their computer systems group related claims rather than consolidating them prior to submission, as was the intent of the system. No evidence was found of much behavioral response by providers, but that could change if the system is implemented by a large payer such as HCFA.

The Second Update of the Geographic Practice Cost Index: Final Report (6611) was produced and is scheduled to be fully phased in by January 1, 1999. One-half of the total change will be implemented in 1998. A comparison between the 1997 and 1999 Geographic Practice Cost Indexes (GPCIs) shows that (1) there is no change between the physician work GPCI or the non-physician employee wage index of the practice expense GPCI, except that they are reweighted with 1994 county relative value units; (2) the office rental index of the practice expense GPCI is updated with FY 1996 Fair Market Rents (from the Department of Housing and Urban Development); (3) county rental adjustment factors are eliminated for all metropolitan areas (except for the New York City Primary Metropolitan Statistical Area); (4) the malpractice GPCI is updated with 1992, 1993, and 1994 premium data, and specialty premiums are weighted by national relative value unit proportions rather than allowed charges; (5) there is no change in the GPCI cost shares, which measure the share of the various inputs in total practice revenues; and (6) the county relative value units used to weight averages to county input prices for each Fee Schedule Area are updated from 1992 to 1994. The project determined that the changes in the Medicare Fee Schedule resulting from implementation of this GPCI update will be less than 2.5 percent for each Fee Schedule Area.

The Payment of Pharmacists for Cognitive Services: Results of the Washington State C.A.R.E. Demonstration Project (6528) is a report on a demonstration project called Cognitive Activities and Reimbursement Effectiveness (C.A.R.E.). It is operated under the premise that (1) direct reimbursement for cognitive services would remove financial barriers associated with the pharmacists' provision of these services; and (2) that this change would result in more consultations, with a subsequent impact on costs and outcomes from drug therapy. Data were drawn from three groups of approximately 100 pharmacies in the State of Washington. The project found that (1) pharmacists who received a financial incentive consistently reported higher cognitive service intervention rates than did those who did not receive financial incentives; (2) about half of all documented service problems were for patient-related matters (about one-third were for drug-related problems, about 17 percent were for prescription-related problems, and the residual 1.4 percent were for non-drug problems); (3) a drug therapy change occurred as a result of 28 percent of all pharmacist cognitive services; (4) for each cognitive service that was associated with any type of drug therapy change, the average downstream drug cost savings was about $13.05 (but these savings varied by type of drug therapy); and (5) the demonstration generated savings of about $78,000 in Medicaid prescription drug expenditure over the 18-month period.

The Consequences of Paying Medicare HMOs and Health Care Prepayment Plans (HCPPs) (6314) compared the cost effectiveness of such plans to fee-for-service (FFS) and health maintenance organizations (HMOs), which operate on a risk basis. The main question examined in this study was whether HCFA payments would have increased or decreased had these organizations remained risk contractors. The evaluator found that costs to HCFA increased substantially under the cost programs, HMO, and Health Care Prepayment Plan (HCPP). Furthermore, most of the cost-based plans were found to have favorable selection. Total dollar losses to HCFA for 1993 due to the cost, HMO, and HCPP programs were estimated to be $185 million relative to fee-for-service, and $4 million relative to risk-based payments. Dropping the seven plans that converted to risk or dropped out of Medicare since 1993 would have resulted in estimated losses of $210 million compared to FFS, and $90 million compared to risk contracting.

The Costs and Consequences of Case Management for Medicare Beneficiaries: Final Report (6291) studied the appropriateness of providing case management services to beneficiaries with catastrophic illnesses and high medical costs. It tested case management as a way of controlling costs in the fee-for-service sector. The target conditions and case management protocols differed in each site, though all three generally focused on increased education regarding proper patient monitoring and management of the target chronic condition. All three sites anticipated reduced hospitalizations and medical costs compared to the beneficiary control groups. The evaluation found that (1) the projects successfully identified and enrolled populations of Medicare beneficiaries likely to have much higher than average Medicare reimbursements during the demonstration period; (2) each project met with unexpectedly low levels of enthusiasm for the demonstration from beneficiaries and their physicians; and (3) despite high levels of satisfaction among the high-cost, chronically ill beneficiaries who eventually participated, the projects failed to improve client self-care or health, or reduce Medicare spending. Comparisons of health status, functional status, and expenditures between the control and intervention groups showed no improvements resulting from the case management intervention.

The Validation of Nursing Home Quality Indicators Study (6612) developed a system by which a series of indicators could be used, in conjunction with claims data, to monitor quality of care in nursing facilities. Fifty such indicators were developed and validated. The indicators focused on three broad problem areas: adverse outcomes, lack of therapy, and inappropriate pharmaceutical treatments. The indicators were designed to be used in an automated system to continuously monitor the quality of care provided. The study found that (1) it is possible to base such indicators on Medicaid and Medicare claims data; (2) as a whole, the indicators were better at predicting the absence of a quality issue (suggesting that the negative finding represented the lack of a quality concern in a specific nursing home); and (3) using covariant diagnoses was useful to adjust for the risk of a resident receiving an indicator tag.

The Evaluation of the Community Nursing Organization Demonstration Interim Evaluation Report (6306) examined a set of demonstration projects that tested a capitated, nurse-managed system of care. It was designed to promote timely and appropriate use of community health services and to reduce the use of costly acute care services. More than 5,400 beneficiaries were enrolled in the four Community Nursing Organization (CNO) demonstration sites. The evaluation report covers the first 15 months of operations. The applicants have been healthier and more independent, on average, than the general Medicare population in their respective market areas. Analyses of measures of death, physical functioning, and satisfaction with care detected no statistically significant differences. Analysis of a subsample of persons who were impaired in Activities of Daily Living indicated that members of the CNO treatment group were more likely to improve (or less likely to deteriorate further) than members of the control group. An even stronger result, in the same direction, was found for individuals who were limited in Instrumental Activities of Daily Living.

The Evaluation of the Utah Prepaid Mental Health Plan (6293) focuses on a demonstration project in which three mental health centers provided services to all Medicaid beneficiaries in their catchment areas (these areas included over 50 percent of all Utah Medicaid beneficiaries). It was expected that this program would control the cost inflation and improve patient outcomes in mental health services. The study examined (1) the organizational and financial characteristics and their evolution over time, (2) the impact on the organization of service delivery and use of services, and (3) the financial impact on providers and the Medicaid program in Utah. The development of the demonstration proceeded fairly smoothly, albeit somewhat more slowly than planned. Three centers chose to enter the demonstration and eight chose to not join. Several changes in the program's environment affected the development during the initial 3 years. There were general expansions in children's programs, day treatment programs, and case management. The community mental health centers began to focus staff attention on ways to increase the efficiency with which discharges from the hospital were managed and on the way outpatient services were delivered--all while maintaining quality of care. The results suggest that the demonstration reduced expenditures on acute inpatient care by reducing admissions, without significant effects on outpatient expenditures and visits. Financial impact was possible in that the demonstration altered the way in which Medicaid funds flowed to the participating community mental health centers and it redefined their responsibilities with respect to the provision of services. In addition, there were relatively few financial differences between contracting and non-contracting community mental health centers. Thus, the decision to participate appears to have had a null effect on the site's financial performance. The payments to the participating sites were less than estimated. In the second method of looking at payments to the sites, there were savings on inpatient care, which were partially offset by increases in outpatient care for a net overall savings. The overall estimates of total program savings showed this same pattern of inpatient savings, diminished by outpatient increases. However, these estimates are very sensitive to the methods used to project trends.

The Limited-Service Hospital Pioneers: Challenges and Successes of the Essential Access Community Hospital/Rural Primary Care Hospital (EACH-RPCH) Program and Medical Assistance Facility (MAF) Demonstration (5200.1) concerned a legislatively mandated attempt to assist States in maintaining access to health care services in rural areas. The program awarded grants to several States to develop rural health plans, establish rural health networks, and create a limited service alternative for communities that could no longer support a full-service hospital. These limited service providers and other health care providers were organized into rural health networks. This evaluation examined the development, implementation, and early operating experience of the program. In addition, it looked at a forerunner program, the Montana Medical Assistance Facility Demonstration. It found that these limited service hospital models occupy a niche in rural health care system by permitting an alternative for outright closure. However, there were considerable limitations to participation. The financial benefits varied by facility, depending on such things as cost structure, provider supply, and patterns of use in the specific community. It found that program flexibility and the linkage of primary care services to developing networks was essential.

The Comparative Study of the Use of EPSDT and Other Preventive and Curative Health Care Services by Children Enrolled in Medicaid: Final Project Synthesis Report (6236.7) included (1) a description of the Medicaid program in the four study States (Michigan, Georgia, Tennessee, and California) and these States' responses to the 1989 legislative changes relating to children's and Medicaid coverage and also to the Early Periodic Screening, Diagnosis, and Treatment (EPSDT) program; (2) an analysis of the impacts of the State's responses on participating providers and enrolled children; and (3) an assessment of the national context. The States' efforts to increase provider participation in Medicaid and EPSDT were successful in increasing provider participation and average caseloads among all types of providers (except for dentists in Michigan). However, the increased supply may not have been adequate to meet the increased demand because the ratios of Medicaid children to participating providers and percentages of counties with provider shortages rose from 1989 to 1992 in all four States. Only 42 to 54 percent of Medicaid children were recommended to have preventive care visits and, of these, only 36 to 59 percent made such visits. Compared with other low-income children, Medicaid coverage did increase service use and improved access to illness-related care. However, Medicaid children had very low completion rates for age-appropriate immunizations.

The Evaluation of the Demonstration to Improve Access to Care for Pregnant Substance Abusers (6297) sought to evaluate project effectiveness in (1) outreach and assessment; (2) expansion, integration, and coordination of program services; and (3) improvement of client case management. The evaluator examined the effects of access to prenatal care and substance abuse treatment services on the health of drug-addicted pregnant women and birth outcomes of their infants. The evaluation showed that the number of abusers enrolled in the demonstrations was low relative to all pregnant substance abusers in the area, since women were reluctant to be identified. The project found higher enrollment rates in States that implemented broad-based outreach efforts, and that higher levels of participation and greater retention in substance abuse treatment resulted in higher-birthweight infants.

An Evaluation of Implementation of the Medicaid Community Supported Living Arrangements (CLSA) Program in Eight States (6304) was designed to test the effectiveness of developing a continuum of care concept as an alternative to the Medicaid-funded residential services provided to individuals with mental retardation and related conditions. The program served individuals who are either living in the community independently, with their families, on in homes with three or fewer other individuals receiving the same services. These services were provided in eight States between fiscal years 1991 and 1995. By the end of the program, only 73 percent of the expected number of individuals were receiving services in the Community Supported Living Arrangement (CSLA), and the expenditures were only 58 percent ($38.4 million) of the amount authorized.

The Impact of Report Cards on Employees: A Natural Experiment (6312) sought to determine whether the dissemination of information about health plans to consumers (who choose health plans within a managed care competition framework) would influence their knowledge of plan characteristics, attitudes toward the plans, or choice of plan. Investigators found that report cards had no effect on employees' knowledge of health plans, their attitude about quality of health plans, or their choices in selecting a managed care health plan. The study did find that health plan characteristics (such as price) were strongly related to health plan choice. The investigators suggest that consumers do not seem to be influenced by the information in any of the ways measured in this study.

Evaluations in Progress

HCFA currently supports 19 evaluation projects in progress, 13 of which are scheduled to be completed in FY 1998. These projects provide information for continued monitoring of the Medicare and Medicaid programs and assess the impacts of HCFA's Medicare and Medicaid demonstration projects.

The Evaluation of Medicare SELECT (5966.3) examines a pilot Medicare supplemental insurance product under which full Medigap benefits are paid only when services are provided by the plan's provider network. Case studies, primary surveys, and administrative data (claims) analyses are conducted. The analytical portion of the project compares cost and use of Medicare and supplemental services, selection effects, beneficiary satisfaction, and physician practice patterns to other Medigap options. To date, the evaluation has found no health status differences or quality of care received by SELECT and non-SELECT beneficiaries. In addition, there were no differences in overall satisfaction levels between these populations. Premiums for SELECT policies were almost always cheaper than the same company's own standard Medigap policy for any age group. In two out of three years examined, SELECT policies were generally more expensive for older beneficiaries when compared to community rated policies. Medicare costs generally increased as a result of Medicare SELECT. The costs increased in five States, decreased in three States; and were not affected in three other States. Since the States studied were not chosen to be representative of the Nation, their experience is not necessarily indicative of a national program. Consequently, the effect of SELECT on Medicare program expenditures appears to be mixed at best.

The Medicare Participating Heart Bypass Center Demonstration Extended Evaluation (5985.5) continues to study the feasibility of a negotiated, all-inclusive pricing arrangement for coronary artery bypass graft surgery while maintaining high-quality care. The project looks for any net cost savings to the Medicare program, any volume increases at the demonstration sites, the aspects of the demonstration that were attractive to beneficiaries and referring physicians, and whether the quality of care at the sites was equivalent to that provided prior to the demonstration.

The Medicare HMO Evaluation (6294) updates the findings of an earlier study of Medicare risk health maintenance organizations. That study found that Medicare paid 5.7 percent more for plan enrollees than would have been spent under fee-for-service. The current study looks at disenrollment, beneficiary satisfaction, quality of care, and selection and savings.

The Evaluation of Medicaid-Managed Care Programs with 1915(b) Waivers (6290) will provide information on the extent to which various features of the managed care waiver projects contribute to the ability of the Medicaid program to deliver cost-effective care to eligible populations.

Maximizing the Effective Use of Telemedicine: A Study of the Effects, Cost Effectiveness, and Utilization Patterns of Consultation via Telemedicine (6303) is a comprehensive evaluation of HCFA's telemedicine demonstration projects. The project assesses alternative payment options for the providers of telemedicine services, such as fee-for-service, facility payments, or capitation. Objective measures of the cost of telemedicine services in different clinical settings are used, both from the payer's and the physician's perspective. Provider and patient satisfaction, utilization measures (e.g., physician visits, hospitalizations or rehospitilizations, and frequencies of complications or comorbidities), and the effect of telemedicine on practice patterns are included.

The Drug Utilization Review Evaluation Contract (6296) is examining the impacts of retrospective and prospective review, which included the payment of pharmacists for cognitive services. Demonstrations of on-line prospective drug utilization review (DUR) in Iowa and of cognitive service by Washington State pharmacists provided the primary data for evaluation. This evaluation looks at the two types of DUR program effects in preventing adverse drug-related events by incorporating and analyzing the program participants' pharmaceutical and medical information.

The Evaluation and Technical Assistance of the Medicare Alzheimer's Disease Demonstration (6305) is being conducted to determine the effectiveness, cost, and impact on health status and functioning of providing comprehensive in-home and community-based services to beneficiaries with dementia. Two models of care will be studied. Both include case management and an extensive package of services not presently covered under Medicare, such as adult day care, homemaker/personal care services, companion services, and caregiver education and training. The models are differentiated by the intensity of the case management provided to beneficiaries and their families and by the monthly amount of Medicare payments for project services. The purpose of the evaluation is to measure the cost and effect on the beneficiaries' functional level and institutionalization rate, as well as its impact on caregiver burden and stress.

Evaluation of Phase Two of the Home Health Agency Prospective Payment Demonstration (6308) examines the second phase of a program designed to test the effectiveness of using prospective payment methods to reimburse Medicare-certified home health agencies (HHAs). In this demonstration a per-encounter payment method is being tested. The study looks at the effects of this payment method on HHAs operations, service quality, and expenditures. It also analyzes the relationship between patient characteristics and the cost and utilization of home health services.

The Examination of the Medicaid Expansions for Children (6300) uses enrollment and expenditure trends to ascertain the impact of the 1989 program changes. It will look at target population penetration, and the impact of State policies (and the eligibility group) on enrollment, expenditures, and utilization of services.

The Project Demonstrating and Evaluating Alternative Methods to Assure and Enhance the Quality of Long-Term Care Services for Persons with Developmental Disabilities through Performance-Based Contracts with Service Providers (6310) tries to determine whether and how well the implementation of new approaches to quality assurance--with outcome-based definitions and measures of quality--will replace input and process measures of quality in this population group.

In future years, several other evaluations are expected to be completed.

The Evaluation of the Medicare Choice Demonstration (6292) assesses the feasibility and desirability of new types of managed care plans for Medicare. These plans can be integrated delivery systems and preferred provider organizations.

HCFA is conducting demonstrations testing the payment of skilled nursing facilities on a prospective basis for Medicare and Medicaid services. Medicare currently reimburses such facilities on a retrospective cost basis. This demonstration uses a case-mix classification system (called Resource Utilization Groups) to classify patients and establish payment rates. The Evaluation of the Nursing Home Case-Mix and Quality Demonstration (6307) seeks to estimate specific behavioral responses to the prospective payment and to test hypotheses about aspects of such responses. The main goal of the project is to estimate the effects on the health and functioning of the nursing home residents, their length of stay, and use of health care services; on the behavior of the facilities; and on the level and composition of Medicare expenditures.

The State of Delaware is experimenting with its Medicaid program. The State believes that by enrolling Medicaid children in a managed care system, it will benefit from a higher level of coordinated care and reduced costs. The Evaluation of the Demonstration Entitled Delaware Health Care Partnership for Children (6288) is examining this project.

HCFA is sponsoring a wide variety of waiver-based demonstrations that give States the opportunity to experiment with their Medicaid programs. As these demonstrations begin, HCFA will undertake examinations of their impact. The Evaluation of the Oregon Medicaid Reform Demonstration (6166), the Evaluation of the State Medicaid Reform Demonstrations (in Hawaii, Rhode Island, and Tennessee) (6289), and the Evaluation of the State Medicaid Reform Demonstrations (in Kentucky; Los Angeles County, California; Minnesota; New York; and Vermont) (6289.1) are three such projects. More evaluations will be initiated as more State demonstrations are begun.

In 1977, the Congress authorized a new type of provider, the Rural Health Clinic, which must be located in a rural health professional shortage area, medically underserved area, or Governor-designated shortage area and must make use of mid-level practitioners. Rural Health Clinics are reimbursed on a cost basis by the Medicare and Medicaid programs. Their numbers have been growing at a relatively rapid rate. The Evaluation of Rural Health Clinics (6299) examines the reasons for this growth, the impact on access to care for these rural populations, and the costs to the Federal government and States.

The Evaluation of the Program of All-Inclusive Care for the Elderly Demonstration (6309), known as PACE, examines replications of a unique model of managed care service delivery for very frail community-dwelling elderly, most of whom are dually eligible for Medicare and Medicaid and all of whom are assessed as being eligible for nursing home placement. The core services include adult day health care and multidisciplinary team care management through which all health and long-term care services are arranged. The evaluator is looking at the demonstration sites before and after assumption of financial risk to see if the replicates are cost effective relative the current Medicare and Medicaid services. They are also looking at the decision to enroll in order to understand how PACE enrollees differ from the eligible beneficiaries who choose not to enroll.

Health Resources and Services Administration

MISSION: To improve the Nation's health by assuring equitable access to comprehensive, quality health care for all.

Evaluation Program

The purposes of the Health Resources and Services Administration (HRSA) evaluation program are to enhance strategic and performance planning, strengthen budget and legislative development, and improve program management. The Agency's evaluation priorities are to develop and support performance measurement, assess program implementation, and conduct crosscutting policy analysis and research.

Performance measurement includes technical and training activities to strengthen the Agency's capacity to assess program performance, as well as studies to assess program outcomes. These activities, mandated by the Government Performance and Results Act (GPRA), will provide data for ongoing program monitoring and evaluation and for developing annual performance plans and reports. HRSA completed a study in September 1995 to establish a performance measurement baseline for all operating programs, with the intent of specifying valid and useful indicators and measures for all programs by September 1998. Over the past three years, HRSA has made significant progress in performance management, such as in strengthened linkages among strategic planning, evaluation, and budget decisionmaking.

Assessment of program implementation includes a broad range of efforts to assist the Agency in adjusting program development and management as the priorities of the Executive Branch and the Congress change. HRSA programs are entering into new arrangements for delivering services, providing health professions education, and encouraging the development of systems reform.

Crosscutting policy analysis and research includes efforts to build capacity and conduct studies to clarify the environmental shifts affecting HRSA programs and to improve HRSA's ability to document performance and impact through better measurement tools and data sources. The term, "crosscutting," refers to issues that bear on the programs of two or more HRSA Bureaus, such as trends in numbers and characteristics of the uninsured, and the viability of the health safety net. Development of this capacity at the Agency level began in FY 1997; work in FY 1998 and beyond will build on this foundation.

HRSA is striving to improve the quality of evaluation products. Potential approaches for this initiative include (1) an increase in the level and amount of technical assistance available to staff across the Agency; (2) expansion in the proportion of work done intramurally or with short-term help from individual experts; (3) encouragement of HRSA staff to participate in evaluation courses offered by local universities; and (4) establishment of an external evaluation advisory committee to consult on evaluation strategies in terms of content, approaches, and dissemination. Evaluation capacity is being strengthened first in the central evaluation unit, with the intent of making similar improvements in evaluation resources of the Bureaus in the future.

HRSA is continuing to place a major emphasis on dissemination of findings and information about the utilization of study results. A broader dissemination strategy, including an assessment of opportunities for using electronic media more effectively, will also be an important component of the quality improvement activities now under development.

Summary of Fiscal Year 1997 Evaluations

Studies related to the three key areas of HRSA's mission are described first, followed by crosscutting projects.

Eliminating Barriers to Care

One significant study, Exploratory Evaluation of Rural Applications of Telemedicine (5749), provides national baseline data on the status of telemedicine in health care for rural populations. The project included a mail survey of all non-Federal hospitals located outside metropolitan areas in the summer of 1995, plus a short survey of hospitals doing only teleradiology and a longer survey of hospitals applying telemedicine for purposes beyond radiology. The major findings were that (1) rural telemedicine is in the early stages of development but is rapidly expanding; (2) telemedicine networks are complex, with an average of four "spoke sites," two hubs, and four facilities that provided and received consults; and (3) Federal and State grants and support from hospitals are common sources of direct funding, but fewer than 25 percent of hub facilities had negotiated payment with insurance carriers and many had not begun negotiations. Study results have been used in forming the FY 1997 guidelines for the Rural Telemedicine Grant Program, in part to encourage the use of telemedicine in community settings. As requested, early findings were shared with congressional staff, who then specified priority areas in the authorizing legislation. The report was distributed to Federal agencies funding telemedicine, to State Offices of Rural Health, and through the World Wide Web. Finally, findings are informing the design of a common telemedicine evaluation instrument for HRSA grantees and for consideration by other Federal sponsors.

Another study relating to access is Institutional Factors That Influence the Likelihood of Living-Related Kidney Transplant Operations--Phase II (4917.1). The purpose of this project was to determine (1) the willingness to conduct, and frequency with which the transplant community does conduct, kidney transplants from living-related donors; (2) the medical and non-medical institutional factors influencing the likelihood that transplant professionals will suggest and conduct such transplants; and (3) the circumstances and conditions under which a transplant team that does not encourage these transplants would refer a patient to another transplant center for evaluation. One finding was that over half of kidney transplant centers lacked patient education materials about live kidney donors. A summary report has been sent to all kidney transplant centers. In addition, discussions have been held with groups such as the National Kidney Foundation about ways to apply the findings, such as through development of educational materials for national distribution.

In the health professions area, a major statistical study produced a technical and an analytic report; the title of the latter is The Registered Nurse Population, March 1996: Findings from the National Sample of Registered Nurses (6367). A contractor prepared the sampling design and conducted the survey, which is the most comprehensive source of statistics on registered nurses (RNs) with current licenses to practice in the United States. The survey resulted in a data base of information on 29,950 nurses, which was then analyzed by HRSA staff. The report provides information on the number of nurses; their educational background and specialty areas; their employment status, including type of employment setting, position level and salary; their geographic distribution; and their personal characteristics, including gender, race and ethnicity, age, and family status. HRSA is using the study products to guide nursing education programs, to prepare biennial reports to Congress on characteristics of the RN population, and for projecting the supply and demand for nurses at a national level. The data base and analytic report have been shared with the nursing community, which looks to the Department to maintain and update this data base.

Eliminating Disparities in Health Status

Another major quantitative effort was the Community Health Center User and Visit Study: Survey Methodology Report (5737). This project gathered, for the first time, nationally representative data about Community Health Center (CHC) users and the services provided to them. The study had two components: (1) personal interviews with 1,932 patients selected randomly from a representative sample of 48 CHCs, and (2) data extraction from 2,878 medical records concerning randomly selected encounters during 1994 at the same centers. Data on users were gathered in a manner to permit comparison with results from the National Health Interview Survey. Similarly, information from the visit (encounter) survey will be compared with results from the National Hospital Ambulatory Medical Care Survey. The CHC data are being analyzed to answer questions on such topics as the race and ethnicity of users, health-risk behaviors presented, most prevalent diagnoses, services utilized, and the extent of monitoring of compliance with treatment prescribed for chronic conditions. In addition, the data are being reviewed to form a profile of CHC visits in comparison with studies of the general population in terms of conditions, diagnoses, type of provider seen, and disposition. Analysis of data from the initial surveys continues intramurally, as well as through a current study noted below.

Assuring Quality of Care

A major concern in recent years has been the spread of tuberculosis (TB) among persons with HIV/AIDS. Examining Relationships Between HIV/AIDS and Tuberculosis Programs in Title I Eligible Metropolitan Areas (6359) was designed (1) to determine the extent to which Ryan White Comprehensive AIDS Resources Emergency (CARE) Act grantees were giving guidance to providers on TB screening and prophylactic and therapeutic regimes, and (2) to describe models of collaboration between HIV and TB control programs within Eligible Metropolitan Areas (EMAs). The study found that TB services varied across EMAs and across care settings within them. As of 1995/1996, Federal and State funding policies and staffing patterns hampered collaboration between HIV and TB programs. However, the study identified mechanisms to promote TB-HIV services collaboration, fostered in some places by outbreaks of TB and multidrug-resistant TB. HRSA has used the results to inform Title I program guidance and to develop the training and technical assistance plan for FY 1998. The report has been sent to Title I grantees and to the Centers for Disease Control and Prevention (CDC) TB control grantees. Results were also presented at the 1995 International AIDS conference in Vancouver, Canada, and in September 1997 at the CDC conference entitled, "Issues for HIV and TB Co-Infected People."

The increasing adoption of managed care by public insurance programs, such as Medicaid, has raised concerns about whether capitated reimbursement affects quality of care provided to people with chronic illnesses such as HIV/AIDS, or with other special health care needs. One way to address these concerns is to create risk-adjusted capitation rates to provide equitable reimbursement for the care of patients with costly illnesses, an approach several States had proposed or implemented by 1997. In May of last year, HRSA convened an invited conference to examine issues surrounding risk adjustment for HIV/AIDS, in collaboration with other HHS organizations (the Office of the Assistant Secretary of Planning and Evaluation and the Health Care Financing Administration), the Henry J. Kaiser Family Foundation, and the National Academy of State Health Policy. The HIV Capitation Risk Adjustment Conference Report (6894) presents perspectives of participants (representing persons living with HIV/AIDS, providers, managed care organizations, and States); methodological issues; and research needs. Conference findings have been used in technical assistance to States on coordinating CARE Act programs with managed care. HRSA has distributed the report to all CARE Act grantees and State Medicaid directors.

Crosscutting Issues

One recently completed study concerned managed care and its relevance for vulnerable populations. The Availability of Primary Care Services Under Medicaid Managed Care: How 14 Health Plans Provide Access and the Experience of 23 Safety Net Providers and Their Communities (6366) examined strategies used by health maintenance organizations (HMOs) to generate access to health care services for Medicaid enrollees, who often live in inner-city and rural, underserved areas. The project also explored the implications of these strategies for overall access in underserved areas and for existing care providers. Third, the study considered how these strategies affect traditional Medicaid providers in the community. The study concluded that Medicaid managed care has had an overall positive effect on the availability of primary care to low-income residents in the 14 communities, and that traditional safety net providers have not had to reduce services. But the future viability and range of services of public sector-supported providers might be endangered in a more competitive health system. The findings will be useful to policymakers concerned with the provision of care for vulnerable populations, providers in underserved areas, and those attempting to assure an adequate supply of health personnel. The study has been noted in the professional press and will be referenced in a journal article now under preparation.

Evaluations in Progress

Projects in progress are presented first according to HRSA's three mission goals; crosscutting projects follow.

Eliminating Barriers to Care

Begun in September 1994, the Evaluation of the Effectiveness of Community Health Centers: Implementation Phase (5736) is describing CHC delivery model(s), measuring the impacts of the model(s), and identifying factors that contribute to the observed impacts. The study is examining the experience of CHC users and non-users in both fee-for-service and managed care arrangements though a sample of 50 CHCs in 10 States. The project has three components: (1) an analysis of utilization and expenditures, based on HCFA State Medicaid Research Files for eight States; (2) an analysis of outcomes and content of care for four conditions (otitis media, asthma, hypertension, and diabetes) based on CHC medical records and the literature on medical treatment norms; and (3) a description of CHC delivery systems using primary data collection (a mail survey) and secondary sources. The analysis of utilization and expenditures is addressing casemix and differences between exclusive and regular CHC users.

Effectiveness of the National Health Service Corps (6357) is examining the various mechanisms for training and recruiting providers, placing them in underserved areas, and retaining them in primary care and service to vulnerable populations. The project includes mail surveys of three random samples: (1) National Health Services Corps (NHSC) alumni clinicians; (2) current NHSC clinicians; and (3) administrators of sites staffed with NHSC clinicians. (Some of the sites are CHCs and others are free-standing.) The study also involves review of secondary data on individual NHSC participants, sites that are eligible for NHSC assignees, characteristics of designated shortage areas, and site vacancies available for assignment of obligated scholarship providers.

In another critical policy area, HRSA is conducting a study entitled Enrollment, Utilization and Cost-Forecasting Models for Use in State AIDS Drug Assistance Programs (6806). This study is exploring the effect of the introduction of protease inhibitors on medical service utilization for use in State AIDS Drug Assistance Programs (ADAP). During 1996, five protease inhibitors became available; this class of drugs has been shown to have superior outcomes in treating HIV. However, although pharmaceutical interventions represent a sizeable portion of the cost of care for HIV-infected patients, the efficacy of these drugs has not been shown outside of controlled clinical trials. This study is using client-specific data from a geographically diverse population to identify personal factors that may modify the link between the introduction of protease inhibitors and services use.

Eliminating Disparities in Health Status

The National Evaluation of the Healthy Start Program (5610) continues as a longitudinal study of the development, implementation, and outcomes of the comprehensive maternal and child health delivery systems at the original 15 Healthy Start sites, which were funded initially as 5-year demonstration projects. A sixth year was added in FY 1996. This study includes an assessment of changes in the health status of pregnant women and infants across the sites. The study was designed to answer four questions: Did the Healthy Start initiative succeed? If so, why? If not, why not? What would be required for a similar intervention to succeed in other settings? The study includes both process and outcome components. Findings from both will contribute to analyses bearing on replication of successful approaches for reducing infant mortality, and on improving program effectiveness in general. Study results will be used to guide the development and implementation of Healthy Start projects funded subsequently, and will be widely shared with the public health community for application in settings funded under other auspices.

Analysis of CHC User Survey: Selected Conditions (6805) is assessing portions of the survey data from the Community Health Center User and Visit Study (see PIC ID No. 5737). The purpose is to describe the self-reported process of care and outcome measures for CHC users with certain conditions: hypertension, high serum cholesterol, diabetes, asthma, and pregnancy. The analysis includes the demographic characteristics, income and educational levels, health status, and activity limitations of these patients as compared with (1) CHC users without the selected conditions, and (2) the general population. Data on the general U.S. population will be from the National Health Interview Survey and the National Hospital Ambulatory Medical Care Survey. Findings will be used, in part, to determine whether care provided at CHCs is appropriate and conforms to the specifications of Healthy People 2000.

Assuring Quality of Care

One of HRSA's critical concerns is cultural competence of care. A study begun in FY 1997, Development and Refinement of Tools for Monitoring Cultural Competence in Managed Care (6817), is analyzing successful experiences of managed care organizations in California in meeting the needs of culturally and linguistically diverse Medicaid populations. A second purpose of this study is to plan for the validation of tools for measuring cultural competence among individual providers. The new study builds on Tools for Monitoring Cultural Competence in Health Care (6352), which was completed in FY 1996.

Direct Support for Conduct of Evaluation Studies at Selected CARE Act Pilot Sites (6807) is providing assistance to six Title I and Title II grantees in analyzing data sets and developing models to assure the effectiveness of their primary care and supportive services. These local studies constitute a first step in developing future "sentinel" studies to create a picture of the national impact of these CARE Act programs. These initial projects include three types: (1) examining the association of CARE Act-funded services with clinical and other outcomes for patients with HIV; (2) developing models to assess the effects of managed care on fund allocations and service delivery; and (3) developing and pilot testing models to analyze the effectiveness of Title II care consortia in evaluating needs, setting priorities, and delivering services.

In addition to studies relating mainly to one mission area, HRSA has ongoing projects focused on the capacity of two HRSA Bureaus to measure performance across the organization. The first of these projects, which concerns the Maternal and Child Health Bureau (MCHB), is the MCHB GPRA Maturity Project (6811). This study, building on a series of program-specific efforts, is providing coordination of performance measurement activities to produce coherent performance data for the Bureau as a whole. The project also provides technical assistance in working with grantees and others to promote acceptance and implementation of performance measures throughout the maternal and child health community, and training for MCHB program staff in the ongoing use of performance measures. The intent is to complete the development of a fully operational performance management system.

A Bureau of Health Professions (BHPr) project also illustrates the use of evaluation to develop performance measures that encompass all Bureau programs. Development of BHPr's Comprehensive Performance Monitoring System (6800) is helping the Bureau implement its monitoring system. In preparation for meeting GPRA requirements, BHPr has developed goals, outcomes, and indicators with respect to workforce quality, supply, diversity, and distribution. This project, begun in FY 1997, is using a data collection instrument that was tested through an earlier pilot study to begin nationwide implementation of the data collection. The study includes technical assistance to applicants and grantees in gathering the required data and completing the data collection instrument. This project will involve grantees for the first time in submitting data electronically using the new instrument. The study includes development of software, training in its use for Bureau staff and grantees, and a help desk to provide ongoing technical assistance during implementation of the reporting system. The study will also include an evaluation of the data collection instrument and electronic implementation, the training and technical assistance, and the quality of data collected. Results of the project will be used to determine whether changes are needed in the current approach for collecting and analyzing program data, and will be shared with the public through an article in a professional journal.

Crosscutting Issues

Through Managed Care and the Safety Net Providers (6815), HRSA is examining the impact of Medicaid managed care on the fiscal and operational viability of safety net providers in primarily ambulatory and primary care settings. Among the types of providers to be included are Community and Migrant Health Centers, Maternal and Child Health programs, Ryan White CARE Act grantees, and rural health centers. A key topic being explored is the effect of a more competitive, market-oriented health care system on capacity to serve vulnerable populations. Policy and program interventions to deal with possible shortfalls will be identified.

A collaborative effort with NCHS, Person-Place Approximations and Utilization vs. Health Status Analysis of the National Health Interview Survey (5614) is addressing three questions: (1) What common characteristics are shared by communities with problems in access to health care? (2) Can these characteristics/

problems be measured directly, or through good proxies from data that can be easily obtained? (3) What is the appropriate aggregation of geographical units to use in describing small areas? The intent is to gain a better understanding of the ways in which the characteristics of particular areas influence the behavior of respondents to the National Health Interview Survey in order to use the knowledge for predicting access problems in small areas for the entire U.S. population.

Finally, HRSA Crosscutting Community Case Studies (6814) is examining the nature, extent, and impact of the interaction of HRSA programs in selected "sentinel" communities in order to determine whether inflows of HRSA resources at the local level combine in a mutually reinforcing way, operate independently, or work at cross-purposes. This project builds on two ongoing studies sponsored by other organizations. The Robert Wood Johnson Foundation is developing baseline information for tracking changes in the health care market in 12 metropolitan statistical areas. The other project, sponsored by the HHS Office of Disease Prevention and Health Promotion, is addressing how market-driven changes are affecting the level of effort of public health departments in providing ten essential public health services. Sites for the HRSA study are three urban communities (Boston, Cleveland, and Phoenix) in the Johnson Foundation study. This project is HRSA's first attempt to test a methodology for measuring the interactions and collective impacts of the Agency's programs on communities, which can then to used for conducting prospective and long-term evaluations.

Indian Health Service

MISSION: The mission of the Indian Health Service, in partnership with American Indian and Alaska Native people, is to raise their physical, mental, social, and spiritual health to the highest level.

Evaluation Program

The goal of the Indian Health Service (IHS) is to assure that comprehensive, culturally acceptable, personal and public health services are available and accessible to American Indian and Alaska Native people. The importance of evaluation in supporting this goal has increased significantly in recent years and includes American Indians and Alaska Natives as the primary stakeholders in defining the purpose, design, and execution of evaluations. The stakeholders use the end product of the evaluations, and are typically the population or groups most likely to be affected by the evaluation findings. The IHS has formally adopted the principles of a responsive evaluation practice to address the needs and concerns of American Indians and Alaska Natives.

The evaluation program of the IHS is managed by the Office of Public Health, Staff Office of Planning, Evaluation, and Research (OPH-SOPER), which provides national leadership and consultation for IHS and Area Offices on strategic and tactical planning, program evaluation and assessment, public health and medical services, research agendas, and special public health initiatives for the Agency. The OPH-SOPER serves as the principal advisory office on issues of national health policy and coordinates the following four evaluation functions.

• Health Program Evaluations-- Collect and analyze information useful for assisting IHS officials in determining the need for improving existing programs or creating new programs to address health needs.
• Policy Analysis-- Conduct analyses when a change in the IHS health service delivery system must be considered, when issues emerge in an area where no policy currently exists, or when current policies are perceived as inappropriate or ineffective.
• Health Services Research--Undertake analyses of the organization, financing, administration, effects, and other aspects of the IHS delivery of health services.
• Special Studies and Initiatives-- Conduct studies and prepare special reports required by Congress in response to pending legislation or policies, often using a roundtable, whenever an issue or a health problem requires immediate action and it is unclear what type of action should be taken.

The evaluation needs of the IHS service components are coordinated by OPH-SOPER using two major types of short-term studies: policy assessments and program evaluation studies. The IHS policy assessments contribute to decisionmaking about budget, legislation, and program modifications and include background information to support the Agency's initiatives. Evaluation studies are focused at the program level, or Area Offices, and focus on specific program needs.

Each year OPH-SOPER selects high-priority health care and management studies for funding through the submission of proposals to headquarters and Area Offices. These proposals are reviewed and rated by a panel of subject-matter experts, evaluation experts, and IHS staff for concurrence with IHS strategic goals, objectives, and priority areas. The proposals are then prioritized and forwarded to the IHS Director, who reviews the projects that are recommended for funding and determines the respective funding levels.

Summary of Fiscal Year 1997 Evaluations

During FY 1997, IHS completed two major evaluation projects.

The Evaluation of the Impact of the Alaska Tribal Health Compact on Programs and Services in the Anchorage Service Unit, Alaska Area (6407) project is the first research to systematically evaluate the perceptions of a wide range of beneficiaries, officials, and employees of the IHS and the tribal system regarding the Alaska Tribal Health Compact, Title III Self-Governance Demonstration Project. There are almost 200 federally recognized tribes in a single Areawide Compact, organized via 13 separate consigners, each of which has its own Annual Funding Agreement. Seven of these consigners are in the Anchorage Service Unit, which is one of nine Service Units in the Alaska Area. The evaluation concludes that more village representatives have seen either no change or have seen a negative change in in-village services than those who have seen a positive change. The perceptions of the officials and employees of the Alaska Native Medical Center and the Alaska Area Native Health Service have revealed problems in staff morale and shifts in workload resulting from Compact consigners transferring patients to the Alaska Native Medical Center who were previously treated outside the Anchorage Service Unit. These changes are considered as having a negative impact on the Anchorage Service Unit. The "microcosmic" view from the Ahtna Region indicates that there is a distinct difference in the perception of those who are participating in the Compact and those who are not. The analysis of the village respondent data, comparing the information that came from the Title III villages with the information from the Title I villages, further solidified the conclusion that individuals in villages that are not served by a Compact consigner hold a consistently more negative view than those who live in villages that are served by a Compact consigner.

The Prior Trauma Care of Intoxicated Patients as a Predictor of Subsequently Fatal Injury (6451) study was undertaken to identify intervention opportunities that are the result of nonfatal, alcohol-related injury contacts in emergency departments and clinics that could, over time, decrease alcohol-related injury deaths in the IHS area in Billings, Montana. The concept was to determine the frequency of injury treatment recidivism, both alcohol- and non-alcohol related, and further to determine if patterns can be established to suggest that injury death is predictable based on those previous nonfatal contacts. The findings indicate a high degree of recidivism and a strong correlation between nonfatal incidents and eventual injury death. This study further reveals that there is a high prevalence of alcohol in nonfatal injury-related incidents among this cohort. There appears to be a strong relationship between repeated episodes of nonfatal injury with alcohol as a contributing factor and subsequently fatal events among this cohort. These nonfatal episodes should serve as warning signals and opportunities for alcohol intervention strategies to take place.

Evaluations in Progress

The IHS currently has several major evaluations under way in the following general areas: the IHS Resource Requirement Methodology (RRM), aftercare services provided by IHS regional treatment centers, and effectiveness of alcohol and substance abuse services for American Indian and Alaska Native women. The IHS also has a number of smaller evaluations in progress at several area offices.

The Elderly Wellness (6452) study focuses on collecting data to determine if providing health care and health promotion services specially designed for the elderly will increase their use of these services and ultimately improve their health status. Comprehensive care remains an unmet need for the elderly. Recent reviews show that less that 50 percent of the elderly population residing in the Fort Peck Service Unit (Billings, Montana, Area Office) has had a health care visit of any kind in the last two fiscal years.

The Cross Sectional Evaluation of the Prevalence of Childhood Obesity Among American Indians in the Aberdeen Area (6859) studies the trend in the prevalence of diabetes among American Indians in the Aberdeen, South Dakota area over a 10-year period. Findings suggest that not only are diabetes prevalence rates increasing, but the gap is widening between Aberdeen, South Dakota area rates and national IHS rates. This study will establish baseline rates of childhood obesity and will facilitate early intervention efforts among 100 schools attended by Indian children. Obesity is the most important risk factor for diabetes in American Indian communities. The study results will be used by service unit directors and their staff (diabetes coordinators, public health nurses, and nutritionists) to assess their progress toward the Healthy People 2000 objectives and the Aberdeen Area Clinical Objectives to determine childhood obesity prevalence and plan primary prevention programs.

The Evaluating the Effects of Medical Nutrition Therapy on Patient Outcomes Among Native Americans with Newly Diagnosed Type II Diabetes Mellitus (6746) study will evaluate the effectiveness of medical nutrition therapy (MNT) on clinical/behavioral outcomes of American Indians/Alaska Natives with newly diagnosed Type II diabetes. The evaluation will include the creation of a uniform methodology for collecting nutrition-oriented outcome data that is consistent among participating tribes, IHS service units, and urban programs. The outcome data will be analyzed to determine differences in the type and amount of MNT provided with regard to patient outcomes. They will also be linked with current clinical data from the IHS Diabetes Program that identifies continuous quality and performance improvement and measures. The analysis will also describe individual cases in which MNT has contributed to improved patients perceived quality of life.

The purpose of the Evaluation of IHS Capacity for Epidemiologic Surveillance (6745) is to determine the epidemiologic surveillance capacity of the Indian/Tribal/Urban (I/T/U) health system and to make recommendations for improving the capacity to meet the needs of the IHS and its stakeholders. The IHS health care environment is undergoing unprecedented change; reorganizing, consolidating, and downsizing are part of the overall restructuring of services to its customers. Limited resources will be available to conduct epidemiologic surveillance, and the I/T/U health system will become more reliant on local capacity. The evaluation will be designed to collect data from a representative sample of IHS direct service providers, P.L. 93-638 contracts, self-governance compacts, and urban programs to determine the availability of epidemiological useful data and the capacity to perform disease surveillance and epidemiologic analyses of data at the local level.

The objective of the Evaluation of the Effectiveness of the Indian Health Service Alcohol and Substance Abuse Treatment Programs, Phase I (6744) is to evaluate the effectiveness of aftercare and continuing care services provided to youth and adult patients who have received treatment for alcohol and substance abuse at a primary treatment facility, a regional treatment center, or other IHS-funded program. The study will examine the availability, accessibility, acceptability, and cost of community-based followup services, as well as the potential for coordinating and integrating these services with the other aspects of alcohol and substance abuse treatment, such as detoxification, residential treatment, and halfway houses. The purpose of Phase I is to develop a methodology to assess the effectiveness of the IHS-sponsored aftercare/continuing care program to meet the needs of tribal and urban programs. Phase II will include a national assessment to measure the outcomes of aftercare and continuing care services attained by patients/clients of alcohol/substance abuse treatment programs in a representative sample of Indian communities that have IHS-sponsored alcohol and substance abuse programs. The study will include recommendations and an implementation plan to improve aftercare/continuing care services in Indian communities.

In the early 1970s, the IHS formulated the Resource Requirements Methodology (RRM) as a management tool to provide a comprehensive, systematic, and uniform process for estimating the level of resource requirements necessary to provide adequate health care to IHS customers and to assist in the allocation of unearmarked resources. To reaffirm the purpose of the RRM, a 1995 study determined the validity and accuracy of the present methodology for use in today's health care environment. The study findings support the need to update the current methodology to meet the future program demands of the IHS. The Resource Requirement Methodology Update (6450) will include a complete revision and documentation of the RRM and will consist of the following phases: (1) Update Staffing Criteria and Modules, (2) Formulate Needs Assessment Cost Model, and (3) Needs Assessment Model Training.

The Evaluation of Indian Health Service-Supported Alcohol and Substance Abuse Treatment Program for American Indian/Alaska Native Women (6201) is part of a four-phase evaluation of alcohol and substance abuse services for adult American Indian and Alaska Native women in alcohol and other drug treatment centers that are funded by the IHS. Phases I and II of this study were completed; a report was prepared in December 1995. The primary objectives of this project are to assess and measure the treatment services provided; to describe the organization and provision of substance abuse treatment and aftercare services available for adult American Indian and Alaska Native women; and to identify common strengths, problems, and recommendations for improvement. Phase I of this effort focused on designing the study, and Phase II collected descriptive data on the women who use IHS-sponsored substance abuse treatment centers. Phase III will be a prospective client outcomes study, and Phase IV will consist of an evaluation of the efficacy of IHS-funded treatment centers.

Development of a Health Services Research Agenda for American Indian and Alaska Native Populations (6153). The IHS and the Agency for Health Care Policy and Research co-sponsored a health services research conference as a first step in a long-term agenda-setting process to identify the most important health services research issues facing American Indian and Alaska Native communities and their health care systems over the next 5 to 10 years. The health services research agenda is intended to promote collaboration between American Indians and Alaska Native organizations, tribal and urban health systems, medical communities, foundations, and government agencies to increase communications and produce research information on health program services for American Indian and Alaska Native patients. The health services research agenda is also intended to provide a forum for discussing health care reform changes that are creating new directions in the Indian health care system.

Malignant neoplasm survival rates for well-defined American Indian populations are scarce. Although a national cancer surveillance system is in place, information relevant to Native Americans in the Northern Plains is not available. The Evaluation of Available Cancer Surveillance Systems for Montana Native Americans and Correlates of Decreased Cancer Survival Among Native Americans in Montana (5944) will examine the utility of the IHS Resource and Patient Management System and the Montana Central Tumor Registry as a cancer surveillance system for the Native Americans in Montana. The results of this evaluation may determine the current level of cancer detection and treatment for Montana Native Americans compared to non-Native Americans in the same region. The development of health promotion and disease prevention activities requires an efficient surveillance system and knowledge of the current deficiencies and strengths of the current system. A better understanding of the sites of late stage cancer at diagnosis will help providers direct education and early detection efforts to Montana Native Americans.

The Study of the Interaction of Persons Treated for Alcoholism with the Health Care Delivery System (6742) will evaluate the results of alcohol treatment for American Indians who abuse alcohol, focusing on the overall health care delivery system. The evaluation design and methodology will be developed and applied to the population served by the Crow Agency Service Unit. Data retrieval, audit instruments, and analytical strategies will be piloted for two years.

National Institutes of Health

MISSION: To sponsor and conduct medical research that leads to better health for all Americans.

Evaluation Program

The National Institutes of Health (NIH) generates scientific knowledge that leads to improved health. This is done by conducting medical research in its intramural laboratories and by supporting research in universities, medical and health professional schools, and other health research organizations. NIH fosters the widespread dissemination of the results of medical research, facilitates the training of research investigators, and ensures the viability of the research infrastructure. The NIH Evaluation Program is an integral part of how NIH sponsors and conducts medical research.

The complexities and challenges of evaluating and assessing fundamental science have become more widely recognized as science agencies have begun implementation of the Government Performance and Results Act (GPRA). The GPRA stresses the use of evaluation to develop measures for reporting on program results. However, measures for evaluating the results of fundamental science have limitations. These are discussed in the March 1997 General Accounting Office report, Measuring Performance: Strengths and Limitations of Research Indicators. The report compares private versus public research and discusses the strengths and limitations of indicators used to measure research. The NIH evaluation program is addressing these challenges as part of its strategy for implementing the GPRA.

Philosophy and Priorities

The NIH Evaluation Program provides information to assist the NIH Director and Institute, Center, and Division (ICD) Directors in determining whether NIH goals and objectives are being achieved and to help guide policy development and program direction. Evaluations are planned and conducted from two sources of funds: 1-percent set-aside evaluation funds used to fund trans-NIH projects, and ICD program funds used for program evaluations for use by various committees, working groups, task forces, workshops, conferences, and symposia to assist the ICD in program management and development. This approach ensures that planning and priority setting specific to the mission of each ICD are fully developed and implemented and that there is central leadership for developing crosscutting initiatives and promoting collaboration among the ICDs.

NIH's major evaluation priority areas fall within three broad program areas: basic research, research training and career development, and facilities maintenance. These broad program areas form the basis for the strategies and activities that NIH undertakes to achieve the goals identified in the GPRA planning process.

Policies and Operations

A distinguishing feature of the NIH Evaluation Program is the utilization of a variety of evaluation strategies that include the use of national advisory councils, boards of scientific counselors, consensus development conferences, and ad hoc committees that help to chart scientific directions and select the most promising research to support.

A two-tier system is used to review project requests that will use 1-percent evaluation set-aside funding. The first tier involves a review and recommendations by the NIH Technical Merit Review Committee (TMRC) on the technical aspects of project proposals and whether a project fits within HHS guidelines for use of the set-aside fund. The second tier involves the NIH Evaluation Policy Oversight Committee (EPOC), which considers TMRC recommendations, conducts policy level reviews, and makes final funding recommendations to the NIH Director or his designee.

Summary of Fiscal Year 1997 Evaluations

Completed evaluations addressed a cross-section of the NIH research program, training, and career development. Three of the NIH evaluations completed are highlighted below.

A blue-ribbon panel of experts, the National Institute of Mental Health (NIMH) Intramural Research Program Planning Committee (IRPPC), was charged with evaluating and, as necessary, redefining the role of the Intramural Research Program (IRP) in fulfilling the mission of the NIMH. The IRPPC report, Finding the Balance: Report of the National Institute of Mental Health Intramural Research Program (IRP) Planning Committee (6697), provided the NIMH Director and Scientific Director with tools for fundamental renewal of the IRP. For example, the IRPPC recommended the consolidation of the Neuroscience Center of St. Elizabeth's Hospital with the NIMH IRP program at the Bethesda campus. Such a consolidation will provide better access to the special facilities the Bethesda campus offers, more opportunities for collaboration, and better access to patients.

Insuring a strong scientific infrastructure--

with a suitable talent base of highly qualified, highly trained, and diverse investigators capable of yielding the scientific discoveries of the future--and excellent research facilities is essential for medical progress. To increase the number of underrepresented minorities entering careers in biomedical research, the National Institute of General Medical Sciences (NIGMS) established two "bridges" programs to encourage minority students to go beyond the associate degree to obtain the bachelor's degree, or beyond the masters degree to obtain a doctorate. NIGMS established an Electronic Student Tracking and Reporting system to capture the information needed to assess the educational outcomes of students participating in the Bridges Program. The results from this evaluation, Evaluation of the Bridges to the Future Program (6093), indicated relatively high rates of transfer from 2-year to 4-year institutions, or from M.S. to Ph.D. programs, when compared to national norms. As a result, NIGMS is continuing support of this program.

To examine how research results translate into practice, A Study to Determine the Changes of Physician Prescribing of Antimicrobial Agents, Omeprazole, and H₂-Receptor Antagonists for Peptic Ulcer Disease Two Years Following the NIH Consensus Conference on Helicobacter Pylori (6284) was designed to evaluate the effect of the consensus conference in prescribing patterns of physicians for the treatment of peptic ulcer disease (PUD), focusing on prescriptions for antimicrobial and antisecretory agents and H₂-receptor antagonists. In particular, the study focused on whether the recommendations of a February 1994 NIH Consensus Development Conference advocating the use of antimicrobial agents for PUD patients with H. pylori infection were coincident with a change in physician prescribing patterns. The study revealed that two years after the highly publicized conference on the eradication of H. pylori, antimicrobial agents were not widely prescribed for the treatment of PUD within the Pennsylvania Medicaid population. The need to eradicate H. pylori to reduce and, ultimately, eliminate the recurrence of PUD in association with the findings of this research suggest that physicians in this specific population do not appear to be using national recommendations based on state-of-the-art information and clinical studies pertaining to PUD, as presented by the NIH. Restricting the study to Medicare patients limits its generalizability and could be related to issues of equity and quality of care for a disadvantaged population. If findings indicate significant cost savings associated with the use of the NIH recommended drug regimens in the treatment of PUD, this approach could be applicable to other populations.

Other NIH evaluations were completed on the following topics: the status of biomedical research facilities (3562.2); National Research Service Award Research Training Programs (5571); the National Heart, Lung and Blood Institute (NHLBI) article citation and patent references study (5577); the Rare Disease Clinical Research Database (6043); laboratory animal use, facilities, and resources (6048); promoting safe and effective genetic testing (6090); user satisfaction of the Physician Data Query data base (6092); research instrumentation needs (6121); toxicology and environmental health information resources (6534); evaluation of the international cooperative bio-diversity groups (6641); and the Federal Interagency Forum on Aging-Related Statistics (6652).

Evaluations in Progress

NIH has over two dozen evaluation projects in progress, as listed in Appendix B. Several of the major evaluations, soon to be completed, are summarized below.

An ongoing study that will provide information in the research program area is the Surgeon General's Report on Oral Health (6861). This study will provide information on how the public health elements of education, daily preventive self-care, and collaboration between individuals and their care providers can help produce a high level of health in a particular area. In spite of tremendous oral health progress throughout the last five decades, data demonstrate that results are distributed unevenly across populations. There still remain populations and communities with devastating levels of oral, dental, and craniofacial diseases. This report will identify specific steps that individuals and communities can take to improve their oral health status.

Another study under way in the research program area is a project titled, Evaluation of Simplification of the Human Resources Management System at the National Institutes of Health (6862). The Secretary has designated NIH as a pilot project for a period of 5 years to simplify personnel management and personnel administration policies and procedures. The main platform for the evaluation is the delegation of human resources management requirements and responsibilities; the long-range thrust is to strengthen NIH management and integrate human resources management into strategic planning. The evaluation seeks to determine the linkage between the impact of the human resources management delegations and the achievement of major NIH goals, such as advancing research to improve health and prevent disease.

An ongoing study that will provide information in the research training and career development area is, Study of National Needs for Biomedical and Behavioral Research Personnel (6860). This congressionally-mandated study determines the overall national need for research personnel in the biomedical, behavioral, and clinical sciences; and assesses the efforts of the National Research Service Awards (NRSA) programs in ensuring an adequate supply of qualified investigators. Earlier studies have included recommendations concerning both the number of NRSA research training positions needed in the future and the importance of systematic program monitoring and assessment. These recommendations have been used by NIH in formulating program priorities and budget requests, and by the Congress during the appropriations process. For example, it is not uncommon for congressional appropriations reports to cite the number of positions recommended by this study and to appropriate funds accordingly.

In the facilities area, NIH collaborates with the National Science Foundation (NSF) to assess the condition of existing facilities and identify needs for new and refurbished research facilities nationwide. Since its inception in 1988, the biennial Survey of Scientific and Engineering Research Facilities at Colleges and Universities (6863), prepared jointly by NIH and NSF, has provided considerable data on the amount, condition, and capital project activity by the Nation's research-performing institutions. For example, the 1996 Status of Biomedical Facilities reports that 47 percent of all biomedical research-performing institutions classified the amount of biological science research space as inadequate, and 51 percent indicated that they had an inadequate amount of medical science research space.

Office of the Assistant Secretary for Planning and Evaluation

OFFICE OF THE ASSISTANT SECRETARY FOR PLANNING AND EVALUATION

MISSION: To provide analytical support and advice to the Secretary on policy development and assist the Secretary with the development and coordination of departmentwide program planning and evaluation activities.

Evaluation Program

• The Assistant Secretary for Planning and Evaluation (ASPE) functions as a principal advisor to the Secretary on policy development and, in this capacity, conducts a variety of evaluation and policy research studies on issues of national importance. ASPE also is responsible for departmentwide coordination of legislative, planning, and evaluation activities. In its evaluation coordination role, ASPE has the following tasks:
• Provide annual guidance to all HHS agencies and staff offices regarding evaluation priorities, procedures, and review requirements.
• Review evaluation priorities proposed by HHS agencies, providing advice about the focus or method of proposed projects and identifying opportunities for collaboration, and effective use of resources.
• Prepare planning and summary reports on evaluation activities as required by Congress.

Through the departmental evaluation planning process, ASPE has the capacity to identify crosscutting program or policy issues of particular concern to the Secretary and specific program and policy areas not covered by the HHS Agency evaluation plans. In these instances, ASPE initiates evaluations or collaborates with the agencies to conduct evaluations or policy assessments. For example, in recent years, ASPE has initiated projects to develop cost estimates for health financing issues in general, and specifically for the Medicare and Medicaid programs; the effects of managed care expansion on public health infrastructure; welfare-to-work approaches; long-term care alternatives; and studies evaluating alternative services for children at risk of harm from drug abuse, crime, child abuse, and other pathologies.

Another continuing evaluation objective of ASPE is to support and promote the development and improvement of data bases that HHS agencies and ASPE use to evaluate health care programs and health trends. For example, ASPE has been the major initiator in collaboration with the National Center for Health Statistics at the Centers for Disease Control and Prevention of the first comprehensive survey of people with disabilities in the United States. The first component of these new data was completed in FY 1996, and national prevalence data on disability are now available. The ASPE co-chairs and provides support to the HHS Data Council, which is charged with integrating key national surveys, such as linking health status indicators with indicators of well-being. The Department needs more comprehensive data sources to assess anticipated transformations in health and human services.

Finally, ASPE uses evaluation funds to promote effective use of evaluation-generated information in program management and policymaking. The latter is accomplished through the dissemination of evaluation findings and other activities, such as providing technical assistance to agencies in the development of performance measures.

Summary of Fiscal Year 1997 Evaluations

During FY 1997, ASPE completed a wide range of studies and reports on many issues, providing information useful to the Secretary and the HHS divisions for program planning and budget and legislative development.

Health Policy

Market Barriers to the Development of Pharmacotherapies for the Treatment of Cocaine Abuse and Addiction Final Report (6694) presents an analysis of the market barriers to development of pharmacotherapies for substance abuse and addiction, particularly for abuse of and addiction to cocaine. The study examined the development of pharmacotherapies for cocaine abusers in light of the size of the potential market, market penetration estimates, and the basic relationships between price, market size, and revenues. The report outlines several critical market barriers that must be taken into account. The substance abuse treatment market relies heavily on State and Federal reimbursement. Most substance abuse treatment services are subsumed under the mental health benefits of entitlement programs, and drug companies are reluctant to rely upon this kind of reimbursement in an age of shrinking budgets for mental health services.

Public Health Laboratories and Health Systems Change (6458) explored how the relationships among public health laboratories, managed care organizations, and other stakeholders in the laboratory services market have recently changed and the implications of these changes for Federal and State governments. The dynamic health care environment is posing many new challenges for public health laboratories. Changes in both the public and private sectors have made it increasingly difficult for public health laboratories to fulfill their mission. In the private sector, managed care and independent laboratories are expanding; hospital-based laboratories are consolidating; and private sector laboratories are experiencing rapid technological change, both in a clinical sense and in information processing. In the public sector, public health laboratories are experiencing a redefinition of the public health safety net, with greater reliance on managed care to address public health needs and shrinking budgets as State coffers have decreased.

Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health (6177). ASPE formed the Panel on Performance Measures and Data for Public Health Performance Partnership Grants (PPGs) to examine the state-of-the-art in performance measurement for public health and to recommend measures that could be used to monitor the PPG agreements to be negotiated between each State and the Federal Government. Performance measures were identified in ten public health areas: chronic diseases, sexually transmitted diseases (STDs), human immunodeficiency virus (HIV) infection, tuberculosis (TB), mental health, immunization, substance abuse, sexual assault, disabilities, and emergency medical services. More than 3,200 measures were proposed to the panel through various outreach efforts. The panel used the following four guidelines to assess the measures: (1) the measure should be specific and result oriented; (2) the measure should be meaningful and understandable; (3) data should be adequate to support the measure; and (4) the measure should be valid, reliable, and responsive. The panel found that many Federal efforts to collect health-related data provide national rates, but many do not collect data that provide State-level rates, and that much State-level data may not be comparable.

Information Needs Associated with the Changing Organization and Delivery of Health Care: Summary of Perceptions, Activities, Key Gaps, and Priorities (6388) reports on unmet needs for supply-side information (information provided by health care suppliers and insurers) on the health system, focusing on helping HHS identify how information needs associated with a changing health care system are perceived by a wide variety of user groups and constituencies. Based on information gathered from over 50 individuals and from a review of relevant literature and Web sites, the report examines perceived gaps, their causes, and the activities generated in response to those perceptions. The report identifies 23 ongoing efforts within the private sector, States, and foundations to address these gaps, and studies 11 of these more thoroughly. The study concludes that private sector stakeholders and States cannot alone sufficiently address supply-side information needs, giving importance to Federal leadership and convening functions.

State Regulatory Experience with Provider-Sponsored Organizations: Final Report (6628) describes the experience of selected State governments in regulating provider-owned health care delivery systems that accept insurance risk for the provision or arrangement of health care services (provider-sponsored organizations, or PSOs). The States included in the study are California, Colorado, Illinois, Iowa, Minnesota, Ohio, Pennsylvania, Texas, and Washington. The report finds that the two models that States use to regulate the activities of PSOs contracting directly with purchasers of health care (e.g., employers) are (1) licensing the organizations as they would a nonprovider-owned entity, and (2) creating a separate licensing category for risk-assuming PSOs. The study found that the States vary considerably in their regulation of PSOs that assume risk from self-funded, ERISA-exempt employer plans, and concludes that since State policy toward PSOs is still in the early stages of development, the lessons to be learned by Federal policymakers are limited.

Establishing an Analytical Framework for Measuring the Role of Reinsurance in the Health Insurance Market (6573). Reinsurance is widely used by underwriters of health benefits to limit their risk exposure. Reinsurance contracts can be structured in many ways, and insurers can use reinsurance to limit their risk for an individual contract or exposure, for a group of contracts or exposures, or for a whole portion of the insurer's business. This report presents a limited analytical study of the level of risk borne by reinsurers in three markets: the indemnity insurance market, the managed care market, and the self-insured employer benefit plan market. The purpose of the study is to determine to what extent these data can be used to measure the role of reinsurance in the various markets and to suggest ways whereby better assessments of reinsurance might be made. The report was generally unable to identify either data sources or existing research allowing quantification of the risk being ceded to reinsurers in the three markets. While a substantial amount of information is available about reinsurance in indemnity (non-HMO) health insurers, the data are aggregated in ways that make it difficult to specifically identify transactions related to primary health insurance offered to groups and individuals.

A Framework for Assessing Insurer Responses to Health Care Market Changes (6572). Where once a few large commercial carriers dominated the market with fee-for-service plans, there are now dozens of plans in most markets. In highly competitive markets, plans and carriers develop more sophisticated strategies and arrangements to identify and retain target markets. To better

integrate government programs and develop public policy, the variations and rationales for insurers' business strategies need to be better understood. This project developed an analytical framework for analyzing insurer behavior, responses to insurance market reforms, and the likely impact of insurer behavior on the availability and cost of insurance.

Deriving State-Level Estimates From Three National Surveys: A Statistical Assessment and State Tabulations (6380). HHS programs are rapidly changing their focus from the national to the State level. Federal roles and responsibilities traditionally played by HHS will be dramatically altered. Recent events bear this out. Landmark legislation eliminating entitlement to Aid to Families with Dependent Children (AFDC) benefits and block granting it to the States was just passed. States have vast new flexibility to administer Medicaid as the result of the 1115 waivers. The basic data needed to address departmental policy and research issues have typically come from major, national, Federally-sponsored population-based surveys, which are designed to provide national (not State) estimates. This project focused on three major Federal surveys--the Survey of Income and Program Participation (SIPP), the Current Population Survey (CPS), and the National Health Interview Survey (NHIS)--selected for their widespread applicability, use, and potential for analyzing policy issues. The purpose of this project was to (1) understand the statistical issues involved in addressing policy and research questions at the State level with national survey data; (2) assess the current capabilities of these three major surveys to provide answers to these questions at the State level; and (3) present options for obtaining valid reliable State-level data from relatively minimal enhancements and changes to survey design.

Improving Health in the Community: A Role for Performance Monitoring (6583) draws on lessons from a number of current community health efforts to outline the elements of a community health improvement process, discusses the role that performance monitoring can play in this process, and proposes tools to help communities develop performance indicators. The report finds that a community health improvement process (CHIP) can be an important tool for developing a shared vision and supporting a planned and integrated approach to improving community health. The report suggests that a CHIP should include two principal interacting cycles based on analysis, action, and measurement. The first is a problem identification and prioritization cycle; the second is an analysis and implementation cycle.

Privacy and Health Research: A Report to the U.S. Secretary of Health and Human Services (6605) identifies privacy issues surrounding research on personally identifiable health data, paying special attention to the international aspects; reviews the ethical, legal, and general social context surrounding the privacy and confidentiality of health data; describes relevant privacy-protection practices and problems, and identifies emerging issues; analyzes the implications of the new European Union Data Privacy Directive and related policy and legal changes; and recommends policy approaches and technical processes for ensuring that the privacy of individuals is respected. The report examines four major groups of issues that are growing rapidly in scale and complexity, and that must be urgently attended to: (1) secondary uses of data and data linking, (2) research on private-sector health data, (3) cybersecurity, and (4) genetic privacy. The international flow of data, new health data privacy laws in Europe, the United States' new Health Insurance Portability and Accountability Act, and the need for dialogue between the United States and Europe are also examined in detail.

Disability, Aging and Long-Term Care Policy

State Assisted Living Policy: 1996 (4719.3) describes assisted living facilities as an expanding source of supportive housing and services that represent a new model of long-term care. Consumer choice and control are central to assisted living models, which seek to allow consumers to control key features of their living environments and to direct their own receipt of services. This study reviews policies on assisted living and board and care in each of the 50 States. One of the major difficulties associated with assisted living is the lack of a common or standard definition of this kind of care. The report provides specific information about policies in each State.

The Role of Home and Community-Based Services in Meeting the Health Care Needs of People with AIDS: Literature Review (6418.1) provides an overview of the literature (published since 1991) describing the delivery and financing of home- and community-based services for people with AIDS. The review draws inferences from literature on the provision of these services to these elderly for their potential application to the AIDS population.

Consumer-Directed Personal Assistance Services: Key Operational Issues for State CD-PAS Programs Using Intermediary Service Organizations (6728). This project examined approaches and solutions for implementing consumer-directed service models for the delivery of home and community-based personal assistance services (PAS) for persons with disabilities. It addressed issues in this area arising from regulatory compliance requirements as well as issues concerning potential tort liability faced by payers, consumers, and providers of these services. The project included case studies of up to 20 programs that use a variety of intermediaries and the services they provide; meetings with Federal officials to identify and discuss tax and labor regulations affecting PAS providers and consumers; and development of model contracts of Federal regulations and requirements in clear, simple, and accurate terms.

Human Services Policy

Evaluating Two Welfare-to-Work Program Approaches: Two-Year Findings on the Labor Force Attachment and Human Capital Development Programs in Three Sites (6576.1) examines the relative strengths and limitations of particular versions of the labor force attachment (LFA) strategy and human capital development (HCD) strategies. It includes the findings from one part of a multi-year, 7-site evaluation and draws on the advantages of a unique experimental design implemented in three sites: Atlanta, Georgia; Grand Rapids, Michigan; and Riverside, California. The evaluation had its origins in the Family Support Act of 1988, which marked a major shift in the philosophy of welfare by establishing a system of mutual obligation--between government and recipients--within the AFDC entitlement structure. The report is part of a larger study called the National Evaluation of Welfare-to-Work Strategies (formerly known as the JOBS Evaluation). It contains an analysis of the implementation, participation patterns, and costs of the two types of programs operated in each site and an assessment of the effectiveness of the LFA and HCD program approaches in promoting employment and reducing welfare.

Setting the Baseline: A Report on State Welfare Waivers (6651). The Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA) of 1996 eliminated the entitlement to Aid to Families with Dependent Children (AFDC) and replaced it with Temporary Assistance for Needy Families (TANF), a block grant that States can use to provide cash and services to low-income families with children. This report is one in a series designed to provide baseline information on the AFDC program as it existed prior to enactment of the new law. The report focuses on the waivers of AFDC requirements that were granted to States in the years preceding enactment of PRWORA. It provides an overview of the types of waivers that States requested and received, and provides details on specific State policies.

Approaches to Evaluating Welfare Reform: Lessons from Five State Demonstrations (6050). Prior to the enactment of PRWORA, many States obtained waivers from AFDC and Food Stamp Program rules in order to implement welfare reform. Between 1993 and 1996, the Clinton administration approved waivers for 43 States. This project examined waiver demonstration projects in five States (California, Colorado, Michigan, Minnesota, and Wisconsin). Four of the five States used an experimental evaluation design, while the fifth State used a quasi-experimental evaluation design. The report addresses issues in five areas: (1) the choice between an experimental and quasi-experimental evaluation design, (2) sample design, (3) implementation of experimental evaluations, (4) data collection, and (5) analytical methods.

Implementing Welfare Reform Requirements for Teenage Parents: Lessons from Experience in Four States (6228). Federal welfare policy requires minor custodial parents receiving cash assistance to attend school and to live with their parents or in an adult-supervised setting. These requirements were established as part of the PRWORA, which created the program for Temporary Assistance for Needy Families (TANF) and abolished the Aid to Families with Dependent Children (AFDC) program. This report summarizes lessons learned from the operational experiences in four States (Arizona, California, Massachusetts, and Virginia) that implemented waivers requiring school attendance and certain living arrangements under the AFDC program.

Determinants of AFDC Caseload Growth: Final Report (5953). This project examined the dynamics of Aid to Families with Dependent Children (AFDC) caseload growth and analyzed the effects of economic, demographic, and programmatic changes at the State level on States' AFDC caseload from the early 1980's to the present. The project addressed the interaction of these factors by analyzing the economic, demographic, and programmatic factors simultaneously. Compared to earlier studies of AFDC caseload determinants, many of which are outdated, this study focused much more on State-level indicators, and included a more comprehensive set of economic, demographic, and programmatic factors.

Indicators of Welfare Dependence and Well-Being: Interim Report to Congress (6634). The Welfare Indicators Act of 1994 directed the Secretary of Health and Human Services to conduct a study to determine which statistics would be most useful in tracking and predicting dependence on three means-tested cash and nutritional assistance programs: Aid to Families with Dependent Children (AFDC), Food Stamps, and Supplemental Security Income (SSI). Enactment of the PRWORA of 1996 makes this task even more important. The report recommends two kinds of indicators: indicators of dependence, self-sufficiency, and family conditions; and indicators of child achievement, health, and well-being. In most cases, the recommended indicators should be tabulated separately for children, male non-elderly adults, female non-elderly adults, and other important population subgroups wherever possible. To measure dependence, self-sufficiency, and family conditions, the report recommends that indicators reflect points on a continuum, from total dependence to complete self-sufficiency.

Iowa's Limited Benefit Plan (6390). Iowa's Family Investment Program (FIP) replaced Aid to Families with Dependent Children (AFDC) in 1993. FIP participants who are able-bodied and not caring for young children--yet who do not develop and carry out a Family Investment Agreement (FIA)--are sanctioned with assignment to the Limited Benefit Plan (LBP). Some clients may also choose LBP over FIP. LBP originally provided 3 months of benefits at FIP levels, followed by 3 months of reduced levels and 6 months of no cash benefits; the program was later changed to eliminate the original 3-month period. At the end of the 6-month period, the client can reapply for FIP, but must meet its requirements. This report presents findings from a study of the original LBP, showing how families enter the LBP and how their well-being changes after benefits are cut. It examines records for over 4,200 cases assigned to LBP records during 6 months in 1994 and 1995, a survey of 137 cases whose cash benefits were terminated, and case studies of 12 LBP families. The authors note that there are several features of the LBP program that mitigate against extreme deprivation when cash assistance has been terminated, and that the absence of these safeguards in a national welfare reform program could result in extreme hardship for many families.

Family and Children's Social Services Policy

Informal and Formal Kinship Care (6016). A growing number of States prefer to place children who have been removed from their homes of origin in the home of a relative. This practice is known as kinship foster care and it is a growing component of family foster care. It comprises between one-third and one-half of all foster care days provided in the States with the largest foster care populations. This study uses several data sources to examine the characteristics of informal kinship care arrangements and how they compare with those initiated through or subsidized by the State child welfare system. It considers national patterns, examines living arrangement patterns by State for 1990, compares formal and informal kinship care in four States, and uses data from Illinois to provide a more focused picture of kinship care.

Trends in the Well-Being of America's Children and Youth: 1997 (6170.1). This is the second edition of an annual report on the well-being of children and youth in the United States. The report's purpose is to provide the policy community, the media, and all interested citizens with a statistical overview describing the condition of children. The report focuses on trends in the well-being of children and youth in 5 areas: (1) population, family, and neighborhood; (2) economic security; (3) health conditions and health care; (4) social development, behavioral health, and teen fertility; and (5) education and achievement.

Efforts by Child Welfare Agencies to Address Domestic Violence: The Experiences of Five Communities (6155). Historically, child protective services and domestic violence services have not been closely linked. However, child welfare agencies across the country are beginning to consider how families in their child protection caseload are affected by domestic violence and are re-examining their policies accordingly. This report documents how child welfare agencies in five communities (Massachusetts; Michigan; San Diego County, California; Hilo, Hawaii; and Oregon) are attempting to integrate domestic violence concerns into their services. The communities were chosen for study because they are extending their services well beyond educating staff in domestic violence concerns. The report includes a discussion and evaluation of approaches to addressing domestic violence within child welfare agencies; the role of the larger community in preventing domestic violence; and more complex policy questions, such as whether a child's witnessing of domestic violence is itself abusive.

Evaluability Assessment of Responsible Fatherhood Programs (6159). Responsible fatherhood programs provide fathers, particularly those who are young, unmarried, and unemployed, with the support and skills to become active participants in their children's lives. An increased interest in fatherhood programs and a lack of information on their accessibility and effectiveness led to this evaluability assessment. The study visited five fatherhood program sites to assess the readiness of the programs for outcome evaluation and to determine the evaluation process and mechanisms necessary for a full evaluation.

Chapin Hall Center for Children at the University of Chicago: Administrative Data Guide (6389). This report is a guide to administrative data resources on health, social services, and related programs available through the Chapin Hall Center for Children at the University of Chicago. Each data section in the guide provides an overview of a particular data set. Included is information to assist the researcher in interpreting the data sections. The guide was designed for use by an audience of informed policy analysts in order to consider what data are available to answer particular policy questions. ASPE will use the guide to better design short-turnaround analyses on a variety of topics of current policy interest.

Evaluations in Progress

ASPE's evaluations in progress cover a broad range of program and policy areas. In health policy, ASPE is concentrating its efforts in the four areas described below.

Studying Changes in Health and Human Services Programs. America's health care system is undergoing rapid and dramatic change. States are using a variety of laws (e.g., the Balanced Budget Act of 1997 and PRWORA of 1996), as well as Medicaid and welfare policy waivers to make substantial changes to their welfare and health financing and delivery programs. In this area, ASPE's goal is to put into place mechanisms to assess the impact of such legislative changes, the interactive effects of waivers and changes in the law, and the effects of the rapidly changing private sector environment. ASPE is supplementing ongoing private sector and foundation efforts to create a baseline of health information at the State level with projects that will help define key measurements and themes with which to monitor the effect of both program changes and changes in the environment. For example, ASPE has projects under way to examine child health insurance patterns (6721) and innovative State strategies (6729) to insure children.

Understanding Private Sector Changes. In recent years, rapid changes to health care in the private sector have transformed the landscape of health financing and service delivery systems. ASPE's goal is to understand the private sector environment in which HHS programs operate and to be able to describe and monitor ongoing changes. A better understanding of how the private sector works and the dynamics of these changes are essential to developing Federal policy. For example, ASPE is currently managing projects to evaluate the effects of "direct-to-consumer" advertising on consumer knowledge, attitudes, and behaviors concerning prescription drugs (6343) and innovative coverage of health promotion, clinical preventive services, and other coverage issues in the private sector (6347).

.Improving Data and Analytic Capability. Current projects that focus on describing, evaluating, and analyzing information to develop estimates for the decisionmaking process require resources for data collection and analytic capability. An actuarial services contract, for example, supports a wide range of analyses, including cost estimates related to legislation. In addition, the rapid evolution of the delivery system has created information gaps that need to be addressed in a comprehensive way. ASPE has several projects that are part of the HHS "Data Initiative," an effort to fill gaps in the data needed to analyze and monitor the health and well-being of the population, particularly given recent policy changes in HHS programs. This effort focuses on the need for State-level data, better linkages between health data and human service data, data on the impact of policy changes on the well-being of low-income children, and data on health care providers and the supply of employer-provided fringe benefits. For example, ASPE is exploring the linkage of data files from the National Health Interview Survey and the Survey of Income and Program Participation (6419) to be used in program evaluation. This effort emphasizes making better use of existing data.

Agency Related Priority Studies. ASPE continues to collaborate with various HHS agencies on program evaluation and technical assistance. As resources continue to shrink, the need to develop and share information on program efficiency and effectiveness assumes even greater importance. For example, ASPE is working on a number of projects with the Public Health Service (PHS) agencies regarding the food additive review process (6342), Indian health programs (6345), and the role of enabling services in the provision of health care (6750). These continue a tradition of joint projects with both the PHS agencies, as well as with HCFA on Medicare and Medicaid payment policies such as a study of managed delivery systems for Medicare beneficiaries with disabilities and chronic illnesses (6391).

In the area of disability, aging, and long-term care issues, ASPE is focusing its efforts in several areas.

Disability, Aging, and Long-Term Care Trends. ASPE has current projects to assess the profound demographic changes brought about by the aging population and the implications of these changes for active aging and health, long-term care, and retirement policies for the next century. ASPE is looking at the implications of disability for the Nation's health, long-term care and personal assistance, welfare, income security, housing, transportation, educational, civil rights, and employment policies. For example, one project is examining two large claims data sets to assess the health and long-term care expenditure patterns of children with disabilities (5758). Another project is reviewing information on how health insurance coverage and the particular provisions of health care policies are related to employment by people with disabilities. The review will consist of an examination of the available research literature on the relationship of health care coverage and employment among people with disabilities (6240).

Promoting Active Lifestyles and Maximum Independence. More seniors and individuals with disabilities can and should be able to expect to lead active and productive lifestyles of their own choosing and direction than ever before in history. There is increasing evidence that disability rates among the elderly are falling. New advances in technology, the availability of personal assistance services in the community, and the stimulus of new civil rights are empowering individuals with disabilities to live independent lives. Through policy analysis and research and demonstration functions, ASPE is identifying what is known and what remains to be learned about how to assist children and adults with the full range of disabilities to become contributing members of their families, communities, and the Nation (6719).

Managed Care and People with Disabilities. Managed care can provide unique opportunities, but it also has potential pitfalls for people with disabilities. ASPE has projects in progress to determine the impact of managed care on access, quality, and satisfaction for people of all ages who experience disabilities (6398). Of particular interest are two policy issues: (1) there is a concern about the impact of extending Medicaid managed care to the SSI-eligible disabled population, (2) the impact of managed care on elderly persons with physical and cognitive impairments needs to be examined. Evaluation of a comprehensive program for capitating acute and long-term care services for children with disabilities will be initiated.

Assisted Living. ASPE has commissioned a national study of assisted living, or the residential settings that combine adapted housing, assistive technologies, personal assistance, and other supportive services for persons with disabilities (4719.5). Assisted living is considered an important component of services to the aging American population and an effective response to the rising costs of nursing home care.

The following projects are focused on human services issues.

Moving Welfare Recipients to Work. ASPE continues to work with the Administration for Children and Families (ACF) to conduct a comprehensive, multi-year study of the different strategies designed for moving people off welfare and into employment, looking particularly at the cost-effectiveness of different approaches. The evaluation will address questions about the long-term effects of different welfare reform approaches, including whether any approach significantly improves the economic circumstances of the most disadvantaged recipients (6576.3) and the effects of welfare-to-work programs on the children of welfare recipients (6576.4). ASPE has also funded a project to look at one-stop shopping models for welfare-to-work programs (6891) and a project to examine the impact of local labor markets on welfare-to-work transitions (6892).

Measuring the Outcomes of Welfare Reform. ASPE recently initiated a project to look at the impact of employment on AFDC recipients in Wisconsin (6727), comparing the characteristics and outcomes of two groups of women who have left the AFDC program. In collaboration with ACF, HCFA, the Immigration and Naturalization Service, and the Department of Agriculture, ASPE is now studying the impact of welfare reform on the economic and health status of immigrants, their communities, and the organizations that serve them (6747). ASPE has also funded a study in New Jersey to look at changes in family well-being and family coping strategies following benefit sanctions (6893).

Welfare Reform Implementation Issues. ASPE has several projects examining human service delivery policy questions related to the implementation of welfare reform. As welfare programs require employment or participation in work-related activities, a better understanding of the low wage labor market is critical. ASPE has commissioned a series of papers to summarize and interpret the results of previous research on the low-wage labor market (6717). ASPE will examine the relationship between domestic violence and welfare reform (6724) by looking at implementation of the TANF Family Violence Option, changes in child support enforcement programs, and the use of good cause waivers for battered women. Evidence suggests that some battered women seek employment and child support while others need specialized assistance in addressing the abuse of partners. A third example is ASPE's study of child health and development programs in the context of welfare reform (6754). This project will identify and present profiles of promising Federal, State and/or community-based health and human services programs believed to be enhancing the health and development of children in the context of welfare reform.

Child Support. ASPE will examine the impact of a change in child support review and modification policy under PRWORA (6749). The law prior to the enactment of PRWORA required that all AFDC child support cases be reviewed every 3 years and adjusted if the child support award was inadequate. PRWORA made review and modification in TANF cases optional. A number of States may discontinue review and modification in TANF cases. This project will provide estimates of the financial impacts to the Federal and State governments, or how such a discontinuance would impact on child support offsets to cash assistance payments. In addition, the financial impacts on families are to be estimated, paying particular attention to those who have left the welfare rolls.

Family Preservation Services. ASPE and ACF are now examining the following types of family preservation programs: placement prevention services aimed at preventing children from entering substitute care; broader family preservation services that may be less intensive and of longer duration than placement prevention services; and reunification services to speed the return of children to their homes after entering substitute care (5337.2). Measures of program success will include prevention of placement of children into substitute care (for pre-placement services), successful reunification (for reunification services), improved child behavior, improved family functioning, and reduction of the recurrence of child abuse and neglect.

Teen Pregnancy. ASPE is coordinating several efforts with HHS agencies to develop and evaluate strategies to reduce teen pregnancy. The first project is part of the National Strategy to Prevent Teen Pregnancy, aimed at furthering our knowledge about teen pregnancy prevention programs and the process of building and strengthening partnerships to reduce teen pregnancy and provide a positive future for youth (6725). ASPE will provide information and tools to help communities develop teen pregnancy prevention programs and partnerships. Next, ASPE will conduct a project on understanding and preventing abusive intimate relationships among adolescents (6751). It will examine the issue through a literature review; analysis of newly available data from the National Longitudinal Study of Adolescent Health, the National Survey of Family Growth, and other surveys; identification of data needs; focused discussion groups; a summary of promising approaches to prevention and intervention; and recommendations for further research, policy, and discussion. Finally, ASPE has two teen pregnancy prevention efforts that target young men. First is a project to develop a research and policy information strategy to inform regional, State, and local policy officials and community-based organizations serving children and youth of the program models--primarily directed at boys and young men-- that can be implemented to help teenagers avoid premature sexual activity and unintended pregnancies (6766). Second, there is a project is to identify abstinence-based pregnancy prevention programs that target boys or both boys and girls (6726). It will use a framework to assess and summarize the state of the field and provide the information in a format useful to State, local, and community policymakers. States and communities have shown interest in providing services and programs for boys and young men, but have also expressed frustration that they do not have access to information about what strategies have been tried, if they have been successful, and how they can access funding for the development and operation of these programs.

Office of Public Health and Science

MISSION: To provide advice on public health and science to the Secretary of Health and Human Services, to provide executive direction to program offices within the Office of Public Health and Science (OPHS), and, at the direction of the Secretary, to coordinate crosscutting public health and science initiatives in the Department.

Evaluation Program

The Office of Public Health and Science (OPHS) provides advice, policy and program coordination, and leadership in the implementation, management, and development of activities related to public health and science, as directed by the Secretary. OPHS helps HHS conduct broad-based public health assessments to better define public health problems and solutions. It assists other components of HHS in anticipating future public health issues and problems and helps ensure that HHS designs and implements appropriate approaches, interventions, and evaluations that will maintain, sustain, and improve the health of the Nation. OPHS provides leadership and policy recommendations on population-based public health and science and, at the Secretary's direction, leads or coordinates initiatives that cut across agencies and operating divisions. In addition, OPHS communicates and interacts, on behalf of the Secretary, with national and international professional and constituency organizations on matters of public health and science. Finally, OPHS's unique role allows it to use its resources to link important HHS programs or fill gaps in areas needing better policy formulation and coordination.

In keeping with its role within the Department, OPHS has developed an evaluation plan that avoids duplication of efforts that might more appropriately and effectively be undertaken by operating divisions of HHS or by the Assistant Secretary for Planning and Evaluation (ASPE). Thus, the FY 1998 evaluation strategy for OPHS focuses on public health and science issues that cut across multiple interests of the operating divisions, and requires a coordinated approach to achieve the most effective results. In addition, OPHS will continue its commitment to carry out every project proposed as part of this year's strategy in collaboration with relevant operating divisions.

OPHS conducts evaluations requested of the Secretary by Congress that are most appropriately managed by staff with medical or health science credentials, and that cannot be assigned to an operating division. Further, evaluations will be conducted to support the Surgeon General and the Assistant Secretary for Health in their respective roles as the Nation's top doctor and senior advisor for public health and science to the Secretary. These roles include the responsibility to assist the Secretary in developing a policy agenda for the Department to address major population-based public health, prevention, and science issues and to provide leadership and a focus for coordination of population-based health, clinical preventive services and science initiatives that cut across operating divisions. In addition, OPHS will conduct evaluations specific to the needs of the programs operated from the offices located within OPHS, such as Women's Health, Minority Health, Disease Prevention and Health Promotion, International Health, and Emergency Preparedness. A portion of the evaluation funds will be made available to the ten HHS Regional Health Administrators.

Summary of Fiscal Year 1997 Evaluations

In FY 1997, OPHS completed five evaluations in an effort to track the impact of the transformation in the Nation's health care system on underserved and vulnerable populations or to improve OPHS program performance. Studies completed this year will better inform the public health and science communities concerning the return on their investment of scarce public health resources and will help shape future budget decisions.

In an effort to quantify investments made by the various levels of government in essential public health services--both personal health services and population-based services--OPHS worked with State and local public health, mental health, substance abuse, and environmental agencies in nine States to define, measure, and monitor pubic health expenditures (6194). Using the essential services framework, the study concluded that expenditures could indeed be determined. Some rather surprising conclusions were reached concerning the source of the largest portion of public health expenditures, indicating that more than 2 out of every 3 dollars spent on essential public health services went for personal health services and that only 1 percent of the total health care expenditures went for population-based health services.

OPHS funded development of a study to identify the types of information and data necessary to successfully protect and monitor the human health hazards of environmental incidences (6322). The study concluded that the Federal government could help State and local public health agencies in environmental surveillance by assuring integration of public health, environmental health, and environmental protection agencies' information. However, any information system developed must be evaluated based on its usefulness to State and local agencies that respond to adverse effects to human health arising from exposure to environmental hazards.

In FY 1997, OPHS asked stakeholders in the Healthy People 2000 process to evaluate the framework of national health goals and disease prevention objectives in anticipation of the development of the Healthy People 2010 plan (6491). By reviewing the successes and failures of the Healthy People 2000 benchmarks, OPHS hoped to make this national framework of performance indicators on health status more results-oriented. In general, participants in the various forums liked the overall structure and content of the Healthy People 2000 document, although some recommended reducing the size of the document for Heathy People 2010 and encouraged the use of new communication avenues for this valuable assessment tool, making it available to a wider range of professionals and community leaders. Both health care purchasers and managed care providers agreed that the number of objectives should be reduced to permit adequate focus on a limited number of problem areas.

Using resources made available through the HHS Regional Health Administrators, one study examined the health and well-being of women in the State of Washington (6684). Through a broad coalition of health care providers, researchers, and activists, as well as focus groups of women from diverse geographical areas, economic levels, and social strata, the report concluded that in general, women in Washington State are living longer and enjoy a better standard of living than women elsewhere in the United States. Although access to health care overall was relatively good, the report indicated that elderly women, those with less than a high school education, and poor women were less likely to use mammography and have regular Pap smears. The report will be used to provide recommendations for addressing several problem areas for Washington women's health.

Finally, in a continuing effort to evaluate program effectiveness, the Office of Minority Health concluded an analysis of sponsored program offices at Historically Black Colleges and Universities (HBCUs) (6246). The evaluation determined that grant funds and technical assistance had achieved the program aims begun in 1992 to increase the involvement of HBCUs in health and social service programs funded by the Public Health Service and other Federal agencies. On campuses where sponsored program offices were funded, there was an increase in proposal submissions and awards. The report indicated a continuing need to invest in both programmatic and administrative infrastructure.

Evaluations in Progress

In reviewing evaluation projects for FY 1997, OPHS gave highest priority to efforts aimed at tracking the enormous transformation occurring in the Nation's health care system. The shifting emphasis on managed care presents new dilemmas about how to ensure that, in the quest to control cost, overall quality of care is not unduly compromised. Ensuring a workforce that is properly trained and fully capable of performing essential public health services in this dynamic health care environment is a task that OPHS must, at a minimum, also monitor. Monitoring this entire process presents new challenges to HHS, OPHS, and their partners in the public health community.

The FY 1997 OPHS evaluation plan emphasized the need for a strong foundation for public health in the 21stcentury. Priorities focused attention on (1) the information systems and workforce that comprise the Nation's public health infrastructure and that are necessary to effectively provide the essential services of public health; and (2) the impact of managed care arrangements on the resources available for this infrastructure and on the health of all Americans, especially those most vulnerable. In support of broader HHS efforts to improve departmental programs, OPHS evaluations also reflect the assessment of various programs and activities funded through PHS.

Several studies were continued from previous years. The Commission on Dietary Supplement Labels, appointed by the President, continued its evaluation of factors relevant to Food and Drug Administration regulation and possible legislation of label claims and statements for dietary supplements as requested by Congress (6193). The National Academy of Sciences also continued its study of dietary reference intakes (6323). In the area of program improvement, OPHS continued supporting an evaluation of a new model of coordinating and integrating HIV prevention and primary care services among high-risk populations (6321), an evaluation of the Minority Health Resource Center (6244.1), and the assessment of the efficiency and effectiveness of the Office of Minority Health's Bilingual/Bicultural Service Demonstration Grant Program (6247). In the area of communication, OPHS is designing and evaluating applications of interactive communications for consumer health information and examining the utility and impact of timed dissemination of public health information from PHS agencies (6327). OPHS continued its support of a study to identify evaluations of school health programs and make available to school boards, administrators, health personnel, and health educators an updated compilation of methodologically sound studies of these programs (6198).

In an effort to track the impact of managed care arrangements on the health care system, OPHS collaborated in a broader study of health systems change (being funded by the Robert Wood Johnson Foundation) to track changes in the public health sector in 12 randomly selected communities over a 4 year period (6325). OPHS also developed a research agenda to address gaps in scientific knowledge related to cultural competence in health care and linking measures to outcomes (6675), and assessed managed care organizations serving racially/ethnically diverse communities to determine the extent and nature of linguistically and culturally appropriate services (6674). When completed, the study will analyze and report the range of services being provided, identify best practices and model approaches, and explain the community and organizational factors conducive to providing linguistically and culturally appropriate services.

In FY 1997, OPHS undertook a project in cooperation with the United Kingdom that examines efforts to address the health concerns of racial and ethnic minorities in the two countries and, for six issue areas, conducts comparative analyses of strategies and approaches to identify strengths and limitations, best practices, lessons learned, and future areas for collaboration. For each of the six issues, a paper incorporating these comparative analyses into an integrated piece suitable for publication is forthcoming (6676).

In a continuing effort to improve the usefulness of Healthy People 2000 objectives, OPHS worked with States and localities to determine their ability to assess health trends in their communities (6488). Although Healthy People 2000 has driven the development of some surveillance and data systems at the national level, the usefulness of this information is dependent on State and local ability to measure objectives locally. Information from this study will be used to develop objectives for Healthy People 2010, in an effort to make them more usable for local performance and health assessment. In this area, OPHS is also assessing linguistically and culturally appropriate community health promotion programs in local health departments to gather the baseline data necessary to support the review of Healthy People 2000 objectives (6798).

The balance of the ongoing evaluation projects funded by OPHS were conducted on a regional basis, examining programmatic impact in significant Presidential or departmental priority areas or in vulnerable or special populations. Specifically, four studies examined effectiveness of services in the area of women's health: preventive health screening among older women (6677), breast and cervical cancer screening in Region X (6682), averted pregnancies and associated cost savings in Region X (6683), and a retrospective study of the preventive health practices of former Title X (Family Planning) clients (6685). Other studies are looking at the impact on public health services of an increased migrant population in northwest Arkansas (6687) and evaluating the impact of the loss of Federal funds in the AoA and ACF programs on the Freely Associated States (6688). Studies reviewing activities concerning Presidential or departmental priorities include an evaluation of an immunization education program for child care centers (6680), a study of males who father children born to teenagers (6681), an assessment of Region IX programs to promote positive images in girls ages 9-14 (6689), and an analysis of local capacity to respond to the health and medical consequences of welfare reform (6686). One project is evaluating the community-based public health practice training program (6678).

Substance Abuse and Mental Health Services Administration

MISSION: To improve the quality and availability of prevention, treatment, and rehabilitation services for substance abuse and mental illness.

Evaluation Program

The Substance Abuse and Mental Health Services Administration (SAMHSA) is committed to evaluating its overall programs and individual grant projects to assess the effectiveness of prevention, treatment, and rehabilitation approaches and systems of care; the accountability of Federal funds; and the achievement of SAMHSA's programmatic and policy objectives.

To the greatest extent appropriate and feasible, SAMHSA encourages the use of comparable data elements and instruments across its evaluations in order to work toward a comprehensive evaluation system and to minimize respondent burden.

SAMHSA conducts grant programs under a variety of legislative authorities. These authorities can generally be grouped into two types: (1) services and (2) knowledge development and application (KDA). The evaluation required for a particular grant program is dependent on the type and purpose of the program. SAMHSA evaluates each of its service programs so as to provide information to program managers about the accountability of Federal funds. In addition, the evaluations of KDA programs will generate new knowledge to lead the field in the development of policies that improve services.

The two types of grant programs (service delivery and KDA) represent the two facets of SAMHSA's mission. SAMHSA's leadership in the field depends on the successful interaction of these two facets. Through KDAs, SAMHSA must identify effective approaches to prevention, treatment, and rehabilitation. Through service delivery funds, SAMHSA must provide incentives to the field to implement effective approaches. Major emphases of SAMHSA's mission are to develop, identify, and disseminate effective strategies and systems for prevention, treatment, and rehabilitation.

SAMHSA is implementing an integrated model of evaluation and planning. Strategic planning identifies priorities, such as managed care, that drive the development of grant programs and evaluations. In compliance with the Government Performance and Results Act (GPRA), SAMHSA is improving performance management by identifying annual performance objectives and measures. The formulation of programmatic and evaluation priorities includes consultation with SAMHSA, Center Advisory Councils, and with other experts in the fields of evaluation and service delivery. Early and continuous coordination of program planning and evaluation design will result in the articulation of program objectives that may be evaluated. Evaluations will demonstrate the extent to which the grant programs have achieved their overall objectives, and SAMHSA will translate these results into information that can be used for program and policy development. The strategic planning and policy development processes will then use these results to refine SAMHSA's priorities and performance objectives.

This evaluation policy will help SAMHSA achieve its goal of continually informing policy and program development with knowledge culled from past performance. In this way, SAMHSA can best serve its customers by enhancing the quality of publicly-funded substance abuse and mental health services.

In compliance with the Public Health Service (PHS) guidelines for the technical review of evaluations, SAMHSA has established a standing committee of PHS staff who are evaluation specialists. Representatives of the Office of the Assistant Secretary for Planning and Evaluation serve as ex officio members of the committee. The SAMHSA evaluation officer is the committee chair. The committee does not generally review the evaluation proposals of individual grantees; rather, it reviews proposals for broader, more comprehensive evaluations, such as the cross-project evaluations of grant programs.

Evaluation project proposals are generally prepared by SAMHSA program staff in the various Centers. The standing committee reviews each proposal on the following criteria: clarity of evaluation objectives and research questions, appropriateness and feasibility of the specifications for evaluation design and methods, appropriateness of the plans for dissemination of results, and use of previous relevant evaluations and existing program data systems. Each proposal must clearly state the relationship of the evaluation to SAMHSA's overall policies, priorities, and evaluation program.

Summary of Fiscal Year 1997 Evaluations

During FY 1997, SAMHSA completed six evaluations. A summary of the resulting reports follows.

The NTIES: National Treatment Improvement Evaluation Study: Final Report (5346.1), highlighted in Chapter II, was a comprehensive assessment of the results of 157 three-year demonstration grants and cooperative agreements funded by the Center for Substance Abuse Treatment (CSAT) beginning in 1990-91. These grants were intended to improve alcohol and drug treatment in selected target cities with severe substance abuse problems, among critical populations such as minority groups and women, and among persons in the criminal justice system. Based on a final sample of 6,600 clients who entered treatment, the results showed a substantial reduction in the selling of drugs and the commission of violent crimes, high-risk sexual behaviors related to HIV transmission, medical visits, homelessness, and inpatient mental health visits. Results also showed a substantial increase in employment.

The CSAT Demonstration: Evaluation of Job Corps Drug Treatment Enrichment Program (4523) was a 4-year demonstration project sponsored by CSAT and put in place within the Department of Labor's Job Corps program. The Drug Treatment Enrichment Program (DTEP) was implemented in four experimental Job Corps Centers. These were matched with four control centers that provided the standard Job Corps Alcohol and Other Drugs of Abuse program. The evaluation found that DTEP students reduced marijuana and crack or cocaine use following their Job Corps experience significantly more than did the students in the control groups. DTEP students were less likely to engage in moderate or extensive use of marijuana or to use other drugs, but no differences were found for alcohol abuse. DTEP students were less likely to report selling or helping to sell drugs after Job Corps. No differences between DTEP and controls were found for other criminal activities. DTEP improved students' mental health. DTEP students with mental health problems experienced a significantly higher job placement rate after Job Corps than did control students, although the program had no effect on job placement rates of students without mental health problems. DTEP appeared to have no significant positive effect on duration of stay in Job Corps or on overall educational and vocational gains. The final report makes recommendations regarding the integration of DTEP into Job Corps and other related policy implications.

The Center for Substance Abuse Prevention (CSAP) released the findings of a demonstration evaluation, Evaluation of Model Projects for Pregnant and Postpartum Women and Their Infants (5674). The evaluation assessed the effectiveness of the demonstration program in enhancing services coordination and in increasing the availability and accessibility of substance abuse prevention services delivered to clients. The study also documented the effectiveness of program models in decreasing alcohol and drug use among substance-abusing women, and enhancing the healthy development of their children. Ten projects funded by CSAP in 1992 were selected to participate in the evaluation. Data collected by the grantees were used for cross-site analyses. The major evaluation finding was that from intake to delivery, the treatment group members reported significantly lower use of marijuana, crack, and other illicit substances than comparison group members. These differences, however, were not significant by the 6-month postpartum followup assessment. Overall, treatment and comparison group members gave birth to infants with similar birth-related outcome characteristics. However, when the history of the mother's use of crack was taken into account, treatment babies had significantly higher birth weights than the babies in the comparison group.

CSAP also conducted a Report on the CTS National Evaluation (4514.1). The training system included the development and delivery of more than 30 curricula to more than 10,000 participants. Recipients included community teams, health professionals, State administrators, and volunteers engaged in prevention activities. The report found that the training system was innovative in several important respects. It focused on community coalitions and associations of health professionals that are responsible for planning and delivering prevention services, rather than concentrating on target populations. Curricula were developed quickly in response to emerging issues in the field. The training system had a continuing commitment to quality control and continual curricular improvement.

CSAP has completed Findings from the Evaluation of the Faculty Development Program, Volume I: Final Report (4510). This program offers training to faculty members from schools of medicine, nursing, and social work, and from graduate psychology programs. The purpose of the development program was to enhance research, teaching, and clinical practices on issues of alcohol, tobacco, and other drugs (ATOD). The evaluation found that faculty fellows, after participating in the program, reported (1) spending significantly more time on ATOD-related activities, (2) conducting more ATOD research, (3) incorporating more ATOD information in courses, (4) joining ATOD professional organizations, (5) increasing ATOD teaching or clinical skills, (6) producing more ATOD publications, and (7) making more ATOD presentations at conferences and other events than they had prior to their participation in the program.

The Center for Mental Health Services (CMHS) produced Final Report: Review of the CMHS Mental Health Care Provider Education in HIV/AIDS (6695). The program supports the provision of state-of-the-art training targeted to (1) traditional mental health care providers (psychiatrists, psychologists, nurses, and social workers); (2) other first-line providers of mental health care services (primary care physicians and medical students); and (3) nontraditional providers of such services (clergy or alternative health care workers). The goal of the program was to improve access to appropriate and adequate mental health services for people living with HIV/AIDS and for their families and partners. The evaluation found that training participants increased their knowledge about the mental health aspects of the disease, their willingness to serve HIV-infected individuals, and their confidence that they could do so effectively. The recommendations of the report stressed the need for training in the mental health aspects of HIV/AIDS throughout the country, especially as affected areas and populations change over time.

Evaluations in Progress

SAMHSA currently has major evaluation projects under way in two areas. First, CSAT's National Evaluation Database Service (5994) is providing centralized data management and analysis for the evaluation of several large demonstrations targeted to special populations, including female, adolescent, criminal justice, culturally distinct, and rural populations. The service represents part of an evaluation strategy that builds upon prior findings, and which seeks to identify a set of consistent evaluation questions that apply across similar substance abuse treatment programs targeted to special populations. The service has identified data elements to provide uniform information across sites so that comparisons of effectiveness could be made.

Second, CSAP is conducting a National Evaluation of the Community Partnership Demonstration Project (4997). This project was designed to identify successful prevention and partnership strategies and common inhibitors to forming successful partnerships and prevention programs. This impact evaluation has focused on assessments of short-term outcomes specific to each interview site, long-term outcomes reliant upon uniform surveys, and community-wide indicators of alcohol and drug abuse across several sites. Data have been collected through a questionnaire administered in schools and a telephone survey for adults.

Other evaluations in progress are looking at services integration for homeless persons with chronic mental illness (4980.1), effects of managed care on substance abuse treatment outcomes (6454), evaluation of substance abuse treatment improvement protocols (6737), long-term effects of drug abuse treatment (6738), and an evaluation of opioid treatment program accreditation (6739).

Appendix A. Abstracts of HHS Evaluations Completed in Fiscal Year 1997

The abstracts included in this appendix describe the evaluation reports completed by the U.S. Department of Health and Human Services (HHS) during fiscal year (FY) 1997. They are listed in alphabetical order by agency. The sponsoring agency, report title, abstract, and Federal contact person(s) are listed for each report. For more information on any of the reports, please call the contact person listed at the end of each abstract. Copies of the reports may be obtained by contacting either the Policy Information Center or the National Technical Information Service, addresses for which are given below.

Policy Information Center

Final reports for most of the evaluations listed in this appendix have been submitted to the HHS Policy Information Center (PIC), a centralized source of information on in-process, completed, and ongoing HHS evaluations; short-term evaluative research; and policy-oriented projects. The PIC maintains a resource data base containing information on more than 6,000 completed and ongoing studies sponsored by HHS, other Federal agencies, and private-sector entities. The PIC provides executive summaries (if available) at no cost.

The PIC identification number appears after the Federal contact's phone number. For information about using PIC resources, please contact:

Policy Information Center
Office of the Assistant Secretary for Planning and Evaluation
Department of Health and Human Services
Room 438F, Hubert H. Humphrey Building
200 Independence Avenue, SW
Washington, DC 20201
(202) 690-6445

National Technical Information Service

The National Technical Information Service (NTIS) is an information clearinghouse run by the Department of Commerce. It provides reports in both paper and microform formats for a fee. If a final report has been registered with this service, the NTIS accession number follows the PIC ID entry. For more information about ordering copies of reports from NTIS, please contact:

National Technical Information Service
Department of Commerce
5285 Port Royal Road
Springfield, VA 22161
(703) 487-4650

Administration for Children and Families

Contents

National Study of Protective, Preventive, and Reunification Services Delivered to Children and Their Families

Self-Sufficiency Project Implementation Manual: Lessons Learned from Eight Years of Office of Community Services Demonstration Partnership Programs

Examination of Special Needs Adoption in New York State--Phase III Report: Subsidized Adoptions in New York State, 1989-1993

Child Maltreatment 1995: Reports from the States to the National Child Abuse and Neglect Data System

Descriptive Study of the Head Start Health Component

Evaluation of Nine Comprehensive Community-Based Child Abuse and Neglect Prevention Programs: Cross-Site Evaluation Report

Preventing Child Abuse and Neglect: A Case Study of Community Coalition Acting for Positive Parenting

Preventing Child Abuse and Neglect: A Case Study of Family Care Connection

Preventing Child Abuse and Neglect: A Case Study of Community Coalition Acting for Positive Parenting

Preventing Child Abuse and Neglect: A Case Study of PARE (Physical Abuse and Neglect Reduction Effort)

Preventing Child Abuse and Neglect: A Case Study of North Lawndale Family Support Initiative

Preventing Child Abuse and Neglect: A Case Study of Dorchester CARES

Preventing Child Abuse and Neglect: A Case Study of Maine Families

Preventing Child Abuse and Neglect: A Case Study of I CARE

Responsible Fatherhood: An Overview and Conceptual Framework

Responsible Fatherhood: An Overview and Conceptual Framework

Arkansas Prenatal and Postnatal Paternity Acknowledgment Project

LEAP: Final Report on Ohio's Welfare Initiative to Improve School Attendance Among Teenage Parents- Ohio's Learning, Earning, and Parenting Program

First Progress Report on the Head Start Program Performance Measures

REPORT: National Study of Protective, Preventive, and Reunification Services Delivered to Children and Their Families

ABSTRACT: This study determined the number and percentage of children and families in the child welfare system receiving protective, preventive reunification, out-of-home care, and/or aftercare services. It also obtained national data on the number, types, and dynamics of the services provided. Case record abstracts were completed on a nationally-representative sample of 3,000 children and their families served by public child welfare agencies. A subsample was followed for a 9-month period. These findings were compared with findings from a previous study, the 1977 National Study of Social Services to Children and Their Families. The study found that (1) between 1977 and 1994, there was a dramatic decline in the number of children receiving child welfare services, reflecting the evolution of the child welfare system from a broad based social services system to one primarily serving abused and neglected children and their families; (2) the child welfare system has not evolved into an in-home family-based system from a foster care system, as had been envisioned by the Adoption Assistance and Child Welfare Act of 1980; (3) foster care drift remains a problem, with more than one-third of the children in foster care remaining there for more than 18 months; (4) minority children, particularly African-American children, are more likely to be in foster care placement than to receive in-home services, even when they have the same characteristics and problems as white children; and (5) kinship care does not explain the dramatically longer stays in foster care for African-American and Hispanic children compared to white children. The final report for this project is also available from the National Clearinghouse on Child Abuse and Neglect Information at 1-800-FYI-3366.

AGENCY SPONSOR: Administration on Children, Youth, and Families

FEDERAL CONTACT: Penelope L. Maza

PHONE NUMBER: (202) 205-8172

PIC ID: 3874

PERFORMER ORGANIZATION: Westat, Incorporated, Rockville, MD

REPORT: Self-Sufficiency Project Implementation Manual: Lessons Learned from Eight Years of Office of Community Services Demonstration Partnership Programs

ABSTRACT: The Self-Sufficiency Project Implementation Manual is a synopsis of lessons drawn from an assessment of eight years of Office of Community Services Demonstration Partnership Program (DPP) projects. DPP projects were designed to demonstrate the effectiveness of various innovative services that promote self-sufficiency among low-income individuals and families who rely on or are at risk of relying on public assistance. DPP has concentrated on five issues in their projects: (1) case management, (2) micro-enterprise development, (3) minority male employment, (4) homelessness, and (5) youth at risk. The manual presents generic models for establishing effective community-based programs in these areas. All models are presented in the form of logic models in order to provide a consistent framework for understanding. The manual also presents material on evaluating such programs. Each section of the manual is organized into general lessons learned, followed by specific lessons learned regarding each of the five project types. The manual is designed for use by community action agencies, community-based organizations, and local community program planners who are interested in developing a self-sufficiency project. (Final report: 107 pages.)

AGENCY SPONSOR: Office of Community Services

FEDERAL CONTACT: Richard Saul

PHONE NUMBER: (202) 401-9341

PIC ID: 4336.4

PERFORMER ORGANIZATION: BHM International, Incorporated, Silver Spring, MD

REPORT: Examination of Special Needs Adoption in New York State--Phase III Report: Subsidized Adoptions in New York State, 1989-1993

ABSTRACT: Large urban states like New York accounted for two-thirds of the precipitous national growth in the number of children in foster care between 1986 and 1991. The growth of caseloads and the need for more concentrated permanency planning necessitates new and innovative data collection efforts related to special needs adoption. This project was initiated in order to pilot a statewide data collection system for special needs adoptions. Information was collected on characteristics of the adopted child and the adoptive family, procedural issues, and the exact nature and amount of subsidy payments provided to the adopted child. This information was collected for the entire population of children being placed in New York via both public and private agencies. The project resulted in five phase reports. This (Phase III) report contains a descriptive analysis of statistics on the number of children placed between 1989 and 1993 in New York State, the level of this support, and the nature of the children's special needs. The report finds that (1) 12,858 new adoption subsidies were approved in New York State during the study period; (2) 62 percent of children receiving an adoption subsidy in the State qualified because they were hard to place; (3) personality or behavioral problems were the most prevalent reason for a child's classification as handicapped; (4) about one-fourth of children qualified for a subsidy had physical handicaps or severe medical conditions; (5) more male than female children were adopted with a subsidy in New York State; and (6) about half of all special needs children are placed in adoptive homes headed by a single mother. See also PIC ID No. 4380. (Final report: 81 pages, plus appendices.)

AGENCY SPONSOR: Administration on Children, Youth, and Families

FEDERAL CONTACT: Cecelia Sudia

PHONE NUMBER: (202) 205-8764

PIC ID: 4380.1

PERFORMER ORGANIZATION: Cornell University, Ithaca, NY

REPORT: Child Maltreatment 1995: Reports from the States to the National Child Abuse and Neglect Data System

ABSTRACT: The Child Abuse and Neglect Data System collects data on child maltreatment from States, territories, and other reporting jurisdictions. This report, which presents data collected from reports of child maltreatment investigated by States in 1995, finds that (1) in 1995, more than 1 million children were identified as victims of abuse or neglect (a rate of 15 per 1,000 children); (2) about 80 percent of the perpetrators of child maltreatment were the parents of the victims, while another 10 percent were other relatives and about 2 percent were people in other caretaking roles; (3) 52 percent of victims suffered from neglect, about twice as many as were subjected to the next most frequent kind of maltreatment--physical abuse (25 percent); (3) about 13 percent of victims were sexually abused; (4) more than half of all victims were 7 years of age or younger, about 26 percent younger than 4 years old, and about 21 percent were teenagers; (5) the majority of victims of neglect and medical neglect were younger than 8 years old, while most victims of other forms of maltreatment were older than 8 years old; and (6) 45 States reported that 996 children were known by the child protection services (CPS) agency to have died as a result of abuse or neglect--most of these deaths were children 3 years of age or younger. The report also finds that (1) CPS agencies investigated nearly 2 million reports alleging maltreatment of about 3 million children (a rate of 43 per 1,000 children); (2) reports were received from professionals (53 percent), family members (19 percent), friends and neighbors (9 percent), and anonymous sources (19 percent); and (3) nationally, about 36 percent of investigations resulted in a disposition of either substantiated or indicated maltreatment, while 58 percent of allegations were not substantiated. The report is available on the Internet at http://www.acf.dhhs.gov/programs/cb/. See also PIC ID Nos. 5387-5387.3.

AGENCY SPONSOR: Administration on Children, Youth, and Families

FEDERAL CONTACT: Gail E. Collins

PHONE NUMBER: (202) 205-8087

PIC ID: 5387.4

PERFORMER ORGANIZATION: Walter R. McDonald & Associates, Incorporated, Rockville, MD

REPORT: Descriptive Study of the Head Start Health Component

ABSTRACT: This study provides a description of the Head Start health component, which encompasses medical, dental, nutritional, and mental health domains. It describes health screening, examination, referral, treatment, and followup procedures across the four health domains. It reviews the case records of 1,200 4-year-olds enrolled in a national sample of 40 randomly selected Head Start programs and interviews their parents regarding the child's health status and health service utilization patterns. It also describes Head Start programs' staffing patterns and staff training, utilization of community health resources, and barriers to the provision of health services through interviews with appropriate Head Start personnel. The final report was completed in November 1996.

AGENCY SPONSOR: Administration on Children, Youth, and Families

FEDERAL CONTACT: James Griffin

PHONE NUMBER: (202) 205-8138

PIC ID: 5849

PERFORMER ORGANIZATION: CDM Group, Incorporated, Chevy Chase, MD

REPORT: Evaluation of Nine Comprehensive Community-Based Child Abuse and Neglect Prevention Programs: Cross-Site Evaluation Report

ABSTRACT: This evaluation (1) designed and implemented a process and impact evaluation of nine comprehensive community-based child abuse and neglect prevention projects funded by the National Center for Child Abuse and Neglect (NCCAN), (2) provided technical assistance to the nine projects in meeting the requirements of the evaluation, and (3) aided the programs in their efforts to design and implement their own internal program evaluations. The evaluation was conducted in three phases over a 3-year period. The nine projects examined had up to ten service components, many of which were different across the programs. Therefore, a series of individual experimental designs were developed for each service component. Process and impact data were collected across programs and through a series of studies conducted in each site. All grantees were aided in refining their evaluation and research plans. The report provides a context for understanding the experiences of the nine projects. A literature review, an examination of study methodology, the projects' implementation experiences, an accounting of the study findings, and policy recommendations are offered. The report finds that several program elements are vital to the success of the projects. These include (1) an emphasis on community involvement and ownership, (2) employing a positive approach, (3) starting on a small scale, and (4) implementing a strong evaluation and using it as a program management tool. The report recommends that NCCAN implement several strategies to enhance the success of these kinds of projects, including (1) focusing future grant programs on more narrowly defined target populations, (2) stressing the importance of community involvement, and (3) providing to grantees with a research framework and priorities delineating key research questions on child maltreatment and requiring them to implement appropriate process and outcome evaluation designs. See also PIC ID Nos. 5851.1-5851.9.

AGENCY SPONSOR: Administration on Children, Youth, and Families

FEDERAL CONTACT: Emily Cooke

PHONE NUMBER: (202) 205-8709

PIC ID: 5851

PERFORMER ORGANIZATION: CSR, Incorporated, Washington, DC

REPORT: Preventing Child Abuse and Neglect: A Case Study of Families First in Fairfax

ABSTRACT: This report describes Families First in Fairfax, one of nine demonstration projects funded by the National Center on Child Abuse and Neglect. These projects were intended to provide models of collaborative, community-based strategies for effectively preventing child maltreatment. Developed by the Fairfax County, Virginia Department of Human Development, the program addresses the prevention needs of a diverse and growing multi-ethnic population. Families First in Fairfax consisted of three major components (1) public awareness and community education, (2) early identification and intervention, and (3) crisis intervention. The report finds that (1) the program being operated in Fairfax County at the end of the grant period differed substantially from the one originally proposed--a number of child abuse prevention strategies attempted during the first 2 years were ultimately discontinued; (2) the greatest barrier to program implementation during the first 2 years was a lack of specificity on how to achieve the program's mission; (3) during the last 2 years, program development was dynamic and yielded many effective strategies that were made permanent at the end of the demonstration period; (4) the demonstration program fostered growth in parenting programs, family and early infant health care programs, neighborhood resource centers, and directories of services made available to ethnic minority populations; (5) the program also encouraged increased collaboration and information sharing between the agencies and organizations working in the area of child abuse and neglect prevention; and (6) the biggest problems during the 5-year demonstration period were frequent leadership changes and staff turnover. See also PIC ID Nos. 5851 and 5851.2-5951.9. (Final report: 19 pages.)

AGENCY SPONSOR: Administration on Children, Youth, and Families

FEDERAL CONTACT: Emily Cooke

PHONE NUMBER: (202) 205-8709

PIC ID: 5851.1

PERFORMER ORGANIZATION: CSR, Incorporated, Washington, DC

REPORT: Preventing Child Abuse and Neglect: A Case Study of Community Coalition Acting for Positive Parenting

ABSTRACT: This report describes the Community Coalition Acting for Positive Parenting (CCAPP), one of nine demonstration projects funded by the National Center on Child Abuse and Neglect to develop community-based, collaborative models to effectively prevent child maltreatment. Located in North Philadelphia, Pennsylvania, CCAPP was developed by Temple University's Center for Social Policy and Community Development as a university-community collaboration to provide child abuse prevention interventions at the community level. The program attempted to stimulate and support a wide variety of prevention interventions in the community by providing small grants and technical assistance. The report finds that (1) CCAPP affected three different groups: the community, child abuse professionals, and the University, allowing members of each group to learn from members of other groups and fostering an expanded community cooperation; (2) CCAPP sparked the institutionalization of community-based efforts to support families and inform the community about its role in preventing child abuse; (3) the creation of a community-based planning and development council or community board through which key leaders can both receive and disseminate information is very important; (4) CCAPP shifted its attention away from child abuse and neglect to a more positive, family support approach by fostering Family Life Festivals, library programs, and parent-child activities, which reduced parental isolation; (5) using community-based providers as agents for prevention activities enabled the program to attract more of the target population; and (6) universities that have experience working with social issues facilitate program implementation and lend legitimacy to community-based efforts. See also PIC ID Nos. 5851-5851.1 and 5851.3-5851.9. (Final report: 19 pages.)

AGENCY SPONSOR: Administration on Children, Youth, and Families

FEDERAL CONTACT: Emily Cooke

PHONE NUMBER: (202) 205-8709

PIC ID: 5851.2

PERFORMER ORGANIZATION: CSR, Incorporated, Washington, DC

REPORT: Preventing Child Abuse and Neglect: A Case Study of Family Care Connection

ABSTRACT: This report describes Family Care Connection (FCC), based in Allegheny County, Pennsylvania and one of nine demonstration projects funded by the National Center on Child Abuse and Neglect (NCCAN). Developed and administered by Community Health at the Children's Hospital of Pittsburgh, the program provided family support incorporating neighborhood drop-in centers, neighborhood-based task forces, parenting classes and support groups, home visits, substance abuse counseling, outreach, a school-based program, and public awareness activities. FCC sought to replicate in several high-risk communities a preexisting program that used community-based parent education to improve parent-child relationships. FCC expanded services at an existing drop-in center and established four centers in three more at-risk communities. The centers provided services such as respite care, child development, recreation, education, and counseling. An evaluation of the program found that (1) 279 parents had graduated from the parenting classes between September 1989 and June 1991, and that these graduates reported that they would be less likely to use physical punishment with their children; (2) the rate of low-birthweight babies decreased after program implementation in two communities; and (3) FCC was successfully established as an institution in the communities through aggressive pursuit of funding and by using staff that were hired by and paid by "partner agencies." The report concludes that FCC achieved notable success in institutionalizing its services in the communities in which it established drop-in centers under the NCCAN grant. It carefully selected community agencies as partners, and established personal contacts and networking connections with agencies and organizations in its target communities. See also PIC ID Nos. 5851-5851.2 and 5851.4-5851.9. (Final report: 16 pages.)

AGENCY SPONSOR: Administration on Children, Youth, and Families

FEDERAL CONTACT: Emily Cooke

PHONE NUMBER: (202) 205-8709

PIC ID: 5851.3

PERFORMER ORGANIZATION: CSR, Incorporated, Washington, DC

REPORT: Preventing Child Abuse and Neglect: A Case Study of Community Lifelines Program

ABSTRACT: This report describes the Community Lifelines Program (CLP), one of nine demonstration projects funded by the National Center on Child Abuse and Neglect. Developed by the Family Life Development Center of Cornell University in collaboration with the government and human service agencies of Chemung County, New York, the program implemented activities designed to alleviate community conditions that lead to isolation, poor self-image, and economic stress, thereby reducing some of the underlying causes of child abuse and neglect. CLP was responsible for several program initiatives in the city of Elmira and the rural Van Etten/Spencer school district, which included Parent Partner Programs (PPPs), parent support groups, activities encouraging better parent-child communication, and family support programs. The report describes the community, the grantee agency, the program design and changes over time, barriers to program implementation, and strategies used to overcome these barriers. It also describes the program's effects and efforts to institutionalize the various components of the CLP. The report finds that the Elmira PPP was very successful, as was the CLP in Van Etten/Spencer. In Elmira, the PPP fostered better relationships between schools and parents, between individual parents, and between parents and children. In Van Etten/Spencer, several initiatives were successful, including a program for parents of teenagers (Parent Lifelines), an after-school program, and a playgroup. However, several other initiatives had little impact or were initiated too late in the demonstration period to be measured. These included the Family Portraits component, the Family Connections Room, and Food Stamps Outreach. The public awareness campaign launched in conjunction with CLP was successful and useful, according to a reader survey. See also PIC ID Nos. 5851-5851.3 and 5851.5-5851.9. (Final report: 25 pages.)

AGENCY SPONSOR: Administration on Children, Youth, and Families

FEDERAL CONTACT: Emily Cooke

PHONE NUMBER: (202) 205-8709

PIC ID: 5851.4

PERFORMER ORGANIZATION: CSR, Incorporated, Washington, DC

REPORT: Preventing Child Abuse and Neglect: A Case Study of PARE (Physical Abuse and Neglect Reduction Effort)

ABSTRACT: This report describes the Physical Abuse and Neglect Reduction Effort (PARE), one of nine demonstration projects funded by the National Center on Child Abuse and Neglect. This project was developed by the Exchange Club Center for the Prevention of Child Abuse (ESCAPE), located in Carolina, Puerto Rico. The report discusses the environmental factors and indicators of social disruption associated with increased risk for child abuse and neglect in Puerto Rico. It provides a comprehensive overview of all aspects of PARE's program design and operations, as well as evaluation findings. The PARE program provided intervention strategies in an effort to stop the cycle of child abuse in the future, while preserving the family. PARE was designed to be family oriented and emphasized self-help and volunteer action. The demonstration model emphasized interventions directed toward the individual, the family, and the community, as well as cultural factors. The report describes several of these interventions, including (1) an interagency task force, (2) a public awareness campaign, (3) respite centers, (4) a life skills curriculum, (5) an educational curriculum for prenatal clinics, (6) a Parent Aid Program, and (7) a Parent Laboratory. The report finds that (1) the development and implementation of three respite centers pioneered new concepts and approaches to providing secondary prevention; (2) 702 volunteers provided support services and served as role models; (3) the program gained the support of the community, government officials, public and private agencies, the business community, and the media; and (4) the public awareness campaign was broadcast throughout Puerto Rico and reached a large segment of the population. The report concludes that PARE increased child abuse awareness in Puerto Rico, but that more work needs to be done before child abuse prevention becomes a priority for leaders and politicians on the island. See also PIC ID Nos. 5851-5851.4 and 5851.6-5851.9.

AGENCY SPONSOR: Administration on Children, Youth, and Families

FEDERAL CONTACT: Emily Cooke

PHONE NUMBER: (202) 205-8709

PIC ID: 5851.5

PERFORMER ORGANIZATION: CSR, Incorporated, Washington, DC

REPORT: Preventing Child Abuse and Neglect: A Case Study of North Lawndale Family Support Initiative

ABSTRACT: This report describes the North Lawndale Family Support Initiative (NLFSI), one of nine demonstration projects funded by the National Center on Child Abuse and Neglect (NCCAN). Developed by the Greater Chicago Council of the National Committee to Prevent Child Abuse, the program tested the efficacy of a model that advocates comprehensive prevention strategies aimed at both the community and its residents. The model called for the provision and coordination of comprehensive prevention services, including community education, parent education, parent support groups, school-based prevention services for children, and therapeutic services for abused children and their families. The report finds that (1) the most successful components of the NLFSI were the advisory council, the community needs assessment, public awareness, and life skills training; (2) the least effective components were the parent education program, the parent support groups, and therapeutic care--this ineffectiveness was due to inadequate program design, local implementation difficulties, and the grantee's perception that NCCAN required more direct services; (3) the NLFSI benefitted from stable and credible staff, who had already established themselves in the community before the program's implementation; (4) staff possessed strong outreach skills which served them well in the public awareness campaign, but they lacked the skills and background needed to implement direct services; and (5) the NLFSI lacked a clearly defined or effectively developed relationship with its grantee agency and had difficulty in being acknowledged as a full-fledged member of the community due to its situation on the campus of a community college located at the edge of the target community. The report also discusses barriers to program implementation and efforts to institutionalize the program, and includes several recommendations in key areas. See also PIC ID Nos. 5851-5851.5 and 5851.7-5851.9.

AGENCY SPONSOR: Administration on Children, Youth, and Families

FEDERAL CONTACT: Emily Cooke

PHONE NUMBER: (202) 205-8709

PIC ID: 5851.6

PERFORMER ORGANIZATION: CSR, Incorporated, Washington, DC

REPORT: Preventing Child Abuse and Neglect: A Case Study of Dorchester CARES

ABSTRACT: This report describes Dorchester CARES, one of nine demonstration projects funded by the National Center on Child Abuse and Neglect (NCCAN). Under NCCAN funding, Dorchester CARES was initially a collaborative project of the Massachusetts Committee for Children and Youth, Inc. and Federated Dorchester Neighborhood House; however, by the end of the demonstration period, the project included many other collaborators. The model was designed to demonstrate that through collaboration, advocacy, resource development, education, and services, a community can create opportunities for families to increase their social networks and become more independent and self-sustaining. The report finds that: (1) Dorchester CARES staff stressed that it was not an agency, or a program, but rather a collaborative comprised of member agencies and staff who thought of themselves as a process; (2) the project reflected an ecological approach that combined a psychiatric model and a sociological model (the first was directed toward a "sick parent" and stressed therapy and education, while the second was aimed toward the "sick society" and stressed educational and political strategies); (3) over the demonstration period, the program was able to improve family functioning to provide a more nurturing environment for children's development; (4) the program created opportunities for family members to become more independent and self-sustaining and for community members to work together to achieve common goals; (5) the CARES family support program worked by proactively encouraging social networking, increasing nurturing values and skills, and empowering parents to carry out their caregiving roles; and (6) the program was successfully duplicated in several other neighborhoods and was institutionalized in Dorchester. See also PIC ID Nos. 5851-5851.6 and 5851.8-5851.9. (Final report: 21 pages.)

AGENCY SPONSOR: Administration on Children, Youth, and Families

FEDERAL CONTACT: Emily Cooke

PHONE NUMBER: (202) 205-8709

PIC ID: 5851.7

PERFORMER ORGANIZATION: CSR, Incorporated, Washington, DC

REPORT: Preventing Child Abuse and Neglect: A Case Study of Maine Families

ABSTRACT: This report describes Project Maine Families, one of nine demonstration projects funded by the National Center on Child Abuse and Neglect. The Maine Families project was the result of a collaboration between the urban Cumberland County Child Abuse and Neglect Council (CCCANC) and the rural Franklin County Children's Task Force (FCCTF); other local community, school, and service organizations joined the two county organizations to identify and address the needs of children and their families. CCCANC implemented a school family center, a media program, a public library discussion series, a teenage parent day care and support center, a drop-in laundry program, a parents' speaker and support group, and a parent education program in a low-income housing development. FCCTF implemented a summer reading program, a resource directory, a parent education program for Head Start-eligible families, parent cooperative support and education groups, and a parent mentoring and home visitation program. In both counties, the project sponsored community events to reduce the isolation of families and to encourage a sense of support and togetherness. The evaluation of the Maine Families project consisted primarily of a qualitative assessment of the reactions of program participants and leaders, finding that (1) the project was guided by an effort to structure interventions based upon what parents said they needed, rather than on what project staff or other agency staff believed they needed; (2) focus groups revealed that few parents believed that they were doing an effective parenting job and most felt isolated from support and in need of information and assistance; (3) the project emphasized involving business and industry in designing and implementing community programming; and (4) collaboration is enhanced by using a businesslike approach. See also PIC ID Nos. 5851-5851.7 and 5851.9. (Final report: 18 pages.)

AGENCY SPONSOR: Administration on Children, Youth, and Families

FEDERAL CONTACT: Emily Cooke

PHONE NUMBER: (202) 205-8709

PIC ID: 5851.8

PERFORMER ORGANIZATION: CSR, Incorporated, Washington, DC

REPORT: Preventing Child Abuse and Neglect: A Case Study of I CARE

ABSTRACT: This report describes the I CARE Project, one of nine demonstration projects funded by the National Center on Child Abuse and Neglect. The project was developed by the Ohio Research Institute on Child Abuse and Prevention to model a prevention program in direct response to the needs and existing resources of the target community. The program implemented four interventions: (1) home health visitation for first-time parents and families with newborns, (2) parent education focused on child development and management of specific child behaviors, (3) a public awareness program in the target community, and (4) provision of support in obtaining basic necessities for families in need. I CARE distributed a series of educational Child Behavior Management Cards throughout the Columbus, Ohio area, as well as refrigerator magnets, local resource directories, and other resource and awareness materials throughout the target community. In addition, monthly family get-togethers were held for parents participating in the home visitation program in order to decrease their social isolation and to provide a forum for discussion and education. After the conclusion of the program, I CARE staff made several conclusions and recommendations, including that: (1) a prevention model containing all the components specified for this demonstration grant would be difficult, if not impossible, notwithstanding a budget of $200,000 per year; (2) although a community-based program cannot correct all the societal problems leading to child abuse and neglect, it should be aware of their effects; (3) an interdisciplinary task force from the community should be considered mandatory; and (4) the uncertainty of future funding is a major obstacle for demonstration or pilot programs. See also PIC ID Nos. 5851-5951.8. (Final report: 23 pages.)

AGENCY SPONSOR: Administration on Children, Youth, and Families

FEDERAL CONTACT: Emily Cooke

PHONE NUMBER: (202) 205-8709

PIC ID: 5851.9

PERFORMER ORGANIZATION: CSR, Incorporated, Washington, DC

REPORT: Responsible Fatherhood: An Overview and Conceptual Framework

ABSTRACT: A growing consensus holds that responsible fathering means establishing paternity, being present in the child's --even if divorced or unmarried, sharing economic support, and being personally involved in the child's life in collaboration with the mother. This report, co-sponsored by the Administration for Children and Families and the Office of the Assistant Secretary for Planning and Evaluation, summarizes the research on factors that influence fathering and presents a systemic, contextual framework that highlights multiple interacting influences on the father-child relationship. A principal finding of the report is that fathering is influenced, even more than mothering, by contextual forces in the family and the community. A father in an adversarial relationship with the mother is at risk of becoming an irresponsible father, as is a father who lacks adequate employment and income. By the same token, responsible fathering can be fostered by positive changes in cultural, economic, institutional, and interpersonal influences. Thus, the report contends that fathering programs should involve a wide range of interventions. In particular, these programs should (1) involve the mother where feasible, and for unmarried fathers, the families of origin; (2) promote collaborative coparenting; (3) emphasize critical transitions, such as the birth of a child and divorce of the parents; (4) deal with employment, economic issues, and community systems; (5) promote father-to-father learning; and (6) promote the viability of committed and collaborative marriage. By developing a theoretical underpinning to guide empirical research, program development, and program evaluation, this project is intended to help inform policymakers about what is necessary to enable fathers to support and nurture a child.

AGENCY SPONSOR: Office of Planning, Research, and Evaluation

FEDERAL CONTACT: Mark Fucello

PHONE NUMBER: (202) 401-5074

PIC ID: 5981

PERFORMER ORGANIZATION: University of Minnesota, Minneapolis, MN

REPORT: Arkansas Prenatal and Postnatal Paternity Acknowledgment Project

ABSTRACT: In Arkansas, close to 1 in 3 births are to unmarried mothers. Prior to the establishment of the paternity acknowledgment project described in this report, half of the nonmarital birth certificates did not identify the fathers of these children. The Arkansas paternity acknowledgment project was multifaceted, ambitious in scope, and considerably changed over the life of the 3-year demonstration period. There were five basic models used in the project: (1) basic in-hospital paternity acknowledgment with a genetic testing option; (2) prenatal and multiple postnatal opportunities to acknowledge paternity; (3) interstate paternity acknowledgment transfer project (parents from one State traveled to another State, gave birth, and signed paternity acknowledgments in that State); (4) home visiting nurse program; and (5) alternative parental support program, which abates or reduces child support payments for young fathers who participate in opportunities to increase parental responsibility. The report finds that (1) IV-D (child support) staff identified mothers uncooperative in establishing paternity and surveyed them regarding their lack of cooperation: these mothers stated that they did not want to establish paternity because the child's father was not involved, that the father gave the mother money already, or that the mother did not know where the father was; (2) by the end of the project, 13,688 paternity acknowledgments had been received, and the ratio of paternity acknowledgments to unmarried births ranged from 48 percent in 1995 to 56 percent for the first three months of 1996; and (3) 38 percent of the acknowledgments were matched to children in the Arkansas child support data base, and 27 percent of these had closed by September 1996 (payments were received on 16 percent of matched cases). (Final report: 111 pages, plus appendix.)

AGENCY SPONSOR: Office of Child Support and Enforcement

FEDERAL CONTACT: David Arnaudo

PHONE NUMBER: (202) 401-5364

PIC ID: 6221

PERFORMER ORGANIZATION: State of Arkansas, Division of Revenue, Little Rock, AR

REPORT: LEAP: Final Report on Ohio's Welfare Initiative to Improve School Attendance Among Teenage Parents--Ohio's Learning, Earning, and Parenting Program

ABSTRACT: Ohio's Learning, Earning, and Parenting (LEAP) program is a statewide initiative that uses financial incentives as a way to increase school enrollment and attendance among pregnant teenagers and custodial teen parents on welfare. The program, which began in 1989, requires these parents (mostly mothers) to stay in school and attend regularly, or, if they have dropped out, to return to school or to prepare for the General Educational Development (GED) test. During the period of this study, teens who met LEAP's requirements received increases to their welfare checks ($62 for school enrollment and an additional $62 each month they attended school regularly). Teens who did not meet program requirements had the same amount deducted from their welfare grant until they complied with program rules. This report is based on a study of 4,151 teens who were identified as eligible for LEAP during the program's second year of operation (August 1990 through September 1991). The report finds that (1) even the initial implementation of LEAP was a considerable achievement as an unprecedented statewide effort requiring new linkages between county welfare and education systems; (2) the program achieved smooth operations after the implementation of a sophisticated statewide public assistance computer system; (3) to a large degree, LEAP met its initial goals to significantly increase school enrollment and attendance for in-school teens and drop-outs, which were directly linked to the incentives and sanctions provided; (4) in general, the impacts on school completion and employment were more positive for teens who were still in school when they were identified as eligible for the program (initially-enrolled participants) than for drop-outs who returned to school; and (6) LEAP increased the employment rates for initially-enrolled teens and had positive impacts on their earnings during the first 2 years of the 4-year followup. (Final report 130 pages plus appendices.)

AGENCY SPONSOR: Office of Planning, Research, and Evaluation

FEDERAL CONTACT: Nancye Campbell

PHONE NUMBER: (202) 401-6659

PIC ID: 6668

PERFORMER ORGANIZATION: Manpower Demonstration Research Corporation, New York, NY

REPORT: First Progress Report on the Head Start Program Performance Measures

ABSTRACT: The Head Start program, established in 1965, has served almost 14 million preschool children. In 1995, the program joined with other Federal programs in developing performance measures to promote accountability through the assessment of program quality and outcomes. This report is the first program assessment using these performance measures. It describes the development of the performance measures, noting that they focus on outcomes rather than on processes. In other words, the outcomes to be measured concern how well the children in the program do, rather than on such things as the credentials of teachers. The Head Start program also has in place a series of Program Performance Standards, which define program activities, while the Program Performance Measures define program results. The report describes the background of the Program Performance Measures development, including the conceptual model used and data sources. Finally, the report includes an account of how the Head Start program will gather data for the measures and establish a "feedback loop" for policy and resource decisions. The report concludes that the performance measures will provide a snapshot of Head Start's program performance at a given point in time and will allow the program to compare its performance to the past. At the national level, it will allow the program to determine how well it is doing through the production of periodic national Head Start progress reports. This first report provides a benchmark against which future performance can be measured and furnishes program accountability information to be used in appropriations decisions. (Final report: 31 pages, plus appendices.)

AGENCY SPONSOR: Administration on Children, Youth, and Families

FEDERAL CONTACT: John Connigan

PHONE NUMBER: (202) 401-5916

PIC ID: 6693

PERFORMER ORGANIZATION: Caliber Associates, Fairfax, VA

TOP OF DOCUMENT

Agency for Health Care Policy and Research

Contents

Publications of the Patient Outcomes Research Teams (PORTS), May 1996 and Medical Treatment Effectiveness Projects 1989 through 1995

Final Report on Changes Needed in CONQUEST's Software Based Programs

Final Report on Changes Needed in CONQUEST 1.0's Structure and Classification Scheme

Assessment of the Feasibility of Creating a Managed Care Encounter-Level Data base

Customer Satisfaction Survey of AHCPR Publications Recipients

Effects of Informatics Tools and Decision Aids to Help Patient Decision-making about Medical Screening and Treatment

Diagnostic Accuracy in Primary Care: Review of the Literature on Five Chronic Conditions-- Final Report

Evaluation of AHCPR Minority Health Services Research Training Activities

Outcomes and Effectiveness Research in the Private Sector

Partners in Research: Identifying Common Interests

Performance Measurement Inventory Phases II & III Workplan

Final Report: Small Conference Grant Program Evaluation

TITLE: Publications of the Patient Outcomes Research Teams (PORTS), May 1996 and Medical Treatment Effectiveness Projects 1989 through 1995

ABSTRACT: This project tracked the activities and effects of research conducted under the Medical Effectiveness Treatment Program (MEDTEP). Using secondary data sources such as newspaper and journal articles, the project collected information on the products of Agency for Health Care Policy and Research-sponsored medical effectiveness research and the diffusion of these findings into other sources. A bibliography entitled "Publications of the Patient Outcomes Research Teams (PORTS): PORT and PORT II Projects" provides information about publications arising from each PORT, listed by the condition that each addressed. Also included is information about the grant or contract number, project title, name and institution of the principal investigator, project start and completion dates, and a brief description of the project. A second bibliography entitled "Medical Treatment Effectiveness Program Research Projects: 1989 through 1995" presents a comprehensive list of MEDTEP research projects. Projects are categorized under Clinical Studies, with subcategories; Guideline Implementation and Evaluation; and Methods and Measures. Within each category, projects are in alphabetical order by the name of the principal investigator. PORT and PORT II projects are marked. As in the first bibliography, there is information about grant or contract number, project title, name and institution of the principal investigator, project start and completion dates, and a very brief description of the project. For those projects that have been completed, executive summaries and final reports may be available through the National Technical Information Center. (Bibliography: 49 pages; bibliography: 73 pages.)

AGENCY SPONSOR: Center for Outcomes and Effectiveness Research

FEDERAL CONTACT: Mary Cummings

PHONE NUMBER: (301) 594-1485

PIC ID: 4311

PERFORMER ORGANIZATION: Walcoff & Associates Incorporated, Fairfax, VA

TITLE: Final Report on Changes Needed in CONQUEST's Software Based Programs

ABSTRACT: This report provides the results of an evaluation of the CONQUEST data base in order to determine the best short-term, intermediate, and long-range strategies for its continued development. Three important findings emerged from this study. First, Microsoft Access was judged to be the best platform (out of 13 alternatives) for CONQUEST in the short and immediate terms. Furthermore, many opportunities exist to improve CONQUEST over the short and immediate terms. Problems identified by users can be corrected by minor enhancements, error corrections, and superficial changes. More substantial changes are also needed, including restructuring of the data base and rewriting of the queries and Access Basic code that perform the underlying CONQUEST search and retrieval operations. Finally, the report determines that the best long-range strategy for disseminating CONQUEST information to the broadest possible audience over the widest variety of platforms is via World Wide Web. This finding also lists the advantages of using the Internet, describes the labor and equipment needed, and estimates that the project would be a multi-year effort costing more than one million dollars. The report concludes that moving CONQUEST in its current form to another stand-alone data base is not desirable. Investments in software should be limited to necessary, cost-effective changes. Further development of the data base should be designed based on the findings of this report, which should also be used in the selection of a new implementation platform. See also PIC ID Nos. 5961 and 5961.2. (Final report: 21 pages, plus appendices.)

AGENCY SPONSOR: Center for Quality Measurement and Improvement

FEDERAL CONTACT: Marge Keyes

PHONE NUMBER: (301) 594-1352

PIC ID: 5961.1

PERFORMER ORGANIZATION: MEDSTAT Group, Cambridge, MA

TITLE: Final Report on Changes Needed in CONQUEST 1.0's Structure and Classification Scheme

ABSTRACT: This report presents the findings of an evaluation of the structure and classification scheme of CONQUEST 1.0. The evaluation identified changes needed to enhance the use and effectiveness of the data base's structure and classification scheme for codifying clinical performance measures and conditions. The evaluation addressed three broad questions: (1) how does the current application of CONQUEST 1.0 compare with its intended use; (2) what are the impediments to potential users; and (3) which structural and classification scheme improvements are needed in the Condition and Measures data bases? The report finds that the data base is used as intended. Users access it to learn which processes and outcomes to measure for particular conditions, to compare and select clinical performance measures, and to find measures related to practice guidelines. The report also finds that the data base must be kept current and comprehensive in order for it to be of use. Advanced and mid-level users would like additional types of measures, particularly measures on satisfaction with care and access. They would also like more functional status measures and clinical measures for non-acute settings. In regard to the structure of the data base and the classification scheme, (1) many performance measures lack enough similarities to be grouped into batches, (2) the quality of the match between conditions and measures designed for them should be improved, (3) most searches for clinical performance measures for specific conditions returned at least one measure that was not well-matched, (4) codes for clinical events may be too broad, (5) the condition list presented to users should be broadened, and (6) users would like additional information about the performance measures. The report also includes several recommendations to improve the quality of the data base. See also PIC ID Nos. 5961 and 5961.1. (Final report: 16 pages, plus appendices.)

AGENCY SPONSOR: Center for Quality Measurement and Improvement

FEDERAL CONTACT: Marge Keyes

PHONE NUMBER: (301) 594-1352

PIC ID: 5961.2

PERFORMER ORGANIZATION: MEDSTAT Group, Cambridge, MA

TITLE: Assessment of the Feasibility of Creating a Managed Care Encounter-Level Data base

ABSTRACT: The rapid growth of managed care in recent years is transforming the Nation's health care system. In the process, it is also changing the ability of health services researchers and policy analysts to answer fundamental questions about access, utilization, cost, and quality of care. Data produced as a "byproduct" of the fee-for-service system (including plan-, provider-, person-, and encounter-level data) are either no longer available or vary in comprehensiveness and accuracy. As a result, there is widespread public and private interest in managed care organizations' (MCOs') ability to collect encounter-level data and in the development of an encounter-level data base that could be used for the purposes of research. This report (1) identified models that the Agency for Health Care Policy and Research (AHCPR) could employ to work with plans to facilitate the availability of data for research, (2) examines the availability and comprehensiveness of encounter-level data in MCO data bases, and (3) identifies the problems and challenges that might be encountered when trying to use or combine data from MCOs. Among key findings are that: (1) MCO data bases vary substantially in the availability and comprehensiveness of encounter-level data and the ability to link these data with administrative and financial data; (2) MCOs increasingly view their information systems and the data they store, as proprietary, and there are a number of disincentives for MCOs to share their data; (3) computer systems, which were originally touted as an inexpensive way to capture data, are technically complex and costly--problems exacerbated by the lack of industry-wide definitions and reporting standards; and (4) despite these barriers, MCOs are interested in exploring ways to work with AHCPR and other Federal agencies to facilitate the availability of data for the purposes of research. (Final report: 52 pages, plus appendices.)

AGENCY SPONSOR: Center for Organization and Delivery Studies

FEDERAL CONTACT: Kelly Devers

PHONE NUMBER: (301) 594-1410

PIC ID: 6374

PERFORMER ORGANIZATION: Research Triangle Institute, Research Triangle Park, NC

TITLE: Customer Satisfaction Survey of AHCPR Publications Recipients

ABSTRACT: The Agency for Health Care Policy and Research (AHCPR) provides most information services to consumers through its Publication Clearinghouse. This report assesses consumer satisfaction with the content, format, and procedures for AHCPR publication distribution. The study statistically analyzes the results of a mail survey protocol for individuals who requested one of four patient/consumer guides: Prescription Medicines and You, Understanding Incontinence, Early Alzheimer's Disease, and You Can Quit Smoking. The report finds that respondents (1) find the guides of greater interest to elderly persons; (2) make 75 percent of their requests by telephone; (3) are pleased with the services of the Clearinghouse in distributing patient/consumer guides; (4) with access to the Internet (17 percent) still prefer to order the guides by telephone; (5) order the guides for themselves (70 percent) and that two-thirds keep it for reference; (6) respond positively to different editorial and typographical features of the guides; (7) express little interest in making any of the nine suggested changes to the guides; and (8) are satisfied overall with the guides, and that the 22 percent who are not satisfied cite insufficient information. Although the diversity of the four guides makes generalizations difficult, the report concludes that (1) due to the large number of phone requests, AHCPR should give Clearinghouse staff information about the contents of the patient/consumer guides in order to provide better customer service; (2) AHCPR should make no changes to the organization of the guides or how information is presented; and (3) AHCPR could pursue methods to identify and answer the questions and concerns of individuals who feel that the guides provide insufficient information. (Final report: 15 pages, plus tables and appendices.)

AGENCY SPONSOR: Center for Health Information Dissemination

FEDERAL CONTACT: Donna Rae Castillo

PHONE NUMBER: (301) 594-1362

PIC ID: 6375

PERFORMER ORGANIZATION: Battelle Corporation, Arlington, VA

TITLE: Effects of Informatics Tools and Decision Aids to Help Patient Decision-making about Medical Screening and Treatment

ABSTRACT: This report identifies, evaluates, and synthesizes published and unpublished research on the effects of informatics tools and decision aids used to help patients make decisions about their screening and treatment. These tools provide treatment- and disease-specific information to patients in computerized or noncomputerized form. The report finds that few controlled studies have been conducted on the effects of these tools and aids. Most only consider effects on patient knowledge and satisfaction, and examine only a few treatment choices. Few studies have investigated the effects of tools on patient-clinician communications and health behavior and health outcomes. There are no comparative studies of the cost and effectiveness of different types of tools, and the studies that have been done are of varying methodological quality. Despite these limitations, certain conclusions can be drawn. For example, (1) studies suggest that the use of informatics tools can increase patients' knowledge of treatment alternatives, and patients report that they like these tools; (2) limited evidence suggests that tools influence treatment selection; (3) greater attention to rigorous controls, adequate sample size, and standardized measurement is crucial; (4) use of tools by patients can be limited, especially if there are operational barriers to their use (such as a requirement for a second visit); (5) patients who use informatics tools generally increase their knowledge of treatment alternatives, but the comparative effectiveness of different types of tools in increasing knowledge has not been well assessed; (6) some trials find that patients who use tools are significantly more satisfied with the decisionmaking process than others and that these patients ask their physicians more questions; and (7) scattered evidence suggests that these tools can improve health outcomes by increasing patients' adherence to treatment regimens. (Final report: 43 pages, plus appendices.)

AGENCY SPONSOR: Center for Organization and Delivery Studies

FEDERAL CONTACT: Denise Dougherty

PHONE NUMBER: (301) 594-1321

PIC ID: 6376

PERFORMER ORGANIZATION: Research Triangle Institute, Research Triangle Park, NC

TITLE: Diagnostic Accuracy in Primary Care: Review of the Literature on Five Chronic Conditions--Final Report

ABSTRACT: This report presents the findings of an iterative literature search and analysis on diagnostic errors and inefficiencies in primary care for five clinical conditions: cutaneous melanoma, breast cancer, heart failure, depression in children, and pediatric asthma. The report points out several gaps in the health services and the clinical knowledge base. It also suggests that these gaps can be addressed by a comprehensive research agenda on primary care, which can be used by the Agency for Health Care Policy and Research for future research agendas on each of the specific conditions. Specific research issues include the extent of inaccurate (missed, erroneous, or delayed) diagnoses, and the causes of such deficiencies; the health and monetary consequences of diagnostic errors and inefficiencies; and the effectiveness of interventions aimed at preventing or overcoming the problems. (Final report variously paginated, plus appendices.)

AGENCY SPONSOR: Center for Organization and Delivery Studies

FEDERAL CONTACT: James Cooper

PHONE NUMBER: (301) 594-1357

PIC ID: 6381

PERFORMER ORGANIZATION: Research Triangle Institute, Research Triangle Park, NC

TITLE: Evaluation of AHCPR Minority Health Services Research Training Activities

ABSTRACT: This report reviews Agency for Health Care Policy and Research (AHCPR) initiatives in health services research training to determine the degree to which they have increased minority participation in health services research. It examines two types of initiatives: (1) pre- and post-doctoral training provided through National Research Service Award grants; and (2) supplementary funding made available to individual AHCPR grantees for the purpose of providing training to individual minority candidates or investigating an area related to minority issues in health services. The report finds that AHCPR has been somewhat successful in increasing minority participation in health services research training. It identified five criteria that are necessary for program success: recruitment, leadership, networking, mentoring, and building a community of scholars. The best practices of successful non-Federal programs were identified as (1) developing sufficient infrastructure to support training, (2) providing sufficient financial support to attract the brightest trainees, and (3) providing necessary support for faculty mentors involved in the program. (Final report variously paginated.)

AGENCY SPONSOR: Office of the Administrator

FEDERAL CONTACT: Morgan Jackson

PHONE NUMBER: (301) 594-1455

PIC ID: 6384

PERFORMER ORGANIZATION: Barents Group, KPMG Peat Marwick LLP, Washington, DC

TITLE: Outcomes and Effectiveness Research in the Private Sector

ABSTRACT: This project analyzed and evaluated private sector activities in performing outcomes and effectiveness research (OER). It was intended to help clarify the Agency for Health Care Policy and Research's (AHCPR's) future priorities in outcomes and effectiveness research, and to provide insights regarding future opportunities for public-private partnerships in this area. The project reviewed the published literature and other documentation, conducted telephone interviews, conducted case studies, and hosted an expert panel with private sector OER leaders. The study finds that (1) organizations reporting OER activities include large insurers, managed care organizations, health product companies, employer groups, associations, and foundations; (2) private sector organizations conduct OER primarily to improve their strategic and financial positions; (3) private sector OER does not overlap considerably with government-sponsored OER; (4) health product companies usually focus their OER efforts on their new drugs and devices, while less attention is given to other health care interventions, particularly those that are not new or those used primarily for traditionally underserved or vulnerable populations; (5) although provider organizations increasingly conduct a variety of activities involving outcomes measurement, including some OER as defined in the report, much of the outcomes measurement is for monitoring health care performance against certain standards (for instance, in the case of accreditation and quality assurance); and (5) AHCPR has supported the development and validation of many of the research tools used in private sector OER. The study concludes that government-sponsored OER can continue to address certain major topics that are not covered by the private sector, and can continue to advance research tools, training, and other aspects of OER that will benefit private sector efforts.

AGENCY SPONSOR: Center for Outcomes and Effectiveness Research

FEDERAL CONTACT: Yen-pin Chiang

PHONE NUMBER: (301) 594-1485

PIC ID: 6385

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Partners in Research: Identifying Common Interests

ABSTRACT: The Agency for Health Care Policy and Research (AHCPR) undertook this project to develop a methodology for systematically matching research interests of private sector organizations with its own in order to identify potential research partners. The methodology was developed in several stages: (1) a segmentation was developed to classify private sector entities by research focus; (2) a matrix identified promising overlap between private sector research and AHCPR research interests; and (3) sample profiles were developed for nine organizations in four health industry segments (pharmaceutical/biotechnology; manufacturing/suppliers; health information systems; and philanthropy). The profiles were created for Amgen, Bristol Myers Squibb, Eli Lilly, American Home Products, Medtronic, HCIA, HBO and Company, and the California Endowment. These profiles demonstrate how available sources of information on organizations can be tapped to provide a detailed background for screening these organizations to assess opportunities for research partnering. The methodology focused on publicly traded companies and private foundations, although it is recognized that attractive partnering opportunities exist with privately held companies and a variety of nonprofit organizations other than those with philanthropic missions. The report recommends that AHCPR more fully develop, and then maintain, a matrix of private sector organizations with research interests overlapping its own research portfolio. A deliberate strategy for undertaking one or more collaborative research activities in each cluster should be undertaken in order to provide further experience in the development and management of successful research partnering. Considerations should be given to soliciting input from a cross-section of companies in the various health care market sectors when planning new research initiatives. (Final report variously paginated, plus appendices.)

AGENCY SPONSOR: Center for Organization and Delivery Studies

FEDERAL CONTACT: Eric Katz

PHONE NUMBER: (301) 594-1321

PIC ID: 6386

PERFORMER ORGANIZATION: Center for Health Policy Studies, Columbia, MD

TITLE: Performance Measurement Inventory Phases II & III Workplan

ABSTRACT: The Performance Measurement Workgroup of the U.S. Department of Health and Human Services (HHS) Managed Care Forum requested an inventory of performance measures (i.e., clinical, population, and system) currently being used, under consideration for use, or being developed for use by HHS agencies for regulating, monitoring, comparing, and/or evaluating health care services and systems of care. The inventory, which will be created in a CONQUEST-compatible format, will be used to identify opportunities for better coordination and improved efficiency, and to prepare recommendations concerning the direction and coordination of departmental activities in regard to performance measurement. Ideally the project will include one or more non-HHS agencies, such as the Department of Veterans Affairs (VA) or the U.S. Department of Defense. The project includes three phases: Phase I is workplan development, Phase II is pilot testing of the data collection instruments and instructions, and Phase III is data collection and final report writing. As of October 1997, Phase I has been completed; a draft report for Phase II, which includes participation by the VA is in progress; and a task order for Phase III has been awarded and work is scheduled to begin in the near future. See also PIC ID Nos. 6691.1 and 6691.2. (Final report: 13 pages.)

AGENCY SPONSOR: Center for Quality Measurement and Improvement

FEDERAL CONTACT: Marge Keyes

PHONE NUMBER: (301) 594-1352

PIC ID: 6691

PERFORMER ORGANIZATION: Center for Health Policy Studies, Columbia, MD

TITLE: Final Report: Small Conference Grant Program Evaluation

ABSTRACT: In 1991, the Agency for Health Care Policy and Research (AHCPR) established the current Small Conference Grant Program. The program supports conferences, doctoral dissertations, and small research projects. Under the program, grants of $50,000 or less are awarded to nonprofit organizations to support conferences on issues relevant to health services research and to AHCPR's mission. In January 1996, AHCPR made revisions to streamline the program, which included the implementation of concept letter procedures. The concept letter saves time for both reviewers and applicants because only those conferences that are of interest to AHCPR move to the full review process. The report finds that (1) small grant conference procedures are efficient and result in conferences that meet AHCPR's goals, (2) the objectives of the conferences are being met, and (3) outcomes of the conferences are in accord with their stated purpose. The study recommends that (1) future grant announcements should set a realistic, experience-based time frame for both the review of concept letters and grant applications; (2) AHCPR consider instituting a time limit on application submission once the applicant has been notified of concept letter acceptance to help the agency develop a more realistic list of expected or pending applications; (3) AHCPR be more proactive in planning the dissemination strategy of a conference to increase the impact of a conference; and (4) in the future, AHCPR might consider lending its name, rather than its money, to conferences, as a way for the Agency to extend its reach without increasing funds to the program. (Final report variously paginated, plus appendices.)

AGENCY SPONSOR: Office of Health Care Information

FEDERAL CONTACT: Donna Rae Castillo

PHONE NUMBER: (301) 594-1362

PIC ID: 6692

PERFORMER ORGANIZATION: Center for Health Policy Studies, Columbia, MD

TOP OF DOCUMENT

AGENCY FOR TOXIC SUBSTANCES AND DISEASE REGISTRY

Contents

Did We Make A Difference? ATSDR's Framework for Community-Focused Evaluation

Site-Specific Outcome Measurement Guidelines for ATSDR Cooperative Agreement States

REPORT TITLE: Did We Make A Difference? ATSDR's Framework for Community-Focused Evaluation

ABSTRACT: In 1994, the Environmental Health Policy Committee, part of the Public Health Service, recommended that all Federal agencies develop guidelines to establish outcome and impact data for use in the evaluation of their activities and products. This report outlines a framework for implementing community-focused evaluation, a program of activities undertaken to assess the outcomes and impacts of the Agency for Toxic Substances and Disease Registry (ATSDR) site-specific activities. The seven steps of the framework include (1) gathering community information, (2) defining the public health problem, (3) determining and implementing the public health action, (4) establishing evaluation goals and objectives, (5) developing outcome measures as indicators of success, (6) collecting and analyzing data, and (7) using feedback to improve public health actions. The report concludes that (1) ATSDR's community-focused evaluation framework provides a comprehensive, flexible, and timely approach to assess the short- and long-term effects of ATSDR's site-specific activities; (2) community-focused evaluation must be viewed as effective and taken seriously as a central part of responsible public health practice, and not merely as an activity to conduct if sufficient funds are left when the program is completed; (3) the collaborative and educational functions of evaluation create promising prospects for facilitating change because of their inherent emphasis on the interactive relationship between ATSDR and the community; and (4) ATSDR's community-focused evaluation program provides the Agency with the opportunity to communicate the value of its products and services to key stakeholders, while allowing for critical feedback to management and staff. See also PIC ID No. 6853.1. (Final report: 16 pages.)

AGENCY SPONSOR: Division of Health Education

FEDERAL CONTACT: Maureen Lichtveld

PHONE NUMBER: (404) 639-6204

PIC ID: 6853

PERFORMER ORGANIZATION: Agency for Toxic Substances and Disease Registry, Atlanta, GA

REPORT TITLE: Site-Specific Outcome Measurement Guidelines for ATSDR Cooperative Agreement States

ABSTRACT: The Agency for Toxic Substances and Disease Registry (ATSDR) developed this report to help State cooperative agreement personnel evaluate their efforts in the protection of public health. A number of States collaborated with ATSDR to develop guidelines for assessing the impacts and outcomes of site-specific health education activities. This report was prepared by a group of Federal and State ATSDR Cooperative Agreement staff and was intended for an audience of State and territorial personnel working under ATSDR Cooperative Agreement 607 auspices. The document deals with measurement of community members' response to the Agency's Superfund activities and includes a short section that evaluates the Agency's effect/influence on other agencies with each State. The community member section is divided into five possible areas of evaluation, which include measurements of knowledge, opinion, behavior, exposure, and health. The agency section has only one area of measurement, which is the opinion area. This area was selected in an attempt to measure the influence of ATSDR on environmental decision making and the subsequent protection of public health. This report is part of a set of tools that will allow cooperative agreement participants to assess the overall improvement in public health as a result of site-specific health education activities and to demonstrate the cooperative agreement program's effectiveness.

AGENCY SPONSOR: Division of Health Education

FEDERAL CONTACT: Maureen Lichtveld

PHONE NUMBER: (404) 639-6204

PIC ID: 6853.1

PERFORMER ORGANIZATION: Agency for Toxic Substances and Disease Registry, Atlanta, GA

TOP OF DOCUMENT

CENTERS FOR DISEASE CONTROL AND PREVENTION

Contents

Sexually Transmitted Disease (STD) Prevention in the United States: Integrated Evaluation of Public and Private Sector Disease Reporting and Service Delivery

Evaluation of ICD-10 for Morbidity Reporting in the United States

Suicide Intervention and Prevention: Evaluation of Community-Based Programs in Three American Indian Communities

Evaluation of the Fatality Assessment and Control Evaluation (FACE) Program: Phase II Final Report

Assessment of the National Laboratory Training Network

Race Reporting on Birth Records by Multiracial Women: An Evaluation of Question Format Alternatives

Management and Operational Impacts of HIV Prevention Community Planning: Lessons Learned at Mid-Course and Implications for Future Activities

CDC's Early Implementation of GPRA: A Case Study-- July 1995 to November 1996

Collaborating with Managed Care Organizations for Mammography Screening and Rescreening: Guidance for NBCCEDP Grantees

Reaching Women for Mammography Screening: Successful Strategies of National Breast and Cervical Cancer Early Detection Program (NBCCEDP) Grantees

Evaluation Design for the Business Responds to AIDS (BRTA) Program

Case Studies of the Prevention Marketing Initiatives (PMI) Local Demonstration Site Projects: Experiences During Planning and Transition Phases

Evaluability Assessment of CDC-Funded Traumatic Brain Injury and Spinal Cord Injury Programs

Community Context Study: Minors' Access to Tobacco

Directions for Research on the National Immunization Survey: Final Report

In Her Own Words: A Focus Group Study of Risk and Protective Factors in Intimate Partner Violence

Linking Research and Public Health Practice: A Review of CDC's Program of Centers for Research and Demonstration of Health Promotion and Disease Prevention

Syphilis in the South: A Case Study Assessment in Eight Southern Communities

Evaluation of the Effectiveness of CDC Surveillance for Drug-Resistant Streptococcus pneumoniae (Baltimore, MD)

Evaluation of the Effectiveness of CDC Surveillance for Drug-Resistant Streptococcus pneumoniae (San Antonio, TX)

Assessment of the Programmatic Impact of One-Percent Evaluation Studies

Directory of Evaluations Administered by the CDC Office of Program Planning and Evaluation 1990-1995

TITLE: Sexually Transmitted Disease (STD) Prevention in the United States: Integrated Evaluation of Public and Private Sector Disease Reporting and Service Delivery

ABSTRACT: The Centers for Disease Control and Prevention (CDC) Division of Sexually Transmitted Disease (STD) Prevention provides financial and technical assistance to State and local agencies to support a range of prevention, research, policy development, and surveillance activities. Two parallel STD surveillance systems are currently operated by CDC, both of which rely upon regular, voluntary reporting of STD incidence by State and local agencies. Both systems are thought to suffer from less-than-complete reporting of STDs by diagnosing practitioners. Estimates of this underreporting range from 25 to 90 percent. The focus of this project, therefore, was to design and pilot test a methodology for validating the STD data routinely collected, analyzed, and published by the Division. The study's main objective was to test the feasibility of using a provider-based survey (1) to quantitatively validate CDC's data on STD diagnoses, and (2) to assess the extent to which clinicians follow CDC's treatment guidelines for STDs. The study was designed to generate estimates of STD incidence through a survey of a random sample of STD data providers in four States, and to compare these estimates with State-reported STD incidence for the same time period. The report finds that (1) the four pilot States differ in terms of their STD data-gathering policies and practices, including which diseases are reported, how cases are defined for reporting purposes, how laboratory reporting is utilized, and which date information is recorded for each case; (2) syphilis, gonorrhea, and chlamydia are reportable diseases in all four States, and in two States many additional diseases are reportable; (3) in two States, physicians must report diseases for them to be recorded, while in two others laboratory reports are sufficient; (4) STDs are considerably underreported by providers; and (5) most STD cases are seen by individual private practitioners. (Final report: 82 pages, plus appendices.)

AGENCY SPONSOR: National Center for HIV, STD and TB Prevention

FEDERAL CONTACT: John Moran

PHONE NUMBER: (404) 639-8272

PIC ID: 5348

PERFORMER ORGANIZATION: Abt Associates, Incorporated, Boston, MA

TITLE: Evaluation of ICD-10 for Morbidity Reporting in the United States

ABSTRACT: This report evaluates (1) whether the International Classification of Disease (ICD)-10 is a significant improvement over the ICD-9; (2) whether these improvements warrant its implementation in the United States; (3) the modifications needed to improve its use for morbidity applications; and (4) whether there are any codes or concepts in ICD-9, but not in ICD-10, that should be considered for inclusion. The Technical Advisory Panel and project staff conclude that there are many strengths in ICD-10 in comparison with ICD-9. Many of the chapters contain a wealth of useful expansion in detail for which diagnostic information is typically available and which represent important clinical distinctions. These expansions will facilitate future specificity in statistical tabulations of diseases and conditions, provide some additional means for severity coding, and provide for enhanced applicability of the classification to primary care reporting. The logic and detail of ICD-10 will also contribute to progress toward the computerization of medical records. Although a number of limitations in ICD-10 are identified, the recommended modifications will overcome most of them. Therefore, it is strongly recommended that the National Center for Health Statistics implement a revised version as soon as possible.

AGENCY SPONSOR: National Center for Health Statistics

FEDERAL CONTACT: Amy Blum

PHONE NUMBER: (301) 436-7050

PIC ID: 5503

PERFORMER ORGANIZATION: Center for Health Policy Studies, Columbia, MD

TITLE: Suicide Intervention and Prevention: Evaluation of Community-Based Programs in Three American Indian Communities

ABSTRACT: Suicide is the second leading cause of death for Native Americans aged 15 to 24. American Indian/Alaska Native young people are more likely to attempt suicide; to choose a highly lethal method, such as firearms or hanging; and to be successful at committing suicide. This report evaluates the local community-based suicide intervention and prevention programs of three geographically diverse American Indian tribes (the Jicarilla Apache Tribe, the Western Athabaskan Tribe, and the Northern Woodlands Tribe). In two tribes, suicide intervention efforts had been in place since 1989 and 1990, respectively; in the third, a suicide crisis in late 1992 precipitated community response. The report finds that (1) the local nature of the evaluation process opened itself to problems of diffuse authority and lack of focus on the evaluation process; (2) suicide prevention issues, tribal identity issues, and staffing issues took precedent over the actual evaluation activities; (3) a major recommendation arising from this study was to have process evaluation activities conducted by an external (non-community member) evaluator not supervised by local staff; (4) the Jicarilla Apache program represents a suicide prevention program in a very small, cohesive, and well-coordinated community response, which depends upon a staff model; (5) at the Western Athabaskan reservation, the prevention program used several special initiatives loosely coordinated with an existing Behavioral Health Services program using a staff model; (6) using a volunteer network, the Northern Woodlands tribe produced an excellent manual describing a grassroots community approach to suicide prevention; and (7) in all three tribes, attempted and successful suicide rates were reduced. The report concludes that the staff models used in the two small and centralized reservation communities were more successful than the diffuse, volunteer effort. (Final report: 120 pages.)

AGENCY SPONSOR: National Center for Injury Prevention and Control

FEDERAL CONTACT: Lloyd Potter

PHONE NUMBER: (770) 488-1557

PIC ID: 5504

PERFORMER ORGANIZATION: Indian Health Service, Rockville, MD

TITLE: Evaluation of the Fatality Assessment and Control Evaluation (FACE) Program: Phase II Final Report

ABSTRACT: The Fatality Assessment and Control Evaluation (FACE) program of the National Institute for Occupational Safety and Health seeks to prevent workplace fatalities by identifying and investigating fatal occupational injuries and disseminating prevention strategies to those who can intervene in similar situations. Publications based on information from fatality investigations identify occupational hazards and prevention measures, and are disseminated to persons who can influence work conditions and information availability in related occupations and work settings.

This report (1) describes the dissemination of five recent FACE documents (4 tear-sheet Alerts and one monograph), (2) compiles users' assessments of their usefulness, and (3) identifies examples of prevention efforts and other impacts attributable to the documents. The report finds that (1) publications are generally considered technically accurate and focused on significant occupational hazards by users; (2) workers participating in focus groups see job-related chronic illnesses and nonfatal injuries as more significant than the risk of fatality, and are less convinced of tear sheets' relevance to them; (3) workers consider the tear sheets' recommendations accurate and appropriate, but are concerned by the inclusion of "common sense" recommendations; (4) the tear sheets' terminology and generally high literacy level may limit their usefulness to their target audiences; (5) the publications are generally disseminated by distributing them directly or by modifying them for inclusion in other informational materials; and (6) information from the publications is used to improve worker training, identify existing hazards, and improve safety standards. The report concludes that the FACE publications are a useful source of authoritative occupational safety information. The report includes several recommendations. See also PIC ID No. 5506. (Final report: 30 pages, plus appendices.)

AGENCY SPONSOR: National Institute for Occupational Safety and Health

FEDERAL CONTACT: Theodore Pettit

PHONE NUMBER: (304) 285-5972

PIC ID: 5506.1

PERFORMER ORGANIZATION: Research Triangle Institute, Research Triangle Park, NC

TITLE: Assessment of the National Laboratory Training Network

ABSTRACT: The National Laboratory Training Network (NLTN) was established in 1989 to provide training in new methodologies and theoretical concepts for laboratory workers. This report assesses whether the NLTN has had a beneficial impact on public health by improving laboratory practice. Data were gathered through a series of surveys of local health department and physician office laboratory employees who participated in training, as well as comparison groups of those who did not. Other surveys of State training coordinators (STCs), laboratory inspectors, microbiology directors, microbiology program participants, and NLTN co-sponsors were also conducted. The report categorizes its findings in three groups, based upon the responses of (1) STCs, laboratory inspectors, and co-sponsors; (2) microbiologists; and (3) course participants and their comparison groups. The report finds that: (1) STCs felt supported in their jobs and used NLTN resources (88 percent were somewhat or very satisfied with NLTN services, and 90 percent were satisfied with programs); (2) laboratory inspectors often recommended training as a means to correct deficiencies, but only a small percentage used the NLTN as a training source; (3) co-sponsors rate relevance, quality, and currency of course content very highly; (4) microbiology directors rated NLTN fees, convenience, relevance, quality, currency of content, and pertinence of issues addressed favorably; (5) one-third of microbiology course participants instituted a change in practice as a result of training (80 percent rated quality and currency of content highly); and (6) course participants in the third group rated their training highly and most instituted changes or saw positive results subsequent to training. The report includes recommendations for further improving the NLTN. (Final report: 147 pages, appendices bound separately in Volume II.)

AGENCY SPONSOR: Public Health Practice Program Office

FEDERAL CONTACT: Eunice Rosner

PHONE NUMBER: (770) 488-4129

PIC ID: 5507

PERFORMER ORGANIZATION: Battelle Corporation, Arlington, VA

TITLE: Race Reporting on Birth Records by Multiracial Women: An Evaluation of Question Format Alternatives

ABSTRACT: This report examines the implications of changing the format of the race item on the birth record filled out for every birth in the United States at the location where the birth occurred. Procedures for filling out birth certificates vary from State to State and in some States, from hospital to hospital. To provide uniform information to the vital registration system, the Federal government, in cooperation with the States, developed the U.S. Standard Certificate of Live Birth that collects relevant information about the race and Hispanic origin of the parents, but not of the baby. This study (1) evaluates how women of multiracial and Hispanic backgrounds interpret the race question on the birth certificate; (2) evaluates experimental race questions, such as the use of a multiracial cue and the use of a "mark all that apply" format; and (3) learns more about the response processes that women use when answering questions about race. Nine States and the District of Columbia recruited 763 women to participate in this study. The study tests the effects of different questions on race, both in a mail and a followup telephone survey. The mother's self-reported race is compared with what she reported on the actual birth record. Further, for those residents with two children, the study compared consistency of race reporting between the last two birth records. The report concludes that (1) the number of women reporting more than one race increases when the race questions include a multiracial cue or "mark all that apply" format; (2) women who report more than one race are more likely to enter two specific races rather than a term like multiracial; (3) among the mothers whose parents who were two or more races, self-reports of race on the last two birth records are inconsistent 25 percent of the time; and (4) self-reports of race are different 40 percent of the time between the birth record and the standard version of the mail survey. (Final report: 48 pages, plus appendices.)

AGENCY SPONSOR: National Center for Health Statistics

FEDERAL CONTACT: Susan Schechter-Ryan

PHONE NUMBER: (301) 436-7111

PIC ID: 5918

PERFORMER ORGANIZATION: Westat, Incorporated, Rockville, MD

TITLE: Management and Operational Impacts of HIV Prevention Community Planning: Lessons Learned at Mid-Course and Implications for Future Activities

ABSTRACT: HIV Prevention Community Planning, implemented in January 1994, represented a major policy change in Centers for Disease Control and Prevention funding policy. Prior to that time, State, territorial, and local health department grantees with prevention funds were required to spend most of their cooperative agreement funds on counseling and testing services. A new emphasis on planning through HIV Prevention Community Planning requires grantees to set their own HIV prevention priorities locally through a process that emphasizes input from the affected communities and relevant technical experts. This project broadly evaluated the policy impact of this new planning process on grantees' HIV prevention programs as demonstrated by budgetary and operational indicators. This is an issue of national importance, as the HIV prevention community planning process affects over $200 million dollars in HIV prevention funds across the country. This first phase of a two-phase project ascertained how community planning is confirming, enhancing, and changing HIV prevention programs. A methodology for obtaining information on the indicators for a retrospective study was also developed and piloted during this phase of the study. The report finds that (1) a prospective and iterative approach to planning, implementation, and assessment is the most valuable approach; (2) in a prospective approach, the baseline is the set of circumstances extant at the beginning of the process, and site-specific goals and objectives are developed and tracked; and (3) more detailed guidance addressing these issues could help operationalize this kind of approach. Finally, the report finds that the indicators offering the most promise are (1) correspondence between the community plan, proposed activities, and undertaken activities; (2) quality assurance for implementation; and (3) capacity building and infrastructure development. (Final report: 28 pages, plus appendices.)

AGENCY SPONSOR: National Center for HIV, STD and TB Control

FEDERAL CONTACT: Carol Robinson

PHONE NUMBER: (404) 639-0952

PIC ID: 5927

PERFORMER ORGANIZATION: Macro International, Incorporated, Atlanta, GA

TITLE: CDC's Early Implementation of GPRA: A Case Study--July 1995 to November 1996

ABSTRACT: The Government Performance and Results Act of 1993 (GPRA) stresses the importance of government agencies' monitoring their results or outcomes in order to improve their management of associated processes and outputs. GPRA mandates that each agency produce a 5-year strategic plan and a 1-year performance plan by September 1997. This report offers a record of the experience of the Centers for Disease Control and Prevention in its early implementation of GPRA. It documents the process, summarizes key findings, and recommends strategies and approaches for the ongoing implementation of GPRA. The report finds that (1) implementation during the study phase did not proceed at the anticipated rate (instead of a fully developed strategic plan and pilot performance plans, most of the effort has concentrated on a strategic plan that is not yet drafted-little attention has been paid to the performance plans); (2) the role and responsibilities of the GPRA Implementation Team (GIT) evolved over the study period; (3) the GIT suffered from shifting membership, lack of continuity in meeting attendance, and an overall insufficient level of effort; (4) more effective and comprehensive planning and coordination was needed; and (5) despite the large amount of time and effort devoted to GPRA, the levels and types of commitment and resources fell short of what was needed. The report also concludes that insufficient momentum, commitment, and resources were a greater problem than ineffective planning; the process could have benefitted from greater executive staff involvement early on; and the process was often frustrating for participants. The report recommends that (1) organizational change efforts be unified under a high-level champion or team; (2) the Office of Program Planning and Evaluation's charge of implementing GPRA requirements be maintained; (3) the GIT be revamped; and (4) realistic budget be developed, requested, and executed.

AGENCY SPONSOR: Office of the Director

FEDERAL CONTACT: Wilma G. Johnson

PHONE NUMBER: (404) 639-7090

PIC ID: 5928

PERFORMER ORGANIZATION: Research Triangle Institute, Research Triangle Park, NC

TITLE: Collaborating with Managed Care Organizations for Mammography Screening and Rescreening: Guidance for NBCCEDP Grantees

ABSTRACT: Clinical trials show that mammography screening among women aged 50-69 years can reduce mortality from breast cancer by as much as 30 to 40 percent. Thus, more complete mammography screening coverage is needed throughout the Nation. This project was undertaken with a dual purpose: (1) to evaluate strategies used in the National Breast and Cervical Cancer Early Detection Program (NBCCEDP) to reach women 50 years and older for the early detection of breast cancer; and (2) to assess potential partnership opportunities between public health and managed care organizations to increase the coverage of mammography screening. Currently, all States, the 5 territories, and 25 American Indian tribes are funded by the NBCCEDP to establish and manage comprehensive breast and cervical cancer screening services for women who are minorities, low-income, and age 50 or older, a population that seeks mammography services less often than other women because of cost, knowledge, and other barriers. The report notes that the increased use of managed care organizations (MCOs) by State Medicaid programs offers new opportunities and incentives to build partnerships between NBCCEDP grantees and MCOs. The report finds that (1) because grantees give priority to women who are low-income, uninsured, racial minorities, or Native American, special attention must be paid to educating and building relationships with MCOs in outreach, case management, and followup; (2) both sectors will benefit from data partnerships and community collaborations; (3) most current MCO efforts in the area of breast and cervical cancer detection are data-driven and office-based; and (4) health plans target providers more than patients to increase their attention to the prevention effort. The report also addresses opportunities for collaboration, barriers to and facilitators of such collaboration, and strategies that programs might use to enhance screening efforts. See also PIC ID No. 6273.1.

AGENCY SPONSOR: National Center for Chronic Disease Prevention and Health Promotion

FEDERAL CONTACT: Nancy Nowak

PHONE NUMBER: (404) 488-4306

PIC ID: 6273

PERFORMER ORGANIZATION: Macro International, Incorporated, Atlanta, GA

TITLE: Reaching Women for Mammography Screening: Successful Strategies of National Breast and Cervical Cancer Early Detection Program (NBCCEDP) Grantees

ABSTRACT: This guide presents successful strategies used by grantees of the National Breast and Cervical Cancer Early Detection Program (NBCCEDP) to reach women in need of mammography screening-in particular low-income women, underserved and uninsured women, and women from communities of color. This information is presented to help grantees reach these marginalized populations by sharing strategies that their peer grantees have found successful. This information will also be useful for private sector and other organizations that are working to provide prevention services to hard-to-reach groups. Five categories of strategies are examined: inreach, public education, outreach, policies and procedures, and coalition and partnership development. While all of these strategies are different, several common themes underlie them all: (1) organizations must consider broad partnerships to reach marginalized populations who, because of their complicated lives, may not seek out care for themselves; (2) aggressive outreach may be needed to work with agencies and individuals who are most frequently in contact with these populations; (3) organizations should consider their infrastructure, not just outreach, to bring women in (health care organizations or health plans with a designated group of clients will find inreach more effective than outreach in the short term); (4) while the natural tendency is to focus on the relationship between the direct caregiver (especially the physician) and the patient, women are more likely to return for rescreening and to comply with followup regimens if a welcoming atmosphere is created by the entire office; and (5) the successful strategies show that there are diverse resources available for programs, but that innovation is needed when developing partnerships. See also PIC ID No. 6273. (Final report: 104 pages, plus appendices.)

AGENCY SPONSOR: National Center for Chronic Disease Prevention and Health Promotion

FEDERAL CONTACT: Nancy Nowak

PHONE NUMBER: (404) 488-4306

PIC ID: 6273.1

PERFORMER ORGANIZATION: Macro International, Incorporated, Atlanta, GA

TITLE: Evaluation Design for the Business Responds to AIDS (BRTA) Program

ABSTRACT: The Business Responds to AIDS program (BRTA) is a public-private initiative partnership undertaken by the Centers for Disease Control and Prevention's Division of HIV/AIDS Prevention to help businesses create and implement comprehensive HIV/AIDS workplace programs. To better understand how worksites are engaged in health promotion efforts, the BRTA Program conducted a national probability survey (the BRTA Benchmark Survey) to track progress since 1992. This project conducts a secondary analysis of the BRTA Benchmark Survey in order to provide recommendations about future evaluation designs that rely heavily on national communication channels. While much of the survey addressed issues related to HIV/AIDS, a number of questions were included that help track trends in worksite policies, screening programs, and health promotion activities. Results from this survey were compared to those gathered in a 1992 Office of Disease Prevention and Health Promotion survey of health promotion and disease prevention in the worksite. The report finds that (1) recent trends in worksite health promotion efforts seem to show increases in worksite policies, but no changes or decreases in classes or screening programs; (2) having a worksite policy may be less costly to implement than classes or screening programs (these policy interventions may also be more cost-effective than other types of interventions); and (3) the downward trend may result from some measurement bias introduced by asking questions about health promotion classes and screening programs only of those respondents to the BRTA Benchmark Survey who said they had had some health promotion program in the past 12 months. (Final report: 14 pages, plus appendices.)

AGENCY SPONSOR: National Center for Chronic Disease Prevention and Health Promotion

FEDERAL CONTACT: Cynthia Jorgensen

PHONE NUMBER: (404) 639-0952

PIC ID: 6277

PERFORMER ORGANIZATION: Research Triangle Institute, Research Triangle Park, NC

TITLE: Case Studies of the Prevention Marketing Initiatives (PMI) Local Demonstration Site Projects: Experiences During Planning and Transition Phases

ABSTRACT: The Prevention Marketing Initiative (PMI) is a social marketing program intended to influence behaviors that contribute to the sexual transmission of HIV and other sexually transmitted diseases among young people below 25 years of age. PMI includes among its components locally funded demonstration sites, which combine community participation with social marketing and behavioral science methodology and theory in order to design, implement, and sustain viable prevention programs. The PMI case study gathered in-depth information at these sites to describe the experiences of program participants over a 2-year period spanning two major project phases: planning and transition to implementation. The study highlights participants' reflections on what was done to foster achievement of PMI's goals, as well as what could be avoided in the future. The qualitative information gathered will be used to contextualize quantitative evaluation data collected as sites move through the implementation phase of PMI. The report finds that PMI had the following effect: (1) increased collaboration among community organizations and individuals on HIV prevention; (2) increased youth involvement in planning HIV prevention activities after some initial floundering in this area; (3) there is some weak evidence of increased support for HIV prevention programs within the community; and (4) increased participants' knowledge of, and sensitivity to, social marketing methodologies. The report also makes several recommendations on two levels: those that can be applied to new PMI sites and to other community-based ventures and those that pertain to PMI in particular. A,ong these recommendations are (1) define and become familiar with the community, (2) learn to manage issues and be realistic, (3) maintain diversity and enforce rules, (4) be clear with the lead agency, and (5) have the staff in place to provide needed technical and management services. (Final report: 204 pages, plus appendices.)

AGENCY SPONSOR: Office of the Director

FEDERAL CONTACT: Nancy Cheal

PHONE NUMBER: (404) 639-7095

PIC ID: 6337

PERFORMER ORGANIZATION: Battelle Corporation, Arlington, VA

TITLE: Evaluability Assessment of CDC-Funded Traumatic Brain Injury and Spinal Cord Injury Programs

ABSTRACT: In 1988, the Centers for Disease Control and Prevention (CDC) created the Disabilities Prevention Program (DPP) to reduce the incidence and severity of primary and secondary disabilities in the United States and to promote the independence and productivity of persons with disabilities. This report was commissioned to assess the feasibility of conducting an evaluation of the traumatic brain injury and spinal cord injury (TBI/SCI) programs funded through the DPP, the National Center for Environmental Health, and CDC. Interviews were conducted with Federal and State staff with a background or interest in DPP, external stakeholders, and with program staff in eight States. Interview data were used to prepare a logic model of State-level TBI/SCI programs, along with a set of measurable program components that match the activities and the objectives set forth in the logic model. The report finds that (1) there is a consensus among all stakeholders on the goals, activities, and objectives of the TBI/SCI capacity building programs' intended goals and on indicators that can be used to measure attainment; and (2) identifying the logic model and framework of measurable components for these programs offers many benefits. In order to strengthen CDC-State partnerships, the report recommends: (1) an appropriate balance in the partnership that allows States to be flexible but also provides parameters to ensure consistency; (2) the provision of regular, consistent feedback on State plans, activities, quarterly reports, and surveillance data; (3) grounding program design and decisions on valid data, whenever possible (in the absence of data, programs should proceed with a phased approach involving both the CDC and the States in the evolutionary learning process); and (4) the expansion of partnerships at the Federal, State, and community levels. (Final report: 33 pages plus appendices.)

AGENCY SPONSOR: National Center for Injury Prevention and Control

FEDERAL CONTACT: Doug Browne

PHONE NUMBER: (404) 488-7098

PIC ID: 6338

PERFORMER ORGANIZATION: Susan Toal, Atlanta, GA

TITLE: Community Context Study: Minors' Access to Tobacco

ABSTRACT: This project represents the first phase of a national study designed to profile the relationship between youths' access to tobacco and the contextual variables that define communities' policies, attitudes, and activities regarding youth and tobacco. The results of the study have implications for local, State, and Federal policies and programs. The basic conceptual model postulates that community forces will initiate and/or enhance local enforcement efforts. Enforcement will affect vendor behavior, which will then restrict minors' access to-and thereby, use of-tobacco. To address the questions raised by the conceptual model, data collection and analysis for this project were conducted in three parts: (1) telephone interviews with tobacco vendors and community leaders, and purchase attempts in stores in 20 communities; (2) in-person interviews with community leaders in ten case-study communities; and (3) analysis of teen tobacco use data from the 1995 Youth Risk Behavior Study (YRBS), collected by CDC for the same 20 communities. The report finds that (1) adolescents have little difficulty buying cigarettes in retail outlets; (2) even in restrictive communities, some stores would sell tobacco to minors; (3) the most important deterrent of sales to minors is asking for proof of age; (4) the presence of tobacco sales warning signs is associated with a lower sale rate; (5) community support for tobacco control promotes merchant compliance with the law; and (6) even a perception of enforcement acts as a strong deterrent to selling tobacco to minors. The report recommends that (1) the study should be expanded as a followup to the 1997 YRBS in sampled communities, (2) a question about the location of sources of retail purchase should be added to the YRBS, and (3) the expanded community context study should be fielded as soon as possible. (Executive summary: 7 pages; final report: variously paginated, plus appendices.)

AGENCY SPONSOR: National Center for Chronic Disease Prevention and Health Promotion

FEDERAL CONTACT: Gary A. Giovino

PHONE NUMBER: (770) 488-5703

PIC ID: 6474

PERFORMER ORGANIZATION: Battelle Corporation, Arlington, VA

TITLE: Directions for Research on the National Immunization Survey: Final Report

ABSTRACT: The National Immunization Survey (NIS) measures the immunization levels among young children in each State and in large metropolitan areas on a quarterly basis. This survey recently came under review, and questions about the cost of the survey and the utility of the data were raised. This report reviews the survey and develops a plan for determining whether less expensive methods could be used and still yield a statistically sound survey that meets the needs of the National Immunization Program. The report finds that (1) the NIS is a well-designed response to a very difficult set of goals posed by the Centers for Disease Control and Prevention; (2) dramatic cost reductions (i.e., 30 percent or larger) could not be achieved without major changes in survey objectives or quality; (3) there is greater potential for cost savings in the area of data collection than in analysis, since the volume of the survey makes refined systems for data collection economically feasible; (4) the high data analysis costs for the survey are well justified; (5) the survey meets the need for monitoring trends at the national level and can also be used to identify Immunization Action Plan (IAP) areas with either very poor or outstanding immunization levels; and (6) the survey can differentiate the immunization rates for reasonably small demographic groups annually, and can be used to monitor immunization trends at the IAP level, but only on an annual or biennial basis. The report concludes that the relative emphasis on IAP-level statistics and on national statistics should perhaps be adjusted-in the justification of the survey, in the design of the household interview, and in the choice of statistics to publish. It recommends that the schedule for publication of IAP-level statistics be cut back to an annual schedule and that new publications on national statistics for domains of interest be developed to fill the publication schedule.

AGENCY SPONSOR: National Immunization Program

FEDERAL CONTACT: Jacquelyn Williams

PHONE NUMBER: (301) 443-4493

PIC ID: 6477

PERFORMER ORGANIZATION: Research Triangle Institute, Research Triangle Park, NC

TITLE: In Her Own Words: A Focus Group Study of Risk and Protective Factors in Intimate Partner Violence

ABSTRACT: This study was designed to provide insights into how African-American and Caucasian women in urban and rural settings perceive the factors that placed them at risk for an abusive relationship, as well as the factors that protected them from further violence during and after the relationship. The model used as the conceptual framework for the study (Prochaska's and DiClemente's Transtheoretical Model) depicts a sequence of five stages that culminate in lasting behavior change. Twenty-two focus groups were conducted in urban and rural areas in Georgia, Illinois, Pennsylvania, and Oregon. One hundred and sixty-eight women participated in the study, ranging in age from 18 to 60; all had been out of their abusive relationships from 6 months to 24 years. The report finds that (1) the long-term effects of emotional and physical abuse are worse than the actual violence and resulting injuries themselves; (2) a wide range of psychologically and physically abusive acts were described by focus group participants, including abuse not only of themselves but also of children and of pets; (3) most focus group participants concluded that anything could be a trigger to violence, and that this realization was a significant contributor to the terror, fear, and desperation they felt in their relationships; (4) participants agreed that a number of problem behaviors, such as a history of violence at home, extreme charm, possessiveness, and explosive rage, are early warning signs of abuse; and (5) women describe the decision to leave the abusive relationship as the result of a breaking point. The report concludes that this study's findings should be validated in a national telephone survey of women's risk and protective factors for intimate partner violence. (Final report: 135 pages.)

AGENCY SPONSOR: National Center for Injury Prevention and Control

FEDERAL CONTACT: Lynn Short

PHONE NUMBER: (770) 488-4285

PIC ID: 6515

PERFORMER ORGANIZATION: Macro International, Incorporated, Atlanta, GA

TITLE: Linking Research and Public Health Practice: A Review of CDC's Program of Centers for Research and Demonstration of Health Promotion and Disease Prevention

ABSTRACT: In order to advance research in health promotion and disease prevention, the Centers for Disease Control and Prevention (CDC) established a program of university-based Centers for Research and Demonstration of Health Promotion and Disease Prevention. The prevention research center (PRC) program began in 1986 with funding to three universities; in 1996, there were 13 PRCs. This report examines the extent to which the PRC program is providing the public health community with workable strategies to address major public health problems in disease prevention and health promotion. The report finds that (1) by forging links with academia, CDC has created a gateway for access to a cadre of well-trained, university-based researchers who could serve to inform and collaborate with the agency and the public health community regarding health promotion and disease prevention; (2) the PRC has made substantial progress in advancing the scientific infrastructure in support of disease and health promotion policy, programs, and practices; (3) each of the centers has made some contributions toward one or more of the goals of the program (research, dissemination, and developing connections with the community and public health practitioners); (4) there are substantial differences among the PRCs in the kinds of activities they have undertaken and the success realized (only a few have made substantial progress in all areas); and (5) CDC's management of the program has been creative in the face of limited resources relative to its mandate-dogged in pursuing the mandate over a 10-year period in a bureaucratic environment that was not created or structured for the management of university-based research programs, and skilled in enhancing a sense of community and networking among the funded centers in a time of disappointing funding levels. (Final report: 67 pages, plus appendices.)

AGENCY SPONSOR: National Center for Chronic Disease Prevention and Health Promotion

FEDERAL CONTACT: Patricia Riley

PHONE NUMBER: (770) 488-5277

PIC ID: 6535

PERFORMER ORGANIZATION: National Academy of Sciences, Washington, DC

TITLE: Syphilis in the South: A Case Study Assessment in Eight Southern Communities

ABSTRACT: This two-volume report presents the results of a comparative case study of local-level syphilis prevention efforts that are conducted as part of sexually transmitted disease (STD) prevention programs supported by the Centers for Disease Control and Prevention (CDC). The case studies were conducted in eight communities in four Southern States: Alabama, Mississippi, South Carolina, and Tennessee. The project investigated past and current responses of the public and private health sectors to a syphilis epidemic that occurred in the southern United States between 1990 and 1992. Study findings were developed around four key questions: (1) who is at greatest risk of syphilis infection and transmission, (2) what institutions are best able to reach those at greatest risk, (3) what are the barriers to syphilis prevention and control in communities, and (4) what innovative ways have been found to transcend these barriers? The report finds that (1) interviewees perceived African Americans to be the demographic group most at risk for infection and transmission of syphilis, especially those who exchanged sex for drugs, homeless people, and incarcerated individuals; (2) adolescent high-risk sexual behavior was of concern as a possible precursor to infection once practitioners reached their 20s; (3) local health departments are the only community organizations focusing directly on syphilis and other STD control and prevention; (4) public health agencies assigned priority to disease control and engaged in few agency-based prevention activities, and schools were limited in their ability to be venues for STD prevention messages due to the local restrictions on sexually explicit materials in health education curricula; and (5) barriers to those at greatest risk of syphilis infection are cultural, programmatic, and political. Recommendations are offered in a manner designed to allow practical implementation. (Vol. I: 121 pages, plus appendices; Vol. II: 100 pages.)

AGENCY SPONSOR: National Center for HIV, STD and TB Prevention

FEDERAL CONTACT: Janelle Dixon

PHONE NUMBER: (404) 639-8344

PIC ID: 6579

PERFORMER ORGANIZATION: Battelle Corporation, Arlington, VA

TITLE: Evaluation of the Effectiveness of CDC Surveillance for Drug-Resistant Streptococcus pneumoniae (Baltimore, MD)

ABSTRACT: The three aims of this project were to (1) establish population-based surveillance for invasive Streptococcus pneumoniae infection; (2) evaluate the validity of the current Centers for Disease Control and Prevention (CDC) sentinel surveillance system; and (3) identify the optimal methods for determining antimicrobial resistance patterns in the United States. Active, population-based surveillance for invasive pneumococcal infection was established in all hospitals in the Baltimore metropolitan area over a 2-year period. Various additional data collection methods were employed, including bimonthly contact/followup to participating sites to obtain cases not reported spontaneously, yearly laboratory audits to identify cases not reported through active surveillance, and retrospective chart reviews. During the 2-year project period, 1,415 cases were reported: 685 in 1995 and 730 in 1996. Additionally, the contractor conducted two nasopharyngeal culture surveys of 300 children each in 5 pediatric clinics within the Baltimore metropolitan area. Sputum surveys were also conducted in three large Baltimore area hospitals. All data and isolates from the project have been sent to CDC. See also PIC ID Nos. 6580.1 and 6580.2. (Final report: 4 pages.)

AGENCY SPONSOR: National Center for Infectious Diseases

FEDERAL CONTACT: Anne Schuchat

PHONE NUMBER: (404) 639-2215

PIC ID: 6580

PERFORMER ORGANIZATION: Johns Hopkins University, Baltimore, MD

TITLE: Evaluation of the Effectiveness of CDC Surveillance for Drug-Resistant Streptococcus pneumoniae (San Antonio, TX)

ABSTRACT: Five major activities were conducted as part of this study to determine the effectiveness of the Centers for Disease Control and Prevention's (CDC's) current surveillance system for drug-resistant Streptococcus pneumoniae. These activities included (1) active population-based surveillance of invasive pneumococcal isolates in San Antonio and Bexar County, Texas for a period of 2 years; (2) microbiologic studies in which all isolates were submitted to a central laboratory for final identification, storage, and cataloging, and for eventual additional susceptibility testing; (3) microbiologic studies, in which 221 sputum isolates were collected and identified for Streptococcus pneumoniae and antibiotic resistance; (4) period prevalence surveys of non-invasive respiratory isolates; and (5) a case-control study among patients with invasive pneumococcal disease. Additionally, data were entered into appropriate data bases for further analysis at CDC. For example, invasive isolate data were entered into EpiInfo and microbiologic study data were entered into the Public Health Laboratory Information System. Results were disseminated to the community via hospital reports, local meetings, national and State meetings, manuscripts, and newsletters. See also PIC ID Nos. 6580 and 6580.2. (Final report: variously paginated.)

AGENCY SPONSOR: National Center for Infectious Diseases

FEDERAL CONTACT: Anne Schuchat

PHONE NUMBER: (404) 639-2215

PIC ID: 6580.1

PERFORMER ORGANIZATION: University of Texas, Health Science Center, San Antonio, TX

TITLE: Assessment of the Programmatic Impact of One-Percent Evaluation Studies

ABSTRACT: This three-volume report provides a systematic evaluation of the programmatic impact of 1-percent evaluation studies carried out by the Centers for Disease Control and Prevention (CDC) between 1990 and 1995. The study was also designed to elicit information about what the Office of Program Planning and Evaluation could do to enhance the impact of 1-percent evaluation studies. The study resulted in findings in three major areas: (1) the perception at CDC of evaluation in general, (2) the perception at CDC of the 1-percent process in particular, and (3) the programmatic impact to date of 1-percent evaluation studies at CDC. Among the findings were that: (1) although pressure for program accountability is mounting, evaluation still engenders anxiety among some CDC program staff-although these attitudes are changing slowly; (2) program staff are concerned that the 1-percent evaluation process in particular is unpredictable-staff do not know from one year to the next how much 1-percent funding they will receive, and the time allowed between the issuance of 1-percent guidance and the deadline for submissions is perceived as too short; (3) despite numerous obstacles to implementation, nearly 70 percent of the recommendations developed in the evaluation studies examined for this report have been considered for implementation and about 50 percent have been implemented in some form; and (4) CDC evaluations have produced programmatic impact leading to changes in practices and procedures, activities, responsibilities, changes in full-time equivalents, budgets, organizational and administrative structures, and policy. Specific recommendations are made in the report and are arrayed according to project phases (study selection, design, implementation, recommendation, and dissemination stages). See also PIC ID No. 6581.1. (Final report: 69 pages, plus appendices A-E; Appendix F separately bound; Appendices G, H, and I separately bound.)

AGENCY SPONSOR: Office of the Director

FEDERAL CONTACT: Wilma G. Johnson

PHONE NUMBER: (404) 639-7090

PIC ID: 6581

PERFORMER ORGANIZATION: Battelle Corporation, Arlington, VA

TITLE: Directory of Evaluations Administered by the CDC Office of Program Planning and Evaluation 1990-1995

ABSTRACT: This directory is one of the products of a study conducted for the Office of Program Planning and Evaluation (OPPE) of the Centers for Disease Control and Prevention (CDC) entitled "Assessment of the Programmatic Impact of (CDC) One-Percent Evaluation Studies." During a process of reviewing nearly 50 CDC evaluations completed between 1990 and 1995 and about 40 additional studies in various stages of completion, it became apparent that these studies were important to CDC program staff in particular and to the field of public health in general. Many common themes ran through these evaluations: (1) the difficulties of monitoring the performance of public health programs; (2) the growing importance of community outreach and mobilization; (3) the evolving role of health communications in CDC's work; and (4) concerns over the impact of managed care on the public health sector. This directory includes both completed and in-process studies. Project summaries reflect a project's reported status as of August 1995. The volume is arranged according to the study ID number assigned to each project as it is initiated through OPPE. See also PIC ID No. 6581. (Final report: variously paginated.)

AGENCY SPONSOR: Office of the Director

FEDERAL CONTACT: Nancy Cheal

PHONE NUMBER: (404) 639-7095

PIC ID: 6581.1

PERFORMER ORGANIZATION: Battelle Corporation, Arlington, VA

TOP OF DOCUMENT

FOOD AND DRUG ADMINISTRATION

Contents

Report of the Prescription Drug User Fee Act (PDUFA) of 1992-Fiscal Year 1997 Report to Congress

Economic Impact Analysis of Regulations Under the Mammography Quality Standards Act of 1992

Mammography Quality Standards Act: Second Annual Government Entity Declaration Program Audit

Product Review Performance Report

Evaluation of Device Performance in the Determination of Substantial Equivalence

Cost of Restrictions on Gulf Oyster Harvesting for Control of Vibrio vulnificus-Caused Disease

Health Transfers: An Application of Health-Health Analysis to Assess Food Safety Regulation

Preliminary Investigation into the Morbidity and Mortality Associated with the Consumption of Fruit and Vegetable Juices

TITLE: Report of the Prescription Drug User Fee Act (PDUFA) of 1992-Fiscal Year 1997 Report to Congress

ABSTRACT: Fiscal year 1997 (FY 97) marks the fifth and final year of the implementation of the Prescription Drug User Fee Act (PDUFA) of 1992, which authorizes the collection of fees from the pharmaceutical industry in order to facilitate the Food and Drug Administration's (FDA's) timely review of human drug applications. The Act mandates that FDA prepare yearly reports for each fiscal year for which fees are collected. The fifth annual report, prepared in response to that mandate, covers FY 97 and reports on how well the FDA has accomplished the performance goals it established for FY 97. These performance-related goals were jointly established by the FDA and the pharmaceutical industry and fall into three main categories: (1) elimination of overdue backlogs, (2) building excellence into the review process, and (3) achievement of high performance. Seven goals were in effect for FY 97. The findings in the report show that the successes reported last year have been confirmed and surpassed. The report finds that (1) PDUFA has resulted in more and better applications filed, which can be accepted immediately and reviewed and approved more quickly; (2) nearly all applications now go directly into the review process (due to the increased quality of the applications, very few are refused); (3) there are more approvals on first review, which demonstrates that submission quality is a key factor in achieving timely reviews; and (4) the ultimate approval time for applications in the PDUFA years has dramatically decreased. While only a preliminary performance assessment of FY 97 is possible at this time, it is clear that the Act's stringent review goals have assured that new pharmaceutical products are getting onto the market at an unprecedented rate without compromising safety. See also PIC ID Nos. 6079 and 6079.1. (Final report: 10 pages, plus appendices.)

AGENCY SPONSOR: Office of Planning and Evaluation

FEDERAL CONTACT: Dennis Hill

PHONE NUMBER: (301) 827-5255

PIC ID: 6079.2

PERFORMER ORGANIZATION: Office of Planning & Evaluation, Food and Drug Administration, Washington, DC

TITLE: Economic Impact Analysis of Regulations Under the Mammography Quality Standards Act of 1992

ABSTRACT: The Food and Drug Administration (FDA) is charged with ensuring that accessibility to quality mammography services is maintained for all facilities performing mammographies in accordance with the Mammography Quality Standards Act of 1992 (MQSA). Under this statute, the FDA is required to certify and annually inspect over 10,000 mammography facilities. Prior to the passage of the MQSA, FDA's oversight responsibility was limited to approving new mammographic processors, x-rays, and tubing. However, MQSA requires that facilities within its jurisdiction meet standards developed by the agency to ensure a reasonable level of quality in order to achieve certification by the agency. FDA is concerned that recent trends by third-party payers to attract payments for mammography services have severely limited the ability of facilities to absorb any cost increases caused by the quality standards. These cost increases, as well as excess capacity, may cause low-volume facilities to leave the industry. This project addresses these problems in two ways. First, it measures the effect of standards on patient access to mammography, especially in rural areas; second, it develops a cost model that will be used to predict the impact of proposed standards on the costs of providing mammography. The contractor reviewed records from over 10,000 facilities that had applied for accreditation and over 500 that had ceased operations. In the Spring of 1996, the contractor drafted the results of a cost-benefit analysis, which FDA managers will use to develop a performance-oriented regulatory policy that will be coordinated with existing customers.

AGENCY SPONSOR: Office of Planning and Evaluation

FEDERAL CONTACT: Steve Tucker

PHONE NUMBER: (301) 827-5340

PIC ID: 6080

PERFORMER ORGANIZATION: Eastern Research Group, Incorporated, Arlington, VA

TITLE: Mammography Quality Standards Act: Second Annual Government Entity Declaration Program Audit

ABSTRACT: The Food and Drug Administration's (FDA's) Division of Mammography Quality and Radiation Programs requested that the Division of Planning, Analysis, and Finance conduct an audit of the Mammography Quality Standards Act (MQSA) Government Entity (GE) Declaration Program. There are two main goals of the audit: (1) to notify facilities that they will be held accountable for their self-certification of fee exemption, and (2) to determine the rate of compliance with the program. A random sample of 90 facilities out of the 989 claiming GE status was selected for examination, and a random sample of 20 (out of 72) new GE facilities was also examined. The audit revealed that (1) only 3 of the 90 GE facilities had claimed GE status incorrectly, and only 1 of the 20 new GE facilities had claimed incorrect status; and (2) the 3 GE facilities (as well as the one new GE facility) that did not meet FDA's fee-exempt status claimed that exemption based on funding from the Centers for Disease Control and Prevention (CDC) Breast and Cervical Cancer Prevention Program, which is the least common way to claim fee exemption. Based on these results, the audit concludes that the GE Declaration program appears to be quite effective. It also suggests several recommendations based on the findings of the audit. These include that: (1) the results should be published to allow the impact of the audit to reach many, if not all, the facilities, to discourage any future misrepresentations, and to demonstrate that FDA will exercise its authority; (2) the sample size of future audits should be smaller and should contain a larger portion of those facilities claiming fee exemption based solely on funding from the CDC; and (3) the MQSA contractor in charge of billing should continue to thoroughly examine each GE Declaration form to ensure that it identifies the body operating the facility. (Final report: 3 pages, plus appendices.)

AGENCY SPONSOR: Center for Devices and Radiological Health

FEDERAL CONTACT: Stephanie Wilkinson

PHONE NUMBER: (301) 594-1284

PIC ID: 6730

PERFORMER ORGANIZATION: Center for Devices and Radiological Health, Rockville, MD

TITLE: Product Review Performance Report

ABSTRACT: This is the first annual report analyzing the Food and Drug Administration's (FDA's) performance in reviewing applications for human drugs, biological products, medical devices, food and color additives, and veterinary drugs. In the past, FDA has measured the length of review using measures that varied for different product categories, making it difficult to provide an agency-wide assessment of the FDA's performance. Responding to legislation such as the Prescription Drug User Fee Act of 1992 (PDUFA), as well as to requirements under the Federal Food, Drug, and Cosmetic Act, the FDA has developed clear performance goals for reviewing new product applications. Setting these goals has provided a valuable management tool for identifying performance expectations and assessing achievements, and for meeting the requirements of the Government Performance and Results Act of 1993. FDA has also developed performance measures and reporting conventions that can be used to report on product review performance consistently across product categories. The report looks at four different measures of the FDA's performance: (1) how often FDA is meeting statutory and nonstatutory time frames for review of each category of products, (2) the median length of time to approval for each product category, (3) the total number of products approved in each category, and (4) the number of applications for which an action was overdue at the end of the fiscal year (FY). It provides FDA's on-time performance for FY 1993, 1994, and 1995 and performance data for the other three categories for FY 1993 to 1996. The report finds that (1) for most categories of products, FDA has improved its performance since FY 1993; (2) the greatest improvements in performance have occurred for product reviews covered by PDUFA; (3) significant progress has been made in reviewing applications for medical devices; and (4) on-time performance for reviews of non-PDUFA applications should be improved.

AGENCY SPONSOR: Office of Planning and Evaluation

FEDERAL CONTACT: Paul Coppinger

PHONE NUMBER: (301) 827-5292

PIC ID: 6731

PERFORMER ORGANIZATION: Food and Drug Administration, Rockville, MD

TITLE: Evaluation of Device Performance in the Determination of Substantial Equivalence

ABSTRACT: Many diagnostic medical devices that reach the market are based on diagnostic agreement with some device already on the market that targets the same attribute. Agreement is often assessed by side-by-side testing of the same subjects or specimens, even if the true disease status is unknown. If each new test is compared with the most recent predicate, what would be the effect on device performance over time? Would it deteriorate, remain the same, or improve? This analysis attempts to address these questions using statistical measures of sensitivity, specificity, and predictive values. A Monto Carlo simulation was used to study this question of "predicate (pre-Amendments or reclassified post-Amendments) creep." Contingency tables were randomly generated using a range of conditional probabilities and prevalence representing screening and confirmatory test data. When a new test was found to be equivalent to the current predicate, it became the new predicate. After 50 new predicates were found, a forecast procedure was applied to the sequence of 50 performance parameters in order to evaluate the long-term trend.

AGENCY SPONSOR: Center for Devices and Radiological Health

FEDERAL CONTACT: John Dawson

PHONE NUMBER: (301) 827-0201

PIC ID: 6734

PERFORMER ORGANIZATION: Center for Devices and Radiological Health, Rockville, MD

TITLE: Cost of Restrictions on Gulf Oyster Harvesting for Control of Vibrio vulnificus-Caused Disease

ABSTRACT: This study for the Food and Drug Administration (FDA) estimates the costs of a seasonal harvesting restriction on Gulf of Mexico oysters and a seasonal marketing restriction on the sale of Gulf oysters for raw consumption. These measures would control oyster-related illnesses in humans known to be caused by the bacterial species Vibrio vulnificus, which naturally occurs in estuarine waters and is a normal flora for oysters. During this study, the 1995 Interstate Shellfish Sanitation Conference (ISSC) adopted a Vibrio vulnificus interim control plan for oysters and the report compares the costs of complying with the ISSC plan with FDA's suggested alternatives. The ISSC interim control plan (1) states that oysters must be under ambient refrigeration at 45 degrees Fahrenheit within a specified time if they are harvested from State waters that were the confirmed source for two or more Vibrio vulnificus illnesses; (2) establishes action levels for growing areas within affected States; and (3) varies the time-to-refrigeration requirement according to the average monthly maximum water temperature. The report estimates the economic impact of alternative remedies for controlling Vibrio vulnificus-related illnesses and concludes that (1) the in-shell marketing restriction might have imposed large economic impacts on the Gulf oyster industry and other related sectors of the economy; (2) the ISSC time and temperature controls would have smaller aggregate economic impacts than the marketing restriction option but the distributional effects of this policy should be considered; and (3) the ISSC impacts are sensitive to specific assumptions about enforcement of the rule and types of responses, engendered by the regulated entities. (Final report: variously paginated.)

AGENCY SPONSOR: Center for Food Safety and Applied Nutrition

FEDERAL CONTACT: Richard Williams

PHONE NUMBER: (202) 205-5667

PIC ID: 6856

PERFORMER ORGANIZATION: Research Triangle Institute, Research Triangle Park, NC

TITLE: Health Transfers: An Application of Health-Health Analysis to Assess Food Safety Regulation

ABSTRACT: Public health policy decisions usually focus on reducing a target risk. However, complying with regulations to reduce a risk is expensive. Because income influences health, Health-Health Analysis (HHA) was developed to compare the targeted reduction in mortality to the induced mortality from income losses. This study applies HHA to assess a potential Gulf oyster harvesting ban to control mortality from the consumption of oysters contaminated by Vibrio vulnificus. The report estimates the induced deaths using Anderson et al.'s estimate of economic losses and Keeney's estimate of the functional relations between income and mortality. The report shows that while a 7-month Gulf oyster harvesting ban might eliminate the estimated 17 oyster-related deaths annually, it may also cause as many as 12 income-loss-related deaths. The induced deaths occur principally among white males and black females. The report concludes that (1) HHA may give decisionmakers reason to pause considering the high costs of a harvesting ban and that consumption of raw oysters is completely voluntary; and (2) this illustration shows that HHA serves as a useful policy analysis tool. (Final report: 26 pages.)

AGENCY SPONSOR: Center for Food Safety and Applied Nutrition

FEDERAL CONTACT: Richard Williams

PHONE NUMBER: (202) 205-5667

PIC ID: 6857

PERFORMER ORGANIZATION: Center for Food Safety and Applied Nutrition, Washington, DC

TITLE: Preliminary Investigation into the Morbidity and Mortality Associated with the Consumption of Fruit and Vegetable Juices

ABSTRACT: Americans consumed 2.3 billion gallons of the major fruit and vegetable juices in 1995. Recent outbreaks of illnesses associated with the consumption of juices demonstrate their potential as a human health hazard. As part of the cost-benefit analysis for both the Hazard Analysis and Critical Control Point (HACCP) and labeling rules associated with fruit and vegetable juices, the Center for Food Safety and Applied Nutrition's working group was asked to investigate the morbidity and mortality associated with human consumption of juice drinks. The standard procedure for estimating human health benefits is to (1) estimate the baseline numbers of illnesses and deaths associated with the compound to be controlled, (2) estimate the likely reductions in those illnesses and deaths associated with various proposed control options, and (3) estimate the values associated with the reduced illnesses and deaths. This report estimates the baseline numbers of illnesses and deaths likely to be associated with the consumption of juice products. The study (1) describes juice products, (2) estimates juice consumption, (3) discusses production methods, (4) explains how hazards might be introduced into juice, (5) discusses evidence on illness from consuming juices, (6) describes human health effects caused by selected microbial pathogens, and (7) discusses the physical and chemical hazards associated with juices. Juices accounted for 447 laboratory-confirmed cases of illness associated with microbial pathogens, according to Centers for Disease Control and Prevention outbreak data and U.S. Food and Drug Administration recall data from 1993 through 1996. The other hazards, mostly physical and chemical, that have been found in juices have been sporadic and associated with fewer cases than the microbial pathogens. (Final report: 65 pages, plus appendices.)

AGENCY SPONSOR: Center for Food Safety and Applied Nutrition

FEDERAL CONTACT: Richard Williams

PHONE NUMBER: (202) 205-5667

PIC ID: 6858

PERFORMER ORGANIZATION: Center for Food Safety and Applied Nutrition, Washington, DC

TOP OF DOCUMENT

HEALTH CARE FINANCING ADMINISTRATION

Contents

Limited-Service Hospital Pioneers: Challenges and Successes of the Essential Access Community Hospital/Rural Primary Care Hospital (EACH-RPCH) Program and Medical Assistance Facility (MAF) Demonstration

Comparative Study of the Use of EPSDT and Other Preventive and Curative Health Care Services by Children Enrolled in Medicaid: Final Project Synthesis Report

Costs and Consequences of Case Management for Medicare Beneficiaries: Final Report

Evaluation of the Utah Prepaid Mental Health Plan

Monitoring and Evaluation of the Medicare Cataract Surgery Alternate Payment Demonstration: Final Evaluation Report

Evaluation of the Demonstration to Improve Access to Care for Pregnant Substance Abusers

Evaluation of the Medicaid Uninsured Demonstrations

Access to Physicians' Services for Vulnerable Beneficiaries: Impact of the Medicare Fee Schedule

A Study of the Appropriateness, Process of Care, and Outcomes of Medical Care for Medicaid Patients

An Evaluation of Implementation of the Medicaid Community Supported Living Arrangements (CLSA) Program in Eight States

Evaluation of the Community Nursing Organization Demonstration Interim Evaluation Report

Evaluation of the Effectiveness of the Operation Restore Trust Demonstrations

The Impact of Report Cards on Employees: A Natural Experiment

Consequences of Paying Medicare HMOs and Health Care Prepayment Plans (HCPPs)

Toward a Prospective Payment System for Outpatient Services: Implementation for Outpatient Services-- Implementation of APGs by State Medicaid Agencies and Private Payers

Payment of Pharmacists for Cognitive Services: Results of the Washington State C.A.R.E. Demonstration Project

Assessing the Viability of All-Payer Systems for Inpatient Hospital Services

Second Update of the Geographic Practice Cost Index: Final Report

Validation of Nursing Home Quality Indicators Study

1995 Influenza Immunizations Paid for by Medicare: State and County Rates

Impact of the Medicare Fee Schedule on Teaching Physicians

TITLE: Limited-Service Hospital Pioneers: Challenges and Successes of the Essential Access Community Hospital/Rural Primary Care Hospital (EACH-RPCH) Program and Medical Assistance Facility (MAF) Demonstration

ABSTRACT: Small communities in rural America are struggling to maintain health care resources in the face of increasing competition and a dwindling population and economic base. In response to the threatened survival of small rural hospitals, the Health Care Financing Administration has supported two initiatives to introduce innovative limited-service hospitals: the Essential Access Community Hospital/Rural Primary Care Hospital (EACH-RPCH) program, implemented in seven States; and the Medical Assistance Facility (MAF) demonstration in Montana. Both establish a new Medicare provider that is exempt from specific key requirements of participation, and both offer enhanced reimbursement. Other States have looked to these two programs as possible models for their rural communities. This report documents the evaluation of the EACH-RPCH program from its start in October 1991 through September 1994. The report assesses the implementation process, initial operating experience, and potential impact of the programs on access and cost as related to limited hospital service. The introductory chapter reviews the background of the EACH-RPCH and MAF programs, discusses the evaluation issues to be examined, and summarizes the evaluation methodology and data sources. Chapters II and III examine the development of networks and the shift to limited service licensure under the EACH-RPCH program. Chapter IV reviews the implementation and operation of the MAF demonstration. Chapters V and VI focus on the two programs' impact on access to care and mechanisms for quality assurance, respectively. The report concludes with Chapter VII, which assesses the impact of both programs on facility finances and the Medicare program. (Final report: 260 pages, plus appendices.)

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Sheldon Weisgrau

PHONE NUMBER: (410) 786-6675

PIC ID: 5200.1

PERFORMER ORGANIZATION: Mathematica Policy Research, Incorporated, Plainsboro, NJ

TITLE: Comparative Study of the Use of EPSDT and Other Preventive and Curative Health Care Services by Children Enrolled in Medicaid: Final Project Synthesis Report

ABSTRACT: This report includes (1) a description of the Medicaid program in Michigan, Georgia, Tennessee, and California and these States' responses to the Omnibus Budget Reconciliation Act of 1989 provisions relating to children's and Medicaid coverage and on the Early Periodic Screening, Diagnosis, and Treatment (EPSDT) program; (2) results of analyses of the impact of these responses on participating providers and enrolled children; and (3) an assessment of the national context through a review of the national survey data analyses. It finds that (1) State efforts to increase provider participation in Medicaid and EPSDT included recruitment activities, training in billing, and increased fees; (2) the States' efforts were successful in increasing provider participation and average caseloads among all types of providers, except for dentists in Michigan, which declined 10 percent from 1989 to 1992; (3) these increases may not have been adequate to meet the increased demand from a growing caseload; (4) the ratios of Medicaid children to participating providers and the percentages of counties with provider shortages rose from 1989 to 1992 in all four study States; (5) counting all EPSDT and all other well-child visits paid for by Medicaid, only 42 to 54 percent of Medicaid children were recommended to have preventive care visits, and of these, only 36 to 59 percent made the recommended visits; (6) compared with other low-income children, Medicaid coverage increased service use and improved access to illness-related care; and (7) Medicaid children had very low completion rates for age-appropriate immunizations, from 48 percent in Michigan to 61 percent in Georgia in 1992. The report concludes that State efforts to expand the EPSDT provider base and to enhance outreach had significant impacts on the use of preventive, diagnostic, and treatment services. See also PIC ID Nos. 6236-6236.6 and 6236.8-6236.A. (Final report: 49 pages.)

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Feather Davis

PHONE NUMBER: (410) 786-6590

PIC ID: 6236.7

PERFORMER ORGANIZATION: MEDSTAT Group, Cambridge, MA

TITLE: Costs and Consequences of Case Management for Medicare Beneficiaries: Final Report

ABSTRACT: This report studies the appropriateness of providing case management services to beneficiaries with catastrophic illnesses and high medical costs. The study tests case management as a way of controlling costs in the fee-for-service sector. These demonstrations, mandated by the Omnibus Reconciliation Act of 1990, and are aimed at a number of chronic illnesses, including congestive heart failure. The resulting demonstrations were implemented in three sites, began operations in October 1993, and continued through November 1995. The target conditions and case management protocols differed in each site, though all three generally focused on increased education regarding proper patient monitoring and management of the target chronic condition. All three sites anticipated reduced hospitalizations and medical costs compared to the beneficiary control groups. (Final report: 216 pages, plus appendices.)

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Leslie M. Greenwald

PHONE NUMBER: (410) 786-6502

PIC ID: 6291

PERFORMER ORGANIZATION: Mathematica Policy Research, Incorporated, Plainsboro, NJ

TITLE: Evaluation of the Utah Prepaid Mental Health Plan

ABSTRACT: This report is an evaluation of a demonstration project conducted in Utah, in which three mental health centers provided mental health services to all Medicaid beneficiaries in their catchment areas (these areas include over 50 percent of all Utah Medicaid beneficiaries). The hope was that this program would control the cost inflation and improve patient outcomes in mental health services. The evaluator examines three areas: (1) organizational and financial characteristics and evolution over time, (2) impact on the organization of service delivery and use of services, and (3) financial impact on providers and the Medicaid program in Utah. The development of the demonstration proceeded fairly smoothly, albeit somewhat more slowly than planned. Three centers chose to enter the demonstration and eight chose to not join. Several changes in the program's environment affected the development during the initial 3 years. (Final report: 68 pages.)

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Paul J. Boben

PHONE NUMBER: (410) 786-6629

PIC ID: 6293

PERFORMER ORGANIZATION: University of Minnesota, Minneapolis, MN

TITLE: Monitoring and Evaluation of the Medicare Cataract Surgery Alternate Payment Demonstration: Final Evaluation Report

ABSTRACT: This report assisted the Health Care Financing Administration (HCFA) in tracking the 3-year demonstration that tested the concept of a single global Medicare payment for outpatient cataract procedures. The payment amount represented payment for the physician, facility, and intraocular lens costs. The analysis portion of the project tested whether there were any net savings to the Medicare program, changes in the use of services included and excluded from the bundle, beneficiary satisfaction, and quality of care. The evaluation found that the project was reasonably successful in encouraging provider flexibility and in managing the bundle of services. There was ample anecdotal evidence that the cost effectiveness incentives were working; e.g., cost reduction efforts by surgeons in the techniques and time required to perform the surgery and actions to standardize the interocular lenses and other supplies and materials. While the effort to become a part of the demonstration was substantial, once operations began, government activities related to pricing were greatly reduced and limited to simple verification and payment of the global fee. The evaluator was not able to measure appropriateness, but in terms of quality control and utilization review the demonstration was successful in that the outcomes were unchanged. The project did produce some insights into the strengths and weaknesses of particular incentives in encouraging providers to participate. However, it provided limited new information on what factors influenced beneficiaries to participate.

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Cynthia K. Mason

PHONE NUMBER: (410) 786-6680

PIC ID: 6295

PERFORMER ORGANIZATION: Abt Associates, Incorporated, Cambridge, MA

TITLE: Evaluation of the Demonstration to Improve Access to Care for Pregnant Substance Abusers

ABSTRACT: This report evaluates demonstration project effectiveness in (1) outreach and assessment; (2) expansion, integration, and coordination of program services; and (3) improvement of client case management. The evaluator examined access to prenatal care and substance abuse treatment services and assessed the effects of these services on the health of drug-addicted pregnant women and birth outcomes of their infants. The evaluation showed that the number of abusers enrolled in the demonstrations was low relative to all pregnant substance abusers in the area, since women were reluctant to be identified. The project found higher enrollment rates in States that implemented broad-based outreach efforts; higher levels of and greater retention in substance abuse treatment resulted in higher birth weight infants. (Final report: 104 pages, plus appendices.)

AGENCY SPONSOR: Center for Beneficiary Services

FEDERAL CONTACT: Suzanne Rotwein

PHONE NUMBER: (410) 786-6621

PIC ID: 6297

PERFORMER ORGANIZATION: Mathematica Policy Research, Incorporated, Washington, DC

TITLE: Evaluation of the Medicaid Uninsured Demonstrations

ABSTRACT: This project evaluated three demonstration project conducted in Maine, South Carolina, and Washington State. These demonstrations, implemented in response to a Congressional mandate under section 4745 of the Omnibus Budget Reconciliation Act of 1990, tested the effect of allowing States to extend Medicaid coverage to low-income families. The evaluation resulted in a series of annual reports, an interim and a final report to Congress, and a final evaluation report. The project examined intra- and inter-site processes and outcomes, including (1) the ability of the programs to enroll significant numbers of eligible persons; (2) conditions under which eligible persons and their families are willing to participate in such programs, given their scarce financial resources; (3) ability of the programs to induce adequate numbers of providers to ensure the availability of necessary services at appropriate utilization levels; (4) willingness of employers to participate in the programs and the conditions under which they participate or choose not to participate; (5) the program's effect on service utilization and health outcomes of participants; (6) the cost-effectiveness of such programs for the various public and private interests; and (7) the extent to which the demonstration's interventions could be applied nationally to assist in achieving program goals. An initial series of site visits was conducted during 1993. The first annual report in the series used data collected during these site visits and data from State-administered baseline surveys to describe the implementation phase and early operational phase of the demonstrations.

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: James Hadley

PHONE NUMBER: (410) 786-6626

PIC ID: 6298

PERFORMER ORGANIZATION: Health Economics Research, Incorporated, Waltham, MA

TITLE: Access to Physicians' Services for Vulnerable Beneficiaries: Impact of the Medicare Fee Schedule

ABSTRACT: In 1992, Medicare introduced the Medicare Fee Schedule (MFS), which alters the relative prices paid to physicians for services and arguably the incentives to provide one service over another. This project evaluates the impact of the MFS on individual access to health care. Since the introduction of the MFS, the Health Care Financing Administration and the Physician Payment Review Commission have monitored access to physicians' services and have failed to document any barriers to health care access resulting from the MFS. This study builds on this work by (1) oversampling groups of beneficiaries thought to be particularly vulnerable to payment changes, (2) evaluating changes in treatment patterns for specific episodes of care, and (3) conducting multivariate analyses that measure actual payment changes over time rather than expected changes due to MFS. A stratified random sampling design took advantage of the differential impacts of the MFS across geographic areas and ensured adequate numbers of vulnerable beneficiaries. The study divided all geographic areas into six groups based on their expected 1992 payment changes under MFS, compared with the earlier payment scheme. The report finds that (1) there is little evidence that the MFS either improved or worsened health care access for Medicare beneficiaries; (2) while some vulnerable groups experienced MFS-related changes in access, the actual size of these changes is relatively small and is not consistently negative or positive; (3) substantial access gaps exist for vulnerable beneficiaries and there is evidence that access to primary care may have worsened for dual Medicaid-eligibles; and (4) Medicare beneficiaries reduced out-of-pocket payments over the 1991-1993 period, including lower copayments and lower extra billing liability. (Final report: variously paginated, plus appendices.)

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Renee Mentnech

PHONE NUMBER: (410) 786-6692

PIC ID: 6301

PERFORMER ORGANIZATION: Center for Health Economics Research, Waltham, MA

TITLE: A Study of the Appropriateness, Process of Care, and Outcomes of Medical Care for Medicaid Patients

ABSTRACT: The Congress requires the Health Care Financing Administration to report on the relative quality of care in the Medicaid program. This study assesses the variation that exists in the rate of performance of selected treatments and procedures provided to Medicaid beneficiaries for small areas within and between States. The report's first phase shows that the overall rate of inpatient use in the Medicaid population is higher than the rate expected based on non-Medicaid utilization patterns. In the report's second phase, data are abstracted from medical records at 118 hospitals in California, Georgia, and Michigan for a selected series of index diagnoses: pediatric asthma, hysterectomy, and complicated labor and delivery. This second phase develops five indicators to assess quality: (1) appropriateness to hospitalize or perform a procedure, (2) process of care, (3) outcomes of treatments and procedures, (4) severity at time of admission, and (5) intensity of care. Medicaid-covered children in two States, and women in all three States, are younger and represent more minority groups than their privately-insured counterparts. The appropriateness of the decision to hospitalize a child with asthma, perform a hysterectomy, and do a Cesarean section in the presence of labor and delivery complications are comparable between Medicaid and privately-insured patients. Most important, the outcomes of care for the two groups were essentially similar, for each condition, across the three States. Medicaid patients were more seriously ill at admission, which suggests that the Medicaid-covered patients may not be practicing the same level of preventative care as their privately-insured counterparts. As a result, the differences in patient outcomes were not a result of the inpatient quality of care provided. The inpatient care for the Medicaid patients was considered adequate and comparable with care received by privately-insured patients. (Final report: 46 pages.)

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Beth Benedict

PHONE NUMBER: (410) 786-7724

PIC ID: 6302

PERFORMER ORGANIZATION: MEDSTAT Group, Cambridge, MA

TITLE: An Evaluation of Implementation of the Medicaid Community Supported Living Arrangements (CLSA) Program in Eight States

ABSTRACT: This report describes the effectiveness of developing a continuum-of-care concept as an alternative to the Medicaid-funded residential services provided to individuals with mental retardation and related conditions. The program serves individuals who are living in the community either independently, with their families, or in homes with three or fewer other individuals receiving the same services. These services were provided in eight States between fiscal years 1991 and 1995. By the end of the program only 73 percent of the expected number of individuals were receiving services in the Community Supported Living Arrangement (CSLA), and the expenditures were only 58 percent ($38.4 million) of the amount authorized. A number of factors contributed to this smaller-than-expected program: (1) late 1990 authorization of CSLA and subsequent delay in regulations, application procedures, and State selection put back the first program enrollments by 16-22 months; (2) simultaneously expanded Federal latitude allowed rapid growth of Medicaid Home and Community Based Services; (3) implementers found that planning and designing services "one person at a time" were slower than expected; (4) States experienced delays in establishing the new systems of quality assurance required by the CSLA program; and (5) recruitment was sometimes delayed by the inclusion of groups of persons who were not traditional users of developmental disabilities services and who were not already known to the system. The report concludes that specific financing problems limit the effectiveness of the program. (Final report: 54 pages.)

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Samuel L. Brown

PHONE NUMBER: (410) 786-6667

PIC ID: 6304

PERFORMER ORGANIZATION: MEDSTAT Group, Cambridge, MA

TITLE: Evaluation of the Community Nursing Organization Demonstration Interim Evaluation Report

ABSTRACT: This report examines a set of demonstration projects mandated in the Omnibus Budget Reconciliation of 1987. The legislation directed the establishment of demonstration projects to test a capitated, nurse-managed system of care. The two fundamental elements of the demonstration are capitation payment and nurse case management. It was designed to promote timely and appropriate use of community health services and to reduce the use of costly acute care services. By October 1995, more than 5,400 beneficiaries were enrolled in the four Community Nursing Organizations (CNOs). The evaluation report covers the first 15 months of operation. The applicants have been healthier and more independent, on average, than the general Medicare population in their respective market areas. Analyses of measures of death, physical functioning, and satisfaction with care detected no statistically significant differences. Analysis of a subsample of persons who were impaired in Activities of Daily Living indicated that members of the CNO treatment group were more likely to improve (or less likely to deteriorate further) than members of the control. An even stronger result, in the same direction, was found for individuals who were limited in Instrumental Activities of Daily Living. (Final report: variously paginated.)

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Melissa Hulbert

PHONE NUMBER: (410) 786-8494

PIC ID: 6306

PERFORMER ORGANIZATION: Abt Associates, Incorporated, Cambridge, MA

TITLE: Evaluation of the Effectiveness of the Operation Restore Trust Demonstrations

ABSTRACT: Operation Restore Trust (ORT) was jointly developed by the Office of Inspector General (OIG), the Administration on Aging (AoA), and the Health Care Financing Administration (HCFA) as a model to demonstrate improved methods for investigation and prosecution of fraud and abuse in health programs. ORT targets a set of initiatives that focus on a few States, a few areas of the Medicare benefit, and a few standard operating procedures (especially failures to coordinate among the most central agencies and to exploit available data resources). The report finds that (1) ORT is associated with a net increase of more than 100 percent in ORT-type cases investigated by the OIG; (2) these cases should generate substantial incremental receivables, some fraction of which will translate into true savings as collections are made; (3) HCFA and its contractors had a series of successful projects under ORT, with notably large collections or avoidances of payment accompanying many of these efforts, although it cannot be said with certainty that these savings would not have been realized without ORT; (4) ORT encourages several important policy changes made during, or immediately following, the demonstration; (5) ORT may have important "sentinel effects" (e.g., ORT may have deterred providers from certain kinds of suspect behavior); (6) there are considerable process effects of ORT; (7) giving the AoA a fraud and abuse mission led to active training, outreach, and education efforts; and (8) the infrastructure of information for combating fraud and abuse has significantly improved, although these improvements include many efforts outside ORT itself. The evaluation concludes that ORT turned around certain troubling trends of the early 1990s and has brought new vigor to government efforts to combat health care fraud and abuse.

AGENCY SPONSOR: Office of Financial Management

FEDERAL CONTACT: Edward Norwood

PHONE NUMBER: (410) 786-6571

PIC ID: 6311

PERFORMER ORGANIZATION: Abt Associates, Incorporated, Cambridge, MA

TITLE: The Impact of Report Cards on Employees: A Natural Experiment

ABSTRACT: This report seeks to determine whether the dissemination of information about health plans to consumers (who choose health plans within a managed care competition framework) would influence their knowledge of plan characteristics, attitudes toward the plans, or choice of plan. Investigators found no "report card" effect on employees' knowledge of health plans, their attitude about quality of health plans, or their choices in selecting a managed care health plan. The study did find that certain health plan characteristics (such as price) were strongly related to health plan choice. The investigators suggest that the current versions of health plans are works in progress and that consumers do not seem to be influenced by the information in any of the ways measured in this study. (Final report: 41 pages.)

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Sherry Terrell

PHONE NUMBER: (410) 786-6601

PIC ID: 6312

PERFORMER ORGANIZATION: Park Nicollet Medical Foundation, St. Louis Park, MN

TITLE: Consequences of Paying Medicare HMOs and Health Care Prepayment Plans (HCPPs)

ABSTRACT: This study examined the operational aspects of health care prepayment plans (HCPPs) and the cost-effectiveness of cost health maintenance organizations (HMOs) and HCPPs. The report contains a case study of HCPPs: it looks at plans' utilization management and quality assurance programs, marketing practices, and contingency plans in case of insolvency. It also examines the cost-effectiveness to the Health Care Financing Administration (HCFA) of cost HMOs and HCPPs with respect to the fee-for-service and risk contracting program during calendar year 1993. The report found that costs to HCFA increased substantially under both the cost HMO and HCPP programs. Cost increases relative to fee-for-service were larger than those relative to the risk-based approach. Only a handful of the 63 plans examined appeared to have generated savings for HCFA. The main reason for the cost increase appears to be a lack of incentive for plans to drive hard bargains with physicians on prices. Furthermore, the report found that most of the cost-based plans experienced favorable selection, where the relatively more healthy segments of the population are enrolled. (Final report: 178 pages, plus appendices.)

AGENCY SPONSOR: Center for Health Plans and Providers

FEDERAL CONTACT: Ron Lambert

PHONE NUMBER: (410) 786-6624

PIC ID: 6314

PERFORMER ORGANIZATION: Mathematica Policy Research, Incorporated, Plainsboro, NJ

TITLE: Toward a Prospective Payment System for Outpatient Services: Implementation for Outpatient Services-Implementation of APGs by State Medicaid Agencies and Private Payers

ABSTRACT: This report studies a Medicaid outpatient prospective payment system, which groups patients for payment purposes rather than paying on a cost basis. These groups are similar in concept to diagnosis-related groups, which form the heart of Medicare's prospective payment system for inpatient care. It involved a case study of Iowa's implementation of the ambulatory patient group (APG) system and an analysis of the reimbursement methodology. This report has two components: an in-depth study of three operational APG systems (based on site visits) and telephone interviews with other payers that have implemented or planned to implement the system. All six payers with operational experience tailored their APG systems to their own priorities and markets. They all implemented the system without major incident. The payers reported success in reducing outpatient costs, where that was the immediate goal, and they believe the system encourages higher-cost facilities to reduce costs and rewards lower-cost facilities. Providers' views were more mixed. Though they reported generally adequate overall payment, they also viewed the system as complex, had difficulty calculating expected payment, and often let payer computer systems group related claims rather than consolidating them prior to submission, as was the intent of the system. No evidence was found of much behavioral response by providers, but that could change if the system is implemented by a large payer such as HCFA. (Final report: 78 pages.)

AGENCY SPONSOR: Center for Health Plans and Providers

FEDERAL CONTACT: Joe Cramer

PHONE NUMBER: (410) 786-6676

PIC ID: 6320

PERFORMER ORGANIZATION: Mathematica Policy Research, Incorporated, Plainsboro, NJ

TITLE: Payment of Pharmacists for Cognitive Services: Results of the Washington State C.A.R.E. Demonstration Project

ABSTRACT: The Washington Cognitive Activities and Reimbursement Effectiveness (C.A.R.E.) Project was a demonstration authorized under the Omnibus Budget Reconciliation Act of 1990 to assess the impact of a financial incentive on pharmacists' performance of cognitive services. The premise of this demonstration was that direct reimbursement for pharmacists' cognitive services will remove financial barriers associated pharmacists' provision of these services and result in increased performance of cognitive services, with a subsequent impact on costs and outcomes. Data for the demonstration were drawn from three groups of pharmacies located throughout the State of Washington, each of which contained about 100 pharmacy sites. Pharmacists in the first two groups documented cognitive services for Medicaid patients using a problem-intervention-result format. The demonstration phase lasted from February 1994 through September 1995 and resulted in the documentation of 20,240 cognitive service events. The project found that (1) Group A pharmacists, who received financial incentives, consistently reported higher cognitive service intervention rates than did Group B, who did not receive financial incentives; (2) about half of all documented cognitive services problems were for patient-related problems, while 32.6 percent were for drug-related problems, 17.6 percent for prescription-related problems, and 1.4 percent for non-drug-related problems; (3) a drug therapy change occurred as a result of 28 percent of all cognitive services; (4) for each cognitive service associated with any type of drug therapy change, the average downstream drug cost savings was about $13, but these savings varied by type of drug therapy; and (5) the Medicaid program saved about $78,000 due to cognitive services resulting in drug therapy changes. The report concludes that the implementation of a prescription drug-related cognitive services documentation and reimbursement system is feasible and useful.

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Jay Bae

PHONE NUMBER: (410) 786-6591

PIC ID: 6528

PERFORMER ORGANIZATION: University of Washington, School of Medicine, Seattle, WA

TITLE: Assessing the Viability of All-Payer Systems for Inpatient Hospital Services

ABSTRACT: This report explores the feasibility and possible structure of alternative approaches to developing an all-payer prospective system for reimbursing hospitals for inpatient care. The study tests the practicality of adapting or extending Medicare's Prospective Payment System (PPS) to either other payers or all payers as part of the health care reform legislation proposed by the Clinton Administration in 1993. The initial tasks were the development of data files necessary to construct such an all-payer PPS. The most basic components of a PPS are payment weights, rates, and adjustors. Separate analytic steps are required in order to derive each of these components, and several different types of data are required to complete each of these analytic tasks. Admission records are needed and must be categorized according to a specified classification system. Hospital-specific cost data by payer category are required to calculate costs per case, and hospital and area characteristic data are required in order to judge the need for and level of payment adjustors. Once these data were developed, they were used in two analyses: one addressing whether diagnosis-related group (DRG)-based payment rates for Medicare and non-Medicare patients should be adjusted based on hospital characteristics, and one focusing on an alternative approach to defining the Medicare standardized prospective payment rate. The report finds that current PPS rates are based on the mean adjusted cost for a given category of hospital, and that high-cost hospitals are penalized under the assumption that their costs are reflective of presumed inefficiency. The report develops hospital-specific measures of inefficiency and then excludes the costs of inefficient hospitals from the calculation of standardized payment rates. The report concludes that a frontier cost function model approach is superior to the current system and can result in substantial program savings. See also PIC ID No. 6594.1.

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Jesse Levy

PHONE NUMBER: (410) 786-6600

PIC ID: 6594

PERFORMER ORGANIZATION: Urban Institute, Washington, DC

TITLE: Second Update of the Geographic Practice Cost Index: Final Report

ABSTRACT: The Geographic Practice Cost Index (GPCI) is an input price index used to adjust Medicare Fee Schedule (MFS) payments for area variations in physicians' cost of practice. The original GPCI was implemented with the MFS in 1992. In 1995 and 1996, the GPCI was updated with 1990 Decennial Census data. This report documents the second update of the GPCI, which is scheduled to be fully phased in on January 1, 1999. During the previous year, 1998, a transition will occur between the 1997 GPCI and the updated 1999 GPCI, with one-half the total change implemented in 1998. The comparison between the 1997 and the 1999 GPCIs shows that (1) there is no change between the physician work GPCI or the non-physician employee wage index of the practice expense GPCI except that they are reweighted with 1994 county relative value units (RVUs); (2) the office rental index of the practice expense GPCI is updated with fiscal year 1996 Fair Market Rents published by the Department of Housing and Urban Development; (3) county rental adjustment factors are eliminated for all metropolitan areas, except for the New York City Primary Metropolitan Statistical Area; (4) the malpractice GPCI is updated with 1992, 1993, and 1994 premium data, and specialty premiums are weighted by national RVU proportions rather than allowed charges; (5) there is no change in the GPCI cost shares, which measure the share of the various inputs in total practice revenues; and (6) the county RVUs used to weight averages of county input prices for each Fee Schedule Area (FSA) are updated from 1992 to 1994. The report also examines changes in the Geographic Adjustment Factor between 1999 and 1997. It determines that changes in MFS payments resulting from implementation of the 1999 GPCIs will be small (less than 2.5 percent for each FSA). See also 5764 series. (Final report: variously paginated, plus appendices.)

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Benson Dutton

PHONE NUMBER: (410) 786-6603

PIC ID: 6611

PERFORMER ORGANIZATION: Health Economics Research, Incorporated, Waltham, MA

TITLE: Validation of Nursing Home Quality Indicators Study

ABSTRACT: This study developed a system by which a series of Quality Indicators (QIs) could be used in conjunction with claims data to monitor quality of care in nursing home facilities. Fifty QIs were developed and validated using a sample of nursing home residents' medical records for a 6-month period surrounding the time of nursing facility's Federal certification survey. The QIs focus on the occurrence of three broad potential problem areas, including adverse outcomes, lack of therapy, and inappropriate pharmaceutical treatments. The QI algorithm used ICD-9 CM diagnosis codes and CPT-4 and CRVS procedure codes to generate the QIs. These codes represent either medical or surgical conditions that define the indicator events (e.g., inpatient stay or emergency room visit), the covariant diagnoses, or case selection factors. It is anticipated that these validated QIs could be used as the basis for an automated system to continuously monitor the quality of care provided in federally certified nursing homes to assist Federal and State surveyors in implementing the survey process effectively. Such a system could contribute to lowering the costs of care and improving outcomes for Medicaid nursing home residents by reducing the incidence of potentially avoidable hospitalization and over-medication. The two volume report finds that (1) the utility of basing indicators of quality of nursing home care on Medicaid and Medicare claims data is clearly demonstrated; (2) the study QIs as a whole were generally better at predicting the absence of a quality issue, suggesting that the negative finding on the quality indicator represented the lack of a quality concern in the nursing home; and (3) using covariant diagnoses to adjust for the risk of a resident receiving a QI tag is useful. The report concludes with a discussion of the issues raised during the validation of the 50 QIs examined here. (Volume I: 88 pages; Volume II: appendices variously paginated.)

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Kay Lewandowski

PHONE NUMBER: (410) 786-6657

PIC ID: 6612

PERFORMER ORGANIZATION: MEDSTAT Group, Cambridge, MA

TITLE: 1995 Influenza Immunizations Paid for by Medicare: State and County Rates

ABSTRACT: In 1993, the Health Care Financing Administration began paying providers for influenza vaccine and its administration to Medicare beneficiaries aged 65 and older. This data book provides information about the national, State, and county rates of influenza vaccination in the Medicare fee-for-service (FFS) sector. Medicare-managed care beneficiaries are not counted. The data book presents maps that report rates as integers for ease of presentation. It also presents tables that measure rates to one tenth of one percent. State rates are also presented. The data book provides three tables for each State: (1) a table providing the State's rates and numbers of influenza immunizations for 1994 and 1995 according to age, gender, and race; (2) a table showing the State's number of elderly enrolled in Medicare Part B FFS and the number of individuals who received flu shots in 1995 according to race and county; and (3) a table presenting corresponding immunization rates for 1994 and 1995 according to race and county. The data show that (1) about 41 percent of elderly Medicare beneficiaries received an influenza vaccination in 1995 that was paid for by Medicare; (2) for all sectors of this population, there was an increase over 1994 immunization rates ranging from 2.3 percent to 6.4 percent, indicating progress toward the Healthy People 2000 goal of a 60 percent influenza immunization rate for this population; (3) African-American beneficiaries have an immunization rate one-half that of Caucasian beneficiaries, although between 1994 and 1995, the increase in the rate of immunizations was slightly greater for African Americans than for Caucasians; (4) there is no substantial variation in rates by gender; and (5) immunization rates are lowest for those beneficiaries 85 years and older. (Final report: 279 pages.)

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Marsha Davenport

PHONE NUMBER: (410) 786-6693

PIC ID: 6615

PERFORMER ORGANIZATION: Health Care Financing Administration, Baltimore, MD

TITLE: Impact of the Medicare Fee Schedule on Teaching Physicians

ABSTRACT: Prior to the implementation of the Medicare Fee Schedule (MFS) in 1992, several studies were conducted to determine the impact of this payment change on various groups of physicians. However, no studies examined the potential impact of the MFS on teaching physicians, even though anecdotal evidence from academic practices suggested that these physicians would be particularly hard hit for four reasons (1) teaching physicians perform relatively more high-technology procedures and less primary care, which could result in disproportionate net losses for many academic practices; (2) teaching hospitals and medical schools tend to be located in large urban areas, where the Geographic Practice Cost Index portion of the payment is lower; (3) customary charges for teaching physicians may be higher than the area-wide historical payments calculated for fee schedule transition; and (4) a number of academic medical practices had not signed Medicare participation agreements. This study determines the impact of the MFS on teaching physicians and examines variations in the impact across academic medical centers, other major teaching hospitals, and minor teaching hospitals. The net effects of MFS price changes and any observed quantity changes on inpatient physician revenues were examined. A total of 720 hospitals were selected for this study, including all U.S. academic medical centers. The report finds that there is no evidence that teaching physicians have been adversely impacted by the MFS. Inpatient revenues per admission fell by the same percentage amount in all types of hospitals. Revenues fell because of a reduction in prices and in the quantity of services provided per admission. Thus, while the theoretical expectation was that teaching physicians would have experienced a disproportionately large reduction in their average price per service, in practice these physicians have altered that service mix by providing less RVU-intensive visits.

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: William Buczko

PHONE NUMBER: (410) 786-6593

PIC ID: 6666

PERFORMER ORGANIZATION: Center for Health Economics Research, Waltham, MA

TOP OF DOCUMENT

HEALTH RESOURCES AND SERVICES ADMINISTRATION

Contents

Institutional Factors Which Influence the Likelihood of Living-Related Kidney Transplant Operations-- Phase II

Exploratory Study of Reasons Kidney Transplant Waiting List Patients are Unavailable for Organ Offers

Community Health Center User and Visit Study: Survey Methodology Report

Healthy Schools, Healthy Communities National Evaluation: Final Report

Exploratory Evaluation of Rural Applications of Telemedicine

Development of a Performance Management System for Ryan White Titles I, II, and SPNS

Development of a Performance Measurement System for the Division of Organ Transplantation

Profiles of Activities to Reduce Perinatal Transmission of HIV: Assessing the Response

Grantee-Level Evaluation Activities of the HIV/AIDS Programs of the Ryan White CARE Act

Bilingual Assistance Program Evaluation, Part I: Strategies for Reducing Cultural and Linguistic Barriers to Health Care for Hispanic and Asian/Pacific Islander Populations

Examining Relationships Between HIV/AIDS and Tuberculosis Programs in Title I Eligible Metropolitan Areas

Availability of Primary Care Services Under Medicaid Managed Care: How 14 Health Plans Provide Access and the Experience of 23 Safety Net Providers and Their Communities

The Registered Nurse Population, March 1996: Findings from The National Sample Survey of Registered Nurses

Service Integration for Multi-Risk Patients: Final Report

Measuring Expenditures for Personal Health Care Services Rendered by Public Health Departments

HIV Capitation Risk Adjustment Conference Report

TITLE: Institutional Factors Which Influence the Likelihood of Living-Related Kidney Transplant Operations-Phase II

ABSTRACT: Most transplant surgeons and physicians believe that kidneys can be obtained from live donors in a safe and effective manner, and that kidneys from live donors provide superior long-term graft survival rates for transplant recipients. Between 1987 and 1993, for example, the 1- and 2-year survival rates for patients who had received kidney transplants through living-related donations (LRDs) was superior to that of patients who had received cadaveric kidneys. The rate of living-related kidney transplantation varies considerably among States, however. This project assessed (1) the willingness of the transplant community to conduct, and the frequency with which it has conducted, kidney transplants from LRDs; (2) the medical and nonmedical institutional factors that influence decision making by transplant professionals about suggesting and conducting a living-related kidney transplant; (3) the range of transplant center responses to, and procedures for, living-related transplants; and (4) the circumstances and conditions under which a transplant team that does not encourage living-related transplants would refer a patient to another transplant center for evaluation. This study is based on a mail survey of kidney transplant centers in the Fall of 1995, which yielded an 80 percent response rate. A key finding is that while attitudes toward LRDs among transplant professionals are generally positive, survey respondents identified a number of deterrents to LRDs that are not related to characteristics or practices of individual transplant centers. Some of these factors relate to the characteristics of the individual patient and his or her family. Among the recommendations is that transplant centers distribute educationally and culturally appropriate materials about policies and procedures related to LRDs. (Executive Summary: 10 pages; final report: 78 pages, plus appendices.)

AGENCY SPONSOR: Office of Special Programs

FEDERAL CONTACT: D. W. Chen

PHONE NUMBER: (301) 443-4111

PIC ID: 4917.1

PERFORMER ORGANIZATION: George Washington University, Washington, DC

TITLE: Exploratory Study of Reasons Kidney Transplant Waiting List Patients are Unavailable for Organ Offers

ABSTRACT: Studies conducted in the early 1990's to examine the efficiency of the organ allocation system identified an additional factor that might influence the equitability of allocation decisions: patient unavailability and refusal of an organ. This study (1) further examines the extent of the problem of patient unavailability for, or refusal of, an organ offer; (2) explores the reasons underlying patient unavailability and patient refusal when an organ offer is made; (3) highlights strengths and weaknesses in recordkeeping and reporting practices; and (4) makes recommendations for improving the procedures for notifying patients of an organ offer. The contractor conducted telephone interviews with three groups: transplant center staff who make organ offers, transplant waiting list patients who had refused an offer or had been unavailable when staff tried to make an offer, and recent transplant recipients. Interviews with study participants, drawn from 31 randomly selected transplant centers, were conducted between November 1995 and September 1996. The major finding was that both patient unavailability and refusal occur but are not significant issues at most centers. (Final report: 38 pages, plus appendix.)

AGENCY SPONSOR: Office of Special Programs

FEDERAL CONTACT: D. W. Chen

PHONE NUMBER: (301) 443-4111

PIC ID: 5601

PERFORMER ORGANIZATION: Battelle Human Affairs Research Centers, Seattle, WA

TITLE: Community Health Center User and Visit Study: Survey Methodology Report

ABSTRACT: This project collects data on users of Community Health Centers (CHCs) and the services they use. The sample survey has two components: (1) a personal interview survey of 1,932 randomly selected patients at a representative sample of 48 CHCs, and (2) data extraction from 2,878 medical records concerning randomly selected encounters at the same centers. Clients and visits are surveyed using modified versions of two National Center for Health Statistics (NCHS) surveys: the National Health Interview Survey and the National Hospital Ambulatory Medical Care Survey, to capture the full range of users and services provided at a particular point in time. This approach allows for comparison of data on CHC populations with national estimates derived from the NCHS surveys. The project provides demographics of users, their reasons for seeking care, diagnoses, services used, and self-reported outcomes. The User and Visit Survey is part of an overall Health Resources and Services Administration (HRSA) evaluation strategy for collecting data regarding users, services, and outcomes of primary care programs. HRSA will be developing further reports based on the survey findings. (Final report: 147 pages, plus appendices.)

AGENCY SPONSOR: Bureau of Primary Health Care

FEDERAL CONTACT: Jerrilynn Regan

PHONE NUMBER: (301) 594-4280

PIC ID: 5737

PERFORMER ORGANIZATION: Mathematica Policy Research, Incorporated, Plainsboro, NJ

TITLE: Healthy Schools, Healthy Communities National Evaluation: Final Report

ABSTRACT: This project develops and implements a design to collect uniform data on access, use, and continuity of care in the Healthy Schools, Healthy Communities Program (HSHC). The Health Resources and Services Administration's (HRSA's) HSHC program is the first Federal categorical grant program specifically targeted to school-based health centers, and is jointly funded and administered by the Bureau of Primary Health Care and the Maternal and Child Health Bureau. The study includes the 25 (of 27) school-based health centers (SBHCs) funded initial projects that were first fully operational during the 1995-1996 school year. This evaluation, implemented concurrently with the HSHC program, was intended to document change in the students' ealth status, utilization of services, and outcomes related to the SBHC interventions. There are two data sources: a student-based clinical data system and a student/parent survey. Findings are informing the activities of participating grantees and the content of policies for grantees that join the program in FY 1998 or FY 1999. In addition, HRSA has gained new insights into collecting data from children and on working with school-based records.

AGENCY SPONSOR: Bureau of Primary Health Care

FEDERAL CONTACT: Thomas Coughlin

PHONE NUMBER: (301) 594-4450

PIC ID: 5739

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Exploratory Evaluation of Rural Applications of Telemedicine

ABSTRACT: The purposes of the Health Resources and Service Administration's (HRSA's) Rural Telemedicine Grant Program (RTMG) are (1) to facilitate the development of rural health care networks through the use of telemedicine, and (2) to develop a data base for systematic evaluation of rural telemedicine systems. This baseline study concerned (1) the current status of telemedicine in rural health; (2) the effects of telemedicine on access to care, practitioner isolation, and development of health care networks; (3) the organizational factors that aid or impede successful development and implementation of telemedicine systems; and (4) development, testing, and refinement of data collection instruments for use in subsequent evaluations. The study focused on telemedicine projects that link one or more rural hospitals or ambulatory sites ("spoke sites") with a multispecialty entity ("hub site"). A hub site might be a tertiary care hospital, a multispecialty clinic, or a set of facilities that, when combined, provide the required range of specialty consultations. In addition, the study was limited to systems that employ real-time or asynchronous video imaging systems. Data sources for the evaluation include three mail surveys to identify and describe telemedicine systems, and site visits to four networks. The report presents recommendations on the content of future evaluations and on data gathering approaches. Results have influenced the FY 1997 RTMG application guidelines and the content of authorizing legislation. The report has been distributed to other Federal agencies that fund telemedicine projects and to State Offices of Rural Health. Findings are being incorporated into the design of a common telemedicine evaluation instrument for use by HRSA grantees, and by other Federal funders at their option. Finally, five articles based on the study have been published and numerous presentations have been given.

AGENCY SPONSOR: Office of Rural Health Policy

FEDERAL CONTACT: Cathy Wasem

PHONE NUMBER: (301) 443-0835

PIC ID: 5749

PERFORMER ORGANIZATION: Abt Associates, Incorporated, Cambridge, MA

TITLE: Development of a Performance Management System for Ryan White Titles I, II, and SPNS

ABSTRACT: In the public sector, efforts to monitor program effectiveness go under the rubric of "reinventing government." Several legislative and executive initiatives, particularly the Government Performance and Results Act of 1993 (GPRA), spur the movement to establish measurable performance goals that can be reported as part of the budgetary process, thus linking funding decisions to performance. GPRA requires each Federal agency to develop comprehensive strategic plans, annual performance plans with measurable goals and objectives, and annual reports on actual performance compared to goals. This report focuses on the response to GPRA undertaken by the Bureau of Health Resources Development (now called the HIV/AIDS Bureau) within the Health Resources and Services Administration (HRSA) in regard to Titles I, II, and Special Programs of National Significance (SPNS) of the Ryan White Comprehensive AIDS Resources Emergency (CARE) Act of 1990. The major results of this project were the development and articulation of an interactive process to be used in the development of a performance measurement system. This process includes (1) a clear articulation of the purposes and objectives of Ryan White Titles I, II, and SPNS; (2) linkage of these purposes and objectives to the broader functions of HRSA; (3) using program purposes and functions, and a clear understanding of accountability, responsibility, and activity throughout the programs, to enable the performance measurement system to capture the individual properties and components of each; (4) developing standard performance terminology and definitions; (5) identification of seven performance dimensions and linking them to the purposes and functions of the programs; and (6) identifying and narrowing potential measures for program performance. The report concludes with several recommendations for implementing a performance measurement system for these three programs. See also PIC ID Nos. 5954 and 5954.2.

AGENCY SPONSOR: HIV/AIDS Bureau

FEDERAL CONTACT: Katherine Marconi

PHONE NUMBER: (301) 443-6560

PIC ID: 5954.1

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Development of a Performance Measurement System for the Division of Organ Transplantation

ABSTRACT: The Government Performance and Results Act of 1993 (GPRA) links funding decisions to program performance by calling for the designation of measurable performance goals that can be reported as part of the budgetary process. GPRA requires each Federal agency to develop comprehensive strategic plans, annual performance plans with measurable goals and objectives, and annual reports on actual performance compared to goals. This report focuses on the response to GPRA undertaken by Division of Organ Transplantation (DOT) within the Health Resources and Services Administration. The report provides a conceptual framework for the organ and bone marrow transplantation programs; outlines a recommended performance management system (including an articulation of program purposes and functions, language and tools of performance measurement, criteria for selecting measures and indicators, and identification of potential measures); and recommends next steps. These next steps include (1) identification of a process for selection of measures and indicators for DOT programs, (2) selection of the most important measures from the potential group, (3) identifying supporting data and data needs and gaps, (4) preparing an annual performance plan designed to meet GPRA requirements, and (5) identifying technical assistance needs for making a performance management system operational. See also PIC ID Nos. 5954 and 5954.1. (Final report: 26 pages.)

AGENCY SPONSOR: Office of Special Programs

FEDERAL CONTACT: Katherine Marconi

PHONE NUMBER: (301) 443-6560

PIC ID: 5954.2

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Profiles of Activities to Reduce Perinatal Transmission of HIV: Assessing the Response

ABSTRACT: In August 1994, the Public Health Service (PHS) published recommendations for the use of zidovudine (ZDV) to reduce perinatal transmission of HIV in response to findings from the Pediatric AIDS Clinical Trials Group (ACTG) 076 by the National Institutes of Health (NIH). In December 1995, the Health Resources and Services Administration (HRSA) issued a Program Advisory entitled "Use of Zidovudine (ZDV) to Reduce Perinatal HIV Transmission in HRSA Funded Programs," which provides background information and specific strategies and options for implementing the PHS recommendations. This study evaluates HRSA's activities in implementing the recommendations through case studies of Florida, Mississippi, Missouri, New Jersey, Puerto Rico, Texas, and Washington. Qualitative methods were used to describe the following for each site: (1) demographic and socioeconomic factors affecting HIV/AIDS prevention and care services, (2) nature of the HIV/AIDS epidemic, (3) structure of the public health system, (4) structure of the State health department, (5) structure of HIV/AIDS services, (6) issues affecting HIV/AIDS prevention and care services, (7) relevant policies and activities prior to the release of the ACTG 076 findings, and (8) specific responses to the ACTG 076 findings and PHS recommendations. The study finds that where providers have responded to the ACTG 076 findings, the results are extremely positive. Despite the complexities of the ZDV regimen, the majority of women elect to participate in the regimen after being informed about it, believing that the advantages outweigh the disadvantages. The study has been widely disseminated to HRSA HIV/AIDS and primary care providers and has been used in assessing the incidence of perinatal HIV transmission. (Final report: 101 pages, plus appendices.)

AGENCY SPONSOR: HIV/AIDS Bureau

FEDERAL CONTACT: Moses B. Pounds

PHONE NUMBER: (301) 443-6560

PIC ID: 6202

PERFORMER ORGANIZATION: U.S. Conference of Mayors, Washington, DC

TITLE: Grantee-Level Evaluation Activities of the HIV/AIDS Programs of the Ryan White CARE Act

ABSTRACT: This study provides a systematic review and synthesis of current Health Resources and Services Administration (HRSA) HIV-related evaluation activities conducted at the grantee level to identify gaps and set national funding priorities across HRSA HIV programs. The study, requested by the HRSA AIDS Advisory Committee, reviews grantee evaluation activities over the October 1995-October 1996 period. Studies are categorized by type (formative, process, outcome/impact, or economic). The information derived from this study is being used to improve the evaluation of HRSA HIV programs. One of the project outcomes is a current study to assess client satisfaction. In addition, HRSA is developing a series of technical manuals to assist grantees with issues of methodology and capacity for evaluation. The first of these manuals to be published, "Choosing and Using an External Evaluator," has been sent to all CARE Act grantees and will be made available to others upon request. (Final report: variously paginated.)

AGENCY SPONSOR: HIV/AIDS Bureau

FEDERAL CONTACT: Moses B. Pounds

PHONE NUMBER: (301) 443-6560

PIC ID: 6256

PERFORMER ORGANIZATION: Academy for Educational Development, Washington, DC

TITLE: Bilingual Assistance Program Evaluation, Part I: Strategies for Reducing Cultural and Linguistic Barriers to Health Care for Hispanic and Asian/Pacific Islander Populations

ABSTRACT: The Disadvantaged Minority Health Improvement Act of 1990 authorized the Bilingual Assistance Program, a 3-year initiative to improve the health status of racial and ethnic minorities. Under the program, the Health Resources and Services Administration (HRSA) awarded grants to three national organizations (the Association of State and Territorial Health Officials, the National Association of County Health Officials, and the U.S. Conference of Local Health Officials) which, in turn, awarded grants in 1993 to 13 State and local health departments. The purpose of the local projects, which were selected competitively, was to reduce cultural and linguistic barriers to health care among Asian/Pacific Islander and Hispanic populations. This study examines (1) factors that allow the local projects to leverage Federal funding and promote improved interagency working relationships; (2) realistic approaches to setting performance measures; (3) technical assistance needs of grantees, including exploring the use of national minority organizations and mentors; and (4) the replicability of the products or training curricula developed by the grantees. This is the first volume of a two-part final report; the second volume is entitled "Assessment of Three Training Curricula." The study was carried out through site visits conducted during April-June 1996 and a review of background documentation. HRSA's Office of Minority Health has shared the study products with interested groups. (Final report: variously paginated.)

AGENCY SPONSOR: Office of the Administrator

FEDERAL CONTACT: Laura Diaz-Baker

PHONE NUMBER: (301) 443-9966

PIC ID: 6356

PERFORMER ORGANIZATION: MDS Associates, Incorporated, Wheaton, MD

TITLE: Examining Relationships Between HIV/AIDS and Tuberculosis Programs in Title I Eligible Metropolitan Areas

ABSTRACT: This project (1) determines the extent to which specific Ryan White Comprehensive AIDS Resource Emergency (CARE) Act Title I grantees are providing guidance to funded providers on tuberculosis (TB) screening, prophylactic, and therapeutic regimes; and (2) describes working models of collaboration between human immunodeficiency virus (HIV) and TB control programs within individual Eligible Metropolitan Areas (EMAs). In March 1995, the 34 EMAs were asked to provide a brief description of TB epidemiology in the EMA, along with specific information regarding collaboration among programs to treat people co-infected with HIV and TB; 30 responded. A literature review provides background for site visits to seven EMAs. The visits were conducted to gain more detailed information about the nature of partnerships between TB and HIV/AIDS programs that result in the most successful models of TB prevention and control within Title I-funded programs, as well as to identify programmatic configurations that result in poorly implemented TB services at these sites. The study finds that TB was prevalent in EMAs, and that services vary across EMAs and across different care settings within them. Study results are being used in developing Title I program guidance and in planning for training and technical assistance. The report has been sent to CARE Act and TB control grantees.

AGENCY SPONSOR: HIV/AIDS Bureau

FEDERAL CONTACT: Gloria Weissman

PHONE NUMBER: (301) 443-9052

PIC ID: 6359

PERFORMER ORGANIZATION: Matthew McClain, Philadelphia, PA

TITLE: Availability of Primary Care Services Under Medicaid Managed Care: How 14 Health Plans Provide Access and the Experience of 23 Safety Net Providers and Their Communities

ABSTRACT: This study examines strategies used by health maintenance organizations (HMOs) to generate access to health care services for Medicaid enrollees, who often live in inner city and rural underserved areas. The project also explores the implications of these strategies for overall access in underserved areas and for existing care providers. Third, the study considers how these strategies affect traditional Medicaid providers in the community. The study involved interviews and collection of secondary data at 14 HMOs with significant Medicaid enrollment, and at 23 traditional safety net providers, of which 13 were Community Health Centers. The study concluded that Medicaid managed care has had an overall positive effect on the availability of primary care to low-income residents in the 14 communities, and that the traditional safety net providers have not needed to reduce services. However, the future viability and range of services of public sector-supported providers might be endangered in a more competitive health system. The information derived from this study will be useful to policymakers concerned with the provision of appropriate care for vulnerable populations, providers in underserved areas, and those attempting to assure an adequate supply of health care personnel. The study has attracted attention in the professional community, as in a story in the American Medical News. In addition, an article under preparation for submission to a professional journal will reference this study, among other sources. (Appendices to the report are bound separately.)

AGENCY SPONSOR: Office of the Administrator

FEDERAL CONTACT: Jessica Townsend

PHONE NUMBER: (301) 443-0371

PIC ID: 6366

PERFORMER ORGANIZATION: Mathematica Policy Research, Incorporated, Washington, DC

TITLE: The Registered Nurse Population, March 1996: Findings from The National Sample Survey of Registered Nurses

ABSTRACT: The Sixth National Sample Survey of Registered Nurses gathered data to evaluate trends in the number, characteristics, and distribution of registered nurses in the United States. This study builds on the sample design and procedures developed in earlier surveys. Sampling procedures are devised to assure (1) the cooperation of all States and the District of Columbia, and (2) an overall response rate of 80 percent. The study resulted in a data base of 29,950 coded individual record responses, with weights for estimating characteristics of the sample universe, estimates of variances for key variables, and a technical procedures report prepared by the contractor. Health Resources and Services Administration (HRSA) staff then analyzed the data and developed the report named above. Results from this study will (1) be used in biennial reports to Congress to describe characteristics of the registered nurse population, (2) contribute to the data base underlying models used to project nurse supply and requirements; and (3) serve as input to analyses and evaluation of HRSA's nursing education programs and plans for future Federal programs to assure appropriate nursing resources. In addition, the nursing community will use the data base, the most comprehensive source of statistical information on registered nurses in the United States. (Final report: 30 pages, plus appendices.)

AGENCY SPONSOR: Bureau of Health Professions

FEDERAL CONTACT: Evelyn Moses

PHONE NUMBER: (301) 443-1451

PIC ID: 6367

PERFORMER ORGANIZATION: Research Triangle Institute, Research Triangle Park, NC

TITLE: Service Integration for Multi-Risk Patients: Final Report

ABSTRACT: This study examines how selected community-based health care organizations integrate services for people with multiple risk factors (e.g., pregnancy, substance abuse, HIV/AIDS, and homelessness) for poor health outcomes. Since these conditions often overlap, the health care organization must integrate services to meet the needs of these individuals with multiple risks. This study targets six Community Health Centers and local health departments currently receiving funding from a combination of Health Resources and Services Administration (HRSA) programs. The report uses site visits, interviews, focus groups, and discussions to explore several topics, including (1) community-based health care organizations' use of multiple funding sources to provide integrated services to patients with multiple risk factors, (2) barriers to integration of services created by separate funding streams that target specific high-risk populations, (3) successful integration strategies used by organizations, and (4) ways in which HRSA might reduce barriers created through multiple funding streams. Based on these strategies, the report identifies characteristics of organizational approaches that facilitate integration and makes recommendations for addressing barriers to service integration. The report has been distributed to all the HRSA primary care grantees. Results will also be shared through articles in peer-reviewed journals. (Final report: variously paginated.)

AGENCY SPONSOR: Bureau of Primary Health Care

FEDERAL CONTACT: Thomas Coughlin

PHONE NUMBER: (301) 594-4450

PIC ID: 6369

PERFORMER ORGANIZATION: MDS Associates, Incorporated, Wheaton, MD

TITLE: Measuring Expenditures for Personal Health Care Services Rendered by Public Health Departments

ABSTRACT: As changes to the health care delivery system continue to occur, the way in which public health departments provide services may be affected in a very significant manner, making it critical for policymakers, managers, and researchers to be able to assess the impact of these changes on the public health infrastructure. Therefore, the Health Resources and Services Administration commissioned this report, which develops a categorization schema for reporting comparable information on State and local health department personal health care services. The project initiates development and testing of a methodology to collect consistent and complete expenditure data on State and local health department services directed toward individuals. Officials from 11 State and local health departments and 1 State association of local health officials assisted in the development of a survey instrument, which was tested by State health departments in Arizona, Iowa, and Rhode Island and by local health departments in Austin/Travis County, Texas, and New York City, New York. The report finds that (1) the provision of personal health care services accounts for nearly 50 percent of the health department expenditures in Austin/Travis County and 57 percent in New York City; (2) participants considered the methodology generally clear and potentially useful, although problems were encountered in trying to fit personal health care services into some survey categories; (3) among the survey respondents, Austin/Travis County is closest to the managed care end of the service delivery continuum and Iowa is closest to the traditional approach of delivering all services directly, bundling no services, and not using capitation; and (4) Rhode Island contracts for 99 percent of its services, and New York and Austin/Travis County offer a similar mix of department-run and contracted services. (Final report: 29 pages, plus appendices.)

AGENCY SPONSOR: Office of the Administrator

FEDERAL CONTACT: Michael Millman

PHONE NUMBER: (301) 443-0368

PIC ID: 6636

PERFORMER ORGANIZATION: Public Health Foundation, Washington, DC

TITLE: HIV Capitation Risk Adjustment Conference Report

ABSTRACT: Widespread adoption of managed care by public insurance programs (such as Medicaid) has raised concerns about whether capitated reimbursement affects the quality of care provided, especially to people with chronic illnesses or other special health care needs. One way to address these concerns is to create risk-adjusted capitation rates to provide equitable reimbursement for care of patients with costly illnesses, an approach several States had proposed or implemented by 1997. In May 1997, HRSA convened an invited conference to examine issues surrounding risk adjustment for HIV/AIDS, in collaboration with ASPE, HCFA, The Henry J. Kaiser Family Foundation, and the National Academy of State Health Policy. Participants included people living with HIV/AIDS and their advocates, providers of care, managed care plans, and purchasers, including representatives of State Medicaid programs. The report presents findings and recommendations concerning various risk-adjustment methods and their evaluation, issues related to the implementation of risk adjustment, and further research needed in each of these areas. Conference findings have been used in technical assistance to States on coordinating CARE Act programs with managed care. HRSA has distributed the report to all CARE Act grantees and to State Medicaid directors; the Kaiser Family Foundation, which financed production of the report itself, has also distributed it widely.

AGENCY SPONSOR: HIV/AIDS Bureau

FEDERAL CONTACT: Richard Conviser

PHONE NUMBER: (301) 443-3075

PIC ID: 6894

PERFORMER ORGANIZATION: The Henry J. Kaiser Family Foundation, Washington, DC

TOP OF DOCUMENT

INDIAN HEALTH SERVICES

Contents

Evaluation of the Impact of the Alaska Tribal Health Compact on Programs and Services in the Anchorage Service Unit, Alaska Area

Prior Trauma Care of Intoxicated Patients as a Predictor of Subsequently Fatal Injury

TITLE: Evaluation of the Impact of the Alaska Tribal Health Compact on Programs and Services in the Anchorage Service Unit, Alaska Area

ABSTRACT: This evaluation is the first research undertaken to systematically study the perceptions of a wide range of beneficiaries, officials, and employees of the Indian Health Service (IHS) and the tribal system regarding the Alaska Tribal Health Compact, Title III Self-Governance Demonstration Project. There are almost 200 federally recognized tribes in a single Area-wide Compact, organized via 13 separate consigners-each of which has its own Annual Funding Agreement. Seven of these consigners are in the Anchorage Service Unit, which is one of nine Service Units in the Alaska Area. The overall conclusion of the evaluation is that there are more village representatives who have seen either no change or a negative change in in-village services than those who have seen a positive change. The perceptions of the officials and employees of the Alaska Native Medical Center and the Alaska Area Native Health Service have revealed problems in staff morale and shifts in workload resulting from Compact consigners transferring patients to the Alaska Native Medical Center who were previously treated outside the Anchorage Service Unit. These changes are considered as having a negative impact on the Anchorage Service Unit. The "microcosmic" view from the Ahtna Region indicates that there is a distinct difference in the perception of those who are participating in the Compact and those who are not participating. The analysis of the village respondent data, comparing the information that came from the Title III villages with the information from the Title I villages, further solidified the conclusion that individuals in villages that are not served by a Compact consigner hold a consistently more negative view than those who live in villages that are serviced by a Compact consigner.

AGENCY SPONSOR: Office of Public Health

FEDERAL CONTACT: Leo J. Nolan

PHONE NUMBER: (301) 443-4245

PIC ID: 6407

PERFORMER ORGANIZATION: Alaska Native Health Board, Anchorage, AK

TITLE: Prior Trauma Care of Intoxicated Patients as a Predictor of Subsequently Fatal Injury

ABSTRACT: This study was undertaken to identify intervention opportunities associated with nonfatal, alcohol-related injury contacts in Indian Health Service emergency departments and clinics that could, over time, decrease alcohol-related injury deaths in the Billings Area Indian Health Services. The concept was to determine the frequency of injury treatment recidivism, both alcohol- and nonalcohol-related, and further to determine if patterns can be established to suggest that injury death is predictable based on those previous nonfatal contacts. The findings indicate a high degree of recidivism and a strong correlation between non-fatal incidents and eventual injury death. This study further reveals that there is a high prevalence of alcohol in nonfatal injury-related incidents among this cohort. There appears to be a strong relationship between repeated episodes of nonfatal injury with alcohol as a contributing factor and subsequently fatal events among this cohort. These nonfatal episodes should serve as warning signals and opportunities for the application of effective alcohol intervention strategies.

AGENCY SPONSOR: Office of Public Health

FEDERAL CONTACT: Leo J. Nolan

PHONE NUMBER: (301) 443-4245

PIC ID: 6451

PERFORMER ORGANIZATION: Billings Area Indian Health Service, Billings, MT

TOP OF DOCUMENT

NATIONAL INSTITUTES OF HEALTH

Contents

The Status of Biomedical Research Facilities 1996

Evaluation of the National Research Service Award Research Training Programs

National Heart, Lung, and Blood Institute Article Citation and Patent Reference Study: An Analysis of Article Citations and Patent References to Journal Articles Supported by the National Heart, Lung, and Blood Institute

Rare Diseases Clinical Research Data base

The National Survey of Laboratory Animal Use, Facilities, and Resources

Promoting Safe and Effective Genetic Testing in the United States: Final Report of the Task Force on Genetic Testing

National Cancer Institute: Characterization of User Population and User Satisfaction of the PDQ Data base-- Documentation for Tracking System

Bridges to the Future: Program Evaluation

Academic Research Instruments and Instrumentation Needs in the Biological Sciences

A Study to Determine the Changes of Physician Prescribing of Antimicrobial Agents, Omeprazole, and H2-Receptor Antagonists for Peptic Ulcer Disease Two Years Following the NIH Consensus Conference on Helicobacter pylori

Toxicology and Environmental Health Information Resources: The Role of the National Library of Medicine

Evaluation of the International Cooperative Biodiversity Groups (ICBG)

Federal Interagency Forum on Aging-Related Statistics: Data Base News In Aging

Finding the Balance: Report of the National Institute of Mental Health Intramural Research Program (IRP) Planning Committee

TITLE: The Status of Biomedical Research Facilities 1996

ABSTRACT: Since 1986, the National Science Foundation and the National Institutes of Health (NIH) have collected and reported to the Congress biennial information on the quantity and quality of biomedical research space. This report describes findings from the 1996 Survey of Scientific and Engineering Research Facilities at Colleges and Universities. The survey collected data from colleges and universities with research and development (R&D) expenditures exceeding $50,000, from Historically Black Colleges and Universities (HBCUs) with any R&D expenditures, and from a sampling of academic and nonprofit research organizations that received extramural research funding from NIH. The report found that (1) institutions performing biomedical research devoted approximately 67.4 million net assignable square feet to this work; (2) roughly half of survey respondents classified their medical science research space and/or biological science research space as inadequate; (3) expenditures to construct, renovate, and repair biomedical research space declined for the first time since NSF and NIH began collecting data; (4) the largest proportion of new construction was funded by State and local governments, while almost half of all institutions funded their own repair/renovation projects; and (5) 36 percent of all institutions with biomedical research space reported repair renovation projects that had to be deferred because funds were not available. In addition, the survey reported that HBCUs: (1) comprised 12 percent of the Nation's research-performing institutions, but contained only 1.8 percent of the national biomedical research space; (2) reported that 8 percent of their biomedical research space needed major repair/renovation to be used effectively; (3) have experienced a steady decline in new construction funding since the first survey in 1986 which reported expenditures of $42 million, to $0.2 million in 1994-5. (Final report: variously paginated.)

AGENCY SPONSOR: Office of the Director

FEDERAL CONTACT: Charles L. Coulter

PHONE NUMBER: (301) 435-0766

PIC ID: 3562.2

PERFORMER ORGANIZATION: National Science Foundation, Arlington, VA

TITLE: Evaluation of the National Research Service Award Research Training Programs

ABSTRACT: This project had several objectives. First, it conducted an evaluation design study, developing a detailed plan for a comprehensive evaluation of career outcomes of pre- and post-doctoral trainees and fellows and of the National Research Service Act (NRSA) programs in which they have participated. The evaluation developed recommendations promoting approaches that are successful in order to improve the return on Federal investment in research training. The second objective of the project was to characterize the nature and quality of the training actually experienced by the present and former trainees and fellows and to differentiate between a good training program and simply having good trainees. The project's third objective was to tap the perceptions of National Institutes of Health (NIH) staff, present and former NRSA trainees and fellows, and university employees about the nature and impact of the training program. The study developed performance indicators so that future evaluations may be based on data that are consistent across separate awards. The final report for this study will not be available until the Evaluation of NIH Post-doctoral Trainees and Fellows is completed in the summer of 1998. See PIC ID No. 6285.

AGENCY SPONSOR: Office of the Director

FEDERAL CONTACT: Walter Schaffer

PHONE NUMBER: (301) 435-2770

PIC ID: 5571

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: National Heart, Lung, and Blood Institute Article Citation and Patent Reference Study: An Analysis of Article Citations and Patent References to Journal Articles Supported by the National Heart, Lung, and Blood Institute

ABSTRACT: The primary objective of all National Institutes of Health (NIH) activities is to improve the health of the American public. However, it is generally impossible to demonstrate the impact of basic research in terms of reductions in disease mortality or morbidity. Therefore, alternative measures must be relied upon. The objectives of this project were to (1) introduce the concept of examining patents as an additional measure for assessing the impact of NIH-supported research; (2) develop a methodology to use patents in evaluation research; and (3) apply this methodology to a National Heart, Lung, and Blood Institute (NHLBI) project in order to compare the outcomes for individual research project grants (R01s) funded under a Request for Applications (RFA) with individual research project grants funded from unsolicited applications. The methods, data bases, software, and results of this project will be used by other Institutes, Centers, and Divisions to evaluate their own funding mechanisms. The project examines the impact of NHLBI grants during the period 1982 through 1994 with respect to the number of biomedical research articles associated with grant programs, the number of subsequent citations to these articles, and the number of patent references to the original articles. The report finds that (1) about 12,500 grants resulted in over 90,000 research articles cited in over 2 million later articles; (2) 3,000 of the original articles are referred to in over 1,000 patents; (3) half of all cited articles are cited no more than 12 times over the 14-year period studied, although a few were cited over 1,000 times each; (4) two-thirds of articles referenced in patents appeared only once, although 3 articles are each referenced in over 20 patents; and (5) new RFAs result in higher than normal numbers of citations, but lower than normal number of patent references. (Final report: 11 pages, plus appendices.)

AGENCY SPONSOR: National Heart, Lung, and Blood Institute

FEDERAL CONTACT: Carl A. Roth

PHONE NUMBER: (301) 496-3620

PIC ID: 5577

PERFORMER ORGANIZATION: National Heart, Lung, and Blood Institute, Bethesda, MD

TITLE: Rare Diseases Clinical Research Data base

ABSTRACT: The Rare Disease Clinical Research Data base (RDCRD) has been established by the Office of Rare Diseases (ORD), Office of the Director, National Institutes of Health (NIH). NIH made a commitment to the rare disease community and to the Congress to develop a mechanism for identifying and monitoring rare disease research supported by the NIH. The data base is similar in format to the Physician Data Query (PDQ) System of the National Cancer Institute. The International Cancer Information Center of NCI provided assistance in the development of the data base. Contact was made with 520 principal investigators and the 75 General Clinical Research Centers managers to determine those clinical trials in active states. The ORD retrieved 291 protocols from ongoing clinical trials to be included in the data base. Information from these protocols has been placed in abstracts and will be available from ORD's home page on the Internet. This development activity will continue, and the next two phases will include the development of information from clinical trials receiving support from the voluntary patient support organizations and the pharmaceuticals and biotechnology industries. ORD will continue to seek information from newly supported clinical trials receiving support from NIH. Information on specific rare diseases is difficult to locate; and, in many cases, the public, confronted with penetrating the NIH information system, reported difficulties gaining access to it. The RDCRD will eliminate this problem by providing an easy-to-use interface between the user and the data base information system. The data base contains up-to-date information that is easily accessible to a diverse user population. This information will facilitate patient and physician access to this information and lead to improved recruitment of patients with rare diseases or conditions into NIH-sponsored clinical trials. (Final report: 8 pages.)

AGENCY SPONSOR: Office of the Director

FEDERAL CONTACT: Stephen C. Groft

PHONE NUMBER: (301) 443-6156

PIC ID: 6043

PERFORMER ORGANIZATION: Oak Ridge Associated Universities, Oak Ridge, TN

TITLE: The National Survey of Laboratory Animal Use, Facilities, and Resources

ABSTRACT: This report summarizes the results of the survey conducted by the National Institutes of Health (NIH) on animal-based research for fiscal year (FY) 1993. NIH is responsible for oversight of the nationwide infrastructure required to pursue basic biomedical and behavioral research including laboratory animals, as well as the resources and facilities required to ensure their availability, appropriate use, and humane care. This study updates information gathered in the last national survey conducted in 1978. For FY 1993, the report finds that (1) of the 626 respondents, roughly half were academic institutions, and half consisted of research laboratories, independent hospitals, and government agencies; (2) there is a total census of approximately 2.5 million animals; (3) facilities reported acquiring 8.3 million animals, of which 29 percent were acquired through the facilities' own breeding; (4) respondent facilities made approximately 9 million vertebrate animals available for research, education or training, and testing, and animal species were used predominantly for research; (5) among directors of animal care and use programs, 74 percent had a doctoral-level degree; (6) among respondent facilities, 34 percent reported having space in need of renovation, 15 percent reported having space in need of replacement, and enclosed animal housing was the category most frequently cited as in need of replacement; and (7) the 559 respondents who provided cost information spent $521 million on animal care and reported that their costs had increased since FY 1991 by 25 percent. This report contributes to the understanding of the effects of Federal, State, and local laws, regulations, and policies enacted to ensure humane care and to regulate the supply and cost of laboratory animals. See also PIC ID Nos. 1506 and 2675. (Final report: 55 pages.)

AGENCY SPONSOR: National Center for Research Resources

FEDERAL CONTACT: Anne Summers

PHONE NUMBER: (301) 496-5572

PIC ID: 6048

PERFORMER ORGANIZATION: Analytical Sciences, Incorporated, Durham, NC

TITLE: Promoting Safe and Effective Genetic Testing in the United States: Final Report of the Task Force on Genetic Testing

ABSTRACT: Genetic testing is the analysis of human DNA, RNA, chromosomes, proteins, and certain metabolites in order to detect heritable diseases. The Task Force on Genetic Testing was created by the NIH-DOE Joint Working Group on Ethical, Legal, and Social Implications of Human Genome Research (ELSI Working Group) to review genetic testing in the United States and to design policies to minimize the potential negative effects of genetic testing. Their final report recommends that (1) an advisory committee be established on genetic testing in the Office of the Secretary to implement the Task Force recommendations and to determine which genetic tests require stringent scrutiny; (2) protocols for the development of genetic tests that can be used to predict future disease in healthy people receive the approval of an institutional review board (IRB) when the intention is to make the test readily available for clinical use; (3) test developers submit their validation and clinical utility data to external reviewers as well as to interested professional organizations in order to permit informed decisions about their routine use; (4) the newly created genetics subcommittee of the Clinical Laboratory Improvement Advisory Committee consider the creation of a specialty in genetics that would encompass all predictive tests that satisfy criteria for stringent scrutiny; (5) curricula on genetics be incorporated into medical school and residency training; (6) hospitals and managed care organizations require evidence of competence before permitting providers to order predictive genetic tests or counsel about them; and (7) physicians have access to accurate information to enable them to diagnose rare diseases, to know where to turn for assistance in clinical and laboratory diagnosis, and to locate laboratories that competently test for rare diseases. (Final report: 88 pages, plus appendices.)

AGENCY SPONSOR: Office of the Director

FEDERAL CONTACT: Jean McKay

PHONE NUMBER: (301) 594-2740

PIC ID: 6090

PERFORMER ORGANIZATION: Johns Hopkins Medical Institution, Baltimore, MD

TITLE: National Cancer Institute: Characterization of User Population and User Satisfaction of the PDQ Data base-Documentation for Tracking System

ABSTRACT: This study examines the user population and users' satisfaction with the Physician Data Query (PDQ) data base. PDQ was originally targeted to physicians only, but the information included in the data base was recognized to be of value to the larger community of nurses, researchers, health educators, and librarians, as well as patients, their families, and friends. PDQ is distributed to these audiences by the original on-line services, CD-ROM, facsimile, hard copy, and electronic networks, including the Internet. The users surveyed for the study include both primary users, or those who access the PDQ data base to fulfill their own research needs, and secondary users, who only conduct searches for others. The report finds that (1) the data base is widely and repeatedly used and is highly regarded; (2) physicians, other health care professionals, and librarians/information specialists predominate among the user population; (3) patients and their families and friends are also recipients of the PDQ information, even when the National Cancer Institute (NCI) toll-free information center phone service is excluded; (4) users consistently see PDQ as an authoritative and dependable source of cancer information; (5) no sections of the data base are negatively regarded or go unused; and (6) many users share the information they access through PDQ with others. Several recommendations are developed as a result of this report. The report recommends that NCI's International Cancer Information Center should (1) establish formal licensing structures to strengthen its ability to require access to users from the vendors, (2) repeat this survey within three years, (3) consider creating a CD-ROM version of the data base on its own in order to reduce costs to consumers, and (4) work with distributors and intermediaries to develop mechanisms for gathering data on user access. (Final report: variously paginated, plus appendices.)

AGENCY SPONSOR: National Cancer Institute

FEDERAL CONTACT: Bonnie J. Harding

PHONE NUMBER: (301) 496-7406

PIC ID: 6092

PERFORMER ORGANIZATION: Aspen Systems Corporation, Silver Spring, MD

TITLE: Bridges to the Future: Program Evaluation

ABSTRACT: In February 1992, the National Institute of General Medical Sciences (NIGMS) and the National Institutes of Health Office of Research on Minority Health announced the implementation of the Baccalaureate and Doctoral Bridge programs. The objective of the Baccalaureate Bridge program is to facilitate the transition of minority students at two-year colleges into colleges with baccalaureate degree programs in the sciences. The Doctoral Bridge program seeks to facilitate the transition of students into Ph.D. programs when they complete the M.S. degree. Two program outcomes that are particularly relevant to the goals of the Bridges program were examined in this study: (1) the transition of students from two-year to four-year institutions, or from M.S. to Ph.D. programs; and (2) the retention of students in the educational pathway both prior to and after making the transition from one institution to another. Although the quantity and quality of the comparison data is not high, rates of transfer among Bridges students appear to be higher than rates among other students. Success in completing the baccalaureate degree among those students who have transferred is high, and is comparable to the rate found among other minority students in an evaluation of the NIGMS Minority Access to Research Careers program. At the baccalaureate level, approximately 70 percent of the students are majoring in such fields as biology, chemistry and biochemistry, molecular biology, and microbiology. From the current data, it is difficult to develop good estimates of the eventual outcomes of participants in the Doctoral Bridges program. The early cohorts of students were small. Of the 21 students in the first two cohorts, about half have transferred to a doctoral program. While the rate of Bridges student attrition from graduate programs is below the national average, it is still too soon to predict their rate of completing their doctoral programs.

AGENCY SPONSOR: National Institute of General Medical Sciences

FEDERAL CONTACT: James Onken

PHONE NUMBER: (301) 594-2764

PIC ID: 6093

PERFORMER ORGANIZATION: Turner Consulting Group and TYC Associates, Rockville, MD

TITLE: Academic Research Instruments and Instrumentation Needs in the Biological Sciences

ABSTRACT: This report examines the fourth cycle of a national instrumentation survey concerning scientific research instruments and the academic units in which they are found for a range of scientific fields. Conducted in 1994, the survey analyzes data from two distinct populations of institutions. The first population consists of 318 institutions performing more than $3 million in research and development (R&D) in 1991. The second population consists of 44 historically black colleges and universities (HBCUs) that reported R&D expenditures in 1994. The study, in results generalized to the first group, finds that in 1993 (1) expenditures for biological research instrumentation totaled $283 million; (2) Federal funding as a percent of total funds for the purchase of biological research instrumentation declined from 48 percent in 1983 to 39 percent: NIH was the single largest Federal funding source providing $85 million; (3) 67 percent of respondents reported an increase in instrumentation needs between 1992 and 1994; (4) the price tag for the three highest priority research instruments was $363 million; (5) 20,978 research instruments in the biological sciences had a purchase price of $20,000 or more; and (6) principal investigators report that only 28 percent of the biological instruments were state-of-the-art in meeting researchers' needs. For HBCUs, the study finds that (1) the purchase price of research equipment in 1993 was $4.9 million, of which 48 percent was spent on maintenance, repair, and operation costs of the instrumentation; (2) Federal funding provided 85 percent of funding, $4.2 million, to purchase research instrumentation, and NIH was the largest source of funds; and (3) 65 percent of respondents reported inadequate research equipment. See also PIC ID Nos. 2382, 2382.1, and 4659. (Final report: 99 pages, plus appendices.)

AGENCY SPONSOR: Office of the Director

FEDERAL CONTACT: Paul Seder

PHONE NUMBER: (301) 496-4418

PIC ID: 6121

PERFORMER ORGANIZATION: National Science Foundation, Arlington, VA

TITLE: A Study to Determine the Changes of Physician Prescribing of Antimicrobial Agents, Omeprazole, and H₂-Receptor Antagonists for Peptic Ulcer Disease Two Years Following the NIH Consensus Conference on Helicobacter pylori

ABSTRACT: This study evaluates physicians' prescribing patterns for the treatment of peptic ulcer disease (PUD). Its particular focus is whether recommendations of a February 1994 National Institutes of Health (NIH) Consensus Development Conference advocating the use of antimicrobial agents for peptic ulcer patients with Helicobacter pylori infection were coincident with a change in physician prescribing patterns. The study uses computerized Pennsylvania Medicaid administrative data from January 1993 through February 1996 to evaluate prescribing patterns in the year prior and the two years following the NIH conference (12,737 patient records in all). Seven major therapeutic drug classes were analyzed: antimicrobial agents, H₂-receptor antagonists, antisecretory agents, bismuth subsalicylate, sucralfate, misoprostol, and antacids. The report finds that (1) only 10 percent of prescriptions contained an antimicrobial agent, while 62 percent were for an H₂-receptor antagonist only; (2) the two- and three-drug regimens recommended by the NIH conference were infrequently prescribed (1.1 percent and 0.9 percent, respectively); (3) younger and non-white PUD patients were more likely to receive a prescription for antimicrobial agents than older and white patients; (4) duodenal ulcer patients were more likely than non-duodenal ulcer patients to receive an antimicrobial prescriptions; and (5) the prescription of antimicrobial agents for the treatment of PUD significantly increased from 7 percent to 14.2 percent over the study period, while prescriptions for H₂-receptor antagonists significantly decreased. The report concludes that, while there were increases in the prescription of antimicrobial agents to treat PUD over the study period, these could not be directly attributed to the NIH conference. Physicians do not appear to be using national recommendations.

AGENCY SPONSOR: National Institute of Diabetes, Digestive and Kidney Diseases

FEDERAL CONTACT: Charles Sherman

PHONE NUMBER: (301) 496-5641

PIC ID: 6284

PERFORMER ORGANIZATION: Medical Technology and Practice Parameters Institute, Washington, DC

TITLE: Toxicology and Environmental Health Information Resources: The Role of the National Library of Medicine

ABSTRACT: This report examines the utility and accessibility of the National Library of Medicine's (NLM) Toxicology and Environmental Health Information Program (TEHIP). The TEHIP program encompasses 16 online data bases that contain bibliographic and factual information on environmental contaminants, including chemical properties, carcinogenicity, exposure levels, adverse health effects, emergency treatment protocols, and Federal regulations. The report notes that Federal appropriations for the TEHIP program have remained relatively constant over the past 29 years. However, the program's reimbursements from other agencies for collaborative projects have fluctuated. The report also notes that the TEHIP data base represents only a small subset of the many data bases containing information related to toxicology and environmental health. With this in mind, the report recommends that NLM consider expanding its traditional library services by organizing and cataloging the full spectrum of online toxicology and environmental health information resources. The report also recognizes that the health professional community does not have homogenous information needs and that there is wide variation in access to online data bases and other information resources. It recommends that the TEHIP program should set priorities that would allow efforts to be focused on those data bases that meet the information needs of the greatest number of health professionals and that NLM's training and outreach efforts be increased to improve awareness of the TEHIP program. The report notes that access to the data bases and difficulties in navigating the user interfaces are the two primary factors limiting use of the TEHIP data bases. The report also provides recommendations on programmatic issues and future directions for the TEHIP program. (Final report: 139 pages, plus appendices.)

AGENCY SPONSOR: National Library of Medicine

FEDERAL CONTACT: Susan Buyer

PHONE NUMBER: (301) 496-2311

PIC ID: 6534

PERFORMER ORGANIZATION: National Academy of Sciences, Institute of Medicine, Washington, DC

TITLE: Evaluation of the International Cooperative Biodiversity Groups (ICBG)

ABSTRACT: This report evaluates the International Cooperative Biodiversity Groups (ICBG) program. Funded by the National Institutes of Health and the National Science Foundation, the ICBG is a program designed to protect human health, advance the conservation of biologically diverse ecosystems, and promote sustainable economic development through the search for drugs to control diseases of concern to both developed and developing countries. Since 1993, the program has funded five individual cooperative biodiversity groups around the world. The study design attempts to (1) accommodate the varied and nonstandard implementations of the ICBG concept, (2) obtain broad coverage in terms of respondents, (3) maximize response rates and data quality, (4) minimize burden on ICBG Group Leaders and other respondents, and (5) incorporate cost and time efficiencies wherever possible. Among the benefits for source countries, the report finds (1) that scientific capacity has been developed through training and equipment, (2) development of economic activities, (3) equitable financial return and the protection of intellectual property rights of source-country people, and (4) the potential sustainability of the individual ICBG projects is one of the strengths of the overall program. The report recommends that the ICBG program (1) develop systems of technical assistance based on a combination of outside expertise and lessons learned across programs; (2) improve communications among the ICBGs; (3) consider developing a cross-ICBG data base that includes work being done in collection, extractions, and drug development techniques; and (4) bring in the expertise of social scientists and source-country legal experts to the ICGB to ensure that the consensus-building, benefit sharing, and economic goals of this project are achieved as well. (Final report: 177 pages, appendices bound separately.)

AGENCY SPONSOR: Fogerty International Center

FEDERAL CONTACT: J. Kirby Weldon

PHONE NUMBER: (301) 496-2571

PIC ID: 6641

PERFORMER ORGANIZATION: Battelle Corporation, Arlington, VA

TITLE: Federal Interagency Forum on Aging-Related Statistics: Data Base News In Aging

ABSTRACT: This annual report highlights the aging-related surveys and products currently available from member agencies of the Federal Interagency Forum on Aging-Related Statistics. All Federal agencies are invited to contribute to the report and to participate in the Forum. Contributors to this report are (1) Department of Agriculture: Economic Research Service; (2) Department of Commerce: Bureau of the Census; (3) Department of Health and Human Services: Administration on Aging, Centers for Disease Control and Prevention, National Institutes of Health (NIH), Health Care Financing Administration, Agency for Health Care Policy and Research, and Office of the Assistant Secretary for Planning and Evaluation; (4) Department of Housing and Urban Development: Office of Policy Development and Research; (5) Department of Labor: Bureau of Labor Statistics; (6) Department of Veterans Affairs; (7) Federal Reserve System, Board of Governors; (8) Social Security Administration; and (9) the United Nations. Each survey or data base is listed with its sponsoring organization, purpose, description, current activity, plans for future activity, public use data files, and associated reports, if any. The report ends with a tabular listing of the data bases and surveys described in the report, which gives the latest year data were collected or whether data collection is ongoing, the year for which the most recent data were released, whether public use tapes or printed reports are planned or available, and information on the person to contact about the data base or survey. (Final report: 117 pages.)

AGENCY SPONSOR: National Institute on Aging

FEDERAL CONTACT: Gail Jacoby

PHONE NUMBER: (301) 496-3121

PIC ID: 6652

PERFORMER ORGANIZATION: Bureau of the Census, Washington, DC

TITLE: Finding the Balance: Report of the National Institute of Mental Health Intramural Research Program (IRP) Planning Committee

ABSTRACT: The Intramural Research Program (IRP) of the National Institute of Mental Health (NIMH) must be positioned to make scientific advances crucial to easing the burden of mental disorders. This report represents an active effort by the NIMH IRP Planning Committee to help the program make these advances. The Committee evaluated the rationale for the program's continuance, as well as its strengths and weaknesses. It solicited confidential letters from almost 1,000 IRP scientists and staff and met with National Institutes of Health (NIH) leadership, the IRP's past and present leadership, and advocacy organizations. It found strong justification for the continued existence of the IRP, as long as the research it produces is of the highest quality. The Committee also offered recommendations to revitalize the IRP. The report finds that (1) stable scientific leadership is crucial to revitalizing IRP science, thus the program must recruit a Scientific Director for the program; (2) the overall quality of IRP science has declined from its previous place at the top of the scientific endeavor-while there are some areas of research brilliance, some research is of lesser quality, according to members of the Board of Scientific Counselors and intramural and extramural scientists; (3) recruitment and retention of talented scientists is essential, but over the past five years, the program has recruited from outside of NIH only one of its four newly-tenured scientists; (4) the quality of the program's intramural training efforts is uneven; (5) the program should address the fact that post-doctoral fellows have difficulty competing for positions when they leave NIH due to a lack of sufficient grantsmanship and teaching experience; and (6) NIMH's IRP's clinical research program needs to be sustained and revitalized.

AGENCY SPONSOR: National Institute of Mental Health

FEDERAL CONTACT: Emily Gause

PHONE NUMBER: (301) 443-4335

PIC ID: 6697

PERFORMER ORGANIZATION: National Institute of Mental Health, Rockville, MD

TOP OF DOCUMENT

OFFICE OF THE ASSISTANT SECRETARY FOR PLANNING AND EVALUATION

Contents

State Assisted Living Policy: 1996

Evaluation of the Minority Male Consortium for Violence Prevention

Determinants of AFDC Caseload Growth: Final Report

Informal and Formal Kinship Care

Approaches to Evaluating Welfare Reform: Lessons from Five State Demonstrations

Efforts by Child Welfare Agencies to Address Domestic Violence: The Experiences of Five Communities

Evaluability Assessment of Responsible Fatherhood Programs

Trends in the Well-Being of America's Children and Youth: '97

Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health

Registry of State-Level Efforts to Integrate Health Information

Implementing Welfare Reform Requirements for Teenage Parents: Lessons from Experience in Four States

Deriving State-Level Estimates from National Surveys: A Statistical Assessment and State Tabulations

Information Needs Associated with the Changing Organization and Delivery of Health Care: Summary of Perceptions, Activities, Key Gaps, and Priorities

Chapin Hall Center for Children at the University of Chicago: Administrative Data Guide

Iowa's Limited Benefit Plan

Role of Home- and Community-Based Services in Meeting the Health Care Needs of People with HIV/AIDS: Literature Review

Exploring the Feasibility of Linking Health Expenditure and Utilization Data

Public Health Laboratories and Health System Change

Framework for Assessing Insurer Responses to Health Care Market Changes

Establishing an Analytical Framework for Measuring the Role of Reinsurance in the Health Insurance Market

National Evaluation of Welfare-to-Work Strategies: Evaluating Two Welfare-to-Work Program Approaches: Two Year Findings on the Labor Force Attachment and Human Capital Development Programs in Three Sites

Improving Health in the Community: A Role for Performance Monitoring

Privacy and Health Research: A Report to the U.S. Secretary of Health and Human Services

State Regulatory Experience with Provider-Sponsored Organizations: Final Report

Indicators of Welfare Dependence and Well-Being: Interim Report to Congress

Setting the Baseline: A Report on State Welfare Waivers

Market Barriers to the Development of Pharmacotherapies for the Treatment of Cocaine Abuse and Addiction: Final Report

Consumer-Directed Personal Assistance Services: Key Operational Issues for State CD-PAS

Programs Using Intermediary Service Organizations

TITLE: State Assisted Living Policy: 1996

ABSTRACT: Assisted living facilities are an expanding source of supportive housing and services, and represent a new model of long-term care. Consumer choice and control are central to assisted living models, which seek to allow consumers to control key features of their living environments and to direct their own receipt of services. This study reviews the assisted living and board and care policies in each of the fifty States. Fifteen States have existing licensure regulations for assisted living facilities and an additional nine States are developing regulations. Twenty-two States reimburse or plan to reimburse assisted living as a Medicaid service and six States provide Medicaid payments for services in board and care settings. Thirteen states have created a task force or a process within a State agency to make recommendations for the development of assisted living rules. One of the major difficulties associated with assisted living is the lack of a common or standard definition of this kind of care. However, State approaches share common components, including that (1) assisted living is characterized as residential, rather than institutional; (2) health or medical services are provided, either by facility staff or through contracts with community agencies; (3) assisted living is a model emphasizing consumer or resident independence, autonomy, dignity, privacy, and decisionmaking; and (4) many States attempt to combine minimum standards of safety and quality of care while also allowing market forces to shape quality. State policies generally address three major issues: (1) the requirement for a living unit, (2) admission and retention criteria, and (3) the level of services to be provided. The report provides specific information about policies in each State. See also PIC ID Nos. 4719-4719.2 and 4719.4-4719.5. (Final report: 225 pages.)

AGENCY SPONSOR: Office of Disability, Aging, and Long-Term Care Policy

FEDERAL CONTACT: Robert Clark

PHONE NUMBER: (202) 690-6443

PIC ID: 4719.3

PERFORMER ORGANIZATION: Research Triangle Institute, Research Triangle Park, NC

TITLE: Evaluation of the Minority Male Consortium for Violence Prevention

ABSTRACT: This project evaluated the Office of Minority Health Minority Male Consortium for Family and Community Violence Prevention Programs. At the time of this study, 19 historically black colleges and universities (HBCUs) were participating in this program. Features of violence prevention program models at 13 historically black colleges and universities/family life centers (HBCUs/FLCs) were synthesized to determine approaches that could prevent or minimize violence in communities committed on or by minority males. The contractor (1) performed an extensive literature review; (2) collected and analyzed relevant program documentation and materials; (3) conducted focus groups; (4) examined target populations; (5) developed a site visit discussion guide; (6) selected 10 HBCUs for site visits; (6) developed and executed an analysis plan; (7) interpreted analysis findings, including defining program models and analyzing HBCUs' organizational capacity; (8) held briefings with Consortium leaders, members, and government personnel; (9) reviewed the overall organization, design, and implementation of the programs; and (10) evaluated prevention strategies adopted by the HBCU/FLCs. Violence prevention programs suffer from an inability to produce valid performance data demonstrating that prevention investments pay off in the long run. The report recommends that (1) without completely centralizing the design process, the Federal government and the Consortium should move toward a consensus-seeking information model in which individual schools are no longer free to pursue independent designs; and (2) constraints should be introduced on which program models are implemented by which schools. See also PIC ID Nos. 5789 and 5789.1. (Final report: 68 pages, plus appendices.)

AGENCY SPONSOR: Office of Program Systems

FEDERAL CONTACT: Patricia S. Hazard

PHONE NUMBER: (202) 690-8291

PIC ID: 5789.2

PERFORMER ORGANIZATION: Macro International, Incorporated, Calverton, MD

TITLE: Determinants of AFDC Caseload Growth: Final Report

ABSTRACT: This project examined the dynamics of Aid to Families with Dependent Children (AFDC) caseload growth and analyzed the effects of economic, demographic, and programmatic changes at the State level on States' AFDC caseload from the early 1980's to the present. Analysis of economic factors included measures of both wages and unemployment; analysis of demographic factors includes an emphasis on nonmarital childbearing. Analysis of program effects included changes arising from national welfare legislation, changes in State welfare parameters, State waivers, and changes in other programs such as Food Stamps and Medicaid. The project addressed the interaction of these factors by analyzing the economic, demographic, and programmatic factors simultaneously. Compared to earlier studies of AFDC caseload determinants, many of which are outdated, this study focused much more on State-level indicators, and included a more comprehensive set of economic, demographic, and programmatic factors. Recently, many State policymakers have expressed concern over the effect of future recessions or nonmarital birth rates on their caseloads, and this study is intended to be a valuable tool to State policymakers as they continue to assume greater responsibility for their AFDC programs and consider new innovations.

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Kelleen Kaye

PHONE NUMBER: (202) 401-6634

PIC ID: 5953

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Informal and Formal Kinship Care

ABSTRACT: A growing number of States prefer to place children who have been removed from their homes of origin in the home of a relative. This practice is known as kinship foster care, and is a growing component of family foster care. It comprises between one-third and one-half of all foster care days provided in the States with the largest foster care populations. Many children, however, live with relatives who are not their parents in an informal relationship. In all, about 2.15 million children lived in the care of relatives without a parent present in 1994. This study uses several data sources to examine the characteristics of informal kinship care arrangements and how they compare with those initiated through or subsidized by the State child welfare system. It considers national patterns, examines living arrangement patterns by State for FY 1990, compares formal and informal kinship care in four States, and uses Illinois data to provide a more focused picture of kinship care. The two-volume report finds that (1) kinship care is more prevalent in the South and in non-metropolitan areas, and for older children; (2) about two-thirds of kinship caregivers are the child's grandparents-about half are currently married, and over 85 percent of single kinship caregivers are women; (3) kinship caregivers are more likely to be currently unmarried, to be less educated, to be poor, and to receive social welfare benefits than are live-in parents; (4) nationally, just over 2 percent of children lived with relatives not their parents (this percentage ranges by State from under 1 percent to well over 3 percent) (5) higher levels of kinship care (as well as single mother households) appear to be direct products of higher levels of social disruption and family disorganization; and (6) while informal kinship care patterns were roughly similar in the four States examined, the use of formal kinship care varies widely. (Vol. I: 80 pages; Vol. II-tables and figures-58 pages plus appendices.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Laura Feig

PHONE NUMBER: (202) 690-5938

PIC ID: 6016

PERFORMER ORGANIZATION: University of Chicago, Chicago, IL; Urban Institute, Washington, DC

TITLE: Approaches to Evaluating Welfare Reform: Lessons from Five State Demonstrations

ABSTRACT: Most States have obtained waivers from Aid to Families with Dependent Children (AFDC) and Food Stamp Program rules in order to implement welfare reform. Between 1993 and 1996, the Clinton Administration approved waivers for 43 States. This project examined waiver demonstration projects in five States (California, Colorado, Michigan, Minnesota, and Wisconsin). Four of the five States used an experimental evaluation design, while the fifth used a quasi-experimental evaluation design. This two-volume report addresses issues in five areas: (1) the choice between an experimental and quasi-experimental evaluation design, (2) sample design, (3) implementation of experimental evaluations, (4) data collection, and (5) analysis methods. The report finds that: (1) most evaluations of State welfare reform demonstrations used an experimental design, which involved random assignment of cases to an experimental group subject to welfare reform or to a control group subject to pre-reform policies; (2) only one waiver (Wisconsin's) was approved with a quasi-experimental design, but this may become more common in a block grant environment; and (3) when designing a study sample, evaluation planners should ensure that the sample size is adequate and should design the sample so that applicant and recipient subgroups can support separate impact estimates (the sample should also be representative of the State as a whole). The report also finds that four aspects of the implementation of an experimental evaluation require special care: (1) the timing of the random assignment, (2) the method of the random assignment, (3) ensuring that control group policies remain unchanged, and (4) preventing experimental and control group cases from changing status. The report also discusses data collection and analysis. (Final report: Volume I: 128 pages, plus appendices; Volume II: 8 pages.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Audrey Mirsky-Ashby

PHONE NUMBER: (202) 401-6640

PIC ID: 6050

PERFORMER ORGANIZATION: Urban Institute, Washington, DC

TITLE: Efforts by Child Welfare Agencies to Address Domestic Violence: The Experiences of Five Communities

ABSTRACT: Child protective services (CPS) and domestic violence services historically have not been closely linked. However, child welfare agencies across the country are beginning to consider how families in their child protection caseload are affected by domestic violence and are reexamining their policies accordingly. This report documents how child welfare agencies in five communities (Massachusetts; Michigan; San Diego County, California; Hilo, Hawaii; and Oregon) are attempting to integrate domestic violence concerns into their services. The communities were chosen for study because they are extending their services well beyond educating staff in domestic violence concerns. The report finds that (1) child welfare agencies have initiated changes from different organizational points within their agencies and have taken different approaches to changing case practice; (2) these agencies cannot make appropriate changes without major and continuing collaboration with community stakeholders who work with domestic violence victims and perpetrators; (3) chief among the concerns raised is the need to refrain from actions that further endanger mothers and children; (4) changes will also benefit from close collaboration with police, civil and criminal courts, corrections, the schools, and local clinics and hospitals; and (5) ongoing technical assistance to CPS staff after initial training in domestic violence services appears to make a more lasting difference on workers' ability to put the content of training into practice on a daily basis. The report closes with a general discussion of (1) approaches to addressing domestic violence within child welfare agencies; (2) the role of the larger community; (3) more complex policy questions, such as whether a child's witnessing of domestic violence is itself abusive; and (4) evaluating the effectiveness of these efforts. (Final report: 148 pages.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Jerry Silverman

PHONE NUMBER: (202) 690-5654

PIC ID: 6155

PERFORMER ORGANIZATION: Urban Institute, Washington, DC

TITLE: Evaluability Assessment of Responsible Fatherhood Programs

ABSTRACT: Responsible fatherhood programs provide fathers, particularly those who are young, unmarried, and unemployed, with the support and skills to become active participants in their children's lives. An increased interest in fatherhood programs and a lack of information on their accessibility and effectiveness led to this evaluability assessment. This study visited five fatherhood program sites, selected as the most developed programs by HHS, in order to (1) assess the readiness of fatherhood programs for formal evaluation and identify obstacles to their evaluation in the future, and (2) illustrate the evaluation process and mechanisms that must be in place before a formal impact evaluation is undertaken. The project goal was to develop an evaluation design that could be used by HHS and other policymakers in the future to evaluate such programs. The report concludes that at this time formal program evaluation remains premature because the programs (1) are new and still at the stage of refining recruiting methods and program services, (2) lack automated systems for tracking and reporting on clients, and (3) serve a very small number of clients. In order to conduct a rigorous evaluation of fatherhood programs, the report suggests the development of (1) measurable outcomes; (2) defined service components and their hypothesized relationship to outcomes; (3) an established recruiting, enrollment, and participation process; (4) an understanding of the characteristics of the target population, program participants, and program environment; and (5) an ability to collect and maintain an adequate program size. The report concludes that to increase viability, fatherhood programs must (1) develop a core definition of what constitutes a responsible fatherhood program, (2) conduct process evaluations, (3) build basic management information system capacity, and (4) stabilize and enhance funding. (Final report: 114 pages, plus appendices.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Linda Mellgren

PHONE NUMBER: (202) 690-6806

PIC ID: 6159

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Trends in the Well-Being of America's Children and Youth: '97

ABSTRACT: This is the second edition of an annual report from HHS on the well-being of children and youth in the United States. The report's purpose is to provide the policy community, the media, and all interested citizens with a statistical overview describing the condition of children. The report focuses on trends in the well-being of children and youth in five key areas: population, family, and neighborhood; economic security; health conditions and health care; social development, behavioral health, and teen fertility; and education and achievement. The report, as it pertains to teenagers, finds (1) a decrease in the pregnancy rate among young women aged 15-19 since 1991; (2) an increase in alcohol, cigarette, marijuana, and cocaine use in the 1990's; (3) a modest gain in math and science proficiency by 17-year-old students since the early 1980's; (4) a sustained increase in mortality for black youth aged 15-19 since 1991, while there was a decrease in mortality rates for white youth; and (5) a steady increase in receipt of early prenatal care by teen mothers. Federal statistical data lack (1) standardized measures of social development and health-related behaviors for very young and pre-teenaged children; (2) indicators that reflect important social processes affecting child well-being that go on inside the family and within the neighborhood; and (3) sufficient information on child abuse and neglect, youth violent crime, day care quality, learning disabilities, and measures of children in institutionalized care. The Federal Interagency Forum on Child and Family Statistics has adopted a mandate to improve the statistical system. As data become available they will be incorporated into new editions of this report. (Final report: variously paginated.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Matthew Stagner

PHONE NUMBER: (202) 690-5653

PIC ID: 6170.1

PERFORMER ORGANIZATION: Urban Institute and Child Trends, Incorporated, Washington, DC

TITLE: Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health

ABSTRACT: The Panel on Performance Measures and Data for Public Health Performance Partnership Grants (PPGs) was established in order to examine the state-of-the-art in performance measurement for public health and to recommend measures that could be used to monitor the PPG agreements to be negotiated between each State and the Federal government. The panel considered performance measures in ten public health areas: chronic diseases, sexually transmitted diseases, human immunodeficiency virus infection and tuberculosis, mental health, immunization, substance abuse, sexual assault, disabilities, and emergency medical services. More than 3,200 measures were proposed to the panel through various outreach efforts. The panel used four guidelines for assessing the performance measures: (1) the measure should be specific and results oriented; (2) the measure should be meaningful and understandable; (3) data should be adequate to support the measure; and (4) the measure should be valid, reliable, and responsive. The measures that scored the highest are recommended for use in performance monitoring. They cover health status, social functioning, consumer satisfaction, and risk status. In assessing the adequacy of data for specific performance measures, the panel concluded that there are few available data sources that are ideal for performance monitoring. Many Federal efforts to collect health-related data provide national rates, but do not collect data that provide State-level rates, and much State-level data may not be comparable. The report also finds that many of the performance measures presented can and should be subdivided to focus on specific high-risk populations in a State. See also PIC ID No. 6200. (Final report: 54 pages, plus appendices.)

AGENCY SPONSOR: Office of Program Systems

FEDERAL CONTACT: Jeffery J. Koshel

PHONE NUMBER: (202) 401-8238

PIC ID: 6177

PERFORMER ORGANIZATION: National Academy of Sciences, Institute of Medicine, Washington, DC

TITLE: Registry of State-Level Efforts to Integrate Health Information

ABSTRACT: In 1996, HHS contracted The Lewin Group to develop a registry of projects that integrates health information systems at the State level. The registry is located on the World Wide Web at

http://aspe.hhs.gov/statereg/index.htm. The home page compiles, abstracts, and indexes summary information on data integration projects for all 50 States and the District of Columbia. The purpose of the registry is twofold. First, the registry gives Federal policymakers a central source for researching the types of integration activities being pursued by individual States. Second, the registry facilitates communication among States regarding data integration activities and related data policy activities. The project (1) provides a resource for State and local policymakers as they seek to identify others who have pursued information integration activities; (2) facilitates information sharing and collaboration across States; and (3) assists Federal policymakers in monitoring, assessing, and supporting State-level integration activities. Relevant State integration projects were identified through: (1) contacts with State officials responsible for hospital discharge data bases, vital statistics, Medicaid data, and any other major information system or data-related offices; (2) conferences on State data integration activities; (3) grantee information related to specific funds which target information activity; (4) State health department websites; and (5) recommendations from people interviewed. A contract with The Lewin Group has been issued to organize and edit the materials that are submitted and encourage biannual updates of the site. The data base is administered by the Office of the Assistant Secretary for Planning and Evaluation. Final report is a web site: http://aspe.hhs.gov/statereg/index.htm.

AGENCY SPONSOR: Office of Program Systems

FEDERAL CONTACT: Dale Hitchcock

PHONE NUMBER: (202) 690-5882

PIC ID: 6178

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Implementing Welfare Reform Requirements for Teenage Parents: Lessons from Experience in Four States

ABSTRACT: Federal welfare policy requires minor custodial parents receiving cash assistance to attend school and to live with their parents or in an adult-supervised setting; these requirements were established as part of the Personal Responsibility and Work Opportunity Reconciliation Act, which created the program for Temporary Assistance for Needy Families and abolished the Aid to Families with Dependent Children (AFDC) program. This two-volume report summarizes lessons learned from the operational experiences in four States (Arizona, California, Massachusetts, and Virginia) that implemented waivers requiring school attendance and certain living arrangements under the AFDC program. The report finds that (1) identifying teenage parents, particularly those who do not head their own cash assistance cases, can be a major challenge; (2) persistent staff training to correct errors, the use of alternative information sources to identify teenage parents, and establishing positive incentives for identifying and referring all teenage parents can help to address these challenges; (3) a range of education options, including GED programs, should be available; (4) programs must determine the focus and scope of case management in light of goals and costs; (5) monitoring school attendance increases welfare agency workload-sometimes substantially-and can complicate the relationship between welfare agencies and schools; (6) a very restrictive living arrangement requirement may create implementation difficulties; and (7) funding group homes may enable States to have fewer exceptions to the requirement, but despite their advantage for teen parents, many may not choose to reside in them. (Final report: Volume I, 50 pages; Volume II (case studies), 103 pages.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Kelleen Kaye

PHONE NUMBER: (202) 401-6634

PIC ID: 6228

PERFORMER ORGANIZATION: Mathematica Policy Research, Incorporated, Plainsboro, NJ

TITLE: Deriving State-Level Estimates from National Surveys: A Statistical Assessment and State Tabulations

ABSTRACT: The Department of Health and Human Services (HHS) is changing its focus from the national to the State level. Landmark legislation eliminating entitlement to Aid to Families with Dependent Children benefits and block granting it to the States was passed at the time of this report. This project is one component of a larger HHS strategy to generate valid and reliable State-level estimates, which can be used to measure the impact of social policy interventions. This project (1) appreciates the statistical issues involved in addressing policy and research questions at the State level with national survey data; (2) assesses the current capabilities of these major surveys to provide answers to these questions at the State level; and (3) presents options for obtaining valid, reliable State-level data from relatively minimal enhancements and changes to the design of the surveys. The study focused on three major Federal surveys: the Survey of Income and Program Participation (SIPP); the Current Population Survey (CPS); and the National Health Interview Survey (NHIS). These surveys were selected for their widespread applicability, use, and potential for analyzing policy issues. The report evaluates statistical concerns to determine the suitability and potential of using the selected surveys for State-level estimates and to identify differences between States. These technical concerns included variance estimation, standard errors, the construction of State-level weights, minimum sample sizes needed, design effects, pooling several years of data, and longitudinal issues (e.g., response rates and weighting). The report recommends periodic redesign efforts and other survey enhancements.

AGENCY SPONSOR: Office of Health Policy

FEDERAL CONTACT: George Greenberg

PHONE NUMBER: (202) 690-7794

PIC ID: 6380

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Information Needs Associated with the Changing Organization and Delivery of Health Care: Summary of Perceptions, Activities, Key Gaps, and Priorities

ABSTRACT: This report assesses unmet needs for supply-side information (information about health care providers and insurers) on the health system, focusing on helping the Department of Health and Human Services identify how information needs associated with a changing health care system are perceived by a wide variety of user groups and constituencies. Based on information gathered from over 50 individuals and a review of relevant literature and Web sites, the report examines perceived gaps, their causes, and the activities generated in response to those perceptions. It also considers, in a more comprehensive manner, information on a diverse group of 11 entities. The report finds that (1) the serious gaps in "supply-side" information on the health system have grown worse as the system has evolved, presenting major obstacles to stakeholders' ability to serve their constituency, address operational needs, and participate in policy decisions; (2) the growth of managed care and consolidation has contributed to the need for supply-side information, as has the growing interest in accountability, competition, and cost-containment; (3) stakeholders say they need better information on health insurers and plans and the ways in which providers are linked to plans and integrated systems; and (4) better transaction-level data are needed, as are data from the State and local levels. The report identifies 23 ongoing efforts within the private sector, States, and foundations to address these gaps, and studies 11 of these more thoroughly. The report concludes that private sector stakeholders and States alone cannot sufficiently address supply-side information needs, giving importance to Federal leadership and convening functions. (Final report: 61 pages, plus appendices.)

AGENCY SPONSOR: Office of Health Policy

FEDERAL CONTACT: Linda Sanches

PHONE NUMBER: (202) 690-7233

PIC ID: 6388

PERFORMER ORGANIZATION: Mathematica Policy Research, Incorporated, Plainsboro, NJ

TITLE: Chapin Hall Center for Children at the University of Chicago: Administrative Data Guide

ABSTRACT: This contract produced a guide to the administrative data resources regarding health, social services, and related programs available through the Chapin Hall Center for Children at the University of Chicago. Each data section in the guide provides an overview of a particular dataset. The sections offer enough information for the researcher to gain a general understanding of the available variables and the types of analysis the data will support. Data Description and Limitation entries document the larger issues involved with analyses of the data and should not be considered exhaustive. The guide also includes information to assist the researcher in interpreting the data sections. The guide was designed for use by an audience of informed policy analysts in order to consider what data are available to answer particular policy questions. It is not intended that the guide be a detailed programmer's reference book to every code used in each data field, but rather that it describe the data sets available, the basic structure and topics covered by each, and the ability to match clients between the different programs' data. The goal of this guide is to enable the Office of the Assistant Secretary for Planning and Evaluation's policy and research analysts to better design relatively short-turnaround analyses on a variety of topics of current policy interest. The guide may also be useful to other researchers. (Final report: 110 pages, plus appendices.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Laura Feig

PHONE NUMBER: (202) 690-5938

PIC ID: 6389

PERFORMER ORGANIZATION: Chapin Hall Center for Children, University of Chicago, Chicago, IL

TITLE: Iowa's Limited Benefit Plan

ABSTRACT: Iowa's Family Investment Program (FIP) replaced Aid to Families with Dependent Children in 1993. FIP participants who are able-bodied and not caring for children, yet who do not develop and carry out a Family Investment Agreement (FIA), are sanctioned with assignment to the Limited Benefit Plan (LBP); some clients may also choose LBP over FIP. LBP originally provided 3 months of benefits at FIP levels, followed by 3 months of reduced levels and 6 months of no cash benefits; the program was later changed to eliminate the original 3-month period. At the end of the 6-month period, the client can reapply for FIP, but must meet its requirements. This report present finding from a study of the original LBP, showing how some families enter the LBP and how their well-being changes after benefits are cut. It examines records for over 4,200 cases assigned to LBP records during 6 months in 1994 and 1995, a survey of 137 cases whose cash benefits were terminated, and case studies of 12 LBP families. The report finds that (1) most clients are assigned to LBP for failing to develop and sign an FIA, rather than for failing to carry it out; (2) slightly more than half of LBP clients subsequently meet FIP requirements and are reassigned to the program; (3) about half of LBP clients whose cash benefits have been terminated are employed immediately after cash benefits end, but about half are not; (4) 40 percent of these clients increase their monthly incomes by an average of $496 when their benefits end, but 47 percent see a decrease in their monthly incomes averaging $384; and (5) other government programs remain important sources of support for LBP families whose cash benefits have been terminated. The report notes that there are several features of the LBP program that mitigate against extreme deprivation when cash assistance has been terminated, and notes that the absence of these safeguards in a national welfare reform program could result in extreme hardship for many families. (Final report: 132 pages plus appendix.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Kelleen Kaye

PHONE NUMBER: (202) 401-6634

PIC ID: 6390

PERFORMER ORGANIZATION: Mathematica Policy Research, Incorporated, Washington, DC

TITLE: Role of Home- and Community-Based Services in Meeting the Health Care Needs of People with HIV/AIDS: Literature Review

ABSTRACT: This report provides an overview of the literature describing the delivery and financing of home- and community-based services for people with AIDS, focusing on the literature published since 1991. The literature review finds that (1) since the start of the epidemic, the AIDS population has been decreasingly made up of white male homosexuals and increasingly made up of people of color with low incomes and low educational levels; (2) new AIDS treatments combining protease inhibitors and other antiretroviral medications may increase life expectancy among some people with AIDS but annual costs range from $9,000 to $18,000 per person; (3) the lifetime cost of AIDS treatment has been rising, since drugs have been increasing life expectancy even before the introduction of protease inhibitors; (4) AIDS patients who experience periods of disability require nursing and support services to complement medical care, including assistance with daily living activities, child care, transportation, help applying for public assistance, and the like; (5) the increasing number of low-income people with AIDS places a heavier burden on public programs such as Medicaid and Ryan White CARE Act funding; (6) the Medicare program has borne a relatively small proportion of the burden for AIDS care, but this may increase as life expectancy increases; (7) rigorous evaluation of programs that provide home- and community-based services to people with AIDS appears to be lacking; (8) the literature provided little data describing current levels of use of and cost for home- and community-based services by people with AIDS; and (9) there is little information about how effectively managed care is meeting the need for these services. The review examines literature on the provision of these services to these elderly to draw inferences for their application to the AIDS population. See also PIC ID No. 6418. (Final report: 92 pages.)

AGENCY SPONSOR: Office of Disability, Aging, and Long-Term Care Policy

FEDERAL CONTACT: Gavin Kennedy

PHONE NUMBER: (202) 690-6443

PIC ID: 6418.1

PERFORMER ORGANIZATION: Mathematica Policy Research, Incorporated, Plainsboro, NJ

TITLE: Exploring the Feasibility of Linking Health Expenditure and Utilization Data

ABSTRACT: The Health Resources and Services Administration (HRSA) has the responsibility for assessing the adequacy of the Nation's supply of various health professions. HRSA commissioned a "General Services Demand Model" (GSDM) in 1994 to permit State-specific estimates of health utilization, which it then inputs into State-specific provider requirements forecasts. HRSA's GSDM predicts, for each State and the District of Columbia: bed days in short-term general hospitals for the under-65 and over-65 populations, outpatient visits, and emergency visits. HRSA then takes these predictions and estimates requirements for various health professions for each State. The estimates are typically based on assumptions about appropriate provider staffing/utilization ratios; for example, each primary care physician can perform 3,000 office visits in a year. From the data and assumptions, a set of State-specific provider requirement forecasts can be generated. In general, these forecast supply and demand for physicians by speciality type, as well as for registered nurses. The Health Care Financing Administration compiles national health account expenditures for hospital services, physician services, prescription drugs, and other areas. The goal of these projects is to link these two data bases over a 2 to 3-year period and then use the model to simulate the utilization and cost savings potential of reorganizing the delivery of care (for example, using HMO specialist/generalist staffing ratios versus the current fee-for-service norms). The ultimate model could be used to analyze utilization and expenditure patterns. We know, for example, that managed care enrollees, other things being equal, have 20 percent fewer beds and 10 percent fewer specialist visits than indemnity enrollees. The ultimate model could provide an estimate of utilization and cost-savings effects of HMOs entering and achieving a 30 percent share in, for example, Alabama over the next 5 years.

AGENCY SPONSOR: Office of Health Policy

FEDERAL CONTACT: George Greenberg

PHONE NUMBER: (202) 690-7794 or 690-7804

PIC ID: 6425

PERFORMER ORGANIZATION: Urban Institute, Washington, DC

TITLE: Public Health Laboratories and Health System Change

ABSTRACT: All 50 States and the District of Columbia operate their own Public Health Laboratories (PHLs). Changes in the private health care delivery system, especially the growth of managed care organizations (MCOs), hospital consolidations, and large commercial laboratories are causing public health professionals to examine the viability of publicly funded laboratories in this new environment. The study identifies aspects of the public health infrastructure critical for developing assessment and evaluation tools. The report includes a literature review, interviews with stakeholders, an informal poll of State PHL directors, and detailed case studies of three States that have PHLs with relationships with MCOs and/or private laboratories. The report concludes that (1) PHLs are part of the broader public health infrastructure and, as such, must respond to changes in the health care marketplace; (2) services that are not commercially viable within the private sector, and those that are critical to identifying emerging diseases, should remain in the domain of PHLs; (3) assurance of quality and consistency of laboratory testing are critical functions of PHLs; and (4) policy development in laboratories needs to address the growth of managed care and changes in the health care delivery system. The study recommends increased Federal guidance in (1) assessing the regionalization of laboratory services, (2) supporting information infrastructure development, and (3) facilitating communication between private and public sectors. (Final report: variously paginated.)

AGENCY SPONSOR: Office of Health Policy

FEDERAL CONTACT: Cheryl Austein Casnoff

PHONE NUMBER: (202) 690-6102

PIC ID: 6458

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: A Framework for Assessing Insurer Responses to Health Care Market Changes

ABSTRACT: Over the past 15 to 20 years, the health insurance market has changed dramatically. Where once a few large commercial carriers dominated the market with fee-for-service plans, there are now dozens of plans in most markets. In highly competitive markets, plans and carriers develop more sophisticated strategies and arrangements to identify and retain target markets. In order to successfully integrate government programs and develop public policy, insurers' business strategies need to be better understood. Understanding insurers' strategies is important for policy analysis because insurers are often regulated entities; and insurers' responses to changing market and regulatory conditions affect both patients' and providers' opportunities and outcomes. This report (1) identifies insurer and insurance market characteristics that are likely to be important determinants of insurers' response to policy changes (such as insurance market reform) and changes in local market conditions (such as aggressive entry of managed care plans), (2) identifies potential data sources and data needs for future empirical work, and (3) provides a better understanding of the operation of insurance markets and how insurers make business decisions based on the demand for insurance from employers and individuals. In order to improve the analysis of health insurance markets, the report recommends (1) measuring changes in the relative density of the health plan contour "web," (2) tracking premiums and loading factors, (3) monitoring the self-insured market, and (4) watching organized buyers. (Final report: 44 pages.)

AGENCY SPONSOR: Office of Health Policy

FEDERAL CONTACT: Stephen Finan

PHONE NUMBER: (202) 690-7387

PIC ID: 6572

PERFORMER ORGANIZATION: Urban Institute, Washington, DC

TITLE: Establishing an Analytical Framework for Measuring the Role of Reinsurance in the Health Insurance Market

ABSTRACT: Reinsurance is widely used by underwriters of health benefits to limit their risk exposure. Reinsurance contracts can be structured in many ways, and insurers can use it to limit their risk for an individual contract or exposure, for a group of contracts or exposures, or for a whole portion of the insurer's business. This report presents a limited analytical study of the level of risk borne by reinsurers in three markets: the indemnity insurance market, the managed care market, and the self-insured employer benefit plan market. The study focuses on three States using public and private data sources, including surveys, published private data, and information from filings made by private insurance companies. The purpose of the study is to determine to what extent these data can be used to measure the role of reinsurance in the various markets and to suggest ways whereby better assessments of reinsurance might be made. The report was generally unable to identify either data sources or existing research allowing quantification of the risk being ceded to reinsurers in the three markets. While a substantial amount of information is available about reinsurance in indemnity (non-HMO) health insurers, the data are aggregated in ways that make it difficult to specifically identify transactions related to primary health insurance offered to groups and individuals. Information from State HMO insurance filings is more useful, but lack of detail and inconsistencies in the way that HMOs report the information in their annual statements reduces its analytic potential. Finally, despite the substantial amount of information available about the prevalence of self-funding and the use of reinsurance by self-funded employers, there are virtually no reliable data about the level of risk being transferred to reinsurers in these arrangements. The report closes with suggestions for enhancing the data available on this subject. (Final report: 23 pages, plus appendices.)

AGENCY SPONSOR: Office of Health Policy

FEDERAL CONTACT: Stephen Finan

PHONE NUMBER: (202) 690-7387

PIC ID: 6573

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: National Evaluation of Welfare-to-Work Strategies: Evaluating Two Welfare-to-Work Program Approaches: Two Year Findings on the Labor Force Attachment and Human Capital Development Programs in Three Sites

ABSTRACT: This report is part of a larger study called the National Evaluation of Welfare-to-Work Strategies, formerly known as the JOBS Evaluation, which has randomly assigned more than 55,000 individuals in seven sites to groups eligible for specific welfare-to-work programs and to control groups that do not participate in these programs. This study compares two distinct welfare-to-work strategies-labor force attachment (LFA) and human capital development (HCD)-at each of three of sites: Atlanta, GA; Grand Rapids, MI; and Riverside, CA. At each site, Aid to Families with Dependent Children applicants were randomly assigned to one of three groups: a group subject to the LFA program, a group subject to the HCD program, or a control group not subject to any welfare-to-work program. This report presents findings on the implementation, participation patterns, and costs of the two types of programs operated at each site. Additionally, the report assesses the two program approaches in promoting employment and reducing welfare expenditures after 2 years. The study finds that (1) the LFA programs increased participation in the job search and HCD programs increased participation in adult basic education; (2) welfare sanction rates for nonparticipation were higher in the LFA and HCD programs than in previously studied programs, although higher sanction rates were not associated with higher rates of eventually participating in program activities; (3) the HCD programs cost about twice as much as the LFA programs; (4) both programs increased individuals' 2-year cumulative employment and earnings, and (5) both programs reduced welfare expenditures within the 2-year followup period. The report concludes that a period of 2 years is insufficient to make a credible comparison of the two approaches. See also PIC ID series 5776 and PIC ID Nos. 6576 and 6576.2. (Executive summary: 36 pages; final report: 276 pages, plus appendices.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Audrey Mirsky-Ashby

PHONE NUMBER: (202) 401-6640

PIC ID: 6576.1

PERFORMER ORGANIZATION: Manpower Demonstration Research Corporation, New York, NY

TITLE: Improving Health in the Community: A Role for Performance Monitoring

ABSTRACT: This report draws on lessons from a number of current community health efforts to outline the elements of a community health improvement process, discusses the role that performance monitoring can play in this process, and proposes tools to help communities develop performance indicators. A recent emphasis on the interdependence of stakeholders requires collaborative efforts to improve community health: performance monitoring (a continuing process of selecting indicators that can be used to measure the process and outcomes of an intervention and making the results available to the community) has gained increasing currency as a way to do so. The report finds that a community health improvement process (CHIP) can be an important tool for developing a shared vision and supporting a planned and integrated approach to improving community health. The report suggests that a CHIP should include two principal interacting cycles based on analysis, action, and measurement: (1) a problem identification and prioritization cycle; and (2) the analysis and implementation cycle. This system differs from standard models primarily because of its emphasis on measurement to link performance and accountability on a community-wide basis. The report offers recommendations for implementing the CHIP concept: (1) communities should base a health improvement process on a broad definition of health and a comprehensive conceptual model of how health is produced within the community; (2) a CHIP should develop its own set of specific, quantitative performance measures and should seek a balance between strategic opportunities for long-term health improvement and short-term goals; (3) State and local health agencies should assure that an effective community health improvement process is in place in all communities; and (4) the Public Health Service should provide support and guidance in these efforts. (Final report: 179 pages, plus appendices.)

AGENCY SPONSOR: Office of Program Systems

FEDERAL CONTACT: James Scanlon

PHONE NUMBER: (202) 690-7100

PIC ID: 6583

PERFORMER ORGANIZATION: National Academy of Sciences, Institute of Medicine, Washington, DC

TITLE: Privacy and Health Research: A Report to the U.S. Secretary of Health and Human Services

ABSTRACT: This report (1) identifies privacy issues surrounding research on personally identifiable health data, paying special attention to the international aspects; (2) reviews the ethical, legal, and general social context surrounding the privacy and confidentiality of health data; (3) describes relevant privacy-protection practices and problems, and identifies emerging issues; (4) analyzes the implications of the new European Union Data Privacy Directive and related policy and legal changes; and (5) recommends policy approaches and technical processes for ensuring that the privacy of individuals is respected. The report examines four major groups of issues that are growing rapidly in scale and complexity, and that must be urgently attended to: (1) secondary uses of data and data linking, (2) research on private-sector health data, (3) cybersecurity, and (4) genetic privacy. The report finds that (1) as data bases are maturing and increasing in size and quality, their appeal as research resources is also growing; (2) if personally identifiable data must be used in secondary research, consent will become the most difficult issue; (3) for many health data held in the private sector, few legal controls apply in theory or are enforced in practice regarding such matters as data-subject consent, public notification, or transfer of the data for secondary study; (4) protection of the confidentiality and security of health data, especially electronic data, is being reviewed worldwide; and (5) developing ethical guidance over genetic privacy is crucial to the future of both genetic research and applied genetics. The report also examines the international flow of data, new health data privacy laws in Europe, the United States' new "Health Insurance Portability and Accountability Act," and the need for dialogue between the United States and Europe. (Final report: 76 pages plus appendix.)

AGENCY SPONSOR: Office of Program Systems

FEDERAL CONTACT: John Fanning

PHONE NUMBER: (202) 690-7100

PIC ID: 6605

PERFORMER ORGANIZATION: William W. Lowrance, Ph.D., Geneva, Switzerland

TITLE: State Regulatory Experience with Provider-Sponsored Organizations: Final Report

ABSTRACT: This report describes the experience of selected State governments in regulating provider-owned health care delivery systems that accept insurance risk for the provision or arrangement of health care services (provider-sponsored organizations, or PSOs). The States included in the study are California, Colorado, Illinois, Iowa, Minnesota, Ohio, Pennsylvania, Texas, and Washington. The report finds that the main models States use to regulate the activities of PSOs contracting directly with purchasers of health care (e.g., employers) are (1) licensing these organizations as they would a non-provider owned entity, and (2) creating a separate licensing category for risk-assuming PSOs. Regulators in States using the first model regulate the activity of risk assumption; in some cases, they apply regulation pertaining to non-provider managed care entities to all organizations seeking to seeking to provide or arrange for health care delivery. In States where PSOs are licensed separately from other managed care organizations, differences in the standards required for PSOs varied and there was little consistency about the rationale for having separate licensing. The report also finds that the States vary considerably in their regulation of PSOs that assume risk from self-funded, ERISA-exempt employer plans. Most consider the PSO as engaging in the business of insurance (thus they may be regulated by the State despite the plan's ERISA status). States also vary in their regulation of PSOs that accept risk downstream from licensed insurance carriers or managed care organizations: some do not regulate these relationships, while others require special licensing or impose substantial requirements governing risk transfer agreements. The report concludes that, since State policy toward PSOs is still in the early stages of development, the lessons to be drawn for Federal policymakers is limited; however, some observations are offered.

AGENCY SPONSOR: Office of Health Policy

FEDERAL CONTACT: Lisa Rovin

PHONE NUMBER: (202) 690-7800

PIC ID: 6628

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Indicators of Welfare Dependence and Well-Being: Interim Report to Congress

ABSTRACT: The Welfare Indicators Act of 1994 directed the Secretary of Health and Human Services to conduct a study to determine which statistics would be most useful in tracking and predicting dependence on three means-tested cash- and nutritional-assistance programs: Aid to Families with Dependent Children, Food Stamps, and Supplemental Security Income. Enactment of the Personal Responsibility and Work Opportunity Reconciliation Act of 1996 makes this task even more important. As this bill is implemented, traditional indicators of dependence may lose their meaning, making dependence more difficult to track over time. In response to these difficulties, this report recommends indicators that measure family well-being in addition to those that measure dependence. The report recommends two kinds of indicators: indicators of dependence, self-sufficiency, and family conditions; and indicators of child achievement, health, and well-being. In most cases, the recommended indicators should be tabulated separately for children, male non-elderly adults, female non-elderly adults, and other important population subgroups wherever possible. To measure dependence, self-sufficiency, and family conditions, the report recommends that indicators reflect points on a continuum from total dependence to complete self-sufficiency. Recommended indicators are grouped into several categories, including: (1) range of dependency, (2) work and job readiness, (3) poverty and deprivation, (4) family structure, and (5) parenting. To measure child achievement and health, the report recommends using indicators of infant and child mortality, low birthweight, child health limitations, teen birth rates, teen violent crime arrests, and others. Finally, the report discusses data needs for the recommended indicators. (Final report: variously paginated, plus appendices.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Susan Hauan

PHONE NUMBER: (202) 690-8698

PIC ID: 6634

PERFORMER ORGANIZATION: Office of the Assistant Secretary for Planning and Evaluation, Washington, DC

TITLE: Setting the Baseline: A Report on State Welfare Waivers

ABSTRACT: The Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA) of 1996 eliminated the entitlement to Aid to Families with Dependent Children (AFDC) and replaced it with Temporary Assistance for Needy Families (TANF), a block grant which States can use to provide cash and services to low-income families with children. This report is one in a series designed to provide baseline information on the AFDC program as it existed prior to enactment of the new law. The report focuses on the waivers of AFDC requirements that were granted to States in the years preceding enactment of PRWORA. Between 1993 and 1996, waivers were granted to 43 States to help them reform their welfare programs. The report provides an overview of the types of waivers that States requested and received, and provides details on specific State policies. The report finds that, in many regards, the waiver programs were the first phase of welfare reform. States were required to conduct evaluations of waiver impacts, and had to ensure that the programs would be cost-neutral (that is, that they would not require additional Federal spending). This cost-neutrality could be achieved over the entire waiver period, rather than in each year, as had been previously required. This allowed States to make large up-front investments that they expected to gain back through reduced benefit payments in later years. The report provides valuable information about how specific States approached the AFDC program and what the Federal government authorized them to implement so that the effects of the TANF program can be measured State by State. Furthermore, the TANF program allows States to maintain their waiver programs or to delay implementation of certain TANF provisions until their waiver expires. The bulk of the report consists of tables presenting information on various aspects of the waiver programs. (Final report variously paginated.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Elizabeth Lower-Basch

PHONE NUMBER: (202) 690-6808

PIC ID: 6651

PERFORMER ORGANIZATION: Office of the Assistant Secretary for Planning and Evaluation, Washington, DC

TITLE: Market Barriers to the Development of Pharmacotherapies for the Treatment of Cocaine Abuse and Addiction: Final Report

ABSTRACT: This report presents an analysis of the market barriers to development of pharmacotherapies for substance abuse and addiction, particularly for abuse of and addiction to cocaine. There are more than 2 million addicted or "heavy" cocaine users. Of these, almost a million may enter treatment at least once in a given year. On any given day, about 10 percent of heavy cocaine users are enrolled in treatment programs. The report examines the development of pharmacotherapies for cocaine abusers in the light of the size of the potential market, estimates of market penetration, and the basic relationships between price, market size, and revenues. The report finds that there are several critical market barriers that must be taken into account. For example, there is a small and uncertain market for cocaine addiction and abuse pharmacotherapy. Although drug companies agree that the total number of cocaine users is significant, they believe that the feasible market for a cocaine abuse treatment is likely to be much smaller than the absolute number of people who use cocaine. Drug companies also anticipate that market penetration would be uncertain; potential patient compliance problems, limited access to patients, and the fact that most substance abuse treatment providers view drug use as a behavioral (rather than a medical) problem further exacerbate this uncertainty. Furthermore, the limited number and capacity of treatment facilities restricts market opportunities, as does the lack of medical treatment models. Finally, the substance abuse treatment market relies heavily on State and Federal reimbursement. Most substance abuse treatment services are subsumed under the mental health benefits of entitlement programs, and drug companies are reluctant to rely upon this kind of reimbursement in an age of shrinking budgets for mental health services.

AGENCY SPONSOR: Office of Health Policy

FEDERAL CONTACT: Kevin Hennessy

PHONE NUMBER: (202) 690-7272

PIC ID: 6694

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Consumer-Directed Personal Assistance Services: Key Operational Issues for State CD-PAS Programs Using Intermediary Service Organizations

ABSTRACT: This study was undertaken by the Office of the Assistant Secretary for Planning and Evaluation (OASPE) to identify best practices for implementing consumer-directed personal assistance (CD-PAS) programs through the use of various intermediary service organization (ISO) models. This report provides practical advice to State program administrators about the design and implementation of CD-PAS programs with ISOs in order that: (1) they comply with Federal tax and labor laws; (2) they make available supportive services that some consumers may want or need; and (3) they allow for the application and enforcement of any limitation or restrictions on consumer direction that may be required by State laws and regulations. The report draws a number of conclusions that cut across ISO models, including (1) that the most successful State-ISO contract relationships are forged when the State has a clear understanding of the services that it wishes to purchase from the ISO; (2) that consumers' backup plans for relief attendants frequently fail, causing consumers to have a difficult time finding relief attendants over weekends, holidays, and when the primary attendant is absent for any reason; (3) that consumers are reluctant to initiate criminal background checks on prospective attendants because they do not know how to conduct them and they feel these checks compromise the consumer-attendant relationship; and (4) that a number of disability advocates and CD-PAS consumers expressed concern about the potential impact of managed care on the availability of CD-PAS to eligible consumers. The report also makes recommendations on how to address these problems. (Final report: 70 pages; appendices bound separately.)

AGENCY SPONSOR: Office of Disability, Aging, and Long-Term Care Policy

FEDERAL CONTACT: Floyd Brown

PHONE NUMBER: (202) 690-6613

PIC ID: 6728

PERFORMER ORGANIZATION: SysteMetrics, Cambridge, MA

TOP OF DOCUMENT

OFFICE OF PUBLIC HEALTH AND SCIENCE

Contents

Measuring Expenditures for Essential Public Health Services

Evaluation of the HBCU Capacity Building Program

Environmental Health Data Needs: An Action Plan for Federal Public Health Agencies (Workshop Results, March 3-5, 1997-- Linthicum Heights, Maryland)

Stakeholders Revisit Healthy People 2000 to Maximize the Impact for 2010

Health of Washington Women

TITLE: Measuring Expenditures for Essential Public Health Services

ABSTRACT: In 1995, the Public Health Foundation, under contract to the U.S. Public Health Service (PHS), worked with State and local public health, mental health, substance abuse, and environmental agencies in nine States-Arizona, Illinois, Iowa, Louisiana, New York, Oregon, Rhode Island, Texas, and Washington-and with Federal PHS agencies to develop and test a methodology for estimating investments in ten essential public health services. These services include both personal health services (care provided to individuals) and population-based health services (interventions that prevent disease and promote health among entire populations). The report finds that public health expenditures can indeed be defined, measured, and monitored according to the essential services framework. The report also produced State-level estimates of essential public health services spending. It finds that (1) more than 2 of every 3 dollars spent by the 9 States on essential public health services ($6.1 billion of $8.8 billion) was for personal health services; (2) population-based health services spending ($2.7 billion or $36 per capita) was only 1 percent of total health care expenditures in the participating States; (3) of the population-based expenditures, 26 percent was for enforcing laws and regulations protecting the health and safety of the public; (4) States were the largest funders of population-based health services, providing 50 percent of the total funding for these activities (Federal funding accounted for 32 percent, while local and other sources accounted for the remaining 18 percent); and (5) of the $7.1 billion PHS expenditures on essential public health services, $2.4 billion (34 percent) supported prevention-related research and $2 billion (29 percent) supported personal health services. The report makes recommendations to improve the methodology for future data collection. (Final report: 36 pages, plus appendices.)

AGENCY SPONSOR: Office of Disease Prevention and Health Promotion

FEDERAL CONTACT: Deborah R. Maiese

PHONE NUMBER: (202) 401-5809

PIC ID: 6194

PERFORMER ORGANIZATION: Public Health Foundation, Washington, DC

TITLE: Evaluation of the HBCU Capacity Building Program

ABSTRACT: In 1992, the Public Health Service (PHS) initiated the Historically Black Colleges and Universities (HBCUs) Capacity Building Program to increase HBCUs' involvement in health and social service programs funded by PHS and other Federal agencies. The hypothesis was that HBCUs could compete more effectively for grant monies if they had comprehensive and fully functional sponsored program offices (SPOs). This evaluation, conducted by the Office of Minority Health (OMH), addresses a series of questions regarding the appropriateness of program processes and the observable impacts of the program at four participating HBCUs. The OMH designed evaluation methods around a logic model that documents the links between program assumptions, activities, and final outcomes. The OMH collected data through questionnaires, site visits, and telephone interviews. The evaluation includes quantitative and categorical indicators of program implementation and outcomes, as well as comparisons of qualitative case histories. The report finds that the grant funds and technical assistance achieved many of the program's aims. An SPO was established at all four participating HBCUs, resulting in more structured and uniform procedures for obtaining and managing externally sponsored programs. Program impact data show that some desired outcomes, such as increased proposal submissions and awards, were partially achieved. The study concludes by highlighting major findings and recommendations related to program effectiveness, including the need for investments in programmatic, as well as administrative, infrastructure. (Final report variously paginated.)

AGENCY SPONSOR: Office of Minority Health

FEDERAL CONTACT: Joan Jacobs

PHONE NUMBER: (301) 443-9923

PIC ID: 6246

PERFORMER ORGANIZATION: Expand Associates, Silver Spring, MD

TITLE: Environmental Health Data Needs: An Action Plan for Federal Public Health Agencies (Workshop Results, March 3-5, 1997-Linthicum Heights, Maryland)

ABSTRACT: State and local public health agencies have had a long-standing role in monitoring the human health hazards of environmental factors. Public health agencies must be able to collect the necessary information to assess the impact of environmental hazards on community health. The Federal government can help State and local public health agencies in environmental surveillance by identifying the types of information and data necessary to successfully protect community health. This report presents the findings of a conference held in March 1997 to assess the environmental health information needs of State and local governments. The report finds that (1) resources should be used to improve existing data collection systems and to increase data accessibility and usability, rather than to create new data collection systems; (2) the integration of activities and electronic data among public health, environmental health, and environmental protection agencies is vital; (3) the usefulness of environmental health information systems can be enhanced through common data architecture, delineation of core baseline and sentinel data, and a comprehensive, central environmental health data repository; (4) these information systems must be evaluated based on their usefulness to State and local agencies in responding to community demands for timely, accurate, and reliable information, and in achieving their research, surveillance, intervention, and health education goals; and (5) full funding of these systems is necessary so that they may be built into a national network that meets the needs of a national environmental health research agenda. The workshop also developed seven key roles the Federal government could play in helping States and localities improve their environmental health surveillance. Finally, the workshop considered potential short-term strategies to help the Federal government fulfill these roles. (Final report: 12 pages, plus appendices.)

AGENCY SPONSOR: Office of Disease Prevention and Health Promotion

FEDERAL CONTACT: Dalton Paxman

PHONE NUMBER: (202) 205-5829

PIC ID: 6322

PERFORMER ORGANIZATION: Public Health Foundation, Washington, DC

TITLE: Stakeholders Revisit Healthy People 2000 to Maximize the Impact for 2010

ABSTRACT: The objectives articulated in Healthy People 2000 aim to improve the health of Americans through effective prevention strategies. They form a model framework for results-oriented performance measurement that can be made relevant to both population-based and individual health care initiatives. They also serve as a benchmark against which health improvement efforts across the Nation can be measured over time. The development of Healthy People 2010 provides an opportunity to consider the successes and failures of Healthy People 2000 in achieving these important goals. Based on experience with Healthy People 2000 and major changes since 1990 in the Nation's health and health care, this report explores new approaches for the development of Healthy People 2010. The perspectives of the Healthy People Steering Committee and the Healthy People Consortium, made up of 350 member organizations were sought. Through interviews with key Steering Committee members, a retreat for the entire Steering Committee, and seven focus groups of Consortium members, clear themes and suggestions emerged. The report finds that both Steering Committee and Consortium members valued Healthy People 2000 as a framework and a voice for public health concerns and issues. However, both groups had several suggestions for the next version. For example, (1) most members of the Steering Committee thought that the document should be reduced in size, (2) Steering Committee members were more interested than Consortium members in making major changes to the document, (3) Consortium members recommended that current information technology be used in order that the document reach the widest audience possible, and (4) health care purchasers and managed care plans thought that there should be fewer priorities and objectives in order to focus on a limited number of problems. The final stakeholders' report is available on the Internet at http://web.health.gov/healthypeople.

AGENCY SPONSOR: Office of Disease Prevention and Health Promotion

FEDERAL CONTACT: Deborah R. Maiese

PHONE NUMBER: (202) 401-5809

PIC ID: 6491

PERFORMER ORGANIZATION: Academy for State and Local Government, Washington, DC

TITLE: Health of Washington Women

ABSTRACT: This report examines the health and well-being of women in Washington State. A broad coalition of Washington's health care providers, researchers, and activists organized to implement this evaluation. To add to the quantitative data collected and analyzed, the report also includes qualitative information from focus groups of women from diverse geographical areas and economic and social strata. The focus groups were conducted to gather women's perceptions and satisfaction with the health care services they received throughout the State. The data show that the nearly 2.8 million women who live in Washington State are living longer lives, are making many positive decisions about their health, and experience, on average, a better standard of living than women in United States in general. The report finds that the overwhelming majority of women in the State are white and live in urban areas, but about 11.3 percent live below the poverty line. The report also finds that women reported more days of poor mental and physical health than did men, but that they are less likely than men to have a sedentary lifestyle, are less likely to smoke, and are more likely to eat healthy fruits and vegetables. Access to health care for Washington's women is relatively good, but about 10 percent did not participate in a health insurance plan in 1994 and 12 percent reported that they were unable to see a doctor because of the cost. Furthermore, elderly women, those with less than a high school education, and poor women were less likely to use mammography and have regular Pap smears. The report also examines rates of domestic violence and sexually-transmitted diseases, as well as mortality patterns and birth-related indicators. The report provides several policy recommendations to address problem areas for Washington women's health. (Final report: 68 pages, plus appendix.)

AGENCY SPONSOR: Regional Health Administrator-Region X

FEDERAL CONTACT: Karen Matsuda

PHONE NUMBER: (206) 615-2501

PIC ID: 6684

PERFORMER ORGANIZATION: Washington Women's Education Foundation, Seattle, WA

TOP OF DOCUMENT

SUBSTANCE ABUSE AND MENTAL HEALTH SERVICES ADMINISTRATION

Contents

Findings from the Evaluation of the Faculty Development Program, Volume I: Final Report

Report on the CTS National Evaluation

CSAT Demonstration: Evaluation of Job Corps Drug Treatment Enrichment Program

NTIES: National Treatment Improvement Evaluation Study: Final Report

Evaluation of Model Projects for Pregnant and Postpartum Women and Their Infants

Final Report: Review of the CMHS Mental Health Care Provider Education in HIV/AIDS Program

TITLE: Findings from the Evaluation of the Faculty Development Program, Volume I: Final Report

ABSTRACT: The Health Professions Clinical Training Program in Alcohol and Other Drug Abuse (Faculty Development Program, or FDP) was established in 1989 and offers training to faculty from schools of medicine, nursing, social work, and graduate psychology programs. The purpose of the FDP was to enhance research, teaching, and clinical practices on issues of alcohol, tobacco, and other drugs (ATOD) within these fields. Specifically, the program seeks to (1) improve the knowledge and practice of participating faculty on ATOD-related issues; (2) incorporate ATOD-related issues into the curriculum at participating institutions; and (3) increase the level of institutional and professional support for ATOD in both policies and programs. The 5-year projects involved two phases: (1) faculty development and implementation efforts, and (2) an evaluation of the program's impact. The report finds that (1) all projects held meetings and seminars, provided mentoring to fellows, supported fellows' attendance at conferences and meetings, and promoted collaboration between fellows and other faculty members; (2) some project directors with extensive prior ATOD experience provided strong organizational leadership and fostered a group spirit among fellows, while others provided less direct guidance and worked with fellows individually; (3) less experienced directors tended to work with the fellows as peers in organizing activities and formulating a training program; (4) at least two-thirds of the fellows participated in all training activities (except for academic courses) during Years 1-3, while during Year 4, two-thirds continued to meet as a group, attended conferences, and engaged in independent studies; and (5) fellows noted three types of particularly helpful training experiences-literature collection and synthesis, participation in experiential learning, and collaborative curricula development and research. (Draft report: variously paginated.)

AGENCY SPONSOR: Office of the Administrator

FEDERAL CONTACT: Cynthia Prather

PHONE NUMBER: (301) 443-4969

PIC ID: 4510

PERFORMER ORGANIZATION: Cosmos Corporation, Bethesda, MD

TITLE: Report on the CTS National Evaluation

ABSTRACT: The Center for Substance Abuse Prevention national Training System (CTS) included the development and delivery of more than 30 distinct curricula on hundreds of occasions to more than 10,000 participants through 1996. Recipients included community teams, health professionals, State administrators, and a range of volunteers engaged in prevention activities. The report finds that CTS was innovative in several important ways, including (1) a focus on community coalitions and associations of health professionals that are responsible for planning and delivering prevention services, rather than concentrating on target populations; (2) curricula were developed quickly in response to emerging issues in the field; and (3) a continuing commitment to quality control and continual curricular improvement. The report also finds that, of three common levels of prevention training (introductory training, training directed at treatment providers and other health professionals, and approaches that contribute to broad systemic changes, usually directed at teams or coalitions), CTS was far more effective at delivering training at the lower two levels, given the complex nature of training for systemic change. The report then focuses on obstacles to training for systemic change, including (1) the difficulty of disseminating best practices, and (2) resistance to change by the prevention community. The report then discusses the principles of effective training. These include (1) attention to post-training application of the knowledge gained; (2) providing a blueprint or schema by which participants may understand the entire framework for a training effort; (3) focus on barriers to application, as well as advantages; (4) ensuring that training is culturally appropriate; and (5) training should encourage participants to use their intellectual and collective intellectual, verbal, emotional, artistic, and imaginative skills. See also PIC ID No. 4514.

AGENCY SPONSOR: Center for Substance Abuse Prevention

FEDERAL CONTACT: Gwyndolyn Ensley

PHONE NUMBER: (301) 443-0581

PIC ID: 4514.1

PERFORMER ORGANIZATION: Pacific Institute for Research and Evaluation, Bethesda, MD

TITLE: CSAT Demonstration: Evaluation of Job Corps Drug Treatment Enrichment Program

ABSTRACT: The Center for Substance Abuse Treatment within the Substance Abuse and Mental Health Services Administration sponsored a 4-year demonstration project of the Drug Treatment Enrichment Program (DTEP), which was put in place within the Department of Labor's Job Corps program. DTEP was implemented in four experimental Job Corps centers, which were matched with four control centers that provided the standard Job Corps Alcohol and Other Drugs of Abuse (AODA) program. The report finds that (1) the organizational location of DTEP within each Job Corps center's operations was problematic, since DTEP staff reported a lack of managerial support for the program, high staff turnover, and long periods for staff replacement; (2) the average cost per program participant for DTEP was twice the amount required for AODA; (3) DTEP reduced marijuana and crack/cocaine use post-Job Corps significantly more than did AODA; (4) DTEP students were less likely to engage in moderate to extensive use of marijuana or to use other drugs, but no differences were found for alcohol abuse; (5) DTEP improved students' mental health; (6) DTEP students were less likely to report selling or helping to sell drugs after Job Corps; (7) no differences between DTEP and AODA were found for other criminal activities; (8) DTEP students with mental health problems experienced a significantly higher job placement rate after Job Corps, although the program had no effect on job placement rates of other students; and (9) DTEP appeared to have no significant positive effect on duration of stay in Job Corps or on educational and vocational gains. The report makes recommendations regarding the integration of DTEP into Job Corps and other topics. (Final report: variously paginated, plus appendices.)

AGENCY SPONSOR: Center for Substance Abuse Treatment

FEDERAL CONTACT: Ron Smith

PHONE NUMBER: (301) 443-7730

PIC ID: 4523

PERFORMER ORGANIZATION: Caliber Associates, Fairfax, VA

TITLE: NTIES: National Treatment Improvement Evaluation Study: Final Report

ABSTRACT: The National Treatment Improvement Evaluation Study (NTIES) is a comprehensive assessment of the results of 157 3-year demonstration grants and cooperative agreements funded by the Center for Substance Abuse Treatment within the Substance Abuse and Mental Health Services Administration beginning in 1990-1991. These grants were intended to improve alcohol and drug treatment in selected target cities with severe substance abuse problems; among critical populations such as minority groups and women; and in the criminal justice system. Based on a final sample of 6,600 clients, the evaluation produced several results. For clients who entered treatment for alcohol and illegal drug use, the evaluation found that the program produced (1) a 78-percent reduction in selling drugs and committing violent crimes; (2) a 56-percent reduction in high-risk sexual behaviors related to HIV transmission; (3) a 53-percent reduction in medical visits; (4) a 42-percent reduction in homelessness; (5) a 28-percent reduction in inpatient mental health visits; and (6) a 19-percent increase in employment. See also PIC ID No. 5346. (Final report: 310 pages, plus appendices.)

AGENCY SPONSOR: Center for Substance Abuse Treatment

FEDERAL CONTACT: Ron Smith

PHONE NUMBER: (301) 443-7730

PIC ID: 5346.1

PERFORMER ORGANIZATION: National Opinion Research Center (NORC), Chicago, IL

TITLE: Evaluation of Model Projects for Pregnant and Postpartum Women and Their Infants

ABSTRACT: In 1989, the Center for Substance Abuse Prevention (CSAP) implemented a national demonstration program to provide prevention, education, and intervention services to substance-abusing pregnant and postpartum women and infants (PPWIs). CSAP contracted this study in 1993 to (1) assess the effectiveness of the PPWI programs in enhancing services coordination and increasing the availability and accessibility of substance abuse prevention services delivered to clients; and (2) document the extent to which program models are effective in decreasing the use of alcohol and drugs among substance-abusing women and enhancing the healthy development of their children. Ten PPWI programs funded by CSAP in 1992 were selected to participate in a 4-year cross-site evaluation. The wide range of interventions and possible outcomes required that program evaluations were augmented by a process evaluation to explain the relationships between mediating variables and outcomes. The project found that (1) there were significantly lower prevalence rates in the use of alcohol, marijuana, crack, and other illicit substances in the treatment group from intake to delivery, and in recent (30 day) marijuana and crack use in the treatment group from intake to delivery (these differences disappeared by the 6-month postpartum followup); (2) the effect of PPWI treatment group membership for lower recent crack use appears to be attributable to participation in day treatment, rather than case management programs; and (3) in general, treatment and comparison group members gave birth to infants with similar birth-related outcome characteristics, but when the mother's crack use histories were taken into account, treatment babies had significantly higher birthweights than comparison group babies. (Final report: variously paginated, plus appendices.)

AGENCY SPONSOR: Center for Substance Abuse Prevention

FEDERAL CONTACT: Soledad Sambrano

PHONE NUMBER: (301) 443-9136

PIC ID: 5674

PERFORMER ORGANIZATION: Johnson, Bassin and Shaw, Silver Spring, MD

TITLE: Final Report: Review of the CMHS Mental Health Care Provider Education in HIV/AIDS Program

ABSTRACT: This report presents findings from a review of the Center for Mental Health Services' (CMHS) Mental Health Care Provider Education in HIV/AIDS Program, from its inception in 1986 through 1994. The program supports the provision of state-of-the-art training for (1) traditional mental health care providers (psychiatrists, psychologists, nurses, and social workers), (2) other first-line providers of mental health care services (primary care physicians and medical students), and (3) nontraditional providers of such services (clergy or alternative health care workers). The program also supports the development and dissemination of curricula and teaching materials for providing training on the mental health aspects of HIV/AIDS. The goal of the program is to improve access to appropriate and adequate mental health services for people living with HIV/AIDS and for their families and partners. The review examines the activities and outcomes of 27 projects funded by the program. The report finds that (1) the program has trained tens of thousands of mental health service providers, representing the full spectrum of assistance to persons infected with HIV and to their families; (2) training participants are likely to have increased their knowledge about the mental health aspects of the disease, their willingness to serve HIV-infected individuals, and their confidence that they can effectively do so; (3) program funding has served as a catalyst for systemic changes within teaching institutions and hospitals for the development of alliances with concerned groups; (4) the need for training in the mental health aspects of HIV/AIDS is needed throughout the country, especially as affected areas and populations change over time; (5) at the program level, there is a pressing need to obtain systemic knowledge about the efficacy of various approaches to training mental health providers; and (6) the program must identify its priorities and how different projects can meet them.

AGENCY SPONSOR: Center for Mental Health Services

FEDERAL CONTACT: Martha Ann Carey

PHONE NUMBER: (301) 443-3653

PIC ID: 6695

PERFORMER ORGANIZATION: KRA Corporation, Silver Spring, MD

Centers for Disease Control and Prevention

Contents

Sexually Transmitted Disease (STD) Prevention in the United States: Integrated Evaluation of Public and Private Sector Disease Reporting and Service Delivery

Evaluation of ICD-10 for Morbidity Reporting in the United States

Suicide Intervention and Prevention: Evaluation of Community-Based Programs in Three American Indian Communities

Evaluation of the Fatality Assessment and Control Evaluation (FACE) Program: Phase II Final Report

Assessment of the National Laboratory Training Network

Race Reporting on Birth Records by Multiracial Women: An Evaluation of Question Format Alternatives

Management and Operational Impacts of HIV Prevention Community Planning: Lessons Learned at Mid-Course and Implications for Future Activities

CDC's Early Implementation of GPRA: A Case Study-- July 1995 to November 1996

Collaborating with Managed Care Organizations for Mammography Screening and Rescreening: Guidance for NBCCEDP Grantees

Reaching Women for Mammography Screening: Successful Strategies of National Breast and Cervical Cancer Early Detection Program (NBCCEDP) Grantees

Evaluation Design for the Business Responds to AIDS (BRTA) Program

Case Studies of the Prevention Marketing Initiatives (PMI) Local Demonstration Site Projects: Experiences During Planning and Transition Phases

Evaluability Assessment of CDC-Funded Traumatic Brain Injury and Spinal Cord Injury Programs

Community Context Study: Minors' Access to Tobacco

Directions for Research on the National Immunization Survey: Final Report

In Her Own Words: A Focus Group Study of Risk and Protective Factors in Intimate Partner Violence

Linking Research and Public Health Practice: A Review of CDC's Program of Centers for Research and Demonstration of Health Promotion and Disease Prevention

Syphilis in the South: A Case Study Assessment in Eight Southern Communities

Evaluation of the Effectiveness of CDC Surveillance for Drug-Resistant Streptococcus pneumoniae (Baltimore, MD)

Evaluation of the Effectiveness of CDC Surveillance for Drug-Resistant Streptococcus pneumoniae (San Antonio, TX)

Assessment of the Programmatic Impact of One-Percent Evaluation Studies

Directory of Evaluations Administered by the CDC Office of Program Planning and Evaluation 1990-1995

TITLE: Sexually Transmitted Disease (STD) Prevention in the United States: Integrated Evaluation of Public and Private Sector Disease Reporting and Service Delivery

ABSTRACT: The Centers for Disease Control and Prevention (CDC) Division of Sexually Transmitted Disease (STD) Prevention provides financial and technical assistance to State and local agencies to support a range of prevention, research, policy development, and surveillance activities. Two parallel STD surveillance systems are currently operated by CDC, both of which rely upon regular, voluntary reporting of STD incidence by State and local agencies. Both systems are thought to suffer from less-than-complete reporting of STDs by diagnosing practitioners. Estimates of this underreporting range from 25 to 90 percent. The focus of this project, therefore, was to design and pilot test a methodology for validating the STD data routinely collected, analyzed, and published by the Division. The study's main objective was to test the feasibility of using a provider-based survey (1) to quantitatively validate CDC's data on STD diagnoses, and (2) to assess the extent to which clinicians follow CDC's treatment guidelines for STDs. The study was designed to generate estimates of STD incidence through a survey of a random sample of STD data providers in four States, and to compare these estimates with State-reported STD incidence for the same time period. The report finds that (1) the four pilot States differ in terms of their STD data-gathering policies and practices, including which diseases are reported, how cases are defined for reporting purposes, how laboratory reporting is utilized, and which date information is recorded for each case; (2) syphilis, gonorrhea, and chlamydia are reportable diseases in all four States, and in two States many additional diseases are reportable; (3) in two States, physicians must report diseases for them to be recorded, while in two others laboratory reports are sufficient; (4) STDs are considerably underreported by providers; and (5) most STD cases are seen by individual private practitioners. (Final report: 82 pages, plus appendices.)

AGENCY SPONSOR: National Center for HIV, STD and TB Prevention

FEDERAL CONTACT: John Moran

PHONE NUMBER: (404) 639-8272

PIC ID: 5348

PERFORMER ORGANIZATION: Abt Associates, Incorporated, Boston, MA

TITLE: Evaluation of ICD-10 for Morbidity Reporting in the United States

ABSTRACT: This report evaluates (1) whether the International Classification of Disease (ICD)-10 is a significant improvement over the ICD-9; (2) whether these improvements warrant its implementation in the United States; (3) the modifications needed to improve its use for morbidity applications; and (4) whether there are any codes or concepts in ICD-9, but not in ICD-10, that should be considered for inclusion. The Technical Advisory Panel and project staff conclude that there are many strengths in ICD-10 in comparison with ICD-9. Many of the chapters contain a wealth of useful expansion in detail for which diagnostic information is typically available and which represent important clinical distinctions. These expansions will facilitate future specificity in statistical tabulations of diseases and conditions, provide some additional means for severity coding, and provide for enhanced applicability of the classification to primary care reporting. The logic and detail of ICD-10 will also contribute to progress toward the computerization of medical records. Although a number of limitations in ICD-10 are identified, the recommended modifications will overcome most of them. Therefore, it is strongly recommended that the National Center for Health Statistics implement a revised version as soon as possible.

AGENCY SPONSOR: National Center for Health Statistics

FEDERAL CONTACT: Amy Blum

PHONE NUMBER: (301) 436-7050

PIC ID: 5503

PERFORMER ORGANIZATION: Center for Health Policy Studies, Columbia, MD

TITLE: Suicide Intervention and Prevention: Evaluation of Community-Based Programs in Three American Indian Communities

ABSTRACT: Suicide is the second leading cause of death for Native Americans aged 15 to 24. American Indian/Alaska Native young people are more likely to attempt suicide; to choose a highly lethal method, such as firearms or hanging; and to be successful at committing suicide. This report evaluates the local community-based suicide intervention and prevention programs of three geographically diverse American Indian tribes (the Jicarilla Apache Tribe, the Western Athabaskan Tribe, and the Northern Woodlands Tribe). In two tribes, suicide intervention efforts had been in place since 1989 and 1990, respectively; in the third, a suicide crisis in late 1992 precipitated community response. The report finds that (1) the local nature of the evaluation process opened itself to problems of diffuse authority and lack of focus on the evaluation process; (2) suicide prevention issues, tribal identity issues, and staffing issues took precedent over the actual evaluation activities; (3) a major recommendation arising from this study was to have process evaluation activities conducted by an external (non-community member) evaluator not supervised by local staff; (4) the Jicarilla Apache program represents a suicide prevention program in a very small, cohesive, and well-coordinated community response, which depends upon a staff model; (5) at the Western Athabaskan reservation, the prevention program used several special initiatives loosely coordinated with an existing Behavioral Health Services program using a staff model; (6) using a volunteer network, the Northern Woodlands tribe produced an excellent manual describing a grassroots community approach to suicide prevention; and (7) in all three tribes, attempted and successful suicide rates were reduced. The report concludes that the staff models used in the two small and centralized reservation communities were more successful than the diffuse, volunteer effort. (Final report: 120 pages.)

AGENCY SPONSOR: National Center for Injury Prevention and Control

FEDERAL CONTACT: Lloyd Potter

PHONE NUMBER: (770) 488-1557

PIC ID: 5504

PERFORMER ORGANIZATION: Indian Health Service, Rockville, MD

TITLE: Evaluation of the Fatality Assessment and Control Evaluation (FACE) Program: Phase II Final Report

ABSTRACT: The Fatality Assessment and Control Evaluation (FACE) program of the National Institute for Occupational Safety and Health seeks to prevent workplace fatalities by identifying and investigating fatal occupational injuries and disseminating prevention strategies to those who can intervene in similar situations. Publications based on information from fatality investigations identify occupational hazards and prevention measures, and are disseminated to persons who can influence work conditions and information availability in related occupations and work settings.

This report (1) describes the dissemination of five recent FACE documents (4 tear-sheet Alerts and one monograph), (2) compiles users' assessments of their usefulness, and (3) identifies examples of prevention efforts and other impacts attributable to the documents. The report finds that (1) publications are generally considered technically accurate and focused on significant occupational hazards by users; (2) workers participating in focus groups see job-related chronic illnesses and nonfatal injuries as more significant than the risk of fatality, and are less convinced of tear sheets' relevance to them; (3) workers consider the tear sheets' recommendations accurate and appropriate, but are concerned by the inclusion of "common sense" recommendations; (4) the tear sheets' terminology and generally high literacy level may limit their usefulness to their target audiences; (5) the publications are generally disseminated by distributing them directly or by modifying them for inclusion in other informational materials; and (6) information from the publications is used to improve worker training, identify existing hazards, and improve safety standards. The report concludes that the FACE publications are a useful source of authoritative occupational safety information. The report includes several recommendations. See also PIC ID No. 5506. (Final report: 30 pages, plus appendices.)

AGENCY SPONSOR: National Institute for Occupational Safety and Health

FEDERAL CONTACT: Theodore Pettit

PHONE NUMBER: (304) 285-5972

PIC ID: 5506.1

PERFORMER ORGANIZATION: Research Triangle Institute, Research Triangle Park, NC

TITLE: Assessment of the National Laboratory Training Network

ABSTRACT: The National Laboratory Training Network (NLTN) was established in 1989 to provide training in new methodologies and theoretical concepts for laboratory workers. This report assesses whether the NLTN has had a beneficial impact on public health by improving laboratory practice. Data were gathered through a series of surveys of local health department and physician office laboratory employees who participated in training, as well as comparison groups of those who did not. Other surveys of State training coordinators (STCs), laboratory inspectors, microbiology directors, microbiology program participants, and NLTN co-sponsors were also conducted. The report categorizes its findings in three groups, based upon the responses of (1) STCs, laboratory inspectors, and co-sponsors; (2) microbiologists; and (3) course participants and their comparison groups. The report finds that: (1) STCs felt supported in their jobs and used NLTN resources (88 percent were somewhat or very satisfied with NLTN services, and 90 percent were satisfied with programs); (2) laboratory inspectors often recommended training as a means to correct deficiencies, but only a small percentage used the NLTN as a training source; (3) co-sponsors rate relevance, quality, and currency of course content very highly; (4) microbiology directors rated NLTN fees, convenience, relevance, quality, currency of content, and pertinence of issues addressed favorably; (5) one-third of microbiology course participants instituted a change in practice as a result of training (80 percent rated quality and currency of content highly); and (6) course participants in the third group rated their training highly and most instituted changes or saw positive results subsequent to training. The report includes recommendations for further improving the NLTN. (Final report: 147 pages, appendices bound separately in Volume II.)

AGENCY SPONSOR: Public Health Practice Program Office

FEDERAL CONTACT: Eunice Rosner

PHONE NUMBER: (770) 488-4129

PIC ID: 5507

PERFORMER ORGANIZATION: Battelle Corporation, Arlington, VA

TITLE: Race Reporting on Birth Records by Multiracial Women: An Evaluation of Question Format Alternatives

ABSTRACT: This report examines the implications of changing the format of the race item on the birth record filled out for every birth in the United States at the location where the birth occurred. Procedures for filling out birth certificates vary from State to State and in some States, from hospital to hospital. To provide uniform information to the vital registration system, the Federal government, in cooperation with the States, developed the U.S. Standard Certificate of Live Birth that collects relevant information about the race and Hispanic origin of the parents, but not of the baby. This study (1) evaluates how women of multiracial and Hispanic backgrounds interpret the race question on the birth certificate; (2) evaluates experimental race questions, such as the use of a multiracial cue and the use of a "mark all that apply" format; and (3) learns more about the response processes that women use when answering questions about race. Nine States and the District of Columbia recruited 763 women to participate in this study. The study tests the effects of different questions on race, both in a mail and a followup telephone survey. The mother's self-reported race is compared with what she reported on the actual birth record. Further, for those residents with two children, the study compared consistency of race reporting between the last two birth records. The report concludes that (1) the number of women reporting more than one race increases when the race questions include a multiracial cue or "mark all that apply" format; (2) women who report more than one race are more likely to enter two specific races rather than a term like multiracial; (3) among the mothers whose parents who were two or more races, self-reports of race on the last two birth records are inconsistent 25 percent of the time; and (4) self-reports of race are different 40 percent of the time between the birth record and the standard version of the mail survey. (Final report: 48 pages, plus appendices.)

AGENCY SPONSOR: National Center for Health Statistics

FEDERAL CONTACT: Susan Schechter-Ryan

PHONE NUMBER: (301) 436-7111

PIC ID: 5918

PERFORMER ORGANIZATION: Westat, Incorporated, Rockville, MD

TITLE: Management and Operational Impacts of HIV Prevention Community Planning: Lessons Learned at Mid-Course and Implications for Future Activities

ABSTRACT: HIV Prevention Community Planning, implemented in January 1994, represented a major policy change in Centers for Disease Control and Prevention funding policy. Prior to that time, State, territorial, and local health department grantees with prevention funds were required to spend most of their cooperative agreement funds on counseling and testing services. A new emphasis on planning through HIV Prevention Community Planning requires grantees to set their own HIV prevention priorities locally through a process that emphasizes input from the affected communities and relevant technical experts. This project broadly evaluated the policy impact of this new planning process on grantees' HIV prevention programs as demonstrated by budgetary and operational indicators. This is an issue of national importance, as the HIV prevention community planning process affects over $200 million dollars in HIV prevention funds across the country. This first phase of a two-phase project ascertained how community planning is confirming, enhancing, and changing HIV prevention programs. A methodology for obtaining information on the indicators for a retrospective study was also developed and piloted during this phase of the study. The report finds that (1) a prospective and iterative approach to planning, implementation, and assessment is the most valuable approach; (2) in a prospective approach, the baseline is the set of circumstances extant at the beginning of the process, and site-specific goals and objectives are developed and tracked; and (3) more detailed guidance addressing these issues could help operationalize this kind of approach. Finally, the report finds that the indicators offering the most promise are (1) correspondence between the community plan, proposed activities, and undertaken activities; (2) quality assurance for implementation; and (3) capacity building and infrastructure development. (Final report: 28 pages, plus appendices.)

AGENCY SPONSOR: National Center for HIV, STD and TB Control

FEDERAL CONTACT: Carol Robinson

PHONE NUMBER: (404) 639-0952

PIC ID: 5927

PERFORMER ORGANIZATION: Macro International, Incorporated, Atlanta, GA

TITLE: CDC's Early Implementation of GPRA: A Case Study--July 1995 to November 1996

ABSTRACT: The Government Performance and Results Act of 1993 (GPRA) stresses the importance of government agencies' monitoring their results or outcomes in order to improve their management of associated processes and outputs. GPRA mandates that each agency produce a 5-year strategic plan and a 1-year performance plan by September 1997. This report offers a record of the experience of the Centers for Disease Control and Prevention in its early implementation of GPRA. It documents the process, summarizes key findings, and recommends strategies and approaches for the ongoing implementation of GPRA. The report finds that (1) implementation during the study phase did not proceed at the anticipated rate (instead of a fully developed strategic plan and pilot performance plans, most of the effort has concentrated on a strategic plan that is not yet drafted-little attention has been paid to the performance plans); (2) the role and responsibilities of the GPRA Implementation Team (GIT) evolved over the study period; (3) the GIT suffered from shifting membership, lack of continuity in meeting attendance, and an overall insufficient level of effort; (4) more effective and comprehensive planning and coordination was needed; and (5) despite the large amount of time and effort devoted to GPRA, the levels and types of commitment and resources fell short of what was needed. The report also concludes that insufficient momentum, commitment, and resources were a greater problem than ineffective planning; the process could have benefitted from greater executive staff involvement early on; and the process was often frustrating for participants. The report recommends that (1) organizational change efforts be unified under a high-level champion or team; (2) the Office of Program Planning and Evaluation's charge of implementing GPRA requirements be maintained; (3) the GIT be revamped; and (4) realistic budget be developed, requested, and executed.

AGENCY SPONSOR: Office of the Director

FEDERAL CONTACT: Wilma G. Johnson

PHONE NUMBER: (404) 639-7090

PIC ID: 5928

PERFORMER ORGANIZATION: Research Triangle Institute, Research Triangle Park, NC

TITLE: Collaborating with Managed Care Organizations for Mammography Screening and Rescreening: Guidance for NBCCEDP Grantees

ABSTRACT: Clinical trials show that mammography screening among women aged 50-69 years can reduce mortality from breast cancer by as much as 30 to 40 percent. Thus, more complete mammography screening coverage is needed throughout the Nation. This project was undertaken with a dual purpose: (1) to evaluate strategies used in the National Breast and Cervical Cancer Early Detection Program (NBCCEDP) to reach women 50 years and older for the early detection of breast cancer; and (2) to assess potential partnership opportunities between public health and managed care organizations to increase the coverage of mammography screening. Currently, all States, the 5 territories, and 25 American Indian tribes are funded by the NBCCEDP to establish and manage comprehensive breast and cervical cancer screening services for women who are minorities, low-income, and age 50 or older, a population that seeks mammography services less often than other women because of cost, knowledge, and other barriers. The report notes that the increased use of managed care organizations (MCOs) by State Medicaid programs offers new opportunities and incentives to build partnerships between NBCCEDP grantees and MCOs. The report finds that (1) because grantees give priority to women who are low-income, uninsured, racial minorities, or Native American, special attention must be paid to educating and building relationships with MCOs in outreach, case management, and followup; (2) both sectors will benefit from data partnerships and community collaborations; (3) most current MCO efforts in the area of breast and cervical cancer detection are data-driven and office-based; and (4) health plans target providers more than patients to increase their attention to the prevention effort. The report also addresses opportunities for collaboration, barriers to and facilitators of such collaboration, and strategies that programs might use to enhance screening efforts. See also PIC ID No. 6273.1.

AGENCY SPONSOR: National Center for Chronic Disease Prevention and Health Promotion

FEDERAL CONTACT: Nancy Nowak

PHONE NUMBER: (404) 488-4306

PIC ID: 6273

PERFORMER ORGANIZATION: Macro International, Incorporated, Atlanta, GA

TITLE: Reaching Women for Mammography Screening: Successful Strategies of National Breast and Cervical Cancer Early Detection Program (NBCCEDP) Grantees

ABSTRACT: This guide presents successful strategies used by grantees of the National Breast and Cervical Cancer Early Detection Program (NBCCEDP) to reach women in need of mammography screening-in particular low-income women, underserved and uninsured women, and women from communities of color. This information is presented to help grantees reach these marginalized populations by sharing strategies that their peer grantees have found successful. This information will also be useful for private sector and other organizations that are working to provide prevention services to hard-to-reach groups. Five categories of strategies are examined: inreach, public education, outreach, policies and procedures, and coalition and partnership development. While all of these strategies are different, several common themes underlie them all: (1) organizations must consider broad partnerships to reach marginalized populations who, because of their complicated lives, may not seek out care for themselves; (2) aggressive outreach may be needed to work with agencies and individuals who are most frequently in contact with these populations; (3) organizations should consider their infrastructure, not just outreach, to bring women in (health care organizations or health plans with a designated group of clients will find inreach more effective than outreach in the short term); (4) while the natural tendency is to focus on the relationship between the direct caregiver (especially the physician) and the patient, women are more likely to return for rescreening and to comply with followup regimens if a welcoming atmosphere is created by the entire office; and (5) the successful strategies show that there are diverse resources available for programs, but that innovation is needed when developing partnerships. See also PIC ID No. 6273. (Final report: 104 pages, plus appendices.)

AGENCY SPONSOR: National Center for Chronic Disease Prevention and Health Promotion

FEDERAL CONTACT: Nancy Nowak

PHONE NUMBER: (404) 488-4306

PIC ID: 6273.1

PERFORMER ORGANIZATION: Macro International, Incorporated, Atlanta, GA

TITLE: Evaluation Design for the Business Responds to AIDS (BRTA) Program

ABSTRACT: The Business Responds to AIDS program (BRTA) is a public-private initiative partnership undertaken by the Centers for Disease Control and Prevention's Division of HIV/AIDS Prevention to help businesses create and implement comprehensive HIV/AIDS workplace programs. To better understand how worksites are engaged in health promotion efforts, the BRTA Program conducted a national probability survey (the BRTA Benchmark Survey) to track progress since 1992. This project conducts a secondary analysis of the BRTA Benchmark Survey in order to provide recommendations about future evaluation designs that rely heavily on national communication channels. While much of the survey addressed issues related to HIV/AIDS, a number of questions were included that help track trends in worksite policies, screening programs, and health promotion activities. Results from this survey were compared to those gathered in a 1992 Office of Disease Prevention and Health Promotion survey of health promotion and disease prevention in the worksite. The report finds that (1) recent trends in worksite health promotion efforts seem to show increases in worksite policies, but no changes or decreases in classes or screening programs; (2) having a worksite policy may be less costly to implement than classes or screening programs (these policy interventions may also be more cost-effective than other types of interventions); and (3) the downward trend may result from some measurement bias introduced by asking questions about health promotion classes and screening programs only of those respondents to the BRTA Benchmark Survey who said they had had some health promotion program in the past 12 months. (Final report: 14 pages, plus appendices.)

AGENCY SPONSOR: National Center for Chronic Disease Prevention and Health Promotion

FEDERAL CONTACT: Cynthia Jorgensen

PHONE NUMBER: (404) 639-0952

PIC ID: 6277

PERFORMER ORGANIZATION: Research Triangle Institute, Research Triangle Park, NC

TITLE: Case Studies of the Prevention Marketing Initiatives (PMI) Local Demonstration Site Projects: Experiences During Planning and Transition Phases

ABSTRACT: The Prevention Marketing Initiative (PMI) is a social marketing program intended to influence behaviors that contribute to the sexual transmission of HIV and other sexually transmitted diseases among young people below 25 years of age. PMI includes among its components locally funded demonstration sites, which combine community participation with social marketing and behavioral science methodology and theory in order to design, implement, and sustain viable prevention programs. The PMI case study gathered in-depth information at these sites to describe the experiences of program participants over a 2-year period spanning two major project phases: planning and transition to implementation. The study highlights participants' reflections on what was done to foster achievement of PMI's goals, as well as what could be avoided in the future. The qualitative information gathered will be used to contextualize quantitative evaluation data collected as sites move through the implementation phase of PMI. The report finds that PMI had the following effect: (1) increased collaboration among community organizations and individuals on HIV prevention; (2) increased youth involvement in planning HIV prevention activities after some initial floundering in this area; (3) there is some weak evidence of increased support for HIV prevention programs within the community; and (4) increased participants' knowledge of, and sensitivity to, social marketing methodologies. The report also makes several recommendations on two levels: those that can be applied to new PMI sites and to other community-based ventures and those that pertain to PMI in particular. A,ong these recommendations are (1) define and become familiar with the community, (2) learn to manage issues and be realistic, (3) maintain diversity and enforce rules, (4) be clear with the lead agency, and (5) have the staff in place to provide needed technical and management services. (Final report: 204 pages, plus appendices.)

AGENCY SPONSOR: Office of the Director

FEDERAL CONTACT: Nancy Cheal

PHONE NUMBER: (404) 639-7095

PIC ID: 6337

PERFORMER ORGANIZATION: Battelle Corporation, Arlington, VA

TITLE: Evaluability Assessment of CDC-Funded Traumatic Brain Injury and Spinal Cord Injury Programs

ABSTRACT: In 1988, the Centers for Disease Control and Prevention (CDC) created the Disabilities Prevention Program (DPP) to reduce the incidence and severity of primary and secondary disabilities in the United States and to promote the independence and productivity of persons with disabilities. This report was commissioned to assess the feasibility of conducting an evaluation of the traumatic brain injury and spinal cord injury (TBI/SCI) programs funded through the DPP, the National Center for Environmental Health, and CDC. Interviews were conducted with Federal and State staff with a background or interest in DPP, external stakeholders, and with program staff in eight States. Interview data were used to prepare a logic model of State-level TBI/SCI programs, along with a set of measurable program components that match the activities and the objectives set forth in the logic model. The report finds that (1) there is a consensus among all stakeholders on the goals, activities, and objectives of the TBI/SCI capacity building programs' intended goals and on indicators that can be used to measure attainment; and (2) identifying the logic model and framework of measurable components for these programs offers many benefits. In order to strengthen CDC-State partnerships, the report recommends: (1) an appropriate balance in the partnership that allows States to be flexible but also provides parameters to ensure consistency; (2) the provision of regular, consistent feedback on State plans, activities, quarterly reports, and surveillance data; (3) grounding program design and decisions on valid data, whenever possible (in the absence of data, programs should proceed with a phased approach involving both the CDC and the States in the evolutionary learning process); and (4) the expansion of partnerships at the Federal, State, and community levels. (Final report: 33 pages plus appendices.)

AGENCY SPONSOR: National Center for Injury Prevention and Control

FEDERAL CONTACT: Doug Browne

PHONE NUMBER: (404) 488-7098

PIC ID: 6338

PERFORMER ORGANIZATION: Susan Toal, Atlanta, GA

TITLE: Community Context Study: Minors' Access to Tobacco

ABSTRACT: This project represents the first phase of a national study designed to profile the relationship between youths' access to tobacco and the contextual variables that define communities' policies, attitudes, and activities regarding youth and tobacco. The results of the study have implications for local, State, and Federal policies and programs. The basic conceptual model postulates that community forces will initiate and/or enhance local enforcement efforts. Enforcement will affect vendor behavior, which will then restrict minors' access to-and thereby, use of-tobacco. To address the questions raised by the conceptual model, data collection and analysis for this project were conducted in three parts: (1) telephone interviews with tobacco vendors and community leaders, and purchase attempts in stores in 20 communities; (2) in-person interviews with community leaders in ten case-study communities; and (3) analysis of teen tobacco use data from the 1995 Youth Risk Behavior Study (YRBS), collected by CDC for the same 20 communities. The report finds that (1) adolescents have little difficulty buying cigarettes in retail outlets; (2) even in restrictive communities, some stores would sell tobacco to minors; (3) the most important deterrent of sales to minors is asking for proof of age; (4) the presence of tobacco sales warning signs is associated with a lower sale rate; (5) community support for tobacco control promotes merchant compliance with the law; and (6) even a perception of enforcement acts as a strong deterrent to selling tobacco to minors. The report recommends that (1) the study should be expanded as a followup to the 1997 YRBS in sampled communities, (2) a question about the location of sources of retail purchase should be added to the YRBS, and (3) the expanded community context study should be fielded as soon as possible. (Executive summary: 7 pages; final report: variously paginated, plus appendices.)

AGENCY SPONSOR: National Center for Chronic Disease Prevention and Health Promotion

FEDERAL CONTACT: Gary A. Giovino

PHONE NUMBER: (770) 488-5703

PIC ID: 6474

PERFORMER ORGANIZATION: Battelle Corporation, Arlington, VA

TITLE: Directions for Research on the National Immunization Survey: Final Report

ABSTRACT: The National Immunization Survey (NIS) measures the immunization levels among young children in each State and in large metropolitan areas on a quarterly basis. This survey recently came under review, and questions about the cost of the survey and the utility of the data were raised. This report reviews the survey and develops a plan for determining whether less expensive methods could be used and still yield a statistically sound survey that meets the needs of the National Immunization Program. The report finds that (1) the NIS is a well-designed response to a very difficult set of goals posed by the Centers for Disease Control and Prevention; (2) dramatic cost reductions (i.e., 30 percent or larger) could not be achieved without major changes in survey objectives or quality; (3) there is greater potential for cost savings in the area of data collection than in analysis, since the volume of the survey makes refined systems for data collection economically feasible; (4) the high data analysis costs for the survey are well justified; (5) the survey meets the need for monitoring trends at the national level and can also be used to identify Immunization Action Plan (IAP) areas with either very poor or outstanding immunization levels; and (6) the survey can differentiate the immunization rates for reasonably small demographic groups annually, and can be used to monitor immunization trends at the IAP level, but only on an annual or biennial basis. The report concludes that the relative emphasis on IAP-level statistics and on national statistics should perhaps be adjusted-in the justification of the survey, in the design of the household interview, and in the choice of statistics to publish. It recommends that the schedule for publication of IAP-level statistics be cut back to an annual schedule and that new publications on national statistics for domains of interest be developed to fill the publication schedule.

AGENCY SPONSOR: National Immunization Program

FEDERAL CONTACT: Jacquelyn Williams

PHONE NUMBER: (301) 443-4493

PIC ID: 6477

PERFORMER ORGANIZATION: Research Triangle Institute, Research Triangle Park, NC

TITLE: In Her Own Words: A Focus Group Study of Risk and Protective Factors in Intimate Partner Violence

ABSTRACT: This study was designed to provide insights into how African-American and Caucasian women in urban and rural settings perceive the factors that placed them at risk for an abusive relationship, as well as the factors that protected them from further violence during and after the relationship. The model used as the conceptual framework for the study (Prochaska's and DiClemente's Transtheoretical Model) depicts a sequence of five stages that culminate in lasting behavior change. Twenty-two focus groups were conducted in urban and rural areas in Georgia, Illinois, Pennsylvania, and Oregon. One hundred and sixty-eight women participated in the study, ranging in age from 18 to 60; all had been out of their abusive relationships from 6 months to 24 years. The report finds that (1) the long-term effects of emotional and physical abuse are worse than the actual violence and resulting injuries themselves; (2) a wide range of psychologically and physically abusive acts were described by focus group participants, including abuse not only of themselves but also of children and of pets; (3) most focus group participants concluded that anything could be a trigger to violence, and that this realization was a significant contributor to the terror, fear, and desperation they felt in their relationships; (4) participants agreed that a number of problem behaviors, such as a history of violence at home, extreme charm, possessiveness, and explosive rage, are early warning signs of abuse; and (5) women describe the decision to leave the abusive relationship as the result of a breaking point. The report concludes that this study's findings should be validated in a national telephone survey of women's risk and protective factors for intimate partner violence. (Final report: 135 pages.)

AGENCY SPONSOR: National Center for Injury Prevention and Control

FEDERAL CONTACT: Lynn Short

PHONE NUMBER: (770) 488-4285

PIC ID: 6515

PERFORMER ORGANIZATION: Macro International, Incorporated, Atlanta, GA

TITLE: Linking Research and Public Health Practice: A Review of CDC's Program of Centers for Research and Demonstration of Health Promotion and Disease Prevention

ABSTRACT: In order to advance research in health promotion and disease prevention, the Centers for Disease Control and Prevention (CDC) established a program of university-based Centers for Research and Demonstration of Health Promotion and Disease Prevention. The prevention research center (PRC) program began in 1986 with funding to three universities; in 1996, there were 13 PRCs. This report examines the extent to which the PRC program is providing the public health community with workable strategies to address major public health problems in disease prevention and health promotion. The report finds that (1) by forging links with academia, CDC has created a gateway for access to a cadre of well-trained, university-based researchers who could serve to inform and collaborate with the agency and the public health community regarding health promotion and disease prevention; (2) the PRC has made substantial progress in advancing the scientific infrastructure in support of disease and health promotion policy, programs, and practices; (3) each of the centers has made some contributions toward one or more of the goals of the program (research, dissemination, and developing connections with the community and public health practitioners); (4) there are substantial differences among the PRCs in the kinds of activities they have undertaken and the success realized (only a few have made substantial progress in all areas); and (5) CDC's management of the program has been creative in the face of limited resources relative to its mandate-dogged in pursuing the mandate over a 10-year period in a bureaucratic environment that was not created or structured for the management of university-based research programs, and skilled in enhancing a sense of community and networking among the funded centers in a time of disappointing funding levels. (Final report: 67 pages, plus appendices.)

AGENCY SPONSOR: National Center for Chronic Disease Prevention and Health Promotion

FEDERAL CONTACT: Patricia Riley

PHONE NUMBER: (770) 488-5277

PIC ID: 6535

PERFORMER ORGANIZATION: National Academy of Sciences, Washington, DC

TITLE: Syphilis in the South: A Case Study Assessment in Eight Southern Communities

ABSTRACT: This two-volume report presents the results of a comparative case study of local-level syphilis prevention efforts that are conducted as part of sexually transmitted disease (STD) prevention programs supported by the Centers for Disease Control and Prevention (CDC). The case studies were conducted in eight communities in four Southern States: Alabama, Mississippi, South Carolina, and Tennessee. The project investigated past and current responses of the public and private health sectors to a syphilis epidemic that occurred in the southern United States between 1990 and 1992. Study findings were developed around four key questions: (1) who is at greatest risk of syphilis infection and transmission, (2) what institutions are best able to reach those at greatest risk, (3) what are the barriers to syphilis prevention and control in communities, and (4) what innovative ways have been found to transcend these barriers? The report finds that (1) interviewees perceived African Americans to be the demographic group most at risk for infection and transmission of syphilis, especially those who exchanged sex for drugs, homeless people, and incarcerated individuals; (2) adolescent high-risk sexual behavior was of concern as a possible precursor to infection once practitioners reached their 20s; (3) local health departments are the only community organizations focusing directly on syphilis and other STD control and prevention; (4) public health agencies assigned priority to disease control and engaged in few agency-based prevention activities, and schools were limited in their ability to be venues for STD prevention messages due to the local restrictions on sexually explicit materials in health education curricula; and (5) barriers to those at greatest risk of syphilis infection are cultural, programmatic, and political. Recommendations are offered in a manner designed to allow practical implementation. (Vol. I: 121 pages, plus appendices; Vol. II: 100 pages.)

AGENCY SPONSOR: National Center for HIV, STD and TB Prevention

FEDERAL CONTACT: Janelle Dixon

PHONE NUMBER: (404) 639-8344

PIC ID: 6579

PERFORMER ORGANIZATION: Battelle Corporation, Arlington, VA

TITLE: Evaluation of the Effectiveness of CDC Surveillance for Drug-Resistant Streptococcus pneumoniae (Baltimore, MD)

ABSTRACT: The three aims of this project were to (1) establish population-based surveillance for invasive Streptococcus pneumoniae infection; (2) evaluate the validity of the current Centers for Disease Control and Prevention (CDC) sentinel surveillance system; and (3) identify the optimal methods for determining antimicrobial resistance patterns in the United States. Active, population-based surveillance for invasive pneumococcal infection was established in all hospitals in the Baltimore metropolitan area over a 2-year period. Various additional data collection methods were employed, including bimonthly contact/followup to participating sites to obtain cases not reported spontaneously, yearly laboratory audits to identify cases not reported through active surveillance, and retrospective chart reviews. During the 2-year project period, 1,415 cases were reported: 685 in 1995 and 730 in 1996. Additionally, the contractor conducted two nasopharyngeal culture surveys of 300 children each in 5 pediatric clinics within the Baltimore metropolitan area. Sputum surveys were also conducted in three large Baltimore area hospitals. All data and isolates from the project have been sent to CDC. See also PIC ID Nos. 6580.1 and 6580.2. (Final report: 4 pages.)

AGENCY SPONSOR: National Center for Infectious Diseases

FEDERAL CONTACT: Anne Schuchat

PHONE NUMBER: (404) 639-2215

PIC ID: 6580

PERFORMER ORGANIZATION: Johns Hopkins University, Baltimore, MD

TITLE: Evaluation of the Effectiveness of CDC Surveillance for Drug-Resistant Streptococcus pneumoniae (San Antonio, TX)

ABSTRACT: Five major activities were conducted as part of this study to determine the effectiveness of the Centers for Disease Control and Prevention's (CDC's) current surveillance system for drug-resistant Streptococcus pneumoniae. These activities included (1) active population-based surveillance of invasive pneumococcal isolates in San Antonio and Bexar County, Texas for a period of 2 years; (2) microbiologic studies in which all isolates were submitted to a central laboratory for final identification, storage, and cataloging, and for eventual additional susceptibility testing; (3) microbiologic studies, in which 221 sputum isolates were collected and identified for Streptococcus pneumoniae and antibiotic resistance; (4) period prevalence surveys of non-invasive respiratory isolates; and (5) a case-control study among patients with invasive pneumococcal disease. Additionally, data were entered into appropriate data bases for further analysis at CDC. For example, invasive isolate data were entered into EpiInfo and microbiologic study data were entered into the Public Health Laboratory Information System. Results were disseminated to the community via hospital reports, local meetings, national and State meetings, manuscripts, and newsletters. See also PIC ID Nos. 6580 and 6580.2. (Final report: variously paginated.)

AGENCY SPONSOR: National Center for Infectious Diseases

FEDERAL CONTACT: Anne Schuchat

PHONE NUMBER: (404) 639-2215

PIC ID: 6580.1

PERFORMER ORGANIZATION: University of Texas, Health Science Center, San Antonio, TX

TITLE: Assessment of the Programmatic Impact of One-Percent Evaluation Studies

ABSTRACT: This three-volume report provides a systematic evaluation of the programmatic impact of 1-percent evaluation studies carried out by the Centers for Disease Control and Prevention (CDC) between 1990 and 1995. The study was also designed to elicit information about what the Office of Program Planning and Evaluation could do to enhance the impact of 1-percent evaluation studies. The study resulted in findings in three major areas: (1) the perception at CDC of evaluation in general, (2) the perception at CDC of the 1-percent process in particular, and (3) the programmatic impact to date of 1-percent evaluation studies at CDC. Among the findings were that: (1) although pressure for program accountability is mounting, evaluation still engenders anxiety among some CDC program staff-although these attitudes are changing slowly; (2) program staff are concerned that the 1-percent evaluation process in particular is unpredictable-staff do not know from one year to the next how much 1-percent funding they will receive, and the time allowed between the issuance of 1-percent guidance and the deadline for submissions is perceived as too short; (3) despite numerous obstacles to implementation, nearly 70 percent of the recommendations developed in the evaluation studies examined for this report have been considered for implementation and about 50 percent have been implemented in some form; and (4) CDC evaluations have produced programmatic impact leading to changes in practices and procedures, activities, responsibilities, changes in full-time equivalents, budgets, organizational and administrative structures, and policy. Specific recommendations are made in the report and are arrayed according to project phases (study selection, design, implementation, recommendation, and dissemination stages). See also PIC ID No. 6581.1. (Final report: 69 pages, plus appendices A-E; Appendix F separately bound; Appendices G, H, and I separately bound.)

AGENCY SPONSOR: Office of the Director

FEDERAL CONTACT: Wilma G. Johnson

PHONE NUMBER: (404) 639-7090

PIC ID: 6581

PERFORMER ORGANIZATION: Battelle Corporation, Arlington, VA

TITLE: Directory of Evaluations Administered by the CDC Office of Program Planning and Evaluation 1990-1995

ABSTRACT: This directory is one of the products of a study conducted for the Office of Program Planning and Evaluation (OPPE) of the Centers for Disease Control and Prevention (CDC) entitled "Assessment of the Programmatic Impact of (CDC) One-Percent Evaluation Studies." During a process of reviewing nearly 50 CDC evaluations completed between 1990 and 1995 and about 40 additional studies in various stages of completion, it became apparent that these studies were important to CDC program staff in particular and to the field of public health in general. Many common themes ran through these evaluations: (1) the difficulties of monitoring the performance of public health programs; (2) the growing importance of community outreach and mobilization; (3) the evolving role of health communications in CDC's work; and (4) concerns over the impact of managed care on the public health sector. This directory includes both completed and in-process studies. Project summaries reflect a project's reported status as of August 1995. The volume is arranged according to the study ID number assigned to each project as it is initiated through OPPE. See also PIC ID No. 6581. (Final report: variously paginated.)

AGENCY SPONSOR: Office of the Director

FEDERAL CONTACT: Nancy Cheal

PHONE NUMBER: (404) 639-7095

PIC ID: 6581.1

PERFORMER ORGANIZATION: Battelle Corporation, Arlington, VA

TOP OF DOCUMENT

Food and Drug Administration

Contents

Report of the Prescription Drug User Fee Act (PDUFA) of 1992-Fiscal Year 1997 Report to Congress

Economic Impact Analysis of Regulations Under the Mammography Quality Standards Act of 1992

Mammography Quality Standards Act: Second Annual Government Entity Declaration Program Audit

Product Review Performance Report

Evaluation of Device Performance in the Determination of Substantial Equivalence

Cost of Restrictions on Gulf Oyster Harvesting for Control of Vibrio vulnificus-Caused Disease

Health Transfers: An Application of Health-Health Analysis to Assess Food Safety Regulation

Preliminary Investigation into the Morbidity and Mortality Associated with the Consumption of Fruit and Vegetable Juices

TITLE: Report of the Prescription Drug User Fee Act (PDUFA) of 1992-Fiscal Year 1997 Report to Congress

ABSTRACT: Fiscal year 1997 (FY 97) marks the fifth and final year of the implementation of the Prescription Drug User Fee Act (PDUFA) of 1992, which authorizes the collection of fees from the pharmaceutical industry in order to facilitate the Food and Drug Administration's (FDA's) timely review of human drug applications. The Act mandates that FDA prepare yearly reports for each fiscal year for which fees are collected. The fifth annual report, prepared in response to that mandate, covers FY 97 and reports on how well the FDA has accomplished the performance goals it established for FY 97. These performance-related goals were jointly established by the FDA and the pharmaceutical industry and fall into three main categories: (1) elimination of overdue backlogs, (2) building excellence into the review process, and (3) achievement of high performance. Seven goals were in effect for FY 97. The findings in the report show that the successes reported last year have been confirmed and surpassed. The report finds that (1) PDUFA has resulted in more and better applications filed, which can be accepted immediately and reviewed and approved more quickly; (2) nearly all applications now go directly into the review process (due to the increased quality of the applications, very few are refused); (3) there are more approvals on first review, which demonstrates that submission quality is a key factor in achieving timely reviews; and (4) the ultimate approval time for applications in the PDUFA years has dramatically decreased. While only a preliminary performance assessment of FY 97 is possible at this time, it is clear that the Act's stringent review goals have assured that new pharmaceutical products are getting onto the market at an unprecedented rate without compromising safety. See also PIC ID Nos. 6079 and 6079.1. (Final report: 10 pages, plus appendices.)

AGENCY SPONSOR: Office of Planning and Evaluation

FEDERAL CONTACT: Dennis Hill

PHONE NUMBER: (301) 827-5255

PIC ID: 6079.2

PERFORMER ORGANIZATION: Office of Planning & Evaluation, Food and Drug Administration, Washington, DC

TITLE: Economic Impact Analysis of Regulations Under the Mammography Quality Standards Act of 1992

ABSTRACT: The Food and Drug Administration (FDA) is charged with ensuring that accessibility to quality mammography services is maintained for all facilities performing mammographies in accordance with the Mammography Quality Standards Act of 1992 (MQSA). Under this statute, the FDA is required to certify and annually inspect over 10,000 mammography facilities. Prior to the passage of the MQSA, FDA's oversight responsibility was limited to approving new mammographic processors, x-rays, and tubing. However, MQSA requires that facilities within its jurisdiction meet standards developed by the agency to ensure a reasonable level of quality in order to achieve certification by the agency. FDA is concerned that recent trends by third-party payers to attract payments for mammography services have severely limited the ability of facilities to absorb any cost increases caused by the quality standards. These cost increases, as well as excess capacity, may cause low-volume facilities to leave the industry. This project addresses these problems in two ways. First, it measures the effect of standards on patient access to mammography, especially in rural areas; second, it develops a cost model that will be used to predict the impact of proposed standards on the costs of providing mammography. The contractor reviewed records from over 10,000 facilities that had applied for accreditation and over 500 that had ceased operations. In the Spring of 1996, the contractor drafted the results of a cost-benefit analysis, which FDA managers will use to develop a performance-oriented regulatory policy that will be coordinated with existing customers.

AGENCY SPONSOR: Office of Planning and Evaluation

FEDERAL CONTACT: Steve Tucker

PHONE NUMBER: (301) 827-5340

PIC ID: 6080

PERFORMER ORGANIZATION: Eastern Research Group, Incorporated, Arlington, VA

TITLE: Mammography Quality Standards Act: Second Annual Government Entity Declaration Program Audit

ABSTRACT: The Food and Drug Administration's (FDA's) Division of Mammography Quality and Radiation Programs requested that the Division of Planning, Analysis, and Finance conduct an audit of the Mammography Quality Standards Act (MQSA) Government Entity (GE) Declaration Program. There are two main goals of the audit: (1) to notify facilities that they will be held accountable for their self-certification of fee exemption, and (2) to determine the rate of compliance with the program. A random sample of 90 facilities out of the 989 claiming GE status was selected for examination, and a random sample of 20 (out of 72) new GE facilities was also examined. The audit revealed that (1) only 3 of the 90 GE facilities had claimed GE status incorrectly, and only 1 of the 20 new GE facilities had claimed incorrect status; and (2) the 3 GE facilities (as well as the one new GE facility) that did not meet FDA's fee-exempt status claimed that exemption based on funding from the Centers for Disease Control and Prevention (CDC) Breast and Cervical Cancer Prevention Program, which is the least common way to claim fee exemption. Based on these results, the audit concludes that the GE Declaration program appears to be quite effective. It also suggests several recommendations based on the findings of the audit. These include that: (1) the results should be published to allow the impact of the audit to reach many, if not all, the facilities, to discourage any future misrepresentations, and to demonstrate that FDA will exercise its authority; (2) the sample size of future audits should be smaller and should contain a larger portion of those facilities claiming fee exemption based solely on funding from the CDC; and (3) the MQSA contractor in charge of billing should continue to thoroughly examine each GE Declaration form to ensure that it identifies the body operating the facility. (Final report: 3 pages, plus appendices.)

AGENCY SPONSOR: Center for Devices and Radiological Health

FEDERAL CONTACT: Stephanie Wilkinson

PHONE NUMBER: (301) 594-1284

PIC ID: 6730

PERFORMER ORGANIZATION: Center for Devices and Radiological Health, Rockville, MD

TITLE: Product Review Performance Report

ABSTRACT: This is the first annual report analyzing the Food and Drug Administration's (FDA's) performance in reviewing applications for human drugs, biological products, medical devices, food and color additives, and veterinary drugs. In the past, FDA has measured the length of review using measures that varied for different product categories, making it difficult to provide an agency-wide assessment of the FDA's performance. Responding to legislation such as the Prescription Drug User Fee Act of 1992 (PDUFA), as well as to requirements under the Federal Food, Drug, and Cosmetic Act, the FDA has developed clear performance goals for reviewing new product applications. Setting these goals has provided a valuable management tool for identifying performance expectations and assessing achievements, and for meeting the requirements of the Government Performance and Results Act of 1993. FDA has also developed performance measures and reporting conventions that can be used to report on product review performance consistently across product categories. The report looks at four different measures of the FDA's performance: (1) how often FDA is meeting statutory and nonstatutory time frames for review of each category of products, (2) the median length of time to approval for each product category, (3) the total number of products approved in each category, and (4) the number of applications for which an action was overdue at the end of the fiscal year (FY). It provides FDA's on-time performance for FY 1993, 1994, and 1995 and performance data for the other three categories for FY 1993 to 1996. The report finds that (1) for most categories of products, FDA has improved its performance since FY 1993; (2) the greatest improvements in performance have occurred for product reviews covered by PDUFA; (3) significant progress has been made in reviewing applications for medical devices; and (4) on-time performance for reviews of non-PDUFA applications should be improved.

AGENCY SPONSOR: Office of Planning and Evaluation

FEDERAL CONTACT: Paul Coppinger

PHONE NUMBER: (301) 827-5292

PIC ID: 6731

PERFORMER ORGANIZATION: Food and Drug Administration, Rockville, MD

TITLE: Evaluation of Device Performance in the Determination of Substantial Equivalence

ABSTRACT: Many diagnostic medical devices that reach the market are based on diagnostic agreement with some device already on the market that targets the same attribute. Agreement is often assessed by side-by-side testing of the same subjects or specimens, even if the true disease status is unknown. If each new test is compared with the most recent predicate, what would be the effect on device performance over time? Would it deteriorate, remain the same, or improve? This analysis attempts to address these questions using statistical measures of sensitivity, specificity, and predictive values. A Monto Carlo simulation was used to study this question of "predicate (pre-Amendments or reclassified post-Amendments) creep." Contingency tables were randomly generated using a range of conditional probabilities and prevalence representing screening and confirmatory test data. When a new test was found to be equivalent to the current predicate, it became the new predicate. After 50 new predicates were found, a forecast procedure was applied to the sequence of 50 performance parameters in order to evaluate the long-term trend.

AGENCY SPONSOR: Center for Devices and Radiological Health

FEDERAL CONTACT: John Dawson

PHONE NUMBER: (301) 827-0201

PIC ID: 6734

PERFORMER ORGANIZATION: Center for Devices and Radiological Health, Rockville, MD

TITLE: Cost of Restrictions on Gulf Oyster Harvesting for Control of Vibrio vulnificus-Caused Disease

ABSTRACT: This study for the Food and Drug Administration (FDA) estimates the costs of a seasonal harvesting restriction on Gulf of Mexico oysters and a seasonal marketing restriction on the sale of Gulf oysters for raw consumption. These measures would control oyster-related illnesses in humans known to be caused by the bacterial species Vibrio vulnificus, which naturally occurs in estuarine waters and is a normal flora for oysters. During this study, the 1995 Interstate Shellfish Sanitation Conference (ISSC) adopted a Vibrio vulnificus interim control plan for oysters and the report compares the costs of complying with the ISSC plan with FDA's suggested alternatives. The ISSC interim control plan (1) states that oysters must be under ambient refrigeration at 45 degrees Fahrenheit within a specified time if they are harvested from State waters that were the confirmed source for two or more Vibrio vulnificus illnesses; (2) establishes action levels for growing areas within affected States; and (3) varies the time-to-refrigeration requirement according to the average monthly maximum water temperature. The report estimates the economic impact of alternative remedies for controlling Vibrio vulnificus-related illnesses and concludes that (1) the in-shell marketing restriction might have imposed large economic impacts on the Gulf oyster industry and other related sectors of the economy; (2) the ISSC time and temperature controls would have smaller aggregate economic impacts than the marketing restriction option but the distributional effects of this policy should be considered; and (3) the ISSC impacts are sensitive to specific assumptions about enforcement of the rule and types of responses, engendered by the regulated entities. (Final report: variously paginated.)

AGENCY SPONSOR: Center for Food Safety and Applied Nutrition

FEDERAL CONTACT: Richard Williams

PHONE NUMBER: (202) 205-5667

PIC ID: 6856

PERFORMER ORGANIZATION: Research Triangle Institute, Research Triangle Park, NC

TITLE: Health Transfers: An Application of Health-Health Analysis to Assess Food Safety Regulation

ABSTRACT: Public health policy decisions usually focus on reducing a target risk. However, complying with regulations to reduce a risk is expensive. Because income influences health, Health-Health Analysis (HHA) was developed to compare the targeted reduction in mortality to the induced mortality from income losses. This study applies HHA to assess a potential Gulf oyster harvesting ban to control mortality from the consumption of oysters contaminated by Vibrio vulnificus. The report estimates the induced deaths using Anderson et al.'s estimate of economic losses and Keeney's estimate of the functional relations between income and mortality. The report shows that while a 7-month Gulf oyster harvesting ban might eliminate the estimated 17 oyster-related deaths annually, it may also cause as many as 12 income-loss-related deaths. The induced deaths occur principally among white males and black females. The report concludes that (1) HHA may give decisionmakers reason to pause considering the high costs of a harvesting ban and that consumption of raw oysters is completely voluntary; and (2) this illustration shows that HHA serves as a useful policy analysis tool. (Final report: 26 pages.)

AGENCY SPONSOR: Center for Food Safety and Applied Nutrition

FEDERAL CONTACT: Richard Williams

PHONE NUMBER: (202) 205-5667

PIC ID: 6857

PERFORMER ORGANIZATION: Center for Food Safety and Applied Nutrition, Washington, DC

TITLE: Preliminary Investigation into the Morbidity and Mortality Associated with the Consumption of Fruit and Vegetable Juices

ABSTRACT: Americans consumed 2.3 billion gallons of the major fruit and vegetable juices in 1995. Recent outbreaks of illnesses associated with the consumption of juices demonstrate their potential as a human health hazard. As part of the cost-benefit analysis for both the Hazard Analysis and Critical Control Point (HACCP) and labeling rules associated with fruit and vegetable juices, the Center for Food Safety and Applied Nutrition's working group was asked to investigate the morbidity and mortality associated with human consumption of juice drinks. The standard procedure for estimating human health benefits is to (1) estimate the baseline numbers of illnesses and deaths associated with the compound to be controlled, (2) estimate the likely reductions in those illnesses and deaths associated with various proposed control options, and (3) estimate the values associated with the reduced illnesses and deaths. This report estimates the baseline numbers of illnesses and deaths likely to be associated with the consumption of juice products. The study (1) describes juice products, (2) estimates juice consumption, (3) discusses production methods, (4) explains how hazards might be introduced into juice, (5) discusses evidence on illness from consuming juices, (6) describes human health effects caused by selected microbial pathogens, and (7) discusses the physical and chemical hazards associated with juices. Juices accounted for 447 laboratory-confirmed cases of illness associated with microbial pathogens, according to Centers for Disease Control and Prevention outbreak data and U.S. Food and Drug Administration recall data from 1993 through 1996. The other hazards, mostly physical and chemical, that have been found in juices have been sporadic and associated with fewer cases than the microbial pathogens. (Final report: 65 pages, plus appendices.)

AGENCY SPONSOR: Center for Food Safety and Applied Nutrition

FEDERAL CONTACT: Richard Williams

PHONE NUMBER: (202) 205-5667

PIC ID: 6858

PERFORMER ORGANIZATION: Center for Food Safety and Applied Nutrition, Washington, DC

TOP OF DOCUMENT

Health Care Financing Administration

Contents

Limited-Service Hospital Pioneers: Challenges and Successes of the Essential Access Community Hospital/Rural Primary Care Hospital (EACH-RPCH) Program and Medical Assistance Facility (MAF) Demonstration

Comparative Study of the Use of EPSDT and Other Preventive and Curative Health Care Services by Children Enrolled in Medicaid: Final Project Synthesis Report

Costs and Consequences of Case Management for Medicare Beneficiaries: Final Report

Evaluation of the Utah Prepaid Mental Health Plan

Monitoring and Evaluation of the Medicare Cataract Surgery Alternate Payment Demonstration: Final Evaluation Report

Evaluation of the Demonstration to Improve Access to Care for Pregnant Substance Abusers

Evaluation of the Medicaid Uninsured Demonstrations

Access to Physicians' Services for Vulnerable Beneficiaries: Impact of the Medicare Fee Schedule

A Study of the Appropriateness, Process of Care, and Outcomes of Medical Care for Medicaid Patients

An Evaluation of Implementation of the Medicaid Community Supported Living Arrangements (CLSA) Program in Eight States

Evaluation of the Community Nursing Organization Demonstration Interim Evaluation Report

Evaluation of the Effectiveness of the Operation Restore Trust Demonstrations

The Impact of Report Cards on Employees: A Natural Experiment

Consequences of Paying Medicare HMOs and Health Care Prepayment Plans (HCPPs)

Toward a Prospective Payment System for Outpatient Services: Implementation for Outpatient Services-- Implementation of APGs by State Medicaid Agencies and Private Payers

Payment of Pharmacists for Cognitive Services: Results of the Washington State C.A.R.E. Demonstration Project

Assessing the Viability of All-Payer Systems for Inpatient Hospital Services

Second Update of the Geographic Practice Cost Index: Final Report

Validation of Nursing Home Quality Indicators Study

1995 Influenza Immunizations Paid for by Medicare: State and County Rates

Impact of the Medicare Fee Schedule on Teaching Physicians

TITLE: Limited-Service Hospital Pioneers: Challenges and Successes of the Essential Access Community Hospital/Rural Primary Care Hospital (EACH-RPCH) Program and Medical Assistance Facility (MAF) Demonstration

ABSTRACT: Small communities in rural America are struggling to maintain health care resources in the face of increasing competition and a dwindling population and economic base. In response to the threatened survival of small rural hospitals, the Health Care Financing Administration has supported two initiatives to introduce innovative limited-service hospitals: the Essential Access Community Hospital/Rural Primary Care Hospital (EACH-RPCH) program, implemented in seven States; and the Medical Assistance Facility (MAF) demonstration in Montana. Both establish a new Medicare provider that is exempt from specific key requirements of participation, and both offer enhanced reimbursement. Other States have looked to these two programs as possible models for their rural communities. This report documents the evaluation of the EACH-RPCH program from its start in October 1991 through September 1994. The report assesses the implementation process, initial operating experience, and potential impact of the programs on access and cost as related to limited hospital service. The introductory chapter reviews the background of the EACH-RPCH and MAF programs, discusses the evaluation issues to be examined, and summarizes the evaluation methodology and data sources. Chapters II and III examine the development of networks and the shift to limited service licensure under the EACH-RPCH program. Chapter IV reviews the implementation and operation of the MAF demonstration. Chapters V and VI focus on the two programs' impact on access to care and mechanisms for quality assurance, respectively. The report concludes with Chapter VII, which assesses the impact of both programs on facility finances and the Medicare program. (Final report: 260 pages, plus appendices.)

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Sheldon Weisgrau

PHONE NUMBER: (410) 786-6675

PIC ID: 5200.1

PERFORMER ORGANIZATION: Mathematica Policy Research, Incorporated, Plainsboro, NJ

TITLE: Comparative Study of the Use of EPSDT and Other Preventive and Curative Health Care Services by Children Enrolled in Medicaid: Final Project Synthesis Report

ABSTRACT: This report includes (1) a description of the Medicaid program in Michigan, Georgia, Tennessee, and California and these States' responses to the Omnibus Budget Reconciliation Act of 1989 provisions relating to children's and Medicaid coverage and on the Early Periodic Screening, Diagnosis, and Treatment (EPSDT) program; (2) results of analyses of the impact of these responses on participating providers and enrolled children; and (3) an assessment of the national context through a review of the national survey data analyses. It finds that (1) State efforts to increase provider participation in Medicaid and EPSDT included recruitment activities, training in billing, and increased fees; (2) the States' efforts were successful in increasing provider participation and average caseloads among all types of providers, except for dentists in Michigan, which declined 10 percent from 1989 to 1992; (3) these increases may not have been adequate to meet the increased demand from a growing caseload; (4) the ratios of Medicaid children to participating providers and the percentages of counties with provider shortages rose from 1989 to 1992 in all four study States; (5) counting all EPSDT and all other well-child visits paid for by Medicaid, only 42 to 54 percent of Medicaid children were recommended to have preventive care visits, and of these, only 36 to 59 percent made the recommended visits; (6) compared with other low-income children, Medicaid coverage increased service use and improved access to illness-related care; and (7) Medicaid children had very low completion rates for age-appropriate immunizations, from 48 percent in Michigan to 61 percent in Georgia in 1992. The report concludes that State efforts to expand the EPSDT provider base and to enhance outreach had significant impacts on the use of preventive, diagnostic, and treatment services. See also PIC ID Nos. 6236-6236.6 and 6236.8-6236.A. (Final report: 49 pages.)

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Feather Davis

PHONE NUMBER: (410) 786-6590

PIC ID: 6236.7

PERFORMER ORGANIZATION: MEDSTAT Group, Cambridge, MA

TITLE: Costs and Consequences of Case Management for Medicare Beneficiaries: Final Report

ABSTRACT: This report studies the appropriateness of providing case management services to beneficiaries with catastrophic illnesses and high medical costs. The study tests case management as a way of controlling costs in the fee-for-service sector. These demonstrations, mandated by the Omnibus Reconciliation Act of 1990, and are aimed at a number of chronic illnesses, including congestive heart failure. The resulting demonstrations were implemented in three sites, began operations in October 1993, and continued through November 1995. The target conditions and case management protocols differed in each site, though all three generally focused on increased education regarding proper patient monitoring and management of the target chronic condition. All three sites anticipated reduced hospitalizations and medical costs compared to the beneficiary control groups. (Final report: 216 pages, plus appendices.)

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Leslie M. Greenwald

PHONE NUMBER: (410) 786-6502

PIC ID: 6291

PERFORMER ORGANIZATION: Mathematica Policy Research, Incorporated, Plainsboro, NJ

TITLE: Evaluation of the Utah Prepaid Mental Health Plan

ABSTRACT: This report is an evaluation of a demonstration project conducted in Utah, in which three mental health centers provided mental health services to all Medicaid beneficiaries in their catchment areas (these areas include over 50 percent of all Utah Medicaid beneficiaries). The hope was that this program would control the cost inflation and improve patient outcomes in mental health services. The evaluator examines three areas: (1) organizational and financial characteristics and evolution over time, (2) impact on the organization of service delivery and use of services, and (3) financial impact on providers and the Medicaid program in Utah. The development of the demonstration proceeded fairly smoothly, albeit somewhat more slowly than planned. Three centers chose to enter the demonstration and eight chose to not join. Several changes in the program's environment affected the development during the initial 3 years. (Final report: 68 pages.)

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Paul J. Boben

PHONE NUMBER: (410) 786-6629

PIC ID: 6293

PERFORMER ORGANIZATION: University of Minnesota, Minneapolis, MN

TITLE: Monitoring and Evaluation of the Medicare Cataract Surgery Alternate Payment Demonstration: Final Evaluation Report

ABSTRACT: This report assisted the Health Care Financing Administration (HCFA) in tracking the 3-year demonstration that tested the concept of a single global Medicare payment for outpatient cataract procedures. The payment amount represented payment for the physician, facility, and intraocular lens costs. The analysis portion of the project tested whether there were any net savings to the Medicare program, changes in the use of services included and excluded from the bundle, beneficiary satisfaction, and quality of care. The evaluation found that the project was reasonably successful in encouraging provider flexibility and in managing the bundle of services. There was ample anecdotal evidence that the cost effectiveness incentives were working; e.g., cost reduction efforts by surgeons in the techniques and time required to perform the surgery and actions to standardize the interocular lenses and other supplies and materials. While the effort to become a part of the demonstration was substantial, once operations began, government activities related to pricing were greatly reduced and limited to simple verification and payment of the global fee. The evaluator was not able to measure appropriateness, but in terms of quality control and utilization review the demonstration was successful in that the outcomes were unchanged. The project did produce some insights into the strengths and weaknesses of particular incentives in encouraging providers to participate. However, it provided limited new information on what factors influenced beneficiaries to participate.

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Cynthia K. Mason

PHONE NUMBER: (410) 786-6680

PIC ID: 6295

PERFORMER ORGANIZATION: Abt Associates, Incorporated, Cambridge, MA

TITLE: Evaluation of the Demonstration to Improve Access to Care for Pregnant Substance Abusers

ABSTRACT: This report evaluates demonstration project effectiveness in (1) outreach and assessment; (2) expansion, integration, and coordination of program services; and (3) improvement of client case management. The evaluator examined access to prenatal care and substance abuse treatment services and assessed the effects of these services on the health of drug-addicted pregnant women and birth outcomes of their infants. The evaluation showed that the number of abusers enrolled in the demonstrations was low relative to all pregnant substance abusers in the area, since women were reluctant to be identified. The project found higher enrollment rates in States that implemented broad-based outreach efforts; higher levels of and greater retention in substance abuse treatment resulted in higher birth weight infants. (Final report: 104 pages, plus appendices.)

AGENCY SPONSOR: Center for Beneficiary Services

FEDERAL CONTACT: Suzanne Rotwein

PHONE NUMBER: (410) 786-6621

PIC ID: 6297

PERFORMER ORGANIZATION: Mathematica Policy Research, Incorporated, Washington, DC

TITLE: Evaluation of the Medicaid Uninsured Demonstrations

ABSTRACT: This project evaluated three demonstration project conducted in Maine, South Carolina, and Washington State. These demonstrations, implemented in response to a Congressional mandate under section 4745 of the Omnibus Budget Reconciliation Act of 1990, tested the effect of allowing States to extend Medicaid coverage to low-income families. The evaluation resulted in a series of annual reports, an interim and a final report to Congress, and a final evaluation report. The project examined intra- and inter-site processes and outcomes, including (1) the ability of the programs to enroll significant numbers of eligible persons; (2) conditions under which eligible persons and their families are willing to participate in such programs, given their scarce financial resources; (3) ability of the programs to induce adequate numbers of providers to ensure the availability of necessary services at appropriate utilization levels; (4) willingness of employers to participate in the programs and the conditions under which they participate or choose not to participate; (5) the program's effect on service utilization and health outcomes of participants; (6) the cost-effectiveness of such programs for the various public and private interests; and (7) the extent to which the demonstration's interventions could be applied nationally to assist in achieving program goals. An initial series of site visits was conducted during 1993. The first annual report in the series used data collected during these site visits and data from State-administered baseline surveys to describe the implementation phase and early operational phase of the demonstrations.

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: James Hadley

PHONE NUMBER: (410) 786-6626

PIC ID: 6298

PERFORMER ORGANIZATION: Health Economics Research, Incorporated, Waltham, MA

TITLE: Access to Physicians' Services for Vulnerable Beneficiaries: Impact of the Medicare Fee Schedule

ABSTRACT: In 1992, Medicare introduced the Medicare Fee Schedule (MFS), which alters the relative prices paid to physicians for services and arguably the incentives to provide one service over another. This project evaluates the impact of the MFS on individual access to health care. Since the introduction of the MFS, the Health Care Financing Administration and the Physician Payment Review Commission have monitored access to physicians' services and have failed to document any barriers to health care access resulting from the MFS. This study builds on this work by (1) oversampling groups of beneficiaries thought to be particularly vulnerable to payment changes, (2) evaluating changes in treatment patterns for specific episodes of care, and (3) conducting multivariate analyses that measure actual payment changes over time rather than expected changes due to MFS. A stratified random sampling design took advantage of the differential impacts of the MFS across geographic areas and ensured adequate numbers of vulnerable beneficiaries. The study divided all geographic areas into six groups based on their expected 1992 payment changes under MFS, compared with the earlier payment scheme. The report finds that (1) there is little evidence that the MFS either improved or worsened health care access for Medicare beneficiaries; (2) while some vulnerable groups experienced MFS-related changes in access, the actual size of these changes is relatively small and is not consistently negative or positive; (3) substantial access gaps exist for vulnerable beneficiaries and there is evidence that access to primary care may have worsened for dual Medicaid-eligibles; and (4) Medicare beneficiaries reduced out-of-pocket payments over the 1991-1993 period, including lower copayments and lower extra billing liability. (Final report: variously paginated, plus appendices.)

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Renee Mentnech

PHONE NUMBER: (410) 786-6692

PIC ID: 6301

PERFORMER ORGANIZATION: Center for Health Economics Research, Waltham, MA

TITLE: A Study of the Appropriateness, Process of Care, and Outcomes of Medical Care for Medicaid Patients

ABSTRACT: The Congress requires the Health Care Financing Administration to report on the relative quality of care in the Medicaid program. This study assesses the variation that exists in the rate of performance of selected treatments and procedures provided to Medicaid beneficiaries for small areas within and between States. The report's first phase shows that the overall rate of inpatient use in the Medicaid population is higher than the rate expected based on non-Medicaid utilization patterns. In the report's second phase, data are abstracted from medical records at 118 hospitals in California, Georgia, and Michigan for a selected series of index diagnoses: pediatric asthma, hysterectomy, and complicated labor and delivery. This second phase develops five indicators to assess quality: (1) appropriateness to hospitalize or perform a procedure, (2) process of care, (3) outcomes of treatments and procedures, (4) severity at time of admission, and (5) intensity of care. Medicaid-covered children in two States, and women in all three States, are younger and represent more minority groups than their privately-insured counterparts. The appropriateness of the decision to hospitalize a child with asthma, perform a hysterectomy, and do a Cesarean section in the presence of labor and delivery complications are comparable between Medicaid and privately-insured patients. Most important, the outcomes of care for the two groups were essentially similar, for each condition, across the three States. Medicaid patients were more seriously ill at admission, which suggests that the Medicaid-covered patients may not be practicing the same level of preventative care as their privately-insured counterparts. As a result, the differences in patient outcomes were not a result of the inpatient quality of care provided. The inpatient care for the Medicaid patients was considered adequate and comparable with care received by privately-insured patients. (Final report: 46 pages.)

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Beth Benedict

PHONE NUMBER: (410) 786-7724

PIC ID: 6302

PERFORMER ORGANIZATION: MEDSTAT Group, Cambridge, MA

TITLE: An Evaluation of Implementation of the Medicaid Community Supported Living Arrangements (CLSA) Program in Eight States

ABSTRACT: This report describes the effectiveness of developing a continuum-of-care concept as an alternative to the Medicaid-funded residential services provided to individuals with mental retardation and related conditions. The program serves individuals who are living in the community either independently, with their families, or in homes with three or fewer other individuals receiving the same services. These services were provided in eight States between fiscal years 1991 and 1995. By the end of the program only 73 percent of the expected number of individuals were receiving services in the Community Supported Living Arrangement (CSLA), and the expenditures were only 58 percent ($38.4 million) of the amount authorized. A number of factors contributed to this smaller-than-expected program: (1) late 1990 authorization of CSLA and subsequent delay in regulations, application procedures, and State selection put back the first program enrollments by 16-22 months; (2) simultaneously expanded Federal latitude allowed rapid growth of Medicaid Home and Community Based Services; (3) implementers found that planning and designing services "one person at a time" were slower than expected; (4) States experienced delays in establishing the new systems of quality assurance required by the CSLA program; and (5) recruitment was sometimes delayed by the inclusion of groups of persons who were not traditional users of developmental disabilities services and who were not already known to the system. The report concludes that specific financing problems limit the effectiveness of the program. (Final report: 54 pages.)

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Samuel L. Brown

PHONE NUMBER: (410) 786-6667

PIC ID: 6304

PERFORMER ORGANIZATION: MEDSTAT Group, Cambridge, MA

TITLE: Evaluation of the Community Nursing Organization Demonstration Interim Evaluation Report

ABSTRACT: This report examines a set of demonstration projects mandated in the Omnibus Budget Reconciliation of 1987. The legislation directed the establishment of demonstration projects to test a capitated, nurse-managed system of care. The two fundamental elements of the demonstration are capitation payment and nurse case management. It was designed to promote timely and appropriate use of community health services and to reduce the use of costly acute care services. By October 1995, more than 5,400 beneficiaries were enrolled in the four Community Nursing Organizations (CNOs). The evaluation report covers the first 15 months of operation. The applicants have been healthier and more independent, on average, than the general Medicare population in their respective market areas. Analyses of measures of death, physical functioning, and satisfaction with care detected no statistically significant differences. Analysis of a subsample of persons who were impaired in Activities of Daily Living indicated that members of the CNO treatment group were more likely to improve (or less likely to deteriorate further) than members of the control. An even stronger result, in the same direction, was found for individuals who were limited in Instrumental Activities of Daily Living. (Final report: variously paginated.)

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Melissa Hulbert

PHONE NUMBER: (410) 786-8494

PIC ID: 6306

PERFORMER ORGANIZATION: Abt Associates, Incorporated, Cambridge, MA

TITLE: Evaluation of the Effectiveness of the Operation Restore Trust Demonstrations

ABSTRACT: Operation Restore Trust (ORT) was jointly developed by the Office of Inspector General (OIG), the Administration on Aging (AoA), and the Health Care Financing Administration (HCFA) as a model to demonstrate improved methods for investigation and prosecution of fraud and abuse in health programs. ORT targets a set of initiatives that focus on a few States, a few areas of the Medicare benefit, and a few standard operating procedures (especially failures to coordinate among the most central agencies and to exploit available data resources). The report finds that (1) ORT is associated with a net increase of more than 100 percent in ORT-type cases investigated by the OIG; (2) these cases should generate substantial incremental receivables, some fraction of which will translate into true savings as collections are made; (3) HCFA and its contractors had a series of successful projects under ORT, with notably large collections or avoidances of payment accompanying many of these efforts, although it cannot be said with certainty that these savings would not have been realized without ORT; (4) ORT encourages several important policy changes made during, or immediately following, the demonstration; (5) ORT may have important "sentinel effects" (e.g., ORT may have deterred providers from certain kinds of suspect behavior); (6) there are considerable process effects of ORT; (7) giving the AoA a fraud and abuse mission led to active training, outreach, and education efforts; and (8) the infrastructure of information for combating fraud and abuse has significantly improved, although these improvements include many efforts outside ORT itself. The evaluation concludes that ORT turned around certain troubling trends of the early 1990s and has brought new vigor to government efforts to combat health care fraud and abuse.

AGENCY SPONSOR: Office of Financial Management

FEDERAL CONTACT: Edward Norwood

PHONE NUMBER: (410) 786-6571

PIC ID: 6311

PERFORMER ORGANIZATION: Abt Associates, Incorporated, Cambridge, MA

TITLE: The Impact of Report Cards on Employees: A Natural Experiment

ABSTRACT: This report seeks to determine whether the dissemination of information about health plans to consumers (who choose health plans within a managed care competition framework) would influence their knowledge of plan characteristics, attitudes toward the plans, or choice of plan. Investigators found no "report card" effect on employees' knowledge of health plans, their attitude about quality of health plans, or their choices in selecting a managed care health plan. The study did find that certain health plan characteristics (such as price) were strongly related to health plan choice. The investigators suggest that the current versions of health plans are works in progress and that consumers do not seem to be influenced by the information in any of the ways measured in this study. (Final report: 41 pages.)

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Sherry Terrell

PHONE NUMBER: (410) 786-6601

PIC ID: 6312

PERFORMER ORGANIZATION: Park Nicollet Medical Foundation, St. Louis Park, MN

TITLE: Consequences of Paying Medicare HMOs and Health Care Prepayment Plans (HCPPs)

ABSTRACT: This study examined the operational aspects of health care prepayment plans (HCPPs) and the cost-effectiveness of cost health maintenance organizations (HMOs) and HCPPs. The report contains a case study of HCPPs: it looks at plans' utilization management and quality assurance programs, marketing practices, and contingency plans in case of insolvency. It also examines the cost-effectiveness to the Health Care Financing Administration (HCFA) of cost HMOs and HCPPs with respect to the fee-for-service and risk contracting program during calendar year 1993. The report found that costs to HCFA increased substantially under both the cost HMO and HCPP programs. Cost increases relative to fee-for-service were larger than those relative to the risk-based approach. Only a handful of the 63 plans examined appeared to have generated savings for HCFA. The main reason for the cost increase appears to be a lack of incentive for plans to drive hard bargains with physicians on prices. Furthermore, the report found that most of the cost-based plans experienced favorable selection, where the relatively more healthy segments of the population are enrolled. (Final report: 178 pages, plus appendices.)

AGENCY SPONSOR: Center for Health Plans and Providers

FEDERAL CONTACT: Ron Lambert

PHONE NUMBER: (410) 786-6624

PIC ID: 6314

PERFORMER ORGANIZATION: Mathematica Policy Research, Incorporated, Plainsboro, NJ

TITLE: Toward a Prospective Payment System for Outpatient Services: Implementation for Outpatient Services-Implementation of APGs by State Medicaid Agencies and Private Payers

ABSTRACT: This report studies a Medicaid outpatient prospective payment system, which groups patients for payment purposes rather than paying on a cost basis. These groups are similar in concept to diagnosis-related groups, which form the heart of Medicare's prospective payment system for inpatient care. It involved a case study of Iowa's implementation of the ambulatory patient group (APG) system and an analysis of the reimbursement methodology. This report has two components: an in-depth study of three operational APG systems (based on site visits) and telephone interviews with other payers that have implemented or planned to implement the system. All six payers with operational experience tailored their APG systems to their own priorities and markets. They all implemented the system without major incident. The payers reported success in reducing outpatient costs, where that was the immediate goal, and they believe the system encourages higher-cost facilities to reduce costs and rewards lower-cost facilities. Providers' views were more mixed. Though they reported generally adequate overall payment, they also viewed the system as complex, had difficulty calculating expected payment, and often let payer computer systems group related claims rather than consolidating them prior to submission, as was the intent of the system. No evidence was found of much behavioral response by providers, but that could change if the system is implemented by a large payer such as HCFA. (Final report: 78 pages.)

AGENCY SPONSOR: Center for Health Plans and Providers

FEDERAL CONTACT: Joe Cramer

PHONE NUMBER: (410) 786-6676

PIC ID: 6320

PERFORMER ORGANIZATION: Mathematica Policy Research, Incorporated, Plainsboro, NJ

TITLE: Payment of Pharmacists for Cognitive Services: Results of the Washington State C.A.R.E. Demonstration Project

ABSTRACT: The Washington Cognitive Activities and Reimbursement Effectiveness (C.A.R.E.) Project was a demonstration authorized under the Omnibus Budget Reconciliation Act of 1990 to assess the impact of a financial incentive on pharmacists' performance of cognitive services. The premise of this demonstration was that direct reimbursement for pharmacists' cognitive services will remove financial barriers associated pharmacists' provision of these services and result in increased performance of cognitive services, with a subsequent impact on costs and outcomes. Data for the demonstration were drawn from three groups of pharmacies located throughout the State of Washington, each of which contained about 100 pharmacy sites. Pharmacists in the first two groups documented cognitive services for Medicaid patients using a problem-intervention-result format. The demonstration phase lasted from February 1994 through September 1995 and resulted in the documentation of 20,240 cognitive service events. The project found that (1) Group A pharmacists, who received financial incentives, consistently reported higher cognitive service intervention rates than did Group B, who did not receive financial incentives; (2) about half of all documented cognitive services problems were for patient-related problems, while 32.6 percent were for drug-related problems, 17.6 percent for prescription-related problems, and 1.4 percent for non-drug-related problems; (3) a drug therapy change occurred as a result of 28 percent of all cognitive services; (4) for each cognitive service associated with any type of drug therapy change, the average downstream drug cost savings was about $13, but these savings varied by type of drug therapy; and (5) the Medicaid program saved about $78,000 due to cognitive services resulting in drug therapy changes. The report concludes that the implementation of a prescription drug-related cognitive services documentation and reimbursement system is feasible and useful.

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Jay Bae

PHONE NUMBER: (410) 786-6591

PIC ID: 6528

PERFORMER ORGANIZATION: University of Washington, School of Medicine, Seattle, WA

TITLE: Assessing the Viability of All-Payer Systems for Inpatient Hospital Services

ABSTRACT: This report explores the feasibility and possible structure of alternative approaches to developing an all-payer prospective system for reimbursing hospitals for inpatient care. The study tests the practicality of adapting or extending Medicare's Prospective Payment System (PPS) to either other payers or all payers as part of the health care reform legislation proposed by the Clinton Administration in 1993. The initial tasks were the development of data files necessary to construct such an all-payer PPS. The most basic components of a PPS are payment weights, rates, and adjustors. Separate analytic steps are required in order to derive each of these components, and several different types of data are required to complete each of these analytic tasks. Admission records are needed and must be categorized according to a specified classification system. Hospital-specific cost data by payer category are required to calculate costs per case, and hospital and area characteristic data are required in order to judge the need for and level of payment adjustors. Once these data were developed, they were used in two analyses: one addressing whether diagnosis-related group (DRG)-based payment rates for Medicare and non-Medicare patients should be adjusted based on hospital characteristics, and one focusing on an alternative approach to defining the Medicare standardized prospective payment rate. The report finds that current PPS rates are based on the mean adjusted cost for a given category of hospital, and that high-cost hospitals are penalized under the assumption that their costs are reflective of presumed inefficiency. The report develops hospital-specific measures of inefficiency and then excludes the costs of inefficient hospitals from the calculation of standardized payment rates. The report concludes that a frontier cost function model approach is superior to the current system and can result in substantial program savings. See also PIC ID No. 6594.1.

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Jesse Levy

PHONE NUMBER: (410) 786-6600

PIC ID: 6594

PERFORMER ORGANIZATION: Urban Institute, Washington, DC

TITLE: Second Update of the Geographic Practice Cost Index: Final Report

ABSTRACT: The Geographic Practice Cost Index (GPCI) is an input price index used to adjust Medicare Fee Schedule (MFS) payments for area variations in physicians' cost of practice. The original GPCI was implemented with the MFS in 1992. In 1995 and 1996, the GPCI was updated with 1990 Decennial Census data. This report documents the second update of the GPCI, which is scheduled to be fully phased in on January 1, 1999. During the previous year, 1998, a transition will occur between the 1997 GPCI and the updated 1999 GPCI, with one-half the total change implemented in 1998. The comparison between the 1997 and the 1999 GPCIs shows that (1) there is no change between the physician work GPCI or the non-physician employee wage index of the practice expense GPCI except that they are reweighted with 1994 county relative value units (RVUs); (2) the office rental index of the practice expense GPCI is updated with fiscal year 1996 Fair Market Rents published by the Department of Housing and Urban Development; (3) county rental adjustment factors are eliminated for all metropolitan areas, except for the New York City Primary Metropolitan Statistical Area; (4) the malpractice GPCI is updated with 1992, 1993, and 1994 premium data, and specialty premiums are weighted by national RVU proportions rather than allowed charges; (5) there is no change in the GPCI cost shares, which measure the share of the various inputs in total practice revenues; and (6) the county RVUs used to weight averages of county input prices for each Fee Schedule Area (FSA) are updated from 1992 to 1994. The report also examines changes in the Geographic Adjustment Factor between 1999 and 1997. It determines that changes in MFS payments resulting from implementation of the 1999 GPCIs will be small (less than 2.5 percent for each FSA). See also 5764 series. (Final report: variously paginated, plus appendices.)

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Benson Dutton

PHONE NUMBER: (410) 786-6603

PIC ID: 6611

PERFORMER ORGANIZATION: Health Economics Research, Incorporated, Waltham, MA

TITLE: Validation of Nursing Home Quality Indicators Study

ABSTRACT: This study developed a system by which a series of Quality Indicators (QIs) could be used in conjunction with claims data to monitor quality of care in nursing home facilities. Fifty QIs were developed and validated using a sample of nursing home residents' medical records for a 6-month period surrounding the time of nursing facility's Federal certification survey. The QIs focus on the occurrence of three broad potential problem areas, including adverse outcomes, lack of therapy, and inappropriate pharmaceutical treatments. The QI algorithm used ICD-9 CM diagnosis codes and CPT-4 and CRVS procedure codes to generate the QIs. These codes represent either medical or surgical conditions that define the indicator events (e.g., inpatient stay or emergency room visit), the covariant diagnoses, or case selection factors. It is anticipated that these validated QIs could be used as the basis for an automated system to continuously monitor the quality of care provided in federally certified nursing homes to assist Federal and State surveyors in implementing the survey process effectively. Such a system could contribute to lowering the costs of care and improving outcomes for Medicaid nursing home residents by reducing the incidence of potentially avoidable hospitalization and over-medication. The two volume report finds that (1) the utility of basing indicators of quality of nursing home care on Medicaid and Medicare claims data is clearly demonstrated; (2) the study QIs as a whole were generally better at predicting the absence of a quality issue, suggesting that the negative finding on the quality indicator represented the lack of a quality concern in the nursing home; and (3) using covariant diagnoses to adjust for the risk of a resident receiving a QI tag is useful. The report concludes with a discussion of the issues raised during the validation of the 50 QIs examined here. (Volume I: 88 pages; Volume II: appendices variously paginated.)

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Kay Lewandowski

PHONE NUMBER: (410) 786-6657

PIC ID: 6612

PERFORMER ORGANIZATION: MEDSTAT Group, Cambridge, MA

TITLE: 1995 Influenza Immunizations Paid for by Medicare: State and County Rates

ABSTRACT: In 1993, the Health Care Financing Administration began paying providers for influenza vaccine and its administration to Medicare beneficiaries aged 65 and older. This data book provides information about the national, State, and county rates of influenza vaccination in the Medicare fee-for-service (FFS) sector. Medicare-managed care beneficiaries are not counted. The data book presents maps that report rates as integers for ease of presentation. It also presents tables that measure rates to one tenth of one percent. State rates are also presented. The data book provides three tables for each State: (1) a table providing the State's rates and numbers of influenza immunizations for 1994 and 1995 according to age, gender, and race; (2) a table showing the State's number of elderly enrolled in Medicare Part B FFS and the number of individuals who received flu shots in 1995 according to race and county; and (3) a table presenting corresponding immunization rates for 1994 and 1995 according to race and county. The data show that (1) about 41 percent of elderly Medicare beneficiaries received an influenza vaccination in 1995 that was paid for by Medicare; (2) for all sectors of this population, there was an increase over 1994 immunization rates ranging from 2.3 percent to 6.4 percent, indicating progress toward the Healthy People 2000 goal of a 60 percent influenza immunization rate for this population; (3) African-American beneficiaries have an immunization rate one-half that of Caucasian beneficiaries, although between 1994 and 1995, the increase in the rate of immunizations was slightly greater for African Americans than for Caucasians; (4) there is no substantial variation in rates by gender; and (5) immunization rates are lowest for those beneficiaries 85 years and older. (Final report: 279 pages.)

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: Marsha Davenport

PHONE NUMBER: (410) 786-6693

PIC ID: 6615

PERFORMER ORGANIZATION: Health Care Financing Administration, Baltimore, MD

TITLE: Impact of the Medicare Fee Schedule on Teaching Physicians

ABSTRACT: Prior to the implementation of the Medicare Fee Schedule (MFS) in 1992, several studies were conducted to determine the impact of this payment change on various groups of physicians. However, no studies examined the potential impact of the MFS on teaching physicians, even though anecdotal evidence from academic practices suggested that these physicians would be particularly hard hit for four reasons (1) teaching physicians perform relatively more high-technology procedures and less primary care, which could result in disproportionate net losses for many academic practices; (2) teaching hospitals and medical schools tend to be located in large urban areas, where the Geographic Practice Cost Index portion of the payment is lower; (3) customary charges for teaching physicians may be higher than the area-wide historical payments calculated for fee schedule transition; and (4) a number of academic medical practices had not signed Medicare participation agreements. This study determines the impact of the MFS on teaching physicians and examines variations in the impact across academic medical centers, other major teaching hospitals, and minor teaching hospitals. The net effects of MFS price changes and any observed quantity changes on inpatient physician revenues were examined. A total of 720 hospitals were selected for this study, including all U.S. academic medical centers. The report finds that there is no evidence that teaching physicians have been adversely impacted by the MFS. Inpatient revenues per admission fell by the same percentage amount in all types of hospitals. Revenues fell because of a reduction in prices and in the quantity of services provided per admission. Thus, while the theoretical expectation was that teaching physicians would have experienced a disproportionately large reduction in their average price per service, in practice these physicians have altered that service mix by providing less RVU-intensive visits.

AGENCY SPONSOR: Office of Strategic Planning

FEDERAL CONTACT: William Buczko

PHONE NUMBER: (410) 786-6593

PIC ID: 6666

PERFORMER ORGANIZATION: Center for Health Economics Research, Waltham, MA

TOP OF DOCUMENT

Health Resources and Services Administration

Contents

Institutional Factors Which Influence the Likelihood of Living-Related Kidney Transplant Operations-- Phase II

Exploratory Study of Reasons Kidney Transplant Waiting List Patients are Unavailable for Organ Offers

Community Health Center User and Visit Study: Survey Methodology Report

Healthy Schools, Healthy Communities National Evaluation: Final Report

Exploratory Evaluation of Rural Applications of Telemedicine

Development of a Performance Management System for Ryan White Titles I, II, and SPNS

Development of a Performance Measurement System for the Division of Organ Transplantation

Profiles of Activities to Reduce Perinatal Transmission of HIV: Assessing the Response

Grantee-Level Evaluation Activities of the HIV/AIDS Programs of the Ryan White CARE Act

Bilingual Assistance Program Evaluation, Part I: Strategies for Reducing Cultural and Linguistic Barriers to Health Care for Hispanic and Asian/Pacific Islander Populations

Examining Relationships Between HIV/AIDS and Tuberculosis Programs in Title I Eligible Metropolitan Areas

Availability of Primary Care Services Under Medicaid Managed Care: How 14 Health Plans Provide Access and the Experience of 23 Safety Net Providers and Their Communities

The Registered Nurse Population, March 1996: Findings from The National Sample Survey of Registered Nurses

Service Integration for Multi-Risk Patients: Final Report

Measuring Expenditures for Personal Health Care Services Rendered by Public Health Departments

HIV Capitation Risk Adjustment Conference Report

TITLE: Institutional Factors Which Influence the Likelihood of Living-Related Kidney Transplant Operations-Phase II

ABSTRACT: Most transplant surgeons and physicians believe that kidneys can be obtained from live donors in a safe and effective manner, and that kidneys from live donors provide superior long-term graft survival rates for transplant recipients. Between 1987 and 1993, for example, the 1- and 2-year survival rates for patients who had received kidney transplants through living-related donations (LRDs) was superior to that of patients who had received cadaveric kidneys. The rate of living-related kidney transplantation varies considerably among States, however. This project assessed (1) the willingness of the transplant community to conduct, and the frequency with which it has conducted, kidney transplants from LRDs; (2) the medical and nonmedical institutional factors that influence decision making by transplant professionals about suggesting and conducting a living-related kidney transplant; (3) the range of transplant center responses to, and procedures for, living-related transplants; and (4) the circumstances and conditions under which a transplant team that does not encourage living-related transplants would refer a patient to another transplant center for evaluation. This study is based on a mail survey of kidney transplant centers in the Fall of 1995, which yielded an 80 percent response rate. A key finding is that while attitudes toward LRDs among transplant professionals are generally positive, survey respondents identified a number of deterrents to LRDs that are not related to characteristics or practices of individual transplant centers. Some of these factors relate to the characteristics of the individual patient and his or her family. Among the recommendations is that transplant centers distribute educationally and culturally appropriate materials about policies and procedures related to LRDs. (Executive Summary: 10 pages; final report: 78 pages, plus appendices.)

AGENCY SPONSOR: Office of Special Programs

FEDERAL CONTACT: D. W. Chen

PHONE NUMBER: (301) 443-4111

PIC ID: 4917.1

PERFORMER ORGANIZATION: George Washington University, Washington, DC

TITLE: Exploratory Study of Reasons Kidney Transplant Waiting List Patients are Unavailable for Organ Offers

ABSTRACT: Studies conducted in the early 1990's to examine the efficiency of the organ allocation system identified an additional factor that might influence the equitability of allocation decisions: patient unavailability and refusal of an organ. This study (1) further examines the extent of the problem of patient unavailability for, or refusal of, an organ offer; (2) explores the reasons underlying patient unavailability and patient refusal when an organ offer is made; (3) highlights strengths and weaknesses in recordkeeping and reporting practices; and (4) makes recommendations for improving the procedures for notifying patients of an organ offer. The contractor conducted telephone interviews with three groups: transplant center staff who make organ offers, transplant waiting list patients who had refused an offer or had been unavailable when staff tried to make an offer, and recent transplant recipients. Interviews with study participants, drawn from 31 randomly selected transplant centers, were conducted between November 1995 and September 1996. The major finding was that both patient unavailability and refusal occur but are not significant issues at most centers. (Final report: 38 pages, plus appendix.)

AGENCY SPONSOR: Office of Special Programs

FEDERAL CONTACT: D. W. Chen

PHONE NUMBER: (301) 443-4111

PIC ID: 5601

PERFORMER ORGANIZATION: Battelle Human Affairs Research Centers, Seattle, WA

TITLE: Community Health Center User and Visit Study: Survey Methodology Report

ABSTRACT: This project collects data on users of Community Health Centers (CHCs) and the services they use. The sample survey has two components: (1) a personal interview survey of 1,932 randomly selected patients at a representative sample of 48 CHCs, and (2) data extraction from 2,878 medical records concerning randomly selected encounters at the same centers. Clients and visits are surveyed using modified versions of two National Center for Health Statistics (NCHS) surveys: the National Health Interview Survey and the National Hospital Ambulatory Medical Care Survey, to capture the full range of users and services provided at a particular point in time. This approach allows for comparison of data on CHC populations with national estimates derived from the NCHS surveys. The project provides demographics of users, their reasons for seeking care, diagnoses, services used, and self-reported outcomes. The User and Visit Survey is part of an overall Health Resources and Services Administration (HRSA) evaluation strategy for collecting data regarding users, services, and outcomes of primary care programs. HRSA will be developing further reports based on the survey findings. (Final report: 147 pages, plus appendices.)

AGENCY SPONSOR: Bureau of Primary Health Care

FEDERAL CONTACT: Jerrilynn Regan

PHONE NUMBER: (301) 594-4280

PIC ID: 5737

PERFORMER ORGANIZATION: Mathematica Policy Research, Incorporated, Plainsboro, NJ

TITLE: Healthy Schools, Healthy Communities National Evaluation: Final Report

ABSTRACT: This project develops and implements a design to collect uniform data on access, use, and continuity of care in the Healthy Schools, Healthy Communities Program (HSHC). The Health Resources and Services Administration's (HRSA's) HSHC program is the first Federal categorical grant program specifically targeted to school-based health centers, and is jointly funded and administered by the Bureau of Primary Health Care and the Maternal and Child Health Bureau. The study includes the 25 (of 27) school-based health centers (SBHCs) funded initial projects that were first fully operational during the 1995-1996 school year. This evaluation, implemented concurrently with the HSHC program, was intended to document change in the students' ealth status, utilization of services, and outcomes related to the SBHC interventions. There are two data sources: a student-based clinical data system and a student/parent survey. Findings are informing the activities of participating grantees and the content of policies for grantees that join the program in FY 1998 or FY 1999. In addition, HRSA has gained new insights into collecting data from children and on working with school-based records.

AGENCY SPONSOR: Bureau of Primary Health Care

FEDERAL CONTACT: Thomas Coughlin

PHONE NUMBER: (301) 594-4450

PIC ID: 5739

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Exploratory Evaluation of Rural Applications of Telemedicine

ABSTRACT: The purposes of the Health Resources and Service Administration's (HRSA's) Rural Telemedicine Grant Program (RTMG) are (1) to facilitate the development of rural health care networks through the use of telemedicine, and (2) to develop a data base for systematic evaluation of rural telemedicine systems. This baseline study concerned (1) the current status of telemedicine in rural health; (2) the effects of telemedicine on access to care, practitioner isolation, and development of health care networks; (3) the organizational factors that aid or impede successful development and implementation of telemedicine systems; and (4) development, testing, and refinement of data collection instruments for use in subsequent evaluations. The study focused on telemedicine projects that link one or more rural hospitals or ambulatory sites ("spoke sites") with a multispecialty entity ("hub site"). A hub site might be a tertiary care hospital, a multispecialty clinic, or a set of facilities that, when combined, provide the required range of specialty consultations. In addition, the study was limited to systems that employ real-time or asynchronous video imaging systems. Data sources for the evaluation include three mail surveys to identify and describe telemedicine systems, and site visits to four networks. The report presents recommendations on the content of future evaluations and on data gathering approaches. Results have influenced the FY 1997 RTMG application guidelines and the content of authorizing legislation. The report has been distributed to other Federal agencies that fund telemedicine projects and to State Offices of Rural Health. Findings are being incorporated into the design of a common telemedicine evaluation instrument for use by HRSA grantees, and by other Federal funders at their option. Finally, five articles based on the study have been published and numerous presentations have been given.

AGENCY SPONSOR: Office of Rural Health Policy

FEDERAL CONTACT: Cathy Wasem

PHONE NUMBER: (301) 443-0835

PIC ID: 5749

PERFORMER ORGANIZATION: Abt Associates, Incorporated, Cambridge, MA

TITLE: Development of a Performance Management System for Ryan White Titles I, II, and SPNS

ABSTRACT: In the public sector, efforts to monitor program effectiveness go under the rubric of "reinventing government." Several legislative and executive initiatives, particularly the Government Performance and Results Act of 1993 (GPRA), spur the movement to establish measurable performance goals that can be reported as part of the budgetary process, thus linking funding decisions to performance. GPRA requires each Federal agency to develop comprehensive strategic plans, annual performance plans with measurable goals and objectives, and annual reports on actual performance compared to goals. This report focuses on the response to GPRA undertaken by the Bureau of Health Resources Development (now called the HIV/AIDS Bureau) within the Health Resources and Services Administration (HRSA) in regard to Titles I, II, and Special Programs of National Significance (SPNS) of the Ryan White Comprehensive AIDS Resources Emergency (CARE) Act of 1990. The major results of this project were the development and articulation of an interactive process to be used in the development of a performance measurement system. This process includes (1) a clear articulation of the purposes and objectives of Ryan White Titles I, II, and SPNS; (2) linkage of these purposes and objectives to the broader functions of HRSA; (3) using program purposes and functions, and a clear understanding of accountability, responsibility, and activity throughout the programs, to enable the performance measurement system to capture the individual properties and components of each; (4) developing standard performance terminology and definitions; (5) identification of seven performance dimensions and linking them to the purposes and functions of the programs; and (6) identifying and narrowing potential measures for program performance. The report concludes with several recommendations for implementing a performance measurement system for these three programs. See also PIC ID Nos. 5954 and 5954.2.

AGENCY SPONSOR: HIV/AIDS Bureau

FEDERAL CONTACT: Katherine Marconi

PHONE NUMBER: (301) 443-6560

PIC ID: 5954.1

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Development of a Performance Measurement System for the Division of Organ Transplantation

ABSTRACT: The Government Performance and Results Act of 1993 (GPRA) links funding decisions to program performance by calling for the designation of measurable performance goals that can be reported as part of the budgetary process. GPRA requires each Federal agency to develop comprehensive strategic plans, annual performance plans with measurable goals and objectives, and annual reports on actual performance compared to goals. This report focuses on the response to GPRA undertaken by Division of Organ Transplantation (DOT) within the Health Resources and Services Administration. The report provides a conceptual framework for the organ and bone marrow transplantation programs; outlines a recommended performance management system (including an articulation of program purposes and functions, language and tools of performance measurement, criteria for selecting measures and indicators, and identification of potential measures); and recommends next steps. These next steps include (1) identification of a process for selection of measures and indicators for DOT programs, (2) selection of the most important measures from the potential group, (3) identifying supporting data and data needs and gaps, (4) preparing an annual performance plan designed to meet GPRA requirements, and (5) identifying technical assistance needs for making a performance management system operational. See also PIC ID Nos. 5954 and 5954.1. (Final report: 26 pages.)

AGENCY SPONSOR: Office of Special Programs

FEDERAL CONTACT: Katherine Marconi

PHONE NUMBER: (301) 443-6560

PIC ID: 5954.2

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Profiles of Activities to Reduce Perinatal Transmission of HIV: Assessing the Response

ABSTRACT: In August 1994, the Public Health Service (PHS) published recommendations for the use of zidovudine (ZDV) to reduce perinatal transmission of HIV in response to findings from the Pediatric AIDS Clinical Trials Group (ACTG) 076 by the National Institutes of Health (NIH). In December 1995, the Health Resources and Services Administration (HRSA) issued a Program Advisory entitled "Use of Zidovudine (ZDV) to Reduce Perinatal HIV Transmission in HRSA Funded Programs," which provides background information and specific strategies and options for implementing the PHS recommendations. This study evaluates HRSA's activities in implementing the recommendations through case studies of Florida, Mississippi, Missouri, New Jersey, Puerto Rico, Texas, and Washington. Qualitative methods were used to describe the following for each site: (1) demographic and socioeconomic factors affecting HIV/AIDS prevention and care services, (2) nature of the HIV/AIDS epidemic, (3) structure of the public health system, (4) structure of the State health department, (5) structure of HIV/AIDS services, (6) issues affecting HIV/AIDS prevention and care services, (7) relevant policies and activities prior to the release of the ACTG 076 findings, and (8) specific responses to the ACTG 076 findings and PHS recommendations. The study finds that where providers have responded to the ACTG 076 findings, the results are extremely positive. Despite the complexities of the ZDV regimen, the majority of women elect to participate in the regimen after being informed about it, believing that the advantages outweigh the disadvantages. The study has been widely disseminated to HRSA HIV/AIDS and primary care providers and has been used in assessing the incidence of perinatal HIV transmission. (Final report: 101 pages, plus appendices.)

AGENCY SPONSOR: HIV/AIDS Bureau

FEDERAL CONTACT: Moses B. Pounds

PHONE NUMBER: (301) 443-6560

PIC ID: 6202

PERFORMER ORGANIZATION: U.S. Conference of Mayors, Washington, DC

TITLE: Grantee-Level Evaluation Activities of the HIV/AIDS Programs of the Ryan White CARE Act

ABSTRACT: This study provides a systematic review and synthesis of current Health Resources and Services Administration (HRSA) HIV-related evaluation activities conducted at the grantee level to identify gaps and set national funding priorities across HRSA HIV programs. The study, requested by the HRSA AIDS Advisory Committee, reviews grantee evaluation activities over the October 1995-October 1996 period. Studies are categorized by type (formative, process, outcome/impact, or economic). The information derived from this study is being used to improve the evaluation of HRSA HIV programs. One of the project outcomes is a current study to assess client satisfaction. In addition, HRSA is developing a series of technical manuals to assist grantees with issues of methodology and capacity for evaluation. The first of these manuals to be published, "Choosing and Using an External Evaluator," has been sent to all CARE Act grantees and will be made available to others upon request. (Final report: variously paginated.)

AGENCY SPONSOR: HIV/AIDS Bureau

FEDERAL CONTACT: Moses B. Pounds

PHONE NUMBER: (301) 443-6560

PIC ID: 6256

PERFORMER ORGANIZATION: Academy for Educational Development, Washington, DC

TITLE: Bilingual Assistance Program Evaluation, Part I: Strategies for Reducing Cultural and Linguistic Barriers to Health Care for Hispanic and Asian/Pacific Islander Populations

ABSTRACT: The Disadvantaged Minority Health Improvement Act of 1990 authorized the Bilingual Assistance Program, a 3-year initiative to improve the health status of racial and ethnic minorities. Under the program, the Health Resources and Services Administration (HRSA) awarded grants to three national organizations (the Association of State and Territorial Health Officials, the National Association of County Health Officials, and the U.S. Conference of Local Health Officials) which, in turn, awarded grants in 1993 to 13 State and local health departments. The purpose of the local projects, which were selected competitively, was to reduce cultural and linguistic barriers to health care among Asian/Pacific Islander and Hispanic populations. This study examines (1) factors that allow the local projects to leverage Federal funding and promote improved interagency working relationships; (2) realistic approaches to setting performance measures; (3) technical assistance needs of grantees, including exploring the use of national minority organizations and mentors; and (4) the replicability of the products or training curricula developed by the grantees. This is the first volume of a two-part final report; the second volume is entitled "Assessment of Three Training Curricula." The study was carried out through site visits conducted during April-June 1996 and a review of background documentation. HRSA's Office of Minority Health has shared the study products with interested groups. (Final report: variously paginated.)

AGENCY SPONSOR: Office of the Administrator

FEDERAL CONTACT: Laura Diaz-Baker

PHONE NUMBER: (301) 443-9966

PIC ID: 6356

PERFORMER ORGANIZATION: MDS Associates, Incorporated, Wheaton, MD

TITLE: Examining Relationships Between HIV/AIDS and Tuberculosis Programs in Title I Eligible Metropolitan Areas

ABSTRACT: This project (1) determines the extent to which specific Ryan White Comprehensive AIDS Resource Emergency (CARE) Act Title I grantees are providing guidance to funded providers on tuberculosis (TB) screening, prophylactic, and therapeutic regimes; and (2) describes working models of collaboration between human immunodeficiency virus (HIV) and TB control programs within individual Eligible Metropolitan Areas (EMAs). In March 1995, the 34 EMAs were asked to provide a brief description of TB epidemiology in the EMA, along with specific information regarding collaboration among programs to treat people co-infected with HIV and TB; 30 responded. A literature review provides background for site visits to seven EMAs. The visits were conducted to gain more detailed information about the nature of partnerships between TB and HIV/AIDS programs that result in the most successful models of TB prevention and control within Title I-funded programs, as well as to identify programmatic configurations that result in poorly implemented TB services at these sites. The study finds that TB was prevalent in EMAs, and that services vary across EMAs and across different care settings within them. Study results are being used in developing Title I program guidance and in planning for training and technical assistance. The report has been sent to CARE Act and TB control grantees.

AGENCY SPONSOR: HIV/AIDS Bureau

FEDERAL CONTACT: Gloria Weissman

PHONE NUMBER: (301) 443-9052

PIC ID: 6359

PERFORMER ORGANIZATION: Matthew McClain, Philadelphia, PA

TITLE: Availability of Primary Care Services Under Medicaid Managed Care: How 14 Health Plans Provide Access and the Experience of 23 Safety Net Providers and Their Communities

ABSTRACT: This study examines strategies used by health maintenance organizations (HMOs) to generate access to health care services for Medicaid enrollees, who often live in inner city and rural underserved areas. The project also explores the implications of these strategies for overall access in underserved areas and for existing care providers. Third, the study considers how these strategies affect traditional Medicaid providers in the community. The study involved interviews and collection of secondary data at 14 HMOs with significant Medicaid enrollment, and at 23 traditional safety net providers, of which 13 were Community Health Centers. The study concluded that Medicaid managed care has had an overall positive effect on the availability of primary care to low-income residents in the 14 communities, and that the traditional safety net providers have not needed to reduce services. However, the future viability and range of services of public sector-supported providers might be endangered in a more competitive health system. The information derived from this study will be useful to policymakers concerned with the provision of appropriate care for vulnerable populations, providers in underserved areas, and those attempting to assure an adequate supply of health care personnel. The study has attracted attention in the professional community, as in a story in the American Medical News. In addition, an article under preparation for submission to a professional journal will reference this study, among other sources. (Appendices to the report are bound separately.)

AGENCY SPONSOR: Office of the Administrator

FEDERAL CONTACT: Jessica Townsend

PHONE NUMBER: (301) 443-0371

PIC ID: 6366

PERFORMER ORGANIZATION: Mathematica Policy Research, Incorporated, Washington, DC

TITLE: The Registered Nurse Population, March 1996: Findings from The National Sample Survey of Registered Nurses

ABSTRACT: The Sixth National Sample Survey of Registered Nurses gathered data to evaluate trends in the number, characteristics, and distribution of registered nurses in the United States. This study builds on the sample design and procedures developed in earlier surveys. Sampling procedures are devised to assure (1) the cooperation of all States and the District of Columbia, and (2) an overall response rate of 80 percent. The study resulted in a data base of 29,950 coded individual record responses, with weights for estimating characteristics of the sample universe, estimates of variances for key variables, and a technical procedures report prepared by the contractor. Health Resources and Services Administration (HRSA) staff then analyzed the data and developed the report named above. Results from this study will (1) be used in biennial reports to Congress to describe characteristics of the registered nurse population, (2) contribute to the data base underlying models used to project nurse supply and requirements; and (3) serve as input to analyses and evaluation of HRSA's nursing education programs and plans for future Federal programs to assure appropriate nursing resources. In addition, the nursing community will use the data base, the most comprehensive source of statistical information on registered nurses in the United States. (Final report: 30 pages, plus appendices.)

AGENCY SPONSOR: Bureau of Health Professions

FEDERAL CONTACT: Evelyn Moses

PHONE NUMBER: (301) 443-1451

PIC ID: 6367

PERFORMER ORGANIZATION: Research Triangle Institute, Research Triangle Park, NC

TITLE: Service Integration for Multi-Risk Patients: Final Report

ABSTRACT: This study examines how selected community-based health care organizations integrate services for people with multiple risk factors (e.g., pregnancy, substance abuse, HIV/AIDS, and homelessness) for poor health outcomes. Since these conditions often overlap, the health care organization must integrate services to meet the needs of these individuals with multiple risks. This study targets six Community Health Centers and local health departments currently receiving funding from a combination of Health Resources and Services Administration (HRSA) programs. The report uses site visits, interviews, focus groups, and discussions to explore several topics, including (1) community-based health care organizations' use of multiple funding sources to provide integrated services to patients with multiple risk factors, (2) barriers to integration of services created by separate funding streams that target specific high-risk populations, (3) successful integration strategies used by organizations, and (4) ways in which HRSA might reduce barriers created through multiple funding streams. Based on these strategies, the report identifies characteristics of organizational approaches that facilitate integration and makes recommendations for addressing barriers to service integration. The report has been distributed to all the HRSA primary care grantees. Results will also be shared through articles in peer-reviewed journals. (Final report: variously paginated.)

AGENCY SPONSOR: Bureau of Primary Health Care

FEDERAL CONTACT: Thomas Coughlin

PHONE NUMBER: (301) 594-4450

PIC ID: 6369

PERFORMER ORGANIZATION: MDS Associates, Incorporated, Wheaton, MD

TITLE: Measuring Expenditures for Personal Health Care Services Rendered by Public Health Departments

ABSTRACT: As changes to the health care delivery system continue to occur, the way in which public health departments provide services may be affected in a very significant manner, making it critical for policymakers, managers, and researchers to be able to assess the impact of these changes on the public health infrastructure. Therefore, the Health Resources and Services Administration commissioned this report, which develops a categorization schema for reporting comparable information on State and local health department personal health care services. The project initiates development and testing of a methodology to collect consistent and complete expenditure data on State and local health department services directed toward individuals. Officials from 11 State and local health departments and 1 State association of local health officials assisted in the development of a survey instrument, which was tested by State health departments in Arizona, Iowa, and Rhode Island and by local health departments in Austin/Travis County, Texas, and New York City, New York. The report finds that (1) the provision of personal health care services accounts for nearly 50 percent of the health department expenditures in Austin/Travis County and 57 percent in New York City; (2) participants considered the methodology generally clear and potentially useful, although problems were encountered in trying to fit personal health care services into some survey categories; (3) among the survey respondents, Austin/Travis County is closest to the managed care end of the service delivery continuum and Iowa is closest to the traditional approach of delivering all services directly, bundling no services, and not using capitation; and (4) Rhode Island contracts for 99 percent of its services, and New York and Austin/Travis County offer a similar mix of department-run and contracted services. (Final report: 29 pages, plus appendices.)

AGENCY SPONSOR: Office of the Administrator

FEDERAL CONTACT: Michael Millman

PHONE NUMBER: (301) 443-0368

PIC ID: 6636

PERFORMER ORGANIZATION: Public Health Foundation, Washington, DC

TITLE: HIV Capitation Risk Adjustment Conference Report

ABSTRACT: Widespread adoption of managed care by public insurance programs (such as Medicaid) has raised concerns about whether capitated reimbursement affects the quality of care provided, especially to people with chronic illnesses or other special health care needs. One way to address these concerns is to create risk-adjusted capitation rates to provide equitable reimbursement for care of patients with costly illnesses, an approach several States had proposed or implemented by 1997. In May 1997, HRSA convened an invited conference to examine issues surrounding risk adjustment for HIV/AIDS, in collaboration with ASPE, HCFA, The Henry J. Kaiser Family Foundation, and the National Academy of State Health Policy. Participants included people living with HIV/AIDS and their advocates, providers of care, managed care plans, and purchasers, including representatives of State Medicaid programs. The report presents findings and recommendations concerning various risk-adjustment methods and their evaluation, issues related to the implementation of risk adjustment, and further research needed in each of these areas. Conference findings have been used in technical assistance to States on coordinating CARE Act programs with managed care. HRSA has distributed the report to all CARE Act grantees and to State Medicaid directors; the Kaiser Family Foundation, which financed production of the report itself, has also distributed it widely.

AGENCY SPONSOR: HIV/AIDS Bureau

FEDERAL CONTACT: Richard Conviser

PHONE NUMBER: (301) 443-3075

PIC ID: 6894

PERFORMER ORGANIZATION: The Henry J. Kaiser Family Foundation, Washington, DC

TOP OF DOCUMENT

Indian Health Service

Contents

Evaluation of the Impact of the Alaska Tribal Health Compact on Programs and Services in the Anchorage Service Unit, Alaska Area

Prior Trauma Care of Intoxicated Patients as a Predictor of Subsequently Fatal Injury

TITLE: Evaluation of the Impact of the Alaska Tribal Health Compact on Programs and Services in the Anchorage Service Unit, Alaska Area

ABSTRACT: This evaluation is the first research undertaken to systematically study the perceptions of a wide range of beneficiaries, officials, and employees of the Indian Health Service (IHS) and the tribal system regarding the Alaska Tribal Health Compact, Title III Self-Governance Demonstration Project. There are almost 200 federally recognized tribes in a single Area-wide Compact, organized via 13 separate consigners-each of which has its own Annual Funding Agreement. Seven of these consigners are in the Anchorage Service Unit, which is one of nine Service Units in the Alaska Area. The overall conclusion of the evaluation is that there are more village representatives who have seen either no change or a negative change in in-village services than those who have seen a positive change. The perceptions of the officials and employees of the Alaska Native Medical Center and the Alaska Area Native Health Service have revealed problems in staff morale and shifts in workload resulting from Compact consigners transferring patients to the Alaska Native Medical Center who were previously treated outside the Anchorage Service Unit. These changes are considered as having a negative impact on the Anchorage Service Unit. The "microcosmic" view from the Ahtna Region indicates that there is a distinct difference in the perception of those who are participating in the Compact and those who are not participating. The analysis of the village respondent data, comparing the information that came from the Title III villages with the information from the Title I villages, further solidified the conclusion that individuals in villages that are not served by a Compact consigner hold a consistently more negative view than those who live in villages that are serviced by a Compact consigner.

AGENCY SPONSOR: Office of Public Health

FEDERAL CONTACT: Leo J. Nolan

PHONE NUMBER: (301) 443-4245

PIC ID: 6407

PERFORMER ORGANIZATION: Alaska Native Health Board, Anchorage, AK

TITLE: Prior Trauma Care of Intoxicated Patients as a Predictor of Subsequently Fatal Injury

ABSTRACT: This study was undertaken to identify intervention opportunities associated with nonfatal, alcohol-related injury contacts in Indian Health Service emergency departments and clinics that could, over time, decrease alcohol-related injury deaths in the Billings Area Indian Health Services. The concept was to determine the frequency of injury treatment recidivism, both alcohol- and nonalcohol-related, and further to determine if patterns can be established to suggest that injury death is predictable based on those previous nonfatal contacts. The findings indicate a high degree of recidivism and a strong correlation between non-fatal incidents and eventual injury death. This study further reveals that there is a high prevalence of alcohol in nonfatal injury-related incidents among this cohort. There appears to be a strong relationship between repeated episodes of nonfatal injury with alcohol as a contributing factor and subsequently fatal events among this cohort. These nonfatal episodes should serve as warning signals and opportunities for the application of effective alcohol intervention strategies.

AGENCY SPONSOR: Office of Public Health

FEDERAL CONTACT: Leo J. Nolan

PHONE NUMBER: (301) 443-4245

PIC ID: 6451

PERFORMER ORGANIZATION: Billings Area Indian Health Service, Billings, MT

TOP OF DOCUMENT

National Institute of Health

Contents

The Status of Biomedical Research Facilities 1996

Evaluation of the National Research Service Award Research Training Programs

National Heart, Lung, and Blood Institute Article Citation and Patent Reference Study: An Analysis of Article Citations and Patent References to Journal Articles Supported by the National Heart, Lung, and Blood Institute

Rare Diseases Clinical Research Data base

The National Survey of Laboratory Animal Use, Facilities, and Resources

Promoting Safe and Effective Genetic Testing in the United States: Final Report of the Task Force on Genetic Testing

National Cancer Institute: Characterization of User Population and User Satisfaction of the PDQ Data base-- Documentation for Tracking System

Bridges to the Future: Program Evaluation

Academic Research Instruments and Instrumentation Needs in the Biological Sciences

A Study to Determine the Changes of Physician Prescribing of Antimicrobial Agents, Omeprazole, and H2-Receptor Antagonists for Peptic Ulcer Disease Two Years Following the NIH Consensus Conference on Helicobacter pylori

Toxicology and Environmental Health Information Resources: The Role of the National Library of Medicine

Evaluation of the International Cooperative Biodiversity Groups (ICBG)

Federal Interagency Forum on Aging-Related Statistics: Data Base News In Aging

Finding the Balance: Report of the National Institute of Mental Health Intramural Research Program (IRP) Planning Committee

TITLE: The Status of Biomedical Research Facilities 1996

ABSTRACT: Since 1986, the National Science Foundation and the National Institutes of Health (NIH) have collected and reported to the Congress biennial information on the quantity and quality of biomedical research space. This report describes findings from the 1996 Survey of Scientific and Engineering Research Facilities at Colleges and Universities. The survey collected data from colleges and universities with research and development (R&D) expenditures exceeding $50,000, from Historically Black Colleges and Universities (HBCUs) with any R&D expenditures, and from a sampling of academic and nonprofit research organizations that received extramural research funding from NIH. The report found that (1) institutions performing biomedical research devoted approximately 67.4 million net assignable square feet to this work; (2) roughly half of survey respondents classified their medical science research space and/or biological science research space as inadequate; (3) expenditures to construct, renovate, and repair biomedical research space declined for the first time since NSF and NIH began collecting data; (4) the largest proportion of new construction was funded by State and local governments, while almost half of all institutions funded their own repair/renovation projects; and (5) 36 percent of all institutions with biomedical research space reported repair renovation projects that had to be deferred because funds were not available. In addition, the survey reported that HBCUs: (1) comprised 12 percent of the Nation's research-performing institutions, but contained only 1.8 percent of the national biomedical research space; (2) reported that 8 percent of their biomedical research space needed major repair/renovation to be used effectively; (3) have experienced a steady decline in new construction funding since the first survey in 1986 which reported expenditures of $42 million, to $0.2 million in 1994-5. (Final report: variously paginated.)

AGENCY SPONSOR: Office of the Director

FEDERAL CONTACT: Charles L. Coulter

PHONE NUMBER: (301) 435-0766

PIC ID: 3562.2

PERFORMER ORGANIZATION: National Science Foundation, Arlington, VA

TITLE: Evaluation of the National Research Service Award Research Training Programs

ABSTRACT: This project had several objectives. First, it conducted an evaluation design study, developing a detailed plan for a comprehensive evaluation of career outcomes of pre- and post-doctoral trainees and fellows and of the National Research Service Act (NRSA) programs in which they have participated. The evaluation developed recommendations promoting approaches that are successful in order to improve the return on Federal investment in research training. The second objective of the project was to characterize the nature and quality of the training actually experienced by the present and former trainees and fellows and to differentiate between a good training program and simply having good trainees. The project's third objective was to tap the perceptions of National Institutes of Health (NIH) staff, present and former NRSA trainees and fellows, and university employees about the nature and impact of the training program. The study developed performance indicators so that future evaluations may be based on data that are consistent across separate awards. The final report for this study will not be available until the Evaluation of NIH Post-doctoral Trainees and Fellows is completed in the summer of 1998. See PIC ID No. 6285.

AGENCY SPONSOR: Office of the Director

FEDERAL CONTACT: Walter Schaffer

PHONE NUMBER: (301) 435-2770

PIC ID: 5571

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: National Heart, Lung, and Blood Institute Article Citation and Patent Reference Study: An Analysis of Article Citations and Patent References to Journal Articles Supported by the National Heart, Lung, and Blood Institute

ABSTRACT: The primary objective of all National Institutes of Health (NIH) activities is to improve the health of the American public. However, it is generally impossible to demonstrate the impact of basic research in terms of reductions in disease mortality or morbidity. Therefore, alternative measures must be relied upon. The objectives of this project were to (1) introduce the concept of examining patents as an additional measure for assessing the impact of NIH-supported research; (2) develop a methodology to use patents in evaluation research; and (3) apply this methodology to a National Heart, Lung, and Blood Institute (NHLBI) project in order to compare the outcomes for individual research project grants (R01s) funded under a Request for Applications (RFA) with individual research project grants funded from unsolicited applications. The methods, data bases, software, and results of this project will be used by other Institutes, Centers, and Divisions to evaluate their own funding mechanisms. The project examines the impact of NHLBI grants during the period 1982 through 1994 with respect to the number of biomedical research articles associated with grant programs, the number of subsequent citations to these articles, and the number of patent references to the original articles. The report finds that (1) about 12,500 grants resulted in over 90,000 research articles cited in over 2 million later articles; (2) 3,000 of the original articles are referred to in over 1,000 patents; (3) half of all cited articles are cited no more than 12 times over the 14-year period studied, although a few were cited over 1,000 times each; (4) two-thirds of articles referenced in patents appeared only once, although 3 articles are each referenced in over 20 patents; and (5) new RFAs result in higher than normal numbers of citations, but lower than normal number of patent references. (Final report: 11 pages, plus appendices.)

AGENCY SPONSOR: National Heart, Lung, and Blood Institute

FEDERAL CONTACT: Carl A. Roth

PHONE NUMBER: (301) 496-3620

PIC ID: 5577

PERFORMER ORGANIZATION: National Heart, Lung, and Blood Institute, Bethesda, MD

TITLE: Rare Diseases Clinical Research Data base

ABSTRACT: The Rare Disease Clinical Research Data base (RDCRD) has been established by the Office of Rare Diseases (ORD), Office of the Director, National Institutes of Health (NIH). NIH made a commitment to the rare disease community and to the Congress to develop a mechanism for identifying and monitoring rare disease research supported by the NIH. The data base is similar in format to the Physician Data Query (PDQ) System of the National Cancer Institute. The International Cancer Information Center of NCI provided assistance in the development of the data base. Contact was made with 520 principal investigators and the 75 General Clinical Research Centers managers to determine those clinical trials in active states. The ORD retrieved 291 protocols from ongoing clinical trials to be included in the data base. Information from these protocols has been placed in abstracts and will be available from ORD's home page on the Internet. This development activity will continue, and the next two phases will include the development of information from clinical trials receiving support from the voluntary patient support organizations and the pharmaceuticals and biotechnology industries. ORD will continue to seek information from newly supported clinical trials receiving support from NIH. Information on specific rare diseases is difficult to locate; and, in many cases, the public, confronted with penetrating the NIH information system, reported difficulties gaining access to it. The RDCRD will eliminate this problem by providing an easy-to-use interface between the user and the data base information system. The data base contains up-to-date information that is easily accessible to a diverse user population. This information will facilitate patient and physician access to this information and lead to improved recruitment of patients with rare diseases or conditions into NIH-sponsored clinical trials. (Final report: 8 pages.)

AGENCY SPONSOR: Office of the Director

FEDERAL CONTACT: Stephen C. Groft

PHONE NUMBER: (301) 443-6156

PIC ID: 6043

PERFORMER ORGANIZATION: Oak Ridge Associated Universities, Oak Ridge, TN

TITLE: The National Survey of Laboratory Animal Use, Facilities, and Resources

ABSTRACT: This report summarizes the results of the survey conducted by the National Institutes of Health (NIH) on animal-based research for fiscal year (FY) 1993. NIH is responsible for oversight of the nationwide infrastructure required to pursue basic biomedical and behavioral research including laboratory animals, as well as the resources and facilities required to ensure their availability, appropriate use, and humane care. This study updates information gathered in the last national survey conducted in 1978. For FY 1993, the report finds that (1) of the 626 respondents, roughly half were academic institutions, and half consisted of research laboratories, independent hospitals, and government agencies; (2) there is a total census of approximately 2.5 million animals; (3) facilities reported acquiring 8.3 million animals, of which 29 percent were acquired through the facilities' own breeding; (4) respondent facilities made approximately 9 million vertebrate animals available for research, education or training, and testing, and animal species were used predominantly for research; (5) among directors of animal care and use programs, 74 percent had a doctoral-level degree; (6) among respondent facilities, 34 percent reported having space in need of renovation, 15 percent reported having space in need of replacement, and enclosed animal housing was the category most frequently cited as in need of replacement; and (7) the 559 respondents who provided cost information spent $521 million on animal care and reported that their costs had increased since FY 1991 by 25 percent. This report contributes to the understanding of the effects of Federal, State, and local laws, regulations, and policies enacted to ensure humane care and to regulate the supply and cost of laboratory animals. See also PIC ID Nos. 1506 and 2675. (Final report: 55 pages.)

AGENCY SPONSOR: National Center for Research Resources

FEDERAL CONTACT: Anne Summers

PHONE NUMBER: (301) 496-5572

PIC ID: 6048

PERFORMER ORGANIZATION: Analytical Sciences, Incorporated, Durham, NC

TITLE: Promoting Safe and Effective Genetic Testing in the United States: Final Report of the Task Force on Genetic Testing

ABSTRACT: Genetic testing is the analysis of human DNA, RNA, chromosomes, proteins, and certain metabolites in order to detect heritable diseases. The Task Force on Genetic Testing was created by the NIH-DOE Joint Working Group on Ethical, Legal, and Social Implications of Human Genome Research (ELSI Working Group) to review genetic testing in the United States and to design policies to minimize the potential negative effects of genetic testing. Their final report recommends that (1) an advisory committee be established on genetic testing in the Office of the Secretary to implement the Task Force recommendations and to determine which genetic tests require stringent scrutiny; (2) protocols for the development of genetic tests that can be used to predict future disease in healthy people receive the approval of an institutional review board (IRB) when the intention is to make the test readily available for clinical use; (3) test developers submit their validation and clinical utility data to external reviewers as well as to interested professional organizations in order to permit informed decisions about their routine use; (4) the newly created genetics subcommittee of the Clinical Laboratory Improvement Advisory Committee consider the creation of a specialty in genetics that would encompass all predictive tests that satisfy criteria for stringent scrutiny; (5) curricula on genetics be incorporated into medical school and residency training; (6) hospitals and managed care organizations require evidence of competence before permitting providers to order predictive genetic tests or counsel about them; and (7) physicians have access to accurate information to enable them to diagnose rare diseases, to know where to turn for assistance in clinical and laboratory diagnosis, and to locate laboratories that competently test for rare diseases. (Final report: 88 pages, plus appendices.)

AGENCY SPONSOR: Office of the Director

FEDERAL CONTACT: Jean McKay

PHONE NUMBER: (301) 594-2740

PIC ID: 6090

PERFORMER ORGANIZATION: Johns Hopkins Medical Institution, Baltimore, MD

TITLE: National Cancer Institute: Characterization of User Population and User Satisfaction of the PDQ Data base-Documentation for Tracking System

ABSTRACT: This study examines the user population and users' satisfaction with the Physician Data Query (PDQ) data base. PDQ was originally targeted to physicians only, but the information included in the data base was recognized to be of value to the larger community of nurses, researchers, health educators, and librarians, as well as patients, their families, and friends. PDQ is distributed to these audiences by the original on-line services, CD-ROM, facsimile, hard copy, and electronic networks, including the Internet. The users surveyed for the study include both primary users, or those who access the PDQ data base to fulfill their own research needs, and secondary users, who only conduct searches for others. The report finds that (1) the data base is widely and repeatedly used and is highly regarded; (2) physicians, other health care professionals, and librarians/information specialists predominate among the user population; (3) patients and their families and friends are also recipients of the PDQ information, even when the National Cancer Institute (NCI) toll-free information center phone service is excluded; (4) users consistently see PDQ as an authoritative and dependable source of cancer information; (5) no sections of the data base are negatively regarded or go unused; and (6) many users share the information they access through PDQ with others. Several recommendations are developed as a result of this report. The report recommends that NCI's International Cancer Information Center should (1) establish formal licensing structures to strengthen its ability to require access to users from the vendors, (2) repeat this survey within three years, (3) consider creating a CD-ROM version of the data base on its own in order to reduce costs to consumers, and (4) work with distributors and intermediaries to develop mechanisms for gathering data on user access. (Final report: variously paginated, plus appendices.)

AGENCY SPONSOR: National Cancer Institute

FEDERAL CONTACT: Bonnie J. Harding

PHONE NUMBER: (301) 496-7406

PIC ID: 6092

PERFORMER ORGANIZATION: Aspen Systems Corporation, Silver Spring, MD

TITLE: Bridges to the Future: Program Evaluation

ABSTRACT: In February 1992, the National Institute of General Medical Sciences (NIGMS) and the National Institutes of Health Office of Research on Minority Health announced the implementation of the Baccalaureate and Doctoral Bridge programs. The objective of the Baccalaureate Bridge program is to facilitate the transition of minority students at two-year colleges into colleges with baccalaureate degree programs in the sciences. The Doctoral Bridge program seeks to facilitate the transition of students into Ph.D. programs when they complete the M.S. degree. Two program outcomes that are particularly relevant to the goals of the Bridges program were examined in this study: (1) the transition of students from two-year to four-year institutions, or from M.S. to Ph.D. programs; and (2) the retention of students in the educational pathway both prior to and after making the transition from one institution to another. Although the quantity and quality of the comparison data is not high, rates of transfer among Bridges students appear to be higher than rates among other students. Success in completing the baccalaureate degree among those students who have transferred is high, and is comparable to the rate found among other minority students in an evaluation of the NIGMS Minority Access to Research Careers program. At the baccalaureate level, approximately 70 percent of the students are majoring in such fields as biology, chemistry and biochemistry, molecular biology, and microbiology. From the current data, it is difficult to develop good estimates of the eventual outcomes of participants in the Doctoral Bridges program. The early cohorts of students were small. Of the 21 students in the first two cohorts, about half have transferred to a doctoral program. While the rate of Bridges student attrition from graduate programs is below the national average, it is still too soon to predict their rate of completing their doctoral programs.

AGENCY SPONSOR: National Institute of General Medical Sciences

FEDERAL CONTACT: James Onken

PHONE NUMBER: (301) 594-2764

PIC ID: 6093

PERFORMER ORGANIZATION: Turner Consulting Group and TYC Associates, Rockville, MD

TITLE: Academic Research Instruments and Instrumentation Needs in the Biological Sciences

ABSTRACT: This report examines the fourth cycle of a national instrumentation survey concerning scientific research instruments and the academic units in which they are found for a range of scientific fields. Conducted in 1994, the survey analyzes data from two distinct populations of institutions. The first population consists of 318 institutions performing more than $3 million in research and development (R&D) in 1991. The second population consists of 44 historically black colleges and universities (HBCUs) that reported R&D expenditures in 1994. The study, in results generalized to the first group, finds that in 1993 (1) expenditures for biological research instrumentation totaled $283 million; (2) Federal funding as a percent of total funds for the purchase of biological research instrumentation declined from 48 percent in 1983 to 39 percent: NIH was the single largest Federal funding source providing $85 million; (3) 67 percent of respondents reported an increase in instrumentation needs between 1992 and 1994; (4) the price tag for the three highest priority research instruments was $363 million; (5) 20,978 research instruments in the biological sciences had a purchase price of $20,000 or more; and (6) principal investigators report that only 28 percent of the biological instruments were state-of-the-art in meeting researchers' needs. For HBCUs, the study finds that (1) the purchase price of research equipment in 1993 was $4.9 million, of which 48 percent was spent on maintenance, repair, and operation costs of the instrumentation; (2) Federal funding provided 85 percent of funding, $4.2 million, to purchase research instrumentation, and NIH was the largest source of funds; and (3) 65 percent of respondents reported inadequate research equipment. See also PIC ID Nos. 2382, 2382.1, and 4659. (Final report: 99 pages, plus appendices.)

AGENCY SPONSOR: Office of the Director

FEDERAL CONTACT: Paul Seder

PHONE NUMBER: (301) 496-4418

PIC ID: 6121

PERFORMER ORGANIZATION: National Science Foundation, Arlington, VA

TITLE: A Study to Determine the Changes of Physician Prescribing of Antimicrobial Agents, Omeprazole, and H₂-Receptor Antagonists for Peptic Ulcer Disease Two Years Following the NIH Consensus Conference on Helicobacter pylori

ABSTRACT: This study evaluates physicians' prescribing patterns for the treatment of peptic ulcer disease (PUD). Its particular focus is whether recommendations of a February 1994 National Institutes of Health (NIH) Consensus Development Conference advocating the use of antimicrobial agents for peptic ulcer patients with Helicobacter pylori infection were coincident with a change in physician prescribing patterns. The study uses computerized Pennsylvania Medicaid administrative data from January 1993 through February 1996 to evaluate prescribing patterns in the year prior and the two years following the NIH conference (12,737 patient records in all). Seven major therapeutic drug classes were analyzed: antimicrobial agents, H₂-receptor antagonists, antisecretory agents, bismuth subsalicylate, sucralfate, misoprostol, and antacids. The report finds that (1) only 10 percent of prescriptions contained an antimicrobial agent, while 62 percent were for an H₂-receptor antagonist only; (2) the two- and three-drug regimens recommended by the NIH conference were infrequently prescribed (1.1 percent and 0.9 percent, respectively); (3) younger and non-white PUD patients were more likely to receive a prescription for antimicrobial agents than older and white patients; (4) duodenal ulcer patients were more likely than non-duodenal ulcer patients to receive an antimicrobial prescriptions; and (5) the prescription of antimicrobial agents for the treatment of PUD significantly increased from 7 percent to 14.2 percent over the study period, while prescriptions for H₂-receptor antagonists significantly decreased. The report concludes that, while there were increases in the prescription of antimicrobial agents to treat PUD over the study period, these could not be directly attributed to the NIH conference. Physicians do not appear to be using national recommendations.

AGENCY SPONSOR: National Institute of Diabetes, Digestive and Kidney Diseases

FEDERAL CONTACT: Charles Sherman

PHONE NUMBER: (301) 496-5641

PIC ID: 6284

PERFORMER ORGANIZATION: Medical Technology and Practice Parameters Institute, Washington, DC

TITLE: Toxicology and Environmental Health Information Resources: The Role of the National Library of Medicine

ABSTRACT: This report examines the utility and accessibility of the National Library of Medicine's (NLM) Toxicology and Environmental Health Information Program (TEHIP). The TEHIP program encompasses 16 online data bases that contain bibliographic and factual information on environmental contaminants, including chemical properties, carcinogenicity, exposure levels, adverse health effects, emergency treatment protocols, and Federal regulations. The report notes that Federal appropriations for the TEHIP program have remained relatively constant over the past 29 years. However, the program's reimbursements from other agencies for collaborative projects have fluctuated. The report also notes that the TEHIP data base represents only a small subset of the many data bases containing information related to toxicology and environmental health. With this in mind, the report recommends that NLM consider expanding its traditional library services by organizing and cataloging the full spectrum of online toxicology and environmental health information resources. The report also recognizes that the health professional community does not have homogenous information needs and that there is wide variation in access to online data bases and other information resources. It recommends that the TEHIP program should set priorities that would allow efforts to be focused on those data bases that meet the information needs of the greatest number of health professionals and that NLM's training and outreach efforts be increased to improve awareness of the TEHIP program. The report notes that access to the data bases and difficulties in navigating the user interfaces are the two primary factors limiting use of the TEHIP data bases. The report also provides recommendations on programmatic issues and future directions for the TEHIP program. (Final report: 139 pages, plus appendices.)

AGENCY SPONSOR: National Library of Medicine

FEDERAL CONTACT: Susan Buyer

PHONE NUMBER: (301) 496-2311

PIC ID: 6534

PERFORMER ORGANIZATION: National Academy of Sciences, Institute of Medicine, Washington, DC

TITLE: Evaluation of the International Cooperative Biodiversity Groups (ICBG)

ABSTRACT: This report evaluates the International Cooperative Biodiversity Groups (ICBG) program. Funded by the National Institutes of Health and the National Science Foundation, the ICBG is a program designed to protect human health, advance the conservation of biologically diverse ecosystems, and promote sustainable economic development through the search for drugs to control diseases of concern to both developed and developing countries. Since 1993, the program has funded five individual cooperative biodiversity groups around the world. The study design attempts to (1) accommodate the varied and nonstandard implementations of the ICBG concept, (2) obtain broad coverage in terms of respondents, (3) maximize response rates and data quality, (4) minimize burden on ICBG Group Leaders and other respondents, and (5) incorporate cost and time efficiencies wherever possible. Among the benefits for source countries, the report finds (1) that scientific capacity has been developed through training and equipment, (2) development of economic activities, (3) equitable financial return and the protection of intellectual property rights of source-country people, and (4) the potential sustainability of the individual ICBG projects is one of the strengths of the overall program. The report recommends that the ICBG program (1) develop systems of technical assistance based on a combination of outside expertise and lessons learned across programs; (2) improve communications among the ICBGs; (3) consider developing a cross-ICBG data base that includes work being done in collection, extractions, and drug development techniques; and (4) bring in the expertise of social scientists and source-country legal experts to the ICGB to ensure that the consensus-building, benefit sharing, and economic goals of this project are achieved as well. (Final report: 177 pages, appendices bound separately.)

AGENCY SPONSOR: Fogerty International Center

FEDERAL CONTACT: J. Kirby Weldon

PHONE NUMBER: (301) 496-2571

PIC ID: 6641

PERFORMER ORGANIZATION: Battelle Corporation, Arlington, VA

TITLE: Federal Interagency Forum on Aging-Related Statistics: Data Base News In Aging

ABSTRACT: This annual report highlights the aging-related surveys and products currently available from member agencies of the Federal Interagency Forum on Aging-Related Statistics. All Federal agencies are invited to contribute to the report and to participate in the Forum. Contributors to this report are (1) Department of Agriculture: Economic Research Service; (2) Department of Commerce: Bureau of the Census; (3) Department of Health and Human Services: Administration on Aging, Centers for Disease Control and Prevention, National Institutes of Health (NIH), Health Care Financing Administration, Agency for Health Care Policy and Research, and Office of the Assistant Secretary for Planning and Evaluation; (4) Department of Housing and Urban Development: Office of Policy Development and Research; (5) Department of Labor: Bureau of Labor Statistics; (6) Department of Veterans Affairs; (7) Federal Reserve System, Board of Governors; (8) Social Security Administration; and (9) the United Nations. Each survey or data base is listed with its sponsoring organization, purpose, description, current activity, plans for future activity, public use data files, and associated reports, if any. The report ends with a tabular listing of the data bases and surveys described in the report, which gives the latest year data were collected or whether data collection is ongoing, the year for which the most recent data were released, whether public use tapes or printed reports are planned or available, and information on the person to contact about the data base or survey. (Final report: 117 pages.)

AGENCY SPONSOR: National Institute on Aging

FEDERAL CONTACT: Gail Jacoby

PHONE NUMBER: (301) 496-3121

PIC ID: 6652

PERFORMER ORGANIZATION: Bureau of the Census, Washington, DC

TITLE: Finding the Balance: Report of the National Institute of Mental Health Intramural Research Program (IRP) Planning Committee

ABSTRACT: The Intramural Research Program (IRP) of the National Institute of Mental Health (NIMH) must be positioned to make scientific advances crucial to easing the burden of mental disorders. This report represents an active effort by the NIMH IRP Planning Committee to help the program make these advances. The Committee evaluated the rationale for the program's continuance, as well as its strengths and weaknesses. It solicited confidential letters from almost 1,000 IRP scientists and staff and met with National Institutes of Health (NIH) leadership, the IRP's past and present leadership, and advocacy organizations. It found strong justification for the continued existence of the IRP, as long as the research it produces is of the highest quality. The Committee also offered recommendations to revitalize the IRP. The report finds that (1) stable scientific leadership is crucial to revitalizing IRP science, thus the program must recruit a Scientific Director for the program; (2) the overall quality of IRP science has declined from its previous place at the top of the scientific endeavor-while there are some areas of research brilliance, some research is of lesser quality, according to members of the Board of Scientific Counselors and intramural and extramural scientists; (3) recruitment and retention of talented scientists is essential, but over the past five years, the program has recruited from outside of NIH only one of its four newly-tenured scientists; (4) the quality of the program's intramural training efforts is uneven; (5) the program should address the fact that post-doctoral fellows have difficulty competing for positions when they leave NIH due to a lack of sufficient grantsmanship and teaching experience; and (6) NIMH's IRP's clinical research program needs to be sustained and revitalized.

AGENCY SPONSOR: National Institute of Mental Health

FEDERAL CONTACT: Emily Gause

PHONE NUMBER: (301) 443-4335

PIC ID: 6697

PERFORMER ORGANIZATION: National Institute of Mental Health, Rockville, MD

TOP OF DOCUMENT

Office of the Assistant Secretary for Planning and Evaluation

Contents

State Assisted Living Policy: 1996

Evaluation of the Minority Male Consortium for Violence Prevention

Determinants of AFDC Caseload Growth: Final Report

Informal and Formal Kinship Care

Approaches to Evaluating Welfare Reform: Lessons from Five State Demonstrations

Efforts by Child Welfare Agencies to Address Domestic Violence: The Experiences of Five Communities

Evaluability Assessment of Responsible Fatherhood Programs

Trends in the Well-Being of America's Children and Youth: '97

Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health

Registry of State-Level Efforts to Integrate Health Information

Implementing Welfare Reform Requirements for Teenage Parents: Lessons from Experience in Four States

Deriving State-Level Estimates from National Surveys: A Statistical Assessment and State Tabulations

Information Needs Associated with the Changing Organization and Delivery of Health Care: Summary of Perceptions, Activities, Key Gaps, and Priorities

Chapin Hall Center for Children at the University of Chicago: Administrative Data Guide

Iowa's Limited Benefit Plan

Role of Home- and Community-Based Services in Meeting the Health Care Needs of People with HIV/AIDS: Literature Review

Exploring the Feasibility of Linking Health Expenditure and Utilization Data

Public Health Laboratories and Health System Change

Framework for Assessing Insurer Responses to Health Care Market Changes

Establishing an Analytical Framework for Measuring the Role of Reinsurance in the Health Insurance Market

National Evaluation of Welfare-to-Work Strategies: Evaluating Two Welfare-to-Work Program Approaches: Two Year Findings on the Labor Force Attachment and Human Capital Development Programs in Three Sites

Improving Health in the Community: A Role for Performance Monitoring

Privacy and Health Research: A Report to the U.S. Secretary of Health and Human Services

State Regulatory Experience with Provider-Sponsored Organizations: Final Report

Indicators of Welfare Dependence and Well-Being: Interim Report to Congress

Setting the Baseline: A Report on State Welfare Waivers

Market Barriers to the Development of Pharmacotherapies for the Treatment of Cocaine Abuse and Addiction: Final Report

Consumer-Directed Personal Assistance Services: Key Operational Issues for State CD-PAS

Programs Using Intermediary Service Organizations

TITLE: State Assisted Living Policy: 1996

ABSTRACT: Assisted living facilities are an expanding source of supportive housing and services, and represent a new model of long-term care. Consumer choice and control are central to assisted living models, which seek to allow consumers to control key features of their living environments and to direct their own receipt of services. This study reviews the assisted living and board and care policies in each of the fifty States. Fifteen States have existing licensure regulations for assisted living facilities and an additional nine States are developing regulations. Twenty-two States reimburse or plan to reimburse assisted living as a Medicaid service and six States provide Medicaid payments for services in board and care settings. Thirteen states have created a task force or a process within a State agency to make recommendations for the development of assisted living rules. One of the major difficulties associated with assisted living is the lack of a common or standard definition of this kind of care. However, State approaches share common components, including that (1) assisted living is characterized as residential, rather than institutional; (2) health or medical services are provided, either by facility staff or through contracts with community agencies; (3) assisted living is a model emphasizing consumer or resident independence, autonomy, dignity, privacy, and decisionmaking; and (4) many States attempt to combine minimum standards of safety and quality of care while also allowing market forces to shape quality. State policies generally address three major issues: (1) the requirement for a living unit, (2) admission and retention criteria, and (3) the level of services to be provided. The report provides specific information about policies in each State. See also PIC ID Nos. 4719-4719.2 and 4719.4-4719.5. (Final report: 225 pages.)

AGENCY SPONSOR: Office of Disability, Aging, and Long-Term Care Policy

FEDERAL CONTACT: Robert Clark

PHONE NUMBER: (202) 690-6443

PIC ID: 4719.3

PERFORMER ORGANIZATION: Research Triangle Institute, Research Triangle Park, NC

TITLE: Evaluation of the Minority Male Consortium for Violence Prevention

ABSTRACT: This project evaluated the Office of Minority Health Minority Male Consortium for Family and Community Violence Prevention Programs. At the time of this study, 19 historically black colleges and universities (HBCUs) were participating in this program. Features of violence prevention program models at 13 historically black colleges and universities/family life centers (HBCUs/FLCs) were synthesized to determine approaches that could prevent or minimize violence in communities committed on or by minority males. The contractor (1) performed an extensive literature review; (2) collected and analyzed relevant program documentation and materials; (3) conducted focus groups; (4) examined target populations; (5) developed a site visit discussion guide; (6) selected 10 HBCUs for site visits; (6) developed and executed an analysis plan; (7) interpreted analysis findings, including defining program models and analyzing HBCUs' organizational capacity; (8) held briefings with Consortium leaders, members, and government personnel; (9) reviewed the overall organization, design, and implementation of the programs; and (10) evaluated prevention strategies adopted by the HBCU/FLCs. Violence prevention programs suffer from an inability to produce valid performance data demonstrating that prevention investments pay off in the long run. The report recommends that (1) without completely centralizing the design process, the Federal government and the Consortium should move toward a consensus-seeking information model in which individual schools are no longer free to pursue independent designs; and (2) constraints should be introduced on which program models are implemented by which schools. See also PIC ID Nos. 5789 and 5789.1. (Final report: 68 pages, plus appendices.)

AGENCY SPONSOR: Office of Program Systems

FEDERAL CONTACT: Patricia S. Hazard

PHONE NUMBER: (202) 690-8291

PIC ID: 5789.2

PERFORMER ORGANIZATION: Macro International, Incorporated, Calverton, MD

TITLE: Determinants of AFDC Caseload Growth: Final Report

ABSTRACT: This project examined the dynamics of Aid to Families with Dependent Children (AFDC) caseload growth and analyzed the effects of economic, demographic, and programmatic changes at the State level on States' AFDC caseload from the early 1980's to the present. Analysis of economic factors included measures of both wages and unemployment; analysis of demographic factors includes an emphasis on nonmarital childbearing. Analysis of program effects included changes arising from national welfare legislation, changes in State welfare parameters, State waivers, and changes in other programs such as Food Stamps and Medicaid. The project addressed the interaction of these factors by analyzing the economic, demographic, and programmatic factors simultaneously. Compared to earlier studies of AFDC caseload determinants, many of which are outdated, this study focused much more on State-level indicators, and included a more comprehensive set of economic, demographic, and programmatic factors. Recently, many State policymakers have expressed concern over the effect of future recessions or nonmarital birth rates on their caseloads, and this study is intended to be a valuable tool to State policymakers as they continue to assume greater responsibility for their AFDC programs and consider new innovations.

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Kelleen Kaye

PHONE NUMBER: (202) 401-6634

PIC ID: 5953

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Informal and Formal Kinship Care

ABSTRACT: A growing number of States prefer to place children who have been removed from their homes of origin in the home of a relative. This practice is known as kinship foster care, and is a growing component of family foster care. It comprises between one-third and one-half of all foster care days provided in the States with the largest foster care populations. Many children, however, live with relatives who are not their parents in an informal relationship. In all, about 2.15 million children lived in the care of relatives without a parent present in 1994. This study uses several data sources to examine the characteristics of informal kinship care arrangements and how they compare with those initiated through or subsidized by the State child welfare system. It considers national patterns, examines living arrangement patterns by State for FY 1990, compares formal and informal kinship care in four States, and uses Illinois data to provide a more focused picture of kinship care. The two-volume report finds that (1) kinship care is more prevalent in the South and in non-metropolitan areas, and for older children; (2) about two-thirds of kinship caregivers are the child's grandparents-about half are currently married, and over 85 percent of single kinship caregivers are women; (3) kinship caregivers are more likely to be currently unmarried, to be less educated, to be poor, and to receive social welfare benefits than are live-in parents; (4) nationally, just over 2 percent of children lived with relatives not their parents (this percentage ranges by State from under 1 percent to well over 3 percent) (5) higher levels of kinship care (as well as single mother households) appear to be direct products of higher levels of social disruption and family disorganization; and (6) while informal kinship care patterns were roughly similar in the four States examined, the use of formal kinship care varies widely. (Vol. I: 80 pages; Vol. II-tables and figures-58 pages plus appendices.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Laura Feig

PHONE NUMBER: (202) 690-5938

PIC ID: 6016

PERFORMER ORGANIZATION: University of Chicago, Chicago, IL; Urban Institute, Washington, DC

TITLE: Approaches to Evaluating Welfare Reform: Lessons from Five State Demonstrations

ABSTRACT: Most States have obtained waivers from Aid to Families with Dependent Children (AFDC) and Food Stamp Program rules in order to implement welfare reform. Between 1993 and 1996, the Clinton Administration approved waivers for 43 States. This project examined waiver demonstration projects in five States (California, Colorado, Michigan, Minnesota, and Wisconsin). Four of the five States used an experimental evaluation design, while the fifth used a quasi-experimental evaluation design. This two-volume report addresses issues in five areas: (1) the choice between an experimental and quasi-experimental evaluation design, (2) sample design, (3) implementation of experimental evaluations, (4) data collection, and (5) analysis methods. The report finds that: (1) most evaluations of State welfare reform demonstrations used an experimental design, which involved random assignment of cases to an experimental group subject to welfare reform or to a control group subject to pre-reform policies; (2) only one waiver (Wisconsin's) was approved with a quasi-experimental design, but this may become more common in a block grant environment; and (3) when designing a study sample, evaluation planners should ensure that the sample size is adequate and should design the sample so that applicant and recipient subgroups can support separate impact estimates (the sample should also be representative of the State as a whole). The report also finds that four aspects of the implementation of an experimental evaluation require special care: (1) the timing of the random assignment, (2) the method of the random assignment, (3) ensuring that control group policies remain unchanged, and (4) preventing experimental and control group cases from changing status. The report also discusses data collection and analysis. (Final report: Volume I: 128 pages, plus appendices; Volume II: 8 pages.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Audrey Mirsky-Ashby

PHONE NUMBER: (202) 401-6640

PIC ID: 6050

PERFORMER ORGANIZATION: Urban Institute, Washington, DC

TITLE: Efforts by Child Welfare Agencies to Address Domestic Violence: The Experiences of Five Communities

ABSTRACT: Child protective services (CPS) and domestic violence services historically have not been closely linked. However, child welfare agencies across the country are beginning to consider how families in their child protection caseload are affected by domestic violence and are reexamining their policies accordingly. This report documents how child welfare agencies in five communities (Massachusetts; Michigan; San Diego County, California; Hilo, Hawaii; and Oregon) are attempting to integrate domestic violence concerns into their services. The communities were chosen for study because they are extending their services well beyond educating staff in domestic violence concerns. The report finds that (1) child welfare agencies have initiated changes from different organizational points within their agencies and have taken different approaches to changing case practice; (2) these agencies cannot make appropriate changes without major and continuing collaboration with community stakeholders who work with domestic violence victims and perpetrators; (3) chief among the concerns raised is the need to refrain from actions that further endanger mothers and children; (4) changes will also benefit from close collaboration with police, civil and criminal courts, corrections, the schools, and local clinics and hospitals; and (5) ongoing technical assistance to CPS staff after initial training in domestic violence services appears to make a more lasting difference on workers' ability to put the content of training into practice on a daily basis. The report closes with a general discussion of (1) approaches to addressing domestic violence within child welfare agencies; (2) the role of the larger community; (3) more complex policy questions, such as whether a child's witnessing of domestic violence is itself abusive; and (4) evaluating the effectiveness of these efforts. (Final report: 148 pages.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Jerry Silverman

PHONE NUMBER: (202) 690-5654

PIC ID: 6155

PERFORMER ORGANIZATION: Urban Institute, Washington, DC

TITLE: Evaluability Assessment of Responsible Fatherhood Programs

ABSTRACT: Responsible fatherhood programs provide fathers, particularly those who are young, unmarried, and unemployed, with the support and skills to become active participants in their children's lives. An increased interest in fatherhood programs and a lack of information on their accessibility and effectiveness led to this evaluability assessment. This study visited five fatherhood program sites, selected as the most developed programs by HHS, in order to (1) assess the readiness of fatherhood programs for formal evaluation and identify obstacles to their evaluation in the future, and (2) illustrate the evaluation process and mechanisms that must be in place before a formal impact evaluation is undertaken. The project goal was to develop an evaluation design that could be used by HHS and other policymakers in the future to evaluate such programs. The report concludes that at this time formal program evaluation remains premature because the programs (1) are new and still at the stage of refining recruiting methods and program services, (2) lack automated systems for tracking and reporting on clients, and (3) serve a very small number of clients. In order to conduct a rigorous evaluation of fatherhood programs, the report suggests the development of (1) measurable outcomes; (2) defined service components and their hypothesized relationship to outcomes; (3) an established recruiting, enrollment, and participation process; (4) an understanding of the characteristics of the target population, program participants, and program environment; and (5) an ability to collect and maintain an adequate program size. The report concludes that to increase viability, fatherhood programs must (1) develop a core definition of what constitutes a responsible fatherhood program, (2) conduct process evaluations, (3) build basic management information system capacity, and (4) stabilize and enhance funding. (Final report: 114 pages, plus appendices.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Linda Mellgren

PHONE NUMBER: (202) 690-6806

PIC ID: 6159

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Trends in the Well-Being of America's Children and Youth: '97

ABSTRACT: This is the second edition of an annual report from HHS on the well-being of children and youth in the United States. The report's purpose is to provide the policy community, the media, and all interested citizens with a statistical overview describing the condition of children. The report focuses on trends in the well-being of children and youth in five key areas: population, family, and neighborhood; economic security; health conditions and health care; social development, behavioral health, and teen fertility; and education and achievement. The report, as it pertains to teenagers, finds (1) a decrease in the pregnancy rate among young women aged 15-19 since 1991; (2) an increase in alcohol, cigarette, marijuana, and cocaine use in the 1990's; (3) a modest gain in math and science proficiency by 17-year-old students since the early 1980's; (4) a sustained increase in mortality for black youth aged 15-19 since 1991, while there was a decrease in mortality rates for white youth; and (5) a steady increase in receipt of early prenatal care by teen mothers. Federal statistical data lack (1) standardized measures of social development and health-related behaviors for very young and pre-teenaged children; (2) indicators that reflect important social processes affecting child well-being that go on inside the family and within the neighborhood; and (3) sufficient information on child abuse and neglect, youth violent crime, day care quality, learning disabilities, and measures of children in institutionalized care. The Federal Interagency Forum on Child and Family Statistics has adopted a mandate to improve the statistical system. As data become available they will be incorporated into new editions of this report. (Final report: variously paginated.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Matthew Stagner

PHONE NUMBER: (202) 690-5653

PIC ID: 6170.1

PERFORMER ORGANIZATION: Urban Institute and Child Trends, Incorporated, Washington, DC

TITLE: Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health

ABSTRACT: The Panel on Performance Measures and Data for Public Health Performance Partnership Grants (PPGs) was established in order to examine the state-of-the-art in performance measurement for public health and to recommend measures that could be used to monitor the PPG agreements to be negotiated between each State and the Federal government. The panel considered performance measures in ten public health areas: chronic diseases, sexually transmitted diseases, human immunodeficiency virus infection and tuberculosis, mental health, immunization, substance abuse, sexual assault, disabilities, and emergency medical services. More than 3,200 measures were proposed to the panel through various outreach efforts. The panel used four guidelines for assessing the performance measures: (1) the measure should be specific and results oriented; (2) the measure should be meaningful and understandable; (3) data should be adequate to support the measure; and (4) the measure should be valid, reliable, and responsive. The measures that scored the highest are recommended for use in performance monitoring. They cover health status, social functioning, consumer satisfaction, and risk status. In assessing the adequacy of data for specific performance measures, the panel concluded that there are few available data sources that are ideal for performance monitoring. Many Federal efforts to collect health-related data provide national rates, but do not collect data that provide State-level rates, and much State-level data may not be comparable. The report also finds that many of the performance measures presented can and should be subdivided to focus on specific high-risk populations in a State. See also PIC ID No. 6200. (Final report: 54 pages, plus appendices.)

AGENCY SPONSOR: Office of Program Systems

FEDERAL CONTACT: Jeffery J. Koshel

PHONE NUMBER: (202) 401-8238

PIC ID: 6177

PERFORMER ORGANIZATION: National Academy of Sciences, Institute of Medicine, Washington, DC

TITLE: Registry of State-Level Efforts to Integrate Health Information

ABSTRACT: In 1996, HHS contracted The Lewin Group to develop a registry of projects that integrates health information systems at the State level. The registry is located on the World Wide Web at

http://aspe.hhs.gov/statereg/index.htm. The home page compiles, abstracts, and indexes summary information on data integration projects for all 50 States and the District of Columbia. The purpose of the registry is twofold. First, the registry gives Federal policymakers a central source for researching the types of integration activities being pursued by individual States. Second, the registry facilitates communication among States regarding data integration activities and related data policy activities. The project (1) provides a resource for State and local policymakers as they seek to identify others who have pursued information integration activities; (2) facilitates information sharing and collaboration across States; and (3) assists Federal policymakers in monitoring, assessing, and supporting State-level integration activities. Relevant State integration projects were identified through: (1) contacts with State officials responsible for hospital discharge data bases, vital statistics, Medicaid data, and any other major information system or data-related offices; (2) conferences on State data integration activities; (3) grantee information related to specific funds which target information activity; (4) State health department websites; and (5) recommendations from people interviewed. A contract with The Lewin Group has been issued to organize and edit the materials that are submitted and encourage biannual updates of the site. The data base is administered by the Office of the Assistant Secretary for Planning and Evaluation. Final report is a web site: http://aspe.hhs.gov/statereg/index.htm.

AGENCY SPONSOR: Office of Program Systems

FEDERAL CONTACT: Dale Hitchcock

PHONE NUMBER: (202) 690-5882

PIC ID: 6178

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Implementing Welfare Reform Requirements for Teenage Parents: Lessons from Experience in Four States

ABSTRACT: Federal welfare policy requires minor custodial parents receiving cash assistance to attend school and to live with their parents or in an adult-supervised setting; these requirements were established as part of the Personal Responsibility and Work Opportunity Reconciliation Act, which created the program for Temporary Assistance for Needy Families and abolished the Aid to Families with Dependent Children (AFDC) program. This two-volume report summarizes lessons learned from the operational experiences in four States (Arizona, California, Massachusetts, and Virginia) that implemented waivers requiring school attendance and certain living arrangements under the AFDC program. The report finds that (1) identifying teenage parents, particularly those who do not head their own cash assistance cases, can be a major challenge; (2) persistent staff training to correct errors, the use of alternative information sources to identify teenage parents, and establishing positive incentives for identifying and referring all teenage parents can help to address these challenges; (3) a range of education options, including GED programs, should be available; (4) programs must determine the focus and scope of case management in light of goals and costs; (5) monitoring school attendance increases welfare agency workload-sometimes substantially-and can complicate the relationship between welfare agencies and schools; (6) a very restrictive living arrangement requirement may create implementation difficulties; and (7) funding group homes may enable States to have fewer exceptions to the requirement, but despite their advantage for teen parents, many may not choose to reside in them. (Final report: Volume I, 50 pages; Volume II (case studies), 103 pages.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Kelleen Kaye

PHONE NUMBER: (202) 401-6634

PIC ID: 6228

PERFORMER ORGANIZATION: Mathematica Policy Research, Incorporated, Plainsboro, NJ

TITLE: Deriving State-Level Estimates from National Surveys: A Statistical Assessment and State Tabulations

ABSTRACT: The Department of Health and Human Services (HHS) is changing its focus from the national to the State level. Landmark legislation eliminating entitlement to Aid to Families with Dependent Children benefits and block granting it to the States was passed at the time of this report. This project is one component of a larger HHS strategy to generate valid and reliable State-level estimates, which can be used to measure the impact of social policy interventions. This project (1) appreciates the statistical issues involved in addressing policy and research questions at the State level with national survey data; (2) assesses the current capabilities of these major surveys to provide answers to these questions at the State level; and (3) presents options for obtaining valid, reliable State-level data from relatively minimal enhancements and changes to the design of the surveys. The study focused on three major Federal surveys: the Survey of Income and Program Participation (SIPP); the Current Population Survey (CPS); and the National Health Interview Survey (NHIS). These surveys were selected for their widespread applicability, use, and potential for analyzing policy issues. The report evaluates statistical concerns to determine the suitability and potential of using the selected surveys for State-level estimates and to identify differences between States. These technical concerns included variance estimation, standard errors, the construction of State-level weights, minimum sample sizes needed, design effects, pooling several years of data, and longitudinal issues (e.g., response rates and weighting). The report recommends periodic redesign efforts and other survey enhancements.

AGENCY SPONSOR: Office of Health Policy

FEDERAL CONTACT: George Greenberg

PHONE NUMBER: (202) 690-7794

PIC ID: 6380

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Information Needs Associated with the Changing Organization and Delivery of Health Care: Summary of Perceptions, Activities, Key Gaps, and Priorities

ABSTRACT: This report assesses unmet needs for supply-side information (information about health care providers and insurers) on the health system, focusing on helping the Department of Health and Human Services identify how information needs associated with a changing health care system are perceived by a wide variety of user groups and constituencies. Based on information gathered from over 50 individuals and a review of relevant literature and Web sites, the report examines perceived gaps, their causes, and the activities generated in response to those perceptions. It also considers, in a more comprehensive manner, information on a diverse group of 11 entities. The report finds that (1) the serious gaps in "supply-side" information on the health system have grown worse as the system has evolved, presenting major obstacles to stakeholders' ability to serve their constituency, address operational needs, and participate in policy decisions; (2) the growth of managed care and consolidation has contributed to the need for supply-side information, as has the growing interest in accountability, competition, and cost-containment; (3) stakeholders say they need better information on health insurers and plans and the ways in which providers are linked to plans and integrated systems; and (4) better transaction-level data are needed, as are data from the State and local levels. The report identifies 23 ongoing efforts within the private sector, States, and foundations to address these gaps, and studies 11 of these more thoroughly. The report concludes that private sector stakeholders and States alone cannot sufficiently address supply-side information needs, giving importance to Federal leadership and convening functions. (Final report: 61 pages, plus appendices.)

AGENCY SPONSOR: Office of Health Policy

FEDERAL CONTACT: Linda Sanches

PHONE NUMBER: (202) 690-7233

PIC ID: 6388

PERFORMER ORGANIZATION: Mathematica Policy Research, Incorporated, Plainsboro, NJ

TITLE: Chapin Hall Center for Children at the University of Chicago: Administrative Data Guide

ABSTRACT: This contract produced a guide to the administrative data resources regarding health, social services, and related programs available through the Chapin Hall Center for Children at the University of Chicago. Each data section in the guide provides an overview of a particular dataset. The sections offer enough information for the researcher to gain a general understanding of the available variables and the types of analysis the data will support. Data Description and Limitation entries document the larger issues involved with analyses of the data and should not be considered exhaustive. The guide also includes information to assist the researcher in interpreting the data sections. The guide was designed for use by an audience of informed policy analysts in order to consider what data are available to answer particular policy questions. It is not intended that the guide be a detailed programmer's reference book to every code used in each data field, but rather that it describe the data sets available, the basic structure and topics covered by each, and the ability to match clients between the different programs' data. The goal of this guide is to enable the Office of the Assistant Secretary for Planning and Evaluation's policy and research analysts to better design relatively short-turnaround analyses on a variety of topics of current policy interest. The guide may also be useful to other researchers. (Final report: 110 pages, plus appendices.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Laura Feig

PHONE NUMBER: (202) 690-5938

PIC ID: 6389

PERFORMER ORGANIZATION: Chapin Hall Center for Children, University of Chicago, Chicago, IL

TITLE: Iowa's Limited Benefit Plan

ABSTRACT: Iowa's Family Investment Program (FIP) replaced Aid to Families with Dependent Children in 1993. FIP participants who are able-bodied and not caring for children, yet who do not develop and carry out a Family Investment Agreement (FIA), are sanctioned with assignment to the Limited Benefit Plan (LBP); some clients may also choose LBP over FIP. LBP originally provided 3 months of benefits at FIP levels, followed by 3 months of reduced levels and 6 months of no cash benefits; the program was later changed to eliminate the original 3-month period. At the end of the 6-month period, the client can reapply for FIP, but must meet its requirements. This report present finding from a study of the original LBP, showing how some families enter the LBP and how their well-being changes after benefits are cut. It examines records for over 4,200 cases assigned to LBP records during 6 months in 1994 and 1995, a survey of 137 cases whose cash benefits were terminated, and case studies of 12 LBP families. The report finds that (1) most clients are assigned to LBP for failing to develop and sign an FIA, rather than for failing to carry it out; (2) slightly more than half of LBP clients subsequently meet FIP requirements and are reassigned to the program; (3) about half of LBP clients whose cash benefits have been terminated are employed immediately after cash benefits end, but about half are not; (4) 40 percent of these clients increase their monthly incomes by an average of $496 when their benefits end, but 47 percent see a decrease in their monthly incomes averaging $384; and (5) other government programs remain important sources of support for LBP families whose cash benefits have been terminated. The report notes that there are several features of the LBP program that mitigate against extreme deprivation when cash assistance has been terminated, and notes that the absence of these safeguards in a national welfare reform program could result in extreme hardship for many families. (Final report: 132 pages plus appendix.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Kelleen Kaye

PHONE NUMBER: (202) 401-6634

PIC ID: 6390

PERFORMER ORGANIZATION: Mathematica Policy Research, Incorporated, Washington, DC

TITLE: Role of Home- and Community-Based Services in Meeting the Health Care Needs of People with HIV/AIDS: Literature Review

ABSTRACT: This report provides an overview of the literature describing the delivery and financing of home- and community-based services for people with AIDS, focusing on the literature published since 1991. The literature review finds that (1) since the start of the epidemic, the AIDS population has been decreasingly made up of white male homosexuals and increasingly made up of people of color with low incomes and low educational levels; (2) new AIDS treatments combining protease inhibitors and other antiretroviral medications may increase life expectancy among some people with AIDS but annual costs range from $9,000 to $18,000 per person; (3) the lifetime cost of AIDS treatment has been rising, since drugs have been increasing life expectancy even before the introduction of protease inhibitors; (4) AIDS patients who experience periods of disability require nursing and support services to complement medical care, including assistance with daily living activities, child care, transportation, help applying for public assistance, and the like; (5) the increasing number of low-income people with AIDS places a heavier burden on public programs such as Medicaid and Ryan White CARE Act funding; (6) the Medicare program has borne a relatively small proportion of the burden for AIDS care, but this may increase as life expectancy increases; (7) rigorous evaluation of programs that provide home- and community-based services to people with AIDS appears to be lacking; (8) the literature provided little data describing current levels of use of and cost for home- and community-based services by people with AIDS; and (9) there is little information about how effectively managed care is meeting the need for these services. The review examines literature on the provision of these services to these elderly to draw inferences for their application to the AIDS population. See also PIC ID No. 6418. (Final report: 92 pages.)

AGENCY SPONSOR: Office of Disability, Aging, and Long-Term Care Policy

FEDERAL CONTACT: Gavin Kennedy

PHONE NUMBER: (202) 690-6443

PIC ID: 6418.1

PERFORMER ORGANIZATION: Mathematica Policy Research, Incorporated, Plainsboro, NJ

TITLE: Exploring the Feasibility of Linking Health Expenditure and Utilization Data

ABSTRACT: The Health Resources and Services Administration (HRSA) has the responsibility for assessing the adequacy of the Nation's supply of various health professions. HRSA commissioned a "General Services Demand Model" (GSDM) in 1994 to permit State-specific estimates of health utilization, which it then inputs into State-specific provider requirements forecasts. HRSA's GSDM predicts, for each State and the District of Columbia: bed days in short-term general hospitals for the under-65 and over-65 populations, outpatient visits, and emergency visits. HRSA then takes these predictions and estimates requirements for various health professions for each State. The estimates are typically based on assumptions about appropriate provider staffing/utilization ratios; for example, each primary care physician can perform 3,000 office visits in a year. From the data and assumptions, a set of State-specific provider requirement forecasts can be generated. In general, these forecast supply and demand for physicians by speciality type, as well as for registered nurses. The Health Care Financing Administration compiles national health account expenditures for hospital services, physician services, prescription drugs, and other areas. The goal of these projects is to link these two data bases over a 2 to 3-year period and then use the model to simulate the utilization and cost savings potential of reorganizing the delivery of care (for example, using HMO specialist/generalist staffing ratios versus the current fee-for-service norms). The ultimate model could be used to analyze utilization and expenditure patterns. We know, for example, that managed care enrollees, other things being equal, have 20 percent fewer beds and 10 percent fewer specialist visits than indemnity enrollees. The ultimate model could provide an estimate of utilization and cost-savings effects of HMOs entering and achieving a 30 percent share in, for example, Alabama over the next 5 years.

AGENCY SPONSOR: Office of Health Policy

FEDERAL CONTACT: George Greenberg

PHONE NUMBER: (202) 690-7794 or 690-7804

PIC ID: 6425

PERFORMER ORGANIZATION: Urban Institute, Washington, DC

TITLE: Public Health Laboratories and Health System Change

ABSTRACT: All 50 States and the District of Columbia operate their own Public Health Laboratories (PHLs). Changes in the private health care delivery system, especially the growth of managed care organizations (MCOs), hospital consolidations, and large commercial laboratories are causing public health professionals to examine the viability of publicly funded laboratories in this new environment. The study identifies aspects of the public health infrastructure critical for developing assessment and evaluation tools. The report includes a literature review, interviews with stakeholders, an informal poll of State PHL directors, and detailed case studies of three States that have PHLs with relationships with MCOs and/or private laboratories. The report concludes that (1) PHLs are part of the broader public health infrastructure and, as such, must respond to changes in the health care marketplace; (2) services that are not commercially viable within the private sector, and those that are critical to identifying emerging diseases, should remain in the domain of PHLs; (3) assurance of quality and consistency of laboratory testing are critical functions of PHLs; and (4) policy development in laboratories needs to address the growth of managed care and changes in the health care delivery system. The study recommends increased Federal guidance in (1) assessing the regionalization of laboratory services, (2) supporting information infrastructure development, and (3) facilitating communication between private and public sectors. (Final report: variously paginated.)

AGENCY SPONSOR: Office of Health Policy

FEDERAL CONTACT: Cheryl Austein Casnoff

PHONE NUMBER: (202) 690-6102

PIC ID: 6458

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: A Framework for Assessing Insurer Responses to Health Care Market Changes

ABSTRACT: Over the past 15 to 20 years, the health insurance market has changed dramatically. Where once a few large commercial carriers dominated the market with fee-for-service plans, there are now dozens of plans in most markets. In highly competitive markets, plans and carriers develop more sophisticated strategies and arrangements to identify and retain target markets. In order to successfully integrate government programs and develop public policy, insurers' business strategies need to be better understood. Understanding insurers' strategies is important for policy analysis because insurers are often regulated entities; and insurers' responses to changing market and regulatory conditions affect both patients' and providers' opportunities and outcomes. This report (1) identifies insurer and insurance market characteristics that are likely to be important determinants of insurers' response to policy changes (such as insurance market reform) and changes in local market conditions (such as aggressive entry of managed care plans), (2) identifies potential data sources and data needs for future empirical work, and (3) provides a better understanding of the operation of insurance markets and how insurers make business decisions based on the demand for insurance from employers and individuals. In order to improve the analysis of health insurance markets, the report recommends (1) measuring changes in the relative density of the health plan contour "web," (2) tracking premiums and loading factors, (3) monitoring the self-insured market, and (4) watching organized buyers. (Final report: 44 pages.)

AGENCY SPONSOR: Office of Health Policy

FEDERAL CONTACT: Stephen Finan

PHONE NUMBER: (202) 690-7387

PIC ID: 6572

PERFORMER ORGANIZATION: Urban Institute, Washington, DC

TITLE: Establishing an Analytical Framework for Measuring the Role of Reinsurance in the Health Insurance Market

ABSTRACT: Reinsurance is widely used by underwriters of health benefits to limit their risk exposure. Reinsurance contracts can be structured in many ways, and insurers can use it to limit their risk for an individual contract or exposure, for a group of contracts or exposures, or for a whole portion of the insurer's business. This report presents a limited analytical study of the level of risk borne by reinsurers in three markets: the indemnity insurance market, the managed care market, and the self-insured employer benefit plan market. The study focuses on three States using public and private data sources, including surveys, published private data, and information from filings made by private insurance companies. The purpose of the study is to determine to what extent these data can be used to measure the role of reinsurance in the various markets and to suggest ways whereby better assessments of reinsurance might be made. The report was generally unable to identify either data sources or existing research allowing quantification of the risk being ceded to reinsurers in the three markets. While a substantial amount of information is available about reinsurance in indemnity (non-HMO) health insurers, the data are aggregated in ways that make it difficult to specifically identify transactions related to primary health insurance offered to groups and individuals. Information from State HMO insurance filings is more useful, but lack of detail and inconsistencies in the way that HMOs report the information in their annual statements reduces its analytic potential. Finally, despite the substantial amount of information available about the prevalence of self-funding and the use of reinsurance by self-funded employers, there are virtually no reliable data about the level of risk being transferred to reinsurers in these arrangements. The report closes with suggestions for enhancing the data available on this subject. (Final report: 23 pages, plus appendices.)

AGENCY SPONSOR: Office of Health Policy

FEDERAL CONTACT: Stephen Finan

PHONE NUMBER: (202) 690-7387

PIC ID: 6573

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: National Evaluation of Welfare-to-Work Strategies: Evaluating Two Welfare-to-Work Program Approaches: Two Year Findings on the Labor Force Attachment and Human Capital Development Programs in Three Sites

ABSTRACT: This report is part of a larger study called the National Evaluation of Welfare-to-Work Strategies, formerly known as the JOBS Evaluation, which has randomly assigned more than 55,000 individuals in seven sites to groups eligible for specific welfare-to-work programs and to control groups that do not participate in these programs. This study compares two distinct welfare-to-work strategies-labor force attachment (LFA) and human capital development (HCD)-at each of three of sites: Atlanta, GA; Grand Rapids, MI; and Riverside, CA. At each site, Aid to Families with Dependent Children applicants were randomly assigned to one of three groups: a group subject to the LFA program, a group subject to the HCD program, or a control group not subject to any welfare-to-work program. This report presents findings on the implementation, participation patterns, and costs of the two types of programs operated at each site. Additionally, the report assesses the two program approaches in promoting employment and reducing welfare expenditures after 2 years. The study finds that (1) the LFA programs increased participation in the job search and HCD programs increased participation in adult basic education; (2) welfare sanction rates for nonparticipation were higher in the LFA and HCD programs than in previously studied programs, although higher sanction rates were not associated with higher rates of eventually participating in program activities; (3) the HCD programs cost about twice as much as the LFA programs; (4) both programs increased individuals' 2-year cumulative employment and earnings, and (5) both programs reduced welfare expenditures within the 2-year followup period. The report concludes that a period of 2 years is insufficient to make a credible comparison of the two approaches. See also PIC ID series 5776 and PIC ID Nos. 6576 and 6576.2. (Executive summary: 36 pages; final report: 276 pages, plus appendices.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Audrey Mirsky-Ashby

PHONE NUMBER: (202) 401-6640

PIC ID: 6576.1

PERFORMER ORGANIZATION: Manpower Demonstration Research Corporation, New York, NY

TITLE: Improving Health in the Community: A Role for Performance Monitoring

ABSTRACT: This report draws on lessons from a number of current community health efforts to outline the elements of a community health improvement process, discusses the role that performance monitoring can play in this process, and proposes tools to help communities develop performance indicators. A recent emphasis on the interdependence of stakeholders requires collaborative efforts to improve community health: performance monitoring (a continuing process of selecting indicators that can be used to measure the process and outcomes of an intervention and making the results available to the community) has gained increasing currency as a way to do so. The report finds that a community health improvement process (CHIP) can be an important tool for developing a shared vision and supporting a planned and integrated approach to improving community health. The report suggests that a CHIP should include two principal interacting cycles based on analysis, action, and measurement: (1) a problem identification and prioritization cycle; and (2) the analysis and implementation cycle. This system differs from standard models primarily because of its emphasis on measurement to link performance and accountability on a community-wide basis. The report offers recommendations for implementing the CHIP concept: (1) communities should base a health improvement process on a broad definition of health and a comprehensive conceptual model of how health is produced within the community; (2) a CHIP should develop its own set of specific, quantitative performance measures and should seek a balance between strategic opportunities for long-term health improvement and short-term goals; (3) State and local health agencies should assure that an effective community health improvement process is in place in all communities; and (4) the Public Health Service should provide support and guidance in these efforts. (Final report: 179 pages, plus appendices.)

AGENCY SPONSOR: Office of Program Systems

FEDERAL CONTACT: James Scanlon

PHONE NUMBER: (202) 690-7100

PIC ID: 6583

PERFORMER ORGANIZATION: National Academy of Sciences, Institute of Medicine, Washington, DC

TITLE: Privacy and Health Research: A Report to the U.S. Secretary of Health and Human Services

ABSTRACT: This report (1) identifies privacy issues surrounding research on personally identifiable health data, paying special attention to the international aspects; (2) reviews the ethical, legal, and general social context surrounding the privacy and confidentiality of health data; (3) describes relevant privacy-protection practices and problems, and identifies emerging issues; (4) analyzes the implications of the new European Union Data Privacy Directive and related policy and legal changes; and (5) recommends policy approaches and technical processes for ensuring that the privacy of individuals is respected. The report examines four major groups of issues that are growing rapidly in scale and complexity, and that must be urgently attended to: (1) secondary uses of data and data linking, (2) research on private-sector health data, (3) cybersecurity, and (4) genetic privacy. The report finds that (1) as data bases are maturing and increasing in size and quality, their appeal as research resources is also growing; (2) if personally identifiable data must be used in secondary research, consent will become the most difficult issue; (3) for many health data held in the private sector, few legal controls apply in theory or are enforced in practice regarding such matters as data-subject consent, public notification, or transfer of the data for secondary study; (4) protection of the confidentiality and security of health data, especially electronic data, is being reviewed worldwide; and (5) developing ethical guidance over genetic privacy is crucial to the future of both genetic research and applied genetics. The report also examines the international flow of data, new health data privacy laws in Europe, the United States' new "Health Insurance Portability and Accountability Act," and the need for dialogue between the United States and Europe. (Final report: 76 pages plus appendix.)

AGENCY SPONSOR: Office of Program Systems

FEDERAL CONTACT: John Fanning

PHONE NUMBER: (202) 690-7100

PIC ID: 6605

PERFORMER ORGANIZATION: William W. Lowrance, Ph.D., Geneva, Switzerland

TITLE: State Regulatory Experience with Provider-Sponsored Organizations: Final Report

ABSTRACT: This report describes the experience of selected State governments in regulating provider-owned health care delivery systems that accept insurance risk for the provision or arrangement of health care services (provider-sponsored organizations, or PSOs). The States included in the study are California, Colorado, Illinois, Iowa, Minnesota, Ohio, Pennsylvania, Texas, and Washington. The report finds that the main models States use to regulate the activities of PSOs contracting directly with purchasers of health care (e.g., employers) are (1) licensing these organizations as they would a non-provider owned entity, and (2) creating a separate licensing category for risk-assuming PSOs. Regulators in States using the first model regulate the activity of risk assumption; in some cases, they apply regulation pertaining to non-provider managed care entities to all organizations seeking to seeking to provide or arrange for health care delivery. In States where PSOs are licensed separately from other managed care organizations, differences in the standards required for PSOs varied and there was little consistency about the rationale for having separate licensing. The report also finds that the States vary considerably in their regulation of PSOs that assume risk from self-funded, ERISA-exempt employer plans. Most consider the PSO as engaging in the business of insurance (thus they may be regulated by the State despite the plan's ERISA status). States also vary in their regulation of PSOs that accept risk downstream from licensed insurance carriers or managed care organizations: some do not regulate these relationships, while others require special licensing or impose substantial requirements governing risk transfer agreements. The report concludes that, since State policy toward PSOs is still in the early stages of development, the lessons to be drawn for Federal policymakers is limited; however, some observations are offered.

AGENCY SPONSOR: Office of Health Policy

FEDERAL CONTACT: Lisa Rovin

PHONE NUMBER: (202) 690-7800

PIC ID: 6628

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Indicators of Welfare Dependence and Well-Being: Interim Report to Congress

ABSTRACT: The Welfare Indicators Act of 1994 directed the Secretary of Health and Human Services to conduct a study to determine which statistics would be most useful in tracking and predicting dependence on three means-tested cash- and nutritional-assistance programs: Aid to Families with Dependent Children, Food Stamps, and Supplemental Security Income. Enactment of the Personal Responsibility and Work Opportunity Reconciliation Act of 1996 makes this task even more important. As this bill is implemented, traditional indicators of dependence may lose their meaning, making dependence more difficult to track over time. In response to these difficulties, this report recommends indicators that measure family well-being in addition to those that measure dependence. The report recommends two kinds of indicators: indicators of dependence, self-sufficiency, and family conditions; and indicators of child achievement, health, and well-being. In most cases, the recommended indicators should be tabulated separately for children, male non-elderly adults, female non-elderly adults, and other important population subgroups wherever possible. To measure dependence, self-sufficiency, and family conditions, the report recommends that indicators reflect points on a continuum from total dependence to complete self-sufficiency. Recommended indicators are grouped into several categories, including: (1) range of dependency, (2) work and job readiness, (3) poverty and deprivation, (4) family structure, and (5) parenting. To measure child achievement and health, the report recommends using indicators of infant and child mortality, low birthweight, child health limitations, teen birth rates, teen violent crime arrests, and others. Finally, the report discusses data needs for the recommended indicators. (Final report: variously paginated, plus appendices.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Susan Hauan

PHONE NUMBER: (202) 690-8698

PIC ID: 6634

PERFORMER ORGANIZATION: Office of the Assistant Secretary for Planning and Evaluation, Washington, DC

TITLE: Setting the Baseline: A Report on State Welfare Waivers

ABSTRACT: The Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA) of 1996 eliminated the entitlement to Aid to Families with Dependent Children (AFDC) and replaced it with Temporary Assistance for Needy Families (TANF), a block grant which States can use to provide cash and services to low-income families with children. This report is one in a series designed to provide baseline information on the AFDC program as it existed prior to enactment of the new law. The report focuses on the waivers of AFDC requirements that were granted to States in the years preceding enactment of PRWORA. Between 1993 and 1996, waivers were granted to 43 States to help them reform their welfare programs. The report provides an overview of the types of waivers that States requested and received, and provides details on specific State policies. The report finds that, in many regards, the waiver programs were the first phase of welfare reform. States were required to conduct evaluations of waiver impacts, and had to ensure that the programs would be cost-neutral (that is, that they would not require additional Federal spending). This cost-neutrality could be achieved over the entire waiver period, rather than in each year, as had been previously required. This allowed States to make large up-front investments that they expected to gain back through reduced benefit payments in later years. The report provides valuable information about how specific States approached the AFDC program and what the Federal government authorized them to implement so that the effects of the TANF program can be measured State by State. Furthermore, the TANF program allows States to maintain their waiver programs or to delay implementation of certain TANF provisions until their waiver expires. The bulk of the report consists of tables presenting information on various aspects of the waiver programs. (Final report variously paginated.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Elizabeth Lower-Basch

PHONE NUMBER: (202) 690-6808

PIC ID: 6651

PERFORMER ORGANIZATION: Office of the Assistant Secretary for Planning and Evaluation, Washington, DC

TITLE: Market Barriers to the Development of Pharmacotherapies for the Treatment of Cocaine Abuse and Addiction: Final Report

ABSTRACT: This report presents an analysis of the market barriers to development of pharmacotherapies for substance abuse and addiction, particularly for abuse of and addiction to cocaine. There are more than 2 million addicted or "heavy" cocaine users. Of these, almost a million may enter treatment at least once in a given year. On any given day, about 10 percent of heavy cocaine users are enrolled in treatment programs. The report examines the development of pharmacotherapies for cocaine abusers in the light of the size of the potential market, estimates of market penetration, and the basic relationships between price, market size, and revenues. The report finds that there are several critical market barriers that must be taken into account. For example, there is a small and uncertain market for cocaine addiction and abuse pharmacotherapy. Although drug companies agree that the total number of cocaine users is significant, they believe that the feasible market for a cocaine abuse treatment is likely to be much smaller than the absolute number of people who use cocaine. Drug companies also anticipate that market penetration would be uncertain; potential patient compliance problems, limited access to patients, and the fact that most substance abuse treatment providers view drug use as a behavioral (rather than a medical) problem further exacerbate this uncertainty. Furthermore, the limited number and capacity of treatment facilities restricts market opportunities, as does the lack of medical treatment models. Finally, the substance abuse treatment market relies heavily on State and Federal reimbursement. Most substance abuse treatment services are subsumed under the mental health benefits of entitlement programs, and drug companies are reluctant to rely upon this kind of reimbursement in an age of shrinking budgets for mental health services.

AGENCY SPONSOR: Office of Health Policy

FEDERAL CONTACT: Kevin Hennessy

PHONE NUMBER: (202) 690-7272

PIC ID: 6694

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Consumer-Directed Personal Assistance Services: Key Operational Issues for State CD-PAS Programs Using Intermediary Service Organizations

ABSTRACT: This study was undertaken by the Office of the Assistant Secretary for Planning and Evaluation (OASPE) to identify best practices for implementing consumer-directed personal assistance (CD-PAS) programs through the use of various intermediary service organization (ISO) models. This report provides practical advice to State program administrators about the design and implementation of CD-PAS programs with ISOs in order that: (1) they comply with Federal tax and labor laws; (2) they make available supportive services that some consumers may want or need; and (3) they allow for the application and enforcement of any limitation or restrictions on consumer direction that may be required by State laws and regulations. The report draws a number of conclusions that cut across ISO models, including (1) that the most successful State-ISO contract relationships are forged when the State has a clear understanding of the services that it wishes to purchase from the ISO; (2) that consumers' backup plans for relief attendants frequently fail, causing consumers to have a difficult time finding relief attendants over weekends, holidays, and when the primary attendant is absent for any reason; (3) that consumers are reluctant to initiate criminal background checks on prospective attendants because they do not know how to conduct them and they feel these checks compromise the consumer-attendant relationship; and (4) that a number of disability advocates and CD-PAS consumers expressed concern about the potential impact of managed care on the availability of CD-PAS to eligible consumers. The report also makes recommendations on how to address these problems. (Final report: 70 pages; appendices bound separately.)

AGENCY SPONSOR: Office of Disability, Aging, and Long-Term Care Policy

FEDERAL CONTACT: Floyd Brown

PHONE NUMBER: (202) 690-6613

PIC ID: 6728

PERFORMER ORGANIZATION: SysteMetrics, Cambridge, MA

TOP OF DOCUMENT

Office of Public Health and Science

Contents

Measuring Expenditures for Essential Public Health Services

Evaluation of the HBCU Capacity Building Program

Environmental Health Data Needs: An Action Plan for Federal Public Health Agencies (Workshop Results, March 3-5, 1997-- Linthicum Heights, Maryland)

Stakeholders Revisit Healthy People 2000 to Maximize the Impact for 2010

Health of Washington Women

TITLE: Measuring Expenditures for Essential Public Health Services

ABSTRACT: In 1995, the Public Health Foundation, under contract to the U.S. Public Health Service (PHS), worked with State and local public health, mental health, substance abuse, and environmental agencies in nine States-Arizona, Illinois, Iowa, Louisiana, New York, Oregon, Rhode Island, Texas, and Washington-and with Federal PHS agencies to develop and test a methodology for estimating investments in ten essential public health services. These services include both personal health services (care provided to individuals) and population-based health services (interventions that prevent disease and promote health among entire populations). The report finds that public health expenditures can indeed be defined, measured, and monitored according to the essential services framework. The report also produced State-level estimates of essential public health services spending. It finds that (1) more than 2 of every 3 dollars spent by the 9 States on essential public health services ($6.1 billion of $8.8 billion) was for personal health services; (2) population-based health services spending ($2.7 billion or $36 per capita) was only 1 percent of total health care expenditures in the participating States; (3) of the population-based expenditures, 26 percent was for enforcing laws and regulations protecting the health and safety of the public; (4) States were the largest funders of population-based health services, providing 50 percent of the total funding for these activities (Federal funding accounted for 32 percent, while local and other sources accounted for the remaining 18 percent); and (5) of the $7.1 billion PHS expenditures on essential public health services, $2.4 billion (34 percent) supported prevention-related research and $2 billion (29 percent) supported personal health services. The report makes recommendations to improve the methodology for future data collection. (Final report: 36 pages, plus appendices.)

AGENCY SPONSOR: Office of Disease Prevention and Health Promotion

FEDERAL CONTACT: Deborah R. Maiese

PHONE NUMBER: (202) 401-5809

PIC ID: 6194

PERFORMER ORGANIZATION: Public Health Foundation, Washington, DC

TITLE: Evaluation of the HBCU Capacity Building Program

ABSTRACT: In 1992, the Public Health Service (PHS) initiated the Historically Black Colleges and Universities (HBCUs) Capacity Building Program to increase HBCUs' involvement in health and social service programs funded by PHS and other Federal agencies. The hypothesis was that HBCUs could compete more effectively for grant monies if they had comprehensive and fully functional sponsored program offices (SPOs). This evaluation, conducted by the Office of Minority Health (OMH), addresses a series of questions regarding the appropriateness of program processes and the observable impacts of the program at four participating HBCUs. The OMH designed evaluation methods around a logic model that documents the links between program assumptions, activities, and final outcomes. The OMH collected data through questionnaires, site visits, and telephone interviews. The evaluation includes quantitative and categorical indicators of program implementation and outcomes, as well as comparisons of qualitative case histories. The report finds that the grant funds and technical assistance achieved many of the program's aims. An SPO was established at all four participating HBCUs, resulting in more structured and uniform procedures for obtaining and managing externally sponsored programs. Program impact data show that some desired outcomes, such as increased proposal submissions and awards, were partially achieved. The study concludes by highlighting major findings and recommendations related to program effectiveness, including the need for investments in programmatic, as well as administrative, infrastructure. (Final report variously paginated.)

AGENCY SPONSOR: Office of Minority Health

FEDERAL CONTACT: Joan Jacobs

PHONE NUMBER: (301) 443-9923

PIC ID: 6246

PERFORMER ORGANIZATION: Expand Associates, Silver Spring, MD

TITLE: Environmental Health Data Needs: An Action Plan for Federal Public Health Agencies (Workshop Results, March 3-5, 1997-Linthicum Heights, Maryland)

ABSTRACT: State and local public health agencies have had a long-standing role in monitoring the human health hazards of environmental factors. Public health agencies must be able to collect the necessary information to assess the impact of environmental hazards on community health. The Federal government can help State and local public health agencies in environmental surveillance by identifying the types of information and data necessary to successfully protect community health. This report presents the findings of a conference held in March 1997 to assess the environmental health information needs of State and local governments. The report finds that (1) resources should be used to improve existing data collection systems and to increase data accessibility and usability, rather than to create new data collection systems; (2) the integration of activities and electronic data among public health, environmental health, and environmental protection agencies is vital; (3) the usefulness of environmental health information systems can be enhanced through common data architecture, delineation of core baseline and sentinel data, and a comprehensive, central environmental health data repository; (4) these information systems must be evaluated based on their usefulness to State and local agencies in responding to community demands for timely, accurate, and reliable information, and in achieving their research, surveillance, intervention, and health education goals; and (5) full funding of these systems is necessary so that they may be built into a national network that meets the needs of a national environmental health research agenda. The workshop also developed seven key roles the Federal government could play in helping States and localities improve their environmental health surveillance. Finally, the workshop considered potential short-term strategies to help the Federal government fulfill these roles. (Final report: 12 pages, plus appendices.)

AGENCY SPONSOR: Office of Disease Prevention and Health Promotion

FEDERAL CONTACT: Dalton Paxman

PHONE NUMBER: (202) 205-5829

PIC ID: 6322

PERFORMER ORGANIZATION: Public Health Foundation, Washington, DC

TITLE: Stakeholders Revisit Healthy People 2000 to Maximize the Impact for 2010

ABSTRACT: The objectives articulated in Healthy People 2000 aim to improve the health of Americans through effective prevention strategies. They form a model framework for results-oriented performance measurement that can be made relevant to both population-based and individual health care initiatives. They also serve as a benchmark against which health improvement efforts across the Nation can be measured over time. The development of Healthy People 2010 provides an opportunity to consider the successes and failures of Healthy People 2000 in achieving these important goals. Based on experience with Healthy People 2000 and major changes since 1990 in the Nation's health and health care, this report explores new approaches for the development of Healthy People 2010. The perspectives of the Healthy People Steering Committee and the Healthy People Consortium, made up of 350 member organizations were sought. Through interviews with key Steering Committee members, a retreat for the entire Steering Committee, and seven focus groups of Consortium members, clear themes and suggestions emerged. The report finds that both Steering Committee and Consortium members valued Healthy People 2000 as a framework and a voice for public health concerns and issues. However, both groups had several suggestions for the next version. For example, (1) most members of the Steering Committee thought that the document should be reduced in size, (2) Steering Committee members were more interested than Consortium members in making major changes to the document, (3) Consortium members recommended that current information technology be used in order that the document reach the widest audience possible, and (4) health care purchasers and managed care plans thought that there should be fewer priorities and objectives in order to focus on a limited number of problems. The final stakeholders' report is available on the Internet at http://web.health.gov/healthypeople.

AGENCY SPONSOR: Office of Disease Prevention and Health Promotion

FEDERAL CONTACT: Deborah R. Maiese

PHONE NUMBER: (202) 401-5809

PIC ID: 6491

PERFORMER ORGANIZATION: Academy for State and Local Government, Washington, DC

TITLE: Health of Washington Women

ABSTRACT: This report examines the health and well-being of women in Washington State. A broad coalition of Washington's health care providers, researchers, and activists organized to implement this evaluation. To add to the quantitative data collected and analyzed, the report also includes qualitative information from focus groups of women from diverse geographical areas and economic and social strata. The focus groups were conducted to gather women's perceptions and satisfaction with the health care services they received throughout the State. The data show that the nearly 2.8 million women who live in Washington State are living longer lives, are making many positive decisions about their health, and experience, on average, a better standard of living than women in United States in general. The report finds that the overwhelming majority of women in the State are white and live in urban areas, but about 11.3 percent live below the poverty line. The report also finds that women reported more days of poor mental and physical health than did men, but that they are less likely than men to have a sedentary lifestyle, are less likely to smoke, and are more likely to eat healthy fruits and vegetables. Access to health care for Washington's women is relatively good, but about 10 percent did not participate in a health insurance plan in 1994 and 12 percent reported that they were unable to see a doctor because of the cost. Furthermore, elderly women, those with less than a high school education, and poor women were less likely to use mammography and have regular Pap smears. The report also examines rates of domestic violence and sexually-transmitted diseases, as well as mortality patterns and birth-related indicators. The report provides several policy recommendations to address problem areas for Washington women's health. (Final report: 68 pages, plus appendix.)

AGENCY SPONSOR: Regional Health Administrator-Region X

FEDERAL CONTACT: Karen Matsuda

PHONE NUMBER: (206) 615-2501

PIC ID: 6684

PERFORMER ORGANIZATION: Washington Women's Education Foundation, Seattle, WA

TOP OF DOCUMENT

Substance Abuse and Mental Health Services Administration

Contents

Findings from the Evaluation of the Faculty Development Program, Volume I: Final Report

Report on the CTS National Evaluation

CSAT Demonstration: Evaluation of Job Corps Drug Treatment Enrichment Program

NTIES: National Treatment Improvement Evaluation Study: Final Report

Evaluation of Model Projects for Pregnant and Postpartum Women and Their Infants

Final Report: Review of the CMHS Mental Health Care Provider Education in HIV/AIDS Program

TITLE: Findings from the Evaluation of the Faculty Development Program, Volume I: Final Report

ABSTRACT: The Health Professions Clinical Training Program in Alcohol and Other Drug Abuse (Faculty Development Program, or FDP) was established in 1989 and offers training to faculty from schools of medicine, nursing, social work, and graduate psychology programs. The purpose of the FDP was to enhance research, teaching, and clinical practices on issues of alcohol, tobacco, and other drugs (ATOD) within these fields. Specifically, the program seeks to (1) improve the knowledge and practice of participating faculty on ATOD-related issues; (2) incorporate ATOD-related issues into the curriculum at participating institutions; and (3) increase the level of institutional and professional support for ATOD in both policies and programs. The 5-year projects involved two phases: (1) faculty development and implementation efforts, and (2) an evaluation of the program's impact. The report finds that (1) all projects held meetings and seminars, provided mentoring to fellows, supported fellows' attendance at conferences and meetings, and promoted collaboration between fellows and other faculty members; (2) some project directors with extensive prior ATOD experience provided strong organizational leadership and fostered a group spirit among fellows, while others provided less direct guidance and worked with fellows individually; (3) less experienced directors tended to work with the fellows as peers in organizing activities and formulating a training program; (4) at least two-thirds of the fellows participated in all training activities (except for academic courses) during Years 1-3, while during Year 4, two-thirds continued to meet as a group, attended conferences, and engaged in independent studies; and (5) fellows noted three types of particularly helpful training experiences-literature collection and synthesis, participation in experiential learning, and collaborative curricula development and research. (Draft report: variously paginated.)

AGENCY SPONSOR: Office of the Administrator

FEDERAL CONTACT: Cynthia Prather

PHONE NUMBER: (301) 443-4969

PIC ID: 4510

PERFORMER ORGANIZATION: Cosmos Corporation, Bethesda, MD

TITLE: Report on the CTS National Evaluation

ABSTRACT: The Center for Substance Abuse Prevention national Training System (CTS) included the development and delivery of more than 30 distinct curricula on hundreds of occasions to more than 10,000 participants through 1996. Recipients included community teams, health professionals, State administrators, and a range of volunteers engaged in prevention activities. The report finds that CTS was innovative in several important ways, including (1) a focus on community coalitions and associations of health professionals that are responsible for planning and delivering prevention services, rather than concentrating on target populations; (2) curricula were developed quickly in response to emerging issues in the field; and (3) a continuing commitment to quality control and continual curricular improvement. The report also finds that, of three common levels of prevention training (introductory training, training directed at treatment providers and other health professionals, and approaches that contribute to broad systemic changes, usually directed at teams or coalitions), CTS was far more effective at delivering training at the lower two levels, given the complex nature of training for systemic change. The report then focuses on obstacles to training for systemic change, including (1) the difficulty of disseminating best practices, and (2) resistance to change by the prevention community. The report then discusses the principles of effective training. These include (1) attention to post-training application of the knowledge gained; (2) providing a blueprint or schema by which participants may understand the entire framework for a training effort; (3) focus on barriers to application, as well as advantages; (4) ensuring that training is culturally appropriate; and (5) training should encourage participants to use their intellectual and collective intellectual, verbal, emotional, artistic, and imaginative skills. See also PIC ID No. 4514.

AGENCY SPONSOR: Center for Substance Abuse Prevention

FEDERAL CONTACT: Gwyndolyn Ensley

PHONE NUMBER: (301) 443-0581

PIC ID: 4514.1

PERFORMER ORGANIZATION: Pacific Institute for Research and Evaluation, Bethesda, MD

TITLE: CSAT Demonstration: Evaluation of Job Corps Drug Treatment Enrichment Program

ABSTRACT: The Center for Substance Abuse Treatment within the Substance Abuse and Mental Health Services Administration sponsored a 4-year demonstration project of the Drug Treatment Enrichment Program (DTEP), which was put in place within the Department of Labor's Job Corps program. DTEP was implemented in four experimental Job Corps centers, which were matched with four control centers that provided the standard Job Corps Alcohol and Other Drugs of Abuse (AODA) program. The report finds that (1) the organizational location of DTEP within each Job Corps center's operations was problematic, since DTEP staff reported a lack of managerial support for the program, high staff turnover, and long periods for staff replacement; (2) the average cost per program participant for DTEP was twice the amount required for AODA; (3) DTEP reduced marijuana and crack/cocaine use post-Job Corps significantly more than did AODA; (4) DTEP students were less likely to engage in moderate to extensive use of marijuana or to use other drugs, but no differences were found for alcohol abuse; (5) DTEP improved students' mental health; (6) DTEP students were less likely to report selling or helping to sell drugs after Job Corps; (7) no differences between DTEP and AODA were found for other criminal activities; (8) DTEP students with mental health problems experienced a significantly higher job placement rate after Job Corps, although the program had no effect on job placement rates of other students; and (9) DTEP appeared to have no significant positive effect on duration of stay in Job Corps or on educational and vocational gains. The report makes recommendations regarding the integration of DTEP into Job Corps and other topics. (Final report: variously paginated, plus appendices.)

AGENCY SPONSOR: Center for Substance Abuse Treatment

FEDERAL CONTACT: Ron Smith

PHONE NUMBER: (301) 443-7730

PIC ID: 4523

PERFORMER ORGANIZATION: Caliber Associates, Fairfax, VA

TITLE: NTIES: National Treatment Improvement Evaluation Study: Final Report

ABSTRACT: The National Treatment Improvement Evaluation Study (NTIES) is a comprehensive assessment of the results of 157 3-year demonstration grants and cooperative agreements funded by the Center for Substance Abuse Treatment within the Substance Abuse and Mental Health Services Administration beginning in 1990-1991. These grants were intended to improve alcohol and drug treatment in selected target cities with severe substance abuse problems; among critical populations such as minority groups and women; and in the criminal justice system. Based on a final sample of 6,600 clients, the evaluation produced several results. For clients who entered treatment for alcohol and illegal drug use, the evaluation found that the program produced (1) a 78-percent reduction in selling drugs and committing violent crimes; (2) a 56-percent reduction in high-risk sexual behaviors related to HIV transmission; (3) a 53-percent reduction in medical visits; (4) a 42-percent reduction in homelessness; (5) a 28-percent reduction in inpatient mental health visits; and (6) a 19-percent increase in employment. See also PIC ID No. 5346. (Final report: 310 pages, plus appendices.)

AGENCY SPONSOR: Center for Substance Abuse Treatment

FEDERAL CONTACT: Ron Smith

PHONE NUMBER: (301) 443-7730

PIC ID: 5346.1

PERFORMER ORGANIZATION: National Opinion Research Center (NORC), Chicago, IL

TITLE: Evaluation of Model Projects for Pregnant and Postpartum Women and Their Infants

ABSTRACT: In 1989, the Center for Substance Abuse Prevention (CSAP) implemented a national demonstration program to provide prevention, education, and intervention services to substance-abusing pregnant and postpartum women and infants (PPWIs). CSAP contracted this study in 1993 to (1) assess the effectiveness of the PPWI programs in enhancing services coordination and increasing the availability and accessibility of substance abuse prevention services delivered to clients; and (2) document the extent to which program models are effective in decreasing the use of alcohol and drugs among substance-abusing women and enhancing the healthy development of their children. Ten PPWI programs funded by CSAP in 1992 were selected to participate in a 4-year cross-site evaluation. The wide range of interventions and possible outcomes required that program evaluations were augmented by a process evaluation to explain the relationships between mediating variables and outcomes. The project found that (1) there were significantly lower prevalence rates in the use of alcohol, marijuana, crack, and other illicit substances in the treatment group from intake to delivery, and in recent (30 day) marijuana and crack use in the treatment group from intake to delivery (these differences disappeared by the 6-month postpartum followup); (2) the effect of PPWI treatment group membership for lower recent crack use appears to be attributable to participation in day treatment, rather than case management programs; and (3) in general, treatment and comparison group members gave birth to infants with similar birth-related outcome characteristics, but when the mother's crack use histories were taken into account, treatment babies had significantly higher birthweights than comparison group babies. (Final report: variously paginated, plus appendices.)

AGENCY SPONSOR: Center for Substance Abuse Prevention

FEDERAL CONTACT: Soledad Sambrano

PHONE NUMBER: (301) 443-9136

PIC ID: 5674

PERFORMER ORGANIZATION: Johnson, Bassin and Shaw, Silver Spring, MD

TITLE: Final Report: Review of the CMHS Mental Health Care Provider Education in HIV/AIDS Program

ABSTRACT: This report presents findings from a review of the Center for Mental Health Services' (CMHS) Mental Health Care Provider Education in HIV/AIDS Program, from its inception in 1986 through 1994. The program supports the provision of state-of-the-art training for (1) traditional mental health care providers (psychiatrists, psychologists, nurses, and social workers), (2) other first-line providers of mental health care services (primary care physicians and medical students), and (3) nontraditional providers of such services (clergy or alternative health care workers). The program also supports the development and dissemination of curricula and teaching materials for providing training on the mental health aspects of HIV/AIDS. The goal of the program is to improve access to appropriate and adequate mental health services for people living with HIV/AIDS and for their families and partners. The review examines the activities and outcomes of 27 projects funded by the program. The report finds that (1) the program has trained tens of thousands of mental health service providers, representing the full spectrum of assistance to persons infected with HIV and to their families; (2) training participants are likely to have increased their knowledge about the mental health aspects of the disease, their willingness to serve HIV-infected individuals, and their confidence that they can effectively do so; (3) program funding has served as a catalyst for systemic changes within teaching institutions and hospitals for the development of alliances with concerned groups; (4) the need for training in the mental health aspects of HIV/AIDS is needed throughout the country, especially as affected areas and populations change over time; (5) at the program level, there is a pressing need to obtain systemic knowledge about the efficacy of various approaches to training mental health providers; and (6) the program must identify its priorities and how different projects can meet them.

AGENCY SPONSOR: Center for Mental Health Services

FEDERAL CONTACT: Martha Ann Carey

PHONE NUMBER: (301) 443-3653

PIC ID: 6695

PERFORMER ORGANIZATION: KRA Corporation, Silver Spring, MD

Appendix B. Inventory of HHS Evaluations in Progress

Listed below are evaluations that were in progress during fiscal year 1997, organized by HHS agency. The expected date of completion is given for each project. For more information about a particular project, one may search the HHS evaluation data base accessible through the Internet at the URL address http://www.hhs.gov.

Administration for Children and Families

PIC ID	TITLE	EXPECTED DATE OF COMPLETION
5042.1	Factors Related to Gang Membership Resistance	12/28/1997
6756	Analyzing the Employment and Wage Patterns of Welfare FACE="MS Sans Serif"> Recipients	02/16/1998
5974	Post-Employment Services Demonstration	04/20/1998
5980	Home Visiting Services Demonstration	04/20/1998
6838	Wisconsin Pay for Performance/Self-Sufficiency First Evaluation	04/30/1998
4393	Evaluation of the Head Start/Public School Early Childhood Transition Demonstration	06/20/1998
6837	Welfare Policy Typology Project	06/30/1998
6842	Study of the Impact of the Child Support Enforcement (CSE)Program on Avoiding Costs to Public Programs	07/15/1998
6843	Evaluation and Reporting on State Access and Visitation FACE="MS Sans Serif"> Grant Programs	08/01/1998
5848	National Evaluation of Family Support Programs	09/20/1998
6763	Neighbors, Service Providers, and Welfare Reform in FACE="MS Sans Serif"> Los Angeles County	09/29/1998
6849	Factors Related to the Quality of Family Foster Care	09/29/1998
6851	Project REFRESH: Research and Evaluation of Foster Children's Reception FACE="MS Sans Serif"> into Environmentally Supportive Homes	09/29/1998
3868	Evaluation of the Comprehensive Child Development Programs	09/30/1998
3869	Profiles of Head Start Families: Development of a Head Start Family Information System	09/30/1998
4361	Evaluation of Runaway and Homeless Youth Programs--A FACE="MS Sans Serif"> Follow-Up Study	09/30/1998
4390.1	National Child Welfare Research Center: University of Chicago	09/30/1998
4390.2	National Child Welfare Research Center: University of California FACE="MS Sans Serif"> at Berkeley	09/30/1998
4394	Evaluation of the Head Start Family Service Center Demonstrations	09/30/1998
4396	Evaluation of the Impact of Homelessness on ACYF Programs	09/30/1998
4397	Evaluation of the Transitional Living Program for Homeless Youth	09/30/1998
4974	Study of the Characteristics of Families Served by Head Start Migrant FACE="MS Sans Serif"> Programs	09/30/1998
4975	Evaluation of Head Start Family Child Care Homes	09/30/1998
5845	Descriptive Study of Head Start Bilingual/Multicultural FACE="MS Sans Serif"> Program Services	09/30/1998
5846	National Study of Outcomes for Children Placed in Foster FACE="MS Sans Serif"> Care with Relatives	09/30/1998
6764	Welfare Restructuring Project (WRP)	09/30/1998
6832	Welfare-to-Work: Monitoring the Impact of Welfare Reform FACE="MS Sans Serif"> on American Indian Families with Children	09/30/1998
6836	Management-Focused Welfare Reform Evaluation Data Access and Policy Manual Development	09/30/1998
6846	Role of Child Care in Low-Income Families' Labor Market Participation	09/30/1998
6848	How Decisions to Change the Case Plan Goal Are Initiated	09/30/1998
6850	Assessing the Quality of Foster Family Care: An Initiative for the Integration of Research and Practice	09/30/1998
6883	Demonstration Partnership Program	09/30/1998
6852	Evaluating Quality of Out-of-Home Care in Kinship Foster Families	10/01/1998
6759	Examination of State Diversion Programs	11/30/1998
6834	Welfare Reform Studies and Analyses (Rural TANF)	02/28/1999
5975	Family Preservation and Family Support Services (FP/FS)Implementation Study	06/20/1999
6331	Descriptive Study of Families Served by Head Start	07/18/1999
5982	New Hope Project	09/20/1999
4378	Prevention of Foster Care Placement of Children at Risk for Domestic Violence	10/01/1999
6757	Prenatal and Early Childhood Nurse Home Visitation Program--Replication/Dissemination	02/29/2000
6761	Family Investment Program (FIP)	04/30/2000
6820	Florida Family Transition Program (FTP) Evaluation Project	04/30/2000
6821	Youth Employment and Training Initiative (YETI)	07/01/2000
6760	Evaluation of Community-Based Job Retention Programs	09/29/2000
6762	State Welfare Reform Evaluation Track 2	09/29/2000
6823	Maryland Family Investment Program Evaluation	09/29/2000
6829	Evaluation of the North Carolina Work First Program	09/29/2000
3570	Early Head Start Research and Evaluation National Study	09/30/2000
6839	Partner and Father Involvement in the Lives of Low-Income FACE="MS Sans Serif"> First-Time Mothers and Their Children	09/30/2000
6840	Evaluation of Los Angeles Jobs-First GAIN	09/30/2000
6841	Front-Line Management and Practice Study	09/30/2000
6844	Partners for Fragile Families	09/30/2000
6824	Minnesota Family Investment Program (MFIP)	12/31/2000
6827	New Hampshire Employment and Training Program Process FACE="MS Sans Serif"> and Outcome Study	09/29/2001
6830	Training, Education, Employment, and Management (TEEM) Project	09/29/2001
6847	Child Impact Studies	09/29/2001
6765	Achieving Change for Texans	03/31/2002
6818	Evaluation of Arizona Employing and Moving People Off Welfare and FACE="MS Sans Serif"> Encouraging Responsibility Program (EMPOWER)	04/30/2002
6819	State Welfare Reform Evaluation Project (Jobs First)	04/30/2002
6869	Indiana Welfare Reform Evaluation Project	04/30/2002
6430	Job Opportunities for Low-Income Individuals (JOLI) Program	09/29/2002
6825	Minnesota WorkFIRST Program (Track 2)	09/29/2000
6826	Evaluation of the Employment First Program	09/29/2002
6828	Impact Study of the New Hampshire Employment Program	09/29/2002
6831	Virginia Independence Program	09/29/2002
6833	Assessing Effective Welfare-to-Work Strategies for Domestic FACE="MS Sans Serif"> Violence Victims and Survivors in the Options/Opciones Project	09/29/2002
6845	National Study of Low-Income Child Care	09/29/2002
6758	Ohio Works First Evaluation	09/30/2002
6748	National Longitudinal Study of Children and Families in the Child Welfare FACE="MS Sans Serif"> System	09/30/2003
6835	Jobs-Plus: Community Revitalization Initiative for Public FACE="MS Sans Serif"> Housing Families	12/31/2003
6755	Capitalizing the Bridge from Welfare to Independence (CBWI)	06/30/2007

Administration on Aging

PIC ID	TITLE	EXPECTED DATE OF  COMPLETION
6855	Evaluation of the Supportive Services and Senior Centers Provided Under the Older Americans Act (Title III-B): Design Phase	10/01/1998

Agency for Health Care Policy and Research

PIC ID	TITLE	EXPECTED DATE OF COMPLETION
6382	Evaluation Synthesis: Research Related to the Implementation and Use of Guidelines and Other Quality Improvement Research	11/28/1997
6691.1	Inventory of DHHS Performance Measures. Phase II--Data Collection/Testing and Analysis	11/30/1997
6690	Review of Functions and Operations of the Office of the Administrator and Assessment of the Status of AHCPR Reorganization	12/31/1997
6691.2	Inventory of DHHS Performance Measures. Phase III--Data Collection, Analysis, and Reporting	02/28/1998
6383	Assessment of Data Release Strategies	03/31/1998
6387	Health Care Informatics Standards of Selected Federal Agencies:   A Revised and Updated Report and Conference	05/01/1998
6696	Evaluation of Improvements to Grant Application Receipt and Review (Large and Small Grants)	09/22/1998

Agency for Toxic Substances and Disease Registry

PIC ID	TITLE	EXPECTED   DATE OF COMPLETION
6854	Hazardous Substances Emergency Events Surveillance (HSEES)	12/31/1997

Centers for Disease Control and Prevention

PIC ID	TITLE	EXPECTED DATE OF  COMPLETION
5925	Atlanta Empowerment Zone	10/01/1997
6584	Diabetes Evaluation Protocol	10/01/1997
6707	HIV Prevention Indicators	11/05/1997
5518	Management and Operational Indicators of HIV Community Planning	11/15/1997
5929	Assessment of Violence Prevention Technical Assistance Efforts for State and Local Health Departments	11/30/1997
6336	Evaluating Small Business Interventions	11/30/1997
6708	Elder Care Focus Group	11/30/1997
6713	Evaluation of the Use of Data from Immunization Information Systems	12/15/1997
6705	Feasibility Study: Pap Screening in STD Clinics	12/30/1997
6278	Community Indicators for Assessing HIV Prevention	12/31/1997
6711	STD Partner Notification Protocol	12/31/1997
5502	Evaluation of Tuberculosis Outreach Worker Activities	01/31/1998
5930.1	Immunizations KAPs of Primary Care Providers in the U.S.	01/31/1998
6332	Evaluation of Data Use of National Electronic Telecommunications System for Surveillance (NETSS) and Public Health Laboratory Information System (PHLIS)	01/31/1998
6710	Skin Cancer Survey	01/31/1998
6702	Evaluating Lead Poisoning Prevention Training	02/15/1998
6334	Evaluation of Dental Treatment/Fluoridation	02/28/1998
6275	Development and Implementation of Strategic Planning	03/01/1998
5923.1	Evaluation of Field Epidemiology Training Program (FETP)--Phase II	03/31/1998
6716	Survey of Assisted Reproductive Technology (ART) Embryo Lab Procedures	05/31/1998
6700	Evaluation of MMWR Recommendations/Bike Helmets	06/01/1998
6712	Pregnancy-Related Violence	06/30/1998
4230	Evaluation of the Medical Certification Process for Death Certificates	09/30/1998
6276	Urban Health Systems Sentinel Network	09/30/1998
6333	Evaluation of Group B Strep Guidelines	09/30/1998
6335	Evaluation of the Impact of the Clinical Laboratories Improvement Amendments (CLIA)	09/30/1998
6699	Evaluation of the U.S. Standard Certificates	09/30/1998
6714	Coordinated Evaluation of Tobacco Control Efforts	09/30/1998
6274	Evaluation of Teen Pregnancy Interventions	10/01/1998
6706	Evaluation of the HIV TA Network	11/30/1998
6709	Evaluation of C. Everett Koop Community Health Center (CHIC)	12/30/1998
6701	Community Indicators--Literature Search	12/31/1998
6704	PCP Guidelines in Underserved Communities	12/31/1998
6715	Program Evaluation Technical Assistance Needs	02/01/1999
6703	Evaluation of Implementation/Impact of VRE Guidelines	08/07/1999
6279	Evaluating Guidelines for Preventing Perinatal HIV	09/30/1999
5919	Determining the Effectiveness of the Prevention Centers Program	12/20/1999
5922	Assessment of the Public Health Impact of the NIOSH Health Hazard Evaluation Program	12/20/1999
5923	Evaluation of the Field Epidemiology Training Program	12/31/1999

Food and Drug Administration

PIC ID	TITLE	EXPECTED   DATE OF  COMPLETION
6461	FDA Penalty Reduction Program for Small Businesses	03/20/1998
6462	FDA Inquiries Assistance Program for Small Businesses	03/20/1998
4355	Evaluation of Influenza Virus Vaccines in High-Risk Patient Categories	09/30/1998
6079.3	Prescription Drug User Fee Act (PDUFA) Management	09/30/1998
6080.1	Mammography Quality Standards Act of 1992	09/30/1998
6733	Safe Alert Satisfaction Survey	09/30/1998
6864	Harmonizing Nutritional Goals and Food Processing	09/30/1998
6865	Estimating the Benefits of Various Kinds of Food Standards	09/30/1998
6866	Survey of Current Hazard Analysis and Critical Control Point FACE="MS Sans Serif"> (HACCP) Practices of the Food Industry	09/30/1998
6867	Economic Characterization of the Dietary Supplement Industry	09/30/1998
6880	Dietary Supplement Sales Information	09/30/1998
6881	Evaluating Federal Standard of Identity Regulations	09/30/1998
6890	Impact of Tobacco Initiative on Youth	12/31/1998
4984	Nationwide Evaluation of X-Ray Trends (NEXT)	12/20/1999
5711	Food Labeling and Package Survey (FLAPS)	12/31/1999
6732	Real-Time PMA Supplement Program Evaluation	09/30/2000
6736	Review of Therapeutic Agents as an Antimicrobial Adjunct FACE="MS Sans Serif"> in Treatment of Sepsis	10/01/2000

Health Care Financing Administration

PIC ID	TITLE	EXPECTED DATE OF  COMPLETION
6294	Medicare HMO Evaluation	10/10/1997
6299	Evaluation of Rural Health Clinics	10/31/1997
6305	Evaluation and Technical Assistance of the Medicare Alzheimer's Disease Demonstration	11/30/1997
6290	Evaluation of Medicaid-Managed Care Programs with 1915(b) Waivers	03/31/1998
5958.5	Medicare Participating Heart Bypass Center Demonstration Extended Evaluation	02/28/1998
6296	Drug Utilization Review Evaluation Contract	02/28/1998
6300	Examination of the Medicaid Expansions for Children	03/30/1998
5735	Development of a Self-Assessment Tool for HIV Health Services Planning Councils and HIV Care Consortia	10/08/1997
6358	Measuring the Costs of Medical Education in Ambulatory FACE="MS Sans Serif"> Training Sites	11/30/1997
6309	Evaluation of the Program of All-Inclusive Care for the Elderly Demonstration	03/31/1998
6310	Project Demonstrating and Evaluating Alternative Methods to Assure and Enhance the Quality of Long-Term Care Services for Persons with Developmental Disabilities through Performance- Based Contracts with Service Providers	09/01/1998
6287	Evaluation of HMO Outlier Demonstration	09/28/1998
5966.3	Evaluation of Medicare SELECT	09/30/1998
6303	Maximizing the Effective Use of Telemedicine: A Study of the Effects, Cost Effectiveness, and Utilization Patterns of Consultation via Telemedicine	09/30/1998
6166	Evaluation of Oregon Medicaid Reform Demonstration	09/20/1999
6289	Evaluation of the State Medicaid Reform Demonstrations  (HI, RI, TN)	09/29/1999
6288	Evaluation of the Demonstration Entitled Delaware Health Care Partnership for Children	09/30/1999
6307	Evaluation of the Nursing Home Case-Mix and Quality Demonstration	09/30/1999
6308	Evaluation of Phase Two of the Home Health Agency Prospective Payment Demonstration	09/30/1999
6292	Evaluation of the Medicare Choice Demonstration	01/30/2000
6289.1	Evaluation of the State Medicaid Reform Demonstrations (OH, MN, KY, CA [L.A. County], NY, and VT)	09/30/2000

Health Resources and Services Administration

6355	Impact of Community Health Advisors on Access, Utilization, and Patient Knowledge/Behavior	01/30/1998
6806	Enrollment, Utilization and Cost-Forecasting Models for Use in State AIDS FACE="MS Sans Serif"> Drug Assistance Programs	02/13/1998
6353	Medicaid Managed Care and FACE="MS Sans Serif"> Federally Qualified Health Centers: Experiences of Plans, Networks and Individual Health Centers	03/12/1998
6362	Review and Synthesis of HIV-Related Consumer-Level FACE="MS Sans Serif"> Evaluations	05/14/1998
5614	Person-Place Approximations and Utilization vs. Health Status FACE="MS Sans Serif"> Analysis of the National Health Interview Survey	05/29/1998
6251	Consortia Development for Health Professions Training in FACE="MS Sans Serif"> Community-Based Settings	05/29/1998
4918.1	Evaluation of the Effectiveness and Impact of Community and Migrant Health FACE="MS Sans Serif"> Centers: Implementation Phase	05/31/1998
6254	Meeting Industry Standards for Performance Measurement FACE="MS Sans Serif"> through Clinical Outcome Measures at Community and Migrant Health Centers	05/31/1998
5747	Training and Educational Needs of the Public Health Workforce under Health Care Reform	06/30/1998
6249	Strategies for the Recruitment, Retention, and Graduation of Hispanics into Baccalaureate Levels of Nursing	06/30/1998
6354	Performance Assessment of Managed Care in Community and Migrant Health FACE="MS Sans Serif"> Centers	06/30/1998
6363	Case-mix Differences in Health Centers and Other Providers and Their FACE="MS Sans Serif"> Relationship to Cost	06/30/1998
6364	Health Care Status Outcome Measures for the Bureau of Primary Health Care: The Assessment of Ambulatory Care Sensitive Conditions through State Medicaid Recipient Files	06/30/1998
6810	Continued Performance Measurement Development for MCHB Programs	07/31/1998
6814	HRSA Crosscutting Community Case Studies	07/31/1998
6817	Development and Refinement of Tools for Monitoring Cultural Competence in Managed Care	09/07/1998
6808	Development of Estimates of Unduplicated AAR Client Counts	09/29/1998
6361	Community Care Network Evaluation	09/30/1998
6801	Customer Survey of Covered Entities Under Section 602 of the Veterans Health Care Act of 1992	09/30/1998
6800	Development of BHPr's Comprehensive Performance Monitoring System	09/30/1998
6802	Health Status Outcomes for the Bureau of Primary Health Care: A Pilot Study Assessing Physiologic Measures through Medical Record Review	09/30/1998
6803	Effect of Medicaid Managed Care Beneficiary Enrollment and Autoassignment Practices on FQHCs and Their Patients	09/30/1998
6807	Direct Support for Conduct of Evaluation Studies at Selected CARE Act Pilot Sites	09/30/1998
6809	Evaluation Strategies and Performance Measures for SPRANS FACE="MS Sans Serif"> and CISS	09/30/1998
6811	MCHB GPRA Maturity Project	09/30/1998
6816	Evaluation of Managed Care and Vulnerable Populations	09/30/1998
6804	Development and Testing of Emergency Department Utilization FACE="MS Sans Serif"> as a Measure of Effectiveness for the Health Care for the Homeless FACE="MS Sans Serif"> Program	10/30/1998
6805	Analysis of CHC User Survey: Selected Conditions	11/13/1998

6812	Effectiveness of Diverse Methods of Technical Assistance to Historically FACE="MS Sans Serif"> Black Colleges and Universities	11/20/1998
6368	Assessment of Bi/Multilingual Services Offered at Selected Community and Migrant Health Centers	12/31/1998
6813	Development and Testing of Data Collection Instruments for FACE="MS Sans Serif"> HRSA-Supported Minority Health Professions Programs	12/31/1998
6815	Managed Care and the Safety Net Providers	01/29/1999
6357	Effectiveness of the National Health Service Corps	12/31/1999
5610	National Evaluation of the Healthy Start Program	03/31/2001

TOP OF DOCUMENT

Indian Health Service

PIC ID	TITLE	EXPECTED   DATE OF  COMPLETION
6859	Cross-Sectional Evaluation of the Prevalence of Childhood Obesity among American Indians in the Aberdeen Area	03/31/1998
6746	Evaluating the Effects of Medical Nutrition Therapy on Patient Outcomes among Native Americans with Newly Diagnosed Type II Diabetes Mellitus	05/31/1998
5944	Evaluation of Available Cancer Surveillance Systems for Montana Native Americans and Correlates of Decreased Cancer Survival among Native Americans in Montana	06/30/1998
6153	Development of a Health Services Research Agenda for American Indian and Alaska Native Populations	06/30/1998
6452	Elderly Wellness	06/30/1998
6744	Evaluation of the Effectiveness of the IHS Alcohol and Substance Abuse Treatment Programs, Phase I	09/30/1998
6742	Study of the Interaction of Persons Treated for Alcoholism with the Health Care Delivery System	12/01/1998
6745	Evaluation of IHS Capacity for Epidemiologic Surveillance	12/31/1998
6201	Evaluation of IHS-Supported Alcohol and Substance Abuse Treatment Program for American Indian/Alaska Native (AI/AN)Women	05/30/2000
6450	Resource Requirement Methodology Update	09/29/2001

TOP OF DOCUMENT

National Institute of Health

PIC ID	TITLE	EXPECTED  DATE OF COMPLETION
6122	NIH AIDS Research Program Evaluation	0/20/1997
5586	Comprehensive School Health Programs in Grades K-12	11/30/1997
6280	Career Status and Satisfaction with NIH Research Grant Application and Award Processes: A Sample Survey of FY 1992 R01 and R29 Applicants	11/30/1997
6091	Design and Pilot Test of an Inventory of Cancer Research Across Federal Agencies	12/06/1997
4663.1	Evaluation of Knowledge-Based Expert System for Subject Indexing	12/20/1997
5569	Evaluation of New Technology: Audio-Casi	12/20/1997
5580	Evaluation of the NCRR Research Centers in Minority Institutions (RCM) Program, Phase I: Planning and Methodology Development for a Full-Scale Evaluation	12/20/1997
6046	Evaluation of the Institutional Award (IDeA) Program, Phase I: Evaluation Design	12/20/1997
6047	Evaluation of the Resource Centers Activities with the Biomedical Research Technology Program of the NCRR, Phase I: Planning and Methodology Development	12/20/1997
6097	Strategic Plan for Public Health Agencies	12/20/1997
5584	Evaluation of the NHLBI Programs of Excellence in Molecular Biology	1 2/31/1997
5984	Vaccine Development: Establishing Priorities for the United States for the 21st Century	12/31/1997
6098	Follow-Up Study of Young Investigators in the Biological and Biomedical Sciences	12/31/1997
6282	National Institute of Mental Health (NIMH) Training Program for Minority Students Evaluation	12/31/1997
4509	Evaluation of the Research Demonstration Projects for Homeless Persons with Alcohol and Other Drug Problems	01/31/1998
6096	Evaluation of the National Institute of Mental Health (NIMH) National Research Service Awards for Predoctoral Individual M.D./Ph.D. Fellows--Phase I	03/01/1998
5588	Career Outcomes of a Matched Sample of Men and Women Ph.D.s in Science and Engineering	03/30/1998
5578	Planning Phase for an Evaluation of the NIDR Center Program	04/01/1998
6095	Evaluation of the National Research Service Training Programs	06/01/1998
6118.1	Evaluation of the NHLBI Short-Term Training for Minority Students Program	06/19/1998
6285	Evaluation of NIH Postdoctoral Trainees and Fellows	06/30/1998
4230	Evaluation of the Medical Certification Process for Death Certificates	09/30/1998
6861	Surgeon General's Report on Oral Health	09/30/1998
6045.1	Full-Scale Evaluation of the Regional Primate Research Centers (RPRC) Program	12/20/1998
6860	Study of National Needs for Biomedical and Behavioral Research Personnel	12/30/1998
6863	Survey of Scientific and Engineering Research Facilities at Colleges and Universities	09/30/1999
6862	Evaluation of Simplification of Human Resources Management System at the National Institutes of Health	10/30/2001

TOP OF DOCUMENT

Office of the Assistant Secretary for Planning and Evaluation

PIC ID	TITLE	EXPECTED   DATE OF  COMPLETION
6227	Learning from Ongoing Welfare Waiver Evaluations: Lessons from Implementing Work-Focused Employment Programs for Welfare Recipients	11/30/1997
2771.9	Teenage Parent Demonstration and Evaluation	12/20/1997
6049	Research and Evaluation Framework for the New Pharmaceutical Marketplace	12/20/1997
6168	A Study of Home Health Practice Variations Study	12/20/1997
4528.1	Consequences and Cost of Intimate Partner Violence	12/31/1997
6017	State Development and Use of Insurance Coverage Surveys	12/31/1997
6176	Panel on Performance Measures and Data to Support Public Health Performance Partnership Grants	12/31/1997
6179	Strategies for Obtaining Public Health Infrastructure FACE="MS Sans Serif"> Data at Federal, State, and Local Levels	12/31/1997
6345	Assessment of Access to Liability Insurance for Tribes and Tribal Organizations with Self-Determination Contracts/Compacts	12/31/1997
6346	State Linked Data Meeting and Monograph	12/31/1997
6418	Role of Home and Community-Based Services in Meeting the Health Care Needs FACE="MS Sans Serif"> of People with HIV/AIDS	12/31/1997
6729	Innovative State Strategies to Insure Children	12/31/1997
6347	Innovative Coverage of Health Promotion, Clinical Preventive FACE="MS Sans Serif"> Services, and Other Coverage Issues in the Private Sector	01/30/1998
6240	Exploratory Study of Health Care Coverage and the FACE="MS Sans Serif"> Employment of People with Disabilities	01/31/1998
6721	Analysis of Children's Health Insurance Policy	02/28/1998
6767	Design and Feasibility of a Child Support Enforcement FACE="MS Sans Serif"> Customer Survey	02/28/1998

6173	Client-Directed In-Home Care/Personal Assistance Services	03/31/1998
6343	Evaluating the Effects of Direct-to-Consumer Advertising on Consumers	03/31/1998
6722	Preventing Teen Pregnancy, HIV, and STDs	04/15/1998
6342	Resource Evaluation of the FDA Food Additive Petition Review Process	04/30/1998
6891	One-Stop Shop Employment and Training Models for AFDC Recipients	04/30/1998
6892	Labor Market Conditions, Job Search Strategics, and Welfare-to-Work Transitions	04/30/1998
6719	Analysis of Trends in Elderly Disability Rates	05/14/1998
6576.3	Implementation, Participation Patterns, Costs, and Two-Year Impacts of the Portland, Oregon Welfare-to-Work Program	05/15/1998
6878	What Works for Youth--An Examination of Evaluations	05/31/1998
6740	Beyond Education and Training: The Need for and Impact of Ancillary FACE="MS Sans Serif"> Services	06/01/1998
6750	Enabling Services Methodology Workshop	06/05/1998
4719.5	National Study of Assisted Living for the Frail Elderly	06/30/1998
5758	Health and Long-Term Care Expenditure Patterns of FACE="MS Sans Serif"> Children with Disabilities	06/30/1998
6725	Teen Pregnancy Prevention Programs and Partnerships	07/01/1998
6752	Health and Adjustment of Immigrant Children and Families	07/30/1998
6698	Research Consultation on Review of HHS Services Research, Demonstrations, and Evaluations Targeted to Hispanic/Latino FACE="MS Sans Serif"> Americans	07/31/1998
6877	Adolescent Decision Making Workshop	07/31/1998
6893	Making Ends Meet: How Mothers Manage When Their Welfare FACE="MS Sans Serif"> Grants Are Cut	07/31/1998
6399	Study of Private Long-Term Care Insurance Claims	08/01/1998
6341	Evolving Health Care Delivery Systems: A Workshop	08/31/1998
6717	Paper Series on the Low-Wage Labor Market	08/31/1998
6726	Abstinence-Based Teen Pregnancy Prevention Efforts Targeting Boys	08/31/1998
6724	Domestic Violence and Welfare: An Early Assessment	09/15/1998
6723	Analysis of Prescription-to-Over-the-Counter Switch Movement	09/24/1998
6727	Wisconsin Data Project on Former AFDC Recipients	09/29/1998
5759	Analysis of Data on Use of Nursing Homes in Florida	09/30/1998
6160	Home Visitor Services Demonstration: Home Visiting for Teen FACE="MS Sans Serif"> Parents Required to Participate in JOBS	09/30/1998
6391	Study of Managed Delivery Systems for Medicare Beneficiaries with Disabilities and Chronic Illnesses	09/30/1998
6420	State Telephone Survey Estimates for Evaluation and Monitoring	09/30/1998
6634.1	Indicators of Welfare Dependence: 1997 Annual Report	10/31/1997
6753	Resource Guide on Quality Assurance Systems for SSI Disabled FACE="MS Sans Serif"> Populations	09/30/1998
6766	Pregnancy Prevention Programs Targeting Boys and Young Men: Policy Information Dissemination Strategy	09/30/1998
6768	Analysis of Patterns of Post-Acute and Chronic Care Services Use Among FACE="MS Sans Serif"> Disabled and Non-Disabled Medicare Beneficiaries 1989-1994	09/30/1998
6889	Impact of Medicare HMO Enrollment on Health Care Costs in California	09/30/1998

6162	Institute on Consumer Directed Services (Long-Term Care FACE="MS Sans Serif"> Infrastructure)	10/20/1998
6749	Mandatory Review and Modification in TANF Cases	11/10/1998
6754	Child Health and Development Programs in the Context of FACE="MS Sans Serif"> Welfare Reform	12/22/1998
6419	Statistical Linkage of Data Files for Evaluation: National FACE="MS Sans Serif"> Health Interview Survey and Survey of Income and FACE="MS Sans Serif"> Program Participation	12/30/1998
6576.4	Child Outcome Study of the National Evaluation of Welfare-to-Work Strategies: Two-Year Impacts	12/30/1998
6718	Analysis of Employer-Group Long-Term Care Insurance	12/31/1998
6751	Preventing Abusive Intimate Relationships among Adolescents	01/01/1999
6172	Assisted Living: Description, Quality, and Standards	03/31/1999
6166.1	Evaluation of the Oregon Medicaid Reform Demonstration	08/31/1999
6289.2	Evaluation of the State Health Reform Demonstrations	08/31/1999
6429	Estimating an Analytical Framework for Assessing Insurer Response to the Health Care Market	09/30/1999
5337.2	Review of Family Preservation and Family Reunification Programs	12/31/1999
6720	Medicare Home Health Practice Variations	03/31/2000
5888	Evaluation of Practice in Care (EPIC)	06/30/2000
6398	Private Payers Serving Disabled Individuals and Research Synthesis on FACE="MS Sans Serif"> Managed Care for Persons with Disabilities	09/28/2000
6161	Personal Assistance Service Cash Payment Demonstration	09/30/2000
6747	Economic and Health Status of Immigrants, Their Communities, and the Organizations that Serve Them	10/01/2000

TOP OF DOCUMENT

Office of Public Health and Science

PIC ID	TITLE	EXPECTED DATE OF COMPLETION
6688	Status Report and Evaluation of AoA and ACF Programs Following Terminations of Federal Funding in the Freely Associated States (FAS)	10/30/1997
6193	Commission on Dietary Supplement Labels	11/01/1997
6488	States' and Localities' Ability to Measure Healthy People 2000 Objectives	11/30/1997
6678	Evaluation of a Community-Based Public Health Practice Training Program	11/30/1997
6198	Evaluation Synthesis of School Health Programs	12/01/1997
6676	Examination and Assessment of U.S. and U.K. Strategies and Approaches for Addressing Racial/Ethnic Minority Health Concerns	12/01/1997
6492	Role of Health Plans in Community-Level Health Improvement	12/31/1997
6677	Evaluation of Preventive Health Care Screening among Older Women	12/31/1997
6679	Evaluation of Managed Care Plans' Coverage of Women's Health Initiatives	12/31/1997
6681	Evaluation of Males Who Are Fathers of Children Born to Teenage Girls	01/01/1998
6682	Evaluation of the Quality and Effectiveness of Breast and Cervical Cancer Screening in Region X	01/01/1998
6683	Evaluation of Pregnancies Averted and Cost Savings in Federally-Funded Clinics in Region X	01/01/1998
6247	Evaluation of the Effectiveness and Efficacy of the Office of Minority Health's Bilingual/Bicultural Service Demonstration Grants Program	03/31/1998
6685	Retrospective Study of the Preventive Health Practices of Former Title X (Family Planning) Clients	04/30/1998
6689	Assessment of Region IX Programs Promoting Positive Images in Girls Ages 9-14	05/15/1998
6875	Conference and Proceedings to Explore the Uses of Summary Measures of Health Status in Assessing Population Health	05/30/1998
6680	Evaluation of an Immunization Education Program for Child Care Centers	07/01/1998
6675	Special Session at the First National Conference on Cultural Competence in Health Care to Address Issues of Measurement and Linkages to Health/Patient Outcomes	09/01/1998
6326	Analysis of State and Local Public Health Policy and Program Activities	09/28/1998
6244.1	Evaluation of the Office of Minority Health Resource Center--Phase II	09/30/1998
6321	Evaluation of Models to Integrate HIV Prevention and Treatment	09/30/1998
6324	Integrated Information Action Plan Services within PHS-Supported Programs	09/30/1998
6686	Develop Local Department Early Detection Capabilities for the Health and Medical Consequences of the New Welfare Reform	09/30/1998
6687	Study of the Impact on Public Health Services of Increased Migrant Populations in Northwest Arkansas	09/30/1998
6798	Assessment of Linguistically and Culturally Appropriate Community Health Promotion Programs in Local Health Departments	09/30/1998
6876	Enabling Performance Measurement Activities	09/30/1998
6325	County-Level Tracking of Public Health Functions and Policy Issues	12/15/1998
6327	Science Panel on Interactive Communication Technology and Health	12/31/1998
6674	National Assessment of Linguistically and Culturally Appropriate Services in Managed Care Organizations Serving Racially and Ethnically Diverse Communities	09/30/1999
6323	Scientific Evaluation of Dietary Reference Intakes	09/30/2000

TOP OF DOCUMENT

Substance Abuse and Mental Health Services Administration

4997	National Evaluation of the Community Partnership Demonstration Project	01/30/1998
5994	National Evaluation Database Service	09/20/1998
6044	Evaluation of High-Risk Youth Substance Abuse Prevention Initiatives Funded in 1994	04/20/1999
4980.1	Evaluation of the HHS Access to Community Care and Effective Services and Supports (ACCESS) Program	12/20/1999
6042	Cross-Site Evaluation of the Community Prevention Coalitions Demonstration Grant Program	05/24/2000
6454	State Substance Abuse Managed Care Evaluation Program	09/26/2000
6737	Treatment Improvement Protocols Field Evaluation	09/01/2001
6738	Persistent Effects of Treatment Studies (PETS)	05/31/2002
6739	Evaluation of Opioid Treatment Program Accreditation Project	09/29/2002

Appendix C. Evaluation Review Panel's Criteria for Assessing Program Evaluations

OVERALL SIGNIFICANCE

• The study addresses a significant issue of policy relevance.
• Evaluation findings are likely to be useful.

CONCEPTUAL CRITERIA

Conceptual Foundations

• A literature review is included.
• The project is shown to be logically based on previous findings; the report uses either theory, or models, or both.
• The program assumptions are stated.
• The evaluation draws from any previous evaluation.
• The report is linked with a program and describes the program.
• The report presents multiple perspectives.
• Multiple relevant stakeholders are consulted and involved.
• The timing is appropriate because the program is ready for evaluation.

Questions for Evaluation

• The aims of the evaluation are clear, well-specified, and testable.
• The questions are feasible, significant, linked to the program, appropriate for the resources and audience, and derive logically from the conceptual foundations.
• The questions show ingenuity and creativity.

Findings and Interpretation

• The conclusions are justified by the analyses.
• The summary does not go beyond what the data will support.
• The appropriate qualifiers are stated.
• The conclusions fit the entire analysis.
• Equivocal findings are handled appropriately.
• The initial questions are answered.
• The interpretation ties in with the conceptual foundation.
• The report notes that the findings are either consistent with or deviate from the relevant literature.
• The presentation is understandable.
• The results have practical significance.
• The extent of program implication is assessed.

Recommendations

• The recommendations follow from findings, are worth carrying out, and are affordable, timely, feasible, useful, and appropriate.
• The recommendations are shown to be relevant to the questions asked.
• The breadth of specificity of the recommendations is addressed.
• Any recommendations for either future evaluations, or improvements, or both are clearly presented.

METHODS

Evaluation Design

• Design considerations include overall appropriateness, soundness, feasibility, funding and time constraints, generalizability, applicability for cultural diversity, assessment of the extent of program delivery, validity, feasibility for data collection, reliability of selected measurements, use of multiple measures of key concepts, and appropriateness of the sample.
• Variables are clearly specified and fit with the questions and concepts.
• The design permits measurement of the extent of program implementation and answering of the evaluation questions.

Data Collection

• Data are collected using appropriate units of measurement for analysis, controls for participant selection and assignment bias, and proper handling of missing data and attrition.
• Data collection is characterized by use of an appropriate comparison group of control; adequate sample size, response rate, and information about the sample; a data collection plan; data collection that is faithful to the plan; attention to and cooperation with the relevant community; project confidentiality; and consistency.
• The quality of the data (including the quality of any extant data sets used in the study) and the efficiency of sampling are addressed.
• The data collection is appropriate to evaluation questions.

Data Analysis

• The data analysis addresses the handling of attrition, the matching of the analysis to the design, the use of appropriate statistical controls, the use of methodology and levels of measurement appropriate to the type of data, and estimation of effect size.
• The analysis shows sensitivity to cultural categories.
• The analysis makes appropriate generalizability of inferences.
• The chosen analysis type is simple and efficient.

CROSS-CUTTING FACTORS

The following are cross-cutting factors that are likely to be important at all stages of a report: clarity, presentation, operation at a state-of-the-art level, appropriateness, understandability, innovation, generalizability, efficiency of approach, logical relationships, and discussion of the report's limitations. The report should also address ethical issues, possible perceptual bias, cultural diversity, and any gaps in study execution.

Appendix D. Glossary of Acronyms

Glossary of Acronyms

ABCDEFGHIJKLMNOPQRSTUWY

ACCESS Access to Community Care and Effective Services and Supports

ACF Administration for Children and Families

ACYE Administration for Children, Youth and Families

ADAP AIDS Drug Assistance Programs

AFDC Aid to Families with Dependent Children

AHCP Agency for Health Care Policy and Research

AIDS Autoimmune Deficiency Syndrome

AoA Administration on Aging

AODA Alcohol and Other Drugs of Abuse

APGs Ambulatory Patient Groups

ART Assisted Reproductive Technology

ASPE (Office of the) Assistant Secretary for Planning and Evaluation

ATOD Alcohol, Tobacco, and Other Drugs

ATSDR Agency for Toxic Substances and Disease Registry

TOP OF DOCUMENT

BHPr Bureau of Health Professions

BRTA Business Responds to AIDS
 

CAAS Community Action Agencies

CARE Comprehensive AIDS Resources Emergency Act

C.A.R.E. Cognitive Activity and Reimbursement Effectiveness

CBWI Capitalizing the Bridge from Welfare to Independence

CCAPP Community Coalition Acting for Positive Parenting

CCCANC Cumberland County Child Abuse and Neglect Council

CDC Center for Disease Control and Prevention

CD-PAS Consumer-Directed Personal Assistance

CERCLA Comprehensive Environmental Response, Compensation, and Liability Act

CHC Community Health Centers

CHIC C. Everett Koop Community Health Center

CHIP Community Health Improvement Project

CLIA Clinical Laboratory Improvement Amendments

CLP Community Lifeline Programs

CMHS Community Mental Health Services

CNOs Community Nursing Organizations

CONQUEST Computerized Needs-Oriented Quality Measurement Evaluation System

CPS (a) Child Protection Services

CPS (b) Current Population Survey

CSAT Center for Substance Abuse Treatment

CSE (Study for the Impact of the) Child Support Enforcement

CSLA Community Supported Living Arrangement

CTS Center for Substance Abuse Prevention National Training System

TOP OF DOCUMENT

DOT Division of Organ Transplants

DPP (a) Demonstration Partnership Program

DPP (b) Disabilities Prevention Program

DTEP Drug Treatment Enrichment Program

DUR Drug Utilization Review
 

EACH-RPCH Essential Access Community Hospital/Rural Primary Care Hospital

EMPOWER Evaluation of Arizona Employing and Moving People off Welfare and Encouraging Responsibility Program

EMAs Eligible Metropolitan Areas

EPA Environmental Protection Agency

EPIC Evaluation of Practice in Care

EPOC Evaluation Policy Oversight Committee

EPSDT Early Periodic Screening, Diagnosis, and Treatment

TOP OF DOCUMENT

FACE Fatality Assessment and Control Evaluation

FDP Faculty Development Program

FAP Family Advocate Plan

FAS Freely Associated States

FCC Family Care Connection

FCCTF Franklin County Children's Task Force

FDA Food and Drug Administration

FETP Field Epidemiology Training Program

FFS Fee-For-Service

FIA Family Investment Agreement

FIP Family Investment Program

FLAPS Food Label and Package Survey

FP/FS Family Preservation/Family Support

FTP Florida Family Transition Program
 

GAPS Pittsburgh Foundation

GAIN Evaluation of Los Angeles Job-First

GE Government Entity

GIT GPRA Implementation Team

GPCIs Geographic Practice Cost Indexes

GPRA Government Performance and Results Act

TOP OF DOCUMENT

HAACP Survey of Current Hazard Assessment Quality Control Points

HBCUs Historically Black Colleges and Universities

HCBS Home Care and Community-Based Service

HCD Human Capital Development

HCFA Health Care Finance Administration

HCPPs Health Care Prepayment Plans

HHA Health-Health Analysis

HHS (Department of) Health and Human Services

HIV Human Immunovirus

HRSA Health Resources and Services Administration

HSHC Healthy Schools, Healthy Communities Program

HSEES Hazardous Substance Emergency Events Surveillance
 

IAP Immunization Action Plan

ICBG International Cooperative Biodiversity Group

ICD Institute, Centers, and Divisions

IHS Indian Health Service

IIS Immunization Information Systems

IRP Intermural Research Program

IRPPC Intermural Research Program Planning Committee

ISO Intermediary Service Organization

I/T/U Indian Tribal Urban

TOP OF DOCUMENT

JOBS Job Opportunity and Basic Skills

JOLI Job Opportunity for Low-Income Individuals
 

KDA Knowledge, Development, and Application
 

LAAM Levo-Alpha-Acetylmethadol- Hydrochloride

LBP Limited Benefit Plan

LEAP Learning, Earning, and Parenting

LFA Labor Force Attachment

LIHEAP Low-Income Home Energy Assistance Program

LRDs Living Related Donations
 

MAF Medical Assistance Facility

MCOs Managed Care Organizations

MEDTEP Medical Treatment Effectiveness Program

MFIP Minnesota Family Investment Program

MFS Medicare Fee Schedule

MNT Medical Nutrition Therapy

MQSA Mammography Quality Standards Act

TOP OF DOCUMENT

NBCCEDP National Breast and Cervical Cancer Early Detection Program

NCCAN National Center for Child Abuse and Neglect

NCHS National Center for Health Statistics

NCI National Cancer Institute

NETSS National Electronic Telecommunications System for Surveillance

NEXT Nationwide Evaluation of X-Ray Trends

NHLBI National Heart, Lung, and Blood Institute

NHSC National Health Service Corps

NHIS National Health Interview Survey

NIGMS National Institute of General Medical Sciences

NIH National Institutes of Health

NIMH National Institute of Mental Health

NIS National Immunization Survey

NLM National Library of Medicine

NLFSI North Lawndale Family Support Initiative

NLTN National Laboratory Training Network

NTIES National Treatment Improvement Evaluation Study

NTIS National Technical Information Service

NRSA National Research Service Award

NSF National Science Foundation

TOP OF DOCUMENT

ODE Office of Device Evaluation

OIG Office of Inspector General

OPHS Office of Public Health and Science

OPH-SOPER Office of Public Health, Staff Office of Planning, Evaluation, and Research

OPPE Office of Program Planning and Evaluation

ORT Operation Restore Trust

OSP Office of Strategic Planning
 

PACE Program of All-Inclusive Care for Elderly Demonstration

PARE Physical Abuse and Neglect Reduction Effort

PAS Personal Assistance Services

PDQ Physician Data Query

PDUFA Prescription Drug User Fee Act

PET Persistent Effect of Treatment Studies

PFP/SSF Pay for Performance/Self-Sufficiency First

PHLIS Public Health Laboratory Information System

PHS Public Health Service

PIC Policy Information Center

PMA Premarket Approval

PMI Prevention Marketing Initiative

PORTS Publication of the Patient Outcome Research Teams

PPGs Public Health Performance Partnership Grants

PPP Parent Partner Programs

PRWORA Personal Responsibility and Work Opportunity Reconciliation Act

PSOs Provider-Sponsored Organizations

PUD Peptic Ulcer Disease

TOP OF DOCUMENT

QI Quality Indicators

R&D Research and Development

RCM Research Centers in Minority Institutions

RDCRD Rare Disease Clinical Research Database

RDUs Relative Value Units

RFA Request for Applications

RPRC Regional Primate Research Centers

RTC Regional Treatment Center

RRM Resource Requirement Methodology
 

SIPP Survey of Income and Program Participation

SPO Sponsored Program Office

STC State Training Cooridinators

STD Sexually Transmitted Disease

TOP OF DOCUMENT

TA Technical Assistance

TANF Temporary Assistance for Needy Families

TB Tuberculosis

TBI/SCI Traumatic Brain Injury and Spinal Cord Injury

TEEM Training, Education, Employment, and Management

TEHIP Toxicology and Environmental Health Information Program

TMRC Technical Merit Review Committee
 

USDA United States Department of Agriculture
 

WRP Welfare Restructuring Project
 

YETI Youth Employment and Training Initiative

YRBS Youth Risk Behavior Study