Performance Improvement 1996

Publication Date

Jan 31, 1996

Foreword

The Department of Health and Human Services (HHS) promotes and protects the health and well-being of all Americans and provides world leadership in biomedical and public health sciences. It does so through a diverse array of programs in basic and applied science, public health, income support, child and adolescent development, the financing of health and social services, and regulation.

Healthy individuals, families, and communities are the foundation of our nation's current and future security and prosperity. The success of this Department's programs should be measured against the objective of steady, broad-based improvements in the health of the people and communities of this country. Individual, family, and community health should be measured against a standard of well-being that includes the capacity and resources to enjoy a fulfilling and productive life. The Department's programs are in a continuous cycle of renewal and reinvention in response to changing internal and external assessments of what is working well and what needs to be improved, as well as in response to emerging needs, priorities, and opportunities. We welcome change and renewal, guided by our core commitment to be vigilant in safeguarding the nation's health and welfare and to administer with compassion and integrity the laws and programs entrusted to the Department.

Evaluations are an important source of knowledge for the process of renewal and re-invention. They provide knowledge for planning new directions and for adapting programs to meet new demands. HHS evaluations are designed to determine whether our programs and services achieve their intended results and how they can be improved. Evaluation is one of the most important tools we use to measure the success of programs in reaching and helping our customers.

Recent HHS evaluations, presented in this report, have been important in planning, budgeting, and developing legislation. For example, in 1995, the Health Care Financing Administration completed an evaluation of the Medicare Participating Heart Bypass Centers' demonstration of new approaches to cost containment. The study found negotiated package pricing helped reduce costs and improve coordination of services while maintaining quality of care. Based on the success of that demonstration, the Administration has requested new legislative authority to implement negotiated package pricing under the regular Medicare program, which should result in savings of more than $38 million annually.

Evaluation activities are also playing a major role in HHS's preparations for implementing the Government Performance and Results Act (GPRA). The Health Resources and Services Administration (HRSA) has used evaluation funds to design a system of performance measurement for assessing HRSA's program activities and outcomes. The performance measurement framework developed in this project can be used by other Federal agencies as they develop their planning strategies to implement GPRA.

Future HHS evaluations will place even greater emphasis on examining the impact of transformations in health and human services, the development of performance measures, and overall program performance improvement. A major collaboration involving Substance Abuse and Mental Health Services Administration (SAMHSA), the Centers for Disease Control and Prevention (CDC), and Office of the Secretary has developed health outcome-based performance measures for assessing the success of public health programs managed by CDC and SAMHSA. The effort involved a stakeholder outreach effort that engaged the participation of more than 1,400 individuals representing consumers; providers; State, local, and tribal officials; and program experts nationwide. A panel of technical experts at the National Academy of Sciences has refined the output of these consultations into proposed performance measures. The lessons learned in this process will underpin multiple efforts in the Department and at the State, local, and tribal level to manage programs on the basis of outcomes.

In keeping with the HHS commitment to produce quality and useful evaluations, the Department is pleased to present Performance Improvement 1996: Evaluation Activities of the U.S. Department of Health and Human Services--the first annual report documenting comprehensive evaluation efforts across all HHS agencies and offices. The report has three audiences: decisionmakers who need information on program results across the broad spectrum of health and human service activities; program managers who want to know how other service programs are operating and how performance can be improved; and the community of researchers, advocates, and practitioners who will use the program information and evaluation tools.

The report contains information on projects completed in fiscal 1995. It also provides information on HHS agency and office evaluations in progress, and future directions for evaluation.

Program Performance 1996 is organized into three chapters. Chapter I describes the organization of HHS evaluations--its activities, funding, planning and management, and future directions. Chapter II highlights the results from 11 fiscal 1995 evaluations selected for their potential application by the health and human services community. Chapter III presents the evaluation activities of the 11 HHS agencies and the Office of the Secretary, including information on their evaluation programs, evaluations completed in fiscal 1995, evaluations in progress, and future directions. A complete inventory of the 148 HHS evaluation projects completed in fiscal 1995 is provided in appendix A, and the HHS agency projects currently in progress are listed in appendix B.

We hope that you will find this report useful and informative.

Donna E. Shalala
Secretary
U.S. Department of Health and Human Services

David F. Garrison
Principal Deputy Assistant Secretary for Planning and Evaluation

Senior Editorial Advisors

The following individuals provided editorial guidance in developing Performance Improvement 1996:

Heather Becker, Ph.D.
Professor, School of Nursing
University of Texas
Austin, Texas

Eleanor Chelimsky, Ph.D.
President, American Evaluation Association
Washington, D.C.

Gary Cox, Ph.D.
Alcohol and Drug Abuse Institute
University of Washington
Seattle, Washington

Michael Hendricks, Ph.D.
MH Associates
Bethesda, Maryland

Holly Korda, Ph.D.
Evaluation Consultant
Boston, Massachusetts

John Kralewski, Ph.D.
Director, Institute for Health Services Research
School of Public Health
University of Minnesota
Minneapolis, Minnesota

Mark Krushat, Ph.D.
Office of Inspector General
Department of Health and Human Services
Washington, D.C.

Lynne Lackey, Ph.D.
Professor, Department of Health Sciences
Towson State University
Towson, Maryland

Kenneth McLeroy, Ph.D.
Health Promotion Sciences Department
University of Oklahoma
Oklahoma City, Oklahoma

Donna Mertens, Ph.D.
Professor
Gallaudet University
Washington, D.C.

Ira Raskin, Ph.D.
Health Policy Consultant
EEI Communications
Alexandria, Virginia

Mary Ann Scheirer, Ph.D.
Evaluation Consultant
Washington, D.C.

Acknowledgments

Performance Improvement 1996: Evaluation Activities of the U.S. Department of Health and Human Services describes the continuous efforts of the various HHS agencies to examine service and research programs for the efficiency of their operations and their effectiveness in achieving objectives. The planning, development, and coordination of those evaluations is largely the responsibility of the following HHS planning and evaluation offices:

Administration for Children and Families
Office of Policy and Planning
Howard Rolston, Director

Administration on Aging
Office of Program Operations and Development
Edwin Walker, Director

Agency for Health Care Policy and Research
Office of Planning and Evaluation
Phyllis Zucker, Director

Centers for Disease Control and Prevention
Office of Program Planning and Evaluation
Jeff Harris, Director

Food and Drug Administration
Office of Planning and Evaluation
Paul L. Coppinger, Associate Commissioner

Health Care Financing Administration
Office of Research and Demonstrations
Thomas Gustafson, Deputy Director

Health Resources and Services Administration
Office of Planning, Evaluation, and Legislation
Ronald H. Carlson, Associate Administrator

Indian Health Service
Office of Planning, Evaluation, and Legislation
Leo Nolan, Director

National Institutes of Health
Office of Science Policy and Technology Transfer
Lana Skirboll, Associate Director

Office of the Assistant Secretary for Planning and Evaluation
Office of Program Systems
Susanne A. Stoiber, Deputy Assistant Secretary

Office of Public Health and Science
Carol Roddy, Senior Program Advisor

Substance Abuse and Mental Health Services Administration
Office of Policy and Program Coordination
Frank Sullivan, Director

Preparation of this report and the noted appendixes was coordinated by staff in the Office of the Assistant Secretary for Planning and Evaluation, under the supervision of Ms. Susanne A. Stoiber, Deputy Assistant Secretary for Program Systems. Staff responsible include Mike Herrell, Paul Johnson, Colleen Monaghan, Tricia Quiller, Carolyn Solomon, and Joan Lee Turek. Development and production of the report was supported, under contract, by the services of Deborah Burke and the staff of EEI Communications.

The following persons from HHS agencies contributed to the writing, collection of information on evaluation activities, and review of the report:

Administration for Children and Families
James V. Dolson
Helen Howerton

Administration on Aging
Saadia Greenberg
Harry Posman

Agency for Health Care Policy and Research
Irma Arispe, Ph.D.

Centers for Disease Control and Prevention
Wilma Johnson
Nancy Cheal

Food and Drug Administration
Mary Gamunev

Health Care Financing Administration
William Saunders
Sydney Galloway

Health Resources and Services Administration
Anabel Crane
Casandra Lyles
Susan Hammersten

Indian Health Service
Frank Marion
Sarah Crazythunder

National Institutes of Health
John Uzzell
Bonnie Kalberer
Anita Harris

Office of Public Health and Science
Carol Roddy

Substance Abuse and Mental Health Services Administration
Anna Marsh, Ph.D.
Andrew Freeman

Executive Summary

Performance Improvement 1996 is the first in a series of annual reports on evaluation activities of the U.S. Department of Health and Human Services (HHS). As a report to Congress, it summarizes the findings of HHS evaluations completed in fiscal 1995. In that year, HHS agencies produced 148 evaluation reports and supported more than 200 evaluation projects in progress.

In general, the report is intended for three audiences: decisionmakers who need information on program results across the broad spectrum of health and human services; program managers who need information on how they can make improvements in program operations and outcomes; and the health and human services community, which can benefit by applying the knowledge and lessons learned from HHS evaluations.

In the Foreword, Secretary Donna Shalala and Principal Deputy Assistant Secretary for Planning and Evaluation David Garrison stress that evaluation is the primary source of knowledge for improving the performance of HHS programs, which is the theme of this report. Every year, program managers are engaged in efforts to determine whether programs and services reach intended populations or communities and achieve desired results. The information obtained from HHS evaluations is useful for planning new directions and adapting programs to meet new demands.

Chapter I

Chapter I provides an overview of evaluation at HHS. Described are the variety of evaluation activities; sources of funding; and evaluation management--including planning procedures, project management, quality assurance, dissemination, and effective use of results. The last section of the chapter discusses future directions for HHS evaluation. First, in coming years, HHS will examine the transformations now taking place in health and human services. Second, the transformations underscore the need for HHS to play a leadership role in developing and applying performance measures. Third, HHS agencies remain committed to seeking information to continuously improve the operation of current programs.

Chapter II

Chapter II highlights 11 HHS evaluations completed during fiscal 1995 and selected by the panel of Senior Editorial Advisors as evaluations considered as particularly useful to the public health and human services community.

Strengthening Families and Neighborhoods: A Community-Centered Approach

This Administration for Children and Families (ACF) report describes the design, implementation, and evaluation of the Iowa Patch project, which was undertaken to improve the quality, accessibility, and accountability of social services to children, youth, and families in a small "patch," or geographic area, of Cedar Rapids, Iowa.

Youth With Runaway, Throwaway, and Homeless Experiences: Prevalence, Drug Use, and Other At-Risk Behaviors

This report, also produced by ACF, describes the most detailed and generalizable study undertaken to date on runaway, throwaway, and homeless youth in the United States, and their associations with high-risk behaviors such as substance abuse, suicide attempts, unsafe sexual behavior, and criminal activities, both inside and outside the home.

Real People, Real Problems: An Evaluation of the Long-Term Care Ombudsman Programs of the Older Americans Act

This Administration on Aging report describes a 1-year project to evaluate long-term care ombudsman programs, which investigate complaints made by or on behalf of residents of nursing facilities and board and care homes.

Understanding and Choosing Clinical Performance Measures for Quality Improvement: Development of a Typology

Increasingly, private purchasers of health care, consumer groups, Federal and State agencies, and health care plans are searching for methods to compare clinical performance among providers and health care systems. This Agency for Health Care Policy and Research project develops a framework for evaluating the structure and content of measures currently used to assess clinical quality so that users can select measures that will help guide choices, ensure accountability, provide data for quality improvement, and track changes in the health care utilization and quality.

The Effect of Mandated Managed Care for Medicaid Populations on the Practice of Public Health: The Example of Childhood Lead Poisoning Prevention

This Centers for Disease Control and Prevention study examined the likely effects of managed care reforms on the delivery of childhood lead poisoning prevention programs. The results suggested that as managed care increases, appropriate childhood lead screening will decrease, use of public health laboratories for testing will decrease, and followup programs will be performed outside the traditional health department.

Medicare Participating Heart Bypass Center Demonstration: Evaluation Design

This negotiated bundled payment demonstration was implemented in May 1991 by the Health Care Financing Administration (HCFA) as one of many cost-containment demonstrations launched in response to increases in Federal spending under Medicare. The study found that both Medicare and hospitals could benefit from this type of payment arrangement through reduced costs and better coordination of care without sacrificing quality of care.

Monitoring the Impact of Medicare Physician Payment Reform on Utilization and Access

This report to Congress prepared by HCFA examined the impact of legislation that reformed Medicare payment methodology to physicians. As Congress intended, the legislation shifted utilization away from surgical procedures and toward medical evaluation and management. In addition, the study showed that the legislation did not create new barriers to care for vulnerable patient groups. The report documents the strong racial and income disparities in utilization.

Performance Indicators for Government Performance and Results Act: Initial Assessment of Health Resources and Services Administration Programs

Evaluation of Diabetes Services Provided by IHS Model Diabetes Program: Final Report

This study evaluates the utility of a model diabetes demonstration program created by the Indian Health Service in 1979. The critical element of the model program was a team of health professionals who augmented primary care by providing a combination of education, outreach, service delivery, and referrals.

Fetal Alcohol Syndrome: Diagnosis, Epidemiology, Prevention, and Treatment

A committee of experts was convened by the National Academy of Sciences, Institute of Medicine, to evaluate published studies on fetal alcohol syndrome and related disorders. The evaluation synthesis was requested by the National Institute on Alcohol Abuse and Alcoholism at the National Institutes of Health.

Consequences of Whistleblowing for the Whistleblower in Misconduct of Science

The Office of Research Integrity (ORI) in the Office of Public Health and Science conducted a mail questionnaire survey of individuals who had made allegations of scientific misconduct to ORI to better understand the personal consequences of such actions.

Chapter III

Chapter III provides an overview of the evaluation activities of the HHS agencies, including information on the evaluation program, a summary of evaluations completed during fiscal 1995 and evaluations in progress, and a discussion of future directions for agency evaluations.

Administration for Children and Families (ACF)

ACF administers a broad range of entitlement and discretionary programs. The objectives of its evaluations are to provide information on the program design and operations; to test service delivery approaches; to conduct policy analyses; and to disseminate and apply evaluation results. In fiscal 1995, evaluations were completed on welfare reform demonstrations; welfare dependency reduction; foster care placement programs; runaway/homeless youth; family strengthening; child maltreatment; and low-income energy assistance. Major evaluations in progress include welfare-to-work programs; responsible fatherhood initiatives; teen pregnancy; intergenerational dependency; the Head Start Program; foster care placement; and women and infant nurturing services.

Administration on Aging (AoA)

AoA supports studies that provide information on the implementation of the Older Americans Act of 1992 (OAA). The major evaluation completed in fiscal 1995 examined the effectiveness of State long-term care ombudsman programs. Results documented the public purpose served by these programs and endorsed the continuation of their mandate. A second AoA study developed the framework and design for two studies authorized by OAA to examine the quality of care in board and care facilities and the quality of home care services for older and disabled individuals. AoA is currently evaluating the Elderly Nutrition Program.

Agency for Health Care Policy and Research (AHCPR)

The goals of the AHCPR evaluation program are to assess agency effectiveness in meeting major and long-term priorities; to obtain information to respond to critical agency and departmental concerns; and to conduct internal evaluations to improve the efficacy of key program areas. Evaluation projects completed in fiscal 1995 focus on clinical practice guidelines; clinical performance measures; and consumer assessments of health care services. In progress are evaluation activities to generate information to improve the health care system, such as surveys to collect data on consumers' satisfaction with access to and use of health services. AHCPR also is developing methods to disseminate this information for improved consumer choice.

Centers for Disease Control and Prevention (CDC)

CDC places high priority on evaluations seeking to answer policy, program, and strategic planning questions related to its mission. The major evaluations in fiscal 1995 include data policy/surveillance studies such as the injury risk factor surveillance system and the influenza vaccination coverage. Program evaluations include the Medical Examiner/Coroner Information Sharing Program, the Fatality and Assessment and Control Program, and the Lead Poisoning Prevention Program. Evaluations of educational interventions were completed in the areas of diabetes and adolescent school health. Projects in progress are evaluating data management and reporting systems; surveillance systems for drug-resistant Streptococcus pneumoniae; occupational health hazards; epidemiology training; and dengue hemorrhagic fever prevention and control in San Juan, Puerto Rico.

Food and Drug Administration (FDA)

Evaluation at FDA is largely a line management responsibility. Projects focus on performance management, customer participation, and rigorous rulemaking. In fiscal 1995, FDA identified performance measures jointly with industry customers as part of the drug and biologic review processes necessary for implementation of the Prescription Drug User Fee Act. In addition, FDA completed an assessment of the Mammography Quality Standards Act of 1992. This program is FDA's second major user fee, performance-oriented, Government Performance and Results Act-style initiative. During fiscal 1995, FDA program managers were evaluating their performance measures in light of GPRA standards.

Health Care Financing Administration (HCFA)

The Office of Research and Demonstrations performs and supports HCFA evaluations to develop new health care financing policies and provide information on the impact of HCFA's programs. In fiscal 1995, HCFA completed evaluations in four areas: monitoring the impact of Medicare physician payment reform on utilization and access; the Medicare participating Heart Bypass Center demonstration evaluation; the Essential Access Community Hospital/Rural Primary Care Hospital Program; and the Rural Health Care Transition Grant Program. The major evaluations in progress include examinations of Medicaid 1915(b) waivers; Community Support Living Arrangements; and evaluations of four demonstration projects: Medicare Cataract Surgery Alternate Payment, Medicare Case Management, Medicaid Uninsured, and Medicare Alzheimer's Disease.

Health Resources and Services Administration (HRSA)

The objectives of the HRSA evaluation program are to enhance strategic planning, budget decisions, and legislative planning, and to improve program management. In fiscal 1995, HRSA reviewed its performance measurement capacity for all line programs and bureaus. Completed evaluations included studies in the following areas: services for prevention and treatment of HIV/AIDS; the cost-effectiveness of the Community Health Centers Program; and grants to foster a permanent infrastructure for child and adolescent injury prevention. Ongoing evaluations include projects to enhance performance measurement capacity; the J-1 visa program; the National Health Service Corps; the future of primary care; the Healthy Start Program; and the role of telemedicine in improving the availability and quality of care to underserved populations.

Indian Health Service (IHS)

The IHS evaluation program serves the Agency's program and policy objectives, developed in consultation with the tribal communities. IHS's major evaluations for fiscal 1995 concentrated on family violence in Native American communities; the IHS Model Diabetes Program; the IHS Mid-Level Health Providers; and child abuse and neglect in American Indian and Alaska Native communities. Projects in progress include the development and implementation of the IHS Strategic Planning Model; an analysis of underreporting of American Indian and Alaska Native deceased persons on State death certificates; mental health service delivery models for urban Native Americans; the IHS Adolescent Regional Treatment Centers; and the IHS substance abuse treatment programs for American Indian/Alaska Native women.

National Institutes of Health (NIH)

Evaluation is an integral part of the role of NIH in supporting biomedical research, training, and public education. In fiscal 1995, NIH's evaluations focused on fetal alcohol syndrome; nursing research training; an assessment of the National Cancer Program; measuring social inequalities in health; evaluating outreach activities of the National Library of Medicine; and public and private funding for basic bioengineering research. The major evaluations in progress are the National Research Service Award Research Training Program; the Minority Access to Research Careers Research Training Program; an assessment of NIH implementation of Section 491 of the Public Health Service Act mandating a program of protection of research subjects; the utilization and regulation of genetic tests; a methodology study to assess the impact of National Heart, Lung, and Blood Institute research; the National Cancer Institute's Physician Data Query Program; an assessment of the facilities and resources for laboratory animals; and a survey of academic research equipment.

Office of the Assistant Secretary for Planning and Evaluation (ASPE)

ASPE functions as principal advisor to the Secretary on policy development, conducting evaluations and policy research studies. It is also responsible for HHS-wide coordination of legislative, planning, and evaluation activities. During fiscal 1995, ASPE's evaluations looked at adolescent sexuality and parenthood; privacy of private-sector health records; substance abuse problems of individuals served by HHS programs; and transition to work for welfare recipients. The major evaluations in progress include a national study of assisted living; an analysis of the effect of regulations on the quality of care provided in board and care homes; the costs of domestic violence to the health care system; family preservation programs; health care technology assessment; physician information use; future long-term care needs; managed care and people with disabilities; the JOBS Program; performance measurement for the State service delivery systems; managed care and the pharmaceutical marketplace; programs to promote responsible fathering; and the cost and quality of subacute care.

Office of Public Health and Science (OPHS)

OPHS provides advice, policy and program coordination, and leadership in the implementation, management, and development of HHS public health and science activities. Several major OPHS-supported evaluations were completed in fiscal 1995 on the following: blood supply safety; HHS standards for methadone programs; clinical trials for the experimental drug fialuridine and its parent drug fiacitabine; Healthy People 2000 baseline data for disease prevention activities; measuring State and local public health expenditures; HIV/AIDS prevention within minority communities; breast and reproductive cancer prevention efforts at community and migrant health centers; hepatitis B virus education and outreach efforts; competitive health plans and academic medical centers; and the supply, training, and distribution of primary care providers. Evaluations and policy studies in progress include dietary supplement labels; framework for Healthy People 2010; school health programs; public health infrastructure; cost-effectiveness methodology; and public health performance measures.

Substance Abuse and Mental Health Services Administration (SAMHSA)

SAMHSA is committed to evaluating its programs and grant projects to assess the effectiveness of prevention, treatment, and rehabilitation approaches and systems of care. In fiscal 1995, SAMHSA completed evaluations on mental health services for children and adolescents with serious emotional disabilities, including the effectiveness of case management services and the impact of multisystemic family preservation therapy on family functioning and delinquent adolescent behavior. Ongoing evaluations include evaluations of SAMHSA's demonstration programs: the Job Corps program of enriched substance abuse treatment for adolescents; the services integration experiment for chronically mentally ill homeless persons; and the National Treatment Improvement Evaluation Study. Evaluations are also under way to examine the impact of managed care on access, cost, and quality of substance abuse treatment.

Appendixes

A complete inventory of the 148 HHS evaluations completed in fiscal 1995 is provided in Appendix A, and HHS agency projects currently in progress are listed in Appendix B.

Chapter I. The Evaluation Program in the U.S. Department of Health and Human Services

The mission of the U.S. Department of Health and Human Services (HHS) is to protect and promote the health and social and economic well-being of all Americans and, in particular, those least able to help themselves--children, the elderly, persons with disabilities, and the disadvantaged--by helping them and their families develop and maintain healthful, productive, and independent lives. Accomplishing this mission through program activities and evaluating their performance is the task of the following HHS agencies and offices:

Administration for Children and Families (ACF).
Administration on Aging (AoA).
Agency for Health Care Policy and Research (AHCPR).
Agency for Toxic Substances and Disease Registry (ATSDR).
Centers for Disease Control and Prevention (CDC).
Food and Drug Administration (FDA).
Health Care Financing Administration (HCFA).
Health Resources and Services Administration (HRSA).
Indian Health Service (IHS).
National Institutes of Health (NIH).
Office of the Secretary (OS).
Substance Abuse and Mental Health Services Administration (SAMHSA).

The Assistant Secretary for Planning and Evaluation (ASPE), located in OS, coordinates evaluation activities throughout HHS.

Evaluation plays an integral role in carrying out the HHS mission by assessing various aspects of program performance of the HHS agencies and by identifying means of improving that performance. The HHS evaluation function has three goals:

To provide information on HHS programs that helps government officials and members of Congress make decisions related to program, policy, budget, and strategic planning.
To help HHS managers improve program operations and performance.
To disseminate HHS evaluations--study results and methodological tools--that are useful to the larger health and human services community of State and local health and human services officials, researchers, advocates, and practitioners for improving the performance of their programs.

The last goal is very important to HHS. The Department believes it has an important obligation to foster the development of new knowledge about the effectiveness of health and human services programs, interventions, and evaluation tools for use by the larger health and human services community. Although the findings and recommendations of HHS evaluations are usually first used by the Administration and Congress, they can also be applied by others in the research and practice communities to improve the performance of programs at the State and community levels. The purpose of this report is to widely disseminate information about recent HHS evaluations and to make sure the potential for wider application is realized.

This chapter describes the organization and operation of evaluation at HHS. It first provides an overview of the kinds of evaluation activities supported by HHS agencies and then describes the funding mechanisms used to support them. It details HHS evaluation management, including planning procedures, project management, quality assurance, dissemination of reports, and effective uses of evaluation results. The chapter concludes with a discussion of future directions for evaluation at HHS.

HHS Evaluation Activities

The evaluation activities sponsored by HHS and described in this report assess program performance (efficiency, effectiveness, responsiveness), analyze results on the basis of those assessments, and use the resulting information in policymaking and program management. These activities are diverse and include the full spectrum of evaluation methodologies developed over the last quarter century. The classification of HHS evaluation activities presented in figure I-1 summarizes that diversity.

HHS evaluation projects typically fall into a combination of these categories. For example, comprehensive HHS evaluations generally examine both process and outcome or impact. Knowing only whether goals and objectives are achieved is insufficient without also knowing how well the program was implemented and whether goals and objectives were appropriate in the first place. Similarly, evaluation feasibility and design activities generally represent the crucial first phase of major HHS process and outcome/impact evaluations.

Evaluation Funding

Evaluation activities of the various HHS agencies are largely supported through two funding mechanisms: (1) direct use of programs funds and (2) special use of legislative set-aside authorities for evaluation. The first is a common mechanism by which programs managers have discretionary authority to use appropriated program funds to support contracts that will design, implement, and analyze evaluation data. In some cases, a program's legislative authority calls for a special mandated evaluation, and the program funds are used directly to support the evaluation.

Figure I-1. Range of HHS Evaluation Activities

Evaluation projects

Outcome evaluations: assessing the immediate or intermediate effects of a program with respect to the stated goals or objectives.
Impact evaluations: assessing the broader results, intended or unintended, of a program on populations or institutions involved.
Implementation or process evaluations: assessing the nature of program inputs and outputs and their relationship to stated goals and objectives.
Policy assessments: examining health policies with respect to their development, implementation, or the impact on public health or program activity.
Cost-benefit or cost-effectiveness analyses: developing methodology and its application to assess the relationship of program results to program costs (direct and indirect), often in comparison with alternative programs.
Survey data analyses: evaluating the results of HHS programs or policies by conducting or analyzing data obtained from surveys.
Performance measurement and data systems: identifying and testing the validity and reliability of process, output, and outcome indicators to measure the performance of programs and develop data systems supporting implementation of the Government Performance and Results Act (GPRA).
Simulations and models: using computer simulations and modeling techniques to analyze the impact of policy changes on service delivery systems and beneficiaries.
Management studies: examining the effectiveness or efficiency of the administration or operation of HHS programs and offices.
Evaluation syntheses: integrating the results from multiple independent evaluation studies within a defined program or policy area in a fashion that improves the accessibility and application of those results.

Methodology projects

Evaluation feasibility studies: assessing the clarity and importance of program goals and objectives, the consensus of program stakeholders on the potential use of evaluation information, and the availability of relevant performance data before committing to a full-scale program evaluation.
Evaluation design projects: procuring assistance in the development of an evaluation design, measurement tools, or analytic models in preparation for full implementation of an evaluation.
Instrument development projects: developing evaluation instruments (design, measurement, or analytic) for a specific HHS program or for general use by the health and human services community.

Evaluation support activities

Evaluation technical assistance: providing assistance to HHS program managers or office directors on any aspect of evaluation planning, project design-implementation-analysis, or use of results.
Evaluation dissemination: identifying target audiences and mechanisms to inform program constituencies and evaluation stakeholders about evaluation results.
Evaluation training/conferences: maintaining the professional skills and expertise of evaluation staff through training opportunities, as well as promoting the dissemination of HHS evaluations through conference symposia.

The second mechanism for evaluation funding is legislative set- aside authorities permitting the Secretary of HHS to use a proportion of overall program funds for evaluation purposes. The largest of such set-aside authorities is one established for evaluations conducted by several agencies of the U.S Public Health Service (AHCPR, CDC, HRSA, NIH, and SAMHSA), ASPE, and the Office of Public Health and Science (OPHS) in the Office of the Secretary. It is called the 1 percent evaluation set-aside legislative authority, provided in Section 241 of the Public Health Service (PHS) Act. This authority was established in 1970 when Congress amended the Act to permit the HHS Secretary to use up to 1 percent of appropriated funds to evaluate authorized programs. Section 241 limits the base from which 1 percent of appropriated funds can be reserved for evaluations of programs authorized by the PHS Act. This limitation excludes all funds appropriated for FDA,¹ IHS,¹ and certain other programs that are managed by PHS agencies but not authorized by the Act (e.g., HRSA's Maternal and Child Health Block Grant and CDC's National Institute for Occupational Safety and Health).

In fiscal 1995, HHS invested more than $41 million in set-aside evaluation funds to conduct evaluation activities. These resources amount to approximately two-tenths of 1 percent of the total appropriated for programs authorized by the Act ($18 billion). An additional $46 million in set-aside funds was earmarked by Congress for use by CDC's National Center for Health Statistics and AHCPR in those agency's appropriations.²

In fiscal 1996, HHS estimates it will use approximately $33.5 million in the PHS evaluation set-aside funds to continue current evaluation activities and to initiate new evaluation projects. This amount is somewhat lower than the comparable fiscal 1995 figure. However, $100.2 million in set-aside funds was earmarked by Congress for CDC and AHCPR, as stated. This figure represents a substantial increase over past years. Table I-1 provides a breakdown of the estimates for fiscal 1996 and the actual usage for fiscal 1995 by PHS agencies and the Office of the Secretary.

Evaluation Management

The management of HHS evaluations, carried out on a regular basis by HHS agencies and offices and coordinated by ASPE, involves these five basic functions:

Evaluation planning and coordination.
Project management.
Quality assurance.
Dissemination of evaluation reports.
Assurance of effective use of evaluation results.

A description of each function in general terms follows. Additional information on the individual HHS agencies, ASPE, and OPHS evaluation functions is found in chapter III.

Table I-1. Agency Use of Evaluation Set- Aside Funds,
in Thousands of Dollars
	FY 1995	FY 1996
Agency evaluation use:
AHCPR	$450	$115
CDC	2,000	2,000
HRSA	7,114	6,677
NIH	4,510	4,510
SAMHSA	1,978	996
ASPE	15,500	15,500
OPHS¹	9,525	3,852
Total use	$41,077	$33,650

1. OASH in fiscal 1995.

Evaluation Planning

HHS Agencies, ASPE, the Office of the Inspector General (OIG), and OPHS develop evaluation plans annually in concert with HHS's program planning, legislative development, and budgeting cycles. Plan development is coordinated by ASPE. Before the start of each fiscal year, evaluation guidance is issued by ASPE to signal HHS program priorities for evaluation. Typically, the priorities include evaluations of Secretarial program or policy initiatives, new programs, programs undergoing major change, programs that are candidates for reauthorization, and programs for which key budget decisions are anticipated.

Recently, emphasis has been given to evaluations that support strategic planning program goals and objectives. Congress has requested HHS to coordinate all of its research, demonstration, and evaluation programs to ensure that the results of these projects address HHS's program goals and objectives. ASPE and the Assistant Secretary for Management and Budget are now working with HHS agencies to provide Congress with annual research, demonstration, and evaluation budget plans, beginning with the fiscal 1996 President's budget, that outline each agency's research, demonstration, and evaluation priorities as related to overall HHS program goals and objectives.

Project Management

The execution of evaluation at HHS is principally decentralized-- the various HHS agencies, OIG, and ASPE are all responsible for executing annual evaluation plans, developing evaluation contracts, and disseminating and applying evaluation results. Even within agencies, while there is some oversight responsibility and execution capability in the Office of the Director or Administrator, the various subunits (centers, institutes, bureaus) conduct much of the day-to-day evaluation activity.

OIG performs independent evaluations through its Office of Evaluations and Inspections (OEI). OEI's mission is to improve HHS programs by conducting inspections that provide timely, useful, and reliable information and advice to decisionmakers. This information (findings of deficiencies/vulnerabilities and recommendations for corrective action) is usually disseminated through inspection reports issued by the Inspector General. Since its inception in April 1985, OEI has produced more than 600 inspection reports. A summary of individual inspection reports and other OIG reports can be viewed on the Internet (http://www.sbaonline. sba.gov/ignet). OEI also provides technical assistance to HHS agencies in conducting their evaluations. A recent example is their joint work with AoA to help train, provide technical assistance, and develop an action plan to address weaknesses in their stewardship of the Older Americans Act.

Quality Assurance

Most evaluation projects are developed at the program level, and the initial review is conducted by a committee of agency-level policy and planning staff members. Before a project is approved, it is reviewed for technical quality, generally by a second staff committee that is skilled in evaluation methodology. Technical review committees follow a set of criteria for quality evaluation practice established by each agency. Some HHS agencies also have external evaluation review committees composed of evaluation researchers and policy experts from universities and research centers. More details on the quality assurance procedures for the various agencies, ASPE, and OPHS are presented in chapter III.

Dissemination of Evaluation Reports

Maintaining and sharing information on the various projects conducted by HHS agencies, ASPE, and OPHS is an important component of evaluation management. Project information is continuously reported to the HHS Policy Information Center (PIC)- -the departmental evaluation database and library maintained by ASPE. As an information database and library resource, the PIC contains nearly 6,000 completed, ongoing, and planned evaluation and policy research studies conducted by HHS, as well as key studies completed outside HHS by the U.S. General Accounting Office (GAO) and private foundations.

Typically, the results of HHS evaluations are disseminated through targeted distribution of final reports, articles in refereed journals, and presentations at professional meetings and conferences. Although the individual HHS agencies have primary responsibility for disseminating results, there is a department- wide effort under way to expand dissemination to the larger research and practice communities through centralized computer communications and publications. First, abstracts of all studies maintained in the PIC database are now accessible through HHS's World Wide Web server (http://www.os. dhhs.gov) on the Internet. Once into the HHS Home Page, one can click on "Policy Information" and then on "Research and Data Provided by HHS" to gain access to the PIC database. It is possible to obtain information on reports available from completed projects and the name and telephone number of an HHS official responsible for the project.

Second, HHS is widely distributing copies of its first annual report on evaluation (Performance Improvement 1995: Evaluation Activities of the Public Health Service). The report's theme of performance improvement reflects the numerous changes and initiatives throughout HHS to increase the effectiveness and efficiency of public health programs. As the first report to Congress, it summarizes the findings of PHS evaluations completed during fiscal 1994. Of the approximately $14 billion in the fiscal 1994 budget for program activities, PHS agencies used almost $27 million to conduct evaluations useful for understanding the outcomes and improving the performance of PHS programs. In FY 1994, PHS Agencies produced 71 evaluation reports and supported more than 180 evaluation projects in progress. The report provides summaries or abstracts of these reports and contacts for further information.

In addition to providing the report to members of Congress, HHS sent copies to State and local health officials, schools of public health, and other national public health research and practice associations. A similar plan has been developed to distribute Performance Improvement 1996: Evaluation Activities of the U.S. Department of Health and Human Services, which contains information on all HHS evaluations completed and in progress during fiscal 1995. These reports are also available on the previously described HHS Home Page in three computer formats: ASCII, HTML, and PDF for downloading information.

Ensuring Effective Use of Evaluation Results

HHS is committed to ensuring that evaluations yield a high return on the investment of available program funds. In the last decade, HHS evaluations were used primarily by program managers for internal purposes of improving program operations and efficiency. In the 1990's, however, the need for more program-outcome and impact-type evaluations has increased because of fiscal pressures to accomplish more with fewer resources. The stakeholders for HHS evaluations have expanded beyond the boundaries of program management to include decisionmakers at the top levels of government, both the Administration and Congress, and health and human service researchers and practitioners at State and community levels.

To meet the needs of these expanding stakeholder groups, HHS has encouraged its agencies to give high priority to outcome/impact evaluations, especially programs that are coming up for reauthorization or are instrumental to strategic planning goals and objectives. The need for this major shift in priorities was documented by GAO in its April 1993 review of the PHS Evaluation Program, focusing on the 1 percent set-aside authority (see Publication No. GAO/PEMD-93-13). GAO recommended that HHS target more of its evaluation resources to outcome/impact evaluations that can be used by Congress and others for program planning, budgeting, and legislative action. In addition, GAO recommended that HHS initiate special projects to synthesize multiple evaluation efforts to better communicate to Congress and others the aggregate lessons learned over the years in a particular program area. Several evaluation syntheses of HHS programs were completed during fiscal 1995 and are reported in chapter III.

Future Directions in HHS Evaluation

In upcoming years, HHS agencies, ASPE, and OPHS will focus their evaluation portfolios on three principal themes: (1) the impact of transformations in health and human services, (2) the development of performance measures, and (3) overall program performance improvement.

Impact of Transformations

In December 1995, the Secretary formed a working group to develop a research strategy to examine the transformations now taking place in health and human services and the impact of those transformations on the well-being of Americans--especially the vulnerable populations that are high priority for HHS programs, such as disadvantaged or low-income children and families, the elderly, racial and ethnic minorities, and individuals with disabilities.

These transformations refer to the nationwide changes in the organization, financing, and availability of health services delivery, including the new managed care arrangements and a growing emphasis on quality of care. Managed care arrangements are affecting virtually every health program funded by HHS. For example, HCFA is granting waivers to States under Section 1115 of the Social Security Act to redesign their Medicaid programs, with most programs having a managed care component. Some of the new evaluation questions being proposed are as follows:

Has the Nation's progress toward the Healthy People 2000 Goals been facilitated or slowed by the transformations to date? Can future impacts of the changes be estimated?
Are HHS's programs of health care for vulnerable populations performing more or less effectively with respect to these changes?
What is the impact of managed care arrangements on the effectiveness of State- and community-level public health programs?

Like health services, human services programs are undergoing transformations in their organization, financing, and availability. In August 1996, new welfare reform legislation was enacted that eliminated the entitlement to cash assistance and replaced it with a fixed block grant to States, placed a 5-year time limit on benefits, imposed strict work requirements on recipients, reduced benefits and services available to legal immigrants, and greatly expanded States' authority over welfare programs. This new legislation raises important evaluation issues, particularly as policy decisions will increasingly be made at the State and local level. Critical questions include the following:

How do States organize and implement the new welfare system?
What are the effects of the legislation on the well-being of families and children?
What approaches are States taking to move families from welfare to economic dependence? How effective are these approaches?

HHS's evaluation function has an important role to fulfill in this research strategy. HHS agencies have already initiated evaluation projects that focus directly on these transformations in health and human services. The evaluations of State-specific Medicaid and welfare reform demonstration are examples. Other projects include the ongoing evaluations of the Head Start program; the evaluation of the national welfare-to-work program (Job Opportunities and Basic Skills Training) which examines the effectiveness of different approaches to moving recipients into work and the impact of the program on the well-being of children; the effectiveness and efficiency of community health centers; alternatives for health care for Native Americans; cost and quality of and access to mental health services and treatment programs for substance abusers.

The evaluations provide an excellent base on which to build an expanded array of studies related to the role of HHS as both sponsor and provider of services. Future HHS program evaluations offer an excellent opportunity to examine the effects of devolution, considering such questions as whether improvements have been made in efficiency and accountability, and to examine the impact on vulnerable populations.

Development of Performance Measurements

The transformations also underscore the need for HHS to expand its leadership role in developing and applying better performance measures regarding health and human services. Stakeholders, from Congress to community leaders, are demanding increased attention to results and the concomitant development of program outcome measures that are meaningful, quantifiable, and reliable. In addition, consensus on performance measurement--at all levels of stakeholders--is likely to be a precondition for effective data sharing across governmental levels and between governments and private sector organizations.

HHS agencies are now engaged in evaluation projects to promote the development and use of performance measures related to health and human services. Recent examples include quality assurance measures within the health care industry and scorecards to help consumers rate health and mental health services.

One of HHS's most ambitious projects to involve States, communities, and service recipients in identifying program performance measurement is called Performance Partnerships. The initiative, which has involved consultations with more than 1,400 stakeholders nationwide, will identify performance measures for program activities within SAMHSA and CDC. The measures will be used as management tools at the Federal, State, and local levels to clarify program goals and objectives and to document the performance of specific programs. This is the most comprehensive effort yet mounted to fully involve States, communities, and service recipients in identifying program measures.

HHS will also invest its evaluation resources in performance indicators to ensure implementation of GPRA. The evaluation strategies of the HHS agencies, mentioned in chapter III, include the priorities of projects that examine program objectives and develop useful measures of program outputs and outcomes. GPRA offers HHS agencies an opportunity to develop performance measurement systems that will eventually link program evaluation activities to budgeting. HHS's evaluation set-aside authority, such as the 1 percent authority for some PHS agencies, is an important resource to help program managers identify performance objectives and test the validity and reliability of indicators to measure progress.

For example, HRSA has completed a major project to assess its capacity to develop and implement a performance measurement and management system, and is conducting followup activities. HRSA's objective is to document program inputs, processes, and outputs and to analyze the link between key program elements and outcomes for the target populations and community health objectives. This investment of evaluation funds will yield a high return on GPRA's objective to have performance measurement systems in place when agency strategic planning and performance budgeting systems are scheduled to be operational in fiscal 1999. HRSA's experience in developing performance measurement for GPRA has potential as a model for other agencies.

Performance Improvement

The Department also encourages evaluations initiated by program managers for improving the performance of HHS programs, such as the use of customer surveys to measure satisfaction with program services or outputs. The evaluations are designed to ensure that program operations are efficient and effective. They are also an essential resource for HHS's Continuous Improvement Program and will be used to support the development and operation of information systems and special studies that will enable program managers to measure customer satisfaction with HHS services.

Several projects illustrate the Department's evaluation priority of continuous improvement of services. CDC is working with the States to look at the efficiency and impact of two disease- surveillance systems. HRSA will use evaluation funds to develop performance measures for grantee assessment of program outcomes in projects funded by the Ryan White Comprehensive AIDS Resources Emergency Act. NIH will evaluate the National Research Service Award training program to determine whether its objectives are being met. SAMHSA will develop performance measures to monitor the generation of new knowledge from its demonstration programs.

Notes

1. FDA programs are principally authorized by legislation other than the PHS Act, specifically the Federal Food, Drug and Cosmetic Act, and appropriated under the Agriculture, Rural Development, Food and Drug Administration and Related Agencies Appropriations Act. IHS programs are authorized under the Indian Health Care Improvement Act and the Indian Self- Determination Act, and appropriated under the Department of the Interior and Related Agencies Appropriations.

2. In the past, the 1 percent congressional earmarks for AHCPR and CDC have been used to support national health surveys. Within AHCPR these funds have supported the Medical Expenditure Panel Surveys (MEPS), formerly called the National Medical Expenditure Survey. MEPS is the Nation's only representative survey regarding the use and payment of health care services. These surveys provide the data needed to develop economic models for national and regional estimates of the impact of changes in financing, coverage, and reimbursement policy. Because of the increase in the 1 percent earmarks in fiscal 1996 appropriations, AHCPR used these funds to support studies in the areas of health care systems, cost and access, and outcomes and effectiveness in addition to MEPS. The congressional earmark of 1 percent funds to CDC supports the programs of the National Center for Health Statistics, consisting mainly of national surveys and data systems designed to monitor and evaluate the health of the American people, their use of health services, and other related issues. For fiscal 1996, CDC received an earmark of $40,063,000 from 1 percent evaluation funds, and AHCPR received an earmark of $60,124,000.

Chapter II. Highlights of Selected Evaluations Completed During Fiscal 1995

In this chapter, the U.S. Department of Health and Human Services (HHS) highlights evaluations of general interest to the public health and human services community and illustrates the diversity of HHS evaluations completed in fiscal 1995. Included are summaries of 11 evaluation projects selected by the Senior Editorial Advisors on the basis of criteria identified in appendix C and applied to 38 reports nominated by HHS agencies. These criteria are as follows:

Is the report important? Does it address a significant issue or problem for which evaluation would help confirm or change program direction, or measure program impact? Are the findings likely to be useful and generalizable?
Is the report methodologically sound? Are its concepts, designs, data collection, and analyses conducted and reported in a competent manner?
Is the report faithful to the data? Do the conclusions and recommendations logically follow from the data and analyses, and are they relevant to the questions asked?

Each summary includes a brief abstract; a description of the study, including its purpose, background, methods, findings, and use of results; the names of any publications that resulted; and the name and phone number of the person to contact for additional information.

Administration for Children and Families

Strengthening Families and Neighborhoods: A Community-Centered Approach

Highlights

This final report describes the design, implementation, and evaluation of the Iowa Patch project, which was undertaken to improve the quality, accessibility, and accountability of social services to children, youth, and families in a small "patch," or geographic area, of Cedar Rapids. By the end of the project, services had been integrated and improved, a community-based interagency service team had been developed, and agency turfism had been reduced. The patch team, including line workers, were more likely to use assessment tools as a team, to share tasks, and to support one another with greater role flexibility. The focus of services shifted from a crisis orientation to a more preventive, proactive approach.

Purpose

A 3-year demonstration grant was provided to the University of Iowa by HHS's Administration for Children and Families. Its purpose was to overcome categorical barriers that prevent the pooling and use of resources to deliver flexible services that build on the strengths of individuals, families, and neighborhoods. As defined by its advisory committee, the Iowa Patch project's goal was "to improve the quality of life of children and families by providing accessible services that are responsive to family and individual needs in the context of the consumer's local community, developing the existing resources and building on the strengths of the neighborhood."

Background

Funded from October 1991 through September 1994, the project replicated elements of the patch system of community-centered service delivery developed in the United Kingdom during the 1970's. Teams of human services workers with different levels and types of skills were placed in neighborhood offices that served about 10,000 people in Linn County, Iowa. Using a community-oriented approach to social work practice, these teams were able to provide more accessible, flexible, and holistic services based on their knowledge of the local cultural and physical environment and on the formal and informal relationships they established.

In Iowa, patch was viewed as a concept that, if appropriately replicated, could complement other approaches to system restructuring, including the Child Welfare Decategorization project that was developed to reshape the service system. Selected Iowa counties, including Linn County, were permitted to merge their share of various funding streams into one child welfare fund. From the outset, the Iowa Patch project was adopted as an initiative of the Linn County Decategorization project. The demographics of the 9,921 residents of the patch area included higher rates of poverty, unemployment, child placement, single-parent homes, and poor housing than those in the rest of Cedar Rapids or in the State. Many children and families had multiple needs, and human services agencies were heavily involved in the communities at the project's outset.

Methods

To achieve project goals, a patch process model was designed and implemented using five primary methods: (1) integrated governance involving agencies, community organizations, neighborhood residents, and service users; (2) localized service delivery; (3) a core interagency team generating new roles and methods; (4) an extended team coordinating service delivery across a wide range of other public and private organizations; and (5) a single point of entry to the service network so consumers could access help with fewer barriers.

A logic model was developed for the project evaluation that included problems endemic to service delivery in the United States; substitutionism (formal services substitute for, rather than support, informal systems of care); turfism (agencies protect their service domains); fragmentation; unresponsiveness to local conditions and consumer needs; deficit orientation (a focus on weaknesses rather than strengths of families and neighborhoods); crisis orientation; and bureaucratic rigidity.

The evaluation focused on three aspects of the project:

The geographic and demographic profiles of the patch area were taken from block-level statistics from the 1990 U.S. census.
The process and results of Patch team development were assessed through focus groups, a 1-year participant-observer study of the team, and a questionnaire completed at the beginning and end of the demonstration period. The questionnaire was also administered to a comparison group of direct service staff in another county.
Systemic impacts of the project were studied through two sets of personal interviews with senior administrators of agencies directly or indirectly involved with the project, through focus groups, and through the administration of a brief paper-and-pencil assessment with the same administrators.

Findings

The project demonstrated the feasibility of integrating services through a neighborhood-based, collocated interagency team of human service workers representing both public and private nonprofit agencies and was accomplished without a large input of new resources.

The patch approach to practice and service delivery reduced problems of fragmentation, deficit- and crisis-orientation, bureaucratic rigidity, turfism, and substitution of formal for informal systems. Researchers observed the interweaving of formal and informal supports, innovative resource mixes, and consumer and citizen involvement. More emphasis was given to prevention, early intervention, and increasing collaboration among service workers. Putting a local team in the neighborhood made human service workers more accessible, more approachable, more able to develop informal relationships with consumers, and more able to establish and work with informal helping networks.

After the grant ended, the Patch project was integrated into an initiative to develop family resource centers. The project established a strong desire to maintain and further develop the patch approach and a commitment to support it financially and administratively.

Use of Results

For State or county agencies with statutory responsibility for the protection and welfare of children, the patch process offers an innovative approach to building new partnerships. It may be appropriate for replication in other geographic areas, in other populations, and in other types of service delivery systems. The findings also have implications for institutions that educate human service workers in terms of developing curricula, field placements, and research opportunities that prepare faculty and students for integrated practice models such as patch.

Agency sponsor:

Administration on Children, Youth and Families

Federal contact:

James Dolson
202/401-0133
PIC ID: 5977

Performer organization:

University of Iowa, Iowa City, IA

Youth With Runaway, Throwaway, and Homeless Experiences: Prevalence, Drug Use, and Other At-Risk Behaviors

Highlights

This report describes the most detailed and generalizable study undertaken to date on runaway, throwaway, and homeless youth in the United States, and their associations with high-risk behaviors such as substance abuse, suicide attempts, unsafe sexual behavior, and criminal activities, both inside and outside the home. A high correlation was found between runaway, throwaway, and homeless experience and high-risk behaviors. Researchers concluded that community-based youth shelter services, including prevention, intervention, and treatment components, are critical to this vulnerable population.

Purpose

This study was undertaken in response to a congressional mandate "to support research on illicit drug use by runaway and homeless youth, the effects on such youth of drug abuse by family members, and any correlation between such use and attempts at suicide" (Public Law 100-690, Section 3511[a][6]). It is the first national study of substance use, suicide, and other at-risk behaviors among youth with runaway, throwaway, and homeless experiences. The study is also the first nationally representative survey on substance use and other at-risk behaviors among youth residing in both federally and nonfederally funded shelters. In addition, it is the first study of its kind with a multicity sample of street youth.

Background

Runaway, throwaway, and homeless youth comprise a vulnerable population at risk for substance abuse, suicide, sexual exploitation, and physical violence. Because of the need for valid, national statistics about these young people, researchers sought to determine the prevalence of such high-risk behaviors in this population, the extent of exposure to drug prevention efforts and treatment services, needs for and access to health care, the relationship between substance use by youth and their families and youth suicide attempts and other risky behaviors, and the number of available youth shelter beds and the proportion of beds occupied.

Methods

The study was conducted by the Research Triangle Institute under a contract with the Research, Demonstration, and Evaluation Branch of the Administration on Children, Youth and Families (ACYF). Four data collection components were developed. The first component was a survey conducted in the fall of 1992 of 640 youth between the ages of 12 and 21 who were residing in 23 federally and nonfederally funded youth shelters located in 17 urban areas. The sample was nationally representative.

The second component was a purposive survey of 600 youth who reported a runaway or homeless episode in the previous 12 months. These young people were found on streets or in other public places in 10 urban areas. A small substudy designed to validate self-reports of substance use also was undertaken.

The third component was a probability sample of youth in households surveyed through an interagency agreement between ACYF and the National Center for Health Statistics. Nine questions concerning runaway experiences were added to the Youth Risk Behavior Supplement of the National Health Interview Survey. In 1992 and 1993, 6,496 youth between the ages of 12 and 17 responded to in-home audiocassette interviews.

The fourth component was a mailed survey to directors of a nationally representative sample of youth shelters in 1992 and 1993. Surveys were returned by 160 shelter directors, an overall response rate of 97 percent from directors of eligible shelters.

Findings

Researchers found that many youth in the shelter and street study components reported runaway, throwaway, and homeless experiences. Single parents were raising 52 percent of the youth in the shelter component (48 percent in the street component). In the household component, 34 percent of youth did not live with both parents. In the 30 days prior to leaving home, 32 percent of youth in the shelter component (46 percent in the street component) had at least one family member who got drunk or used marijuana or other drugs.

Fifty-six percent of youths in the street component had never used a shelter. In the household component, 75 percent of youth with a runaway experience had not used either a youth or an adult shelter. An average of 10 beds were available in shelters serving only youth aged 12 to 17, while an average of 28 beds were available in shelters serving youth aged 12 to 21. On any given night, an average of 55 percent of the beds in youth shelters were occupied. Only 10 percent of youth in the shelter component and 14 percent of youth in the street component had ever been turned away from a shelter.

In the household component, lifetime substance use among youth with runaway and homeless experiences was high compared with other youth. Eighteen percent of substance users in the shelter component had been in treatment for an alcohol- or drug-related problem (24 percent in the street component). Of the young people who had ever received treatment, 44 percent of those in the shelter component (23 percent in the street component) were in treatment when data were collected.

Suicide attempts were reported by 26 percent of youth in the shelter component and 32 percent in the street component. More youth reported attempting suicide at home than away from home.

Many young people had been robbed (12 percent in the shelter component, 30 percent in the street component), assaulted (15 percent, shelter; 29 percent, street), or sexually victimized (8 percent, shelter; 14 percent, street). Sex was traded for money, food, shelter, or drugs by 7 percent of the shelter component and 21 percent of the street component. Reported crimes committed by youth in the study included theft-related activity (66 percent, shelter; 81 percent, street); assault-related activity (30 percent, shelter; 42 percent, street); and drug-related activities (32 percent, shelter; 50 percent, street). In the household component, youth with any runaway experience were three times as likely as other youth to have carried a weapon in the past 30 days and to have been in a physical fight in the past year.

In the household component, youth with any runaway experience were four times as likely as other youth to ever have had sexual intercourse. At the time of the interview, 12 percent of the females in the shelter and street components were pregnant; 41 percent of females in the shelter component (50 percent in the street component) reported having ever been pregnant.

A strong relationship was found between youth substance use and suicide attempts. Having at least one family member who got drunk or used marijuana or other drugs in the 30 days before a youth left home substantially increased the likelihood that the youth had ever attempted suicide.

Use of Results

This study should be of interest to Federal and State policymakers, community leaders, residential and outreach service providers, law enforcement officials, and other individuals concerned for the well-being of this population. Findings may be useful in shaping public policy, directing resources to this population and their families, improving the design and delivery of prevention and treatment services, strengthening community-based systems and linkages, and undertaking additional research.

Agency sponsor:

Administration on Children, Youth and Families

Federal contact:

James Griffin, Ph.D.
202/205-8138
PIC ID: 3871

Performer organization:

Research Triangle Institute, Research Triangle Park, NC

Administration on Aging

Real People, Real Problems: An Evaluation of the Long-Term Care Ombudsman Programs of the Older Americans Act

Highlights

This report describes a 1-year project to evaluate long-term care ombudsman programs, which investigate complaints made by or on behalf of residents of nursing facilities and board and care (B&C) homes. The report sets forth recommendations related to program availability, adequacy, and effectiveness for residents of long-term care facilities; compliance with Federal program mandates; adequacy of current Federal, State, and local program resources; and the feasibility of extending ombudsman services to older persons who do not reside in long-term care facilities. Overall, the report strongly supports long-term care ombudsman programs and their continuation and urges a stronger Federal Government role in providing program leadership, guidelines, and oversight.

Purpose

The purpose of this study is to assess the effectiveness of State-based long-term care ombudsman programs in providing advocacy for the health, safety, welfare, and rights of those who reside in long-term care facilities and B&C homes. Under contract with the Administration on Aging (AoA), the Institute of Medicine convened a multidisciplinary expert committee to evaluate long-term care ombudsman programs and to consider the potential impact of these programs on other target populations, in other settings, and in other circumstances. This evaluation project was mandated by the 1992 amendments to the Older Americans Act.

Background

Long-term care ombudsman programs were created more than two decades ago as a response to serious concerns about the quality of nursing home care. Currently operating in all 50 States, the District of Columbia, and Puerto Rico, these programs are funded from a combination of Federal, State, and local sources.

Over the years, the role of the ombudsman has evolved to include both individual and systemic advocacy. Individual advocacy activities are designed to ensure that ombudsman services are accessible to all residents, systematically receive and respond to resident concerns about their long-term care services, and work cooperatively with a variety of agencies to resolve problems.

Systemic advocacy activities include evaluation of laws and regulations related to long-term care services for older adults, education of the public and facility staff, dissemination of data about the ombudsman programs, and promotion of the development of citizen organizations and resident and family councils. Both roles are important, but the ombudsman programs primarily identify, investigate, and resolve individual resident complaints.

Methods

The committee developed a formative evaluation strategy to identify program issues, strengths, and weaknesses. Both qualitative and quantitative data were collected and analyzed using a triangulated approach and a form of recta-analysis, which incorporates multiple dimensions of program effectiveness, individual interpretations, and expert opinions. A technical panel was convened twice and consulted as needed.

Although there is no national database or source of standardized information, numerous sources of primary and secondary data were used. Two- to three-day site visits were made to programs in California, Colorado, Florida, Massachusetts, Minnesota, and Virginia. Seven papers were commissioned, and a thorough literature review was undertaken.

Structured, systematic contacts were made with directors of State units on aging, State and local long-term care ombudsman program offices, long-term care physicians, and consumer advocacy groups. A 1-day invitational symposium, a public hearing, and two "open-mike" sessions were conducted. Discussions were also initiated with four national associations of long-term care facility providers.

Findings

The report underscored the vital public purpose served by long-term care ombudsman programs and endorsed the continuation of their mandate, offering several examples of ways in which the programs have provided valuable assistance to individuals and improved the service system. The committee could not offer a definitive assessment of program effectiveness, however, because of the lack of uniform process and outcome data for all States and the lack of theoretical consensus regarding the use of appropriate effectiveness measures. Instead, the committee reported selected findings and encouraged the development of a process to facilitate uniform process and outcome evaluation in the future.

In some areas, the committee found that meaningful access to ombudsman services does not exist for all eligible nursing facility residents because of infrequent site visitation and insufficient information outreach efforts. In general, complaints receive a timely response. Program implementation in B&C homes is much more limited than in nursing facilities.

Considerable variation among States in the extent of program implementation in nursing facilities is noted. At the systemic level, the committee observed that too many States operate fragmented programs that only respond to individual complaints rather than functioning as unified, integrated programs. Many States lack access to adequate, independent legal advice and backup for ombudsmen and for residents who need assistance in courts and regulatory hearings.

The committee also found that current resources are inadequate to support full implementation of program mandates. It recommends that Federal funds for the program be increased from the current $25 million to $39.5 million by fiscal 1998. This increase would provide funding to achieve the committee's recommended staffing level of one ombudsman full-time equivalent for every 2,000 long-term care beds. (The current ratio is one to every 2,700 beds.) The committee also recommended that resources be apportioned among the States on the basis of the number of beds rather than the number of persons aged 60 years and over, as is currently done.

The committee recommended a strengthened leadership and oversight role for the Federal Government, including the implementation of an objective compliance review method for States. An improved data system is needed that includes information about program awareness, as well as the extent of complaint resolution, the extent of ombudsman input into systemic improvement activities, and the extent of ombudsman impact on the overall system of long-term care and services.

Concern was expressed that expanding the program to cover persons who do not reside in long-term care facilities may hinder the ability of ombudsmen to serve their current constituencies in nursing facilities and B&C homes. The committee recommended that expansion capacity, target populations, organizational placement, staffing needs, and funding receive careful consideration before any expansion efforts to non-institutional care settings are undertaken. The importance of developing and supporting research and demonstration projects in this arena was also noted, with attention to demographic and health care system trends affecting the long-term care system.

The report indicates that it was premature to evaluate the impact on residents of B&C homes because long-term care ombudsman programs have been only partially implemented there.

Use of Results

The development of a nationwide database of key structure, process, and outcome program measures is proposed. This study can be used by local and State ombudsman programs to assess their own strengths and weaknesses and to improve their performance. The Federal Government can consider recommendations to strengthen its role as leader, researcher, funder, guide, and monitor. Findings can also be useful to agencies that consider developing, adapting, or expanding ombudsman programs for other populations and services. It is also suggested that program expansion be delayed until existing mandates are fully implemented.

Agency sponsor:

Office of Program Operations and Development

Federal contact:

Nancy Wartow
202/619-1058
PIC ID: 5819
NTIS Accession Number: PB95-174223

Performer organization:

National Academy of Sciences, Institute of Medicine, Washington, DC

Agency for Health Care Policy and Research

Understanding and Choosing Clinical Performance Measures for Quality Improvement: Development of a Typology

Highlights

Increasingly, private purchasers of health care, consumer groups, Federal and State agencies, and health care plans are searching for methods to compare clinical performance among providers to guide choices, ensure accountability, provide data for quality improvement, and track change. Unfortunately, currently used clinical performance measures do not provide meaningful comparisons of clinical performance or, at worst, are actually misleading because they are limited in scope, insufficiently detailed, methodologically flawed, or not standardized across providers. This project is a first step toward identifying gaps between the clinical performance measures that exist and those sought by potential users. Knowledge of gaps permits prioritizing needs for clinical performance measurement and assessing the feasibility of addressing those needs through future research and development.

Purpose

The objectives of this 6-month study were to (1) collect information on the range of clinical performance measures currently in use; (2) summarize the resulting information; (3) assess the feasibility of deriving various clinical performance measures from existing databases; (4) explore the cost of different strategies for clinical performance measurement; and (5) explore the sampling issues associated with the application of selected clinical performance measures that would be useful in measuring quality of health care plans.

Background

Clinical performance measures are instruments that estimate the extent to which a health care provider delivers clinical services that are appropriate for each patient's condition; provides the services safely, competently, and in an appropriate timeframe; and achieves desired outcomes in terms of those aspects of patient health and patient satisfaction that can be affected by clinical services. Clinical performance measures concern the technical content of health care and assess health care in terms of individual patients. Clinical performance measurement requires summing data about the health care given to many patients to create a rate or score for average performance. Performance can be measured by identifying a representative sample of similar patients and collecting data about the care received by those patients within a given time period. By applying criteria for quality of performance to these data for each patient, good and poor quality care can be determined. Results are then aggregated to form a performance rate or score.

Current measurement techniques are plagued by a variety of flaws. A significant obstacle is that many types of data currently used as indicators of quality are not directly usable for comparisons of clinical performance. For example, utilization statistics (e.g., hospital admissions or rates of surgery) are not helpful unless applied to the individual patient. Measures of health status or patient outcomes are not useful without allowing for the probability that each case would experience a good outcome if good clinical care were provided. Measures of patient satisfaction are flawed because of subjectivity, although patient surveys are useful if patients are asked about the facts of their care.

Although many existing indicators of quality provide inaccurate comparisons of clinical performance, they can serve as an intermediary step toward better measures. For instance, measures of patient outcomes that cannot currently be used to compare clinical performance may become useful for that purpose as methods are developed to allow for patient differences in the likelihood of achieving a good outcome. Similarly, users who currently have inadequate data sources to construct precise clinical performance measures may find it essential to use crude indicators while working to improve data sources.

To be serviceable, measures must be useful internally within health care organizations and must have adequate levels of sensitivity, specificity, and predictive value. Measures must be reliable and valid for their intended purposes, as well as affordable. A reliable typology of performance measurements would allow potential users to select an approach--quality control, choice of health care plan, or accountability--that is appropriate for their own purpose.

Methods

A typology of measurements was created, as was a data set that included the relationship of measures to each other, the aspects of clinical performance that the measures addressed, the properties of measures that determine appropriateness for specific uses, and the data needed to create the measures.

Existing and evolving clinical performance measures were identified by two approaches: (1) a literature search conducted by using the Medical Analysis and Retrieval System® database of the National Library of Medicine; and (2) direct personal contact by phone with 112 individuals or agencies known to be involved with performance measurement research, use, or evaluation. The contacts yielded 40 sets of measures consisting of 1,287 clinical performance measures. Data concerning measure attributes were extracted, coded, and entered into six relational databases.

Findings

The project developed a classification scheme to assist users in identifying and evaluating clinical performance measures. It developed and defined key attributes of clinical performance measures and applied this framework to 40 measure sets used by public and private organizations to measure and improve clinical quality. The 40 measure sets were classified on 7 dimensions:

Rigor of development (e.g., detailed specification of measure, availability of reliability or validity tests).
Organization type for which the set is used or developed (e.g., managed care, fee-for-service, government agency).
Type of review for which the set is used or developed (e.g., internal quality management, outcome management, technology assessment, purchaser review).
Extent of use (e.g., single system, multiple system, in test phase).
Practicality (e.g., cost, implementation, or utility information available).

In addition to describing general characteristics about the type of measure set, the Typology framework classified measures with respect to their structure (factors such as data requirements, sampling, time window, scoring, risk adjustment, and interpretation) as well as their clinical content (e.g., whether a measure addresses health promotion, early detection, or treatment of a disease; whether it is a process or outcome measure, etc.).

The objective was to develop and test a prototype framework sufficiently flexible to encompass the structural and clinical characteristics of the wide variety of clinical performance measures currently used. The result was a series of interlinked databases containing information on measure sets, batches of measures (measures with similar structure or content), and clinical conditions and events that are associated with the measures.

The interrelated nature of the databases enables users to access data by measure or clinical event. Frequencies or percentages of measures in the various categories were computed and the results presented in graphs. A broad range of performance measures were included in the database (i.e., process and outcome measures, health care setting, demographics of patient population, data derived from clinician judgment and patient perception, and mental and physical health measures). The development of the prototype raised other issues to be considered (e.g., cost and sampling) when developing future databases for clinical performance measurement.

Performance measures were constructed from administrative data (e.g., enrollee files, claims data files, disease registers, pharmacy records, medical records), and from special data collections (e.g., patient or provider surveys).

Use of Results

The typology is proposed as a starting point for a data system that would permit users to find out what measures are available for given conditions and associated clinical events, what data resources are required, and which measures are suitable for the users' specific purpose. The classification framework and its definitions form the basis of a common or uniform language to describe and compare the thousands of clinical performance measures under development and in use today. The framework also serves as a teaching tool to help those interested in learning about how to construct, compare, and evaluate the utility of measures.

The study also concluded that future work is needed to test the framework and prototype databases against the needs of users. To accomplish this objective, the Agency for Health Care Policy and Research (AHCPR) is using the typology framework as the basis for a follow-on project, CONQUEST 1.0, the COmputerized Needs-Oriented QUality Measurement Evaluation SysTem. This project builds on the typology framework in three ways. First, it evaluates the typology by verifying the content of the measure database with measure developers. The verification effort has resulted in enhancements to the typology structure and content. Second, CONQUEST 1.0 builds on the typology by creating a database of information on clinical conditions that can be used to steer the search for appropriate measures. The condition database contains information from AHCPR-supported clinical practice guidelines, clinical practice guidelines produced by other organizations, and medical effectiveness research findings. Third, CONQUEST 1.0 translates the typology into a more useful system by making it available on computer. The project develops a computerized system with a user-friendly interface to link measures to clinical information and guide the selection of measures.

Follow-on efforts are currently under way at AHCPR. One such related project involves evaluating this product by convening users to pilot test CONQUEST 1.0 and participate in focus groups about its usefulness. Also, AHCPR has issued a Request for Contract Proposals for a project called QM-Net, to use the typology and CONQUEST 1.0 as the basis for a national data source for information on clinical quality measures. Information on CONQUEST 1.0 can be obtained through AHCPR's web site (http://www.ahcpr.gov) or through the Agency's clearinghouse at (800) 358-9295.

Agency sponsor:

Office of Planning and Evaluation

Federal contact:

Irma Arispe, Ph.D.
301/594-1455
PIC ID: 5630
NTIS Accession Number: PB 96-144639

Performer organization:

Center for Health Policy Studies, Columbia, MD

Centers for Disease Control and Prevention

The Effect of Mandated Managed Care for Medicaid Populations on the Practice of Public Health: The Example of Childhood Lead Poisoning Prevention

Highlights

This study examined the likely effects of managed care reforms on the delivery of childhood lead poisoning prevention (CLPP) programs. Three major service-related components of CLPP programs--screening, laboratory analysis, and case management--were evaluated at selected State and local public health departments (S/L PHDs). Results suggest that, as managed care continues to increase its share of the health care market, there is a possibility that appropriate childhood lead screening will decrease, that the participation of public laboratories in performing blood lead analysis will decline, and that followup services for children with elevated lead levels will be delivered outside the traditional health department setting. Assurance that these activities continue to be performed adequately will require public health monitoring of screening and followup and carefully written Medicaid managed care contracts. This report focuses on CLPP programs, but the underlying question it asks--how can we ensure that important public health services to children (and others in need) will continue as managed care takes over health care systems, including Medicaid--has broad implications for many other programs.

Purpose

This study was undertaken to (1) gain a thorough perspective on how various State reform efforts using different types of managed care arrangements have influenced, either positively or negatively, the three major service components of CLPP programs; (2) gather detailed information about how selected programs are handling the challenges presented by managed care and, where possible, identify patterns and themes; and (3) identify strategies that might help lead poison prevention grantees and other providers respond creatively to the challenges of a new health care environment.

Background

State health care reforms are diverse, but current trends suggest that privatization of direct services previously delivered by the public sector is on the rise. Marked increases in the cost of health care are driving this trend. By 1993, Medicaid was the single largest and fastest growing component of most State budgets, accounting for 18.4 percent of the States' total expenditures.

In an effort to stem rising health care costs, many States have turned to managed care models to deliver personal health care services to low-income persons and to other vulnerable populations, including those eligible for Medicaid. At this time, all but eight States offer some type of managed care to Medicaid enrollees; approximately 25 percent of all Medicaid enrollees are part of a managed care arrangement, and most of these individuals are in full-risk capitation programs, which generally are responsible for all care rendered. The shift from the traditional indemnity insurance system to a managed care system is expected to alter markedly the delivery of direct and nondirect services. The type of managed care organization (MCO) models that predominate within a State will likely further affect the role of public health agencies.

At both the State and local levels, CLPP programs serve primarily the most vulnerable of populations--children of low-income families. Several mechanisms provide ongoing support for these programs. For example, the Centers for Disease Control and Prevention (CDC), which launched its CLPP initiative in 1989, funds 37 S/L PHDs in an effort to develop comprehensive programs to screen, identify, and provide adequate medical and environmental followup to children with elevated blood lead levels (EBLLs). The future of this and other programs is uncertain, however, because of changes in our health care delivery system. In October 1993, the Lead Poisoning Prevention Branch (LPPB) of CDC commissioned a study to examine the effects of managed care reforms on CLPP programs, with a focus on CDC-funded activities. The results of the CDC-initiated study are presented in this report.

Methods

The five sites chosen include three states (Indiana, Rhode Island, and Tennessee), one county (Pinellas County, Florida), and one city (Minneapolis, Minnesota). They serve as models for how S/L PHDs can adapt to a managed care environment, and were selected on the basis of their involvement in a range of managed care environments and their varied responses to managed care. Teams of two contractor staff members and at least one CDC/LPPB staff member conducted the site visits. Initial site visits took place between July and September 1994; followup interviews with key personnel from all sites were conducted by phone in February 1995. Team members interviewed CLPP program staff responsible for each of the major service components of the program. They also interviewed maternal and child health staff, Medicaid staff, and key administrators. A mix of State and local respondents was sought. Where possible, the contractor/CDC team interviewed MCO staff. The report's authors acknowledge that the study's findings are based on data from a limited number of case study sites.

Findings

This report recognizes the extent to which managed care has been incorporated into our health care system. For the three components studied (screening, lab analysis, case management), data indicate that services, and therefore the individuals who receive these services, will be adversely affected in the absence of legislated mandates or highly specific contracts written between the State Medicaid agency and the MCOs operating within the State.

Data on the first component--screening--indicated that the number of lead screenings in the surveyed States decreased between 1993 and 1994. In Tennessee, for example, approximately 40 percent fewer screens were reported in 1994 than were projected for that year, but decreases in lead screenings occurred studywide. Possible explanations for the decrease include inadequate reporting and communication between private and State laboratories; poor provider compliance (e.g., with Early and Periodic Screening, Diagnosis, and Treatment program [EPSDT]); and difficulty in obtaining approval for screening.

The report recommends the following strategies to help States ensure that lead screenings are conducted: increase provider-based education and advocacy; legislate a screening mandate; expand reporting requirements; add lead screening to the State's quality assurance indicators; enforce EPSDT; retain some public sector service delivery role, including joining MCO networks as primary care providers; and try to obtain blanket authorization for lead screening within the private sector.

Data on the second component studied--lab analysis--indicated that data collection in general is likely to be severely affected by managed care, particularly in States without a strong and comprehensive reporting law. Historically, CLPP programs have received the data on blood lead tests primarily from public laboratories, whether or not a State had a reporting mandate. Surveyed sites reported success using the following strategies: establish a reporting law; mandate the use of public laboratories for selected tests; establish price (and other) incentives; and focus on the importance that surveillance serves in maintaining and improving public health.

Case management services, including necessary followup treatment and retesting of children with EBLLs, are currently performed by CLPP staff either directly or through links with other public agencies. Data from site visits revealed that private providers and MCOs had a number of concerns about CLPP case management services and admitted to a lack of knowledge of comprehensive care and followup, a lack of ability, and a lack of incentive. The report suggests the following strategies to ensure appropriate and adequate case management: establish reporting laws; identify and separately fund core public health functions; require data collection at the individual person level; require that case management responsibilities be detailed in the MCO contract; and encourage mandated or contractual relationships between MCOs and S/L PHDs.

Finally, the report recommends that S/L PHDs seek to maintain public sector service delivery, build a legal infrastructure to influence the behavior of private providers, and fund core public health functions with monies derived from non-Medicaid sources.

Use of Results

In the evaluation of the activities of the lead programs funded by CDC, MACRO International, Inc., studied the effect on lead programs of the shift to managed care of formerly public-sector patients and assessed three lead poisoning prevention program functions: screening, lab analysis, and case management.

As a result, LPPB was able to improve its guidance to State and local childhood lead poisoning prevention offices about the appropriate language for State Medicaid managed care contracts requiring lead poisoning prevention services. LPPB was also able to provide guidance on improving data systems to monitor managed care performance. Thus, some programs have quickened the pace of their efforts to complete monitoring systems, and others have formed ties with Medicaid agencies.

In addition, investigators studied the community perspective on the childhood lead poisoning prevention efforts of public health agencies. Representatives of two types of community groups were consulted: umbrella organizations, which generally have national or State-level structure and support, and community-based organizations, which are generally local in origin and support. Representatives participated in focus groups to discuss several topics, including their relations with public health agencies and their visions of how to improve these relationships. As a result, some programs have been able to change their tactics for collaborating with community-based organizations, and some have broadened the focus of their collaborative efforts.

Agency sponsor:

National Center for Environmental Health

Federal contact:

Nancy Tips
770/488-7277
PIC ID: 4743.1
NTIS Accession Number: PB 96-144670

Performer organization:

MACRO International, Inc., Silver Spring, MD

Health Care Financing Administration

Medicare Participating Heart Bypass Center Demonstration: Evaluation Design

Highlights

This negotiated bundled payment demonstration was implemented in May 1991 by the Health Care Financing Administration (HCFA) as one of many cost-containment demonstrations launched in response to increases in Federal spending under Medicare. Its goal was to assess the benefits of a negotiated package pricing arrangement for heart bypass surgery with hospitals and associated physician groups that have capabilities and incentives to manage and coordinate the complex services needed for high-quality care. Participating hospitals and physicians accepted the negotiated global package price as payment in full. The evaluation of the demonstration focused on several factors including quality and appropriateness of care, savings to Medicare, savings to hospitals, changes in patient care and hospital management, and hospital competition and marketing. The study found that both Medicare and hospitals could benefit from this type of payment arrangement through reduced costs and better coordination of services while maintaining high quality of care.

Purpose

This study examined the impact of a Medicare demonstration project to test the overall feasibility and cost-effectiveness of paying hospitals and physicians a single negotiated amount for all hospital and inpatient professional services associated with coronary artery bypass graft (CABG) surgery. Among the key questions assessed were the following:

Could a fair and efficient process be developed for the government to negotiate discounts with providers that include both hospital and physician services?
Did participation in the demonstration result in increased volume of bypass procedures for the centers?
Did the overall level of appropriateness of patient care or the extent of disease among demonstration patients change over the term of the demonstration?
Were there changes in physician or hospital management of patient care and services under the demonstration?
How did participants market their selection as a demonstration hospital, and what impact did marketing have on case volumes? How did competitors respond?

Background

Throughout the 1980's, Federal spending under Medicare increased at an average rate of 11 percent annually. Expenditures in hospital care increased threefold and spending on physician services increased fourfold during the same period. Every year, the government spends several billion dollars on inpatient care for coronary artery bypass procedures provided to Medicare beneficiaries. A major concern of both hospital managers and policymakers in controlling inpatient costs for high-technology procedures is the asymmetry of financial incentives faced by hospital staff versus physicians. Currently, hospitals are paid for bypass surgery on a per case basis, primarily within the Diagnosis Related Groups (DRGs 106 and 107). Except for extraordinary outlier costs, hospitals are paid a fixed amount regardless of the intensity of care provided each patient. Although surgeons, like hospitals, receive a bundled fee for inpatient services, other physicians, by contrast, are paid for every additional service they provide, including routine daily hospital visits and consultations. In addition, surgeons receive higher compensation for more complicated bypass surgeries. Moreover, physicians do not bear the hospital's financial risks of keeping patients in the intensive care unit longer or using more expensive pharmaceuticals. The rationale for the demonstration was that a global fee that includes physician services would align hospital and physician incentives and encourage physicians to use institutional resources in a more cost-effective manner.

The research demonstration was conducted from May 1991 through June 1996. HCFA ultimately selected seven hospitals to serve as demonstration sites during the term of the study. Selections were based on the completeness of the package of services each hospital was willing to provide Medicare beneficiaries under the demonstration, the quality of care provided by the hospital and its physicians, and the size of the discount the hospital and physicians were willing to accept for their services. The demonstration was implemented at four sites in May 1991: Saint Joseph Hospital of Atlanta, St. Joseph Mercy Hospital in Ann Arbor, the Ohio State University Hospital in Columbus, and Boston University Medical Center Hospital. In the spring of 1993, three more sites were added: St. Luke's Episcopal Hospital in Houston; St. Vincent Hospital and Medical Center in Portland, Oregon; and Methodist Hospital in Indianapolis.

Under the demonstration, participating hospitals and physicians received a global payment covering hospital and related physician services, including outliers, for each CABG procedure (DRGs 106 and 107). In each case, the negotiated rate represented a discount from what Medicare paid, on average, for these procedures. Depending on the package proposed by the individual hospital, the package of services also included pre- and postdischarge physician services and some cardiac rehabilitation services as well as readmissions within a specified time after discharge.

Selected sites received the designation of Medicare Participating Heart Bypass Centers and were encouraged to market their services to referring physicians and beneficiaries and to offer incentives to attract patients. Quality assurance was carefully monitored by HCFA as well as by project evaluators. Hospitals not participating in the demonstration continued to provide services under the traditional Medicare fee-for-service program, and Medicare beneficiaries were free to choose between demonstration and nondemonstration hospitals.

A two-phase evaluation strategy evolved. Phase I covers implementing the demonstration and evaluating the original four sites for a 2-year period while focusing on marketing, volume increases, quality of care, and developing and implementing the appropriateness model. In addition, the phase I evaluation gathered baseline data and first-year operational statistics for the three sites that began the demonstration in May 1993.

Methods

A 3-year evaluation contract was awarded to The Lewin Group and its subcontractor, Health Economics Research, Inc. After the first evaluation contract ended, a new evaluation contract was awarded in 1994 to Health Economics Research, Inc., which then continued the evaluation for the remaining 2 years of the demonstration.

An interdisciplinary group of economists, physicians, and marketing experts assembled a variety of databases and conducted numerous onsite interviews with participants as part of an extensive quantitative and qualitative evaluation of the demonstration. Methodologies included two quasi-experimental designs, time-series studies, case studies, surveys, and interviews. Several major literature reviews also were undertaken, including one on efficacy and risks of CABG surgery. A separate appropriateness study was conducted, in which an expert technical advisory panel was convened to help formulate a model of appropriateness indicators for CABG surgery. Information from the literature reviews and the appropriateness study was used to draft materials to assess quality of care at the demonstration sites.

Medicare claims were used to document national trends in Medicare bypass volumes, patient demographics, lengths of stay, mortality rates, and costs. Physician costs were broken into three categories: 30 days prior to bypass surgery, inpatient, and 90 days post-discharge. Prices negotiated with each demonstration site were compared with predicted Medicare prospective payment rates and physician inpatient outlays to determine the immediate savings from the demonstration. Market share savings also were calculated.

Every demonstration hospital provided a set of clinical information on each patient, including discharge status, risk indicators, admission priority, age, gender, and previous bypass operations. Additional information was provided on disease anatomy. The seven demonstration hospitals were compared extensively by using logistic analyses.

Findings

HCFA received over 200 letters of interest and preapplications from the initial request for participants in the demonstration, indicating that many hospitals are willing to work jointly with their medical staffs to develop the data necessary to submit a single proposed price. Of the four initial demonstration participants selected, the two nonacademic medical centers experienced statistically significant increases in Medicare bypass market shares. The third had a significant increase in market share, and the fourth had no increase.

In the first 2.5 years of the demonstration, total Medicare savings (Medicare program and beneficiaries and their insurers) were estimated at $17.6 million. Three of the four original hospitals were able to make major changes in physician practice patterns and in hospital operations that generated significant cost savings. Physicians in the three hospitals were able to quickly and dramatically reduce the length of inpatients' stay, substitute generic for brand drugs, and reduce unnecessary testing and other services. In the one hospital where surgeons resisted attempts to change practice patterns, costs continued to rise. The study found that alignment of physician and hospital incentives facilitated a closer working relationship between physicians and hospital staff.

One of the first four demonstration sites significantly reduced its patient mortality rate during the course of the demonstration. Most participating hospitals reduced intensive care unit stays by 1 full day and total hospital stays by 2 days. For the first 3 years of the demonstration, there was some evidence of a growing severity in case mix, including a higher percentage of patients over the age of 80 with comorbid conditions. Complication rates appeared to increase commensurately during this period. Nevertheless, these factors did not produce an upward trend in mortality.

The demonstration involved major changes in Medicare payment arrangements. According to providers, patients were satisfied with the single copayment amount. Hospitals, in general, were pleased with HCFA's prompt payment, which was received within 30 days by wire. Supplemental insurers responsible for paying patient deductibles and coinsurance amounts were uniformly displeased with the flat actuarial payment calculated by the government because it was incompatible with their computer systems, and patient policies differed in their deductibles, coinsurance amounts, and so forth. However, HCFA intentionally had calculated an artificial copayment amount that was comparatively low, intending to share the savings with the beneficiary--although in most cases, ironically, their supplemental insurers reaped the benefit.

Use of Results

By January 1996, more than 9,900 cases had been performed under the demonstration, with an estimated savings of nearly $38 million to the Medicare program. The demonstration has shown that it is feasible for Medicare to negotiate a bundled payment for an episode of care that represents an appreciable savings to Medicare while maintaining quality of care for the beneficiary. The demonstration also has shown that an all-inclusive global fee for both hospital and physician services can align incentives to encourage hospital managers and physicians to cooperate in using institutional resources in a more cost-effective manner.

The success of the demonstration has led to the inclusion of language in the President's 1997 proposed budget package calling for new legislation to implement negotiated bundled payment arrangements under the regular Medicare program. The experience gained from this demonstration has led HCFA to develop a new bundled payment demonstration that expands the concept to a group of orthopedic and a group of cardiovascular procedures. The Participating Centers of Excellence Demonstration for Orthopedic and Cardiovascular Services is expected to be implemented at selected hospitals beginning in mid-1997.

Agency sponsor:

Office of Research and Demonstrations

Federal contact:

Armen H. Thoumaian, Ph.D.
410/966-6672
PIC ID: 5958 and 5958.1

Performer organization:

The Lewin Group, Fairfax, VA

Monitoring the Impact of Medicare Physician Payment Reform on Utilization and Access

Highlights

This report to Congress examined the impact of legislation that reformed Medicare payment methodology to physicians. Through analysis of Medicare databases and national surveys, the report examined the use of, and access to, Medicare services before and after the new payment system was launched. Among the major findings was that legislative reforms in physician payments led to the outcome that Congress intended: the legislation shifted utilization away from surgical procedures and toward medical evaluation and management. According to the report, the legislation did not create new barriers to care for vulnerable patient groups. The report also found strong racial and income disparities in utilization. Finally, the report provided directions for future study of the relationship among income, race, and access to care.

Purpose

The purpose of this report was to keep Congress abreast of changes in Medicare patients' access and utilization following the passage of legislation in 1989 that introduced changes in Medicare physician payment policies. The report sought to provide detailed analyses of whether the legislation resulted in a shift to greater reliance on primary care and medical specialties and less reliance on surgical specialties. The report also sought to analyze barriers to, and satisfaction with, physicians' care and the relationships among race, income, health status, and utilization.

Background

This report represented the fifth in a series of annual reports required by Congress under the Omnibus Budget Reconciliation Act of 1989 (Public Law 101-239). One key purpose of this legislation was to create a more rational and equitable payment system for physicians' services under Medicare. The law mandated a Medicare Fee Schedule (MFS) that fundamentally changed the way physicians were paid. Instead of doctors' payments being based on what they charged for each service, payments are now based on a relative value scale, which reflects the relative costs of the resources needed to provide various physician services. The MFS shifted physician payments from surgical procedures to evaluation and management services. The law also placed limits on billing for amounts above the MFS fee and instituted target rates of growth in Medicare physician expenditures.

Methods

Researchers from the Health Care Financing Administration prepared the main body of the report, which summarized and integrated all analyses conducted by HCFA and by the Center for Health Economics Research. The analyses, contained in 12 separate appendixes, were based on several Medicare databases and ongoing national surveys. For example, utilization information was derived primarily from Medicare's national claims history files and its Medicare provider analysis and review (MEDPAR) file. Other utilization information came from the National Health Interview Survey conducted by the National Center for Health Statistics and from the Medicare Current Beneficiary Survey conducted by HCFA. Income information was estimated from U.S. census data based on beneficiary zip codes from the claims files. Information on patient satisfaction was derived from the Medicare Current Beneficiary Survey. To determine the impact of instituting MFS, the analyses examined trends during the years after the new system was established (1992-94) and made comparisons with data from the year before (1991).

Findings

The report's major finding was that the legislation achieved its intended objective to enhance the use of medical services and lower the use of surgical services. For example, from 1992 to 1993, claims for medical visits and consultations increased by 9.3 percent, while claims for doctors' services related to surgery, radiation therapy, and anesthesiology declined by 2.4 percent (see table II-1). These trends were consistent with those found during the first year of operation (1992) relative to the final year of the conventional fee-for-service system (1991).

**Table II-1. Medicare Part B Fee-for-Service Claims: Allowed Charges by Type of Service, 1991, 1992, and 1993**
Year	Total	Medical Visits and Consultations	Surgery, Rad. Ther., Anesthesia, & Assistants	X-ray and Lab Tests	All Other
Allowed Charges (in millions)
1993	$46,124	$16,311	$12,984	$8,962	$7,868
1992	43,942	14,926	13,301	8,785	6,930
1991	42,915	13,885	14,116	8,727	6,186
Percent Changes
1992-93	5.0	9.3	-2.4	2.0	13.5
1991-92	2.4	7.5	-5.8	0.7	12.0
Adjusted for Changed Population
1992-93	3.5	7.8	-3.8	0.6	12.0
1991-92	0.7	5.7	-7.3	-1.0	10.2

Source: Medicare Part B Monitoring System: Allowed charges derived from Medicare National Claims History File. Population information from June 30 Medicare Part B enrollment files.

The report amplified these general trends with specific information about utilization rates for numerous procedures. From 1992 to 1993, there was a decline in rates of many procedures. For example, the rate of sigmoidoscopies (per 1,000 beneficiaries) declined by 8.9 percent; the rate of hip replacements declined by 1.8 percent; and the rate of hysterectomies declined by 5.4 percent.

Overall, the passage of the legislation was associated with improved patient access. There were general increases in the number of visits per beneficiary for outpatient, nursing home visits, and consultations. But disparities emerged when these data were analyzed for vulnerable populations within the overall Medicare population. Vulnerable populations are defined as beneficiaries who live in rural areas, poor areas, or areas with a shortage of health providers; who are African-American, Medicaid-eligible, disabled, or older than 85 years. These vulnerable populations still were found to face the barriers to care that they faced before MFS was in place. Yet MFS did not create any new barriers. The report concluded that greater understanding of barriers to care for vulnerable Medicare populations is needed.

Physicians' caseloads--the total number of different Medicare patients treated in a year--remained stable or increased during the period studied. Caseload is viewed as an indicator of physician willingness to deliver services to Medicare patients. From 1992 to 1993, the mean caseload for primary care and medical specialties tended to increase more (+4-5 percent) than that for surgical specialties (+2 percent). The trends in the mean-allowed physician charges revealed a somewhat similar pattern. Physicians in primary care and medical specialties tended to experience the greatest increases in caseload, while surgeons, radiologists, and anesthesiologists tended to experience little or no growth. Some groups of surgical specialists--ophthalmologists and otolaryngologists--may have witnessed decreases in mean-allowed charges.

According to a survey analyzed in the report, Medicare beneficiaries reported greater satisfaction with their care in the years after the introduction of MFS than before. They were more satisfied with the quality of care, its availability, the ease of getting to a doctor, and the cost. In addition, fewer beneficiaries reported a health problem for which they did not receive care.

The report also showed dramatic differences in income between White and African-American Medicare beneficiaries. Based on inferences from median household income by zip code, 73 percent of African-American, but only 19 percent of White, beneficiaries were classified in the lowest income quartile. In comparison with Whites, African-Americans experienced higher mortality rates, lower rates of ambulatory visits (see figure II-1), lower use of referral-sensitive procedures, and higher rates of hospitalization. These patterns suggest that African-Americans encountered greater barriers to comprehensive and continuous care.

Figure II-1. Ambulatory Visits per Person by Race and Income: Persons Age 65 Years and Older, 1993

Some of the racial differences in utilization were attributable to income. For example, the rate of ambulatory visits declined with income, almost irrespective of race. Whites in the lowest income quartile not only had far fewer visits than wealthier Whites, but their rate also was just as low as that for the poorest African-Americans (see figure II-1). Other analyses revealed that if incomes for Whites and African-Americans were equal, some racial differences in utilization would diminish. Race, however, continues to play a role in access and utilization.

Use of Results

This report is intended to help Congress monitor the impact of legislative changes to physicians' payments under Medicare. It is expected to build on findings presented in previous reports to Congress under the same legislation. Future studies by HCFA are to concentrate on developing new utilization measures. These measures will be used to provide Congress with greater information for policy purposes.

Agency sponsor:

Office of Research and Demonstrations

Federal contact:

Marion Gornick
410/786-6696
PIC ID: 5493

Performer organization:

Health Care Financing Administration, Baltimore, MD

Health Resources and Services Administration

Performance Indicators for GPRA: Initial Assessment of Health Resources and Services Administration Programs

Highlights

There are increasing expectations across the Federal Government that agency programs, with a given set of resources and through a series of actions and decisions, will produce the desired outcomes and products for the intended audiences or beneficiaries. This study provides an assessment of the Health Resources and Services Administration's (HRSA) current capacity to respond to such expectations in compliance with the Government Performance and Results Act (GPRA) of 1993. The findings and recommendations in this study address the development of the kind of performance management system needed to respond to the governmentwide search for greater effectiveness.

Purpose

HRSA commissioned this study to assess its ability to develop and implement a performance measurement and management system in response to the GPRA. Under GPRA, each Federal agency is required to develop comprehensive strategic plans as well as annual performance plans that set specific performance goals for each program activity. Moreover, each agency must report annually on actual performance as it compares with goals. The specific objectives of the study were to provide information about the current status (as of summer 1995) of the development of performance indicators in the four HRSA bureaus and in the Office of Rural Health Policy (ORHP); the adequacy of data and data sources for applying the indicators; and the potential for the agency to use the performance measures in strategic planning or other efforts requiring indicators and data that can be aggregated across its organizational units.

Background

A variety of economic and political forces have increased emphasis on performance in both the public and private sectors. This larger societal phenomenon has produced converging political, budgetary, and management environments that collectively result in a systematic search for effective Federal programs. One of the basic underlying purposes of GPRA is to provide information about the comparative performance of the many Federal programs and agencies competing for budgetary resources, so that available resources may be allocated to those with the best return on the Nation's investment.

Under GPRA, each Federal agency is required to report annually (beginning in March 2000) on actual program performance as it compares with program goals. A preliminary set of performance measures for program and spending is to be developed for fiscal 1997 budget submissions, with actual performance data obtained from fiscal 1995 to fiscal 1997. The intent of GPRA is to motivate Federal agencies to identify programs that are effective in achieving goals and to reallocate resources away from those that are ineffective. GPRA focuses on the quality of performance, quantity of products, timeliness of products, and program efficiency.

Methods

The Lewin Group undertook this study for HRSA, organizing the approach around a central question: Can this organization, with a given set of resources, through a series of actions and decisions, produce outputs that have the desired effects and outcomes for the intended audiences or beneficiaries? Preliminary design activities included meetings with HRSA staff, a literature review, and other background research.

An interview protocol was developed and administered to each of the four bureaus and ORHP. Information was solicited about each program's objectives, inputs (e.g., dollars, staff), processes (e.g., training approaches), outputs (e.g., number of people trained), impacts (e.g., number of trainees who would not have found comparable work), outcomes (e.g., number of trainees who find and retain work), and current use of performance measures and indicators. In addition, a wide variety of program materials was collected and reviewed. Draft assessments were developed and shared with staff from the relevant bureaus and offices to ensure that the information was complete and accurate. Findings from the interviews and background research were synthesized, analyzed, and then compiled into assessments of the program clusters and individual programs or budget line items.

The study used an overall assessment approach and analytical framework to document the linkages and commonalities among various HRSA organizational components with respect to resources, processes, products, outcomes, and populations served. Application of a program logic model that illustrates the rationale, structure, and division of labor of each program was critical to the assessments. The program logic model uses a matrix to represent the interactions and relationships among a program's objectives, illustrating the flow of inputs, activities, outputs, outcomes, and impacts required to achieve them.

Findings

HRSA's current program structures and measurement efforts form a base for a HRSA-wide measurement system. As part of this system, significant indicators that are common to many programs could be used to measure achievement of agencywide goals. The study identified three principles that should underlie the performance measurement strategy:

The measurement strategy should emphasize a coordinated, interdependent system of health care resources, rather than a disparate set of programs.
The measurement strategy should be clearly linked to strategic planning, program activities, evaluation planning and findings, and budgeting.
The measurement strategy should be developed incrementally, with initial focus on the GPRA requirements that are easiest to meet.

The study found that it is both feasible and desirable to use a standard approach to measuring program performance across HRSA programs. In addition, progress in measuring and collecting performance data is quite varied across HRSA programs and program clusters. Within most programs, there is inadequate linkage along the continuum of performance measurement, with a general emphasis on producing outputs rather than evaluating program performance or outcomes. The study also concludes that data and data sources are not treated as strategic HRSA resources and that approaches to specification, collection, and use of data vary widely. Furthermore, the budget does not provide resources specifically for the data collection and evaluation necessary to build and sustain the kind of performance measurement system recommended in the study.

Recommendations are made for both short- and long-term actions. Short-term recommendations include the appointment of a central coordinator responsible for performance management efforts; the establishment of intra-agency working groups to specify performance measures, data collection, support strategies, and involvement of service beneficiaries; the provision of resources; and the provision of technical assistance tailored to specific HRSA components for developing and implementing systems of performance measurement.

Long-term recommendations address the development of a performance management system infrastructure. First, the HRSA strategic planning process should be continued and refined. Second, the performance management process needs to be linked to the budget process, to evaluation, and to grants and contracts management. Third, mechanisms need to be developed to incorporate feedback and data from grantees and populations served by HRSA. Fourth, some further restructuring of HRSA programs will be needed to facilitate performance measurement in light of HRSA-wide strategic priorities. Finally, the commitment and ongoing support of senior management is needed if staff members are to undertake the changes necessary to implement and sustain the performance system.

Use of Results

The study produced a framework for assessing HRSA functions and related performance measures and for how individual program activities and outcomes relate to the Agency's overall strategic goals. Subsequent technical assistance to HRSA on performance indicator development is proceeding, based on needs identified in this initial status report. The logic models developed for the four bureaus and ORHP are serving as an internal guide to the assessment of program inputs, outputs, and outcomes. Assistance to individual bureaus to expand staff capacity in performance measurement began at the end of fiscal 1995. The results of this study have broader applications as well. Many Federal agencies could use the performance measurement framework to assess the linkages between their strategic goals and individual program components, activities, and performance. The identified performance measures, the recommended questions on assessing program effectiveness, and the information on how to develop performance indicators are all valuable tools for any Federal agency developing planning and performance measurement strategies in compliance with GPRA.

Agency sponsor:

Office of Planning, Evaluation, and Legislation

Federal contact:

Karen Thiel Raykovich, Ph.D.
301/443-7718
PIC ID: 5954

Performer organization:

The Lewin Group, Fairfax, VA

Indian Health Service

Evaluation of Diabetes Services Provided by IHS Model Diabetes Program: Final Report

Highlights

This study evaluated the utility of a model diabetes demonstration program created by the Indian Health Service (IHS) in 1979. Through analysis of 634 medical records, focus groups, and interviews, the model program was found to improve patients' blood sugar control and to reduce their hospitalizations. A critical element of the model program was a team of health professionals who augmented primary care by providing a combination of education, outreach, service delivery, and referrals. Recommendations were made to patients, providers, administrators, and policymakers that would refine and expand this demonstration program.

Purpose

The purpose of this study was to evaluate a model diabetes demonstration program at two of the five sites at which the program was launched by IHS in 1979. The hallmark of the program is a team of three to five midlevel health professionals, including a physician's assistant or nurse practitioner, nurse, and dietitian/nutritionist. The team educated patients in preventive care, referred patients to specialists when needed, coordinated services, performed community screening, and maintained patient registries. The study measured two key patient outcomes--blood sugar control and hospitalizations--in relation to a control group of patients who received usual IHS diabetes services.

Background

The Indian Health Care Improvement Act (Public Law 94-437) of 1992 mandated an evaluation of a model diabetes demonstration program established by IHS in 1979. Since its inception, this model program of intensive primary and preventive care has been extended to at least one site within each of the 12 regional areas of IHS. The impetus for the formation of the model program was to improve patient management and to reduce hospitalizations for a costly condition that disproportionately affects American Indians.

The prevalence of diabetes in this population (6.9 percent) is almost three times higher than that in the U.S. population as a whole (2.4 percent). American Indians also suffer from disproportionately higher rates of diabetic complications. Complications include loss of vision or blindness, lower extremity amputations, and end-stage renal disease. Mortality rates among diabetic American Indians also are higher compared with those of other ethnic groups. Moreover, diabetics are intensive consumers of medical services, and the costs per patient are among the highest of any medical condition.

The underlying philosophy behind the creation of the model program was that a team of primary care professionals could effectively deliver patient education, outreach, and treatment. Patient education was designed to address diet, exercise, foot care, administration of medications, and monitoring of blood sugar. Equipped with more knowledge and skills, patients were thought to be in a better position to control their blood sugar levels. Achieving better control over blood sugar levels was expected to lower the rate of diabetic complications and other conditions that lead to hospitalization.

Methods

The study methodology consisted of quantitative and qualitative components. Abstracted medical records from a total of 634 patients who entered treatment between 1983 and 1992 were analyzed. Information from medical records was supplemented by data on inpatient and outpatient contacts from IHS health services utilization databases. Additional analyses were conducted to determine the generalizability of findings to other IHS sites. Using 1993 IHS audit data, patient demographics, severity of illness, and extent of services at model sites were compared with similar data at other model sites. The analyses also compared the control site with other IHS sites that provided usual treatment.

Focus groups with patients and interviews with IHS and tribal staff supplemented quantitative analyses. During four focus group meetings--two at model sites and two at control sites--patients were asked about problems they encountered in complying with medical recommendations and their views of the services they received. Interviews with medical personnel yielded information about the content and the perceived quality of diabetes services.

Findings

The study found the model program to be successful at controlling patients' blood sugar levels and at reducing their hospitalizations. Fewer patients (11-35 percent) at model versus control sites experienced poor blood sugar control, as defined by fasting levels of more than 220 milligrams per deciliter of blood or by random levels of more than 250 milligrams per deciliter of blood. Patients at model sites were at lower risk of being hospitalized, operationally defined as the time from diagnosis to the first diabetes-related hospitalization.

The study identified several elements of the program that were associated with good blood sugar control: consultations with a dietitian and a podiatrist in the first year after diagnosis and receipt of a hemoglobin A1c test one or more times in 2 years. This test for blood sugar levels is more reliable than others because it reflects average levels over the previous 3- to 4-month period. Similar programmatic elements helped prevent hospitalizations. As to the generalizability of all study findings to other IHS sites, the analysis of audit data could not provide a definitive answer because of undersampling at the control site.

Observations from patients and providers at focus groups and interviews, respectively, offered important insights about the model program. Patients and providers both claimed changes in diet to be the most difficult for patients to implement. Patients stressed the importance of good communication with their providers, and those receiving the usual care reported more communication problems. This finding was considered noteworthy because earlier published research revealed that the quality of the patient-physician relationship is predictive of diabetic patients' compliance with lifestyle changes and medication dosing.

Use of Results

The study offered a battery of recommendations to refine the model program and to enhance care at IHS sites that do not have the program. The study is being prepared for publication. Knowledgeable clinical staff who are current with diabetes management were considered vital to the delivery of primary care to diabetes patients. Those IHS sites that do not have the model program can greatly benefit from creating one new staff position that is devoted to diabetes education, coordination, and outreach.

The finding that patient education in the first year after diagnosis had important and long-lasting impacts led to a recommendation for early patient education that focused on difficult lifestyle changes, particularly dietary changes. Blood sugar evaluation by both standard tests and by the hemoglobin A1c test were requirements for good clinical management and were correlated with better sugar control and lower hospitalizations. Only a small percentage of patients at model sites, and even fewer at the control site, received both tests once a year.

The study recommended more time for direct patient contact with providers even in lieu of some educational services. A strong bond between patients and providers was viewed as essential for improving patient compliance. Finally, the study recommended training for providers in counseling patients who are especially stressed by the diagnosis of diabetes.

Agency sponsor:

Office of Planning, Evaluation, and Legislation

Federal contact:

Frank E. Marion
PIC ID: 6005

Performer organization:

Native American Consultants, Inc., Washington, DC

National Institutes of Health

Fetal Alcohol Syndrome: Diagnosis, Epidemiology, Prevention, and Treatment

Highlights

A committee of experts was convened by the National Academy of Sciences, Institute of Medicine, to evaluate published studies on fetal alcohol syndrome (FAS) and related disorders. The committee offered new diagnostic categories to resolve confusion and thus improve diagnosis. Through recommendations for policy, research, active and passive surveillance, and enhanced prevention and treatment, the committee sought to intensify the public health response to a completely preventable group of disorders with serious developmental consequences. Recognizing that the problems of FAS and related disorders cross medical disciplines, the committee recommends establishing an interagency task force, led by the National Institute on Alcohol Abuse and Alcoholism (NIAAA), to coordinate national efforts.

Purpose

The FAS committee was responsible for enhancing the understanding of available research and clinical knowledge by reviewing and evaluating studies on FAS and related disorders with respect to diagnosis; prevalence; and effectiveness of surveillance, prevention, and treatment programs. Because of uncertainty on fetal outcome associated with moderate or low levels of maternal alcohol consumption, the committee restricted its analysis to the effects on the fetus of maternal consumption of large amounts of alcohol.

Background

As part of the Alcohol, Drug Abuse, and Mental Health Administration Reorganization Act (Public Law 102-32 1), Congress mandated a study by the National Academy of Sciences' Institute of Medicine of FAS and related disorders that result from heavy alcohol use during pregnancy. FAS is a serious disorder with physical and mental deficiencies that are costly to treat and rehabilitate and that often require long-term care. The costs of FAS and related conditions can be quite high for the individual, for the family, and for society. Alcohol's effects on the fetus are considered more severe than cocaine, heroin, and marijuana, yet this serious public health condition is rarely diagnosed and treated.

FAS and related disorders are characterized by a constellation of physical malformations, growth deficiencies, and neurodevelopmental abnormalities. The most obvious malformations are found on the face, often in the form of a flat upper lip and midface, but other malformations can be present in other parts of the body. Neuro-developmental defects include impairments in intelligence, motor skills, gait, hearing, and eye-hand coordination. Although not all FAS victims experience mental retardation, FAS is considered the most common totally preventable cause of mental retardation. Variability in the expression and permanence of these defects is dependent on numerous factors, including the amount and pattern of maternal drinking, the timing of drinking during pregnancy, and a host of biological and environmental factors before and after birth.

The incidence of FAS and related disorders appears to be 0.5 to 3 cases per 1,000 births, although the data are far from ideal, in part because of inaccuracies in diagnosis. While up to 14 percent of women drink heavily when they drink, only a small percentage of women, about 4 percent, are dependent on alcohol according to accepted clinical criteria. The rates of heavy drinking during pregnancy vary, but most studies find that less than 1 percent of pregnant women drink heavily. Although these percentages appear small in relative terms, the absolute number of women engaging in hazardous drinking is large. The annual costs of FAS and related disorders are estimated at $75 million to $9.7 billion. The great range of costs reflects uncertainty about the long-term consequences, such as the need for medical and educational interventions, as well as uncertainty about the actual incidence of FAS and related disorders.

Methods

The 14-member committee of experts reviewed and evaluated published studies on FAS and related disorders. Committee expertise included pediatrics, developmental psychology and neurology, obstetrics, nosology, teratology, epidemiology, psychiatry, and substance abuse prevention and treatment.

Findings

The committee formulated new diagnostic criteria for FAS and related disorders. The purpose of the new criteria was to capture the range of defects, both obvious and subtle, and to improve clarity, rigor, and consistency of diagnosis for clinicians and researchers. Symptoms were presented in five major diagnostic categories, with three categories describing partial characteristics of full FAS. The two categories for subtler related disorders, collectively referred to as "alcohol-related effects," are for either physical malformations or neurodevelopmental abnormalities. Alcohol-related effects are thought to occur more frequently than full-blown FAS.

Passive and active surveillance measures were compared. With passive surveillance, data are collected retrospectively through reviews of medical charts, although more expensive active surveillance in which data collection takes place prospectively through experimental protocols could identify more cases and provide superior data. The committee recognized that passive surveillance measures could be heightened but were inadequate to capture the incidences of FAS and related disorders.

Identified risk factors for maternal drinking were a major research gap. Understanding risk factors that predict drinking behavior during pregnancy was deemed vital for developing targeted prevention programs. The committee concluded that more study was needed of the psychological and social-environmental factors that lead to heavy drinking during pregnancy, including dependence on alcohol, depression, low self-esteem, family history of alcoholism, sexual abuse, other violent victimization, and dietary habits.

The committee constructed a conceptual framework for prevention that takes into account cultural, sociological, behavioral, public health, and medical disciplines. This new framework was seen as an important guide to the field, given the scarcity of prevention-related research on FAS and related disorders. The proposed prevention model is a continuum of interventions designed for targeting the general risks of all women who drink, to the specific risks incurred by some women who drink heavily during pregnancy. The committee endorsed the universal prevention message to all women that alcohol consumption should be avoided during pregnancy, although it noted a paucity of studies to determine the behavioral impact of this message.

The committee stressed that an important aspect of preventing FAS is treating the alcohol-dependent woman. There is a compelling need for studies that assess the effectiveness of specialized programs that treat pregnant women who are alcohol dependent. The committee recommended a high priority be placed on efforts to design, implement, and evaluate prevention programs that effectively guide pregnant women who drink heavily into alcohol treatment. The committee further recommended continued research on biological methods of detecting alcohol consumption by pregnant women and continued basic research in animals to elucidate mechanisms of alcohol-induced fetal abnormalities so that medications can be developed that ease the symptoms of FAS.

The availability of treatment for individuals affected by FAS and related disorders was viewed by the committee as inadequate. The committee was unable to identify specific treatment programs. Because studies have clearly shown the value of early intervention in ameliorating symptoms of other neurodevelopmental disorders, the committee stressed the importance of identifying children with FAS and related disorders as early as possible. The committee also concluded that there was inadequate and insufficient attention given to preventing secondary disabilities such as deteriorating mental health, criminal behavior, and substance abuse.

Use of Results

The committee recommended research to assess the utility, reliability, and validity of its new diagnostic criteria for FAS and related disorders. Other research recommendations included active surveillance through national surveys that are repeated at periodic intervals; identification of psychological and social-environmental risk factors to guide the development of more targeted prevention programs; and studies to determine the effectiveness of prevention and treatment programs geared toward pregnant women, families, and individuals with FAS and related disorders.

In terms of major policy recommendations, the committee embraced the Surgeon General's 1981 position of total abstinence from alcohol prior to conception and throughout pregnancy and recommended the establishment of clinical practice guidelines for treatment of children with FAS and related disorders. While recognizing that NIAAA has historically led research efforts on FAS, the committee noted that NIAAA is a small institute and FAS is a complicated and diverse problem. In the absence of a single organized discipline of health professionals with responsibility for FAS prevention and treatment, the committee recommended forming an interagency task force to coordinate national efforts. The task force, which would be spearheaded by NIAAA, would include appropriate Federal research, surveillance, and service agencies.

Agency sponsor:

National Institute on Alcohol Abuse and Alcoholism

Federal contact:

Lura Abbott, M.S.
301/443-2369
PIC ID: 6006

Performer organization:

National Academy of Sciences, Institute of Medicine, Washington, DC

Office of Public Health and Science

Consequences of Whistleblowing for the Whistleblower in Misconduct of Science

Highlights

In this study, a contractor for the Office of Research Integrity (ORI) contacted and interviewed by mail individuals who had made allegations of scientific misconduct to ORI. In this first examination of its kind, ORI sought to better understand the impacts of whistleblowing and develop an empirical base by which to examine the consequences of whistleblowing on whistleblowers. More than two-thirds of the whistleblowers surveyed reported experiencing at least one negative outcome as a result of their action, with the most serious negative consequences being actions taken by institutional officials. Adverse consequences were even more likely to be experienced if the alleged misconduct developed into a high-profile case that gained notoriety outside the institution.

Purpose

Data were collected on whistleblowers from closed cases to provide an empirical base to consider the consequences of whistleblowing. These data were used to inform the Commission on Research Integrity and ORI staff as they sought to deal seriously and effectively with whistleblowers' allegation of misconduct and to protect them from retaliation or negative consequences resulting from their action. Data were collected on the types of actions experienced by whistleblowers before and after their allegation and the effects these actions had on their personal and professional lives. In addition, more detailed information was gathered on the particular circumstances of the allegation (i.e., the relationship of the accused, the type of allegation, the outcome of the allegation, and the amount of publicity the allegation received).

Background

Uncovering misconduct in science often depends on the willingness of those who are aware of or suspect the misconduct to report it. Before reporting misconduct, potential whistleblowers must consider whether administrators will take the allegation seriously, treat it confidentially, and protect them from retaliation.

Current Federal Regulations (42 CFR 50 103) require that institutions develop policies and procedures to handle allegations of misconduct. These policies must include provisions for "undertaking diligent efforts to protect the positions and reputations of those persons who, in good faith, make allegations" (42 CFR 50 103 [D][13]). If there is an immediate need to protect the interests of the person(s) making the allegations, Federal policy requires institutions to notify HHS.

The National Institutes of Health Revitalization Act of 1993 created the Commission on Research Integrity, which was established in March 1994 to make recommendations to the Secretary of HHS and Congress on how the Public Health Service (PHS) should deal with research misconduct in research funded by the PHS Act. In an interim report released in January 1995, the Commission identified "retaliation against whistleblowers" as one of three problem areas affecting the responsible conduct of scientific research.

Congressional hearings revealed that some whistleblowers may have suffered retaliation by the accused or others in the institution, which led to mandated additional regulations to protect whistleblowers. Despite legislative and executive efforts, however, empirical evidence and anecdotal reports continue to suggest that some students and faculty engaged in scientific research are, or feel they are, vulnerable to retaliation if they report misconduct.

Methods

The Research Triangle Institute conducted the study for ORI. Individuals listed in ORI files (closed cases only) as having made allegations of scientific misconduct were identified. There were two phases of data collection. In the first phase, whistleblowers were contacted by mail; and a followup telephone call was made to create an up-to-date mailing list, which resulted in current addresses for 105 of the 127 individuals (82 percent) listed in the files.

A second-phase questionnaire was mailed to individuals, and a followup telephone call was made to remind them to return the completed survey. Eighty-nine individuals responded to the questionnaire; of these, 68 whistleblowers were considered eligible and willing to participate. Ninety-one percent of respondents reported that they held doctoral degrees; 78 percent worked in an academic setting; 9 percent were postdoctoral or graduate students.

Findings

Whistleblowers in scientific misconduct cases are highly likely to experience one or more negative consequences as a result of their whistleblowing, but most perceive these consequences as having had a neutral impact on their careers, professional activities, and personal lives. Sixty-nine percent of whistleblowers reported experiencing at least one negative outcome. Twenty-five percent reported serious consequences such as loss of position or denial of tenure, promotions, or salary increases. Other negative consequences included reduction in research support or travel funds, counterallegations, delays in reviewing manuscripts or processing grant applications, and ostracism.

The majority of the negative consequences experienced by whistleblowers were due, they said, to the actions of institution officials, respondents, colleagues, and professional societies; the most serious consequences felt by whistleblowers were attributed to the actions of institutional officials while the institution was responding to their allegations (i.e., while the case was still open) and after the inquiry or investigation was completed. Consequences were most likely to involve pressure on the whistleblower to drop the allegations of misconduct.

Data revealed that whistleblowing was most likely to have adverse outcomes in situations in which fabrication of data was alleged, the case received publicity, the allegations were made to a senior administrative official or misconduct official of the institution or to the funding agency, the allegations were made both within and outside the institution, the allegations were made to many different types of individuals, and the allegations were subjected to an investigation without recourse to an initial inquiry.

About 62 percent of whistleblowers perceived their whistleblowing to have had a neutral impact on their careers, professional activities, and personal lives; 28 percent perceived a negative impact; and 10 percent reported a mixed impact. Although few whistleblowers perceived positive consequences of their actions, 68 percent reported a willingness to make another allegation; 12 percent said they probably would make another allegation; 10 percent were uncertain; and 10 percent said they would not.

Not every whistleblower suffers substantial negative consequences as a result of reporting misconduct, but most individuals who report allegations of misconduct frequently face the prospect of significant hardship because of their efforts.

Use of Results

The study results will be used by ORI to develop the mandated regulation on the protection of whistleblowers, educate institutional officials and scientists about current abuses, and create a system of monitoring the treatment of whistleblowers in scientific misconduct cases.

These data suggest that ORI focus first on limiting adverse actions while the case is still active. To prevent the most serious consequences of whistleblowing, regulations and enforcement approaches will need to primarily target institutional officials. Finally, potential whistleblowers should be counseled about the likely harm they will suffer if they make their case a cause celebre by taking their concerns outside their institution or getting their case publicized by the media.

Agency sponsor:

Office of Research Integrity

Federal contact:

Lawrence J. Rhoades, Ph.D.
301/443-5300
PIC ID: 5659
NTIS Accession Number: PB 96-200449

Performer organization:

Research Triangle Institute, Research Triangle Park, NC

Chapter III. Agency Evaluation Activities

10 agencies of the U.S. Department of Health and Human Services (HHS) and the Office of the Assistant Secretary for Planning and Evaluation (ASPE) and Office of Public Health and Science (OPHS) in the Office of the Secretary each maintain their own evaluation program, including the functions of evaluation planning and policy review, quality assurance through technical review, project coordination and management, dissemination of reports, and utilization of results. This chapter provides an agency-by-agency summary of the evaluation activities during fiscal 1995. Each summary begins with a description of the relevant evaluation program--its philosophy, policies, and procedures. Next is an overview of the major evaluations completed in fiscal 1995 and the major evaluations in progress during that time. Finally, each agency offers a discussion of new directions for its evaluations, including priorities for future program and policy evaluation projects. As stated in the foreword, abstracts and contact persons for all evaluation reports completed in fiscal 1995, organized by agency, are presented in appendix A. Listings, by agency, of all evaluation projects in progress are located in appendix B.

Administration for Children and Families

MISSION: To promote the economic and social well-being of families, children, individuals, and communities.

ACF Evaluation Program

ACF administers a broad range of entitlement and discretionary programs including the welfare programs (Aid for Dependent Children [AFDC], Job Opportunities and Basic Skills Training [JOBS], Child Support); children and family services (Head Start, Child Welfare, Family Preservation and Support, youth programs, child care); four block grants; and special programs for targeted populations, such as the developmentally disabled and Native Americans.

The objectives of ACF's evaluations are to provide information on the design and operation of effective programs; test new service delivery approaches that build on the success of completed demonstrations; apply evaluation data to policy development, legislative planning, budget decisions, program management, and strategic planning and performance measures development; and disseminate findings of completed studies and promote application of results by State and local governments.

ACF actively engages with other Federal agencies, State and local policy and program officials, national organizations, foundations, professional groups and practitioners, and consumers to stay current on the emerging issues affecting its programs and to identify questions for evaluation studies. Study designs are carefully negotiated with the States and other interest groups. Studies are frequency funded as joint ventures with ASPE and other Federal agencies and foundations. Such collaborations enable efforts that are better informed, more representative of varying perspectives, and larger in scale. Proposals are reviewed by multidisciplinary experts. Work groups of various types are used throughout the projects to monitor progress and to advise on refinements in design and presentation of the findings.

Summary of Fiscal 1995 ACF Evaluations

ACF's evaluations are closely linked to its two strategic goals:

Measurably improve the economic independence and productivity of families by reforming the welfare system and by stimulating the changes in attitude and behavior necessary to achieve results.
Increase the number of children, youth, and families who have improved health, development, and well-being and live in stable communities.

During fiscal 1995, ACF produced 23 evaluation reports on its various programs. Several major evaluations were related to the first goal of "Economic Independence and Productivity of Families." One study focused on increasing participation in work and work-related activities and the lessons learned from welfare reform demonstrations in five States: Colorado, Iowa, Michigan, Utah, and Vermont. The study identified a number of successful approaches that could be used by other States. The second study, "Something Old, Something New: A Case Study of the Post-Employment Services Demonstration in Oregon," evaluated a demonstration intended to help recently employed persons keep their jobs, help those losing their jobs return to work quickly, and reduce the amount of time spent receiving AFDC.

ACF's Office of Refugee Resettlement looked at its Key States Initiative Program, begun in fiscal 1987, to increase employment and reduce welfare dependency in refugee communities in Minnesota, New York, Pennsylvania, Washington, and Wisconsin. The study documented program design features, program participant characteristics, program outcomes, and lessons learned from each State.

Ten evaluations important to the goal of "Healthy Development of Children and Families" were completed in fiscal 1995. Three evaluations focused on children in foster care. The first, "Foster Youth Mentors," examined factors characterizing successful relationships between foster youth and older citizen mentors. The findings have been disseminated to independent living programs throughout the United States to facilitate use of mentors in older youths' transitions from foster care. The second study, "Outcomes of Permanency Planning for 1,165 Foster Children," examined foster care programs in San Diego County, California, and Pierce County, Washington. The study found an overrepresentation of minority children and a sizable number of children with mental health, physical health, and behavioral problems, calling for a more culturally sensitive practice in child welfare systems and appropriate services to this population with special health needs. The third study, "Update From the Multi-State Foster Care Data Archive: Foster Care Dynamics 1983-1993," contains foster care career histories for all children in State-supervised substitute care living arrangements. The study found that infants and young children are the fastest growing age groups in the foster care population.

The other seven studies focused on specific ACF program activities. "Strengthening Families and Neighborhoods: A Community-Centered Approach," highlighted in chapter II, is an innovative method of testing services improvement in a distressed urban neighborhood in Linn County, Iowa, modeled on the British patch system of community-centered service delivery. The 3-year demonstration project aimed to overcome categorical barriers that prevent the pooling and use of informal and formal resources needed for flexible services--services building on the strengths of individuals, families, and neighborhoods.

"Youth With Runaway, Throwaway, and Homeless Experiences: Prevalence, Drug Use, and Other At-Risk Behaviors" is a national study that examines substance use, suicide attempts, and other at-risk behaviors, which is also highlighted in chapter II. The findings were drawn from youth living in shelters, on the street, in family households, and from youth shelter directors. The study recommends that services address a continuum of need from primary prevention to intervention and treatment at the community level.

"Child Maltreatment 1993: Reports From the States to the National Center on Child Abuse" found that almost 2 million reports of child abuse and neglect were received by child protective services agencies and referred for investigation in 1993; neglect is the most common type of maltreatment, followed by physical, sexual, medical, and emotional abuse; and 1993 was the first year since 1976 that the rate of reported child abuse and neglect cases did not increase.

"Children on Hold: Improving the Response to Children Whose Parents Are Arrested and Incarcerated" surveyed patrol officers, narcotics officers, child protective services workers, foster care parents, and corrections staff in 100 counties and conducted site visits to four communities selected for their exemplary responses to children whose parents are arrested. The report assesses existing policies, procedures, and practices of child welfare, law enforcement, and local correctional agencies regarding children whose primary caretaker is arrested or incarcerated. It also assesses how well these agencies coordinate with each other in dealing with these families, identifies promising strategies to improve coordination, and assesses statutory enactments and case law regarding the termination of parental rights for incarcerated parents. The report recommends ways in which relevant agencies can improve services to these children. It highlights areas of concern within law enforcement and child protective services agencies and in the interaction of these agencies.

Next, ACF supported two literature review studies. First, the "Study of the Impact on Service Delivery of Family Substance Abuse" reviews research literature from 1986 through 1994, including intensive case studies of the effects of family alcohol and other drug (AOD) abuse programs funded by the Administration for Children, Youth and Families (ACYF). Child Protective Services, of all ACYF programs, is the most negatively affected by family AOD abuse. AOD abuse affects the mandates of all ACYF programs including accelerated termination of parental rights policies, postadoption and foster placement support programs, and Head Start's services to nonparental caretakers. The second study, "Selected Annotated Bibliography on Youth and Gang Violence Prevention, Community Team Organizing and Training, and Cultural Awareness Curriculums," developed a resource for youth-serving organizations and individuals, researchers, and policymakers concerned with youth issues.

Finally, ACF assessed its Low-Income Home Energy Assistance Program (LIHEAP) in a report to Congress for fiscal 1993, looking at State and Indian tribal use of funds to provide heating and cooling assistance, energy crisis intervention or assistance, low-cost home weatherization, or other energy-related home repairs. An estimated 5.6 million households received help with heating costs through heating or winter crisis assistance in fiscal 1993. This figure represents 20 percent of the 28.4 million households estimated to have incomes under the maximum income eligibility standard established by the LIHEAP statute. The mean home energy group burden for all eligible households (i.e., the ratio of home energy expenditures to income) was 4.0 percent of income compared with 1.1 percent for all U.S. households. Thirty-eight percent of all LIHEAP recipients received public assistance, and 65 percent received food stamps.

ACF Evaluations in Progress

During fiscal 1995, ACF had 18 evaluation projects in progress. These projects are also linked to ACF's two strategic goals.

Three studies pertinent to the goal of "Economic Independence and Productivity of Families" are examining employment of welfare recipients. The JOBS evaluation, a major long-term study of the processes, impacts, and cost-effectiveness of the JOBS program, is designed to evaluate the effectiveness of alternative strategies for moving welfare recipients to work. Within the past year, the evaluation produced preliminary impacts on employment and welfare receipt at three sites (Atlanta, Georgia; Grand Rapids, Michigan; and Riverside, California), providing separate results for program models that are education focused (human capital development) and employment focused (labor force attachment).

The second study, which looks at the Parents' Fair Share (PFS) Demonstration, is testing the effects of requiring unemployed noncustodial fathers of children on AFDC to participate in employment and other services designed to increase their earnings so they can adequately support their children. PFS programs in seven States have developed effective procedures to identify eligible fathers, enroll them into employment services, and enforce regular participation. Preliminary data also show that PFS work and training requirements provide States with a promising mechanism to discover previously unreported income of nonpaying, noncustodial parents; approximately 25 percent of the men in the program had previously unreported income.

The third study, an evaluation of the Oregon Post-Employment Services Demonstration, addresses job loss among newly employed welfare recipients in four sites: Riverside, California; Chicago, Illinois; Portland, Oregon; and San Antonio, Texas. Each site furnishes job retention and reemployment services to recently employed JOBS program participants randomly assigned to receive the additional services, regardless of continued AFDC receipt.

Two studies under way address parenthood in welfare families. "Responsible Fatherhood: Theoretical and Empirical Foundations for Policy and Program Development," a joint project with the Office of the Assistant Secretary for Planning and Evaluation, is aimed at systematically developing credible information for States and localities about how to encourage and increase responsible conduct among fathers of disadvantaged children. Five community-based organizations are currently operating programs to teach fathers how to understand their children's development and positively affect their children's behavior.

A second study is assessing the effectiveness of the Home Visiting Services Demonstration in Chicago, Illinois; Dayton, Ohio; and Portland, Oregon. This demonstration requires first- time teenage parents on AFDC to participate in the JOBS program. It will test whether adding weekly home visitor services to mandatory JOBS programs will substantially strengthen the effectiveness of JOBS programs in helping young mothers better support themselves and their children while promoting positive parenting and reductions in repeat childbearing.

Two other studies are looking at child support enforcement issues. The first, "Evaluation of Child Support Guidelines," is funded by the Office of Child Support Enforcement (OSCE) and evaluates presumptive child support guidelines. The study will assess the impact of shifting from voluntary to presumptive guidelines by using the Current Population Survey--Child Support and Alimony Supplement of 1992 and will examine changes and activities of State Guideline Commissions.

A second OSCE study, the "Evaluation of Child Access Demonstration Projects," assesses demonstration projects in Idaho, Indiana, and Florida set up to test mediation services as a means to assist divorced, separated, and nonmarried parents reach parenting plans, as well as encourage greater involvement by noncustodial parents (usually fathers) with their children after divorce or separation from the custodial parent.

The final report in this group of related studies is titled "Identification and Prevention of Intergenerational AFDC Dependency: Promoting Long-Term Child Welfare." It will examine the causes of intergenerational welfare dependency, looking specifically at the critical age at which girls are most likely to be caught up in the dependency cycle. Factors distinguishing those who break free from dependency compared with those who cannot will be identified.

ACF has nine evaluations related to its second major strategic goal of "Healthy Development of Children and Families." Three of them address various facets of the Head Start program. First, the "Evaluation of the Head Start/Public School Early Childhood Transition Demonstration" is intended to assess the effectiveness of providing comprehensive, continuous, and coordinated services to Head Start families and children from the time of Head Start enrollment through the third grade in public school. The study will provide data on the effectiveness of the transition project models in maintaining the gains that children and families achieve while in Head Start.

A second study, "Evaluation of Head Start Family Child Care (FCC) Homes," will assess the quality of Head Start services provided in FCC homes and determine whether these services meet quality standards, including Head Start program performance standards. It will also compare services delivered in FCC homes with those delivered in Head Start centers.

The third study, a "Descriptive Study of Head Start Bilingual/Multicultural Program Services," will determine the number, geographic distribution, and sociodemographic characteristics of the Head Start eligible population for different cultural and linguistic groups, by region and nationally; will determine the number and nature of bilingual and multicultural children served by Head Start; and will identify the range of bilingual and multicultural services provided by Head Start nationally. The study includes an indepth assessment of service models, staff training approaches, community partnerships, and administrative plans and processes of a sample of 30 Head Start programs that use innovative methods to address the unique program needs of one or more of the diverse cultural and linguistic groups.

Several ACF projects in progress are looking at family protective services. The "National Study of Outcomes for Children Placed in Foster Care With Relatives" is examining the outcomes, including costs, for children and families in various configurations of relative foster care compared with similar configurations of nonrelative foster care. The "Family Preservation and Family Support Services (FP/FS) Implementation Study" will examine how FP/FS program funds for services to strengthen families have been used across States and communities and among different stakeholders. The main component of the study analyzes and synthesizes first-year State applications.

The "National Study of Protective, Preventive, and Reunification Services Delivered to Children and Their Families" will determine the number and percentages of children and families in the child welfare system that receive protective, preventive reunification, out-of-home care, and/or aftercare services. Case record abstracts will be completed on a nationally representative sample of 3,000 children and their families served by public child welfare agencies.

The "Evaluation of Nine Model Comprehensive Community-Based Child Abuse and Neglect Prevention Programs" is taking place in three phases over 3 years. Process and impact data are being collected across programs and through a series of studies conducted at each of the nine sites. Because each project has up to 10 service components, many of which vary across programs, a series of experimental designs has been developed for each service component.

ACF also is supporting two studies looking at family services from an international perspective. The "Transfer of International Innovations--Development of a Clinical Monitoring System To Support Foster Care in Michigan" features a computerized system based on a model combining structured and systematic monitoring of each individual child, with the aggregation of this information across the whole agency. Developed and now mandated in Israel, and modified for the U.S. foster care system, the system provides an integrated response to the needs of all partners in the agency, including administrators, managers, and policymakers. A second project is a knowledge transfer project assessing the application of a preteen Bedouin Arab dropout prevention program in a tribal community in the Negev Desert of Southern Israel to the Pascua Yaqui Indian Tribe in the Sonoran Desert of southern Arizona. The assessment will examine the effectiveness of a cooperative community/university model for empowering economically disadvantaged minority communities to respond to the unique needs of their at-risk children.

One ACF evaluation project addresses an important crosscutting issue of children and family services with substance abuse treatment. The "Woman and Infant Nurturing Services (WINGS)" program, designed to counteract the upward spiral of female incarceration and substance abuse, is being assessed. This demonstration project at the Rose M. Singer Correctional Facility, Riker's Island, New York City, targets pregnant, substance-abusing inmates and uses incarceration as a point of treatment intervention. The evaluation will test the effectiveness of a comprehensive service program, including substance abuse treatment, prenatal health and nutritional care, human immunodeficiency virus (HIV) education, parenting classes, mental health services, and assistance with entitlement preparation.

New Direction for ACF Evaluation

The passage of the Personal Responsibility and Work Opportunity Reconciliation Act of 1996 intensifies ACF's need to work even more closely with States and others to help improve the delivery, organization, and financing of human services to ensure positive impacts for children and families.

In recent years, State-initiated welfare reforms approved through waivers have been accompanied by carefully designed evaluation studies. For example, many States are testing the following: (1) requiring teen parents to attend school or training, (2) requiring minor parents to live at home or in other structured living settings, (3) increasing participation in JOBS, (4) increasing regular applications of larger sanctions for noncooperation with child support and JOBS requirements, and (5) encouraging savings by establishing special purpose accounts. One project described earlier has already provided States with very useful operational lessons on how they have increased participation in employment and training activities.

The law provides for continuation of these studies as well as for the initiation of new studies. For example, an area of current interest concerns the impact that welfare reforms will have on children. Several States have already expressed their interest to work in collaboration with ACF, ASPE, and other agencies to address this and related issues. Efforts are under way to develop measures of child well-being, to explore administrative databases as informational sources for evaluation, to identify methods for measuring outcomes and performance, and to move the field forward to benchmark, measure performance, and track results.

Administration on Aging

MISSION: To foster the development of services to help older persons maintain their independence.

AoA Evaluation Program

AoA is the Federal focal point and advocate agency for older persons and their concerns. Under the Older Americans Act (OAA) of 1992, AoA administers programs to help vulnerable older persons remain in the own homes by providing supportive services, and offers opportunities for older Americans to enhance their health and to be active contributors to their families, communities, and the Nation through employment and volunteer programs. AoA works closely with a nationwide network of Federal regional offices, State and area agencies on aging, tribal organizations, national organizations, and representatives of business to plan, coordinate, and develop community-based systems of services that meet the unique needs of individual older persons and their caregivers.

As the responsibilities of this nationwide network of governmental and private organizations continue to grow, it is essential that they have the necessary information to meet these responsibilities. The overall evaluation priorities of AoA are to support studies that provide information on the success of existing program implementation in meeting the goals of the OAA; the design and operation of effective programs; and policy development, legislative planning, and program management.

Summary of Fiscal 1995 AoA Evaluations

During fiscal 1995, AoA completed two studies related to the assessment of the quality and effectiveness of services for older persons. The first project examined the effectiveness of the long-term care ombudsman programs. These programs were created more than two decades ago as a response to serious concerns about the quality of nursing home care. Currently operating in all 50 States, the District of Columbia, and Puerto Rico, these programs are funded from a combination of Federal, State, and local sources.

The role of the ombudsman has evolved to include both individual and systemic advocacy. Individual activities are designed to ensure that ombudsman services are accessible to all residents, systematically receive and respond to resident concerns about their long-term care services, and work cooperatively with a variety of agencies to resolve problems.

Systemic advocacy activities include evaluation of laws and regulations related to long-term care services for older adults, education of the public and facility staff, dissemination of data about the ombudsman program, and promotion of the development of citizen organizations and resident and family councils. Both roles are important, but the primary activity of ombudsman programs is identifying, investigating, and resolving individual resident complaints.

The evaluation was conducted by the Institute of Medicine (IoM) of the National Academy of Sciences/National Research Council. An IoM committee developed a formative evaluation strategy to identify program issues, strengths, and weaknesses. The evaluation results documented the public purpose served by long- term care ombudsman programs and endorsed the continuation of their mandate, offering several examples of ways in which they have provided valuable assistance to individuals and improved the service system. In some areas, the committee found that meaningful access to ombudsman services does not exist for all eligible nursing facility residents because of infrequent site visits and insufficient information outreach efforts. In general, complaints receive a timely response. Program implementation in board and care (B&C) homes is much more limited than in nursing facilities. Because the committee could not make a definitive assessment of program effectiveness, they recommend development of a process to facilitate uniform process and outcome evaluation in the future.

The IoM committee also recommended a strengthened leadership and oversight role for the Federal Government, including implementing an objective compliance review method for States. An improved data system is needed that includes information about program awareness, as well as the extent of complaint resolution, the extent of ombudsman input into systemic improvement activities, and the extent of ombudsman impact on the overall system of long- term care and services.

The second project completed by AoA developed the conceptual framework and provisional design for two studies that the 1992 OAA amendments authorize the Secretary of HHS to conduct on the quality of care provided by B&C facilities and on the quality of home care services for older and disabled individuals. The evaluation was conducted by IoM with the understanding that funding for the larger studies would need to come from other public and private resources. These recommendations were based on the deliberations of an outside expert study committee that directed the study, a literature review, and presentations and comments made by consumers, leading researchers, and State officials at an invitational workshop.

The final report recommends that one study be conducted that looks at quality assurance in both home care and residential care settings. It further recommends the focus be on all users of home- and community-based long-term care, reaching beyond just the elderly or disabled populations. The report lays out a study design to examine the key features that define home care services and the consumers receiving them, the frequency and severity of quality problems in home care and in residential care, the factors that enhance or impede quality care, the roles of consumers and their informal caregivers, and the need for national minimum standards or model standards to ensure the quality of home and residential care.

AoA Evaluations in Progress

AoA is currently supporting an evaluation of the Elderly Nutrition Program (ENP). With the aging of the U.S. population, increased attention has been given to designing efficient service delivery systems for older people. Of particular concern is the development of service networks that can provide elders with a continuum of home- and community-based long-term care, thus avoiding premature or unnecessary institutionalization. An important component of any long-term care system is the provision of adequate nutrition services to ensure that optimal nutritional status in the older population is achieved and maintained.

The 1992 OAA amendments directed AoA and HHS's Assistant Secretary for Planning and Evaluation to evaluate the nutrition programs funded under OAA Title III and VI, ENP. ENP has not been evaluated since 1983, and the nutrition program funded under Title VI had never been evaluated. The project is being conducted by Mathematica Policy Research, Inc. Congress mandated that the evaluation address four objectives: (1) evaluate who is using the program and how effectively the program serves targeted groups, (2) evaluate the program's effects on participants' nutritional status and socialization, (3) assess how efficiently and effectively the program is administered and delivers services, and (4) clarify program funding sources and allocation of funds among program components.

The project data collection and preliminary analysis were completed during fiscal 1995; the final analysis and report are expected to be completed in fall 1996. The final report will be disseminated to Congress, to the aging network, and to the public, with public-use tapes also available for further research.

New Directions for AoA Evaluation

Given the evolving roles of AoA and State and area agencies on aging together with the projected growth of the Nation's elderly population, AoA's evaluation efforts will continue to focus on program planning and service delivery as well as continued monitoring of the effectiveness of programs in addressing the goals of OAA.

The changes occurring in the aging service delivery network present an opportunity to learn critical lessons in program planning and system development. These changes include the more systematic focus on home- and community-based long-term care, increased sophistication in addressing a variety of needs of the older population, and changes in the health care system such as the growth of managed care and the implementation of enhanced program information systems. Future evaluations will need to consider the impact of these and other similar developments on the delivery of OAA services to the Nation's older persons.

Agency for Health Care Policy and Research

MISSION: To generate and disseminate information that improves the health care system.

AHCPR Evaluation Program

AHCPR has designed a portfolio that responds to the evaluation needs for the following:

Comprehensive information to assess AHCPR's effectiveness in meeting its major and long-term priorities and goals.
Information from "fast-track" or quick turnaround projects to respond to critical AHCPR, Public Health Service, and HHS concerns.
Information from internal evaluations to improve the efficiency with which AHCPR performs its work.

Evaluation efforts are built into virtually all AHCPR program activities. The evaluation mechanisms used by the agency include targeted evaluation studies undertaken through contracts or grants; peer review of grant applications and technical review of contract proposals for scientific integrity; obtaining information on the usefulness of AHCPR research efforts and findings through AHCPR's User Liaison Program, which provides information to State policymakers, health departments, and officials; and other targeted efforts such as focus groups and surveys to provide baseline information and assist the design of future AHCPR work.

AHCPR established a two-tier system for assessing proposed evaluation projects. The process begins with an executive-level review to assess policy relevance and relative priority of proposed projects. The review is conducted by the Administrator and senior staff. The second, or technical merit, review assesses policy-relevant project proposals for feasibility, soundness of design, costs, potential importance of the findings, and relation to ongoing evaluation activities. The second review is conducted by the Task Force on AHCPR Evaluation Projects, a group consisting of one individual with evaluation expertise from each office and center in AHCPR.

Summary of Fiscal 1995 AHCPR Evaluations

The 12 AHCPR evaluation projects completed in fiscal 1995 reflect AHCPR's mission of increasing scientific knowledge for improved clinical decisionmaking and for organizing public and private systems of health care delivery. The projects are part of AHCPR's efforts to develop information that will improve health care quality, reduce health care costs, and enhance access to health services; respond to information needs of consumers, clinical practitioners, payers, and policymakers; and promote dissemination and use of new information about health care systems, methods, and technologies.

The first group of projects consists of four studies to advance methodologies for evaluating and improving the quality of clinical care. The first project, a contract with the Center for Clinical Quality Evaluation in Washington, D.C., translated three AHCPR-supported clinical practice guidelines (on urinary incontinence, acute postoperative pain, and benign prostatic hyperplasia, or BPH) into clinical performance measurement systems. The project found it is possible to develop valid and reliable guideline-based performance measures from AHCPR- supported guidelines, and the structure of guideline recommendations can have a strong influence on developing such measures. The project also found that targeted educational efforts conducted by Medicare peer review organizations and based on the BPH guideline resulted in improved performance on specific aspects of clinical care.

The second project, a contract with Rand of Santa Monica, California, developed a quality review system based on the AHCPR- supported cataract guideline. This review system was designed for use in a range of settings including ophthalmologists in fee-for- service private practice, salaried ophthalmologists on staff at model health maintenance organizations (HMOs) or other managed care organizations, and ophthalmologists providing eye care services under capitated contracts or subcontracts with capitated primary care groups or other managed care organizations. The project found that it is feasible to develop performance measures based on the cataract guideline and that a medical record-based review system can be very useful for estimating guideline adherence and can highlight variations in practice and opportunities for quality improvement.

The third project, also conducted by Rand, developed a review system based on the guideline for prediction and prevention of pressure ulcers. The pressure ulcer guideline review system was developed and tested in collaboration with the Department of Veterans Affairs (VA) and was tested among VA and non-VA hospitals. The study revealed that because pressure ulcer care is multidisciplinary and recommends daily performance of many behaviors, the review systems required collection of data elements from a variety of data sources over a statistical sampling of hospital days.

In addition to developing clinical performance measures from science-based guidelines, these first three studies pointed to the need for organized and comprehensive information on the types of clinical performance measures in existence as well as the need for methods for evaluating the structure and quality of these measures. The fourth project therefore developed a typology or framework for collecting and evaluating clinical performance measures. AHCPR's fiscal 1995 measurement typology projects, described in chapter II, established a common language that can be used to compare measures. It also identified areas where additional clinical quality measures are needed.

Another category of projects completed in fiscal 1995 responds to the information needs of consumers, clinical practitioners, payers, and policymakers. The first of these projects, the Survey Design Project (SDP), developed prototype survey modules to obtain consumer assessments of their access to care, use of services, health outcomes, and patient satisfaction. The SDP forms the basis of AHCPR's Consumer Assessments of Health Plans Study (CAHPS), a five-year project to develop and test questionnaires that assess health plans and services, produce easily understandable reports for communicating survey data to consumers, and evaluate the usefulness of these reports for consumers in selecting health care plans and services. The second project in this category, using a qualitative case study approach, identified factors related to the selection, implementation, and use of information management systems. Project findings guided the development of an AHCPR research solicitation focused on understanding computerized decision support systems for clinical quality improvement. The third project in this category focused on improved clinical information from managed care settings. Conducted by the Institute for Health Policy Studies at the University of San Francisco, this study assessed the feasibility of using managed care data to assess the differential use of selected tests and procedures. The final project in this category developed a directory of minority health and human services data resources that can be used by researchers and others interested in examining data that contain race and ethnic identifiers.

The final group of projects completed in fiscal 1995 concentrated on evaluating the usefulness of AHCPR-supported clinical practice guidelines. A project conducted by George Washington University developed analytical designs and methods that could be used to evaluate the process of developing clinical practice guidelines. The project identified key variables related to the process of guideline development and assessed the feasibility of measuring the variables across guidelines. The next study, conducted by the American Institutes for Research, collected detailed and structured information from guideline users on issues associated with implementation of guidelines and their perceived impact. Approximately 100 physicians in office-based settings, physicians in academic settings, and nurses participated in 10 focus groups across the United States and provided information on how better to construct and disseminate information on improving clinical practice. A third project, conducted by the Institute of Medicine, described and evaluated methods for setting priorities for guideline topics. The final report in this category, AHCPR Clinical Practice Guidelines Program: Report to Congress, summarizes the Agency's evaluation work to date and highlights future directions for the Agency's work.

These findings, as well as those from other AHCPR-supported evaluation projects, suggest that the methods for introducing and implementing guidelines into clinical settings are extremely important in fostering conformance with science-based clinical recommendations. As a result of findings from this and other projects, AHCPR will no longer be developing clinical practice guidelines but instead will focus on science-based partnerships to improve health care delivery. This new three-faceted strategy will continue AHCPR's commitment to the synthesis of scientific and medical effectiveness information, establish public-private partnerships to make guidelines and other quality-related information accessible through a national clearinghouse, and continue to support research and evaluation focusing on methods to integrate scientific and medical effectiveness information into everyday clinical practice.

AHCPR Evaluations in Progress

AHCPR's ongoing evaluation activities also concentrate on generating information to improve the health care system. In the area of information for clinical quality improvement, AHCPR has supported two followup efforts to the fiscal 1995 measurement typology project, described in chapter II. The first is the development of CONQUEST 1.0, the COmputerized Needs-Oriented QUality Measurement Evaluation SysTem. The CONQUEST project builds on its predecessor project in three ways. First, the project evaluates and strengthens the framework developed in the typology project by verifying the data with measure developers. Second, it creates an interlocking database of clinical conditions with coded information from AHCPR-supported and other clinical practice guidelines, so users can identify practice recommendations and link them to clinical quality measures. Third, the project creates a computerized system with a user- friendly interface to link measures to clinical information and guide the selection of measures. A related project currently under way involves evaluating this product by convening users to pilot test CONQUEST and to participate in focus groups about its usefulness.

Another project under way in fiscal 1996 builds on AHCPR's consumer choice work. The survey design project conducted by Research Triangle Institute and CAHPS develop surveys to collect data on consumers' satisfaction with access to and use of health services and methods to disseminate this information for improved consumer choice. Through the Survey Users Network, a related contract, AHCPR will conduct needs assessments with users to obtain information on how they use consumer surveys as well as their technical assistance needs in issues such as sampling, survey administration, data analysis, and reporting. The needs assessments conducted under the survey will help inform AHCPR of the CAHPS' work.

New Directions for AHCPR Evaluation

In the coming years AHCPR will focus its evaluation activities on three general areas. The first area is the need to develop AHCPR- wide performance measures that can be used to evaluate the quality of the Agency's work as well as to inform policymaking, budget planning, and program management. Efforts are under way to develop evaluation designs that will identify key customers or users of AHCPR information, identify their information needs, and obtain feedback on the usefulness of the Agency's products and reports.

The second general area centers on building AHCPR's research portfolio to translate research findings into forms of information that actively help consumers, practitioners, payers, and others make effective health care decisions. This area builds on research related to decision-support systems and hospital information systems, as well as psychosocial and organizational research examining factors that influence provider practice and clinical quality improvement.

The third area will build on AHCPR's work to examine and evaluate changes in the health care system, and how those changes affect access to care, use of services, quality of care, and patient outcomes. Through the use of targeted evaluation projects, analyses of data from the Medical Expenditure Panel Survey, and other media, this area of research will help HHS evaluate the effect of policy changes. For example, one area might focus on comprehensive evaluations and evaluation syntheses of the effect of managed care on vulnerable populations--and specifically, the impact of system changes such as the Section 1115 demonstration waivers on Medicaid beneficiaries, on health professionals, and on public health organizations and services within and across States.

Centers for Disease Control and Prevention and Agency for Toxic Substances and Disease Registry

MISSION: To promote health and quality of life by preventing and controlling disease, injury, and disability.

Agency for Toxic Substances and Disease Registry (ATSDR)

MISSION: To prevent exposure and adverse human health effects and diminished quality of life associated with exposure to hazardous substances from waste sites, unplanned releases, and other sources of pollution in the environment.

CDC Evaluation Program

CDC places high priority on evaluations seeking to answer policy, program, and strategic planning questions. Performance improvement studies are of particular interest and importance to CDC. Evaluation studies are developed and selected based on CDC's eight strategies to achieve its mission. These strategies are as follows:

Monitor health.
Detect and investigate health problems.
Conduct research to enhance prevention.
Develop and advocate sound public health policies.
Implement prevention methods.
Promote healthy behaviors.
Foster safe and healthful environments.
Provide leadership and training.

CDC's evaluation philosophy is consistent with its overall focus on the practice and application of sound public health strategies. This orientation is exemplified by CDC's "bottom up" nature of project development. Projects are conceptualized, developed, and monitored by CDC's public health professionals who are most closely aligned with the everyday practice of public health--the programmatic staff.

Annually, the Director of CDC provides guidance to the various center, institute, and office (CIO) directors on 1 percent set- aside evaluation activities. This memorandum generally includes information about the types of studies to be carried out with 1 percent evaluation funds. Each proposal undergoes multiple levels of review. Initial review is conducted by the Office of Program Planning and Evaluation (OPPE). Subsequent reviews are completed by the Office of the Assistant Secretary for Planning and Evaluation staff. Study authors are provided with comments, questions, and recommendations made by reviewers. In addition to providing their responses, authors may revise their proposals at this time.

A panel of CDC evaluators, scientists, and program managers are convened to review and rank proposals. Review criteria include (1) relevance to prevention effectiveness, (2) relative importance of the public health problem being addressed, (3) probability that the proposed project will accomplish its objectives, and (4) extent to which other CDC programs will derive benefit from the project. Results from the panel review are converted into a comprehensive ranking that is provided to the Director of CDC. Final funding decisions are made at that time.

Finally, staff within OPPE work closely with program staff to ensure development of a clear statement of work for selected projects. Prior to initiation of procurements, a final ad hoc review of the project statement of work is completed.

ATSDR Evaluation Program

ATSDR receives its funds from the U.S. Environmental Protection Agency/Superfund appropriations rather than Public Health Service appropriations; therefore ATSDR does not receive a 1 percent evaluation set-aside. Nevertheless, ASTDR is responding to the changes mandated in its program planning and evaluation efforts by the National Performance Review and the Government Performance and Results Act (GPRA). To meet those requirements, ATSDR staff members modified ATSDR's planning process by incorporating implementation strategies and outcome/performance measures.

Prominent issues addressed in the new planning system emphasize ATSDR's priority to improve the health of people affected by hazardous substances polluting the environment. Using exposure assessments and demographic data to identify people at risk and, more directly, assessing/addressing the concerns of the Agency's customers represent some of its improvements. The new planning system provides the basis for measuring ATSDR performance and making systematic improvements a part of its internal evaluation activities.

Summary of Fiscal 1995 CDC Evaluations

Fourteen evaluations were completed in fiscal 1995. Of this number, 10 were funded through the 1 percent evaluation set- aside, and 4 were funded with program dollars. These studies are of four types: data policy/surveillance studies, program evaluations, evaluations of educational interventions, and organizational assessments.

Data Policy/Surveillance Studies

Five studies sought to assess the implications of policy decisions on various existing or proposed data/surveillance systems. One of these studies addressed automated analysis of the four major data systems maintained by the National Center for Health Statistics: vital statistics, health examination, health interview, and health care. The primary domain of this project was health care and health interview data. Another study assessed the feasibility of developing an injury risk factor surveillance system; a third project entailed the conduct of two distinct surveys to explore issues related to influenza vaccination coverage.

Program Evaluations

Three program evaluation studies were completed in fiscal 1995. These studies included evaluations of the Medical Examiner/Coroner Information-Sharing Program, the Fatality and Assessment and Control Program, and the Lead Poisoning Prevention Program (LPPP). These program evaluations were context-specific. For example, the evaluation of LPPP examined an emerging trend in the program--the effects of managed care reforms on childhood lead poisoning prevention (CLPP) programs, focusing on CLPP programs funded by CDC.

Evaluations of Educational Interventions

Of the three evaluations of educational interventions completed in fiscal 1995, two of them focused on guides to diabetes education. One guide, written in Spanish, was evaluated for the appropriateness of its use for diabetic patients in Hispanic populations. The second diabetes educational intervention focused on health care providers. This project provided information to the Division of Diabetes Translation on the impact of three different provider interventions related to the dissemination of a provider guide. The final study evaluated the progress and activities conducted by the 71 HIV education programs funded by CDC's Division of Adolescent and School Health through cooperative agreements with State and local education agencies.

Organizational Assessments

Three organizational assessments were carried out in fiscal 1995. Two of these studies addressed management issues of discrete programmatic areas: immunization and women's health. The third study sought to develop a comprehensive evaluation strategy that can be incorporated into planning budget and legislative processes in the National Center for Chronic Disease Prevention and Health Promotion.

CDC Evaluations in Progress

During fiscal 1995, 18 evaluation studies were in progress. These studies consist of three types: evaluations of data policy/surveillance systems, program evaluations, and assessments of specialized aspects of proposed programs.

Evaluations of Data Policy/Surveillance Systems

Six studies are under way that relate to either data management or reporting systems, or both. These studies include evaluations of racial and ethnic identification data, surveillance data for drug-resistant pneumococcal infections, and an assessment of the National Nosocomial Infections Surveillance (NNIS) Program.

In the study of racial and ethnic identification data, three major activities will be performed to improve the data quality of natality data systems. These activities include (1) a systematic evaluation of how respondents for birth records interpret current questions about their ethnic and racial identity; (2) an assessment of how respondents classify themselves on birth certificates regarding the current racial and ethnic categories used in the National Center for Health Statistics' natality statistics data systems; and (3) a comparison of respondents' interpretation and classification in response to the current categories of how they would interpret questions and classify themselves by using other racial and ethnic categories under consideration by the Office of Management and Budget.

The study evaluating CDC's current surveillance system for drug- resistant streptococcus pneumoniae will evaluate the system's hospital-based sentinel surveillance system for sensitivity and representativeness. Active population-based surveillance for invasive pneumococcal infections in two geographically distinct areas will be conducted. Surveillance areas for this evaluation are either adjacent to a sentinel surveillance hospital or in a community with one or more hospitals that are sufficiently large enough to serve as a comparison for the rest of the community.

Finally, the major purpose of the assessment of the NNIS Program is to develop a methodology for assessing NNIS hospitals' accuracy in identifying and reporting nosocomial infections in NNIS hospitals and to assess the utility of criteria used to diagnose these infections. Specifically, the study is designed to (1) confirm the presence of nosocomial infections, by site, in patients previously reported to NNIS as infected; (2) confirm the absence of infections, by site, in a hospital-selected population of patients who were at risk for acquiring nosocomial infections but for whom no infections were reported; (3) identify certain risk factors and nosocomial infection case-finding criteria to the identified infection site; and (4) determine whether consistent findings can be achieved by two similarly trained observers who apply specific nosocomial infection case-finding criteria to a retrospective review of a given medical record.

Program Evaluations

Seven major program evaluations are currently in progress. Included in this number are evaluations of the National Institute for Occupational Safety and Health's Health Hazard Evaluation (HHE) Program, the Field Epidemiology Training Program (FETP), the National Laboratory and Training Network (NLTN), and the Dengue Hemorrhagic Fever Prevention and Control Program in San Juan, Puerto Rico.

The HHE study is a process evaluation. As such, the overall purpose of the study is to develop a tool and procedure by which NIOSH can conduct an ongoing evaluation of the effectiveness of the HHE Program, which responds to 400 to 500 requests for onsite health hazard evaluations each year from employers; employees; employee representatives; or Federal, State, or local agencies.

By contrast, two of the studies mentioned are outcome evaluations. The evaluation of FETP will ascertain whether the program has achieved its objectives to train public health professionals in applied epidemiologic skills, to promote the sustainability of autonomous FETPs, and to develop a global network of national programs.

The evaluation of NLTN will assess the degree to which NLTN achieves its goals and mission. Study questions fall into five general categories: (1) offerings related to needs of laboratories and their staff, (2) quality of the training provided, (3) impact of the training provided, (4) quality of outreach and marketing, and (5) barriers to training.

Finally, the evaluation of the Dengue Hemorrhagic Fever Prevention and Control Program will assess three major components of the program: (1) proactive disease surveillance, (2) education of the medical community, and (3) community-based mosquito control.

Assessments of Specialized Aspects of Proposed Programs

Five specialized studies relating to discrete aspects of current or proposed programs are in progress. These studies include an evaluation of tuberculosis (TB) outreach worker activities as well as a community-context study seeking to understand minors' access to tobacco products. Two studies related to violence prevention are also included in this category. One of these studies relates to suicide in Native American communities, and the other considers various implementation aspects of youth violence prevention programs.

Each of the studies mentioned has as its core a desire to understand particular human behaviors and motivations. A clear understanding of these issues must be present before wholehearted program and fiscal commitments are made. For example, the Advisory Committee for the Elimination of TB recommended increased funding for TB outreach and workers conducting the outreach. However, prior to committing resources to this strategy, CDC must have a clear understanding of how outreach work is currently conducted, and what outreach workers do in their everyday work. It is through the evaluation of TB outreach workers that the National Center for HIV, STD, and TB Prevention will be provided with baseline data about how outreach workers function.

Similarly, a community context study is also being conducted that will help refine CDC's understanding of the relationships between public policies prohibiting minors' access to tobacco, the implementation and enforcement of such policies, tobacco-vendor perceptions and actions that may influence the sale of tobacco products to minors, and the use of tobacco products by minors.

Finally, two evaluations addressing violence prevention issues are also under way. The first study is being conducted jointly by CDC and the Indian Health Service. The study will evaluate surveillance systems, process indicators, and outcomes of four multifaceted suicide prevention programs. The second violence prevention study will identify key factors in the successful development and implementation of CDC-funded youth violence prevention programs. As part of the study, a document will be developed that will assist individuals, groups, or communities as they implement their own youth violence prevention programs.

New Directions for CDC Evaluation

CDC's evaluation program will continue to support studies aimed at improving public health programs. This focus on improvement is consistent with the congressional intent of GPRA. Efforts to develop and implement performance indicators are currently under way at CDC. Similarly, as programs have been developing and implementing performance indicators, projects that assess the effectiveness and efficacy of such indicators have been initiated. CDC's evaluation priorities in upcoming years will be based on five crosscutting strategies that address the programmatic needs of its 11 CIOs.

Strengthen Core Public Health Functions

A number of key activities at CDC center around its commitment to strengthen the Nation's core public health functions. Tangible results of this commitment are reflected by CDC's work with State and local health departments and other partners throughout the country to obtain information necessary for monitoring and evaluating health, conducting epidemiologic and laboratory studies, developing new technologies, and providing training and technical assistance to help ensure that States and communities can protect the health of their citizens. Ongoing and proposed evaluation studies seek to answer how well CDC is fulfilling this commitment.

Enrich Its Capacity To Respond to Urgent Threats to Health

Epidemiologic investigations and laboratory work done by CDC and State and local health departments enable CDC to address urgent threats to public health in a timely and effective manner. Similarly, in a world increasingly threatened by emerging infections, CDC's leadership role in this area is critical. In fiscal 1996, a number of evaluation studies address important programmatic issues related to infectious disease.

Develop Nationwide Prevention Strategies

One of CDC's key roles is to translate knowledge about effective methods of preventing disease and injuries into nationwide strategies that reach people in communities throughout the country. Several studies under way seek to assess the extent to which guidelines developed by CDC impact disease prevention activities throughout the Nation.

Promote Women's Health

Recognizing that women's health issues have not always received the attention they have warranted, CDC has identified promotion of women's health as one of its top priorities. Although a number of projects are funded by CDC's Office on Women's Health, there are several ongoing evaluation studies that address important women's health issues. An example of one such study is an evaluation of guidelines for preventing perinatal HIV infection by measuring how recommendations are translated into routine clinical practice. Successful implementation of recommendations depends on the ability to maximize the opportunities for HIV- infected women to learn their HIV infection status, to be offered and receive preventive therapy, and to gain access to health and social services for themselves and their infants.

Invest in the Nation's Youth

CDC's commitment to investing in the Nation's youth is exemplified by its school health education programs. However, CDC's investment is not limited to school-based intervention programs. For example, one recently initiated study is designed to assess the effectiveness of various teen pregnancy prevention interventions. This study will result in a systematic methodology to identify and evaluate community intervention components that show promise based on the behavioral science theory, available evaluation information, and consensus of leaders in the field.

Food and Drug Administration

MISSION: To protect and promote public health through food, drug, medical device, and cosmetic regulation.

FDA Evaluation Program

Systematic changes in the government management environment are strongly influencing the setting, conduct, and use of evaluation activities in FDA. Three forces--all related to the goals of the Government Performance and Results Act--are reshaping the evaluation function within FDA.

Performance Management

The reorientation of all government managers toward performance management has shifted the responsibility for program evaluation from specialized staff offices and contractor studies to the day- to-day line managers. Picking appropriate program goals, establishing a valid measure for those goals, and collecting management information to record the measured progress toward the goals are now integral parts of the new government manager's responsibility. Relearning the role of management regarding these shifted responsibilities is a key priority.

Customer Participation

Most of FDA's management performance measures are the same as those for regulated industries. Thus, although the beneficiary of FDA's performance is ultimately the general public, FDA operates in a manner that supplies industry with an essential component of commercial success. FDA's approval of a new drug, for example, not only satisfies a legal requirement but also assures the public of the safety and efficacy of the drug. Collaboration between FDA and its regulated customers regarding the design and coordination of the joint responsibilities to ensure effective high-quality products was a revolutionary concept, but it is becoming the new norm under the customer-conscious GPRA directives.

Increasing Rigor of the Rulemaking Process

FDA establishes standards of safety and efficacy through rules published in the Federal Register. Many safety and efficacy standards are also performance standards that industry is obliged to meet. Examples are the Good Manufacturing Practices regulations. Today, virtually every FDA final rule of significant magnitude includes elements found in classic program evaluations- -a critique of the existing system, alternatives for better performance, performance and cost tradeoffs, reactions and suggestions of customers, and conclusions with an action timetable for implementation. The new directives of the Administration as well as the possible legislative action by Congress will bring the rulemaking process of the future even closer to the classic evaluation process.

In summary, FDA's evaluation efforts are driven by the mandates of GPRA and its corollaries; are carried out by line managers rather than specialized evaluation staffs; and are focused on performance management, customer participation, and rigorous rulemaking. Because it receives its funds from the U.S. Department of Agriculture's appropriations rather than Public Health appropriations, FDA does not conduct evaluations under the 1 percent evaluation set-aside authority.

Summary of Fiscal 1995 FDA Evaluations

The past year was the second year under the new evaluation paradigm. The following examples illustrate the manner in which managers have integrated evaluation into their line responsibilities and have conducted evaluation in cooperation with the affected customer.

"Implementation of the Prescription Drug User Fee Act." In fiscal 1995, FDA's drug and biologic review processes involving more than 1,000 Field Test Evaluations (FTEs) completed their third full year of successful performance management toward goals and performance measures jointly established by FDA managers and industry customers. The goals were ambitious. Line managers agreed to clear an overdue backlog equivalent to half a year of submissions, review a pending workload equal to nearly 2 years of submissions, and build a review capacity that could meet a phased schedule of substantially accelerated review goals. FDA managers met or exceeded all of the performance goals for the year.

"Assessment of the Mammography Quality Standards Act of 1992 (MQSA)." This program is FDA's second major user-fee, performance-oriented, GPRA-style initiative. MQSA requires facilities that perform mammography services to be certified as meeting standards developed by FDA. FDA managers want to ensure that as MQSA is implemented, patients can maintain access to quality services, especially patients in areas that are short of health professionals. To achieve this goal, FDA managers contracted for a cost-benefit analysis of the program. The contractor reviewed records from more than 10,000 facilities that had applied for accreditation and from more than 500 that had ceased operations. The contractor presented the results of the cost-benefit analysis to FDA managers who then used them to develop a performance-oriented regulatory policy coordinated with participating customers.

FDA Evaluations in Progress

The evaluation agenda of line managers in fiscal 1996 will be influenced by the objectives discussed below, which merely reflect the forces shaping their evaluation role.

Performance Management

Managers of all FDA programs are evaluating their performance measures in light of GPRA standards. Broad-based training of managers to enable them to implement the performance design of their responsibilities is under way. This multiyear process toward a new standard of performance measure will end its first phase with the formulation of FDA's 1997 budget.

Customer Participation

FDA managers are identifying further opportunities to involve their customers in the design and testing of alternative ways of doing business. The design and implementation of the Prescription Drug User Fee Act (PDUFA) is the most notable example of the successful pursuit of this objective. Initiatives with import brokers to facilitate the entry of safe products into the country is another example. Managers plan to involve customers in the redesign of several establishment inspection functions.

Negotiated Rulemaking

FDA managers are adding a new customer-sensitive dimension to their increasingly rigorous rulemaking function by implementing the President's directive to promote negotiated, consensual rulemaking. This is a marriage of customer participation and rigorous rulemaking that is attracting serious attention by FDA managers.

One example of the evaluation projects managers will undertake in fiscal 1996 to implement the stated objectives is the Fourth Annual Evaluation of PDUFA. Fiscal 1996 is a critical year for implementing PDUFA. Negotiations on its renewal will begin shortly thereafter. FDA managers will use performance data from this GPRA-style program to convince customers and Congress of the merits of continuing the program.

New Directions for FDA Evaluation

Changes in government management have created a systemic change in FDA's evaluation function. The new paradigm driven by line managers' performance responsibilities, by the imperatives for involving customers, and by directives for analytically rigorous rulemaking has replaced the traditional practice of evaluation guided and managed by centralized, specialized evaluation staffs.

Health Care Financing Administration

MISSION: To promote the timely delivery of appropriate, quality health care to the Nation's aged, disabled, and poor through administration of the Medicare and Medicaid programs.

HCFA Evaluation Program

The research arm of HCFA, the Office of Research and Demonstrations (ORD), performs and supports research and demonstration projects (through intramural studies, contracts, and grants) to develop and implement new health care financing policies and to provide information on the impact of HCFA's programs. The scope of ORD's activities embraces all areas of health care: costs, access, quality, service delivery models, and financing approaches. ORD's research responsibilities include evaluations of the ongoing Medicare and Medicaid programs and of demonstration projects testing new health care financing and delivery approaches. These projects address four major themes:

Monitoring and Evaluating Health System Performance. HCFA's research program produces information and descriptive statistics on the infrastructure of the health system, on populations of health care users, and on service and expenditure patterns; examines differences in costs, quality, and access to care; and assesses the effects of HCFA programs on beneficiary health status.
Improving Health Care Financing and Delivery Models. HCFA performs research and demonstrations to develop and test new payment and delivery models intended to increase the efficiency and effectiveness of Medicare. ORD's evaluations of these demonstration projects provide policymakers with information about their impact.
Meeting the Needs of Vulnerable Populations. HCFA's evaluation program includes studies to examine the factors affecting access to health care delivery, as well as evaluations of demonstration projects that test new financing and delivery systems to improve access to cost-efficient and appropriate health services for vulnerable populations.
Providing Information To Improve Consumer Choice and Health Status. Improving beneficiaries' knowledge and ability to make informed health care choices, both in the health plans they select and in the services they use, is part of HCFA's commitment to improve communication of information to beneficiaries. ORD examines variations in the use of services and treatments and the impact of new information tools and technologies in making health care decisions and improving beneficiaries' health status.

Summary of Fiscal 1995 HCFA Evaluations

During fiscal 1995, HCFA completed three major evaluations.

"Monitoring the Impact of Medicare Physician Payment Reform on Utilization and Access" represents a combination of a wide series of intramural and extramural projects and results in an annual report to Congress. This annual report was initiated when the new Medicare fee schedule for physician services was implemented to track the changes that might be caused by such a major shift in the Medicare payment approach. The fiscal 1995 report indicates that the new payment system has produced the kinds of shifts in payments that were anticipated. There are continuing indications that many vulnerable population groups face barriers to care; additional understanding of these barriers is needed to improve access to care.

The "Medical Participating Heart Bypass Center Demonstration Evaluation" covers the first 3 years of a demonstration carried out at seven hospitals across the United States. Its findings support the feasibility of an all-inclusive negotiated bundled payment arrangement for heart bypass surgery at high- volume hospitals. This evaluation focused on the assessment of surgical appropriateness, quality of care, hospital and Medicare program savings, and patient satisfaction.

Finally, an evaluation of the Rural Health Care Transition Grant Program produced information for an annual report to Congress on this HCFA grant program for rural hospitals. The report describes the new awards made in fiscal 1994, the characteristics of grantees compared with earlier cohorts, and the impacts of the grants on the hospitals and their communities.

HCFA Evaluations in Progress

ORD currently supports 39 evaluation projects in progress. These projects provide information for continued monitoring of the Medicare and Medicaid programs and assess the impacts of HCFA's Medicare and Medicaid demonstration projects. In-progress evaluations that are scheduled to be completed in fiscal 1996 include examinations of Medicaid program initiatives such as 1915(b) waivers and Community-Supported Living Arrangements and evaluations of the following several demonstration projects:

The Medicare cataract surgery alternate payment demonstration.
The Medicare case management demonstrations.
The Medicaid uninsured demonstrations.
The Medicare Alzheimer's disease demonstration.

HCFA's evaluations in progress that are scheduled for completion in later years include evaluations of State health reform demonstrations, including Oregon and Tennessee, and of the following ongoing demonstrations:

The program for all-inclusive care for the elderly demonstration.
The Medicare home health agency prospective payment demonstration.
The nursing home case-mix and quality demonstration.
The drug utilization review demonstration.

New Directions for HCFA Evaluations

As the U.S. health care system continuously changes, there is a clear need for the development, design, and testing of new ways to monitor and evaluate the performance of the system. It is important that monitoring and evaluation efforts for the Medicare and Medicaid programs include a number of critical dimensions to provide an understanding, on an ongoing basis, of how well these programs are performing in terms of access to care, quality, efficiency, costs, and beneficiary satisfaction. ORD is working to develop a comprehensive monitoring and evaluation plan for systematically examining the Medicare and Medicaid programs. It also will continue to work to develop a wider array of evaluation and measurement tools. Evaluation activities will continue to examine specific policy issues within the HCFA programs. For example, as Medicare and Medicaid continue to pursue managed care options, ongoing work will examine the cost-effectiveness and quality of and beneficiary satisfaction with managed care. HCFA also plans to carry out projects to monitor and compare the health status and/or health risk behaviors of beneficiaries in various delivery systems and how these change over time.

Finally, as HCFA develops and implements new high-priority demonstrations that will test new payment and health care delivery models for the future, it will continue to evaluate these programs and provide information to policymakers about the impacts of these alternatives. The new projects include the Medicare Choices Demonstrations, which will test the feasibility and desirability of new types of managed care plans for Medicare; demonstrations of Medicare payment for telemedicine services in rural areas; Operation Restore Trust, which is demonstrating improved methods for investigation and prosecution of health care fraud and abuse; a demonstration of Centers for Excellence that replace separate fee-for-service payments with capitated payments for the entire medical costs associated with certain expensive medical/surgical procedures; a competitive pricing demonstration to evaluate an array of cost-saving approaches of paying for managed care; and the continued monitoring and evaluation of State health reform demonstrations.

Health Resources and Services Administration

MISSION: Improve the health of the Nation by assuring quality health care to underserved and vulnerable populations and by promoting primary care education and practice.

HRSA Evaluation Program

The purposes of the HRSA evaluation program are to enhance strategic planning, budget decisions, and legislative planning, and to improve program management. Consequently, major emphases during fiscal 1996 and beyond will be performance measurement and assessment of program implementation and policy.

Performance measurement includes (1) technical and training activities to strengthen HRSA's capacity to assess program performance, and (2) studies to assess the outcomes of individual programs or groups of programs. Projects to enhance measurement capacity have assumed greater emphasis in relation to the Government Performance and Results Act. During fiscal 1995, HRSA established a performance measurement baseline for all operating programs; the ultimate objective is to ensure that HRSA establishes valid and useful indicators and measures for all programs by the end of fiscal 1998. Over the past year, HRSA has made significant progress in performance management that provides operational linkages between strategic planning, program activity, and the budget process--as intended by GPRA.

Assessment of program implementation and policy includes a broad range of efforts to assist HRSA during a time of significant change caused by new policy directions initiated by the Administration or Congress. Specifically, HRSA-funded programs are entering into new arrangements for delivering services, providing health professions education, and encouraging the development of systems reform within seven program priority areas. These priorities reflect HRSA's focus on underserved populations. They include academic and community partnerships to foster clinical training in community-based settings; new arrangements to bring poor, uninsured, rural, and chronically ill persons into the mainstream of managed care; assistance to communities in strengthening their health care infrastructure; and new activities with States. HRSA's studies relate to one or more of these priorities and include efforts to describe and assess the initial or later implementation of a program; compare alternative approaches to delivering services; assess the benefits of a current or potential policy; examine the effectiveness or efficiency of resources management; and conduct evaluation syntheses.

The objective of HRSA's evaluation work is to provide useful and timely information to the Administrator, the four bureaus, and the Office of Rural Health Policy. To ensure that this objective is met, all study proposals are reviewed first by a committee composed of the bureau directors and four other senior executives who consider the proposals' relevance to important policy, budgetary, or legislative issues; potential to answer questions about program effectiveness or impact; and degree of attention to crosscutting topics. This committee makes recommendations to the Administrator about study approval and in so doing establishes relative priorities for funding. Subsequently, expanded materials needed for developing contracts for approved studies are reviewed by a committee of senior analysts from HRSA, the Agency for Health Care Policy and Research (AHCPR), the National Center for Health Statistics (NCHS), and the Office of the Assistant Secretary for Planning and Evaluation (ASPE). The role of the latter group is to offer suggestions, as needed, on ways to strengthen study design and/or data sources.

HRSA places major emphasis on the effective dissemination of findings and information about the use of completed studies. Dissemination is done in a number of ways and is especially designed to ensure that evaluation information promptly reaches the Administrator and other agency executives. In addition, attention is given to a broader dissemination through articles in professional literature and presentations at professional conferences. An annual report on completed studies describes purpose, findings, and uses; a second annual volume provides brief summaries of studies initiated the preceding fiscal year.

Summary of Fiscal 1995 HRSA Evaluations

HRSA completed an agencywide review of performance measurement capacity for all line programs last year through a project that is highlighted in chapter II, "Performance Indicators for GPRA: Initial Assessment of HRSA Programs." This work is the foundation for current component-specific efforts and for developing the fiscal 1998 budget submission. Three projects reflect differing aspects of health professions work force issues. A study of user satisfaction found that information from the National Practitioner Data Bank was clearly important in identifying practitioners who had not voluntarily revealed their "problem records" to hiring and certifying authorities. HRSA has used the study to develop the improved data bank system implemented in June 1995 and to strengthen the reporting process. Another health professions study, "Development of Integrated Requirements for PAs, NPs, CNMs, and Physicians (MDs and DOs)," has produced a model for estimating the national demands for primary care practitioners by year through 2020. HRSA advisory groups (the Council on Graduate Medical Education and the National Advisory Council on Nursing Education and Practice) are using the model to form recommendations for data and education program development. Another external user is the Utah State Health Department, which is adapting the model to project State-level needs.

A third example is a study that describes factors influencing decisions about the numbers and types of primary care personnel used by health maintenance organizations. The growth of managed care is altering the ways in which members of the health care work force are used and thus is modifying the demand for, and the costs of, various groups of health care professionals. Understanding how primary care staffing decisions are made in managed care organizations will improve health professions' forecasts by helping determine how to adjust to the influence of managed care. The study also investigates ways in which managers alter their primary care staffing patterns to accommodate the needs of Medicaid enrollees. It has given an initial insight into changes in the demand for primary care personnel that will accelerate as more States move into managed Medicaid.

Major priority is placed on improving access to high-quality services for prevention and treatment of HIV/AIDS. A study of Ryan White CARE Act Title I assessed grantee strategies used to reach African-Americans not in care in four metropolitan areas. Among the strategies described in the report are establishing and maintaining broad planning council representation from all populations affected by epidemic; enhancing the capacity of community-based agencies to successfully compete for local funding; adapting case management systems to changing priorities; recruiting African-American physicians as clinicians in community-oriented primary care; and building community awareness and participation in planning through African-American institutions, such as communities of faith. Two key findings of the report were that the methods used here also could be used to assess strategies for other underserved populations and that developing effective strategies requires the grantee, the planning body, and constituents of services to recognize that each acts on distinctive underlying values and principles.

Another study, "Impact of Ryan White CARE Act Title I on Capacity Building in Latino Community-Based Organizations," developed and pilot tested a methodology for evaluating whether Title I funds have influenced the development, expansion, or enhancement of HIV/AIDS services in San Diego and Boston. (This methodology also can be used to study capacity building in organizations serving other populations.)

A careful analysis was conducted of the cost-effectiveness of the Community Health Centers (CHCs) Program. It showed that AFDC recipients who used a CHC as their main source of primary care incurred lower Medicaid costs and used fewer hospital days than other recipients living in the service areas of selected CHCs in California and New York. Study findings, which reflect only care reimbursed on a fee-for-service basis, have been used in presentations to the Office of Personnel Management and Congress. Building on this study, HRSA has contracted for a national evaluation of the effectiveness and impact of CHCs, including examination of the experience of centers involved in managed care.

Representatives of the Substance Abuse and Mental Health Services Administration (SAMHSA) participated in planning and oversight of an additional CHC study, "Community and Migrant Health Centers and the Assessment of and Response to Mental Health and Developmental Needs in Primary Care Patients." The final report has been distributed to State primary care organizations (which administer the primary care cooperative agreements with States) and to primary care associations for use in advising member organizations on improving their mental health services.

Essential to achieving more effective program evaluation is the production of useful data. To help produce this data, a third primary care project developed a Uniform Data System (UDS) covering five programs, including Community and Migrant Health Centers and Health Care for the Homeless, and a user manual for the UDS. This new system provides uniformly defined data for related programs and eliminates duplication and inconsistency in reporting, thus reducing grantee burden.

Also in the services area, a maternal and child health study assessed the outcomes of grants that were intended to foster creation of a permanent infrastructure for child and adolescent injury prevention in seven States. This study, which analyzed and compared the State strategies, found that recipients in general exhibited a considerable increase in injury prevention capacity several years after the expiration of the special incentive grant funding. Results will be used by policymakers in considering future funding for injury prevention and will influence the design of such efforts.

HRSA Evaluations in Progress

Ongoing studies include a range of projects concerned with enhancing performance measurement capacity and assessing the effectiveness or implementation of specific programs. Illustrative studies in particular topic areas follow.

In the AIDS area, HRSA is continuing a longitudinal examination of the effects of Ryan White CARE Act Title I funding on services for active or recovering drug users with HIV. Staff of the National Institute of Drug Abuse are collaborating in the design and oversight of this study. Another AIDS-related project provides a synthesis of local evaluations sponsored by Ryan White grantees to facilitate dissemination nationwide of findings and experience with various methodologies as applied to locally identified issues.

A study concerning organ transplantation, titled "Reasons African-American and White Waiting List Patients Are Unavailable for an Organ Offer," reflects HRSA's concern with cultural competence in health service delivery. An Inspector General's report prepared in 1991 showed that African-American patients on the waiting list for a kidney transplant waited nearly twice as long as did Caucasian patients. Only part of this difference can be explained by biological and medical factors. HRSA, through the Organ Procurement and Transplantation Network, is responsible for ensuring that the U.S. organ allocation system operates equitably. This current study will help determine whether changes in Federal policy governing the organ procurement system are needed.

Continued major attention is directed toward community-based service programs. "Effectiveness of the National Health Service Corps" is a 3-year study to assess the Corps' performance by using such indicators as satisfaction of communities where Corps members are working; increases in numbers of people served because of placement of a Corps member in a site; and long-term retention of former Corps assignees in a primary care or related profession.

As noted, HRSA is conducting a national study of the effectiveness and impact of Community and Migrant Health Centers through a sample of 50 centers in 10 States. The data will be analyzed separately for users served under managed care arrangements. The "Community Health Center User and Visit Survey" involves interviews with 2,000 users and a review of 3,000 medical records for visits at the same centers to provide information about the demographics of CHC users, their reasons for seeking care, their diagnoses, services used, and outcomes of care. Another study is assessing the impact of Medicaid waivers on HRSA-funded, federally qualified health centers in States that have instituted mandatory managed care for Medicaid beneficiaries.

In addition, "The Future of Primary Care" was intended to define the place of primary care in the changing health care environment and to develop normative goals for primary care. Results will be used to assess the adequacy of the primary care system in the United States and strategies needed to influence the supply and distribution of primary care providers.

An emerging policy issue concerns the management of the J-1 visa program, a matter that cuts across health professions and primary care issues, therefore calling for a collaborative effort with the Educational Commission for Foreign Medical Graduates and the Appalachian Regional Commission. This project, "Tracking of J-1 Visa Exchange Students," is examining the postresidency experience of exchange students who have secured waivers (to the requirement to return home for 2 years following completion of residency) to remain in the United States. Experience of those remaining will be compared with experience of those who return to their home country. Results, expected in 1997, will provide information for guiding U.S. policy about such waivers for physicians.

The "National Evaluation of the Healthy Start Program" continues as a 5-year longitudinal study of the development, implementation, and outcomes of comprehensive, coordinated perinatal care systems in the initial 15 Healthy Start demonstration sites. This project, the largest study funded by HRSA, includes an assessment of changes in the health status of pregnant women and infants across the sites. The study is designed to answer four questions: Did the Healthy Start initiative succeed? If so, why? If not, why not? What would be required for a similar intervention to succeed in other settings?

Finally, with the continued and projected emphasis on technology, HRSA is asking whether investment in telemedicine improves the availability and quality of care to underserved populations and provides easier access to continuing education and consultation for providers in isolated settings. "Rural Applications of Telemedicine" is constructing a broad base of knowledge about telemedicine upon which further assessment of the HRSA telemedicine grant program will be built. The four main objectives are (1) to determine the current status of telemedicine in rural health; (2) to explore the effects of telemedicine on access to care, practitioner isolation, and the development of health care networks; (3) to explore the organizational factors that aid or impede the successful development and implementation of telemedicine systems; and (4) to develop, test, and refine data-collection instruments that can be used in subsequent evaluation efforts. Representatives of several other Federal departments and agencies are participating in the conduct of this study.

New Directions for HRSA Evaluation

Major evaluation priorities in fiscal 1996 include managed care; such primary care programs as Community and Migrant Health Centers and the National Health Service Corps; care for mothers and children, exemplified by the Healthy Start initiative; HIV/AIDS services, including new approaches to delivering and financing services through the Special Projects of Regional and National Significance; health professions efforts to foster community-based training for primary care practitioners; and strengthening of the health care infrastructure at the community level, partly through collaboration with States and external organizations such as the American Hospital Association. Underlying the work in all of these topical areas will be continuing efforts to further strengthen HRSA's performance measurement capacity through projects targeted to the needs of particular components and programs and to the integration of performance management approaches agencywide in the context of budget decisionmaking and strategic planning. Finally, HRSA will continue efforts to broaden the dissemination of evaluation results and information about ongoing studies to the public health community and to Congress.

Indian Health Service

MISSION: To provide a quality health services delivery system for American Indians and Alaska Natives with opportunity for maximum tribal involvement in developing and managing programs to meet health needs.

IHS Evaluation Program

The goal of IHS is to raise the health status of its principal beneficiary, American Indians and Alaska Natives to the highest level possible. The importance of evaluation in supporting this goal has increased significantly in recent years. The IHS evaluation process seeks to include American Indians and Alaska Natives as primary stakeholders in defining the purpose, design, and execution of evaluations. Stakeholders are the users of the end product of evaluations and typically are the population or groups most likely to be affected by evaluation findings. The principles of responsive evaluation practice have been adopted by IHS because they can best address these needs and concerns of American Indians and Alaska Natives.

The purpose of the Office of Planning, Evaluation, and Legislation (OPEL) is to advise the Director of IHS on policy formulation; conduct and manage program planning, operations research, program evaluation, health services research, legislative affairs, and programs statistics; develop the long- range program and financial plan for IHS in collaboration with appropriate agency staff; coordinate with HHS, Indian tribes, and organizations on matters that involve planning, evaluation, research, and legislation; and develop and implement long-range goals, objectives, and priorities for all activities related to resource requirements and allocation methodologies and models. OPEL serves as the principal advisory office to IHS on issues of national health policy. It also coordinates these four evaluation functions:

Health Program Evaluations--collect and analyze information useful for assisting IHS officials in determining the need for improving existing programs or creating new programs to address health needs.
Policy Analysis--conduct analyses when a change in the IHS health services delivery system must be considered, when issues emerge in an area where no policy currently exists, and when current policies are perceived as being inappropriate or ineffective.
Health Services Research--undertake analyses of the organization, financing, administration, effects, and other aspects of IHS health services.
Special Studies and Initiatives--conduct studies and prepare special reports required by Congress in response to pending legislation or policies, often using a roundtable whenever an issue or a health problem requires immediate action and it is unclear what type of action should be taken.

OPEL meets part of IHS evaluation needs with two types of short- term studies: policy or program assessments and evaluation studies. The policy study contributes to IHS decisionmaking about budget, legislation, and program modifications and includes background information to support IHS initiatives. Evaluation studies are carried out at the program level, or area offices, and focus on specific program needs.

Annually, OPEL identifies the high-priority health care and health management issues and concerns through the submission of headquarters and area office proposals for assessment or evaluation. IHS area and associate directors are asked to submit proposals for possible areas of evaluation study. These proposals are reviewed and rated by a panel of subject matter experts and evaluation experts. They are also reviewed by IHS staff for more specific concurrence with IHS strategic goals, objectives, and priority areas. The proposals are then prioritized and forwarded to the OPEL Associate Director for review and approval. The Director of IHS reviews the final proposals and decides the respective funding levels.

Summary of Fiscal 1995 IHS Evaluations

During fiscal 1995, OPEL completed four evaluation projects of several major Indian health topics.

"Case Study of Family Violence in Four Native American Communities: Final Report." Family violence on Indian reservations is devastating for individuals, families, and reservation communities. There are many families in American Indian communities who have experienced violent behaviors, who have coped with violent behaviors positively, or who wish to learn more about violent behaviors and their prevention. IHS sponsored this study to produce the information and data needed to guide program planning and development.

"Evaluation of Diabetes Services Provided by IHS Model Diabetes Program: Final Report." This evaluation of the IHS diabetes model projects used data from 634 patient medical records, four focus groups, and 20 informant interviews to describe these projects and examine their effect on two patient health outcomes--blood sugar control and hospitalizations. Two diabetes project sites (Winnebago, Nebraska, and Fort Totten, North Dakota) and one "usual care" site (Rosebud, South Dakota) were selected for the evaluation. Data from the 1993 Diabetes Program Audit were used to assess whether or not the findings from this evaluation of two diabetes projects could generally represent other diabetes team approaches in other IHS areas.

"Evaluation of IHS Midlevel Health Providers: Final Report." IHS must determine its needs for midlevel health providers (MLHPs), such as physician assistants, nurse practitioners, certified nurse midwives, and clinical nurse specialists through the year 2000 and address recruitment and retention of MLHPs. This study resulted in obtaining responses from 119 MLHPs and 14 primary care managers. The survey confirmed that MLHPs are making a major contribution to the IHS primary care program and are well utilized. However, IHS must begin immediately to address the shortfall of MLHPs that is projected to be approximately 51 percent over the next 6 years.

"Phase III Final Report: Child Abuse (CA) and Child Neglect (CN) in American Indian and Alaska Native Communities and the Role of the Indian Health Service." Indian child and adolescent abuse and neglect are issues of widespread concern; however, no reliable statistics exist on the prevalence of abuse or neglect. Recent data have indicated that more than 6,500 referrals for suspected child abuse and neglect were made to the Bureau of Indian Affairs (BIA), reflecting a minimum of 1 percent of Indian children in the BIA service area. This study provided comprehensive assessments of the effectiveness of IHS and tribal policies, procedures, and personnel in recognizing and treating CA/CN, and facilitated the design of an intervention program flexible enough to be used by American Indian and Alaska Native communities across the country.

IHS Evaluation in Progress

During fiscal 1995, IHS funded more than 10 studies, including health program evaluations, policy analysis projects, health services research, and special studies and initiatives. Examples of these projects follow.

"Analyzing the Underreporting of American Indian and Alaska Native Deceased Persons on State Death Certificates, 1986- 1988." The fiscal 1990 phase of the project funded the matching of the IHS patient registration records with State death certificate records. The records of American Indians and Alaska Natives matched with certificates from the National Death Index (NDI) for 1986, 1987, and/or 1988. The output was a computer tape and hard copy printout of probable and possible matched records. The results from the NDI match are used to develop estimates for each IHS area and each Reservation State on the number and proportion of deaths occurring each year that are misreported by race. These data will indicate which States have significant problems in underreporting American Indians and Alaska Natives on death certificates.

"A Mental Health Service Delivery Model for Urban Native Americans: An Evaluation of Utilization Rates and Mental Health Treatment Factors in an Urban Setting." This study will investigate utilization rates for mental health services over a 9-year period in an urban Indian population so that a profile of service use can be assessed over time. The approach will include the use of historical time analysis to examine the patterns of use over a set period within the context of events taking place during the those years and the effect on the population being served. The study will also examine the interaction of a service delivery model with traditional and western approaches within the same operational framework.

"Evaluation of the IHS Adolescent Regional Treatment Centers." This study evaluates the effectiveness of the regional treatment centers (RTCs) that provide alcoholism rehabilitation for American Indian and Alaska Native youth. The study will define the issues facing RTCs and help establish the parameters of what these programs have accomplished and plan to accomplish over the next 10 years. It will also provide guidance on how the success rate of RTCs can be improved.

"Evaluation of the IHS-Supported Substance Abuse Treatment Program for American Indian/Alaska Native Women." This study assesses the current IHS substance abuse and treatment program. It seeks to improve program effectiveness by monitoring behaviors that enhance risk, because research is virtually nonexistent on the scope of the problem of alcohol and other substance use among American Indian and Alaska Native women. The study will also evaluate the effectiveness of treatment.

New Directions for IHS Evaluation

IHS is responding to dramatic changes taking place inside and outside the government. The causes of the many changes include a decrease in the funding level of discretionary Federal programs, greater involvement of tribal governments in the Indian health care system, technological innovations, the changing patterns of disease to more chronic conditions, and the transfer of many Federal programs and resources to individual States.

These changes will affect the IHS evaluation strategy in the coming years. Nevertheless, IHS remains committed to comprehensive community-based, preventive, and culturally sensitive projects that empower tribes and communities to overcome health issues. The Director of IHS has placed increased emphasis on several initiatives to focus attention on specific health areas and to serve as a management tool to prioritize IHS' workload. These initiatives include emphasis on women's health, youth, traditional medicine, and elder care and on establishing working relationships with Federal and State government agencies. The initiatives will undoubtedly affect new directions for evaluation.

Research and evaluation proposals to be considered in upcoming years include the following topics: childhood obesity; elderly wellness; program review of training the practitioner in the assessment and treatment of adolescent sexual perpetrators; impact of the Alaska tribal health compact on programs and services in the Anchorage Service Unit; Pueblo of Zuni end-stage renal disease quality of life; and village-based women's preventive health services delivered by community health aides/practitioners.

National Institutes of Health

MISSION: To sponsor and conduct medical research that leads to better health for all Americans.

NIH Evaluation Program

NIH develops scientific knowledge that leads to improved means to prevent illness, cure disease, and treat disability. It accomplishes its mission by conducting medical research in its own intramural laboratories and supporting research in universities, medical and health professional schools, hospitals, and other health research organizations. NIH fosters the widespread dissemination of the results of medical research, facilitates the training of research investigators, and ensures the viability of the research infrastructure. The NIH Evaluation Program is an integral part of how NIH manages its support and conduct of medical research.

Philosophy and Priorities

NIH evaluation activities assess program performance (efficiency, effectiveness, responsiveness); analyze both quantitative and qualitative results based on those assessments; and use the resulting information in policymaking, strategic planning, budgeting, and program development and management.

The quantity and diversity of diseases, disorders, and biological systems in the NIH portfolio make strategic planning and evaluation at NIH a complex task that is continuously evolving. Priorities are set and research programs are planned and evaluated at two levels: the institutes and centers (ICs), and centrally by the NIH Director, with whom the ultimate responsibility lies for the course of NIH-funded medical research.

This two-level approach ensures that planning and priorities specific to the mission of each IC are fully developed and implemented with a clear vision and within the fiscal constraints set by the IC budget, and that there is central leadership for developing crosscutting initiatives and promoting collaboration among the ICs. The NIH Evaluation Program provides information to assist the NIH Director and IC directors in determining whether NIH goals and objectives are being achieved and to help guide policy development and program direction.

Policies and Operations

A distinguishing feature of the NIH Evaluation Program is the extent to which it employs a variety of evaluation strategies and inputs that include the use of national advisory councils, boards of scientific counselors, consensus development conferences, and ad hoc committees that help chart scientific directions and select the most promising research.

Program evaluations are funded through both a trans-NIH mechanism, the 1 percent set-aside authority, and by individual IC program funds. A two-tier system reviews project requests to use set-aside funding. The first tier involves review and recommendations by the NIH Technical Merit Review Committee (TMRC) on the technical aspects of project proposals and on whether a project fits within HHS guidelines for set-aside funding. The second tier involves the NIH Evaluation Policy Oversight Committee (EPOC) that considers TMRC recommendations and makes final funding recommendations to the NIH Director or designee. It also conducts policy-level concept reviews of proposals for NIH-wide evaluation studies, establishes the overall NIH set-aside budget, and oversees the process. EPOC recommendations are approved by the NIH Director or designee. Concurrently, the ICs fund program evaluations from their budgets; these evaluations are used by various committees, working groups, task forces, workshops, conferences, and symposia for program management and development.

An important characteristic of NIH's Evaluation Program is solicitation from a number of sources to provide information to the NIH Director and the IC directors. Discussions are continuously held with extramural grantees, intramural investigators and other NIH staff, members of Congress and the Administration, and members of the public, including professional societies and voluntary health organizations. These individuals and groups provide valuable input on pressing public health needs, important scientific opportunities, knowledge gaps, and the balance between patient-oriented and laboratory research.

Summary of Fiscal 1995 NIH Evaluations

The evaluations completed in fiscal 1995 addressed a cross- section of the NIH research program, as demonstrated by the following sample of studies.

"Fetal Alcohol Syndrome: Diagnosis, Epidemiology, Prevention, and Treatment" responded to the congressional mandate that a study of fetal alcohol syndrome (FAS) and related birth defects be carried out. A complete description of this study, funded by the National Institute of Alcohol Abuse and Alcoholism, is in chapter II of this report.

"Directions in Nursing Research Training" summarizes information on National Institute of Nursing Research (NINR) (1) funding that permits nurse scientists to follow research career paths; (2) recommendations for nursing research training in a National Research Council report; and (3) the views and recommendations of the scientific community. The report also discusses research training issues in the broad context of Federal and academic perspectives. The report recommends that NINR disseminate the report widely, analyze the research career paths of NINR trainees and fellows to determine the criteria for successful training experiences, emphasize the need to increase the number of nurses with doctoral degrees, and support research consortia and partners.

"Cancer at a Crossroads: A Report to Congress for the Nation" describes the results of an evaluation undertaken at the request of Congress to assess the achievements of the National Cancer Program, identify barriers to reducing the burden of cancer, and recommend future research and program directions. A subcommittee appointed to address these issues concluded that the strongest strategy for a renewed "war on cancer" should (1) apply currently available knowledge about cancer prevention and care to all segments of the population; (2) increase support for transnational research that develops basic cancer knowledge into preventive strategies, new technologies, and effective treatments; and (3) increase support for basic cancer research to ensure the continued flow of new discoveries that lead to better cancer prevention and care.

"Measuring Social Inequalities in Health" provides the results of a National Institute of Child Health Development conference held to develop recommendations for improving or changing measures of socioeconomic gradients in federally supported health data sets. The conferees recommended (1) the availability of existing social class data be publicized, and (2) researchers be encouraged to analyze the data. The conference also provided investigators wishing to apply for NIH funds with improved tools for implementing the revised guidelines for the inclusion of women and minorities in clinical research.

"Report on Outreach Activities of the National Library of Medicine" details a 5-year review of National Library of Medicine (NLM) outreach activities in response to congressional concerns that hospitals and health professionals in isolated areas lack access to recent scientific and biomedical information. The report found that (1) the number of outreach projects had increased from 16 in 1989 to almost 300 at the end of 1994; (2) approximately 30,000 user codes issued in 1989 had increased to nearly 100,000 by 1994; (3) 4 million searches of the NLM databases in 1989 increased to 7 million searches in 1994; and (4) similar changes have occurred in the use of the "Grateful Med" system.

"Support for Bioengineering Research" responded to a request from Congress that the HHS Secretary, acting through the NIH Director, conduct a detailed inventory of sources and amounts of public and private funding for basic bioengineering research in fiscal 1993. The report recommended (1) establishing an Interagency Bioengineering Coordinating Committee; (2) including basic bioengineering research within appropriate intramural programs; and (3) providing, through the Federal Register, a "comment period" notice to suggest research topics for inclusion in the annual Small Business Innovation Research Omnibus solicitation.

NIH Evaluations in Progress

NIH currently supports more than 30 evaluations. They range from small- to large-scale assessments, from evaluative studies to comprehensive evaluations. Examples illustrating this range include the following:

"Evaluation of the National Research Service Award (NRSA) Research Training Program" is the first phase of a longer term evaluation effort directed at examining the extent to which the objectives of NIH/NRSA programs are being met. It is an update and extension of earlier work performed by the National Academy of Sciences published in a 1984 report titled "Career Achievements of NIH Predoctoral Trainees and Fellows." The study will address the major evaluation questions frequently posed by constituencies of NIH research training; make efficient use of extant data relevant to these evaluation questions; and identify gaps in existing databases, thus guiding the development of the data-collection efforts planned for the second and third stages of the evaluation.

"Implementation Phase for an Evaluation of the Minority Access to Research Careers (MARC) Honors Undergraduate Research Training Program" will evaluate the degree to which the intended goals of the MARC undergraduate program are being met and will result in the establishment of a general information database on the program and its trainees. Information is being collected on the programs implemented at MARC institutions, the trainees, the training experience, and former trainees' subsequent education and career paths.

"Evaluation of NIH Implementation of Section 491 of the PHS Act Mandating a Program of Protection of Research Subjects" is examining NIH's implementation of a program to ensure adequate protection to all individuals who are used as research subjects. The study is attempting (1) to determine the impact of the program on the administrative operations and research activities of awardee institutions and (2) to examine information on some financial costs, administrative burden, levels of effort, and selected program measures related to the protection of human subjects, the education of researchers, and the facilitation of research.

The Task Force on Genetic Testing is examining issues surrounding the use and regulation of genetic tests. The objectives of the task force are to review the state of the art of genetic testing; examine the strengths and weaknesses of current practices and policies relating to testing; and, if needed, recommend changes or policy options to ensure that the public is protected so that only the appropriate tests are done by qualified laboratories.

The study, "Methodology To Assess the Impact of National Heart, Lung and Blood Institute (NHLBI) Research," is examining alternative measures to demonstrate the impact of basic research in terms of reductions in disease mortality and morbidity. The objectives of this study are to introduce the concept of examining patients as an additional measure for assessing the impact of NIH-supported research; to develop a methodology to use patients in evaluation research; and to apply the methodology to an NHLBI project to compare the outcomes for individual research project grants funded under requests for applications with individual research grants funded from unsolicited applications.

The study, "Characterization of User Population and User Satisfaction of the Physician Data Query (PDQ) Database," is being conducted to obtain information on PDQ database users and their satisfaction with the comprehensive cancer database. The objectives of the study are to survey PDQ database users to characterize who is using the database, and to determine how the information is used and if users are satisfied with the information they receive and with the method of retrieval (e.g., CD-ROM, online, or hard copy).

The "Evaluation of Laboratory Animal Use, Facilities, and Resources" will acquire, analyze, aggregate, and report objective data for fiscal 1993 on the number and characteristics of animals used and on animal-related facilities and resources at those institutions and organizations that have animal welfare assurances currently on file with the NIH Office of Protection From Research Risks.

The "Survey of Academic Research Equipment" responds to a congressional directive under Public Law 96-44, Section 7, that the National Science Foundation, in cooperation with NIH, conducts an ongoing instrumentation survey and issue biennial reports. The purpose of the study is to assess national trends in medical research instrumentation.

New Directions for NIH Evaluation

NIH will continue to engage in a wide variety of evaluation activities. Priorities for future evaluation activities reflect input from a number of sources: the HHS Strategic Plan, the NIH Strategic Plan, and the NIH GPRA Performance Plan--all in the developmental phase; the NIH Director's Areas of Emphasis; the Administration's High Priority Areas; recommendations of NIH/IC directors; and recommendations of the Evaluation Policy and Oversight Committee. As a result, NIH will give priority funding to the following areas:

Medical Research. This area focuses on issues such as the economic impact of biomedical research, clinical research including the impact of managed care, the health of special populations, and the development of measures for medical research accomplishments.
Information Dissemination. This area focuses on issues such as expanded data collections to address emerging priorities, the health information dissemination infrastructure and strategy, and technology transfer.
Research Training. This area focuses on issues involving national needs for medical research personnel, the quality of NIH training programs, and diversity among trainees.
Research Infrastructure. This area focuses on issues involving the maintenance and construction of buildings and facilities, computers in biomedical research and medicine, and informatics in medical research.

NIH evaluation studies provide a rational basis for managerial decisionmaking and responding to public concerns for accountability in government. In addition, it is through such studies that NIH is able to determine its progress in meeting scientific objectives, strengthening research and administrative activities, and contributing to its mission of sponsoring and conducting research that leads to better health for all Americans.

Office of the Assistant Secretary for Planning and Evaluation

MISSION: To provide analytical support and advice to the Secretary on policy development and assist the Secretary with the development and coordination of HHS-wide program planning and evaluation activities.

ASPE Evaluation Program

ASPE functions as a principal advisor to the Secretary on policy development and conducts a variety of evaluation and policy research studies on issues of national importance. ASPE is also responsible for HHS-wide coordination of legislative, planning, and evaluation activities. In its evaluation coordination role, ASPE does the following:

Provides annual guidance to all HHS agencies and staff offices about evaluation priorities, procedures, and review requirements.
Reviews evaluation priorities proposed by HHS agencies, providing advice about the focus or method of proposed projects and identifying opportunities for collaboration and effective use of resources.
Prepares planning and summary reports on evaluation activities as required by Congress.

Through the departmental evaluation planning process, ASPE has the capacity to identify crosscutting program or policy issues of particular concern to the Secretary and specific program and policy areas not covered by HHS agency evaluation plans. In these instances, ASPE initiates evaluations or collaborates with the agencies to conduct evaluations or policy assessments. For example, in recent years ASPE has initiated projects to develop cost estimates for health financing issues in general and specifically for Medicare and Medicaid programs; the effects of managed care expansion on public health infrastructure; welfare- to-work approaches; long-term care alternatives; and studies evaluating alternative services for children at risk of harm from drug abuse, crime, abuse, and other pathologies.

Another continuing ASPE evaluation objective is to support and promote the development and improvement of databases that HHS agencies and ASPE use to evaluate health care programs and health trends. ASPE provides support to the HHS Data Council charged with integrating key national surveys, such as linking health status indicators with indicators of well-being. HHS needs more comprehensive data sources to assess the anticipated transformation in health and human services.

Finally, ASPE uses evaluation funds to promote the effective use of evaluation-generated information in program management and policymaking. The latter is accomplished through the dissemination of evaluation findings and other activities such as providing technical assistance to agencies in the development of performance measures.

Summary of Fiscal 1995 ASPE Evaluations

During 1995, ASPE completed 28 studies and reports on a number of issues that provided information useful to the Secretary and HHS divisions for purposes of program planning and budget and legislative development. A description of some of those completed studies that have broad potential application follow.

Adolescent Sexuality and Parenthood

Several studies were aimed at providing the Department with an expanded understanding of the causes, impacts, and possible ways to address the pervasive national triad of adolescent sexual activity, pregnancy, and teen parenthood. One study examined prevention and intervention strategies designed to reduce adolescent pregnancy and parenthood. A second report summarized recent research on adolescent sex, contraception, and childbearing. The report summarized the factors that lead to teenage childbearing. The varied antecedents of sexual activity include biological factors, race and gender, family characteristics, use of alcohol or drugs, and other behavioral factors. The studies documented the increased rate of sexual intercourse among teens. The earlier the age of sexual initiation, the more likely the experience is coercive and the more likely the teen will become pregnant or contract a sexually transmitted disease (STD). Another study analyzing the increases in nonmarital births found that one-third of all births occur outside of marriage, with teenagers accounting for about one-half of all nonmarital births; and a smaller proportion of nonmarital pregnancies are ended in abortion. Most parents in nonmarital births are disadvantaged before the birth.

Health Information

As the pressure on expanding data-collection systems continues and the ability to store, manipulate, and transmit such growing amounts of information accelerate, it becomes more important to protect the privacy of this information. ASPE supported a task force on the privacy of private sector health records. The task force examined the extent of the problems with the collection, storage, and use of health information in the private sector. The task force also examined the social, legal, and economic issues affecting the privacy of people who use the health care system. The report recommended a coordinated Federal policy on medical records, confidentiality, universal identifiers, effective security standards and guidance, the establishment of a data- protection entity, and an education program about the issue.

Substance Abuse

Substance abuse problems blight the lives of individuals served by a number of HHS programs. It is therefore important to increase understanding of the etiology of substance abuse and of effective modes of intervention. One study looked at patterns of substance use and substance-related impairment among recipients of Aid for Families with Dependent Children (AFDC). It found that while AFDC recipients have higher rates of substance use and substance-related impairments than the general population, the vast majority of substance users and impaired people do not receive AFDC. A second study of substance abuse among women and parents also examined the number of children potentially at risk because of parental drug abuse. The study also found that children of drug-using parents tend to be younger than children overall and women and men with young children in their homes tend to report one-half as much illicit drug use as those with no young children at home.

Transition to Work

Another area of significant concern and interest is the movement of individuals from various dependent arrangements, including welfare, into employment. Three complementary studies were carried out to learn more about barriers to and models for facilitating transition of disabled individuals to work.

One study reviewed literature on barriers and incentives to improving the labor force participation of persons with significant disabilities. The second study evaluated the methods used in transition-to-work demonstration projects aimed at helping schools and adult providers obtain integrated employment, using natural supports for students with very severe disabilities. The report details the barriers encountered by the demonstration sites, such as difficulty locating jobs, difficult funding procedures, transportation problems, and staff turnover.

A third study examined approaches States have taken to move significant numbers of welfare recipients into work-related activities, with the goal of identifying operational lessons for program administrators and policymakers. The report documented strategies States used to increase participation in work, the role of child care in achieving this objective, and how States changed the culture of welfare to have a stronger focus on employment.

ASPE Evaluations in Progress

ASPE currently has 26 studies in progress on a broad range of program and policy areas. The major areas of study include the following:

Assisted Living

ASPE has commissioned a national study of assisted living, or the residential settings that combine adapted housing, assistive technologies, personal assistance, and other supportive services for persons with disabilities. Assisted living is considered an important component of services to the increasingly aging American population and an effective response to the rising costs of nursing home care. The study will examine the role of assisted living from the perspective of consumers, owners and operators, workers, regulators, developers, investors, and others with a stake in the long-term care system.

Board and Care Homes

As many as 1 million mostly elderly or disabled individuals are dependent on personal services and supervision provided by board and care homes. To examine one significant way in which public agencies seek to influence the care provided in these facilities, ASPE is carrying out an analysis of the effect of regulations on the quality of care provided in board and care homes. In addition to raising concerns about the unwillingness or inability of most homes to meet changing resident needs and inadequate staffing ratios, the study's preliminary findings show that extensive regulation did reduce the use of psychotropic drugs and increases operator training and the availability of social aids and supportive devices. However, regulation does not affect operators' requirements for staff training, the availability of licensed nurses, or the cleanliness of the home.

Domestic Violence

Two studies are looking at the costs of domestic violence to the health care system and the domestic violence policy and programs of selected communities. Together, the studies will provide information about the economic consequences of costs of domestic violence to the health care system and will develop an economic model for determining these costs. The studies will also examine how selected communities have built community-based comprehensive family violence programs.

Evaluation of Family Preservation Services

ASPE and the Administration for Children and Families (ACF) are evaluating selected family preservation programs, including placement prevention services aimed at preventing children from entering substitute care, broader family preservation services that may be less intensive and of longer duration than placement prevention services, and reunification services to speed the return of children to their homes after entering substitute care. Measures of program success will include reduced placements of children into substitute care (for preplacement services), successful reunification (for reunification services), improved child psychological well-being, improved child behavior, improved family functioning, and reduced recurrences of child abuse and neglect.

Health Care Technology Assessment

ASPE is preparing current information on the demand for technology assessment. The study will examine the performers of technological assessment, the methods of conducting assessments, the uses of the results, and the unmet needs that might be met by further cooperation between the public and private sectors of health care.

Information Used by Physicians

The purpose of this study is to understand the relationship between the increasing amounts of information that are being produced for physicians and the types of information that physicians actually use. In particular, this study is assessing physician use of available computerized health and medical information sources, such as those on the World Wide Web and the Internet, and how access to such information influences medical practice.

Long-Term Care

It is difficult to estimate how often people who enter nursing homes down their assets and become eligible for Medicaid. Because some elderly people enter nursing homes more than once, a longitudinal study to help provide this information is being conducted.

Managed Care and People With Disabilities

Managed care can provide unique opportunities but also has potential pitfalls for people with disabilities. Research is under way to determine the impact of managed care on access, quality, and satisfaction for people of all ages who experience disabilities. Of particular interest are the experiences of children with disabilities in managed care, best practices in managed care for the disabled, and the impact of extending Medicaid managed care to the disabled population eligible for Supplemental Security Income (SSI).

Moving Welfare Recipients to Work

HHS, with ASPE's assistance, is conducting a comprehensive, multiyear study of the government's principal program for moving people off welfare and into employment--the Job Opportunities and Basic Skills Training (JOBS) program. The evaluation addresses the long-term effects of different welfare reform approaches including whether any approach significantly improves the economic circumstances of the most disadvantaged recipients, the effects of welfare-to-work programs on the children of welfare recipients, and the cost effectiveness of different approaches.

Performance Data

The Federal grants relationships to States in public health are evolving into outcomes-based performance management. A health outcomes-based monitoring approach requires data system development, and several projects are being conducted for that purpose. One project, with the National Academy of Sciences, will work on identifying which results of a performance-based approach can be measured at the State and Federal level and will recommend specific steps that can be taken to improve these measurement capabilities. A related project will develop approaches to obtaining comprehensive baseline and trend data on public health infrastructure. Finally, a third study, building on a Robert Wood Johnson Foundation project, will assess the quality of data that States collect and determine whether a network can be established to share health data among the States and thus improve the health policy decisions they make.

Pharmaceutical Marketplace

The growing influence of managed care has had a profound influence on the pharmaceutical marketplace and has highlighted a number of concerns among pharmaceutical companies, third-party payers, and the Federal and State governments. ASPE is engaged in a project to develop a framework to assess the impact of managed care on the pharmaceutical marketplace, as well as on consumer access to newly developed drugs.

Promoting Father Involvement

Three studies are looking at various aspects of father involvement, especially among fathers who are not living with their children. One study examines the relationship between various child custody, visitation, and support payment patterns and the effect that payment and contact have on child well-being. The second is developing a design for the evaluation of community-based programs for vulnerable fathers. Although programs to increase and enhance father involvement exist in some communities, very few have been evaluated. The third study will develop a theoretical framework to assist in understanding and implementing programs and activities to promote responsible fathering. The last two projects are being conducted in collaboration with ACF.

Subacute Care--Market Analysis, Cost, and Quality

Subacute care is considered a cost-effective alternative to acute nursing home care services, that is, it can be provided in lower cost settings with no diminution of quality. ASPE has commissioned a study to identify and understand the definitions of subacute care; examine the provision of subacute care in select market areas by a variety of providers (e.g., hospital- based and freestanding skilled nursing facilities, rehabilitation and long-term care hospitals, and home health agencies); evaluate policy issues about who receives, provides, and pays for subacute care; and assess the cost, quality, and cost-effectiveness of this type of care.

Trends in the Well-being of America's Children and Youth

In fiscal 1995, ASPE provided funds for creating the first annual report on the health and well-being of America's children. The report will be a single volume that shows up-to-date trends on how our Nation's children are faring, pulling together information on all available national trends in the lives of children and youth. The volume will fill a crucial gap and will become an annual series to keep the Nation apprised of the well- being of its children and youth.

New Directions for ASPE Evaluation

Extensive new legislation, enacted during the current 104th Congress, will lead to the consideration of a range of new evaluations. First, understanding the effects of profound structural changes taking place in health care will be a majority priority for ASPE evaluation activity. The objective is to understand the changes on health status of families and individuals, access to quality health care, patterns of health care utilization and spending. Included is the need to evaluate the impact of Medical Savings Accounts on spending and savings and the connection between levels of income and amounts of savings under this option. A continuing and important evaluation objective is to support and promote the development and improvement of databases that ASPE and others can use to conduct evaluations of health care and human services programs and health and social trends.

Second, the welfare reform legislation also will require a varied evaluation response in order to assess the success of meeting the objectives of this legislation as well as to gather effective objective information on impacts on current and future recipients. It will also be critical to document and assess the impact of the recently enacted welfare legislation. The evaluation activities will focus on understanding the operation and organization of the new welfare system and assessing the impact of the changes on low-income families and children.

In other areas, ASPE will examine long-term initiatives that focus on the development and implementation of systems of acute care, subacute care, assisted living, long-term care, and personal assistance services for people with disabilities. ASPE will continue to develop effective indicators of the well-being of children in order to measure changes in the conditions of our children. ASPE will support the Departments' efforts to develop outcomes-based performance measures for health and social service programs, intended to provide the framework for new types of grant relationships with States.

Finally, one of ASPE's continuing and important evaluation objectives is to support and promote the development and improvement of databases that ASPE and others can use to conduct evaluations of health care and human services programs and health and social trends. All of the above activities will be supported with a combination of program appropriations, policy research, and evaluation funds.

Office of Public Health and Science

MISSION: To provide advice on public health and science to the Secretary of HHS, executive direction to program offices within OPHS, and, at the direction of the Secretary, coordinate crosscutting public health and science initiatives in the Department.

OPHS Evaluation Program

OPHS provides advice, policy and program coordination, and leadership in the implementation, management, and development of activities related to public health and science, as directed by the Secretary of HHS. OPHS provides advice to ensure that the Department conducts broad-based public health assessments designed to better define public health problems and solutions to those problems. It assists other components of HHS in anticipating future public health issues and problems and provides assistance to ensure that HHS designs and implements appropriate approaches, interventions, and evaluations that will maintain, sustain, and improve the health of the Nation. OPHS provides a focus for leadership on matters including recommendations for policy on population-based public health and science and, at the Secretary's direction, leads and coordinates initiatives that cut across agencies and operating divisions. In addition, OPHS communicates and interacts, on behalf of the Secretary, with national professional and constituency organizations on matters of public health and science.

In keeping with its newly defined role within HHS, OPHS has specifically sought to develop an evaluation plan that avoids duplication of efforts that might more appropriately and effectively be undertaken by the operating divisions of the Department or by the Office of the Assistant Secretary for Planning and Evaluation. Thus, the direction of the fiscal 1996 evaluation strategy for OPHS is toward public health and science issues that cut across multiple interests of the operating divisions and that require a coordinated approach to achieve the most effective results. In addition, OPHS commits itself to carrying out every project proposed as part of this year's strategy in collaboration with relevant operating divisions.

OPHS specifically conducts evaluations requested of the Secretary by Congress that are most appropriately managed by staff with medical or health sciences credentials and that cannot be assigned to an operating division. Also, evaluations are conducted to support the Assistant Secretary for Health's role as senior advisor to the Secretary on matters of public health and science. This role includes the responsibility to "assist the Secretary in developing a policy agenda for the Department to address major population-based public health, prevention and science issues" and to provide "leadership and a focus for coordination of population-based health, clinical preventive services and science initiatives that cut across operating divisions." In addition, OPHS conducts evaluations specific to the needs of the program offices located within OPHS, such as Women's Health, Minority Health, Disease Prevention/Health Promotion, International Health, and Emergency Preparedness.

Summary of Fiscal 1995 OPHS Evaluations

In fiscal 1995, OPHS's predecessor, the Office of the Assistant Secretary for Health (OASH), completed 12 evaluations in an effort to better inform policy decisions throughout the Public Health Service (PHS) as the health care sector went through a period of rapid change. Studies were in diverse areas of public health in an effort to strengthen the public health infrastructure, including primary health care, HIV/AIDS, work force training, women's health, and substance abuse. OASH efforts focused on issues that cut across PHS program areas, as follows.

In an effort to study and learn from past experiences, one study, performed by the Institute of Medicine (IoM), analyzed crisis decisionmaking by the Food and Drug Administration, the Centers for Disease Control and Prevention, and the National Institutes of Health in response to a threat to the blood supply that emerged in the early 1980s--HIV. These Federal entities, as well as the plasma fractionation industry, community blood banks, and other groups, have responsibilities to protect the supply of blood from communicable disease. The report concludes that the system did not deal well with blood safety issues, that strong leadership is required to counteract legal and competitive concerns that may inhibit effective agency action when a crisis is not well defined, and that agencies need to formulate a systematic approach to advisory committees and should not rely on the entities they regulate for data analysis or modeling. These lessons are being used by the Assistant Secretary for Health in a cross-departmental review on blood safety.

Another project, also performed by the IoM, examined current HHS standards for methadone programs and evaluated the effects of Federal regulations on the provision of methadone treatment services. Unlike other controlled substances, methadone use is controlled by a multitiered system of regulations, and Federal oversight of methadone programs is provided by FDA, the Drug Enforcement Administration, the National Institute on Drug Abuse, and the Substance Abuse and Mental Health Services Administration. The IoM report examines the underlying ideology of methadone regulations that societal risks of methadone outweigh its benefits to such an extent that its use must be extraordinarily regulated. The IoM committee concludes such an approach is not valid in the current environment, and the scope of Federal regulation should be scaled back to untie clinicians' hands in determining the best course of a patient's treatment.

A third report examined the clinical background for trials involving the experimental drug fialuridine (FIAU) and its parent drug fiacitabine to determine whether any rules governing the trial process should be changed, and if so, what burdens or costs these changes might place on future clinical trials. The study followed up on an NIH clinical drug trial in which several hepatitis B virus (HBV) outpatients who had been administered FIAU were hospitalized with liver failure; five patients died and two were saved only by liver transplantation. The report finds that the entire set of trials reviewed was an ethically sound clinical research project designed and carried out by highly competent investigators who frequently exceeded regulation requirements imposed by institutional review boards. The report discusses conclusions and recommendations issued by FDA and concurred with them without ascribing blame for the tragedy to any party.

Among projects in the Office of Disease Prevention and Health Promotion (ODPHP), one report obtained baseline data on the disease-prevention activities of 1,310 nurse practitioners as primary care providers and compared the findings to target goals from 17 objectives of Healthy People 2000: National Health Promotion and Disease Prevention Objectives. The 892 respondent reported spending 50 percent or more of their time providing primary care. In the areas of assessment services, screening and immunizations, cancer-screening services, obstetrics/gynecology, and family medicine, nurse practitioners generally met or exceeded the target percentages of the Healthy People 2000 objectives for selected services.

In an effort to document the relative investment in public health in the United States, nine States participated in another ODPHP project to develop a tool for collecting Federal, State, and local public health expenditures using the essential services of public health terminology. Data-collection instruments were distributed to State health officials who, in turn, coordinated the data collection for State and local public health, substance abuse, and environmental agencies. Mental health data collection was coordinated through the National Association of State Mental Health Program Directors. The report is intended to lay important groundwork for tracking these expenditures over time and across various agencies and levels of government.

An evaluation by the Office of Minority Health (OMH) was designed to document, describe, and assess the effectiveness of community- based projects that address HIV/AIDS prevention within minority communities. The study compares various community-based approaches to disease prevention in nine projects and will be used by OMH and other agencies to identify successful practices and common themes to apply as innovative community-based risk- reduction strategies when targeting minority communities and at- risk populations. The study found that viable linkages with community-based providers and social service agencies are critical to the provision of effective, coordinated, and comprehensive HIV/AIDS services in the community; that projects that build upon existing AIDS activities are better able to influence the local health care and social service agencies and are more likely to continue after Federal funding ends; and that projects must conduct and document internal formative/process evaluations.

Four OASH evaluation projects were concerned with the operation of programs in the PHS regions. One project evaluated the impact of factors that impede the implementation of breast and reproductive cancer prevention efforts at nine Community and Migrant Health Centers (C/MHCs) located in California. In reviewing screening, referral, diagnosis, treatment, and case management protocols, the report finds that the majority of C/MHCs have weaknesses in the areas of community education and outreach; designation of staff to perform specific duties; attention to ethnic or culturally specific barriers to health care; and overall lack of understanding about the importance of protocols. To correct these weaknesses, a Primary Care Effectiveness Review has been scheduled for each center.

After identifying a gap in reproductive health care information within the C/MHC setting, an interagency task force in PHS Region IX began an evaluation project to strengthen the regional public health information infrastructure. The study focused on a representative sample of women 20 to 44 years of age who received reproductive health care services from 1 of 36 study sites. Using a 220-item data-collection instrument designed to collect and analyze the frequency and extent of services provided, the study gives regional programs an information base that will help improve quality of care by disclosing demographic and utilization patterns of the typical client. The system provides immediate feedback on quality of care through documentation analysis; for example, preventive health screenings are highly documented, while services related to the report of abuse and violence are the most poorly documented. Also, prenatal documentation tends to be complete, and documentation of health promotion education and counseling appears low.

Copies of the PHS Region IX report were sent to all C/MHCs involved in the study and to executive and clinical staff in the region. Technical assistance in the areas of violence and abuse, reproductive health, and depression is being made available to the centers as part of a continuous quality-improvement plan. In addition, forms are being redesigned to improve staff use. A domestic violence task force has been organized in the region to address the issues raised in the report.

A report from Region IX assesses HBV education and outreach efforts among primary care centers serving Asians and Pacific Islanders, who constitute more than 50 percent of the infectious HBV carriers in the United States. The report summarizes programs at five centers, which completed and returned a self-assessment tool. It finds that program staffing is the strongest component of HBV education and outreach programs, and media-based outreach is the weakest. The report recommends sensitizing staff to cultural beliefs and attitudes impacting on HBV, using translated health education materials, integrating HBV activities with standard patient care protocols, using ethnic media, and developing linkages with school-based health programs and other community-based programs that help high-risk Asian/Pacific Islanders overcome the barriers identified by the study. The centers are now working to incorporate these findings into ongoing staff training and development.

From surveys mailed to nine C/MHCs in Region VI, another project assessed the level of interest in primary care, practice-based research. The report finds that significant barriers to research include lack of research skills and expertise, too many other clinical responsibilities, and lack of access to research consultation. The report recommended further evaluation of PHS and non-PHS clinic sites to determine topics of interest; encouragement of collaborative arrangements with academic researchers; and, where possible, the use of electronic forums for clinicians. As a result, the Region VI office convened a primary care research meeting in San Antonio to provide clinicians with an opportunity to discuss research interests and provided technical assistance to clinicians in accessing electronic bulletin boards as a means of acquiring research information.

Two studies were completed demonstrating the impact of the rapidly changing health care delivery system. One project evaluated responses to a Federal Register notice soliciting comments on the extent to which competitive health plans contract with academic medical centers (AMCs). The objective was to obtain insight into the potential impact of further growth in managed care organizations (MCOs) on the viability of AMCs and their teaching hospitals and to determine what additional research would be useful to further explore these issues. The study finds that the growth of managed care enrollment will change the mix of patients admitted to AMCs, thus affecting training opportunities for resident physicians and other health professionals. Both AMCs and MCOs suggest that the costs of teaching and research be separated from patient care costs and funded through separate mechanisms.

Another project analyzed issues raised by proposals for improving the supply, training, and distribution of primary care providers. Many proposals considered by Congress included provisions that would have altered the mode of funding graduate medical education, thus changing the numbers and types of new physicians. The report includes a chart book on the supply, training, and distribution of physicians; background on the need for an increased number of generalist physicians; and estimates of the number of advanced practice nurses and physicians' assistants that would be needed if the number of physician residents were reduced.

OPHS Evaluations in Progress

In reviewing evaluation projects for 1996, OPHS considered the following priority areas: congressional mandates, cross- departmental initiatives, OPHS strategic plan-related initiatives, program improvements, and the Government Performance and Results Act. OPHS selected 20 program and policy evaluations for funding for 1996, which are described below.

The Commission on Dietary Supplement Labels, appointed by the President, is evaluating factors relevant to possible FDA regulation of labels for dietary supplement regarding health claims. This project was requested by Congress.

An evaluation of the Cooperative Agreements for Demonstration Projects for Capacity Building at Historically Black Colleges and Universities (HBCUs) is under way. This cross-departmental initiative will examine the extent to which the overall program and individual projects have resulted in the establishment of offices of sponsored programs and the adoption of uniform processes. The evaluation will also assess how well the program has achieved increased funding for health-related research and training at the institutions and how effectively it has enhanced current research, training, and services.

The OPHS evaluation priorities related to strategic planning initiatives will focus on several areas. First, the organization of the Healthy People 2000 objectives reflects categorical funding streams and the concern of special interests in the field of health. To promote an integrated public health message, OPHS will reconsider the basic framework of national health goals and objectives in preparing for the Healthy People 2010 plan. The study will collect insights on the current objectives framework from the Healthy People 2000 consortium members, State health agencies, managed care industry representatives, and major Fortune 500 purchasers of health care plans.

School health programs, both comprehensive and categorical, continue to be developed and implemented throughout the Nation. OPHS will fund a study to identify evaluations of school health programs and make available an updated compilation of methodologically sound studies and their findings to assist school boards, administrators, health personnel, and health educators trying to maximize limited resources for effective health programs.

OPHS will continue to expand efforts to improve nationally available data on public health infrastructure, especially focusing on local public health capacity. In addition, funds will be used to support a similar effort conducted by several pilot States to categorize funding for population-based health activities, as distinct from funding of personal clinical services. The results will provide local health officials and planners with a validated instrument for estimating the level of support for essential public health functions.

In the area of health data and the environment and in continuing support for public health infrastructure revitalization, OPHS will fund development of two data-tracking systems: one related to sentinel public health indicators, related health outcomes, levels of health risks, and the health protection infrastructure, and the second on national and State-level data concerning environmental health outcomes and risks. OPHS will also fund an assessment of health plan involvement in community-based initiatives and a survey of clinicians to assess the level of provision of and the importance placed on preventive clinical services by primary care providers.

OPHS will support several projects to help improve program operations and management. These include a study of management alternatives for emergency preparedness and response, an evaluation of a new model of coordinating and integrating HIV- prevention messages and primary care in high-risk populations, an evaluation of the Minority Health Resource Center, and the assessment of the Adolescent Family Life Program.

OPHS will continue support for the work of the Panel on Cost- Effectiveness in Health and Medicine, a non-Federal expert panel appointed in 1993, which will publish its report this year. The report will be discussed at a conference to explore the panel's recommendations and conduct workshops on the application of cost- effectiveness analysis for specific medical and public health applications. OPHS will also provide support through the National Academy of Sciences for a critical reassessment and revision of dietary reference intakes (recommended dietary allowances).

The implementation of GPRA is a priority for OPHS. The development of performance measures for public health programs will be crucial for program planning, budgeting, and legislative development. OPHS will target efforts in three program areas to develop these measures. First, it will support completion of the consultation begun in fiscal 1995 with State and local health officials concerning the development of performance measurement systems that can be used to improve accountability for expenditure of Federal funds. OPHS will help States and local governments determine the impact Federal programs are having on improving the health status of Americans. OPHS will also discern the extent to which States and major American cities have current data available to measure their own year 2000 objectives and selected national objectives. Finally, the Office of Minority Health will examine its Bilingual/Bicultural Service Demonstration Grant Program, which intends to build the capacity of community-based organizations to address access to health services for limited English-speaking minority populations.

New Directions for OPHS Evaluation

The next few years will be a time of enormous transformation for this Nation's health care system. The need to manage skyrocketing health care costs and rein in the Federal budget deficit--while trying to address the health needs of more than 40 million uninsured Americans and a general population that is becoming older as well as more linguistically and culturally diverse-- presents enormous challenges to HHS, OPHS, and their partners in the public health community.

The shifting emphasis on managed care presents new opportunities coupled with new dilemmas about how best to ensure that, in the quest to manage the cost of care, overall quality of care is not unduly compromised. It raises new questions of how to ensure a work force that is properly trained and fully capable of performing the essential services of public health for increasingly diverse populations in this increasingly dynamic health care environment. Furthermore, transformations in the health care and public health arena necessitate monitoring and assessing (and the data systems to do so) effects and impacts of these changes on the financing, organization, and availability of population- and clinic-based preventive services and medical treatment.

OPHS has given highest priority in its fiscal 1997 strategic plan and evaluation efforts to addressing these issues. The plan emphasizes the need for a strong foundation for public health in the 21st century. The priorities focus attention on (1) the information systems and work force that constitute the Nation's public health infrastructure and that are necessary to effectively provide the essential services of public health; and (2) the impact of managed care arrangements on the resources available for this infrastructure and on the health of all Americans, especially those most vulnerable.

Future OPHS evaluation planning efforts will reflect the OPHS strategic plan as well as the broader HHS programmatic priorities related to improving the health of all Americans and assessing the effects of health sector transformation. Planned investigations in support of the public health infrastructure include analyses of existing and needed data systems and data to adequately monitor health status and address potential threats to the public's health before they become actual threats; better assessments of public health expenditures at the Federal, State, and local level; and examinations of policies and programs that impact population-based services in the States and communities. Efforts related to managed care include evaluations of service delivery to Medicaid managed care beneficiaries; identification of deficiencies in the "safety net" (e.g., the uninsured) as a result of managed care; an assessment of outreach efforts to racial/ethnic minority populations in the implementation of Medicaid managed care; and collection of baseline data on the extent to which managed care organizations and State and local health agencies with linguistically and culturally diverse populations in their service areas are able to provide services.

Other planned activities will focus on addressing specific health needs of the most vulnerable populations. These include evaluations of successful models for overcoming system barriers to better integration of prevention and treatment services for populations at high risk of HIV infection and those with HIV disease; the effectiveness of demonstration projects intended to promote abstinence among teens who are not sexually active and a project to improve parenting skills among pregnant or parenting teens; guidelines and tools to enhance the development and assessment of health information and materials for linguistically and culturally diverse audiences; the impact of HBCUs consortium activities on reducing family and domestic violence; and the research base for post-traumatic stress disorders in women, children, and minorities.

Finally, in response to the increased need to measure the performance and effects of public health programs and activities, OPHS will support efforts to integrate national Healthy People 2000 objectives and performance measures in grants with program and department performance plans; develop and implement measures that assess the capacity of State and local health agencies to provide essential public health services and to meet OPHS goals; and ensure expansion of the public health knowledge base through scientific investigations in the behavioral and social sciences, preventive medicine, public health practice, nutrition, environmental and occupational health, and health systems and services.

Substance Abuse and Mental Health Services Administration

MISSION: To improve the quality and availability of prevention, treatment, and rehabilitation services for substance abuse and mental illness.

SAMHSA Evaluation Program

SAMHSA is committed to evaluating its overall programs and individual grant projects in order to assess the effectiveness of prevention, treatment, and rehabilitation approaches and systems of care; the accountability of Federal funds; and the achievement of SAMHSA's programmatic and policy objectives.

To the greatest extent appropriate and feasible, SAMHSA will encourage the use of comparable data elements and instruments across its evaluations in pursuit of a comprehensive evaluation system and to minimize respondent burden.

SAMHSA conducts grant programs under a variety of legislative authorities. These authorities can generally be grouped into two types: services and demonstrations. The evaluation required for a particular grant program is dependent on the type and purpose of the program. SAMHSA will evaluate each of its service programs to provide information to program managers about the accountability of Federal funds. In addition, the evaluations of demonstration programs will generate new knowledge to lead the field in developing policies that improve services. Program and evaluation staff must work together to identify clearly the questions or goals each grant program will address and to propose appropriate evaluation strategies.

The two types of grant programs represent two facets of SAMHSA's mission: service delivery and knowledge development. SAMHSA's leadership in the field is dependent upon the successful interaction of these two facets. Through evaluation, SAMHSA must identify effective approaches to prevention, treatment, and rehabilitation. Through service delivery funds, SAMHSA must provide incentives to the field to implement effective approaches. Major emphases of SAMHSA's mission are the development, identification, and dissemination of effective strategies and systems for prevention, treatment, and rehabilitation.

SAMHSA is now implementing a new integrated model of evaluation and planning. Strategic planning will identify priorities, such as managed care, that drive the development of grant programs and evaluations. In compliance with the Government Performance and Results Act, SAMHSA is attempting to improve performance by identifying performance goals as part of its strategic planning process. The formulation of programmatic and evaluation priorities will include consultation with the SAMHSA and Center Advisory Councils, and with other experts in the fields of evaluation and service delivery. Early and continuous coordination of program planning and evaluation design will result in the articulation of program objectives that can be evaluated. Evaluations will show how well the overall grant programs have achieved their objectives. SAMHSA will translate these results into information that can be used for program and policy development. The strategic planning and policy development processes will then use the results to refine SAMHSA's priorities and objectives.

This evaluation policy will help SAMHSA achieve its goal of continually informing policy and program development with knowledge culled from past performance. In this way SAMHSA can best serve its customers by enhancing the quality of public substance abuse and mental health services.

In compliance with the PHS guidelines for the technical review of evaluations, SAMHSA has established a standing committee of PHS staff who are evaluation specialists. Representatives of the Office of the Assistant Secretary for Planning and Evaluation serve as ex officio members of the committee. The SAMHSA evaluation officer is the committee chair. The committee does not generally review the evaluation proposals of individual grantees. It does review proposals for broader, more comprehensive evaluations, such as the cross-project evaluations of grant programs.

Evaluation project proposals are generally prepared by SAMHSA program staff in the various centers. The standing committee reviews each proposal on the following criteria: clarity of evaluation objectives and research questions, appropriateness and feasibility of the specifications for evaluation design and methods, appropriateness of the plans for dissemination of results, and use of previous relevant evaluations and existing program data systems. Each proposal must clearly state the relationship of the evaluation to SAMHSA's overall policy and priorities and evaluation program.

Summary of Fiscal 1995 SAMHSA Evaluations

During fiscal 1995, SAMHSA completed several reports on its evaluations in the Center for Mental Health Services. Some of these studies focused on youth, one of the population groups that SAMHSA has identified as in greatest need of services. In particular, services for children and adolescents with serious emotional disabilities were the subject of a series of research studies funded by the Center for Mental Health Services (CMHS). The evaluations were highlighted in a special issue of the Journal of Emotional and Behavioral Disorders.

In one CMHS research study, investigators developed, implemented, and evaluated an intensive, adolescent-centered case management approach to treating homeless adolescents in Washington State. Results of a 3-month followup showed a significant decrease in symptoms of depression, problem behavior, and substance abuse; significant increases in self-esteem; and reports from the youth of an increase in quality of life. This study is significant because it is the first to formally assess the effectiveness of mental health-related services to homeless youths.

A second research study examined the effectiveness of an individualized, intensive case-managed approach to improving adjustment outcomes for foster children with emotional and behavioral disorders. Results of this community-based, case- controlled experiment indicate that children who received the additional individualized services showed significantly greater short-term improvements in some behavioral and emotional adjustments than did children who received standard practice foster care services.

Another CMHS-funded study assessed the impact of multisystemic family preservation therapy (MFP) on family functioning and problem behavior of delinquent adolescents in rural and urban South Carolina communities. MFP is a nonoffice-based therapy. Interventions are directed toward individuals and families, peer relations, school relationships, academic performance, and any other social system believed to be involved in the problem behaviors. In general, the MFP group demonstrated improvements in the amount of problem behavior, level of mothers' psychological distress, and family functioning following MFP treatment.

SAMHSA Evaluations in Progress

SAMHSA has 12 major evaluations under way in the following general areas: program accountability, evaluation of demonstrations, and managed care. A description of each type follows, with some examples.

Program Accountability

SAMHSA conducts evaluations for program accountability in compliance with GPRA. These evaluations are undertaken to inform program management and help managers refine program operations. This is the primary type of evaluation conducted on SAMHSA's service grant programs. For example, the children's mental health service program has an ongoing evaluation. The evaluation will yield continuous information on program implementation and on outcomes for children and families served. The information will be used for reports to Congress, feedback to grantees, program development, and performance improvement.

Evaluations of Demonstrations

Evaluations of demonstrations are designed to generate new knowledge for policy development. The primary purpose of SAMHSA's demonstration programs is to generate new knowledge to lead the field in developing policies that improve substance abuse and mental health services. For example, evaluation results on substance abuse prevention for high-risk youth will allow policymakers to draw inferences about the effectiveness of certain interventions for this population. The Job Corps evaluation, which will be completed in fiscal 1996, is assessing an enriched substance abuse treatment program for adolescents. An evaluation of the program for Access to Community Care and Effective Services and Supports (ACCESS) will provide information for the design of ongoing service programs at the Federal, State, and local levels.

The National Treatment Improvement Evaluation Study (NTIES) is a cross-site evaluation study examining the effectiveness of demonstration grants funded by the Center for Substance Abuse Treatment (CSAT). The study will assess the extent to which treatment enhancements improve substance abuse treatment outcomes over time. Preliminary results from NTIES provide strong support for the efficacy of substance abuse treatment in reducing levels of substance use, reducing involvement in criminal activities, and increasing employment. The final report will be available in fiscal 1997.

Managed Care

In fiscal 1996, CSAT is initiating an important managed care evaluation to assess the impact of the States' managed care initiatives on substance abuse treatment in terms of access, cost, and quality. Currently, there is little information at the State, provider, or client levels about the impact of managed care on the provision and outcomes of substance abuse treatment services. Five States will be selected for rigorous, statistical evaluation, which will include measures of costs, access, quality, and treatment outcomes. Results will be used to improve the States' substance abuse managed care programs and for Federal policymaking related to managed care and health care reform.

New Directions for SAMHSA Evaluation

SAMHSA is designing its evaluation activities to guide programmatic and policy decisions. The evaluation activities will complement the SAMHSA and HHS strategic plans and will respond to emerging trends such as managed care.

One SAMHSA evaluation priority is to evaluate demonstration programs and the impact of demonstration findings and knowledge- transfer activities on service delivery. In fiscal 1996, SAMHSA proposed a new demonstration program designed to answer specific, important policy-relevant questions. These questions will be designed to provide critical information to improve the Nation's mental health and substance abuse treatment and prevention services. Evaluations of the demonstrations planned for fiscal 1996 will generate knowledge on such topics as the relative effectiveness of alternative models of managed care for treatment and prevention of substance abuse and mental disorders, the relative effectiveness of alternate models for preventing homelessness among adults with serious mental illnesses, and the efficacy of a brief intervention for marijuana dependence.

Another of SAMHSA's evaluation priorities is to assess the impact of managed care on the availability of services to populations in need and to develop standards for measuring quality and outcome. SAMHSA's managed care evaluation strategy is designed to leverage important knowledge that directly impacts its mission. This knowledge will be acquired through focused applied health services research, evaluations, demonstrations, and epidemiological and service capacity studies. SAMHSA is currently considering an evaluation strategy that would first establish baseline information about the incidence, prevalence, functional disability, and negative consequences associated with mental health and substance abuse; characteristics of the service systems; and characteristics of individuals serviced by those systems. Evaluations would then assess the impact of changes in organization and financing services through managed care and health care system reforms.

Appendix A. Abstracts of HHS Evaluations Completed in Fiscal 1995

The abstracts included in this appendix briefly describe the evaluation reports completed by the Department of Health and Human Services (HHS) during fiscal 1995. They are listed in alphabetical order by agency. The sponsoring agency, report title, abstract, and Federal contact person(s) are listed for each report. For more information on any of the reports, please call the contact listed at the end of each abstract. Copies may be obtained by contacting either the Policy Information Center or the National Technical Information Service. Their addresses are given below.

Policy Information Center

Final reports for most of the evaluations listed in this appendix have been submitted to the HHS Policy Information Center (PIC), a centralized source of information on in-process, completed, and ongoing HHS evaluations; short-term evaluative research; and policy-oriented projects. The PIC maintains a resource database containing information on more than 6,000 completed and ongoing studies sponsored by HHS, other Federal agencies, and private sector entities. The PIC provides executive summaries (if available) at no cost.

The PIC identification number appears after the Federal contact's phone number. For information about using PIC resources, please contact the following:

Policy Information Center
Office of the Assistant Secretary of Planning and Evaluation
Department of Health and Human Services
Room 438F, Hubert H. Humphrey Building
200 Independence Avenue, S.W.
Washington, DC 20201
(202) 690-6445

National Technical Information Service

The National Technical Information Service (NTIS) is an information clearinghouse run by the Department of Commerce. It provides reports in both paper and microform formats for a fee. If a final report has been registered with this service, the NTIS accession number follows the PIC ID entry. For more information about ordering copies of reports from NTIS, please contact the following:

National Technical Information Service
Department of Commerce
5285 Port Royal Road
Springfield, VA 22161
(703) 487-4650

Administration for Children and Families

Contents

Child Access Demonstration Projects: Final Wave I Report

Child Maltreatment 1993: Reports From the States to the National Center on Child Abuse

Children on Hold: Improving the Response to Children Whose Parents Are Arrested and Incarcerated

Demonstration Partnership Program Projects: Project Design and Evaluation Guidebook (Third Revised Edition)

Demonstration Partnership Program Projects: Self-Sufficiency Project Implementation Manual

Demonstration Partnership Program Projects: Summary of Final Evaluation Findings From 1990--Homeless and Youth at Risk

Demonstration Partnership Program Projects: Summary of Final Evaluation Findings From FY 1991--Case Management/Family Development

Demonstration Partnership Program Projects: Summary of Final Evaluation Findings From FY 1991--Micro-Business and Self-Employment

Demonstration Partnership Program Projects: Summary of Final Evaluation Findings from FY 1991--Minority Males

Demonstration Partnership Program Projects: Summary of Final Evaluation Findings From FY 1991--Youth at Risk

Evaluation of the Key States Initiative

Foster Youth Mentors

Increasing Participation in Work and Work-Related Activities: Lessons From Five State Welfare Reform Demonstration Projects (Two Volumes)

Low-Income Energy Assistance Program: Report to Congress for Fiscal Year 1993

Matching Opportunities to Obligations: Lessons for Child Support Reform From the Parents' Fair Share Pilot Phase

Outcomes of Permanency Planning for 1,165 Foster Children

Selected Annotated Bibliography on Youth and Gang Violence Prevention, Community Team Organizing and Training, and Cultural Awareness Curriculums

Something Old, Something New: A Case Study of the Post-Employment Services Demonstration in Oregon

Strengthening Families and Neighborhoods: A Community-Centered Approach--Final Report on the Iowa Patch Project

Study of the Impact on Service Delivery of Family Substance Abuse

Update From the Multistate Foster Care Data Archive: Foster Care Dynamics 1983-93

Youth With Runaway, Throwaway, and Homeless Experiences: Prevalence, Drug Use, and Other At-Risk Behaviors (Three Volumes)

TITLE: Child Access Demonstration Projects: Final Wave I Report

ABSTRACT NUMBER: 001

ABSTRACT: Since the passage of the 1984 Child Support Enforcement amendments and the 1988 Family Support Act, courts and child support agencies have vastly increased their efforts to establish a child's paternity and establish and enforce child support orders. However, many noncustodial parents' groups have complained that there has not been a parallel increase in enforcement of their rights to parental access. This report presents information from child access demonstration projects in Florida, Idaho, and Indiana that began in October 1990 (wave I). These projects were designed to determine whether mediation services for child access problems would reduce parent conflict; reduce interference with visitation rights; and encourage full, voluntary compliance with child support obligations. The report finds that (1) problems with parental access to children arise frequently (12 to 24 percent in Florida and Idaho), primarily in divorce cases; (2) access problems are more complex than visitation denial; (3) disagreements about visitation seem to increase over time; (4) mediation produced an agreement between parents in about two-thirds of cases, but this conclusion is complicated by the fact that a high proportion of cases assigned to mediation were never attended; and (5) there is no way to tell how long mediated agreements between parents might last. The project was extended through June 1996. (Final report 239 pages, plus appendixes.)

AGENCY SPONSOR: Office of Child Support Enforcement

FEDERAL CONTACT: David Arnaudo

PHONE NUMBER: 202/401-5364

PIC ID: 5972

PERFORMER ORGANIZATION: Policy Studies, Inc., Denver, CO; Center for Policy Research, Denver, CO

TITLE: Child Maltreatment 1993: Reports From the States to the National Center on Child Abuse

ABSTRACT NUMBER: 002

ABSTRACT: The National Child Abuse and Neglect Data System (NCANDS) collects data on child maltreatment reported from States, territories, and other jurisdictions. This study presents data collected from reports of child maltreatment investigated by States in 1993. NCANDS compiles aggregate data from States in its Summary Data Component (SDC) and also contains case-level data that allow more detailed analyses in its Detailed Case Data Component (DCDC). Pilot testing of the DCDC data collection instrument has been completed, and the DCDC is being phased in nationally. An analysis of the data from the first 10 participating States was to be available in late 1995. The study finds that (1) almost 2 million reports of child abuse and neglect were received by child protective service agencies and referred for investigation in 1993; (2) 1993 is the first year since 1976 in which the rate of reported child abuse and neglect cases has not increased; (3) 53 percent of all reports come from professionals, including educators, law enforcement and justice officials, medical professionals, social service professionals, and child care providers; (4) 18 percent of all reports come from family members; (5) 38 percent of the 1.6 million investigations of alleged abuse or neglect resulted in a disposition of substantiated or indicated child abuse; 53 percent were not substantiated or indicated child abuse; and (6) neglect is the most common type of maltreatment, followed by physical, sexual, medical, and emotional abuse.

AGENCY SPONSOR: Administration on Children, Youth and Families

FEDERAL CONTACT: Gail Collins

PHONE NUMBER: 202/205-8087

PIC ID: 5387.1

PERFORMER ORGANIZATION: Bowers and Associates, Reston, VA; American Humane Association, Englewood, CO

TITLE: Children on Hold: Improving the Response to Children Whose Parents Are Arrested and Incarcerated

ABSTRACT NUMBER: 003

ABSTRACT: An estimated 1.5 million children in the United States have an incarcerated parent. As a result of "get tough on crime" policies, the number of incarcerated women has tripled. It is estimated that, on any given day, 167,000 children have mothers in prisons and jails. This report explores what communities are doing nationwide to meet the needs of children whose caretakers are arrested or incarcerated. The report finds that the typical incarcerated woman is likely to be young, single, unemployed, and African-American or Hispanic, with limited education, job skills, and income; a history of drug abuse; and two to three young children. The report uses national telephone surveys of 500 law enforcement, child welfare, and corrections officials in 100 counties. It examines survey responses from patrol officers, narcotics officers, child protective services, foster care parents, and corrections staff. It reports on site visits to communities with exemplary responses to children whose parents are arrested: Galveston, Texas; Mobile, Alabama; San Antonio, Texas; and St. Louis, Missouri. Site visits to exemplary programs for children whose parents are incarcerated are reported on for Albany, New York; Bethel Bible Village, Tennessee; San Antonio, Texas; and San Francisco, California. (Final report 288 pages, plus appendixes.)

AGENCY SPONSOR: Administration on Children, Youth and Families

FEDERAL CONTACT: Cecelia Sudia

PHONE NUMBER: 202/205-8764

PIC ID: 5853

PERFORMER ORGANIZATION: American Bar Association, Center on Children and the Law, Chicago, IL

TITLE: Demonstration Partnership Program Projects: Project Design and Evaluation Guidebook (Third Revised Edition)

ABSTRACT NUMBER: 004

ABSTRACT: The Demonstration Partnership program (DPP) represents the first formal research and development component within the Community Service Block Grant. Federal guidelines require that the projects funded under DPP include a strong third-party evaluation component to determine whether they are worthy of replication. The objectives of the program are to (1) stimulate community action agencies to develop new approaches that provide for greater self-sufficiency among the poor; (2) test and evaluate new approaches; (3) disseminate project results and evaluation findings; and (4) strengthen the ability of eligible entities to integrate, coordinate, and redirect activities that promote maximum self-sufficiency among the poor. DPP grants are made to innovative projects that can be coordinated with a grantee's ongoing program. The projects can also combine resources, including partnerships with other community agencies. This guidebook is intended to help grant applicants and grantees understand the role of evaluation in the grants process, in selecting and working with a third-party evaluator, and in preparing the required evaluation reports. See also the series under PIC ID 4336, 6054, 6055, 6056, 6057, and 6058. (Final report variously paginated, plus appendixes.)

AGENCY SPONSOR: Office of Community Services

FEDERAL CONTACT: Richard Saul

PHONE NUMBER: 202/401-9341

PIC ID: 4336.4

PERFORMER ORGANIZATION: BHM International, Inc., Silver Spring, MD

TITLE: Demonstration Partnership Program Projects: Self-Sufficiency Project Implementation Manual

ABSTRACT NUMBER: 005

ABSTRACT: "The Self-Sufficiency Project Implementation Manual" is a synopsis of lessons drawn from 8 years of DPP projects. DPP projects were designed to demonstrate the effectiveness of innovative services that promote self-sufficiency among low-income individuals and families who rely on or are at risk of relying on public assistance. DPP has concentrated on five issues in its projects: (1) case management; (2) micro-enterprise development; (3) minority male employment; (4) homelessness; and (5) youth at risk. The implementation manual presents generic models for establishing effective community-based programs in these areas and offers ideas on evaluating such programs. All models are presented in the form of logic models for consistency. Each section of the manual is organized into general lessons and specific lessons learned regarding each of the five project types. The manual is designed for use by community action agencies, community-based organizations, and local community program planners who are interested in developing a self-sufficiency project.

AGENCY SPONSOR: Office of Community Services

FEDERAL CONTACT: Richard Saul

PHONE NUMBER: 202/401-9341

PIC ID: 4336.3

PERFORMER ORGANIZATION: BHM International, Inc., Silver Spring, MD

TITLE: Demonstration Partnership Program Projects: Summary of Final Evaluation Findings From 1990: Homeless and Youth at Risk

ABSTRACT NUMBER: 006

ABSTRACT: This report presents the findings from three grants that focused on homeless adults and youth at risk. Two of the projects addressed the needs of homeless adults, while one concentrated on homeless youths. The projects focused on adults were directed toward very different populations. The Tacoma, Washington program worked with adult males who, while homeless, were generally free from most problems that prevent reintegration into the workplace. The project had considerable success in collecting data and in moving clients to the workforce. At follow-up, 80 percent of the case-managed group had jobs. About one-third of these jobs included benefits, and 50 percent were permanent. Half of the clients who were homeless at the beginning of the project were not at follow-up. Unlike the Tacoma project, the Duluth, Minnesota project, which targeted residents of low-income housing, did not exclude residents with special problems, such as a history of mental illness, drug abuse, or criminal involvement. This project was less successful in collecting data and in integrating clients into the workforce. However, participants achieved significantly higher incomes than the control group, although their wages were still low. The project targeted to youth was located in Portland, Oregon. It focused on helping homeless youth at risk find a more stable environment. Many of the youths were using illicit drugs, had criminal histories or mental problems, and came from violent homes. Stabilizing the youths and meeting their emergency needs were the first priorities of the project. The project had problems tracking clients, since many of the youths did not follow up with counselors. Many did not return to the program out of fear that their parents might be contacted. As a consequence, the outcome analysis was difficult to develop. Despite these difficulties, some conclusions were reached, such as: (1) 75 percent of the youths who completed the program have a stable residence unconnected to street life; and (2) 83 percent of those who completed the program are employed, are participating in an educational or vocational training program, or are serving in the Armed Forces. See also the series under PIC ID 4336, 6054, 6055, 6056, 6057, and 6058. (Final report variously paginated.)

AGENCY SPONSOR: Office of Community Services

FEDERAL CONTACT: Richard Saul

PHONE NUMBER: 202/401-9341

PIC ID: 6056.1

PERFORMER ORGANIZATION: BHM International, Inc., Silver Spring, MD

TITLE: Demonstration Partnership Program Projects: Summary of Final Evaluation Findings From FY 1991--Case Management/Family Development

ABSTRACT NUMBER: 007

ABSTRACT: This report describes two efforts designed to increase the self-sufficiency of public assistance recipients through case management. The two DPPs were located in DuPage, Illinois, and Stevens Point, Wisconsin. In each project several agencies cooperated to provide coordinated, integrated, and appropriate services to individuals or families receiving public assistance. A case manager coordinated the care received by clients. The target population in DuPage was single-parent families; and in Stevens Point the target population was families whose total income was below 125 percent of the poverty level. Both projects were intended to increase the economic self-sufficiency of clients. The DuPage project included 83 participants in its experimental group. Despite numerous attempts to engage them, about 45 percent of the treatment group never presented themselves for service, never followed up, or were unable to participate because of mental health problems. The Stevens Point program focused on helping working poor families pursue home ownership. Three groups were compared: a control group, a comparison group of participants in a low-income buyers group, and a target group who were enrolled in the home-buyers group and receiving integrated case-management services. See also the series under PIC ID 4336, 6054, 6055, 6056, 6057, and 6058.

AGENCY SPONSOR: Office of Community Services

FEDERAL CONTACT: Richard Saul

PHONE NUMBER: 202/401-9341

PIC ID: 6054.2

PERFORMER ORGANIZATION: BHM International, Inc., Silver Spring, MD

TITLE: Demonstration Partnership Program Projects: Summary of Final Evaluation Findings From FY 1991--Micro-Business and Self-Employment

ABSTRACT NUMBER: 008

ABSTRACT: This report presents the results of three programs designed to help low-income individuals start their own business or pursue self-employment to achieve economic self-sufficiency. Two of the programs, one located in Ukiah, California, and one on Wetumpka, Alabama, were relatively close to major urban centers. The third program in Bozeman, Montana, by contrast, was located in one of the more sparsely populated areas of the United States. The populations served by the three programs were quite dissimilar: Ukiah targeted only women; Bozeman targeted only Aid to Families with Dependent Children (AFDC) and food stamp recipients; and Wetumpka targeted a blue-collar and semirural population of mixed gender and race. The Ukiah program was a great success, forcing some participants to wait several weeks before receiving services. Unfortunately, some participants that could not be served immediately did not return to the program when space became available. The Bozeman project assisted low-income individuals in obtaining small business loans. By the end of the project, 39 loans had been awarded to 37 recipients. Additionally, the program awarded loans to 12 Native Americans, or 86 percent of the goal. All programs were relatively successful and had positive impacts on their communities. See also the series under PIC ID 4336, 6054, 6055, 6056, 6057, and 6058. (Final report variously paginated.)

AGENCY SPONSOR: Office of Community Services

FEDERAL CONTACT: Richard Saul

PHONE NUMBER: 202/401-9341

PIC ID: 6055.2

PERFORMER ORGANIZATION: BHM International, Inc., Silver Spring, MD

TITLE: Demonstration Partnership Program Projects: Summary of Final Evaluation Findings From FY 1991--Minority Males

ABSTRACT NUMBER: 009

ABSTRACT: This report presents the results of three DPPs focused on minority males. Two of the projects, located in Milwaukee, Wisconsin, and Boston, Massachusetts, organized their approaches around a combination of a job club and case management. The third project, located in Lexington, Kentucky, used a combination of mentoring and case-management support. The important differences between the programs lay in their target audiences. The Milwaukee and Lexington projects focused almost exclusively on African-Americans, while the Boston project expanded this focus to include Hispanics. Participants' ages ranged from 14 to 34 years; the Lexington project focused on the younger end of the spectrum, the Boston project on the middle, and the Milwaukee program on the older group. The Milwaukee program concentrated on issues of racial bias in education and employment as a means to reduce minority male unemployment. Individual training and education was also used to lessen the negative effects of family and community dysfunction. In Lexington, problems included a lack of suitable mentors. Only 10 percent of 300 possible mentors completed the application, and this number was further reduced after a check of police records. However, the program demonstrated that mentors and participants must be matched one-to-one. The Boston program achieved a 58 percent completion rate. See also the series under PIC ID 4336, 6054, 6055, 6056, 6057, and 6058.

AGENCY SPONSOR: Office of Community Services

FEDERAL CONTACT: Richard Saul

PHONE NUMBER: 202/401-9341

PIC ID: 6058

PERFORMER ORGANIZATION: BHM International, Inc., Silver Spring, MD

TITLE: Demonstration Partnership Program Projects: Summary of Final Evaluation Findings From FY 1991--Youth at Risk

ABSTRACT NUMBER: 010

ABSTRACT: This report presents the findings from four types of youth-at-risk projects funded under DPP: school-based, teen parents, employment training, and homeless youth. Regardless of the approach taken, all products had some core service components in common, including case management, social support services, educational services, and employment training. Four projects were school based. The Yakima, Washington, program targeted low-income high school youth to increase their educational success and wage rates and to reduce their reliance on public assistance. The Morristown, Tennessee, program provided school-based interventions to 148 preteen girls to reduce adolescent pregnancy rates. The Greenfield, Massachusetts, program provided case-management services, as well as nontraditional interventions (cooperative games, outdoor challenge activities). The Everett, Washington, program helped teenage mothers increase their education and job readiness. Two projects focused on employment training (Tacoma, Washington, and Austin, Texas). The Tacoma program served JTPA youths aged 17 to 21 at risk of gang or drug involvement or dropping out of school and eligible for Job Training Partnership Act services. The second project was intended to increase the potential for self-sufficiency and to strengthen service linkages. See also the series under PIC ID 4336, 6054, 6055, 6056, 6057, and 6058. (Final report variously paginated.)

AGENCY SPONSOR: Office of Community Services

FEDERAL CONTACT: Richard Saul

PHONE NUMBER: 202/401-9341

PIC ID: 6057.2

PERFORMER ORGANIZATION: BHM International, Inc., Silver Spring, MD

TITLE: Evaluation of the Key States Initiative

ABSTRACT NUMBER: 011

ABSTRACT: After passage of the Refugee Act of 1980, it became apparent that a large percentage of refugee families were remaining in welfare programs for extended periods of time. In response, the Office of Refugee Resettlement implemented the Key States Initiative (KSI) in fiscal 1987. The initiative was intended to increase employment among the refugee community and reduce dependence on welfare. Eight States emerged with high welfare dependency characteristics, and of these States, five chose to participate in this new program: Minnesota, New York, Pennsylvania, Washington, and Wisconsin. For each State, the report provides information on program design features, program participant characteristics, program outcomes, and lessons learned. The report also discusses the conclusions that can be drawn from a composite view of KSI program principles. The report finds that (1) programs that emphasize self-sufficiency goals, such as welfare termination, are more successful than those focusing only on job placement; (2) employment-related services provided to all potential wage earners in the family increased the odds that the family would leave the welfare rolls; (3) results-oriented vocational training programs worked better when they obtained employer commitments to training, program design, teaching resources, and job placement; and (4) clients are more cooperative with staff who have cultural backgrounds similar to their own.

AGENCY SPONSOR: Office of Refugee Resettlement

FEDERAL CONTACT: Loren Bussert

PHONE NUMBER: 202/401-5364

PIC ID: 5959

PERFORMER ORGANIZATION: Office of Refugee Resettlement, Washington, DC

TITLE: Foster Youth Mentors

ABSTRACT NUMBER: 012

ABSTRACT: The University of Illinois Urbana-Champaign School of Social Work examined the factors characteristic of successful relationships between foster youths and older citizen mentors. The program compared 250 successful mentor/foster youth matches with 250 unsuccessful matches. Data focused on the characteristics of the mentors, the foster youths, and the mentoring program. The study also includes evaluations of each match from both participants. Study samples were taken from programs that offer mentoring as part of their service delivery. Programs include The Peoria Children's Home Association (Illinois); Covenant House Rites of Passage (New York); Judge Baker Children's Center; Fairfax County Mentor Program (Virginia); and various mentoring programs from Montana, New Hampshire, Alabama, and Kentucky. Project findings were disseminated to independent living programs throughout the United States in order to facilitate the use of mentors in efforts to transition older youths from foster care.

AGENCY SPONSOR: Administration on Children, Youth and Families

FEDERAL CONTACT: Cecelia Sudia

PHONE NUMBER: 202/205-8764

PIC ID: 4388

PERFORMER ORGANIZATION: University of Illinois, School of Social Work, Urbana, IL

TITLE: Increasing Participation in Work and Work-Related Activities: Lessons From Five State Welfare Reform Demonstration Projects (Two Volumes)

ABSTRACT NUMBER: 013

ABSTRACT: This study uses data from State and local officials, case record reviews, and management reports to examine key implementation and operational issues in five States currently conducting welfare reform demonstration projects: Colorado, Iowa, Michigan, Utah, and Vermont. Reforms in these five States emphasize strategies designed to increase participation in work and work-related activities. Specifically, the report looks at (1) the strategies States use to increase participation in the Job Opportunities and Basic Skills (JOBS) training program (or similar programs); (2) the role of child care in increasing program participation; (3) the extent to which the culture of welfare offices has changed from one of check issuance to one of finding employment for recipients; and (4) the early lessons to be drawn from the experiences of these States. The report finds that States use many strategies to increase participation, including (1) reducing exemptions and serving more participants; (2) redefining participation to include unsubsidized employment and activities, such as substance abuse and mental health counseling and parenting classes; and (3) increasing penalties for nonparticipation. All five States budgeted for additional expenditures for staff and child care. All States reported that it is extremely difficult to change the culture of welfare offices. Volume 2 presents a site visit summary for each State. (Final report: volume 1, 96 pages; volume 2, 174 pages.)

AGENCY SPONSOR: Administration on Children, Youth and Families

FEDERAL CONTACT: Peter G. Germanis

PHONE NUMBER: 202/401-9316

PIC ID: 5815

PERFORMER ORGANIZATION: Urban Institute, Washington, DC

TITLE: Low-Income Home Energy Assistance Program: Report to Congress for Fiscal Year 1993

ABSTRACT NUMBER: 014

ABSTRACT: This report examines how States used funding they received under the Low-Income Home Energy Assistance Program (LIHEAP), describes trends in home energy consumption, reviews demographic data on LIHEAP beneficiaries, and assesses State compliance with the LIHEAP legislation for fiscal 1993. During that year, States spent $1.525 billion from LIHEAP, 86 percent of which came from Federal net allotments. States used the funds to provide heating and cooling assistance, energy crisis intervention or assistance, low-cost home weatherization, or other energy-related home repairs. The report finds that (1) 32 percent of home energy is used for space heating or cooling; (2) low-income households are more likely to heat with fuel oil, kerosene, or liquified petroleum gas and are less likely to heat with electricity; (3) low-income households consume less energy than other households (about 10 percent less for heating, 32 percent less for cooling, 17 percent less for appliances, and 5 percent less for water heating); (4) although these households consume less energy, a greater percentage of their annual income is spent on energy needs; (5) about 20 percent of those households eligible for LIHEAP received assistance in 1993 (they were among the poorest of all eligible households); and (6) States generally comply with requirements for setting benefit levels and legibility. See also PIC ID 4638. (Final report 37 pages, plus appendixes.)

AGENCY SPONSOR: Office of Community Services

FEDERAL CONTACT: Donald Sykes

PHONE NUMBER: 202/401-9333

PIC ID: 4638.1

PERFORMER ORGANIZATION: Office of Community Services, Division of Energy Assistance, Washington, DC

TITLE: Matching Opportunities to Obligations: Lessons for Child Support Reform From the Parents' Fair Share Pilot Phase

ABSTRACT NUMBER: 015

ABSTRACT: The Parents' Fair Share (PFS) demonstration program requires that the noncustodial parents of children on welfare participate in employment related and other services when they are unemployed and unable to meet child support obligations. This study evaluates the operational feasibility of the PFS approach, ascertains whether a full-scale evaluation is warranted, and studies the target population. This effort studied 4,000 noncustodial parents in nine States who were not able to meet their child support obligations. Baseline data from enrollment forms, management information data, child support payment records, child support orders, site visits, peer support feedback forms, and participant and staff interviews were used to evaluate the pilot phase. The report finds that (1) the PFS program appears to be operationally feasible and shows enough promise to warrant a full-scale evaluation of its impacts and cost-effectiveness; (2) the program shows most ability to effect changes in individuals in poor support groups with a set curriculum; (3) cases received attention that were neglected prior to PFS; (4) employment and training systems were more difficult to change because of institutional barriers and limitations in program design choices, but the number of options in this area increased over time; and (5) many PFS parents appeared to be living in poverty, but most saw their children regularly and wished to support them. See also PIC ID 5952.1.

AGENCY SPONSOR: Office of Child Support Enforcement

FEDERAL CONTACT: Mark Fucello

PHONE NUMBER: 202/401-4538

PIC ID: 5952

PERFORMER ORGANIZATION: Manpower Demonstration Research Corporation, New York, NY

TITLE: Outcomes of Permanency Planning for 1,165 Foster Children

ABSTRACT NUMBER: 016

ABSTRACT: This report presents the findings of a study of 1,165 children in foster care. It examines the distribution of the children in four permanency planning (PP) exit categories after up to 18 months in care, and any changes in the child's PP status within 12 months of leaving the foster family. The report gives information on the demographic characteristics of the children, the reasons for their removal from the family, and characteristics of their stays in foster care. The report finds that many significant differences among children in the two study sites (San Diego County, California, and Pierce County, Washington) are explained in part by differences in child and family characteristics, but also by differences in the service systems in the sites. Pierce County's system has greater judicial system involvement, greater use of voluntary placements, and administrative rather than court reviews. The report also finds that the over-representation of minority children calls for an ethnicity-sensitive practice in child welfare systems. Furthermore, the large number of children with mental health, physical health, and behavioral problems in the study population reveals the need for appropriate services for this population. The report also finds that (1) two-thirds of children in both sites were reunited with their families; (2) adoptions were more frequent in Pierce County than in San Diego County, while long-term foster care was more frequent in the latter site; and (3) guardianship was used sparingly in both sites.

AGENCY SPONSOR: Administration on Children, Youth and Families

FEDERAL CONTACT: Cecelia Sudia

PHONE NUMBER: 202/205-8764

PIC ID: 5874

PERFORMER ORGANIZATION: San Diego State University, Child and Family Research Group, San Diego, CA; Manpower Demonstration Research Corporation, New York, NY

TITLE: Selected Annotated Bibliography on Youth and Gang Violence Prevention, Community Team Organizing and Training, and Cultural Awareness Curriculums

ABSTRACT NUMBER: 017

ABSTRACT: This report is intended as a resource for youth-serving organizations, as well as individuals, researchers, and policymakers concerned with youth issues. This annotated bibliography provides brief summaries of materials ranging from reports and monographs to curriculums, training manuals, articles, and other literature and products. The entries in the bibliography were selected in a search of local and national print and electronic libraries and databases. Items were selected, annotated, and organized in the following general sections: (1) youth violence/gang violence; (2) youth violence prevention programs; (3) youth violence, gang, and drug prevention curriculums; (4) interpersonal skills training; (5) peer mediation; (6) multicultural awareness curriculums and cultural sensitivity training; (7) immigrants; (8) team organizing; (9) community collaboration; (10) interagency cooperation and partnerships; (11) community empowerment; and (12) training methodologies and strategies. Author, subject, geographic area, organization, project, and center indexes are provided. See also PIC ID 5875 and 5875.1. (Final report, 172 pages.)

AGENCY SPONSOR: Administration on Children, Youth and Families

FEDERAL CONTACT: Terry Lewis

PHONE NUMBER: 202/205-8102

PIC ID: 5875.2

PERFORMER ORGANIZATION: Development Services Group, Inc., Bethesda, MD

TITLE: Something Old, Something New: A Case Study of the Post-Employment Services Demonstration in Oregon

ABSTRACT NUMBER: 018

ABSTRACT: The Federal Post-Employment Services Demonstration (PESD) was initiated in response to indications that many people who leave welfare for work lose their jobs fairly quickly and return to the welfare rolls. The demonstration is intended to help those who become employed keep their jobs, to help those who lose their jobs return to work quickly, and to reduce the amount of time families receive Aid to Family with Dependent Children (AFDC). PESD operates in Riverside, California; Chicago, Illinois; Portland, Oregon; and San Antonio, Texas. This report provides an indepth look at the program's operations in Oregon, gives the background for PESD and contrasts it with programs such as JOBS, which focuses on preemployment services. It describes the AFDC/JOBS program in Oregon and the State's postemployment initiative, the JOBS Employment Retention Initiative (JERI). The report then describes early patterns of job loss and reemployment in JERI, finding that job turnover is significant for participants (61 percent lost their first jobs during the first year of the demonstration). However, participants who lose their jobs return to work fairly quickly. The report describes the structure of the JERI program and makes general recommendations about how States might move from a preemployment services model to a post-employment services model. (Final report 48 pages, plus appendixes.)

AGENCY SPONSOR: Office of Planning, Research, and Evaluation

FEDERAL CONTACT: Nancye Campbell

PHONE NUMBER: 202/401-5760

PIC ID: 6002

PERFORMER ORGANIZATION: Project Match, Erikson Institute, Chicago, IL

TITLE: Strengthening Families and Neighborhoods: A Community-Centered Approach--Final Report on the Iowa Patch Project

ABSTRACT NUMBER: 019

ABSTRACT: The Iowa Patch project, modeled on the British patch system of community-centered service delivery, was an innovative method of testing services improvement in a distressed urban neighborhood in Linn County, Iowa. The project attempted to overcome categorical barriers that prevent the pooling and use of informal and formal resources needed for flexible social services. The project ran in tandem with, and was supported by the Linn County Decategorization project, an Iowa initiative merging child welfare funding streams and involving the community in comprehensive services planning. The project showed that, overall, services were integrated and improved using the patch model. The project overcame normal turfism to successfully collocate an interagency team of human services workers; it successfully devolved responsibility for service delivery to the team and to line workers on the team; and it shifted the workers' practice from a deficit and crisis orientation to one that is proactive, preventive, and holistic. Services were more accessible and workers were more approachable and were able to develop informal relationships with consumers and local helping networks. Workers also did more in-home work with consumers. The Patch project has developed well beyond the level of a demonstration program and has been integrated with other agencies in Linn County's Family Research Center program. (Final report 148 pages, plus appendix.)

AGENCY SPONSOR: Administration on Children, Youth and Families

FEDERAL CONTACT: James Dolson

PHONE NUMBER: 202/401-0133

PIC ID: 5977

PERFORMER ORGANIZATION: University of Iowa, Iowa City, IA

TITLE: Study of the Impact on Service Delivery of Family Substance Abuse

ABSTRACT NUMBER: 020

ABSTRACT: This report was originally intended to provide the Administration on Children, Youth and Families (ACYF) with information on the incidence of family drug abuse, and the effects of this abuse on ACYF programs. The comprehensive survey and study first planned was not approved by the Office of Management and Budget. Therefore, the scope of the study was reduced to include a literature review of relevant research from 1986 through 1994 and intensive case studies of the effects of family alcohol and other drug (AOD) abuse on ACYF-funded programs. The literature review and site visits show that (1) Child Protective Services (CPS) are the ACYF programs most seriously negatively affected by family AOD abuse, followed by foster care, Head Start and other early childhood education programs, runaway and homeless youth (RHY) programs, and adoption services; (2) front-line staff in most ACYF-funded programs are ill-equipped and poorly trained for dealing with substance-abusing families; (3) ACYF-funded programs that effectively intervene in cases of family AOD abuse are significantly hampered by the lack of appropriate AOD treatment outlets; and (4) family AOD abuse has resulted in new challenges to all ACYF-funded programs in meeting their mandated functions, including accelerated termination of parental rights policies, postadoption and foster placement support programs, and modification of the Head Start program to address the needs of nonparental caretakers. (Final report 33 pages, plus appendixes.)

AGENCY SPONSOR: Administration on Children, Youth and Families

FEDERAL CONTACT: James Dolson

PHONE NUMBER: 202/260-6165

PIC ID: 4399

PERFORMER ORGANIZATION: Cygnus Corporation, Inc., Washington, DC

TITLE: Update From the Multistate Foster Care Data Archive: Foster Care Dynamics 1983-93

ABSTRACT NUMBER: 021

ABSTRACT: The Multistate Foster Care Data Archive contains foster care career histories for all children who have been in State-supervised substitute-care living arrangements. The data for the archive are extracted directly from the administrative data systems operated by each State's child welfare agency. The information is comprehensive and longitudinal at the individual level. Five States participated in the original archive: California, Illinois, Michigan, New York, and Texas. These States provide services to almost one-half of the total child substitute-care population in the United States. This report covers 1983-93, updating a 1994 report examining foster care dynamics between 1983 and 1992. The report is descriptive and is not a policy analysis. The report finds that (1) the five States described here have different levels of child care placement activity; (2) the trend in each of the States has been one of significant growth in the numbers of children receiving State-supported care; (3) admissions have been far greater than discharges, and most episodes of rapid caseload change follow a shift in admission levels; (4) most child care placement activity was concentrated in major urban centers; (5) much of the recent growth in foster care has involved the placement of children with relatives, while the number of children placed in institutional and group care settings has remained stable; and (6) infants and young children are the fastest growing age groups in the foster care population.

AGENCY SPONSOR: Administration on Children, Youth and Families

FEDERAL CONTACT: Penelope L. Maza

PHONE NUMBER: 202/205-8172

PIC ID: 5885.1

PERFORMER ORGANIZATION: Chapin Hall Center for Children, University of Chicago, Chicago IL

TITLE: Youth With Runaway, Throwaway, and Homeless Experiences: Prevalence, Drug Use, and Other At-Risk Behaviors (Three Volumes)

ABSTRACT NUMBER: 022

ABSTRACT: This report is the first national study of substance use, suicide attempts, and other at-risk behaviors among youth with runaway, throwaway, or homeless experience. Information comes from four sources: youths in shelters, youths on the street, youths in households, and directors of youth shelters. The survey of youths and youth shelter directors shows that (1) these youths are exposed to high-risk environments before as well as after they leave home; (2) about half of the youths classified as runaways could also be considered throwaways by their families; (3) these youths are much more likely than youths in general to engage in problem behaviors, including substance abuse, suicide attempts, unsafe sexual behavior, and criminal activity; (4) the youths from families who use drugs are particularly likely to engage in high-risk behaviors; (5) a high proportion of these youths spend time in institutions such as foster homes, psychiatric hospitals, and juvenile justice facilities; and (6) these youths are more likely to engage in many problem behaviors after leaving home than before, perhaps because doing so is a means of survival. The report recommends improvements in services for youths with these experiences. It notes that services must address all needs, from primary prevention to intervention and treatment. Furthermore, services must be community based, accessible, and comprehensive. (Final report three volumes variously paginated, plus appendixes.)

AGENCY SPONSOR: Administration on Children, Youth and Families

FEDERAL CONTACT: James Griffin, Ph.D.

PHONE NUMBER: 202/205-8138

PIC ID: 3871

PERFORMER ORGANIZATION: Research Triangle Institute, Research Triangle Park, NC

Administration on Aging

Contents

Quality of Long-Term Care in Home and Community-Based Settings: Defining the Issues

Real People, Real Problems: An Evaluation of the Long-Term Care Ombudsman Programs of the Older Americans Act

TITLE: Quality of Long-Term Care in Home and Community-Based Settings: Defining the Issues

ABSTRACT NUMBER: 023

ABSTRACT: This study is the first step in implementing the 1992 amendments to the Older Americans Act, which authorize the Department of Health and Human Services to conduct two studies. The first study is on the quality of care provided by board and care (B & C) facilities, while the second study is on home care services for older and disabled individuals. The study listed here developed the conceptual framework and provisional design for the two mandated studies.

AGENCY SPONSOR: Office of Program Operations and Development

FEDERAL CONTACT: James Steen

Room 4262, Cohen Building

PHONE NUMBER: 202/619-0075

PIC ID: 5962

PERFORMER ORGANIZATION: National Academy of Sciences, Institute of Medicine, Washington, DC

TITLE: Real People, Real Problems: An Evaluation of the Long-Term Care Ombudsman Programs of the Older Americans Act

ABSTRACT NUMBER: 024

ABSTRACT: The long-term care ombudsmen program, administered by AoA, provides ombudsmen who advocate to protect the health, safety, welfare, and rights of the institutionalized elderly in nursing facilities and board and care homes. This report examines (1) State compliance with program mandates; (2) conflicts of interest; (3) effectiveness of the program; and (4) adequacy of resources for the program. The report finds that (1) the ombudsman program as a whole has not been fully implemented with regard to the provisions of the Older Americans Act; (2) not all residents of long-term care facilities have meaningful access to ombudsmen, and many are not aware of the program's existence; (3) ombudsmen provide timely responses to complaints; (4) implementation of the ombudsman program for residents of nursing facilities is uneven, and for residents of B&C homes it has not been implemented in any meaningful way; (5) because the ombudsman program is part of State government and its hierarchy, conflicts of interest and loyalty exist within the program, yet regulations do not reflect this reality; and (6) the ombudsman programs function well in many States, serving many thousands of institutionalized elderly, despite the obstacles of inadequate funding, staff shortages, low salary levels, and conflicts of interest. The report recommends improvements in each of the main areas it discusses. (Final report 248 pages, plus appendixes.)

AGENCY SPONSOR: Office of Program Operations and Development

FEDERAL CONTACT: Nancy Wartow

PHONE NUMBER: 202/619-1058

PIC ID: 5819

PERFORMER ORGANIZATION: National Academy of Sciences, Institute of Medicine, Washington, DC

Agency for Health Care Policy and Research

Contents

Develop, Apply, and Evaluate Medical Review Criteria and Educational Outreach Based Upon Clinical Practice Guidelines: Acute Postoperative Pain and Urinary Incontinence

Developing Quality and Utilization Review Criteria for Management of Cataracts in Adults: Phase II Final Report

Developing Quality and Utilization Review Criteria for Pressure Ulcers in Adults: Phase II Final Report

Understanding and Choosing Clinical Performance Measures for Quality Improvement: Development of a Typology

Design of a Survey to Monitor Consumers' Access to Care, Use of Health Services, Health Outcomes, and Patient Satisfaction: Final Report

Overcoming Barriers to Implementation and Integration of Clinical Information Management Systems: Feasibility Study

Practice Patterns and Expenditures in Managed Care and Indemnity Plans: A Condition-Specific Analysis

Directory of Minority Health and Human Services Data Resources

Development of Designs for Evaluation of the Process of Clinical Guideline Development

Users' Perceptions of Four Clinical Practice Guidelines and Their Development Process

Setting Priorities for Clinical Practice Guidelines

AHCPR Clinical Practice Guideline Program: Report to Congress

TITLE: Develop, Apply, and Evaluate Medical Review Criteria and Educational Outreach Based Upon Clinical Practice Guidelines: Acute Postoperative Pain and Urinary Incontinence

ABSTRACT NUMBER: 025

ABSTRACT: This project, a contract with the Center for Clinical Quality Evaluation in Washington, DC, was designed to translate three AHCPR-supported clinical practice guidelines into tools that could be used to evaluate quality of care. Using the methodology developed in "Using Clinical Practice Guidelines to Evaluate Quality of Care: Issues and Methods" (AHCPR Publication Numbers 95-0045 and 95-0046), the project developed quality/utilization review criteria and clinical performance measures based on three guidelines: urinary incontinence; acute postoperative pain; and benign prostatic hyperplasia, or BPH. The project used five Medicare Peer Review Organizations to develop and test the criteria and performance measures. These criteria and measures were used to assess the care provided to a sample of randomly selected cases of Medicare patients. The peer review organizations also played an integral role in developing, implementing, and evaluating alternative educational outreach strategies based on the BPH guideline.

The project found that it is possible to develop valid and reliable guideline-based performance measures from AHCPR-supported guidelines. The structure of guideline recommendations can strongly influence the development of clinical performance measures. The project also found that targeted educational efforts based on the BPH-supported guidelines resulted in improved performance on specified measures. (Final report 107 pages, plus appendixes.)

AGENCY SPONSOR: Office of Planning and Evaluation

FEDERAL CONTACT: Irma Arispe

PHONE NUMBER: 301/594-1455

PIC ID: 5973

PERFORMER ORGANIZATION: Center for Clinical Quality Evaluation, Washington, DC

TITLE: Developing Quality and Utilization Review Criteria for Management of Cataracts in Adults: Phase II Final Report

ABSTRACT NUMBER: 026

ABSTRACT: This project developed and tested quality and utilization review criteria and review systems based on AHCPR-supported clinical practice guidelines on cataracts in adults. The project included two phases. In phase I, project teams developed and tested initial drafts of review criteria and review systems. These drafts were then refined and key components tested again in phase II. Conclusions from phase I are (1) guideline-based review criteria and systems can be developed and used in reviewing quality and utilization in a range of health care delivery settings; (2) review criteria derived from a guideline can be organized into many types of review systems with different goals and purposes; (3) guidelines must contain several critical elements to be useful for clinical decisionmaking and for development of review criteria and review systems; (4) operationalization and measurement are key challenges in review system development. Phase II conclusions are (1) the medical record-based review system designed to assess adherence to AHCPR-supported cataract guidelines is feasible and useful; (2) guidelines must be more specific and have more comprehensive information to optimize their development; and (3) use of a review system may improve documentation practices and adherence to recommendations.

AGENCY SPONSOR: Office of Planning and Evaluation

FEDERAL CONTACT: Irma Arispe

PHONE NUMBER: 301/594-1455

PIC ID: 5633

PERFORMER ORGANIZATION: Rand Corporation, Santa Monica, CA

TITLE: Developing Quality and Utilization Review Criteria for Pressure Ulcers in Adults: Phase II Final Report

ABSTRACT NUMBER: 027

ABSTRACT: This project developed guideline-based quality and utilization review criteria that can be used to assess clinical care to prevent pressure ulcers. These criteria can be used to assess conformance with recommendations in clinical practice guidelines and to target areas for clinical quality improvement. The project developed and tested measures that can be used to assess aspects of care that may be related to the development of pressure ulcers. The measures included the development of care plans and nutritional assessments, as well as care behavior related to positioning patients, alleviating pressure, and cleansing and moving patients. The focus of the review system is on patients with limited mobility, specifically patients with stroke or hip fracture. However, the review system can be adapted for use with other patient groups. The review system based on the pressure ulcer guideline was designed in collaboration with the Department of Veterans Affairs (VA) and focused on testing the review criteria for use in both VA and non-VA hospitals. The criteria will be used as part of future collaborative research between AHCPR and the VA. AHCPR and the VA are involved in a VA-wide effort to disseminate the guideline to 45 hospitals.

AGENCY SPONSOR: Office of Planning and Evaluation

FEDERAL CONTACT: Irma Arispe

PHONE NUMBER: 301/594-1455

PIC ID: 5634

PERFORMER ORGANIZATION: Rand Corporation, Santa Monica, CA

TITLE: Understanding and Choosing Clinical Performance Measures for Quality Improvement: Development of a Typology

ABSTRACT NUMBER: 028

ABSTRACT: Clinical performance measures are instruments that estimate the extent to which a health care provider (1) delivers appropriate clinical services in a safe, competent, and timely manner; and (2) achieves desired outcomes in terms of patient health and satisfaction. This project formulates a typology for analyzing the properties and uses of clinical performance measures that have been developed by government and nongovernment organizations, including AHCPR. The report stresses that the typology developed does not classify measures in terms of good and bad, but rather according to the setting for which they were developed, the purpose for which they were developed, and the properties that relate to the measures' potential suitability for its original or other purpose. The report also discusses using performance measures for quality improvement and consumer education. It discusses the use of performance measure sets, ranging in size from less than 10 measures to hundreds. Furthermore, the report examines the desired properties of measures, including (1) detailed specifications to ensure uniformity of application; (2) testing for reliability and validity as measures of performance; (3) allowance for patient differences in making comparisons; and (4) well-planned graphical displays of comparisons. Finally, the report discusses making the measures affordable and offers possible uses for them. (Final report variously paginated, plus appendixes.)

AGENCY SPONSOR: Office of Planning and Evaluation

FEDERAL CONTACT: Irma Arispe

PHONE NUMBER: 301/594-1455

PIC ID: 5630

NTIS ACCESSION NUMBER: PB 96-144639

PERFORMER ORGANIZATION: Center for Health Policy Studies, Columbia, MD

TITLE: Design of a Survey to Monitor Consumers' Access to Care, Use of Health Services, Health Outcomes, and Patient Satisfaction: Final Report

ABSTRACT NUMBER: 029

ABSTRACT: This project developed survey instruments to be used by health plans, employers, health alliances, States, accrediting agencies, the Federal Government, and others. The surveys measure consumers' assessments of their health insurance plans, in order to provide meaningful and useful information to other consumers choosing a health insurance plan. The report presents a set of prototype instruments containing an initial set of questions and procedures to assess a range of consumer issues. The report also contains an overview of the conceptual development process and the rationale for the current survey design. The methods and results of cognitive testing are described, as is the logic and evolution of the items contained in the prototype survey. The report concludes with a discussion of the prototype survey's strengths and limitations and a brief overview of recommendations for additional development and testing. (Final report 85 pages, plus appendixes.)

AGENCY SPONSOR: Center for Quality Measurement and Improvement

FEDERAL CONTACT: Sandra Robinson

PHONE NUMBER: 301/594-1352

PIC ID: 6011

PERFORMER ORGANIZATION: Research Triangle Institute, Research Triangle Park, NC

TITLE: Overcoming Barriers to Implementation and Integration of Clinical Information Management Systems: Feasibility Study

ABSTRACT NUMBER: 030

ABSTRACT: Clinical information systems capture and retrieve information at the point of health care delivery, display clinically meaningful results from multiple services, display chronological histories of patients' medical interventions, compare data on patients with similar characteristics, and share information between sites of care. Although such systems are commercially available, hospitals have been slow to implement them. This report examines the sociological, organizational, technological, and financial barriers to the implementation and optimal use of integrated clinical information systems. The report provides details on the feasibility of a study of 8 to 16 sites representing 4 to 8 major vendor products. It also provides details on site selection, research design, and a data collection plan. The report finds that the study should use focused case studies to carry out four main objectives: (1) identify target systems and provider site characteristics; (2) identify and recruit provider sites; (3) identify and solicit vendor participation; and (4) design an evaluation plan for the study, including a process and structure evaluation and an outcomes study. The report also gives a schedule for the implementation of the study. (Final report variously paginated, plus appendixes.)

AGENCY SPONSOR: Office of Planning and Evaluation

FEDERAL CONTACT: Irma Arispe

PHONE NUMBER: 301/594-1455

PIC ID: 5039

PERFORMER ORGANIZATION: Abt Associates, Inc., Cambridge, MA

TITLE: Practice Patterns and Expenditures in Managed Care and Indemnity Plans: A Condition-Specific Analysis

ABSTRACT NUMBER: 031

ABSTRACT: This project studied overall costs and utilization in managed care settings and how differences in practice patterns affect these variables. The project focused, in particular, on differences in practice patterns in managed care settings. The project assessed the use of selected tests and procedures for patients with heart attacks, congestive heart failure, delivery, back pain, or asthma. One of the conclusions of the project was that although the volume of claims studied was enormous, the coding used on the claim forms appeared to allow the selection of specific target procedures and tests. Also, the prevalence of such interventions was enough to detect important differences in practice patterns. In general, the use of claims to assess differences in practice patterns seems promising. However, during the project, problems continued in the receipt of usable data, causing one of the three participating carriers to withdraw from the project. This withdrawal terminated the project because confidentiality could no longer be guaranteed. General lessons from these experiences relate to project design and data management. For example, projects of this sort should be segmented, with initial data acquisition and cleaning coming first. Thus, carriers will be participating in a funded project, but large amounts of resources will not be expended until all the data are acquired.

AGENCY SPONSOR: Center for Primary Care Research

FEDERAL CONTACT: Linda A. Siegenthaler

PHONE NUMBER: 301/594-1357

PIC ID: 5664

PERFORMER ORGANIZATION: University of California, San Francisco, San Francisco CA

TITLE: Directory of Minority Health and Human Services Data Resources

ABSTRACT NUMBER: 032

ABSTRACT: This directory is intended for use by policymakers, researchers, and the public as a reference to resources within the Department of Health and Human Services that contain race and ethnicity data. Databases with broad utility, from ongoing departmental data projects to program administrative or evaluation activities, are included in the directory. The directory includes information from the Office of the Secretary, the Administration for Children and Families, the Health Care Financing Administration, the nonindependent Social Security Administration, and the Public Health Service. Each entry includes the following information: (1) the title of the data source; (2) the sponsoring office; (3) an abstract summarizing key characteristics of the database; (4) the race and ethnicities categorized in the database; (5) data limitations; (6) other data; (7) the program supported by the data project or system; (8) the general purpose of the database; (9) the status of the database; (10) the start date, start date period, or start/end dates; (11) the availability of the database; (12) constraints on use of the database; (13) the data media; (14) the distributor of public use files; (15) the agency contact person; and (16) the available data products. (Final report 278 pages.)

AGENCY SPONSOR: Center for Information Technology

FEDERAL CONTACT: Harvey Schwartz

PHONE NUMBER: 301/594-1483

PIC ID: 5911

PERFORMER ORGANIZATION: Moshman Associates, Inc., Bethesda, MD

TITLE: Development of Designs for Evaluation of the Process of Clinical Guideline Development

ABSTRACT NUMBER: 033

ABSTRACT: This project was a response to AHCPR Reauthorization Act of 1992. The project focused on the development and testing of designs to evaluate AHCPR's process for guideline development. The project identified key dimensions of the process of clinical practice guideline development and compared these dimensions among AHCPR and non-AHCPR-funded guidelines. The key dimensions selected were chosen because they varied across guideline development efforts; and they were viewed by experts as likely to affect the quality of guidelines and their acceptance by clinicians and other targeted audiences. The report recommends that AHCPR (1) consider pursuing its legislative mandate to review and assess guidelines developed by other organizations using appropriate methodologies; (2) examine mechanisms to streamline the organizational arrangements of expert panels (one of two types of panels used by AHCPR); (3) consider the development of standing panels, each addressing a broad class of conditions or technologies; (4) consider obtaining some front-end external review during the guideline development process to obtain information that could enhance the usefulness of the guideline to clinicians; (5) consider identifying and training a small pool of consumer representatives who could serve on multiple panels; and (6) undertake research to determine how the development process relates to the use of guidelines by key end users.

AGENCY SPONSOR: Office of Planning and Evaluation

FEDERAL CONTACT: Irma Arispe

PHONE NUMBER: 301/594-1455

PIC ID: 5629

PERFORMER ORGANIZATION: George Washington University Medical Center, Washington, DC

TITLE: Users' Perceptions of Four Clinical Practice Guidelines and Their Development Process

ABSTRACT NUMBER: 034

ABSTRACT: Clinical practice guidelines assist practitioners, health care providers, and consumers in making decisions about health care interventions for specific clinical circumstances. This report determines if aspects of various guideline development processes are associated with typical users' perceptions about the guideline's validity, reliability, cost, and utility. The report finds that (1) clinical applicability and ease of use are two major criteria by which participants assess the potential value of clinical guidelines and upon which they base their decisions to adopt and use them; (2) most participants want guidelines that help them solve problems in a clinical setting by updating their knowledge, outlining the latest developments, or providing guidance in areas where the participants' experience is limited; (3) panel selection, topic selection, and literature review are critical to a guideline's validity; (4) participants believe that recommendations made in the clinical guidelines must be verified by their own judgment and experience; and (5) some participants also feel that inappropriate implementation of clinical guidelines can have a negative impact on practice. The report recommends that (1) AHCPR continues to develop--under rigorous conditions--guidelines that reflect changes in the field of medicine. (Final report 110 pages, plus appendixes.)

AGENCY SPONSOR: Office of Planning and Evaluation

FEDERAL CONTACT: Irma Arispe

PHONE NUMBER: 301/594-1455

PIC ID: 5956

PERFORMER ORGANIZATION: American Institutes for Research, Washington, DC

TITLE: Setting Priorities for Clinical Practice Guidelines

ABSTRACT NUMBER: 035

ABSTRACT: This project described and evaluated possible methods for setting priorities for guideline topics. The report concludes that the current priority-setting approach used by AHCPR is relatively open, fairly explicit, and generally defensible. The report recommends that six general criteria be applied in considering topics for guidelines and technology assessment: prevalence of the problem; burden of illness imposed by the problem; cost; variability in practice; potential of a guideline or assessment to improve health outcomes; and the potential of a guideline or assessment to reduce costs. The report recognizes that the data used to score particular topics on these six criteria will often be incomplete. The report recommends possible ways to improve the procedures used in setting priorities for clinical practice guidelines. These recommendations focus on (1) methods for obtaining expert judgment and developing consensus positions; and (2) the desirability of a basic procedure manual. The report also recommends that AHCPR more precisely and narrowly define the topics selected for guideline development. The report concludes that AHCPR could play a useful role as a clearinghouse that collects and disseminates guidelines developed by other organizations. It also concludes that AHCPR needs an explicit, open, and defensible process for assessing the soundness of guidelines developed by other organizations.

AGENCY SPONSOR: Center for Primary Care Research

FEDERAL CONTACT: David Lanier, M.D.

PHONE NUMBER: 301/594-1357

PIC ID: 5631

PERFORMER ORGANIZATION: Institute of Medicine, Washington, DC

TITLE: AHCPR Clinical Practice Guideline Program: Report to Congress

ABSTRACT NUMBER: 036

ABSTRACT: The AHCPR Reauthorization Act of 1992 (Public Law 102-410) required that AHCPR report to Congress on three aspects of its clinical practice guideline program: methods for selecting guideline topics, methods for developing guidelines, and methods for assessing guideline quality. This report summarizes AHCPR's clinical practice guideline program to date in areas including guideline development, research and evaluation activities, and analyses of the guideline program conducted by organizations such as the Physician Payment Review Commission and the Office of Technology Assessment. It also examines potential directions for the future. (Final report 39 pages, plus appendixes.)

AGENCY SPONSOR: Office of the Forum for Quality and Effectiveness in Health Care

FEDERAL CONTACT: Kathryn Rickard

PHONE NUMBER: 301/594-1455

PIC ID: 5949

PERFORMER ORGANIZATION: Agency for Health Care Policy and Research, Rockville, MD

Centers for Disease Control and Prevention

Contents

Development of a National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP) Evaluation for Decisionmaking Strategy: Final Report

Development of a National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP) Evaluation for Decisionmaking Strategy: Interim Final Report

Effect of Mandated Managed Care for Medicaid Populations on the Practice of Public Health: The Example of Childhood Lead Poisoning Prevention (CLPP)

Evaluation of External Cause-of-Injury Codes: Final Report (Three Volumes)

Evaluation of the Activities of the Medical Examiner/Coroner Information-Sharing Program (MECISP)

Evaluation of the Content and Conduct of Health Examination Surveys

Evaluation of Encarguese de Su Diabetes: Una Guia Para Su Cuidado

Evaluation of the Fatality Assessment and Control Evaluation (FACE) Program: Dissemination Component

Evaluation of the Feasibility of an Injury Risk Factor

Evaluation of the Impact of a Diabetes Guide for Patients

Evaluation of the Impact on Practice of the Prevention and Treatment of Complications of Diabetes: A Guide for Primary Care Providers

Evaluation of the Policies, Practices, and Implementation of HIV Education Programs in Schools

Final Report: Assessment of CDC's Press Relations Functions

Final Report: A Blueprint for CDC's Office of Women's Health: Functions, Strategies, and Activities

Final Report: Evaluation of the Operating Processes, Communication Techniques, and Management Approaches in the National Immunization Program (NIP)

Rapid Assessment of Influenza Vaccination in the United States

Report of Completion for the Automated Analysis on National Center for Health Statistics (NCHS) Data

Working With Community-Based Organizations To Advance the Childhood Lead Poisoning Prevention Agenda

TITLE: Development of a National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP) Evaluation for Decisionmaking Strategy: Final Report.

ABSTRACT NUMBER: 037

ABSTRACT: This project develops a comprehensive evaluation strategy to be incorporated into planning, budgeting, and legislative decisionmaking processes of the National Center for Chronic Disease Prevention and Health Promotion. The final report discusses the major findings of a 1993 review of the planning and evaluation environment at NCCDPHP. It finds that (1) getting programs into the field is a higher priority at CDC than is assessing the efficacy and effectiveness of these programs; (2) funding decisions are unrelated to evaluation; (3) there is little connection between planning and evaluation at NCCDPHP or CDC; and (4) a stronger commitment to evaluation is needed. A pilot test of the evaluation strategy developed by the project shows that (1) data transfers between Federal, State, and local levels are costly and imperfect, and they require an approach linked to public health objectives; (2) States are concerned that program performance indicators fail to capture the complexity of the States' problems and accomplishments; (3) a too-detailed and overly specific evaluation methodology poorly assesses State and local implementations of chronic disease prevention programs; and (4) the Government Performance and Review Act and block grant funding will have a substantial impact on NCCDPHP programs. The report recommends a flexible approach to evaluation for NCCDPHP. See also PIC ID 4746. (Final report 95 pages, plus appendixes.)

AGENCY SPONSOR: National Center for Chronic Disease Prevention and Health Promotion

FEDERAL CONTACT: Lisa Daily

PHONE NUMBER: 770/488-5706

PIC ID: 4746.1

PERFORMER ORGANIZATION: Battelle Corporation, Arlington, VA

TITLE: Development of a National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP) Evaluation for Decisionmaking Strategy: Interim Final Report

ABSTRACT NUMBER: 038

ABSTRACT: This project develops a comprehensive evaluation strategy that can be incorporated into planning, budgeting, and legislative decisionmaking processes at NCCDPHP. This strategy will provide direction on whether Center programs can be evaluated. It will guide decisions about which program components should be evaluated, will provide direction regarding the scope of the evaluation, will clarify the types of data that should be routinely collected to support planning and evaluation activities, and will indicate when and how evaluation results should be fed back into the decisionmaking process. A six-step iterative process was used to develop the Evaluation for Decisionmaking Strategy. The strategy incorporates the views of public health stakeholders and was developed with the advice of NCCDPHP managers, policymakers, and evaluators. The final concept for the strategy attempts to balance the scientific rigor needed for proper evaluations and the flexibility required when dealing with real programs. The strategy links evaluation to the decisionmaking process by defining five program decision points that are associated with stages of program development. These decision points are (1) identification of needs; (2) examination of the policy context; (3) program design; (4) program implementation; and (5) program maintenance. A user's guide is provided to help NCCDPHP planners and evaluators implement the strategy. (Interim report 73 pages, plus appendixes.)

AGENCY SPONSOR: National Center for Chronic Disease Prevention and Health Promotion

FEDERAL CONTACT: Lisa Daily

PHONE NUMBER: 770/488-5403

PIC ID: 4746

PERFORMER ORGANIZATION: Battelle Corporation, Arlington, VA

TITLE: Effect of Mandated Managed Care for Medicaid Populations on the Practice of Public Health: The Example of Childhood Lead Poisoning Prevention

ABSTRACT NUMBER: 039

ABSTRACT: This report examines how the move toward managed care by State Medicaid programs affects the provision of the major service components of CDC Childhood Lead Poisoning Prevention (CLPP) programs. It also examines how CLPP programs have responded to the challenges of managed care. Site visits to Florida, Indiana, Minnesota, Rhode Island, and Tennessee were conducted. The report finds that (1) managed care will affect all components of CLPP programs because it leads to decentralization and privatization of functions that are currently centralized; (2) State and local health departments will spend less time on assurance and more on assessment and policy development under managed care, and they will also have a diminished role in direct service delivery; and (3) many of the changes in the managed care environment may make it difficult to obtain the sort of data needed to perform new functions such as assessment and quality assurance. The report discusses several strategies that CDC can use to adapt to the managed care environment. In general, these strategies are in the areas of (1) maintaining public sector service delivery; (2) building a legal infrastructure, whether this involves a screening law, a reporting law, or an enforcement of protocols and data collection to support enforcement; and (3) funding core public health functions. See also PIC ID 4373 and 4373.2. (Final report 29 pages.)

AGENCY SPONSOR: National Center for Environmental Health

FEDERAL CONTACT: Nancy Tips

PHONE NUMBER: 770/488-7277

PIC ID: 4743.1

NTIS ACCESSION NUMBER: PB 96-144670

PERFORMER ORGANIZATION: MACRO International, Inc., Silver Spring, MD

TITLE: Evaluation of External Cause-of-Injury Codes: Final Report (Three Volumes)

ABSTRACT NUMBER: 040

ABSTRACT: This report evaluates the current system used to classify external causes of injury in order to facilitate the correct use of the International Classification of Diseases "Supplemental Classification of External Causes of Injury and Poisoning" (E-Codes). The study evaluates the E-Code tabular list and alphabetical index and recommends improvements; (2) evaluates existing E-Code coding guidelines and recommends improvements; (3) evaluates E-Code training materials and recommends improvements; and (4) develops and evaluates prototype training materials, including a model training syllabus for E-Code use. E-Code stakeholders were asked to participate in the evaluation by serving on a technical advisory panel. The panel evaluated and recommended changes to the E-Code tabular list and alphabetical index. These changes were field tested by training 138 subjects: medical record department coding staff from eight hospitals (four trauma hospitals and four community hospitals); and health information management students. Test sites in four States were used. The field test demonstrated that experienced coders liked the revisions and could use the new codes with minimal training. Participants praised expanded guidelines, the addition of clarifying examples, the new alphabetic index, and the new tabular list. (Final report three volumes, variously paginated, plus appendixes.)

AGENCY SPONSOR: National Center for Health Statistics

FEDERAL CONTACT: Perrianne Lurie

PHONE NUMBER: 301/436-4216

PIC ID: 4742

PERFORMER ORGANIZATION: Center for Health Policy Studies, Columbia, MD

TITLE: Evaluation of the Activities of the Medical Examiner/Coroner Information-Sharing Program (MECISP)

ABSTRACT NUMBER: 041

ABSTRACT: Of all deaths in the United States, about 20 percent are investigated by medical examiner or coroner (ME/C) offices. In all, more than 2,200 jurisdictions are responsible for death investigation and certification. Records from these jurisdictions are timely and detailed, and constitute the ultimate source of information on sudden and unexpected deaths. CDC established MECISP in 1987 to coordinate these data for researchers and public health officials. MECISP's main product is a database containing data collected from ME/C sites. MECISP also produces a biannual directory listing ME/C jurisdictions and summarizing death investigation laws.

This report reviews the activities at MECISP in order to evaluate and suggest improvements to the process of sharing information on death investigation in the United States. The report finds that (1) MECISP's main role is to be a broker between death investigators and people who can best use data from these investigations to improve public health; (2) death investigation data, particularly MECISP data, are underused, because of problems with access, visibility, relevance, coverage, and quality; and (3) correcting these problems requires resources that MECISP does not have, such as staff with technical expertise in the area. The report makes several recommendations, chiefly one that calls for greater resources.

AGENCY SPONSOR: National Center for Environmental Health

FEDERAL CONTACT: Gib Parrish, M.D.

PHONE NUMBER: 404/488-3310

PIC ID: 5350

PERFORMER ORGANIZATION: MACRO International, Inc., Silver Spring, MD

TITLE: Evaluation of the Content and Conduct of Health Examination Surveys

ABSTRACT NUMBER: 042

ABSTRACT: The Third National Health and Nutrition Examination Survey (NHANES III) is the seventh in a series of surveys with health examination components conducted by the National Center for Health Statistics (NCHS) since 1960. The survey asked 30,000 persons about their demographic, socioeconomic, dietary, and health history. An examination component for each participant includes phlebotomy, as well as examinations by a physician, a dentist, specialized interviewers, and health technicians. About 5,000 sample persons are examined each year in two 3-year cycles of data collection. NCHS is considering a design for NHANES-97 that includes recurring 2-year data collection cycles, which will examine a sample of about 7,500 each year (a 50 percent increase). This report (1) explores methods for increasing the sample size by 50 percent; (2) examines the core content of the examination; (3) evaluates other health examination studies; and (4) discusses the feasibility of collecting specified nutrition assessment measures in a setting other than a mobile examination center. For each of these tasks, the report presents objectives, methodology, and major findings and recommendations. This report is available from NTIS only. (Final report two volumes, variously paginated.)

AGENCY SPONSOR: National Center for Health Statistics

FEDERAL CONTACT: Natalie Fuller-Dupree

PHONE NUMBER: 301/436-7080

PIC ID: 5512

NTIS ACCESSION NUMBER: PB 95-221974

PERFORMER ORGANIZATION: Westat, Inc., Rockville, MD

TITLE: Evaluation of Encarguese de Su Diabetes: Una Guia Para Su Cuidado

ABSTRACT NUMBER: 043

ABSTRACT: This project evaluated a pilot edition of a Spanish-language diabetes patient guide targeted to Hispanic populations. This guide, "Encarguese de Su Diabetes: Una Guia Para Su Cuidado," was developed by staff of the Division of Diabetes Translation at CDC. The rationale for this evaluation was to ensure that the final guide would be culturally relevant and readable for individuals with at least a sixth-grade level of education. Specifically, the project used focus group discussions to determine the understandability, relevance, usefulness, and adaptability of the guide among diverse Hispanic populations in the United States. The focus group discussions were held throughout the United States. Phase I of the project resulted in a revised Spanish-language version of the guide, and phase II was directed at refining issues related to finalization, production, and distribution of the guide.

AGENCY SPONSOR: National Center for Chronic Disease Prevention and Health Promotion

FEDERAL CONTACT: Patricia Mitchell

PHONE NUMBER: 770/488-5015

PIC ID: 5352

PERFORMER ORGANIZATION: Casals and Associates, Arlington, VA

TITLE: Evaluation of the Fatality Assessment and Control Evaluation (FACE) Program: Dissemination Component

ABSTRACT NUMBER: 044

ABSTRACT: FACE identifies and investigates fatal occupational injuries, with the goal of preventing them. It identifies work situations at high risk for fatal injury and formulates and disseminates prevention strategies to those who can intervene in the workplace. This report assesses the effectiveness of information dissemination for FACE programs in three States. The report (1) describes communication channels and information users; (2) identifies prevention efforts resulting from information provided by FACE; (3) assesses indicators of changed work behaviors that might be attributed to the program; and (4) assesses the potential effectiveness of specific dissemination strategies within each of the three States. The report finds that (1) employers generally assess the program favorably; (2) some employers are concerned that the inclusion of specific points might cast an unfavorable light on their companies, and that some recommendations are economically or technically unfeasible; (3) most employers who have made workplace changes did so immediately following a fatality, and many reported these changes to other managers in the company; (4) FACE materials are widely used for professional education, public information, advocacy, regulation, surveillance, and vocational training; and (5) organizational and economic influences govern the dissemination of FACE materials. The report concludes that the program is effective and of high quality in each State reviewed.

AGENCY SPONSOR: National Institute of Occupational Safety and Health

FEDERAL CONTACT: Theodore Pettit

PHONE NUMBER: 304/284-5796

PIC ID: 5506

PERFORMER ORGANIZATION: Research Triangle Institute, Research Triangle Park, NC

TITLE: Evaluation of the Feasibility of an Injury Risk Factor

ABSTRACT NUMBER: 045

ABSTRACT: This report evaluates the feasibility of a national telephone survey as (1) a surveillance system for injury risk factors; (2) a means through which to monitor progress toward injury objectives for the year 2000; and (3) a tool for evaluating injury control programs. A national telephone survey of 5,238 adults was conducted. The response rate for the survey was 54 percent of households determined to be eligible for the study. While telephone response rates have been declining over the past decade, the 29 percent refusal rate for this survey was higher than anticipated. Analyses suggest that neither the sampling methodology nor the survey length were related to problems with response rates; but instead that these problems were related to (1) the perceived (rather than actual) length of the survey; (2) sensitivity of topics; and (3) informed consent procedures.

AGENCY SPONSOR: National Center for Injury Prevention and Control

FEDERAL CONTACT: Jeffrey Sacks, M.D.

PHONE NUMBER: 770/488-4652

PIC ID: 4522

PERFORMER ORGANIZATION: Battelle Corporation, Arlington, VA

TITLE: Evaluation of the Impact of a Diabetes Guide for Patients

ABSTRACT NUMBER: 046

ABSTRACT: This project provided information to CDC's Division of Diabetes Translation about how different dissemination approaches affected the behavior of providers, and how health care practice was impacted when the guides were presented to primary care practitioners as part of a specialized training seminar at community health centers. The study recommended that (1) programs have a champion at each local intervention site; (2) an adequate market for the intervention be present prior to program implementation; (3) sources of capital be targeted and linked to the unit of analysis; (4) agreements with each intervention site be obtained to ensure necessary levels of human and structural support; and (5) sufficient time be allowed for system change.

AGENCY SPONSOR: National Center for Chronic Disease Prevention and Health Promotion

FEDERAL CONTACT: Dawn Satterfield

PHONE NUMBER: 770/488-5020

PIC ID: 5934

PERFORMER ORGANIZATION: Research Triangle Institute, Research Triangle Park, NC

TITLE: Evaluation of the Impact on Practice of the Prevention and Treatment of Complications of Diabetes: A Guide for Primary Care Providers

ABSTRACT NUMBER: 047

ABSTRACT: In 1991, CDC's Division of Diabetes Translation published two guides for diabetes care. One, "The Prevention and Treatment of Complications of Diabetes: A Guide for Primary Care Providers," was aimed at health care providers. The second, "Take Charge of Your Diabetes: A Guide for Care," was aimed at patients. This project provides information to the Division of Diabetes Translation on the dissemination approaches for the provider guide. Three urban community health centers served as intervention sites. Two of them received the provider guide and a 4-hour education and activation program; one of these two also received the patient guide. The third center received the provider guide without explanation. The project determined how different dissemination approaches affect the behavior of providers, and how health care practice is impacted when the guides are presented to primary care practitioners as part of a specialized training seminar. The report finds no conclusive evidence that the intervention had a strong effect on practitioner behavior. Although there were some improvements, these could not be traced to the intervention. Furthermore, the intervention may not have been strong enough to affect provider behavior, especially given the high staff turnover rates at these centers. (Final report 56 pages, plus appendixes.)

AGENCY SPONSOR: National Center for Chronic Disease Prevention and Health Promotion

FEDERAL CONTACT: Dawn Satterfield

PHONE NUMBER: 404/488-5020

PIC ID: 4108

PERFORMER ORGANIZATION: Research Triangle Institute, Research Triangle Park, NC

TITLE: Evaluation of the Policies, Practices, and Implementation of HIV Education Programs in Schools

ABSTRACT NUMBER: 048

ABSTRACT: This project evaluated the progress and activities conducted by the 71 HIV education programs funded by CDC's Division of Adolescent and School Health (DASH). These programs are funded through cooperative agreements with State and local education agencies. Data elements were obtained from reading the program file documents and were abstracted for each funded year of the 5-year period, 1987-92, for each program. A database was composed from these elements that will allow CDC to answer specific evaluation questions regarding the development of programs and the extent to which CDC policies and guidelines were followed during the funding period.

AGENCY SPONSOR: National Center for Chronic Disease Prevention and Health Promotion

FEDERAL CONTACT: Peter Hunt

PHONE NUMBER: 404/488-5343

PIC ID: 6070

PERFORMER ORGANIZATION: MACRO International, Inc., Atlanta, GA

TITLE: Final Report: Assessment of CDC's Press Relations Functions

ABSTRACT NUMBER: 049

ABSTRACT: CDC's News Division within the Office of Public Affairs (OPA) was established in 1987 to handle inquiries from the press. The volume of calls to this office has more than doubled between 1991 (8,000 inquiries) and 1994 (more than 20,000 inquiries), but the number of press officers available to respond to calls has remained constant. This report examines OPA's News Division and its relationship with CDC centers, institutes, and offices (CIOs) and with the press. The key finding is that the News Division is understaffed for the volume of calls. The report also finds that some CIOs recognize that OPA is overwhelmed, yet they are pleased with the services it provides; others are generally dissatisfied and desire more proactive media relations services. Reporters say that they have difficulty reaching the News Division and that their inquiries are not answered in a timely way. Furthermore, a review of external press offices finds that OPA staff handle more than twice the number of inquiries than do staff of comparable organizations. The report recommends that (1) communications technology such as voice mail, batch faxing, electronic mail, and automated tracking of press inquiries be instituted; (2) staffing levels be increased to 9 or 10 press officers; (3) internal policies and procedures guiding current and future activities be developed; and (4) individual CIO needs be determined and met.

AGENCY SPONSOR: Office of the Director

FEDERAL CONTACT: Helen T. Kuykendall

PHONE NUMBER: 404/639-3286

PIC ID: 5915

PERFORMER ORGANIZATION: MACRO International, Inc., Silver Spring, MD

TITLE: Final Report: A Blueprint for CDC's Office of Women's Health: Functions, Strategies, and Activities

ABSTRACT NUMBER: 050

ABSTRACT: CDC established the Office of Women's Health (OWH) in 1994. This report identifies the role, functions, strategies, activities, and benchmarks for OWH. The report maintains that OWH has five key functions: (1) leadership; (2) advocacy; (3) internal coordination; (4) external relationship building at the national, State, and local levels; and (5) research and program development. The report also discusses the guiding principles of OWH and suggests how it should position itself in light of these principles. For example, the report stresses that OWH must have the goodwill of other offices within CDC, and suggests that the Associate Director for Women's Health be made a member of CDC's senior management team in order to highlight the vital role OWH will play. The report suggests that OWH (1) create a vision at CDC for women's health; (2) increase the visibility of women's health programs within CDC and to external audiences; (3) develop formal, systematic communication mechanisms for internal exchange of information about CDC's women's health activities; (4) facilitate coordination and collaboration between CDC and other Public Health Service offices that deal with women's health; and (5) dedicate allocated funding to support projects that address priorities and gaps in women's health efforts. The report also discusses such issues as building relationships, staffing, and priorities for the coming year. (Final report 24 pages, plus appendixes.)

AGENCY SPONSOR: Office of the Director

FEDERAL CONTACT: Wanda Jones

PHONE NUMBER: 404/639-7230

PIC ID: 5913

PERFORMER ORGANIZATION: MACRO International, Inc., Silver Spring, MD

TITLE: Final Report: Evaluation of the Operating Processes, Communication Techniques, and Management Approaches in the National Immunization Program (NIP)

ABSTRACT NUMBER: 051

ABSTRACT: Three events have placed significant demands upon CDC's National Immunization Program: (1) the measles outbreak in the late 1980's and early 1990's, which brought the importance of childhood immunization to the forefront of public attention; (2) the President's Childhood Immunization Initiative (CII) of 1994, which made the immunization of preschool children a key health priority; and (3) mandates to eliminate indigenous cases of specific vaccine-preventable diseases and to increase vaccine coverage to at least 90 percent among 2-year-old children. An increase in the NIP operation budget of 160 percent and a staff increase of approximately 50 percent occurred between fiscal 1993 and 1995. This report examines NIP's management structure and style, its communication and decisionmaking processes and strategies, its patterns of authority, and its delegation of work and responsibility. The report offers recommendations in the following areas: (1) strategic planning, (2) day-to-day management, and (3) staff issues. (Final report 43 pages, plus appendixes.)

AGENCY SPONSOR: National Immunization Program

FEDERAL CONTACT: Victoria Westberg

PHONE NUMBER: 404/639-8204

PIC ID: 5916

PERFORMER ORGANIZATION: MACRO International, Inc., Silver Spring, MD

TITLE: Rapid Assessment of Influenza Vaccination in the United States

ABSTRACT NUMBER: 052

ABSTRACT: This project determines the 1992-93 and 1993-94 influenza vaccine coverage in sites where CDC plans to conduct active surveillance for Guillain-Barre syndrome (GBS). This determination should permit CDC to (1) assess whether receipt of influenza vaccine in these years was associated with an increased risk for GBS; (2) determine whether a change in recommendations for influenza vaccination is warranted; and (3) provide insight into usage patterns of influenza vaccination.

AGENCY SPONSOR: National Immunization Program

FEDERAL CONTACT: Robert Chen

PHONE NUMBER: 301/402-3497

PIC ID: 5894

PERFORMER ORGANIZATION: MACRO International, Inc., Atlanta, GA

TITLE: Report of Completion for the Automated Analysis on National Center for Health Statistics (NCHS) Data

ABSTRACT NUMBER: 053

ABSTRACT: This project was intended to improve the analysis of NCHS data. The project evaluated NCHS's present analytical methods and developed a prototype automated data analysis system, the Automated System for Survey Information and Statistical Tools (ASSIST). During the project, ASSIST was used as a communications tool between NCHS analysts and the contractor staff to explore and assess data access. NCHS also considered the accessibility of the data to users whose computing expertise ranged from novice to expert. The prototype was also used in (1) evaluating and assessing problems associated with multiple data set storage; (2) determining the requirements for the use of data across data systems; and (3) determining whether both user and provisional data should be employed. (Final report five pages.)

AGENCY SPONSOR: National Center for Health Statistics

FEDERAL CONTACT: Marcus Sanchez

PHONE NUMBER: 301/436-7904

PIC ID: 3575

PERFORMER ORGANIZATION: American Management Systems, Inc., Arlington, VA

TITLE: Working With Community-Based Organizations To Advance the Childhood Lead Poisoning Prevention Agenda

ABSTRACT NUMBER: 054

ABSTRACT: As public health services evolve, CDC must recognize that at least three important changes in emphasis must occur within its CLPP programs. These three changes are (1) a shift from direct service delivery to assessment, policy development, and quality assurance; (2) a shift from secondary prevention to primary prevention; and (3) a new goal of coordinating the care and followup of lead-poisoned children and their families.

Joining into partnership with community-based organizations (CBOs) may help ease the impact of these changes. This report is intended to help CLPP grantees and local health departments build better relationships with CBOs and to provide suggestions as to how they can broaden their mission to include a community-based prevention agenda. The report gives examples of successful collaborations and concludes that (1) grantees may have difficulty finding appropriate CBOs to work with, since not all CBOs will be interested in preventing lead poisoning; (2) CBOs have a different accountability and organizational structure than public sector organizations and may have different working styles; (3) grantees should choose long-established CBOs with good records in the community; (4) CBOs, like the public sector, must compete for limited resources; and (5) CBOs may have high staff turnover and little institutional memory. See also PIC ID 4743 and 4743.1. (Final report 22 pages.)

AGENCY SPONSOR: National Center for Environmental Health

FEDERAL CONTACT: Nancy Tips

PHONE NUMBER: 404/488-7277

PIC ID: 4743.2

NTIS ACCESSION NUMBER: PB 96-144688

PERFORMER ORGANIZATION: MACRO International, Inc., Silver Spring, MD

Food and Drug Administration

Contents

Assessment of the Mammography Quality Standards Act of 1992 (MQSA)

Nationwide Evaluation of X-Ray Trends (NEXT)

Third Annual Performance Report on the Prescription Drug User Fee Act of 1992: Fiscal Year 1995 Report to Congress

TITLE: Assessment of the Mammography Quality Standards Act of 1992 (MQSA)

ABSTRACT NUMBER: 055

ABSTRACT: FDA is charged with ensuring that accessibility to quality mammography services is maintained for all facilities performing mammographies, in accordance with MQSA. Under this statute, FDA is required to certify and annually inspect over 10,000 mammography facilities. Prior to the passage of MQSA, FDA's oversight responsibility was limited to approving new mammographic processors, X rays, and tubing. However, MQSA requires that facilities within its jurisdiction meet quality standards developed by FDA if these facilities are to obtain certification by the Agency. FDA is concerned that recent trends by third-party payers to captivate payments for mammography services have severely limited the ability of facilities to absorb any cost increases caused by the quality standards. These cost increases, as well as excess capacity, may cause low-volume facilities to leave the industry. This project addresses these problems in two ways. First, it measures the effect of standards on patient access to mammography, especially in rural areas. It also develops a cost model that will be used to predict the impact of proposed standards on the costs of providing mammography. This project addresses these issues by collecting and analyzing data from facilities that provide mammography services.

AGENCY SPONSOR: Office of Planning and Evaluation

FEDERAL CONTACT: Steve Tucker

PHONE NUMBER: 301/443-1839

PIC ID: 6080

PERFORMER ORGANIZATION: Eastern Research Group, Inc., Rosslyn, VA

TITLE: Nationwide Evaluation of X-Ray Trends (NEXT)

ABSTRACT NUMBER: 056

ABSTRACT: NEXT is a collaborative program conducted jointly between FDA and State radiation control agencies. The study estimates the radiation dosage from diagnostic X-ray examinations, using data collected from previous NEXT surveys of different types of X-ray examinations. The study generates data on image quality, photographic processing quality, and types of practice. The data are used by public health and professional organizations to set policy. The studies are conducted in cooperation with the Conference of Radiation Control Directors. Products generated in the study include (1) "National Survey of Mammographic Facilities," in 1985, 1988, and 1992; (2) "Mammography (and Radiology?) Is Still Plagued With Poor Quality in Photographic Processing and Darkroom Fog"; (3) "Assessment of Skin Entrance Kerma in the United States: The Nationwide Evaluation of X Ray Trends (NEXT)"; (4) "How Good (or Bad) Is Film Processing?"; and (5) "Automatic Film Processing: Analysis of 9 Years of Observations."

AGENCY SPONSOR: Center for Devices and Radiological Health

FEDERAL CONTACT: Orhan Suleiman

PHONE NUMBER: 301/594-3332

PIC ID: 4984

PERFORMER ORGANIZATION: Conference of Radiation Control Directors, Washington, DC

TITLE: Third Annual Performance Report on the Prescription Drug User Fee Act of 1992: Fiscal Year 1995 Report to Congress

ABSTRACT NUMBER: 057

ABSTRACT: Fiscal 1995 marked the third year of the 5-year implementation of the Prescription Drug User Fee Act (PDUFA) of 1992. This Act authorized the collection of fees from the pharmaceutical industry in order to facilitate FDA's timely review of human drug applications. The Act mandates that FDA prepare yearly reports for each fiscal year in which fees are collected. This third annual report, for fiscal 1995, was prepared in response to that mandate. It presents findings as to how well FDA has accomplished its performance goals for fiscal 1995. These performance-related goals were jointly established by FDA and the pharmaceutical industry and fall into three main categories: eliminating overdue backlogs, building excellence into the review process, and achieving high performance. The report finds that (1) New Drug Applications (NDAs) approval was 23 percent higher after PDUFA's enactment than in the 2 years prior; (2) a high percentage of first reviews were accomplished within the 12-month timeframe specified in the PDUFA; and (3) other reasons for quicker approval times include high rates of affirmative first actions, quick action on approvables, and effective "refuse-to-file" screening. (Final report 24 pages, plus appendixes.)

AGENCY SPONSOR: Office of Planning and Evaluation

FEDERAL CONTACT: Paul Coppinger and Kate McEvoy

PHONE NUMBER: 301/443-4230

PIC ID: 6079

PERFORMER ORGANIZATION: Office of Planning and Evaluation, Food and Drug Administration, Rockville, MD

Health Care Financing Administration

Contents

Comparative Study of the Use of Early and Periodic Screening, Detection, and Treatment and Other Preventive and Curative Health Care Services by Children Enrolled in Medicaid

Evaluation of the Maryland Access to Care Demonstration: Managed Care for Medicaid Recipients

Evaluation of the Medicaid Extension Demonstrations

Evaluation of United Mine Workers of America Demonstration

Expanded Cross-Cutting Evaluation on Medicare Prevention Demonstrations Under Consolidated Omnibus Budget Reconciliation Act

Final Evaluation Report of the 1989 Grant Program for Rural Health Transition: Send Us More Doctors, Please

Interim Report of the Evaluation of the Essential Access Community Hospital/Rural Primary Care Hospital Programs

Medicare Participating Heart Bypass Center Demonstration: Appropriateness Study

Medicare Participating Heart Bypass Center Demonstration: Data Collection Design

Medicare Participating Heart Bypass Center Demonstration: Evaluation Design

Medicare Participating Heart Bypass Center Demonstration: Final Evaluation Report--Volume One, The First Three Years

Medicare Participating Heart Bypass Center Demonstration: Final Evaluation Report--Volume Two, Marketing Activities of Participating Hospitals

Medicare Risk Program for HMOs: Final Summary Report on Findings From the Evaluation

Monitoring the Impact of Medicare Physician Payment Reform on Utilization and Access

TITLE: Comparative Study of the Use of Early and Periodic Screening, Detection, and Treatment and Other Preventive and Curative Health Care Services by Children Enrolled in Medicaid

ABSTRACT NUMBER: 058

ABSTRACT: This project was comprised of a series of individual research projects that studied the effect of the changes in the Early and Periodic Screening, Diagnosis, and Treatment (EPSDT) Program introduced by the Omnibus Budget Reconciliation Act of 1989 on the process of providing health services to children and on the appropriateness of expenditures for the services provided in Tennessee, Georgia, Michigan, and California. These projects compared Medicaid-enrolled children in four EPSDT programs with other Medicaid-enrolled children in the four States who are not receiving EPSDT services, regarding enrollment patterns, service utilization, and expenditures, with a particular emphasis on preventive health services. They also compared Medicaid-enrolled children with non-Medicaid-enrolled children, insured and uninsured, on the use of and expenditures for preventive services and other health care services, using national survey data.

AGENCY SPONSOR: Office of Research and Demonstrations

FEDERAL CONTACT: Feather Ann Davis, Ph.D.

PHONE NUMBER: 410/786-6590

PIC ID: 6236

PERFORMER ORGANIZATION: SysteMetrics Division of MedStat, Inc., Ann Arbor, MI

TITLE: Evaluation of the Maryland Access to Care Demonstration: Managed Care for Medicaid Recipients

ABSTRACT NUMBER: 059

ABSTRACT: The Maryland Access to Care (MAC) demonstration became operational in December 1991 and eventually covered about two-thirds of Medicaid recipients in the State. The targeted population was Aid to Families with Dependent Children (AFDC) recipients, Supplemental Security Income (SSI) participants, and Sixth Omnibus Budget Reconciliation Act-eligible children. The MAC program was mandatory for recipients in the MAC-eligible categories. The program matched MAC recipients with a primary medical provider (PMP) that acted as the recipient's gatekeeper to the health care system. These PMPs also continued under the standard Medicaid fee-for-service reimbursement systems, but, to encourage their participation, Medicaid fees for primary care services were increased by an average of 50 percent under the MAC program. Specialists were reimbursed for nonemergency specialty care provided to MAC patients only if these services were referred by the patients' PMPs. The evaluation employed a pre-/post-test comparison and a post-test description of program operations. The data came primarily from Medicaid enrollment and claims files and from provider surveys. The project was extended to allow the performance of additional analyses to compare the performance of office-based and hospital outpatient providers under the MAC program.

AGENCY SPONSOR: Office of Research and Demonstrations

FEDERAL CONTACT: Paul J. Boben, Ph.D.

PHONE NUMBER: 410/786-6629

PIC ID: 6235

PERFORMER ORGANIZATION: People-to-People Health Foundation, Inc., Bethesda, MD

TITLE: Evaluation of the Medicaid Extension Demonstrations

ABSTRACT NUMBER: 060

ABSTRACT: This project was an evaluation of three demonstrations mandated under section 6407 of the Omnibus Budget Reconciliation Act of 1989. They evaluated alternative models for extending health insurance coverage to children under 20 years of age who lacked insurance. The demonstrations occurred in the States of Florida, Maine, and Michigan. Each State used a different strategy for providing the new coverage. Florida tested the effectiveness of marketing a school-based affordable insurance package that delivers services through a managed care network. Maine conducted a statewide program that subsidized comparable private employer-based group coverage, where such insurance was believed to be cost effective. Michigan tested the effectiveness of a public/private partnership between the State and Michigan Blue Cross and Blue Shield, using donated funds to subsidize a mainstream outpatient insurance package. The evaluation examined the effect of these demonstrations on various outcome and process measures of access to care, private insurance coverage, and cost of care. The methodology took into account the distinctiveness of the three demonstrations while incorporating a strategy that allowed for comparisons between programs in terms of penetrating the eligible population. Case studies were coupled with analysis of program data to describe the structure and processes of the demonstrations. In addition, primary data were collected through surveys of both program participants and controls. Separate analyses of program costs and program effectiveness were included.

AGENCY SPONSOR: Office of Research and Demonstrations

FEDERAL CONTACT: Paul J. Boben, Ph.D.

PHONE NUMBER: 410/786-6629

PIC ID: 6234

PERFORMER ORGANIZATION: Abt Associates, Inc., Cambridge, MA

TITLE: Evaluation of United Mine Workers of America Demonstration

ABSTRACT NUMBER: 061

ABSTRACT: This is an evaluation of the United Mine Workers of America (UMWA) Health and Retirement Funds (the UMW Funds) Medicare Part B capitation demonstration. This demonstration replaced the Funds' Health Care Prepayment Plan arrangement with the Health Care Financing Administration (HCFA), in which it had been reimbursed for Medicare Part B services on a cost basis. In its place, the Funds assumed risk for Medicare Part B services under a capitated payment mechanism. The evaluation addressed the cost-effectiveness of capitation based on an analysis of changes in utilization and cost resulting from the demonstration. It also undertook a detailed case study describing the cost management programs and changes occurring in the organization as a result of the demonstration. Based on this approach, the evaluation was unable to discern a causal relationship between the demonstration and the events that occurred subsequent to its implementation. Events that happened during the demonstration may have been independent or could have been related in complicated ways. Thus, in an interim report the evaluation concluded that the demonstration had no measurable operational effect on the Funds beyond the obvious effect of ending the reimbursement dispute between HCFA and UMW Funds. The final report indicated that there was no evidence to clearly support a conclusion about the cost-effectiveness of capitation for UMW Funds' beneficiaries.

AGENCY SPONSOR: Office of Research and Demonstrations

FEDERAL CONTACT: Ronald W. Lambert

PHONE NUMBER: 410/786-6624

PIC ID: 6233

PERFORMER ORGANIZATION: Abt Associates, Inc., Cambridge, MA

TITLE: Expanded Cross-Cutting Evaluation of Medicare Prevention Demonstrations Under Consolidated Omnibus Budget Reconciliation Act

ABSTRACT NUMBER: 062

ABSTRACT: This project was a cross-cutting evaluation of the five Medicare prevention demonstrations mandated by the Consolidated Omnibus Budget Reconciliation Act (COBRA) of 1985. These demonstrations tested the effectiveness of providing disease prevention and health promotion services to Medicare beneficiaries. Congress stipulated that the preventive health service package was to include health screenings, health risk appraisals, immunizations, and counseling and instruction in diet and nutrition, stress reduction, exercise programs, sleep regulation, injury prevention, substance abuse and mental disorders prevention, self-care (including medication use), and smoking cessation. In May 1988, cooperative agreements were awarded to five schools of public health to implement the demonstrations. Waivered services were provided between May 1989 and April 1991. A preliminary Report to Congress was submitted in July 1989. While the original legislation mandated 4-year demonstrations, the Omnibus Budget Reconciliation Act (OBRA) of 1990 extended them to 5 years. This extension allowed for an additional year of followup for purposes of evaluation, added an interim Report to Congress (which was submitted in September 1993), and required that the final Report include a comprehensive evaluation of the long-term effects of the demonstration.

AGENCY SPONSOR: Office of Research and Demonstrations

FEDERAL CONTACT: Deborah C. Van Hoven

PHONE NUMBER: 410/786-6625

PIC ID: 6237

PERFORMER ORGANIZATION: Abt Associates, Inc., Cambridge, MA

TITLE: Final Evaluation Report of the 1989 Grant Program for Rural Health Transition: Send Us More Doctors, Please

ABSTRACT NUMBER: 063

ABSTRACT: This report evaluates the impact of the Rural Health Care Transition (RHCT) grant program on the cohort of 181 rural hospitals that received grants in September 1989. The report uses three methods to evaluate the program: (1) a trend analysis of all grantees to determine whether the program improved their utilization rates and finances; (2) a pregrant/postgrant comparison to examine managerial improvements; and (3) a case study, descriptive analysis of projects implemented. The report finds that (1) the grant program does not seem to affect hospital finances or managerial capacity in a measurable way, but it does help some hospitals and areas where the majority of patients served would have had to travel or go without services in the absence of the grant program projects; (2) adult day care programs are least likely to be implemented and, if implemented, to be retained; (3) patient services such as community education programs are fairly easy to implement, but are likely to be abandoned because of high cost; and (4) outpatient, home health, emergency rooms, and emergency transportation programs are most frequently implemented and retained. The report concludes that the grant program does not address fundamental problems facing rural hospitals. The program cannot change the hospitals' financial weakness, their high numbers of uninsured patients, the low demand for their services, the high cost of providing low-volume emergency services, or the shortage of rural physicians. (Final report 168 pages, plus appendixes.)

AGENCY SPONSOR: Office of Research and Demonstrations

FEDERAL CONTACT: Siddhartha Mazumdar

PHONE NUMBER: 410/786-6673

PIC ID: 5283

NTIS ACCESSION NUMBER: PB 94-142312

PERFORMER ORGANIZATION: Mathematica Policy Research, Inc., Plainsboro, NJ

TITLE: Interim Report of the Evaluation of the Essential Access Community Hospital/Rural Primary Care Hospital Programs

ABSTRACT NUMBER: 064

ABSTRACT: This report documents the initial development of the Essential Access Community Hospital (EACH) and the Rural Primary Care Hospital (RPCH) programs, which are Federal-State initiatives to maintain access to health care services in rural areas by developing rural health networks and by integrating health care. Through qualitative and quantitative analysis of background information provided by program grantees, secondary data from HCFA's Medicare Cost Reports (1987-89) and Market Area File (1988-89), and site visits to State agencies and hospitals in California, Colorado, Kansas, New York, North Carolina, South Dakota, and West Virginia, the report examines program development at the State and local levels. It finds that (1) States have made uneven progress in program development; (2) program planning and network development are difficult, labor intensive, and time consuming; and (3) the program has encouraged the development of other non-EACH, non-RPCH networks. However, the report also finds that (1) hospital commitment to RPCH conversion is weak; (2) financial stresses encourage hospitals to convert; (3) networks are shaped by competitive relationships between hospitals and by local circumstances; and (4) physicians' support, or at least their nonopposition, is critical to the success of the program. (Final report 115 pages, plus appendixes.)

AGENCY SPONSOR: Office of Research and Demonstrations

FEDERAL CONTACT: Sheldon Weisgrau

PHONE NUMBER: 410/786-6675

PIC ID: 5200

NTIS ACCESSION NUMBER: PB 94-117959

PERFORMER ORGANIZATION: Mathematica Policy Research, Inc., Plainsboro, NJ

TITLE: Medicare Participating Heart Bypass Center Demonstration: Appropriateness Study

ABSTRACT NUMBER: 065

ABSTRACT: This series of reports was prepared as part of the preapproval package for HCFA's Medicare Participating Heart Bypass Center Demonstration. The reports in this series deal with the appropriateness of coronary artery bypass graft (CABG) surgery and percutaneous transluminal coronary angioplasty (PTCA) surgery prior to approval of the demonstration. The reports in this series include "Model for the Use of CABG and PTCA"; "Appropriateness Rating Scale for CABG and PTCA"; "Indications for Coronary Artery Bypass Graft Surgery"; "Review of Literature for Efficacy and Risks of CABG Surgery"; and "Review of Literature for Efficacy and Risks of PTCA." See also PIC ID 5958-5958.3.

AGENCY SPONSOR: Office of Research and Demonstrations

FEDERAL CONTACT: Armen H. Thoumaian, Ph.D.

PHONE NUMBER: 410/966-6672

PIC ID: 5958.4

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Medicare Participating Heart Bypass Center Demonstration: Data Collection Design

ABSTRACT NUMBER: 066

ABSTRACT: This report provides information on the data collection methodology to be used in the HCFA Medicare Participating Heart Bypass Center Demonstration. This report presents the rationale for the evaluation, the data elements required, and the procedures for collecting them. It discusses several economic issues of interest to HCFA, including sources of volume increases at the demonstration sites and the relevant savings to the Medicare program (if any), and the demonstration administrative costs anticipated at the original four participating hospitals. The report also presents data collection related to the evaluation of the appropriateness of coronary artery bypass graft surgery. Finally, the report discusses the assessment of the hospitals' marketing activities in order to measure their varying levels of success at promoting the demonstration. See also PIC ID 5958-5958.2 and 5958.4. (Final report 32 pages, plus appendixes.)

AGENCY SPONSOR: Office of Research and Demonstrations

FEDERAL CONTACT: Armen H. Thoumaian, Ph.D.

PHONE NUMBER: 410/966-6672

PIC ID: 5958.3

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Medicare Participating Heart Bypass Center Demonstration: Evaluation Design

ABSTRACT NUMBER: 067

ABSTRACT: This report provides information about the design of the evaluation of HCFA's Medicare Participating Heart Bypass Center Demonstration. The report summarizes the nature of the demonstration, the number of sites that will participate, and other background information. It also discusses how the evaluation will measure (1) the economics of the demonstration; (2) the quality and appropriateness of care received under the demonstration; and (3) the marketing of the program carried out by the participating hospitals. Three primary data collection efforts are proposed, including (1) onsite case studies; (2) beneficiary telephone surveys; and (3) referring physician telephone surveys. The report also explicates a model of behavior under a global payment mechanism. See also PIC ID 5958-5958.1 and 5958.3-5958.4. (Final report 56 pages.)

AGENCY SPONSOR: Office of Research and Demonstrations

FEDERAL CONTACT: Armen H. Thoumaian, Ph.D.

PHONE NUMBER: 410/966-6672

PIC ID: 5958.2

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Medicare Participating Heart Bypass Center Demonstration: Final Evaluation Report--Volume One, The First Three Years

ABSTRACT NUMBER: 068

ABSTRACT: This report presents findings from the first three years of the Medicare Participating Heart Bypass Center Demonstration. This program is one of HCFA's cost-containment demonstrations. It was implemented to test the feasibility and cost-effectiveness of a negotiated package price for coronary artery bypass graft (CABG) surgery. Hospitals and physicians participating in the demonstration received a global payment for all hospital and physician services related to CABG surgery. The report finds that (1) the Medicare program saved $15.3 million on bypass patients treated in four of the seven participating hospitals from May 1991 through December 1993; (2) beneficiaries and their insurers saved another $2.3 million in Part B coinsurance payments during this same period; (3) no statistically significant trend in inpatient mortality rates was found (holding many patient risk factors constant); and (4) 97.7 percent of demonstration patients were treated appropriately with CABG surgery (assuming that none of the patients was a candidate for angioplasty--if all were assumed to be angioplasty candidates, then only 72.7 percent of the surgeries would have been considered appropriate). The report concludes that the results of the demonstration were mixed: only two of the four hospitals increased their market shares for CABG surgery, but hospital costs were significantly reduced. See also PIC ID 5958.1-5958.4.

AGENCY SPONSOR: Office of Research and Demonstrations

FEDERAL CONTACT: Armen H. Thoumaian, Ph.D.

PHONE NUMBER: 410/966-6672

PIC ID: 5958

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Medicare Participating Heart Bypass Center Demonstration: Final Evaluation Report--Volume Two, Marketing Activities of Participating Hospitals

ABSTRACT NUMBER: 069

ABSTRACT: This report presents information on the marketing activities of hospitals participating in the Medicare Participating Heart Bypass Center Demonstration program, in which Medicare was charged a global fee for all hospital and inpatient physician services related to coronary artery bypass graft surgery. Seven hospitals participated in the demonstration. The report presents an overview of the participating hospitals and the markets in which they are located; the focus, structure, and content of the hospitals' marketing programs; patient volume and physician referral patterns; and consumer satisfaction. The report finds that (1) all seven hospitals have developed and are implementing marketing plans for their CABG surgery programs to increase community awareness of their institutions and promote the benefits of choosing their facilities for CABG surgery; (2) the hospitals have not addressed product attributes, price, and consumer needs and desires, but have changed the ways in which they define the content and duration of services provided in conjunction with CABG surgery; and (3) while hospitals want to use their participation in the demonstration to increase their general standing in the community and to increase patient volumes in every category, HCFA wants to determine if the program can attract CABG patients away from higher priced competitors. See also PIC ID 5958 and 5958.2-5958.4.

AGENCY SPONSOR: Office of Research and Demonstrations

FEDERAL CONTACT: Armen H. Thoumaian, Ph.D.

PHONE NUMBER: 410/966-6672

PIC ID: 5958.1

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Medicare Risk Program for HMOs: Final Summary Report on Findings From the Evaluation

ABSTRACT NUMBER: 070

ABSTRACT: Since the early 1980's, HCFA has been encouraging health maintenance organizations (HMOs) to provide Medicare coverage to enrolled beneficiaries in return for fixed prepaid premiums. This report evaluates these Medicare risk plans and their impacts on beneficiaries, the Federal Government, and participating plans. The report finds that (1) risk plans attract healthier-than-average Medicare beneficiaries, thus saving the program less than expected--in fact, HCFA paid HMOs approximately 5.7 percent more than it would have spent for these patients under a fee-for-service (FFS) plan; (2) HMOs reduce the number of hospital days and average lengths of stay, but do not reduce admissions; (3) Medicare risk plans increase the likelihood that beneficiaries receive some services, but reduce intensity or frequency of the service; (4) risk plans may spend about 10 percent less than HCFA would spend for all medical services; (5) HMOs and FFS plans produce similar outcomes for inpatients, but HMOs use fewer resources; (6) HMOs provide comparable access to ambulatory care and produce similar patient results, with fewer resources; (7) HMO enrollees are somewhat less satisfied with their care than others, but are happier with costs and benefit coverage; and (8) one-half of HMO risk plans appear to be profitable. (Final report 192 pages, plus appendixes.)

AGENCY SPONSOR: Office of Research and Demonstrations

FEDERAL CONTACT: James Hadley

PHONE NUMBER: 410/786-6626

PIC ID: 5957

PERFORMER ORGANIZATION: Mathematica Policy Research, Inc., Plainsboro, NJ

TITLE: Monitoring the Impact of Medicare Physician Payment Reform on Utilization and Access

ABSTRACT NUMBER: 071

ABSTRACT: This report monitors changes in use of and access to physician services by population groups, geographic areas, and types of services. It also monitors possible sources of inappropriate use. The report seeks to discover whether reforms in Medicare physician payment policies have negatively influenced use of and access to services. Several measures are used to discover whether access to care has been affected, and several vulnerable population groups are monitored. The report finds that (1) the introduction of the Medicare fee schedule (MFS) has resulted in a relative increase in allowed charges for visits and consultations and a relative decrease in allowed charges for procedure-based services; (2) MFS has not caused new barriers to care for vulnerable populations, although these groups still face difficulties in accessing and using services; (3) MFS has resulted in a move away from delivery of physician services in the inpatient setting to delivery of those services in the outpatient setting; (4) the disabled have a higher percentage of those needing medical care and not receiving it than any other group; (5) vulnerable populations are at risk of not receiving Medicare-covered preventive services; and (6) beneficiaries who live in low-income areas or who are eligible for Medicaid are likely to be in poor health, and many need more continuous and appropriate ambulatory care to avoid hospitalization. (Final report 47 pages, plus appendixes.)

AGENCY SPONSOR: Office of Research and Demonstrations

FEDERAL CONTACT: Marian Gornick

PHONE NUMBER: 410/786-6686

PIC ID: 5493

PERFORMER ORGANIZATION: Health Care Financing Administration, Baltimore, MD

Health Resources and Services Administration

Contents

Bureau of Primary Health Care Evaluation Strategy: Design of the User and Visit Surveys

Bureau of Primary Health Care User Manual: Uniform Data System

Community and Migrant Health Centers and the Assessment of and Response to Mental Health and Developmental Needs in Primary Care Patients (Three Volumes)

Community Health Centers' Performance Under Managed Care

Development of Integrated Requirements for Physician Assistants, Nurse Practitioners, Certified Nurse-Midwives, and Physicians

Enrolling Minorities, Women, and Children/Adolescents Into NIH AIDS Clinical Trials: Models and Practices

Evaluation of Maternal and Child Health Bureau Childhood Injury Prevention Implementation Incentive Grants

Health Services Utilization and Costs to Medicaid of AFDC Recipients in New York and California Served and Not Served by Selected Community Health Centers

HMO Primary Care Staffing Patterns and Processes: A Cross-Site Analysis of 23 HMOs

Impact of FQHC Implementation on Community and Migrant Health Center Revenue and Utilization

Impact of Ryan White CARE Act Title I on Capacity Building in Latino Community-Based Organizations: Findings From a Study of Two Cities

Impact of Ryan White Title III(b) Funding on the Provision of HIV Primary Care

Infant Mortality Among U.S. Women of Mexican Descent: Evaluating the Validity of Current Estimates

Linkages Between Federally Qualified Health Centers and Hospitals: A Guide

National Practitioner Data Bank: User Satisfaction With Reporting and Querying and Usefulness of Disclosure Information for Decisionmaking 1992-94

Performance Indicators for GPRA: Initial Assessment of HRSA Programs (Final Report)

Ryan White CARE Act Title I Strategies for Underserved African-Americans: Methods, Analysis, and Interpretations of Four Cities

Second National Primary Care Conference: Executive Summary

Second National Primary Care Conference: Proceedings--Volume I

Second National Primary Care Conference: Proceedings--Volume II, Case Studies

Uniform Reporting System Field Test

Ryan White Title IV Program for Children, Youth, Women, and Families: National Evaluation

TITLE: Bureau of Primary Health Care Evaluation Strategy: Design of the User and Visit Surveys

ABSTRACT NUMBER: 072

ABSTRACT: This project was a pilot study in preparation for a study of Community Health Centers. The pilot had two components: a personal interview survey of health center users, and a separate, medical record-based study of visits to health centers. HRSA's Bureau of Primary Health Care developed an agreement with the National Center for Health Statistics to assist in the design of the personal health interview survey modeled after the National Health Interview Survey (NHIS) and the medical record-based study of visits based on the National Hospital Ambulatory Medical Care Survey (NHAMCS). The final survey design resulted from visits to seven Community Health Centers to develop, evaluate, and revise the survey instruments and procedures. Lessons learned from the pilot test strengthened the design of the full survey. Based on the pilot test results, a contract for full implementation of the Community Health Center User and Visit Survey was awarded in September 1994. (Final report 28 pages, plus attachments.)

AGENCY SPONSOR: Bureau of Primary Health Care

FEDERAL CONTACT: Jerrilynn Regan

PHONE NUMBER: 301/594-4280

PIC ID: 5613

PERFORMER ORGANIZATION: Center for Health Policy Studies, Columbia, MD

TITLE: Bureau of Primary Health Care User Manual: Uniform Data System

ABSTRACT NUMBER: 073

ABSTRACT: This report reviews the existing reporting requirements for grantees of HRSA's Bureau of Primary Health Care (BPHC). It also provides a structure for the Uniform Data System (UDS), which integrates reporting for five primary care programs: the Community Health Center Program; the Migrant Health Center Program; the Health Care for the Homeless Program; the Outreach and Primary Health Services for Homeless Children Program; and the Public Housing Primary Care Program. The study developed a manual for using UDS. UDS consists of two components: the "Universal Report," which has nine tables and is required of all grantees; and the "Grant Reports," which repeat three of the Universal Report tables to provide comparable data on characteristics of users whose services fall within the scope of a project funded under a particular grant. The report notes that UDS will replace other reporting requirements for the programs and that its implementation is planned to facilitate transition to the new system. Instructions for each table are given, as are appendixes dealing with personnel and service definitions. UDS is being implemented, with the first data collection covering calendar year 1996. The first reports are due on March 31, 1997. (Final report 70 pages, plus appendixes.)

AGENCY SPONSOR: Bureau of Primary Health Care

FEDERAL CONTACT: Jerrilynn Regan

PHONE NUMBER: 301/594-4280

PIC ID: 5611

PERFORMER ORGANIZATION: Mathematica Policy Research, Inc., Plainsboro, NJ

TITLE: Community and Migrant Health Centers and the Assessment of and Response to Mental Health and Developmental Needs in Primary Care Patients (Three Volumes)

ABSTRACT NUMBER: 074

ABSTRACT: This study explores the extent to which Community and Migrant Health Centers (C/MHCs) provide screening, assessment, and treatment for the mental health and developmental needs of primary care patients. For this study, 25 health centers in 10 States were chosen to represent geographic diversity and different approaches to Medicaid coverage of mental health services. Structured telephone interviews were conducted at these 25 centers. Case studies written for each State describe selected characteristics of the health center and how the center conducts screening, assessment, treatment, and followup for the mental health and developmental needs of both children and adults. The study found that most of the C/MHCs provide some level of mental health screening and treatment. The study also found that (1) C/MHCs identified a broad range of mental health problems as prevalent in their patient populations and viewed these problems as interfering with the patient's health, quality of life, and ability to benefit from primary care; (2) in response to the mental health needs of their patients, C/MHCs took approaches that were distributed over a continuum from screening to provision of comprehensive mental health services onsite; and (3) States that reimburse for services delivered by a broad range of mental health professionals in different settings facilitate C/MHC delivery of mental health services, either directly or by referral. The final report has been distributed to primary care organizations and primary care associations that can use the successful models to improve the delivery of mental health services in underserved communities. A manuscript is being developed for submission to a professional journal. Volume One contains the final report, while Volumes Two and Three contain appendixes detailing case studies.

AGENCY SPONSOR: Bureau of Primary Health Care

FEDERAL CONTACT: James Macrae

PHONE NUMBER: 301/594-4315

PIC ID: 5617

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Community Health Centers' Performance Under Managed Care

ABSTRACT NUMBER: 075

ABSTRACT: This study examined the effects of managed care on seven Community Health Centers (CHCs) with varying degrees of managed care experience. The sites selected had prepaid managed care arrangements involving some financial risk. The study chose centers with significant managed care enrollment and several years' experience with managed care contracts. An important determinant in choosing the final site cohort was whether the HMOs that had contracted with the selected CHCs were willing to participate in the study. The study found that (1) CHCs perform as well as or better than comparable providers in their network based on key utilization and financial indicators; (2) the CHCs studied offer strong primary care services accompanied by cultural sensitivity that is highly valued by HMOs; and (3) CHCs need to pursue changes in their operations more aggressively to better accommodate managed care. Health centers have used the findings of this report to negotiate with managed care organizations and with State Medicaid agencies. An article on the report has been published in the July 1995 issue of the Journal of Ambulatory Care Management. (Final report 44 pages, plus appendixes.)

AGENCY SPONSOR: Bureau of Primary Health Care

FEDERAL CONTACT: Bethann Tutunjian

PHONE NUMBER: 301/594-4060

PIC ID: 5596

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Development of Integrated Requirements for PAs, NPs, CNMs, and Physicians (MDs and DOs)

ABSTRACT NUMBER: 076

ABSTRACT: This project developed a personal computer-based model to forecast national integrated requirements for physicians, physician assistants (PAs), nurse practitioners (NPs), and certified nurse-midwives (CNMs) who deliver primary health care services. The Integrated Requirements Model provides estimates of practitioner needs under varying scenarios regarding managed care, insurance coverage, and alternative staffing configurations. The model comes prepackaged with six scenarios based on differing sets of assumptions concerning potential changes in primary care delivery. Users of the model can develop an unlimited number of scenarios by editing the projection measures underlying the assumptions. Study results are being used by the Council on Graduate Medical Education and the National Advisory Council on Nursing Education and Practice to form recommendations for data and education program development. The model is also being used by a number of external entities, including the Utah State Health Department, the Virginia Medical College, and the Wisconsin Consortium for Primary Care. Findings were presented at a conference, and a manuscript is planned for submission to a professional journal. (Final report variously paginated, plus appendix.)

AGENCY SPONSOR: Bureau of Health Professions

FEDERAL CONTACT: Edward Sekscenski

PHONE NUMBER: 301/443-6663

PIC ID: 5745

PERFORMER ORGANIZATION: Vector Research, Inc., Ann Arbor, MI

TITLE: Enrolling Minorities, Women, and Children/Adolescents Into NIH AIDS Clinical Trials: Models and Practices

ABSTRACT NUMBER: 077

ABSTRACT: This study qualitatively documented the HRSA-funded Clinical Trials Linkage Projects. This funding was awarded to eight Ryan White Title III(b) sites to increase the enrollment of persons from underserved NIH-sponsored populations (ethnic/racial minorities, women, children, and adolescents) in HIV/AIDS clinical trials sponsored by the National Institutes of Health (NIH). Enrollment was increased by establishing linkages between primary care clinics and clinical trials research sites. Each of the eight sites (called Linkage Projects) were visited. The report concluded that, while the Linkage Projects were successful in establishing a working relationship between a primary care facility and a clinical trials research center, the recruitment efforts depended on the natural constituency of the primary care institution. One of the major recommendations was to expand outreach and education efforts to improve client recruitment for and retention in clinical trials. Based on study findings and related workshops, more than 90 percent of Title III(b) grantees now refer clients to a local clinical trial unit. (Final report 95 pages, plus appendixes.)

AGENCY SPONSOR: Bureau of Primary Health Care

FEDERAL CONTACT: Deborah Parham

PHONE NUMBER: 301/594-4444

PIC ID: 6099

PERFORMER ORGANIZATION: LTG Associates, Inc., Turlock, CA

TITLE: Evaluation of Maternal and Child Health Bureau Childhood Injury Prevention Implementation Incentive Grants

ABSTRACT NUMBER: 078

ABSTRACT: HRSA's Maternal and Child Health Bureau (MCHB) awarded incentive grants in 1987 and 1988 to seven States to develop programs in childhood injury prevention. This study analyzed and compared the State strategies for establishing and maintaining focal points for injury prevention after Federal funding ended. Site visits to each State included document review and interviews with injury prevention personnel. Findings show that, in general, recipients of the implementation incentive grants exhibited a considerable increase in injury prevention capacity. All seven States increased their dedicated injury prevention staff during the grant period and six have maintained or increased that staffing level. Advocacy for injury prevention has increased. Several respondents credit the MCHB program with enabling them to acquire funding from other Federal and State sources. Most also greatly increased their capacity to collect and use injury morbidity and mortality data, but have not fully incorporated data analysis into the process of planning injury prevention programs. These findings will be used by policymakers and HRSA program personnel in considering future allocations of program funding. Volume one contains the final report, and volume two contains a cross-site analysis report. (Final report 5 pages, plus appendix; cross-site analysis report variously paginated, plus appendixes.)

AGENCY SPONSOR: Maternal and Child Health Bureau

FEDERAL CONTACT: David Maglott

PHONE NUMBER: 301/443-2778

PIC ID: 5599

PERFORMER ORGANIZATION: Center for Health Policy Studies, Columbia, MD

TITLE: Health Services Utilization and Costs to Medicaid of AFDC Recipients in New York and California Served and Not Served by Selected Community Health Centers

ABSTRACT NUMBER: 079

ABSTRACT: This report presents the findings from studies of the use of Medicaid-covered services and costs to Medicaid for Aid to Families With Dependent Children (AFDC) and AFDC-like recipients living in the service areas of four selected Community Health Centers in California and six in New York. The California study was based on calendar year 1989 Medi-Cal eligibility, claims, and payment data, while the New York study used Federal fiscal 1991 Medicaid data. The hypotheses tested in the study were that (1) patterns of use would differ among recipients who use CHCs as their customary source of primary care and those who do not; and (2) regular users of CHCs would experience reduced inpatient hospital care and would incur lower costs to Medicaid. The study confirmed this second hypothesis, finding statistically significant reductions in use (hospital admissions and inpatient days) and costs to Medicaid for regular CHC users, compared with nonusers. In California, regular CHC users incurred costs to Medicaid that were 14 percent lower than for nonusers; if maternity cases are excluded, the costs were 33 percent lower; and for maternity cases alone, the costs were 9 percent lower. In New York, these figures were 24 percent, 30 percent, and 12 percent, respectively. The report concludes that CHCs are cost-effective, perhaps because of organized utilization review, case management, and preventive services.

AGENCY SPONSOR: Bureau of Primary Health Care

FEDERAL CONTACT: Judy Rodgers

PHONE NUMBER: 301/594-4343

PIC ID: 6001

PERFORMER ORGANIZATION: Center for Health Policy Studies, Columbia, MD

TITLE: HMO Primary Care Staffing Patterns and Processes: A Cross-Site Analysis of 23 HMOs

ABSTRACT NUMBER: 080

ABSTRACT: This study examined HMO primary care staffing patterns and the decisionmaking processes used to establish them. The study design consisted of onsite interviews with senior health plan officials in 23 HMOs nationwide. The study found that the three key factors that affect how HMOs determine the number of primary care physicians they need are (1) current growth in enrollment and opportunities for further growth; (2) enrollee experience with access; and (3) employer desires and requirements. Although generally successful in recruiting the desired number of primary providers (some plans expressed recruiting difficulties in rural and underserved urban areas), plans report that new graduates who join are not well-prepared for practice in either managed care settings or primary care settings more generally. To address this problem, HMOs typically monitor new recruits closely. The sampled plans included five with Medicaid enrollment at 25 percent or more of total enrollment. The purpose was to investigate how needs for primary care staffing might change if managed care becomes more dominant for low-income populations. Although utilization differs for the Medicaid population--relative to other enrollees, Medicaid enrollees have higher walk-in and no-show appointment rates, higher hospital admission rates, and underutilize preventive health care services--the organizations did not plan their primary care staffing very differently. But HMOs serving Medicaid populations place providers near these enrollees, provide outreach services, and seek providers with experience in treating Medicaid beneficiaries. Findings from this study will improve the ability of HRSA managers and the public health community to forecast work force needs and to ensure an adequate supply of practitioners. (Final report 118 pages, plus appendixes.)

AGENCY SPONSOR: Office of Planning, Evaluation, and Legislation

FEDERAL CONTACT: Jessica Townsend

PHONE NUMBER: 301/443-1900

PIC ID: 5606

PERFORMER ORGANIZATION: Mathematica Policy Research, Inc., Plainsboro, NJ

TITLE: Impact of FQHC Implementation on Community and Migrant Health Center Revenue and Utilization

ABSTRACT NUMBER: 081

ABSTRACT: This study assessed the impact of the Medicaid Federally Qualified Health Center (FQHC) legislation on Community Health Center (CHC) revenue and utilization of services. Multivariate analysis was used to examine the experience of federally funded CHCs between 1989 and 1992. Analyses suggest that the FQHC legislation, in the Omnibus Budget Reconciliation Act of 1989, had a positive impact on health center financing and on access to care for both Medicaid recipients and the uninsured. Study results also suggest that FQHC impact on service utilization differed according to center size. In particular, for smaller centers experiencing significantly higher growth rates for all insurers and for Medicaid users, higher reimbursement rates under FQHC may have provided an incentive to increase the center's Medicaid beneficiaries. Conversely, larger centers experienced significantly greater absolute increases in Medicaid beneficiaries, but not in total user populations. It was also found that centers in States with fast-growing Medicaid programs experienced significantly higher increases in the number of Medicaid beneficiaries than centers in other States. Study results are being used in the program planning process to shape policy directions at the agency and departmental levels, and within States. (Final report 94 pages, plus appendixes; executive summary, 31 pages, separately bound.)

AGENCY SPONSOR: Bureau of Primary Health Care

FEDERAL CONTACT: Ronda Hughes

PHONE NUMBER: 301/594-4120

PIC ID: 5616

PERFORMER ORGANIZATION: MDS Associates, Inc., Wheaton, MD

TITLE: Impact of Ryan White CARE Act (RWCA) Title I on Capacity Building in Latino Community-Based Organizations: Findings From a Study of Two Cities

ABSTRACT NUMBER: 082

ABSTRACT: This study was designed to develop and test a methodology for evaluating whether and how CARE Act Title I funds have helped develop, expand, or enhance HIV/AIDS services to Latino community-based organizations (CBOs). Information for the study was gathered from surveys and key informant interviews conducted at Latino CBOs in two eligible metropolitan areas: San Diego and Boston. The study found that 29 percent of the Latino CBOs in the study areas received Title I funding and that an atmosphere of openness and inclusiveness in the planning process was critical for Latino CBOs to feel that they were full participants. The majority of persons with HIV/AIDS served by the Latino CBOs were Spanish speaking, emphasizing the need for linguistically and culturally appropriate services. Study participants noted that receipt of Title I funding is critical for enhancing and expanding services. Recommendations were for HRSA to (1) develop incentives for eligible metropolitan areas to increase Latino CBO representation on planning councils; (2) create active outreach mechanisms to engage nonmedical agencies; (3) fill data gaps for planning council directors regarding service and capacity needs of communities of color; (4) develop clear guidelines for capacity-building activities that should be funded under the Ryan White CARE Act; and (5) conduct studies that investigate factors that impede or facilitate participation of ethnic minority CBOs in the RWCA planning process. The findings from this study are being used to provide technical assistance to grantees on issues regarding capacity building in both minority and nonminority CBOs.

AGENCY SPONSOR: Bureau of Health Resources Development

FEDERAL CONTACT: Gloria Weissman

PHONE NUMBER: 301/443-3478

PIC ID: 6197

PERFORMER ORGANIZATION: Boston University, School of Public Health; The National Coalition of Hispanic Health and Human Services Organizations

TITLE: Impact of Ryan White Title III(b) Funding on the Provision of HIV Primary Care

ABSTRACT NUMBER: 083

ABSTRACT: The Ryan White Comprehensive AIDS Resources Emergency Act of 1990 authorizes a variety of grants. These grants include AIDS Early Intervention Services grants to community-based primary care facilities under Title III(b). This study assessed the impact of Title III(b) funding in terms of HIV-related services, models of service delivery, and the utilization of Title III(b) funds. The study was based on a mail survey of all organizations participating in the Title III(b) program. Survey data indicate that the client populations served by Title III(b) are changing. Nearly 60 percent of respondents reported serving more women; 29 percent reported serving increased numbers of injection drug users. Fifteen percent of respondents reported increased numbers of African-American or Hispanic clients. Many grantees attribute the shift in population mix to substantial outreach efforts launched under Title III(b). The survey also found that Title III(b) funds have enabled HIV projects to increase the number of sites by nearly 60 percent. In addition, the projects were able to add or expand medical, social, and support services. Funds have also allowed grantees to address and overcome many of the traditional barriers to health care, including inadequate transportation; cultural and language barriers; lack of client financial resources; and client, provider, and community ignorance about and denial of AIDS/HIV. Title III(b) funds have also helped projects establish and maintain linkages with health care and social services, and responding Title III(b) grantees have established collaborative relationships with over 1,300 agencies.

AGENCY SPONSOR: Bureau of Primary Health Care

FEDERAL CONTACT: Thomas M. Coughlin

PHONE NUMBER: 301/594-4450

PIC ID: 5597

PERFORMER ORGANIZATION: CDM Group, Inc., Chevy Chase, MD

TITLE: Infant Mortality Among U.S. Women of Mexican Descent: Evaluating the Validity of Current Estimates

ABSTRACT NUMBER: 084

ABSTRACT: Researchers and HRSA policymakers have been puzzled for a number of years by the anomalously low infant mortality rate (IMR) reported for the Mexican-American community. If IMRs are used as measures of medical underservice, the low IMRs for Mexican-Americans will mean that fewer resources will be channeled to areas with large Mexican-American populations. An analysis of NCHS matched birth-death data tapes for 1983-87 indicate that the rate of very low and low birthweight infants for Mexican-Americans (5.7 percent) is about the same as that of non-Hispanic white infants but significantly lower than those of non-Hispanic black infants (13 percent). The difference in IMRs between first and later generation Mexican-American infants indicates that attachment to Mexican culture may be a reasonable explanation for low infant mortality. With the exception of congenital anomalies and short gestation/low birthweight, first generation infants have a lower IMR than later generation infants. The study suggests that ethnic misclassification and underreporting may be less serious problems than some observers believe. The approaches used to analyze the NCHS 1983-87 data are now being applied to the newly available 1988-90 linked birth-death records. Two journal manuscripts are being developed based on this study and the 1988-90 linked birth-death records.

AGENCY SPONSOR: Bureau of Primary Health Care

FEDERAL CONTACT: Barbara Wells

PHONE NUMBER: 301/594-4284

PIC ID: 6053

PERFORMER ORGANIZATION: The George Washington University, Center for Health Policy Research, Washington, DC

TITLE: Linkages Between Federally Qualified Health Centers and Hospitals: A Guide

ABSTRACT NUMBER: 085

ABSTRACT: Hospitals and community-based primary health care centers of all types, including Community and Migrant Health Centers (C/MHCs) and other Federally Qualified Health Centers (FQHCs), are increasingly entering linkages that go beyond traditional arrangements in which center physicians have admitting privileges and can follow their patients while they are inpatients in local hospitals. This study suggests both legal and organizational approaches to these new relationships. Four factors have led to these increased linkages: (1) recognition by most providers that their survival is best assured within the system of care rather than as freestanding, limited service providers; (2) pressure from purchasers of care to build integrated delivery systems; (3) implementation of cost-based reimbursement for Medicaid and Medicare beneficiaries in FQHCs; and (4) the presence of new opportunities resulting from State or private financial incentives. Five sites (Albany, Georgia; Plainfield, New Jersey; Blossburg, Pennsylvania; Erie, Pennsylvania; and Seattle, Washington) were selected to collect the experiences of FQHC or hospital networks, ranging in joint activities and organizational forms. The study found that FQHCs and hospitals should determine the purpose of their linkage before they determine the organizational form that will legally embody agreed-upon activities. The report is in the form of a guidebook for staff and board members of FQHCs who are considering linkages with hospitals. Study results are being used to identify legal and financial issues that FQHCs should consider in forming vertical arrangements with hospitals.

AGENCY SPONSOR: Bureau of Primary Health Care

FEDERAL CONTACT: Ronda Hughes

PHONE NUMBER: 301/594-4120

PIC ID: 5615

PERFORMER ORGANIZATION: The George Washington University, Center for Health Policy Research, Washington, DC

TITLE: National Practitioner Data Bank: User Satisfaction With Reporting and Querying and Usefulness of Disclosure Information for Decisionmaking 1992-94

ABSTRACT NUMBER: 086

ABSTRACT: This study represents phase II of an evaluation study to assess the impact of the National Practitioner Data Bank (NPDB). Phase I developed the survey instrument under HRSA Contract No. 240-91-0017, Delivery Order No. 240-92-0508. The primary purpose of the Phase II study was to implement the evaluation methodology. NPDB's impact was assessed in terms of its ability to serve as a national repository and to distribute useful information effectively. The telephone survey found that (1) most entities had a high level of satisfaction with NPDB's reporting form and had a favorable reaction toward implementing technical advances; (2) querying entities generally found the information they received to be useful; (3) approximately one-fourth of entities received new information from NPDB responses; (4) three-quarters of malpractice insurers indicated that NPDB had impacted claims resolution between 1992 and 1994; (5) not all State licensing boards received adverse action reports to pass through to NPDB between 1992 and 1994; (6) hospitals are required to query NPDB, but HMOs, State boards, group practices, PPOs, and professional societies had voluntary query rates during the study period of from 7 percent to 58 percent. The report also provides information on costs, disclosures, and decision making based on NPDB information. Study results have been used to design NPDB's second generation system, and have been made available to medical malpractice insurers, State health care practitioner licensing boards, hospitals, and other health care entities that query and report to NPDB, and the general public. (Final report 111 pages, plus appendixes.)

AGENCY SPONSOR: Bureau of Health Professions

FEDERAL CONTACT: Robert Oshel

PHONE NUMBER: 301/443-2300

PIC ID: 5609

PERFORMER ORGANIZATION: Walcoff & Associates Inc., Fairfax, VA

TITLE: Performance Indicators for GPRA: Initial Assessment of HRSA Programs (Final Report)

ABSTRACT NUMBER: 087

ABSTRACT: The Government Performance and Results Act of 1993 requires that each Federal agency develop comprehensive strategic plans, annual performance plans that set specific performance goals for each program activity, and annual reports on the actual performance achieved compared with the performance goals. This study assessed HRSA ability to develop and implement a system of performance measurement and management as of summer 1995. The report focuses on individual programs or program clusters identified in the President's budget for fiscal 1996. The study used a literature review, background research, and interviews within HRSA to assess program clusters, programs, and budget line items and to develop a technical assistance plan that identifies what is needed to further help HRSA develop and implement a performance management system. The study found that (1) HRSA's current program structures and measurement efforts form a base for a HRSA-wide measurement system; and (2) indicators common to many programs could be used to measure agencywide performance. The performance measurement strategy should stress (1) primary emphasis on a coordinated, interdependent system of health care resources, rather than a disparate set of programs; (2) links between the measurement strategy, strategic planning, program activities, evaluation planning and findings, and budgeting; and (3) incremental development of the system. This study has served as the basis for technical assistance to all HRSA Bureaus, designed to meet their individual needs in performance measurement. (Final report 68 pages, plus appendixes.)

AGENCY SPONSOR: Office of Planning, Evaluation, and Legislation

FEDERAL CONTACT: Karen Thiel Raykovich, Ph.D.

PHONE NUMBER: 301/443-7718

PIC ID: 5954

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Ryan White CARE Act Title I Strategies for Underserved African-Americans: Methods, Analysis, and Interpretations of Four Cities

ABSTRACT NUMBER: 088

ABSTRACT: This study was intended to provide a better understanding of the local strategies currently being used by Ryan White CARE Act grantees to reach and serve African-Americans living with HIV. A secondary aim of the study was to assess the suitability of using qualitative data collection methods to identify local service delivery strategies that could be applied to other underserved populations. The study was designed to capture the social and organizational contexts in which HIV service strategies emerged in local communities. The study uses indepth, semistructured discussions with key stakeholders, including grantees, service providers, and people living with HIV. Study respondents identified three types of service strategies to increase the number of underserved African-Americans living with HIV: improving administrative procedures, building service capacity, and raising public education and awareness. An essential component of effective strategies was working through and with African-American institutions to develop a comprehensive system of quality care. Study findings will be presented at the 1996 International AIDS and American Public Health Association conferences. (Final report 93 pages, plus appendixes.)

AGENCY SPONSOR: Bureau of Health Resources Development

FEDERAL CONTACT: Moses B. Pounds

PHONE NUMBER: 301/443-6560

PIC ID: 5734

PERFORMER ORGANIZATION: MACRO International, Inc., Silver Spring, MD

TITLE: Second National Primary Care Conference: Executive Summary

ABSTRACT NUMBER: 089

ABSTRACT: This executive summary provides information on the Second National Primary Care Conference, which was held September 11-13, 1994, in Dallas, Texas. The conference was designed to answer questions and to generate understanding on the following primary care topics: (1) the training, composition, and preparedness of the primary care work force; (2) the formation, responsiveness, and comprehensiveness of the primary care delivery system; and (3) the relationship between primary care and special populations. The executive summary describes the events that took place during the conference, and includes summaries of plenary speeches and panel presentations. Copies of the conference program, a list of program participants, and a letter to conferees from First Lady Hillary Rodham Clinton are also included. See also PIC ID 4435, 5750.1, and 5750.2. (Executive summary 47 pages.)

AGENCY SPONSOR: Office of Planning, Evaluation, and Legislation

FEDERAL CONTACT: Pearl Perry

PHONE NUMBER: 301/443-1126

PIC ID: 5750

PERFORMER ORGANIZATION: Social and Scientific Systems, Inc., Bethesda, MD

TITLE: Second National Primary Care Conference: Proceedings--Volume I

ABSTRACT NUMBER: 090

ABSTRACT: This report provides details of the Second National Primary Care Conference, held in Dallas, Texas, on September 11-13, 1994. The report includes (1) a letter to conferees from First Lady Hillary Rodham Clinton; (2) the executive summary; (3) the conference program; (4) the keynote address delivered by the Honorable Richard D. Lamm; (5) plenary speeches presented by Ron J. Anderson, M.D., Paul M. Ellwood, M.D., Neal Vanselow, M.D., and Bruce Vladeck, Ph.D.; (6) a list of program participants; and (7) a list of conference attendees. See also PIC ID 4435, 5750, and 5750.2. (Final report 152 pages.)

AGENCY SPONSOR: Office of Planning, Evaluation, and Legislation

FEDERAL CONTACT: Pearl Perry

PHONE NUMBER: 301/443-1126

PIC ID: 5750.1

PERFORMER ORGANIZATION: Social and Scientific Systems, Inc., Bethesda MD

TITLE: Second National Primary Care Conference: Proceedings--Volume II, Case Studies

ABSTRACT NUMBER: 091

ABSTRACT: This report presents case studies used in panels at the Second National Primary Care Conference held in Dallas, Texas, on September 11-13, 1994. The case studies cover a range of topics, such as the Adirondack Rural Health Network, the California Cultural Competency Task Force, the Changing Role of Academic Health Centers Under Health Care Reform, Linking School-Based Programs and Primary Care, and Using Quality to Build Capitated Delivery Systems, among others. See also PIC ID 4435, 5750, and 5750.1. (Final report 233 pages.)

AGENCY SPONSOR: Office of Planning, Evaluation, and Legislation

FEDERAL CONTACT: Pearl Perry

PHONE NUMBER: 301/443-1126

PIC ID: 5750.2

PERFORMER ORGANIZATION: Social and Scientific Systems, Inc., Bethesda, MD

TITLE: Uniform Reporting System Field Test

ABSTRACT NUMBER: 092

ABSTRACT: In September 1992, HRSA began a year-long field test of a Uniform Reporting System (URS) that was intended to obtain data on the populations reached by Ryan White CARE Act Title I and Title II services providers, and on the types and amounts of services delivered to clients. This study assessed (1) the feasibility of unduplicated, client-level reporting; (2) the value of the resulting information; (3) the level of effort and costs required of all participants; (4) the adequacy of measures; (5) the types and amounts of technical assistance HRSA would have to provide; and (6) the refinements needed in URS data elements or procedures. Analysis of the field test, which included 15 grantees, shows that (1) URS client-level data systems are feasible and valuable; (2) effective implementation of URS client-level reporting would require significant effort by grantees and service providers; and (3) a high level of technical support from HRSA would be needed for full implementation of URS nationwide. Based on these conclusions, HRSA submitted to the Office of Management and Budget a request for approval of mandatory implementation of the Annual Administrative Report calling for aggregated data. HRSA also decided to proceed with client-level URS reporting on a voluntary basis and to establish demonstration sites for client-level URS reporting. Eight Title I and Title II grantees are conducting the demonstration for a 3-year period.

AGENCY SPONSOR: Bureau of Health Resources Development

FEDERAL CONTACT: Robert Baitty

PHONE NUMBER: 301/443-0652

PIC ID: 4916

PERFORMER ORGANIZATION: Mathematica Policy Research, Inc., Plainsboro, NJ

TITLE: Ryan White Title IV Program for Children, Youth, Women, and Families: National Evaluation

ABSTRACT NUMBER: 093

ABSTRACT: The purpose of this project was to assess the effectiveness of HRSA's Grants for Coordinated Services and Access to Research for Children, Youth, and Families, authorized by Title IV of the Ryan White Comprehensive AIDS Resources Emergency Act. The study was focused on the following questions: Have systems of care been established, improved, maintained, or expanded? Do the systems have the desired attributes? Are the systems reaching and serving intended unserved or underserved target populations? Are multidisciplinary providers accessible, available, and trained to serve the target populations? Are clients receiving the array of medical and social services they need? Has the program had an impact on increasing participation in clinical drug trials? Has the program had a positive impact on client and family satisfaction? Methodologies included review of grant applications, self-study questionnaires from grantees, site visits to 18 grantees, and focus groups in ten sites. Findings confirm that the Title IV program has established, improved, maintained, or expanded systems of care that provide family-centered, culturally competent, and community-based services to the target populations. Multidisciplinary providers are accessible and available, and are specifically trained to meet client needs. Clients reported that they generally receive the array of services they need, though some gaps exist. The program has led to increased participation in clinical trials. Finally, the program has had a significant and lasting impact on client and family satisfaction. Results have supported targeted program planning to fill service gaps identified through this first national snapshot of the Title IV program. Information about barriers to and successes in building models for enhanced access to services and research is being used by other Ryan White programs.

AGENCY SPONSOR: Maternal and Child Health Bureau

FEDERAL CONTACT: David Maglott

PHONE NUMBER: 301/443-2778

PIC ID: 5600

PERFORMER ORGANIZATION: Macro International, Inc., Silver Spring, MD

Indian Health Service

Contents

Case Study of Family Violence in Four Native American Communities: Final Report

Evaluation of Diabetes Services Provided by IHS Model Diabetes Program: Final Report

Evaluation of IHS Midlevel Health Providers: Final Report

Phase III Final Report: Child Abuse and Neglect in American Indian and Alaska Native Communities and the Role of the Indian Health Service

TITLE: Case Study of Family Violence in Four Native American Communities: Final Report

ABSTRACT NUMBER: 094

ABSTRACT: This study examines family violence on four American Indian reservations (the Confederated Tribes of Warm Springs, the Eastern Band of Cherokee, the Navajo Nation, and the Rosebud Sioux). It uses a case study approach to collect primary and secondary data about the nature and prevalence of family violence, and the intervention and prevention measures planned or in place on each reservation. The report's strengths include the broad range of informants interviewed, the wide variation in characteristics of the sites, and the objectiveness used by the contractor to gather data. Its limitations include the facts that case study data, by nature, are not representative and that informants feel pressure not to disclose unfavorable information. The report finds that the eight components of family violence interventions include (1) adoption of a family violence code, with mandatory arrest and incarceration or treatment of offenders; (2) establishment of a victim support system; (3) institution of new police procedures for dealing with family violence; (4) increase in community education and involvement; (5) coordination of resources and programs across agencies; (6) development of an information tracking system; (7) staff training initiatives; and (8) establishment of an abuser treatment protocol. The report also offers six recommendations based on these interventions. See also PIC ID 6000.1. (Final report 52 pages, plus appendixes.)

AGENCY SPONSOR: Office of Planning, Evaluation, and Legislation

FEDERAL CONTACT: Leo Nolan

PHONE NUMBER: 301/443-4700

PIC ID: 6000

PERFORMER ORGANIZATION: Indian Health Service, Office of Planning, Evaluation, and Legislation, Rockville, MD

TITLE: Evaluation of Diabetes Services Provided by IHS Model Diabetes Program: Final Report

ABSTRACT NUMBER: 095

ABSTRACT: This project evaluates the IHS diabetes model projects. It uses data from 634 patient medical records, 4 focus groups, and 20 information interviews to describe these projects and to examine their effects on patient health outcomes, blood sugar control, and hospitalizations. Two diabetes project sites (Winnebago, Nebraska, and Fort Totten, North Dakota) and on "usual care" site (Rosebud, South Dakota) were selected for the evaluation. Data from the 1993 Diabetes Program Audit were used to assess whether the findings from this evaluation could generally represent other diabetes team approaches in other IHS areas. The report finds that (1) patients followed at diabetes project sites experienced lower rates of poor blood sugar control than patients followed at the usual care site; (2) patients at the project sites were at reduced risk of having a first diabetes-related hospitalization over a followup period than patients at the usual care site; (3) data from the 1993 audit cannot provide a clear answer about whether the results from this evaluation can be generalized to the larger IHS service population, but projects in general seem to be similar in their rates of poor control and diabetes care characteristics; and (4) focus groups and informant interviews indicated that patients desired more education and that medical staff believe that patients' refusal to accept their diagnosis is a barrier to their adherence to a care regimen. (Final Report variously paginated, plus appendixes.)

AGENCY SPONSOR: Office of Planning, Evaluation, and Legislation

FEDERAL CONTACT: Frank E. Marion

PHONE NUMBER: 301/443-4700

PIC ID: 6005

PERFORMER ORGANIZATION: Native American Consultants, Inc., Washington, DC

TITLE: Evaluation of IHS Midlevel Health Providers: Final Report

ABSTRACT NUMBER: 096

ABSTRACT: IHS must determine its needs for midlevel health providers (MLHPs), such as physician assistants, nurse practitioners, certified nurse midwives, and clinical nurse specialists, through the year 2000 and must address recruitment and retention of these professionals. A survey of 254 MLHPs and 25 primary care managers and visits to three sites yielded responses from 119 MLHPs (47 percent of the current MLHPs) and 14 primary care managers. The report finds that MLHPs say that the key retention points in their consideration to remain with IHS are increases in salaries, Continuing Medical Education benefits, and recognition as health care professionals with policymaking privileges. Serious dissatisfaction in these areas indicates a need for IHS action. The report also finds that (1) the quality of health care in clinics will deteriorate significantly if sufficient MLHPs are not provided by the year 2000; (2) the recruitment of more local Native Americans will bolster this already good situation; (3) significant decline in the numbers of IHS physicians will negatively impact physician performance; and (4) more administrative support will be required to free MLHPs for almost exclusive attention to primary health care duties. The report provides several recommendations. (Final report 15 pages, plus appendixes.)

AGENCY SPONSOR: Office of Planning, Evaluation, and Legislation

FEDERAL CONTACT: Frank E. Marion

PHONE NUMBER: 301/443-4700

PIC ID: 6013

PERFORMER ORGANIZATION: Native American Consultants, Inc., Washington, DC

TITLE: Phase III Final Report: Child Abuse and Neglect in American Indian and Alaska Native Communities and the Role of the Indian Health Service

ABSTRACT NUMBER: 097

ABSTRACT: In 1990, IHS instituted a study of its response to child abuse and neglect in American Indian and Alaska Native communities. This report provides a comprehensive assessment of the effectiveness and impact of IHS policies, procedures, and protocols in this area. It also evaluates the capacity of IHS personnel to recognize and treat the abuse and neglect of American Indian and Alaska Native children and proposes a culturally sensitive intervention program to address child abuse and neglect in these communities. The report finds that (1) 34.4 percent of American Indian and Alaska Native children are at risk of abuse, neglect, or both, according to IHS and Bureau of Indian Affairs estimates; (2) staff are reluctant to report abuse, even though reporting is mandatory for Federal employees; (3) staff lack adequate training and computerized record systems; (4) a disproportionate number of abuse and neglect victims are under age 5, and although boys and girls are equally likely to be victims of physical abuse and neglect, girls are more likely to be victims of sexual abuse; and (5) abusers are equally likely to be male or female, although sexual abusers are more likely to be male. A model intervention program of the secondary prevention type has been piloted and includes home visitor services, the promotion of healthy child growth and development, and other key program elements. The report concludes that IHS can profoundly reduce the incidence of abuse and neglect in its role as a public health provider.

AGENCY SPONSOR: Office of Planning, Evaluation, and Legislation

FEDERAL CONTACT: Leo Nolan

PHONE NUMBER: 301/443-4700

PIC ID: 6009

PERFORMER ORGANIZATION: National Indian Justice Center, Petaluma, CA

National Institutes of Health

Contents

Cancer at a Crossroads: A Report to Congress for the Nation

Career Development Applications and Awards: Medical Department Sources and Patterns of NIH Funding

Development of Medications for the Treatment of Opiate and Cocaine Addictions: Issues for the Government and Private Sector

Directions in Nursing Research Training

Evaluation Design Study for the National Research Service Awards (NRSA)

Fetal Alcohol Syndrome: Diagnosis, Epidemiology, Prevention, and Treatment

Measuring Social Inequalities in Health

Report on Outreach Activities of the National Library of Medicine: A Five-Year Review

The Status of Biomedical Research Facilities

Study of the Minority Access to Research Careers Honors Undergraduate Research Training Program

Support for Bioengineering Research

TITLE: Cancer at a Crossroads: A Report to Congress for the Nation

ABSTRACT NUMBER: 098

ABSTRACT: This evaluation of the National Cancer Program (NCP) was undertaken at the request of Congress to assess the achievements of NCP, identify barriers to reducing the burden of cancer, and make recommendations for future research and program directions. NCP comprises not just the cancer research community, but government at all levels, business and industry, the total health care system, and every United States citizen. The report concludes that the strongest strategy for a renewed war on cancer should include three essential elements: (1) applying currently available knowledge about cancer prevention and care to all segments of the population; (2) increasing support for translational research that develops basic cancer knowledge into preventive strategies, new technologies, and effective treatments; and (3) increasing support for basic cancer research to ensure continued new discoveries that lead to better cancer prevention and care. The report notes that six major issues must be addressed to mount an effective war on cancer: (1) need for health care reform with universal cancer care coverage written in statute; (2) absence of coordination in NCP; (3) inadequate cancer care: (4) current laws, policies, and regulations that hinder the development of effective cancer treatments; (5) lack of support for translational research; and (6) need to increase basic science research. (Final report 34 pages, plus appendixes.)

AGENCY SPONSOR: National Cancer Institute

FEDERAL CONTACT: Cherie Nichols

PHONE NUMBER: 301/496-5515

PIC ID: 6008

PERFORMER ORGANIZATION: National Cancer Institute, Bethesda, MD

TITLE: Career Development Applications and Awards: Medical Department Sources and Patterns of NIH Funding

ABSTRACT NUMBER: 099

ABSTRACT: This study was undertaken to find out, first, if the graduate medical school and residency enrollment in departments that have been sources of National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) and other NIH Career Development (K) awards and applications has continued to increase over time, or is leveling off or decreasing; and second, to examine whether that potential supply of applicants is in fact reflected in applications received and applications funded by NIDDK and by other institutes. The supply of potential applicants for K awards may be leveling off or decreasing, based on 1994 data of the American Medical Association, but an encouraging aspect of these data is that enrollment in postdoctoral programs in departments from which NIDDK applicants have come is not decreasing, for the most part. Applications to NIH for individual support by K awards have increased in the last 8 years, but this increase is primarily from clinical sciences departments. Applications from basic sciences have declined slightly. The relative standing of NIDDK among NIH ICDs, with respect to applications received, applications funded, and both received and funded in proportion to agency budget, has remained constant. NIDDK is third or fourth among ICDs by these criteria, and its support of Career K trainees is consistently above the NIH average.

AGENCY SPONSOR: National Institute of Diabetes and Digestive and Kidney Diseases

FEDERAL CONTACT: Carol Feld

PIC ID: 6019

PHONE NUMBER: 301/496-6623

PERFORMER ORGANIZATION: National Institute of Diabetes and Digestive and Kidney Diseases, Bethesda, MD

TITLE: Development of Medications for the Treatment of Opiate and Cocaine Addictions: Issues for the Government and Private Sector

ABSTRACT NUMBER: 100

ABSTRACT: The Medications Development Division (MDD) at the National Institute on Drug Abuse (NIDA) was established to coordinate and encourage academic, private, and Federal regulatory involvement in developing and bringing to market new medications to treat drug abuse. This report examines MDD's progress and explores the factors that hinder the development of antiaddiction medications. The report finds that pharmacotherapy for the treatment of drug addiction has received little attention. In fact, despite the success of methadone for the treatment of opiate addiction, only two additional drugs have been approved, both of which were developed in the late 1960's and early 1970's. Furthermore, there is still no approved medication for the treatment of cocaine addiction. The report also finds that the major disincentives to pharmaceutical research and development for antiaddiction medications include (1) an inadequate scientific base on the mechanisms of addiction and on the prevention of relapse, particularly for cocaine addiction; (2) lack of financing for treatment and trained treatment specialists; (3) narrow Federal and State regulations; (4) relatively small market for such drugs; (5) pricing issues and social stigma against drug abusers; (6) liability issues and difficulties in conducting clinical research; and (7) lack of sustained Federal leadership. The report provides recommendations to help remove these barriers. (Final report 199 pages, plus appendixes.)

AGENCY SPONSOR: National Institute on Drug Abuse

FEDERAL CONTACT: Laura Rosenthal

PHONE NUMBER: 301/443-6487

PIC ID: 5831

PERFORMER ORGANIZATION: National Academy of Sciences, Institute of Medicine, Washington, DC

TITLE: Directions in Nursing Research Training

ABSTRACT NUMBER: 101

ABSTRACT: The National Institute of Nursing Research (NINR) supports mechanisms to permit nurse scientists to follow research career paths. This report summarizes information on NINR's funding of these efforts, its recommendations for nursing research training in a National Research Council report, and the views and recommendations of the scientific community. This report also discusses research training issues in the broad context of Federal and academic perspectives. The report finds that (1) during its formative years, NINR placed a high priority on training at the predoctoral level in order to ensure a beginning cadre of nurse scientists; (2) the numbers of Nursing Research Service Awards (NRSA) applications and part-time enrollment are decreasing for most nursing doctoral students; (3) the number of NRSA full-time training positions must increase in the next 5 years; (4) NINR and the nursing research community must design and implement creative research training programs to ensure that scientists in nursing research have access to high-quality training in research-intensive environments. The report recommends that NINR (1) disseminate this report widely; (2) analyze the research career paths of NINR trainees and fellows to determine the criteria for successful training experiences; (3) emphasize the need to increase the number of nurses with doctoral degrees; and (4) support research consortia and partners. (Final report 19 pages.)

AGENCY SPONSOR: National Institute on Nursing Research

FEDERAL CONTACT: Suzanne L. Feetham

PHONE NUMBER: 301/402-1446

PIC ID: 6051

PERFORMER ORGANIZATION: National Institute of Nursing Research, Bethesda, MD

TITLE: Evaluation Design Study for the National Research Service Awards (NRSA) (Volume I)

Description of National Research Service Awards Program (Volume II)

ABSTRACT NUMBER: 102

ABSTRACT: The NRSA program is designed to meet the need for scientists and institutions to carry out the Nation's biomedical research agenda. This evaluation design study provides a detailed plan for the evaluation of career outcomes of predoctoral and postdoctoral trainees and fellows and the NRSA programs in which they have participated. The design study is divided into three parts: the first part describes the program and its history; the second part details previous evaluation efforts; and the third part discusses the proposed evaluation design. The report finds that (1) in 1994, NRSA provided $373 million, distributed among 2,356 individual fellowships and 12,032 traineeships awarded by institutions; (2) approximately 15 percent of the graduate students in the biomedical and behavioral science area are supported by NRSA funds; and (3) NRSA is the largest research training program in the United States. The evaluation design study proposes three approaches for the evaluation: (1) an analysis of existing databases to provide initial data, to help define sampling strategies, and to be the basis for comparisons between NRSA participants and nonparticipants; (2) a survey of prior NRSA recipients and of institutions; and (3) site visits to institutions. See also PIC ID 6012.1. (Final report 75 pages.)

AGENCY SPONSOR: Office of the Director

FEDERAL CONTACT: John Uzzell

PHONE NUMBER: 301/496-9285

PIC ID: 6012

PERFORMER ORGANIZATION: National Institutes of Health, Division of Strategic Evaluation, Bethesda, MD

TITLE: Fetal Alcohol Syndrome: Diagnosis, Epidemiology, Prevention, and Treatment

ABSTRACT NUMBER: 103

ABSTRACT: When fetal alcohol syndrome (FAS) was first described in published medical literature, hopes were high for its prevention. However, preventing this condition has been very complicated. This report responds to the mandate of Congress in the ADAMHA Reorganization Act that a study of FAS and related birth defects be carried out. The committee convened to write the report has members whose expertise includes pediatrics, developmental psychology and neurology, obstetrics, nosology, teratology, epidemiology, substance abuse prevention and treatment, and psychiatry. The committee was charged with improving the understanding of available research knowledge and experience on diagnosis, prevalence, surveillance, and prevention and treatment of FAS and related birth defects. The committee-revised diagnostic criteria aim to increase clarity, rigor, and consistency by expanding the traditional designations of FAS and other alcohol-related effects. Recommendations include: (1) cross-sectional and longitudinal research to assess the characteristics and clinical expression of these syndromes across the patient's life span; (2) research to identify potential structural or functional brain abnormalities that may be associated with FAS and related birth defects; (3) further clinical research on the adverse developmental effects of prenatal alcohol exposure; and (4) the development of more specific biologic diagnostic markers. (Prepublication report 40 pages, plus appendixes.)

AGENCY SPONSOR: National Institute on Alcohol Abuse and Alcoholism

FEDERAL CONTACT: Lora Abbott, M.S.

PHONE NUMBER: 301/443-2369

PIC ID: 6006

PERFORMER ORGANIZATION: National Academy of Sciences, Institute of Medicine, Washington, DC

TITLE: Measuring Social Inequalities in Health

ABSTRACT NUMBER: 104

ABSTRACT: This report presents the findings of a conference held in Annapolis, Maryland, in September 1994. The conference provided the conceptual background for measures of socioeconomic gradients in federally supported health data sets and recommended improvements. It also recommended methods for implementation in the context of data gathering under health care reform. The socioeconomic gradients under consideration were those used in federally supported surveys, vital statistical and disease registries, and hospital discharge data. These gradients are subsequently used to evaluate programs or as variables to help assess factors affecting health outcomes. The conference made very specific recommendations for improving existing measures or adding new ones in ongoing general purpose surveys. It evaluated current measures and recommended modifications to the vital statistical and disease registries and to hospital discharge data critical to future health care reform discussions. The conference also provided investigators wanting to apply for NIH funds with improved tools for implementing the newly revised guidelines for the inclusion of women and minorities in clinical research. The most important recommendation made was that the availability of existing social class data be advertised and that researchers be encouraged to analyze them. (Final report 5 pages.)

AGENCY SPONSOR: National Institute of Child Health and Human Development

FEDERAL CONTACT: Nancy Moss

PHONE NUMBER: 301/496-3136

PIC ID: 6020

PERFORMER ORGANIZATION: National Institute of Child Health and Human Development, Bethesda, MD

TITLE: Report on Outreach Activities of the National Library of Medicine: A Five-Year Review

ABSTRACT NUMBER: 105

ABSTRACT: In October 1987, Congress encouraged the National Library of Medicine (NLM) to develop an outreach program to transfer the latest scientific findings to all health professionals. Congressional concern was underscored in 1988 when the Senate Subcommittee on Appropriations expressed particular interest in the development of an outreach program for rural and remote health care professionals. This report evaluates NLM's outreach efforts between 1990 and 1994. The report finds that (1) the number of outreach projects increased from 16 in 1989 to almost 300 at the end of 1994; (2) in 1989, approximately 30,000 user codes had been issued, and by 1994, user codes numbered close to 100,000; (3) in 1989, there were 4 million searches of the NLM databases, and in 1994, there were 7 million searches; (4) similar changes have occurred in the use of Grateful Med; and (5) although these increases in use are impressive, and although measurement of the precise number of users of NLM's information systems from all sources is not yet available, it is reasonable to postulate that the number of users is far short of the total population of United States health care professionals who could benefit from such use. The study was carried out jointly by NLM, the University of Texas Health Science Center/San Antonio, Texas, Meharry Medical College, and Oak Ridge Associated Universities in Tennessee. (Final report 100 pages.)

AGENCY SPONSOR: National Library of Medicine

FEDERAL CONTACT: Karen Wallingford

PHONE NUMBER: 301/496-2311

PIC ID: 6010

PERFORMER ORGANIZATION: National Library of Medicine, Bethesda, MD

TITLE: The Status of Biomedical Research Facilities

ABSTRACT NUMBER: 106

ABSTRACT: The National Science Foundation (NSF) has been authorized and directed by the House of Representatives in H.R. 1210 to "design, establish, and maintain a permanent data collection and analysis capability for the purpose of assessing and identifying the research facilities needs of universities." This survey is conducted every 2 years, and the results are reported to Congress by September 1. In order to meet NIH needs, additional samples of medical schools, hospitals, and independent research organizations were added to NSF's sample of research universities. This arrangement is efficient and cost-effective for NIH because of the overlap between NSF's needs and those of NIH. NIH has input on all aspects of the biomedical facilities survey and complete control of the biomedical facilities report. "The Status of Biomedical Research Facilities" was prepared for NIH, assessing trends in aspects of extramural research facilities. The results were used by the Director of NIH, other HHS agencies, and the Office of the Secretary to plan research programs and review current policies regarding construction programs.

AGENCY SPONSOR: Office of the Director

FEDERAL CONTACT: Paul Seder, Ph.D.

PHONE NUMBER: 301/496-5011

PIC ID: 3562.1

PERFORMER ORGANIZATION: National Science Foundation, Washington, DC

TITLE: Study of the Minority Access to Research Careers Honors Undergraduate Research Training Program

ABSTRACT NUMBER: 107

ABSTRACT: The National Institute of General Medical Sciences (NIGMS) sponsors the Minority Access to Research Centers (MARC) Honors Undergraduate Research Training Program. Under the program, highly qualified minority institutions receive support to provide science courses and research training for honors students in their third and fourth years of college. Each year, there are 60 to 70 undergraduate training programs supporting approximately 560 trainees. This report presents the results of a survey on the outcomes of the MARC program and the training pathways of MARC students. The survey included all 3,062 former MARC students: those who had received support prior to, but were no longer receiving it in, fiscal year 1994. The report finds that: (1) a substantial number of former students (over 700) have already obtained research or clinical doctorates; (2) since the program began in 1977, about 1,400 blacks, Hispanics, and Native Americans obtained a bachelor's degree and had received a Ph.D. in the biological sciences by the end of the 1993 school year; (3) 92.2 percent of former MARC students had received bachelor's degrees and almost one-half of these entered graduate programs and one-third entered medical school; and (4) MARC students have pursued and obtained Ph.D.s, M.D.s, and other doctoral degrees at greater rates than minority biology and chemistry students who did not participate in the MARC program. See also PIC ID Nos. 4462.1 and 4462.2. (Final report 50 pages plus appendix.)

AGENCY SPONSOR: National Institute of General Medical Sciences

FEDERAL CONTACT: James Onken

PHONE NUMBER: 301/594-2764

PIC ID: 4462

PERFORMER ORGANIZATION: Research Triangle Institute, Research Triangle Park, NC

TITLE: Support for Bioengineering Research

ABSTRACT NUMBER: 108

ABSTRACT: The NIH Revitalization Act of 1993 mandated that the HHS Secretary, acting through the NIH Director, conduct a study of support for bioengineering research. The legislation directed evaluations of the Federal commitment to innovation, the Federal role for enhancing innovation, and the coordination among Federal agencies and between the public and private sectors. NIH is the largest source of support for bioengineering research within the Federal Government. It conducted a detailed inventory of sources and amounts of public and private funding for basic bioengineering research for fiscal 1993. To assist in understanding how innovation in bioengineering proceeds, a case study was conducted in a representative field of bioengineering: implantable prostheses. Data were collected on the science base, patents, and new health care products. One of the most important findings of the effort was that relevant data are inadequate to assess the innovation process with confidence. The report recommends that NIH (1) establish an interagency Bioengineering Coordinating Committee; (2) include basic bioengineering research within appropriate intramural programs; and (3) provide, through the Federal Register, a "comment period" notice to solicit research topics suggested for inclusion in the annual Small Business Innovation Research Omnibus solicitation. See also PIC ID 5581.1. (Final report 11 pages.)

AGENCY SPONSOR: National Heart, Lung, and Blood Institute

FEDERAL CONTACT: John T. Watson, M.D.

PHONE NUMBER: 301/496-1586

PIC ID: 5581

PERFORMER ORGANIZATION: National Heart, Lung, and Blood Institute, Rockville, MD

Office of the Assistant Secretary for Planning and Evaluation

Contents

Adolescent Time Use, Risky Behavior, and Outcomes: An Analysis of National Data

Beginning Too Soon: Adolescent Sexual Behavior, Pregnancy, and Parenthood

Case Management in Service Integration: An Annotated Bibliography

Child Support Enforcement

Evaluation of Transition To Work Demonstration Projects Using a Natural Supports Model

Exploratory Study of the Barriers and Incentives To Improving Labor Force Participation Among Persons With Significant Disabilities

Family and Community Violence Prevention Program: Evaluability Assessment and Technical Assistance Conference

Fifty-State Health Reform Database

Final Report of the Task Force on the Privacy of Private Sector Health Records

JOBS Evaluation: Early Findings on Program Impacts in Three Sites

JOBS Evaluation: How Well Are They Faring? AFDC Families With Preschool-Aged Children in Atlanta at the Outset of the JOBS Evaluation

JOBS Evaluation: Monthly Participation Rates in Three Sites and Factors Affecting Participation Levels in Welfare-to-Work Programs

Managed Care for People With Disabilities: Developing a Research Agenda

Minnesota Learning Readiness Initiative: Reforming the Delivery of Social Services to School-Aged Children and Families in Hennepin County, Minnesota

New Approaches To Evaluating Community Initiatives: Concepts, Methods, and Contexts

Noncustodial Parents' Participation in Their Children's Lives: Evidence From the Survey of Income and Program Participation

Options for Full-Day Services for Children Participating in Head Start

Patterns of Substance Use and Substance-Related Impairment Among Participants in the Aid to Families With Dependent Children Program

Performance Measurement for the U.S. Department of Health and Human Services, the Administration for Children and Families, and Its Other Operating Divisions

Report to Congress on Out-of-Wedlock Childbearing

Review of Family Preservation and Family Reunification Programs

Simulation of Trends in Employment, Welfare, and Related Dynamics

Sources of Support for Young Latina Mothers

Substance Abuse Among Women and Parents

Systematic Thinking About Government Programs for Children With Disabilities: Size, Participation, Benefits, and Expenditures

Child Support Payment Patterns Among AFDC Mothers in Massachusetts and Implications for TRIM2

Use of Administrative Data Kept by States To Study the Duration of Program Participation

TITLE: Adolescent Time Use, Risky Behavior, and Outcomes: An Analysis of National Data

ABSTRACT NUMBER: 109

ABSTRACT: This report uses several large-scale databases to examine the time-use patterns of United States adolescents in the late 1980's and early 1990's. The report compares these patterns with those exhibited by young people 10 to 20 years ago, and tests whether participation in extracurricular activities reduces the chances that they will engage in various risky behaviors. The report finds that (1) most teenagers have a lot of unfilled discretionary time; (2) teenagers use this time to watch television, talk on the telephone, gather with friends in malls or other local hangouts, or work at low-skill jobs; (3) compared with earlier cohorts from the mid-1970's or early 1980's, teenagers spend no more time doing homework--despite supposedly more rigorous courses--they read fewer books, do fewer household chores, attend fewer religious services, and participate less in traditional school-based activities; (4) students who spend no time in extracurricular activities are 57 percent more likely to have dropped out of school by senior year, 49 percent more likely to have used drugs, 37 percent more likely to have become teen parents, 35 percent more likely to have smoked cigarettes, and 27 percent more likely to have been arrested than those who spend 1 to 4 hours per week in extracurricular activities; and (5) the effectiveness of organized youth activities depends upon the extent to which the activities develop skills, create challenges, and provide fulfilling experiences for participants. (Final report 64 pages, plus appendixes.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Emily Novick

PHONE NUMBER: 202/690-5937

PIC ID: 6014

PERFORMER ORGANIZATION: Westat, Inc., Rockville, MD

TITLE: Beginning Too Soon: Adolescent Sexual Behavior, Pregnancy, and Parenthood

ABSTRACT NUMBER: 110

ABSTRACT: This series of three reports discusses trends in adolescent sexual behavior and fertility, as well as trends in adolescent pregnancy prevention programs. The reports discuss recent trends in sexual behavior among this population, its use of contraceptives, and outcomes of pregnancy. The reports highlight findings in the following areas: (1) sexual intercourse among teens; (2) contraceptive use; (3) pregnancy outcomes; and (4) prevention programs. The reports find that the rate of sexual intercourse among teens is increasing, but that contraceptive use is consistently low. The reports stress that the younger a teen is at the time of sexual initiation, the more likely the experience is to have been coercive, and the more likely it is that the teen will become pregnant or contract a sexually transmitted disease. The reports also note that many factors, such as race, family income, developmental characteristics, educational success and goals, and neighborhood mores, influence the age of first sexual intercourse. The reports indicate that contraceptive use among teens varies: older teens are more likely to use contraceptives than are risk-taking teens; contraceptive methods requiring male involvement, such as condoms and withdrawal, are more common at first intercourse, while female-based methods are more common in longer standing relationships. Furthermore, there is generally a lag time between first sexual intercourse and obtaining contraceptive services. The reports note that prevention programs emphasizing behavioral skills-oriented sex education and recommending abstinence and consistent contraceptive use are most effective, as are other programs that address alternate life options for teens. However, most of the prevention and intervention strategies designed to affect the teen birth rate are small, ad hoc, and poorly designed short-term projects lacking a useful evaluation strategy.

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Elisa Koff and Amy Nevel

PHONE NUMBER: 202/690-5880

PIC ID: 5877

NTIS ACCESSION NUMBER: PB 95-256475; PB 95-256483; PB 95-256491

PERFORMER ORGANIZATION: Child Trends, Inc., Washington, DC

TITLE: Case Management in Service Integration: An Annotated Bibliography

ABSTRACT NUMBER: 111

ABSTRACT: Case management is integral to many service integration projects and programs, which use case managers as the primary link between clients and services. While no standard definition of case management exists, some consensus on a case manager's functions is emerging. These functions include (1) assessing client needs; (2) creating an action plan; (3) brokering services; and (4) ensuring that appropriate services are accessed and delivered. This annotated bibliography includes information on (1) case management practices and case manager functions; (2) evaluations that consider the process and outcomes of case management; and (3) case management operations. The articles and books selected focus on practices that cross organizational boundaries and that generally include young children in the population served. Items included in the bibliography were published after 1983 and are currently in print and available through public libraries or for purchase. The bibliography excludes works dealing with geriatrics, mental illness, or medical care. The abstracts do not critique or analyze the works described. See also PIC ID 5307-5307.9 and 5307.B-5307.C. (Annotated bibliography 35 pages.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Richard Silva

PHONE NUMBER: 202/690-6805

PIC ID: 5307.A

PERFORMER ORGANIZATION: National Center for Children in Poverty and National Center for Service Integration, New York, NY

TITLE: Child Support Enforcement

ABSTRACT NUMBER: 112

ABSTRACT: This report examines the problem of child support enforcement and its impact on welfare reform. The report was prepared by the interagency Working Group on Welfare Reform, Family Support, and Independence. It presents information on the state of child support enforcement in the Nation and provides a rationale for reform of the system. The report finds that (1) a gap exists between current child support collections and the amount that, theoretically, could be collected (this amount is about $33.7 billion); (2) this gap exists because not all child support awards are paid, awards are generally inadequate, and many eligible custodial parents do not have a child support order or award in place; (3) the number of children potentially eligible for child support has grown, primarily because of out-of-wedlock births; and (4) there have been some improvements in collection of child support payments (for children eligible for AFDC) and in paternity establishment. However, the report also asserts that fundamental system reform is needed; children have the right to receive support from both parents, and reform will save welfare dollars. This reform should be effected through universal, immediate paternity establishment; standardization of child support awards throughout the Nation (regularly updated); and vigorous interstate enforcement of child support orders. (Final report 18 pages, plus appendix.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Paul Legler

PHONE NUMBER: 202/690-7507

PIC ID: 5777

PERFORMER ORGANIZATION: Working Group on Welfare Reform, Family Support, and Independence, Washington, DC

TITLE: Evaluation of Transition To Work Demonstration Projects Using a Natural Supports Model

ABSTRACT NUMBER: 113

ABSTRACT: This report describes and evaluates the methods used by six demonstration projects to help schools and adult providers obtain integrated employment using natural supports for students with very severe disabilities (severe mental retardation, autism, learning disability, or other condition). The major categories of service provided include (1) student-centered planning, which helps the student identify goals and interests and assesses the behavior and knowledge the student needs to attain them; (2) community-based job experience; (3) job placements, ideally before the client has left school; and (4) transition out of the school system, including long-term vocational support and locating jobs when needed. The report also finds that (1) almost 335 persons with disabilities were identified as having participated in the projects; (2) sites attempt to select those with the most severe disabilities for inclusion; (3) most participants work less than half-time; (4) jobs are usually located by using direct telephone calls to potential employment sites; and (5) costs increase for schools using this method, but not for most parents and employers. The report details barriers encountered by sites, including difficulty in locating jobs, complex and rigid funding procedures, transportation problems, and staff turnover. The report suggests several areas for consideration in the future expansion and improvement of the program. (Final report variously paginated, plus appendix.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Richard Silva

PHONE NUMBER: 202/690-6805

PIC ID: 5836

PERFORMER ORGANIZATION: Pelavin Research Institute, Washington, DC

TITLE: Exploratory Study of the Barriers and Incentives To Improving Labor Force Participation Among Persons With Significant Disabilities

ABSTRACT NUMBER: 114

ABSTRACT: In recent years, substantial interest has emerged in improving opportunities for people with disabilities to join, remain in, or return to the work force. Disability income support programs like Supplemental Security Income and Social Security Disability Insurance now have work incentive programs. The Americans With Disabilities Act has made it illegal for employers to discriminate against persons with disabilities. This report provides background information on the barriers and incentives to improving the work force participation of persons with significant disabilities. The report includes (1) a literature review on various aspects of the competitive labor market for persons with disabilities; (2) the use of personal assistance services and assistive devices by disabled workers; (3) the impact of welfare reform strategies on persons with disabilities; and (4) selected programs that offer comprehensive services to help persons with significant disabilities participate in competitive employment. The report also provides a research agenda, noting that a substantial amount is known about the employment of persons with disabilities, but much more information is needed to help the government analyze the impact of current policies on this population. More research would also assist in formulating policies that are designed to increase the participation of persons with disabilities in competitive employment. (Final report variously paginated.)

AGENCY SPONSOR: Office of Disability, Aging, and Long Term Care Policy

FEDERAL CONTACT: Kathleen Bond

PHONE NUMBER: 202/690-6443

PIC ID: 5757

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Family and Community Violence Prevention Program: Evaluability Assessment and Technical Assistance Conference

ABSTRACT NUMBER: 115

ABSTRACT: This report presents information about the Family and Community Violence Prevention Program workshop held in January 1995 in Washington, D.C., for a consortium of historically black colleges and universities (HBCUs). The consortium has been developing and demonstrating interventions aimed at curbing the incidence of family and community violence. Participants included 30 HBCU participants, 6 Federal personnel, and 8 members of the contractor's staff. Originally planned for HCBUs as a 1-day training session on evaluability assessment, the conference was extended to 1.5 days. The first day included a review of the goals of the training workshop, a viewing of antiviolence public service announcements, an overview of evaluability assessment, the identification of stakeholders and environmental factors surrounding the demonstration projects, and a review of evaluability assessment logic models. The second day offered an opportunity for HCBU participants to develop evaluability assessment action plans for use in their demonstration projects. After the conference, participants completed a process evaluation of the training workshop. Process evaluation results show that participants' expectations were reasonable and were met to their satisfaction. Participants gained an understanding of the need for evaluability assessments and were prepared to conduct them. After the conference, the contractor also offered technical assistance to all HCBUs, based on lessons learned during the conference.

AGENCY SPONSOR: Office of Health Policy

FEDERAL CONTACT: Burke Fishburn

PHONE NUMBER: 202/690-7807

PIC ID: 5789

PERFORMER ORGANIZATION: Macro International, Inc., Silver Spring, MD

TITLE: Fifty-State Health Reform Database

ABSTRACT NUMBER: 116

ABSTRACT: This project constructed a database consisting of data elements related to health reform for all 50 States. This database supports the generation of customized State profiles for a variety of purposes. For example, Administration officials need information specific to each State in order to better support a State's health care reform efforts. The database will also allow comparison of individual State health care report efforts to the Federal effort and to each other. Some of the questions that can be answered by the database include (1) how many States have adopted small group market reforms or risk pools for uninsurables and (2) how Federal reform efforts might affect each State. The database has not been updated since the end of 1994, but can be updated for a small additional investment of resources.

AGENCY SPONSOR: Office of Health Policy

FEDERAL CONTACT: Sarah Jane Holcombe

PHONE NUMBER: 202/690-7804

PIC ID: 5394

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Final Report of the Task Force on the Privacy of Private Sector Health Records

ABSTRACT NUMBER: 117

ABSTRACT: The mandate of the Task Force on the Privacy of Private Sector Health Records was to examine the extent to which problems exist with the collection, storage, and use of health information in the private sector. The changing policy environment refocused the mission to include examination of ways to protect the privacy of all health care information within the context of health care reform and the development of electronic health information networks. The task force examined the social, legal, and economic issues affecting the privacy of people who use the health care system. It addressed several policy questions, including (1) the kinds of records to be protected; (2) the treatment of especially sensitive records; (3) the level of legislation that should enact privacy provisions; (4) the circumstances under which a record keeper should disclose health information; (5) the impact of automation on the privacy of health records; (6) the use of unique identifiers for health records; (7) the oversight structure needed to protect privacy, confidentiality, and security matters and violations; and (8) the training, education, and awareness programs needed for health care users and providers. The report recommends a coordinated Federal policy on the privacy of health care records; the use of universal identifiers; effective security standards and guidance; the establishment of a data protection entity; and an education program about the issue. (Final report 128 pages, plus appendix.)

AGENCY SPONSOR: Office of Program Systems

FEDERAL CONTACT: Joan Turek-Brezina

PHONE NUMBER: 202/690-6141

PIC ID: PIC 5879

PERFORMER ORGANIZATION: Kunitz and Associates, Inc., Rockville, MD

TITLE: JOBS Evaluation: Early Findings on Program Impacts in Three Sites

ABSTRACT NUMBER: 118

ABSTRACT: This report presents impact results of an evaluation of the Job Opportunities and Basic Skills Training (JOBS) program in three sites: Atlanta, Georgia; Grand Rapids, Michigan; and Riverside, California. The JOBS program examines two interventions: the labor force attachment (LFA) approach, emphasizing rapid job entry; and the human capital development (HCD) approach, which offers longer, skill-building education and training activities. Participants were randomly assigned to one of these approaches or to one of two control groups. The report is based on telephone and in-person surveys of 2,604 people in the three sites. The report finds that (1) the three sites successfully operated two distinct, well-run, and highly mandatory LFA and HCD versions of JOBS; (2) the LFA approach increased participation in job-search activities dramatically, slightly increased participation in other work-directed services, and resulted in a high rate of sanctioning (removal of a portion of welfare benefits because of nonparticipation); (3) the HCD approach also resulted in a high rate of sanctioning, and the basic services provided were primarily basic education and job search; and (4) the LFA approach substantially increased the number of people who found work and left the welfare rolls within 2 years, and the LFA's impact on AFDC receipts and payments and on earnings was substantial, while the HCD approach had not yet translated into higher earnings.

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Audrey Mirsky-Ashby

PHONE NUMBER: 202/401-6640

PIC ID: 5776.4

PERFORMER ORGANIZATION: Manpower Demonstration Research Corporation, New York, NY

TITLE: JOBS Evaluation: How Well Are They Faring? AFDC Families With Preschool-Aged Children in Atlanta at the Outset of the JOBS Evaluation

ABSTRACT NUMBER: 119

ABSTRACT: This report provides a descriptive summary of a Child Outcomes Study (a component of the larger JOBS program evaluation) in Fulton County, Georgia, near the start of the JOBS evaluation. The Fulton County JOBS evaluation uses a human capital development approach, a labor force attachment approach, and a control group. The descriptive sample included 790 respondents from the JOBS Child Outcomes Study in Fulton County. All respondents were mothers whose youngest child was between age 3 and 5 at the time of random assignment, all were 20 years of age or older, and 96 percent were African-American. The report finds that the mothers in the Fulton County sample are highly disadvantaged in many ways. Their reading and math literacy levels are low, they report minimal economic or other assistance from the fathers of their children, and they have high rates of depressive symptoms. On the other hand, most report social support from family and friends, have completed high school or the graduate equivalent degree, have positive attitudes about maternal work, and have taken steps toward limiting their childbearing. The children in the sample are also disadvantaged: their receptive vocabulary is substantially below the national mean and many lack the skills and knowledge needed for school readiness. Finally, the report notes that the heterogeneity of the sample mothers will lead to varied outcomes for their children.

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Audrey Mirsky-Ashby

PHONE NUMBER: 202/401-6640

PIC ID: 5776.2

PERFORMER ORGANIZATION: Manpower Demonstration Research Corporation, New York, NY

TITLE: JOBS Evaluation: Monthly Participation Rates in Three Sites and Factors Affecting Participation Levels in Welfare-to-Work Programs

ABSTRACT NUMBER: 120

ABSTRACT: The JOBS program provides an array of job search, work experience, education, and training services to families who receive AFDC. The program requires participation; those who do not participate are sanctioned with the loss of part of their welfare grant. The legislation authorizing JOBS also requires States to meet specific, and incrementally increasing, participation standards. This report uses data collected from the case files of 1,113 AFDC recipients in three JOBS evaluation sites (Atlanta, Georgia; Grand Rapids, Michigan; and Riverside, California) to examine participation rates. The report notes that the definitions of participation and the classes of people included in the measure greatly affect the feasibility of achieving a particular standard. The report finds that (1) only 5 to 10 percent of all single-parent AFDC clients participated in the JOBS program or worked the required hours during each week in a month; (2) including sanctioning in the definition of participation raises participation rates to between 9 and 21 percent; (3) using only JOBS-mandatory individuals in the measure increases the participation rate; and (4) if participation rates remained at 1992 levels at the sites, the rates would fall far short of the ultimate standards contained in the 1995 welfare reform bills, despite the JOBS program's tough standards, and its success compared with other programs. (Final report 59 pages, plus appendixes.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Audrey Mirsky-Ashby

PHONE NUMBER: 202/401-6640

PIC ID: 5776.3

PERFORMER ORGANIZATION: Manpower Demonstration Research Corporation, New York, NY

TITLE: Managed Care for People With Disabilities: Developing a Research Agenda

ABSTRACT NUMBER: 121

ABSTRACT: This report provides background information for a meeting held on the subject of managed care and people with disabilities, hosted by the Office of the Assistant Secretary for Planning and Evaluation's Office of Aging, Disability, and Long Term Care Policy. The purpose of the meeting was to (1) review the implications of the movement toward managed health care for people with physical and mental disabilities; and (2) flesh out a research and demonstration program to improve the understanding of the impact of managed care on people with disabilities. The report highlights critical policy areas and related research issues around managed care and disability. These policy areas include (1) functional and demographic characteristics of persons with disabilities who receive managed care; (2) the impact of managed care, including how participation affects access to needed services; (3) targeting managed care plans to people who are disabled; (4) financing and reimbursement; (5) service coverage and organization of the delivery system; and (6) quality assurance. For each of these areas, the report discusses the policy issues involved, research completed and under way on the subject, and suggested directions for research. The report stresses that little information has been generated about how any of these areas affect persons with disabilities. The report also notes that many persons with disabilities fear the role managed care will play in their access to services and the quality of those services, and that some of these fears are justifiable. (Final report 28 pages.)

AGENCY SPONSOR: Office of Disability, Aging and Long Term Care Policy

FEDERAL CONTACT: Mary Harahan

PHONE NUMBER: 202/690-6172

PIC ID: 4927

PERFORMER ORGANIZATION: Office of the Assistant Secretary for Planning and Evaluation, Washington, DC

TITLE: Minnesota Learning Readiness Initiative: Reforming the Delivery of Social Services to School-Aged Children and Families in Hennepin County, Minnesota

ABSTRACT NUMBER: 122

ABSTRACT: This report presents information on a major collaborative service integration initiative in Hennepin County, Minnesota. The two-track initiative was intended to mobilize community planning and collaboration around the development of school-linked services. The first track, the Learning Readiness Initiative, concentrated on research and development. The second track, the School Human Services Redesign Initiative, was a large-scale effort to reform delivery of social and other support services to families. The Learning Readiness Initiative began with a feasibility study, "When Kids and Systems Collide," which found that, despite over $1 billion spent on education and human services in Hennepin County, a massive policy collaboration was needed to make these services effective. The initiative used research and development-oriented grant making to develop new approaches and to foster collaboration. The School Human Services Redesign Initiative, begun while the Learning Readiness Initiative was still under way, had four goals: (1) to increase equity on all levels among students in school achievement, school service, and health outcomes; (2) to strengthen families' ability to support their children; (3) to enhance the responsiveness of human services and schools to the needs of students and families; and (4) to strengthen community bonds. (Final report 47 pages, plus appendixes.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Richard Silva

PHONE NUMBER: 202/690-6805

PIC ID: 5340

PERFORMER ORGANIZATION: Minneapolis Youth Coordinating Board, Minneapolis, MN

TITLE: New Approaches To Evaluating Community Initiatives: Concepts, Methods, and Contexts

ABSTRACT NUMBER: 123

ABSTRACT: This report examines issues in evaluating comprehensive community initiatives (CCIs). CCIs have their origins in the settlement houses of the late 19th century and the War on Poverty, among other influences. Most CCIs seek to provide social and health care services to low-income communities, as well as improve general conditions in the communities. Most operate under the belief that authority and responsibility must reside at the local level, rather than with the municipal, State, or Federal governments. The report's introduction addresses several factors that make CCIs difficult to evaluate, including horizontal and vertical complexity; contextual issues, such as the macroeconomic climate that is out of the control of CCIs; flexible and evolving interventions; the broad range of outcomes sought; and the absence of a comparison community or control group. The articles included in the report attempt to address some of these difficulties. The article titles are (1) "Evaluating CCIs: A View From History"; (2) "Nothing as Practical as Good Theory: Exploring Theory-Based Evaluation for CCIs for Children and Families"; (3) "How Do Urban Communities Affect Youth? Using Social Science Research to Inform the Design and Evaluation of CCIs"; (4) "Problems in the Evaluation of Community-Wide Initiatives"; (5) "Using Community-Level Indicators of Children's Well-Being in CCIs"; and (6) "The Role of the Evaluator in CCIs." (Final report 225 pages.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Barbara Bromar

PHONE NUMBER: 202/690-6461

PIC ID: 5895

PERFORMER ORGANIZATION: Aspen Institute, New York, NY

TITLE: Noncustodial Parents' Participation in Their Children's Lives: Evidence From the Survey of Income and Program Participation

ABSTRACT NUMBER: 124

ABSTRACT: This study examines the relationship between noncustodial parental involvement, child well-being, child support, custody and visitation arrangements, parental income, and family structure. In examining these interactions, the report focuses on father involvement and child well-being. The report includes (1) a literature review and annotated bibliography and (2) a secondary analysis using the Survey of Income and Program Participation. Because the majority of noncustodial parents are fathers, it is hoped that the findings from this report will provide new insights for promoting father involvement for children in divorced, separated, and never-married families.

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Anne Benson

PHONE NUMBER: 202/690-7409

PIC ID: 6158

PERFORMER ORGANIZATION: The Lewin Group, Fairfax, VA

TITLE: Options for Full-Day Services for Children Participating in Head Start

ABSTRACT NUMBER: 125

ABSTRACT: This study examines full-day services offered by eight Head Start grantees that fund extended hours by combining resources from a variety of sources. Data were collected during 2-day site visits to each grantee. The grantees provide services in one of three ways: (1) wraparound care, which uses funding from sources other than Head Start to expand services; (2) wrap-in care, in which Head Start grantees contract with another provider and fund the set of services needed to bring the contracted program up to Head Start standards; and (3) connected care, in which grantees contract with existing child care programs to provide children with supervised care before and after the Head Start day. The report examines program differences in full-day services, funding, fiscal management, and collaboration. The report finds that grantees (1) develop innovative ways to train staff and to integrate parent activities and home visits into the full-day programs; (2) develop ways to allocate funding from various sources, but also deal with several funding issues, including parental loss of eligibility for full-day funding, retroactive reimbursements, and funding shortfalls; and (3) encounter quality assurance problems using wrap-in or connected care. The report lists several issues for consideration by Federal policymakers and points toward needed research. (Final report 47 pages, plus appendix.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Laura Feig

PHONE NUMBER: 202/690-5938

PIC ID: 5333

PERFORMER ORGANIZATION: Pelavin Research Institute, Washington, DC

TITLE: Patterns of Substance Use and Substance-Related Impairment Among Participants in the Aid to Families With Dependent Children Program

ABSTRACT NUMBER: 126

ABSTRACT: This report was sponsored and performed jointly by ASPE, the Substance Abuse and Mental Health Services Administration (SAMHSA), and NIDA. This report provides data on substance use and substance-related impairments among participants in the AFDC program. The study uses data from the 1991 and 1992 National Household Surveys on Drug Abuse to gain information about the prevalence of substance abuse in the AFDC population and to investigate the rate of substance-related impairment in this population. Substance-related impairment is categorized as either "significant impairment," which may preclude participation in training or education activities, or "some impairment," which may require treatment concurrent with these activities. The report finds that (1) 4.9 percent of female AFDC recipients and 5.2 percent of all AFDC recipients report significant substance-related impairment; (2) an additional 10.6 percent of female AFDC recipients and 11.2 percent of all recipients report some impairment resulting from substance use; and (3) AFDC recipients have higher rates of substance use and substance-related impairments than the general population, but the vast majority of substance users and substance-related impaired people do not receive AFDC. See also PIC ID 5180 and 5180.1. (Final report 26 pages, plus appendixes.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Laura Feig

PHONE NUMBER: 202/690-5938

PIC ID: 5180.2

PERFORMER ORGANIZATION: Office of the Assistant Secretary for Planning and Evaluation, Washington, DC

TITLE: Performance Measurement for the U.S. Department of Health and Human Services, the Administration for Children and Families, and Its Other Operating Divisions

ABSTRACT NUMBER: 127

ABSTRACT: This report presents illustrative, candidate sets of outcome indicators for Child Welfare and Child Health Services, as well as recommendations to the Department of Health and Human Services (HHS) to further develop outcome indicators. Indicators in the area of child health include (1) increasing the number of insured pregnant women and insured children; (2) increasing access and reducing barriers to services; (3) coordinating and simplifying programs and systems; (4) establishing quality standards; (5) implementing public health education; and (6) increasing the numbers of women and children receiving appropriate services. The report also indicates end goals for children's health. The child welfare outcomes goals are grouped in the areas of safety, permanency, child development, and customer service. These goals relate to home-environment safety; permanent placement for children; school-based performance and age-appropriate growth and behavior; and State-, child-, and family-level satisfaction with services. The recommendations presented in the report are clustered around the major elements that should be included in a plan for a departmentwide or operating divisionwide performance measurement process. These elements are (1) establishing high-level policy and working committees to oversee the effort; (2) undertaking comprehensive implementation; and (3) providing training and technical assistance to programs. (Final report variously paginated.)

AGENCY SPONSOR: Office of Program Systems

FEDERAL CONTACT: Mike Herrell

PHONE NUMBER: 202/690-5739

PIC ID: 5081

PERFORMER ORGANIZATION: Urban Institute, Washington, DC

TITLE: Report to Congress on Out-of-Wedlock Childbearing

ABSTRACT NUMBER: 128

ABSTRACT: This report was prepared in response to the Violent Crime Control and Law Enforcement Act of 1994, which requires that HHS prepare an analysis of the increases in nonmarital births. A Department working group was formed of staff from ASPE, NCHS, and NICHD. The report examines the trends in, consequences of, and causes of out-of-wedlock childbearing. The report finds that (1) in 1993, nearly one-third of all births occurred outside of marriage; (2) less than one-half of nonmarital births were first births, but teenagers account for about one-half of all nonmarital first births; (3) nonmarital birth rates are highest during the ages of 18-29, and tend to be higher among disadvantaged and less-educated women and women in urban areas; (4) the proportion of nonmarital pregnancies that ended in abortion declined from 60 percent to 46 percent between 1980 and 1991; and (5) the consequences and causes of nonmarital births are difficult to determine because most of these parents are disadvantaged before the birth. The report examines the role of welfare; economic opportunities; neighborhood influences; individual and family characteristics; and attitudes, values, and norms in the increase in nonmarital births. See also Call Number DOC.HE 20.6202:C43/x in PIC book collection.

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Elisa Koff

PHONE NUMBER: 202/690-5880

PIC ID: 5910

PERFORMER ORGANIZATION: National Center for Health Statistics, Hyattsville, MD

TITLE: Review of Family Preservation and Family Reunification Programs

ABSTRACT NUMBER: 129

ABSTRACT: This report describes the state of the family preservation field and examines in greater depth the characteristics and operations of family preservation programs that are potential sites for future outcome evaluation. The report looks at placement prevention programs in 26 States. Of these States, 22 have one or more statewide models (Colorado, Florida, Iowa, New Hampshire, and Oregon use two family preservation models). Four of the 26 states do not have a specific program model; they make funds available to counties and allow the counties determine their own models. The report also analyzes these family preservation programs' characteristics, including (1) program models; (2) referral sources; (3) referral practices, including definition of imminent risk, decisionmaking processes, and exclusion criteria; (4) program maturity; (5) service providers; and (6) program statistics. The report also discusses family reunification programs. There were 26 such programs on the State and county levels in 15 of the 26 States contacted. Reunification programs that are part of a more general family preservation approach indicate that only a small percentage of cases served are reunification cases. These programs generally provide aftercare services and do not differentiate between preservation and reunification cases. (Final report 82 pages, plus appendixes.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Matt Stagner

PHONE NUMBER: 202/690-5953

PIC ID: 5337

PERFORMER ORGANIZATION: Westat, Inc., Rockville, MD

TITLE: Simulation of Trends in Employment, Welfare, and Related Dynamics

ABSTRACT NUMBER: 130

ABSTRACT: The Simulation of Trends in Employment, Welfare, and Related Dynamics (STEWARD) model was developed to inform the debate on welfare policy by simulating policy options that reflect several key questions: How will policies intended to encourage work affect the duration of welfare dependency? How will education and training programs reduce the duration of welfare receipt? How will AFDC costs and caseloads change as a result of reforms, and in what time frame? This report describes the STEWARD model, including (1) its development; (2) its unique features; (3) the kinds of policy options it can model; (4) the concept of the model; (5) how the concept is translated into an operational model; (6) the sources of key empirical estimates; (7) an overview of the impact measures routinely presented in the model output tables; and (8) the model database. The report describes three scenarios that are simulated by STEWARD: (1) a baseline case that simulates welfare caseloads, dynamics, and costs for a baseline period (here, 1992) under actual policies in effect at the time; (2) what welfare caseloads and costs would have been had the policy changes that have been enacted recently been implemented in the baseline year; and (3) what welfare caseloads and costs would be if welfare reform policies were enacted on top of current policies. (Final report 103 pages, plus appendixes.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Don Oellerich

PHONE NUMBER: 202/690-5877

PIC ID: 5882

PERFORMER ORGANIZATION: Mathematica Policy Research, Inc., Plainsboro, NJ

TITLE: Sources of Support for Young Latina Mothers

ABSTRACT NUMBER: 131

ABSTRACT: In 1992, the total Latina fertility rate was 3.04 births per woman, compared with 1.94 for all non-Latina women. Since 1980, Latina total fertility has risen by about 20 percent, and teen Latina birth rates have increased by 30 percent. These figures contrast sharply with fertility rates of less than 7 percent for all non-Latinas since 1980 and 6 percent for non-Latina teens. This report examines the support strategies used by young Latina mothers in the United States in order to identify Latina subgroups in greatest need of additional support. Using data from the 1990 census, the report considers (1) the young mother's living arrangements and (2) how she supports herself and her child (through employment and public assistance). The report finds that (1) Puerto Rican mothers are less likely to be married or to be living with parents or other adults and are more likely to be living in poverty and receiving welfare; (2) Cuban mothers have the highest household incomes and the lowest rates of receiving welfare; (3) Mexican and Central and South American mothers are similar to Whites in terms of their marriage patterns and living arrangements, but have much higher poverty rates; (4) foreign-born mothers are considerably more likely than their U.S.-born counterparts to use family or kin resources, and they appear to use this strategy rather than welfare; and (5) the most vulnerable mothers are teens raising children alone. (Final report 12 pages, plus appendix.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Elisa Koff

PHONE NUMBER: 202/690-5880

PIC ID: 5917

PERFORMER ORGANIZATION: Urban Institute, Washington, DC

TITLE: Substance Abuse Among Women and Parents

ABSTRACT NUMBER: 132

ABSTRACT: This report uses data from the 1991 National Household Survey on Drug Abuse and the 1991 Drug Abuse Warning System to examine the prevalence of substance abuse by women of childbearing age and by parents. The report also examines the number of children potentially at risk because of parental drug abuse. The report finds that (1) 5.7 percent of women age 15-44 years with children living in their households report past-month illicit drug use, compared with 11.2 percent of women in the same age bracket without children living in their homes, and the rates for men show similar patterns; (2) approximately 6 million children (9 percent) under age 18 have parents who have used illicit drugs in the past month, most commonly marijuana; (3) substance abuse varies little by urban/suburban/rural residency; (4) the prevalence of drug abuse among parents shows similar demographic patterns, as does drug use in the general population; (5) 4 percent of mothers and 13 percent of fathers report consumption of 5 or more alcoholic drinks at one time on at least 3 occasions in the past 30 days; and (6) children of drug-using parents tend to be younger than children overall--20 percent of children of both past-year and past-month drug users are under 3 years old, whereas 17 percent of all children are in this age group. See also PIC ID 5180 and 5180.2. (Final report 51 pages, plus appendixes.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Laura Feig

PHONE NUMBER: 202/690-5938

PIC ID: 5180.1

PERFORMER ORGANIZATION: CSR, Incorporated, Washington, DC

TITLE: Systematic Thinking About Government Programs for Children With Disabilities: Size, Participation, Benefits, and Expenditures

ABSTRACT NUMBER: 133

ABSTRACT: According to one source, about 4.5 million children in the United States have a disability. Furthermore, measures of the prevalence of disabilities among children and adolescents have been growing over time, possibly because of changes in data collection, increased survival of low birth-weight infants and children with terminal chronic illnesses; increased responsiveness to programs that provide assistance; and greater parental, educational, and medical awareness and detection of disability in children. This report attempts to provide a better understanding of how various Federal, State, and local programs serving children with disabilities have evolved and how they fit together. The report contains information on the participation, benefits, and expenditures of major programs serving this population. The report finds that (1) at the Federal level, the largest programs for children with disabilities are Supplemental Security Income (SSI) and Medicaid; (2) participation in SSI has increased 150 percent since 1989, and, since SSI recipients are automatically eligible for Medicaid in most States, the number of children with disabilities receiving Medicaid has also increased; (3) the largest State and local program serving this population is special education, which provides educational and other services (speech therapy, counseling, and physical therapy, among others); and (4) data on this population is limited and hampers policymakers' ability to serve the population. (Final report 119 pages, plus appendixes.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Barbara Broman

PHONE NUMBER: 202/690-6461

PIC ID: 5950

PERFORMER ORGANIZATION: Urban Institute, Washington, DC

TITLE: Child Support Payment Patterns Among AFDC Mothers in Massachusetts and Implications for TRIM2

ABSTRACT NUMBER: 134

ABSTRACT: AFDC and the Federal Child Support Enforcement Program (Title IV-D) are closely linked. In fact, the Title IV-D program was originally established to stem escalating AFDC costs. Two surveys, the Current Population Survey and the Survey of Income and Program Participation collect data on child support collections. However, these self-reported data are subject to limitations endemic to this kind of information. This report presents an analysis of data collected on the State level (in this case, Massachusetts) for a random sample of at least 2,400 AFDC cases covering 12 consecutive months and having at least 1 month in which child support was due during 1993. The sample was reduced to 2,364 cases because of data limitations for some cases. The report presents analyses of these data by ethnic/racial group; marital status and education of the mother; and amount of child support due, the amount paid, and the percent of amount due that was paid. The report then considers the implications of these analyses for the Transfer Income Model-Number 2 (TRIM2) microsimulation model. It describes how TRIM2 models child support awards and payments, and how Title IV-D/AFDC data might be used for TRIM2 validation and improvement. The report concludes that State-level Title IV-D/AFDC data provide a rich source of information on the child support characteristics of families receiving AFDC, and that additional analyses of State Title IV-D/AFDC data could prove very useful for validating and improving the child support simulation procedures in the TRIM2 model. (Final report 45 pages.)

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Linda Mellgren

PHONE NUMBER: 202/690-7507

PIC ID: 4942

PERFORMER ORGANIZATION: Urban Institute, Washington, DC

TITLE: Use of Administrative Data Kept by States To Study the Duration of Program Participation

ABSTRACT NUMBER: 135

ABSTRACT: This study assesses whether administrative data kept by States can be used to study the duration of program participation in the AFDC, food stamps, medical assistance, and Title IV-D programs. The need for longitudinal data is compounded by recent efforts to reform welfare that limit the duration of a welfare spell to 24 months. Two types of data may be obtainable: retrospective data that are currently available, including case histories; and prospective data that are assembled as events occur. Of 15 States contacted, 4 (California, Georgia, New York, and Texas) were selected for the collection of detailed information on their respective data systems. The report finds that California and Texas offer the best possibilities for collecting retrospective data, while New York and California offer the best possibilities for collecting prospective data. The report offers two options for assembling longitudinal data about program participation on the State level: (1) asking States to produce these files themselves and providing them the funding to do so and (2) asking States to provide administrative data in whatever form it exists to a third party, who will assemble the longitudinal files. The report considers the positive and negative aspects of each of these options and concludes that the second option is better from a research standpoint. Under this option, data are more likely to be usable to address relevant policy questions and to make State comparisons. Furthermore, economies of scale are expected to make this option more cost-effective over time. The report notes that if neither option is considered viable, California and Texas have data available that may be examined, but these data lack information about all programs. Finally, the report cautions that new longitudinal data will not be available for at least 9 months from the date of funding.

AGENCY SPONSOR: Office of Human Services Policy

FEDERAL CONTACT: Linda Mellgren

PHONE NUMBER: 202/690-7507

PIC ID: 4945

PERFORMER ORGANIZATION: Urban Institute, Washington, DC

Office of Public Health and Science

Contents

Baseline Data on the Delivery of Clinical Preventive Services Provided by Nurse Practitioners

Breast and Reproductive Cancer Screening Project for Women 40 Years of Age and Older

Consequences of Whistleblowing for the Whistleblower in Misconduct in Science Cases

Evaluation and Enhancement of Primary Care Research in Community, Migrant, and Homeless Health Centers in the Public Health Service, Region VI

Federal Regulation of Methadone Treatment

Financing of Academic Medical Centers and Graduate Medical Education Under Competitive Systems With Managed Care (Two Volumes)

HIV and the Blood Supply: An Analysis of Crisis Decisionmaking

Improving the Supply, Training, and Distribution of Primary Care Providers Under Federal Health Care Reform

Measuring Expenditures for Essential Public Health Services

Minority HIV/AIDS Programs Community-Based Organization Evaluation: HIV/AIDS Education/Prevention Grant Program 1988-89, A Cross-Site Evaluation

Region IX Hepatitis B Project Final Report

Review of the Fialuridine (FIAU) Clinical Trials

TITLE: Baseline Data on the Delivery of Clinical Preventive Services Provided by Nurse Practitioners

ABSTRACT NUMBER: 136

ABSTRACT: This report obtains baseline data on the disease-prevention activities of nurse practitioners as primary care providers and compares the findings with target goals from 17 objectives of Healthy People 2000: National Health Promotion and Disease Prevention Objectives. The study surveyed 1,310 nurse practitioners. The 892 respondents reported spending 50 percent or more of their time providing primary care, and it appears that nurse practitioners are closer to the Healthy People 2000 target percentages for assessment services than for intervention services (that is, nurse practitioners exceeded assessment goals with regard to preconception care, family planning, and emotional and behavioral functioning; however, they exceeded the intervention goals only in the areas of preconception care and family planning). With regard to the provision of screening and immunization services, nurse practitioners working in pediatrics and family medicine exceeded the target percentages for scheduled hemoglobin and hematocrit testing; blood pressure monitoring; height and weight checks; and diphtheria, pertussis, and tetanus vaccinations. Concerning the provision of cancer screening services, nurse practitioners in obstetrics/gynecology and family medicine exceeded the objectives for breast exams, Pap smears, and mammograms. See also PIC ID 4962.1-4962.3. (Final report 42 pages, plus appendixes.)

AGENCY SPONSOR: Office of Disease Prevention and Health Promotion

FEDERAL CONTACT: Deborah R. Maiese

PHONE NUMBER: 202/401-5809

PIC ID: 4962

PERFORMER ORGANIZATION: National Alliance of Nurse Practitioners, Washington, DC

TITLE: Breast and Reproductive Cancer Screening Project for Women 40 Years of Age and Older

ABSTRACT NUMBER: 137

ABSTRACT: This project evaluated the impact of factors that impede the implementation of breast and reproductive cancer prevention efforts at nine C/MHCs located in California. This report reviews screening, referral, diagnosis, treatment, and case management protocols in the nine C/MHCs. Federal funding requires that C/MHCs file programmatic assurances stating that such protocols are on file. Two sources of established guidelines, the American College of Obstetricians and Gynecologists clinical guidelines and the California State Breast and Cervical Cancer Control Program clinical guidelines, were used as the basis of comparison for the C/MHC guidelines. Eight of the nine C/MHCs submitted protocols; the ninth submitted a letter listing the routines they followed for women over 40. The report finds that, of the protocols submitted, all are either developed by other organizations or developed specifically for the C/MHC. All included information on women over the age of 40. However, several weaknesses existed: (1) community education and outreach was lacking among the majority of C/MHCs; (2) only one of the nine C/MHCs designated staff in the protocols who would be performing the different care duties; (3) no ethnic, culturally specific barriers were addressed in protocols submitted by any of the C/MHCs; and (4) the C/MHCs fail to understand the content of protocols or their importance. (Final report 16 pages, plus appendix.)

AGENCY SPONSOR: Public Health Service, Regional Office IX

FEDERAL CONTACT: Irene Reed

PHONE NUMBER: 415/556-7009

PIC ID: 5625

PERFORMER ORGANIZATION: Associated California Health Centers, Inc., Lamont, CA

TITLE: Consequences of Whistleblowing for the Whistleblower in Misconduct in Science Cases

ABSTRACT NUMBER: 138

ABSTRACT: Whistleblowers in scientific misconduct cases are highly likely to experience one or more negative consequences as a result of their whistleblowing. This report presents the results of a mail survey of those individuals listed in the Office of Research Integrity files as having made allegations of scientific misconduct. Only individuals involved in closed cases were contacted for the study. Of the 127 individuals eligible to be surveyed, 105 whistleblowers were located and mailed a survey. Of these, 89 responded, of whom 68 completed the surveys. Survey results show that (1) most whistleblowers believe that the negative consequences they have experienced as a result of their actions have had a neutral impact on their careers, professional activities, and personal lives; (2) 69 percent report experiencing at least one or more negative outcomes, while 31 percent experienced none; (3) 25 percent report serious consequences, such as the loss of position or denial of tenure, promotions, or salary increases; (4) whistleblowers attribute negative consequences to institutional officials, respondents, colleagues, and professional societies; (5) most consequences were experienced during the response to the allegations and after the inquiry was completed; and (6) despite the negative consequences cited, 68 percent of respondents said they would make another allegation, 10 percent were uncertain, and 10 percent would not. (Final report 61 pages, plus appendixes.)

AGENCY SPONSOR: Office of Research Integrity

FEDERAL CONTACT: Lawrence J. Rhoades, Ph.D.

PHONE NUMBER: 301/443-5300

PIC ID: 5659

NTIS ACCESSION NUMBER: PB 96-200449

PERFORMER ORGANIZATION: Research Triangle Institute, Research Triangle Park, NC

TITLE: Evaluation and Enhancement of Primary Care Research in Community, Migrant, and Homeless Health Centers in the Public Health Service, Region VI

ABSTRACT NUMBER: 139

ABSTRACT: This project assessed the level of interest in primary care practice-based research in PHS Region VI. The project (1) ascertained and evaluated past and present activities and potential sources for primary care research in Region VI; (2) developed and enhanced primary care research linkages; and (3) promoted interest in primary care research and clinical networking in Region VI. Surveys were mailed to the medical director and the executive director in each of nine sites chosen from C/MHCs in Region VI. Analysis of survey responses shows that (1) clinicians are interested in primary care research, electronic mail, and clinical bulletin board systems; (2) respondents are more interested in research on preventive and lifestyle factors than on clinical diseases; and (3) only a minority of clinicians have access to computer lists of patient diagnoses and ordered tests. Significant barriers to research include (1) lack of research skills and expertise; (2) too many clinical responsibilities; and (3) lack of access to research consultation. The report recommends that (1) further evaluation of PHS and non-PHS clinic sites in Region VI be carried out to determine topics of interest; (2) collaborative arrangements with academic researchers be developed; and (3) possible electronic forums for clinician use be investigated. (Final report 10 pages, plus appendixes.)

AGENCY SPONSOR: Public Health Service, Regional Office VI

FEDERAL CONTACT: Robert A. Sappington

PHONE NUMBER: 214/767-3719

PIC ID: 5914

PERFORMER ORGANIZATION: Migrant Clinicians Network, Austin, TX

TITLE: Federal Regulation of Methadone Treatment

ABSTRACT NUMBER: 140

ABSTRACT: Methadone hydrochloride (methadone) has been used to treat opiate addiction since 1972. Unlike other controlled substances, methadone use is controlled by a many-tiered system of regulations. FDA, the Drug Enforcement Administration, NIDA, and SAMHSA provide Federal oversight to methadone programs, while State and local agencies may also regulate them. This report examines current HHS standards for methadone programs and evaluates the effects of Federal regulations on the provision of methadone treatment services. The report also explores options for modifying these regulations. It finds that the underlying ideology of methadone regulations is that the societal risks of methadone outweigh the benefits to such an extent that its use must be extraordinarily regulated. The report contends that this belief is not valid in the current environment and that the stringent regulation of methadone removes power from the hands of clinicians in determining the best course of a patient's treatment and causes unintended negative impacts. The report recommends that the scope of Federal regulations be scaled back to allow greater clinical discretion, and that opiate-addicted patients be regarded holistically by methadone programs and other health and human services providers. (Final report 237 pages.)

AGENCY SPONSOR: Office of Public Health and Science

FEDERAL CONTACT: Carol Roddy

PHONE NUMBER: 202/205-0152

PIC ID: 5627

PERFORMER ORGANIZATION: National Academy of Sciences, Institute of Medicine, Washington, DC

TITLE: Financing of Academic Medical Centers and Graduate Medical Education Under Competitive Systems With Managed Care (Two Volumes)

ABSTRACT NUMBER: 141

ABSTRACT: This report examines how the growth of competitive health care financing and service delivery systems based on managed care have affected or could affect the financial support available to academic medical centers (AMCs) and graduate medical education (GME). Case studies of AMCs in Minneapolis/St. Paul, San Diego, and Washington, D.C., and a review of the literature, as well as the results of an ongoing study of the related financial performance of AMCs nationwide, produce several findings. These findings include the following: (1) AMCs tend to be more expensive than other hospitals, although the extent to which this higher cost is attributable to GME is unclear; (2) implicit subsidies for these higher costs through patient care revenues are becoming less available; (3) relatively few short-term adverse effects on AMCs from the growth of competitive systems can be found, but AMCs are concerned that long-term effects will arise; (4) to the extent that managed care penetrates more heavily into the Medicare market, impacts on AMCs are likely to be intensified; and (5) AMCs are attempting to reduce costs and develop affiliations useful to a managed care environment, even though they are concerned that doing so will detract from their unique missions. The report concludes that, while there is considerable support for pooled funding for GME among diverse parties, AMCs have not reached a consensus about its details. (Volume One final report 15 pages; Volume Two appendixes variously paginated.)

AGENCY SPONSOR: Office of Public Health and Science

FEDERAL CONTACT: Carol Roddy

PHONE NUMBER: 202/205-0152

PIC ID: 5722

PERFORMER ORGANIZATION: Mathematica Policy Research, Inc., Plainsboro, NJ

TITLE: HIV and the Blood Supply: An Analysis of Crisis Decisionmaking

ABSTRACT NUMBER: 142

ABSTRACT: FDA has regulatory authority over blood plasma collection establishments, blood banks, and all blood products. The Centers for Disease Control and Prevention is responsible for surveillance, detection, and warning of potential public health risks within the blood supply. NIH supports these efforts through fundamental research. These Federal entities, as well as the plasma fractionation industry, community blood banks, and other groups, have responsibilities to protect the supply of blood from communicable disease. This report examines the behavior of these entities in response to a threat to the blood supply that emerged in the early 1980's--HIV. The report relates the history of the disease and the bureaucratic response to it in four areas: (1) blood product treatment; (2) donor screening and referral; (3) regulation of the removal of contaminated products from the market; and (4) communication to physicians and patients. The report concludes that (1) the system did not deal well with blood safety issues such as hepatitis and was not prepared for the much greater threat of HIV; (2) strong leadership must counteract legal and competitive concerns that may inhibit effective agency action when a crisis is not well-defined; and (3) agencies need to formulate a systematic approach to advisory committees and should not rely on the entities they regulate for data analysis or modeling.

AGENCY SPONSOR: Office of Public Health and Science

FEDERAL CONTACT: Carol Roddy

PHONE NUMBER: 202/205-0152

PIC ID: 5896

PERFORMER ORGANIZATION: National Academy of Sciences, Institute of Medicine, Washington, DC

TITLE: Improving the Supply, Training, and Distribution of Primary Care Providers Under Federal Health Care Reform

ABSTRACT NUMBER: 143

ABSTRACT: This project analyzes issues related to proposals for improving the supply, training, and distribution of primary care providers as part of health care reform. Many reform proposals considered by Congress include provisions that would change the mechanism used in decisions to fund graduate medical education, thus changing the numbers and types of new physicians. Information provided in this report includes (1) a chart book on the supply, training, and distribution of physicians; (2) technical papers providing background on the need for increased numbers of generalist physicians; (3) the rationale for proposals to control the supply of new physicians; (4) proposals for an all-payer pool for GME; (5) estimates of the number of physicians in research training tracks funded by NIH and elsewhere; and (6) preliminary estimates of the number of advanced practice nurses and physician assistants that would be needed to substitute for physicians if the number of physician residents were reduced.

AGENCY SPONSOR: Office of Public Health and Science

FEDERAL CONTACT: Carol Roddy

PHONE NUMBER: 202/205-0152

PIC ID: 5666

PERFORMER ORGANIZATION: Center for Health Policy Research, Washington, DC

TITLE: Measuring Expenditures for Essential Public Health Services

ABSTRACT NUMBER: 144

ABSTRACT: The purpose of this project is to develop a tool for collecting data on Federal, State, and local public health expenditures that uses the terminology of the essential services of public health. The goal of this effort is to document the relative investment in public health in the United States, laying the groundwork for a system to track these expenditures over time and across agencies and levels of Government. Another purpose of this project is to provide the financial data to relate to public health outcomes, public health infrastructure, and work force measurement. Nine States participated in this effort: Arizona, Illinois, Iowa, Louisiana, New York, Oregon, Rhode Island, Texas, and Washington. On June 29, 1995, 75 State, local, and Federal officials met to discuss and plan the collection of data on public health expenditures. After a plenary session with presentations on several public health initiative projects, the participants from public health, mental health, substance abuse, and environmental agencies met in separate work groups to discuss the tools for estimating expenditures for the essential services of public health. Data collection instruments were distributed to State health officials in September 1995. State health officials coordinated the data collection for State and local public health, substance abuse, and environmental agencies. At the request of the mental health agencies, data collection was coordinated using 1993 data from the National Association of State Mental Health Program Directors.

AGENCY SPONSOR: Office of Disease Prevention and Health Promotion

FEDERAL CONTACT: Deborah R. Maiese

PHONE NUMBER: 202/401-5809

PIC ID: 6194

PERFORMER ORGANIZATION: Public Health Foundation, Washington, DC

TITLE: Minority HIV/AIDS Programs Community-Based Organization Evaluation: HIV/AIDS Education/Prevention Grant Program 1988-89, A Cross-Site Evaluation

ABSTRACT NUMBER: 145

ABSTRACT: This evaluation study uses a multiple case study approach to assess 9 of 38 projects awarded grants in 1988 and 1989. The study documents, describes, and assesses the effectiveness of community-based projects that address HIV and AIDS prevention within minority communities. The grant program was designed to support innovative approaches that minority programs use when targeting their communities and at-risk populations. This report assesses the effectiveness of these projects. The results and findings of this evaluation will be used by the Office of Minority Health and other agencies to identify successful practices and common themes that could be used as innovative community-based risk-reduction strategies. The study compares various community-based approaches to disease prevention and the impact of those approaches on minority communities. Results of the study show that successful prevention models include (1) funding agencies that give priority to community-based organizations (CBOs) with a history of meeting community needs; (2) CBOs with local representation at all levels of program decisionmaking; (3) funding agencies that provide site visits, responses to program reports, and ongoing communication with program managers; and (5) funding agencies that set aside resources for technical assistance. (Final report 121 pages, plus appendixes.)

AGENCY SPONSOR: Office of Minority Health

FEDERAL CONTACT: John Walker, III

PHONE NUMBER: 301/594-0769

PIC ID: 5618

PERFORMER ORGANIZATION: Tonya, Inc., Washington, DC

TITLE: Region IX Hepatitis B Project Final Report

ABSTRACT NUMBER: 146

ABSTRACT: Approximately 300,000 people are infected with the hepatitis B virus (HBV) each year, and about 1 million are infectious HBV carriers. Of these, over 50 percent are Asians or Pacific Islanders. Carrier rates for Asians range from 8 to 15 percent, and for Pacific Islanders, from 5 to 10 percent, while rates for the total U.S. population are 0.2 percent. This report assesses HBV education and outreach efforts among Region IX primary care centers serving Asians and Pacific Islanders. The report summarizes the programs at five centers. These centers completed and returned a self-assessment tool, the results of which show that staffing is the strongest component of HBV education and outreach programs. Programs with bilingual medical staff or medical interpreters, with materials written in patients' languages, and with a high-profile, integrated approach are generally more successful than others. The report finds that media-based outreach is the weakest aspect of the programs. Information gathered in two site visits is also presented. The report covers the following areas: clinic/client profiles; staffing; health education materials; clinic-based, media-based, and community-based outreach and education; and networking and linkages. The report also provides information on the barriers to effective HBV outreach and education, including cost, cultural health beliefs, and lack of personnel and awareness. It provides recommendations to improve HBV education and outreach for Asians and Pacific Islanders.

AGENCY SPONSOR: Public Health Service, Regional Office IX

FEDERAL CONTACT: George Nakama

PHONE NUMBER: 415/556-4926

PIC ID: 5681

PERFORMER ORGANIZATION: Association for Asian Pacific Community Health Organization, Oakland, CA

TITLE: Review of the Fialuridine (FIAU) Clinical Trials

ABSTRACT NUMBER: 147

ABSTRACT: In June 1993, 1 of 15 HBV outpatients taking part in an NIH clinical drug trial was hospitalized with liver failure. Shortly thereafter, six more patients also developed severe toxicity. Five patients eventually died, and two more were saved from death only by a liver transplant. All of these patients were taking the experimental drug fialuridine (FIAU). The Institute of Medicine (IOM) examined the clinical trials involving FIAU and its parent drug, fiacitabine, to determine whether any rules governing the trials process should be changed, and what burdens or costs these changes might place on future clinical trials. Their report presents the history of clinical trials of the drugs, beginning with trials that showed promise in the treatment of HBV and ending with the trial in which several people died from toxic effects of FIAU. It also carefully examines the clinical background for the trials, the informed consent process, and the conduct of the investigators. The IOM found that the entire set of trials reviewed was an ethically sound clinical research project designed and carried out by highly competent investigators who frequently exceeded requirements of regulations imposed by institutional review boards. Furthermore, the IOM report discusses conclusions and recommendations issued by the FDA. The IOM generally concurs with the recommendations without ascribing blame for the tragedy to any party.

AGENCY SPONSOR: Office of Public Health and Science

FEDERAL CONTACT: Carol Roddy

PHONE NUMBER: 202/205-0152

PIC ID: 5660

PERFORMER ORGANIZATION: National Academy of Sciences, Institute of Medicine, Washington, DC

Substance Abuse and Mental Health Services Administration

Contents

Evaluation of Mental Health Service Delivery Systems for Children and Adolescents

TITLE: Evaluation of Mental Health Service Delivery Systems for Children and Adolescents

ABSTRACT NUMBER: 148

ABSTRACT: This issue of the Journal of Emotional and Behavioral Disorders is dedicated to a special series on research projects supported by the Center for Mental Health Services (CMHS) within SAMHSA. The first article summarizes the recent history of research on the service delivery system for children and adolescents who have serious emotional disabilities, and the other articles examine the effectiveness of specific interventions for individual populations. Articles include "Building the Research Base for Children's Mental Health Services"; "Multisystemic Family Preservation Therapy: Preliminary Findings From a Study of Rural and Minority Serious Adolescent Offenders"; "Improving Adjustment Outcomes for Foster Children With Emotional and Behavioral Disorders: Early Findings From a Controlled Study on Individualized Services"; "Effectiveness of Intensive Case Management for Homeless Adolescents: Results of a 3-Month Followup"; "Development and Evaluation of Treatment Foster Care and Family-Centered Intensive Case Management in New York"; "Connecting Low-Income Families to Mental Health Services: The Role of the Family Associate"; "The Vanderbilt School-Based Counseling Program: An Interagency, Primary-Care Model of Mental Health Services"; and "The Challenges of Child Mental Health Research." (Journal articles 71 pages in total.)

AGENCY SPONSOR: Center for Mental Health Services

FEDERAL CONTACT: Diane Sondheimer

PHONE NUMBER: 301/443-1333

PIC ID: 6152

PERFORMER ORGANIZATION: Center for Mental Health Services, Rockville, MD

Appendix B. Inventory of HHS Evaluations in Progress

Listed below are projects that are in progress for the various HHS agencies and offices. Included is the project's expected date of completion and the Policy Information Center (PIC) abstract number. In the case of the Health Care Financing Administration, the agency's project identification numbers are given instead of PIC abstract numbers. Additional information about these projects is found in Health Care Financing: Active Projects, October 1, 1995. HCFA Pub. 03381. For additional information about an individual project in progress, contact the Policy Information Center at (202) 690-6445. The PIC database can also be accessed through the Department's World Wide Web server (http://www.hhs. gov) on the Internet.

Administration for Children and Families

Project Title	Expected Date of Completion	PIC Abstract Number
Social and Educational Development of Tribal-Based Communities for the Sonoran and Negev Deserts	12/95	5978
Women and Infant Nurturing Services (WINGS)	12/95	979
Transfer of International Innovations--Development of a Clinical Monitoring System To Support Foster Care in Michigan	2/96	5976
Descriptive Study of Head Start Bilingual/Multicultural Program Services	3/96	5845
Evaluation of Child Support Guidelines	3/96	5983
Evaluation of Nine Model Comprehensive Community-Based Child Abuse and Neglect Prevention Programs	3/96	5851
Responsible Fatherhood: Theoretical and Empirical Foundations for Policy and Program Development	5/96	5981
Evaluation of Child Access Demonstration Projects	6/96	972
National Study of Protective, Preventive, and Reunification Services Delivered to Children and Their Families	7/96	3874
Evaluation of the Comprehensive Child Development Programs	9/96	3868
Evaluation of Head Start Family Child Care Homes	9/96	975
National Study of Outcomes for Children Placed in Foster Care With Relatives	9/96	5846
Identification and Prevention of Intergenerational AFDC Dependency: Promoting Long-Term Child Welfare	10/96	4385
Home Visiting Services Demonstration	4/98	5980
Post-Employment Services Demonstration	4/98	5974
Evaluation of the Head Start/Public School Early Childhood Transition Demonstration	6/98	4393
Family Preservation and Family Support Services (FP/FS) Implementation Study	6/99	5975
New Hope Project	9/99	5982

Administration on Aging

Project Title	Expected Date of Completion	PIC Abstract Number
Elderly Nutrition Program	9/96	6175

Agency for Health Care Policy and Research

Project Title	Expected Date of Completion	PIC Abstract Number
Computerized Needs-Oriented Quality Measurement Evaluation System (CONQUEST)	2/96	5961
Oregon Scorecard Project	7/96	5960
Assessment of the Nature and Extent of the Diffusion and Impact of MEDTEP Research Activities and Findings	12/96	4311

Centers for Disease Control and Prevention

Project Title	Expected Date of Completion	PIC Abstract Number
Evaluation of the CDC Injury Grant Program	11/95	166
Assessing the Cost Effectiveness and Cost Benefit of CDC Funded Smoke Detector Programs	12/95	5351
Evaluation of CDC WONDER/PC Public Health Decisionmaking	3/96	5505
Georgia State Clinic Assessments of Immunization Coverage: Impact of Management and Clinic Immunization Practices	3/96	354
Case Study Evaluation of the Henry J. Kaiser Family Foundation's Community Health Promotion Grants Program	4/96	4109.1
Consent for Adolescent Immunizations: Issues and Current Practices	6/96	5933
Evaluation of the Dengue Hemorrhagic Fever (DHF) Prevention and Control Program at the San Juan Laboratories, Puerto Rico	6/96	355
Participation in the 1993-94 National Health Interview Survey on Disability	6/96	4532
Evaluation of ICD 10 for Morbidity Reporting Purposes in the United States	7/96	5503
Assessment of the Programmatic Impact of 1% Evaluation Studies, 1990- 95	9/96	5924
Determining the Effectiveness of the Prevention Centers Program	9/96	5919
Evaluation of the Effectiveness of CDC Surveillance for Drug-Resistant Pneumococcal Infections	9/96	5501
Evaluation of the Medical Certification Process for Death Certificates	9/96	4230
Evaluation of Tuberculosis Outreach Worker Activities	9/96	5502
Community Context Study of Minors' Access to Tobacco Products	10/96	5921
Assessment of the Effectiveness of the National Laboratory Training Network	12/96	5507
Assessment of the Public Health Impact of the NIOSH Health Hazard Evaluation Program	12/96	5922
Evaluation of Suicide Prevention Interventions in Four Native American Communities	12/96	5504
Sexually Transmitted Disease (STD) Prevention in the U.S.	12/96	5348
Assessment of Violence Prevention Technical Assistance Efforts for State and Local Health Departments	1/98	5929
Development of a Public Health Surveillance Plan to Evaluate CDC Activities in the Atlanta Empowerment Zone	12/99	5925
Evaluation of the Data for Decisionmaking Project in Bolivia	12/99	5926
Evaluation of the Field Epidemiology Training Program	12/99	5923
Evaluation of the Fiscal and Programmatic Impact of HIV Prevention Community Planning	12/99	5927
Evaluation of Racial and Ethnic Identification Data	12/99	5918
Identification of Early Warning Signs and Protective Factors in Intimate Violence Prevention	12/99	5931
Immunization Related Knowledge, Attitudes, and Practices of Primary Health Care Providers in the U.S.	12/99	5930
Implementing GPRA at CDC	12/99	5928
Steps to Successful Implementation of Youth Violence Prevention Programs	12/99	5920
Summarizing Educational Outcomes of School Health Interventions	12/99	5932

Food and Drug Administration

Project Title	Expected Date of Completion	PIC Abstract Number
Consumer Use of Food Labels	1/96	4988
FY 1995 Food Label and Package Survey	8/96	5711
Fourth Annual Evaluation of PDUFA	12/96	6079.1
Evaluation of Influenza Virus Vaccines in High Risk Patient Groups	9/98	4355

Health Care Financing Administration

Project Title	Expected Date of Completion	HCFA Project Number
Evaluation of the Home Health Prospective Payment Demonstration	11/95	90-065
Evaluation of Medicare SELECT	2/96	93-031
Evaluation of the Ventilator-Dependent Unit Demonstration	2/96	90-012
Evaluation of Cost Health Maintenance Organizations and Health Care Prepayment Plans	3/96	93-075
Evaluation of Medicaid Managed Care Programs with 1915(b) Waivers	5/96	93-073
Evaluation of the Program for All-Inclusive Care for the Elderly (PACE) Demonstration	6/96	91-017
Community-Supported Living Arrangements (CSLA) Program: Process Evaluation	8/96	93-077
Monitoring and Evaluation of the Medicare Cataract Surgery Alternate Payment Demonstration	8/96	94- 130
Evaluation and Technical Assistance of the Medicare Alzheimer's Disease Demonstration	9/96	95-073
Evaluation of the Arizona Health Care Cost Containment System	9/96	89-033
Evaluation of the Impact of Health Plan Report Cards on Consumer Knowledge, Attitudes, and Choice in a Managed Competition Setting	9/96	95-001
Evaluation of the Medicaid Uninsured Demonstrations	9/96	92-064
Implementation and Evaluation of APGs as an Outpatient Measurement and Financing Methodology in Maine	9/96	94-112
Medicaid Quality of Life Study	11/96	88-015
Evaluation of the Medicare Case Management Demonstrations	12/96	93-056
Evaluation of Rural Health Clinics	2/97	95-058
Evaluation of the Community Nursing Organization Demonstration	2/97	92-068
Evaluation of the Essential Access Community Hospital/Rural Primary Care Hospital Program	3/97	91- 078
Evaluation of the Utah Prepaid Mental Health Plan: Coordinated Care System as Alternative to Traditional Fee for Service	3/97	2-037
Medicare Participating Heart Bypass Center Demonstration Extended Evaluation	3/97	94-010
Evaluation of the Iowa Implementation of Ambulatory Patient Groups (APGs)	4/97	95-060
Evaluation of the Effectiveness of the Operation Restore Trust Demonstrations	7/97	95-003
Evaluation of the Demonstration Entitled Delaware Health Care Partnership for Children	9/97	95- 028
Evaluation of the Demonstration for Improving Access to Care for Pregnant Substance Abusers	9/97	92-069
Project Demonstrating and Evaluating Alternative Methods to Assure and Enhance the Quality of Longterm Care Services for Persons with Developmental Disabilities through Performance-Based Contracts with Service Providers	9/97	94-096
Drug Utilization Review Evaluation Contract	2/98	3-033
Evaluation of the Medicare Choice Demonstration	6/98	5-018
Evaluation of Health Maintenance Organization (HMO) Outlier Demonstration--Maximizing the Effective Use of Telemedicine: A Study of the Effects, Cost-Effectiveness, and Utilization Patterns of Consultation via Telemedicine	9/98	95-006
Evaluation of the Demonstration Entitled Delaware Health Care Partnership for Children	9/98	95- 028
Maximizing the Effective Use of Telemedicine: A Study of the Effects, Cost Effectiveness, and Utilization Patterns of Consultation via Telemedicine	9/98	95-023
Evaluation of the Nursing Home Case-Mix and Quality Demonstration	9/99	94-081
Evaluation of the Oregon Medicaid Demonstration	9/99	4-127
Evaluation of the State Medicaid Reform Demonstrations for Hawaii, Oklahoma, Rhode Island, Tennessee, Vermont	9/99	94-126
Evaluation of the State Medicaid Reform Demonstrations for Ohio, Minnesota and Three Additional States	9/2000	95-052

Health Resources and Services Administration

Project Title	Expected Date of Completion	PIC Abstract Number
Technical Support for Need Designation and State Primary Care Access Plan Initiatives	1/96	6249
Evaluation of the Health Care for the Homeless Program	2/96	5595
Impact of Section 1115 Medicaid Waivers on Federally Qualified Health Centers	2/96	5738
Changes in Provision of Services for Active/Recovering Drug Users With HIV in Ryan White CARE Act Title I Cities	3/96	6250
Development of a Study Design for Evaluating Variations in OPO Organ Procurement Rates	3/96	6251
Evaluation of Learning Needs Using Computerized Standard Educational Needs Assessment Protocol for Education Training Centers	3/96	5604
The Future of Primary Care	3/96	5612
Lasting Impact of MCHB's Emergency Medical Services for Children (EMSC) Funding	3/96	5748
Quality Assurance Procedures for the Uniform Reporting System for Titles I and II of the Ryan White CARE Act	3/96	5602
Training and Educational Needs of the Public Health Work Force Under Health Care Reform	3/96	5747
A Test of Local-Level Health Care Reform Strategies in Selected California Counties	5/96	6252
Enhancing State Health Department Policy Development Capacity Through Technical Assistance	5/96	6253
Reasons African-American and White Waiting List Patients Are Unavailable for an Organ Offer	5/96	5601
Rural Applications of Telemedicine	5/96	5749
Person-Place Approximations and Utilization vs. Health Status Analysis of the National Health Interview Survey	6/96	5614
Technical Support for Closing Gaps in Low-Income and Minority Populations	7/96	6254
Collaborative Models That Work: Examining the Relationship Between HIV/AIDS and TB Programs in Selected Title I Eligible Metropolitan Areas	8/96	6255
Synthesis of Local Evaluations Sponsored by Ryan White Grantees	8/96	6256
Analysis of HRSA Participation in the HBCU Initiative: FY 1992-FY 1995	9/96	6257
Community and Migrant Health Centers' Immunization Efforts--A Joint Study with the Centers for Disease Control	9/96	4921
Development of a Self-Assessment Tool for HIV Health Services Planning Councils and HIV Care Consortia	9/96	5735
Evaluation of Activities To Reduce Perinatal HIV Transmission: Case Studies and Feasibility Assessment	9/96	6258
HMO Staffing Strategies in Underserved Areas	9/96	259
Institutional Factors Which Influence the Likelihood of Living-Related Kidney Transplant Operations--Part II	9/96	4917.1
Ability of Community-Based Health Organizations To Integrate Services for Individuals With Multiple Risk Factors	10/96	6260
Bilingual Assistance Program: Evaluation of Strategies for Reducing Cultural and Linguistic Barriers to Health Care for Hispanic and Asian/Pacific Islander Populations	10/96	6261
Community Health Center User and Visit Survey	10/96	737
Impact of Section 1115 Medicaid Waivers on Federally Qualified Health Centers and Underserved Populations	11/96	5738 (I)
Sixth National Sample of Registered Nursing	1/97	262
Evaluation of the Healthy Schools, Healthy Communities Program	3/97	5739
Measuring the Costs of Medical Education in Ambulatory Training Sites	3/97	6263
Tracking of J-1 Visa Exchange Students	3/97	6264
Assessment of Bilingual/Multilingual Services Offered at Selected Community and Migrant Health Centers	5/97	6265
Evaluation of Effectiveness and Impact of Community and Migrant Health Centers: Implementation Phase	9/97	4918.1
Effectiveness of the National Health Service Corps	9/98	6266
National Evaluation of the Healthy Start Program	9/98	610

Indian Health Service

Project Title	Expected Date of Completion	PIC Abstract Number
Mental Health Service Delivery Model for Urban Native Americans	9/99	5943
Analyzing the Underreporting of American Indian and Alaska Native Decreased Persons on State Death Certificates, 1986-1988	10/96	946
Development of a Health Services Research Agenda for American Indian and Alaska Native Populations	12/96	6053
Evaluation of the Indian Health Service Adolescent Regional Treatment Centers	12/96	5941
Development and Implementation of a Strategic Planning Evaluation Model: Phase I and Phase II	5/97	5548
Evaluation of IHS-Supported Substance Abuse Treatment Program for American Indian/Alaska Native Women	7/98	TBE

National Institutes of Health

Project Title	Expected Date of Completion	PIC Abstract Number
Methodology To Assess the Impact of NHLBI Research	9/96	5577
Implementation Phase for an Evaluation of the Minority Access to Research Careers (MARC) Honors Undergraduate Research Training Program	10/96	5574
Career Outcomes of a Matched Sample of Men and Women Ph.D.s in Science and Engineering	12/96	5588
Electronic Student Tracking and Reporting System: The Bridges Program	12/96	6093
Evaluation of Knowledge-Based Expert System for Subject Indexing	12/96	4663.1
Evaluation of the Regional Primate Research Centers Program Phase I Planning and Methodology Development for a Full-Scale Evaluation	12/96	6045
Phase II: Survey To Assess the Impact of the NICHD Antenatal Steriods Concensus Development Conference	12/96	6120
Planning Phase for an Evaluation of the NIDR Center Program	12/96	5578
Design and Pilot Test of an Inventory of Cancer Research Across Federal Agencies	3/97	6091
Evaluation of the National Institute of Mental Health (NIMH) National Research Service Awards for Predoctoral Individual M.D./Ph. D. Fellows, Phase I	6/97	6096
Followup Study of Young Investigators in the Biological and Biomedical Sciences	6/97	6098
Evaluation of the National Research Service Award Research Training Program	7/97	5571
Evaluation of the NHLBI Programs of Excellence in Molecular Biology	9/97	5584
Improving the Non Response in the Panel Study of Income Dynamics (PSID) to Enhance Inter and Intra Generational Analyses	9/97	555
Survey of Academic Research Equipment	9/97	6121
Survey of Biomedical Research Facilities	9/97	562.2
Vaccine Development: Establishing Priorities for the United States for the 21st Century	9/97	5984
NIH AIDS Research Program Evaluation	10/97	6122
Characterization of User Population and User Satisfaction of the PDQ Database	12/97	6092
Comprehensive School Health Programs in Grades K-12	12/97	5586
Development of a Database for NIH-Supported Rare Disease Research	12/97	6043
Evaluation of New Technology: Audio Casi	12/97	5569
Evaluation of the Institutional Award (IDeA) Program, Phase I: Evaluation Design	12/97	6046
Evaluation of the NCRR Research Centers in Minority Institutions Program Phase I: Planning and Methodology Development for a Full-Scale Evaluation	12/97	5580
Evaluation of the Resource Centers Activities With the Biomedical Research Technology Program of the NCRR, Phase I; Planning and Methodology Development	12/97	6047
Strategic Plan for Public Health Agencies	12/97	6097
Evaluation of Laboratory Animal Use, Facilities, and Resources, Phase II	1/98	6048
Evaluation of NIH Extramural Shared Instrumentation Activities, Stage II: Survey Execution	12/98	5083
Evaluation of NIH Implementation of Section 491 of the PHS Act Mandating a Program of Protection of Research Subjects	12/98	460
Full-Scale Evaluation of the Regional Primate Research Centers Program	12/98	6045.1
Task Force on Genetic Testing	12/98	6090

Office of the Assistant Secretary for Planning and Evaluation

Project Title	Expected Date of Completion	PIC Abstract Number
Subacute Care--Market Analysis, Cost, and Quality	11/95	6140
Analysis of the Effect of Regulation on the Quality of Care in Board and Care Homes	12/95	4720
Cost of Domestic Violence to the Health Care System: A Methodology	4/96	4528
Analysis of AFDC Caseload Growth and Decline	5/96	953
Innovative and Interdisciplinary Education and Training Programs for Professionals Care for Persons With Disabilities	5/96	052
Registry of Projects To Integrate Health Information Systems at the State Level	5/96	6178
Domestic Violence Policy and Programs in Selected Communities	7/96	6154
Information To Guide Physician Practice	7/96	4499
Research and Demonstration Strategy to Test Models for Integrating Acute and Long-Term Care for Children With Disabilities	7/96	760
Study of Welfare Waiver Evaluations	7/96	6050
Health Care in Transition: Technology Assessment in the Private Sector	8/96	6180
Home Visitor Services Demonstration: Home Visiting for Teen Parents Required To Participate in JOBS	9/96	6160
Research and Evaluation Framework for the New Pharmaceutical Marketplace	9/96	6049
Teenage Parent Demonstration and Evaluation	9/96	771.9
Alcohol and Drug Treatment Outcomes for Welfare and Child Welfare Clients in California	10/96	4511
Characteristics of "Informal" Kinship Care	10/96	6016
Child Welfare and Domestic Violence: An Exploratory Study	10/96	6155
Public Health Performance Monitoring and Evaluation	12/96	6177
Strategies for Obtaining Public Health Infrastructure Data at Federal, State, and Local Levels	3/97	6179
Study of Medicare Home Health Practice Variations	9/97	888
Analysis of Data on Use of Nursing Homes in Florida	10/97	5759
Analysis of Health and Mental Health of Immigrant Children and Their Families	10/97	6150
Health and Long-Term Care Expenditure Patterns of Children with Disabilities	10/97	5758
Panel on Performance Measures and Data to Support Public Health Performance Partnership Grants	12/97	6176
State Development and Use of Insurance Coverage Surveys	9/98	6017
National Evaluation of Family Preservation Services	1/99	5337.1
Evaluability Assessment for Responsible Fatherhood Projects	9/99	6159

Office of Public Health and Science

Project Title	Expected Date of Completion	PIC Abstract Number
Evaluation of Clients of Adolescent Family Life Demonstration Projects	11/96	6243
Regional Meetings to Identify Desired Results for Performance Measurement in Selected Public Health Programs	12/96	6200
Evaluation of Health Needs of Newly Arriving Refugees	1/96	5622
Support of Efforts to Improve the Methodology for Evaluating the Cost- Effectiveness of Clinical Preventive Services	2/96	619
Development of Planning and Evaluation Criteria for the Functions and Components of a "Put Prevention into Practice" Online Interactive Application	4/96	5661
Formative Evaluation of an Automated Telecommunication System To Support Care Managers Providing Health and Human Services to High-Risk Families	4/96	5683
Survey of Accused but Exonerated Individuals in Research Misconduct Cases	6/96	5621
Measuring Expenditures for Essential Public Health Services	9/96	6194
Evaluation of the Office of Minority Health Resource Center, Phase I: Design Development	4/97	6244
Evaluation Synthesis of School Health Programs	5/97	198
Evaluation of Models To Integrate HIV Prevention and Treatment Services Within PHS-Supported Programs	9/97	6245
Evaluation of the Historically Black Colleges and Universities Capacity Building Program	10/97	6246
Evaluation of the Effectiveness and Efficacy of the OMH Bilingual/Bicultural Service Demonstration Grant Program	4/98	6247
Commission of Dietary Supplement Labels	9/97	6193

Substance Abuse and Mental Health Services Administration

Project Title	Expected Date of Completion	PIC Abstract Number
Evaluation and Technical Assistance for the Health Professions Clinical Training Program in Alcohol and Other Drug Abuse (Faculty Development Program)	9/96	4510
Evaluation of Campus Treatment Demonstration Programs	9/96	4518
Evaluation of CSAP's Training System	9/96	4514.1
Evaluation of Demonstration Treatment Programs in Job Corps	9/96	4523
National Treatment Improvement Evaluation Study (NTIES)	9/96	5346
Evaluation of the Pregnant and Post-Partum Women and Their Infants Demonstration Program	9/97	5674
Outcome Evaluation for the Community Partnership Program	12/97	4997
DC Courts Initiative Evaluation	6/98	4998
National Evaluation Data and Technical Assistance Center (NEDTAC)	9/98	5994
Evaluation of High-Risk Youth Substance Abuse Prevention Initiatives Funded in 1994	4/99	6044
Cross-Site Evaluation of the Community Prevention Coalitions Demonstration Grant Program	5/99	6042
Evaluation of the HHS Access to Community Care and Effective Services and Supports (ACCESS) Program	12/99	4980.1

Appendix C. Senior Editorial Advisors' Review Criteria for Assessing Program Evaluations

The study addresses a significant issue of policy relevance; evaluation findings are likely to be useful.

Conceptual Criteria

Conceptual Foundations

A literature review is included; the project is shown to be logically based on previous findings; the report uses either theory or models or both; the program assumptions are stated; the evaluation draws from previous evaluations (if any); the report is linked with a program and describes the program; the report presents multiple perspectives; multiple relevant stakeholders are consulted and involved; the timing is appropriate because the program is ready for evaluation.

Questions for Evaluation

The aims of the evaluation are clear, well-specified, and testable; the questions are feasible, significant, linked to the program, appropriate with respect to resources and audience, and derive logically from the conceptual foundations. The questions show ingenuity and creativity.

Findings and Interpretation

The conclusions are justified by the analyses; the summary does not go beyond what the data will support; the appropriate qualifiers are stated; the conclusions fit the entire analysis; equivocal findings are handled appropriately; the initial questions are answered; the interpretation ties in with the conceptual foundation; the report notes that the findings are either consistent with, or deviate from, the relevant literature; the presentation is understandable; the results have practical significance; the extent of program implementation is assessed.

Recommendations

The recommendations follow from findings and are worth carrying out and are affordable, timely, feasible, useful, and appropriate; the recommendations are shown to be relevant to the questions asked; the breadth or specificity of the recommendations is addressed. Any recommendations for either future evaluations or improvements or both are clearly presented.

Methods

Evaluation Design

Design considerations include overall appropriateness; soundness; feasibility; funding and time constraints; generalizability; applicability for cultural diversity; assessment of the extent of program delivery; validity; feasibility for data collection; reliability of selected measurements; use of multiple measures of key concepts; and appropriateness of the sample. In addition, variables are clearly specified and fit with the questions and concepts; the design permits measurement of the extent of implementation of the program and answering of the evaluation questions.

Data Collection

Data are collected using appropriate units of measurement for analysis, controls for participant selection and assignment bias, and proper handling of missing data and attrition. Other considerations include use of an appropriate comparison group or control; adequate sample size, response rate, and information about the sample; a data collection plan; data collection that is faithful to the plan; attention to and cooperation with the relevant community; project confidentiality; and consistency in data collection. The quality of the data (including the quality of any extant data sets used in the study) and the efficiency of sampling are addressed. The data collection is appropriate to evaluation questions.

Data Analysis

Among the factors that the data analysis addresses are the handling of attrition; the matching of the analysis to the design; the use of appropriate statistical controls; the use of methodology and levels of measurement appropriate to the type of data; and estimation of effect size. The analysis shows sensitivity to cultural categories; appropriate generalizability of inferences; and choice of an analysis type that is simple and efficient.

Cross-Cutting Factors

The following are crosscutting factors that are likely to be important at all stages of a report: clarity, presentation, operation of a state-of-the-art level, appropriateness, understandability, innovativeness, generalizability, efficiency of approach, logical relationships, and discussion of the report's limitations. The report should also address ethical issues, possible perceptual bias, cultural diversity, and any gaps in study execution.

Appendix D. Glossary of Acronyms

ACCESS	Access to Community Care and Effective Services and Supports Program
ACF	Administration for Children and Families
ACYF	Administration on Children, Youth and Families
AHCPR	Agency for Health Care Policy and Research
AMC	academic medical center
AoA	Administration on Aging
AOD	alcohol and other drugs
ASPE	Assistant Secretary for Planning and Evaluation
ASSIST	Automated System for Survey Information and Statistical Tools
ATSDR	Agency for Toxic Substances and Disease Registry
BDMP	Birth Defects Monitoring Program
BHP	benign prostatic hyperplasia
BPHC	Bureau of Primary Health Care
CABG	coronary artery bypass graft
CA/CN	child abuse/child neglect
CAHPS	Consumer Assessments of Health Plans Study
CBOs	community-based organizations
CCIs	comprehensive community initiatives
CDC	Centers for Disease Control and Prevention
CHC	Community Health Center
CIO	center, institute, office
CLPP	childhood lead poisoning prevention
C/MHCs	Community and Migrant Health Centers
CMHS	Center for Mental Health Services
CSAP	Center for Substance Abuse Prevention
DASH	Division of Adolescent and School Health
DCDC	Detailed Case Data Component
DPP	Demonstration Partnership Program
DRGs	Diagnosis-Related Groups
EACH	Essential Access Community Hospital
EBLLs	elevated blood lead levels
ENP	Elderly Nutrition Program
EPOC	Evaluation Policy Oversight Committee
EPSDT	early and periodic screening, diagnosis, and treatment
FAS	fetal alcohol syndrome
FDA	Food and Drug Administration
FETP	Field Epidemiology Training Program
FIAC	fiacitibine
FIAU	fialuridine
FP/FS	Family Preservation and Family Support Services
FQHCs	Federally Qualified Health Centers
FY	fiscal year
GAO	General Accounting Office
GBS	Guillain-Barre syndrome
GME	graduate medical education
GPRA	Government Performance and Results Act
HBCUs	historically black colleges and universities
HBV	hepatitis B virus
HCD	human capital development
HCFA	Health Care Financing Administration
HHE	Health Hazard Evaluation program
HHS	Department of Health and Human Services
HMOs	health maintenance organizations
HRSA	Health Resources and Services Administration
ICs	institutes and centers
IHS	Indian Health Service
IMRs	infant mortality rates
IoM	Institute of Medicine
IV-D	child support enforcement programs
JOBS	Job Opportunities and Basic Skills Training
K awards	NIH Career Development awards
KSI	Key States Initiative
LFA	labor force attachment
LIHEAP	Low-Income Home Energy Assistance Program
LPPB	Lead Poisoning Prevention Branch (of CDC)
LPPP	Lead Poisoning Prevention Program
MARC	Minority Access to Research Careers
MCHB	Maternal and Child Health Bureau
MCO	managed care organization
MDD	Medications Development Division
ME/C	medical examiner or coroner
MECISP	Medical Examiner/Coroner Information-Sharing Program
MEDPAR	Medicare Provider Analysis and Review
MEDTEP	Medical Treatment Effectiveness Program
MEPS	Medical Expenditure Panel Surveys
MFP	multisystem family preservation
MFS	Medicare fee schedule
MLHP	midlevel health providers
MQSA	Mammography Quality Standards Act of 1992
NPDB	National Practitioner Data Bank
NCANDS	National Child Abuse and Neglect Data System
NCCDPHP	National Center for Chronic Disease Prevention and Health Promotion
NCHS	National Center for Health Statistics
NCP	National Cancer Program
NCRR	National Center for Research Resources
NDAs	New Drug Applications
NDI	National Death Index
NHAMCS	National Hospital Ambulatory Medical Care Survey
NHANES	National Health and Nutrition Examination Survey
NHIS	National Health Interview Survey
NHLBI	National Heart, Lung, and Blood Institute
NIAAA	National Institute on Alcohol Abuse and Alcoholism
NICHD	National Institute of Child Health and Human Development
NIDA	National Institute on Drug Abuse
NIDDK	National Institute of Diabetes and Digestive and Kidney Diseases
NIDR	National Institute of Dental Research
NIH	National Institutes of Health
NIMH	National Institute of Mental Health
NINR	National Institute of Nursing Research
NIP	National Immunization Program
NLM	National Library of Medicine
NLTN	National Laboratory Training Network
NNIS	National Nosocomial Infections Surveillance program
NRSA	National Research Service Award
NSF	National Science Foundation
NTIES	National Treatment Improvement Evaluation Study
NTIS	National Technical Information Service
OAA	Older Americans Act
OASH	Office of the Assistant Secretary for Health
OASPE	Office of the Assistant Secretary of Planning and Evaluation
OCS	Office of Community Services
OCSE	Office of Child Support Enforcement
ODPHP	Office of Disease Prevention and Health Promotion
OEI	Office of Evaluations and Inspections
OIG	Office of the Inspector General
OMH	Office of Minority Health
OPA	Office of Public Affairs
OPEL	Office of Planning, Evaluation, and Legislation
OPHS	Office of Public Health and Science
OPO	Organ Procurement Organization
OPPE	Office of Program Planning and Evaluation
ORD	Office of Research and Demonstrations
ORHP	Office of Rural Health Policy
ORI	Office of Research Integrity
OWH	Office of Women's Health
PDQ	Physician Data Query
PDUFA	Prescription Drug User Fee Act
PESD	Post-Employment Services Demonstration
PFS	Parents' Fair Share
PIC	Policy Information Center
PP	permanency planning
PTCA	percutaneous transluminal coronary angioplasty
RHCT	Rural Health Care Transition
RPCH	Rural Primary Care Hospital
RWCA	Ryan White Care Act
SAMHSA	Substance Abuse and Mental Health Services Administration
SDC	Summary Data Component
SSI	Supplemental Security Income
TMRC	Technical Merit Review Committee
TRIM2	Transfer Income Model-Number 2
UDS	Uniform Data System
URS	Uniform Reporting System
VA	Department of Veterans Affairs
WINGS	Women and Infant Nurturing Services

Topics

Program Evaluation | Modeling & Simulation | Key Indicators

Performance Improvement 1996

Foreword

Senior Editorial Advisors

Acknowledgments

Executive Summary

Chapter I. The Evaluation Program in the U.S. Department of Health and Human Services

Chapter II. Highlights of Selected Evaluations Completed During Fiscal 1995

Administration for Children and Families

Strengthening Families and Neighborhoods: A Community-Centered Approach

Youth With Runaway, Throwaway, and Homeless Experiences: Prevalence, Drug Use, and Other At-Risk Behaviors

Administration on Aging

Real People, Real Problems: An Evaluation of the Long-Term Care Ombudsman Programs of the Older Americans Act

Agency for Health Care Policy and Research

Understanding and Choosing Clinical Performance Measures for Quality Improvement: Development of a Typology

Centers for Disease Control and Prevention

The Effect of Mandated Managed Care for Medicaid Populations on the Practice of Public Health: The Example of Childhood Lead Poisoning Prevention

Health Care Financing Administration

Medicare Participating Heart Bypass Center Demonstration: Evaluation Design

Monitoring the Impact of Medicare Physician Payment Reform on Utilization and Access

Health Resources and Services Administration

Performance Indicators for GPRA: Initial Assessment of Health Resources and Services Administration Programs

Indian Health Service

Evaluation of Diabetes Services Provided by IHS Model Diabetes Program: Final Report

National Institutes of Health

Fetal Alcohol Syndrome: Diagnosis, Epidemiology, Prevention, and Treatment

Office of Public Health and Science

Consequences of Whistleblowing for the Whistleblower in Misconduct of Science

Chapter III. Agency Evaluation Activities

Administration for Children and Families

Administration on Aging

Agency for Health Care Policy and Research

Centers for Disease Control and Prevention and Agency for Toxic Substances and Disease Registry

Food and Drug Administration

Health Care Financing Administration

Health Resources and Services Administration

Indian Health Service

National Institutes of Health

Office of the Assistant Secretary for Planning and Evaluation

Office of Public Health and Science

Substance Abuse and Mental Health Services Administration

Appendix A. Abstracts of HHS Evaluations Completed in Fiscal 1995

Administration for Children and Families

Administration on Aging

Agency for Health Care Policy and Research

Centers for Disease Control and Prevention

Food and Drug Administration

Health Care Financing Administration

Health Resources and Services Administration

Indian Health Service

National Institutes of Health

Office of the Assistant Secretary for Planning and Evaluation

Office of Public Health and Science

Substance Abuse and Mental Health Services Administration

Appendix B. Inventory of HHS Evaluations in Progress

Administration for Children and Families

Administration on Aging

Agency for Health Care Policy and Research

Centers for Disease Control and Prevention

Food and Drug Administration

Health Care Financing Administration

Health Resources and Services Administration

Indian Health Service

National Institutes of Health

Office of the Assistant Secretary for Planning and Evaluation

Office of Public Health and Science

Substance Abuse and Mental Health Services Administration

Appendix C. Senior Editorial Advisors' Review Criteria for Assessing Program Evaluations

Appendix D. Glossary of Acronyms

Connect with Us