IN THIS ISSUE:
- Evaluation of the Research Centers in Minority Institutions Program: Final Report 2000
- Evaluation of Head Start Family Child Care Demonstration: Final Report
- Evaluation of the National Electronic Telecommunication System for Surveillance and the Public Health Laboratory Information System
- Report to the President: Prescription Drug Coverage, Spending, Utilization, and Prices
- HIV Prevention Indicators
The Research Centers in Minority Institutions (RCMI) Program was initiated by the National Institutes of Health (NIH) and mandated by the US Congress in 1985 to enable predominantly minority institutions to become more competitive in obtaining support for the conduct of biomedical and/or behavioral research relevant to the mission of the U.S. Public Health Service.
The RCMI Program assists health professional schools and graduate institutions with more than 50 percent minority students to strengthen and expand their human and physical resources to conduct research. The program provides for faculty development, enrichment, and expansion; renovation of laboratories and animal facilities; acquisition of state-of-the art instrumentation; enhancement of grants management and research development activities; improvement of biostatistical and computer resources; development of new technologies; initiation and expansion of research disciplines; and other institutional infrastructure activities. It should be added that RCMI funds are not directed toward student training.
In 1995 NCRR began a two-phase evaluation of the RCMI Program to assess the extent to which the goals and objectives of the Program had been achieved during its first 10 years. Phase I consisted of an 8-month feasibility study. In 1998 a contract to conduct the Phase II evaluation began and incorporated the research plan and methodology developed in Phase I. RCMI grants have been awarded on a competitive basis by the National Center for Research Resources (NCRR) to 21 academic institutions. The Phase II study focused on the 15 institutions that had received RCMI funding for at least 10 years.
The study was designed to accomplish the following goals:
- produce a scientifically sound evaluation of the 15 RCMI centers that have received funding for at least 10 years
- foster a better understanding of the relationship between long-term programmatic success and specific program characteristics and activities
- identify “best practices” implemented by different RCMI centers; and
- develop an improved process for selecting and monitoring the progress of RCMI centers.
The results of the evaluation revealed that as a group, the 15 RCMI centers had demonstrated substantial progress toward the expected successful outcome of increased competitiveness for research grants and increased peer-reviewed publications after 10 years of RCMI Program funding. The expert advisory panel for the study noted that the challenges faced by the centers were substantial and 10 years is a relatively short length of time to build research capacity and change organizational culture.
The report recommends that NCRR assist the RCMI centers by playing a more active role in helping the centers develop their capabilities particularly in the areas of strategic planning skills, and management and communications systems, and by encouraging the development of strong scientific and administrative leadership.
This study was conducted by the Quantum Research Corporation, a Division of Macro International, Inc. for the National Center for Research Resources (NCRR), National Institutes of Health. The project officer, Patricia Vorndran, can be reached at (301) 435-0866. Copies of the final report, PIC ID #5580.1, are available from the PIC.
The Administration on Children, Youth, and Families (ACYF) awarded family child care demonstration grants to 18 Head Start agencies to establish and implement family child care home programs and provide Head Start services. Two co-horts of 80 four-year-old children (1993-94 and 1994-95) were randomly assigned to either family child care homes or center classrooms. The evaluation team measured program quality and adherence to Head Start Performance Standards in the two settings and conducted assessments of the children’s cognitive, social-emotional, and physical development at the beginning and end of the Head Start year and in the Spring of the kindergarten year. Parent perspectives on the program and overall satisfaction with the program were also sought. Eligibility was limited to parents who were either working, attending school, or in a job training program.
The findings indicate that there were no significant differences in observed quality in family child care homes and center classrooms. Based on observations during on-site assessments, there were also no significant differences in caregiver and child interactions. Children in the family child care homes performed as well as children in center classrooms on assessments of cognitive performance, social-emotional and physical outcomes. Children in both settings performed equally well in kindergarten on measures of cognitive, social-emotional, and physical outcomes. Parents were also pleased with both settings (90% of the parents reported being very happy), although there had been initial concern by parents that the family child care home setting would not provide the same quality of education.
The report concludes that family child care homes are viable settings for the delivery of comprehensive Head Start services. The integrity of the Head Start program is well maintained in both the family child care home and the center classroom settings, as evidenced by the lack of setting difference for child outcome measures both at the post-test evaluation and at the kindergarten follow-up.
Family child care can be a particularly good Head Start services option for outlying rural areas, areas with a shortage of classroom facilities, children whose needs are better met in a small-group setting, and families who need full-day care for their children. The report also offers the following recommendations, among others, for those agencies considering offering a Head Start family child care program:
- Quality matters, regardless of the preschool setting.
- Assess the need for family child care in the community so as not to overestimate or underestimate the demand.
- Provide opportunities and release time for family child care providers to receive in-service training, complete paperwork, and conduct home visits.
- Allocate adequate funds for materials and equipment and circulate to the homes, possibly through a lending library.
- Ensure that family child care providers are well supervised and supported, receive technical assistance, and are encouraged to coordinate activities with center classroom teachers in their agency.
This study was prepared for ACYF by RMC Research Corporation. The project officer, Louisa Tarullo, can be reached at (202) 205-8324. Copies of the final report, PIC ID # 4975, can be obtained on the web at: http://www2.acf.dhhs.gov/programs/CORE/dox/fcc_eval.htm
Evaluation of the National Electronic Telecommunication System for Surveillance and the Public Health Laboratory Information System
The National Telecommunications System for Surveillance (NETSS) is a computerized public health surveillance system that assembles, analyzes and disseminates weekly data on about 50 nationally notifiable diseases. Providers, clinical laboratories, and public health officials collect NETSS data at the state and local level.
This evaluation study was undertaken to provide the Centers for Disease Control and Prevention (CDC) with a better understanding of the utilization of data collected by state and territorial health agencies and transmitted to CDC’s Epidemiology Program Office (EPO) over the NETSS. Case reports are transmitted over NETSS for most nationally notifiable diseases. All case reports contain a core component with event date, county of residence, and age, sex, race and ethnicity of the patient. The records for some diseases also contain an enhanced component with additional disease-specific data needed to support disease control activities.
This project’s aim was to discover (1) how communicable disease data transmitted to CDC over NETSS are used by state and local public health officials; (2) barriers to use of the data at CDC and at the state and local levels; and (3) ways in which CDC can improve the utilization of NETSS data.
The methods used to collect the data included a survey of state epidemiologists; a survey of 29 CDC staff members who were identified by CDC’s Epidemiology Program Office (EPO) as being significantly involved in working with NETSS data; six focus groups with staff from the offices of state and territorial epidemiologists in18 states and territories; and four focus groups with vaccine-preventable disease staff at the state and local level from four states.
The findings reveal that the states make use of data collected under NETSS to track trends in communicable diseases, to detect epidemiological patterns emerging across several states, and to compare observed levels of disease to expected levels. One of the most important uses for the data among state epidemiologists is information dissemination through bulletins, reports and newsletters.
Frequently cited barriers to utilization of NETSS data in states include limited staff time to analyze the data, limited staff capabilities in data analysis and computer systems and technical barriers. A major technical problem is that NETSS is a DOS-based system designed for centralized data entry in a world that has increasingly moved to networked applications in a Windows environment.
Regarding access and availability of NETSS data, most state epidemiologists and the focus groups would like to have data from other states available for analysis and most were willing to release their own data for use by other states if personal identifiers were removed. Both those willing to share and those who were hesitant to do so expressed caveats about data quality, confidentiality, and potential embarrassment of the state health agency if errors were made.
The report offers a number of recommendations to CDC for facilitating use of NETSS data, removing barriers to use of NETSS data, improving access and availability of the data and features of an integrated notifiable disease surveillance system.
Some of the recommendations are as follows:
- CDC should facilitate coordination of multiple systems at the state level.
- CDC should build toward the goal of training for staff in state offices in accessing and using NETSS data.
- CDC should technically update NETSS to support modern applications.
- CDC should make NETSS data accessible on a protected Intranet for analysis by states or consortia of states.
This study was prepared by Battelle. The project officers, Nancy Cheal and Nancy Chalmers, can be reached at (404) 639-7095 and (404) 639-7085, respectively. Copies of the final report, PIC ID #6332, can be obtained from the PIC.
Prescription drugs are critical in American life. These medications are improving health outcomes and quality of life, replacing surgery and other invasive treatments, and quickening recovery for patients who receive these treatments. However, millions of Americans – particularly elderly and disabled Medicare beneficiaries–have inadequate or no insurance coverage for drugs. Nearly a third of all Medicare beneficiaries have no financial protection for the costs of drugs, if they can obtain them at all.
President Clinton has proposed a new, voluntary Medicare drug benefit that would offer all beneficiaries access to affordable, high quality prescription drug coverage.
This study is designed to assist policymakers by providing comprehensive information on drug coverage, drug spending, and drug prices.
The key findings are:
- Individuals without drug coverage pay a higher price at the retail pharmacy than the total price paid on behalf of those with drug coverage.
- Cash customers (including those without coverage and those with indemnity coverage) pay more for a given drug than those with third party payments at the point of sale.
- In 1999, excluding the effect of rebates, the typical cash customer paid nearly 15 percent more than the customer with third party coverage. For a quarter of the most common drugs, the price difference between cash and third parties was even higher–over 20 percent.
- Medicare beneficiaries with drug coverage fill nearly one-third more prescriptions than those without coverage. Although drug spending for beneficiaries with coverage is nearly two-thirds higher, those without coverage pay nearly twice as much out of pocket. These differences in utilization and spending between beneficiaries with and without coverage generally hold up across different income levels, ages, health status and other categories.
Future research will examine the more complex interrelationships between demographic factors, insurance status, and prescription drug use.
This report represents the work of individuals and agencies through the Department of Health and Human Services, including the Agency for Healthcare Research and Quality (AHRQ), the Food and Drug Administration (FDA), the Health Care Financing Administration (HCFA), and the Office of the Assistant Secretary for Planning and Evaluation (ASPE).
The project officer, Jack Hoadley, can be reached at (202) 401-8401. The full report is available on-line at: http://aspe.hhs.gov/health/reports/drugstudy/
The Centers for Disease Control and Prevention (CDC) supports 65 jurisdictions for HIV prevention. Within each jurisdiction there is an array of interventions for each target population that are expected to result in changes in the determinants of HIV risk behavior and HIV transmission. The data for this report were gathered to assess the cumulative effects of HIV prevention efforts within a jurisdiction.
The intended audience for the indicators is public health officials and HIV prevention community planning groups at the state and local levels. CDC must also be able to describe and assess the national status of HIV prevention to help guide national policies and programs. Meeting the needs of both these audiences requires a standard set of core indicators that can be addressed with data uniformly available across the country. Thus, the indicators in this report are limited to those for which data are currently available for at least half of CDC's HIV prevention grantees.
The HIV Prevention Indicators (HPI) are designed to monitor the key biological, behavioral, socio-political, and service- oriented "vital signs" of the four primary HIV sub-epidemic populations, which are: (1) Childbearing Women; (2) High-Risk Heterosexuals; (3) Injection Drug Users; and (4) Men Who Have Sex with Men.
Initially, the methods workgroup was formed to develop guidelines and a methodology in the indicator development process. That group felt the indicators should be developed through a consensus-driven, evidence-based process, tapping expertise from a broad range of scientists, program managers, public policy-makers, and community members. The development process also involved convening an expert panel to: draft potential core HIV prevention indicators; conduct background and literature reviews on the core indicators; pre-convene the expert panel to review the indicators based on the scientific literature search; pilot the indicators for availability, feasibility and usefulness of the data; and, finally, develop protocols for the remaining indicators.
Over 200 indicators were brought forward, subsequently narrowed to 44, and finally, as of publication of this report, 37 which are believed to meet the minimal criterion of having a data source(s) available in at least half of the grantees' jurisdictions. There are eight (8) indicators for the sub-epidemic population group "Injection Drug Users"; nine (9) indicators in the category of "Men Who Have Sex With Men"; 14 indicators for the category, "High-Risk Heterosexuals"; and six indicators for the category "Childbearing Women."
This report represents the first phase of the overall HIV prevention indicators process on the use of existing data sets for impact indicators. Since this activity began in 1996, however, there has been acknowledgment that many of the critical behavioral and socio-political impacts of HIV prevention have little or no way to be measured. Future phases of the HPI will want to consider the need for new data sources.
This study was conducted by the Program Evaluation Research Branch of Macro International Inc. The project officer, Nancy Chalmers, can be reached at (404) 639-7085. Copies of the final report, PIC ID #6707, can be obtained at the PIC.
SERVICES AVAILABLE FROM THE PIC
The Policy Information Center (PIC) is a centralized source of information on in-process, completed, and on-going evaluations; short-term evaluative research and; policy-oriented projects conducted by HHS as well as other Federal departments and agencies. The PIC on-line database provides project descriptions of these studies. It is available on-line at: http://aspe.hhs.gov/PIC/. Inquiries regarding PIC services should be directed to Juanita Butler, Technical Information Specialist, at 202-690-6445. Or E-mail PIC at: email@example.com