Performance Improvement 2006

Chapter I - Program Evaluation at the Department of Health and Human Services

This Performance Improvement 2006 report presents to Congress a comprehensive summary of evaluation projects completed by the Department, The Office of the Secretary, and agencies comprising the Department, engage in extensive evaluation activities; some are required by statute, while others are determined to be essential by an individual agency, the Department, or the President. Evaluation is a core Federal program management responsibility along with strategic planning, policy and budget development, and program operation (Figure 1, Performance Management System).

Performance Management System 

Figure 1

The Department of Health and Human Services (HHS) is responsible for more than 300 separate programs, with over $580 billion in annual funding for health and social service support payments, and approximately $67 billion for discretionary programs. HHS spends about $2.6 billion for research, demonstration, and evaluation activities. Successful evaluations increase the likelihood of effective delivery of public services through these programs and insure that programs are efficient, targeted to their intended clients, and well managed.

Structure of this report

Evaluations summarized in this year’s report are organized under the Department’s eight broad Strategic Goals consisting of 40 essential Objectives (Appendix A). As articulated in the Department’s FY 2004- 2009 Strategic Plan, the goals are as follows:

1. Prevent Disease and Illness
2. Protect Our Homeland
3. Close the Gaps in Health Care
4. Improve Health Science
5. Realize the Possibilities of 21^st Century Health Care
6. Work Toward Independence
7. Leave No Child Behind
8. Improve Department Management

Performance Improvement 2006, the 12^th annual report in this series, provides summaries of findings of HHS evaluation projects completed or released during Fiscal Year (FY) 2005 (October 1, 2004 through September 30, 2005) that were funded under Section 241 of the Public Health Service Act. This Chapter provides an overview  of  the  Department’s evolving  evaluation  responsibilities.  Chapter  2 presents the summaries of the evaluations. In addition to the Department’s Strategic Plan Goals and Objectives found in Appendix A, Appendix B contains  a statement of each  agency’s Mission and Evaluation Program, Appendix C provides a table of the studies by each agency and the Objective in this report where the study may be found, and Appendix D acknowledges the HHS officials who contributed to the report.

Role of Evaluation

Programs need to provide better results for the individuals served, assure that tax dollars are wisely spent, and achieve the aims set for them by elected representatives. This report to Congress on Performance Improvement continues the transition to a more strategic and analytic presentation of evaluation studies. With the implementation of a unified Strategic Plan, as required by the Government Performance and Results Act of 1993, the Department recognizes its responsibility both to evaluate programs and to assure that evaluation funds are targeted to address the core goals and objectives of both the Congress and Executive branch. This re-engineering of management practice underscores the important role of evaluations’ potential to test, weigh, measure and judge the success of management performance, program outputs, and social outcomes.

Evaluations play an integral role in carrying out the HHS mission. Assessing various aspects of agency functioning allows staff to identify means of improving individual program performance. HHS evaluations directly support the following efforts: (1) Helping government officials and members of the Congress make decisions related to programs, policies, budgets, and strategic planning; (2) Enabling managers to improve program operations and performance; (3) Disseminating evaluation results and methodological tools useful to the larger health and human services community of state and local officials, researchers, advocates, and practitioners to improve the performance of their programs.

Types of Evaluation

For HHS, evaluation is the assessment of the performance (Comprising efficiency, effectiveness, and responsiveness) of the Department’s programs or strategies through the analysis of data or information collected systematically and ethically; and the effective use of resulting information in strategic planning, program or policy decision-making and program improvement.  Evaluations serve one or more of the following objectives (Figure 2, Types of Evaluation):

• Enhance Program Effectiveness and Support Policy Analysis — Determine the impact of HHS programs on achieving intended goals and objectives and examine the impact of alternative policies on the future direction of HHS programs or services.
• Improve Performance Measurement — Monitor annual progress in achieving departmental strategic and performance goals. We invest evaluation funds to develop and improve performance measurement systems and improve the quality of the data that support those systems. Performance measurement is a high priority for HHS agencies. The emphasis during development, implementation, and refinement of programs is on results and specific measurements are required under the Government Performance and Results Act.
• Assess Environmental Factors — Seek to understand the forces of change in the health and human services environment that influence the success of our programs. Such understanding allows us to adjust our strategies and continue to deliver effective health and human services.
• Strengthen Program Management and Development — Address the need of program managers to obtain information or data that will help them effectively design and manage programs more efficiently and ensure successful results. Focus on developmental or operational aspects of program activities and provide understanding of services delivered and populations served.

Types of Evaluation:

Figure 2

Evaluation Resources

Evaluation activities of the various HHS agencies are largely supported through two funding mechanisms: direct use of program funds and use of special legislative set-aside authorities for evaluation. The first is a common mechanism by which program managers have discretionary authority to use appropriated program funds to support contracts that will design and implement evaluation studies, and analyze evaluation data. In some cases, a program’s legislative authority calls for a specially mandated evaluation, and program funds are used directly to support the evaluation.

The second mechanism for evaluation funding is the legislative set-aside authority that permits the Secretary of HHS to use a portion of overall program funds for evaluation purposes. The largest of the set-aside authorities at HHS is Section 241 of the Public Health Service (PHS) Act for evaluations conducted by several of HHS’ agencies:

• Administration for Children and Families (ACF)
• Agency for Healthcare Research and Quality (AHRQ
• Centers for Disease Control and Prevention (CDC)
• Health Resources and Services Administration (HRSA)
• National Institutes of Health (NIH)
• Substance Abuse and Mental Health Services Administration (SAMHSA)

In addition, two staff components in the Office of the Secretary receive funds under this authority to carry out evaluations:

• Office of the Assistant Secretary for Planning and Evaluation (ASPE)
• Office of Public Health and Science (OPHS)

This authority was originally established in 1970, when the Congress amended the Act to permit the HHS Secretary to use up to 1 percent of appropriated funds to evaluate authorized programs. Section 241 limited the base from which funds could be reserved for evaluations to programs authorized by the PHS Act. Excluded were funds appropriated for the Food and Drug Administration, the Indian Health Service, and certain other programs that were managed by PHS agencies but not authorized by the Act (e.g., HRSA’s Maternal and Child Health Block Grant and CDC’s National Institute for Occupational Safety and Health).¹   In addition, programs may have other authorized sources of funding for evaluation activities.² Section 207 of Division F of the Consolidated Appropriations Act, 2005 increased the amount the Secretary could use for evaluation to 2.4 percent.

¹ FDA programs are principally authorized by the Food, Drug and Cosmetic Act, and appropriations are provided by the Appropriations Subcommittee on Agriculture, Rural Development, Food and Drug Administration and Related Agencies.  IHS programs are principally authorized by the Indian Health Care Improvement Act and the Indian Self- Determination Act Appropriations are provided by the Appropriations Subcommittee on Interior and Related Agencies.

² The Older Americans Act (OAA) specifies that $1.5 million from Title III, and $1.5 million from Title IV are to be available from its annual appropriations to be used for the evaluation of OAA programs.  Since 2000, the Administration on Aging (AoA) has used those funds for the Performance Outcome Measures Project and its annual national performance measurement surveys.  AoA initiated new evaluation studies of Title III-D Health Promotion and Disease Prevention, and Title III-B Supportive Services in FY 2004 and intends to continue evaluating all OAA titles on a rotating basis in the future.

Evaluation Management

Management of evaluations carried out by HHS agencies and offices and coordinated by ASPE involves:

(1) planning and coordination, (2) project oversight, (3) quality assurance, and (4) dissemination of results (Figure 3, Evaluation Management Cycle). A description of each function follows.

Evaluation Mangement Cycle 

Figure 3

Evaluation Planning and Coordination

HHS agencies, ASPE, the Office of the Inspector General (OIG), and OPHS develop evaluation plans annually in concert with HHS program planning, legislative development, and budgeting cycles. Plan development is coordinated by ASPE. Each agency or office evaluation plan generally states the evaluation priorities, or projects under consideration for implementation. Typically, HHS evaluation priorities include: congressionally-mandated program evaluations, evaluations of Secretarial program or policy initiatives, assessments of new programs and ones that are candidates for reauthorization, and evaluations that support program performance management and accountability.

HHS evaluation planning activities are coordinated with three department-wide planning initiatives. First, HHS evaluation activities support the Department’s strategic planning and performance management activities in several ways. Completed evaluation studies are used in shaping the specific HHS strategic goals and objectives. Evaluation findings provide important sources of information and evidence about the success of various HHS programs or policies. The HHS Strategic Plan highlights evaluations that document efficacy or effectiveness of strategic programs or policies and lists future evaluations that will benefit strategic planning. HHS agencies use findings from their evaluations to support the Government Performance and Results Act (GPRA) annual performance reporting to Congress and program budget justifications.

Second, Congress requests that HHS coordinate and report to Congress regarding all of its research, demonstration, and evaluation (RD&E) programs to ensure that the results of these projects address HHS program goals and objectives. ASPE and the Assistant Secretary for Budget, Technology and Finance work together with HHS agencies to provide the Congress with a special annual research, demonstration, and evaluation budget plan that coincides with the preparation of the President’s fiscal year budget. The plan outlines HHS agency research, demonstration, and evaluation priorities as related to the Department’s strategic goals and objectives (Figure 4, Evaluation Reporting Cycle).

Evaluation Reporting Cycle

Figure 4

The Secretarially-created Research Coordination Council (chaired by the ASPE, and containing representatives of HHS agencies) fosters greater interactions among the research programs. The Council’s work includes streamlining research and evaluating Department-wide research priorities to ensure greater efficiencies in research, demonstration, and evaluation. The Council seeks to strengthen HHS research coordination and planning around key Departmental priorities and themes.

Third, the Secretary must report to the Congress his plan for using PHS evaluation set-aside funds before implementing the plan (Figure 4). Those agencies and offices that use the PHS evaluation set-aside authority—ACF, AHRQ, CDC, HRSA, NIH, ASPE, OPHS, SAMHSA—submit a formal plan to ASPE, which coordinates and develops the individual plans into the HHS report to the Congress on the use of the PHS authority.

Project Oversight

The OIG performs independent evaluations through its Office of Evaluations and Inspections (OEI). OEI’s mission is to improve HHS programs by conducting inspections that provide timely, useful, and reliable information and advice to decision makers. These findings of deficiencies or vulnerabilities and recommendations for corrective action are usually disseminated through inspection reports issued by the Inspector General. A summary of individual inspection reports and other OIG reports can be viewed on the World Wide Web at http://oig.hhs.gov/reports.html. OEI provides technical assistance to HHS agencies in conducting their evaluations.

Quality Assurance and Improvement

Most evaluation projects are developed at the program or office level. A committee of agency- or office- level policy and planning staff members generally conducts the initial quality review. Before a project is approved, a second committee reviews it for technical quality with expertise in evaluation methodology. Technical review committees follow a set of criteria for quality evaluation practice established by each agency. ASPE, for example, has a formalized peer review process in which experienced evaluators on staff review, discuss and approve all proposed research projects before they are submitted for funding. Some HHS agencies have external evaluation review committees composed of evaluation experts from universities and research centers.

Since HHS began reporting to Congress in 1995 on completed evaluations through the Performance Improvement report series, the Department has focused attention on improving the quality of evaluation studies performed. An Evaluation Review Panel, convened periodically, has contributed insights to HHS evaluation officers on the strengths and challenges of ensuring quality evaluation studies. HHS evaluation officers have had opportunities to discuss these strengths and challenges and identify steps to improve agency evaluation projects.

Dissemination of Evaluation Reports

Maintaining on-line electronic report libraries and distributing information on evaluation results is an important component of HHS evaluation management. The Department’s information and reports on major evaluations is available centrally through the website of the HHS Policy Information Center (PIC), located at the following Web address: http://aspe.hhs.gov/pic. ASPE’s PIC website offers users an opportunity to search – by key word, selected program, or policy topics, – the departmental evaluation report database and electronic report library maintained by ASPE. The PIC contains over 8,500 completed and in-progress evaluation and policy research studies conducted by the Department, as well as key studies completed outside of HHS by the U.S. Government Accountability Office (GAO) and private foundations.

The results of HHS evaluations are disseminated through targeted distribution of final reports, articles in refereed journals, and presentations at professional meetings and conferences. Although individual HHS agencies have primary responsibility for disseminating results, ASPE continues its Department-wide efforts to expand dissemination of evaluation results to the larger research and practice communities through e-mail lists, e-newsletters, and publications available on the PIC Web site.

Chapter II – Summaries with Findings for Completed Evaluations

This Chapter presents brief abstracts summarizing the purposes and findings for each HHS evaluation completed between October 1, 2004 and September 30, 2005.  As discussed in Chapter 1, each study is presented under the Strategic Objective (Appendix A) it most supports. This thematic grouping provides an immediate view of the program and research areas emphasized during the previous year and aids in identifying areas for possible future inquiry. Future evaluation priorities will also be guided by the results of oversight not only by the Congress but also by the Executive Branch through such management mechanisms as the Performance Assessment Reporting Tool (PART), which is already yielding specific recommendations for future evaluative inquiry and program validation.

To find abstracts in this Chapter for studies funded by particular agencies, refer to the Objectives identified in Appendix C. Not all Objectives are represented among the completed studies reported in Chapter 2; a multitude of studies are also already in progress and development of the new fiscal year’s research, demonstration, and evaluation agenda will add other studies to the roster throughout the year. Further information about completed and in-progress reports can be accessed on-line at the Policy Information Center Web site, located at http://aspe.hhs.gov/pic

Goal 1 - Reduce The Major Threats to the Health and Well-being of Americans

Objective 1.1 - Reduce Behavioral and Other Factors That Contribute to the Development of Chronic Diseases

National Kidney Disease Education Program (NKDEP) Evaluation Survey

PIC ID: 7923; Agency Sponsor: NIH, National Institutes of Health; Federal Contact: Gladstone, Elisa, 301- 435-8116; Performer: Equals Three Communications, Bethesda, MD

Assessment of Diabetic Eye Disease Education and Resource Needs of Pharmacists

Diabetes has reached epidemic proportions in the United States and prevalence rates are expected to increase over the coming years. Most people with diabetes develop diabetes complications, one of which is diabetic eye disease, particularly diabetic retinopathy. The National Eye Institute (NEI) conducted a series of discussion groups with pharmacists to determine their needs regarding diabetic eye disease education, and what resources and materials would be most useful to pharmacists and their patients.

Pharmacists have the potential to educate their patients about the complications of diabetes, specifically about diabetic eye disease and glaucoma, but only if they are themselves educated about these diseases. Because language and health literacy were identified as barriers, careful consideration should be given to the wording used in any products developed. Content and calls to action must be written simply and clearly. Pharmacists also suggested that the National Eye Health Education Program (NEHEP) collaborate with pharmacy associations and organizations to create a free Continuing Education (CE) course on diabetic eye disease. Printed materials could also be distributed to pharmacists after they complete the CE course.

PIC ID: 8202; Agency Sponsor: NIH-NEI, National Eye Institute; Federal Contact: Janiszewski, Rosemary, 301-496-5248; Performer: ORC Macro, Calverton, MD

Diabetes: A National Plan for Action

PIC ID: 8235; Agency Sponsor: ASPE-OHP, Office of Health Policy; Federal Contact: Tilson, Wilma, 202- 205-8841; Performer: Research Triangle Institute, Research Triangle Park, NC

Center for Food Safety and Applied Nutrition (CFSAN) Program Evaluation Model

The model provides guidance for preparing program evaluations, which provides the following: (1) A mechanism for the agency to address the condition of and specific regulatory concerns about regulated industry and products; (2) Provides for risk-based utilization of CFSAN and Office of Regulatory Affairs (ORA) field resources; (3) Furnishes the information necessary to develop or support regulations, policies, regulatory strategies, and industry and consumer outreach activities; (4) Provide a feedback mechanism for modifying, improving, and maintaining the quality of compliance programs and field assignments; (5) Provides an information source on FDA program area activities to an interested public. The model provides general instructions, content, and format guidance for preparing CFSAN program evaluations.

The model identifies “Program Design and Methodology” as a required heading and discussion topic for each program evaluation. This section of the evaluation should describe the program requirements, indicate the method used for evaluation of the program area, (including the risk prioritization model that will be used, the type of data that will be collected, the data sources, and how the data will be analyzed), and indicate data and information sources used for the evaluation. The model identifies “Results” or “Findings” as a required heading and discussion topic for each program evaluation. This section of the evaluation should indicate the findings and use tables and figures to demonstrate support for each  finding as appropriate. It also should include a summary of risks associated with the industry or with products covered by the program. The model identifies “Recommendations” as a required heading and discussion topic for each program evaluation. Recommendations for activities that are necessary to achieve the program objective or that will enhance CFSAN’s ability to determine progress toward achieving the program objective should be addressed in this section of each program evaluation.

PIC ID: 8249; Agency Sponsor: FDA-CFSAN, Center for Food Safety and Applied Nutrition; Federal Contact: Ledet, Mischelle, 301-436-2070; Performer: FDA, Center for Food Safety and Applied Nutrition, Washington, DC

Objective 1.2 - Reduce the Incidence of Sexually Transmitted Diseases and Unintended Pregnancies

Evaluating the Impact of Viral Hepatitis Integration on HIV and STD Prevention Services

The primary objectives were to describe how and to what extent viral hepatitis services have been integrated into STD clinics, HIV Counseling and Testing Sites (CTSs), jails, and substance abuse treatment; the impact of integration on existing HIV and STD prevention activities; and the costs, benefits, and lessons learned from integration. The evaluation was conducted in two phases. The formative phase encompassed engaging stakeholders, describing the programs, and focusing the evaluation design. The main activities in the formal phase were to collect and analyze data, prepare the case studies, and analyze and summarize data within and across sites. The results indicate that a substantial level of viral hepatitis services can be achieved in STD clinics, HIV CTSs, jails, and substance abuse treatment facilities, and also documented multiple examples of successful integration into existing public health settings. Results also demonstrate the importance of providing hepatitis A and B vaccinations, hepatitis C virus testing, and clinical care options for persons identified with HCV infection in public health settings.

PIC ID: 7979; Agency Sponsor: CDC-NCHSTP, National Center for HIV, STD, and TB Prevention; Federal Contact: Dietz, Sue, 404-639-0210; Performer: Research Triangle Institute, Research Triangle Park, NC

AIDSInfo Process Evaluation Study: Analysis of Trends in Service Usage, Accuracy and Customer Satisfaction

PIC ID: 8164; Agency Sponsor: NIH-NIAID, National Institute of Allergy and Infectious Diseases; Federal Contact: Siskind, Rona, 301-435-3732; Performer: Aspen Systems Corporation, Rockville, MD

Youth Development Approaches in Adolescent Family Life Demonstration Projects

The purpose of this project was to examine how youth development (YD) strategies in conjunction with abstinence education may affect adolescents’ sexual decision making. The Urban Institute (UI) explored the relationship between YD activities and the prevention of sexual risk taking. Through this undertaking, UI also examined the evaluability of several Adolescent Family Life (AFL) grantees. The project conducted a literature review, a document review, and site visits to AFL projects to gather information. The UI also utilized feedback from AFL Project Officers to augment information for this report. The literature review identified several programs that demonstrated direct effects of YD approaches on sexual risk taking or on antecedents to sexual risk taking behaviors. All of these programs were clear in their interventions and conducted short and long-term evaluations. The project also found that several AFL grantees identified the use of specific YD strategies in their programming. The predominant theme of the programs was a total integration of YD approaches with the abstinence education components. This comprehensive approach makes it difficult to extract the YD strategies for individual evaluation. The UI recommends that the AFL projects submit more specific and intensive End of Year reports, thereby helping programs to better identify successes and failures. The project also recommends that individual programs should enhance their evaluations and the OAPP should conduct a cross-site evaluation.

PIC ID: 8242; Agency Sponsor: OPHS-OPA, Office of Population Affairs; Federal Contact: Roper, Allison, 240-453-2806; Performer: Urban Institute, Washington, DC

SAFE Know Now Campaign Evaluation

The objective of the Know Now campaign evaluation was to understand the effectiveness of the targeted campaign efforts. Specifically, the objectives were to understand whether campaign messages were reaching the targeted clusters based on degree of exposure; ascertain how the targeted clusters have been affected by the message, measuring their cognitive, affective, and behavioral responses; and determine whether campaign messages have reached other non-targeted clusters (message spillover) and if so, how people in these clusters have been affected and whether increases in requests for HIV testing by persons at low risk of infection were observed.

PIC ID: 8251; Agency Sponsor: CDC, Centers for Disease Control and Prevention; Federal Contact: Davis, David, 404-639-0938; Performer: Aeffect, Deerfield, IL

Objective 1. 3 – Increase Immunization Rates Among Adults and Children (National Immunization Program)

Institute of Medicine (IOM) Immunization Review

The key objectives of this evaluation were to: (1) Evaluate whether the IOM Committee has met the requirements and needs of CDC and NIH as described in the original scope of work and by informed agency staff members; (2) Assess the knowledge of the IOM Committee and its reports by various external partners and stakeholder groups; (3) Assess the impact of the IOM reports on immunization decisions of vaccine recipients and/or their parents; (4) Assess the impact of the IOM reports on information relayed to vaccine recipients or their patients by their health care providers. The methods consisted of primary (in-depth key informant surveys and interviews) and secondary (abstractions of IOM Committee reports to evaluate whether the Committee met its scope of work) data analysis. The analysis concluded that the IOM Committee generally met the scope of work, but there were incidents in which the Committee appeared to exceed or fail to meet the mission set out for it. There is a clear consensus that a continuing need exists for a credible vaccine safety review group. However, there is no consensus on the membership or structure of such a group.

PIC ID: 7986; Agency Sponsor: CDC, Centers for Disease Control and Prevention; Federal Contact: Dietz, Sue, 404-639-0210; Performer: Research Triangle Institute, Research Triangle Park, NC

Objective 1.4 - Reduce Substance Abuse

Alcohol and Drug Services Study (ADSS) Phase II: Client Record Abstract Report

This report presents findings for the Phase II of the Alcohol and Drug Services Study sponsored by SAMHSA. Data reported was abstracted from client treatment records in a stratified sample of 280 alcohol and drug treatment facilities. This report is based on a nationally representative sample of client  treatment episodes from August 1997 through April 1999. Treatment experiences could be different depending on where an individual was treated, according to variations by client type of care. The objective of this report was to provide detailed national data on current substance abuse treatment practices, clients in treatment and treatment length of stay for use by providers, policymakers, and  researchers. This Phase consisted of administrator interview and client record abstracting. The sample design for ADSS was a multi-stage stratified clustered design, which involved site visits to a sub-sample of facilities. This report was then organized into two main sections: the findings for the ADSS discharged clients and the findings for the in-treatment methadone clients. All findings presented were weighted national estimates.

PIC ID: 8060; Agency Sponsor: SAMHSA, Substance Abuse and Mental Health Services Administration; Federal Contact: Gadzuk, Anita, 240-276-1266; Performer: Brandeis University, Waltham, MA

Feasibility Study for the NIDA Science-Based Drug Education Project Entitled “NIDA Goes Back to School.” - Final Report

In September 2003, the National Institute on Drug Abuse (NIDA) launched its NIDA Goes Back to School (NGBTS) Campaign. NGBTS is designed to promote NIDA as the source for science-based information on drug abuse, announce the NIDA for Teens Website, and highlight opportunities to order NIDA’s free science-based materials for school-aged children. This feasibility study refined the evaluation questions, identified the optimal evaluation methodology, and determined the feasibility of, and resources necessary for, a full-scale evaluation. The contractor collected and analyzed data from focus groups, individual interviews, school site visits, and secondary data sources. The school site visits and interviews demonstrated the usefulness of the NGBTS materials, but also illuminated obstacles teachers face incorporating new materials into their curriculum. Based on the evaluation of existing data, such as website usage and publication orders, it was determined that a full-scale evaluation is feasible. A mixed- method data collection strategy and multilevel study design was recommended. By examining NIDA’s current dissemination strategies and identifying more efficient and cost-effective approaches to reach a multitude of target audiences, a full-scale evaluation of the NGBTS program will provide NIDA with the tools necessary to ensure the rapid and effective dissemination of NIDA’s science education materials.

PIC ID: 8191; Agency Sponsor: NIH-NIDA, National Institute on Drug Abuse; Federal Contact: Palmquist, Lanette, 301-443-6036; Performer: CSR Incorporated, Washington, DC

Results from the 2004 National Survey on Drug Use and Health (NSDUH): National Findings

This report presents the results from the 2004 National Survey on Drug Use and Health (NSDUH), an annual survey of the civilian, non-institutionalized population of the United States aged 12 years old or older. This initial report on the 2004 data presents national estimates of rates of use, numbers of users, and other measures related to illicit drugs, alcohol, and tobacco products. Measures related to mental health problems also are presented, including data on the co-occurrence of substance use and mental health problems, and new data on depression among youths and adults. Among youths aged 12 to 17, the rate of illicit drug use declined between 2002 and 2004 (11.6 percent in 2002, 11.2 percent in 2003, and 10.6 percent in 2004). An estimated 22.5 million Americans aged 12 or older in 2004 were classified with past year alcohol or illicit drug dependence or abuse (9.4 percent of the population), about the same number as in 2002 and 2003.

http://oas.samhsa.gov/NSDUH/2k4nsduh/2k4Results/2k4Results.pdf

PIC ID: 8259; Agency Sponsor: SAMHSA-OAS, Office of Applied Studies; Federal Contact: Hughes, Arthur, 240-276-1261; Performer: Research Triangle Institute, Research Triangle Park, NC

Drug Abuse Warning Network (DAWN), 2003: Area Profiles of Drug-Related Mortality

http://dawninfo.samhsa.gov/pubs/mepubs/default.asp

PIC ID: 8260; Agency Sponsor: SAMHSA-OAS, Office of Applied Studies; Federal Contact: Ball, Judy K, 240-276-1256; Performer: Westat, Inc., Rockville, MD

National Survey of Substance Abuse Treatment Services (N-SSATS): 2004 Data on Substance Abuse Treatment Facilities

The N-SSATS is an annual census of facilities providing substance abuse treatment conducted by the Substance Abuse and Mental Health Services Administration. The survey collects data on the location, characteristics, and use of alcoholism and drug abuse treatment facilities and services throughout the United States and other U.S. jurisdictions. The 2004 survey was the 27th in a series of national surveys of treatment providers and was conducted with a reference date of March 31, 2004. Data was collected by mail and web-based questionnaires and by telephone interviews. A total of 13,454 facilities reported a one-day census of 1,072,251 clients enrolled in treatment, 91,112 (eight percent) of which were under the age of 18. Nearly half (46 percent) of all clients were in treatment for both an alcohol and drug abuse problem. Approximately one third (34 percent) of clients were in treatment for drug abuse only, and 20 percent were in treatment for abuse of alcohol alone. Nationwide, there were 459 clients in treatment per 100,000-population aged 18 and older. 89 percent of all clients were in outpatient treatment, nine percent in non-hospital residential treatment, and one percent in hospital inpatient treatment. Opioid Treatment Programs were available at eight percent of all treatment facilities and clients receiving methadone accounted for 22 percent of all clients in treatment. Sixty percent of facilities were operated by private-nonprofit organizations, and 26 percent were operated by private for-profit organizations. The remaining 14 percent were operated by Federal, State, local, or tribal governments.

http://wwwdasis.samhsa.gov/04nssats/nssats_rpt_04.pdf

PIC ID: 8261; Agency Sponsor: SAMHSA-OAS, Office of Applied Studies; Federal Contact: Trunzo, Deborah, 240-276-1267; Performer: Synectics for Management Decisions, Inc., Arlington, VA

Goal 2 - Enhance the Ability of the Nation’s Health Care System to Effectively Respond to Bioterrorism and Other Public Health Challenges

Objective 2.1 - Build the Capacity of The Health Care System to Respond to Public Health Threats, Especially Bioterrorism Threats, in a More Timely and Effective Manner

Evaluation of Non-Participants in the Smallpox Vaccination Program

The purpose of this project was to determine: (1) Why persons choose not to be vaccinated with smallpox (vaccinia) vaccine in the Stage I vaccination program; (2) Why individuals opt out of the vaccination program after the pre-screening process but before presenting for vaccination; (3) Why individuals who present for vaccination choose not to be vaccinated. The study includes health care workers (HCWs) and public health personnel. Results will be used to evaluate the performance of the State I vaccination program and to help identify areas of potential later stages. By identifying the rates and reasons for nonparticipation, CDC will be able to better understand the perceived barriers and educational needs. The threat of the use of biological agents (including smallpox) as biological weapons remains a major concern for the United States. Information collected from this evaluation will be used to enhance planning for other vaccination programs that might need to be directed toward adults and/or health care staff in preparation for or in response to a deliberate, malicious, and catastrophic release of bioterrorism agents or emerging infectious diseases.

PIC ID: 7935; Agency Sponsor: CDC-NIP, National Immunization Program; Federal Contact: Dietz, Sue, 404-639-0210; Performer: Research Triangle Institute, Research Triangle Park, NC

Objective 2.2 - Improve the Safety of Food, Drugs, Biological Products, and Medical Devices

FDA’s Evaluation of the Seafood HACCP Program for Fiscal Years 2002/2003

http://www.cfsan.fda.gov/~comm/haccpsea.html

PIC ID: 8243; Agency Sponsor: FDA-CFSAN, Center for Food Safety and Applied Nutrition; Federal Contact: Creeden, John, 301-436-1419; Performer: FDA, Center for Food Safety and Applied Nutrition, Washington, DC

Infant Formula Program (CP 7321.006)

http://intranet.cfsan.fda.gov/oc/pages/programs/infantpe.pdf

PIC ID: 8244; Agency Sponsor: FDA-CFSAN, Center for Food Safety and Applied Nutrition; Federal Contact: Greenberg, Beatrice, 301-436-1690; Performer: FDA, Center for Food Safety and Applied Nutrition, Washington, DC

Office of Cosmetics and Colors Program Evaluation of the Domestic Cosmetics Compliance Program (CP 7329.001) for Fiscal Year 2004 (October 1, 2003 through September 30, 2004)

The purpose of this project was to evaluate the Food and Drug Administration’s (FDA) Field inspection findings for cosmetic manufacturing firms in order to assist in setting inspection priorities. Since FDA does not have pre-market review authority for cosmetics or cosmetic labeling, exercising our inspectional authority under the FD&C Act is especially important in the regulation of cosmetic products. Thus, the inspections and samples required under this program are necessary to help ensure that the industry is producing cosmetics that are both safe and properly labeled before they reach the market. The  evaluation involved the review of FDA’s “Field Accomplishments and Compliance Tracking System” (FACTS) database for Fiscal Year 2004 (FY04) “cosmetic establishment inspections.” Data reviewed included numbers of inspections, inspection classifications, product types, and sample collections. 14 FDA District Offices conducted 120 inspections under the Domestic Cosmetics Compliance Program in FY04.

Inspections covered a broad range of cosmetic products. Inspections were classified as follows: 69 percent “No Action Indicated,” 23 percent “Voluntary Action Indicated,” six percent “Official Action Indicated,” and two inspections were not classified. The number (seven) and reasons (mainly microbiological and colors related) for recalls in FY04 did not diverge significantly from the number and reasons for recalls as the previous three fiscal years. Eighty-five samples were collected during these inspections. Two inspections involved samples for potential microbial contamination (shampoo and hair styling products), one inspection collected samples of tattoo inks for which multiple adverse events were reported to the agency, and one inspection involved the collection of decorative, non-vision-correcting contact lenses (for label review and to determine if the lenses were distributed without the involvement of a qualified eye care professional). Overall, the objectives of the Domestic Cosmetics Compliance Program (CP 7329.001) appear to have been met and remain as valid objectives for the FY05 cosmetics program. The program evaluation data indicated that program objectives could be enhanced through increased inspectional coverage of eye area cosmetic products, sampling for microbiological analysis, and the submission of the bovine-derived tissue questionnaire per program directions. Additional coverage is suggested for establishments that produce solely cosmetic products (e.g., firms that do not also produce drug products), and of products marketed as cosmetics but that bear drug claims.

http://intranet.cfsan.fda.gov/oc/pages/programs/cosmetpe.pdf

PIC ID: 8245; Agency Sponsor: FDA-CFSAN, Center for Food Safety and Applied Nutrition; Federal Contact: Lambert, Lark A, 301-436-1143; Performer: FDA, Center for Food Safety and Applied Nutrition, Washington, DC

Evaluation of the Food and Color Additives Compliance Program for Imported Foods

The purpose of this project was to evaluate the performance of the program against its primary objective of determining compliance of imported foods with the requirements of the Federal Food, Drug, and Cosmetic Act and the food and color additive regulations. The performance of this compliance program was evaluated based on data collected by the Food and Drug Administration (FDA) field offices in FY 2004 from analyzing samples of imported food products for food and color additive violations. The study found continued high violation rates in 2004 for undeclared sulfites (45 percent), cyclamates (45 percent),  and coumarin (36 percent). The violation rates for all targeted food and color additives were 44 percent and 58 percent, respectively. The report makes a number of recommendations: (1) Target cyclamates    in all foods, not just mixed fruit products; (2) Target 3-MCPD (a carcinogenic impurity) in Asian-style sauces and acid-hydrolyzed protein; (3) Target unsafe herbal and other dietary supplement- type ingredients in conventional foods; (4) Spend more resources on medium and high risk violations (e.g., unapproved color additives such as Sudan I) and less on low risk problems (e.g., undeclared color additives that are approved for food use); (5) Use moderate surveillance program with focus on problems identified by foreign food-control authorities (e.g., the illegal color additive Sudan I); (6) Delete undeclared benzoates, sorbates, and nitrates/nitrites as violations that should be targeted; (7) Keep abreast of software for finding food and color additive violation trends; (8) Determine whether betel nuts are still being offered for import; (9) Implement an equivalent of the EC’s Rapid Alert System.

http://intranet.cfsan.fda.gov/OC/pages/progrevaln.htm

PIC ID: 8247; Agency Sponsor: FDA-CFSAN, Center for Food Safety and Applied Nutrition; Federal Contact: Zajac, Andrew, 301-436-1267; Performer: FDA, Center for Food Safety and Applied Nutrition, Washington, DC

Program Evaluation of the Domestic and Imported Cheese and Cheese Products October 1, 2003 - September 30, 2004 (FY04)

PIC ID: 8248; Agency Sponsor: FDA-CFSAN, Center for Food Safety and Applied Nutrition; Federal Contact: Metz, Monica, 301-436-2041; Performer: FDA, Center for Food Safety and Applied Nutrition, Washington, DC

Goal 3 - Increase the Percentage of the Nation’s Children and Adults Who Have Access to Health Care Services, and Expand Consumer Choices

Objective 3.1 - Encourage the Development of New, Affordable Health Insurance Option

The Role of Health Insurance in Successful Labor Force Entry and Employment Retention

This report is one in a series developed by The Lewin Group and its subcontractors, Berkeley Policy Associates and Cornell University, for the Office of the Assistant Secretary for Planning and Evaluation (ASPE) within the U.S. Department of Health and Human Services (HHS), in connection with the study, “Research on Employment Supports for People with Disabilities.” For this project, we reviewed available studies and literature, and collected detailed information on the experiences of people with significant disabilities who are successfully and competitively employed, the events and factors affecting their employment decisions, the relative importance of specific factors, and the reasons for successful and unsuccessful employment attempts. The goal of the study was to gain a better understanding of the role supports play in the employment of people with disabilities. In particular, the study aimed to identify the role(s) supports play in two key areas: providing assistance to people with significant disabilities to participate successfully in competitive employment, and improving employment outcomes for people with disabilities. This project is unique among the many investigations of the factors affecting the employment of people with disabilities, in that it focuses on those who have achieved a measure of success in employment, and the factors contributing to their success.

http://aspe.hhs.gov/daltcp/reports/lfentry.htm

PIC ID: 7346; Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy; Federal Contact: Frank, Andreas, 202-690-6443; Performer: The Lewin Group, Fairfax, VA

Evaluation of Parity in the Federal Employees Health Benefits (FEHB) Program: Final Report

http://aspe.hhs.gov/daltcp/reports/parity.htm

PIC ID: 8208; Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy; Federal Contact: Kennedy, Cille, 202-690-6443; Performer: Northop Grumman Information Technology, Health Solutions, Rockville, MD

Health Savings Accounts: Early estimations on national take-up from the 2003 MMA and future policy proposals

The Medicare Modernization Act (MMA) established Health Savings Accounts (HSAs), which are tax- advantaged savings vehicles that can be used to pay for medical expenses incurred by individuals and their dependents. The purpose of this project was to examine the potential impact of HSAs on insurance coverage. This examination included: (1) Developing baseline estimates of the numbers of individuals that will be covered by HSAs; (2) Developing estimates on the number of additional low-income individuals that would obtain insurance coverage, if HSAs were offered in combination with either tax credits or vouchers; (3) Developing estimates of the costs to the government associated with tax credits or vouchers. Using a combination of public and private data sources to examine the impact of consumer directed health plans generally and HSAs more specifically, the analysis found that the national adoption of these plans might be significant. The baseline (unsubsidized) simulation indicates that there could be approximately 3.2 million HSA-covered members of the U.S. population between the ages of 19 and 64 who are not students and not enrolled in public health insurance programs. Depending upon the generosity of the assumed subsidy, additional simulations found that upwards of 12.8 million previously uninsured individuals could enroll in an HSA plan.

PIC ID: 8255; Agency Sponsor: ASPE-OHP, Office of Health Policy; Federal Contact: Cox, Donald, 202- 690-6597; Performer: University of Minnesota, Minneapolis, MN

Objective 3.2 - Strengthen and Expand the Health Care Safety Net

Evaluation of Health Resources and Services Administration (HRSA)’s Clinical Pharmacy Demonstration Projects

Future research may include an examination of the extent to which these services are cost-effective and effective relative to other competing “best practice” approaches.

PIC ID: 7959; Agency Sponsor: HRSA-BPHC, Bureau of Primary Health Care; Federal Contact: McGee, Kathy, 301-594-0318; Performer: Mathematica Policy Research, Inc., Plainsboro, NJ

Using Medicaid in Support of Working Age Adults with Serious Mental Illnesses in the Community: A Handbook

This Handbook is designed to improve understanding and provide greater clarity concerning Medicaid’s contribution in supporting working-age adults with serious mental illnesses in the community. The Handbook focuses on working-age adults between the ages of 21 and 64 with serious mental illnesses, whose need for support extends beyond mental health services that can be effectively provided by primary care physicians or periodic visits to outpatient settings.

http://aspe.hhs.gov/daltcp/reports/handbook.htm

PIC ID: 8209; Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy; Federal Contact: Kennedy, Cille, 202-690-6443; Performer: Research Triangle Institute, Research Triangle Park, NC

Objective 3.3 - Strengthen and Improve Medicare

Analysis of the Medicare Advantage Stabilization Fund

PIC ID: 8229; Agency Sponsor: ASPE-OHP, Office of Health Policy; Federal Contact: Cox, Donald, 202- 690-6597; Performer: Mathematica Policy Research, Inc., Washington, DC

Issues in the Design and Implementation of Drug Formularies and Therapeutic Classes

This report provided an examination of public and private sector formulary and classification schemes compared with USP Model Formulary and CMS formulary approval guidance. It used data from the Medicare Current Beneficiary Survey (MCBS) to develop a simulation model of the cost impacts of formulary changes. One finding was that without modification, some formularies that existed prior to the implementation of Part D would have failed CMS' rules for formulary approval. Another finding suggested that some formularies could have been approved although they did not include some popular drugs.

Neither finding accounted for the potential effect of market competition that would be expected to provide incentives for plans to offer richer benefit packages. The simulation model suggests that co- insurance provides patients with stronger behavioral incentives than does co-payments, and tiered cost sharing can strengthen those incentives.

PIC ID: 8254; Agency Sponsor: ASPE-OHP, Office of Health Policy; Federal Contact: Sheingold, Steven, 202-690-7387; Performer: National Opinion Research Center (NORC), Washington, DC

Objective 3.4 - Eliminate Racial and Ethnic Health Disparities

Racial, Ethnic and Primary Language Data: An Assessment of State Laws, Regulations and Practices Affecting Their Collection and Reporting by Health Insurers and Managed Care Plans

PIC ID: 7479; Agency Sponsor: OPHS-OMH, Office of Minority Health; Federal Contact: Woo, Violet Ryo- Hwa, 301-496-5343, and Welsh, Valerie, 240-453-8222; Performer: National Health Law Program, Washington, DC

Evaluation of the Office of Minority Health Resource Center (Final Report)

PIC ID: 7862; Agency Sponsor: OPHS-OMH, Office of Minority Health; Federal Contact: West, John I., 240-453-8833; Performer: Cosmos Corporation, Bethesda, MD

A Patient-Centered Guide to Implementing Language Access Services in Health Care Organizations

The purpose of this project was to develop a “Health Care Language Services Implementation Guide” (HCLSIG) that provides practical, action-oriented, ground-level suggestions and alternatives for how health care organizations can implement language access services for Limited English Proficiency (LEP) individuals, and that also serves as a companion piece to Office of Minority Health (OMH)’s National Standards for Culturally and Linguistically Appropriate Services (CLAS). This implementation guide was intended to meet the needs of a variety of physician practice settings and health care organizations, including community health centers, clinics, hospitals, mental health institutions, dental offices, long-term care facilities, substance abuse treatment centers, and the like. It was also intended to be patient- centered and family-focused, regardless of the type of clinical setting in which patients and families receive health care services. The guide was developed collaboratively with input from a wide range of experts representing the field of language services, provider groups and health care entities, and  patients. It contains four steps and six resource units that provide guidance on the process of implementing Language Access Services (LAS) at a health care organization. The first two steps involve assessment. The third step focuses on the planning, implementation, and evaluation of the various components of LAS. The fourth and final step provides a detailed discussion of specific methods for monitoring, evaluation, and improving the organization’s LAS. Each step and resource unit describes two case studies that depict a situation and reactions to the situation from patient and provider perspectives. The placement of the cases is designed to help readers apply what they have learned through the guide content. Placed at the beginning of each step and resource unit, Case 1 illustrates what an organization and its patients might experience in a setting where LAS are not well established. Case 1 is followed by a brief synopsis of the patient’s perspective as well as the provider’s perspective. Placed at the end of each step and resource unit, Case 2 illustrates what an organization and its patients might experience in a setting where LAS are better established as compared with Case 1. Case 2 is also followed by a brief synopsis of both the patient and provider’s perspectives. Each step and resource unit concludes with a summary of insights from the cases that illustrate the benefits of LAS for health care organizations. The guide provides emphasis on patient-centered care by highlighting the process a patient and his or her family members go through when seeking health care services, specifically at each point of contact throughout the delivery of a continuum of care. This guide should be beneficial to not only providers but to LEP patients as well.

PIC ID: 7864; Agency Sponsor: OPHS-OMH, Office of Minority Health; Federal Contact: Pacheco, Guadalupe, 240-453-2882; Performer: American Institutes for Research, Washington, DC

Identification of Variables that Influence Access to Eye Care

PIC ID: 8196; Agency Sponsor: NIH-NEI, National Eye Institute; Federal Contact: Janiszewski, Rosemary, 301-496-5248; Performer: National Eye Institute, Bethesda, MD

Evaluation of the Impact of a Computerized Screening Tool on Preventative Care and Research Recruitment

The goal of this project was to improve the quality of preventive health care and to increase recruitment in research trials, especially African-Americans, by using a computerized screening tool. By linking patient report information to medical records data, computerized technology was used to enhance the potential for identifying women eligible for research trials and of obtaining initial consent to contact. Patients received a personalized print out with advice on their preventive health needs and a report was generated for the patient chart. In addition, the patient and clinician were alerted to potential research  projects for which the patient may be eligible. Qualitative interviews and chart reviews were used to evaluate the impact of the screening, including acceptability to patients and clinic staff, impact on referrals/provision of preventive health care, and impact on research recruitment. The use of the computerized intake method increased assessment completion, identified more positives (particularly in the areas of intimate partner violence and depression), and increased provider response with counseling and referrals. Smoking and drug use were not as significantly affected by use of the computer except for counseling in pregnant women and written information to gynecology patients. The computerized intake process shows promise for supporting more complete assessments, identifying positive risk behaviors  with increased counseling and referrals.

PIC ID: 8241; Agency Sponsor: OPHS-OWH, Office of Women’s Health; Federal Contact: Newman, Eileen P., 301-443-1393; Performer: Magee-Women’s Hospital, NCE Women’s Health, Pittsburgh, PA

Objective 3.5 - Expand Access to Health Care Services for Targeted Populations with Special Health Care Needs

Evaluation Guidance for TB Programs

PIC ID: 7980; Agency Sponsor: CDC, Centers for Disease Control and Prevention; Federal Contact: Lobato, Mark, 404-639-8131; Performer: Research Triangle Institute, Research Triangle Park, NC

A Compendium of Intervention and Descriptive Studies Designed to Promote the Health of Caregivers for Older Adults

An important component of the Assistant Secretary for Planning and Evaluation (ASPE) Health Promotion and Aging Project is the development of an inventory of health promotion, disease prevention, and health education activities targeted at informal caregivers for HHS. The specific focus of this report is to  highlight some state and local initiatives that have developed promising programs to support, educate, and motivate caregivers of persons age 60 and older to maintain and improve their health and functioning.

http://aspe.hhs.gov/daltcp/CaregiverEvent/compendium.htm

PIC ID: 8077; Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy; Federal Contact: Frank, Andreas, 202-690-6443; Performer: RTI International, Research Triangle Park, NC

Recent Findings on Frontline Long-Term Care Workers: A Research Synthesis 1999-2003

This synthesis paper reviews, summarizes, and discusses the significance of available research findings on the front-line long-term care (LTC) workforce since 1999, in both home and community-based and nursing home settings. This paper provides an updated review of the status of empirical findings, focusing on what had been learned between 1999 and 2003. The primary goal was to learn what initiatives have worked to reduce LTC direct-care work force recruitment and retention problems. A secondary goal was to provide empirically based insights on the factors that contribute to recruitment and retention problems. This paper was intended to help policymakers, providers, worker and consumer groups, and researchers create a framework for future evidence-based policy, practice, and applied research initiatives to address LTC direct-care work force shortages.

http://aspe.hhs.gov/daltcp/reports/insight.htm

PIC ID: 8079; Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy; Federal Contact: Frank, Andreas, 202-690-6443; Performer: Institute for the Future of Aging Services, Washington, DC

State-Based Initiatives to Improve the Recruitment and Retention of the Paraprofessional Long-Term Care Workforce

The Department of Health and Human Services (HHS) awarded a contract to launch a national initiative designed to improve recruitment and retention of direct care workers in the long-term care field. As part of this initiative, the Institute for the Future of Aging Services and the Paraprofessional Health Care Institute have identified a range of workforce improvement initiatives at the state and sub-state level to reduce high vacancy and turnover rates among direct care workers, and improve the quality of their jobs. This report summarizes the experiences of five states—California, Massachusetts, North Carolina, Pennsylvania, and Wisconsin—that have pursued several strategies to address this issue to inform policymakers, long-term care providers, organized labor and other worker groups, and consumers across the country grappling with serious shortages of direct care workers.

http://aspe.hhs.gov/daltcp/reports/pltcwf.htm

PIC ID: 8080; Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy; Federal Contact: Frank, Andreas, 202-690-6443; Performer: Institute for the Future of Aging Services, Washington, DC

Evaluation of the Alzheimer’s Disease Demonstration Grants to States Program - First Phase

The Alzheimer’s Disease Demonstration Grants to States Program was established in 1991 with the goal of supporting individuals with Alzheimer’s disease and their family caregivers in their own homes and communities. The purpose of the first phase evaluation was to determine the effects on individuals with Alzheimer’s disease and their family caregivers, of the home and community-based supportive services that they receive through this program. The evaluation identifies differences in service effectiveness and consumer satisfaction based on racial, ethnic, and geographical diversity.

PIC ID: 8136; Agency Sponsor: AOA, Administration on Aging; Federal Contact: Stalbaum, Lori, 202-357- 3452; Performer: University of Kansas, Lawrence, KS

Evaluation of Health Info Web Resources for and about Specific Populations

PIC ID: 8193; Agency Sponsor: NIH-NLM, National Library of Medicine; Federal Contact: Love, Cindy, 301-496-5306; NIH, National Library of Medicine

Feasibility Study of the Optimal Approached for Evaluating the Cancer Disparities Research Partnership Program

This study examined and identified the most appropriate evaluation methodologies, techniques, and tools to measure the relevance, effectiveness, and impact of the Cancer Disparities Research Partnership (CDRP) Program. The study resulted in the identification of consistent evaluation methods to collect and analyze data over the life of the grants to effectively evaluate program progress and to produce meaningful annual reports to stakeholders of interest within and outside the National Cancer Institute (NCI). The evaluation plan focuses on two different sets of outcomes based on their timing. First, the CDRP Program logic model specifies implementation outcomes that are to be measured annually. Second, the logic model provides for summative outcomes that assess the short- and long-term results of a project and seeks to measure the changes brought about by the project. Justifying conclusions on the basis of evidence will include comparing predetermined measures of success or Program objectives with analysis and synthesis of information, interpretation of evidence, and recommendations for consideration. When appropriate, conclusions will be strengthened by summarizing plausible mechanisms of change, delineating temporal sequences between activities and effects, and showing that the effects can be repeated.

PIC ID: 8206; Agency Sponsor: NIH-NCI, National Cancer Institute; Federal Contact: Govern, Frank, 301- 496-6111; Performer: NOVA Research Company, Bethesda, MD

The Size of the Long-Term Care Population in Residential Care: A Review of Estimates and Methodology

This report is part of a project that will attempt to better understand the source of differences in estimates of the older population in residential care by analyzing key national surveys. In the report, the authors present results of a review of existing estimates of the number of residential care settings, generally divided into nursing homes and alternative residential care settings, and the number of persons residing in them. The review has three purposes. First, the authors describe the data that have been used to generate estimates and summarize the range of estimates. Second, they identify methodological issues that contribute to differences among estimates, focusing on those that can be investigated using available national surveys. Finally, the authors identify four surveys that appear to offer the best opportunity to conduct such an investigation, and to provide a better understanding of the size and characteristics of the residential care population and insights into how collection of data on residential care settings can be improved.

http://aspe.hhs.gov/daltcp/reports/ltcpopsz.htm

PIC ID: 8236; Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy; Federal Contact: Drabek, John, 202-690-6443; Performer: Urban Institute, Washington, DC

Objective 3.6 - Increase Access to Health Services for American Indians and Alaska Natives (AI/AN)

Revised Methodology to Adjust AI/AN Mortality Data to Compensate for Under Reporting of AI/AN Race on State Death Certificates

The project purpose is to replicate the previous study completed in November 1996 using current National Death Index (NDI) data and current Indian Health Service (IHS) patient registration data to develop   more current race reporting adjustment factors to use for state mortality record adjustment purposes. This study was completed in two phases. Phase I involved using the NDI, which is maintained by the National Center for Health Statistics (NCHS), to develop estimates for each Indian Health Service Area and for each reservation state or the proportion of American Indian and Alaska Native deaths that occur each year that are reported by state Vital Statistics Offices to the NCHS as non-Indian deaths. The data indicated which states have significant problems in under-reporting of Indian race on state death certificates, and thereby warrant the attention of IHS to mount corrective actions at the state and local levels. In the second phase of this project a contractor performed an analysis of the data obtained during the first phase. This analysis involved aggregation of data by the deceased’s age and sex, IHS Area Office and state of occurrence, and cause of death (cause of death is contained on records contained in the NDI file).

PIC ID: 7964; Agency Sponsor: IHS, Indian Health Service; Federal Contact: Melton, Debbie, 301-443- 4700; Performer: Indian Health Service, Rockville, MD

Violence and Suicide Reduction-Methodologic Training and Information Synthesis

This proposal responded to the Director’s Child and Adolescent Initiative and addressed five of the 21 critical objectives for youth in Healthy People 2010 objectives. Given the focus of the present application is capacity building for tribal leaders to collect information as it relates to self-directed and other directed violence, this proposal addressed the following 2010 objectives: (1) Reduce homicides from year 2000 baselines; (2) Reduce suicide attempts to 1 percent; (3) Reduce suicide rates; 4) Reduce the percent of youth involved in physical fights; 5) Reduce the percent carrying a weapon to school to 4.9 percent.

PIC ID: 7965; Agency Sponsor: IHS, Indian Health Service; Federal Contact: Melton, Debbie, 301-443- 4700; Performer: Indian Health Service, Rockville MD

Assessing the Effect of Health Care Delivery Among the Emerging Elders of Fort Peck

PIC ID: 7968; Agency Sponsor: IHS, Indian Health Service; Federal Contact: Melton, Debbie, 301-443- 4700; Performer: Indian Health Service, Rockville MD

Goal 4 - Enhance the Capacity and Productivity of the Nation’s Health Science Research Enterprise

Objective 4.1 - Advance the Understanding of Basic Biomedical and Behavioral Science and How to Prevent, Diagnose, and Treat Disease and Disability

National Eye Institute VISION Public Information Network Final Evaluation March 30, 2005

This feasibility study was a self-initiated evaluation request. The NEI has supported the VISION Public Information Network (Network) for a 10 year-period and it appears to be an appropriate time to confirm its value to the NEI and its grantee institutions. The National Eye Institute (NEI) coordinates the Network for Eye Institutes and departments of ophthalmology and schools and colleges of optometry. Network members appoint a Public Information Officer to work with NEI and the Network to develop an ongoing program to inform and educate the public about the benefits of vision research. A review of the  Network’s various reports and activities was undertaken, but limited to the following: the Network’s participation in the dissemination of clinical trial study results to the media, the amount of media coverage generated, the annual business meeting, survey reports from the Network membership, and website activity. Also, input from representatives of the numerous stakeholders had to be obtained.

These stakeholders included, but were not limited to the following: Network Members, Deans and Presidents of the Schools and Colleges of Optometry, and the Chairs of the Departments of Ophthalmology, Directors of Public Affairs Offices at NEI grantee institutions, NEI senior administrative staff, and Principal Investigators at NEI grantee institutions. In addition, a search for comparable programs within the Federal and state governments as well as in the private sector needed to be undertaken. Evaluation results were reviewed to determine whether an outcome or process evaluation is needed. This study will help in the design of a process and/or outcome evaluation to assess the programs operation and/or to assess the programs effects.

PIC ID: 8192; Agency Sponsor: NIH-NEI, National Eye Institute; Federal Contact: Horrigan, Jean, 301- 496-5248; Performer: GYMR Public Relations, Washington, DC

Diabetic Education in Tribal Schools (DETS) Evaluation Project Report

PIC ID: 8194; Agency Sponsor: NIH-NIDDK, Diabetes and Digestive and Kidney Diseases; Federal Contact: Garfield, Sanford A, 301-594-8803; Performer: Doug Coulson, Arnold, MD 21012

A Feasibility Study for the Evaluation of Parkinson’s Disease Research Centers: Assessment of Approaches and Development of an Evaluation Plan

In September 2004, the National Institute of Neurological Disorders and Stroke (NINDS) contracted with ORC Macro, Inc. to assess the feasibility of conducting a full-scale outcome evaluation of its original 11 Morris K. Udall Centers for Excellence in Parkinson’s Disease Research. The feasibility study explored potential questions that could be addressed in such an evaluation, and assessed the availability of the  data necessary to address these questions. At its conclusion, the contractors performing the study determined that an evaluation was feasible and articulated an approach for conducting such an  evaluation. This approach involved a conceptual framework for the anticipated function of the Udall Centers and a set of ten study questions focused on the selection process for the Centers; the administration of the Centers program by (NINDS); the achievement of short-term and long-term research and related (e.g., collaboration, sharing, etc.) goals by the Centers; an assessment of why some Centers were more successful than others; and a comparison of the Centers’ performance to that of other Parkinson’s disease researchers in the community (who are not affiliated with the Udall Centers).

PIC ID: 8200; Agency Sponsor: NIH-NINDS, Nat’l Institute of Neurological Disorders & Stroke; Federal Contact: Scott, Paul A, 301-496-9271; Performer: ORC Macro, Calverton, MD

Objective 4.3 - Strengthen and Diversify the Pool of Qualified Health and Behavioral Science Researchers

Feasibility Study for Evaluating Research Training in NIMH, NIDA and NINDS

PIC ID: 8195; Agency Sponsor: NIH-NIMH, National Institute of Mental Health; Federal Contact: Hann, Della, 301-443-4335; Performer: Westat, Inc., Rockville, MD

A- NLM Local Legends Usability Test Report Round 1; 11-19-04 & B-Final: NLM Local Legends Usability Test Report Round 2; 5-06-05

The Woman Physician Local Legends (WPLL) Web site was a new resource in 2004, and The National Library of Medicine (NLM) sought to assure that it is highly usable and accessible. The site has been developed in collaboration with Congress and the American Women’s Medical Association Foundation (AMWA). The site represents a collection of interesting physician bibliographies available as a searchable database, includes inspiring video/audio segments and a guest book feature which can serve as a collaboration tool for the nomination process of woman physicians, aiding the general public and those involved in the approval process of nominated candidates. A form-filling feature for congressional staff has also been implemented to enable electronic submissions of congressional nominations. Usability and Accessibility testing was conducted blending Needs Assessment with Focused Outcome Evaluation. A combination of the following evaluation methods was employed: (1) Semi-structured stakeholder interviewing; (2) Heuristic inspection of prototype features; (3) Structured scenario walkthroughs with design prototypes; (4) Pre- and post-usability testing surveys; (5) Analysis of statistical usage/tracking data for the parent Local Legends site; (6) Testing with assistive technology (AT) devices. The evaluation project produced several interface re-designs and additional site functionality. The redesigned site continues to help ensure that users can maintain site orientation and that the site meets or exceeds the US Access Board’s Section 508 Accessibility Compliance Standards. The site can be found at:

http://www.nlm.nih.gov/exhibition/locallegends/index.html.

PIC ID: 8198; Agency Sponsor: NIH-NLM, National Library of Medicine; Federal Contact: Ma, Wei, 301- 496-8436; Performer: UserWorks, Silver Spring, MD

Review of the Joint National Institutes of Health / National Science Foundation Ecology of Infectious Disease Program

http://www.fic.nih.gov/about/eid_review2005.pdf

PIC ID: 8203; Agency Sponsor: NIH-FIC, Fogarty International Center; Federal Contact: Kupfer, Linda, 301-496-3288; Performer: Abt Associates Inc., Cambridge, MA

Objective 4.4 - Improve the Coordination, Communication, and Application of Health Research Results

Genome.gov: Redesign Outcome Evaluation

This report summarized the results of an evaluation of the National Human Genome Research Institute (NHGRI)’s website performed in the last half of 2004. The purpose of the evaluation was to determine the effectiveness of the redesign that was launched in June 2004. Although the entire site was included in the evaluation, the “Research” and “Careers & Training” tiers were specifically targeted for review since they received the most radical restructuring in the redesign. The evaluation included feedback from both external visitors and internal staff members. Many internal staff members not only use the site regularly, they also provide the material for the site. Their participation in the ongoing curation of the site’s content was key to the success of the site. Major findings include the following strengths: (1) The new navigation scheme worked well for helping visitors find information; (2) The new layout in the “Careers & Training” section worked well; (3) The revamping of the “Research” section was mostly successful, but needed some adjustments. Major findings include the following weaknesses: (1) The extramural program pages were confusing for visitors; (2) External visitors noticed that the research tools hosted on a related server were out of date and poorly maintained; (3) There were ongoing difficulties in fully engaging internal staff members in the maintenance of the site’s content. The remainder of the report provides details for the findings, and recommendations for addressing them.

PIC ID: 8170; Agency Sponsor: NIH-NHGRI, National Human Genome Research Institute; Federal Contact: Bartlett, Maggie, 301-594-0632; Performer: Lynn Baumeister, Silver Spring, MD

An Application of MASSCsm to the Disclosure Treatment of Selected NCHS Survey Data

A Statistical Disclosure Limitation methodology known as MASSC was applied to two sets of data from the public use files of the third National Health and Nutrition Examination Survey and the 2000 National Health Interview Survey. For each set of data, an assessment of the ability of the MASSC methodology to reduce the risk of disclosure while minimizing the impact on data quality was carried out. Since public use files used in this study were previously treated in order to limit disclosure, it was not possible to determine the level of disclosure risk associated with each set of public use files. Accordingly, the pre- MASSC treatment level of “risk” should be regarded as being simulated and not be considered true risk. Under  this condition, the degree to which the MASSC treatment lowered this simulated risk was  measured. Pre-treatment estimates, including point estimates and regression estimates, were compared with the corresponding post-treatment quantities. The level of agreement between the pre- and post- treatment estimates was measured.

PIC ID: 8175; Agency Sponsor: OPHS, Office of Public Health and Science; Federal Contact: Harris, Kenneth; Performer: Research Triangle Institute, Research Triangle Park, NC

National Institutes of Health Usage and Attitudes Study

Purpose: Are health information seekers aware of the National Institute of Health (NIH)’s research mission? To what degree? How satisfied are health information seekers with NIH health content? How satisfied are users with the depth of health topics covered by NIH? Do users have different expectations for the NIH Health Information Pages than they do for commercial health Web sites? Methodology: Email survey fielded to Nielsen/NetRatings MegaPanel January, 2004: 35 percent response rate, Weighted n = 1,095. Data weighted based on Age, Income and Gender. Findings and Recommendations: Trust is clearly a key component to which sites consumers visit for health content. NIH could increase visitors’ trust of their content simply through online brand awareness campaign. Since health visitors find new sites mostly through search engines and links from other respected sites, it may be most effective to develop deeper partnerships with respected health related sites to further develop brand awareness.

Clearly, building traffic should be a key goal for NIH. Users who have visited NIH in the past month have a significantly better opinion of the quality of NIH content and have a much better understanding of NIH’s core message. NIH is doing an adequate job conveying its message to health content consumers. Improving the messaging is important for long-term success of NIH.gov.

PIC ID: 8197; Agency Sponsor: NIH-OD, Office of the Director; Federal Contact: Rodrigues, Dennis, 301- 435-2932; Performer: Nielsen/NetRatings, Haymarket, VA

Evaluation of the Health and Research Outcomes of Technologies Licensed by the National Institutes of Health

PIC ID: 8204; Agency Sponsor: NIH-OD, Office of the Director; Federal Contact: Elliott, Cerise L., 301- 435-5294; Performer: Research Triangle Institute, Waltham, MA

Modular Grants Application Process Outcome Evaluation

The program evaluated is the modular grant application process. This National Institutes of Health (NIH) policy was introduced on June 1, 1999. Designed to simplify the grant application process, the program was predicated on the belief that NIH provides research institutions with grant-in-aid for research endeavors, and therefore it is not necessary for NIH to have an accounting for all dollars spent, i.e. a detailed budget. By simplifying budgetary requests within an application, the initiative was designed to focus the attention of investigators, their institutions, peer reviewers and the NIH staff on the proposed research rather than on budgetary details. The modular program specifies that budget requests be made in modules that consist of $25,000 increments. In addition, long budgetary justifications are to be replaced with short descriptions of personnel needs. The modular grant program was developed as an extension of NIH’s streamlining and reinvention activities. The process is intended to provide an alternative to “cost analysis” by focusing review on scientific merit and disengaging from complex budgetary negotiations. It was designed to meet the following long-term goals: to focus the efforts of the investigators, institutional officials, and NIH staff on the science of the applications; to reduce administrative burden; to accommodate investigators’ need for flexibility; and to facilitate science and simplify administration. Through survey questionnaires of the various stakeholders, this evaluation assesses the effectiveness of the program in meeting these goals.

http://grants.nih.gov/grants/funding/modular/eval/index.htm

PIC ID: 8237; Agency Sponsor: NIH-OD, Office of the Director; Federal Contact: Schaffer, Walter, 301- 435-2770; Performer: Westat, Inc., Rockville, MD

Objective 4.5 - Strengthen the Mechanisms for Ensuring the Protection of Human Subjects and the Integrity of the Research Process

Review of the Fogarty International Center International Bioethics Education and Career Development Award Program

The Fogarty International Center’s (FIC) International Bioethics Education and Career Development  Award Program (referred to hereinafter as the “Bioethics program”), addresses an important need in clinical research in developing countries and has already made important contributions in its first five years of existence. Clinical research in developing countries is essential to better understand and treat    diseases whose burdens are disproportionately great in these regions. Ethical issues in research need to be resolved in order for scientists to gain the trust of the public and research participants. FIC conducts qualitative process evaluations of its programs every five years. In May 2005, the Fogarty International Center convened a panel of six experts in research ethics and curriculum development to review progress of the first five years of the Bioethics program and to make suggestions regarding its future. This review served as the first formal evaluation of the Bioethics program. With the program goals in mind, the panel spoke with Bioethics Principal Investigators (PIs), past and current program trainees, FIC program officer, National Institutes of Health (NIH) partners, and outside experts in bioethics and clinical research ethics. During these interviews, the panel explored the impact of the program with respect to FIC’s mission and program goals as well as the impact on developing country researchers and institutions in order to suggest improvements and modifications to all areas of the Bioethics program. In accordance with the Fogarty Evaluation Framework, the panelists explored five aspects of the program: Program Planning, Program Management, Partnerships and Program Results with regard to Re-entry and Post-training Activities, and Program Results with regard to program Outcomes and Metrics of Success. The panelists noted that even though it is relatively new, the trainee from the Bioethics program has published papers in high-impact journals, successfully obtained NIH funding, and established national ethics review committees. The panel strongly recommended the continuation and support of the Bioethics program, and suggested several modifications to strengthen its impact in the developing world and more closely align  its objectives with the goals of FIC and the other NIH partners. The review panel made nine formal recommendations on modifications and improvements to the Bioethics program.

http://www.fic.nih.gov/about/bioethics%20review.pdf

PIC ID: 8201; Agency Sponsor: NIH-FIC, Fogarty International Center; Federal Contact: Kupfer, Linda, 301-496-3288; Performer: Abt Associates Inc., Cambridge, MA

Evaluating the Efficacy of Informed Consent Forms and the Impact on Recruitment of Minorities into Clinical Research

PIC ID: 8240; Agency Sponsor: OPHS-OWH, Office of Women’s Health; Federal Contact: Newman, Eileen P., 301-443-1393; Performer: Drexel University - Med School NCE Women’s Health, Philadelphia, PA

Goal 5 - Improve the Quality of Health CARE Services

Objective 5.2 - Increase the Appropriate Use of Effective Health CARE Services by Medical Providers

The Value of Hospital Discharge Databases

The objectives for this contract are: (1) To obtain encounter-level data from statewide information sources; (2) To maintain and expand a uniform, multi-state health care database for health services research and health policy analysis; and (3) To make these data available to a broad set of public and private users. This contract will extend the capabilities of the Health Care Cost and Utilization Project (HCUP) national data system to assist researchers and decision-makers at the national, state, and community levels by expanding HCUP to include more states and more settings of care, and to develop tools to make better use of administrative data.

http://www.hcup-us.ahrq.gov/reports/final_report.pdf

PIC ID: 7694; Agency Sponsor: AHRQ-CODS, Center for Organization and Delivery Studies; Federal Contact: Schnaier, Jenny, 301-594-6827; Performer: Medstat Group, Washington, DC

Prevention of Surgical Wound Infections

Key objectives were to measure adherence to CDC's SSI Guideline recommendations using eight measures, three related to antibiotic prophylaxis; two, preoperative glucose control; two, surgical site preparation; and one, hand hygiene. RTI performed quantitative and qualitative data collection. Overall, adherence rates are not high, with the exception of type of antibiotic agent used. The degree of compliance was also strongly related to the consequences of post-surgical infection. Steps should be taken to improve compliance with both antimicrobial prophylaxis and perioperative glucose control, and also to standardize site preparation and hand hygiene protocols.

PIC ID: 7877; Agency Sponsor: CDC, Centers for Disease Control and Prevention; Federal Contact: Dietz, Sue, 404-639-0210 Performer: Research Triangle Institute, Research Triangle Park, NC

Objective 5.3 - Increase Consumer and Patient Use of Health CARE Quality Information the Past, Present and Future

The Past, Present and Future of Managed Long-Term Care

This study assesses the state of the managed long-term care market from the perspective of purchasers (states) and suppliers (managed long-term care contractors), addressing the following questions: (1) What is the current state of the managed long-term care market? (2) What value do managed long-term care products offer relative to the fee-for-service system? (3) What is the market outlook in terms of future demand from state purchasers and supply from existing and new organizations? [33 PDF pages]

http://aspe.hhs.gov/daltcp/reports/mltc.htm

PIC ID: 8174; Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy; Federal Contact: McKay, Hunter, 202-690-6443; Performer: Edmund S. Muskie Institute of Public Affairs, Bethesda, MD

Summary report on Needs Assessment of the National Institute of Arthritis and Musculoskeletal and Skin Diseases (NIAMS) Pediatric Rheumatic Diseases CD- ROM

NIAMS developed a CD-ROM to provide pediatric rheumatic diseases health care professionals with information in an easy-to-use format so they could better serve their patients. NIAMS contracted with Circle Solutions, Inc. to assess the content, presentation, and usability of the CD-ROM. The study design entailed mailing the CD and one of two different feedback forms to reviewers, and direct observations of 11 reviewers. An array of health care professionals reviewed the CD-ROM, particularly health care professionals involved in pediatric rheumatology. The reviewers’ consensus was that the CD-ROM provides convenient access to helpful literature. They particularly liked the utility of the print-friendly publications, Web site links, the Arthritis Foundation materials, and the information from the professional reference guides. Several reviewers recommended improvements in navigation: (1) Have documents open in a smaller view; (2) Have an easier means to return to the CD-ROM from a PDF document or Web site; (3) Update the NIAMS publication order page Web site link; (4) Improve the primary means of navigation (welcome page or bookmarks); (5) Make the links more visible in certain sections; (6) incorporate a keyword search function; (7) Make the “Tips for using this CD-ROM” briefer.

PIC ID: 8207; Agency Sponsor: NIH-OD, Office of the Director; Federal Contact: Peterman, Wilma, 301- 496-8271; Performer: Circle Solutions, McLean, VA

Objective 5.5 - Accelerate the Development and Use of an Electronic Health Information Infrastructure

Understanding Our Users: How to Better Deliver Health Information Online to American Indians and Alaska Natives

The Office of Disease Prevention and Health Promotion (ODPHP) conducted this qualitative study to build user-focused Web sites. This report presents findings from September 2002 using research conducted to better understand American Indian and Alaska Native (AIAN) online health information seekers. ODPHP seeks to provide reliable health information for a variety of audiences through  http://www.healthfinder.gov/. Launched in 1997, www.healthfinder.gov is a portal site, linking the general public to quality health information resources. Known for being user friendly,  http://www.healthfinder.gov/is continuously evaluated by its users through interviews, usability tests, online comments, and expert reviews. In fall 2002, ODPHP focused on expanding resources for AIAN populations. Some goals of the study were to (1) Assess the range of health-information-seeking behaviors in the AIAN population by identifying who their authoritative information sources are and discover when they would use health information; (2) To identify tasks that AIAN individuals want to accomplish on a health information Web site; (3) To understand how healthfinder® can best serve the AIAN population; (4) To investigate the overall usability of healthfinder®. According to the results of each goal, it was found that all AIAN interviewees had used computers before, although their skills varied. Intermediaries were asked the question, “How can healthfinder® best serve AIAN populations?” Most respondents were very pleased that ODPHP was conducting these interviews. Face-to-face contact is extremely important when working with AIAN communities. It shows commitment and builds trust.

Web quality is judged by the ease with which diverse users enter the electronic environment and accomplish their goals. Various strategies (such as Advisory/Review boards) should be examined for reviewing the content of health-related Web sites to ensure that it provides these populations with the information they need.

PIC ID: 7823; Agency Sponsor: OPHS-ODPHP, Office of Disease Prevention and Health Promotion; Federal Contact: Hsu, Leslie, 202-401-0732; Performer: Office of Disease Prevention and Health Promotion, Washington, DC

Understanding Our Users: How to Better Deliver Health Information Online to Asian-Americans, Native Hawaiians, and Other Pacific Islanders

http://odphp.osophs.hhs.gov/projects/aanhopi/aanhopi.pdf

PIC ID: 7823.1; Agency Sponsor: OPHS-ODPHP, Office of Disease Prevention and Health Promotion; Federal Contact: Hsu, Leslie, 202-401-0732; Performer: HHS, Department of Health and Human Services, Washington, DC

Pregnancy Risk Assessment Monitoring System (PRAMS) Program Evaluation

The objectives were to inform the operational, analytic, translation, and capacity-building functions of the current PRAMS system and make them more efficient, effective, and capable of meeting future needs; to provide information that will guide the expansion and support of additional state PRAMS programs; and to provide information that will enable the PRAMS system to be more responsive to changes in public health priorities and policies, including the needs of the state programs and the wider Maternal and Child Health Block Grant (MCH) community. The findings from this evaluation show that PRAMS is a strong and vibrant surveillance program facing a range of programmatic challenges. Many of the study’s recommendations have already been incorporated into program operations.

PIC ID: 7976; Agency Sponsor: CDC-NCCDPHP, National Center for Chronic Disease Prevention and Health Promotion; Federal Contact: Dietz, Sue, 404-639-0210; Performer: Research Triangle Institute, Research Triangle Park, NC

Case Studies of Electronic Health Records in Post-Acute and Long-Term Care

This study focused on electronic health records (EHRs) and information systems used to exchange clinical information between providers. The only clinical information systems that were identified that exchanged information across settings that include post-acute and long-term care (PAC/LTC) providers were health delivery systems that were highly integrated, due to either their geographic location (large, rural health systems) or financial structure. Sites selected for this study were sites that used EHRs in the course of providing care and also used their EHRs to exchange clinical information with other health care providers. To learn more about the most advanced EHR systems in PAC/LTC, case studies were conducted at four sites with connectivity to acute hospitals, pharmacies, and/or ambulatory care. The goal was to examine and describe sites representing the various PAC/LTC settings, and organizational and payment arrangements.

http://aspe.hhs.gov/daltcp/reports/ehrpaltc.htm

PIC ID: 8075; Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy; Federal Contact: Harvell, Jennie, 202-690-6443; Performer: University of Colorado, Denver, CO

Evaluation of Confidentiality Protection and Data Utility for NCHS Public Use Microdata Files

The purpose of this evaluation was to perform combined studies and to compare analytical outputs on two publicly released microdata files selected from the National Center for Health Statistics (NCHS) families of data systems. One file involved establishments and the other involved persons. For each selected file, the contractor developed and applied several standard methods and combinations of methods to reduce the identifiability of microdata records and the risk of disclosure of confidential information. Evaluations were based on quantitative methods.

PIC ID: 7874; Agency Sponsor: CDC, Centers for Disease Control and Prevention; Federal Contact: Harris, Kenneth, 301-458-4262; Performer: Research Triangle Institute, Research Triangle Park, NC

Evaluation of the Role of Epi Info in Public Health

Epi Info is a series of public domain computer programs for rapid questionnaire design, data entry, statistical analysis, mapping, and graphing designed for use by public health professionals. Recent events related to bioterrorism have brought to the forefront the need for flexible data gathering, management, and analysis software in the hands of communities where new monitoring systems may be necessary within hours. The goals of this evaluation are to provide information on: (1) The impact of Epi Info use on domestic and international public health; (2) The degree of Epi Info use, as well as relevant informatics needs and current technological access capabilities among public health professionals; (3) Improved development methods and an ongoing system for evaluation in the future, with defined outcomes and performance criteria. Information will be used to inform the future software development.

PIC ID: 7871; Agency Sponsor: CDC, Centers for Disease Control and Prevention; Federal Contact: Yoon, Steven, 404-639-8331; Performer: Booz Allen Hamilton, McLean, VA

Goal 6 - Improve the Economic and Social Well- Being of Individuals, Families, and Communities, Especially Those MOST in Need

Objective 6.1 - Increase the Proportion of Low-income Individuals and Families, Including Those Receiving Welfare, WHO Improve Their Economic Condition

Indicators of Welfare Dependence: Annual Report to Congress 2005

The 2005 Indicators of Welfare Dependence report is the eighth annual report that addresses the extent to which Americans are dependent on social welfare programs. The report provides welfare dependence indicators through 2002, reflecting changes that have taken place since enactment of the Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA) in August 1996. The report uses a variety of data sources, including the Survey of Income and Program Participation and the Current Population Survey, Annual Demographic Supplement, analyzed using the TRIM3 micro-simulation model. These indicators show a significant decline in welfare dependency since 1996, concurrent with the more widely reported declines in welfare recipiency and poverty. In 2002, 3.2 percent of the total population was dependent, in that they received more than half of their total family income from Temporary Assistance for Needy Families (TANF), food stamps, and/or Supplemental Security Income (SSI). While marginally higher than the 3.1 percent dependency rate measured in 2001, the 2002 rate is much lower than the 5.2 percent rate measured in 1996. Overall, 4.7 million fewer Americans were dependent on welfare in 2002 compared to 1996.

http://aspe.hhs.gov/hsp/indicators05/

PIC ID: 7281.7; Agency Sponsor: ASPE-OHSP, Office of Human Services Policy; Federal Contact: Isaacs, Julia, 202-690-7507; ASPE, Office of Human Services Policy

Linking TANF Recipients with Paraprofessional Long-Term Care Jobs

http://aspe.hhs.gov/daltcp/reports/linkTANF.htm

PIC ID: 8173; Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy; Federal Contact: Liggins, Charlene, 202-690-6443; Performer: Mathematica Policy Research, Inc., Washington, DC

TANF Recipients as Potential Long-Term Care Workers: An Assessment of the Prospects in the District of Columbia, Illinois, Maryland and South Carolina

The long-term care (LTC) industry’s need for workers and Temporary Assistance for Needy Families (TANF) recipients’ need for jobs could be mutually beneficial if recipients’ characteristics, skills, and circumstances match the requirements, accessibility, and availability of LTC jobs. In this paper, the authors provide a brief overview of what is already known about the characteristics of long-term care jobs and the individuals who hold them, focusing on nurse aides and home health aides. They use survey data from the District of Columbia and three states (Illinois, Maryland, and South Carolina) to examine  the characteristics and circumstances of all single parent TANF case heads, and of those recipients who are or have recently been employed in the long-term care industry. Finally, the authors use multivariate analysis to predict the likelihood of employment among current TANF recipients and use the results to estimate the proportion of the current TANF caseload that could potentially be employed in the long-term care industry and describe their characteristics. They conclude this paper with a brief discussion of the potential policy and/or program initiatives that this analysis suggests.

http://aspe.hhs.gov/daltcp/reports/TANFltc.htm

PIC ID: 8173.1; Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy; Federal Contact: Liggins, Charlene, 202-690-6443; Performer: Mathematica Policy Research, Inc., Washington, DC

Potential Employment Liabilities Among TANF Recipients: a Synthesis of Data From Six State TANF Caseload Studies

PIC ID: 8178; Agency Sponsor: ASPE-OHSP, Office of Human Services Policy; Federal Contact: Hauan, Susan, 202-690-8698; Performer: ASPE, Office of the Assistant Secretary for Planning and Evaluation, Office of Human Services Policy, Washington, DC

Public Assistance Use Among Two-Parent Families: an Analysis of TANF and Food Stamp Program Eligibility and Participation

States increasingly are focusing on the Temporary Assistance for Needy Families (TANF) goals relating to promoting and supporting healthy marriage. To understand how state policies may affect marriage, it is useful to first examine the extent to which existing public assistance programs provide benefits to both married-parent and single-parent families. Specifically, this recent study by Mathematica Policy Research (MPR) examines patterns of married- and single-parent families’ eligibility and participation in the (TANF) program and the Food Stamp Program (FSP). Results are based on data from the 2001 Current Population Survey as well as micro-simulation results from the TRIM3 and MATH models. Among the key findings are: (1) low-income married-parent families are less likely than single-parent families to be eligible for public assistance programs, and once eligible, are less likely to participate; (2) FSP and TANF caseloads have declined over time for both married- and single-parent families, but the reasons differ between the two; and (3) differences between married- and single-parent family participation rates persisted even after controlling for demographic and economic circumstances, suggesting that these family types make fundamentally different decisions about public assistance.

http://aspe.hhs.gov/hsp/05/2parent-part/report.pdf

PIC ID: 8179; Agency Sponsor: ASPE-OHSP, Office of Human Services Policy; Federal Contact: Kaye, Kelleen, 202-401-6634; Performer: Mathematica Policy Research, Inc., Princeton, NJ

Maternity Group Homes Classification and Literature Review

The Office of Human Services Policy (OHSP) in the Office of the Assistant Secretary for Planning and Evaluation (ASPE) contracted with Mathematica Policy Research, Inc. (MPR) to conduct a study of how Maternity Group Home programs operate and to explore options for studying them further. This first report of three provides an overview of past research and develops a preliminary classification framework for Maternity Group Homes. Maternity Group Homes offer an array of on-site social support services for the most at-risk teen families, using trained staff members to provide nurturing, structured guidance.

There is some limited evidence suggesting that Maternity Group Homes may be effective in addressing subsequent teen pregnancies, school-related outcomes, self-sufficiency, and childhood immunization rates. By exploring the history of Maternity Group Homes, identifying program components and service delivery methods, and describing management and funding mechanisms, this report sets the stage for further exploration of the range of Maternity Group Homes currently in the field.

http://aspe.hhs.gov/hsp/grouphomes04/litreview04/report.pdf

PIC ID: 8180; Agency Sponsor: ASPE-OHSP, Office of Human Services Policy; Federal Contact: Trivits, Lisa, 202-205-5750; Performer: Mathematica Policy Research, Inc., Princeton, NJ

The Implementation of Maternity Group Home Programs: Serving Pregnant and Parenting Teens in a Residential Setting

The Office of Human Services Policy (OHSP) in the Office of the Assistant Secretary for Planning and Evaluation (ASPE) contracted with Mathematica Policy Research, Inc (MPR) to conduct a study of how Maternity Group Home programs operate and to explore options for studying them further. Maternity Group Homes offer an innovative and intensive approach to addressing the needs of teenage mothers and their children. This second report of three documents the implementation of maternity group homes in seven programs chosen because they had the highest potential for inclusion in a random assignment evaluation. The MPR researchers conducted two-day site visits to each program, visiting up to five homes per program to tour the facilities, meet with key staff, and observe program activities. The report addresses research questions on management structure, funding sources, population characteristics, the range of services provided, and staffing. Key findings indicated that the homes: (1) tended to be  operated by larger organizations with broader social service missions; (2) were funded by a combination of federal and state monies and private donations; (3) served a highly disadvantaged population with many special needs; (4) provided intensive, comprehensive support programs for pregnant and parenting teens; (5) often required a large staff.

http://aspe.hhs.gov/hsp/grouphomes04/imp05/report.pdf

PIC ID: 8180.1; Agency Sponsor: ASPE-OHSP, Office of Human Services Policy; Federal Contact: Trivits, Lisa, 202-205-5750; Performer: Mathematica Policy Research, Inc., Princeton, NJ

Designing a Rigorous Evaluation of a Maternity Group Home Program

The Office of Human Services Policy (OHSP) in the Office of the Assistant Secretary for Planning and Evaluation (ASPE) contracted with Mathematica Policy Research, Inc (MPR) to conduct a study of how Maternity Group Home programs operate and to explore options for studying them further. This third and final report discusses the feasibility of evaluating the effectiveness of existing maternity group home programs using a random assignment research design. The contractor developed a set of criteria to identify and select promising programs for study. Two researchers visited the site programs for two to three days to gather information about the program operations and service delivery. Georgia’s Second Chance Homes program, operated by the Georgia Campaign for Adolescent Pregnancy Prevention (GCAPP), was determined to be the only viable candidate for such a study. Therefore, the report focuses on a comprehensive evaluation plan for the GCAPP program model. The evaluation plan includes an implementation and in-depth study of participants’ life circumstances. It also includes an impact study that examines the success of the program in improving key outcomes. Finally, the evaluation assesses the cost of serving teens in the program compared to the costs of serving them in alternative settings.

PIC ID: 8180.2; Agency Sponsor: ASPE-OHSP, Office of Human Services Policy; Federal Contact: Trivits, Lisa, 202-205-5750; Performer: Mathematica Policy Research, Inc., Princeton, NJ

Meta-Analysis of Welfare-to-Work Programs

(2)  Time limits tended to reduce caseloads, but had no clear impact on earnings; (5) Impacts appeared  to have been greater for those more disadvantaged in terms of recent work history and time spent receiving benefits; (6) Positive impacts on recipients appear to have been greatest in the first couple of years after treatment; (7) For children, impacts on behavioral and emotional outcomes were less positive for school-age children than for younger children, though all impacts were, on average, very modest. Some caveats to keep in mind in interpreting the results of this study include: (1) The analysis was limited in the number of explanatory variables that could be included, and this could have biased the results in ways that cannot be known; (2) In particular, only variables which indicated whether a service was received and not variables indicating the intensity of service were included; (3) These results are mostly based on conditions under the Aid to Families and Dependent Children (AFDC) program and may not be applicable to the current Temporary Assistance to Needy Families (TANF) environment; (4) A meta-analysis combines data from different demonstrations which may not be strictly comparable.

http://www.acf.hhs.gov/programs/opre/welfare_employ/meta_analysis/reports/report_w  tw/meta_title.html

PIC ID: 8210; Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation; Federal Contact: Sternbach, Leonard, 415-437-7671; Performer: University of Maryland, School of Social Work, Baltimore, MD

Barriers to Subsidies: The Influence of Family Characteristics on Child Care Subsidies Study II: Child Care Preferences and Study III: Child Care Quality and Preferences

http://www.temple.edu/psychology/FacultyWebs/Weinraub/Links/Factorial Survey Draft - Final to ECRQ - July 2004.doc

PIC ID: 8211; Agency Sponsor: ACF-ACYF, Administration on Children, Youth and Families; Federal Contact: Martinez-Beck, Ivelisse, 202-690-7885; Performer: Temple University, Philadelphia, PA

From Welfare to Child Care: What Happens to Young Children When Single Mothers Exchange Welfare for Work- Infant and Toddler Care After Welfare Reform: A Cross State Comparison

The study found descriptive evidence from Miami-Dade County (MDC), Florida and from five representative areas in Massachusetts, that government policies governing welfare reform, the child care subsidy system, and minimum-standards regulations had considerable impact on the availability, use, quality, and price of infant and toddler care, as welfare reform progressed from 1996 to 2000. During this period, more than double the number of low-income infants and toddlers using child care subsidies were placed in formal (licensed) care in MDC. This was likely related to a welfare reform policy in Florida requiring cash assistance recipients with children 3-months or older to engage in work-related activities. There was also evidence during this period, in order to meet state minimum-standards regulations, that child care centers in MDC and Massachusetts had to find other sources of funding for their infant and toddler programs, since neither the prices providers charged families, nor the reimbursements providers received from the state, covered the full costs of providing care for these age groups.

http://www.wellesley.edu/Economics/partner/Witte_Queralt_2004.pdf

PIC ID: 8214; Agency Sponsor: ACF-ACYF, Administration on Children, Youth and Families; Federal Contact: Martinez-Beck, Ivelisse, 202-690-7885; Performer: National Bureau of Economic Research, Cambridge, MA

A Snapshot of Quality in Minnesota's Child Care Centers

This study assessed the quality of center-based care in Minnesota using environmental ratings scales and surveys with directors and classroom teachers. The study found overall quality of preschool classrooms below the “good” benchmark on the Early Childhood Environment Rating Scale, Revised Edition. Areas of particular concern were personal care routines (health, safety, meals, toileting) and the variety and accessibility of activities (art, music, math, science). The correlates of high quality classrooms were a higher percentage of staff with Bachelor’s degrees, lower staff turnover, accreditation by the National Association for the Education of Young Children (NAEYC), and low child-staff ratios. Directors identified resources that would be helpful for improving quality. Findings from this study can inform policy decision- making in Minnesota including identifying targets for child care quality improvement initiatives.

http://edocs.dhs.state.mn.us/lfserver/Legacy/DHS-423-ENG

PIC ID: 8215; Agency Sponsor: ACF-ACYF, Administration on Children, Youth and Families; Federal Contact: Martinez-Beck, Ivelisse, 202-690-7885; Performer: Minnesota Department of Human Services, St. Paul, MN

Child Care Assistance and the Market for Child Care in Minnesota

The study analyzes trends in the price of child care in Minnesota using data from market rate surveys from 1998 through 2004. The approach uses descriptive statistical analysis to compare trends in prices by age, type of care, and region of the state. Trends in the dispersion or variation of prices within regions are also compared. The study found that child care rates in Minnesota have increased about five to six percent per year over the time period, not adjusting for inflation. This rate of increase is similar to the rate of price increases for educational services nationwide (based on the Consumer Price Index). Price trends were similar across age groups and types of care, except for hourly rates of family child care providers outside the Minneapolis-St Paul metro area, which barely rose over the time period. Rising child care prices may have implications for the affordability of child care for families and the type of care they use.

http://edocs.dhs.state.mn.us/lfserver/Legacy/DHS-4234-ENG

PIC ID: 8216; Agency Sponsor: ACF-ACYF, Administration on Children, Youth and Families; Federal Contact: Martinez-Beck, Ivelisse, 202-690-7885; Performer: Minnesota Department of Human Services, St. Paul, MN

Early Care and Education Partnerships: State Actions and Local Lessons

This report presents the findings from a three-year study of a randomly selected sample of 141 child care providers in Ohio. Half of the providers participated in partnerships with Head Start and half served as comparison centers. The report describes the nature of partnerships, including details about the financial resources, supervision, training, and professional development provided to child care centers. The report also provides details about differences in child care teachers who are employed by partnering centers and comparison centers. Differences in parents’ perceptions of services and quality are also presented. The study found that child care centers in partnership with Head Start are more likely than comparison centers to: (A) Offer screenings, referrals, and services such as developmental screenings, referrals to medical services, and other services, such as physical therapy; (B) Offer more services to parents, such as GED classes or adult literacy classes; (C) Offer job benefits, such as vacation and sick time to  teachers; (D) Have lower ratios of children to teachers; (E) Offer a structured curriculum to children.

http://ccf.edc.org/PDF/EDC_ExecBrief.pdf

PIC ID: 8217; Agency Sponsor: ACF-ACYF, Administration on Children, Youth and Families; Federal Contact: Martinez-Beck, Ivelisse, 202-690-7885; Performer: Education Development Center, Newton, MA

Under-Reporting of Medicaid and Welfare in the Current Population Survey

http://www.acf.hhs.gov/programs/opre/welfare_employ/under_cps/reports/under_rep/u nder_title.html

PIC ID: 8222; Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation; Federal Contact: Sternbach, Leonard, 415-437-7671; Performer: Rand Corporation, Santa Monica, CA

The Employment Retention and Advancement Project - Early Results from Four Sites

This report presents early evidence on the effectiveness of four diverse programs that aim to help current or former welfare recipients maintain stable employment and increase their earnings. The four programs are part of the Employment Retention and Advancement (ERA) project, which is testing 15 such programs across the country. The evaluation design is similar in most of the programs. Individuals who meet the ERA eligibility criteria (which vary by program) are randomly assigned to an experimental group or a control group. Members of the experimental group are recruited for - and, in some programs, required to participate in - the ERA services, while those in the control group are not eligible for ERA services. The extent and nature of services and supports available to the control group vary, but it is important to note that, in most cases, the ERA group is not being compared with a “no services” control group. The four programs discussed in this report, like virtually all the other ERA programs, have faced a substantial challenge in increasing participation in ERA services. These early findings reflect a short follow-up period, only one year after individuals entered the four programs. The early effects on employment outcomes are mixed: some of the ERA programs appear to be promoting retention or advancement, while others seem to be less successful.

http://www.acf.hhs.gov/programs/opre/welfare_employ/employ_retention/rep  orts/early_results/early_title.html

PIC ID: 8223; Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation; Federal Contact: Richards, Patrice, 202-205-8324; Performer: MDRC, New York, NY

The Evaluation Data Coordination Project: Common Constructs and Measures Across Nine ACF Studies and Other Key Data Collection Efforts

http://www.acf.hhs.gov/programs/opre/other_resrch/eval_data/reports/common_constru cts/common_constructs.pdf

PIC ID: 8263; Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation; Federal Contact: Yaffe, Alan, 202-401-4537; Performer: American Institutes for Research, Washington, DC

The Welfare Rules Data Book: State Policies as of July 2002

The State Temporary Assistance for Needy Families (TANF) Policies: Welfare Rules Database Expansion project provided resources to facilitate updating and expanding the Urban Institute’s Welfare Rules Database (WRD). The Urban Institute had begun developing the WRD in early 1997, as part of the Assessing New Federalism project. The database was conceived as a single location where information on program rules could be researched across States and/or across years, without the need to consult multiple documents, and it was intended to provide a resource for researchers working on both descriptive and quantitative projects. As of the end of 2000, the Urban Institute had released information on eligibility and benefits rules for 1996 through 2000. This project funded updates to the database for 2001 and 2002. The database is available to the public online at:

http://anfdata.urban.org/drsurvey/login.cfm?CFID=142812&CFTOKEN=83213734.

The reports from the project were three Welfare Rules Data books, which provide summary tables of selected information on state TANF policies as of July of the years 2000, 2001, and 2002.

http://www.acf.hhs.gov/programs/opre/welfare_employ/state_tanf/reports/wel_rules02/ wel02_rules.pf

PIC ID: 8228; Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation; Federal Contact: Yaffe, Alan, 202-401-4537; Performer: The Urban Institute, Washington, DC

Objective 6.2 - Increase the Proportion of Older Americans WHO Stay Active and Healthy

National Survey of Older Americans Act Participants Using Indicators Developed under the Performance Outcomes Measures Project

The purpose of this project was to conduct a statistically valid national survey of recipients of services under the Older Americans Act using survey questions designed in concert with states and Area agencies on aging. Questions were field-tested by states under the Administration on Aging (AoA)’s current performance outcomes measures project. Westat conducted telephone surveys on a sample drawn with the cooperation of approximately 150 area agencies nationwide. Two national surveys have been conducted, focusing on recipients that received specific services funded under the Older Americans Act. A third survey is being finalized at this time.

PIC ID: 7650; Agency Sponsor: AOA, Administration on Aging; Federal Contact: Greenberg, Saadia, 202- 357-3516; Performer: Westat, Inc., Rockville, MD

State Residential Care and Assisted Living Policy: 2004

This compendium describes regulatory provisions and Medicaid policy for residential care settings in all 50 states and the District of Columbia. It updates an earlier report completed in 2002. Information was collected between February and July 2004 by reviewing state websites and regulations and calling key state contacts to verify information. Section 1 provides an overview of residential care and assisted living policy. Section 2 presents six tables, which compare states’ policies in selected areas. Section 3 provides summaries of each state’s regulations and policy for residential care settings, including assisted living facilities. [499 PDF pages]

http://aspe.hhs.gov/daltcp/reports/04alcom.htm

PIC ID: 8171; Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy; Federal Contact: Kennedy, Gavin, 202-690-6443; Performer: Research Triangle Institute, Research Triangle Park, NC

Supportive Services Programs in Naturally Occurring Retirement Communities

http://aspe.hhs.gov/daltcp/reports/NORCssp.htm

PIC ID: 8172; Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy; Federal Contact: Kennedy, Gavin, 202-690-6443; Performer: Urban Institute, Washington, DC

Objective 6.3 - Increase the Independence and Quality of Life of Persons with Disabilities, Including Those with Long-term CARE Needs

Evaluability Assessment of Discharge Planning and the Prevention of Homelessness

The Assistant Secretary for Planning and Evaluation (ASPE) of the Department of Health and Human Services (HHS) supported this study in order to build knowledge for researchers and policy makers regarding the evaluability of discharge planning. The specific focus of this study was on discharge planning as a homeless prevention strategy in institutional and custodial settings including: (1) Adult inpatient psychiatric treatment units in state psychiatric, private psychiatric, or general hospitals; (2) Residential treatment centers serving children and youth with serious emotional disturbances and/or substance use disorders; (3) Residential treatment programs for adults with substance use disorders; (4) foster care independent living programs. The project had three goals: (1) To identify key evaluation questions both by setting and across settings; (2) To determine the feasibility of conducting a rigorous and objective evaluation; (3) To develop alternative evaluation designs. The approach included a literature review, the use of an expert panel, documentary analysis of selected exemplary discharge planning programs, site visits to the exemplary programs, the development of logic models related to the programs, and a final report. Study findings indicated that a rigorous outcome evaluation is not justified at this time. However, several alternative study designs to evaluate issues or activities related to discharge planning and homelessness prevention are feasible and are described.

PIC ID: 8187; Agency Sponsor: ASPE-OHSP, Office of Human Services Policy; Federal Contact: Halpern, Peggy, 202-260-0285; Performer: Westat, Inc., Rockville, MD

Does Consumer Direction Affect the Quality of Medicaid Personal Assistance in Arkansas?

http://aspe.hhs.gov/daltcp/reports/arqual.htm

PIC ID: 6161; Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy; Federal Contact: Doty, Pamela, 202-690-6443; Performer: Boston College, Social Welfare Research Institute, Chestnut Hill, MA

The Effect of Consumer Direction of Personal Assistance Received in Arkansas

The traditional system of providing Medicaid personal care services (PCS) through home care agencies gives consumers few choices about how and when their care is provided. Because some agencies do not provide care on weekends or outside business hours, consumers may not receive care when they need it. Moreover, worker shortages make it difficult to ensure that beneficiaries receive all the care they are entitled to. This study of Arkansas’ Cash and Counseling Demonstration program, IndependentChoices, examined how this model of consumer direction changes the way that consumers meet their personal assistance needs.

http://aspe.hhs.gov/daltcp/reports/Arkpa.htm

PIC ID: 8054.1; Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy; Federal Contact: Doty, Pamela, 202-690-6443; Performer: Boston College, Social Welfare Research Institute, Chestnut Hill, MA

Easing the Burden of Caregiving: The Effect of Consumer Direction on Primary Informal Caregivers in Arkansas

Cash and Counseling, an innovative model of consumer direction, has been implemented as a three-state demonstration designed to weigh the advantages and disadvantages of allowing Medicaid beneficiaries to assume more responsibility for the supportive services they may need. This paper presents estimates of program effects on primary informal caregivers (those providing the most unpaid supportive services at baseline) from Arkansas’s Cash and Counseling demonstration, the first of three such demonstrations to enroll and randomly assign Medicaid beneficiaries to direct their own services or to rely on agencies as usual. Arkansas’s consumer-directed program, known as IndependentChoices, was designed for elderly and non-elderly adults.

http://aspe.hhs.gov/daltcp/reports/easing.htm

PIC ID: 8054; Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy; Federal Contact: Doty, Pamela, 202-690-6443; Performer: Boston College, Social Welfare Research Institute, Chestnut Hill, MA

Enabling Personal Preference: The Implementation of the Cash and Counseling Demonstration in New Jersey

This report describes the design and implementation of Personal Preference, New Jersey’s model of Cash and Counseling. It also draws lessons from the state’s experience. The report is based primarily on in- person interviews conducted in April 2001, about 18 months after the program began enrolling beneficiaries (November 1999). Interviews were conducted with New Jersey state officials, Personal Preference program staff members, officials of organizations representing the personal care industry in New Jersey, and staff members of organizations providing outreach, enrollment, consulting, and fiscal services under Personal Preference.

http://aspe.hhs.gov/daltcp/Reports/enablepp.htm

PIC ID: 8054.2; Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy; Federal Contact: Doty, Pamela, 202-690-6443; Performer: Boston College, Social Welfare Research Institute, Chestnut Hill, MA

Medicaid Costs Under Consumer Direction for Florida Children with Developmental Disabilities

The randomized design of the evaluation of the national Cash and Counseling Demonstration enables researchers to rigorously analyze costs under traditional and consumer-directed approaches. In this report, we use results from Florida’s Consumer-Directed Care for children to investigate the program’s effect on Medicaid waiver costs, other Medicaid costs, and Medicaid service use under consumer-directed and traditional programs for the two-year period after demonstration enrollment.

http://aspe.hhs.gov/daltcp/reports/FLddkids.htm

PIC ID: 8054.5; Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy; Federal Contact: Doty, Pamela, 202-690-6443; Performer: Mathematica Policy Research, Inc., Princeton, NJ

Do Consumer-Directed Medicaid Supportive Services Work for Children with Developmental Disabilities?

The evaluation of Cash and Counseling is the first to use a randomized design to compare service use and quality under traditional versus consumer-directed approaches to Medicaid supportive services. As part of the evaluation, this paper presents results for the children and parents who participated in Florida's Cash and Counseling demonstration program, Consumer Directed Care.

http://aspe.hhs.gov/daltcp/reports/ddkidsMss.htm

PIC ID: 8054.6; Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy; Federal Contact: Doty, Pamela, 202-690-6443; Performer: Mathematica Policy Research, Inc., Princeton, NJ

The Effect of Cash and Counseling on Medicaid and Medicare Costs: Findings for Adults in Three States

Recent research suggests that Florida’s Cash and Counseling model—Consumer-Directed Care - increased the well-being of children with disabilities and their parents in Florida, and the Cash and Counseling programs in Arkansas, Florida, and New Jersey similarly increased the well-being of adults. The program also increased the likelihood that consumers in all three states would receive paid personal care services. However, the results to date on costs are less clear-cut. Arkansas’s Cash and Counseling program increased personal care costs for adults, but, because of savings on other Medicaid services, the higher costs were partially offset during the first post-enrollment year and almost fully offset during the second. Florida’s program for children increased Section 1915(c) waiver costs, although savings on home health services partly offset the higher costs. This report compares results from all three demonstration programs to examine how consumer direction for adults affects Medicaid and Medicare service use and costs.

http://aspe.hhs.gov/daltcp/reports/3stcost.htm

PIC ID: 8054.7; Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy; Federal Contact: Doty, Pamela, 202-690-6443; Performer: Mathematica Policy Research, Inc., Princeton, NJ

Lessons from the Implementation of Cash and Counseling in Arkansas, Florida, and New Jersey

This paper is the first to consider the Cash and Counseling program in all three states. It draws lessons about the structure and policies of a Cash and Counseling program, to provide information useful to other states considering adopting such a program. It draws on the experience of those responsible for the demonstration and evaluation at the Cash and Counseling National Program Office, the Assistant  Secretary for Planning and Evaluation (ASPE), the Robert Wood Johnson Foundation (RWJF), the Centers for Medicare & Medicaid Services (CMS), the three participating states, and Mathematica Policy Research (MPR). Evaluation staff visited each of the three Cash and Counseling programs about 18 months after each began to operate, and these visits provide an important source of data.

http://aspe.hhs.gov/daltcp/reports/cclesson.htm

PIC ID: 8073; Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy; Federal Contact: Doty, Pamela, 202-690-6443; Performer: Boston College, Social Welfare Research Institute, Chestnut Hill, MA

Objective 6.4 - Improve the Economic and Social Development of Distressed Communities

Rural Research Needs and Data Sources for Selected Human Services Topics

The main goal of this study was to develop an inventory of databases that researchers could use to study selected human services-related conditions and the accessibility and utilization of human services in rural areas. Three human services issues were focal topics for the project - work supports for low-income families, substance abuse, and child welfare. The study found that: (1) There is no single, standardized definition that designates populations and places as rural or urban; (2) In contrast to urban areas, less is known about human and social services conditions in rural areas, the social services rural residents need and use, and the effectiveness of those services; (3) One of the major difficulties in conducting rural research is finding suitable data; (4) Several steps could be taken by researchers or their sponsors to strengthen rural data and research. The final report provides a descriptive overview of the social and economic conditions in rural America, including persistent and emerging trends, as well as descriptions (e.g., content, rural identifiers, access restrictions) of 20 federal and nonfederal and 60 state administrative data sources that could be used by rural researchers and policymakers to study one or more of the selected focal conditions.

http://aspe.hhs.gov/hsp/05/rural-data/index.htm

PIC ID: 8188; Agency Sponsor: ASPE-OHSP, Office of Human Services Policy; Federal Contact: Halpern, Peggy & McCormick, Ann; Performer: Mathematica Policy Research, Inc., Princeton, NJ

Objective 6.5 - Expand Community and Faith- Based Partnerships

State and Local Contracting for Social Services Under Charitable Choice - Final Report

http://aspe.hhs.gov/hsp/05/contracting/

PIC ID: 8184; Agency Sponsor: ASPE-OHSP, Office of Human Services Policy; Federal Contact: Landey, Alana & Tambornino, John; Performer: Mathematica Policy Research, Inc., Washington, DC

Goal 7 - Improve the Stability and Healthy Development of Our Nation’s Children and Youth

Objective 7.1 – Promote Family Formation and Healthy Marriages

Service Delivery and Evaluation Design Options for Strengthening and Promoting Healthy Marriages

Increasing access to marriage education services is a major objective within the Healthy Marriage Initiative. A major goal of this project was to provide Administration for Children and Families (ACF) with better information about the range of existing programs, and to identify opportunities for potential expansion of services to low-income couples and individuals. In the course of the project, the researchers conducted telephone discussions with key staff in 58 different programs and conducted visits to 28 programs. The report presents the authors’ assessment of key factors to understanding how marriage education programs are implemented, in terms of the setting and organizational structure, the intervention (approach, format, curriculum), and population served. In examining potential opportunities for expansion, the authors suggest two possible approaches, each with challenges: expanding the number of marriage education programs to include low-income couples, or enabling and encouraging programs otherwise serving low-income couples to incorporate marriage education into their traditional work. Both would need to build internal capacity to address the new situation - addressing the needs of a new target population for marriage education programs, and building knowledge and skills related to marriage education among staff within other service programs. They suggest that ACF consider a framework for expansion that combines the expertise of providers currently offering marriage education services with that of providers with expertise in serving low-income populations, in order to build on both sets of providers’ strengths.

http://www.acf.hhs.gov/programs/opre/strengthen/serv_delivery/index.html

PIC ID: 8224; Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation; Federal Contact: Campbell, Nancye, 202-401-5760; Performer: The Urban Institute, Washington, DC

Systematic Review of the Impact of Marriage and Relationship Programs

http://www.acf.hhs.gov/programs/opre/strengthen/serv_delivery/index.html

PIC ID: 8225; Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation; Federal Contact: Campbell, Nancye, 202-401-5760; Performer: The Urban Institute, Washington, DC

Objective 7.2 - Improve the Development and Learning Readiness of Preschool Children

Child Care for Working Poor Families: Child Development and Parent Employment Outcomes

This study assessed types and quality of care used by low-income working families and explored links between children’s developmental and parents’ employment outcomes and child care quality. The researchers conducted parent focus groups, interviews with community child care leaders, parent and caregiver questionnaires, and structured observations and assessments of 307 children in their child care settings on a volunteer non-random sample in four counties in Indiana. The most common types of child care used were licensed centers (38 percent), licensed family child care homes (24 percent), child care ministries (16 percent), Head Start (nine percent), unlicensed family child care (eight percent), and relative care (five percent). Twenty percent of children started in child care soon after birth and more than 75 percent of children were in some type of child care by age eight months. Low-income working parents reported the primary reason for using child care was to work or attend school. Lack of flexibility in child care and work schedules were reported as a barrier to work. Overall, average global quality was rated below “good,” and 25 percent of classrooms or homes were below “minimal” quality. Child care for preschoolers was of higher quality than child care for infants and toddlers. Many children in this sample scored below national norms in areas of cognitive and language competence. In general, there were few significant links between child care quality and parent employment outcomes but there was scattered evidence that families whose children were enrolled in higher-quality care had more stable employment patterns.

http://www.cfs.purdue.edu/cccrp/

PIC ID: 8212; Agency Sponsor: ACF-ACYF, Administration on Children, Youth and Families; Federal Contact: Martinez-Beck, Ivelisse, 202-690-7885; Performer: Purdue University, West Lafayette IN

The Cost and Quality of Full-Day Year-Round Early Care and Education in Massachusetts: Infant and Toddler Classrooms

This study conducted observations on quality and interviewed directors in a random sample of 100 infant classrooms and 100 toddler classrooms in licensed community centers in Massachusetts. The researchers found that one-quarter of infant classrooms and 31 percent of toddler classrooms provided care that met professional standards for infant/toddler programs. Centers that served predominantly low-income or low- to-moderate income families were less likely to receive the level of early care and education that will prepare them for school and later life, with toddlers in low-income centers at the greatest risk. Parent fees were the most important source of revenues for centers serving low-to-moderate, moderate, and high-income families. Government subsidies were an important revenue source for centers serving low- income families. Labor costs were the single largest component of center expenditures. Higher-quality infant and toddler care costs significantly more than lower-quality infant/toddler care. Classrooms with better child: staff ratios, more experienced and better educated teachers provided better quality care overall, including more developmentally-appropriate stimulation, and better relationships between classroom staff and children. Centers serving different income groups varied considerably in the quality of care they provided. While qualified teachers are clearly an important part of quality early care and education, center directors reported that it was difficult to hire qualified teachers. Data collected in this study provide evidence that higher quality early care and education is associated with greater costs.

http://www.wcwonline.org/earlycare/index.html

PIC ID: 8213; Agency Sponsor: ACF-ACYF, Administration on Children, Youth and Families; Federal Contact: Martinez-Beck, Ivelisse, 202-690-7885; Performer: Wellesley Centers for Women, Wellesley, MA

Assessing the Quality of Child Care Using Longitudinal, Administrative Data: What Can It Tell Us and How Can It Be Used?

PIC ID: 8218; Agency Sponsor: ACF-ACYF, Administration on Children, Youth and Families; Federal Contact: Martinez-Beck, Ivelisse, 202-690-7885; Performer: National Bureau of Economic Research, Cambridge, MA

Head Start Impact Study: First Year Findings

In 1998, as part of the Head Start reauthorization, Congress mandated that the U.S. Department of Health and Human Services (HHS) conduct a national study of the impact of Head Start programs to determine, on a national level, how participation in Head Start affects the school readiness of the children it serves, and identify the types of participants and under what circumstances the program is most effective. The Head Start Impact Study is being conducted across 84 nationally representative grantee/delegate agencies. Approximately 5,000 newly entering 3- and 4-year-old children applying for Head Start were randomly assigned to either a Head Start group that had access to Head Start program services or to a non-Head Start group that could enroll in available community non-Head Start services, selected by their parents. This report examines outcomes for children after one year in the program and compares children who received Head Start services to children who were assigned to a control group.

The report quantifies the impact of Head Start separately for 3- and 4-year-old children across child cognitive, social-emotional, and health domains as well as on parenting practices. For children in the 3- year-old group, the preliminary results from the first year of data collection demonstrate small to moderate positive effects favoring the children enrolled in Head Start for some outcomes in each domain. There were fewer positive impacts found for children in the 4-year-old group.

http://www.acf.hhs.gov/programs/opre/hs/impact_study/reports/first_yr_finds/firstyr_fi nds_title.html

PIC ID: 8226; Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation; Federal Contact: Woolverton, Maria, 202-205-4039; Performer: Westat, Rockville, MD

Role of Early Head Start Programs in Addressing the Child Care Needs of Low- Income Families with Infants and Toddlers

http://www.acf.hhs.gov/programs/opre/ehs/ehs_resrch/reports/role_ehs_cc/role_ehs_cc_title.html

PIC ID: 8252; Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation; Federal Contact: Cohen, Rachel Chazan, 202-205-8810; Performer: Mathematica Policy Research, Inc., Princeton, NJ

Health And Disabilities Services in Early Head Start: Are Families Getting Needed Health Care Services?

The report describes the health status and incidence of health-related problems among 3001 children enrolled in Early Head Start (EHS) research in 17 diverse sites. The data were collected in conjunction with the EHS Research and Evaluation project, conducted by Mathematica Policy Research, Columbia University National Center for Children and Families, and the EHS Research Consortium of researchers from 15 universities, when children were 14, 24, and 36 months of age, and periodically after enrollment into the research project. Immunization rates for EHS children were significantly higher than for control group children, who did not receive EHS. EHS children were also more likely to have visited a doctor for treatment of illness, and were less likely to have visited the emergency room due to accident or injury. Incidence of asthma and respiratory problems were high. Children exposed to household smoking were more likely to have asthma problems, with 57 percent of EHS children exposed to household smoke in all. Most parents used safety precautions, but parents needed more information on poison control measures. Almost all parents used car seats for infants. However, car seat use declined for EHS toddlers, as parents were not always able to replace car seats as children grew. A number of EHS programs have instituted programs for exchanging car seats as children grow larger. Of the women who enrolled during  pregnancy, women enrolled in EHS were more likely to breast feed their infants. Hispanic children fared worse, on average, than African-American and white EHS children. Hispanic children were less likely to have health insurance and regular health care providers than African-American or white children, and Hispanic parents were more likely to report their children were in fair or poor health.

http://www.acf.hhs.gov/programs/opre/ehs/ehs_resrch/reports/ehs_disabilities/ehs_dis_ title.html

PIC ID: 8253; Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation; Federal Contact: Cohen, Rachel Chazan, 202-205-8810; Performer: Mathematica Policy Research, Inc., Princeton, NJ

Objective 7.4 - Increase the Percentage of Children and Youth Living in a Permanent, Safe Environment

Formative Evaluation of the National Academic Centers of Excellence in Youth Violence

PIC ID: 7859; Agency Sponsor: CDC, Centers for Disease Control and Prevention; Federal Contact: Dietz, Sue, 404-639-0210; Performer: Battelle, Arlington, VA

Child Welfare Outcomes 2002: Annual Report

Child Welfare Outcomes 2002: Annual Report to Congress analyzes data on outcomes for children who come into contact with state child welfare systems. Analyses of state performance data suggest that many states are improving performance, particularly with regard to reducing maltreatment of children in foster care by foster parents or facility staff members, achieving permanency for children in foster care, and ensuring placement stability for children during the first 12 months that they are in foster care.

However, many states experience challenges in reducing the incidence of maltreatment recurrence, as well as in finding permanent homes for children who have a diagnosed disability; children who enter foster care when they are adolescents; children who are African-American; and children who are Alaska Native/Native American. Many states are successful in reunifying children in a timely manner, but this success is tempered by the fact that rapid reunification tends to be positively correlated with re-entry into foster care. There was an increase in the number of States achieving adoptions within 24 months of a child’s entry into foster care. Efforts to achieve timely adoptions are often impeded by the reluctance of some judges and agency staff to seek termination of parental rights (TPR), and the lengthy legal procedures to achieve a TPR. Finally, variations in state performance may be influenced by state differences in demographics and other contextual factors.

http://www.acf.hhs.gov/programs/cb/pubs/cwo02/index.htm

PIC ID: 8154.1; Agency Sponsor: ACF-ACYF, Administration on Children, Youth and Families; Federal Contact: Newburg-Rinn, Sharon, 202-205-0749; Performer: Caliber Associates, Fairfax, VA

Re-reporting and Recurrence of Child Maltreatment: Findings from NCANDS

http://aspe.hhs.gov/hsp/05/child-maltreat-rereporting/

PIC ID: 8182; Agency Sponsor: ASPE-OHSP, Office of Human Services Policy; Federal Contact: Radel, Laura, 202-690-5938; Performer: Walter R. McDonald & Associates, Inc., Rockville, MD

Alternative Responses to Child Maltreatment: Findings from NCANDS

Alternative response systems are a new and quickly growing innovation in the child maltreatment field. More defined by what they are not (investigations) than by what they are, states have implemented a range of alternative response models. In general, these models are intended to focus on family strengths and needs and take a more service-oriented approach than do investigations, which are primarily intended to determine forensically whether or not a particular incident or circumstance occurred. While the first alternative response systems emerged less than a decade ago, in some states the majority of child abuse and neglect reports are no longer investigated, but rather are referred to these new response models. And while a few early studies of individual states show promising results, to date there have been no cross-state studies of these programs. This study uses data from six states’ submissions to the National Child Abuse and Neglect Data System (NCANDS) to begin understanding, across states that use these systems, which children no longer receive investigations and whether those children are more or less likely than others to receive services, be placed in foster care, or to be the subject of subsequent maltreatment reports. The analysis described here was conducted by the staff of Walter R. McDonald and Associates, under contract to the Assistant Secretary for Planning and Evaluation (ASPE), and in cooperation with the Administration for Children and Families (ACF).

http://aspe.hhs.gov/hsp/05/child-maltreat-resp/index.htm

PIC ID: 8183; Agency Sponsor: ASPE-OHSP, Office of Human Services Policy; Federal Contact: Radel, Laura, 202-690-5938; Performer: Walter R. McDonald & Associates, Inc., Rockville, MD

Understanding Adoption Subsidies: An Analysis of AFCARS Data

Using data from the federal Adoption and Foster Care Analysis and Reporting System (AFCARS), this report explores patterns of adoption subsidy receipt, how subsidies are related to adoption outcomes such as the rate of adoptions among eligible children and how quickly eligible children are adopted. Questions of interest include how extensively subsidies are used and how they are funded, the relationships between children’s characteristics, foster care experiences, and subsidy receipt and amount, and variations among states in subsidy practice. Three types of analyses are presented: (1) Descriptive analyses of both national trends and variation among states; (2) Correlations among state-level measures, examining relationships among state subsidy practices and adoption outcomes; (3) Multivariate analyses addressing the relationship of the child, family, and state characteristics to subsidy receipt and subsidy amount.

http://aspe.hhs.gov/hsp/05/adoption-subsidies/index.htm

PIC ID: 8189; Agency Sponsor: ASPE-OHSP, Office of Human Services Policy; Federal Contact: Radel, Laura, 202-690-5938; Performer: Research Triangle Institute, Research Triangle Park, NC

Understanding Foster Parenting: Using Administrative Data to Explore Retention

Foster homes are a critical resource within the child welfare system, with more than 260,000 children in non-relative foster care at the end of FY2001. Child welfare agencies are continually challenged to  provide adequate numbers of foster homes that are stable, can accommodate sibling groups, and are located in proximity to family members. However, research on foster parent retention is surprisingly slender, with little known about the length of time served by foster parents and the characteristics associated with varying length of service. This study was designed to extend current understanding of foster parent retention by producing unbiased estimates of length of service and examining factors associated with licensure, provision of care, and length of service. The study used administrative data, applying data management and analytical methods that have previously been used by researchers to describe the length of stay for children in foster care. Principal research questions include: How have the characteristics of foster parents changed over time? How can variations in activity levels be described, and what foster parent characteristics are associated with varying activity levels? What is the typical length of service for foster parents? What characteristics are associated with variations in length of foster parent careers?

http://aspe.hhs.gov/hsp/05/foster-parenting/index.htm

PIC ID: 8190; Agency Sponsor: ASPE-OHSP, Office of Human Services Policy; Federal Contact: Radel, Laura, 202-690-5938; Performer: Research Triangle Institute, Research Triangle Park, NC

Synthesis of Findings: Assisted Guardianship Child Welfare Waiver Demonstrations

PIC ID: 8219; Agency Sponsor: ACF-ACYF, Administration on Children, Youth and Families; Federal Contact: Collins, Gail E., 202-205-8087; Performer: James Bell Associates, Inc., Arlington, VA

Synthesis of Findings: Substance Abuse Child Welfare Waiver Demonstrations

Since 1996, four states have implemented child welfare waiver demonstration projects focusing on the needs of families with substance abuse problems. These waiver demonstration projects allow states to spend federal foster care funds more flexibly in order to improve services and outcomes for children and families. Three states - Delaware, Maryland, and New Hampshire - focused on early identification of parents with substance use disorders and referral to existing treatment resources, seeking to prevent removal of children from the home. One state, Illinois, has used intensive case management and supportive services to support the recovery of caregivers whose children have already been removed from the home and placed in foster care. An independent evaluator evaluated each of the state projects. This report reviews the process and outcome findings from each of those evaluations and presents crosscutting lessons learned. Among the findings is that some states were successful in improving the engagement and retention of parents in substance abuse treatment and in decreasing lengths of stay in foster care for children removed from home. However, none of the state demonstrations has yet been successful either in reducing the rates of removal or in increasing overall permanency rates for children in foster care. Other lessons learned include the need to strengthen the skills of frontline child welfare workers to identify and assess substance abuse, and the need to have strong managerial support and consistent communication in order to foster successful collaboration between child welfare and substance abuse service providers.

http://www.acf.hhs.gov/programs/cb/programs_fund/cwwaiver/substanceabuse/index.ht  m

PIC ID: 8220; Agency Sponsor: ACF-ACYF, Administration on Children, Youth and Families; Federal Contact: Collins, Gail E., 202-205-8087; Performer: James Bell Associates, Inc., Arlington, VA

Synthesis of Findings: Title IV-E Flexible Funding Child Welfare Waiver Demonstrations

http://www.acf.hhs.gov/programs/cb/programs_fund/cwwaiver/flexfund/index.htm

PIC ID: 8221; Agency Sponsor: ACF-ACYF, Administration on Children, Youth and Families; Federal Contact: Collins, Gail E., 202-205-8087; Performer: James Bell Associates, Inc., Arlington, VA

Report on Baseline Findings from the National Survey of Child and Adolescent Well-Being: CPS Sample Component

Goal 8 - Achieve Excellence in Management Programs

Objective 8.5 - Enhance the Use of Electronic Commerce in Service Delivery and Record Keeping

Compendium for the Evaluation of Performance of the Medical Expenditure Panel Survey (MEPS): Final Report

The purpose of the project was to evaluate key aspects of the Medical Expenditure Panel Survey (MEPS) in terms of data quality, cost, and timeliness. The evaluation focuses on areas in which there have been programmatic concerns, especially those in which the data now exists for evaluation, which would be implemented in a timely way to inform quality enhancements for MEPS. Where appropriate, the evaluation identifies priorities for redesign, and outlines potential cost-effective alternatives. The final report is in three volumes.

PIC ID: 7401; Agency Sponsor: AHRQ, Agency for Healthcare Research and Quality; Federal Contact: Hagan, Michael, 301-594-1354; Performer: Research Triangle Institute, Research Triangle Park, NC

Objective 8.6 - Achieve Integration of Budget and Performance Information

A Review of the Population Research Infrastructure Program (PRIP)

The Population Research Infrastructure Program (PRIP) is an ongoing program providing support for centers. It is National Institute of Child Health and Human Development (NICHD) policy that such programs undergo evaluation before they are renewed for another 5-year cycle. Because the PRIP was established only in 2000, it is not yet mature enough for an outcome evaluation. The process evaluation examined the extent to which supported programs have implemented activities that are likely to lead to the fulfillment of program goals. The evaluation also provided early indicators of program productivity that can be used as a benchmark in later evaluations.

PIC ID: 8124; Agency Sponsor: NIH-NICHD, National Institute of Child Health and Human Development; Federal Contact: Bachrach, Christine, 301-496-9485; Performer: Rose Li and Associates, Inc., Washington, DC

Appendix a – HHS FY 2004-2009 Strategic Goals and Objectives

“Preventing Disease and Illness”

“Protecting Our Homeland”

“Closing the Gaps in Health Care”

“Improving Health Science”

“Realizing the Possibilities of 21st Century Health Care"

“Working Toward Independence”

“leaving No Child Behind”

“Improving Department Management”

Appendix B – Agency Mission and Evaluation Program Statements

Administration for Children and Families

Mission

To promote the economic and social well-being of families, children, individuals, and communities.

Evaluation Program

The Administration for Children and Families (ACF) administers a broad range of formula and discretionary programs, including family self-sufficiency (Temporary Assistance for Needy Families), child support, child care, children and family services (Head Start, Child Welfare, Family Preservation and Support, and youth programs), and special programs for community development and for targeted populations, such as the developmentally disabled, refugees, and Native Americans.

ACF’s evaluation objectives are to: furnish information on designing and operating effective programs; test new service delivery approaches capitalizing on the success of completed demonstrations; apply evaluation data to policy development, legislative planning, budget decisions, program management, and strategic planning and performance measures development; and disseminate findings of completed studies and promote application of results by state and local governments.

ACF stays current on emerging issues affecting its programs and identifying questions for evaluation studies by actively engaging other federal agencies, state and local policy and program officials, national organizations, foundations, professional groups and practitioners, and consumers. Of primary concern are systems changes and how they affect vulnerable populations, particularly children. The movement toward devolving responsibility for health and human services to state and local organizations offers both tremendous opportunities and unprecedented challenges in redefining and implementing services for families.

Studies are often funded as joint ventures with the Office of the Assistant Secretary for Planning and Evaluation (ASPE) and other federal agencies and foundations. Such collaborations permit large-scale efforts that are better informed and more representative of varying perspectives. Multidisciplinary experts review proposals. Evaluation study designs are carefully developed in collaboration with project partners and technical experts in order to address specific research questions. Work groups of various kinds are used to monitor the progress of projects and to advise on design refinements and the presentation of findings.

Administration on Aging

Mission

To foster the development of services to help older persons maintain their independence.

Evaluation Program

The Administration on Aging (AoA) is the federal focal point and advocate agency for the concerns of older persons. The agency administers key federal programs mandated under various titles of the Older Americans Act. These programs help vulnerable older persons remain in their own homes by providing supportive services. Other programs offer opportunities for older Americans to enhance their health and to be active contributors to their families, communities, and the nation through employment and volunteer programs. AoA works closely with its nationwide network of regional offices and state and area agencies on aging to plan, coordinate, and develop community-level systems of services that meet the unique needs of individual older persons and their caregivers. AoA collaborates with other federal agencies, national organizations, and representatives of business to ensure that, whenever possible, their programs and resources are targeted to the elderly and coordinated with those of the network on aging. As the responsibilities of this nationwide network of state and area agencies on aging continue to grow, it is essential that they have the necessary information to meet these responsibilities.

The overall evaluation priorities of the AoA are to support studies that provide information on: (1) Successful program implementation in meeting the goals of the Older Americans Act; (2) Design and operation of effective programs; (3) Issues relevant to policy development, legislative planning, and program management.

Agency for Healthcare Research and Quality

Mission

To improve the quality, safety, efficiency and effectiveness of health care for all Americans.

Evaluation Program

Evaluation activities within the Agency for Healthcare Research and Quality (AHRQ) provide executive management, program officers and audiences external to the Agency with evaluative findings concerning the Agency’s effectiveness and efficiency in meeting its Government Performance Results Act (GPRA), Program Assessment Rating Tool (PART), and other performance goals. The work is conducted by external, independent evaluators and complies with the Office of Management and Budget’s (OMB) Paperwork Reduction Act requirements. Evaluation components are built into virtually all major AHRQ programmatic or portfolio activities beginning at the design phase. Among evaluation mechanisms employed by the Agency are targeted evaluation studies undertaken via contract that use a variety of quantitative and qualitative methods and that tend increasingly to provide more real-time monitoring feedback. Evaluation activities also include satisfaction feedback from AHRQ customers regarding the usefulness of its research findings and dissemination products.

Agency for Toxic Substances and Disease Registry

Mission

To serve the public by using the best science, taking responsible public health actions, and providing trusted health information to prevent harmful exposures and disease related to toxic substances.

Evaluation Program

The Comprehensive Environmental Response, Compensation, and Liability Act (CERCLA), more commonly known as Superfund, created the Agency for Toxic Substances and Disease Registry (ATSDR) as a federal agency. ATSDR was created to implement the health-related sections of CERCLA and other laws that protect the public from hazardous waste and environmental spills of hazardous substances. The ATSDR evaluation program is coordinated with the HHS-wide strategic planning process to implement requirements of the Government Performance and Results Act (GPRA), the Program Assessment Rating Tool (PART), and the President’s Management Agenda (PMA). ATSDR’s strategic goals and its annual performance plan are the result of an interactive process that reflects a long-term commitment by  Agency staff to develop stronger relationships among external clients and stakeholders, to assess products and services using relevant data, and to improve our processes and systems for more efficient accomplishment of its mission.

Centers for Disease Control and Prevention

Mission

To promote health and quality of life by preventing and controlling disease, injury, and disability.

Evaluation Program

The Centers for Disease Control and Prevention (CDC) conducts evaluation studies designed to provide essential information about its programs, goals, and priorities. These projects support the assessment of CDC’s strategies, which are to protect the health and safety of Americans provide credible information to enhance health decisions, and promote health through strong partnerships.

CDC emphasizes evaluations that answer policy, program, and strategic planning questions related to the goals and objectives of Healthy People 2010. Performance improvement studies, such as those focusing on the development of key performance indicators consistent with the Government Performance and Results Act (GPRA) and the Office of Management and Budget’s Program Assessment Rating Tool (PART) are of particular interest and import to the Agency.

In addition, CDC supports a variety of activities to enhance evaluation quality, use, and understanding. An example of one such activity completed during FY2004 was a study that looked at case reporting of HIV, AIDS, STD, TB, and Hepatitis B and C in tribally operated health facilities.

Centers for Medicare & Medicaid Services

Mission

To assure health care security for beneficiaries.

Evaluation Program

The research arm of the Centers for Medicare & Medicaid Services (CMS), the Office of Research, Development, and Information (ORDI), performs and supports research and evaluations of demonstrations (through intramural studies, contracts and grants) to develop and implement new health care financing policies and provide information on the impact of CMS’ programs. ORDI’s activities embrace all areas of health care: costs, access, quality, service delivery models, and financing. ORDI’s responsibilities include evaluations of ongoing Medicare and Medicaid programs and demonstration projects testing new health care financing and delivery approaches.

Examples of research themes include state program flexibility, the future of Medicare, provider payment and delivery, and vulnerable populations and dual-eligibles.

Food and Drug Administration

To protect the public health by assuring the safety, efficacy, and security of human and veterinary drugs, biological products, medical devices, our nation’s food supply, cosmetics, and products that emit radiation; to advance public health by helping to speed innovations that make medicines and foods more effective, safer, and affordable; and helping the public get the accurate, science-based information they need to use medicines and foods to improve their health.

Evaluation Program

The Food and Drug Administration (FDA) evaluates programs it is responsible for consistent with the goals established and promulgated by the HHS strategic performance planning process. FDA uses its  own strategic framework to accomplish these goals. This process also satisfies the implementation requirements of the Government Performance and Results Act (GPRA) and the Food and Drug Administration Modernization Act of 1997 (FDAMA). The strategic and performance process is an evolving set of program directions as changes occur in FDA’s dynamic environment. Meeting these challenges, now and in the future, will rest on its ability to leverage its efforts in that environment, which grows increasingly complex and more institutionally networked. The Agency strives to maintain the scientific knowledge base necessary to achieve greater effectiveness in assuring the quality and availability of the products it regulates.

Health Resources and Services Administration

Mission

To provide national leadership, program resources and services needed to improve access to culturally competent, quality health care.

Evaluation Program

The Health Resources and Services Administration (HRSA) supports a wide array of very different programs and activities that promote access to needed health care for all, including primary health care centers, the National Health Service Corps, HIV/AIDS programs, maternal and child health activities, health professions training, rural health programs, organ donation and transplantation initiatives, and telehealth activities. To provide underpinning for these efforts, HRSA’s evaluation program is designed to enhance strategic planning, strengthen budget and legislative development, and improve program performance.

Indian Health Service

Mission

In partnership with American Indian and Alaska Native people, to raise their physical, mental, social, and spiritual health to the highest level.

Evaluation Program

The goal of the Indian Health Service (IHS) is to assure that comprehensive, culturally acceptable, personal and public health services are available and accessible to American Indian and Alaska Native people. The importance of evaluation in supporting this goal has increased significantly in recent years and includes American Indians and Alaska Natives as the primary stakeholders in defining the purpose, design, and execution of evaluations. The stakeholders use the end product of the evaluations, and are the population or groups most likely to be affected by the findings. The IHS has formally adopted the principle of a responsive evaluation practice to address the needs and concerns of Native Americans and Alaska Natives.

Each year IHS selects high-priority health care and management studies for funding through the submission of proposals to headquarters and Area Offices. These proposals are reviewed and rated by a panel of subject-matter experts, evaluation experts, and IHS staff for concurrence with IHS strategic goals, objectives, and priority areas. The proposals are then prioritized and forwarded to the IHS Director, who reviews the projects that are recommended for funding and determines the respective funding levels.

The evaluation needs of the IHS service components are coordinated using two major types of short-term studies: policy assessments and program evaluations. Policy assessments contribute to decision making about budget, legislation, and program modifications including information to support the Agency’s initiatives. Evaluations are focused at the program level, or Area Offices, and focus on specific needs.

The evaluation program of the IHS is managed by the Office of Public Health Support, Staff Office of Planning, Evaluation, and Research, which provides national leadership and consultation for IHS and Area Offices on strategic and tactical planning, program evaluation and assessment, public health and medical services, research agendas, and special public health initiatives for the Agency.

National Institutes of Health

Mission

To uncover new knowledge that will lead to better health for everyone.

Evaluation Program

The National Institutes of Health (NIH) pursue new knowledge about the prevention, detection, diagnosis, and treatment of disease and disability. To that end, NIH has a wide range of programs to support health-related research and training and professional development. Evaluating these numerous and diverse programs is one important tool that NIH administrators use to determine the extent to which these programs are operating efficiently and achieving their intended outcomes.

NIH Institutes and Centers (ICs) and components within the Office of the Director (OD), NIH, use program evaluations and evaluation-related activities to improve decision-making and, ultimately, enhance program performance. Many NIH activities are crosscutting in nature and require trans-NIH program evaluations (i.e., program evaluations that involve more than one IC or OD office) to be examined effectively. Program evaluations are professional systematic investigations or studies that evaluate the merit of particular programs, or contribute to making such an evaluation possible. In most cases, the purpose of program evaluations is to help NIH administrators improve a program or make other programmatic decisions (e.g., how to allocate resources). A “program” is broadly defined as any set of activities funded by the NIH to achieve one or more predefined goals (also referred to as “program goals”).

The NIH recognizes that results-based management as a basic principle for the sound and productive operation of government agencies and their programs. The most notable evidence of this is passage of the Government Performance and Results Act (GPRA), and the use of the Program Assessment Rating Tool (PART). With additional efforts to increase public sector accountability (such as passage of the Chief Financial Officers Act and the Government Management Reform Act), interest in evaluation has increased steadily among program administrators.

A distinguishing feature of the NIH Evaluation Program is its position within a larger institutional framework of several evaluation strategies including the use of national advisory councils, boards of scientific counselors, consensus development conferences, and ad hoc committees. This framework helps to chart scientific directions and select the most promising research to support.

Office of the Assistant Secretary for Planning and Evaluation

Through the departmental evaluation planning process, ASPE has the capacity to identify crosscutting health and human services program or policy issues of particular concern to the Secretary and specific program and policy areas not covered by the HHS Agency evaluation plans. In these instances, ASPE initiates evaluations or collaborates with the agencies to conduct evaluations or policy assessments.

Another continuing evaluation objective of ASPE is to support and promote the development and improvement of databases that HHS agencies and ASPE use to evaluate health care programs and health trends. For example, ASPE has been the major initiator, in collaboration with the National Center for Health Statistics at the Centers for Disease Control and Prevention (CDC), of the first comprehensive survey of people with disabilities in the United States. The first component of this new data was completed in FY 1996, and national prevalence data on disability is now available.

The ASPE chairs the Research Coordination Council (RCC), which will evaluate Department-wide research priorities to ensure that efficiencies are realized and research funding priorities are consistent with the Administration’s priorities. The ASPE also co-chairs and provides support to the HHS Data Council, which is charged with integrating key national surveys, such as linking health status indicators with indicators of well being.

Finally, ASPE uses evaluation funds to promote effective use of evaluation-generated information in program management and policymaking. The latter is accomplished through the dissemination of evaluation findings and other activities, such as providing technical assistance to agencies in the development of performance measures.

Office of Public Health and Science

Substance Abuse and Mental Health Services Administration

Mission

To build resilience and facilitate recovery for people with, or at risk for, substance abuse and mental illness.

Evaluation Program

The Substance Abuse and Mental Health Services Administration (SAMHSA) evaluates the effectiveness of prevention, treatment, and rehabilitation approaches and systems of care used by its overall programs and individual grant projects. SAMHSA conducts evaluations to ensure accountability for federal funds and to measure results toward its programmatic and policy objectives. In compliance with the Government Performance and Results Act (GPRA), SAMHSA is improving performance management and results by identifying annual, long-term and cost-efficiency performance measures to manage its programs.

SAMHSA has an integrated evaluation and planning process. Strategic planning identifies priorities that drive the development of grant programs and evaluations. The formulation of programmatic and evaluation priorities includes consultation with SAMHSA Center Advisory Councils, with other HHS agencies, and with experts in the fields of evaluation and service delivery. Early and continuous coordination of program planning and evaluation design results in the articulation of program objectives that may be evaluated. Evaluations measure achievement of grant programs overall objectives, and these results are used for program and policy development. The strategic planning and policy development processes then use these results to refine SAMHSA’s priorities and performance objectives.

The specific type of evaluation required depends on the type and purpose of the particular grant program. To the greatest extent possible, SAMHSA encourages the use of comparable data elements and instruments across its evaluations to implement a comprehensive evaluation system and to minimize respondent burden. Efforts to improve evaluation are continuing and SAMHSA is committed to using systematic approaches in using data to accomplish its mission.

Appendix C – List of Studies by Agency

Appendix D – Acknowledgment of HHS Officials

The following persons from HHS contributed to preparing information on evaluation projects for Performance Improvement 2006: Evaluation Activities of the U.S. Department of Health and Human Services:

Preparation of this report was managed by the Office of the Assistant Secretary for Planning and Evaluation, Office of Planning and Policy Support.

Project Director: Audrey Mirsky-Ashby.  Technical and Systems Support: Kimberley Berlin.

Additional ASPE staff contributing to the development of these materials included Alana Landey, Bill Marton, and Lynn Nonnemaker. Susan Belsinger of Library Associates, Inc. provided database management. Kim Axelrod of Library Associates, Inc. provided database updating and editing support. Martin Bosworth of STG International provided editing support.