Performance Improvement 2011-2012

Publication Date

Dec 31, 2011

Introduction

Section 241(a) of the Public Health Service Act authorizes evaluation of the implementation and effectiveness of programs funded by the Act. Section 241(b) requires the Secretary of Health and Human Services (HHS) to summarize the findings from these studies in annual reports to the Senate Health, Education, Labor and Pensions Committee and to the House Energy and Commerce Committee.

This report, Performance Improvement 2011-2012, the 17^th in this series, summarizes the key findings from studies completed during the two fiscal-year period ending September 30, 2011.

Researchers, analysts, students and members of the general public are encouraged to access the entire set of HHS evaluations covered by this reporting series, including those summarized in this current report. The HHS Evaluation Database located at http://aspe.hhs.gov/Evaluation/Performance contains abstracts for all these studies. Many entries provide direct links to the reports for the studies described in the abstracts.

The online publicly available and searchable database also contains entries for the studies completed after those included in this report as well as over 150 studies that are in process at the present time. All agencies of the Department of Health and Human Services enter this information directly as studies are initiated as well as completed so that the database is kept current, reflecting both ongoing studies and studies for which reports or other final products have been issued.

As new studies are initiated or completed, the information in the Database is being supplemented or updated. The online database provides the most current roster of evaluations and their status. Database SEARCH and REPORT generation features, newly added to the database, including improved full text searching and categorical search capabilities, make the contents of the HHS Evaluation Database more readily accessible and retrievable.

This report consists of the following:

· Chapter I provides brief background information regarding HHS evaluation work.

· Chapter II presents summaries for each completed study describing:

o Key Study Question (title to the entry);

o Study Background (first paragraph);

o Summary of Findings (second paragraph); and

o Contact and Citation information (staff contact, report title and, where available, the URL linking directly to a report).

Chapter I – Background

The Department of Health and Human Services (HHS) administers the largest number of assistance programs of any Federal department. These are identified and described in the Catalog of Federal Domestic Assistance, www.cfda.gov . For studies funded during Fiscal Years 2010 and 2011, the period covered by this report, Congress specified the use of $1,007 million and $982 million respectively of new funding, and HHS utilized $18 million and $9 million in remaining available funds from previous years, respectively, under the Section 241(a) evaluation set-aside provision of the Public Health Service Act.

Public Health Service Set-Aside Authority

The Public Health Service Act, Section 241 set-aside authority was originally established in 1970, when the Congress amended the Act to permit the HHS Secretary to use up to 1 percent of appropriated funds to evaluate authorized programs. Section 241 limits the base from which funds can be reserved for evaluations to programs authorized by the PHS Act. Excluded are funds appropriated for the Food and Drug Administration, the Indian Health Service, and certain other programs that are managed by PHS agencies but not authorized by the Act (e.g., HRSA's Maternal and Child Health Block Grant and CDC's National Institute for Occupational Safety and Health). [1]

The Appropriations Acts for 2010 and 2011 authorized the Secretary to use up to 2.5 percent of the amounts appropriated for programs authorized by the Public Health Service Act for the evaluation of these programs and purposes outlined in the appropriations bills. A total of $1,025 million and $1,001 million were available in PHS Evaluation Funds for agencies and offices in FY 2010 and FY 2011, respectively. By agency, these funds were appropriated as follows:

Administration for Children and Families (ACF) – $6 million in FY 2010 and FY 2011

Agency for Healthcare Research and Quality (AHRQ) – $397 million in FY 2010 and $372 million in FY 2011

Centers for Disease Control and Prevention (CDC) – $352 million in FY 2010 and FY 2011

Health Resources and Services Administration (HRSA) – $25 million in FY 2010 and FY 2011

National Institutes of Health (NIH) – $8 million in FY 2010 and FY 2011

Substance Abuse and Mental Health Services Administration (SAMHSA) – $132 million in FY 2010 and FY 2011

Staff components in the Office of the Secretary including the Office of the Assistant Secretary for Planning and Evaluation (ASPE), the Office of the Assistant Secretary for Health (ASH), the Office of the Assistant Secretary for Financial Resources (ASFR), the Office of the National Coordinator for Health Information Technology (ONC), and the Office of the Assistant Secretary for Preparedness and Response (ASPR) received $87 million in FY 2010 and again in FY 2011.

Funds obligated for studies in one fiscal year are typically expended during one or more future years. Therefore, a particular year's funding doesn't correlate with specific studies completed during the same year.

Evaluation Management

During each year's budget development cycle, offices and agencies make recommendations regarding the evaluations necessary to comply with statutory requirements and Executive and Department guidance. On a multi-year basis, the Government Performance and Results Act of 1993 requires the Department to establish a new five-year strategic plan every three years. The Department last updated its Strategic Plan in September, 2011. The evaluations summarized in the current report are organized under the Goal and Objective that they support under the Department's Strategic Plan. The statute forms an essential basis for evaluation planning. Typically, HHS evaluation priorities include congressionally mandated program evaluations, evaluations of Secretarial program or policy initiatives, assessments of new programs and ones that are candidates for reauthorization, and evaluations that support program performance management and accountability.

HHS evaluation activities support the Department's strategic planning and performance management activities in several ways. Completed evaluations help shape specific HHS strategic goals and objectives. Evaluation findings provide important sources of information and evidence about the success of HHS programs and policies. The HHS Strategic Plan highlights evaluations that document efficacy or effectiveness of strategic programs or policies and lists future evaluations that will benefit strategic planning. Agencies use findings from their evaluations to support annual performance reporting to Congress and program budget justifications.

As specified in annual appropriations authorizations, the Secretary reports to the Congress, plans for using PHS evaluation set-aside funds before implementing these plans. Congress has included in its appropriations acts that these reports, titled “Use of Public Health Services Set-Aside Authority,” be submitted to the Committee on Appropriations of the House of Representatives and the Senate before expending funds for this purpose. HHS sent the most recent plan–for FY 2012–to the Congress in May, 2012.

Dissemination of Evaluation Reports

Easy public access to searchable and retrievable information regarding both new and completed evaluations is essential. Project officers and evaluation managers work to spread knowledge obtained from studies conducted. As one component of the Department's efforts to assure wide spread dissemination of the evaluation work ongoing and completed, HHS recently upgraded the ease with which studies can be identified and obtained. The Assistant Secretary for Planning and Evaluation took particular interest in assuring that the HHS Evaluation Database was complete and up to date. In addition to assuring that all studies are entered into the Database, agencies of the Department are instructed to post all their evaluations online. The HHS Evaluation Database, found at http://aspe.hhs.gov/Evaluation/Performance, offers users an opportunity to search – by key word, selected program, or policy topics – the departmental evaluation report database and electronic report library.

The results of HHS evaluations are also disseminated on agency and office websites through targeted distribution of printed reports and research briefs, as well as presentations at professional meetings and conferences. HHS researchers also participate in the broader research community by contributing articles in specialist publications and refereed journals.

Chapter II – Highlights of Findings for Completed Studies

This chapter provides summaries of the key findings for the studies for which reports were issued during the two fiscal years ending September 30, 2011. The studies are grouped under the relevant HHS Strategic Plan goal and objective which they support. The entries are also in the HHS Evaluation Database, http://aspe.hhs.gov/Evaluation/Performance.

The studies have been conducted by or funded through the Department of Health and Human Services by the specific agency or office sponsors. Reports may be obtained at the URL hotlinks provided or from the Federal Contact provided in the citation at the end of each entry below, or from the Librarian of the Office of the Assistant Secretary for Planning and Evaluation, at evaluation@hhs.gov .

Goal 1: Strengthen Health Care

Objective: Make coverage more secure for those who have insurance, and extend affordable coverage to the uninsured

How Does Ryan White Care Act Funding Impact Service Use and Patient Health for Those Suffering from Human Immunodeficiency Virus/Acquired Immune Deficiency Syndrome?

This study examined the impact of Title I funds on service utilization patterns and prevalence of HIV-related morbidity and mortality among underserved and vulnerable populations in two metropolitan areas It compared delivery and utilization of HIV-related services before (Phase 1: 3/1/1998-2/28/1999) and after (Phase 2: 3/1/2000-2/28/2001) the receipt of Ryan White CARE Act funding.

Both study sites reported that 100 percent of patients were prescribed antiretroviral therapy at least once every three months, regardless of clinic type in Phase 2 compared to 63 percent in Phase 1.

Report Title: Impact of RWCA Title I Funding on HIV Services Utilization and Health Outcomes in Newly Eligible Metropolitan Areas (EMAs)
Agency Sponsor: HRSA, Health Resources and Services Administration
Federal Contact: Malitz, Faye, 301-443-3259
Performer: Health Resources and Services Administration

Record ID: 7215 (Report issued January 31, 2010)

What are the Attitudes, Perceptions, and Behaviors of Active Private Practice Dentists Towards the Latest Infection Control Recommendations for Dentists?

Researchers surveyed 6,500 active private practice dentists and evaluated the extent to which four new recommendations of the 2003 Centers for Disease Control and Prevention (CDC) Guidelines for Infection Control in Dental Health Care Settings had been implemented. The study assessed how implementation of the guidelines was related to demographic and educational characteristics of the dentists, including knowledge of the guidelines, implementation, and attitudes regarding their importance, sources of instruction regarding the guidelines and additional factors including state licensing requirements for infection control and available opportunities for continuing dental education.

Implementation and knowledge of the recommendations in the CDC 2003 infection control guidelines were neither complete nor uniform across dentists. Of the dentists surveyed, 34 percent had implemented zero or one recommendation, 40 percent had implemented two recommendations and 26 percent had implemented three or four recommendations. Dentists implemented more recommendations if infection control was important to the dentist personally, and if they had multiple exposures to the guidelines. These results suggest that CDC may want to target strategies to increase awareness of the importance of infection control among dentists and develop multiple modes of instructions on the guidelines.

Report Title: Factors Associated with Implementing CDC's “Guidelines for Infection Control in Dental Health Care Settings ─ 2003”

Accepted for publication: jada.ada.org

Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Jennifer Cleveland, 770-488-6066
Performer: Health, Social, and Economics Research, RTI International
Record ID: 9227 (Report issued September 29, 2010)

How Feasible and Desirable Is Working Towards A Set Of Core Indicators Across the Resource Centers Of Excellence Managed By the National Center On Birth Defects and Developmental Disabilities (NCBDDD)?

This program evaluation studied the feasibility and desirability of working towards a set of core indicators across several Resource Centers funded by NCBDDD, the National Center on Physical Activity and Disability (NCPAD), the Paralysis Resource Center of the Christopher and Dana Reeve Foundation and the National Limb Loss Information Center of the Amputee Coalition.

The study found that each Resource Center had a robust evaluation system, including expertise and applications to collect extensive information on output and outcome indicators. The study concluded it was feasible to develop a set of common core indicators as a first step towards more systematic cooperation among the Resource Centers in evaluating the impact of their work. Larger issues regarding inter-organizational issues would need to be studied more fully before making the decision to implement these.

Report Title: Exploring a Common Evaluation Framework for Resource Centers of Excellence

Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Thomas Chapel, 404-639-2116
Performer: Adriane Griffen
ID: 9391 and 9401 (Report issued December 31, 2009)

How Can the Agency for Healthcare Research and Quality Program Be Improved?

Using a sample of selected evaluation reports, researchers identified program goals and other factors against which to measure program success. The evaluation reports were rated and analyzed and conclusions and recommendations for future program planning and evaluation were drawn.

The program evaluation synthesis resulted in the following conclusions and recommendations: 1) improve the clarity of program goals and ensure that they are aligned with program scope and resources; 2) improve the support for product/output dissemination; 3) ensure products and outputs are usable for target audiences; 4) consider the program management structures in relation to the program size, scope, composition, and goals; and, 5) assure program staffing is consistent with program size, scope, and complexity.

Report Title: Evaluation Synthesis
Agency Sponsor: Agency for Healthcare Research and Quality
Federal Contact: Cynthia Boone, 301-427-1212
Performer: IMPAQ International, LLC
ID Number: 9497 (Report issued April 30, 2010)

What are the Key Lessons Learned from the Experience of Network-Enabled Health Information Technology in Federally Funded Health Center Networks?

Through site visits, this project examined the experiences of ten federally funded networks of health centers in their adoption of health information technology. Topics addressed include: associated challenges and opportunities, perceived costs and benefits, care quality and efficiency applications, the nature of support and technical assistance needed at various stages of implementation and the role networks play in implementation.

Networks have been key in facilitating health information technology (HIT) adoption, but even with network support the task is difficult for the following reasons: 1) using HIT for quality improvement takes time and may not be the most direct route to improved quality; 2) some, but not all, network functions can be provided to members in diverse geographic locations; 3) networks are involved in data exchange and integration but not as Health Information Organizations; and, 4) technical support needed for implementation to address meaningful use is substantial and varies according to stage of electronic health record implementation. Support may be needed in planning and vendor selection, prior to and during implementation, for training, and, significantly, post-implementation to better assure full use of system potential.

Report Title: Assessment of Health IT and Data Exchange in Safety Net Providers http://aspe.hhs.gov/sp/reports/2010/chcit2010/report.pdf
Agency Sponsor: OASPE-OHP, Office of Health Policy
Federal Contact: Caroline Taplin, 202-690-7906
Performer: NORC
Record ID: 9635 (Report issued January 1, 2010)

What Process Should the Department of Health and Human Services Use in Defining and Updating Essential Health Benefits as Required by the Affordable Care Act?

The Affordable Care Act identified at least ten essential categories of items and services known as essential health benefits (EHBs) that must be included in packages of benefits to be offered by qualified health plans participating in Exchanges beginning in 2014, issuers in the individual and small group markets, Medicaid benchmark and benchmark-equivalent plans, and State basic health programs for low-income individuals not eligible for Medicaid. The scope of these EHBs should be guided by the content of a typical employer plan. The Institute of Medicine (IOM) at the Department's request convened an expert panel, held two public hearings with over 50 witnesses, received 345 responses to questions posed on line, and issued two reports.

The IOM recommended that the scope and design of EHBs should reflect those packages offered by small employers, that suggested criteria guide the aggregate package of EHBs and specific inclusions and exclusions, and that costs be considered in both the initial design and update of EHBs and that a public deliberative process should guide both efforts. The IOM further proposed a process for updating EHBs, encouraged state innovation, recommended data gathering and research efforts across the health care system to control costs and recommended the creation of a National Benefits Advisory Council.

Report Title: Essential Health Benefits: Balancing Coverage and Cost http://www.iom.edu/Reports/2011/Essential-Health-Benefits-Balancing-Coverage-and-Cost.aspx
Agency Sponsor: OASPE-OHP, Office of Health Policy
Federal Contact: Caroline Taplin, 202-690-7906
Performer: Institute of Medicine
Record ID: 9634 (Report issued October 1, 2010)

Objective: Improve health care quality and patient safety

Did Mental Health HIV Services Collaboration Grants Improve Community-based Treatment for Underserved Individuals?

Substance Abuse and Mental Health Services Administration (SAMHSA) provided funds in 2006 for grants to enhance and expand the provision of effective, culturally competent HIV/AIDS-related mental health services in minority communities for persons living with HIV/AIDS (PLWHA) and having a mental health need. Five-year grants were awarded in FY 2006 to applicants with demonstrated experience providing culturally competent mental health services in their respective communities.

The sixteen grantees assessed 4,569 individuals during the four and one-half year period included in the analysis, primarily from minority communities, and living with HIV/AIDS. Program clients experienced significantly improved mental and physical health as indicated by a decline in depression symptoms and increased mental and physical well-being. Decreases in detectable viral load and increases in CD-4 levels were reported by clients. Clients presented with a wide range of mental health disorders; the most prevalent were depressive disorders, substance-related disorders, and anxiety disorders. Grantees were able to implement expanded or enhanced HIV mental health services in a culturally acceptable manner. The majority of the clients served by the grantees were unemployed and one third was disabled. Moreover, 45 percent of clients were living in unstable housing situations. To accommodate complex and diverse needs, clients were served in a wide variety of service settings with site-specific clinical models that emphasized meeting the clients where they were located. Although grantees were asked to focus on sustainability planning early on in the project period, not all grantees will be able to sustain some or the entire set of project services without additional outside funding.

Report Title: Evaluation of the CMHS Mental Health HIV Services Collaborative (MHHSC) Program
Agency Sponsor: SAMHSA, Substance Abuse and Mental Health Services Administration
Federal Contact: Ilze Ruditis, 240-276-1777
Performer: James Bell Associates, Inc.
Record ID: 8722 (Report issued December 1, 2010)

What Services Should the Comprehensive Women's Preventive Services Guidelines Include and How Should They Be Reviewed and Updated?

The Affordable Care Act requires new health plans to offer certain preventive health services and screenings without imposing cost sharing – including preventive services specifically for women as provided for in comprehensive guidelines supported by the Health Resources and Services Administration. This project convened a panel to identify gaps and propose services in addition to those rated A and B in the U.S. Preventive Services Task Force guidelines, and recommend a process for regular review and updates to such guidelines.

Recommendations to Health and Human Services included coverage of the following services: well-woman visits; screening for gestational diabetes; human papillomavirus testing; counseling for sexually transmitted infections, counseling and screening for human immune deficiency virus; contraceptive methods and counseling; breastfeeding support, supplies and counseling; and screening and counseling for interpersonal and domestic violence.

Report Title: Clinical Preventive Services for Women: Closing the Gaps

http://www.iom.edu/Reports/2011/Clinical-Preventive-Services-for-Women-Closing-the-Gaps.aspx
Agency Sponsor: OASPE-OHP, Office of Health Policy
Federal Contact: Adelle Simmons, 202-690-6870
Performer: Institute of Medicine
Record ID: 9632 (Report issued July 1, 2011)

What Are The Replicable Policy And System Change Strategies Implemented In State Heart Disease And Stroke Prevention Programs That Achieve Intended Outcomes?

This study identified and confirmed policy and system based promising practices in heart disease and stroke prevention. The determination of promising practices is based on evidence that these practices have the potential for public health impact on the burden of heart disease and stroke and are adaptable to other geographic settings. Funded state programs submitted interventions. Two selected interventions were intended to improve emergency response, two were intended to improve control of high blood pressure, and two were intended to improve the quality of care and one to improve the spread of policy and system change across a region. A comprehensive evaluation of each of the selected state-funded interventions was conducted.

One of the seven interventions emerged as a promising practice, and three as practices with promising processes. The evaluation helped identify interventions worth sharing as well as criteria, definitions, and methods for identifying promising practices. Staff responsible for each intervention received a summary report identifying the strengths and weaknesses of the intervention and recommendations to improve the program's outcomes and replicability.

Federal Contact: Susan Ladd 770-488-5448
Performer: RTI International
Record ID: 9221 (Report issued September 29, 2010)

Objective: Emphasize primary and preventive care linked with community prevention services

What Are the Facilitators, Barriers, Benefits and Drawbacks to Family Planning Providers' Participation in Drug Pricing and Prime Vendor Programs?

The family planning program, Title X of the Public Health Service Act, serves approximately 5 million individuals each year. The program provides contraceptive, health screening, and sexually transmitted disease prevention and treatment services. To help safety-net providers, including those supported by Title X continue to serve clients, the federal government has implemented some cost-saving mechanisms and programs. One such effort, the 340B Drug Pricing Program, requires that manufacturers provide outpatient drugs at a discounted price to certain federal grantees, including Title X-supported entities. The study researchers sought to better understand Title X providers' experiences using the programs and alternatives available to achieve pharmaceutical discounts. The research included a literature review and in-depth discussions with grantee and delegate agencies around the country currently receiving Title X funds.

Researchers found that a large portion of Title X grantees were enrolled in the 340B Program and about half of the study participants participated in the Prime Vendor program, though this proportion is increasing.

Report Title: Analysis of the Effectiveness of Title X Family Planning Providers' Use of the 340B Drug Pricing Program

http://www.hhs.gov/opa/pdf/304b-analysis-of-effectiveness.pdf
Agency Sponsor: OASH, Office of the Assistant Secretary for Health
Federal Contact: Eugenia Eckard, 240-453-2831
Performer: The Lewin Group
Record ID: 9200 (Report issued October 16, 2009)

For Consumers with Limited Health Literacy, What are their Food Choice Beliefs, Attitudes, and Behaviors?

Six focus groups were conducted as formative research about food choice attitudes and behaviors among adults with low health literacy. Researchers sought to learn about participants' knowledge, attitudes and behaviors related to their food planning and selection decisions, identify barriers and motivators to healthy eating, examine participants' understanding of food groups and health-related food characteristics, examine reactions to various images that may be used to help identify effective methods to depict food portion sizes, and identify healthy eating goals and messages.

This research informed the communication of the 2010 Dietary Guidelines for Americans. The research yielded the following recommended strategy: To develop a set of materials that provides information about the Dietary Guidelines coupled with images to visualize portion sizes and concrete examples as well as actionable messages that suggest what people can do to meet the guidelines and overcome barriers, and that explain potential benefits from new behaviors. Examples include: (1) provide information and tools enabling people to make ‘healthy eating' choices in everyday situations; (2) use images or information about the health conditions and risk factors of not eating healthy; (3) refer to the actionable goals most often mentioned by the focus groups such as increasing knowledge of nutritional content, eating smaller portions, eating more fruits and vegetables and less fat; (4) use measuring instruments as tools for visualizing portion size; (5) be careful about using reference objects as a way to determine portion size; (6) include low-cost, easy-to-prepare, healthy eating options so people can more easily adopt the dietary guidelines.

Report Title: 2010 Dietary Guidelines for Americans Consumer Focus Group Testing on Adults with Limited Health Literacy
Agency Sponsor: OASH, Office of the Assistant Secretary for Health
Federal Contact: Rachel Hayes, 240 8252
Performer: American Institutes for Research (AIR)
Record ID: 9205 (Report issued December 31, 2010)

How are Health Information Technology Tools Being Used in Nursing Homes and Home Health Agencies?

This case study examined how particular health information technology (HIT) tools are being used in nursing homes and home health agencies, and uses qualitative data to identify the costs and benefits associated with their use.

The types of benefits and costs of HIT implementation in nursing homes and home health agencies are generally reported to be the same as those reported in hospitals and physician offices. The scope of these benefits may be greater in long-term care (LTC) because care is provided by interdisciplinary teams of clinicians who are often geographically dispersed. Benefits were reported to outweigh costs.

Report Title: Understanding the Costs and Benefits of Health Information Technology in Nursing Homes and Home Health Agencies: Case Study Findings http://aspe.hhs.gov/daltcp/reports/2009/HITcsf.htm
Agency Sponsor: OASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Vidhya Alakeson
Performer: University of Colorado
Record ID: 9334 (Report issued October 30, 2009)

What are the Workforce Competencies for Professionals Working in Long-Term Care Settings?

This study provides a snapshot of workforce competencies that have been identified for professionals who work in long-term care settings. This is provided through examination of basic roles and responsibilities of professionals and options presently available for specialization, through an analysis of the long-term care workforce literature, and through identification of initiatives launched by professional associations and providers. Additionally, the paper examines whether there might be differences in the competencies required to care for the geriatric population in long-term care settings compared to acute and ambulatory care settings.

Efforts are underway by various stakeholders to identify competencies and necessary skill-sets for professionals who work with older persons in long-term care settings; however more work is needed. Currently, licensure requirements for most health care professionals are variable and therefore do not include mandatory competencies. Professional competencies that pertain to older populations in long-term care settings are difficult to define. The alternatives for additional or supplemental training come in the form of modifying curricula, and/or adding continuing education; both are viable avenues but would require consensus within individual fields to determine the most optimal path.

Report Title: Examining Competencies for the Long-Term Care Workforce: A Status Report and Next Steps http://aspe.hhs.gov/daltcp/reports/2009/examcomp.htm
Agency Sponsor: OASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Emily Rosenoff, 202-690-6443
Performer: Institute for the Future of Aging Services, HHS Office of the Assistant Secretary for Planning and Evaluation
Record ID: 9338 (Report issued May 21, 2010)

What are Current Trends in Ownership Structure of Nursing Homes?

As private equity firms are increasingly buying and investing in nursing home chains, policymakers are concerned with the effect of corporate structure on the quality of care provided. Based on detailed ownership data from the State of Texas, ASPE's study found that the trend in ownership structure of nursing homes is more complex with increasing numbers of layers of organization between the nursing home and the actual individual owners. The quality of care has nevertheless remained the same as before the changes in corporate structure.

Individual nursing homes are increasingly using limited liability structures. A limited liability structure is one in which each owner has limited personal liability for the debts and actions of the company. By contrast, in a sole or general partnership each owner is personally responsible for all risks, liabilities, and debts of a company. Most of the limited liability corporations and partnerships in this study were for-profit facilities. Corporate structures often have multiple layers of limited liability entities between the individual nursing home and the ultimate owners of multiple facilities. The use of management companies to deliver care has increased in the past 10 years. Texas has experienced an increase in separate ownership of operations and ownership of physical property through lease agreements and real estate investment trusts. Quality of care does not appear to change as a result of changes in corporate structure. Nursing homes that were poorer performing facilities when part of a national nursing home chain, remained poorer performing facilities after the change in corporate structure.

Report Title: Nursing Home Ownership Trends and Their Impact on Quality of Care http://aspe.hhs.gov/daltcp/reports/2009/TXNHown.htm
Agency Sponsor: OASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Susan Polniaszek, 202-690-6443
Performer: Harvard Medical School
Record ID: 9340 (Report issued March 26, 2010)

What Types of Contracting are States Engaged in with Medicare Advantage Special Needs Plans?

The first report provides information on the extent and nature of state contracts with Medicare Advantage Special Needs Plans (SNPs). Review of SNP applications revealed the number of states that are already contracting with SNPs but highlighted a number of policy issues related to how states contract with SNPs.

Existing state Medicaid contracts with SNPs raise important definitional questions about the dual SNP contracting mandate established under the Medicare Improvements for Patients and Providers Act of 2008. A dual SNP is often operated as a distinct product within a larger corporate entity. The corporate parent may or may not also operate a Medicaid managed care contract that includes duals. SNP-state contracts take a variety of approaches for providing incentives for coordination of care. Those that include long-term care services in the contract have explicit provisions for managing care as a single benefit. Those in which long-term care is not included contain a range of proposals for encouraging/requiring coordination between Medicare and Medicaid. Benefits included in SNP-state contracts range from a full slate of services (Medicare cost sharing, drugs, primary and acute care, long-term care, and behavioral health) to a minimum of Medicare cost-sharing responsibilities. Some SNP-state contracts explicitly obtain value-added services for beneficiaries.

Report Title: State Purchasing Strategies Drive State Contracts with Medicare Special Needs Plans http://aspe.hhs.gov/daltcp/reports/2009/stpur.htm
Agency Sponsor: OASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Hunter McKay, 202-690-6443
Performer: Thomson Reuters
Record ID: 9341 (Report issued June 18, 2010)

What Types of Contracting are States Engaged in with Medicare Advantage Special Needs Plans? - Further Questions

The second report provides a more in-depth look at five contracts between Medicare Advantage SNPs and state Medicaid programs. The five featured programs were selected as examples of types of contracts currently being used by states and SNPs.

Report Title: Medicaid Contracts with Medicare Special Needs Plans Reflect Diverse State Approaches to Dually Eligible Beneficiaries http://aspe.hhs.gov/daltcp/reports/2009/SNPdual.htm
Agency Sponsor: OASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Hunter McKay, 202-690-6443
Performer: Thomson Reuters
Record ID: 9342 (Report issued June 18, 2010)

What Have Been the Changes in Numbers of Skilled Nursing Facilities and Long Term Care Hospitals?

In the years immediately following the introduction of prospective payment systems (PPS) for skilled nursing facilities (SNFs), large numbers of hospital-based SNFs closed. During the same time period, long-term care hospitals (LTCHs) expanded rapidly, especially in certain areas of the country. Some of the increase in LTCH services was provided to medically complex beneficiaries who previously might have been treated in SNFs. Because LTCH payments are typically much higher than SNF payments, this trend might well represent an increase in Medicare spending with little or no corresponding benefit to patients.

Between 1997 and 2007, both the number of hospital-based SNFs and the number of beds declined by over 50%; the number of LTCHs more than doubled. The supply of LTCH beds increased more rapidly in cities that lost hospital-based SNFs than in cities that did not. The analysis of PAC episodes found sharp declines in medically complex patients admitted to hospital-based SNFs (from 26% in 1997 to 9% in 2006). In communities that experienced a loss of hospital-based SNF, admissions of medically complex patients to LTCH increased in cities that expanded LTCH. In cities without a LTCH, medically complex care shifted to freestanding SNFs.

Report Title: Substitutability Across Institutional Post-Acute Care Settings: 1998-2006 http://aspe.hhs.gov/daltcp/reports/2009/instPAC.htm
Agency Sponsor: OASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Susan Polniaszek, 202-690-6443
Performer: Mathematica Policy Research
Record ID: 9343 (Report issued June 25, 2010)

What Have We Learned About Residential Care Facilities?

Starting in April 2010, ASPE and National Center for Health Statistics began the first nationally representative survey of residential care facilities (including assisted living facilities). A sample frame was created of all of the licensed residential care facilities in the United States. This study chronicled the sample frame development, including preliminary estimates of the size of the residential care industry and the number of beds.

There are approximately 39,000 residential care facilities, which offer a supply of a little over 1 million beds. These numbers indicate that residential care is starting to rival the nursing home industry in size. Approximately 41% of the facilities on the sample frame were affiliated with a chain. Of those that were part of a chain, only 8% were associated with the top forty chains and the rest were associated with regional or smaller chains. Residential care licensure terminology varies widely by state.

Report Title: National Survey of Residential Care Facilities: Sample Frame Construction and Benchmarking Report http://aspe.hhs.gov/daltcp/reports/2010/sfconst.htm
Agency Sponsor: OASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Emily Rosenoff, 202-690-6443
Performer: RTI International
Record ID: 9345 (Report issued July 16, 2010)

What Do We Know About Implementation of Electronic Health Records?

This study was in response to the Health Information Technology for Economic and Clinical Health Act of the American Recovery and Reinvestment Act of 2009 (Pub. L. 1115). The Act directs the HHS Secretary to conduct several studies and produce reports to Congress. including the “Study and Report on Application of EHR (Electronic Health Record) Payment Incentives for Providers Not Receiving Other Incentive Payments.” This study was conducted to determine the extent to and manner in which payment incentives for implementing and using certified EHR technology should be made available to health care providers who receive minimal or no payment incentives or other funding under HITECH, Medicare, or Medicaid for such purposes.

Report Title: Report to Congress on the Application of EHR Payment Incentives for Providers Not Receiving EHR Incentive Payments http://aspe.hhs.gov/daltcp/reports/2010/EHRcr.htm
Agency Sponsor: OASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Jennie Harvell, 202-690-6443
Performer: HHS Office of the Assistant Secretary for Planning and Evaluation, Centers for Medicare and Medicaid Services
Record ID: 9346 (Report issued August 13, 2010)

How Has the Study of Electronic Health Record Adoption Proceeded?

The health information technology (HIT) questions on nursing home (NH) adoption, use, and barriers to adoption or use that were developed under this project were designed to be fielded as a part of the National Nursing Home Survey (NNHS) or a freestanding survey, and to enable comparisons on health information technology (HIT) or electronic health record (EHR) adoption in other provider settings (e.g., the National Home Health and Hospice Survey, the National Ambulatory Care Survey, etc.).

The project developed two sets of survey questions on NH HIT adoption, use, and barriers: The "core" survey questions were designed for possible administration with the NNHS or other surveys. The "expanded" survey questions include the core questions and follow-up questions designed to obtain additional detail on certain electronic functions in use in NHs. The expanded survey may be of use to NH providers seeking a more information regarding HIT use in targeted NH(s) and/or more details on the use of particular HIT functions (e.g., e-prescribing).

Report Title: Survey Questions for EHR Adoption and Use in Nursing Homes: Final Report http://aspe.hhs.gov/daltcp/reports/2010/EHRques.htm
Agency Sponsor: OASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Jennie Harvell, 202-690-6443
Performer: University of Colorado
Record ID: 9347 (Report issued September 3, 2010)

Objective: Reduce the growth of health care costs while promoting high-value, effective care

What Factors Encourage Certified Nursing Assistants to Stay Working in Nursing Homes?

Certified nursing assistants care for approximately 1.5 million persons in nursing homes in the United States each year. Turnover and vacancy rates are high which may adversely affect residents. Additionally, the demand for services is growing faster than the labor pool. This study focused on wages and benefits, training, and organizational culture as policy interventions that may increase the supply and job tenure of certified nursing assistants. The major advantage of this research over other studies is its use of a nationally representative sample of certified nursing assistants within a nationally representative sample of nursing homes. All previous studies have been of local or regional samples that were not representative of the country as a whole.

This study underscores the importance of the basic economics of job choice by low-income workers. Wages, fringe benefits, job security, and alternative choices of employment are important determinants of job tenure, in addition to improvements in training and organizational culture.

Report Title: Why Do They Stay? Job Tenure among Certified Nursing Assistants in Nursing Homes http://aspe.hhs.gov/daltcp/reports/2009/whystay.htm
Agency Sponsor: OASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Marie Squillace, 202-690-6250
Performer: Research Triangle Institute, HHS Office of the Assistant Secretary for Planning and Evaluation
Record ID: 9335 (Report issued November 20, 2009)

What Is the Prognosis for Achieving and Maintaining Sufficient Numbers of Certified Nursing Assistants in Nursing Homes?

Projections of a substantial workforce imbalance and a myriad of unresolved systemic issues have motivated policymakers, providers, private foundations, and others to seek immediate and sustainable solutions to stabilizing the long-term care workforce. Current demographic, economic, and policy trends suggest that without serious intervention, the supply of certified nursing assistants (CNAs) could significantly worsen in the coming decades. The National Nursing Assistant Survey (NNAS) represents a major advance in the data available about CNAs in nursing homes and provides a rich resource for evidence-based policy to improve recruitment and retention efforts.

Survey data show that: 1) one in three CNAs received some kind of means-tested public assistance; 2) 55% of CNAs incurred at least 1 work-related injury within the past year and 22% were unable to work for at least one day due to the injury; 3) 42% of uninsured CNAs cite not participating in their employer-sponsored insurance plan because they could not afford the plan; and 4) years of experience do not translate into higher wages; CNAs with 10+ years of experience averaged just $2/hr more than aides who started working in the field less than 1 year ago.

Report Title: The National Nursing Assistant Survey: Improving the Evidence Base for Policy Initiatives to Strengthen the Certified Nursing Assistant Workforce http://aspe.hhs.gov/daltcp/reports/2009/NNASeb.htm
Agency Sponsor: OASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Marie Squillace, 202-690-6250
Performer: HHS Office of the Assistant Secretary for Planning and Evaluation, George Mason University, Substance Abuse and Mental Health
Record ID: 9336 (Report issued November 25, 2009)

What Factors Lead to Absenteeism of Certified Nursing Assistants?

Turnover and absenteeism of certified nursing assistants (CNAs) impose substantial costs on nursing homes and also may compromise quality of resident care. Nursing homes endeavoring to improve quality of care and life for residents rely on a committed frontline workforce. These individuals provide eight out of every ten hours of care residents receive in nursing homes. When workers are more satisfied with their jobs, they are less likely to be absent from work or to quit their jobs. The major advantage of this research over other studies is its use of a nationally representative sample of CNAs within a nationally representative sample of nursing homes. All previous studies have been of local or regional samples that were not representative of the country as a whole.

Wages, benefits, and job demands were associated with job satisfaction. Consistent with previous studies, job satisfaction was greater when nursing assistants felt respected and valued by their employers and had good relationships with supervisors.

Report Title: Nursing Home Work Practices and Nursing Assistants' Job Satisfaction http://aspe.hhs.gov/daltcp/reports/2009/NHwork.htm
Agency Sponsor: OASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Marie Squillace, 202-690-6250
Performer: Brandeis University, HHS Office of the Assistant Secretary for Planning and Evaluation, RTI International
Record ID: 9337 (Report issued November 25, 2009)

Objective: Ensure access to quality, culturally competent care for vulnerable populations

Can Clinical Research Infrastructure Be Established at Community-Based Institutions?

The National Cancer Institute's (NCI) Cancer Disparities Research Partnership (CDRP) pilot program sought to help reduce the negative consequences of cancer health disparities seen in targeted U.S. populations: Black/African American, Hispanic/Latino, Native American, Asian, elderly, and low-income. The pilot program was a 5-year planning grant designed to provide state-of-the-art radiation therapy; train new health disparities researchers; develop radiation oncology clinical trials research in community-based institutions with a disproportionate share of medically underserved, low-income, and racial and ethnic minority populations; offer quality cancer care to disparate populations; and build partnerships between academic institutions and community researchers. The pilot program also included community outreach and educational activities to increase knowledge and awareness of cancer and related services and clinical trials. The evaluation measured the relevance, effectiveness, and impact of the pilot program using a mix of qualitative and quantitative data. Data on program processes provided information on program activities, challenges, outputs, and short-term results. An assessment of program outcomes yielded information on project accomplishments and activities that led to attainment of CDRP's intermediate and long-term goals.

Evaluators found that the pilot program accomplished its goals of training community-based clinicians as health disparities researchers, and building radiation oncology clinical trial research infrastructure in community-based institutions. The pilot program created a culture of clinical trial research in community-based institutions and among community physicians where it did not exist before. It demonstrated that enrollment in clinical trials sponsored by community institutions was possible. Tremendous support from NCI, community organizations, physicians and the hospital community are required if new community-based institutions plan to take this path to address cancer health disparities.

Report Title: Cancer Disparities Research Partnership (CDRP) Final Program Evaluation Report
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: NOVA Research Company
Record ID: 9384 (Report issued June 18, 2010)

What Impact has the Adult Treatment Drug Court Program Had on the Existing Treatment System?

Researchers designed and conducted a process and outcome cross-site evaluation of the Grant Program to Expand Substance Abuse Treatment in Adult Drug Courts. The program grants were intended to expand or enhance substance abuse treatment services in problem-solving courts which use the treatment drug court model in order to provide alcohol and drug treatment, recovery support services supporting substance abuse treatment, screening, assessment, case management, and program coordination to adult defendants/offenders.

Drug court clients for whom six month post-discharge GPRA data are available report several positive outcomes including an 89% decrease in arrest rates and 84% decrease in illicit drug use. Improvements in employment, housing, and education also demonstrated positive outcomes for this sample. Drug court clients are most likely in need of services in addition to substance abuse treatment such as job training, housing, and transportation to treatment, among others. Five distinct components of the drug court structures and procedures that the clients and staff felt were most prominent and important in contributing to client success in recovery were the judge's role and practices; the relationship between the judge and the drug court team; drug court team practices; urine screening practices; and treatment practices. Sanctions and rewards issued by the judge only had a minor impact on the outcomes of the drug court clients.

Report Title: Program Evaluation for Adult Treatment Drug Courts
Agency Sponsor: SAMHSA, Substance Abuse and Mental Health Services Administration
Federal Contact: Deepa Avula, 240-276-2961
Performer: Substance Abuse and Mental Health Services Administration
Record ID: 8879 (Report issued September 1, 2011)

What Residual Impacts Remained from a Telephone Care Management Program Eighteen Months After Random Assignment?

Researchers tested a telephonic care management program that tried to encourage depressed parents who were receiving Medicaid to seek treatment from a mental health professional.

Managers effectively engaged people with depression via telephone. The "phone program" was a useful alternative for clients but typically did not lead to in-person treatment. Care managers were rarely able to function as liaisons between clients and clinicians in the community. There were significant barriers to in-person treatment within the target population.

Report Title: Working Toward Wellness Telephone Care Management for Medicaid Recipients With Depression, Eighteen Months After Random Assignment

http://www.acf.hhs.gov/programs/opre/welfare_employ/enhanced_hardto/reports/telephone_care/telephone_care.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Girley Wright, 202-401-5070
Performer: MDRC
Record ID: 9834 (Report completed November 15, 2010)

Would Establishing a Uniform National Elder Abuse Database Be Feasible?

In 2006, as part of the Tax Relief and Health Care Act of 2006, Congress directed the Secretary of HHS to conduct a study, in consultation with the Attorney General, assessing current elder abuse data collection systems and examining the feasibility of establishing a uniform national elder abuse database to improve the quality and accessibility of data (P.L. 109-432).

The study identified current administrative data sources, definitions and laws on elder abuse and for other violence fields including child abuse and intimate partner violence. The study also identified considerations for the development of a national uniform data collection on elder abuse. These considerations included the purposes of the data collection effort, technical issues, definitions, mandatory/voluntary reporting, and populations to include. Considerations also included funding, elements to include, use of aggregated or case-level data, confidentiality, data systems and coding.

Report Title: Congressional Report on the Feasibility of Establishing a Uniform National Database on Elder Abuse http://aspe.hhs.gov/daltcp/reports/2010/elderCR.htm
Agency Sponsor: OASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Gavin Kennedy, 202-690-6443
Performer: HHS Office of the Assistant Secretary for Planning and Evaluation, Lewin Group
Record ID: 9344 (Report issued June 28, 2010)

What Are the Public's Perceptions, Attitudes and Experiences About Racial and Ethnic Differences in Health Status and Health Care?

Despite significant documentation of persistent racial and ethnic disparities in health status and health care in the United States, current research on the subject indicates that the general public, including racial and ethnic minorities, is uninformed about the nature and extent of such disparities. In 1999, the Kaiser Family Foundation and Princeton Survey Research Associates conducted a national survey to measure the public's perceptions, attitudes, and experiences about racial and ethnic disparities in health care. The HHS Office of Minority Health (OMH) and NORC at the University of Chicago conducted a study to replicate and expand the earlier survey to determine if there have been any changes in the public's awareness of racial and ethnic disparities in health care and health status.

The study revealed a significant increase in Americans' awareness of such disparities from 55 percent in 1999 to 60 percent in 2009 and 59 percent in 2010. However, awareness of several key health status and disease indicators, such as infant mortality and HIV/AIDS, decreased or remained steady. Awareness of several important health disparities remains quite low even among the disproportionately affected racial and ethnic minority groups. Lastly, the general public reports relatively low levels of familiarity and contact with important national health disparities reports and awareness campaigns. A significant amount of work remains to be done to inform the U.S. population of health conditions that disproportionately affect specific racial and ethnic minority groups. Future studies need to include American Indian/Alaska Native populations and could also be adapted for other populations (e.g., legislators and other policymakers). To improve response rates, future data collection cycles should employ financial incentives and new sampling techniques and data collection methodologies.

Report Title: Trends in U.S. Public Awareness of Racial and Ethnic Disparities in Health
http://content.healthaffairs.org/content/30/10/1860
Agency Sponsor: OASH, Office of the Assistant Secretary for Health
Federal Contact: Valerie Welsh, 240-453-8222
Performer: NORC at the University of Chicago
Record ID: 8614 (Report issued September 30, 2010)

How Many and What Kind of Individuals Receive Services in Specialty Mental Health Organizations?

The purpose of the Client/Patient Sample Survey was to provide national estimates on the number and demographic, clinical, and service use characteristics of persons who receive services in specialty mental health organizations throughout the U.S. The survey is conducted approximately every 10 years. In the current survey, a nationally representative sample of mental health organizations/programs was selected for participation.

In the United States, an estimated 2.0 million persons were inpatient admissions and 3.3 million were outpatient admissions to specialty mental health organizations. Among inpatient admissions, 86% were adults ages 18 and older; 70% were non-Hispanic or Latino White; 19% non-Hispanic or Latino Black; and 9% Hispanic or Latino. Among outpatient admissions, 71% were adults ages 18 and older; 70% were non-Hispanic or Latino White; 17% non-Hispanic or Latino Black; and 10% Hispanic or Latino. Almost twice as many children under age 18 were admitted to outpatient care (29%) than admitted to inpatient care (14%).

Report Title: Mental Health, United States, 2004 (Chapter 20); DHHS Pub. No. SMA-06-4195, http://store.samhsa.gov/shin/content/MA06-4195.pdf
Agency Sponsor: SAMHSA, Substance Abuse and Mental Health Services Administration
Federal Contact: Laura Milazzo-Sayre, 240-276-1764
Performer: Westat, Inc.
Record ID: 8735 (Report issued December 1, 2010)

Goal 2: Advance Scientific Knowledge and Innovation

Objective: Accelerate the process of scientific discovery to improve patient care

Do Suicide Grants Increase Early Identification, Awareness, and Knowledge of Suicide Prevention on College Campuses?

All grant recipients of the Garrett Lee Smith Suicide (GLS) Prevention Program are required to participate in a national cross-site evaluation. This participation allows for large sample comparisons of early intervention activities and the collection of information on the use of best and promising practices.

Current evaluation results demonstrate that over 182,243 individuals on college campuses have been trained in suicide prevention and that 62.0% of students and 74.6% of faculty and staff reported high knowledge of suicide warning signs and suicide prevention resources. D ata from the cross-site evaluation also indicate that successful college suicide prevention is facilitated by student-serving programs and services that coordinate in the interest of students at risk and their families; a trained and accessible behavioral health system; widespread involvement throughout the college community; and a follow-up system for students most at risk. The GLS grants are laying this foundation, but there is still much work that needs to be done.

Report Title: To assess whether GLS grants increased early identification, awareness, and knowledge related to suicide prevention on college campuses
Agency Sponsor: SAMHSA, Substance Abuse and Mental Health Services Administration
Federal Contact: Richard McKeon, 240-276-1873
Performer: Macro International, Inc.
Record ID: 8725 (Report issued March 31, 2010)

How Well Do National Suicide Prevention Lifeline Grantee Crisis Staff Complete Suicide Risk Assessments and Follow Up Afterwards?

Through a cooperative agreement, the Substance Abuse and Mental Health Services Administration supports the National Suicide Prevention Lifeline (1-800-273-TALK). Lifeline is a system of toll-free telephone numbers that routes calls from anywhere in the United States to a network of more than 150 certified crisis centers that can link callers to local emergency, mental health, and social service resources. Recently, the Lifeline answered its three millionth call. The crisis hotline services were evaluated by conducting a follow-up assessment of suicidal individuals calling the Lifeline network. The follow up assessment includes an evaluation of the caller's suicide risk status at the time of and since the call, depressive symptoms at follow-up, service utilization since the call, barriers to access, and the client's perception of the efficacy of the hotline intervention.

Results show that over 60% of callers report that the calls were important in keeping them safe. The great majority of those who did not report that the calls were important in this way indicate this was because they had experienced no further suicidal thoughts since the original hotline call.

Report Title: Final Progress Report Hotline Evaluation and Linkage Project Category II
Agency Sponsor: SAMHSA, Substance Abuse and Mental Health Services Administration
Federal Contact: Richard McKeon, 240-276-1873
Performer: Macro International, Inc.
Record ID: 8728 (Report issued March 31, 2010)

What is the Level of Participation and Emphasis for Having Children Take Part in Research Grants by the National Institute of Child Health and Human Development?

In 1998, the National Institutes of Health (NIH) implemented a policy requiring scientists to include children in their clinical studies, unless there was justification to not include them. This study examined the implementation of the NIH Policy on the Inclusion of Children in Clinical Research. The study determined the participation rate of children and how much research scientists considered examining differences in disease and treatment for children. Reviews were conducted of grant applications that NICHD did and did not fund.

The majority of grantees were including children as research subjects, and peer review study sections identified only a very small number of compliance issues. However, the percentage of grants that included only children over 18, and the apparent confusion of reviewers and applicants about the definition of children, suggested that further education of the applicant and review communities about the NIH policy might be appropriate.

Report Title: NIH Policy on the Inclusion of Children in Clinical Research: Evaluation of Policy Implementation at the NICHD
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Sarah Glavin, 301-496-7898
Performer: MasiMax Resources Inc
Record ID: 9182 (Report issued July 1, 2010)

How is the National Institute for General Medical Sciences Advisory Council Perceived?

The study examined how the National Institute for General Medical Sciences (NIGMS) Advisory Council was perceived among NIGMS staff, staff at other Institutes, Council members and several institutions outside of the National Institutes of Health. The Council is composed of extramural scientists specializing in areas relevant to NIGMS, lay members with specific expertise, and representatives from the U.S. Department of Veterans Affairs and the U.S. Department of Defense. The Council has two functions: perform second-level peer review of grant applications and advise on policy and program development. The evaluators analyzed existing documents and gathered new data through semi-structured interviews with staff, Executive Secretaries at other NIH institutes, Council members and representatives from the National Science Foundation and the Howard Hughes Medical Institute.

The majority of respondents perceived the Council positively. Several staff described the Council as “basically appropriate” while Executive Secretaries at other NIH institutes characterized the Council as well-run or exemplary, and nearly 90% of the Council members reported being satisfied with their service.

Report Title: Support Services Task1: Assessment of Advisory Council Functions
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: Abt Associates Inc.
Record ID: 9381 (Report issued May 5, 2010)

Is the NIH-wide Human Research Protection Program Meeting Established Standards?

In 2006, National Institutes of Health (NIH) adopted new standards established by the Association for the Accreditation of Human Subjects Protection Programs to maintain its compliance with policies, laws, and regulations related to human subjects' research. This study assessed the extent to which the NIH-wide Human Research Protection Program (HRPP) was meeting these standards. Evaluators surveyed NIH Institutes and Centers to determine if their Institutional Review Boards were performing at the standards set by the Association. They collected and analyzed data from document reviews of policies and procedures of the HRPP, individual institute review boards, the NIH Office of Human Subjects Research, clinical programs, the Medical Administrative Service, and the Medical Executive Committee. Some document reviews were supplemented by interviews, focus groups, expert panels, structured observations, and/or questionnaires depending on the data needed.

The Human Research Protection Program was found deficient in various respects and NIH is following up accordingly.

Report Title: Evaluation of the NIH Human Subjects Protection Program to Acquire Accreditation from the Association of Human Research Protection Program, Inc.
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: HRP Associates, Inc.
Record ID: 9380 (Report issued March 15, 2010)

What Approach is Feasible for the Evaluation of the Science Education Partnership Award Program?

The National Center for Research Resources (NCRR) commissioned a feasibility study to develop an approach for evaluating the Science Education Partnership Award (SEPA) program. The SEPA program provides five-year grants for K-12 educational projects, science centers, and museum exhibits to deliver information about National Institutes of Health funded medical research, to increase career opportunities in science, and to cultivate an understanding about healthy living habits among the general public. The results of the study were based on several activities that included meetings with program officials, and working with NCRR staff to build models to describe the SEPA program and to develop a set of process and outcome questions for an evaluation of the program.

The study concluded that a process and outcome evaluation of the SEPA program was feasible and would be extremely valuable for the program and the field. The evaluators proposed an evaluation plan with assumptions, questions, approaches, and a schedule for carrying out the work. Methods proposed included: a review of SEPA project documents from the last 10 years, a survey of principal investigators of projects funded between 2000 and 2010, and a survey of all partners associated with a sample of projects, interviews with program directors and other science experts, and project-specific case studies.

Report Title: Science Education Partnership Award (SEPA) Feasibility Study
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: Westat, Inc.
Record ID: 9382 (Report issued September 30, 2010)

Is a Large-Scale Evaluation of the President's Cancer Panel Activities Feasible?

The President's Cancer Panel (PCP) identifies barriers to optimal development and implementation of all aspects of the National Cancer Program (NCP). A study was commissioned to determine whether a large-scale evaluation of Panel activities and progress related to Panel recommendations would be possible and informative. The study identified and collected explicit indicators and measures of the impact and degree of implementation of past recommendations. The goals of the study included identifying opportunities to improve activities of the Panel; documenting progress relative to Panel recommendations; and characterizing the Panel's role in the NCP progress.

It would be feasible to conduct a process evaluation of Panel activities and an outcome evaluation of intermediate outcomes such as implementation of Panel recommendations. Although short-term outcomes such as changes in awareness of and support for an issue might be evaluated, this type of analysis is resource-intensive and might not be informative. Direct links between the Panel and recommendation implementation activities might be drawn; however, there are limitations to the practicality and utility of carrying out a large-scale attribution evaluation. A more thorough process evaluation of Panel dissemination activities would identify opportunities to improve and expand current Panel dissemination activities to better reach and influence intended implementers and key stakeholders.

Report Title: President's Cancer Panel Evaluation Feasibility Study
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: NOVA Research Company
Record ID: 9387 (Report issued December 15, 2009)

Does a Two-Stage Editorial Board Review Process Enhance Traditional NIH Peer Review?

The NIH Center for Scientific Review (CSR) conducted a pilot study to examine whether the Two-Stage or Editorial Board Review would enhance traditional peer review of grant applications. At Stage 1, two to three specialist mail reviewers, known as the Editorial Board, evaluated each application on technical merit. At Stage 2, reviewers with broader expertise, called Editors, met in-person to discuss the impact and significance of the science. Stage 1 reviewers provided written critiques but no scores; Stage 2 reviewers scored the applications. Surveys measured the participants' overall level of satisfaction with the process including the rigor and quality of the review, the level of burden on the reviewers, and the adequacy of the instructions provided.

Participants were generally satisfied with the format and outcomes of the Two-Stage review process. Furthermore, the results indicated that the Two-Stage review process offered several enhancements relative to traditional peer review, including greater emphasis on peer review criteria and expanded access to technical expertise.

Report Title: CSR Evaluation Support Services: Editorial Board (Two-Stage) Review Survey Report
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: Abt Associates, Inc.
Record ID: 9394 (Report issued September 21, 2010)

Have NIH Funding Opportunity Announcements Targeted at Human Subjects Stimulated Bioethics Research?

This study examined a series of National Institutes of Health Funding Opportunity Announcements designed to attract research projects studying ethical issues in human subject's research to determine whether the funding announcements stimulated extramural research on ethical issues in NIH-sponsored human subjects' research. Evaluators assessed outputs associated with a series of funding announcements targeted at human subjects' research ethics and a series of announcements related to research integrity and ethical, legal and social implications. Outputs evaluated included total number of grants received and funded, institutions and investigators supported, and topical areas proposed and supported.

In general, human subjects' research ethics funding opportunity announcements stimulated the submission of bioethics research applications and increased submissions from new investigators. Projects studying informed consent were funded more often than projects studying research oversight. Human subjects' research related projects were funded at a higher rate when they were reviewed by scientists who had expertise in ethics and/or human subjects research issues.

Report Title: Analysis of NIH's Funding Opportunity Announcements (FOAs) to Stimulate Research on Human Subjects Research (HSR) Ethics
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: Discovery Logic, Inc.
Record ID: 9395 (Report issued March 5, 2010)

Has Funding for Pioneering Researchers led to Significant Contributions in Medical Research?

The National Institutes of Health (NIH) Director's Pioneer Award (NDPA) was initiated to support innovative investigators with the creativity and talent to pursue high risk and potentially high-impact ideas in the biomedical and behavioral sciences. Over the course of the program, the criteria for selecting awardees have shifted in several ways: from the sole merit of the individual investigator toward a combination of the merits of the candidate and the scientific idea, targeting women, minorities, and early-to-mid-career investigators. The purpose of the evaluation was to assess changes in the pilot program. The evaluation assessed the award selection process and determined whether the program operations were consistent with program goals.

Despite changes in program processes over time, the fundamental purpose and intent of the program did not change. Application reviewers, however, had different understandings of what were meant by key terms in the review criteria such as “pioneering.” Evaluators pointed out that there were challenges defining these terms without losing flexibility and diversity in the applications. They suggested exploring additional ways to identify non-traditional scientists who tend not to apply for NIH grants, and increasing the number of awards.

Report Title: FY2004 - 2008 NIH Director's Pioneer Award Process Evaluation - Comprehensive Report
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: IDA Science and Technology Policy Institute (STPI)
Record ID: 9332 (Report issued January 30, 2010)

How Did the National Cancer Institute Identify Metrics for the Small Business Innovation Research Program?

This study identified metrics to evaluate the National Cancer Institute (NCI) Small Business Innovation Research (SBIR) program. The SBIR program is structured in three phases: Feasibility; Full Research/Research and Development Effort; and, Commercialization without SBIR funds. The Metric Team selected two overlapping sets of metrics for the NCI SBIR program: one set based on innovation or the creation and dissemination of science-based discoveries intended for the market and the other based on commercialization or the process of getting innovations into the marketplace. The team then used the measures to analyze national survey data and compare the performance of the NCI SBIR program to all other NIH SBIR programs.

The team found that the NCI program performed similarly to all other NIH SBIR programs across the innovation and commercialization metrics they had selected. For future implementation of the metrics, the NCI Metric Team recommended that NCI support more robust data gathering by supporting NIH-wide data collection efforts; creating an NCI commercialization database; and collecting information on commercialization from awardees.

Report Title: Metrics for NCI SBIR Program
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: Humanitas, Inc.
Record ID: 9399 (Report issued June 30, 2010)

Is the Health Information National Trends Survey Program at the National Cancer Institute Working as Intended?

This study determined whether or not the Health Information National Trends Survey (HINTS) was achieving its process goals and meeting user needs. HINTS, a nationally representative survey, allows the National Cancer Institute and its partners to gather data about the public's need for cancer information, their preferred and trusted sources of cancer information, and their information-seeking behaviors. The evaluators conducted in-depth phone interviews and focus groups with current and potential users of HINTS, which included academics, cancer control planners, graduate students, Center of Disease Control and Prevention staff members, and cancer center communications directors.

The evaluators determined that user impressions of HINTS were very positive. Both current and potential users provided numerous suggestions for improving HINTS and the HINTS website including improving awareness and outreach; updating website navigation; using plain language principles when designing web pages; and increasing website content by adding information on additional types of cancer, survivorship and end-of-life issues.

Report Title: Process and Needs Assessment Evaluation of the HINTS Program
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: RTI International
Record ID: 9400 (Report issued August 30, 2010)

Do Teen Safe Driving Campaigns Encourage Teens to Adopt Safer Driving Practices and Increase Parental Involvement?

Researchers assessed the appropriateness and impact of the "Parents Are the Key" communication campaign's messages and creative materials. The messages and materials were intended to increase parental involvement in their teens driving education and experience, and encourage teens to adopt safer driving practices.

Parents are aware of the dangers of teen driving, yet not all are taking specific actions to prevent their teens from engaging in the risky behaviors that cause crashes, nor are all parents acting as positive role models for their children.

About six-in-ten parents responded that driving is their biggest worry as it relates to their teen, yet a majority of parents do not model safe driving behaviors themselves. About eight-in-ten parents strongly agreed that parents have a strong influence on their teen driver. Parents who read or heard campaign messages were significantly more likely to talk with their children about distractions such as eating or adjusting the stereo while driving, than parent respondents as a whole. Parents who were aware of the campaign were also more likely to discuss driving issues listed in the survey with their teen driver. CDC revised the campaign based on the evaluation findings, and released the campaign tool kit nationally.

Report Title: Development and Implementation of a Communication Campaign for Teen Safe Driving

Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Ann Dellinger, 770-488-4811
Performer: Ogilvy Public Relations Worldwide
Record ID: 9406 (Report issued April 30, 2010)

Has the Clinical Proteomics Technology for Cancer Program Been Implemented as Intended and What are Its Short-term Outcomes?

This study assessed the extent to which the Clinical Proteomic Technologies for Cancer (CPTC) program was implemented as intended and its short-term effects on the proteomic research community. The program was established to enhance technical abilities to identify and measure proteins accurately and increase reproducibility of proteomic biomarkers in biological systems and to advance proteomics as a reliable, quantitative field that can accelerate discovery and translational research. The program has three interrelated components: the Clinical Proteomic Technology Assessment for Cancer; the Advanced Proteomic Platforms and Computational Sciences; and Proteomic Reagents and Resources Core. This study evaluated processes and short-term outcomes. Data was collected from Clinical Proteomic Technology Assessment for Cancer participants and stakeholders through interviews and on-site visits as well as program reports, publications, and related grant and publication data.

The program achieved significant milestones. Moreover, the program successfully established a collaborative working relationship among the participating institutions and individuals and participating institutions were not in direct competition with one another. Efforts to standardize existing technology were accelerated by the program.

Report Title: An Extended Feasibility Analysis of the Clinical Proteomics Technology for Cancer Initiative
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: Macro International, Inc.
Record ID: 9418 (Report issued December 2, 2009)

How Do Evaluation Support Activities Complement the National Cancer Institute's Program Evaluation Efforts?

The National Cancer Institute (NCI) hired contractors to perform various evaluation support activities to meet the evaluation needs of the NCI division offices and centers. The Institute has a diverse research portfolio ranging from programs that focus on basic science to community-based cancer prevention and care delivery. Therefore, the evaluation goals and methodologies that were adopted for the evaluation activities differed widely across the Institute. Evaluation activities performed by the contractors included resource development, capacity building, evaluation training and related workshops, consultation, and technical assistance and support.

These support activities led to several completed and in-progress evaluation studies throughout the Institute, including the Analysis of NCI Translation of Research, which analyzed the link between Institute projects and drug patents; the Pancreatic Cancer Progress Report, which evaluated the range and extent of research and training programs in pancreatic cancer; and the Role in Milestones in Cancer Research, which assessed the Institute's “return on investment” by examining the link between Institute-supported research and recognized milestones in cancer research.

Report Title: List of Products and Accomplishments for Contractor Support for Multiple NCI Activities Related To Program Evaluations

Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: NOVA Research Company
Record ID: 9423 (Report issued May 31, 2011)

Is an Outcome Evaluation of the Childhood Cancer Survivor Study Feasible and Warranted?

The Childhood Cancer Survivor Study (CCSS) is a multi-institutional collaborative study, with 26 participating clinical centers, that serves as a national resource for investigating the long-term morbidity and mortality associated with cancer treatment regimens that have enabled dramatic increases in the survival of children treated for cancer during the past 30 years.

Most outcomes associated with CCSS were clearly defined and measurable; no extraordinary barriers to additional data collection were anticipated; and existing databases could be used as complementary data sources. An outcome evaluation of the CCSS was thus feasible; however, such an evaluation was not warranted. As the CCSS is a research study, peer review was determined to be the more appropriate mechanism for assessing its merit. Furthermore, evaluators found that a retrospective outcome evaluation of the CCSS would not help NCI address critical strategic issues with respect to the future of its pediatric cancer survivorship research portfolio. Lastly, evaluators identified several process-related issues during the study that might warrant attention in the future.

Report Title: Feasibility Study for Evaluation of the Childhood Cancer Survivor Study (CCSS)
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: Science and Technology Policy Institute (STPI)
Record ID: 9379 (Report issued March 31, 2010)

What Did the National Institute of General Medical Sciences Learn by Conducting a Process Evaluation of the Large-Scale Collaborative Project Awards Initiative?

The National Institute of General Medical Sciences (NIGMS) commissioned a process evaluation of the Large-Scale Collaborative Project Awards initiative, commonly known as the Glue Grants program, to assess program design, management and implementation. The Glue Grants program was launched in 1999 to facilitate collaboration among groups of independently funded investigators in order to solve complex biomedical research problems. The process evaluation focused on three main areas of interest: program planning and design; consortium-level implementation and management; and program-level implementation and management. Data was collected from Glue Grant Principal Investigators (PIs) and NIGMS staff members using a series of semi-structured interviews. Additional data sources included internal NIGMS planning documents, Glue Grant funding opportunity announcements, application materials, annual progress reports submitted by the PIs, NIH databases, and a book chapter authored by Glue Grants program staff members.

The Glue Grants' program-level objectives were not clear. The types of problems that would be most amenable to solution via the Glue Grants were not well-defined and the specific outcomes that were to be achieved through the integration of investigators were not apparent to program participants. As a result, the program's implementation was not entirely consistent with program objectives. Evaluators determined that challenges existed at the program level regarding the program's implementation and management, such as those associated with the ten-year timeframe of the grants, peer review of the Glue Grants due to the size and scope of the grants, and NIGMS' approach to managing the Glue Grants as cooperative agreements. Evaluators presented four recommended options: discontinue the grants program; maintain the program and continue to fund and manage the grants as individual cooperative agreements; transform the grants from a mechanism into a true initiative with clear goals and consistent, centralized management process; build a smaller program around NIGMS' priorities and the core functionality provided by the grants.

Report Title: Process Evaluation of the Large-Scale Collaborative Project Awards (Glue Grants) of the National Institute of General Medical Sciences (NIGMS)
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: IDA Science and Technology Policy Institute (STPI)
Record ID: 9426 (Report issued March 31, 2010)

How Has the Implementation of the National Institutes of Health Enhancing Peer Review Requirements Improved the Funding Opportunity Announcement Process?

Beginning in January 25, 2010, all pending and existing Funding Opportunity Announcements (FOAs) published in the National Institutes of Health (NIH) Guide for Grants and Contracts had to be compliant with the NIH Enhancing Peer Review requirements. These requirements imposed page limits and specified language that must be included in all NIH FOAs.

Evaluators conducted a content analysis of existing FOAs to examine their structure, format, and language and an evaluation task force determined what modifications could be implemented to reduce and simplify the FOA application process.

The efforts to reinvent the FOA templates improved the efficiency of the NIH FOA process by eliminating redundancies and reducing costs. Resources required to finalize the average FOA were also reduced as the 18 templates used in the original FOA development were consolidated into 7 templates.

Report Title: NIH Guide Implementation of the Peer Review Recommendations Overview Committee and Subgroups: FOA Conversion and Template Reinvention
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: Ripple Effect Communications
Record ID: 9430 (Report issued August 18, 2010)

How Are Science and Engineering Research Facility Space Needs Determined?

Evaluators analyzed data from the National Science Foundation survey of Science and Engineering Research Facilities to identify impact on the National Institutes of Health's Research Facilities Improvement Program (RFIP). RFIP provides grants to public and private nonprofit biomedical institutions to expand, remodel, renovate or construct research and animal facilities. Data from the survey informs RFIP about biomedical and behavioral research space needs.

Overall, research space at academic institutions increased by 4% between FY 2007 and FY 2009, and the amount of space for biological and biomedical sciences increased by 12%, the largest increase among all science and engineering fields. Based on the needs identified in this study, program resources may not be sufficient to fully address the number of construction and renovation projects planned for in future years.

Report Title: Science and Engineering Research Facilities http://www.nsf.gov/statistics/nsf11323/pdf/nsf11323.pdf
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: Westat, Inc.
Record ID: 9436 (Report issued September 30, 2011)

What Updates Were Incorporated into the 8th Edition of the Guide for the Care and Use of Laboratory Animals?

The National Academy of Sciences convened a committee of experts to update the Guide for the Care and Use of Laboratory Animals to reflect advances in science and technology that provide effective and humane care of laboratory animals. Previously revised in 1996, the Guide assists institutions in caring for and using animals in ways judged to be scientifically, technically, and humanely appropriate. A committee of experts met in 2008 and heard from various sponsor organizations. The committee conducted three public forums and met in closed session in order to review and address stakeholder comments. The Guide was released to the public in June, 2010.

The 8th edition of the Guide strongly reaffirmed both the performance standards approach for animal care and care practices and the basic precepts of the US Government Principles for the Humane Care and Use of Laboratory Animals. Specific updates included expanded information on environmental enrichment, animal well-being and psychological health as well as transportation, pain and distress, euthanasia, and veterinary medicine. In addition, for the first time, the Guide contained information about the care and use of fish and other aquatic species, and initial guidance on animal biosecurity practices.

Report Title: Update of the Guide for the Care and Use of Laboratory Animals: NIH Task Order 188 Summary of Salient Results

Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: National Academy of Sciences, Institute for Laboratory Animal Research (ILAR)
Record ID: 9437 (Report issued December 31, 2010)

How Well Does the NIH Research, Condition, and Disease Categorization System Compare to Alternative Categorization Packages?

This study compared alternatives for implementing the National Institutes of Health (NIH) Research, Condition, and Disease Categorization (RCDC) System. RCDC is an information technology and document categorization system that uses the Collexis platform to automate and standardize the classification of research projects into specific research and disease categories, and to assist in reporting research expenditures to Congress and the public. The evaluators assessed the relative suitability of the current leading categorization packages by comparing them to the current RCDC/Collexis implementation. The categorization packages were evaluated using business-related criteria, functionality criteria, technical criteria and usability criteria. The evaluators also assessed the levels of agreement between the RCDC system and human raters, using 2005 and 2006 validity feedback provided by subject matter experts within the NIH.

Three of the five alternative categorization packages tested scored higher than Collexis on the business-related criteria, and scored equal to or higher than Collexis on the functionality criteria. In addition, two of the three scored higher on the technical criteria. Collexis scored highest, however, on the usability criteria. The median overlap fraction across categories was approximately 37%. The evaluators recommended further analysis of the validity and categorization process, however, before an alternative categorization package should be considered more feasible for NIH's research categorization.

Report Title: Alternatives for Implementing the National Institutes of Health's Research, Condition, and Disease Categorization System

Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: IDA Science and Technology Policy Institute (STPI)
Record ID: 9438 (Report issued May 31, 2011)

How Satisfied Are Stakeholders with the Changes to the National Institutes of Health Peer Review Process?

This study assessed stakeholder satisfaction with changes made to the National Institutes of Health (NIH) peer review process that were based on the recommendations of the initiative. NIH started the Enhancing Peer Review Initiative in 2007 as a self-study to promote fairness, efficiency, and effectiveness in the peer review process. Changes implemented as a result of the initiative included: 1) a nine-point scoring system; 2) criterion scores; 3) a bulleted critique format and structured critique templates; 4) enhanced review criteria; and, 5) clustering of applications that propose clinical research and clustering of research project grant applications submitted by New Investigators. Evaluators administered satisfaction surveys to five groups of stakeholders, including NIH grant applicants, NIH peer reviewers, Scientific Review Officers (SROs), Program Officials (POs) and Advisory Council members.

Stakeholders rated the nine-point scoring system as adequate for communicating meaningful differences in application quality. Criterion scores were rated by POs to be among the most helpful elements of the initiative for advising applicants. Applicants, however, were more equivocal about the usefulness of criterion scores in the application process. Neither POs nor applicants rated the bulleted format for reviewer critiques as helpful for understanding the recommendations of the review group. The clustering of applications submitted by New Investigators and those proposing clinical research were uniformly judged to be a positive change to the NIH peer review process as a result of the Enhancing Peer Review Initiative.

Report Title: Enhancing Peer Review Survey Results Report
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: RTI International
Record ID: 9439 (Report issued December 31, 2010)

How Should the Management Structure of the National Centers for Biomedical Computing Be Improved?

The National Centers for Biomedical Computing (NCBC) seek to transform the way biomedical research is conducted by overcoming hurdles and filling knowledge gaps. The short-term high impact/high-risk program spans all areas of health and disease research and boundaries of National Institutes of Health Institutes and Centers. The seven centers are intended to be the core of the networked effort to build the computational infrastructure for biomedical computing in the nation. The centers cover systems biology, image processing, biophysical modeling, biomedical ontologies, information integration, and tools for gene-phenotype and disease analysis. They create innovative software programs and other tools that enable the biomedical community to integrate, analyze, model, simulate, and share data on human health and disease. The study sought to understand the strengths and weaknesses of the program's management structure; solicit input and ideas from program stakeholders about how the management processes might be improved; and estimate the level of effort that had been devoted to program management by various participants to inform the budgeting process for the next term of funding.

Through interviews with the Program Officers and Lead Science Officers and the questionnaires sent to the seven Principal Investigators, stakeholders provided several suggestions for improving program management. The role of the Science Officers needed to be better defined. Communication needed to be improved, both within the project team and outwards to the Principal Investigators. The collaborative management structure of the NCBCs was good but could be enhanced to be more effective.

Report Title: Results of Questionnaire for NCBC PIs Regarding NIH Management Practices: UPDATED
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: IDA Science and Technology Policy Institute (STPI)
Record ID: 9476 (Report issued March 31, 2010)

What Evaluation Approach is Most Appropriate for the Pharmacology Research Associate Program?

The Pharmacology Research Associate program is an intramural postdoctoral training program administered by the National Institute of General Medical Sciences (NIGMS). The program is intended for individuals with backgrounds in the basic or clinical sciences who wish to obtain advanced experience in an area of pharmacology, or for those with a pharmacology background to gain experience in new fields. This exploratory study identified the best evaluation approaches and examined available program data.

Evaluators concluded that the program filled an important gap in the postdoctoral training offered at the NIH, of which NIGMS is a component. Overall, Pharmacology Research Associate fellows were positive about program activities, emphasizing their utility for networking with other researchers on campus and with potential employers. The fellows described program directors as accessible, supportive, and very helpful in making vital and, in retrospect, appropriate career choices. The evaluators, however, found that the program focuses more on mentoring and career advancement than research training. The evaluators suggested that a combination process/outcome evaluation would be the most cost effective approach.

Report Title: NIGMS Evaluation Support Services Task 2: Exploratory Assessment of the Pharmacology Research Associate Program (PRAT)
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: Abt Associates, Inc.
Record ID: 9482 (Report issued September 29, 2011)

Does the Interdisciplinary Research Consortium Program Encourage Greater Interdisciplinary Collaboration?

The Interdisciplinary Research (IR) Program is one of eight National Institutes of Health (NIH) Common Fund initiatives developed to overcome barriers to medical research that are inherent in single discipline approaches. The broad goal of the IR program is to change academic research culture to facilitate interdisciplinary approaches that dissolve academic department boundaries within academic institutions, increase cooperation between institutions, train scientists to cultivate interdisciplinary efforts, and build bridges between the biological sciences and the behavioral and social sciences. The Interdisciplinary Research Consortium (IDRC) program is one of four components funded under the IR program. This study was conducted as a mid-course evaluation of the program to examine consortia management structure and management issues at the project and program level, leadership qualities, and the experiences of investigators and post-doctoral trainees.

Most participants rated the IDRC program positively. Investigators rated team leadership favorably with the most positive ratings being observed for supporting innovative projects. They also reported positive perceptions of the team science climate at their consortium. Trainees strongly agreed that the IDRC program exposed them to new scientific approaches and paradigms and that the program was a better way to explore a biomedical problem than single discipline research. Most Program Officers thought the program was a novel and innovative approach to solving complex problems and agreed that the scope and scale of such projects would not be possible without cross-institute collaboration and the use of Common Fund resources.

Report Title: Facilitating and Experiencing Interdisciplinarity in Biomedical Research (Mid-course Evaluation of the Interdisciplinary Research Consortium Program: an NIH Common Fund Program)
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: Abt Associates Inc.
Record ID: 9496 (Report issued August 31, 2011)

Is an Evaluation of the National Cancer Institute's Office of Advocacy Relations Feasible and Warranted?

This study determined the overall feasibility of conducting a comprehensive process and outcome evaluation of the National Cancer Institute's Office of Advocacy Relations (OAR). OAR was created to strengthen the Institute's communications, collaborations, and relationships with national advocacy and voluntary organizations that work with consumer advocates, as well as scientific and professional societies concerned about cancer. The feasibility study used four data collection approaches: review of background materials and previous studies, review of existing OAR databases, review of relevant websites and e-publications, and interviews with current and former OAR staff.

The evaluators concluded that a comprehensive process and outcome evaluation was feasible and warranted. They determined that it was appropriate to assess OAR's internal staff and external advocacy community customers' utilization of and satisfaction with OAR services; and the nature, quality, and impact of relationships and collaborations resulting from OAR activities. They recommended and designed an evaluation consisting of three phases: develop and implement tracking systems, measure short-term outputs and outcomes, and measure intermediate and long-term outcomes.

Report Title: National Cancer Institute Office of Advocacy Relations Feasibility Study

Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: The Madrillon Group, Inc.
Record ID: 9504 (Report issued November 16, 2010)

How Effective Has the Agency for Healthcare Research and Quality's National Guideline Clearinghouse Been on the Dissemination of Evidence-based Clinical Practice Guidelines?

The National Guideline Clearinghouse is a publicly available database of evidence-based clinical practice guidelines and related documents. It provides Internet users with free online access to guidelines at http://www.guideline.gov. The evaluation survey examined the following key dimensions: 1) awareness of the use of the Clearinghouse; 2) use of other guidelines sources to obtain clinical guidelines; 3) general understanding of Clearinghouse non-users; 4) users' assessment of the Clearinghouse, 5) differences among users by stakeholder group; and, 6) the impact and influence of the Clearinghouse on the development, use and dissemination of guidelines by stakeholder groups.

The survey demonstrated that the Clearinghouse is a well-known and trusted source for clinical guidelines across all stakeholder groups. In particular, respondents indicated that the Clearinghouse greatly influenced guideline developers, providers' ongoing learning efforts and identifying guidelines, medical librarians' ability to meet their client's needs, medical librarians' and researchers' ability to identify current and high quality guidelines, measure developers' data collection processes, and policymakers' and purchasers' ability to identify guidelines and convert clinical information. Those that used the Clearinghouse were more satisfied with it relative to other guideline sources, had used it for a long time, were likely to recommend it to others, and found the guidelines trustworthy. Respondents also identified areas for expanding use and impact.

Report Title: Evaluation of the National Guideline Clearinghouse, http://dev.ahrq.gov/about/evaluations/ngceval.htm
Agency Sponsor: AHRQ, Agency for Healthcare Research and Quality
Federal Contact: Boone, Cynthia, 301-427-1212
Performer: AFYA, Inc.
ID Number: 9667 (Report issued September 29, 2011)

Is a Process and Outcome Evaluation of the Network for Translational Research Program Feasible and Warranted?

The National Cancer Institute (NCI) Cancer Imaging Program, within the National Institutes of Health (NIH), commissioned a feasibility study of the Network for Translational Research (NTR): Optical Imaging in Multimodal Platforms program (hereafter referred to as the Network for Translational Research program, or NTR program). The study addressed three objectives: determine whether a process and outcome evaluation of the program was feasible and warranted; determine what form that evaluation should take and how best to conduct it; and gather baseline data on the NTR program's current functioning. Beginning in 2008, the NTR program funded four multidisciplinary, multi-institutional research centers for a five year period. Each center consists of a team of multidisciplinary investigators from two or more academic institutions and one or more industrial partners. Together, the four centers constitute a network that works to bring multimodality cancer imaging to the clinical environment. The overall mission of the NTR program is to develop, optimize, and validate multimodal molecular imaging platforms and methods for entering single- or multisite clinical trials data for human subjects. The goal of the NTR program is to accelerate the translational research of in vivo multimodal imaging and/or spectroscopic platforms from the laboratory to the preclinical level, and ultimately, to the clinical level.

Evaluators concluded that a process and outcome evaluation of the NTR program is both feasible and warranted. Moreover, they determined that the NTR program had reached a level of maturity to make evaluation possible and appropriate data and data sources were available to support an evaluation. Evaluators also cited an urgent need to learn more about programmatic models for conducting translational technology development research. For the process and outcome evaluation, they recommended a cross-sectional design using the multiple-case study approach and they provided NCI with a logic model to guide the evaluation.

Report Title: Network for Translational Research Program Evaluation Feasibility Study: Final Report
Agency Sponsor: OASPE-OPPS, Office of Planning and Policy Support
Federal Contact: Rosanna Ng, 301-496-5367
Performer: The Madrillon Group, Inc.
Record ID: 9734 (Report issued April 30, 2011)

How Can the Diabetes-Based Education in Tribal Schools Project Improve Recruitment and Retention?

This study evaluated the success of recruiting teachers to the Diabetes-Based Education in Tribal Schools (DETS) Project and the success of the distribution of the curriculum materials. The project is a cooperative effort among the National Institutes of Health, the Centers of Disease Control and Prevention, the Indian Health Service and eight Tribal Colleges and Universities. The DETS Project is a K—12 curriculum that uses a multidisciplinary approach as part of a national effort to decrease the incidence and improve the care of type 2 diabetes among American Indians and Alaska Natives (AI/AN).

The study showed a strong and sustained effort to recruit teachers and distribute curriculum materials. The close correlation between recruitment efforts and shipped orders for DETS indicated that the recruitment effort was working. DETS materials were ordered and shipped to locations where teachers had been recruited to use the materials. Efforts to recruit teachers were sustained, showing monthly increases. The most promising recruitment efforts were those that had person-to-person contact and included small presentations, personal development workshops, in-person conferences, large conferences, and exhibit tables at large festivals.

Report Title: Diabetes-based Education in Tribal Schools (DETS) Process Evaluation
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: PS International, Doug Coulson
Record ID: 9736 (Report issued October 31, 2010)

How Well Do the AIDSinfo Website and Its Spanish Sister Site, infoSIDA Meet the Needs of Their Target Audiences?

Evaluators studied the AIDSinfo website and its sister site, infoSIDA, to determine how to improve the websites to better meet the needs of their audiences. The two websites are a U.S. Government source for the latest HIV/AIDS medical practice guidelines, HIV treatment and prevention clinical trials, and other HIV-related research information. Although both websites largely offer the same information, the infoSIDA website places greater emphasis on providing consumer health information, both for patients and family members (who search for Spanish-language materials) and for clinicians (who often need complementary materials for their Spanish-speaking patients). The two websites also attract very different primary audiences: health care providers for AIDSinfo and patients and caregivers for infoSIDA. For the AIDSinfo website, evaluators conducted usability testing with health care providers. For the usability testing of the infoSIDA website, evaluators used another group of participants which consisted of patients, physicians, caregivers, nurses, health educators, and outreach workers.

Both sites were valuable to health care providers, patients and consumers. However, some participants were not sure of the intended audiences for the AIDSinfo and infoSIDA websites or the goals of each site. Some health care providers wondered if the AIDSinfo website was intended for professionals or patients. They also expressed concern that the patient education materials/factsheets were too technical and difficult for patients to understand. Others who participated in the usability testing of the infoSIDA website stated that there was no way to determine which links (for guidelines and factsheets) were for professionals and which were for a general audience. Evaluators provided recommendations for improving both websites.

Report Title: Usability Evaluation of the AIDSinfo and infoSIDA Web Sites
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: ICF International
Record ID: 9746 (Report issued January 6, 2011)

Does the Redesigned Home Page of the Radiation Emergency Medical Management Website Enhance Usability and Navigation?

During the Fukushima nuclear incident, the Radiation Emergency Medical Management website (REMM @ http://remm.nlm.gov ) was accessed more than 170,000 times on the first day and hundreds of thousands of times during incident. It has clearly become a vital resource for information during a radiation incident response. The mobile REMM app in both the Apple and Android app stores has been downloaded thousands of times, especially during the Fukushima incident.

Evaluators studied the Radiation Emergency Medical Management (REMM) website to determine whether a redesigned home page would enhance the usability and navigation of the website. The REMM website provides guidance for health care providers, primarily physicians, about clinical diagnosis and treatment of radiation injury during radiological and nuclear emergencies; just-in-time, evidence-based, usable information with sufficient background and context to make complex issues understandable to those without formal radiation medicine expertise; and web-based information that is also downloadable in advance, so that it would be available during an emergency if the internet is not accessible. Usability tests were conducted using individual in-person and remote sessions.

Participant reactions were positive and some of the redesign suggestions were implemented for enhanced usability of the site. Participants found the content valuable, including the depth of the information, having all information about radiation medical management gathered in one site, and the useful tools, including the algorithms, dose estimators, and detailed procedures. Most participants preferred aspects of the new home page design, stating that they would use the site again, and that they would recommend it to others. Participants recommended that an expanded display of the multimedia library would improve the usability of the website and highlight its value to potential users. This was implemented. A number of the requested enhancements have been added to the site including a multimedia carousel, and a redesign of the “right side” of the page have made new and select resources more accessible.

Has the Redesign of an NIH Public Website Improved Its Usability, Usefulness, and Accessibility?

This study evaluated the usability, usefulness, and accessibility of the redesigned National Institute on Deafness and Other Communication Disorders (NIDCD) public website. The website seeks to educate the general public and to provide communication disorder–related resources for scientists, educators, and health care professionals.

The redesigned website enabled users to navigate quickly to the information they sought. The site scored higher than 80 in satisfaction in eight out of the twelve months a customer satisfaction survey was implemented. Scores for elements of satisfaction—content, functionality, look and feel—were consistently in the mid-to-upper 80's and low 90's, and no scores for the 12 month period were below 78. The Usability Compliance Rating comparing NIDCD's specific site pages against usability best practices, as gauged by the application of approximately 50 usability indicators, was 72%. The Usability Compliance Rating for the research landing page was 64%. Recommendations were presented to NIDCD to help improve the website's navigation and presentation.

Report Title: Final Report of the Evaluation of the NIDCD Web Site Redesign
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: Westat, Inc.
Record ID: 9763 (Report issued September 30, 2011)

Does the Tool to Create Online Interactive Training on the CancerSPACE Website Meet the Needs of Its Users?

The Cancer Institute's CancerSPACE (Simulating Practice And Collaborative Education) website contains e-learning tools for healthcare providers in community health centers, and aims to increase cancer screening rates for underserved and minority populations that bear a disproportional share of the cancer burden. The primary goal of the website is to provide an on-line, engaging, realistic collaborative learning environment that is accessible to trainers and clinical staff.

Participants liked the concept of the authoring tool that allows trainers to develop customized simulations. In particular, they liked that it was interactive. However, when participants created vignettes and stories, they did not quite understand how the vignettes and stories fit together into "playlists" of individual training modules or of related training materials. Additionally, participants found the screen layout and labeling difficult to understand. In particular, the labels used throughout the site to refer to the training tool ("Game") and the authoring tool ("Author") did not make sense to the participants. Based on these results, the evaluators concluded that the authoring tool does not meet the needs of its intended audience. The authoring tool needed more flexibility, better labeling, an improved workflow, and possibly, additional features to improve its usefulness and usability.

Report Title: Innovation for Health Usability Study Findings: A Study to Determine the Suitability and Usability of a Tool to Create Online Interactive Training
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: User Centered Design
Record ID: 9793 (Report issued October 15, 2010)

How Can the Usability of the Surveillance, Epidemiology and End Results Training Website Be Improved?

Evaluators assessed the usability of the National Cancer Institute's Surveillance, Epidemiology and End Results (SEER) Training Website and the functional changes being considered for the website. The website trains and supports Certified Tumor Registrars (CTRs) and serves as a resource to increase knowledge in the cancer registration and surveillance fields. The functional changes are intended to provide CTRs with newer and more relevant information that reflects current trends in the field of cancer registration and surveillance, and to allow the website to serve a wider audience of users. The testing was followed by an expert review to identify areas where the SEER Training Website failed to meet industry standards and best practices.

The quality of information on and usability of the website were both rated highly. However, experienced practitioners raised concerns about the timeliness of content updates. The expert review revealed usability issues concerning content and navigation on the home page and landing pages, functionality with navigation, and layout on specific sub-pages. Evaluators recommended changes to navigation controls, a new "Resources" section, a new "Help" section, and a new format for Case History within Hands on Exercises.

Report Title: Final Report on SEER Training Usability Evaluation Project: Expert Review and Three Rounds of Usability Evaluation
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: Health Data Institute
Record ID: 9794 (Report issued June 30, 2011)

How Has the National Institutes of Health Enhanced the Research Portfolio Online Reporting Tools Website?

The National Institutes of Health (NIH) created the Research Portfolio Online Reporting Tools (RePORT) website as an NIH-wide data system and public-facing website to improve internal and external access to data on NIH research in order to facilitate evaluation and management of research portfolios and to satisfy reporting requirements of the NIH Reform Act of 2006. Following the initial design and development of the RePORT website, NIH implemented a series of system enhancements. These included enhancements to the RePORT website and the creation of the RePORT Expenditures and Results (RePORTER) module and NIH reports (the NIH Readiness Tool) on funds provided by the American Recovery and Reinvestment Act of 2009. The launching of the RePORTER module introduced major enhancements to the database's search and export capabilities. The creation of the ExPORTER site allowed queries of the RePORTER database to be downloaded.

The enhancements to the RePORT website were well-received. NIH received many requests for copies of the database for evaluation and science policy research. Recommendations to further improve and enhance RePORT were to gather additional feedback on the RePORT website from NIH stakeholders (both internal and external), continue to refine the website; and continually promote trans-NIH reporting business rules, reporting processes, and methods through the NIH RePORT initiative.

Report Title: NIH Research Portfolio Online Reporting Tools (RePORT) Phase II Website http://report.nih.gov
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: NETE Solutions, Inc.
Record ID: 9796 (Report issued November 15, 2010)

Is An Outcome Evaluation of the New Innovator Award Program Feasible?

The goal of the New Innovator Award (NIA) program is to stimulate highly innovative research. The program seeks to accomplish this by funding high-risk, high-reward research and supporting promising young investigators who have not yet received a large research award such as the individual investigator research project grant. To examine possible approaches for an appropriate outcome evaluation design, the evaluators conducted interviews with program staff, held several focus groups with select awardees and performed a literature review. They also held discussions with experts who had performed evaluations of NIA-like programs and assessed the programs' degree of similarity in motivation and population to the NIA to identify a comparison group.

The evaluators determined that an outcome evaluation of the NIA program was feasible. They proposed a quasi-experimental design using a comparison group of early stage investigators who were awarded research project grants. They proposed that the outcome evaluation be conducted as an expert review, and supplemented by bibliometric analysis, administrative data, surveys of awardees, and case studies of selected awardees.

Report Title: Feasibility Study of An Outcome Evaluation of the National Institutes of Health's New Innovator Award Program

Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: IDA Science and Technology Policy Institute (STPI)
Record ID: 9822 (Report issued January 31, 2011)

Does the Director's Pioneer Award Program Result Stimulate Pioneering Research?

The National Institutes of Health (NIH) commissioned a study to assess whether the short-term outcomes of the NIH Director's Pioneer Award (NDPA) program were consistent with its original goals, and to evaluate the impact of the NDPA on NIH and its funding of high-risk research. The NIH launched the NDPA program in 2004 to support individual scientists of exceptional creativity who propose pioneering approaches to major contemporary challenges in biomedical research. The study assessed the research conducted by NDPA awardees funded during the first two years of the program (2004 – 2005) and focused on answering two study questions: Did the NDPA awardees conduct pioneering research with the NDPA funds, and what were the spillover effects of the program? The overall goal of the outcome evaluation was to develop a valid set of measures that could be used to evaluate the NDPA program as well as other high-risk, high-reward programs in the future.

Overall, the short-term outcomes of the program were positive. The study showed the diversity of the types of risks and rewards in the research funded through the NDPA, and the variety of outputs emerging from the NDPA. The majority of the awardees pursued their originally proposed ideas and most did not believe that their research would have been funded through traditional mechanisms. The majority of experts agreed that the accomplished research was pioneering; some believed that the research was so scientifically strong that it would have been funded under other mechanisms. Regardless, most experts stated that the NDPA added value to the NIH portfolio of programs. However, despite the range of bibliometric outputs produced, evaluators concluded that the time since the awards was too short to fully assess the impact of the outputs and the small number of awardees inhibited comparisons with the pool of unfunded applicants. Additionally, they determined that standard bibliometric measures were inappropriate for evaluating a program like the NDPA that encourages non-traditional research and expects some level of failure in the projects funded.

Report Title: Outcome Evaluation of the National Institutes of Health (NIH) Director's Pioneer Award (NDPA) FY 2004-2005
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: IDA Science and Technology Policy Institute (STPI)
Record ID: 9823 (Report issued July 22, 2011)

Does the Mind-Body Interactions and Health Program Promote Research Focusing on This Topic?

The outcome evaluation of the Mind-Body Interactions and Health (MBIH) program sought to determine whether the program had met its programmatic goals and objectives, and to assess the contributions the program had made to the broader field of mind-body interactions and health research. The program was established by Congress in 1999 (and ended in 2009) to expand research on interactions among the brain, mind, body and behavior and on the mechanisms by which factors such as emotions, cognitions and attitudes, and social and behavioral phenomena directly affect physical and mental health.

The MBIH program achieved its two main goals, and the research centers and research projects produced clear and positive effects across all five of the research benefit categories examined in the evaluation. The evaluation provided clear evidence that two programmatic research objectives were accomplished (facilitating interdisciplinary collaboration and innovation in research ideas, and building a capacity for mind-body research through cultivation and development of research personnel and funding of research core services).

Report Title: The Mind-Body Interactions and Health Program Outcome Evaluation
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: The Madrillon Group, Inc.
Record ID: 9824 (Report issued September 30, 2011)

Has the Community Cancer Center Program Improved Cancer Care at Participating Hospitals?

The National Cancer Institute Community Cancer Center Program (NCCCP) was launched as a three-year pilot program designed to help build a community-based research platform to support basic, clinical, and population-based research on cancer prevention, screening, diagnosis, treatment, survivorship, and palliative care at community hospitals. Evaluators assessed whether participation in the program changed the quality of cancer care provided over time and in comparison to similar hospitals that did not participate. Five quality of care measures associated with breast and colon cancer treatment were examined in the evaluation.

A small number of hospital and market characteristics—more registry staff, history of performing more quality studies, larger cancer patient population, and hospital location in metro or small metro/suburban counties—were consistently associated with greater quality of care among NCCCP hospitals. Other hospital and market characteristics were likely associated with only one or a subset of quality measures, and might show opposite associations with different measures. NCI concluded that it was reasonable for only a few factors to be consistently associated with improvements for all five quality measures.

Report Title: Comparative Health Outcomes Analysis for the NCCCP Evaluation: Final Report

Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: RTI International
Record ID: 9825 (Report issued October, 2011)

Is an Outcome Evaluation of the National Cancer Institute's Mouse Models of Human Cancers Consortium Feasible?

The Mouse Models of Human Cancers Consortium is a trans-disciplinary program, which focuses on the translational uses of genetically engineered mouse cancer models and other related mouse genetic resources. The initial goal of the program was to stimulate the integration of mouse models into cancer research. Subsequently, the National Cancer Institute (NCI) incorporated increased translational science goals for research-focused awards and modified the Consortium management structure to encourage more science leadership awards. This evaluation quantified the outcomes that would be appropriate for future evaluations of the program.

Evaluators concluded that an outcome evaluation was feasible and warranted. They provided an evaluation plan with an integrated study with two stages: First, that NCI conduct a process evaluation to identify successes and opportunities for mid-course correction regarding Consortium activities and organization; second, that NCI conduct an outcome evaluation near the end of the current award period to capture the scientific impact of the modified Consortium management structure.

Report Title: Mouse Models of Human Cancer Consortium Feasibility Study: Final Report

Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: IDA Science and Technology Policy Institute (STPI)
Record ID: 9826 (Report issued August 26, 2011)

What Program Models, Implementation, Screening Costs and Results Are Associated With Five Colorectal Cancer Screening Demonstration Sites?

From 2005-2009, CDC funded five sites to implement the Colorectal Cancer Screening Demonstration Program to screen low-income, under or uninsured men and women ages 50-64 living in the US at or below 250-350% of the federal poverty level. This multiple component evaluation provided a descriptive summary of program models and implementation processes, screening results, and a cost analysis. Program design was influenced by two critical factors: pre- existing infrastructure and the need to adapt programs to fit local service delivery structures. Partnerships were integral to the demonstration program at all stages of its development.

The implementation experience pointed to the importance of utilizing a comprehensive and systematic screening approach. Well-defined patient pathways and clinical protocols, clear mechanisms for patient referrals, and the development of quality assurance and tracking systems were imperative to ensure high quality screening and patient adherence. A wide variability was found between fecal occult blood tests ( FOBT) and colonoscopies. Polyp, adenoma and cancer detection rates were five to ten times lower for those tested with a single FOBT compared to screening colonoscopy in average risk persons, with even higher colonoscopy polyp, adenoma, and cancer detection rates among those at increased risk for colorectal cancer. Screening using FOBT was found to be much less costly than colonoscopy, both per person and per program. The cost analysis found that start-up costs are significant and that in-kind contributions play a critical role in overall program funding.

Report Title: Development of a Framework for a Demonstration Colorectal Cancer Screening Program with an Evaluation Component

Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Amy DeGroff, 770-488-2415

Performer: CDC, Information Management Services, Research Triangle Institute
Record ID: 8710 (Report issued May 31, 2010)

What are Proximal Performance Indicators for the National Diabetes Prevention and Control Program Efforts Towards Achieving the Diabetes National Goals?

CDC, through its Proximal Indicators Project, developed a set of rated and ranked indicators for planning implementation strategies and measuring the impact of Diabetes Prevention and Control Program's (DPCP) core interventions. Proximal indicators are the short and intermediate-term indicators that map to the outcomes articulated on the Program Logic Model for DPCPs. This Program Logic Model was revised in 2012 to incorporate the four-domain framework being employed by all Divisions in the National Center for Chronic Disease Prevention and Health Promotion. Indicators will support evaluation and program improvement at the grantee level as well as efforts to evaluate the collective impact of state-based DPCPs.

One hundred candidate indicators were identified through an extensive literature review, a review of current measurement systems used by the CDC's Division of Diabetes Translation and other chronic disease programs, and interviews with DPCPs. The project resulted in a guidance document with a set of indicators that will be used by DPCPs.

Report Title: Proximal Indicators for State-based Diabetes Prevention and Control Programs

Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Karen Richard-Lee, 770-488-6387

Performer: Research Triangle Institute
Record ID: 9222 (Report issued June 30, 2010)

Objective: Foster innovation to create shared solutions

Did Institutions take Advantage of the Option to Re-budget National Research Service Award Training Grant and Fellowship Funds?

National Research Service Award (NRSA) training grants and fellowships at the National Institutes of Health (NIH) have been governed by a formal tuition reimbursement policy since the mid-1990s. Increases in tuition, fees, and health insurance, however, began to limit the number of trainees that the NIH could support. In FY 2007, NIH began testing a new funding policy that gave institutions the option to re-budget grant funds. This evaluation assessed the extent to which institutions transferred or re-budgeted training grant funds between stipends and tuition and fees.

Of the 1,514 NRSA institutional training grants eligible to rebudget funds between stipends and tuition/fees, ten reported doing so. For those ten training grants, the amount of funds re-budgeted typically represented less than a full training slot as several of those institutions only re-budgeted modest amounts of funding that became available when a training appointment ended early, or a postdoctoral trainee was appointed at a lower experience level than anticipated. This evaluation did not find widespread re-budgeting of stipend funds for tuition. However, the option to re-budget NRSA training grant funds between the stipend and tuition/fee categories appeared to provide useful flexibility to training program directors when stipend funds unexpectedly become available, and therefore, the option should continue to be offered.

Report Title: Ruth L. Kirschstein National Research Service Award Program Policy Changes for FY 2007: Implementation and Implications
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: Diana Trunnell (Clarksburg, MD)
Record ID: 9386 (Report issued October 31, 2009)

Objective: Increase our understanding of what works in public health and human service practice

How Can the Effectiveness of Risk Communication Be Enhanced?

Communication is a critical component of helping individuals prepare for, respond to, and recover from emergencies. However, there is limited knowledge about how to best communicate with at-risk populations in emergencies. This study examined the current literature, available web-based resources, and the experiences of state and local officials to identify a number of advances in the area of risk communication for at-risk populations. The study also identified a number of remaining challenges to effective risk communication.

Community-based participation strengthens emergency preparedness, response, and recovery for at-risk populations. Training through exercises and drills that include risk communication for at-risk populations may improve response to future disasters. Evaluating the implementation of risk communication programs and impact of risk communication efforts is critical but systematic efforts are lacking. To be effective, risk communicators must be trained to understand emergency risk communication, know their stakeholders, and be trusted in the community. Reaching at-risk populations requires the use of multiple channels, formats, and tools.

Report Title: Analysis of Risk Communication Strategies and Approaches with At-Risk Populations to Enhance Emergency Preparedness, Response, and Recovery: Final Report http://aspe.hhs.gov/daltcp/reports/2008/emergfr.htm
Agency Sponsor: OASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Emily Rosenoff, 202-690-6443
Performer: Rand Health
Record ID: 9333 (Report issued October 16, 2009)

What Is The Status Of Health Disparities, and What Community Based Interventions Are Effective In Eliminating Health Disparities In Minority Communities?

Racial and Ethnic Approaches to Community Health (REACH US) is a community-based multilevel program that approaches the elimination of racial and ethnic health disparities. REACH U.S. Risk Factor Survey is a program evaluation done through an annual survey in 28 minority communities. Within each community, a total of 900 residents are surveyed each year. Individuals are persons over the age of 18 who are within the race and ethnic group (i.e., African American, Hispanic, American Indian, and Asian/Native Hawaii/Pacific Islander) targeted by the specific REACH community. The survey focuses on the following areas: health-related quality of life, access to health care, diabetes screenings and treatment, physical activity, consumption of fruits and vegetables, cigarette smoking, hypertension and cholesterol screening and treatment, knowledge of the signs and symptoms of heart attack and stroke, mammography and cervical cancer screening, and influenza and pneumonia vaccinations.

Data from the REACH Risk Factor Survey showed that the REACH U.S. program helped people to significantly reduce their health risks and manage their chronic diseases. Over a 4-year period, the cholesterol screening rates for Hispanics living in REACH communities increased steadily, from 56.3% in 2002 to 68.6% in 2006. In 2002, the cholesterol screening rate for African Americans living in REACH communities (74.2%) was below the national average (76.2%). By 2006, the rate had risen to 78.8%, which was above the national average of 75.2% for that year. In REACH communities that focused on breast and cervical cancer prevention, the percentage of women who reported having a Pap smear in the previous year increased from 81% in 2002 to 86% in 2006. The rate of cigarette smoking among Asian American men in REACH communities decreased from 42% in 2002 to 20% in 2006.

Report Title: REACH U.S. Risk Factor Survey

http://www.cdc.gov/reach/risk_factor_survey/index.htm

Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Youlian Liao, 770-488-5229

Performer: National Opinion Research Corporation
Record ID: 9403 (Report issued September 30, 2010)

How Successful Were Tribal Motor Vehicle Injury Prevention Programs At Reducing High Rates Of Motor Vehicle Crashes, And Increasing Restraint Use?

This project evaluated the four CDC-funded Tribal Motor Vehicle Injury Prevention Programs. From 2004 to 2009, CDC's National Center for Injury Prevention and Control (NCIPC) funded four Native American Tribes to design, implement, and tailor effective injury prevention programs to reduce motor vehicle-related injuries and deaths among members of their communities. The goals of this evaluation project were to: 1) determine if evidence-based interventions can be successfully tailored to Tribal communities to reduce the high rates of motor vehicle injury and death; 2) determine the impact of these strategies in reducing motor vehicle injuries, reducing crashes, or increasing occupant restraint use among the four Tribes; 3) determine the core elements, key characteristics, and collaborations needed for these strategies to be successful in Tribal communities; 4) determine the barriers that Tribes face in implementing these programs; and 5) learn how to move these effective strategies into widespread use with other Tribes.

Tribal programs increased driver seat belt use by 38% to 315%, passenger seat belt use by 85% to 220%, child safety seat use by 40%, and decreased motor vehicle crashes by 29%.

Report Title: Program Evaluation, Dissemination, and Translation: CDC-Tribal Motor Vehicle Injury Prevention Programs

Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Anne Dellinger, 770-488-4811
Performer: University of North Carolina, Gillings School of Global Public Health, Department of Health Behavior & Health Education
Record ID: 9405 (Report issued January 1, 2010)

Did Business Improvement Districts In Los Angeles Lead To Decreases In Youth Violence?

Researchers evaluated the impact of Business Improvement Districts (BIDs) in Los Angeles on youth violence. Previous research showed that violence is higher in communities with high rates of family disruption, unemployment, concentrated poverty, and limited access to economic opportunities. Less research has identified specific actionable interventions that can change these community-level factors. BIDs used commercial property assessments to improve services such as sanitation, security, marketing, and planning. These services were expected to increase social cohesion and physical neighborhood improvements, which in turn would reduce rates of youth violence. The study aimed to test whether census tracts with BIDs had lower rates of youth violence than census tracts without BIDs; and whether the relations between BIDs and youth violence were mediated by improvements in social, physical, and/or economic characteristics of the communities.

Implementation of BIDs was associated with significant reductions in violent crimes most frequently committed by youth. There was a 12% drop in robbery (one type of violent crime) and an 8% drop in violent crime overall. Due to limitations in the availability of the data, it was not possible to determine whether the reductions in violent crime were attributable to acts committed by youth. However, the reductions associated with implementation of BIDs were for violent crimes are most likely to be committed by youth offenders. A cost analysis found that investments in BID neighborhoods resulted in cost savings due to reduced violent crime rates, reduced arrests, and lower prosecution-related expenditures. Researchers currently have a paper in press that examines changes in mortality rates between Los Angeles neighborhoods.

Federal Contact: Greta Massetti, 770-488-3943
Performer: RAND
Record ID: 9407 (Report issued August 31, 2010)

Will a Research Roadmap Address The Most Significant Scientific Issues In Asbestos Toxicity, Occupational Exposures, Epidemiology, And Sampling/Analytical Methods?

The National Academies' Institute of Medicine conducted a scientific peer review of the draft National Institute for Occupational Safety and Health (NIOSH) Current Intelligence Bulletin titled “Asbestos Fibers and other Elongated Mineral Particles: State of the Science and Roadmap for Research.” The review assessed the technical quality of the draft document, its scope and the methods used to produce its findings. The National Academies also assessed the likelihood that the framework of research envisioned in the draft document would address the most significant scientific issues in the areas of toxicity, occupational exposures, epidemiology, and sampling/analytical methods.

The National Academies made a total of eight recommendations, a few of which included the need to clarify the vision and rationale for the document, improve the use of terminology, and emphasize the need for interdisciplinary collaboration and integration of research. The Roadmap is being used to guide priority research in this area.

Report Title: Review of the NIOSH Research Roadmap on Asbestos and Other Elongated Mineral Fibers

Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Tammy Sawyer 404-498-2522
Performer: Institute of Medicine of the National Academy of Sciences
Record ID: 9409 (Report issued October 26, 2009)

What Is The Role And Impact Of A Biosurveillance System On The Detection And Response To An Outbreak?

The role and impact of a biosurveillance system on the detection and response to disease outbreaks or adverse weather-related health events were studied by the following activities: 1) case studies of biosurveillance system use in North Carolina and Texas in 2007; 2) a case study of an itch mite outbreak in Illinois in 2007; 3) a case study of the North Carolina Heat Wave of 2007; 4) a review of syndromic surveillance systems in the U.S.; 5) data quality improvement activities for NCDETECT to assess wider applicability to other systems; 6) a paper describing a framework for evaluating the costs of biosurveillance; and 7) discussions with biosurveillance users to identify key needs and requirements for future biosurveillance systems.

Biosurveillance systems had some limited utility in early detection of illness and case detection; but did enhance situational awareness of scale, scope and spread of an outbreak or event, and identification of new risk groups. Systems also improved timeliness of response and accuracy and speed of communication to the public. Syndromic surveillance systems complemented traditional surveillance, facilitated monitoring of non-reportable diseases and reportable diseases not well reported, and enhanced public health credibility and authority by getting information out to public and stakeholders quickly. Use of syndromic surveillance systems in one of the cases led to more detailed information on populations at risk in a heat wave and led the state to revise its recommendations accordingly.

Report Title: BioSense Evaluation: The Role of Biosurveillance During An Outbreak

http://isdsnews.blogspot.com/2010/08/upcoming-book-release-biosurveilla… (book)
Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Taha Kass-Hout, 404-498-2014
Performer: Research Triangle Institute
Record ID: 9410 (Report issued January 1, 2010)

What Is The Impact Of Biosurveillance Systems On Public Health System Preparedness, Detection And Response To Public Health Threats?

Researchers sought to understand the characteristics, scope and geographic dispersion of syndromic surveillance systems.

Researchers identified the following key elements which make biosurveillance useful and acceptable to users: 1) automation (does not rely on someone to report the information); 2) flexibility (can create filters and queries to integrate new information), 3) easy to use (easily integrated into day to day monitoring activities and takes only a few minutes to check).

There is wide variation in syndromic surveillance systems in terms of sources of data, the syndromes tracked, and the completeness of syndromic surveillance coverage. Of the 40 systems identified, emergency department (ED) records were collected by 65% of states, followed by hospital admissions records (43% of states). Systems that used ED records covered an average of 47% of all ED visits in the state, and several states have estimated ED coverage rates above 75%. Respiratory (57%) and gastrointestinal (53%) syndromes were most commonly tracked syndromes. In terms of the cost-benefit of syndromic surveillance, researchers indicated that benefits of early event detection (e.g., anthrax attack) alone are unlikely to outweigh the costs. However, when the benefits of situational awareness are included, the cost-benefit conclusions may change and ultimately depend on the systems potential to avoid premature deaths.

Report Title: BioSense Evaluation: Cross-Case Study

http://www.rti.org/pubs/rr-0013-1011-lesesne.pdf

Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Taha Kass-Hout, 404-498-2014
Performer: Research Triangle Institute
Record ID: 9411 (Report issued January 1, 2010)

What Are The Different Approaches Taken By Different Health Departments For The BioSense Program?

Researchers analyzed approaches taken by different health departments to the BioSense program, and evaluated the structural capability of several state departments of health to respond to a public health incident. In early 2008, the John Hopkins team evaluated the BioSense programs for Indiana, Wisconsin and Minnesota. The teams developed a coordinated continuum of BioSense, including both centralized and decentralized public health, and assessed the continuum's strengths, weaknesses, related costs, barriers and successes. The team interviewed the states' departments of health and, based on the findings, conducted case studies to explore the states' use of BioSense.

Syndromic surveillance data, specifically BioSense, can be used as a proxy for detecting and monitoring outbreak events such as influenza-like illness, H1N1, and chickenpox, as a proxy for smallpox, where rapid testing is not available. Additional work assessed the use of natural language processing (NLP) for parsing clinicians' free text notes in relation to influenza like illness and gastrointestinal illness. Researchers found that natural language processing: 1) Increases statistical performance over detection by ICD9 codes alone; 2) Increases statistical performance when combined with ICD9 codes; 3) Can lower the delay and workload requirements needed to detect an influenza like illness outbreak; 4) Increases the sensitivity of influenza like illness detection when compared to using such information as chief complaint, and emergency department and triage notes; 5) Can be of greater assistance to public health investigations as compared to structured data sources; 6) Can increase sensitivity when combined with analysis using chief complaint, and emergency department and triage notes; 7) Improves detection of patients with febrile illness within a study group; and, 8) Using natural language processing that detects negative statements in these notes lowers false positives. An assessment of BioSense's Cumulative Sum algorithm also showed it to provide a shorter detection delay than SatScan.

Report Title: BioSense Evaluation Cooperative Agreement: The Mayo and Hopkins Evaluations

Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Taha Kass-Hout, 404-498-2014
Performer: Mount Sinai Medical Center
Record ID: 9412 (Report issued January 1, 2010)

How Well Does The BioSense Syndrome Categorization Process Identify Potential Category “A” Agents?

CDC outlined an objective for the BioSense program to develop technologies for automated detection of suspected cases of infection with bioterrorism relevant agents from clinical data. To enhance the value of BioSense, it is important to evaluate each case comprising alerts and to assess whether each patient comprising the alert has a presentation compatible with infection by bioterrorism agents.

Disease diagnosis codes were less likely to predict infection with a potential bioterrorism infectious agent than the patient's chief complaints, largely due to inclusion of incorrect ICD9 codes. (ICD9 codes classify diseases and a wide variety of signs, symptoms, abnormal findings, complaints, social circumstances and external causes of injury or disease.) Removing these codes may permit improved specificity and an increase in the predictability of anomalies, resulting in an increase in the value of BioSense.

Federal Contact: Taha Kass-Hout, 404-498-2014
Performer: Johns Hopkins University
Record ID: 9413 (Report issued January 1, 2010)

How Can We Promote High Quality, Effective Evaluation Practice Among Our Asthma Programs?

The National Asthma Control Program (NACP) developed and disseminated an award winning self-study manual: Learning and Growing through Evaluation: State Asthma Program Evaluation Guide, to assist in evaluation planning and capacity building at the state level. The Evaluation Guide provides information that is valuable in constructing and implementing individual state evaluation plans, as well as case studies for each program component area (i.e., surveillance, partnerships, and interventions). The Evaluation Guide serves as a resource and supports the NACP's extensive evaluation training materials, and one-to-one technical assistance. The Evaluation Guide's ready-to-use tools and manuals for grantees have been estimated to have saved approximately $1.2 million in evaluation development costs for state asthma programs.

Report Title: State Program Evaluation Guide-- Building Evaluation Capacity in State Health Departments

URL: http://www.cdc.gov/asthma/program_eval/guide.htm
Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Sheri Disler, 770-488-3713

Performer: Intramural: CDC/EHHE/APRHB
Record ID: 9414 (Report issued April 1, 2010)

In Children With a High Burden of Asthma, Was the Comprehensive Community-Based Approach Effective In Bringing About Institutional and Policy Change, and Improving Health Outcomes?

The National Asthma Control Program (NACP) funded seven cities through the Controlling Asthma in American Cities Project (CAACP). The project was designed to translate success in the clinical management of asthma to community and households in underserved communities with a high asthma burden.

The Project created seven inner-city community laboratories where evidenced-based or promising asthma interventions could be applied in a culturally appropriate, integrated way to improve the health of children with asthma. The Project's other objectives focused on multiple outcomes at the institutional, community, and policy levels. For example, the Northern Manhattan site demonstrated a dose-response to interventions at the daycare level, showing that interventions delivered to children, parents and the child's healthcare providers were more effective than interventions limited to a single audience. The St Louis site carried out an innovative program to create Asthma Friendly Pharmacies which provided brief, focused educational messages to clients and communicated medication problems or concerns to patients' primary care providers. The Richmond site engaged physicians working with inner-city low SES populations through academic detailing and systems support. This approach has been expanded to other parts of Virginia.

Report Title: Journal Supplement--Controlling Asthma in American Cities

URL: http://www.springerlink.com/content/1099-3460/88/s1

Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Elizabeth Herman, 770-488-3711

Performer: Intramural: CDC/EHHE/APRHB
Record ID: 9415 (Report issued February 1, 2011)

What Variables Contributed To the Sustainability of Management Development Programs in Vietnam, Botswana, and Malawi?

The purpose of this evaluation project funded by the President's Emergency Plan for AIDS Relief (PEPFAR) was to identify key variables associated with program sustainability that could be incorporated into the creation of other PEPFAR initiatives. The Sustainable Management Development Program (SMDP) selected three countries of interest (Botswana, Malawi, and Vietnam) that demonstrated success in the development and implementation of public health management capacity development programs. Information for this case-study analysis was collected by conducting in-person interviews with key informants, including public health management capacity development program leaders, ministry of health officials, and CDC in-country staff. The case studies were augmented by data from documents such as trip reports, meeting reports, evaluation reports, and publications related to the subject matter.

Knowledge distilled from the evaluation was incorporated into a technical document from which publications, newsletters, and webinars will be generated that will assist other countries in designing and implementing public health management capacity development programs .

Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Elizabeth Howze, 404-639-7422
Performer: CGH/DPHSWD/Sustainable Management Development Program
Record ID: 9546 (Report issued December 31, 2010)

What Evidence Supports The Contributions Of Management and Leadership To Improved Outcomes In Global Health?

There is growing awareness that well-led and -managed global health solutions are required to achieve effective, efficient, and sustainable health programs, especially at the scale needed to attain Millennium Development Goals and other global targets. Researchers at George Washington University examined more than 300 documents to assess the evidence available about the contribution of good leadership and management to improved global health outcomes. Content analysis was used to determine retrospectively the reported presence in each program of 81 leadership and management characteristics. Program achievements were classified into one of three categories: health effects, increased service outputs, or improved processes. Improvements for processes ranged from 46-483%; for service delivery outputs from 125-571%, and for health effects from 61-175%.

The background review and case studies were assembled into a searchable compendium entitled the “Presence of Leadership and Management in Global Health Programs: A Compendium of Case Studies.” The Compendium, which is being widely disseminated, suggests the potentially substantial magnitude of health returns on investments that can be achieved by improving management and leadership. It also identifies steps that should be taken to strengthen the evidence base for the impact of leadership and management on global health outcomes.

Report Title: Presence of Leadership and Management in Global Health Programs: A Compendium of Case Studies. URL:

URL: http://www.cdc.gov/globalhealth/SMDP/

Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Elizabeth Howze, 404-639-7422
Performer: George Washington University, Washington, D.C.
Record ID: 9547 (Report issued January 31, 2011)

How Reliable And Valid are Key Data Terms Collected By National Program of Cancer Registries and How Complete Is Case Ascertainment By the Registries?

Population-based cancer registries are an essential part of the cancer surveillance system for planning, operating, funding, and evaluating cancer control programs. Complete and accurate data are necessary to estimate variations in population subgroups and changes among population subgroups over time. The National Program of Cancer Registries was audited and assessed for completeness and quality of data. The audits emphasized the existence of appropriate policies and procedures for data ascertainment, data collection, and data quality assessment.

The study showed that registries capture high quality complete data.

Report Title: Data Completeness and Quality Audits

Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Thomas Chapel, 404-639-2116
Performer: ICF Macro
Record ID: 9558 (Report issued September 30, 2011)

Are There Changes in Client Self-Reported HIV Risk Behaviors After Participating in the CDC funded “Many Men, Many Voices” Intervention by Community Based Organizations?

Many Men, Many Voices (3MV) is a 7-session group-level intervention program to prevent HIV and sexually transmitted diseases among black men who have sex with men (MSM) who may or may not identify themselves as gay.

The intervention addresses factors that influence the behavior of black MSM: cultural, social, and religious norms; interactions between HIV and other sexually transmitted diseases; sexual relationship dynamics; and the social influences that racism and homophobia have on HIV risk behaviors. 3MV sessions aim to foster positive self-identity, educate clients about their risk for HIV and sexually transmitted diseases, and teach assertiveness skills. Clients who participated in the evaluation of 3MV were from three CDC-funded community-based organizations.

Report Title: CBO Behavioral Outcomes: Many Men, Many Voices (CBOP-3MV) (PN1212) URL: Data analysis underway. Final report expected in Fall 2012

Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Renee Stein, 404-639-3517 or Gary Uhl 404-639-0950

Performer: Program Evaluation Branch
Record ID: 9581 (Report issued June 30, 2011)

Can County Jail Facility Successfully Implement a Comprehensive Program of Routine, Opt-out HIV Rapid Testing, Sti Screening, and Hepatitis B Vaccination?

The development and programmatic evaluation of a comprehensive, routine, opt-out HIV rapid testing, sexually transmitted infection (STI) screening, and Hepatitis B vaccination initiative within a large county jail facility was studied. Key outcomes included: 1) number of persons screened for HIV with a rapid HIV screening test, STIs (e.g., Syphilis, Gonorrhea, Chlamydia), Hepatitis B and Hepatitis C; and, 2) number and percent of persons diagnosed with HIV, STI, Hepatitis B, or Hepatitis C-positive.

Integrating rapid HIV screening into the medical intake process in jails may be an effective strategy to reach persons with HIV who may not seek HIV testing services in the community. The project received a 1-year no-cost extension and program data are currently being analyzed.

Report Title: Jail-based Integration of HIV/STI/Hepatitis screening, Hepatitis B vaccination, and Linkage to Care and Treatment (PN1468)

Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Robin MacGowan, 404-639-1920
Performer: Prevention Research Branch
Record ID: 9582 (Report issued August 31, 2011)

Does North Carolina's Smoke Alarm Installation And Fire Safety Education Program Have An Impact On Resident's Fire Safety Knowledge And Practices?

The study described North Carolina's CDC-funded Smoke Alarm Installation and Fire Safety Education (SAIFE) program and determined its impact on resident's fire safety knowledge and practices. The North Carolina SAIFE program, “Get Alarmed” was evaluated because it was an established program with the appropriate infrastructure in place to conduct a coordinated evaluation. North Carolina is also located in what has been called the burn belt, an area of the southeast in which people are at higher risk of fire death than in many other areas of the United States.

The program was largely successful in reaching members of its designated target groups. A high number of smoke alarms remained functional after six months, particularly among those that were visually inspected. Changes in terms of fire safety knowledge and behaviors as a result of household participation in the program were mixed and mostly inconclusive.

Report Title: Evaluation of the Effectiveness of the SAIFE Program

Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Robin Lee, PhD, 770-488-3811

Performer: RTI International
Record ID: 9651 (Report issued December 31, 2010)

What Were The Effects Of Imputation On The Current Population Survey Income And Poverty Series?

The Current Population Survey has been the source of the official U.S. poverty estimates since their inception. Since then, many changes have occurred in society and in the willingness of survey respondents to report the cash income used to construct these poverty figures. Improvements have also occurred in the data collection instruments. Poverty rates were examined by type of imputation from 1981 until 2007 focusing particularly on how poverty series for reporters and for those with item and whole imputes have trended over this period. Differences for blacks and whites were determined.

Poverty rates for persons with positive income trended downward. Over most of the period, poverty rates for persons reporting all of their income and for those with at least some of their income imputed paralleled each other, however, poverty rates for imputers were 2.5% to 3% lower than for reporters from year to year. Persons with no imputes have the highest poverty rates and those with item imputes the lowest poverty rates. Blacks have significantly higher poverty rates than whites for all types of imputation.

Report Title: Effects of Imputation on CPS Income and Poverty Series: 1981-2007
Agency Sponsor: OASPE-OSDP, Office of Science and Data Policy
Federal Contact: Joan Turek, 202-690-5965
Performer: Joan Turek, ASPE/OSDP
Record ID: 9245 (Report issued November 30, 2009)

What Analyses Can Be Done with the Homelessness Data that States Collect?

The Homelessness Data in Health and Human Services Mainstream Programs study investigates which states currently collect housing status data from applicants for Medicaid and/or Temporary Assistance for Needy Families (TANF), the two largest HHS mainstream programs that may serve individuals or families experiencing homelessness. Information about data collected at application was collected in a survey of the 50 states and the District of Columbia.

Thirty states currently collect information on homelessness or risk factors for homelessness from applicants for TANF or Medicaid. This report provides ideas for potential uses of this information for policymakers (especially at the state level) and researchers. The report summarizes ideas for data analysis that would help address three research areas that should be of interest to policymakers and researchers. Extent to which people who are homeless participate in TANF and Medicaid, whether or not TANF and Medicaid are protecting people from becoming homeless, and whether or not people experiencing homelessness are using TANF and Medicaid in ways that are different from other program participants.

Report Title: Potential Analyses with Homelessness Data: Ideas for Policymakers and Researchers http://aspe.hhs.gov/hsp/09/HomelessnessDataHHS/PotentialAnalyses/index.shtml
Agency Sponsor: OASPE, Office of the Assistant Secretary for Planning & Evaluation
Federal Contact: Linda Mellgren, 202-690-6806
Performer: National Opinion Research Center (NORC)
Record ID: 9327.2 (Report issued December 1, 2009)

Goal 3: Advance the Health, Safety, and Well-Being of the AmericanPeople

Objective: Promote the safety, well-being, resilience, and healthy development of children and youth

How Does the Health and Well-Being of Adopted Children Compare to the General Population of Children and are There Differences by Adoption Type?

Adoption USA presents findings from the first nationally representative survey of adoptive parents in the United States. About two percent of U.S. children joined their families through adoption. The Chartbook describes the characteristics, adoption experiences, and well-being of these children and their families, making comparisons between adopted children and the general population of children in the U.S. and among children adopted through different adoption types.

Of the 1.8 million adopted children in the U.S., 37 percent were adopted from foster care, 38 percent joined their families through private domestic adoptions, and 25 percent were adopted internationally. Nearly one-quarter of adopted children were adopted by relatives. Just over one-third of children in non-relative adoptions had post-adoption contact with their birth families. Most adopted children (85 percent) are in excellent or very good health, however, they are twice as likely as children in the general population to have special health care needs (39 percent compared with 19 percent). The majority of adopted children also fare well according to measures of social and emotional well-being. However, 26 percent of adopted children have been diagnosed with Attention Deficit Disorder or Attention Deficit with Hyperactivity Disorder (ADD/ADHD) at some point during childhood, and 15 percent have been diagnosed with behavior or conduct problems. Children adopted from foster care were most likely to have these conditions. Parents of 86 percent of adopted children report that their relationship with their child met or exceeded their expectations. However, parents of 11 percent of adopted children report parental aggravation, compared with 6 percent in the general population.

Report Title: Adoption USA: A Chartbook Based on the 2007 National Survey of Adoptive Parents http://aspe.hhs.gov/hsp/09/NSAP/chartbook/index.cfm
Agency Sponsor: OASPE-OHSP, Office of Human Services Policy
Federal Contact: Laura Radel, 202-690-5938
Performer: Child Trends, Inc.
Record ID: 9199 (Report issued November 20, 2009)

How do Community Level Efforts Develop Changes That Result in Beneficial Outcomes for Children, Families, and Schools?

The goals of the Safe Schools/Healthy Students Interdepartmental Initiative were to help students develop the skills and emotional resilience necessary to promote positive mental health, engage in pro-social behavior, and prevent violent behavior and drug use; ensure that all students learn in a safe, disciplined, and drug-free environment; and develop an infrastructure that will institutionalize and sustain integrated services. This evaluation of the initiative documented how coalitions and collaborations at the community level develop systemic change that results in beneficial outcomes for children, families, and schools.

Some degree of improvement was reported for all key outcomes in elementary, middle and high school. Change was more favorable than unfavorable. Elementary school teachers reported improved student behavior and school climate. Over time middle school students witnessed less bullying and felt safer. High school students reported decreased use of tobacco and alcohol. There was a 15 percent decrease in the number of students involved in violent incidents during the grant period. There was a 12 percent decrease in the number of students reporting that they had experienced or witnessed violence from year one of the grant period to year three. Most staff at grantee schools reported that the Initiative had made their schools safer. By year three of the grant, 84 percent said the Initiative had improved school safety. Additionally, 77 percent of respondents said the program had reduced violence on campus, and 75 percent said it had reduced violence in the community.

Report Title: To assess whether SS//HS grants develop systemic change that results in beneficial outcomes for children, families, and schools.
Agency Sponsor: SAMHSA, Substance Abuse and Mental Health Services Administration
Federal Contact: Danyelle Mannix, 240-276-1879
Performer: Research Triangle Institute (RTI)
Record ID: 8723 (Report issued March 31, 2010)

Did Early Head Start Programs Address the Employment Needs Of Low-Income Parents And the Developmental Needs Of Their Children?

The study tested a program model that addressed both the employment and educational needs of parents who are at risk of unemployment and who are dealing with the developmental needs of their young children.

The model increased the program's focus on parental employment and educational needs, but the implementation of the enhancements was weak. The program increased families' receipt of child-focused developmental services, but the control group also reported receiving high levels of such assistance. The short-term impacts of the program 18 months after families entered the study were mixed.

Report Title: A Two-Generational Child-Focused Program Enhanced With Employment Services

http://www.acf.hhs.gov/programs/opre/welfare_employ/enhanced_hardto/reports/two_generational.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Girley Wright, 202-401-5070
Performer: MDRC
Record ID: 9669 (Report completed April 15, 2011)

How Does the Quality of Family Relationships and Performance in School of Adopted Children Compare to That of Other Children?

An analysis was conducted, using data from the National Survey of Adoptive Parents and a comparison group drawn from the National Survey of Children's Health. The study examined data on adopted children and all U.S. children on selected indicators of school performance and family relationship quality. Because the population of adopted children is not homogenous and varies considerably on many dimensions by adoption type, estimates of school performance and relationship quality for adopted children by adoption type were also prepared. Adjusted estimates that controlled for demographic and socioeconomic differences among adoption types were examined to evaluate whether differences in school performance and relationship quality by adoption type were explained by demographic and socioeconomic differences between adoption types.

Adopted children were less likely than children in the general population to excel in reading or math, but the quality of their family relationships was similar. However, children adopted from foster care did show lower relationship quality than other adopted children. This lower quality of family relationships and also seemed to account for much of the difference in school performance. Researchers concluded that differences in school performance and relationship quality among groups from different types of adoption were largely explained by demographic and socioeconomic differences among the groups.

Report Title: National Survey of Adoptive Parents: Benchmark Estimates of School Performance and Family Relationship Quality for Adopted Children (Research Brief) http://aspe.hhs.gov/hsp/09/NSAP/Brief3/rb.shtml
Agency Sponsor: OASPE-OHSP, Office of Human Services Policy
Federal Contact: Laura Radel, 202-690-5938
Performer: National Center for Health Statistics, (NCHS), CDC
Record ID: 9199.1 (Report issued July 15, 2011)

What Are the Characteristics of Children Adopted from Foster Care; What Motivates Their Families to Adopt Them; and How Do They Fare?

Researchers prepared the first of a two-part analysis presenting information on children adopted from foster care in the United States and their families. The analysis used data from the 2007 National Survey of Adoptive Parents and the 2007 National Survey of Children's Health.

Demographic and other characteristics of children adopted from foster care differ from those of children in the general U.S. population. Parents who adopt children from foster care are diverse. Most children adopted from foster care were doing well and their parents were satisfied with the adoptions. However, nearly half lived in households with low incomes, and about one out of ten have parents who were more than 50 years older than the child. While most had parents who were satisfied with the adoption, one quarter of children adopted from foster care had parents who did not describe their relationship with the child as very warm and close, and a small percentage (6 percent) had parents who would definitely or probably not make the same decision to adopt the child, knowing everything they knew after the adoption.

Report Title: Children Adopted from Foster Care: Child and Family Characteristics, Adoption Motivation, and Well-Being http://aspe.hhs.gov/hsp/09/NSAP/Brief1/rb.shtml
Agency Sponsor: OASPE-OHSP, Office of Human Services Policy
Federal Contact: Laura Radel, 202-690-5938
Performer: Child Trends
Record ID: 9199.2 (Report issued May 15, 2011)

What Post-Adoption Supports Are Provided to Children Adopted From Foster Care?

Using the same data sources as in the previous entry, researchers prepared a second analysis presenting information on post-adoption supports and services including adoption subsidy payments, Medicaid coverage, and other services such as support groups for parents and children.

Children adopted from foster care may receive a variety of post-adoption supports. Ninety-two percent of children had adoption agreements with the public child welfare agency, and over three-quarters (78 percent) of children adopted from foster care received a monthly subsidy payment. Adoption subsidies and Medicaid were by far the most common supports received by children adopted from foster care. But a wide variety of both adoption-specific and more general services were utilized by substantial percentages of children. In the case of seven services specifically asked about in the survey, between 10 and 20 percent of children's parents indicated that they desired but did not receive the services. Overall, 35 percent of children adopted from foster care did not receive at least one adoption-specific support their adoptive parents thought would have been helpful. These included tutoring, mentoring, child-care assistance, parenting training, parent support groups, child support groups and respite care. Making such services accessible to more adoptive families could address the stress and isolation often experienced by children adopted from foster care and their adoptive parents.

Report Title: Children Adopted from Foster Care: Adoption Agreements, Adoption Subsidies, and Other Post-Adoption Supports http://aspe.hhs.gov/hsp/09/NSAP/Brief2/rb.shtml
Agency Sponsor: OASPE-OHSP, Office of Human Services Policy
Federal Contact: Laura Radel, 202-690-5938
Performer: Child Trends
Record ID: 9199.3 (Report issued May 15, 2011)

What is the Influence of Media on Adolescent Sexual Activity?

Researchers synthesized the existing information about new media use by adolescents, with an emphasis on identifying measures of new media exposure and their association with sexual behavior. The project has examined the empirical evidence regarding both new and traditional media use and its links to sexual activity, theoretical perspectives and measurement issues regarding the influence of media on sexual activity, and emerging findings on new media's positive and negative impact on adolescent sexual activity.

There is a pressing need for rigorous, systematic content analyses of the sexual content in new media. Theory development is needed because existing theories do not easily account for issues such as the likely effects of users' new roles in creating and distributing content, as well as consuming it. Sexual health interventions based in new media or incorporating new media components should be developed and tested. Experimental and longitudinal survey studies of the effects of exposure to new media content on adolescents are needed to determine whether new media pose risks or confer benefits in the area of sexual health. New measures will be needed that allow researchers to accurately assess the amount of sexual content that youth are exposed to (or create) as a consequence of using new media.

Report Title: Influence of New Media on Sexual Health: Evidence and Opportunities http://aspe.hhs.gov/hsp/11/AdolescentSexualActivity/NewMediaLitRev/index.shtml
Agency Sponsor: OASPE-OHSP, Office of Human Services Policy
Federal Contact: Diana Tyson, 202-401-6670
Performer: Rand Corporation
Record ID: 9527 (Report issued April 11, 2011)

What Are The Impacts Of The Employment Services Program Offered To Youth In Kern County, California?

This study was part of a larger multi-site evaluation seeking to determine the effects of programs funded under the John Chafee Foster Care Independence Program.

The study explored the impacts of the employment services offered to youths in Kern County, California, aged 16 and older in the child welfare system. This study found no significant differences in employment or other key outcomes between the program and control group.

Report Title: Evaluation of the Independent Living - Employment Services Program, Kern County, California: Final Report.

http://www.acf.hhs.gov/programs/opre/abuse_neglect/chafee/reports/eval_kern/eval_kern.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Maria Woolverton, 202-205-4039
Performer: The Urban Institute
Record ID: 9700 (Report completed July 15, 2011)

What Are The Impacts Of The Massachusetts Adolescent Outreach Program for Youths in Intensive Foster Care?

This study was also part of the larger multi-site evaluation seeking to determine the effects of programs funded under the John Chafee Foster Care Independence Program in achieving key outcomes for youth. The study examined concrete measures (education, employment, economic well-being) of the transition to adulthood.

Program participants were more likely than control group members to report having ever enrolled in college, persist in college across more than one academic year, and remain in foster care. The study also found that program participants reported receiving more help than control group members in some areas of educational assistance, employment assistance, money management, and financial support in obtaining housing but program participants did not report better outcomes in employment, economic well-being, housing, delinquency, pregnancy, or self-reported preparedness for independence.

Report Title: Evaluation of the Massachusetts Adolescent Outreach Program for Youths in Intensive Foster Care: Final Report http://www.acf.hhs.gov/programs/opre/abuse_neglect/chafee/reports/eval_mass/eval_mass.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Maria Woolverton, 202-205-4039
Performer: The Urban Institute
Record ID: 9701 (Report completed July 15, 2011)

How Does Head Start Affect The School Readiness Of Children Participating In The Program As Compared To Children Not Enrolled In Head Start?

The National Head Start Impact Study was a longitudinal study involving approximately 5,000 three and four year old preschool children across 84 nationally representative grantee/delegate agencies in communities where there were more eligible children and families than can be served by the program. The children participating were randomly assigned to either a treatment group (which had access to Head Start services) or a comparison group (which did not have access to Head Start services, but could receive other community resources).

Providing access to Head Start led to improvements in the quality of the early childhood settings and programs children experienced. However, while children given access to Head Start showed benefits after a year in Head Start compared to children in the control group, there were only a few statistically significant differences in outcomes between these two groups at the end of grade one for the sample as a whole.

Report Title: Head Start Impact Study: Final Report http://www.acf.hhs.gov/programs/opre/hs/impact_study/reports/impact_study/hs_impact_study_final.pdf
Report Title: Head Start Impact Study: Executive Summary http://www.acf.hhs.gov/programs/opre/hs/impact_study/reports/impact_study/executive_summary_final.pdf

Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Jennifer Brooks, 202-205-8212
Performer: Westat, Inc.
Record ID: 9705 and 9706 (Report completed January 15, 2010)

How do The Characteristics of Families Headed by Kinship Caregivers Compare with Families Headed by Non-Kin Foster Caregivers?

This study examined the parenting provided by kinship caregivers to children 10 years old or younger who have been involved in investigations of child maltreatment.

Kinship caregivers, compared with other foster caregivers, were older and had fewer years of education. Kinship caregivers were less likely to be currently married and more likely to be living below the federal poverty level.

Report Title: National Survey on Child and Adolescent Well-being (NSCAW) Research Brief 15: Kinship Caregivers in the Child Welfare System http://www.acf.hhs.gov/programs/opre/abuse_neglect/nscaw/reports/kinship_caregivers/rb_15_2col.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Mary Webb, 202-205-8628
Performer: Research Triangle Institute
Record ID: 9715 (Report completed July 15, 2010)

What Are The Experiences Of Children And Families Who Come Into Contact With The Child Welfare System?

This study summarized the main findings of the National Survey on Child and Adolescent Well-being.

Most children reported for maltreatment remain living at home with their biological caregivers. Many children have some history of chronic health conditions or special needs that typically require extensive health and related social services. High levels of children's mental health needs go unmet.

Report Title: National Survey on Child and Adolescent Well-being (NSCAW) Research Brief 16: Summary of Main Findings http://www.acf.hhs.gov/programs/opre/abuse_neglect/nscaw/reports/summary_nscaw/nscaw_research_brief_main_findings.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Mary Webb, 202-205-8628
Performer: Research Triangle Institute
Record ID: 9716 (Report completed July 15, 2010)

What Are The Experiences of Children Involved With The Child Welfare System As They Transition to Adolescence?

This study was the third in a series presenting findings from the National Survey on Child and Adolescent Wellbeing Wave 5 follow-up. It provided information about 1,484 adolescents who were reported for maltreatment to the child welfare system when they were between 3 and 11 years old.

Some children's cases were closed after investigation while others had a case opened to child welfare services. Although the majority of the children remained at home after investigation, a small proportion was removed from their homes. Six to seven years after the child protective services investigation, these adolescents were 11 to 17 years old.

Report Title: Children Involved with Child Welfare: A Transition to Adolescence (Wave 5 NSCAW Follow-up) http://www.acf.hhs.gov/programs/opre/abuse_neglect/nscaw/reports/transition_to_adolescence/wave5_adolescence_report_final.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Mary Webb, 202-205-8628
Performer: Research Triangle Institute
Record ID: 9717 (Report completed April 15, 2010)

How Did Grantees Participating In Training-Of-Trainers Event Implement Learning Enhancements?

Investigators examined how grantees that participated in the spring 2006 regional Training-of-Trainers event implemented I Am Moving, I Am Learning enhancements. The evaluation was not designed to assess impact on children's health outcomes.

Programs that participated in the training implemented a wide range of activities for children, staff members, and parents. Music and movement activities were integrated into daily activities in Head Start classrooms and as a part of home visits. Some programs reached out to staff members and parents, with the focus on increasing their movement activities and improving their eating habits to help them serve as better role models for the children.

Report Title: Final Report on I am Moving I am Learning Evaluation http://www.acf.hhs.gov/programs/opre/hs/eval_move_learn/reports/implement_moving_learning/implement_moving_learning.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Laura Hoard, 202-401-4561
Performer: Mathematica Policy Research, Inc
Record ID: 9719 (Report completed February 15, 2010)

What Do Children In the Family And Child Experiences Survey Look Like After A Year In Head Start?

Researchers profiled the 3- and 4-year-old Head Start children and families who were newly enrolled in the program in fall 2006 and were still attending in spring 2007. The study obtained information on the children's characteristics, family demographics, and home life, including language background, educational environment of the home, family routines, and socioeconomic risk status. It included information on parent involvement in Head Start and level of satisfaction with their own and their children's Head Start experiences. It also chronicled children's developmental progress over the Head Start year, considering whether these outcomes vary by age, gender, race/ethnicity, or risk status.

Changes in children's skills and development during the program year reflect a range of influences, including maturation, program and family influences, and other influences in children's lives. Researchers found that both 3- and 4-year old children demonstrated more social skills, cooperative classroom behavior, and positive approaches to learning on average by the end of the program year. They also found that the majority of children were rated as having excellent or very good health.

Report Title: A Year in Head Start: Children, Families and Program

http://www.acf.hhs.gov/programs/opre/hs/faces/reports/year_final/year_final.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Maria Woolverton, 202-205-4039
Performer: Mathematica Policy Research, Inc
Record ID: 9720 (Report completed October 15, 2010)

What Do Children in the Family and Child Experiences Survey Look Like After Their Second Year In Head Start?

This study profiled the second year in the program for 5-year old children and their families who were newly enrolled in Head Start as 3-year old children in fall 2006 and were still attending in spring 2008.

Families of children who entered Head Start as 3-year olds faced many of the same challenges as the children who completed their second year in the program. Families of children entering as 3-year olds actively engaged in activities that supported their children's learning and reported positive experiences and engagement in Head Start. For example, despite the prevalence of household-level poverty, single parenthood, low education, and primarily speaking a language other than English at home, these families generally ensure that important resources and supports for children are in place. The majority of children have health insurance and their parents report active involvement in their learning through home- or community-based activities.

Report Title: A Second Year in Head Start: Characteristics and Outcomes of Children Who Entered the Program at Age Three http://www.acf.hhs.gov/programs/opre/hs/faces/reports/program_agethree/program_agethree.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Maria Woolverton, 202-205-4039
Performer: Mathematica Policy Research
Record ID: 9760 (Report completed February 15, 2011)

What Do Children in the Family And Child Experiences Survey Look Like After Graduating From Head Start And Entering Kindergarten?

This study examined the population of 3- and 4-year-olds who entered head Start for the first time in fall 2006, their families, and their classrooms.

Children made progress in a range of cognitive and non-cognitive skills during their first year in Head Start, with children making the greatest gains relative to their same-age peers in letter-word knowledge. Children were enrolled in Head Start classrooms that, on average, met professional standards for group size and child-to-staff ratio and were in the adequate range on standard measure of classroom quality.

Report Title: Head Start Children go to Kindergarten Report http://www.acf.hhs.gov/programs/opre/hs/faces/reports/hs_kindergarten/hs_kindergarten.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Maria Woolverton, 202-205-4039
Performer: Mathematica Policy Research
Record ID: 9761 (Report completed February 15, 2011)

What Is the Design and Implementation Plan for the Three Interventions Tested In Project Upgrade?

Researchers developed a design and implementation of three interventions tested in Project Upgrade, an experiment conducted as part of the Evaluation of Child Care Subsidy Strategies.

Within six months of training, all three language/literacy interventions resulted in significant changes in teacher behaviors that supported their student's language and literacy development. The number of classroom activities that involved literacy, increased significantly and more literacy resources were made available in the classroom.

Report Title: Evaluation of Child Care Subsidy Strategies: Implementation of Three Language and Literacy Interventions in Project Upgrade http://www.acf.hhs.gov/programs/opre/cc/ccs_strategies/reports/three_language/pu_intervention.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Ivelisse Martinez-Beck, 202-690-7885
Performer: Abt Associates Inc.
Record ID: 9765 (Report completed June 15, 2010)

What Was the Effect Of Reducing Child Care Subsidy Copayments In Washington State?

Evaluators explored the results of the Washington State policy decision to reduce the amount families had to contribute when they received subsidies to help pay for child care, or copayments.

A reduced copayment schedule led families to receive subsidies for a longer period of time. This was especially true for higher-income subsidy recipients, for whom the alternative copayment represented the largest reduction in their copayment amounts. However, reducing copayments did not appear to affect employment.

Report Title: Effects of Reducing Child Care Subsidy Copayments in Washington State http://www.acf.hhs.gov/programs/opre/cc/ccs_strategies/reports/washington/ccse_washington.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Ivelisse Martinez-Beck, 202-690-7885
Performer: MDRC
Record ID: 9766 (Report completed June 15, 2010)

What Impact Has “LearningGames” Had On Providers and Children In Massachusetts?

The Massachusetts Family Child Care study examined the impact on providers and children of an early childhood education program aimed at improving children's development and learning opportunities in the care settings. The early childhood education program,LearningGames, focuses on training caregivers to stimulate children's cognitive, language, and social-emotional development through game-like interactions with individual children across the day.

LearningGames had significant positive impacts on the behavior of the family child care providers the behavior of the family child care providers who received up to two years of support in implementing the program. LearningGames was effective at promoting high-quality, individualized and small group interactions between providers and children, which have been shown in previous research to be associated with children's cognitive and language development.

Report Title: Evaluation of Child Care Subsidy: Massachusetts Family Child Care Study http://www.acf.hhs.gov/programs/opre/cc/ccs_strategies/reports/ma_family_ccs/css_ma_final.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Ivelisse Martinez-Beck, 202-690-7885
Performer: Abt Associates
Record ID: 9767 (Report completed June 15, 2010)

What Does Previous Research Say About Quality, Dosage, and Thresholds of Child Care and Early Childhood Settings?

Researchers examined findings from previous studies regarding the quality, dosage, and threshold of child care and early childhood settings to identify the most promising areas to pursue further.

Low-income children in high quality child care and early childhood settings, had reductions in behavior problems proportionate to increases in the number of hours in the particular setting. An increase in the frequency that young children spent in high quality care settings correlated with a decrease in the achievement gap associated with the income-to-needs ratio of the children's family.

Report Title: Quality, Dosage, Thresholds and Features in Early Childhood Settings: A Review Of The Literature http://www.acf.hhs.gov/programs/opre/cc/q_dot/quality_review.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Ivelisse Martinez-Beck, 202-690-7885
Performer: Mathematica Policy Research
Record ID: 9768 (Report completed August 15, 2010)

What Lessons Are To Be Learned From the Complexity Of Child Care Decision-Making?

Researchers examined the forces that shape parents' child care and early education decisions. The complex decision-making process is shaped by the characteristics of the family and the community in which they reside.

Multiple programs and policies will be needed to reduce the barriers and constraints low-income parents face as they make child care decisions. Child care subsidies cannot provide access if options do not exist in the parent's community, nor will increased information help if the available options do not provide high-quality education and care.

Report Title: Understanding Parents Child Care Decision Making: A Foundation for Child Care Policy Making http://www.acf.hhs.gov/programs/opre/cc/childcare_technical/reports/parents_childcare.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Susan Jekielek, 202-401-7260
Performer: Child Trends
Record ID: 9769 (Report completed February 15, 2011)

Did Effects from Early Head Start Continue for Any Specific Child and Family Outcomes When Children Were in Fifth Grade?

Early Head Start provides educational and developmental activities for children from ages three to five. This study sought to determine whether these early experiences provided long term learning results.

Early Head Start results, when measured at 5th grade, did not continue to have the broad pattern of impacts for child and family outcomes seen at younger ages. However, there were interesting patterns of impacts for some subgroups. There was one significant impact at the trend level for the overall sample on a social-emotional success index that did not reach significance for individuals.

Report Title: Early Head Start Children in Grade 5: Long-Term Follow-up of the Early Head Start Research and Evaluation Project Study Sample, Final Report

http://www.acf.hhs.gov/programs/opre/ehs/ehs_resrch/reports/grade5/grade5.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Amy Madigan, 202-401-5143
Performer: Mathematica Policy Research
Record ID: 9770 (Report completed December 15, 2010)

How Do We Measure the Quality of Early Care And Education Programs; What Challenges Exist?

Researchers explored the changing context of quality measurement and delineated a set of challenges that must be addressed as measures were used for purposes such as making ratings of quality levels widely available, and using quality ratings to make funding and programmatic decisions.

As the measurement of quality goes to scale in quality initiatives, there is a need to reexamine quality measures to ask how they were functioning in a policy context. Information is needed on what aspects of quality existing measures cover well and what they do not cover well, the reliability of data collection at this scale, and the extent to which measures predict to the child outcomes that early childhood policies are focusing on.

Report Title: Measuring the Quality of Early Care and Education Programs at the Intersection of Research, Policy, and Practice http://www.acf.hhs.gov/programs/opre/cc/childcare_technical/reports/quality_measures.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Ivelisse Martinez-Beck, 202-690-7885
Performer: Child Trends
Record ID: 9771 (Report completed December 15, 2010)

How Do We Define and Measure Quality in Home-Based Care Settings?

Researchers explored the importance of designing and evaluating quality measures for home-based settings, and the unique characteristics and variability of home-based care including mixed age groups, supports for parents, and different provider characteristics.

Fifty-seven percent of children under age six who were in non-parental care were cared for in a home, whether that of a relative or a non-relative provider. The continuing expansion in home-based care quality measures must keep up with the rapidly-growing research base on aspects of early development that appear to be linked to children's long-term social and academic success.

Report Title: Defining and Measuring Quality in Home-Based Care Settings http://www.acf.hhs.gov/programs/opre/cc/childcare_technical/reports/define_measures.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Ivelisse Martinez-Beck, 202-690-7885
Performer: Child Trends
Record ID: 9772 (Report completed December 15, 2010)

What Observational Measures Can Be Used to Measure Quality In Center-Based Early Care And Education Programs?

Researchers reviewed several widely used assessments and their relation to each other and to child outcomes. Particular attention was given to purposes for assessment, psychometric properties, inter-rater reliability, applicability of measures across ages, and content and cross-cultural validity.

While several classroom observation methods have been shown to predict later child outcomes, classroom features and experiences still accounted for far less of child variability than family characteristics did. Despite the modest sizes of the associations between child care quality and child outcomes, quality measures consistently and significantly confirmed these links. Further development of quality measurement tools is warranted.

Report Title: Observational Measures of Quality in Center-Based Early Care and Education Programs http://www.acf.hhs.gov/programs/opre/cc/childcare_technical/reports/observe_measures.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Ivelisse Martinez-Beck, 202-690-7885
Performer: Child Trends
Record ID: 9773 (Report completed December 15, 2010)

What Are the Design Options for the Migrant and Seasonal Head Start Survey?

This Design Report for a Migrant and Seasonal Head Start Survey laid the groundwork for a study of Head Start's Migrant and Seasonal Branch by offering custom-designed methodological suggestions regarding sampling, site outreach, instrumentation, data collection, and data analyses.

Researchers provide survey background and a comprehensive literature review. The study identified components that could be included in a final survey design.

Report Title: Design Report for Migrant and Seasonal Head Start Survey http://www.acf.hhs.gov/programs/opre/hs/migrant_mshs/reports/migrant_design.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Wendy DeCourcey, 202-260-2039
Performer: The CMD Group
Record ID: 9774 (Report completed May 13, 2011)

What Evidence Exists For the Effectiveness Of Early Childhood Home Visiting?

Researchers identified key lessons learned from the first year of the home visiting evidence of effectiveness review.

Most prioritized models had at least one randomized controlled trial that met the project's standards and studies used a wide range of outcome measures of quality, making it difficult to compare findings across studies.

Report Title: Lessons Learned from the Home Visiting Evidence of Effectiveness Review
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Lauren Supplee, 202-401-5434
Performer: Mathematica Policy Research
Record ID: 9775 (Report completed January 31, 2011)

What Information is Needed to Determine the Appropriate Design, Timing, and Resources for Evaluating Quality Rating and Improvement Systems for Early Care and Education and School-Age Care?

Researchers sought to identify how quality and improvement systems in early child care and education could best be evaluated.

Researchers identified basic evaluation concepts, useful tools for determining the appropriate design and timing of an evaluation, and evaluation references and resources for those who in positions to authorize, finance, design, and refine quality rating and improvement systems and other quality improvement efforts. Researchers found that the development and testing of a logical model is an essential starting point.

Report Title: Effective Evaluation of Quality Rating and Improvement Systems for Early Care and Education and School-Age Care http://www.acf.hhs.gov/programs/opre/cc/childcare_technical/reports/quality_rating.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Ivelisse Martinez-Beck, 202-690-7885
Performer: Rand Corporation
Record ID: 9776 (Report completed June 15, 2011)

What Are The Basic Features of the FACES 2009 Study Design?

This study highlighted basic features of the FACES 2009 study design for those interested in learning more about the study or who have an interest in using the data for future analyses. It also offered an overview of the dissemination plans for the FACES data and study findings.

The basics features are the sample and sampling design, the composition of the child assessment; classroom observation; and the parent, teacher, and Head Start staff interviews. Descriptive information is provided on each component.

Report Title: FACES 2009 Study Design http://www.acf.hhs.gov/programs/opre/hs/faces/reports/study_2009/study_2009.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Maria Woolverton, 202-205-4039
Performer: Mathematica Policy Research
Record ID: 9779 (Report completed June 15, 2011)

What is the Overall Wellbeing and Degree of Maltreatment Experienced by Children Coming In Contact With the Child Welfare System?

Researchers summarized the characteristics of well-being and maltreatment of children for the first wave of data collection in the National Survey of Child and Adolescent Well-Being.

Neglect was the most prevalent and often considered by caseworkers the most serious type of abuse for an individual child. Physical abuse was the second most prevalent type of abuse children experienced Children reported for maltreatment in 2008 were at higher risk for problems with poor health, mental health, cognitive development, and social competence than children in the general population.

Report Title: NSCAW II Baseline Report: Maltreatment Final Report http://www.acf.hhs.gov/programs/opre/abuse_neglect/nscaw/reports/nscaw2_maltreatment/nscaw2_maltreatment.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Mary Webb, 202-205-8628
Performer: RTI International
Record ID: 9781 (Report completed August 15, 2011)

and

Report Title: NSCAW II Baseline Report: Child Well-Being Final Report http://www.acf.hhs.gov/programs/opre/abuse_neglect/nscaw/reports/nscaw2_child/nscaw2_child.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Mary Webb, 202-205-8628
Performer: RTI International
Record ID: 9782 (Report completed August 15, 2011)

How Healthy Were Caregivers Of Children Reported for Maltreatment?

Researchers summarized the health, well-being, and services received by caregivers of the first wave of data collection from a nationally representative sample of children reported for maltreatment at birth to 17.5 years old.

The majority of caregivers were parents living in-home with their children. Many caregivers lived below the federal poverty level and approximately half were in very good or excellent health.

Report Title: NSCAW II Baseline Report: Caregiver and Health Services http://www.acf.hhs.gov/programs/opre/abuse_neglect/nscaw/reports/nscaw2_cg/nscaw2_cg.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Mary Webb, 202-205-8628
Performer: RTI International
Record ID: 9784 (Report completed September 15, 2011)

What Were The Experiences of Investigative Caseworkers and The Child Welfare System With Children and Families?

Researchers examined experiences children and families had with investigative caseworkers and the child welfare system in the second National Survey of Child and Adolescent Well-Being (NSCAW).

Caseworkers managed on average 15 new investigations monthly and recommended or provided at least one service to nearly two thirds of families they assisted. The most recommended services were caregiver counseling, parenting training, and caregiver substance abuse treatment.

Report Title: NSCAW II Baseline Report: Caseworker Characteristics, Child Welfare Services, and Experiences of Children Placed in Out-of-Home Care http://www.acf.hhs.gov/programs/opre/abuse_neglect/nscaw/reports/nscaw2_cw/nscaw2_cw.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Mary Webb, 202-205-8628
Performer: RTI International
Record ID: 9786 (Report completed September 15, 2011)

How Do Select Quality, Rating, and Improvement Systems Define and Measure Quality?

Researchers conceptualized quality and how it was measured in five Child Care Quality, Rating, and Improvement Systems (QRIS). Researchers identified explanations for the variations in these approaches.

There is greater consistency in environment rating scales than in the procedures for gathering evidence on other quality components or calculating ratings. Among the five systems studied, there was generally greater consistency in the definitions of quality components at the highest rating levels than at the baseline levels. These standards overlap with recommendations of accrediting organizations such as the National Association for the Education of Young Children.

Report Title: Defining and Measuring Quality: An In-Depth Study of Five Child Care Quality Rating and Improvement Systems http://www.acf.hhs.gov/programs/opre/cc/childcare_quality/five_childcare/five_childcare.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Ivelisse Martinez-Beck, 202-690-7885
Performer: Mathematica Policy Research
Record ID: 9789 (Report completed August 15, 2011)

Did The Impacts Of The Original Project Upgrade Study Persist As Children Progress Through Their First, Second, And Third Grade Years of Elementary School?

Researchers tested the effectiveness of three different language/literacy interventions on preschool-age children's language and emergent literacy skills in Project Upgrade. The three interventions had two components: a professional development component and a curriculum component. The professional development model had a staffing plan with several layers of supervision and a training plan that featured three sequenced training sessions over an 18-month period, combined with ongoing mentoring and support over the entire period. Two nationally known curricula and one developed by a local academic were tested.

There were significant impacts prior to school entry for two of the three interventions on children's language and emergent skills. The study also found significant impacts on both students whose teachers had received the curriculum training in English and students whose teachers received the training in Spanish, but the impacts were larger for the students with Spanish dominant teachers.

Report Title: Evaluation of Child Care Subsidy Strategies, Follow-Up Study of the Participants in Project Upgrade in Miami-Dade

http://www.acf.hhs.gov/programs/opre/cc/ccs_strategies/reports/subsidy_followup.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Ivelisse Martinez-Beck, 202-690-7885
Performer: Abt Associates
Record ID: 9812 (Report completed September 15, 2011)

What Is the Quality of Early Head Start Programs?

Researchers examined Early Head Start programs, staff qualifications, services delivered and characteristics of participating families.

Early Head Start programs provided diverse services to address family needs; children were served by well-qualified staff from diverse backgrounds; Early Head Start home visits and classrooms were in the mid-range of quality. There were some relations between family characteristics and service option, and between needs and services received.

Report Title: Learning As We Go: A First Snapshot of Early Head Start Programs, Staff, Families, and Children, Volume I: First Report

http://www.acf.hhs.gov/programs/opre/ehs/descriptive_study/reports/as_we_go.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Amy Madigan, 202-401-5143
Performer: Mathematica Policy Research
Record ID: 9813 (Report completed June 15, 2011)

What Are The Range Of Initiatives Underway Or Recently Implemented To Support Home-Based Child Care?

Researchers profiled a wide range of initiatives, either currently underway or recently implemented that support quality in home-based child care.

The study identified four types of initiatives according to their primary purpose: quality improvement, certificate program, support for licensing, and support for accreditation. Eighty of the initiatives in the compilation aimed to improve quality in home-based care. Some initiatives intended to enhance providers' support for a single development domain such as language or health. Seven of the initiatives aimed to help family, friend and neighbor caregivers become regulated providers. These initiatives provided information about establishing a child care business, offered training that complies with state regulatory requirements, and provided support for improving the home environment to meet state standards.

Report Title: A Compilation of Initiatives to Support Home-Based Child Care http://www.acf.hhs.gov/programs/opre/cc/supporting_quality/reports/compilation/compilation.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: T'Pring Westbrook, 202-401-5660
Performer: Mathematica Policy Research
Record ID: 9839 (Report completed August 15, 2010)

And

Report Title: Supporting Quality in Home-Based Child Care: A Compendium of 23 Initiatives http://www.acf.hhs.gov/programs/opre/cc/supporting_quality/reports/compendium_23/compendium_23.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: T'Pring Westbrook, 202-401-5660
Performer: Mathematica Policy Research
Record ID: 9840 (Report completed August 15, 2010)

And

Report Title: Supporting Quality in Home Based Child Care: Initiative Design and Evaluation Options

http://www.acf.hhs.gov/programs/opre/cc/supporting quality/reports/supporting options/supporting options.pdf

Agency Sponsor: ACF OPRE, Office of Planning, Research and Evaluation

Federal Contact: T'Pring Westbrook, 202-401-5660

Performer: Mathematica Policy Research

Record ID: 9841 (Report completed August 15, 2010)

And

Report Title: Supporting Quality in Home Based Child Care: Final Brief

http://www.acf.hhs.gov/programs/opre/cc/supporting_quality/reports/suppo...

Agency Sponsor: ACF OPRE, Office of Planning, Research and Evaluation

Federal Contact: T'Pring Westbrook, 202-401-5660

Performer: Mathematica Policy Research

Record ID: 9842 (Report completed August 15, 2010)

How Well Are Traumatized Children Receiving Care as a Result of the National Child Traumatic Stress Initiative?

The National Child Traumatic Stress Initiative (NCTSI) focuses on addressing child trauma issues. The Initiative provides Federal support for a national, science-based network of organizations that work to improve treatment and services for child trauma, expand availability and accessibility of effective community services, and promote better understanding of clinical and research issues relevant to effective interventions for children and adolescents exposed to traumatic events. The cross-site evaluation assessed the effectiveness of NCTSI as a whole, including the Network's impact as a national resource for enhancing the standard and quality of care for children and their families affected by traumatic stress.

The Core Data Set (CDS) collects information regarding child and family demographics, presenting problems, trauma history, and psychosocial information. There are currently 1,793 active cases, with a mean age of 10.7, in the CDS. Most common trauma types experienced by youth were domestic violence, traumatic loss or bereavement, and impaired caregiver. Over time, youth show significant improvements in depression, generalized anxiety, and attachment problems. The Trauma-Informed Services Survey found that during the year, 1,623 people were trained. Trainings about clinical interventions and psychoeducation were among the most common training types. Most participants reported positively about the strengths of the training, but nearly 22.3% anticipated barriers that might prevent them from putting knowledge and skills into practice. The Network Collaboration Survey found that a moderate degree of collaboration is occurring for many key activities of the Network, with the greatest level of collaboration occurring for the development of products. Collaboration in this domain appears to have become more diffuse in 2010, with many centers playing a key role in the Network for this core activity. In contrast, although a high level of collaboration is occurring for the delivery of trainings, grantees continue to rely on smaller number of centers for the delivery of their trainings.

Report Title: The National Child Traumatic Stress Initiative (NCTSI) Cross-site Evaluation Annual Progress Report

Agency Sponsor: SAMHSA, Substance Abuse and Mental Health Services Administration
Federal Contact: Ken Curl, 240-276-1779
Performer: ICF Consulting Group, Inc.
Record ID: 8729 (Report issued March 31, 2010)

Does Fire Safety and Prevention Education Presented to Children Change Knowledge and Self-Reported Behavior?

The Department of Homeland Security invited the Centers for Disease Control and Prevention's National Center for Injury Prevention and Control to evaluate fire safety education programs delivered to second grade students in school settings. The program documented changes in children's and parent's fire safety knowledge and behavior over time. Teachers, school administrators, and fire safety instructors (FSIs) were also surveyed. Eligible fire departments were recruited with an even distribution across urban, suburban or rural locations. Instruction took place in either the classroom/assembly or a fire safety trailer and results were compared to participants who received no instruction.

While similar at baseline, second grade students who received fire safety instruction in either the classroom or a trailer had significantly higher knowledge and behavior scores as compared to those who received no instruction, and maintained the knowledge over a six month period. Parents reported changes in fire safety actions, such as creating and practicing an escape plan, between initial reporting and follow up. Parents whose children received instruction planned more changes to fire safety behaviors in the home than those whose children did not receive instruction. More than half of parents indicated that changes resulted from information the child brought home or discussed with the parent after receiving fire safety education. However, there was dramatic variation in the types, structure, content and delivery of programs for both classroom/assembly and trailers limiting the ability to assess the effect of education in different venues. Structured, age-appropriate, and consistently presented curriculum or guidance for program development is needed. Further research should investigate the relative effects of standardized training provided in classrooms versus trailers and should support the development of behavioral and education theory-based guidelines for developing local programs or standardized, age-appropriate education programs.

Report Title: Evaluation and Assessment of the Effectiveness of Activities Supporting Fire Prevention and Safety

Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Julie Gilchrist, MD, 770-488-1178

Performer: McKing Consulting
Record ID: 9650 (Report issued April 30, 2010)

Objective: Promote economic and social well-being for individuals, families, and communities

What Interventions for Homeless Families are Associated with Positive Outcomes?

This study was a two-phase, five-year knowledge development initiative to document and evaluate the effectiveness of time-limited, intensive intervention strategies to provide mental health, substance abuse treatment, trauma recovery, housing, family preservation, and other services to homeless mothers with psychiatric and/or substance use disorders who are caring for their children.

Virtually all the families in the sample had experienced trauma. Significant positive improvement occurred in most outcomes for families. Ongoing conflict and trauma were strongly associated with poorer outcomes. The average rate of change for families on most outcomes was positive, regardless of whether they were served in the target interventions or in comparison interventions. The major factor related to receipt of housing is having a housing subsidy. Differences in the level of the services available were not related to changes in outcomes with the exception of the mental health outcome.

Report Title: Evaluation of the Effectiveness of Time-Limited Interventions for Homeless Families
Agency Sponsor: SAMHSA, Substance Abuse and Mental Health Services Administration
Federal Contact: Pamela Fischer, 240-276-1901
Performer: Policy Research Associates
Record ID: 7725 (Report issued June 30, 2010)

What Welfare Policies Does Each State's Temporary Assistance for Needy Families Program Have?

The Welfare Rules Database was created to track state-specific Temporary Assistance for Needy Families program rules for all states and the District of Columbia.

The Welfare Rules Database is available to the public (link provided below), enabling users to query rules across 30 policy areas, and includes longitudinal tables describing selected state policies from 1996 onward.

Report Title: Welfare Rules Datebook: State TANF Policies as of July 2009; http://www.urban.org/welfare/databook.cfm

Agency Sponsor: OASPE-OHSP, Office of Human Services Policy
Federal Contact: Brian Levy, 202-401-3953
Performer: Urban Institute
Record ID: 9663 (Report issued August 15, 2010)

How Many Single Mothers are Disconnected from Work and Public Support, and for How Long?

Researchers examined information on the population of people who are “disconnected” (not working or living with a working spouse and not receiving welfare or disability benefits). The study used nationally representative longitudinal data from the Survey of Income and Program Participation to examine movement into and out of being disconnected, events associated with becoming disconnected, and the size and characteristics of this population. The study also examined how people fare after being disconnected who return to employment or welfare, or find other forms of support.

About one in five low-income single mothers were disconnected from work and TANF in the mid-2000s. More than 40 percent remain disconnected for over a year. One-third of disconnected mothers live alone without other adults.

Report Title: Dynamics of Being Disconnected from Work and TANF http://aspe.hhs.gov/hsp/11/DisconnectedDynamics/index.shtml
Agency Sponsor: OASPE-OHSP, Office of Human Services Policy
Federal Contact: Laura Chadwick, 202-690-7772
Performer: Urban Institute
Record ID: 9665 (Report issued May 20, 2011)

Does A Marriage and Relationship Education Program Increase the Relationship Quality of Unwed Couples or The Likelihood That They Will Get Married or Remain Romantically Involved?

This study tested relationship skills education programs for unwed parents.

The programs had limited effects on participating couples. Averaging results across the eight local programs that participated in the evaluation, the programs had no effect on the couples' relationship quality or on the likelihood that they would remain romantically involved or get married within 15 months after they enrolled in the program. Only one of the eight programs had a consistent pattern of positive effects on couples' relationships, while another was found to have negative effects.

Report Title: BSF'S Effects On Couples Who Attended Group Relationship Skills Sessions: A Special Analysis of 15-Month Data

http://www.acf.hhs.gov/programs/opre/strengthen/build_fam/reports/bsf_relationships/bsf-relationships.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Nancye Campbell, 202-401-5760
Performer: Mathematica Policy Research
Record ID: 9670 (Report completed June 15, 2011)

What Were the Effects Of Marriage and Relationship Education Programs on Low-Income Parents And Their Children?

This study presented early implementation and operational lessons from the Supporting Healthy Marriage (SHM) evaluation. In the first year of program implementation, SHM providers focused on three main tasks: developing effective marketing and recruitment strategies, keeping couples engaged in the program, and building management structures and systems.

Simply distributing brochures and posters was not a sufficient recruitment technique for most programs. Programs also found that they needed to partner with local agencies and community organizations, and go into their communities to speak directly with couples. Programs made services as accessible and as attractive as possible. They offered activities during evenings and weekends, provided meals, transportation and child care assistance, emergency funds, and modest incentives. The research team held programs accountable by requiring that they achieve particular benchmarks in enrollment and participation. Early participation data showed that within six months of enrollment, more than 80 percent of couples attend at least one workshop and went on to complete an average of 20 workshop hours. More than 85 percent of couples attend at least one family support meeting and complete, on average, 4.5 meetings within six months.

Report Title: Early Lessons From the Implementation of A Relationship and Marriage Skills Program for Low-Income Married Couples

http://www.acf.hhs.gov/programs/opre/strengthen/support_hlthymarr/reports/early_lessons/early_lessons.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Nancye Campbell, 202-401-5760
Performer: MDRC
Record ID: 9671 (Report completed October 15, 2010)

What Were the Effects of the Center For Employment Opportunities Program On Recidivism Rates?

This study expanded on interim results from a rigorous impact evaluation of the New York City-based Center for Employment Opportunities (CEO), a transitional jobs program designed to help former prisoners increase longer-term employment and, consequently, reduce recidivism.

The program reduced recidivism in both the first and the second year of follow-up. The study used a regression-based analysis to identify whether CEO had its greatest impact among low-, medium-, or high-risk offenders. The study found the program had its strongest reductions in recidivism for former prisoners who were at highest risk of recidivism. The program reduced the probability of re-arrest, the number of re-arrests, and the probability of reconviction two years after random assignment.

Report Title: Enhanced Services for the Hard To Employ Demonstration: Recidivism Effects of the Center for Employment Opportunities (CEO) Program Vary By Former Prisoners' Risk of Reoffending

http://www.acf.hhs.gov/programs/opre/welfare_employ/enhanced_hardto/reports/ceo_program/ceo_program.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Girley Wright, 202-401-5070
Performer: MDRC
Record ID: 9672 (Report completed October 15, 2010)

Do Earnings Supplements Increase Employment, Income, and Employment Retention?

This study identified findings and lessons for policy and practice from five programs that provided earnings supplements and that had been rigorously evaluated using a random assignment research design.

Earnings supplements substantially increased employment and income and, in many cases, employment retention.

Report Title: Providing Earnings Supplements To Encourage and Sustain Employment

http://www.acf.hhs.gov/programs/opre/welfare_employ/employ_retention/reports/provide_earnings.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Michael Dubinsky, 202-401-3442
Performer: MDRC
Record ID: 9673 (Report completed June 15, 2011)

Can Federally-Funded Healthy Marriage and Responsible Fatherhood Programs Develop and Implement Culturally Relevant and Appropriate Services for Hispanic Populations?

Researchers examined ways in which nine federally funded healthy marriage and responsible fatherhood programs were developing, adapting and implementing culturally relevant and appropriate services for Hispanic populations.

The study documented the site selection process and described the selected programs.

Report Title: Providing Culturally Relevant Services: Programs in the Hispanic Healthy Marriage Initiative Implementation Evaluation http://www.acf.hhs.gov/programs/opre/strengthen/hispanic_healthy/reports/brief_his_healthy/hhmi_practict_brief_1_final.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Girley Wright, 202-401-5070
Performer: The Lewin Group
Record ID: 9674 (Report completed May 15, 2010)

What Were the Effects of Community-Wide Service Delivery System Education Programs Upon Marriages and Relationships?

This study examined the implementation of five research and demonstration projects funded and managed by the Office of Child Support Enforcement.

Local demonstration sponsors were able to mount sizable initiatives to teach individuals and couples healthy relationship and marriage education skills and provide related services. Large numbers of married couples and unmarried individuals and couples attended classes and other activities to improve their relationships. Demonstration leaders involved many other organizations in support of their goals. They used existing networks or formed new coalitions of partners to help identify and refer individuals and couples for demonstration services or to meet the needs of individuals or couples referred to them by demonstration staff. Partners also supported community awareness campaigns on the relationship between healthy marriage and child well-being, and were a source of volunteers who served in various roles, including conducting classes and workshops.

Report Title: Piloting A Community Approach To Healthy Marriage Initiatives in Five Sites: Minneapolis, Minnesota; Lexington, Kentucky; New Orleans, Louisiana; Atlanta, Georgia; and Denver, Colorado.

http://www.acf.hhs.gov/programs/opre/strengthen/eval_com/reports/piloting_five/piloting_five_exec.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Nancye Campbell, 202-401-5760
Performer: RTI International
Record ID: 9675 (Report completed December 15, 2010)

Did Program Models Designed to Promote Steady Work and Career Advancement Increase Employment and Advancement of Welfare Recipients?

This study tested the effectiveness of programs that attempted to promote steady work and career advancement for current and former welfare recipients and other low-wage workers, most of whom were single mothers.

Three of twelve programs included in the report produced positive economic impacts. Engaging individuals in employment and retention services at levels above what they would have done in absence of the programs was a challenge. Staff spent a lot of time and resources on placing unemployed individuals back into jobs, which made it difficult for them to focus on helping those who were already working to keep their jobs or move up.

Report Title: The Employment Retention and Advancement Project: How Effective were Different Approaches Aiming to Increase Employment Retention and Advancement? Final Impacts for Twelve Models http://www.acf.hhs.gov/programs/opre/welfare_employ/employ_retention/reports/different_approaches/era_core.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Michael Dubinsky, 202-401-3442
Performer: MDRC
Record ID: 9677 (Report completed May 15, 2010)

How Were Relationship Education Programs Being Implemented for Unmarried Couples with Children?

Researchers documented the design and implementation of Building Strong Families programs, described the services received and experiences of program group members, and analyzed characteristics of couples and programs that may affect participation.

The programs were implemented by non-profit and public agencies at 12 locations in seven states, and enrolled more than 5,000 volunteer couples, who were randomly assigned to an intervention or control group. The intervention featured up to 42 hours of multi-couple group sessions led by trained facilitators, focusing on skills that, according to earlier research, were associated with relationship and marital stability and satisfaction.

Report Title: The Building Strong Families Project: Implementation of Eight Programs to Strengthen Unmarried Parent Families http://www.acf.hhs.gov/programs/opre/strengthen/build_fam/reports/eight_programs/final_impl_rpt.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Seth Chamberlain, 202-260-2242
Performer: Mathematica Policy Research, Inc
Record ID: 9679 (Report completed May 5, 2010)

What Interventions Best Help to Strengthen the Families of Unmarried Couples with Children?

Researchers documented the design and implementation of a voluntary intervention for couples, the Couples Employment Project. The project is operated by the Center for Urban Families (CFUF) in Baltimore, Maryland.

The program addresses both employment and relationship issues at the same time. It emphasizes joint employment planning, relationship support, and engagement of the couple in dialogue, planning, and education focused on family economic self-sufficiency and healthy relationships.

Report Title: The Building Strong Families Project: Initial Implementation of A Couples-Focused Employment Program

http://www.acf.hhs.gov/programs/opre/strengthen/build_fam/reports/couples_focused/couples_focused.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Seth Chamberlain, 202-260-2242
Performer: Mathematica Policy Research, Inc.
Record ID: 9680 (Report completed October 15, 2010)

How Did the Welfare Caseload Respond to the Great Recession?

Researchers examined the factors associated with changes in the Temporary Assistance for Needy Families caseload in response to the “Great Recession” of the early years of the 21^st Century. They also explored how the caseload response to the recession compares with caseload responses during previous economic downturns.

TANF responded more strongly to changes in unemployment, economic variables, and work incentives during the Great Recession than it has in the past decade.

Report Title: TANF and the Great Recession: A Multivariate Analysis of Program Responsiveness
Agency Sponsor: OASPE-OHSP, Office of Human Services Policy
Federal Contact: Brian Levy, 202-401-3953
Performer: In-house staff
Record ID: 9698 (Report issued June 15, 2011)

How Many and Which Households Received Assistance with Heating Costs Through the Low Income Home Energy Assistance Program in FY 2007?

Program staff analyzed caseload and fiscal data for the Low Income Home Energy Assistance Program (LIHEAP) for FY 2007. OCS provided block grant funds to the 50 States, the District of Columbia, Indian Tribes and Tribal Organizations, and U.S. Insular Areas. The funds assist low income households primarily in meeting their immediate home energy needs, especially those households with very young children, individuals with disabilities, and/or frail older individuals.

In FY 2007, 5.3 million households received help with heating costs (the largest LIHEAP component) compared to 5.5 million households in FY 2006. These households represented 16 percent of all households with incomes under the Federal LIHEAP income eligibility cutoff. Households that received heating assistance were among the poorer households (median household poverty level of 80 percent) within the LIHEAP income eligible population (median household poverty level of 121 percent). About 32 percent of the heating assistance households had at least one member 60 years or older, compared to 40 percent of all low income households that had at least one member 60 years or older. About 31 percent of the heating-assisted households had at least one member with a disability slightly higher than the 26 percent of all low income households that had at least one disabled member. Twenty-two percent of both the heating-assistance households and all low income households had at least one child 5 years or younger.

Report Title: Low Income Home Energy Assistance Program Report to Congress for FY 2007 http://www.acf.hhs.gov/programs/ocs/liheap/publications/liheap07rc.pdf
Agency Sponsor: ACF-OCS, Office of Community Services
Federal Contact: Leon Litow, 202-401-5304
Performer: Office of Community Services, (OCS), ACF
Record ID: 9727 (Report issued August 2, 2010)

What Guidelines Need to be Used in Planning/Conducting a Quality Rating and Improvement Systems Evaluation?

Researchers identified key elements for evaluating Quality Rating and Improvement Systems of child development.

When planning and conducting a quality rating and improvement system evaluation should factor in several guidelines: develop a program logic model, use only child development or school readiness measures that have norms and solid evidence for reliability and validity with populations similar to those participating in the QRIS, and make sure the QRIS is fully implemented.

Report Title: Evaluation of Quality Rating and Improvement Systems for Early Childhood Programs and School-Age Care: Measuring Children's Development

http://www.acf.hhs.gov/programs/opre/cc/childcare_technical/reports/improv_systems.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Ivelisse Martinez-Beck, 202-690-7885
Performer: Child Trends
Record ID: 9815 (Report completed June 15, 2011)

Do Assisted Low-Income Single Parents Move Up In the Labor Market?

Researchers examined the work, education, and training patterns of single parents in the Employment Retention and Advancement (ERA) project.

Three years after entering the study, only one in four single parents had advanced. Most of the remaining parents either spent long periods out of work or they lost ground. Single parents who advanced worked more consistently over the study period than other parents and, if they were unemployed, they returned to work more quickly. They experienced faster earnings growth while working than other parents, especially when they changed jobs. At the end of the study period, they worked in better jobs, such as those with higher pay and more benefits, than parents who had not advanced. These findings support other research in underscoring the importance of changing jobs and of access to "good" jobs as strategies to help low-wage workers advance.

Report Title: Can Low-Income Single Parents Move Up in the Labor Market' Findings From the Employment Retention and Advancement Project

http://www.acf.hhs.gov/programs/opre/welfare_employ/employ_retention/reports/labor_market.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Michael Dubinsky, 202-401-3442
Performer: MDRC
Record ID: 9816 (Report completed January 15, 2011)

What Are the Paths to Advancement for Single Parents?

Researchers examined the work, education, and training experiences of single parents targeted by the Employment Retention and Advancement project.

Few parents advanced over time, and most of the remaining parents either spent long periods out of work or lost ground; parents who advanced had worked more stably over the period than other parents; parents who did not work during Year 3 had very high rates of employment instability; in terms of demographic characteristics and experiences, parents who worked but had not advanced were between these two extremes (that is, between parents who advanced and those who did not work in Year 3; and job changing is an important route to advancement.

Report Title: Paths To Advancement for Single Parents

http://www.acf.hhs.gov/programs/opre/welfare_employ/employ_retention/reports/single_parent.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Michael Dubinsky, 202-401-3442
Performer: MDRC
Record ID: 9817 (Report completed November 15, 2010)

What Are the Background Characteristics And Patterns Of Employment, Earnings, And Public Assistance Receipt For Adults In Two-Parent Families?

Researchers identified the background characteristics, employment and earnings patterns, and patterns of Temporary Assistance for Needy Families (TANF) and food stamp receipt for adult members of two-parent families in the Employment Retention and Advancement (ERA) project. Not much is known about the low-income two-parent population's need for employment retention and advancement services or about their responses to offered services. This population has particular policy relevance in that two-parent TANF cases include more family members and receive higher average monthly grants than do single-parent recipients. These families therefore require higher income (from employment of one or both parents) to achieve self-sufficiency.

In the ERA sample, retention and advancement was as important an issue for low-income two-parent family members as for single parents. Men and women in two-parent families were equally likely to work during the follow-up period, but men earned more, on average. Adults in low-income single-parent and two-parent families have a roughly equivalent need for services to support employment retention and advancement. This need does not differ substantially between men and women in two-parent families.

Report Title: Background Characteristics and Patterns of Employment, Earnings, and Public Assistance Receipt of Adults in Two-Parent Families

http://www.acf.hhs.gov/programs/opre/welfare_employ/employ_retention/reports/two_parent.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Michael Dubinsky, 202-401-3442
Performer: MDRC
Record ID: 9818 (Report completed November 15, 2010)

What Are the Reemployment Strategies In Retention and Advancement Programs For Current and Former Welfare Recipients?

Researchers examined strategies used in the Employment Retention and Advancement (ERA) project to reemploy program participants who quickly lost jobs.

Employment programs seeking to improve retention and support advancement among workers should be prepared to address job loss quickly. These programs need to help newly unemployed workers learn from their experience. They should also focus on placing individuals into better jobs that – besides providing higher wages – match their skills, interests, and education, to the extent possible.

Report Title: Finding the Next Job: Reemployment Strategies in Retention and Advancement Programs for Current and Former Welfare Recipients
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Michael Dubinsky, 202-401-3442
Performer: MDRC
Record ID: 9819 (Report completed June 15, 2010)

What Are the Benefit-to-Cost Ratios of Three Employment Retention And Advancement Programs?

Researchers analyzed the financial benefits and costs of three diverse Employment Retention and Advancement (ERA) Project programs designed to increase employment stability and career advancement among current and former welfare recipients. The analysis focused on three programs in Corpus Christi and Fort Worth, Texas; Chicago, Illinois; and Riverside County, California. These programs were selected because comparisons between the program and control groups indicated that these programs increased individuals' employment and earnings –– the primary goal of the project.

Program group members were better off financially as a result of ERA. From the perspective of the government budget, Riverside program broke even, but the programs in Chicago and Texas did not produce net savings. All three programs produced financial gains for society as a whole. For every dollar that the government invested in these programs, program group members gained more than one dollar. This suggests that the three programs were cost-effective.

Report Title: Benefit-Cost Findings for Three Programs in the Employment Retention and Advancement (ERA) Project

http://www.acf.hhs.gov/programs/opre/welfare_employ/employ_retention/reports/benefits_cost/benefits_cost.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Michael Dubinsky, 202-401-3442
Performer: MDRC
Record ID: 9820 (Report completed May 15, 2011)

How Effective Are Different Approaches That Aim To Increase Employment Retention And Advancement?

Researchers presented the final impacts for 12 of the 16 Employment Retention and Advancement (ERA) programs and summarized how the 12 programs were implemented and individuals' levels of participation in program services.

Three programs produced positive economic impacts; nine did not. Increases in participation beyond control group levels were not consistent or large. This may have made it difficult for the programs to achieve impacts on employment retention and advancement. Engaging individuals in employment and retention services at levels above what they would have done in the absence of the programs was a consistent challenge. In addition, staff had to spend a lot of time and resources placing unemployed individuals back into jobs, which made it difficult for them to focus on helping those who were already working to keep their jobs or move up.

Report Title: How Effective are Different Approaches Aiming to Increase Employment Retention And Advancement? Final Impacts for Twelve Models http://www.acf.hhs.gov/programs/opre/welfare_employ/employ_retention/reports/different_approaches/era_core.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Michael Dubinsky, 202-401-3442
Performer: MDRC
Record ID: 9821 (Report completed August 10, 2010)

Do Partners And Spouses Influence the Decisions Of Social Services Clients?

Researchers investigated the role of partners and spouses in the decision-making processes of social service clients through direct observation of clients' interactions with their partners/spouses.

Direct observations of low-income couples showed that whether married or unmarried, low-income individuals in couple relationships affect each other's decision-making behavior in ways similar to those in married, middle-class families. The observed influence of partners may extend to program-related decisions such as take-up, participation, and behavior change.

Report Title: Exploring the Role of Partners and Spouses in the Decisions of Social Service Clients

http://www.acf.hhs.gov/programs/opre/strengthen/couples_decision/reports/brief_expl_role/brief_expl_role.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Seth Chamberlain, 202-260-2242
Performer: Mathematica Policy Research
Record ID: 9831 (Report completed December 15, 2010)

Do Couples' Dynamics Affect Their Ability To Take Up And Benefit From Services?

Researchers studied how couples' interactions affected how they used services, such as those that sought to involve fathers, strengthen families, and improve children's outcomes.

Individuals in low-income couples—whether married or unmarried—affected each other's decisions. These patterns of influence may extend to the kinds of decisions that were central to the objectives of social services. The findings also revealed how self-reported survey measures of couple functioning may have reflected actual behavior of low-income couples.

Report Title: Exploratory Study of Decision-Making in Low-Income Couples: Technical Report

http://www.acf.hhs.gov/programs/opre/strengthen/couples_decision/reports/exploratory_st/exploratory_st.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Seth Chamberlain, 202-260-2242
Performer: Mathematica Policy Research
Record ID: 9832 (Report completed December 15, 2010)

What Was the Implementation Strategy Of A Couple-Focused Employment Programs?

Researchers tested a program model for couples that emphasized joint employment planning, relationship support, and engagement of the couple in dialogue, planning, and education focused on family economic self-sufficiency and healthy relationships.

There were four challenges experienced during implementation: defining an integrated program model, identifying strong management, hiring staff with appropriate backgrounds, and maintaining a focus on couples. Program operators have continued to refine the program model as well as its management and operation. This program model has little precedent and is breaking new ground. Social service programs that typically serve individuals often do not address or substantively acknowledge a partners' role.

Report Title: Initial Implementation of a Couples-Focused Employment Program http://www.acf.hhs.gov/programs/opre/strengthen/build_fam/reports/couples_focused/couples_focused.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Nancye Campbell, 202-401-5760
Performer: Mathematica Policy Research
Record ID: 9833 (Report completed October 15, 2010)

What Are The Findings From the Compassion Capital Fund Evaluation?

Researchers evaluated the Compassion Capital Fund.

The Fund increased organizational capacity. The non-profit organizations that received capacity building services under the program experienced significantly higher levels of growth in each of the five critical areas examined compared to organizations assigned to the control group, when contributions of all of the measures for the area were considered together. The program group also had significantly higher capacity growth than the control group on some individual outcome measures in each of the five areas.

Report Title: Building Non-Profit Capacity: Findings from the Compassion Capital Fund Evaluation
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Nancye Campbell, 202-401-5760
Performer: Abt Associates
Record ID: 9835 (Report completed June 15, 2010)

In What Ways Did the Compassion Capital Fund Demonstration Program Improve Its Capacity?

Researchers tested the effectiveness of the Compassion Capital Fund (CCF). CCF builds organizational capacity among non-profit organizations (NPOs) to strengthen their long-term sustainability and ability to provide effective social services. Five critical areas of organizational capacity were examined, organizational development, program development, revenue development, leadership development, and community engagement. The program group members showed higher levels of capacity gain than members of the control group.

Organizations served by the Compassion Capital Fund intermediaries experienced substantial changes in multiple areas of organizational capacity. Statistically significant changes were found for at least one measure in each of the substantive organizational development areas targeted.

Report Title: Improving Capacity among Nonprofits -- Impact Study of the Compassion Fund Demonstration http://www.acf.hhs.gov/programs/opre/other_resrch/ccf/reports/ccf_impact_study/ccf_impact_rprt_final.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Nancye Campbell, 202-401-5760
Performer: Abt Associates
Record ID: 9836 (Report completed June 15, 2010)

and

Report Title: Assessing Changes in Nonprofit Capacity -- Outcome Study of the Compassion Capital Fund Demonstration http://www.acf.hhs.gov/programs/opre/other_resrch/ccf/reports/assessing_technical/outcome_study_rpt_final.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Nancye Campbell, 202-401-5760
Performer: Abt Associates
Record ID: 9837 (Report completed June 15, 2010)

What Is Known About Home-Based Child Care?

Researchers summarized what is known about home-based child care, identified gaps in the research, and identified promising approaches for further exploration.

Family support and home visiting initiatives that aim to improve outcomes for parents and children may have potential for supporting quality in home-based care, because home-based child care shares some characteristics with care within the family. Studies show modest effects on a range of parent and child outcomes. They indicated that family support programs were effective in promoting children's cognitive development and school outcomes only if they supplement services to support parents with services provided directly to children, such as center-based preschool.

Report Title: A Review of the Literature on Home-Based Child Care: Implications for Future Directions http://www.acf.hhs.gov/programs/opre/cc/supporting_quality/reports/lit_review/lit_review.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: T'Pring Westbrook, 202-401-5660
Performer: Mathematica Policy Research
Record ID: 9838 (Report completed August 15, 2010)

What Are the Early Impacts Of the Building Strong Families Project?

Researchers tested interventions designed to improve the quality of unmarried parents' relationships and increase the likelihood that they would remain together. This impact evaluation was designed to identify the extent of success but not to provide an explanation of why one program was more successful than another.

When all the Building Strong Families (BSF) programs were combined, BSF did not improve relationship quality or make couples more likely to remain romantically involved or get married. Fifteen months after entering the program, the relationship outcomes of BSF couples were, on average, almost identical to those of couples in the control group. Program impacts varied substantially across the eight programs included in the evaluation. The program in Oklahoma City had a consistent pattern of positive effects on relationship outcomes, while the Baltimore program had a number of negative effects. The other programs generally had little or no effect on relationships. Given the wide variation in program effects, it is useful to consider what is distinctive about the two programs with the strongest patterns of effects—Oklahoma City and Baltimore.

Report Title: Strengthening Unmarried parents' Relationships: The Early Impacts of Building Strong Families http://www.acf.hhs.gov/programs/opre/strengthen/build_fam/reports/unmarried_parents/15_impact_main_rpt.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Seth Chamberlain, 202-260-2242
Performer: Mathematica Policy Research
Record ID: 9844 (Report completed June 15, 2010)

and

Report Title: Implementation of Eight Programs to Strengthen Unmarried Parent Families
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Seth Chamberlain, 202-260-2242
Performer: Mathematica Policy Research
Record ID: 9845 (Report completed June 15, 2010)

Objective: Promote prevention and wellness

How Do States, Localities and Tribal Communities Use Healthy People 2010?

Researchers examined how HP2010 initiatives including HealthierUS were being used among states, localities, and tribal communities. The assessment explored use, users and why and how user groups described the contribution of the two initiatives to their work in health promotion.

The project helped establish the efficacy of these nationwide initiatives through an understanding of where and how groups with the ability and resources to bring the programs to life were carrying out the program objectives. Data collected from this project was used to establish a Performance Assessment Reporting Tool measure for the Office of Disease Prevention and Health Promotion. This was an annual output measure for the percentage of states that use the national objectives in their health planning processes.

Report Title: Healthy People User Study

http://aspe.hhs.gov/health/reports/09/userstudy/index.shtml .
Agency Sponsor: OASPE-OHP, Office of Health Policy
Federal Contact: Wilma Tilson, 202-205-8841
Performer: NORC
Record ID: 9633 (Report issued March 28, 2010)

How Do States, Territories, and Tribes Use the Healthy People 2020 Framework to Develop Strategic Plans for Improving Their Population's Health?

This project evaluated the effectiveness of the Healthy People 2020 framework in guiding the development of State, Territorial, and Tribal health improvement action plans and their corresponding policies and programs. The project results will inform the Department's strategies for implementing Healthy People 2020 over the coming decade. Each project did an evaluation which was consolidated into a final evaluation report. The 14 projects included two to tribal health entities and one to a territory. Most addressed health disparities, social determinants of health, data infrastructure, and/or emergency preparedness as a part of their efforts relating to the Healthy People 2020 framework. Awardees cited new partnerships, training, data reports, and most importantly, stakeholder input among the most important outcomes of the project.

Overall, states, tribes and territories were able to develop and implement successful projects addressing Healthy People 2020 using only the framework as guidance. They noted that the most resource-intensive as well as the most meaningful activities were building partnerships, and planning, as well as collecting, managing, and reporting data. Emphasizing the need for measurable goals and objectives—particularly for social determinants of health, awardees recommended that the Healthy People 2020 objectives include performance indicators as well as standardized reporting methodologies to enable entities to measure and report on progress. They also recommended development of a mechanism whereby states, territories, and tribes can share best practices and lessons learned. Awardees expressed their interest in continuing to participate in similar future activities with the Office of Disease Prevention and Health Promotion. Some recommended that Healthy People 2020 establish an evaluation workgroup.

Report Title: Achieving a Healthier Nation Through State Action
Agency Sponsor: OASH, Office of the Assistant Secretary for Health
Federal Contact: Carter Blakey, 240-453-8254
Performer: John Snow, Inc.
Record ID: 9288 (Report issued September 21, 2010)

What Methodological Options Exist for Collecting Data on Hard-to-Reach Populations?

This project identified, analyzed, tested, and presented other methodological options that could fill persistent data gaps for small racial and ethnic minority and other hard-to-reach populations in order to more effectively address health disparities impacting these groups. The project began with a systematic review, interpretation, and synthesis of the research literature on population-based surveys to identify and evaluate the extent to which methods of data collection/analysis have been successfully used to collect/analyze data on racial and ethnic minority populations. A cost-effective and statistically reliable approach to deriving estimates of prevalence of various health behaviors, risk factors, and outcomes in small racial and ethnic and other hard-to-reach groups was identified and developed, followed by validation of the proposed approach and identification of its limitations and the conditions under which it is best used.

The study resulted in the modification of an existing statistical tool (the Kalman Filter) and the development of the Modified Kalman Filter, a new statistical tool that uses national data to pool estimates over time to make valid inferences about small populations. The project included the development of software and a step-by-step manual to facilitate use of the Modified Kalman Filter by analysts and improved accuracy of health measures and outcomes for small racial and ethnic minority and other hard-to-reach population groups.

Report Title: Improving Modified Kalman Filter Procedures: Final Report
Agency Sponsor: OASH, Office of the Assistant Secretary for Health
Federal Contact: Audrey Burwell, 240-453-8222
Performer: Rand Corporation
Record ID: 8231 (Report issued April 29, 2011)

Are There Improved Health Outcomes And Cost Savings For Hypertension In Federally Qualified Health Centers That Implement The Chronic Care Model?

This study determined whether the Chronic Care Model (CCM), which offers enhanced patient care, is an effective strategy to improve blood pressure and lipid (cholesterol) levels. The study also assessed CCM cost-effectiveness.

Hypertension and cholesterol control improved among patients when compared to usual care. The costs that most healthcare providers would need are about $500 per-person per-year to implement similar enhanced care programs to a large population. Clinics were also found to be cost-effective. A cost-tool was produced that can be used by state health departments working with Medicaid and Medicare data, and with healthcare provider plans. The tool is being pilot tested in partnership with state health departments who will make it available to health plans and other users who want to determine cost-effectiveness of interventions they are conducting.

Federal Contact: Nell Brownstein 770-488-2570
Performer: RTI International
Record ID: 9220 (Report issued April 1, 2010)

To What Extent do States Health and Human Service Programs Collect Data on Applicant Housing Status and Homelessness?

The study investigated which states collect housing status data from applicants for Medicaid and/or Temporary Assistance for Needy Families (TANF), the two largest HHS mainstream programs that may serve individuals or families experiencing homelessness. Information about data collected at application was collected in a survey of the 50 states and the District of Columbia.

More than half of the states collect information on homelessness indicators and risk factors for applicants. In addition, several that do not collect this information on the application nonetheless address homelessness through interviews that are part of the application process. Only 13 states do not collect any homelessness data from Medicaid applicants and 10 do not collect homelessness data from TANF applicants. Even though homelessness is not a condition of eligibility for TANF and Medicaid, 14 states responded that they have adopted a formal definition of homelessness that is used to document homelessness for a variety of programs. Homelessness data tend to be collected on applications for the following purposes: to document living arrangements and applicant home address; to facilitate eligibility determination and administration of SNAP benefits; and to understand service needs and make referrals to needed services. The study team also reviewed practices related to homelessness data collection in nine other HHS mainstream programs. Five of the nine programs examined already report data annually on the number of homeless persons served. Each of these current data sources yield a limited amount of information about the homelessness status of program beneficiaries, and there may be an opportunity to conduct further analysis on these data sources to learn more.

Report Title: Homelessness Data in Health and Human Services Mainstream Programs: Final Report http://aspe.hhs.gov/hsp/09/HomelessnessData/index.shtml#execsum
Agency Sponsor: OASPE, Office of the Assistant Secretary for Planning & Evaluation
Federal Contact: Flavio Menasce, 202-260-0384
Performer: Abt Associates Inc, Bethesda
Record ID: 9324 (Report issued December 1, 2009)

Which States Collect Housing Status Data?

The Homelessness Data in Health and Human Services Mainstream Programs study investigated which states currently collect housing status data from applicants for Medicaid and/or Temporary Assistance for Needy Families (TANF), the two largest HHS mainstream programs that may serve individuals or families experiencing homelessness. Information about data collected at application was collected in a survey of the 50 states and the District of Columbia. In support of a broad goal of using the benefit assessment process to make referrals to homelessness services, the Housing Status Assessment Guide for State TANF and Medicaid Programs provides recommendations on a set of standardized housing status and homelessness risk questions that could be incorporated into state applications for TANF and/or Medicaid.

The Guide includes a Housing Status Assessment Tool, as well as a Housing Status Summary and an Assistance Priority & Response Matrix that could be used following assessment to determine the relative priority and appropriate intervention for each family or individual, based on present housing needs, as well as links to relevant federal resources.

Report Title: Housing Status Assessment Guide for State TANF and Medicaid Programs http://aspe.hhs.gov/hsp/09/HomelessnessDataHHS/HousingStatusGuide/index.shtml
Agency Sponsor: OASPE, Office of the Assistant Secretary for Planning & Evaluation
Federal Contact: Flavio Menasce, 202-260-0384
Performer: Abt Associates Inc, Bethesda
Record ID: 9324.1 (Report issued December 1, 2009)

To What Extent do States Health and Human Service Programs Collect Data on Applicant Housing Status and Homelessness?

The study found that thirty states currently collect information on homelessness or risk factors for homelessness from applicants for TANF or Medicaid. This report provides ideas for potential uses of this information for policymakers (especially at the state level) and researchers. The report summarizes ideas for data analysis that would help address three research areas that should be of interest to policymakers and researchers. Extent to which people who are homeless participate in TANF and Medicaid, whether or not TANF and Medicaid are protecting people from becoming homeless, and whether or not people experiencing homelessness are using TANF and Medicaid in ways that are different from other program participants.

Report Title: Potential Analyses with Homelessness Data: Ideas for Policymakers and Researchers http://aspe.hhs.gov/hsp/09/HomelessnessDataHHS/PotentialAnalyses/index.shtml
Agency Sponsor: OASPE, Office of the Assistant Secretary for Planning & Evaluation
Federal Contact: Flavio Menasce, 202-260-0384
Performer: Abt Associates Inc, Bethesda
Record ID: 9324.2 (Report issued December 1, 2009)

Goal 4: Increase Efficiency,Transparency,and countability of HHS Programs

Objective: Ensure program integrity and responsible stewardship of resources

How Are Market Forces Operating in the Medicare Advantage Program?

Researchers constructed a multi-year analysis file containing data on plan participation, enrollment, disenrollment, benefit design, quality, and county characteristics. The project included a descriptive analysis of plan-level changes in Medicare Advantage (MA) enrollment and disenrollment, an examination of the relationship between enrollment and disenrollment patterns and plan characteristics such as premiums, benefits and quality, a literature review on the functioning of competitive forces in the marketplace, and an environmental scan of efforts by private purchasers and States to introduce pay for performance (P4P) methods at the health plan level.

MA enrollment grew between 2007 and 2008. One-third of beneficiaries enrolled in MA had some change in plan or status, either voluntary or involuntary. Voluntary disenrollments, which occurred in 15 percent of all enrollments, accounted for the majority of all disenrollments for all plan types except Private Fee-For-Service (PFFS) plans, whose large proportion of involuntary disenrollments were driven by plan terminations. The rapid disenrollment rate—a measure of the proportion of new enrollees who disenrolled within the first three months of enrollment—was about 9 percent on average for individual market MA plans overall. The MA program continued to expand in 2009. Health Maintenance Organizations continued to account for the majority of MA enrollees, and to provide better value in terms of average premium and benefits, while PFFS plan enrollment slowed. Plan premiums and basic benefits, such as prescription drug benefits, are key drivers of enrollment at the plan/county level. Available data on MA quality indicators provided little evidence of improvement over time among those plans reporting. The environmental scan found that some State Medicaid programs and private purchasers have been using P4P to incentivize and improve health plan performance.

Report Title: Analysis of the Medicare Advantage Marketplace
Agency Sponsor: OASPE-OHP, Office of Health Policy
Federal Contact: Audrey McDowell, 202-690-7770
Performer: Mathematica Policy Research
Record ID: 9625 (Report issued November 1, 2009)

What Explains Medicare Payment Differences for the Same Medical Treatment by Outpatient Medical Setting and are they Justified?

This project provided recommendations on how to make payment incentives more uniform across different outpatient settings to discourage shifting the provision of medical procedures to settings with higher payment rates. Researchers looked at a small number of procedures performed in multiple settings in order to first derive a methodology to compare prices across settings for similar procedures and ultimately to develop policy recommendations. The most important methodological development of the project was to develop a methodology to compare prices across physician and hospital outpatient department settings for similar procedures because ancillary services associated with the procedures are bundled differently in these settings.

Payment differentials between settings are large and variable among procedures to an extent that do not appear justified by factors the researchers examined. Policy recommendations to reduce inconsistent payment for the same procedures in different ambulatory care settings included; eliminate duplicate overhead payment when a physician provides services in an Ambulatory Service Center, cap certain hospital outpatient based non-surgical procedures at the physician fee schedule rate, extend bundling for routine pre-procedural tests and other general proposals aimed to reduce or eliminate inappropriate Medicare payments across multiple ambulatory settings.

Report Title: Policy Options for Addressing Medicare Payment Differentials across Ambulatory Care Settings

http://www.rand.org/content/dam/rand/pubs/technical_reports/2011/RAND_TR979.pdf
Agency Sponsor: OASPE-OHP, Office of Health Policy
Federal Contact: George Greenberg, 202-260-6217
Performer: Rand Corporation
Record ID: 9626 (Report issued June 1, 2011)

For Medicare Post-Acute Care Episodes, How Do Definitions Differ Based on Alternative Starting Points and How Do Payments and Service Mix Differ for Longitudinal and Cross-Sectional Samples?

Researchers expanded the beneficiary level episode file developed previously to reflect a longitudinal and cross-sectional analysis of three years of beneficiary level episodes. The current 5% Medicare claims sample was expanded to 30% and 100% for Medicare long-term care hospital and inpatient rehabilitation facility users. The analysis examined hospital initiated and community entrant post-acute care use episodes and bundled payments. The larger sample size will provide insight to differences in payments by geographic area.

The inclusion or exclusion of readmissions and subsequent post-acute care use has a significant impact on total episode length of stay and payments. These analyses demonstrate the shorter term, higher cost nature of service use for beneficiaries initiating a post-acute care episode with an acute hospitalization compared with the longer term, relatively lower cost service use of those entering post-acute care directly from the community. The results of the geographic analysis highlight that provider supply and geography are significant drivers of post acute care utilization and spending.

Report Title: Post-Acute Care Episodes Expanded Analytic File;

http :// aspe.hhs.gov/health/reports/2011/PACexpanded/index.shtml
Agency Sponsor: OASPE-OHP, Office of Health Policy
Federal Contact: Susan Bogasky, 202-401-0882
Performer: Research Triangle Institute
Record ID: 9629 (Report issued April 1, 2011)

How Do Medicare Post-Acute Care Services and Payment Levels Differ Across Alternative Episode Definitions?

This project built on prior Medicare episode payment research by considering eighteen alternative Medicare acute hospital plus post-acute care episode payment definitions. The episodes were defined with fixed end points such as 30, 60, 90 days post-acute hospital discharge and a variety of variable length end points. The analysis modeled the impact of the potential bundled payments. The project examined differences in payments across the episode payment options by geographic area using standardized payments.

The mean post-acute care episode payment level is largely driven by the alternative definitions used to define the end point of an episode; the sample over which the payments are averaged; the mix and volume of post-acute care services included in the episode definition; and the inclusion or exclusion of acute hospital readmissions in the episode definition.

Report Title: Post-Acute Care Episodes

http://aspe.hhs.gov/health/reports/09/pacepifinal/index.shtml
Agency Sponsor: OASPE-OHP, Office of Health Policy
Federal Contact: Susan Bogasky, 202-401-0882
Performer: Research Triangle Institute
Record ID: 9630 (Report issued November 1, 2009)

What Factors Motivate Employers' to Self-Insure and How Might this Change After the Affordable Care Act Takes Full Effect?

Researchers compared the characteristics of firms that chose to self-insure and those that chose to fully-insure their group health plans. Researchers examined the factors that appeared to influence firms' decisions to select a particular method of funding health plans, and determined the extent to which new insurance market reforms are likely to cause adverse selection in the large group market or to encourage small and midsize employers to self-insure.

Larger employers are much more likely to offer a self-insured plan than are smaller employers, with employers in industries like healthcare, manufacturing, transportation, utilities, and communication more likely to self-insure, and those less likely to self-insure in retail and the agriculture, mining, and construction industries. The research observed similar premiums across self and fully-insured health plans for most firm sizes and plan types. Additionally, the study finds little difference in plan generosity between self-insured and fully-insured plans when comparing plans of the same size and concludes with no evidence of systematically lower or higher out-of-pocket payments at self-insured firms compared with fully-insured firms. The research findings suggest that there will be little change in self-insurance rates after the Affordable Care Act's insurance market reforms are fully implemented in 2014, although the results are sensitive to the assumptions made about the availability of attractively priced reinsurance providing coverage beginning at low levels of stop-loss.

Report Title: Large Group Market Study http://aspe.hhs.gov/health/reports/2011/lghpstudy/index.shtml
Agency Sponsor: OASPE-OHP, Office of Health Policy
Federal Contact: Thomas Musco, 202-690-7272
Performer: RAND
Record ID: 9751 (Report issued June 1, 2011)

Is the Community Mental Health Services Block Grant Being Implemented and Is it Achieving the Results Intended?

The Community Mental Health Services Block Grant is a flexible funding source that may be used to provide a range of mental health services. The funds also support service delivery through planning, administration, evaluation, and educational activities. State's develop their block grant plans in collaboration with the State mental health planning councils in which membership is statutorily mandated to include consumers, family members of adult and child consumers, providers, and representatives of other principal state agencies. This evaluation examined the extent to which States and the Federal government are implementing the program according to the authorizing legislation, capacity to leverage block grant funds to implement State mental health system transformation, the State planning process, and implementation reports.

Over six million adults and children accessed mental health services through state mental health agencies during FY 2010. All States had mental health planning and advisory councils which played significant roles in statewide planning, advocacy, and outreach efforts. The block grant program empowered state mental health administrations to better address the needs of adults and children with serious mental illnesses and serious emotional disturbances. States used the funds to initiate or supplement promising practices including peer support, jail diversion, suicide prevention, telemedicine, self-directed care, and disaster response.

Report Title: National Evaluation of the Community Mental Health Services Block Grant (CMHSBG) http://store.samhsa.gov/product/SMA10-4610
Agency Sponsor: SAMHSA, Substance Abuse and Mental Health Services Administration
Federal Contact: John Morrow, 240-276-1783
Performer: Altarum Institute
Record ID: 8719 (Report issued February 28, 2011)

Goal 5: Strengthen the Nation's Health and Human Service Infrastructure and Workforce

Objective: Invest in the HHS workforce to help meet America's health and human service needs today and tomorrow

How Can the National Institutes of Health Streamline Mandatory Training for Intramural Staff?

This study evaluated the National Institutes of Health's (NIH) offering of mandatory training courses to determine if NIH could streamline mandatory training for intramural staff into a singular mandatory training program that would be effective and meet regulatory and program requirements. At the time of this study, NIH's mandatory training consisted of individual courses or course segments that were administered by multiple business owners. The training was not offered in integrated packages and the business owners did not coordinate or collaborate on the training. For this study, evaluators reviewed mandatory training courses, interviewed business owners, distributed an electronic questionnaire to intramural scientific and administrative staff, conducted focus groups and researched best practices.

Evaluators identified several barriers to training, including having to navigate various training sites, uncoordinated delivery of refresher training, excessive delivery frequency, confusing communications, and inappropriate training design. Based on these findings, the evaluators recommended that NIH create a single site for mandatory training to reduce staff frustration and confusion. They also recommended that NIH develop a standard approach to analyze, design, and deliver refresher training; minimum training evaluation standards; online training standards; and an NIH Training Coordinator Network to promote communication and collaboration.

Report Title: The Evaluation of NIH Intramural Mandatory Training
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: Management Analysis, Inc (MAI)
Record ID: 9402 (Report issued April 17, 2010)

Is Complete Citizenship Information Available to Support Evaluations of the Pathway to Independence Award Program?

The National Institutes of Health's Pathway to Independence Award Program was intended to foster the careers of recently trained investigators, and allow them to receive mentoring and independent research support from a single award. Unlike other NIH career development award programs, this award program is open to non-citizens as well as citizens.

Citizenship information needed for the program evaluation was reviewed and found to be lacking for more than 30% of applicants. The study examined the application records of about 600 applicants for whom citizenship status was missing. By reviewing application forms, and contacting applicants, their mentors, and sponsoring institutions, evaluators located their citizenship information and loaded it into NIH's administrative database. Information on the citizenship of these early program participants, once complete, was made available for future evaluations of the program. An electronic application form was introduced that included automatic edit checks to ensure that citizenship information is captured for all future applicants.

Report Title: K99 Applicants and Awardees, FYs 2007-2009
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: Ripple Effect Communications
Record ID: 9383 (Report issued March 31, 2010)

How Well Does the Community for Advanced Graduate Training Website Meet the Needs of its Users?

This study evaluated the National Institute of General Medical Sciences (NIGMS) Community for Advanced Graduate Training (CAGT) website to determine whether it met the needs of its users and how its usability and functionality might be improved. The website provides a "matching service" between undergraduate students from the NIGMS Minority Access to Research Careers (MARC) Undergraduate Student Training in Academic Research (U-STAR) program and graduate research training programs funded through National Research Service Award Institutional Pre-doctoral Training grants.

The evaluation team received positive feedback from program participants. They reported that the CAGT website aided in the recruitment and placement of students by providing institutions with greater access to high-quality students and students with one convenient place to search for research programs across the country. Shortcomings of the website were also identified and several programmatic and functional changes were recommended to improve its robustness and functionality.

Report Title: National Institute of General Medical Sciences Community for Advanced Graduate Training (CAGT) Website Evaluation

Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: Highrise Consulting, Inc.
Record ID: 9422 (Report issued May 2, 2011)

Are Process and Outcome Evaluations Feasible for the National Human Genome Research Institute Summer Internship Program?

This study determined the feasibility of conducting process and outcome evaluations of the National Human Genome Research Institute (NHGRI) Student Internship Program (SIP), a NIH-wide summer training program pairing high school, college, and graduate students with scientists from a variety of NIH Institutes and Centers.

Evaluators concluded that an evaluation was feasible. An evaluation plan was developed. It included specific study questions and measurable objectives to address goals for each evaluation component; key variables required to address objectives; appropriate data sources and target audiences/participants for the process and outcome components; evaluation tools/survey instruments; data collection methods; and statistical analysis plans.

Report Title: Feasibility Study To Conduct Process and Outcome Evaluations of the NHGRI Summer Internship Program

Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: Capital Consulting Corporation
Record ID: 9424 (Report issued March 31, 2011)

How Do Postdoctoral Fellows Rate the Quality of Mentoring Provided by the National Institutes of Health?

This study determined the level and quality of mentoring across 22 National Institutes of Health (NIH) intramural research programs. A web-based survey was sent to all eligible participants. Eligible participants included clinical fellows, Intramural Research Training Award/Cancer Research Training Award recipients, visiting fellows, and research and special volunteer fellows.

Female fellows reported a greater lack of mentor availability, evaluation and promotion of networking opportunities than their male counterparts. They were also less likely to report that their training and career goals were being met. Visiting fellows, who comprise the largest group of fellows at NIH, responded that many of their training goals were not being met. Evaluators made several recommendations: develop a workshop to train Principal Investigators in mentoring; conduct annual evaluations of trainees that assess trainee progress, strengths, and areas requiring improvement; hold roundtable discussions to determine how NIH can address the unique mentoring needs of fellows with an MD degree (the group that reported feeling least prepared and supported); conduct a follow-up survey to assess the impact of cultural diversity on the mentoring needs of visiting fellows; offer a training workshop that specifically addresses how fellows can make their needs known to their mentor; and establish a Hispanic support group, since 25% of Hispanic fellows reported having no mentor.

Report Title: The NIH Intramural Research Postdoctoral Fellows Mentoring Survey
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: Information Management Services, Inc.
Record ID: 9425 (Report issued June 30, 2011)

What Was Learned from the 2010 National Survey of Substance Abuse Treatment Services on Substance Abuse Treatment Facilities?

The National Survey of Substance Abuse Treatment Services, is an annual census of facilities providing substance abuse treatment. The survey collects data on the location, characteristics, and use of alcoholism and drug abuse treatment facilities and services throughout the United States. Over fourteen thousand facilities completed the survey.

The facilities had a one-day census of 1,175,462 clients enrolled in substance abuse treatment on March 31, 2010.

There were 81,863 clients under age 18 in treatment on that day, making up 7 percent of the total population in treatment. Forty-three percent of all clients were in treatment for both alcohol and drug abuse, thirty-nine percent were in treatment for drug abuse only, and eighteen percent were in treatment for abuse of alcohol alone. Forty-three percent of all clients were in treatment for co-occurring mental health and substance abuse disorders. Core structural characteristics of the substance abuse treatment system such as percentages of the type of entity responsible for the operation of the facility (government, private) and the type of care offered (outpatient, residential, hospital inpatient) have remained stable over the past five years.

Report Title: National Survey of Substance Abuse Treatment Services (N-SSATS): 2010, Data on Substance Abuse Treatment Facilities http://www.samhsa.gov/data/DASIS/2k10nssats/NSSATS2010Web.pdf
Agency Sponsor: SAMHSA, Substance Abuse and Mental Health Services Administration
Federal Contact: Cathie Alderks, 240-276-1269
Performer: Synectics for Management Decisions, Inc.
Record ID: 9440 (Report issued September 15, 2011)

Are there Significant Differences in the Leadership Skills of Male and Female Scientific Leaders?

Female scientists at the National Cancer Institute (NCI) have presented anecdotal evidence that their leadership development needs are different from those of male scientific leaders. There is a perception among female scientists that women do not progress in their careers as effectively as their male counterparts. This study explored potential differences between male and female scientific leaders at NIH. Twenty-two scientific leaders were invited to complete the Denison Leadership Development Survey. The specific competencies rated included: Empowering People, Building Teamwork, Developing Organizational Capability, Defining Core Values, Working to Reach Agreement, Managing Coordination and Integration, Creating Change, Promoting Organizational Learning, Emphasizing Customer Focus, Creating Shared Vision, Defining Strategic Direction and Intent, and Defining Goals and Objectives.

Small differences were found for some rater groups; however, the differences were only significant for Direct Reports and Combined Others ratings. No significant differences were observed between male and female leaders in the Self, Peer, Supervisor or Others ratings. Direct Reports of male leaders rated their leaders more favorably than those of females leaders and Combined Others ratings mirrored some of the same patterns. These results suggested that there may be some differences in how male and female leader are perceived at NIH, especially by their Direct Reports.

Report Title: The Denison Leadership Development Survey: Exploring Gender Differences at the National Institutes of Health
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Rosanna Ng, 301-496-5367
Performer: Doyen Consulting Services, Inc.
Record ID: 9490 (Report issued April 21, 2011)

Files

PerformanceImprovement2011-2012.pdf (pdf, 701.44 KB)

Topics

Program Evaluation | Modeling & Simulation | Key Indicators