Performance Improvement 2009

U.S. Department of Health and Human Services
200 Independence Ave., S.W.
Washington, D.C. 20201

Program evaluation:

to study and assess projects, policies and programs and determine if they work. What is a program supposed to do? Is it having unintended or unforeseen consequences? Is it causing what is being measured or are other events or processes causing the outcome, or preventing the hoped-for outcome?

Introduction

This report fulfills a statutory obligation of the Department of Health and Human Services. Section 241 of the Public Health Service Act directs the Secretary to inform Congress annually concerning findings from evaluations conducted by the Department. As required, the report is submitted to the Senate Health, Education, Labor and Pensions Committee and the House Energy and Commerce Committee.

Performance Improvement 2009 is the fifteenth annual report in this series. These reports, and the database from which they are drawn, can be found through http://aspe.hhs.gov/pic/performance. This report cites the most recent evaluations completed during the year ended September 30, 2008.

Summarizing 173 evaluations, the report provides an easy way for Congress and the public to find the latest information about the functioning and impact of departmental programs.

This report was prepared by staff of the Office of the Assistant Secretary for Planning and Evaluation based on study summaries submitted by project officers and departmental staff.

Organization of the Report

Chapter I describes some of the ways program performance is assessed, measured, monitored and evaluated in the Department of Health and Human Services.

Chapter II provides full abstracts for recently completed studies, categorized under the Department’s four Goals and their corresponding Objectives (found at Appendix B). Each entry comprises:

• The Key Question the study sought to address;
• A Summary describing the research and highlighting important findings; and
• Tracking Information, including the sponsoring agency; Federal contact (for questions or copies of reports not available online), and the name of the organization or contractor which performed the work.

Appendices provide a variety of useful supplemental reference information such as a list all the federal domestic assistance programs administered by the Department (Appendix A) and explanations of common acronyms and a glossary of terms in the report (Appendix D).

Chapter I - Assessing Program Effectiveness

Managing a Big Department

The Department of Health and Human Services (HHS) administers 337 programs (listed in Appendix A; also, see descriptions of all Federal programs in the Catalog of Federal Domestic Assistance, www.cfda.gov). The HHS budget included $721 billion for fiscal year (FY) 2008, the period covered by this report. Of this amount, Congress directed more than $800 million for evaluation and related activities through the set-aside provision of the Public Health Service Act (Section 241) which allows the Secretary of HHS to use a portion of the amounts appropriated for programs authorized under the Act for the evaluation, directly, or by grants and contracts, of the implementation and effectiveness of these programs[1]. Additionally, Congress annually appropriates significant other funds for both directed and discretionary evaluations. Since these make up part of the essential evaluation activities of the Department, they too are included in this report.

Why We Evaluate

Successful programs rely on effective evaluation. Thoughtful program evaluation can improve the delivery of public services and ensure that programs are efficient, targeted to their intended clients, and well managed. Important questions addressed by program evaluations include: what are the circumstances in which programs exist, who are the people that need services, and which program activities work best? The initial needs that give rise to programs, the knowledge base on which they are constructed, and the environment in which they operate, are not static. Actions by many individuals and demographic, economic, political, and social conditions are in flux. Legislators, policy makers, and managers must stay abreast of change. Congress and the Executive branch are responsible for the design, modification and implementation of laws and rules governing programs and both branches of government need the results of ongoing research, evaluation, and policy studies to effectively monitor and appropriately modify programs.

Funding Evaluations

Under the Public Health Service Act provision, about 2.4 percent of funds annually appropriated under the Act are used both to conduct evaluations and to fund other activities identified by Congress. The Public Health Service Act requires that the Secretary report annually, to the Senate Health, Labor and Pensions Committee and the House Energy and Commerce Committee, the findings of evaluations conducted under these provisions. As a courtesy to Congress, though not required by the Public Health Service Act, evaluations funded pursuant to other yearly administrative appropriations and statutory administrative authority are also included. This report, the 15^th in this series, provides summaries of recently completed studies funded with both set-aside as well as other program evaluation funds.

What Do We Evaluate?

Evaluation is an essential means of achieving outstanding program performance. Evaluations begin with questions for which we seek answers. Without questions, there would be no need to study how well programs perform, what they accomplish, and what the circumstances are in which they operate. Study results emerge as significant signposts along the path of program review and improvement efforts. Evaluation implies critical judgment and reaching conclusions about merit and value. This year’s collection of completed studies includes a wide range of significant findings that are potentially of broad interest.

Evaluation priorities respond to and are guided by Congressional oversight, Executive branch decision-making, program management needs and performance measurement systems, including Presidential guidance and the Government Performance and Reports Act of 1993 (GPRA). During the past few years, executive agencies have also incorporated challenges and measurement involved in the Performance Assessment Reporting Tool (PART) issued by the Office of Management and Budget. Evaluation activities must also respond to changing realities in the programs themselves and in the environments in which the programs operate as well as in response to advice and recommendations derived from earlier evaluations.

Collaboration and Participation

To continue to refine and strengthen programs administered by HHS and the way they are evaluated reader feedback is welcomed. For this purpose, a one-page evaluation form is included at the end of the report. Please submit comments or recommendations, by either mail or email.

As indicated above also, for interested readers of this report, you may also see the entire database of over 8,000 abstracts and summaries, many with links to the original reports, as well as obtain access to the reports in this Performance Improvement series online: http://aspe.hhs.gov/pic/performance.

Performance Measurement Improvement Context

There are many systems of accountability to help assure effective and efficient functioning of government programs. Some are internal the HHS, others operate across all Executive Branch agencies. Congressional oversight provides performance review, and the public is the final judge of performance. Internally within HHS, the work of each program and agency (see Appendix C) is subject to review and approval by the Office of the Secretary, reflecting the advice gathered from several Assistant Secretaries.

As with most Departments and many individual agencies, HHS has a large Office of the Inspector General (OIG) which annually sets an ambitious study agenda, not only to audit finances of certain programs, but to examine how well programs are meeting their statutory and social objectives. Every OIG, including ours, functions under a dual reporting authority. While the OIG’s budget, staffing, and organizational functioning is subject to oversight by HHS, the OIG is also authorized and required to provide the results of its audit, program evaluation, and inspection to the Congress, uncensored by HHS policy and political leadership, thus assuring that its voice is heard on matters of public urgency.

Across the entire Executive Branch, during the annual budget development and review process, run by the Office of Management and Budget (OMB) on behalf of the President, the success and failure of programs of government is reviewed and correspondingly appropriate future resource levels judged.

Outside the executive branch, the most fundamental and far reaching control and performance assessment results from the work of the Congress through its oversight committee function. The Congress makes the laws, and it can likewise change them, and does, as a result of reassessments regarding the adequacy and performance of programs. Various staff offices support the work of Congress, including, notably, the Government Accountability Office (GAO). The GAO, used to focus on financial audit and accounting. It now engages in a wide array of performance studies at both a micro (individual program) level and macro (broad or cross-cutting governmental function) levels.

Most importantly, informed, engaged public groups and individuals regularly influence the operation and direction of programs, most frequently through the direct actions of elected and appointed officials. Civil servants, providing institutional memory and continuity, act to assist in designing and implementing the wishes of the public and elected and appointed leaders. Non-governmental organizations (NGOs) and active media can and frequently do provide a form of oversight, assessment, and voice for improving both the underlying law and the functioning of programs. These groups help to educate the public about what actions are being taken on their behalf, thereby better enabling an informed citizenry to more fully participate in its government.

How Evaluations Originate

Some evaluations are required by statute, others are considered essential by the President, and the Department, or an individual agency. Evaluation completes other core Federal management responsibilities: strategic planning, policy and budget development, and program operations (Figure 1).

D

In FY 2008, the HHS budget included $711 billion for HHS programs. Of this amount, Congress directed more than $800 million for evaluation and related activities through the Public Health Service Act Set Aside provision [Section 241(a) of the Act]. Successful evaluation increases the likelihood of effective delivery of public services through these programs and insures that programs are efficient, targeted to their intended clients, and well managed. Additional funds, through general and directed authorities, are also available for research, demonstrations, and evaluations by agencies of HHS.

Role of Evaluation

Programs need to provide good results for the individuals served, use resources wisely, and achieve the goals intended by Congress and the President. This obligatory report to Congress on Performance Improvement continues the effort to provide a strategic and analytic presentation of evaluations. Many provisions of public laws and executive orders include the need for evaluations and systematic review of programs and goals across the department.

This report reflects the efforts of departmental staff to measure, test, and evaluate the effect of programs, and to provide information that enables managers and policy makers to address changes that may be needed in existing programs or in revising policies, regulations and statutory provisions.

HHS evaluations directly support several efforts. Evaluations help government officials and members of the Congress make decisions related to programs, policies, budgets, and strategic planning. Evaluations enable managers to improve their program operations and performance. In addition, these results and methodologies are useful to the larger health and human services community of state and local officials, researchers, advocates, and practitioners to improve the performance of their programs.

Types of Evaluation

Classic Evaluation

Traditional program evaluation categories include process/implementation, experimental impact, non-experimental (or quasi-experimental), cost-benefit analysis, and other outcome studies.

• A cost-benefit analysis, examining the advantages and costs of one or more program designs, could be carried out before a program has been implemented.
• During the first several months of a program’s existence, before there are discernable outcomes to measure, a process or implementation evaluation could be carried out to see if the program is being set up as intended.
• Fully experimental evaluations, or random-assignment studies, are considered the gold-standard of evaluation because they include both program and control groups so the results of the program can be compared to a group intended to be identical in every way except for the role of the program being tested.
• Finally, non-experimental or quasi-experimental studies seek to find natural circumstances that mimic to some extent what is created artificially by fully experimental studies so that comparisons can be drawn.

Performance measurement differs somewhat from and can fully complement evaluations. While performance measurement may use some of the same types of evaluative tools, the goal is more directed. While an evaluation will typically test a hypothesis, performance measurement must start with the goal of measuring observed performance against particular expectations or criteria for success.

Type by Use

A second way of thinking about types of evaluations is to examine how the information is intended to be used. HHS evaluations assess performance (efficiency, effectiveness, and responsiveness) of programs or strategies, the use of information in strategic planning, programmatic or policy oriented decision-making, and program improvement. Evaluations serve one or more of these objectives (Figure 2):

• Improve Performance Measurement  — Monitor annual progress in achieving departmental strategic and performance goals. HHS invests in evaluation funds to develop and improve performance measurement systems and the quality of the data that support those systems. The emphasis during development, implementation, and refinement of programs is on results and specific measurements as required under the Government Performance and Results Act.
• Strengthen Program Management and Development  — Address the need of program managers to obtain information or data that will help them effectively design and manage programs more efficiently and ensure successful results. Focus on developmental or operational aspects of program activities and provide understanding of services delivered and populations served.
• Assess Environmental Factors  — Seek to understand the forces of change in the health and human services environment that influence the success of our programs. Such understanding allows us to adjust our strategies and continue to deliver effective health and human services.
• Enhance Program Effectiveness and Support Policy Analysis  — Determine the impact of HHS programs on achieving intended goals and objectives and examine the impact of alternative policies on the future direction of HHS programs or services.

D

Evaluation Resources

Evaluation activities of HHS agencies and offices are supported with both general program funding and with a portion of the funds appropriated under the Public Health Service Act “ set-aside” authority.

General Program Funding

Program managers, operating under either discretionary or directed authority may use program funds to support contracts to design and carry out evaluations and analyze evaluation data. In some cases, a program’s statutory authority calls for specially mandated evaluations, and program funds are used directly to support these studies. Agencies for which one or both examples of such funding apply include the Administration for Children and Families (ACF) and the Centers for Medicare & Medicaid Services (CMS).[2] Such funds for evaluation are also available for the Administration on Aging.[3]
 

Public Health Service Act Set-Aside Authority

The Public Health Service Act, Section 241 set-aside authority was originally established in 1970, when the Congress amended the Act to permit the HHS Secretary to use up to 1 percent of appropriated funds to evaluate authorized programs. Section 241 limited the base from which funds could be reserved for evaluations to programs authorized by the PHS Act. Excluded were funds appropriated for the Food and Drug Administration, the Indian Health Service, and certain other programs that were managed by PHS agencies but not authorized by the Act (e.g., HRSA’s

Maternal and Child Health Block Grant and CDC’s National Institute for Occupational Safety and Health).[4]

The Consolidated Appropriations Act, 2008, authorized the Secretary to use up to 2.4 percent of the amounts appropriated for programs authorized by the Public Health Service Act for the evaluation of these programs. For Fiscal Year 2008, the year reflected in the studies here reported, agencies were budgeted a total of $913 million from the set-aside authority:

Administration for Children and Families (ACF) -- $11 million

Agency for Healthcare Research and Quality (AHRQ) -- $335 million

Centers for Disease Control and Prevention (CDC) -- $327 million

Health Resources and Services Administration (HRSA) -- $28 million

National Institutes of Health (NIH) -- $22 million

Substance Abuse and Mental Health Services Administration (SAMHSA) -- $123 million

Three staff components in the Office of the Secretary received a total of $47 million, shared between the Office of the Assistant Secretary for Planning and Evaluation (ASPE), the Office of Public Health and Science (OPHS), and the Office of the Assistant Secretary for Financial Resources (ASFR). In addition, the Office of the National Coordinator for Health Information Technology (ONC) received $19 million and the Office of the Assistant Secretary for Preparedness and Response (ASPR) received $3 million.

Substantial portions of the above funds are congressionally directed to pay for both general operating expenses and broad research activities.[5]

Most evaluations are started in one budget year, carried out in one or more subsequent years, and final reports, marking the completion of each study, may be delivered and available for the public in a third or subsequent year. Therefore, not all funds for studies completed in a particular year equate to the funds appropriated for that year.

Evaluation Management

Management of evaluations carried out by HHS agencies and offices involves: (1) planning and coordination, (2) project oversight, (3) quality assurance, and (4) dissemination of results (Figure 3). A description of each function follows.

D

Evaluation Planning and Coordination

The Government Performance and Results Act of 1993 (GPRA) requires that the Department establish a new five-year strategic plan every three years. The most recent was prepared last year for 2007-2012. This statute forms an essential basis for evaluation planning. HHS agencies and staff offices develop evaluation plans annually in concert with HHS program planning, legislative development, and budgeting cycles. Each agency or office evaluation plan generally states the evaluation priorities or projects under consideration for implementation. Typically, HHS evaluation priorities include: congressionally-mandated program evaluations, evaluations of Secretarial program or policy initiatives, assessments of new programs and ones that are candidates for reauthorization, and evaluations that support program performance management and accountability.

HHS evaluation planning activities are coordinated with three department-wide planning initiatives. First, HHS evaluation activities support the Department’s strategic planning and performance management activities in several ways. Completed evaluations are used in shaping specific HHS strategic goals and objectives. Evaluation findings provide important sources of information and evidence about the success of various HHS programs or policies. The HHS Strategic Plan highlights evaluations that document efficacy or effectiveness of strategic programs or policies and lists future evaluations that will benefit strategic planning. HHS agencies use findings from their evaluations to support GPRA annual performance reporting to Congress and program budget justifications.

Then, as specified in statute, the Secretary reports to the Congress the plans for using PHS evaluation set-aside funds before implementing these plans.

Project Oversight

HHS agencies and staff offices execute annual evaluation plans that involve developing evaluation contracts and disseminating and applying evaluation results. Where their subject matters relate, agencies seek to coordinate with one another.

The OIG performs independent evaluations through its Office of Evaluation and Inspections (OEI). OEI’s mission is to improve HHS programs by conducting inspections that provide timely, useful, and reliable information and advice to decision makers. Findings of deficiencies or vulnerabilities and recommendations for corrective action are usually disseminated through inspection reports issued by the Inspector General.

Quality Assurance and Improvement

Most evaluation projects are developed at the program or office level. A committee of agency- or office-level policy and planning staff members may conduct an initial quality review. Before a project is approved, a second committee reviews it for technical quality with expertise in evaluation methodology. Technical review committees generally follow a set of criteria for quality evaluation practice established by each agency. ASPE, for example, has a peer review committee that serves to improve the technical merits of ASPE proposals before final approval. Some HHS agencies have external evaluation review committees composed of evaluation experts from universities and research centers.

Since HHS began reporting to Congress in 1995 on completed evaluations through the Performance Improvement report series, the Department has focused attention on improving the quality of evaluations performed. In the past, Evaluation Review Panels, convened periodically, have contributed insights to HHS evaluation officers on the strengths and challenges of ensuring quality evaluations.

Dissemination of Evaluation Reports

Maintaining online electronic report libraries and distributing information on evaluation results is an important component of HHS evaluation management. The Department’s information and reports on major evaluations are available through the Web site of the HHS Policy Information Center (PIC), located at: http://aspe.hhs.gov/pic/performance (Appendix F contains additional information about how to access this information). ASPE’s PIC Web site offers users an opportunity to search  – by key word, selected program, or policy topics – the departmental evaluation report database and electronic report library maintained by ASPE. New entries in the online database focus on effective and clear summaries answering the basic questions: what was the study, why was it conducted, and what was learned. Through the online database, information regarding the work of evaluation within the department can be made known, speeding the dissemination of important factual information regarding work of the Department.

Additionally, the results of HHS evaluations are disseminated on agency and office websites through targeted distribution of printed reports, and research briefs as well as presentations at professional meetings and conferences. HHS researchers also participate in the broader research community through articles in specialist publications and refereed journals.

Chapter II - Summaries of Completed Evaluations and highlights of Findings

This chapter compiles summaries of 173 studies for which reports were issued during the year ending September 30, 2008. Each entry consists of a title posing the central question addressed in the study, a brief synopsis, and key findings. Each study is identified under one of the sixteen critical objectives comprising the Department’s Strategic Goals, shown in Appendix B. For studies carried out by a particular agency or office see Appendix E. All earlier studies and entries for ongoing studies can be found at http://aspe.hhs.gov/pic/performance. Appendix F provides additional guidance on how to obtain more information about these and other studies.

Some studies result in definitive results; others raise as many questions as they answer. Most are concerned with our need to better understand the public programs for which we are responsible. Wide availability of information strengthens agency and program operations, guides management, and drives policy advice. Every effort has been made to communicate information about this body of work as clearly, concisely, and effectively as possible. Most importantly, this report reflects our accountability to the public for the programs we administer.

During Fiscal Year 2008, evaluations examined important management, operational, policy, and factual circumstances faced by programs. These evaluations employed a wide range of methods, including literature reviews, focus groups, surveys, micro-simulation analyses, field visits, and case studies. Some were carried out by agency staff; most were completed with the assistance of contract support expertise; all reflect an intense, creative and collaborative effort.

Goal 1: Health Care: Improve the safety, quality, affordability and accessibility of health care, including behavioral health care and long-term care.

This Strategic Goal targets the need for people to be able to obtain and maintain affordable health care coverage, receive efficient high-quality health care services, and access appropriate information for informed choices.[7]

Objective 1.1: Broaden health insurance and long-term care coverage.

How Has Legislation Changed Medicare Advantage Plan Availability, Participation, Premiums, Benefits Cost Sharing, and Enrollment?

The study examined the impact of legislated changes in the Medicare Modernization Act.

Access to and enrollment in Medicare Advantage plans of all types increased during 2006. Medicare Advantage plans in rural areas increased. Likely causes of higher enrollment included greater availability of plans and plan types offering greater access to providers, the addition of the Part D, and more attractive premiums and benefits. Private-Fee-For-Service and Preferred Provider Organizations were instrumental in achieving universal access. Participation by beneficiaries in the Medicare Advantage program increased significantly between 2005 and 2006. Most beneficiaries received Part C (Medicare Advantage) and Part D (prescription drug) benefits at no additional out of pocket cost to them as a result of changes in plan premiums during 2006.

Report Title: Medicare Advantage Plan Availability, Premiums and Benefits, and Beneficiary Enrollment in 2006, Final Report http://www.cms.hhs.gov/Reports/Downloads/Pope-2007.pdf
Agency Sponsor: CMS-ORDI, Office of Research, Development, and Information
Federal Contact: Melissa Montgomery, 410-786-7596
Performer: Research Triangle Institute
PIC ID: 8959

Do Sponsors Willingly Participate in the Prescription Drug Payment Demonstration; with What Results?

The evaluation examined implementation-phase results from a five-year demonstration intended to allow prescription drug plan sponsors to choose drug reimbursement methodologies. This “ reinsurance” demonstration seeks to provide an incentive for private sector drug payment plans to offer supplemental prescription drug coverage to Medicare beneficiaries.

The study found broad support for the demonstration and almost universal agreement that the alternative reinsurance financing allowed for enhanced benefits or lowered premiums. Organizations set monthly premiums with great care and considered strategies for formulary design and covered drugs. No effects were found on implementation, marketing or education strategies used in the demonstration.

Report Title: Medicare Part D Payment Demonstration Site Visit Report http://www.cms.hhs.gov/DemoProjectsEvalRpts/downloads/PDP_Site_Visit_Report.pdf
Agency Sponsor: CMS-ORDI, Office of Research, Development, and Information
Federal Contact: Aman Bhandari, 410-786-2313
Performer: Research Triangle Institute
PIC ID: 8961

Why Do Medicare Beneficiaries Enroll in Prescription Drug Payment Demonstration Plans; Do Their Experiences Differ from Non-Demonstration Plan Enrollees?

The study focused on experiences among demonstration beneficiaries enrolled in Medicare Advantage-Prescription Drug plans or free-standing prescription drug plans versus non-demonstration beneficiaries in such plans. The five-year Medicare prescription drug (Part D) payment demonstration allows plans to choose alternative methods to pay for reinsurance coverage.

The study found important differences among the enrollees in demonstration versus non-demonstration plans. Enrollees in demonstration plans were more aware of having a range of choices among basic and enhanced benefit packages. Demonstration enrollees were more knowledgeable about Part D plan benefit details, appeared healthier, and reported lower drug consumption than the non-demonstration enrollees. Beneficiaries found the Part D program confusing, complained about drug prices in the coverage gap, but eventually got needed medications.

Report Title: Medicare Part D Payment Demonstration Site Focus Group Report http://www.cms.hhs.gov/DemoProjectsEvalRpts/downloads/PDP_Focus_Report.pdf
Agency Sponsor: CMS-ORDI, Office of Research, Development, and Information
Federal Contact: Aman Bhandari, 410-786-2313
Performer: Research Triangle Institute
PIC ID: 8962

Why and At What Rates Do Beneficiaries Quit Health Care Plans?

This study compared beneficiary voluntary disenrollment (withdrawal) from Medicare Preferred Provider Organization (PPO) demonstration plans with voluntary disenrollment from Medicare health maintenance organization (HMO) plans. The PPO demonstration seeks to provide beneficiaries more and better Medicare options.

Researchers found disenrollment rates were more than 3 percentage points higher among PPO demonstration plan participants than among HMO participants in the same markets. Disenrollees from PPOs were more likely than those from HMOs to cite issues with premiums or co-payment amounts, but, PPO disenrollees were less likely to cite problems with access to care or service.

Report Title: The Medicare Preferred Provider Organization (PPO): PPO Plan Disenrollment Rates in a Market Context; Report may be obtained from Federal Contact
Agency Sponsor: CMS-ORDI, Office of Research, Development, and Information
Federal Contact: Noemi Rudolph, 410-786-6662
Performer: Research Triangle Institute
PIC ID: 8965

How Did Medicare's Preferred Provider Organization Demonstration Affect Beneficiary Prescription Drug Plan Choice and Participation?

This study evaluated the Medicare Preferred Provider Organization (PPO) demonstration, which ended in 2006. The demonstration sought to expand the types of managed care products to Medicare beneficiaries and test the impact of enhanced payment and risk sharing on the range of options and benefits to beneficiaries.

The demonstration succeeded in expanding plan choices in a wide variety of geographic areas. Although many plans were offered, enrollment and market share under the demonstration was modest. PPO enrollees were similar to other Medicare Advantage plan enrollees. They tended to be healthier than the average beneficiary in the traditional Medicare program. The demonstration was a precursor to the local and regional PPOs established by the Medicare Modernization Act of 2003, and many demonstration plans transitioned to Local PPOs in 2006.

Report Title: Evaluation of the Medicare Preferred Provider Organization Demonstration; Report may be obtained from Federal Contact
Agency Sponsor: CMS-ORDI, Office of Research, Development, and Information
Federal Contact: Noemi Rudolph, 410-786-6662
Performer: Research Triangle Institute
PIC ID: 8966

Have Short Stays at Long Term Care Hospitals Declined?

This study provided information on long term care hospitals (LTCH) short-stay outliers. Short-stay outliers are LTCH stays ending before reaching five-sixths of the average length-of-stay for the patient’s diagnosis under the long term care diagnostic related grouping (LTC-DRG) category.

Short-stay outliers decreased from 40 percent of LTCH stays discharged in 2003 to 27 percent of stays discharged in 2006. Some patients may have been inappropriately placed in LTCHs or discharged based on financial incentives. From 2003 through 2006, LTCHs (1) discharged over a third of short-stay outlier patients at least 10 days before patients reached the short-stay outlier threshold, (2) greater proportions of patients were also discharged within 2 days of having qualified for full LTC-DRG payment, and (3) LTCHs discharged short-stay outlier patients to acute care facilities more frequently than other LTCH patients. For 2005 and 2006, short-stay outlier payment errors mirror those of other LTCH claims. Most payment errors Quality Improvement Organizations identified with LTCH claims were inaccurate LTC-DRGs and inappropriate LTCH admissions.

Report Title: Long Term Care Hospitals Short-Stay Outliers http://oig.hhs.gov/oei/reports/oei-01-07-00290.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 8978

Do Nursing Homes and Long-Term Care Pharmacies Assist Their Medicare-Medicaid Dual-Eligible Residents to Select Prescription Drug Payment Plans?

The study analyzed structured interviews with a sample of nursing home administrators and directors of operations for long term care pharmacies.

Some of the practices identified may not be in accordance with CMS guidance that nursing homes not request, require, coach, or steer residents to particular plans. Nursing homes and long term care pharmacies provide different types of assistance to dual-eligible residents who are selecting their Part D plans. Thirty-eight percent of nursing home administrators and thirty-three percent of pharmacy directors reported that their pharmacies identified multiple plans that met dual-eligible residents’ needs or provided a general list of plans that the pharmacy recommended to all residents. About nine percent of nursing home administrators and eight percent of pharmacy directors reported enrolling most dual-eligible residents in a single plan or recommended one plan to each resident. The remaining administrators and directors reported that they provided only general information about the benefit or no assistance.

Report Title: Role of Nursing Homes and Long Term Care Pharmacies in Assisting Dual-Eligible Residents With Selecting Part D Plans http://oig.hhs.gov/oei/reports/oei-02-06-00191.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 8981

How Do States Reduce Unpaid Child Support Obligations?

This study determined the prevalence, characteristics, and outcomes of programs used by State child support enforcement agencies to reduce child support arrearages (unpaid payments by non-custodial parents). In “ debt compromise,” a State agrees to accept reduction or elimination of child support debt owed to it by a non-custodial parent. Reviewers surveyed all States and conducted site visits to five, examining a sample of debt compromise agreements.

In States with debt compromise programs, cases were considered eligible based on a number of factors, including the amount of the arrearages, and willingness of local to negotiate agreements. Agency officials in States with programs reported a largely positive view of debt compromise, although a few expressed the concern that settling debt is contrary to the child support enforcement process. Agencies in 20 States operated fully implemented or pilot programs, 23 States settled arrearage debt on a case-by-case basis, and the remaining 8 States did not allow compromise of arrearages. Debt compromise resulted in an average of $9,383 settled per case. Non-custodial parents in 45 percent of sample cases made lump sum payments averaging $5,515 at the time of the agreements. Following debt compromise, 41 percent of cases closed, either after lump sum payments or with all debt settled. When cases remained open, four of five States did not routinely follow up when non-custodial parents paid irregularly.

Report Title: State Use of Debt Compromise To Reduce Child Support Arrearages http://oig.hhs.gov/oei/reports/oei-06-06-00070.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 8996

How Have Medicare Savings Account Plans Grown?

The study examined early patterns of enrollment and early stage development of Medicare Savings Account (MSA) plans.

Organizations offering an MSA product had positive experiences. Plans reported several challenges but were generally able to work around issues. Organizations offering an MSA plan believed doing so would make their companies full service organizations and 'one-stop shops' for Medicare Advantage products. Health plans that didn’t offer a savings account indicated that their potential customers were not prepared for consumer directed health plans or the area was couldn’t support multiple health plan options. From the member perspective, the most common issue included the timeframe when the member would receive the deposit to their account and be able to use their debit card. Two plans addressed this issue by funding the member account before receiving the deposit from the Centers for Medicare & Medicaid Services. From the plan's perspective, the most significant issues were developing MSA-specific marketing materials, setting up the accounts, recovering funds from disenrollees, and prorating the plan deductible.

Report Title: Evaluation of MSA Plans Under the Medicare Program Case Study Report; Report may be obtained from Federal Contact
Agency Sponsor: CMS-ORDI, Office of Research, Development, and Information
Federal Contact: Melissa Montgomery, 410-786-7596
Performer: L&M Policy Research, Inc
PIC ID: 9011

Do Participants in the Prescription Drug Payment Demonstration Provide More Generous and Varied Benefit Packages?

This study analyzed the impact of the prescription drug payment (Part D) demonstration on beneficiaries, sponsors and Medicare program costs. The 5-year demonstration extends from 2006 through 2011 and allows Part D plans to choose from among three main alternative payment methods for re-insurance: fixed capitation, flexible capitation, and Medicare Advantage rebate. Many enhanced benefit plans exist outside the reinsurance demonstration. Even without the reinsurance demonstration, there would be variety in enhanced benefit plans, including plans that provide gap coverage.

Demonstration plans were consistently more generous than basic plans and enhanced non-demonstration plans.

Report Title: Part D Reinsurance Demonstration Plan Benefit Design Analysis; Report may be obtained from Federal Contact
Agency Sponsor: CMS-ORDI, Office of Research, Development, and Information
Federal Contact: Aman Bhandari, 410-786-2313
Performer: Research Triangle Institute
PIC ID: 9012

Objective 1.2: Increase health care service availability and accessibility.

Does California Provide Comparable Coverage for Mental Health?

This study addressed various questions: what are issues/problems in legislation implementing parity (equivalence between mental health benefits and general health care benefits in health insurance plans); how have costs and use changed as a result of parity; and what are consumer, employer, insurer, and provider opinions about the effects of the law? Federal and state legislation require benefit parity. The scope and application of these legislative efforts are often limited. California implemented parity legislation in 2000 that provides for equal coverage for severe mental illnesses and covers children with one or more mental disorders. Unlike the parity legislation enacted in many other states, small businesses are not exempt. The size and complexity of California's economy and health care market, make its parity mandate especially important to understand.

Health plans reported that outpatient Mental Health utilization increased following passage of the law requiring parity. Cost increases were reported to be nominal due to the use of managed care. Stakeholders did not feel that parity relieved the financial burden on the public mental health system.

Report Title: Assessment of California’s Mental Health Parity Law: A Step Toward Broader Mental Health System Reform; Report may be obtained from Federal Contact

Agency Sponsor: SAMHSA, Substance Abuse and Mental Health Services Administration
Federal Contact: Jeffrey Buck, 240-276-1959
Performer: Mathematica Policy Research, Inc.
PIC ID: 7727

How Strengthen Multi-State Electronic Health Information Exchange?

Researchers established an advisory board comprised of Governors and other executive-level officials to identify ways to resolve issues deterring States from achieving interoperable electronic health information exchange.

The National Governors Association established and managed the State Alliance for e-Health, a consensus-based body of State elected and appointed officials to address State-level health information technology (IT) and health information exchange (HIE) issues and challenges. The State Alliance’s mandate included: identifying, assessing and mapping ways to resolve State-level health IT issues that affect multiples States and impede interoperable HIE; harmonizing or standardizing State laws, regulations, and policies affecting health information exchange, while preserving or enhancing privacy, security, and consumer protections; providing a forum that enables dialogue and partnerships among States to increase the efficiency and effectiveness of their health IT initiatives and facilitates input from experts and others working on health IT and HIE to inform State policymaking.

Report Title: Accelerating Progress: Using Health Information Technology and Electronic Health Information Exchange to Improve Care, First Annual Report and Recommendations from the State Alliance for e-Health. http://www.nga.org/Files/pdf/0809EHEALTHREPORT.pdf
Agency Sponsor: OS-ONC, Office of the National Coordinator for Health Information Technology
Federal Contact: Kathleen Fyffe, 202-205-0670
Performer: National Governors' Association
PIC ID: 8598

What Home- and Community-Based Waivers Have States Designed for People With Intellectual and Developmental Disabilities?

The study explored how waivers emerged to address the needs of people with intellectual and developmental disabilities (I&DD), compile information about comprehensive and supports waivers, and learn about participant characteristics and experiences.

The rapid growth in the number of states operating waivers is noteworthy and will continue. Nearly every state grapples with a large waiting list for services and the prospect that demand for services will continue to climb. States are searching for lower-cost alternatives. Operating supports waivers offer a way to channel demand away from costly residential services. Because supports waivers offer a lower-cost alternative to comprehensive waivers, state policy makers may be more amenable to periodic increases in funding for services for people with I&DD. To make most effective use of supports waivers, states will need to ensure that proper resources are in place to support them, operational difficulties are resolved, and an appropriate role for these waivers is found within the larger state system.

Report Title: Gauging the Use of HCBS Supports Waivers for People with Intellectual and Developmental Disabilities: Final Project Report http://aspe.hhs.gov/daltcp/reports/2007/gaugingfr.htm
Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Ruth Katz, 202-690-6443
Performer: Human Services Research Institute, RTI International
PIC ID: 8893

What Characterizes the Home Health Aide Partnering Collaborative?

Researchers examined the home health aide (HHA) Partnering Collaborative model to identify implementation challenges and successes, and results and implications.

The major impact of the Collaborative was on patients' functional outcomes. Functional improvements were not associated with a significant reduction in length of stay, nor were they associated with greater likelihood of discharge to the community. During the randomized trial, patients in the intervention group had significantly better case-mix adjusted outcomes on two of three targeted areas of daily living — transferring and ambulation — than patients in the control group. Increased focus on communication and common goal-setting yielded better outcomes without the need for more visits.

Report Title: HHA Partnering Collaborative Evaluation: Practice/Research Brief http://aspe.hhs.gov/daltcp/reports/2007/HHAPartrb.htm
Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Marie Squillace, 202-690-6250
Performer: Visiting Nurse Service of New York
PIC ID: 8898

What Medicaid Services Do Beneficiaries Use, At What Cost, and How Does This Vary Across States?

This study examined how successfully States have rebalanced their long-term care systems and how Medicaid enrollees who utilize community-based long-term care services differ from people in institutions.

The study found that only 34% of Medicaid long-term care expenditures paid for persons served were for community-based services in 2002, while almost 59% of long-term care users used community-based services. National estimates mask significant variation across states. While the use-based measure was larger than the expenditure measure in every state, there was significant variation across states in how the two measures compared. Institutional and community long-term care expenditures were much more balanced among young disabled Medicaid enrollees than their aged counterparts in 2002. Over half of long-term care expenditures were for community-based services among disabled enrollees but less than 20% were for community-based care among those over 65. The primary distinguishing factor between people using community-based and those using institutional long-term care was age. Compared with people using community-based services, a higher percentage of people in institutions were non-Hispanic White, female, dual Medicare and Medicaid enrollees, and enrolled in Medicaid for only part of the year--all factors associated with age.

Report Title: A Profile of Medicaid Institutional and Community-Based Long-Term Care Service Use and Expenditures Among the Aged and Disabled Using MAX 2002: Final Report http://aspe.hhs.gov/daltcp/reports/2008/profileMAX.htm
Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: John Drabek, 202-690-6443
Performer: Mathematica Policy Research, Inc.
PIC ID: 8900

How Many Medicaid Nursing Home Users Enroll in Medicaid Before Their Nursing Home Stays Begin; How Does This Pattern Vary Across States?

This study examined nursing home stays, and how they vary across population groups and states. Although Medicaid expenditures for nursing home care are well-documented, little is known about the characteristics of people who use nursing home services. All Medicaid enrollees who reside in nursing homes must meet Medicaid income and asset requirements but pathways to Medicaid eligibility can vary greatly.

The study found that almost 46% of all nursing home users had new spells of nursing home care paid in part by Medicaid during the 18-month period of observation. There is considerable transition in and out of Medicaid nursing home care--due to death, extended hospital stays, Medicare-covered acute care stays, limited need, or availability of community-based services. The primary eligibility pathways for Medicaid-covered nursing home care nationwide were through the long-term care associated eligibility criteria, which include people qualifying under the 300% rule. Almost 48% qualified under this long-term care associated eligibility group, 23% qualified as a result of Supplemental Security Income receipt, 22% qualified under medically needy provisions, and 6% qualified under their state’s poverty criteria. About 64% of enrollees with new nursing home spells were already enrolled in Medicaid at the start of their spell whereas 36% were new enrollees. Almost half of new enrollees were already in a nursing home prior to their spell start, most likely because they entered as Medicare skilled nursing facility residents or spent down their savings while institutionalized to become Medicaid eligible. Over half of enrollees beginning Medicaid-financed nursing home spells were already residing in nursing homes when Medicaid began financing part of their stay. The duration of spells was bimodal, indicating that two distinct types of people utilized Medicaid-covered nursing home care: those needing care for acute conditions and those requiring longer term care. Duration of nursing home spells was negatively associated with availability of community-based services in a state. States with significant community-based programs tended to have a higher percentage of people using community-based services before entering nursing homes.

Report Title: Medicaid-Financed Nursing Home Services: Characteristics of People Served and Their Patterns of Care, 2001-2002 http://aspe.hhs.gov/daltcp/reports/2008/mfNHserv.htm
Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: John Drabek, 202-690-6443
Performer: Mathematica Policy Research, Inc.
PIC ID: 8901

What Regulatory Provisions and Medicaid Policy for Residential Care Settings Apply Across the United States ?

Researchers gathered information describing regulatory provisions and Medicaid policy for residential care settings and assisted living in all 50 states and the District of Columbia current through 2007. Researchers identified current issues in assisted living and residential care, including policy developments, growth trends, changing regulatory models, approaches to quality, and financing and reimbursement.

Compared to 2004, the supply of licensed facilities in 2007 rose 6 percent and the number of units rose 4 percent. About half the states reported an increase in the number of licensed facilities and half reported a decline. Twenty-one states revised their regulations between 2004 and 2007, and 12 reported work on revising regulations. The number of Medicaid beneficiaries served in residential care settings declined modestly from 121,000 in 2004 to about 115,000 in 2007; mostly occurring in Michigan and Florida.

Report Title: Residential Care and Assisted Living Compendium: 2007 http://aspe.hhs.gov/daltcp/reports/2007/07alcom.htm
Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Gavin Kennedy, 202-690-6443
Performer: Research Triangle Institute
PIC ID: 8902

Why Do Individuals Purchase Private Long-Term Care Insurance, Would Additional Tax Incentives Increase Such Purchases and Reduce Medicaid Expenditures?

This study examined the factors influencing the decision to purchase private long-term care insurance coverage using a major national survey--the Health and Retirement Study. Like traditional medical insurance, private long-term care insurance is a financial contract whereby the insurer agrees to provide covered benefits in exchange for regular premium payments by the policyholder. The cost and adequacy of policies vary by the types of services they cover, when they start paying benefits, how much they pay, and for how long. Insurance companies generally price policies as a function of age at issue date, health status, and the comprehensiveness of the plan. Policies are guaranteed renewable, and premiums remain fixed over the life of the contract. However, rates can rise for an entire class of policyholders if insurers can demonstrate that their costs exceed premium revenue, and rate increases have been common in recent years.

The study showed that the decision to purchase private long-term care insurance responds to the expected benefit of coverage. People become significantly more likely to take-up coverage as the net expected benefit increases, the likelihood of using services rises, the cost of services without insurance rises, or the chances of qualifying for Medicaid falls. However, the effects are modest. The net expected benefit of coverage significantly increased individual coverage rates — every $1,000 increase in the net expected benefit of coverage would raise purchase probabilities by about 2.4%. Health, economic, social, and demographic characteristics of older adults significantly influenced the likelihood that people purchased private long-term care insurance. Take-up rates increased with age, and college graduates were much more likely to purchase insurance than those who never attended college. People in good health were significantly more likely to obtain coverage than those in worse health, and African Americans and Hispanics were less likely to purchase than other racial groups. Take-up declined with the number of children. Take-up rates increased significantly with the self-reported probability of using nursing home care in the next five years. Take-up rates did not vary significantly with household income or assets. Liberalizing rules for deducting long-term care insurance premiums from taxable income could modestly increase take-up rates. Granting a full tax deduction to all policyholders, even those who do not itemize their deductions, would boost older adults who take-up coverage by about 36%. Tax incentives would boost long-term care insurance take-up rates for high-income taxpayers, but would have little impact for lower-income groups. Tax incentives that increase private insurance coverage would have little impact on Medicaid costs. Policy reforms allowing all policyholders to deduct all of their long-term care insurance premium payments from taxable income would have negligible effects on Medicaid costs. Only 3% of those who would take-up coverage under this policy reform, but not under current law, would eventually qualify for Medicaid.

Report Title: Modeling the Decision to Purchase Private Long-Term Care Insurance http://aspe.hhs.gov/daltcp/reports/2007/LTCImod.htm
Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: John Drabek, 202-690-6443
Performer: Urban Institute
PIC ID: 8903

What Concerns, Perspectives and Values Do People With Disabilities Have About Advance Care Planning?

This study sought to learn the best way to promote the use of advance directives among adults as a means of specifying their wishes about end-of-life care. A part of the study included topic commissioned papers. This review examined the current status of advance directives and advance care planning in the disability community. The author reviewed professional literature, consumer materials and internet sites pertaining to advance care planning and end-of-life care for people with physical and intellectual disabilities.

The author found that the concerns, perspectives, and values of people with disabilities have often been overlooked in the research, programs, and policies regarding advance directives, advance care planning, and end-of-life care more generally. While the process for advance care planning for people with physical and intellectual disabilities is the same as for non-disabled people, there are unique community perspectives and issues to be addressed in policies seeking to promote such plans.

Report Title: Advance Directives and Advance Care Planning for People with Intellectual and Physical Disabilities http://aspe.hhs.gov/daltcp/reports/2007/adacp.htm
Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Judith Peres, 202-690-6443
Performer: Yeshiva University
PIC ID: 8905

How Does People's Health Insurance Status Change Over Time; What Types of Coverage Do They Have?

This study analyzed data from a major national survey to better understand who had health insurance coverage in recent years, and how their coverage changed. The study examined the impact of risk on ‘point-in-time’ coverage and changes in coverage.

Although the individual market was less stable than the group market, retention rates appeared high for the self-employed — those best-suited for the individual market. High-risk people appeared to be significantly more likely to be covered than healthy people in the group market relative to the individual market. High-risk people initially-uninsured were more likely to obtain coverage in the individual market and the large-group market, and high-risk people who were initially-insured in the large-group market seemed to be significantly more likely to lose their coverage. High-risk individuals tended to obtain more generous coverage than their healthy counterparts. The larger the firm one was employed with, the more generous coverage one obtained. No evidence was found that being high-risk was associated with a higher premium or that the onset of a chronic condition was associated with an increase in the premium from one year to the next. Guaranteed renewability was successful in providing protection against ‘reclassification risk’ in individual insurance markets but low sample size prohibits a conclusive inference.

Report Title: Changes in Coverage in the Individual and Group Health Insurance Markets and the Effect of Health Status http://aspe.hhs.gov/daltcp/reports/2008/HIcover.htm
Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: John Drabek, 202-690-6443
Performer: Johns Hopkins University
PIC ID: 8907

How Well Implemented Is a New Indian Health Service Health Facility; Has It Affected Patient Health, Service Delivery, and the Community?

This study evaluated the implementation and impact of an Indian Health Service (IHS) health care facility, the White Earth Health Center. This was the first effort at long-term follow-up.

The White Earth Center has had an impact on many areas, including a significant increase in patient workload, more services available, improved health outcomes, increased staff, additions to the physical structure, increased revenues and increased investment and infrastructure. Researchers recommended addressing various administrative needs at new health centers, the planning process for new facilities, and guidelines for future long term evaluation studies.

Report Title: Ten Year Follow-Up Evaluation: White Earth Health Center; Report may be obtained from Federal Contact
Agency Sponsor: IHS, Indian Health Service
Federal Contact: Lucie Vogel, 301-443-1133
Performer: Staff; Indian Health Service
PIC ID: 8920

Why Do Few Beneficiaries Enroll in the Home Health Independence Demonstration; With What Experiences; How Do Home Health Agencies View the Homebound Restriction?

The study explored the reasons for unexpectedly low participation in the demonstration. The demonstration was intended to test the effects of deeming severely disabled beneficiaries as homebound to meet requirements for eligibility to receive Medicare home health services. The study relied on interviews with a broad range of sources, a survey of home health agencies in the demonstration states, and an analysis of Medicare claims and routinely collected assessment data.

Researchers identified a number of barriers that apparently hindered a full test of the demonstration concept. A majority of home health agencies declined to participate and some beneficiaries who were offered enrollment declined to participate. Extensive eligibility criteria of the enabling law could have targeted people too severely disabled to take advantage of the waiver. Clarifications to the homebound eligibility criteria of the Medicare, Medicaid, and SCHIP Benefits Improvement and Protection Act of 2000 likely reduced the restrictiveness of the homebound eligibility criteria.

Report Title: Evaluation of the Home Health Independence Demonstration: Barriers to a Successful Experiment Were Multifaceted, and Difficult Policy Issues Remain http://www.cms.hhs.gov/Reports/downloads/homebound.pdf

Agency Sponsor: CMS-ORDI, Office of Research, Development, and Information
Federal Contact: Ann Meadow, 410-786-6602
Performer: Mathematica Policy Research, Inc.
PIC ID: 8952

Do Nursing Homes Make Medicare Prescription Drugs Available to Their Medicare-Medicaid Dual Eligible Residents?

This study provided an assessment of the availability of Medicare Part D prescription drugs to dual-eligible nursing home residents. It focused on issues that arose in the early stages of implementing the new benefit. The study was based on structured interviews with a sample of nursing home administrators, medical directors, and directors of operations for long term care pharmacies.

Dual-eligible nursing home residents receive all necessary drugs covered by Part D plans. However, nursing homes and long term care pharmacies sometimes pay for drugs not covered by Part D plans. Administrators and pharmacy directors explained that the drugs they most commonly paid for either were not on the residents’ plans’ formularies or required prior authorization. In addition, respondents expressed concerns that formularies, the prior authorization process, and copayments might pose problems for dual-eligible nursing home residents. Concerns existed that long term care pharmacies generally did not disclose to physicians the rebates that they receive from drug manufacturers. The agency will work with plans to ensure that formularies meet the needs of dual-eligible nursing home residents and continue to work with plans to improve the prior authorization process; ensure that copayments for dual-eligible nursing home residents are fully subsidized, as appropriate; and consider methods to encourage long term care pharmacies to disclose to physicians information about rebates that they receive from drug manufacturers.

Report Title: Availability of Medicare Part D Drugs to Dual-Eligible Nursing Home Residents http://oig.hhs.gov/oei/reports/oei-02-06-00190.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 8980

Does the Food and Drug Administration Give Timely Review and Approval of Generic Drugs?

This study examined timeliness of generic drug application review and approval. Federal law requires that the agency approve or disapprove original Abbreviated New Drug Applications (ANDA) within 180 days of receipt. Pharmaceutical companies must submit the applications to the agency’s Office of Generic Drugs and receive approval before marketing new generic drugs. Based on reviewers’ survey responses, the Office of Inspector General conducted in-depth analyses of review times for the sample and structured interviews with agency officials to determine factors affecting drug application review times throughout the review process.

FDA disapproved 96 percent of original drug applications under review in 2006 because they did not meet FDA review standards. Nearly half of Chemistry review times exceeded the 180 days required by Federal law. Moreover, many review times in other agency divisions exceeded 180 days. In addition, for a sample of ANDA reviews exceeding 180 days, most reviews did not begin before the 180-day periods expired. Reviewers found that the agency prioritization practices affected drug application review times. The agency has taken actions that address recommendations by providing guidance to assist industry in submitting more easily reviewed applications, developing a focused hiring program to increase staff and decrease review times, and prioritizing some applications based on potential market entry date.

Report Title: The Food and Drug Administration's Generic Drug Review Process http://oig.hhs.gov/oei/reports/oei-04-07-00280.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 8992

Do Prescription Drug Plans Make Generic Drugs Available to Medicare Beneficiaries?

This study determined the extent of generic drugs in the Medicare Part D program.

Reviewers found that Part D achieved a high level of generic drug use during the first two quarters of 2006. Under Part D, generic drugs were dispensed 88 percent of the time when generic substitutes were available. However, 37 percent of prescriptions were for drugs that had no generic substitutes, limiting opportunities to dispense generics. Fifty-six percent of all drugs dispensed were generic drugs.

Report Title: Generic Drug Utilization in the Medicare Part D Program http://oig.hhs.gov/oei/reports/oei-05-07-00130.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 8995

Do Internet Web Sites Sponsored by Medicare Part D Prescription Drug Plans Comply with Federal Regulations?

This study examined Medicare Part D prescription drug plan (PDP) sponsors’ Internet Web sites compliance with Federal regulations regarding content and accessibility. Web sites were examined of all 84 sponsors offering drug plans within the 50 States and the District of Columbia in 2007. Federal regulations require that all sponsors have Web sites that include content about receipt and use of Medicare benefits.

Thirty-three percent of the PDP sponsor Web sites did not contain all federally required content. The most commonly omitted information pertained to enrollee disenrollment rights and responsibilities, the potential for PDP contract termination, and formulary information. Reviewers found that 85 percent of sponsor Web sites did not meet at least one of the Federal requirements for Web site accessibility. These problems could affect access to content by Medicare beneficiaries.

Report Title: Medicare Part D Prescription Drug Plan Sponsor Internet Web Sites" Content and Accessibility http://oig.hhs.gov/oei/reports/oei-06-06-00340.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 8998

Do Providers of Durable Medical Equipment, Prosthetics, Orthotics, and Other Medical Supplies Meet Medicare Enrollment Requirements?

The study determined whether suppliers complied with Medicare equipment, prosthetics, orthotics, and supplies (DMEPOS) standards. DMEPOS are covered under Medicare Part B. Suppliers must enroll in the program to submit claims for reimbursement. Suppliers of DMEPOS have to comply with 21 Medicare DMEPOS supplier standards to enroll in the Medicare program. Reviewers conducted unannounced site visits to suppliers in Los Angeles County in 2006.

Reviewers found that 13 percent of suppliers did not maintain a physical facility or were not open during unannounced site visits. Another 9 percent were open but did not meet at least one of the two additional requirements for the standards we reviewed. An additional 14 percent met the requirements for the standards, but their claims had in common an atypical characteristic. The agency is considering seeking legislative authority to impose temporary moratoriums on supplier enrollment.

Report Title: Los Angeles County Suppliers’ Compliance With Medicare Standards: Results From Unannounced Visits http://oig.hhs.gov/oei/reports/oei-09-07-00550.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 9007

How Can We Best Move Poor Health Care Beneficiaries into the Prescription Drug Payment Program?

This study examined whether the administrative state-to-plan (S2P) process was more efficient than the point-of-sale facilitated enrollment process in moving low-income subsidy beneficiaries to Part D.

The study compared the administrative efficiency of the S2P demonstration to the contractor-based point-of-sale facilitated enrollment (POS FE) process, explored the characteristics of beneficiaries utilizing the two programs, and examined the feasibility of alternative models for transitioning new dual eligible beneficiaries into Part D. Study results were based on key informant interviews and secondary data analysis.

The study documented gains in administrative efficiency within the ongoing POS FE process due, in part, to edits put in place that led to significant reductions in inappropriate claims submissions and served as a valuable safety net for the enrollment of low income subsidy beneficiaries into a Part D plan. The S2P demonstration laid the groundwork for future endeavors moving these beneficiaries into drug coverage, but there remain pitfalls to avoid and options to pursue.

Report Title: Easing the Part D Transition: An Evaluation of Federal and State Efforts to Ensure Dual Eligibles and Other Low-Income Beneficiaries Maintain Prescription Drug Coverage; http://www.cms.hhs.gov/reports/downloads/Bagchi.pdf

Agency Sponsor: CMS-ORDI, Office of Research, Development, and Information
Federal Contact: Iris Wei, 410-786-6539
Performer: Acumen
PIC ID: 9010

What Knowledge Do Consumers Need in the New Genome-Based Health Care Market?

This preliminary exploration explored what information citizens need to become effective consumers of new genome-based “ products.” The Personalized Health Care Initiative sponsored a workshop in order to provide a forum for stakeholders to discuss alternatives for helping consumers make use of new health information from genetics based systems. Three expert panels met to discuss consumer interest topics. One panel concluded that interest in genetic testing may primarily come about through the individual need and curiosity of the consumer. Another discussed the types of genetic testing information currently provided to consumers, and interaction between consumers and health professionals. The third panel discussed information consumers may need in a new health care system involving these types of tests.

Report Title: Personalized Health Care Initiative Workshop: ‘Understanding the Needs of Consumers in the Use of Genome-based Health Information Services’ http://aspe.hhs.gov/sp/reports/2008/cghis/index.html
Agency Sponsor: ASPE-OSDP, Office of Science and Data Policy
Federal Contact: Gregory Downing, 202-260-1911
Performer: Gregory Downing
PIC ID: 9073

Objective 1.3: Improve health care quality, safety, cost, and value.

Does Essential Health Information Get Exchanged in Post-Acute and Long-Term Care Settings; By What Means; What Encourages or Prevents Such Exchange?

This study sought answers to three questions: what type of health information needs to be exchanged on behalf of patients receiving post-acute and long-term care services as they move from one care setting to another and when does information need to be exchanged among their physicians, pharmacies, and other providers; how is health information exchanged, and is health information technology used to support these types of exchanges; and what factors support or hinder timely health information exchange? Growing evidence indicates that medical errors and quality deficiencies occur during care transitions. Failure to exchange needed health information threatens patient safety by contributing to medication errors and potentially increasing health care expenditures due to patients needing to return to higher levels of care and/or the provision of unnecessary and possibly redundant tests.

Report Title: Health Information Exchange in Post-Acute and Long-Term Care: Project Summary http://aspe.hhs.gov/daltcp/reports/2007/HIErpt.htm
Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Jennie Harvell, 202-690-6443
Performer: University of Colorado, Health Sciences Center
PIC ID: 8334

Can Privacy, Security, Policy, and Statutory Variations Be Reduced Through Multi-State Collaboration?

The Health Information Security and Privacy Collaboration (HISPC), is a group of representatives from 42 states and territories that address the privacy and security challenges presented by electronic health information exchange. Each HISPC participant has the support of its state or territorial governor and maintains a steering committee and contact with a range of local stakeholders to ensure that developed solutions accurately reflect local preferences. During 2008, the states and territories worked together in 7 multi-state collaborative privacy and security projects. These projects analyzed consent data elements in state law, studied intrastate and interstate consent policies, developed tools to help harmonize state privacy laws, developed tools and strategies to educate and engage consumers, developed a toolkit to educate providers; recommended basic security policy requirements, and developed inter-organizational agreements. Each project was designed to develop common, multi-state solutions that could be replicated and that would potentially reduce the variation in state privacy and security practices, policies, and laws.

A cross collaborative steering committee was established to ensure that information is transferred among all project participants and to identify points of intersection. The HISPC involves significant participation from a broad range of state- and local-level stakeholders. Through the work of the HISPC, the state and territory subcontractors produced a number of reports and other materials, and the contractor will produce a final report of achievements early in 2009. The HISPC collaboratives created common solutions for all to use to advance their understanding of privacy and security and electronic health information exchange.

Report Title: Health Information Security and Privacy Collaboration (HISPC) http://www.hhs.gov/healthit/privacy/state.html
Agency Sponsor: OS-ONC, Office of the National Coordinator for Health Information Technology
Federal Contact: Kathleen Fyffe, 202-205-0670
Performer: Research Triangle Institute
PIC ID: 8597

Does Health-Specific Physician Productivity Measurement Impact Medicare Economic Index Calculations?

This study examined the impact of developing a health-specific measure of physician productivity for calculating the Medicare Economic Index.

Researchers developed a conceptual framework for different types of productivity and the problems of measuring them. A health care measure of the Medicare Economic Index was then developed. The measure indicated that all-factor non-farm productivity was about 5 to 10 percent higher than the health care measure in the last ten years.

The assumptions needed to create the measure, and the availability of data left most experts convinced that further work was needed before the health measure could be substituted into the current Medicare Economic Index.

Report Title: Accounting More Accurately for Physician Productivity Investments and Capital Investments in the Update Scheme The papers prepared under this project can be obtained from the Federal Contact
Agency Sponsor: ASPE-OHP, Office of Health Policy
Federal Contact: George Greenberg, 202-690-7794
Performer: Zachary Dyckman; Fund Transferred from the Office of the Assistant Secretary for Planning and Evaluation to the Centers for Medicare & Medicaid Services
PIC ID: 8628

How Useful, Feasible, Ethical and Successful Have Been Advance Directives and Advance Care Planning for Patients?

This study examined what the literature says about the utility, feasibility, ethical issues, and success of implementing advance directives and/or advance care planning for a diverse array of patient populations and across health care settings.

Advance care planning and discernment of end-of-life care preferences is an on-going process best accomplished through continuing communication among individuals, clinicians, and family members. In addition, multi-part interventions show that advance care planning and advance directives can be carried out successfully, at least in defined populations. Replication and extension from such interventions and implementation via health information technology hold promise for improving care toward the end of life.

Report Title: Literature Review on Advance Directives http://aspe.hhs.gov/daltcp/reports/2007/advdirlr.htm
Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Judith Peres, 202-690-6443
Performer: Rand Corporation
PIC ID: 8894

What Health Information Technology and Electronic Health Records Functions Do Nursing Homes and Home Health Agencies Use?

This study explored the use of and need for health information technology applications in nursing homes and home health agencies. Through a literature review and stakeholder discussions the authors identified the types of Health Information Technology and Electronic Health Record applications and functions (electronic point-of-care and information exchange) currently used in nursing homes or home health agencies that go beyond the federally-mandated Output and Assessment Information Set (OASIS) and Minimum Data Set (MDS) reporting and claims submission requirements. Nursing home and home health agency providers and vendors provided feedback regarding which health information technology and electronic health record applications they were using.

The stakeholders largely agreed with the description of functions and organization of the taxonomy. Four of the five responding nursing home providers and all five responding home health agency providers indicated that they used automated systems for most of the functions listed under the administration domain. Four of the five nursing facilities reported` some automated quality management tools. Four of the five home health agencies were able to order patient supplies electronically from the field.

Reports:

• Taxonomy of Health Information Technology Functions in Nursing Homes and Home Health Agencies  – Report A: Review by Representatives from Standards Development Organizations, http://aspe.hhs.gov/daltcp/reports/2007/Taxonomy-SDO.htm
• Taxonomy of Health Information Technology Functions in Nursing Homes  – Report B: Review by Representatives from Nursing Homes and Vendors http://aspe.hhs.gov/daltcp/reports/2007/Taxonomy-NH.htm
• Taxonomy of Health Information Technology Functions in Home Health Agencies  – Report C: Review by Representatives from Home Health Agencies and Vendors http://aspe.hhs.gov/daltcp/reports/2007/Taxonomy-HHA.htm
  Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy

Federal Contact: Jennie Harvell, 202-690-6443
Performer: University of Colorado, Health Sciences Center
PIC ID: 8895, 8896, 8897

How Well Do Health Care Delivery Systems Exchange Health Information?

The study examined how well health care delivery systems that use health information technology (HIT) exchange health information on behalf of patients who also received services from post-acute and long-term care (PAC/LTC) providers that also used HIT and were either “ affiliated” or “ unaffiliated” with these health delivery systems. The study (1) described the use of health information technology in state-of-the-art health delivery systems and how health information was or was not exchanged with unaffiliated post-acute and long-term care (PAC/LTC) providers, physician offices, laboratories, pharmacies, and hospitals; (2) identified the factors that supported or deterred timely exchange of health information with unaffiliated PAC/LTC providers and other parts of the health care delivery continuum that use HIT; and (3) identified ways to encourage information exchange between health delivery systems that used HIT with unaffiliated PAC/LTC providers and use of HIT in PAC/LTC.

Report Title: Health Information Exchange in Post-Acute and Long-Term Care Case Study Findings: Final Report http://aspe.hhs.gov/daltcp/reports/2007/HIEcase.htm
Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Jennie Harvell, 202-690-6443
Performer: University of Colorado, Health Sciences Center
PIC ID: 8899

How Has the Law of Advance Directives Advanced?

This study sought the best ways to promote use of advance directives among adults as a way for them to choose their end-of-life care. The study examined the legislative history of advance directives.

Legislative provisions addressing health care advance directives evolved quickly, starting with California’s adoption of the first living will statute in 1976.
States began crafting special durable powers of attorney for health care statutes or adding proxy provisions to their living will statutes. State legislative action took place roughly from the mid-1980s to the mid-1990s. Law and practice are now moving toward a more flexible communications approach, bridging the gap between patient desires — expressed through an advance directive — and plans of care reflected in physician orders (sometimes referred to as Physicians’ Orders for Life-Sustaining Treatment).

Report Title: Advance Directives and Advance Care Planning: Legal and Policy Issues http://aspe.hhs.gov/daltcp/reports/2007/adacplpi.htm
Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Judith Peres, 202-690-6443
Performer: American Bar Association, Commission on Law and Aging
PIC ID: 8906

Do Older Individuals With Children Use More Medicare Covered Services Than Those Without Children?

This study examined Medicare service use and costs of older individuals with and without children using survey data from the National Long Term Care Survey linked to Medicare claims data. The authors considered annual cost and usage data for 17 separate categories of Medicare-reimbursed services, as well as annual costs and outcomes.

Only for 1995 were Medicare costs lower for parents than for childless individuals. In only one service category — physician visits for providing long-term care services, either at home or in a nursing home — was there a consistent pattern of significantly lower costs among parents.

Report Title: Childless Elderly Beneficiaries' Use and Costs of Medicare Services: Final Report http://aspe.hhs.gov/daltcp/reports/2006/childless.htm
Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: William Marton, 202-690-6443
Performer: Syracuse University
PIC ID: 8908

For Adults with Serious Mental Illness, Which States Have Developed Self-Directed Care Programs; With What Impact; and What Barriers Prevent Their Extension?

This study compiled descriptions and impacts of self-directed care programs for adults with serious mental illness. Self-directed care gives consumers greater control over the services they receive to meet their mental health needs. It provides a more consumer- and family-driven mental health system.

In this early evaluation, self-directed care programs satisfied consumers more than traditional services. Self-directed care changed the pattern of mental health services used by consumers. Remaining to be seen are longer term impacts on mental health outcomes and costs. The difficulty of supporting self-directed care through Medicaid was seen as a significant barrier to extending the approach.

Report Title: The Contribution of Self-Direction to Improving the Quality of Mental Health Services http://aspe.hhs.gov/daltcp/reports/2007/MHslfdir.htm
Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Vidhya Alakeson, 202-690-6443
Performer: Staff; Office of the Assistant Secretary for Planning and Evaluation
PIC ID: 8910

How Can the Food and Drug Administration Improve Efforts to Assure That Drugs Approved for Sale Are Safe, Effective, and Optimal?

This study identified possible improvements to the Food and Drug Administration post-marketing commitment (PMC) processes. FDA evaluates new drug and biological products prior to approval for marketing in the United States and may request that a sponsor seeking approval of a new drug or biological product conduct a post-marketing study to provide additional information about the safety, efficacy or optimal use of a drug or biological product. Such studies are important but not necessary for approval to market the product.

The study found that the post-marketing commitment processes positively impacted public health, but need to be used judiciously to ensure that only commitments addressing important issues regarding safety, efficacy and optimal use are requested. Sponsors generally agreed that the PMC program had a positive public health impact. More than half of fulfilled PMCs assessed in the study resulted in a label change. The most common reasons for the label changes were validated safety and efficacy concerns, validated drug-drug interaction concerns, and expanded use in subpopulations. However, 50 percent of sponsors questioned the value and/or rationale of PMCs. These sponsors noted that in some cases, the supporting studies were ongoing at the time of approval of the product, and the PMC was simply a mechanism to ensure the results were submitted to FDA. Others reported that they believed the PMC supported a reviewer's academic interests. PMC milestones agreed on by FDA and the sponsor were not always met. The main reason for FDA failure to meet review goal dates was competing workload priorities.

Report Title: Independent Evaluation of FDAs Prescription Drug User Fee Act III -- Evaluations & Initiatives -- Task Order 4 -- Post Marketing Commitments Study Report http://www.fda.gov/ope/PMC/pmc08.html
Agency Sponsor: FDA, Food and Drug Administration
Federal Contact: Judith Arndt, 240-276-3234
Performer: Booz Allen Hamilton
PIC ID: 8948

What Factors Influence New Drug Application First-Cycle Approvals; What Initiatives Impact These Factors?

This study examined efforts to achieve specific goals to improve the effectiveness and efficiency of review of new drug applications and biologic license applications. Several of these goals focused on improving the review process activities occurring between initial submission of the application and subsequent FDA action regarding the application.

The study found that the Filing Review Notification, or 74-Day Letter, communicated deficiencies to sponsors. Also, priority review designation given to applications for products that offered major advances in treatment or provided a treatment where no adequate therapy exists, had the most significant impact on first-cycle approval rates. Applications were more likely to be approved in the first cycle if a major deficiency was identified pre-submission than if major deficiencies were identified during the review. Applications for which no major deficiency was identified either pre-submission or during the review had a high first-cycle approval rate. Products with a novel mechanism of action targeting life-threatening conditions had a greater first-cycle approval rate. Applications that complied with most or all of the assessed good review manufacturing procedures activities had the highest first-cycle approval rates. Researchers made two key recommendations: FDA should continue with good review manufacturing procedures implementation, ensuring adoption of good review manufacturing procedures activities and timeframes; and, FDA should continue to use the 74-Day Letter to communicate application deficiencies early in the review process.

Report Title: Independent Evaluation of FDA's First Cycle Review Performance - Final Report http://www.fda.gov/oc/pdufa3/firstcyclerept08/default.htm
Agency Sponsor: FDA, Food and Drug Administration
Federal Contact: William Hagan, 301-827-5292
Performer: Booz Allen Hamilton
PIC ID: 8950

How Can Medication Therapy Management Programs Be Improved?

This study explored the evolving field of medication therapy management (MTM) and sought to understand attributes of MTM programs potentially useful for the Medicare program, including structure of organizations providing MTM, services and interventions included, providers involved, how beneficiaries are targeted, differences from and integration with disease management programs, financing, and outcomes. Statute requires that a prescription drug plan or Medicare Advantage plan that offers prescription drug coverage have an MTM program.

There was no clear evidence which practice models or elements contributed to clinical outcomes. Common features found among both public and private MTM programs included reliance on pharmacists as primary service providers, comprehensive medication review that included reconciliation of drug therapies with prescribers’ records, and patient education and monitoring. The study found areas where the program could continue to be refined including coordination with stand-alone prescription drug plans and potential overlap of responsibilities between plan and nursing home consultant pharmacists for institutionalized patients.

Report Title: Exploratory Research on Medication Therapy Management http://www.cms.hhs.gov/Reports/Downloads/Blackwell.pdf
Agency Sponsor: CMS-ORDI, Office of Research, Development, and Information
Federal Contact: Steve Blackwell, 410-786-6852
Performer: Abt Associates, Inc.
PIC ID: 8951

What Changes to the Home Health Prospective Payment System of Case Mix Groups Would Improve Its Predictive Power and Financial Incentives?

The study addressed Medicare's information needs for refining the home health case mix model and monitoring the home health prospective payment system. Topics covered included: trends in patient and payment-related data over time; refinements to diagnosis groups within the original case mix model; reducing reliance on the therapy threshold in the case mix model; and how the system can account for costs of caring for long-stay patients and highly variable costs of bundled nonroutine medical supplies.

Main results were that redefining the system's therapy threshold, adding more thresholds, and accounting for the timing of the 60-day claim in relation to the care episode's entire sequence of claims, could improve the predictive power of the case mix model.

Report Title: Refinement of Medicare's Home Health Prospective Payment System: Final Report; http://www.cms.hhs.gov/Reports/downloads/Coleman_Final_April_2008.pdf

Agency Sponsor: CMS-ORDI, Office of Research, Development, and Information
Federal Contact: Ann Meadow, 410-786-6602
Performer: Abt Associates, Inc.
PIC ID: 8953

How Might Refinements to Cost-to-Charge Ratios Improve the Payment Accuracy of Two Sets of Relative Payment Weights?

The study sought to improve the payment accuracy of the ambulatory payment classification weights used in the hospital outpatient prospective payment system and the Medicare severity adjusted diagnosis related groups weights used in the hospital Inpatient prospective payment system. In order to improve them, researchers analyzed the cost estimates used in the computation of relative resource weights. They analyzed how to better use existing cost report and claims data, making changes to the Medicare cost report and MedPAR data file, and estimating statistical adjustments to address aggregation bias in cost-to-charge ratios.

Key results included corrections of providers’ misclassification of nonstandard cost centers on the Medicare cost report and expansions and revisions of the detailed revenue code crosswalk used to map claims charges to provider-specific cost report cost-to-charge ratios. Researchers created new outpatient cost centers for services not previously recognized in the cost center aggregation tables. Statistical adjustments were estimated for potential use in cost areas such as medical devices, cardiac catheterization, computed tomography (CT) and magnetic resonance imaging (MRI) scanning, radiology, nuclear medicine, and intravenous solutions.

Report Title: Refining Cost to Charge Ratios for Calculating APC and MS-DRG Relative Payment Weights http://www.rti.org/reports/cms/HHSM-500-2005-0029I/PDF/Refining_Cost_to_Charge_Ratios_200804.pdf

Agency Sponsor: CMS-ORDI, Office of Research, Development, and Information
Federal Contact: Philip Cotterill, 410-786-6598
Performer: Research Triangle Institute
PIC ID: 8955

How Do Alternative Methods of Calculating Diagnosis-Related Groups' Relative Weights Impact Their Values and Accuracy?

The study used alternative methods of estimating cost and standardizing for systematic cost differences among hospitals to calculate 5 sets of alternative relative weights which were compared to the relative weights constructed using the current method. This was done to learn the implications of different methods on payments and payment accuracy in the hospital Inpatient Prospective Payment System. To assess the payment implications, researchers compared the relative weights and average payments using each alternative method to the relative weights and average payments across hospital groupings using fully phased-in Medicare severity diagnosis-related group (DRG) cost weights. Researchers assessed each relative weight method in terms of its ability to explain cost differences among DRGs and its impact on payment accuracy.

The study found substantial differences in the weights for DRGs across the alternative methods and large redistributions across hospitals. But there was little difference across the methods in their ability to predict cost at either the discharge or hospital levels. None of the alternative methods represented a marked improvement over the current method. The indirect medical education and disproportionate share payment adjustments had a larger impact on payment accuracy than did calculating DRG relative weights.

Report Title: Evaluation of Alternative Methods to Establish DRG Relative Weights http://www.rand.org/pubs/working_papers/WR560/
Agency Sponsor: CMS-ORDI, Office of Research, Development, and Information
Federal Contact: Philip Cotterill, 410-786-6598
Performer: Rand Corporation
PIC ID: 8957

What Impact Would Shifting from Centers for Medicare & Medicaid Services' Wage Index to the Medicare Payment Advisory Commission's Alternative Have?

This impact analysis compared the Centers for Medicare & Medicaid Services’ (CMS’) wage index with the Medicare Payment Advisory Commission (MedPAC) recommended hospital compensation index. The Tax Relief and Health Care Act of 2006 required the MedPAC to recommend alternatives for revising the hospital wage index. In developing its recommendations, the Act required CMS to consider MedPAC's work.

Under MedPAC's recommended index, the differences between hospitals with the highest and lowest wage index would be reduced. Furthermore, adopting MedPAC's recommended index would lead a substantial number of hospitals to experience change in their index values of more than 5 percent. As part of ongoing work, the study is investigating reasons for underlying differences between the CMS wage index and MedPAC's compensation index. The latter incorporates Bureau of Labor Statistics, not CMS, wage data.

Report Title: Impact Analysis for the 2009 Final Rule: Interim Report Revision of Medicare Wage Index http://www.acumenllc.com/reports/cms/RevisedImpactAnalysisfor2009FinalRule.pdf
Agency Sponsor: CMS-ORDI, Office of Research, Development, and Information
Federal Contact: Craig Caplan, 410-786-4165
Performer: Acumen
PIC ID: 8958

Have the Fifteen Medicare Coordinated Care Demonstration Programs Improved Health Outcomes and Reduced Costs for Beneficiaries?

This study determined whether the 15 individual Medicare Coordinated Care Demonstration (MCCD) programs improved health outcomes and reduced Medicare costs for their targeted diagnostic groups of chronically-ill beneficiaries. The study entailed a randomized clinical trial design with each program having two arms: an intervention group and a ‘usual care’ control group.

Researchers found that five programs in the demonstration had some modest favorable effects on quality without significantly increasing Medicare expenditures. One program was cost neutral; another was probably cost neutral; two had low enrollment and have closed; the fifth had only small, insignificant Part A & B savings, insufficient to offset its high per-beneficiary fee to Medicare. The remaining ten programs significantly increased costs to Medicare, although one reduced hospitalizations by 17 percent. While it failed to attain budget neutrality due to its high fee structure, future fee reduction for the program could potentially provide savings to Medicare. The Centers for Medicare & Medicaid Services granted conditional extensions (until 2010) to the three programs that either are providing, or have potential to provide, savings to Medicare, however, one of these and its parent vendor closed all operations in 2008.

Report Title: Third Report to Congress on the Evaluation of the Medicare Coordinated Care Demonstration (MCCD) ; Report may be obtained from Federal Contact
Agency Sponsor: CMS-ORDI, Office of Research, Development, and Information
Federal Contact: Carol Magee, 410-786-6611
Performer: Mathematica Policy Research, Inc.
PIC ID: 8960

How Can We Improve Health Outcomes and Reduce Medicare Expenditures for Certain Medicare Beneficiaries With Heart Problems?

The evaluation examined the impact of providing disease management services and a prescription drug benefit for Medicare fee-for-service beneficiaries with congestive heart failure, coronary artery disease or diabetes. Three disease management organizations participated in the demonstration. The evaluation used a randomized design where eligible beneficiaries were recruited, then randomized into a treatment or control group.

Results of the evaluation showed that the effect of the prescription drug benefit on beneficiary access to prescription drugs was small. There were some positive impacts of disease management on a few of many care process measures: more chronic heart failure patients served by two of the programs had left ventricular assessments, and more diabetics in one program had more claims for self-monitoring supplies, podiatry visits, therapeutic shoes and urine tests for protein. There were no treatment control group differences in beneficiary satisfaction with general health care or with any of the aspects of care they were asked about, nor were there differences on the measures of physical functioning or perceived mental and physical health quality of life. The analyses indicated that the demonstration had no impacts on hospitalizations, readmissions, emergency room use or on Medicare Part A and B expenditures. The demonstration ended early when it became clear that the programs were unable to generate savings to the Medicare program to offset their fees.

Report Title: Evaluation of Medicare Disease Management Programs; Report may be obtained from Federal Contact

Agency Sponsor: CMS-ORDI, Office of Research, Development, and Information
Federal Contact: Lorraine Johnson, 410-786-9457
Performer: Mathematica Policy Research, Inc.
PIC ID: 8963

How Might Location Be Used to Adjust Medicare Physician Fee Schedule Payments?

This study examined using metropolitan statistical areas (MSAs), grouping counties with similar practice costs, and incremental revisions in the current localities as potential options for defining the Geographic Practice Cost Index and Geographic Adjustment Factors (GAFs) used to adjust payment under the Medicare Physician Fee Schedule.

No recommendations were made on the favored geographic definition. Using MSAs changed the number from the 89 current payment localities to 387 metropolitan payment areas and 51 non-metropolitan payment areas. Grouping counties with similar GAFs resulted in 134 localities, compared to the 89 localities. statewide localities were reduced from 36 to 7. Incremental approaches retained the current localities, but made small changes to address areas generating the most complaints or in which the data suggested the largest payment inaccuracy.

Report Title: Payment Areas for Medicare Physician Services: Selected Alternatives http://www.cms.hhs.gov/Reports/downloads/Adamache_Final_March_2008.pdf

Agency Sponsor: CMS-ORDI, Office of Research, Development, and Information
Federal Contact: Jesse Levy, 410-786-6600
Performer: Acumen
PIC ID: 8964

Have Advanced Imaging Service Payments Increased Under Medicare's Physician Fee Schedule?

This study documented the nature and extent of growth in advanced imaging-computed tomography, magnetic resonance, and positron emission tomography between 1995 and 2005. Advanced imaging used to be the exclusive domain of hospitals; however, in the last 10 to 15 years, use of these services under the Medicare program has proliferated in ambulatory settings. Oversight of these settings includes accreditation and certification for hospital outpatient departments, State licensure for independent diagnostic testing facilities, and doctors’ offices. In December 2006, final regulations were issued with new performance standards.

Advanced imaging paid under the Medicare Physician Fee Schedule grew significantly between 1995 and 2005: (1) Services per year grew more than fourfold, from 1.4 million to 6.2 million; (2) allowed charges grew by more than 5 times, to $3.5 billion in 2005; and (3) the use rate grew from 42 services per 1,000 beneficiaries in 1995 to 163 per 1,000 in 2005. In 2005, significant variation remained in use rates across States, from 8 services per 1,000 beneficiaries in Vermont to 326 per 1,000 in Florida. In every year from 1995 to 2005, a small number of procedure codes consistently accounted for over half of all advanced imaging billed. The share of all advanced imaging services grew from 2.5 percent in 1995 to 23 percent in 2005.

Report Title: Growth in Advanced Imaging Paid Under the Medicare Physician Fee Schedule http://oig.hhs.gov/oei/reports/oei-01-06-00260.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General

Do States Operating Medicaid Managed Care Programs Arrange for External Quality Reviews?

This study assessed ways in which States utilized external quality reviews of Medicaid managed care. Federal statute requires States that operate Medicaid managed care programs to arrange for external quality reviews which States may conduct or contract with an independent external quality review organization (EQRO). States receive matching Federal funds for the costs of the review. Regulations include three mandatory and five optional activities for the review. These regulations specify five deliverables that EQROs must produce based on the review activities.

Most States were using the results of EQRO reviews. Of the 37 States that implemented external quality reviews, 33 required their managed care plans to make changes based on EQRO reports, such as improving how plans conduct performance improvements. Some EQRO reports did not include all required information, despite the States’ oversight. Reports for 15 States were missing at least one of the deliverables. All States reported regular monitoring of their EQROs, through communication, status reports, and contract provisions. More than half of the States cited concerns with the external quality review process regarding staffing issues, EQRO report quality, and redundancy with other monitoring efforts.

Report Title: External Quality Reviews in Medicaid Managed Care http://oig.hhs.gov/oei/reports/oei-01-06-00510.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 8977

Do Medicare Administrative Law Judges Hold Timely Hearings?

This study assessed the use of telephone, video teleconference, and in-person hearings to decide Medicare administrative law judge cases and the timeliness of decisions during the first 13 months of operation of the Office of Medicare Hearings and Appeal (OMHA). Medicare beneficiaries, providers, and suppliers of health care services can appeal decisions about their Medicare claims.

OMHA held approximately three-quarters of its hearings by telephone during its first 13 months. Most sampled appellants were satisfied with their hearing format. Incomplete and inaccurate data limited the ability to manage the caseload. A number of cases were not decided in a timely manner. Fifteen percent of the cases that had a 90-day requirement and a decision date in the appeals system were not decided on time. A training program was implemented for staff to standardize the scheduling process and performs random unannounced screenings of the scheduling procedures and has modified the appeals system to record and track the requested and hearing formats. Steps have been taken to address technical difficulties associated with telephone and video teleconference hearings.

Report Title: Medicare Administrative Law Judge Hearings: Early Implementation, 2005-2006 http://oig.hhs.gov/oei/reports/oei-02-06-00110.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 8979

How Do Medicare Hospice Patients in Nursing Facilities and Other Settings Compare?

This study was analyzed data from Medicare Part A hospice claims, the Minimum Data Set, and the Online Survey Certification and Reporting System. The Medicare hospice benefit allows a beneficiary with a terminal illness to forgo curative treatment for the illness and instead receive palliative care, which is the relief of pain and other uncomfortable symptoms.

Hospice beneficiaries in nursing facilities were more than twice as likely as beneficiaries in other settings to have had a terminal diagnosis of an ill-defined condition, a mental disorder, or Alzheimer’s disease. On average, beneficiaries in nursing facilities spent more time in hospice care and were associated with higher Medicare reimbursements for hospice care than beneficiaries in other settings.

Report Title: Medicare Hospice Care: A Comparison of Beneficiaries in Nursing Facilities and Beneficiaries in Other Settings http://oig.hhs.gov/oei/reports/oei-02-06-00220.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 8982

How Frequently Do Hospice Beneficiaries Use Respite Care?

This study analyzed Medicare Part A claims for hospice care. Respite care is short-term inpatient care that provides respite for the individual’s family or other persons caring for the individual at home.

Insufficient hospice claim information limited the agency’s ability to determine whether hospice agencies were complying with the requirement that they may not be reimbursed for more than 5 consecutive days of respite care at a time. Two percent of all hospice beneficiaries received respite care during 2005 and most of these beneficiaries received the care for 5 days or less; there were a number of instances in which the use of respite care may have been inappropriate. Fifty-four beneficiaries received respite care longer than the 5 consecutive days allowed by Federal regulations, and 62 received respite care while residing in nursing facilities, even though respite care is designed to relieve the beneficiary’s caregiver.

Report Title: Hospice Beneficiaries' Use of Respite Care http://oig.hhs.gov/oei/reports/oei-02-06-00222.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 8983

How Do Physician-Owned Specialty Hospitals Manage Medical Emergencies?

This study provided an assessment of physician-owned specialty hospitals’ ability to manage medical emergencies. The study was based on data from 109 physician-owned specialty hospitals and relies on a review of staffing schedules for nurses and physicians for 8 sampled days, a review of hospitals’ staffing policies, a review of policies for managing medical emergencies, and interviews with hospital administrators.

About half of all physician-owned specialty hospitals had emergency departments, the majority of which had only one emergency bed. Not all hospitals had nurses on duty and physicians on call. Less than one-third of administrators report having physicians onsite at all times. Two-thirds of hospitals use 9-1-1 as part of their emergency response procedures. Some hospitals lacked basic information in their written policies about managing medical emergencies. The Centers for Medicare & Medicaid Services stated that it would add information to its provider enrollment form and the new Provider Enrollment and Chain-Operated System. CMS issued a program memorandum to State Survey Agencies that reiterated its requirements for hospitals and addresses medical emergency requirements.

Report Title: Physician-Owned Specialty Hospitals' Ability To Manage Medical Emergencies http://oig.hhs.gov/oei/reports/oei-02-06-00310.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 8984

Do Prescription Drug Plan Sponsors Owe Money to Medicare Under its Payment Reconciliation Process?

This study assessed the estimated reconciliation amounts that prescription drug plan (Part D) sponsors owed to and received from Medicare for 2006. Results were primarily based on preliminary estimates from the Centers for Medicare & Medicaid Services.

Part D sponsors owed Medicare $4.4 billion for 2006. Eighty percent of sponsors owed money to Medicare, whereas 20 percent of sponsors would receive money from Medicare. The majority of the funds sponsors owed were profits that they must repay to Medicare as a result of risk-sharing requirements. The agency has no mechanisms to collect funds owed by sponsors until it completed reconciliation scheduled to occur months after the plan year ended and had no mechanism to adjust prospective payments prior to reconciliation. As a result, sponsors had the use of these funds for a significant length of time. The Centers for Medicare & Medicaid Services stated that it had the authority to change payment methodologies for the low-income cost-sharing and reinsurance subsidies and that it was examining possible options.

Report Title: Medicare Part D Sponsors: Estimated Reconciliation Amounts for 2006 http://oig.hhs.gov/oei/reports/oei-02-07-00460.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 8985

Do Prescription Drug Plans Accurately Track Beneficiaries' Out-of-Pocket Costs?

This study reviewed the processes that Part D plans and the Coordination of Benefits Contractor uses to help ensure the accurate tracking and oversight of beneficiaries’ true out-of-pocket costs. Part D plans are responsible for tracking these costs.

The study found that information on Part D plan enrollees’ additional prescription drug coverage was not consistently submitted in 2006; that Part D plans cited problems with transferring true out-of-pocket balances when enrollees changed plans; and that the Centers for Medicare & Medicaid Services had conducted limited oversight of Part D plans tracking of true out-of-pocket costs. CMS will continue the Part D plans’ self-attestation process, enforcing compliance with data-sharing agreements.

Report Title: Tracking Beneficiaries' True Out-of-Pocket Costs for the Part D Prescription Drug Benefit http://oig.hhs.gov/oei/reports/oei-03-06-00360.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 8986

Do Average Sales Prices for Five Inhalation Drugs Exceed Market Prices?

This study determined whether the volume-weighted average sales price exceeded market price by at least 5 percent for of the five inhalation drugs under review.

The volume-weighted average sales price for two of the five inhalation drugs under review (albuterol and levalbuterol) exceeded market price by at least 5 percent in the second quarter of 2007. The Medicare payment amount for albuterol in the third quarter of 2007 was 13 times greater than the market price in the previous quarter because of the agency’s decision to reestablish a single drug code for albuterol and levalbuterol beginning July 1, 2007. After the analysis, the Centers for Medicare & Medicaid Services separated albuterol and levalbuterol back into two codes, thereby establishing separate payment amounts for the two drugs.

Report Title: A Comparison of Average Sales Prices to Widely Market Prices for Inhalation Drugs http://www.oig.hhs.gov/oei/reports/oei-03-07-00190.pdf

Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 8988

Do Average Sales Prices Exceed Average Manufacturer Prices for Specific Prescription Drugs?

Pursuant to Federal statute, the Office of Inspector General must notify the Secretary of the Department of Health and Human Services (the Secretary) if the average sales price (ASP) for a drug exceeds the drug’s average manufacturer price (AMP) by a threshold of 5 percent. If that threshold is met, the statute grants the Secretary authority to disregard the ASP for that drug and substitute the payment amount for the drug code with the lesser of the market price for the drug (if any) or 103 percent of the AMP.

For the second quarter of 2007, reviewers identified 22 of 292 drug codes with ASPs that exceeded AMPs by at least 5 percent. If reimbursement amounts for all 22 codes were based on 103 percent of AMP, reviewers estimated that Medicare expenditures would have been reduced by $8 million during the fourth quarter of 2007.

For the third quarter of 2007, using the current ASP payment methodology, reviewers identified 41 of 369 drug codes with ASPs that exceeded AMPs by at least 5 percent. If reimbursement amounts for all 41 codes were based on 103 percent of AMP, reviewers is estimated that Medicare expenditures would have been reduced by $16 million during the first quarter of 2008. Under the revised payment methodology, ASPs for 35 of 369 drug codes would have exceeded at least 5 percent. Of these 35 codes, 32 met the 5-percent threshold under the Centers for Medicare & Medicaid Services’ current method for volume-weighting data. An additional three codes would have met the 5-percent threshold using the revised ASP payment methodology but not the current methodology.

Report Title: Comparison of Second-Quarter 2007 Average Sales Prices to Average Manufacturer Prices: Impact on Medicare Reimbursement for Fourth Quarter 2007 http://oig.hhs.gov/oei/reports/oei-03-08-00010.pdf
Report Title: Comparison of Third-Quarter 2007 Average Sales Prices to Average Manufacturer Prices: Impact on Medicare Reimbursement for First-Quarter 2008 http://oig.hhs.gov/oei/reports/oei-03-08-00130.pdf

Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 8989 and 8990

Do Medicare Fee Schedule Amounts for Power Wheelchairs Exceed Internet Prices?

This study compared the Medicare fee schedule amounts for power wheelchairs to Internet prices. In 2004, Medicare and its beneficiaries paid higher prices for power wheelchairs than consumers or suppliers. In November 2006, the agency implemented a revised fee schedule for power wheelchairs as part of a strategy to reform Medicare payments for power wheelchairs. The revised fee schedule was designed to improve the accuracy of Medicare pricing for power wheelchairs. Reviewers (1) collected and analyzed prices of power wheelchairs from the Internet sites of power wheelchair suppliers during the first quarter of 2007, (2) then compared the median Internet prices of power wheelchairs to the Medicare fee schedule amounts during the same time period, and (3) calculated the potential savings to Medicare and its beneficiaries during the first quarter of 2007 had power wheelchair claims submitted to Medicare been reimbursed at the median Internet prices we collected.

Reviewers found that Medicare fee schedule amounts for power wheelchairs were 45 percent higher than median Internet prices in the first quarter of calendar year 2007. Medicare and its beneficiaries could have achieved savings during the first quarter of 2007 had power wheelchairs been reimbursed at median Internet prices. On average, each beneficiary could have saved $233 of his or her power wheelchair copayment. Researchers concluded that consumers were able to purchase most power wheelchairs over the Internet at lower prices than the Medicare fee schedule amounts for the same power wheelchairs.

Report Title: A Comparison of Medicare Program and Consumer Internet Prices for Power Wheelchairs http://oig.hhs.gov/oei/reports/oei-04-07-00160.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 8991

What Accounts for Improper Medicaid Rebate Payments?

This study estimated inappropriate Medicaid rebate claims caused by unit of measure inconsistencies and determined how often States converted their Medicaid use data to correct for unit of measure inconsistencies. Prescription drugs are defined using two types of unit of measure standards. Inconsistencies between the two standards have potential financial implications for Medicaid rebates.

Reviewers estimated that unit of measure inconsistencies resulted in $11.8 million in inappropriately claimed Medicaid rebates during the first 6 months of 2006. Most inconsistencies involved the unit type “ each” (for example pricing for “ each” pill). On average, States converted 45 percent of their use data for drugs with unit of measure inconsistencies; and that States could not use package size data from the agency to efficiently detect or correct for unit of measure inconsistencies. Reviewers recognized that inappropriate Medicaid rebate claims caused by unit of measure inconsistencies represented less than 1 percent of rebate claims. However, as the analysis of the National Council for Prescription Drug Programs and the agency data showed, unit of measure inconsistencies remained a problem within the Medicaid prescription drug program. The agency made efforts to prevent and correct these inconsistencies.

Report Title: Unit of Measure Inconsistencies in the Medicaid Prescription Drug Program http://oig.hhs.gov/oei/reports/oei-05-07-00050.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 8994

Do Safeguards Prevent or Detect Prescription Drug Plan Fraud and Abuse?

This study determined what safeguards were implemented during fiscal year 2006 to prevent and detect fraud and abuse in Medicare prescription drug plans. Statute requires performance of financial audits of drug plans contracted to provide drug benefits to Medicare beneficiaries and holds considerable discretion in structuring safeguards for the program.

The Office of Inspector General found that the agency implemented safeguard activities throughout fiscal year 2006; however, further development or application of these activities is needed. The agency relied largely on complaints to identify potential fraud and abuse; however, not all complaints were investigated timely. Further, limits to legal authority, jurisdiction, and the agency’s ability to monitor enrollees switching plans complicated its efforts to safeguard Medicare Part D prescription drug plans.

Report Title: CMS's Implementation of Safeguards During Fiscal Year 2006 To Prevent and Detect Fraud and Abuse in Medicare Prescription Drug Plans http://oig.hhs.gov/oei/reports/oei-06-06-00280.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 8997

Does Medicare Correctly Process Denial of Payment Remedies for Skilled Nursing Facilities?

This study determined if fiscal intermediaries appropriately processed denials of Medicare payment remedies for skilled nursing facilities that have been found noncompliant with Federal program participation standards. Denial of payment is an enforcement remedy that the agency may used to address noncompliance with Federal quality of care standards in skilled nursing facilities. The agency is responsible for imposing denial of payment remedies but relies on its fiscal intermediary’s to identify and reject the relevant Medicare claims.

Three-quarters of the denial of payments for new admission remedies were processed incorrectly. In 40 percent of the total cases, errors resulted in one or more inappropriate payments to nursing facilities. These overpayments exceeded $5 million. In the other 34 percent of the total cases, processing errors occurred but did not result in claims paid in error, either because the facilities did not have new admissions during the remedy periods or the facilities were aware of the remedies and did not submit claims for new admissions during the remedy periods. Errors were attributable primarily to late processing and problems with agency’s provision of denial of payment instructions to the appropriate fiscal intermediaries. Approximately half of claims involving readmissions lacked codes indicating the readmission status, which made these claims appear to be new admissions that should be denied. CMS plans to develop new internal procedures to ensure that it effectively communicates denials of payment for new admissions instructions to fiscal intermediaries and Medicare administrative contractors.  CMS has created a protocol between itself and the fiscal intermediaries and Medicare administrative contractors to ensure follow-up notification and CMS is updating its manual to clarify coding and verification requirements.

Report Title: Nursing Home Enforcement: Processing Denials of Medicare Payment http://oig.hhs.gov/oei/reports/oei-06-03-00390.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 8999

Do Contractors Meet Requirements for Processing Medicare Claims Reconsiderations?

This study examined whether qualified independent contractors met Medicare Parts A and B claims reconsiderations requirements.

From May 2005 to July 2006, qualified independent contractors handling Medicare Part A claims reconsiderations met 60-day processing requirements and 58 percent of Medicare Part B reconsiderations did not meet processing timeframes. Four contractors did not follow all requirements relating to correspondence, such as letters acknowledging appeal requests, notifications of the reconsideration decision, and notifications of processing delays. For 54 percent of reconsiderations, contractors did not enter accurate information in the Medicare Appeals System, which stores reconsideration data and is used by the agency to monitor contractor adherence to processing requirements. Contractors had not been operational for long at the time of our review and the agency awarded a contract to a private entity to conduct its own performance evaluation and made several changes to the second level of Medicare appeals to improve the reconsiderations process.

Report Title: Early Implementation Review of Qualified Independent Contractor Processing of Medicare Appeals Reconsiderations http://oig.hhs.gov/oei/reports/oei-06-06-00500.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 9000

Are Erroneous Fee-for-Service Payments Made for Capitated Medicaid Managed Care Services?

This study determined which Medicaid programs paid non-institutional fee-for-service claims for services provided to beneficiaries enrolled in capitated Medicaid managed care plans. In capitated Medicaid managed care arrangements, State Medicaid programs pay managed care plans a fixed rate per Medicaid beneficiary in exchange for services included in the plan.

Four of the five State Medicaid programs included in the review reimbursed fee-for-service claims $864,000 in error, and two State Medicaid programs potentially paid an additional $974,006 in error for the same quarter. The agency planned to (1) remind States of the importance of eliminating erroneous payments and recommend that States make necessary edits to their payment systems at the next Medicaid Managed Care Technical Advisory Group call and (2) work with States to voluntarily collect the overpayments associated with the erroneous fee-for-service payments.

Report Title: Fee-For-Service Payments For Services Covered By Capitated Medicaid Managed Care http://oig.hhs.gov/oei/reports/oei-07-05-00320.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 9002

Do Medicare and Medicaid Duplicate Each Others' Payments for Supplies or Services?

This study examined whether duplicate payments were being made by Medicare and Medicaid. Medicare and Medicaid pay home health providers for services but should not pay for the same medical supplies or services for the same beneficiary. Home health services seek to restore health and minimize the effects of illness and disability, thereby enabling beneficiaries to reside in community settings and avoid institutionalization.

In four of the five States reviewed, Medicaid inappropriately paid $1 million in 2005 for nonroutine medical supplies and therapeutic services that were paid by Medicare; Medicaid paid $6.6 million for routine medical supplies on the same dates as home health services; and States controls to prevent duplicate payments did not eliminate all inappropriate payments. State officials reported that they lacked direct access to Medicare claims data to determine whether Medicare had already paid. The Centers for Medicare & Medicaid Services offered a simplification which involves Medicare sending a copy of the denial of payment notice to the State Medicaid program and to clarify policy on coverage of routing medical supplies under Medicare’s home health prospective payment system.

Report Title: Duplicate Medicaid and Medicare Home Health Payments: Medical Supplies and Therapeutic Services http://oig.hhs.gov/oei/reports/oei-07-06-00640.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 9004

Have Repeated Deficiencies Been Found During Subsequent Surveys of Home Health Agencies?

This study examined which Medicare home health agencies repeated the same deficiency citations. Medicare’s home health benefit provides treatment for beneficiaries who have short or long term illnesses or injuries and who are confined to their homes. This benefit has grown in terms of Medicare beneficiaries receiving home health services, expenditures, and number of home health agencies.

Reviewers found that 15 percent of home health agencies repeated the same deficiency citation on three consecutive surveys. Inadequate patient care plans were the most frequent citations at repeatedly deficient home health agencies. These HHAs received twice as many deficiency citations on past surveys compared to others that did not repeat citations. Among repeatedly deficient home health agencies, most were located in six States and tended to be concentrated in highly populated areas. Reviewers found that deficiency history beyond the most recent survey can be an important indicator of performance on the next survey and can improve identification of at-risk home health agencies. The agency has implemented improvements to the oversight of home health agencies, many of which address the issue of repeated deficiencies.

Report Title: Deficiency History and Recertification of Medicare Home Health Agencies http://oig.hhs.gov/oei/reports/oei-09-06-00040.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 9006

Could Commercially Available Software Generate Resource Use Reports for Physicians?

The study appraised two commercial software packages that allow the generation of episode treatment groups. The study was conducted using Medicare claims data, with a focus on understanding the properties of the grouper algorithms in forming episodes of care and in assigning costs to these episodes. Analyses were conducted with Medicare claims from samples of beneficiaries in several states.

Researchers found that each software package had its own approach for classifying medical care use into episodes of care, and episodes were not typically comparable across the groupers. Considerable variation in cost across episodes within episode type for the algorithms were observed, and there were challenges in using the grouping algorithms to handle common practice patterns under the Medicare payment system.

Report Title: Evaluating the Functionality of the Symmetry ETG and Medstat MEG Software in Forming Episodes of Care Using Medicare Data http://www.cms.hhs.gov/Reports/Downloads/MaCurdy.pdf
Agency Sponsor: CMS-ORDI, Office of Research, Development, and Information
Federal Contact: Fred Thomas, 410-786-6675
Performer: Acumen
PIC ID: 9009

What Facilitated or Hindered Start-Up of a Colorectal Cancer Screening Demonstration Program; What Does Starting the Program in a Community Based Setting Cost?

An in-depth evaluation of the Colorectal Cancer Screening Demonstration Program (CRCSDP) is being conducted to inform current program implementation and support decision making about future screening efforts. The CRCSDP was established to explore the feasibility of implementing a national program for the underserved U.S. population and to learn which settings and which program models are most viable and cost-effective. Five sites were funded to provide colorectal cancer screening to low-income men and women, ages 50 to 64 years, who had inadequate or no health insurance coverage for these services. The evaluation is being conducted for two distinct time periods: program start-up and program implementation.

Report Title (challenges): Facilitators and Challenges to Start-Up of the Colorectal Cancer Screening Demonstration Program http://www.cdc.gov/pcd/issues/2008/apr/07_0205.htm

Report Title (costs): Cost of Starting Colorectal Cancer Screening Programs: Results from Five Federally Funded Demonstration Programs. http://www.cdc.gov/pcd/issues/2008/apr/07_0202.htm
Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Julie Zajac, 404-498-4381
Performer: Research Triangle Institute
PIC ID: 9032, 9037

What Aspects of Independent Living for the Disabled Should Be Included in Future Surveys?

This study used data on how older Americans spend their time to seek insights into enhancements to common survey measures of activity limitation and participation restrictions experienced in late life.

Researchers found several areas of activity participation not commonly measured but that may be important features of independent living. For future surveys, researchers recommended adding several "quality of life" activities like socializing with others, travel and leisure, administrative activities like handling the mail/e-mail, home repairs or arranging for and using services, "helping" activities like volunteering and caring for others, and self care activities like physical exercise and health self-care.

Report Title: The Daily Activities of the Community-Dwelling Elderly: Evidence from the American Time Use Survey http://aspe.hhs.gov/daltcp/reports/2006/cdelderly.htm
Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: William Marton, 202-690-6443
Performer: Urban Institute, University of Medicine and Dentistry of New Jersey
PIC ID: 9039

How Can We Measure Implementation and Use of Health Information Technology?

This study identified and evaluated methods for measuring adoption, implementation, and use of health information technology (HIT) in the United States . The study identified strengths and weaknesses of measures for and measurement of attitudes toward HIT. The study examined HIT implementation progress and impact including variations in incentives and barriers influencing health providers’ decisions to adopt HIT.

Researchers found that surveys had been the principal source of data for measuring levels of HIT adoption. HIT survey research methodologies used were diverse and data definitions of the variables were inconsistent across the numerous studies. A potential and promising measurement source, which may reduce the cost and effort of research involves the use of proxy metrics e.g. vendors and systems certified by the Commission for Certification of Health Information Technology, pay-for-performance HIT programs supported each year by health plans, or participants in a national e-prescribing program.

Report Title: Assessing Success in Health Information Technology Adoption: Metrics and Data; Report may be obtained from Federal Contact
Agency Sponsor: ASPE-OSDP, Office of Science and Data Policy
Federal Contact: Suzie Burke-Bebee, 202-401-8266
Performer: Westat, Inc.; Rockville, MD
PIC ID: 9069

Will Physicians With Small Practices Adopt Electronic Health Records?

The study developed and evaluated a way to accurately predict adoption rates of electronic health records (EHR) in physicians’ small practices (less than ten physicians). This study included an extensive literature review, development of a preliminary economic framework and a roadmap for EHR implementation in physicians’ small practice settings. A microeconomic framework was drafted incorporating EHR adoption factors (barriers and incentives). The model was further developed using Bayesian network learning and incorporated third-party surveys with heterogeneous datasets into the model for conducting ‘what-if’ scenarios to better understand the factors influencing adoption.

Single factor or overly simplified initiatives are likely to have unintended results for facilitating EHR adoption. The model’s relevance or ‘half-life’ is considered to be two-three years.

Report Title: Economic Analysis of Health Information Technology in the Ambulatory Setting; Report may be obtained from Federal Contact

Agency Sponsor: ASPE-OSDP, Office of Science and Data Policy
Federal Contact: Suzie Burke-Bebee, 202-401-8266
Performer: MDM Strategies, Inc.;
PIC ID: 9070

Objective 1.4: Recruit, develop, and retain a competent health care workforce.

Graduate Medical Education: What Are We Paying For?

This study examined the current state of graduate medical education (GME) in the United States . It compiled quantitative data on the financing of GME and resident population demographics. Based on site visits to five academic medical centers, the report described the system of GME financing, administration, and oversight from the perspective of the institutions that train medical residents. The report examined Medicare's role in supporting GME.

Report Title: Graduate Medical Education: What Are We Paying For? http://aspe.hhs.gov/health/reports/06/GradMedicalEdu/index.html
Agency Sponsor: ASPE-OHP, Office of Health Policy
Federal Contact: Lee Wilson, 202-690-6051
Performer: National Opinion Research Center
PIC ID: 8293

Can Development Policies and Practices Increase Retention and Performance of Long-Term Care Workers?

This evaluation assessed the impact of the Home Health Aide (HHA) Partnering Collaborative, a quality improvement initiative implemented at the Visiting Nurse Service of New York. The study examined three areas: patient outcomes; patient service use; and aide job perceptions and retention. The Collaborative was designed to improve the quality of work life and retention of home care paraprofessionals, as well as increase clinicians' and aides' support for patients' self-management and improvement in key activities of daily living.

Report Title: Home Health Aide (HHA) Partnering Collaborative Evaluation: Final Report http://aspe.hhs.gov/daltcp/reports/2007/HHAPartfr.htm
Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Marie Squillace, 202-690-6250
Performer: Visiting Nurse Service of New York
PIC ID: 8904

Goal 2: Prevent and control disease, injury, illness, and disability across the lifespan, and protect the public from infectious, occupational, environmental, and terrorist threats.

Within HHS, multiple operating and staff divisions work together to develop and implement strategies to achieve the goal of preventing and controlling disease, injury, illness, and disability across the lifespan and of protecting the public from infectious occupational, environmental, and terrorist threats.[8]

Objective 2.1: Prevent the spread of infectious diseases.

What Impact Does Housing the Homeless Have on Disease Progression, Transmission Risk, and Access to and Utilization of Medical Care?

The Housing and Health study was a multi-site, multi-agency research collaboration. The goal of the project was to examine the impact of providing housing for people living with HIV who were homeless or at imminent risk of homelessness on their disease progression, their risks of transmitting HIV, and medical care use.

Report Title: Assessing the Role of Housing in HIV/AIDS Prevention; Report may be obtained from Federal Contact
Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Delia Easton, 404-639-1912
Performer: Research Triangle Institute
PIC ID: 7977

Does Water Fluoridation in Indian Country Work?

This study evaluated the effectiveness of water fluoridation in Indian Country. The project focused on health program evaluation and health services research. It proposed an epidemiologic evaluation of the efficacy of water fluoridation in Indian Country. Water fluoridation has succeeded in reducing the prevalence of tooth decay in middle class America . The effectiveness of fluoridation in Indian Country, however is unknown. The excellent opportunity to evaluate a long-term initiative with significant ramifications future direction and resource allocation will not present again in the foreseeable future. The combination of easily current data, widespread initiation of fluoridation and plans for a subsequent national oral health assessment in about five years outline a research plan of action that is efficient and of significant value.

Report Title: The Effects of Fluoridation on the Prevalence of Oral Disease in American Indian Youngsters; Reports may be obtained from Federal Contact
Agency Sponsor: IHS, Indian Health Service
Federal Contact: Lucie Vogel, 301-443-1133
Performer: Dr. Tim Ricks, former IHS DPH Resident, Principal investigator; Dr. Blahut, Residency Mentor; and Dr. Bruce Dye, NCHS, Other Key Mentor
PIC ID: 8095

Do Consumers Understand Latex Condom Labeling?

The study measured and compared consumer understanding of the labeling recommended for latex condoms under FDA's 1998 guidance document, "Latex Condoms for Men, Information for 510(k) Premarket Notifications: Use of Consensus Standards for Abbreviated Submissions," which is found on currently marketed latex condoms, and the latex condom labeling proposed in the 2005 draft guidance document, "Class II Special Controls Guidance Document: Labeling for Male Condoms Made of Natural Rubber Latex." The results of the study were used in FDA's final rulemaking process.

The study found that readers with lower reading levels and those with less education (two variables not highly correlated) had lower comprehension scores than those with higher reading levels. However, there were no differences based on age, race, ethnicity, income, or the type of neighborhoods where the respondents resided. The study found that most participants understood the basic message in the current and proposed labeling that latex condoms help protect against transmission of sexually transmitted infections (>80% correct responses). When comparing equivalent questions between the current and proposed latex condom labeling, for every comparison with a significant difference in rates of comprehension, the difference favored the current latex condom labeling over the proposed latex condom labeling. Most study participants did not understand the more complex messages about the relative degree of protection provided by condoms against different sexually transmitted infections (<30% correct responses). FDA's proposed labeling was lengthier, with considerably more information than current labeling. Shorter and simpler labeling will more likely result in better consumer comprehension.

Report Title: Food and Drug Administration (FDA) Condom Label Comprehension Study: Stage One Report of Findings; Report may be obtained from Federal Contact
Agency Sponsor: FDA, Food and Drug Administration
Federal Contact: Paula Silberberg, 240-276-3234
Performer: M. Davis and Company, Inc.
PIC ID: 8956

What Features Do High Performing Immunization Programs Share?

This study evaluated the most improved and highest performing immunization programs in an effort to identify strategies that could close the gaps in immunization levels. Childhood immunization levels in the United States are at an all-time high; however those levels vary across state and local immunization programs. Using immunization coverage rates from the 2000-2005 National Immunization Surveys, staff identified the seven most improved and 10 highest performing state and local immunization programs and conducted 166 key informant interviews with internal program staff and community partners at these sites. Key informants were asked about the immunization program characteristics and initiatives that contributed to their success, challenges that they faced in conducting their work, and their advice for other programs that were trying to improve or sustain childhood immunization coverage.

Themes that emerged from preliminary analysis of improved and high performing programs included the importance of knowing the community and understanding the barriers, taking advantage of data (especially data from immunization information systems), building relationships and partnerships with healthcare providers and community organizations, and understanding the importance of face-to-face contact with them.

Report Title: Closing the Gap: How do Immunization Programs Achieve and Sustain High Immunization Coverage? http://cdc.confex.com/cdc/nic2008/webprogram/Paper15418.html
Agency Sponsor: CDC, Centers for Disease Control
Federal Contact: Julie Zajac, 404-498-4381
Performer: CDC's Immunization Services Division; ,
PIC ID: 9027

Can School-Based Clinics Provide Mass Vaccination of Children?

This study consisted of constructing a purposive sample of elementary, middle and high schools where mass influenza vaccination campaigns were conducted in 2005 and 2006, and conducting a semi-structured interview with key informants. Recommendations for influenza vaccination of children were expanded in 2007 to include all children 6 months through 18 years, and identifying feasible means of vaccinating children every year is a priority. Because children can be easily reached at school, school-based vaccination clinics may be a feasible approach.

The project achieved vaccination rates of 40%. Children missed at most 20 minutes of class time. This level of success in obtaining parental consent depended on the superintendent, principal, school nurse, and school office staff support. The principle barrier to program sustainability was vaccine cost, specifically for privately insured children who do not meet federal qualifications for the Vaccines for Children Program. Evaluations of the feasibility of billing third party payers for vaccines administered in school-based seasonal influenza vaccination clinics are beginning at this time.

Report Title: Management and Outcome of School-based Mass Vaccination Clinics for Seasonal Influenza: A National Perspective http://cdc.confex.com/cdc/nic2008/webprogram/Paper15856.html
Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Julie Zajac, 404-498-4381
Performer: Staff; Centers for Disease Control and Prevention; National Center for Immunization and Respiratory Diseases
PIC ID: 9028

Do Theoretical Models Aid Health Care Workers Prepare for or Prevent Infectious Disease Outbreaks?

This evaluation determined whether the Models of Infectious Disease Agent Study (MIDAS) program was operating as intended or whether modifications were necessary. MIDAS is a consortium of researchers who develop mathematical and computational tools to assist policymakers and public health professionals prepare the nation for outbreaks of infectious diseases. The evaluation focused on three aspects of the program: adequacy and appropriateness of policies and infrastructure; effectiveness of internal and external collaborations and communications; and perceived value of outputs.

Report Title: Models of Infectious Disease Study (MIDAS) Process Evaluation Report Summary; Report may be obtained from Federal Contact
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Jennifer Villani, 301-451-6446
Performer: Abt Associates, Inc.
PIC ID: 9045

Do State Officials Accomplish Critical Public Health Laboratory Testing Tasks?

This study surveyed State public health laboratory officials to determine the extent to which they carried out the eight critical tasks for public health laboratory testing, as required by the Pandemic Influenza Guidance Supplement to the 2006 Cooperative Agreement, Phase II (the Guidance). Four of these critical tasks require coordination with clinical laboratories. All States reported that their public health laboratories met the first two critical tasks to conduct year-round influenza testing, and to detect and subtype influenza viruses (i.e., the ability to distinguish one influenza strain from another). Although not specifically required by the Guidance, all States reported public health laboratory capability to subtype H5 influenza.

Forty-four States reported that they had no clinical laboratories with the capability to perform H5 subtyping, and another 4 States reported that they did not know if clinical laboratories in their State had H5 influenza subtyping capability. The H5 strain of influenza normally infects birds, but has the potential to cause a human pandemic. No State reported that its public health laboratory met all of the six remaining critical tasks. For the tasks involving public health and clinical laboratories, States reported performing the required activities for public health laboratories to a greater extent than for clinical laboratories.

Report Title: Laboratory Preparedness for Pandemic Influenza http://oig.hhs.gov/oei/reports/oei-04-07-00670.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 8993

Objective 2.2: Protect the public against injuries and environmental threats.

Do Warning Statements on Indoor Tanning Devices Help Consumers?

This study evaluated the effectiveness of warning labels on indoor tanning equipment. The Food and Drug Administration Amendments Act of 2007 requires FDA to conduct consumer testing of labeling information for indoor tanning devices. Forty-eight respondents participated in six consumer focus groups. Changes to the warning label for tanning devices were in progress prior to the request for FDA to conduct this consumer testing, and this testing promoted appropriate changes.

A majority of participants reacted more positively to the alternative warning statement label than the current one because they found it easier to understand. Study participants indicated they found the alternative message to be streamlined, and not as ambiguous as the current label. Most participants said they would be more likely to read the alternative label because the shorter length and bulleted format made it easier to focus on the risks and directives. Participants found the alternative label easier to understand because of its clarity and simplicity, streamlined format, and messaging which made it more attention-grabbing and easier to process. Participants said the alternative statement sent a stronger message about the dangers associated with indoor tanning equipment. The study found the format of the current label to be its greatest weakness. Participants recommended the warning statement label be placed away from other labels on the tanning bed, so as not to detract from the label's importance.

Report Title: Findings from Six Consumer Focus Groups on Indoor Tanning Equipment Warning Statement Label; Report may be obtained from Federal Contact
Agency Sponsor: FDA, Food and Drug Administration
Federal Contact: Judith Arndt, 240-276-3234
Performer: House Market Research/Edge Research
PIC ID: 8954

Objective 2.3: Promote and encourage preventive health care, including mental health, lifelong healthy behaviors, and recovery.

For WISEWOMAN Projects, What Are the Best Practices and How Disseminate Them?

This study identified best practices among WISEWOMAN projects and disseminated these practices for translation among all programs and populations.

The best practices identified in this study included successful recruitment methods, strategies for engaging program participants, delivering lifestyle interventions, facilitating and maintaining behavior change, and retaining participants in the WISEWOMAN program. This study influenced several projects to adopt some of the best practices as a way to improve their programs. We anticipate that the broader implementation of these best practices will enhance our program and help us achieve the greatest public health impact.

Report Title: 1. Best Practices in Implementing Lifestyle Interventions in the WISEWOMAN Program: Adaptable Strategies for Public Health Programs http://www.center-trt.org/index.cfm?fa=wisewoman.toolkit
Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Susan Cleveland, 404-498-1721
Performer: Mathematica Policy Research, Inc.
PIC ID: 7924

What Relationship Exists Among Obesity, Disability, and Other Health Conditions in the Elderly?

This study explored the relationship between excess weight and obesity in a series of cross sectional and longitudinal analyses in an attempt to evaluate these concerns. The authors used data from the 1998-2004 waves of the Health and Retirement Study. The recent rise in the prevalence of obesity and overweight in the U.S. population has raised many concerns about the future. In addition to concerns about the medical costs of treating obesity-related illness, an apparent correlation between obesity and disability has led to concern that the recent declines in rates of disability among the elderly may cease or reverse.

This study found that the risks of developing difficulties with activities of daily living (ADLs) increased with the prevalence of obesity, but the effects appeared to be less dramatic than the effects on the limitations in physical functioning, which can be precursors of ADL disability. Researchers found only a weak relationship between excess weight and the onset of difficulties with instrumental ADLs.

Report Title: Examining the Relationships between Excess Body Weight, Health and Disability http://aspe.hhs.gov/daltcp/reports/2008/weight.htm
Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Hakan Aykan, 202-690-6443
Performer: University of Medicine and Dentistry of New Jersey, Syracuse University and Urban Institute
PIC ID: 8909

What Screening Instruments and Effective Interventions Exist for Older Adults with Depression?

Investigators identified interventions and screening instruments for depression that were suitable for dissemination to older adults through public health and aging services networks. An expert panel examined the findings of this literature review to identify interventions that are ready for translation to older adults in the community. The investigators constructed a logic model, conducted a search of the published scientific literature using explicit criteria, and abstracted and rated the evidence from the 116 eligible articles. Panelists rated the depression care management interventions as effective. Investigators presented the project&rsquo;s findings to the Task Force on Community Preventive Services, an independent, non-governmental, volunteer body of public health and prevention experts.

Report Title: Review of Community-based Interventions to Manage Depression among Older Adults http://www.cdc.gov/pcd/issues/2008/jan/07_0154.htm

Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Julie Zajac, 404-498-4381
Performer: University of Washington, Prevention Research Center
PIC ID: 9016

Objective 2.4: Prepare for and respond to natural and manmade disasters.

Have Public Health Departments Made Progress in Preparedness; What Challenges Remain?

Researchers collected and assessed data on the progress since 2001 of state and local public health preparedness. For the review, researchers compiled and validated data, identified trends, and identified gaps in data availability.

Researchers found that the number of epidemiologists working in emergency response had increased. All states could receive and evaluate reports of urgent health threats 24/7/365 (that is, any day, time, every day) compared to 12 states in 1999. The number of public health laboratories that could detect biological and chemical agents as members of the Laboratory Response Network increased. All states trained public health staff in the Incident Command System compared to only 14 did so in 1999. Remaining were several challenges: difficulty recruiting and retaining qualified epidemiologists and laboratory scientists; limited ability to electronically exchange health data; and difficulty sustaining a system of all-hazards planning, training, exercising, and improving in order to be ready to help at-risk populations. The study highlighted the need for more comprehensive data on public health preparedness. The analysis reinforced the complex nature of public health preparedness, as different jurisdictions must plan for the unique characteristics of their communities and respond to varying threats.

Report Title: Public Health Preparedness: Mobilizing State by State http://emergency.cdc.gov/publications/feb08phprep/

Agency Sponsor: CDC, Centers for Disease Control
Federal Contact: Andrea Baeder, 404-498-4002
Performer: Centers for Disease Control and Prevention; ,
PIC ID: 9094

Goal 3: Human Services: Promote the economic and social well-being of individuals, families and communities.

This Strategic Goal seeks to protect life, family, and human dignity by promoting the economic and social well-being of individuals, families, and communities; enhancing the safety and well-being of children, youth, and other vulnerable populations; and strengthening communities.[9]

Objective 3.1: Promote the economic independence and social well-being of individuals and families across the lifespan.

How Has the Longest-Running Statewide Marriage Initiative Developed and Been Implemented?

This process evaluation analyzed the Oklahoma Marriage Initiative (OMI), which aims to promote and strengthen marriage for low income families, primarily by providing relationship skills education. The OMI uses service delivery infrastructure to provide services and has a growing cadre of volunteer relationship skills instructors trained under the program. It focuses on serving low-income families but is open to all. Interviews were conducted with key program stakeholders and other community leaders. Data, including OMI program records, were analyzed.

Preliminary findings were that the OMI was successfully implemented in high schools, providing relationship skills classes in 90% of OK high schools and serving over 62,500 youth. Successful implementation was linked to a well-identified avenue for implementation: a marriage and family course in high schools, ready access to youth, making training materials easily accessible, and providing teacher support.

Report (1) Title: The Oklahoma Marriage Initiative: An Overview of the Longest-Running Statewide Marriage Initiative in the U.S. http://aspe.hhs.gov/hsp/06/OMI/index.htm

Report (2) Title: How Was the Oklahoma Marriage Initiative, The Longest-Running Statewide Marriage Initiative in the U.S. , Developed and Implemented? http://aspe.hhs.gov/hsp/06/OMI/index.htm

Report (3) Title: Putting Marriage on the Agenda: How Oklahoma Laid the Foundation for Its Marriage Initiative http://aspe.hhs.gov/hsp/06/OMI/index.htm
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Jennifer Burnszynski, 202-690-8651
Performer: Mathematica Policy Research, Inc.
PIC ID: 8533, 8533.1, 8533.2

What Child Care Services Exist for Low-Income Households?

This project examined low-income child care after welfare reform, studying the low-income child care market in 25 communities in 17 States with a sub-study examining the family child care market in 5 neighborhoods drawn from these communities. A survey was conducted of 2,500 low-income families in the same 25 communities to determine how child care decisions are made and to study the relationship of child care subsidies to family child care choices.

Report Title: National Study of Child Care for Low-Income Families http://www.acf.hhs.gov/programs/opre/cc/nsc_low_income/index.html
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Seth Chamberlain, 202-260-2242
Performer: Abt Associates, Inc.
PIC ID: 8547

What Do Sites Look Like That Implement Healthy-Marriage Services for Unmarried Expectant Couples?

This implementation evaluation examined the operation of the healthy marriage services for expectant low-income couples as part of the Building Strong Families (BSF) project. BSF project sites modified a program to incorporate BSF principles, added BSF practices as a program within an array of multiple programs, or established a new entity with BSF methods as the sole program.

Sites implementing healthy-marriage services were successful at enrolling couples during pregnancy or within three months of delivery. The maternal health care system was the most common source for BSF recruitment (e.g. hospitals, clinics and doctors offices serving pregnant women or newborns and their mothers). The majority of couples were African American (58 percent), about a quarter was Hispanic, and about 14 percent were White. Seventy-two percent of BSF couples reported that they cohabited all or most of the time. Two-thirds of men and women were between 20 and 29 years of age. Over half of couples had no children from other relationships. Three-quarters of the fathers reported being employed at enrollment and 93 percent of them reported earnings in the year prior to enrollment. Sites had been successful in involving couples in the program. Ensuring regular attendance by couples at group meetings was a substantial challenge for some sites. Lack of participation was often due to scheduling problems. Couples found the healthy marriage services appealing and were enthusiastic about the facilitators and interacting with other couples. They reported that the program helped them improve their communication skills, and, ultimately, their relationships with their partners and children.

Report Title: Implementation of the Building Strong Families Program http://www.acf.hhs.gov/programs/opre/strengthen/build_fam/reports/bsf_program/bsf_program.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Seth Chamberlain, 202-260-2242
Performer: Mathematica Policy Research, Inc.
PIC ID: 8828

How Can We Help Ex-prisoners Find and Keep Employment?

This random assignment evaluation examined impacts of the Center for Employment Opportunities (CEO) Prisoner Reentry Program. CEO uses a distinctive transitional employment model. After a four-day job readiness class, participants are placed in temporary, minimum wage jobs with work crews that perform maintenance or repair work under contract to city and state agencies. Participants are paid daily. Within weeks, they receive help finding permanent jobs and follow-up services to promote employment retention. The evaluation targeted a key subset of CEO's population - ex-prisoners who showed up at the program after being referred by a parole officer. CEO is one of four sites in the Enhanced Services for the Hard-to-Employ Demonstration and Evaluation project.

CEO generated a large, but short-lived increase in employment covered by unemployment insurance (UI). During the early months of follow-up, when many in the program group worked in CEO crews (jobs covered by UI), the employment rate for the program group was 30 to 40 percentage points higher than for the control group. However, the program group's employment rate dropped as people left CEO jobs, and the difference between the groups disappeared by the end of the year. Nevertheless, there was a small but statistically significant decrease in felony convictions and incarceration for new crimes during Year 1. Among those who came to CEO within three months after release, the program produced statistically significant decreases in parole revocations, felony convictions, re-incarceration for new crimes, and re-incarceration. Effects on these measures are rarely seen in rigorous evaluations. The pattern of employment impacts is similar to that for the full sample, though the impacts seem to have declined more slowly for this subgroup.

Report Title: The Center for Employment Opportunities Prisoner Reentry Program: Early Impacts from a Random Assignment Evaluation http://www.acf.hhs.gov/programs/opre/welfare_employ/enhanced_hardto/reports/transitional_jobs/transitional_jobs.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Girley Wright, 202-401-5070
Performer: MDRC
PIC ID: 8832

How Can We Improve Employment and Other Outcomes for Low-Income Parents and Others Who Face Serious Barriers to Employment?

The Enhanced Services for the Hard-to-Employ project is evaluating four diverse strategies designed to improve employment and other outcomes for low-income parents and others who face serious barriers to employment. The evaluation utilizes an experimental, random assignment design and includes three main parts: an implementation study, an impact study, and a financial benefit and costs study.

• Rhode Island Working Toward Wellness Project aims to improve employment outcomes through simultaneously providing employment services and an intensive telephonic care management program for single parents who are Rhode Island Medicaid recipients with serious depression.
• Kansas/Missouri Two-Generation Early Head Start Evaluation program provides enhanced employment services to TANF recipients who are Early Head Start participants. It aims to improve the economic circumstances of parents and the well-being of their children.
• Test of Alternative Employment Strategies for TANF Recipients in Philadelphia involves two alternative employment strategies for long-term welfare recipients in Philadelphia, one emphasizing upfront assessment and pre-employment services to remove barriers to employment, and the other based on paid transitional employment.
• Success Through Employment Preparation program includes extensive participant assessments, design of a plan to address the participant's barriers, and counseling with behavioral health specialists, as well as ongoing case management meetings.
• Transitional Work Corporation administers the transitional employment program in which participants receive intensive job-readiness activities, a transitional job, professional development activities, permanent, unsubsidized jobs and job retention services, including financial bonuses for retaining employment.
• Center for Employment Opportunities Evaluation, New York City, is a comprehensive transitional employment program for ex-prisoners in which participants complete a job readiness class, are placed in paid jobs, paid minimum wage, meet with job coaches to identify permanent jobs, and participate in a fatherhood and family relationships program.

Report Title: Four Strategies to Overcome Barriers to Employment: An Introduction to the Enhanced Services for the Hard-to-Employ Demonstration and Evaluation Project http://www.acf.hhs.gov/programs/opre/welfare_employ/enhanced_hardto/reports/four_strategies/four_stategies.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Girley Wright, 202-401-5070
Performer: MDRC
PIC ID: 8833

What Long-Term Employment Patterns Do Youth Too Old for Foster Care Exhibit?

Looking at California, North Carolina and Minnesota, this study assessed employment outcomes for former youth through age 24 by linking child welfare, Unemployment Insurance (UI) and TANF administrative data outcomes for former youth through age 24 in three states. A primary task for youth in transitioning to adulthood is sustaining employment to be self-sufficient. Studies of former youth who age out of foster care find that they generally experience unstable employment patterns and earn low incomes between ages 18 and 20. However, less is known about whether these youths&rsquo; initial patterns of employment instability and low earnings persist. Descriptive, multivariate, and trajectory analysis techniques were employed to describe employment patterns.

Low rates of employment persisted through age 24; Low earnings persisted through age 24 though few received TANF benefits; and youth show four patterns of connectedness to the workforce that may provide insights to program planners considering how to best tailor services to youths&rsquo; needs. Four employment patterns were observed consistently across the three states studied. Consistently Connected Youth maintained high probabilities of employment between the ages of 18 and 24 and had earnings that were comparable to youth nationally. These youth represented 25% of former foster youth in California, 22% in Minnesota and 16% in North Carolina. Later Connected Youth got a slow start in the labor market, but steadily increased their probability of employment throughout their early twenties. This group included 20% of youth who aged out of foster care in California, 21% in Minnesota and 16% in North Carolina. Never Connected Youth had low probabilities of employment and hardly any earnings at time between ages 18 and 24 and did not have earnings prior to age 18. This group included 33% of former foster care youth in California, 29% in Minnesota and 22% in North Carolina. Finally, Initially Connected Youth began making connections to the workforce prior to adulthood and maintained a high probability of employment through their late teens, but their probabilities of employment then declined in their early twenties. This pattern was seen in 22% of youth who aged out of foster care in California, 29% in Minnesota and 46% in North Carolina. Some of these youth may have moved to jobs not covered by unemployment insurance wage data (such as military employment) or moved between states in ways we cannot track in the data.

Report Title: Coming of Age: Employment Outcomes for Youth Who Age Out of Foster Care Through Their Middle Twenties http://aspe.hhs.gov/hsp/08/fosteremp/index.html
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Laura Radel, 202-690-5938
Performer: Urban Institute
PIC ID: 8835

Do Programs Use Vouchers and Other Indirect Funding Mechanisms to Improve Client Choices?

This study culminated an examination titled “ Understanding Vouchers as a Tool to Expand Client Choices in HHS Programs.” The study represented a first step toward better understanding the role of indirect funding mechanisms such as vouchers to increase client choice of service provider and expanding the array of providers to include faith-based and community organizations (FBCOs).

Voucher use varied greatly between the Temporary Assistance for Needy Families (TANF) and the Child Care Development Fund (CCDF) programs. The federal framework for CCDF required the use of vouchers, and as a result most states offered certificates to families that could be redeemed with providers of their choice. In TANF, legislative authority is given for voucher use, but there is no requirement to use them and we found only a few examples of their use. Some TANF programs integrated client-choice concepts into their contract-based service delivery system by offering clients a choice from among a set of contracted providers. Such models preserved an element of financial stability for providers who depended on the consistency of contracts to create the organizational and staffing capacity. The use of vouchers alone did not maximize client choice; program policies and procedures influenced choice. The value of vouchers, the provision of information to allow clients to make informed decisions, provider qualifications for program participation, the voucher-funded service and the client&rsquo;s interest in receiving the service all affected client choice. While program officials recognized and appreciated the role of FBCOs in providing child care and services to the low-income, they did not seem to consider vouchers as a means of expanding the role of FBCOs in the service delivery network.

Report Title: Using Vouchers to Deliver Social Services: Considerations Based on the CCDF and TANF Program Experiences http://aspe.hhs.gov/hsp/07/vouchers/experiences/
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Alana Landey, 202-401-6636
Performer: Mathematica Policy Research, Inc.
PIC ID: 8860

Does the Federal Faith-Based and Community Initiative Work?

This study consisted of several topics: descriptive studies of Faith-Based and Community Initiative (FBCI) innovations such as intermediary models, technical assistance, capacity building and public-private collaborations; descriptive examinations of faith-based organizations' service parts, funding, accessibility to clients, barriers, and the faith orientation of organizations and clients; studies focusing on participant outcomes; and an analysis of the legal and regulatory issues that govern the FBCI and influence the social service environment.

Report Title: Innovations in Effective Compassion: Compendium of Research Papers presented at the Faith-Based and Community Initiatives Conference on Research, Outcomes, and Evaluation http://aspe.os.dhhs.gov/fbci/comp08/index.htm
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Alana Landey, 202-401-6636
Performer: Research Triangle Institute
PIC ID: 8861

How Can We Best Study Migrant and Seasonal Head Start Programs, Children and Families?

This study first reviewed the challenges of studying Migrant and Seasonal Head Start (MSHS) programs and then piloted a selection of assessment and research methods. Challenges highlighted included family mobility, bilingual language development, geographical distribution, age range served (i.e., zero to five years) and extensive program schedule variations (e.g., start dates range from Spring to Winter; length of open seasons ranges from 6 weeks to 12 months).

Methods piloted included interviews of staff, teachers, and parents; direct assessment and ratings of the children; and tracking of a small sample to ascertain viable options for longitudinal follow-up. Additional piloting work, building on these early efforts, will be required for identification of reliable and practical methods for studying MSHS programs and families.

Report Title: Migrant and Seasonal Head Start Research Design Development Project, Executive Summary, July 2004 http://www.acf.hhs.gov/programs/opre/hs/migrant_designproj/index.html

Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Wendy DeCourcey, 202-260-2039
Performer: Westat, Inc.
PIC ID: 8864

How Did Sites Apply Revised Work Participation Rate Requirements?

This follow-up study sought to determine what changes five sites (in Arizona, Georgia , Missouri, New Jersey, and Wisconsin) were making toward meeting more stringent work participation requirements as set forth in the Deficit Reduction Act (DRA). This study was based on information gathered in mid-2007 and represents early efforts to address DRA requirements.

Four of the five states studied low had pre-DRA work participation levels so that meeting the new requirements was challenging. Researchers found the following state level changes: making sanctions provisions more stringent, targeting the hard-to-employ, moving some clients to solely state funded programs so that those clients would not count in the work participation rate calculation, making engagement in work activities an eligibility requirement, and updating data systems. Changes initiated at the local level included designating specialized staff to monitor work participation to ensure missed hours were documented and to free up case manager staff to focus on working directly with clients, making greater efforts to identify cases with significant barriers to work, and initiating efforts to more closely monitor caseworker performance.

Report Title: Local Implementation of TANF in Five Sites: Changes Related to the Deficit Reduction Act http://www.acf.hhs.gov/programs/opre/welfare_employ/local_impl/index.html
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Marc Fucello, 202-401-5750
Performer: The Lewin Group
PIC ID: 8867

What Is the Status of the Federal Assets for Independence Program?

Staff updated the status of the Assets for Independence (AFI) program as of the end of the program's 6^th and 7^th years. AFI enables community-based nonprofits and government agencies assist low-income individuals and families with the information and resources necessary to achieve economic self-sufficiency by accumulating assets. The community-based AFI projects provide financial management education as well as other supportive services aiding low-income participants the opportunity to save earned income in matched-fund savings accounts called individual development accounts (IDAs), for the expressed purposes to acquire such appreciating economic assets as a first home, a small business, or enrollment in post-secondary education. The report, developed by the Office of Community Services based on information provided by AFI program grantees, is descriptive in nature; it does not present recommendations.

As of the end of the program&rsquo;s seventh year, the Administration for Children and Families was supporting more than 368 AFI projects across the nation; nearly 44,000 people had opened IDA through the program; more than $38,800,000 had been deposited into the IDAs, and more than 13,000 participants had used their IDA savings and matching funds to purchase an appreciating economic asset.

Report Title: Assets for Independence Program: Status at the Conclusion of the Seventh Year http://www.acf.hhs.gov/programs/ocs/afi/seventh_interim_report.pdf
Report Title: Assets for Independence Program: Status at the Conclusion of the Sixth Year; Report may be obtained from Federal Contact

Agency Sponsor: ACF-OCS, Office of Community Services
Federal Contact: Jim Gatz, 202.401-5284
Performer: Office of Community Services, (OCS), ACF; Washington, DC
PIC ID: 8629, 8868

Do State And Local Agencies Enable Individuals and Families To Become Self-Sufficient?

The study examined the extent to which the Community Services Block Grant (CSBG) Network measured and reported performance outcomes of Community Action. The researchers surveyed the 50 States, District of Columbia, Puerto Rico and CSBG eligible entities nationally and analyzed how the CSBG Network measured and reported outcomes.

As a result of CSBG assistance, 142,641 low-income participants obtained child care in order to acquire or maintain employment, over 1.1 million low-income participants obtained food assistance, over 230,000 low-income households were helped to receive $160 million in tax credits, 13,298 low-income households were helped to obtain over $17 million in child support payments, 7,212 low-income households opened accounts and saved $4 million, just under 600,000 housing units were improved or preserved, and 52 million hours of service were volunteered to CSBG eligible entities. The use of the 12 National Performance Indicators enabled State and local CSBG eligible entities receiving CSBG funds to document program outcomes in a manner that captured the scope and depth of anti-poverty work performed in more than 1,000 communities across the nation.

Report Title: Community Services Block Grant Program Performance Measurement Report to Congress Fiscal Year 2005 http://www.acf.hhs.gov/programs/ocs/csbg/index.html
Agency Sponsor: ACF-OCS, Office of Community Services
Federal Contact: Anita Wright, 202-690-5660
Performer: National Association for State Community Services Programs
PIC ID: 8881

With What Frequency Do Children In Contact With the Child Welfare System Qualify for Social Supplemental Security Income?

The child welfare system was reviewed. Children who have been placed in foster care have been found to be at a high risk of having a medical, social or behavioral disability. Researchers examined access to services for children in the child welfare system, and Supplemental Security Income (SSI) eligibility among children living in out-of-home placements in the child welfare system, using data from the National Survey of Child and Adolescent Wellbeing.

The analysis confirmed that a large number of children living in foster care may be eligible for SSI. The rates of SSI eligibility vary depending on children&rsquo;s age, race/ethnicity, gender and locality of placement.

Report Title: Estimates of Supplemental Security Income Eligibility for Children in Out-of-home Placements http://www.acf.hhs.gov/programs/opre/abuse_neglect/nscaw/reports/est_suppl/est_suppl.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Lauren Supplee, 202-401-5434
PIC ID: 8882

How Well Has the Older Americans Act Been Implemented?

This study assessed the Older Americans Act&rsquo;s supportive service (Title III-B) program and its role in planning, coordinating, and providing community service for older people. The Older Americans Act (OAA) was established in 1965 to help provide older Americans with the resources they need to live independently in the community for as long as possible. The OAA works through the Aging Network, the system of state agencies, called State Units on Aging (SUAs), area agencies of aging (AAAs), and local community service providers that plan, coordinate and deliver services. The project evaluated the Aging Network&rsquo;s involvement with key services: case management, information and assistance, personal care, chore services, homemaker services, transportation, and assisted transportation services.

Researchers concluded that the Title III-B program was a key part of the Older Americans Act and it was performing as intended; assisting vulnerable older adults to remain independent and active in their communities. The percent of program participants that were at high risk of institutionalization increased. People who received home care services were older (aged 75+), lived alone, and had three or more Activities of Daily Living (ADL) impairments. Users of transportation services relied heavily on these services, with over half reporting that the service was used for at least 75% of their trips. Most of these participants lived alone and were at least 75 years old. In addition to reaching the program&rsquo;s target population, participants were highly satisfied. For example, over 80% of survey respondents rated home care services as positive. Finally, Title III-B program funds were highly leveraged. Depending on the service, the study found that for every $1 of Title III-B funding, local programs leveraged $2 to $6 from other sources.

Report Title: Final Report for the Evaluation of Select Consumer, Program, and System Characteristics under the Supportive Services Program (Title III-B) of the Older Americans Act http://www.aoa.gov/about/results/index.aspx
Agency Sponsor: AOA, Administration on Aging
Federal Contact: Jennifer Klocinski, 202-357-0146
Performer: Research Triangle Institute
PIC ID: 8883

Does Language and Literacy Training Work?

This experimental study assessed the effectiveness of three different language and literacy interventions, implemented in child care centers in Miami-Dade County, FL, that served children from low-income families.

Within six months of training all three language/literacy interventions had produced significant impacts on teacher behaviors and interactions with children that supported language and literacy development. By 18 months after training, these impacts were more pronounced and there were significant impacts in classroom activities that involved supports for literacy development. Two of the three interventions had significant impacts on all four measures of emergent literacy outcomes for children, i.e., definitional vocabulary; phonological awareness; knowledge and understanding; and an index of early literacy. The impact of the two interventions was much greater for children in classrooms with Spanish-speaking teachers than for children in classrooms with English-speaking teachers. These two interventions brought children close to or above the national norms on three of the four measured outcomes. Training and mentoring eliminated differences found — before the intervention — between less and more educated teachers&rsquo; child literacy practices.

Report Title: Evaluation of Child Care Subsidy Strategies: Findings from an Experimental Test of Three Language/Literacy Interventions in Child Care Centers in Miami-Dade County; Report may be obtained from Federal Contact
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Ivelisse Martinez-Beck, 202-690-7885
Performer: Abt Associates, Inc.
PIC ID: 8884

Has a Program Serving Fragile Families, Including Fathers Succeeded?

This evaluation of the Partners for Fragile Families (PFF) demonstrations included implementation and outcome analyses and case studies. The PFF projects tested new ways for state-run child support enforcement programs and community-based organizations to work together to help young fathers obtain employment, make child support payments and learn parenting skills. Services were targeted at young, never-married, non-custodial parents who did not have a child support order in place.

The study found that the proportion of PFF participants with child support orders increased considerably, the amount of child support paid increased, and the average number of months that PFF participants made a child support payment increased. Most PFF participants fared poorly in the labor market, but child support outcomes were more positive, especially in light of the modest employment gains. Program eligibility criteria needed to avoid being too narrow; child support services, including education about the system, were critical; and linkage to successful programs helped as did providing a comprehensive range of services.

Report Title: Partners for Fragile Families Demonstration Projects:

Employment and Child Support Outcomes and Trends

http://aspe.hhs.gov/hsp/07/PFF/
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Jennifer Burnszynski, 202-690-8651
Performer: Urban Institute
PIC ID: 8885.3

How Can Relationship Skills Education, Financial Literacy And Asset Development Fields Collaborate To Improve Family Well-Being?

Researchers and practitioners from the marriage and relationship skills education, financial literacy and asset development fields met to discuss the relationship between family strengthening and financial practices to improve programmatic efforts and encourage collaborative policies designed to address long-term family and economic stability. Participants explored future research and collaborative needs and opportunities.

Participants agreed on the need for comprehensive, longitudinal research; identification of appropriate outcome measures; definitions of success in these fields; discovering how services fit together; and more sharing of evidence-based practices and collaborations on smaller scales in order to gain more understanding and establish relationships and referral networks.

Report Title: Marriage Education, Financial Literacy, and Asset Development Roundtable Meeting Summary
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Jennifer Burnszynski, 202-690-8651
Performer: Research Triangle Institute
PIC ID: 8886

How Do People Served by the Target Populations of Healthy Marriage Programs Vary By State and Region?

Policy-relevant statistics on marriage, divorce, childbearing, and low-income children were compiled, including national and regional benchmark estimates to help identify the distinctive characteristics of each state, information policymakers can use to better understand marriage patterns in their state and to design an approach that best serves the needs of their local populations. Since healthy marriage programs aim to serve a broad mix of people, including expectant unmarried parents, low-income married parents, high school students, engaged couples, single adults, and other groups, the design and content of the programs varies substantially. Researchers drew on survey data and vital statistics to provide policymakers and program operators with a broad range of state-level statistical information to use to better assess the characteristics and needs of their state populations, identify high-priority target populations, and make informed decisions about the design and implementation of their healthy marriage programs.

The study provided individual analyses of each state on indicators including the number and percent of births to unmarried women, the number of divorces granted and the divorce rate, and the distribution of children living in low-income families by urban and rural and racial/ethnic group. For example, in the most populous state, California, the percentage of adults who were divorced was 14 percent, compared to the national average of 15 percent. The percentage of adults who were divorced was 16.0 percent in rural areas and 14 percent in urban areas. California had nearly 4 million children living in families with incomes below 200 percent of the federal poverty level in 2006. Just over half of these children were living with married parents. The state&rsquo;s largest group of low-income children lived with married parents in urban areas. Sixty-seven percent of low-income children in California were Hispanic, 15 percent were white, and 8 percent were African American.

Report Title: : The Marriage Measures Guide of State-level Statistics; Report may be obtained from Federal Contact
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Jennifer Burnszynski, 202-690-8651
Performer: Mathematica Policy Research, Inc.
PIC ID: 8887

What Do We Know About Promoting Asset Development?

Poor Finances is a series of examinations of various aspects of poverty, asset building, and social policy focusing on asset accumulation and asset-based policies for low-income individuals and families. One inquiry determined the effects of varying asset limits across state TANF programs, treatment of different types of assets, and state efforts to encourage asset accumulation among TANF recipients. Another identified data sets that were the most reliable and informative sources for understanding low-income households' assets and liabilities. A third synthesized current research and other information on the assets and liabilities of low-income households. A fourth provided a policy-oriented conceptual framework that has the potential to explain saving and asset accumulation across the entire population and to account for the low levels of saving and asset accumulation in the low-income population.

Report Title: Determinants of Asset Building http://aspe.hhs.gov/hsp/07/PoorFinances/determinants/index.html
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Linda Mellgren, 202-690-6806
Performer: Urban Institute
PIC ID: 8888.3

What Do We Know About Welfare Time Limits?

This study examined what has been learned about welfare time limits: the implementation of state policies, families affected by time limits, the effects of time limits on employment and welfare receipt, and the circumstances of families whose welfare cases have been closed because they reached a time limit. An earlier study included a survey of state welfare administrators to obtain information on states&rsquo; time-limit policies and their experiences. Through this study, findings from the earlier survey were updated. Researchers categorized States according to their time limit policies and how they implemented these policies.

The study found that about half of all assistance cases were subject to the federal time limit. Families who had reached 60 months were headed by individuals who were older, on average, had lower levels of education, were more likely to have a disabled family member, and more likely living in public housing or receiving a rent subsidy than individuals who had accumulated fewer months. In a given month in FY 2005, approximately 4.5 percent of assistance cases (and 8 percent of all adult-headed families) had received at least 60 months of assistance. No state had reached the 20 percent cap for granting extensions beyond 60 months due to hardships by FY 2005.

Report Title: Welfare Time Limits: An Update on State Policies, Implementation, and Effects on Families http://www.acf.hhs.gov/programs/opre/welfare_employ/sep_state/reports/time_limits/time_limits.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Michael, Dubinsky, 202-401-3442
Performer: The Lewin Group and MDRC
PIC ID: 8892

How Does the Assets for Independence Program Impact Participants' New Worth, Employment, Income and Means-Tested Benefits Receipt?

This evaluation examined the Assets for Independence (AFI), the largest individual development account program administered by the Administration for Children and Families. Individual development accounts are personal savings accounts targeted to low-income persons that encourage participants to save for types of asset building, typically home purchase, post-secondary education, or small business start-up. Savings is encouraged by matching the deposits of participants, providing them with financial education, and other forms of support.

Preliminary evidence indicated that AFI participants were more likely to purchase a home, advance their education, or start a business than they would have been, absent the program.

Report Title: Assets for Independence Act Evaluation, Impact Study: Final Report http://www.acf.hhs.gov/programs/ocs/afi/AFI_Final_Impact_Report.pdf
Agency Sponsor: ACF-OCS, Office of Community Services
Federal Contact: James Gatz, 202-401-5284
Performer: Abt Associates Inc, Bethesda; Bethesda, MD
PIC ID: 8914

Does an Innovative Program Combining Education and Mentoring Improve Employment and Other Outcomes for the Rural Poor?

This study examined impacts of Building Nebraska Families (BNF), part of a broader Rural Welfare-to-Work Strategies (RWtW) demonstration evaluation assessing whether innovative programs could improve employment and other outcomes for rural low-income people. BNF provided individualized education, mentoring, and service coordination support with the goal of improving TANF clients&rsquo; basic life skills, self-sufficiency, family functioning, and well-being.

BNF improved employment near the end of the 30-month follow-up. The program group was significantly more likely to retain employment and advance in their jobs. There were not significant impacts on sample members&rsquo; earnings or public assistance receipt, but BNF significantly improved family income and reduced poverty. More disadvantaged program group members worked significantly more months and hours than more disadvantaged control group members. The more disadvantaged program group was significantly more likely to work in higher-paying jobs with better benefits, to retain employment, and to move to a better job. BNF led to significant, robust impacts on earnings, with the impacts growing.

Report Title: Teaching Self-Sufficiency: An Impact and Benefit- Cost Analysis of a Home Visitation and Life Skills Education Program. http://www.acf.hhs.gov/programs/opre/welfare_employ/rural_wtw/reports/te... Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Michael Dubinsky, 202-401-3442
Performer: Mathematica Policy Research, Inc.
PIC ID: 8915

How Best Examine the Effects of Hurricane Katrina?

Researchers examined the effects of Hurricane Katrina to identify ways of answering four questions: where did people go when Hurricane Katrina struck in August 2005 (migration and housing), how were they doing (income and employment), what were their needs for social service program support, and how did the disaster affect the programs of the Administration for Children and Families?

Report Title: Understanding the Consequences of Hurricane Katrina for ACF Service Populations: A Feasibility Assessment of Study Approaches http://www.acf.hhs.gov/programs/opre/other_resrch/feasibility_assmt/reports/katrina_final/katrina_final.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Mark Fucello, 202-401-5750
Performer: Urban Institute
PIC ID: 8922

How Did Hurricane Katrina Affect Agency for Children and Family Program Beneficiaries?

Researchers undertook a literature review of works that addressed the human, social, and economic dimensions of the storm. Researchers identified a mix of studies, articles, conference proceedings, reports, speeches, essays, opinion pieces, issue analyses, and fact sheets.

Report Title: Studying the Consequences of Hurricane Katrina for ACF Service Populations: Annotated Bibliography http://www.acf.hhs.gov/programs/opre/other_resrch/feasibility_assmt/reports/katrina/katrina_title.html
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Mark Fucello, 202-401-5750
Performer: Urban Institute
PIC ID: 8921

How Have Healthy Marriage Initiatives Been Implemented?

This study evaluated implementation of healthy marriage demonstrations. The study examined three different approaches to implementing a healthy marriage initiative and showed how various organizations leveraged their strengths and abilities to get their projects up and running.

The three programs exhibited several similarities: all provided 10-12 hours in training that emphasized relationship skills and other attributes of successful couples and families; all had a graduation that participants could achieve by attending 10-12 hours of training; all provided referrals to other agencies for problems ranging from employment and housing to drug rehabilitation and education; all built on other local programs; each sponsor began with partners, community contact and support, and an understanding of the grant requirements; each had experience with recruiting and delivering services to the community; all three took considerable time to start serving people and all involved changes in partnerships; nonetheless, each site made progress toward its goals and served at least one third of the participants it proposed to reach; and all three programs are reaching low-income individuals and couples.

Report Title: Piloting a Community Approach to Healthy Marriage Initiatives in Three Sites: Chicago, Illinois, Boston, Massachusetts and Jacksonville, Florida; Report may be obtained from Federal Contact
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Erica Zielewski, 202-401-5995
Performer: Research Triangle Institute
PIC ID: 8923

How Do Employers View the Low-Wage Workforce?

This nationally representative survey of private-sector employers sought information about employers&rsquo; practices and workplace policies relevant for less-skilled workers. The survey gathered information on employer characteristics, job requirements, wages and benefits, hiring practices, and potential for advancement, focusing on employers&rsquo; most recently filled jobs noncollege jobs (jobs that required no more education than a high school degree or GED).

The study found that interpersonal skills and attitude were important factors in hiring. Willingness among noncollege employers to hire former welfare recipients was high. Many noncollege jobs were not unskilled but required prior job experience or skills training. Only a third of noncollege jobs were entry-level and thus readily accessible to job seekers with minimal prior experience and training. The median wage for noncollege jobs was $11 an hour, with jobs that required more skill and experience paying substantially better than other noncollege jobs. About two-thirds of recently filled noncollege jobs offered health insurance and pension plans to workers. An additional fifth of these jobs were in firms that provided these benefits to some workers, but the recently hired worker was ineligible due to short tenure or too few hours. More than two-thirds of recently filled noncollege jobs offered paid leave, although only half allowed use of this paid leave to care for a sick child or family member. few employment problems were reported for recently hired workers; the most common was tardiness, reported for nine percent of workers. The majority of noncollege jobs were reported by employers to offer an &lsquo;excellent&rsquo; or &lsquo;good&rsquo; chance of promotion. Many workers on the job for at least six months had been promoted or had received raises. The median raise received was $1 per hour on a median starting wage of $9.50. Entry-level jobs with higher promotion possibilities included those in for-profit firms (versus nonprofits) and those with employers that offered formal training

Report Title: Understanding the Demand Side of the Low-Wage Labor Market: Final Report. http://www.acf.hhs.gov/programs/opre/welfare_employ/unders_demand/reports/wage_labor_mkt/wage_labor_mkt.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Mark Fucello, 202-401-5750
Performer: Urban Institute
PIC ID: 8924

How Can We Better Assist Temporary Assistance for Needy Families Program Recipients Who Have Disabilities Get and Keep Jobs?

This process and implementation analysis in nine sites provided information on innovative strategies used by local Temporary Assistance for Needy Families (TANF) and Vocational Rehabilitation agencies to assist TANF recipients with disabilities obtain and maintain employment.

Some key ingredients to a successful partnership were small caseload sizes, highly-trained staff, and formal interagency agreements and performance standards. On creating work opportunities, the research team found that work opportunities came in several forms - unpaid work experience, subsidized employment, and unsubsidized transitional employment. They shared several common features, including specialized and comprehensive assessments, support for specialized treatment, intensive personal and employment support, and flexible and progressive paths to employment, with small caseloads and highly trained staff critical to the success of these programs. The research team found that regardless of structure, work programs targeted to TANF recipients living with a disability shared several common features, including specialized and comprehensive assessments, support for specialized treatment, intensive personal and employment support, and flexible and progressive paths to employment. Program administrators believed that small caseloads and highly trained staff were critical to the success of these programs. While there was no evidence to suggest that one approach to assessment is better than another, an important lesson learned from these initiatives is that assessments serve different purposes and an assessment approach should be chosen to fit the purpose for which it is intended. The research team examined the background and policy context of TANF recipients with disabilities and reviewed of initiatives and promising practices.

Report Title: Identifying Promising Practices for Helping TANF Recipients with Disabilities Enter and Sustain Employment, 2006-2008 http://www.acf.hhs.gov/programs/opre/welfare_employ/identify_promise_prac/index.html
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Tim Baker, 202-260-6165
Performer: Mathematica Policy Research, Inc.
PIC ID: 8925

Do Welfare Recipients Receiving Career and Job Search Assistance Stay Employed and Get Promoted?

The study assessed the interim impacts of the Enhanced Job Club (EJC) program in Los Angeles, California. EJC was part the Employment Retention and Advancement (ERA) project, which was designed to identify and test innovative models designed to promote employment stability and wage progression among welfare recipients and other low-income groups. EJC targeted applicants of Temporary Assistance for Needy Families (TANF) cash assistance benefits who were unemployed and who were in Greater Avenues for Independence (GAIN) program, California&rsquo;s mandatory Welfare-to-Work program.

The program did not increase employment retention or advancement over the follow-up period. Nor did it did affect public assistance receipt or income levels over the follow-up period.

Report Title: The Employment Retention and Advancement Project: A Comparison of Two Job Club Strategies: The Effects of Enhanced Versus Tradition Job Clubs in Los Angeles http://www.acf.hhs.gov/programs/opre/welfare_employ/employ_retention/reports/era_la/era_la.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Michael Dubinsky, 202-401-3442
Performer: MDRC
PIC ID: 8926

Does a Pre-post Employment Program Help Unemployed Temporary Assistance for Needy Family Applicants Keep Jobs and Advance?

This study targeted applicants of Temporary Assistance for Needy Families cash assistance benefits who were unemployed. This study, conducted in Salem, Oregon, was part the Employment Retention and Advancement project, designed to identify and test innovative models designed to promote employment stability and wage progression among welfare recipients and other low-income groups. The Job Opportunities and Basic Skills Training (JOBS) program the regular welfare-to-work program offered job search and related support; the other, Valuing Individual Success and Increasing Opportunities Now (VISION) provide more than basic pre-employment services.

During the first year and a half of follow-up, neither JOBS nor VISION increased employment retention or advancement above the control group. VISION increased public assistance receipt but had no statistically significant effect on income.

Report Title: The Employment Retention and Advancement Project: Results from the Valuing Individual Success and Increasing Opportunities Now (VISION) Program in Salem, Oregon http://www.acf.hhs.gov/programs/opre/welfare_employ/employ_retention/reports/vision_salem/vision_salem.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Michael Dubinsky, 202-401-3442
Performer: MDRC
PIC ID: 8927

Do Alternative Education Strategies Increase Employment Retention and Advancement?

This study examined two approaches to providing education and training services to employed welfare recipients in Riverside County, California as part of the national Employment Retention and Advancement Project (ERA). The two approaches, called Work Plus and Training Focused, enrolled recipients of Temporary Assistance for Needy Families (TANF) benefits (primarily single parents) who worked for 20 or more hours per week but earned too little to leave assistance. approaches encouraged working TANF recipients to attend courses in remedial education, postsecondary education, or vocational training, depending on recipients&rsquo; levels of educational attainment and career aspirations.

During the first follow-up year, RFS led to a small increase in employment retention relative to the PES program. This retention impact was not sustained in the second year and no other impacts on employment or earnings were found.

Report Title: The Employment Retention and Advancement Project: A Comparison of Two Job Club Strategies: The Effects of Enhanced Versus Traditional Job Clubs in Los Angeles. http://www.acf.hhs.gov/programs/opre/welfare_employ/employ_retention/reports/era_la/era_la.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Michael Dubinsky, 202-401-3442

Performer: MDRC
PIC ID: 8928

Do Former Temporary Assistance for Needy Families Recipients Return, Stay Employed, or Receive Unemployment Insurance Benefits?

This study examined subsequent joblessness, application for Unemployment Insurance (UI) benefits, eligibility for UI benefits, and rates of UI benefit receipt among Temporary Assistance for Needy Families (TANF) recipients who left the program for employment. Researchers compared the levels of TANF and UI income support, and calculated the rate of return to TANF between UI beneficiaries, non-applicants, and ineligible applicants. Findings were compared to results from earlier studies measuring UI eligibility and receipt among those who left social assistance programs.

The study estimated that among TANF leavers who became newly unemployed and applied for UI benefits, nearly 91 percent would be eligible for monetary reasons, 36 percent would be eligible for non-monetary reasons, and 55 percent would ultimately draw UI benefits. In previous research, the highest estimated rate of UI benefit receipt among TANF leavers was 33 percent. Results suggested that UI may serve as a safety net for TANF leavers. Researchers found evidence that receipt of UI benefits is associated with a lower rate of return to TANF. Among TANF leavers who applied for UI, receipt of benefits reduced the rate of return to TANF by 22 percent compared to the rate observed for applicants who did not qualify and receive UI benefits. On a monthly basis, UI benefits were two to five times more generous than TANF payments. But small changes in the relative generosity of UI-to-TANF did not affect the rate of return to TANF. Taken together, these results suggest that UI benefit receipt might be serving as a proxy for strong labor force attachment. It might not be the income replacement function of UI that reduces return to TANF, but instead those who receive UI benefits might simply have better prospects for maintaining self-sufficiency through employment. Further investigation into the relative importance of UI income support and labor force attachment could inform policy.

Report Title: UI as a Safety Net for Former TANF Recipients http://aspe.hhs.gov/hsp/08/UI-TANF/index.htm
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Reuben Snipper, 202-401-6615
Performer: W.E. Upjohn Institute for Employment Research
PIC ID: 8929

How Has the ESTEP Independent Living Program Impacted Foster Care Youth?

This was the final process and impact study from one of four programs evaluated as part of the Multi-Site Evaluation of Foster Youth Programs. Impact findings were based on a two-year follow-up of youth who participated in a random assignment evaluation of the Early Start to Emancipation Preparation (ESTEP) -- Tutoring Program. The ESTEP Tutoring Program was designed to improve reading and math skills of foster youth aged 14 and 15 who are one to three years behind grade level in reading or math. Youth who participated in the evaluation were randomly assigned to either a treatment group that was offered access to ESTEP Tutoring, or to a control group. Outcomes evaluated aligned closely with the program&rsquo;s primary goals of improving reading and math skills and empowering youth to use other educational resources.

The program had no impacts on educational outcomes. No statistically significant differences were observed between the treatment and control groups in outcomes at the two year follow-up.

Report Title: Evaluation of the Early Start to Emancipation Preparation Tutoring Program; LA County, CA Final Report http://www.acf.hhs.gov/programs/opre/abuse_neglect/chafee/reports/eval_estep/eval_estep.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Maria Woolverton, 202-205-4039
Performer: Urban Institute
PIC ID: 8940

How Has the Life Skills Independent Living Program Impacted Foster Care Youth?

This was the final process and impact study from another one of four programs evaluated as part of the Multi-Site Evaluation of Foster Youth Programs. Impact findings were based on a two-year follow-up of youth in foster care in Los Angeles County who participated in a random assignment evaluation of the Life Skills Training Program. Youth were 17 years old at the time of random assignment to either a treatment group that was offered access to Life Skills Training, or to a control group. Concrete measures of the transition to adulthood were examined. Education and employment measures included completion of a high school diploma or general equivalency diploma and current employment status. Economic well-being was measured by reported earnings and current net worth, economic hardship, and receipt of formal and informal financial assistance.

There were found few impacts on outcome. After adjusting significance levels to account for the possibility of false positive results, no significant impacts remained.

Report Title: Evaluation of the Life Skills Training Program: Los Angeles County, California http://www.acf.hhs.gov/programs/opre/abuse_neglect/chafee/reports/eval_lst/eval_lst.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Maria Woolverton, 202-205-4039
Performer: Urban Institute
PIC ID: 8941

How Can Social Services Be Designed to Better Help Families At Risk of Poverty?

In this exploratory study, researchers examined the factors influencing the functioning of families, especially those at risk of poverty, to identify lessons about how to design possible human services demonstrations to improve the well-being of at-risk families and children. They synthesized research relevant to decision-making and behavior about marriage, family formation, employment and earnings and family time use with a focus on low-income populations. It developed a conceptual framework for further research on these aspects of family and work life.

Report Title: Marriage, Employment, and Family Functioning: Conceptual Framework for Interventions http://www.acf.hhs.gov/programs/opre/strengthen/marr_employ/index.html
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Mark Fucello, 202-401-5750
Performer: The Urban Institute
PIC ID: 8942

Does Participation in the "I am Moving, I am Learning" Train the Trainer Program Result in Program Implementation?

Researchers examined the extent to which grantees who participated in a training-of-trainers implemented a Head Start program enhancement called I Am Moving, I Am Learning (IM/IL). The IM/IL program was intended to (1) increase the quantity of time children spend in moderate to vigorous physical activity (MVPA) during their daily routine; (2) improve the quality of structured movement activities that are helped by teachers and adults; and (3) promote healthy food choices for children each day.

Most (96 percent) programs tried to implement IM/IL in the year following training. Two-thirds offered activities to alter the eating and physical activity behaviors of parents, and half did so with their staff. Almost half of the programs reported that they were successful in implementing IM/IL. Staff enthusiasm and training quality were the two most commonly factors contributing to successful implementation.

Report Title: Results from the &lsquo;I Am Moving, I Am Learning&rsquo; Stage 1 Survey: Final Interim Report. http://www.acf.hhs.gov/programs/opre/hs/eval_move_learn/reports/stage1_survey/stage1_survey.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Laura Hoard, 202-205-4561
Performer: Mathematica Policy Research, Inc.
PIC ID: 8944

How Has Abstinence Education Research and Education Been Promoted?

Researchers assessed how to promote abstinence research and evaluation.

Investigators determined that a website could offer training and resources in program evaluation and one was created to encourage use of quality evaluations using widely-accepted scientific practices for sampling, measurement, design, analysis, and interpretation of findings. The website features training modules describing the major stages involved in conducting abstinence education program evaluation, including planning, designing, and implementing an evaluation, as well as analyzing data, interpreting findings and reporting findings to stakeholders. The website includes videotaped clips of consultations between evaluators and practitioners, and video and audio taped interviews with various experts, and evaluation planning exercises.

Report Title: Center for Research and Evaluation on Abstinence Education http://abstinenceevaluation.hhs.gov/tiki-index.php

Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Lisa Trivits, 202-205-5750
Performer: The Lewin Group
PIC ID: 9013

How Many Children Are Eligible for Child Care Subsidies?

Analysts estimated the number of children meeting eligibility requirements for child care assistance under the Child Care and Development Fund. The estimates were produced by the Transfer Income Model (TRIM). The analysis estimated the number of children receiving HHS-funded child care subsidies, and the percentage served.

Over eight million children were eligible for child care subsidies in 2005, under the eligibility rules of the Child Care and Development Fund (CCDF). While we do not know how many of these children were in families that needed help paying for child care, 29 percent of the potentially eligible children received subsidized care through CCDF or other funding streams in fiscal year 2005. An even larger percentage of children in families with income below poverty were served.

Report Title: Child Care Eligibility and Enrollment Estimates for Fiscal Year 2005 http://aspe.hhs.gov/hsp/08/cc-eligibility/ib.htm
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Kendall Swenson, 202-690-6888
Performer: Staff; Office of the Assistant Secretary for Planning and Evaluation
PIC ID: 9014

How Do Child Care Arrangements Differ in Urban and Rural Areas of the Country?

Analysts compared non-parental care arrangements of pre-school age children in urban and rural areas of the United States using data from the 2005 National Household Education Survey (NHES), Early Childhood Program Participation Survey (ECPP). Data from the NHES show that among preschool-age children, those in rural areas were about as likely as those in urban areas to receive care from someone other than their parents on a weekly basis.

When rural children participated in non-parental care they were more likely than urban children to receive this care from relatives and were less likely to receive care in center programs. Additionally, rural children were in families that, on average, made fewer out-of-pocket contributions toward the cost of their care.

Report Title: Child Care Arrangements in Urban and Rural Areas http://aspe.hhs.gov/hsp/08/cc-urban-rural/index.shtml
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Kendall Swenson, 202-690-6888
Performer: Staff; Office of the Assistant Secretary for Planning and Evaluation
PIC ID: 9015

What Early Implementation Experiences Have Head Start Oral Health Initiative Grantees Had?

This study examined was designed to describe the oral health promotion strategies developed by the 52 Head Start Oral Health Initiative (OHI) grantees and evaluated implementation. The study is based primarily on telephone interviews with grantees conducted in 2007 and on information collected in the program record-keeping system by grantees on the characteristics of children, their families, and pregnant women enrolled in the OHI and the oral health services they received.

Most grantees implemented the OHI in their entire service area and served diverse children and families. A key service delivery theme was the education of parents, children, pregnant women and staff about oral health. Grantees partnered with a combination of direct service providers, local oral health coalitions, and advocacy groups. Grantees using OHI funds for new staff positions often hired individuals with clinical dental experience. Grantees expanded the types of preventive services offered to Head Start children and most grantees referred children and pregnant women to community providers for follow-up treatments. Partnerships with direct service providers were important factors in service delivery approaches. To reduce barriers to care, grantees provided a range of support services; grantees distributed oral hygiene supplies to reinforce educational messages.

Report Title (volume 1): Oral Health Promotion, Prevention, & Treatment Strategies for Head Start Families: Early Findings from the Oral Health Initiative Evaluation Volume 1: Final Interim Report http://www.acf.hhs.gov/programs/opre/hs/eval_oral_health/reports/early_findings_vol1/early_findings_vol1.pdf

Report Title (volume 2): Oral Health Promotion, Prevention, & Treatment Strategies for Head Start Families: Early Findings from the Oral Health Initiative Evaluation Volume 2: Site Profiles http://www.acf.hhs.gov/programs/opre/hs/eval_oral_health/reports/early_findings_vol2/early_findings_vol2.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Laura Hoard, 202-401-4561
Performer: Mathematica Policy Research, Inc.; Princeton, NJ
PIC ID: 8945, 8945.1

Objective 3.2: Protect the safety and foster the well-being of children and youth.

What Have We Learned about Domestic Violence and Child Maltreatment?

This national, cross-site evaluation assessed whether the Greenbook Initiative succeeded in changing systems to better meet families' needs. The Greenbook Initiative is a multi-agency collaborative demonstration addressing the co-occurring problems of domestic violence and child maltreatment. Through grants to several local communities, the initiative sought to bring about changes in the primary systems — child welfare agencies, family and juvenile courts, and domestic violence service providers — so that parents who were victims of abuse and their children could become safer and more stable. . The study used stakeholder surveys, interviews with caseworkers and other front line workers, case record reviews, focus groups with survivors of domestic violence, and community descriptive information.

Sites undertook major collaborative efforts aimed at improving practices, services, and outcome for children and families. There were changes in practice at work with families and children.

Report Title: The Greenbook Initiative Final Evaluation Report
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Jerry Silverman, 202-690-5654
Performer: Caliber Associates
PIC ID: 8185

How Does Involvement of Nonresident Fathers Affect What Happens to Their Children in Foster Care?

This analysis followed-up a previous study, What About the Dads. The original study examined child welfare agencies' efforts to identify, locate, and involve nonresident fathers of children in foster care. This study explored whether variation in (1) child welfare agencies&rsquo; contact with nonresident fathers and (2) fathers&rsquo; support and visitation improved outcomes for children in foster care.

Nonresident fathers&rsquo; involvement with their children was associated with a higher likelihood of reunification and a lower likelihood of adoption. Children whose nonresident fathers were highly involved were discharged from foster care more quickly than those with less or no involvement. The study found that nonresident fathers&rsquo; contact with the child welfare agency and involvement with their children was not associated with subsequent maltreatment allegations. Among children whose case outcome was reunification, usually with their mothers, higher levels of nonresident father involvement were associated with a substantially lower likelihood of subsequent maltreatment allegations.

Report Title: More about the Dads: Exploring Associations between Nonresident Father Involvement and Child Welfare Case Outcomes http://aspe.hhs.gov/hsp/08/MoreAboutDads/index.htm
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Laura Radel, 202-690-5938
Performer: Urban Institute
PIC ID: 8361.1

What Do We Know About Child Welfare Privatization Initiatives?

This analysis described choices faced by agencies as they design child welfare privatization initiatives. It illustrated how various initiatives have defined their target populations and program scope, as well as how they have structured payments and distributed financial risk. Conclusions focused on how each program and fiscal design element must be considered in conjunction with others.

Privatization is first and foremost a systemic reform, which has implications for and requirements of multiple features of a social services system. Partnerships between public and private agencies require ongoing collaboration, information exchange and adjustments as reforms mature and system goals evolve.

Report Title: Program and Fiscal Design Elements of Child Welfare Privatization Initiatives http://aspe.hhs.gov/hsp/07/CWPI/models/index.htm
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Laura Radel, 202-690-5938
Performer: Planning and Learning Technologies, Inc.
PIC ID: 8509.1

Do Staff Roles Change When Child Welfare Services Are Privatized?

This study examined several child welfare systems transition of case management functions from public to private agencies and analyzed how roles and responsibilities were shared and divided once privatization occurred. The history and complexity of defining privatization in child welfare services was reviewed. The study looked at how jurisdictions in seven states divided key case management activities for their out-of-home care population including initial case assessments, roles in dependency hearings, and ongoing case decision making. Researchers reviewed the experience of a group of states that used private agencies to deliver foster care case management and that had operational State Automated Child Welfare Information Systems. Researchers considered challenges faced by public and private agencies with their new information systems and offered examples of how states have helped the transition.

Report Title: Evolving Roles of Public and Private Agencies in Privatized Child Welfare Systems http://aspe.hhs.gov/hsp/07/CWPI/roles/index.htm
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Laura Radel, 202-690-5938
Performer: Planning and Learning Technologies, Inc.
PIC ID: 8509.2

What Factors Do State and Local Officials Consider When Contracting for Child Welfare Services?

This study examined the decisions regarding child welfare privatization that must be made in cooperation with the provider community. Child welfare privatization (initiation and renewal) is accomplished through contractual agreements between local or state public agencies and private providers. An overarching theme of the study were these partnership arrangements.

When public agencies contract for services, they seek one or more partners to share the risks, rewards, and responsibilities of delivering services to children and families in the child welfare system. To the extent allowed by state procurement rules, a collaborative public-private planning process can ensure that consensus is reached on the broad goals and expectations of the procurement, paving the way for explicit, fairly negotiated, enforceable and outcome-based contracts.

Report Title: Preparing Effective Contracts in Privatized Child Welfare Systems http://aspe.hhs.gov/hsp/07/CWPI/contracts/index.shtml
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Laura Radel, 202-690-5938
Performer: Planning and Learning Technologies, Inc
PIC ID: 8509.3

How Should Privatized Child Welfare Services Be Evaluated?

This pre-evaluation sought information on evaluating child welfare privatization initiatives, to be used by state and local program managers. Researchers identified key features of program evaluation and the tasks that program managers could perform to ensure successful evaluation. The study examined the value of cost-effectiveness analysis and the kinds of information that cost analyses can generate.

Report Title: Evaluating Privatized Child Welfare Programs: A Guide for Program Managers http://aspe.hhs.gov/hsp/07/CWPI/guide/index.shtml
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Laura Radel, 202-690-5938
Performer: Planning and Learning Technologies, Inc.
PIC ID: 8509.4

What Development Needs Do Children Investigated by Child Protective Services Have and Do These Children Receive Early Intervention?

This descriptive study characterized the extent to which maltreated children had developmental problems or were subject to factors associated with poor developmental outcomes, what services these children might be eligible to receive, what factors influenced service receipt, and what solutions had been devised to address barriers to service provision. A growing body of evidence suggests that many infants and toddlers in the child welfare system have developmental and behavioral problems but do not receive the services they need. Federal law requires that infants and toddlers with substantiated child maltreatment reports be referred to early intervention services funded.

The study found that children ages birth to three who have been maltreated were at substantial risk of experiencing developmental problems and a higher proportion of these children tended to be described as having environmental and biomedical risks or having a low score on a developmental measure. Maltreated children between two and three years old had high levels of behavior problems. While families were receiving parent training and family counseling services through child welfare service or by referral, the extent to which these services provided interventions focused on enhancing child development was unclear.

Report Title: Developmental Status and Early Intervention Service Needs of Maltreated Children: Final Report http://aspe.hhs.gov/hsp/08/devneeds/index.htm
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Sarah Potter, 202-260-0382
Performer: Institute for Social and Economic Development
PIC ID: 8570

How Do States and Communities Respond to Statutory Rape Incidents?

This study profiled nine sites addressing the problem of statutory rape through various interdisciplinary approaches. Site profiles and summary findings from an analysis of state statutory rape laws were used to illustrate approaches in three categories: cooperative legislative initiatives, law enforcement, and education and prevention programs.

Data to understand the issue were limited and interest in addressing the problem was often based on anecdotal information. Laws varied by state, as did consensus. No agreement existed about what was illegal, despite laws, and what behavior should be reported. The complexity of the issue necessitated broad stakeholder involvement; no single agency or department within states had ownership of the issue. Multifaceted response was needed; victims and offenders often needed an array of services to prevent and address the underlying risk factors as well as the effect of the harmful relationships.

Report Title: Exploring Community Responses to Statutory Rape: Final Report; Report may be obtained from Federal Contact
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Jerry Silverman, 202-690-5654
Performer: The Lewin Group
PIC ID: 8577

Does a Child's Well-Being and Service Receipt Depend on Verifying Maltreatment?

Researchers examined the well-being of children in substantiated and unsubstantiated maltreatment cases. Researchers collected information about children&rsquo;s access to child welfare, mental health, and special education services.

Children in substantiated and unsubstantiated maltreatment cases appeared to have similar social, behavioral, and emotional needs. When compared with needs among children in unsubstantiated cases, needs among children with substantiated maltreatment cases were perceived by caseworkers to be greater. Children with substantiated cases of maltreatment received more child welfare services after investigation than those with unsubstantiated cases, but substantiation status did not appear to affect access to mental health or special education services.

Report Title: National Survey of Child and Adolescent Well-Being, Brief 9: Does Substantiation of Child Maltreatment Relate to Child Well-Being and Service Receipt. http://www.acf.hhs.gov/programs/opre/abuse_neglect/nscaw/index.html
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Mary Bruce Webb, 202-205-8628
Performer: Research Triangle Institute
PIC ID: 8973

Are Caseworkers' Subjective Judgments Likely Used as a Basis for Determining That a Child Has Been Maltreated?

Researchers examined the relationship of caseworker judgments to the “ substantiation” decision. Substantiation denotes child welfare system (CWS) services&rsquo; official decision about the validity of maltreatment allegations.

The majority of cases investigated by child protective services were unsubstantiated. However, caseworkers did take into account their judgments of harm to the child, future risk to the child, and evidence of maltreatment when they made substantiation decisions. Substantiation rates clearly rose as the rating of harm or risk increased but evidence played an important role, even when caseworkers believed children were harmed or at risk. Substantiation was unlikely unless evidence of maltreatment was sufficient. The study found that harm, risk, and evidence did not perfectly predict substantiation indicating that other factors, like caseworker workload, may be affecting outcomes.

Report Title: National Survey of Child and Adolescent Well-Being, Brief 6: How Do Caseworker Judgments Predict Substantiation of Child Maltreatment. http://www.acf.hhs.gov/programs/opre/abuse_neglect/nscaw/index.html
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Mary Bruce Webb, 202-205-8628
Performer: Research Triangle Institute
PIC ID: 8974

What Proportion of Eligible Low-Income Children Are Not Insured?

This Congressionally-mandated study sought to identify the percent of State Children&rsquo;s Health Insurance Program (SCHIP) enrollees eligible for Medicaid.

Reviewers determined that 4 percent of children enrolled in separate SCHIPs were eligible for their States&rsquo; Medicaid program. These enrollment errors involved miscalculations of the families&rsquo; net income, clerical mistakes, and other unclassified errors. Projected to the population of all children enrolled in separate SCHIPs in 2006, the 4-percent error rate corresponded to about 105,000 children nationally. An additional 4.5 percent lacked sufficient documentation to make a determination regarding Medicaid eligibility, which leaves open the possibility that the number of children enrolled in separate SCHIPs who were eligible for Medicaid in 2006 could have been higher than our projection. The Centers for Medicare & Medicaid Services stated that it would continue to undertake a number of activities to prevent the types of errors identified.

Report Title: Separate State Children's Health Insurance Program Enrollees' Eligibility for Medicaid in 2006 http://oig.hhs.gov/oei/reports/oei-06-07-00310.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 9001

Do We Detain and Release Children From Other Countries as Required?

This study assessed the placement, care, and release of unaccompanied alien children. Most were placed and released in accordance with the Flores Agreement, with 84 percent of children admitted to a detention facility within 3 days of apprehension. The Flores Agreement — named after a child that became the center of a series of exchanges both in and outside the courts — requires special handling of youth when in immigration custody in the United States (such as not intermingling children and adults in detention centers).

All children&rsquo;s case files lacked at least one required document that would indicate whether a child received medical or mental health services or participated in educational or recreational activities. The Division of Unaccompanied Children&rsquo;s Services provided limited oversight of facilities. No explicit agreement exists between the Departments of Health and Human Services and Homeland Security regarding information exchange and post-release activities. The Administration for Children and Families will include random interviews with children and case file reviews as part of the routine responsibilities for Federal field specialists.

Report Title: Division of Unaccompanied Children&rsquo;s Services: Efforts To Serve Children http://oig.hhs.gov/oei/reports/oei-07-06-00290.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 9003

What Challenges Do Boys Face; How Best Address These Challenges?

This analysis reviewed the literature on the risks and assets that affect boys ages 10 through 18. It pinpointed what strengths make some boys likely to succeed and what risks, or challenges, increased the likelihood that they would struggle. A major goal of the project was to provide information about approaches that helped boys stay on the right track, ranging from formal programs to environmental interventions.

The review found that boys were doing better than they had done a decade or more previously across a variety of indicators, including juvenile justice involvement, dropout rates, and substance use. However, boys were still facing challenges in many areas. Recently, boys committed fewer property crimes than they had in the 1980s, but in 2004, almost three-quarters of young people prosecuted in juvenile courts were boys. Since 1995, boys&rsquo; dropout rates had fallen, but boys still represented more than half (56 percent) of school dropouts for ages 16 to 24. Recently, boys&rsquo; smoking rates were lower than in the mid to late 1990s, and their drinking rates had declined. However, about 1 in every 6 eighth grade boys surveyed in 2006 had drunk alcohol in the previous month, and nearly 40 percent of the twelfth grade boys had used illegal drugs during the previous year.

Report Title: What Challenges are Boys Facing, and What Opportunities Exist to Address Those Challenges? http://aspe.hhs.gov/hsp/08/boys/

Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Sarah Potter, 202-260-0382
Performer: JBS International
PIC ID: 9008

What is the Adoption Status of Infants in Maltreatment Cases?

Based on available survey data, researchers described adoption among infants involved with the child welfare system.

Report Title: National Survey of Child and Adolescent Well-Being, Brief 14: Need for Adoption Among Infants Investigated for Child Maltreatment http://www.acf.hhs.gov/programs/opre/abuse_neglect/nscaw/reports/need_adoption/need_for_adoption.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Mary Bruce Webb, 202-205-8628
Performer: Research Triangle Institute; Research Triangle Park, NC
PIC ID: 8969

For Young Victims of Child Maltreatment, How Prevalent and What Predicts Depression Among Their Caregivers?

Researchers explored depression among young mothers of young children reported to the child welfare system. They reported on female caregivers of 1,244 children who had encounters with child protective services.

Between 22% and 25% of caregivers had a score indicating major depression at some point in time. In addition, having been a victim of intimate-partner violence, being in fair or poor health, being single, or being White correlated with symptoms of major depression.

Report Title: National Survey of Child and Adolescent Well-Being, Brief 13: Depression Among Caregivers of Young Children Reported for Child Maltreatment http://www.acf.hhs.gov/programs/opre/abuse_neglect/nscaw/reports/depression_caregivers/depression_caregivers.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Mary Bruce Webb, 202-205-8628
Performer: Research Triangle Institute; Research Triangle Park, NC
PIC ID: 8970

How Well Do Young Adults Transition to Adulthood Who Were Previously Involved With the Child Welfare System as Adolescents?

Analysts focused on adolescents transitioning to young adulthood. They identified information about 620 young adults who were adolescents (12 to 15 years old) at baseline. Analysts examined the characteristics of these young adults, the types of maltreatment they experienced, and the risks they faced.

Report Title: National Survey of Child and Adolescent Well-Being, Brief 11: Adolescents Involved with Child Welfare, A Transition to Adulthood http://www.acf.hhs.gov/programs/opre/abuse_neglect/nscaw/reports/adolescents_involved/adolescents_involved.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Mary Bruce Webb, 202-205-8628
Performer: Research Triangle Institute; Research Triangle Park, NC
PIC ID: 8971

How Are Children Doing Five to Six Years After Their Initial Involvement as Infants With the Child Welfare System?

Analysts provided information about 962 children who were infants (zero to 12 months old) when they first became involved in investigations for child abuse or neglect. They examined the characteristics of these children.

Report Title: National Survey of Child and Adolescent Well-Being, Brief 10: From Early Involvement with Child Welfare Services to School Entry http://www.acf.hhs.gov/programs/opre/abuse_neglect/nscaw/reports/wave_five_followup/wave_five_followup.pdf
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Webb, Mary, 202-205-8628
Performer: Research Triangle Institute; Research Triangle Park, NC
PIC ID: 8972

Objective 3.3: Encourage the development of strong, healthy, and supportive communities.

Do Faith-Based and Community Organizations Report Positive Changes in Organizational Capacity?

Evaluators assessed the organizational capacity building outcomes associated with receipt of a Targeted Capacity Building grant under the Compassion Capital Fund (CCF). These grant funds are awarded to faith-based and community organizations for organizational capacity building to increase their effectiveness, enhance their ability to provide social services, expand their organizations, diversify their funding sources, and create collaborations to better serve those in need.

Ninety-two percent of the grantees reported that the grant funds contributed to improvements in organizational capacity and 81 percent reported that CCF assistance helped improve participant outcomes. Almost three-quarters (72%) indicated the Targeted Capacity Building grant made a positive difference in their revenue development strategy.

Report Title: An Assessment of the Compassion Capital Fund Targeted Capacity Building Program: Findings from a Retrospective Survey of Grantees http://www.acf.hhs.gov/programs/ccf/surveys/capacity_assessment.html
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Nancye Campbell, 202-401-5760
Performer: Branch Associates, Inc.
PIC ID: 8859

Objective 3.4: Address the needs, strengths, and abilities of vulnerable populations.

How Many and Who Receive What Kind of Energy Assistance under the Low Income Home Energy Assistance Program?

Program staff analyzed service and expenditure data for the Low Income Home Energy Assistance Program (LIHEAP). In FY 2006, LIHEAP provided assistance through block grants to the States, the District of Columbia, Indian Tribes and Tribal Organizations, and United States Insular Areas. These households represented 16 percent of all households with incomes under the Federal LIHEAP income eligibility cutoff.

Households that received heating assistance were among the poorer households within the LIHEAP income eligible population. Of households receiving heating assistance 31 percent had at least one member 60 years or older, 29 percent included at least one member with a disability, and 21 percent included at least one child 5 years or younger. Households receiving weatherization assistance had the highest concentration of elderly members. Households receiving cooling assistance (as opposed to other types of LIHEAP assistance) had the highest concentration of disabled members. Households receiving summer crisis assistance had the highest concentration of young children.

Report Title: Low Income Home Energy Assistance Program Report to Congress for FY 2006 http://www.acf.hhs.gov/programs/ocs/liheap/report/index.html
Agency Sponsor: ACF-OCS, Office of Community Services
Federal Contact: Leon Litow, 202-401-5304
Performer: Staff; Administration for Children and Families
PIC ID: 8858

What Protocols and Tools Can Stakeholders Use to Assess the Impact of Funded Efforts to Address Racial/Ethnic Minority Health Needs?

This project developed a protocol for evaluating efforts funded by States and other stakeholders aimed at improving racial/ethnic minority health and reducing racial/ethnic health disparities, and a plan for identifying "best practices" based on evaluation of the effectiveness of these efforts. The intent was to ensure systematic development and implementation of evaluation plans relative to these OMH-funded efforts to assess whether the initiatives made a difference, and whether new/revised strategies and systems approaches to health disparities contributed to that difference.

A logic model approach was used to develop a tool entitled A Strategic Framework for Improving Racial/Ethnic Minority Health and Eliminating Racial/Ethnic Health Disparities. Then, a preliminary set of performance measures for outcomes and impacts identified in the Strategic Framework was developed. Lastly, an evaluation protocol for systematically evaluating efforts to improve racial and ethnic minority health, reduce health disparities, and effect systems approaches (the evaluation protocol) was developed to assess the nature and extent of the outcomes and impacts being achieved by efforts being funded by the Office of Public Health and Science, its grantees and partners, and other stakeholders. Use of these three tools to inform funding and other policy-relevant decisions in a more coordinated way will promote the effectiveness, efficiency, and impact of individual and collective efforts aimed at improving the health of racial/ethnic minorities and the Nation overall.

Report Title: Development of an Evaluation Protocol for Assessing the Impacts of OMH-funded Initiatives; Report may be obtained from Federal Contact
Agency Sponsor: OPHS, Office of Public Health and Science
Federal Contact: Valerie Welsh, 240-453-8222
Performer: Development Services Group, Inc.
PIC ID: 8234

What Factors Need to be Considered in Promoting the Effective Identification and Use of Health Data by Tribal Communities?

This study examined the mediators and barriers that exist within tribal communities that affect the translation of quality health data into health programs and policy that can reduce health disparities. The project supported a systematic investigation of the kinds of data and used among Northwest Tribes, and of the mediators and barriers that affect the use and translation of such data into health programs and policies conducive to improved health status and reduced health disparities among American Indians/Alaska Natives at the Tribal level.

Successful use of data to guide programmatic and policy-relevant decision-making depended on several keys: strong leadership, a team vision, and investments in time and funding to prepare staff on how to access and use information. Data collection efforts encountered several barriers: staff shortages, rapid staff turnover, and inadequate staff training in data collection, management, analysis, and interpretation; concerns about data quality and whether conclusions drawn from the data are reliable; ineffective coordination among programs collecting or using data on the same local population; and communication breakdowns between tribal leadership and tribal health programs; and insufficient funding underlying and reinforcing other barriers.

Report Title: From Data to Action: An Evaluation of Tribal Data Use to Eliminate Health Disparities among Northwest Tribes; Report may be obtained from Federal Contact
Agency Sponsor: OPHS, Office of Public Health and Science
Federal Contact: Julie Moreno, 240-453-8222
Performer: Northwest Portland Area Indian Health Board
PIC ID: 8615

Have We Successfully Protected and Advocated for the Mentally Ill?

This evaluation examined the context in which the Protection and Advocacy for Individuals with Mental Illness (PAIMI) program operated, the processes established to implement the program at the state and federal levels, activities and tasks undertaken as part of the program, and the process and outcomes achieved. The PAIMI program seeks to extend the protections of the Developmental Disabilities Assistance and Bill of Rights Act of 1975 to individuals with significant mental illness by providing funds to support advocacy and protection activities.

Eighty-two percent of clients surveyed believe the advocate/attorney listened to their story and truly understood their circumstance. Ninety-two percent believe their advocate/attorney did everything they could do to obtain the outcome s/he wanted. Seventy percent felt the quality of their representation was “ excellent,” and twenty-four percent felt it was “ good.” Ninety-three percent of grantees met their target goals and objectives.

Report Title: National Evaluation of the Protection and Advocacy for Individuals with Mental Illness (PAIMI) Program; Report may be obtained from Federal Contact
Agency Sponsor: SAMHSA, Substance Abuse and Mental Health Services Administration
Federal Contact: John Morrow, 240-276-1783
Performer: Human Services Research Institute
PIC ID: 8720

Do Permanent Housing And Supportive Services Improve Conditions for the Chronically Homeless?

This performance assessment of the Collaborative Initiative to Help End Chronic Homelessness monitored client and system outcomes of a collaborative community-based approach that provided permanent supportive housing, case management, mental health and substance abuse treatment, primary health care, and veteran&rsquo;s health services to disabled individuals who have experienced chronic homelessness. The Collaborative Initiative to Help End Chronic Homelessness was implemented in response to the goal of eliminating chronic homelessness and to further the goal that federal agencies increase their level of collaboration.

Interim client-level data indicated that the proportion of the six services received by each client rose during the first year, the average number of days clients were housed increased, the mean monthly public assistance income increased, and clients alcohol and drug problems remained unchanged.

Interim system-level findings indicated a significant increase in practices that encourage system integration; a significant increase in the availability of information on client and service delivery, management information systems, and the use of evidence-based mental health practices; and no significant association between client outcomes and the use of evidence-based practices or measures of collaboration and trust among the network agencies.

Report Title: Evaluation of the Collaborative Initiative to Help End Chronic Homelessness http://aspe.hhs.gov/hsp/homelessness/CICH07/index.htm
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Canta Pian, 202-690-7149
Performer: Department of Veteran's Affairs Northeast Program Evaluation Center
PIC ID: 8844

What do we Know About Homelessness; What Areas Need More Study?

This research symposium focused on innovative prevention and intervention models to end homelessness.

Report Title: Toward Understanding Homelessness: The 2007 National Symposium on Homelessness Research http://aspe.hhs.gov/hsp/homelessness/symposium07/index.htm
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Anne Fletcher, 202-690-5739
Performer: Abt Associates, Inc.
PIC ID: 8856

Does Existing Research Reliably Characterize Homeless Families?

This study explored the research on homeless families with children, identified key knowledge gaps and considered whether these gaps might most efficiently be filled through secondary analysis of data, adding questions or a module to planned surveys that include low-income people, or whether new primary data collections were needed.

The data were insufficient to evaluate and develop a typology of homeless family and child characteristics. Researchers recommended adding questions or a module to planned evaluations or surveys that included low-income people as the best way to collect more information to inform a typology.

Report Title: Characteristics and Dynamics of Homeless Families with Children http://aspe.hhs.gov/hsp/homelessness/improving-data08/index.htm
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Anne Fletcher, 202-690-5739
Performer: Westat, Inc.
PIC ID: 8857

How Well Do We Measure Income, Family Structure and Poverty?

This study examined whether the same picture of the U.S. population was presented by four different surveys that had sought to measure income, family structure and poverty. Policy makers use national surveys to paint a picture of the U.S. population along a variety of dimensions such as poverty status, receipt of program benefits, demographic characteristics and health insurance coverage. Inferences are drawn about need and eligibility for Federal programs based on estimates produced by these surveys.

Measures of income and income recipiency (income received as a result of eligibility under a public program; term often used in the context of welfare receipt) varied substantially among the surveys even when comparable estimates of income in 2002 were constructed for each survey. Policy analysts may not be able to use the surveys with the best income data because other essential data are not collected, for example, lack of health information on the Current Population Survey, which is the official source of poverty statistics. Similarly, policy analysis requires information on each person in the family to calculate eligibility for program units smaller than the family or for &lsquo;what-if&rsquo; scenarios to determine persons eligible but not participating in a program, or who would become eligible if the program were changed. The National Health Interview Survey (NHIS) measures income only at the family level and uses a definition of family (treating unmarried partners as families) not currently used by Federally-funded transfer programs. Other important differences occur with respect to wages and salaries, poverty levels and for the elderly.

Report Title: Measuring Income and Poverty in Four Surveys: An Overview; Report may be obtained from Federal Contact
Agency Sponsor: ASPE-OSDP, Office of Science and Data Policy
Federal Contact: Joan Turek, 202-690-5965
Performer: Gabrielle Denmead, Denmead Services and Consulting
PIC ID: 9071

Goal 4: Scientific Research and Development: Advance scientific and biomedical research and development related to health and human services.

Basic science is the foundation for improved health and human services. However, once a basic discovery is made, the findings must be applied and translated into practices for health and human service improvement to result. This continuum from basic and applied research to practice is a significant emphasis of HHS&rsquo; scientific research and development enterprise.[10]

Objective 4.1: Strengthen the pool of qualified health and behavioral science researchers.

What Global Health Research and Training Needs Exist?

This study assessed global health research needs of the National Institute of Health (NIH); likewise, it assessed the health research capacity-building needs of developing countries. The study examined programs that addressed global health research and identified gaps between needs and activities. Fogarty International Center addresses global health challenges through collaborative research and training programs, and international partnerships. This study was conducted to re-align the Fogarty&rsquo;s portfolio of extramural research and training programs with the global health research and training agenda. A data-rich portfolio analysis and environmental scan with the resulting gap analysis of global health research formed the basis of the study.

In response to the study, Fogarty will emphasize chronic disease research, implementation science, and research training for U.S. and developing country researchers. Fogarty staff used the analysis and data collected to enhance their own understanding of research and training opportunities, needs, and potential partnerships and synergies within and across programs.

Report Title: Needs Assessment for the Fogarty International Center; Report may be obtained from Federal Contact
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Linda Kupfer, 301.496.3288
Performer: Science and Technology Policy Institute
PIC ID: 9042

Should NIH Establish a Broad Unified Information System?

Researchers documented and assessed information technology needs of the National Institutes of Health (NIH) technology transfer community, focusing on greater information integration and sharing.

Stakeholders suggested the need to improve information technology in four broad areas: functionality and productivity; systems; intersystem information transfer; and miscellaneous. To meet these needs, researchers considered four options: maintain the status quo, make targeted architecture improvements, create a shared &lsquo;situational awareness&rsquo; website and database for non-proprietary information, or make additional improvements to Office of Technology Transfer. Researchers recommended that the NIH technology transfer community make architecture improvements in the medium term. In the long term, the technology transfer community should aim to evolve towards greater integration of information systems.

Report Title: Information Technology Needs Assessment for the NIH Technology Transfer Community; Report may be obtained from Federal Contact
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Bonny Harbinger, 301-594-7700

Performer: Science and Technology Policy Institute
PIC ID: 9047

Objective 4.2: Increase basic scientific knowledge to improve human health and human development.

How Can We Improve Management of Allergy and Infectious Diseases Small Business Programs?

This needs assessment examined management process and policies of National Institute for Allergies and Infectious Diseases (NIAID) Small Business Innovation Research (SBIR) and Small Business Technology Transfer Research (STTR) programs. The domestic small business program supports research and development of products or services that prevent, diagnose, and treat allergic, immunologic, and infectious diseases.

The study identified two primary areas for program improvement: 1) changes in organizational and portfolio management; and (2) development of a contract award mechanism. The study highlighted the importance of transparency, continued communication, and trans-Institute collaboration to maintain a vigorous program.

Report Title: Needs Assessment of the NIAID Small Business Innovation Research (SBIR) and Small Business Technology Transfer Research (STTR) Programs; Report may be obtained from Federal Contact
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Judith Brooks, 301-594-6626
Performer: NOVA Research Company; Bethesda, MD
PIC ID: 9074

Objective 4.3: Conduct and oversee applied research to improve health and well-being.

Does Research on the Demography and Economics of Aging Yield Adequate Results?

The Behavioral and Social Research Program at the National Institute on Aging evaluated its research centers on the Demography and Economics of Aging — often referred to as the "Demography Centers" — to assess the effectiveness of the centers and to determine changes warranted for a future funding cycle, including potential adjustments to program scope, goals, and objectives.

The evaluation found that the Demography Centers have been outstanding and have had a tremendous impact on aging research. They recommended no major substantive changes to the program.

Report Title: Review of the Centers on the Demography and Economics of Aging; Report may be obtained from Federal Contact
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Georgeanne Patmios, 301-496-3138
Performer: Rose Li and Associates, Inc.
PIC ID: 8752

Has Occupational Safety and Health Research Improved Workplace Health and Safety?

The evaluations of programs in the National Institute of Occupational Safety and Health had three objectives: assess the relevance of the programs&rsquo; activities to the most important workplace health and safety problems, assess the programs&rsquo; impacts on worker safety and health, and assess the programs&rsquo; ability to respond to emerging issues.

Report Title: The National Academies Systemic Evaluation of NIOSH Research Programs http://www.cdc.gov/niosh/nas/
Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Julie Zajac, 404-498-4381
Performer: National Academy of Sciences
PIC ID: 9030

How Does the International Collaborative Genetics Research Training Program Work?

This study examined the implementation and processes of the International Collaborative Genetics Research Training Program. The program enables U.S. universities and non-profit research institutions to support training for scientists from developing countries in human genetics research, in areas that are relevant to their home countries&rsquo; needs. The review helped guide the development of the program&rsquo;s new Request for Proposals; and results were shared with program partners, sponsors, investigators, and students.

The experts expressed a consensus view that the idea and vision for the Genetics program was timely and appropriate.

Report Title: A Review of the Fogarty International Center&rsquo;s International Collaborative; Report may be obtained from Federal Contact
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Linda Kupfer, 301.496.3288
Performer: Science and Technology Policy Institute
PIC ID: 9040

What Discoveries Has Parkinson's Disease Research Made?

This study assessed 11 Udall Parkinson's Disease Research Centers, to evaluate their operations in terms of their scientific productivity, collaborations, and training. The study provided additional insight on the reasons some of the centers were more successful than others in achieving the program&rsquo;s goals.

Report Title: 1) Evaluation of the NINDS Morris K. Udall Parkinson's Disease Research; and 2) Report of the Working Group of the National Advisory Neurological Disorders; Report may be obtained from Federal Contact
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Paul Scott, 301-496-9271
Performer: Booz Allen Hamilton
PIC ID: 9041

Can We Evaluate Electronic Communications Networks for Oral Health?

This evaluability assessment documented the readiness of the National Institute of Dental and Craniofacial Research (NIDCR) Dental Practice-Based Research Network Program for a comprehensive evaluation. NIDCR created the Network program in response to the lack of research data to guide treatment decisions in dental practice, in order to establish practice-based research networks that would investigate with greater scientific rigor the everyday issues in the delivery of oral healthcare.

The Network program was found to be sufficiently well implemented and stable so that a comprehensive evaluation could be planned and undertaken. A plan for the evaluation was developed. Researchers advised that an evaluation include several process and outcome components: dissemination of research findings, translation of research findings to community practices, and changes in community dental practices.

Report Title: NIDCR Dental Practice-Based Research Network Program (Dental PBRN); Report may be obtained from Federal Contact
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Sue Hamann 301-594-4849

Performer: Dennis Gehley and Associates
PIC ID: 9043

What Preclinical Development Resources Do Researchers Need?

This study assessed the need in the research community for access to preclinical development resources; and determined if the National Institutes of Health (NIH) Rapid Access to Interventional Development (NIH-RAID) Pilot Program is structured and targeted to meet the need and address critical gaps in early stage drug development. The NIH- RAID Pilot Program was established, to make available, on a competitive basis, critical preclinical development resources needed for the development of new therapeutic agents.

There is a strong need for a program like NIH-RAID. No significant funding alternative to the program could be identified. Programs offering similar services were limited in scope, funding, and access, while NIH-RAID was more comprehensive and available to a wider range of eligible investigators. Recommended modifications to the program included broadening the range of services to include development of biologics, and opening up the eligibility criteria to include for-profit entities. Researchers recommended that NIH-RAID consider a multi-pronged marketing approach to improve awareness of the program in the research community.

Report Title: National Institutes of Health Rapid Access to Interventional Development (NIH-RAID) Pilot Program Needs Assessment Evaluation; Report may be obtained from Federal Contact
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Deshiree Belis, 301-496-5860
Performer: Tunnell Consulting, Government Services Group
PIC ID: 9048

How Best Evaluate Clinical and Translational Science Programs?

This study designed a methodology to evaluate the operations of the Clinical and Translational Science Awards (CTSA) consortium. The National Institutes of Health&rsquo;s Roadmap for Medical Research Program launched the consortium to re-engineer the clinical research enterprise in the United States . This re-engineering was accomplished by transforming the local, regional and national environment for clinical and translational science, thereby increasing the efficiency and speed of clinical and translational research.

This feasibility study highlighted the complexity of the national CTSA initiative; and raised several overarching issues that need to be considered in the design and implementation of the prospective process evaluation study, including defining key terms, transitioning from the former General Clinical Research Center (GCRC) model to the CTSA model, capturing relevant contextual variables, and determining the critical timeframes for assessing the results of program activities and their outcomes.

Report Title: Clinical and Translational Science Awards Program: National CTSA Process Evaluation Feasibility Study; Report may be obtained from Federal Contact

Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Patricia Newman, 301-435-0864
Performer: MasiMax Resources, Inc.
PIC ID: 9049

Do Laboratories Adequately Report Findings That Help Clinicians Detect Kidney Disease?

This study asked to what extent and under what conditions accredited US laboratories report estimated glomerular filtration rate (eGFR). Reporting eGFR is important because it could help clinicians detect kidney disease. At present the extent of reporting of eGFR done by clinical laboratories is not known. The study evaluated a national random sample of 6350 clinical laboratories. Laboratories surveyed by paper mail and the Internet were then followed-up with telephone calls.

Physician office laboratories were not as likely to report eGFR as much as other types of laboratories. For the laboratories that reported eGFR, it was routinely reported along with adult serum creatinine results. Independent laboratories tended to report eGFR only upon request. Laboratories with a high volume were more likely to report eGFR than other laboratory types. The evaluation concluded that the National Kidney Disease Education Program (NKDEP) should continue working with its Laboratory Working Group and others in the clinical laboratory community to promote and improve routine reporting of eGFR.

Report Title: Prevalence of Estimated Glomerular Filtration Rate (eGFR) Reporting among US Clinical Laboratories; Report may be obtained from Federal Contact
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Eileen Newmann, 301-435-8116
Performer: Ogilvy Public Relations Worldwide; ,
PIC ID: 9076

How Can NIH Better Track Training Programs Across All Institutes and Centers?

This study assessed whether an extension of the Query/View/Reporting (QVR) system (a web-based tool to help users of the electronic research administration system search and view grant application/approval information) could provide a mechanism for tracking in greater detail all National Institutes of Health (NIH) trainees. The expanded QVR system developed interfaces with NIH databases so that multiple measures of trainee success and training programs could be obtained without additional burden on individual Institutes and Centers. NIH&rsquo;s IMPAC II system was used as the primary source of data for the interfaces, but other databases were considered as well. The study evaluated the expanded system&rsquo;s capability to customize search criteria, review results in tabular format, save the results, provide information at aggregate and individual levels, and identify individuals associated with grants.

QVR successfully interfaces with the new Information for Management, Planning, Analysis and Coordination (IMPAC II) system so that an individual NIH institute or center could easily retrieve information on its trainees. This enabled the history of a trainee to be tracked as they moved through the NIH system. The extended system could track numbers of trainees who successfully competed for grants or published in scientific journals. QVR experienced varying degrees of success with other databases, but was unable to complete an assessment of its interface capability with some. The study demonstrated that the expanded QVR system created a user-friendly way of tracking trainees and allowed for additional enhancements, if needed.

Report Title: NIH-Wide Trainee Tracking System via QVR; Report may be obtained from Federal Contact
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Janet Guthrie, 919-541-4258
Performer: QVR
PIC ID: 9077

Objective 4.4: Communicate and transfer research results into clinical, public health, and human service practice.

Do Grantee Institutions Comply with Federal Financial Conflicts-of-Interest Regulations?

This study determined the number and nature of financial conflicts of interest reported by grantee institutions to the National Institutes of Health (NIH). For this analysis, NIH provided 438 financial conflict-of-interest reports for fiscal years 2004 through 2006. Federal regulations require grantee institutions to report financial conflicts of interest; the regulations do not require them to report the details.

At least 89 percent of financial conflict-of-interest reports reviewed did not state the conflicts or how they would be managed. Reviewers found few cases in which Institutes followed up with grantee institutions regarding a financial conflict of interest and/or the management plan for a conflict. Many Institutes relied on the good faith of the grantee institution to ensure compliance with Federal financial conflict-of-interest regulations and did not directly oversee or review grantee institutions&rsquo; management of financial conflicts of interest.

Report Title: National Institutes of Health: Conflicts of Interest in Extramural Research http://oig.hhs.gov/oei/reports/oei-03-06-00460.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 8987

Have Research Project Grants Been Correctly Monitored?

This study determined which parts of the National Cancer Institute (NCI) monitor their research project grants in accordance with Federal regulations. In 2007, the Institute disbursed 54 percent of its $29.1 billion budget in funding more than 38,000 research project grants.

All grant files contained progress reports that had evidence of agency review; however, 41 percent of progress reports were received late. In addition, deficiencies existed in financial oversight of Research Project Grants. Five of the nine required grant closeouts in the sample were not completed within the general timeframes specified in departmental guidelines. Insufficient documentation impeded third-party review of grant files in some cases. NCI indicated that it would continue to monitor the use of its electronic grant system and update procedures for file documentation.

Report Title: National Cancer Institute&rsquo;s Monitoring of Research Project Grants http://oig.hhs.gov/oei/reports/oei-07-07-00120.pdf
Agency Sponsor: OS-OIG, Office of Inspector General
Federal Contact: Erin Lemire, 202-205-9523
Performer: Staff; Office of Inspector General
PIC ID: 9005

Which Health Information Dissemination Campaigns Have Been Evaluated and Which Have Succeeded?

This project reviewed the scientific literature and synthesized findings regarding stroke signs and symptoms awareness campaigns. Additionally, the project identified which of the 34 Heart Disease and Stroke Prevention (HDSP) programs were currently implementing campaigns and what evaluation of them has been conducted.

Both campaigns were able to produce positive statistically significant results in increased stroke knowledge and behavioral intent to call 911 when witnessing a stroke. The campaigns worked well among a diverse target population. Analysts recommended that the goal of stroke awareness campaigns should be to increase individuals who call 911 when they experience or observe stroke signs and symptoms. This synthesis helps to inform the decision-making process on what type and relative factors are most important in awareness campaigns, especially when planning new campaigns or enhancing current efforts.

Report Title: Science Synthesis of Stroke Signs and Symptoms Awareness Campaigns; Report may be obtained from Federal Contact
Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Julie Zajac, 404-498-4381
Performer: Staff; Centers for Disease Control and Prevention, Division of Heart Disease and Stroke Prevention
PIC ID: 9021

Can Evaluation Help Public Health Partnerships Succeed?

Staff identified technical assistance tools to assist in the evaluation of heart disease and stroke prevention activities within States. States funded by the Centers for Disease Control and Prevention are charged with providing evidence of capacity, of intervention, and of change within their state. This analysis offers a consistent definition of terms, guidance on evaluation methods, and aid skill building on a wide range of general evaluation topics and selected topics.

Guidance was developed in four areas: writing objectives, developing an evaluation plan, developing and using a program logic model, and fundamentals of evaluating partnerships.

Report Title: Fundamentals of Evaluating Partnerships http://www.cdc.gov/dhdsp/state_program/evaluation_guides/index.htm
Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Julie Zajac, 404-498-4381
Performer: Staff; Centers for Disease Control and Prevention, Division of Heart Disease and Stroke Prevention
PIC ID: 9022

What Criteria Can Be Used to Identify Effective Evidence-Based Outcome Indicators of Program Progress?

Researchers reviewed scientific studies to identify indicators (data points) that could effectively assess progress in reaching heart disease and stroke prevention goals. A comprehensive set of indicators was identified for four key areas: hypertension, high cholesterol, stroke signs and symptoms and improving emergency response, and quality of care.

The indicator project has made it possible for public health entities to quickly and efficiently identify scientifically valid indicators to measure the effectiveness and progress of their programs. For assessing outcome indicators, researchers used the following criteria: (1) quality relative to evaluating state programs, (2) resources needed to collect and analyze data, (3) strength evidence that the indicator supports the assumption that implementing interventions will effect change, (4) utility for answering evaluation questions, (5) validity of data derived, and (6) consistency with accepted practice.

Report Title: Evaluation Indicators for the National Heart Disease and Stroke Prevention Program: High Blood Pressure Control; Report may be obtained from Federal Contact
Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Julie Zajac, 404-498-4381
Performer: Research Triangle Institute
PIC ID: 9023

How Do Two Highly Successful Community Clinics Control Patients' Blood Pressure?

Researchers evaluated the Stroke and Heart Attack Prevention Program (SHAPP). The program provided treatment and medicines for poor Georgians. The average rate of blood pressure control among patients in SHAPP clinics was about 60 percent, compared with the national average of 35 percent. This evaluation found out what made the SHAPP clinics so successful. Patients increased their awareness of their blood pressure numbers and improved in taking their medicines and in keeping their clinic appointments.

Enrolling in the clinic was easy and patients were able to get needed medicines free or at a low-cost. The clinics used the most up-to-date medical guidelines and patient tracking systems. They regularly followed-up with patients to see how they were doing and to remind them to come in for their appointments. There was good communication between the staff and patients. The patients found that the staff made time for them, were accepting and nonjudgmental of their low-income status, and treated them well. The staff treated the patients with respect and taught patients and their families about what to do to keep blood pressure under control. The staff was dedicated and knew that the SHAPP clinics filled a need in the community. The patients trusted the staff and were satisfied with the care they got. The patients said that without SHAPP they would not have help they needed to keep their blood pressure under control. Georgia is the only state that supports blood pressure clinics for the poor. Better blood pressure control among patients will result in less kidney disease, stroke and heart attack, preventable suffering, and early deaths for Georgians.

Report Title: Implementing successful blood pressure control strategies in low income populations: evaluation of a stroke and heart attack prevention program. http://www.cdc.gov/pcd/issues/2008/apr/07_0200.htm
Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Julie Zajac, 404-498-4381
Performer: Research Triangle Institute
PIC ID: 9024

What Factors Successfully Control High Blood Pressure and High Cholesterol?

In this qualitative case study, the Centers for Disease Control and Prevention identified clinical practices in health care organizations that used policy, environmental, or systems-level interventions to improve patient outcomes for high blood pressure and high cholesterol. Policy, environmental, and systems-level interventions are part of a comprehensive approach to managing high blood pressure and high cholesterol, which are key risk factors for heart disease and stroke.

Health care organizations succeeded because they supported patient self-management, integrated interventions into the practice&rsquo;s daily work flow to make implementation easier for staff, had effective leadership and committed staff, and facilitated community involvement. These results have important implications for clinicians and for policy makers. They show clinicians that establishing comprehensive systems of care matters and lead to better patient health. Policy makers may want to consider initiatives that require health care practices to adopt these systems of care.

Report Title: Strategies for Establishing Policy, Environmental, and Systems-Level Interventions for Managing High Blood Pressure and High Cholesterol in Health Care Settings: A Qualitative Case Study http://www.cdc.gov/pcd/issues/2008/jul/07_0218.htm
Agency Sponsor: CDC, Centers for Disease Control and Prevention
Federal Contact: Julie Zajac, 404-498-4381
Performer: Associations of Schools of Public Health; Emory University/Rollins School of Public Health
PIC ID: 9026

Can the National Institutes for Health and the Food and Drug Administration Integrate their Intellectual Property Portfolios?

The National Institutes of Health (NIH) Office of Technology Transfer, evaluates, protects, markets, licenses, monitors, and manages NIH and Food and Drug Administration (FDA) invention portfolios. The large number of inventions and their interdisciplinary nature make extracting and evaluating this information difficult. A proof-of-concept system (for verifying that an idea or technology likely works) was developed for analyzing, synthesizing, and visualizing NIH&rsquo;s and FDA&rsquo;s intellectual property portfolios.

The capabilities of the prototype tool developed to analyze and integrate data were significantly greater than capabilities available at the onset of the project. A number of network visualization tools were studied but none were technically adequate. Researchers recommended that the agencies take advantage of existing capabilities, using the advances provided as a result of having the prototype tool in hand.

Report Title: Project Catapult: Data System for Evaluation and Visualization of Relationships between Technologies; Report may be obtained from Federal Contact
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Bonny Harbinger, 301-594-7700

Performer: Discovery Logic, Inc.
PIC ID: 9044

How Evaluate Internet Based Health Services Research Information?

A web portal, Information Central (providing health services research information) was evaluated qualitatively. This new web portal is operated by the National Information Center on Health Services Research and Health Care Technology (NICHSR). NICHSR coordinates the development and management of information resources and services at the National Library of Medicine in the fields of health services research and public health. In 2005, NICHSR launched the Health Services Research (HSR) Information Central Web Portal, designed to centralize access to health services research information. Of especial interest, were determining the usefulness of the portal&rsquo;s content, and the usability of the web for the target users.

Online focus groups agreed that the information within the site was valuable but that accessing it was challenging. Its main strengths were the portal&rsquo;s depth and breadth of content, and its &lsquo;one-stop&rsquo; nature. Participants recommended several improvements: restructure and reorganize the site to make finding information easier and navigating the site clearer, use clarifying labels and terminology that provide brief clear explanations of the information in each category.

Report Title: Qualitative Research Study on HSR Information Central; Report may be obtained from Federal Contact
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Deshiree Belis, 301-496-5860
Performer: Solomon Solutions
PIC ID: 9046

How Best Evaluate a Health Information Program?

Study researchers explored whether it was possible and appropriate to design and conduct an evaluation of the Health Information National Trends Survey (HINTS) program. A communication evaluation could provide the National Cancer Institute with information that would help future access and use of HINTS program data. Researchers telephoned and emailed users and non-users of HINTS data, and reviewed the program website and publications.

Researchers advised that because many potential users did not yet know about HINTS, a full communications evaluation should be delayed until a later date. Researchers found that users of HINTS information were from academia or research areas, and non-users from public health practice.

Report Title: Feasibility of Conducting a Communications Evaluation of the Health Information National Trends Survey (HINTS) Program; Report may be obtained from Federal Contact
Agency Sponsor: NIH, National Institutes of Health
Federal Contact: Richard Moser, 301-496-0273
Performer: Academy for Educational Development (AED); ,
PIC ID: 9078

How Can Family Health Data Be Compiled Efficiently and Practically?

Researchers reviewed efforts of the American Health Information Community&rsquo;s (AHIC&rsquo;s) Multi-Stakeholder Workgroup. The Workgroup created the core data set for information on family health history, and worked on how to transfer such information into electronic formats for use in making health care decisions. Researchers examined electronic tools for recording family health information and noted that the current widely used tools do not provide a mechanism to transfer this data into systems used in clinical settings.

The reviewers reached the conclusion that the main goal of this project would be achieved if health data could be made accessible to all health care officials involved in continuous care for a patient, even as the patient moves between different medical facilities. This would require an electronic format which allows the user to collect, represent and interpret structured data on patient health which might aid in health care decisions. This core data set is a step towards that goal.

Report Title: Perspectives on Informatics: New Standards and Enhanced Utility for Family Health History Information in the Electronic Health Record http://www.jamia.org/cgi/reprint/15/4/391.pdf
Agency Sponsor: ASPE-OSDP, Office of Science and Data Policy
Federal Contact: Gregory Downing, 202-260-1911
Performer: American Medical Informatics Association
PIC ID: 9068

Appendix A - Federal Domestic Assistance Programs of the Department of Health and Human Services

Federal Domestic Assistance Programs of the Department of Health and Human Services[11]

Office of the Secretary

Office of Minority Health

Presidents Council on Physical Fitness and Sports

Office of Disease Prevention and Health Promotion

Agency for Healthcare Research and Quality (AHRQ)

Office of Population Affairs

Administration on Aging (AoA)

Administration for Children and Families (ACF)

Centers for Medicare & Medicaid Services (CMS)

Food and Drug Administration (FDA)

Centers for Disease Control and Prevention (CDC)

Health Resources and Services Administration (HRSA)

Indian Health Service (IHS)

Substance Abuse and Mental Health Services Administration (SAMHSA)

National Institutes of Health (NIH)

Agency for Toxic Substances and Disease Registry (ATSDR)

Appendix B - Strategic Goals and Objectives

HHS FY 2007-2012 Strategic Goals and Objectives

We cannot discover what ought to be the case by examining what is the case.

We must decide what ought to be the case.  – Paul Taylor

Goal 1: Health Care: Improve the safety, quality, affordability and accessibility of health care including behavioral health care and long term care.

Objective 1.1: Broaden health insurance and long-term care coverage.

Objective 1.2: Increase health care service availability and accessibility.

Objective 1.3: Improve health care quality, safety, cost, and value.

Objective 1.4: Recruit, develop, and retain a competent health care workforce.

Goal 2: Public Health Promotion and Protection, Disease Prevention, and Emergency Preparedness: Prevent and control disease, injury, illness, and disability across the lifespan, and protect the public from infectious, occupational, environmental, and terrorist threats.

Objective 2.1: Prevent the spread of infectious diseases.

Objective 2.2: Protect the public against injuries and environmental threats.

Objective 2.3: Promote and encourage preventive health care, including mental health, lifelong healthy behaviors and recovery.

Objective 2.4: Prepare for and respond to natural and manmade disasters.

Goal 3: Human Services: Promote the economic and social well-being of individuals, families and communities.

Objective 3.1: Promote the economic independence and social well-being of individuals and families across the lifespan.

Objective 3.2: Protect the safety and foster the well-being of children and youth.

Objective 3.3: Encourage the development of strong, healthy, and supportive communities.

Objective 3.4: Address the needs, strengths, and abilities of vulnerable populations.

Goal 4: Scientific Research and Development: Advance scientific and biomedical research and development related to health and human services.

Objective 4.1: Strengthen the pool of qualified health and behavioral science researchers.

Objective 4.2: Increase basic scientific knowledge to improve human health and human development.

Objective 4.3: Conduct and oversee applied research to improve health and well-being.

Objective 4.4: Communicate and transfer research results into clinical, public health, and human services.

Appendix C - Agency Mission and Evaluation Program Statements

Agency Mission and
Evaluation Program Statements

This appendix describes the mission and evaluation program for each agency and office in the Department of Health and Human Services that conducts evaluations. For those that have a dedicated evaluation web site, this is provided below. These resources supplement what is available from the Policy Information Center database at http://aspe.hhs.gov/pic/performance/.

Every agency and office seeks to maximize effectiveness and efficiency, consistent with the provisions of the Government Performance Results Act (GPRA), and the Program Assessment Rating Tool (PART). During the year covered by this report, a new component was added to this effort, Executive Order 13450, Improving Government Program Performance. The executive order established a Performance Improvement Council and directed each Department to appoint a Performance Improvement Officer to represent it on the Council and to promote effective and efficient work of the agencies. Robust evaluation contributes to an evolving effort to assure that programs function well.

OPERATING DIVISIONS

Administration for Children and Families

Mission

Promote the economic and social well-being of families, children, individuals, and communities.

Evaluation Program

The Administration for Children and Families (ACF) administers a broad range of formula and discretionary programs, including family self-sufficiency, child support, children and family services (Head Start, Child Welfare, Child Care Subsidies, Family Preservation and Support, and youth programs), and special programs for targeted populations, such as the developmentally disabled, refugees, and Native Americans.

ACF&rsquo;s evaluation objectives are to: furnish information on designing and operating effective programs; test new service delivery approaches capitalizing on the success of completed demonstrations; apply evaluation data to policy development, budget decisions, program management, and strategic planning and performance measures development; and disseminate findings of completed studies and promote application of results by state and local governments.

ACF stays current on emerging issues affecting its programs and identifying questions for evaluation studies by actively engaging other federal agencies, state and local policy and program officials, national organizations, foundations, professional groups and practitioners, and consumers.

Studies are often funded as joint ventures with the Office of the Assistant Secretary for Planning and Evaluation (ASPE) and other federal agencies and foundations. Such collaboration permits large-scale efforts that are better informed and more representative of varying perspectives. Multidisciplinary experts review proposals. Evaluation study designs are carefully developed in collaboration with project partners and technical experts in order to address specific research questions. Work groups of various kinds are used to monitor the progress of projects and to provide advice about design refinements and the presentation of findings.

Evaluation Web Site: http://www.acf.hhs.gov/programs/opre/index.html

Administration on Aging

Mission

Foster development of services to help older persons maintain their independence.

Evaluation Program

The Administration on Aging (AoA) is the federal focal point and advocate agency for the concerns of older persons. The agency administers key federal programs mandated under various titles of the Older Americans Act. These programs help vulnerable older persons remain in their own homes by providing supportive services. Other programs offer opportunities for older Americans to enhance their health and to be active contributors to their families, communities, and the nation through employment and volunteer programs. AoA works closely with its nationwide network of regional offices and state and area agencies on aging to plan, coordinate, and develop communitylevel systems of services that meet the unique needs of individual older persons and their caregivers. AoA collaborates with other federal agencies, national organizations, and representatives of business to ensure that, whenever possible, their programs and resources are targeted to the elderly and coordinated with those of the network on aging. As the responsibilities of this nationwide network of state and area agencies on aging continue to grow, it is essential that they have the necessary information to meet these responsibilities.

The overall evaluation priorities of the AoA are to support studies that provide information on: successful program implementation in meeting the goals of the Older Americans Act; design and operation of effective programs; and, issues relevant to policy development, and program management.

Web Site: http://www.aoa.gov/

Agency for Healthcare Research and Quality

Mission

Improve the quality, safety, efficiency and effectiveness of health care for all Americans.

Evaluation Program

The Agency for Healthcare Research and Quality (AHRQ) provides executive management, program officers and audiences external to the Agency with evaluative findings concerning the Agency&rsquo;s effectiveness and efficiency in order to meet its performance goals.  The work is conducted by external, independent evaluators.  Evaluation components are built into virtually all major AHRQ programmatic or portfolio activities beginning at the design phase.  Among evaluation mechanisms used by the Agency are targeted evaluation studies undertaken through contracts, using a variety of quantitative and qualitative methods, and that tend increasingly to provide more real-time monitoring feedback.  Evaluation activities also include satisfaction feedback from AHRQ customers regarding the usefulness of its research findings and dissemination products. Evaluation Web Site: http://www.ahrq.gov/research/

Agency for Toxic Substances and Disease Registry

Mission

Serve the public by using the best science, taking responsible public health actions, and providing trusted health information to prevent harmful exposures and disease related to toxic substances.

Evaluation Program

The Comprehensive Environmental Response, Compensation, and Liability Act (CERCLA), more commonly known as Superfund, created the Agency for Toxic Substances and Disease Registry (ATSDR) as a federal agency. ATSDR was created to carry out the healthrelated sections of CERCLA and other laws that protect the public from hazardous waste and environmental spills of hazardous substances. The ATSDR evaluation program is coordinated with the HHS-wide strategic planning process. ATSDR&rsquo;s strategic goals and its annual performance plan are the result of an interactive process that reflects a longterm commitment by Agency staff to develop stronger relationships among external clients and stakeholders, to assess products and services using relevant data, and to improve our processes and systems for more efficient accomplishment of its mission.

ATSDR Data Resources Web Site: http://www.atsdr.cdc.gov/2p-data-resources.html

Centers for Disease Control and Prevention

Mission

Promote health and quality of life by preventing and controlling disease, injury, and disability.

Evaluation Program

The Centers for Disease Control and Prevention (CDC) conducts evaluation studies designed to provide essential information about its programs, goals, and priorities. These projects support the assessment of CDC&rsquo;s strategies, which are to protect the health and safety of Americans, provide credible information to enhance health decisions, and promote health through strong partnerships.

CDC emphasizes evaluations that advance its health protection goals and answer policy, program and strategic planning questions related to the goals and objectives of Healthy People 2010. Performance improvement studies, such as those focusing on the development of key performance indicators are of particular interest and import to the Agency. CDC supports a variety of activities to enhance the quality, use, and understanding of evaluations.

Evaluation Web Site: http://www.cdc.gov/eval/index.htm

Centers for Medicare & Medicaid Services

Mission

Assure health care security for beneficiaries.

Evaluation Program

The research arm of the Centers for Medicare & Medicaid Services (CMS), the Office of Research, Development, and Information (ORDI), performs and supports research and evaluations of demonstrations (through intramural studies, contracts and grants) to develop and carry out new health care financing policies and provide information on the impact of CMS&rsquo; programs. ORDI&rsquo;s activities embrace all areas of health care: costs, access, quality, service delivery models, and financing. ORDI&rsquo;s responsibilities include evaluating ongoing Medicare and Medicaid programs and demonstration projects that test new health care financing and delivery approaches.

Examples of research themes include state program flexibility, the future of Medicare, provider payment and delivery, and vulnerable populations and dual-eligibles.

Evaluation Web Site: http://www.cms.hhs.gov/Reports/Reports/list.asp#TopOfPage

Food and Drug Administration

Mission

The FDA is responsible for protecting the public health by assuring the safety, efficacy, and security of human and veterinary drugs, biological products, medical devices, our nation&rsquo;s food supply, cosmetics, and products that emit radiation. The FDA is also responsible for advancing public health by helping to speed innovations that make medicines and foods more effective, safer, and more affordable; and helping the public get the accurate, science-based information they need to use medicines and foods to improve their health.

Evaluation Program

Evaluation plays an integral role in carrying out the FDA mission. Assessing various aspects of Agency program performance allows staff to identify means of improving that performance. The evaluation function has three goals: 1) provide information and analyses that helps Agency officials, the Department, and members of Congress make decisions related to programs, policies, budgets, and strategic planning; 2) help FDA managers improve program operations and performance; and 3) disseminate evaluation results and methodological tools useful to FDA program managers and, in some cases, to the larger public health community.

FDA evaluation activities serve one or more of the following purposes:

Performs program and policy evaluations and analytical studies of significantly broad Agency issues;

Recommends alternative courses of action to increase effectiveness of agency allocation of resources and to improve program and project performance;

Monitors program evaluation activities in Agency components and collaborates with DHHS in the development of the annual DHHS evaluation plan;

Applies quantitative and qualitative techniques to assess systems, processes, and operations to help Agency officials discover optimal courses of action; and

Assists and consults with Agency components to design, develop and complete FDA User Fee performance reports for Congress.

Evaluation Web Site: http://www.fda.gov/ope/org.html

Health Resources and Services Administration

Mission

Provide national leadership, program resources and services needed to improve access to culturally competent, quality health care.

Evaluation Program

The Health Resources and Services Administration (HRSA) supports a wide array of very different programs and activities that promote access to needed health care for underserved populations, including primary health care centers, the National Health Service Corps, HIV/AIDS programs, maternal and child health activities, health professions training, rural health programs, organ donation and transplantation initiatives, and telehealth activities. To provide underpinning for these efforts, HRSA&rsquo;s evaluation program is designed to enhance strategic planning, strengthen budget and legislative development, and improve program performance.

Evaluation Web Site: http://www.hrsa.gov

Indian Health Service

Mission

To raise the physical, mental, social, and spiritual health of American Indians and Alaska Natives to the highest level.

Evaluation Program

The goal of the Indian Health Service (IHS) is to assure that comprehensive, culturally acceptable, personal and public health services are available and accessible to American Indian and Alaska Native people. The importance of evaluation in supporting this goal has increased significantly in recent years and includes American Indians and Alaska Natives as the primary stakeholders in defining the purpose, design, and execution of evaluations. The stakeholders use the end product of the evaluations, and are the population or groups most likely to be affected by the findings. The IHS has formally adopted the principle of a responsive evaluation practice to address the needs and concerns of Native Americans and Alaska Natives.

The evaluation needs of the IHS service components are coordinated using two major types of shortterm studies: policy assessments and program evaluations. Policy assessments contribute to decision making about budget and program modifications including information to support the Agency&rsquo;s initiatives. Evaluations are focused at the program level, or Area Offices, and focus on specific needs.

The evaluation program of the IHS is managed by the Office of Public Health Support, Division of Planning, Evaluation, and Research, which provides national leadership and consultation for IHS and Area Offices on strategic and tactical planning, program evaluation and assessment, public health and medical services, research grants for Native Centers for Healthcare Research, and special public health initiatives for the Agency.

Planning and Evaluation Web Site:

http://www.ihs.gov/NonMedicalPrograms/PlanningEvaluation/index.asp

Research Web Site

http://www.ihs.gov/MedicalPrograms/Research/

National Institutes of Health

Mission

Uncover new knowledge that will lead to better health for everyone.

Evaluation Program

The National Institutes of Health (NIH) pursue new knowledge about the prevention, detection, diagnosis, and treatment of disease and disability. To that end, NIH has a wide range of programs to support health-related research and training and professional development. Evaluating these numerous and diverse programs is one important tool that NIH administrators use to determine the extent to which these programs are operating efficiently and achieving their intended outcomes.

NIH Institutes and Centers and components within the Office of the Director, use program evaluations and evaluation-related activities to improve decision-making and, ultimately, enhance program performance. Many NIH activities are crosscutting in nature and require program evaluations that involve more than one Institute, Center, or Office of the Director office to be examined effectively. Program evaluations are professional systematic investigations or studies that evaluate the merit of particular programs, or contribute to making such an evaluation possible. In most cases, the purpose of program evaluations is to help NIH administrators improve a program or make other programmatic decisions (e.g., how to allocate resources). A “ program” is broadly defined as any set of activities funded by the NIH to achieve one or more predefined goals.

NIH recognizes that results-based management is a basic requirement for the sound and productive operation of government agencies and their programs. With additional efforts to increase public sector accountability, such as passage of the Chief Financial Officers Act and the Government Management Reform Act, interest in evaluation has increased steadily among program administrators.

A distinguishing feature of the NIH Evaluation Program is its position within a larger institutional framework of several evaluation strategies including the use of national advisory councils, boards of scientific counselors, consensus development conferences, and ad hoc committees. This framework helps to chart scientific directions and select the most promising research to support.

Evaluation Web Site: http://opasi.nih.gov/desa/eb/intranet/setaside/fundedstudies.asp

Substance Abuse and Mental Health Services Administration

Mission

Build resilience and facilitate recovery for people with, or at risk for, substance abuse and mental illness.

Evaluation Program

The Substance Abuse and Mental Health Services Administration (SAMHSA) conducts evaluations to ensure accountability for federal funds and to measure results toward its programmatic and policy objectives. SAMHSA is improving performance management and results by identifying annual, long-term and cost-efficiency performance measures to manage its programs.

SAMHSA has a strategic planning process through which it identifies priorities that drive the development of grant programs and evaluations. The formulation of programmatic and evaluation priorities includes consultation with SAMHSA Center Advisory Councils, with other HHS agencies, and with experts in the fields of evaluation and service delivery. Early and continuous coordination of program planning and evaluation activities results in the articulation of program objectives that may be evaluated. Evaluations measure achievement of grant programs overall objectives, and these results are used for program and policy development. The strategic planning and policy development processes then use these results to refine SAMHSA&rsquo;s priorities and performance objectives.

The specific type of evaluation required depends on the type and purpose of the particular grant program. To the greatest extent possible, SAMHSA encourages the use of comparable data elements and instruments across its programs. Efforts to improve evaluation are continuing and SAMHSA is committed to using systematic approaches in using data to accomplish its mission.

Evaluation Web Site: http://ncadistore.samhsa.gov/catalog/results.aspx?h=drugs&topic=48

OFFICE OF THE SECRETARY

Office of the Assistant Secretary for Planning and Evaluation

Mission

Provide the Secretary analyses and advice for policy development, and help the development and coordination of department-wide program planning and evaluations.

Evaluation Program

The Office of the Assistant Secretary for Planning and Evaluation (ASPE) independently funds or conducts necessary policy and evaluation research; in partnership with others, especially HHS agencies, plans and carries out evaluations; and as required, provides oversight and advice to the Secretary regarding evaluation across the Department. To support its role as a principal advisor to the Secretary on policy development, ASPE conducts a variety of health and human services evaluation and policy research studies on issues of national importance. In its evaluation coordination role, ASPE provides annual guidance to all HHS agencies and staff offices regarding evaluation priorities, procedures, and review requirements and prepare planning and summary reports on evaluation activities as required by Congress; identifies crosscutting health and human services program or policy issues of particular concern to the Secretary and specific program and policy areas not covered by the HHS Agency evaluation plans; and conducts collaborative exploration of ways to strengthen evaluation activities across the Department.

ASPE supports and promotes the development and improvement of databases that HHS agencies and ASPE use to evaluate health care programs and health trends. ASPE co-chairs and provides support to the HHS Data Council, which is charged with integrating key national surveys, such as linking health status indicators with indicators of well being. ASPE uses evaluation funds to study and promote effective use of evaluation-generated information in program management and policymaking. The Office accomplishes this through disseminating evaluation findings and other activities, such as providing technical assistance to agencies developing performance measures. Working with ASFR, ASPE staff prepares the annual Evaluation Set-Aside Report that must be submitted to Congress before funds authorized by Section 241(a) of the Public Health Service Act are expended. ASPE also is responsible for coordinating and providing to the Secretary for transmittal to Congress, this annual Performance Improvement Report summarizing the findings of evaluations carried out by the Department.

Evaluation Web Site: http://aspe.hhs.gov/pic/performance/

Office of the Assistant Secretary for Financial Resources

Mission

The Office of the Assistant Secretary for Financial Resources (ASFR) provides advice and guidance to the Secretary on budget, financial management, and information technology, and grants management and provides direction and coordination of these activities throughout the Department.

Evaluation Program

ASFR systematically evaluates the effectiveness of HHS programs and strategies, including examination of program purpose and design, strategic planning systems, program management, and program results. ASFR oversees the development of the HHS annual performance plans and reports pursuant to the requirements of the Government Performance Results Act. The position of HHS Performance Improvement Officer, established by the Presidential Executive Order, Improving Government Program Performance, is housed in ASFR. In conjunction with ASPE, to prepare the annual Evaluation Set-Aside Report; ASFR staff generates the budget allocation tables for this report.

Office Web Site: http://www.hhs.gov/asrt/

Office of the Assistant Secretary for Preparedness and Response

Mission

The Office of the Assistant Secretary for Preparedness and Response (ASPR) is responsible for policy formulation, analysis, coordination, and evaluation for preparedness, response, and recovery planning and implementation.

Evaluation Program

In coordination with other Departmental offices, ASPR analyzes proposed policies, Presidential directives, and regulations, discharging those action items that fall within its authority. ASPR undertakes studies of preparedness, response, and recovery issues, identifying gaps in policy and initiating policy, research agendas, evaluation, planning and formulation of enterprises to fill these gaps. ASPR takes the lead on special projects, initiatives, and policy analysis and evaluation directly related to its areas of responsibility.

Office Web Site: http://www.hhs.gov/aspr/

Office of Inspector General, Office of Evaluation and Inspections

Mission

To protect the integrity of HHS programs, as well as the health and welfare of beneficiaries, by conducting evaluations that provide timely, useful, and reliable information and recommendations to decision makers and the public.[12]

Evaluation Program

OEI develops evaluation techniques and coordinates projects with other Office of Inspector General and HHS components. It provides programmatic expertise and information on new programs, procedures, regulations and statutes. It maintains liaison with other components in HHS, follows up on implementation of corrective action recommendations, evaluates the actions taken to resolve problems and vulnerabilities identified, and provides additional data or corrective action options, where appropriate.[13]

Evaluation Web Site: http://www.oig.hhs.gov/oei/oeisearch.html

Office of the National Coordinator for Health Information Technology

Mission

The Office of the National Coordinator for Health Information Technology (ONC) provides counsel to the Secretary and Departmental leadership of HHS for the development and nationwide implementation of an interoperable health information technology infrastructure. Use of this infrastructure will improve the quality, safety and efficiency of health care and the ability of consumers to manage their health information and health care.

Evaluation Program

The Office of the National Coordinator for Health Information Technology is engaged in funding research and programs to foster the development of interoperable nation-wide health information exchange. Initiatives are being funded to: harmonize standards for interoperability; certify electronic health record systems; evaluate the variation of organization-level business practices, policies, and state law that relate to privacy and security; and develop best practices and consensus-based policies for health information exchange.

Office Web Site: http://www.hhs.gov/healthit/

Office of Public Health and Science

Mission

Provide advice to the Secretary on public health and science, provide executive direction to program offices within the Office of Public Health and Science (OPHS), and, at the Secretary&rsquo;s direction, coordinate crosscutting public health and science initiatives in the Department. The Assistant Secretary for Health heads OPHS and is responsible for oversight of and policy development for the Public Health Service Commissioned Corps. The Surgeon General implements Corps policy and manages Corps operations.

OPHS Evaluation Program

The Office of Public Health and Science, provides advice, policy and program coordination, and leadership in the implementation, management, and development of activities related to public health and science, as directed by the Secretary. OPHS helps HHS conduct broadbased public health assessments to better address and solve public health problems. It assists other components of HHS anticipate future public health issues and helps ensure that HHS designs and carries out appropriate approaches, interventions, and evaluations that will maintain, sustain, and improve the health of the Nation. OPHS provides leadership and policy recommendations on populationbased public health and science and, at the Secretary&rsquo;s direction, leads or coordinates initiatives that cut across agencies and operating divisions. The Office communicates and interacts, on behalf of the Secretary, with professional and constituency organizations on matters of public health and science. It links important HHS programs or fill gaps in areas needing better policy formulation and coordination.

OPHS&rsquo; evaluation strategy focuses on public health and science issues that cut across multiple interests of the operating divisions and require a coordinated approach to achieve effective results. OPHS evaluations support the Assistant Secretary for Health as the Secretary&rsquo;s senior advisor for public health and science. OPHS conducts evaluations specific to the needs of the programs operated from the offices located within OPHS, such as women&rsquo;s health, minority health, disease prevention and health promotion, and research integrity. Some evaluation funds are also made available to the ten HHS Regional Health Administrators.

Evaluation Web Site: http://www.hhs.gov/ophs/index.html

Appendix D - Acronyms and Glossary

Acronyms and Glossary

1915(c) waivers  – refers to section 1915(c) of the Medicaid program allowing the Secretary of HHS to waive certain program requirements in the law. Waivers permit States greater flexibility to target program eligibility and provide home and community based services for the disabled and/or elderly populations.

Accountability  – responsibility for the expenditure, administrative and programmatic activities that occur in organizational units over which one has formal authority.

ACF  – Administration for Children and Families.

Acquired Immune Deficiency Syndrome &ndash; set of symptoms and infections resulting from the damage to the human immune system caused by human immunodeficiency virus.

Advance (health) care planning &ndash; timely discussions (and possibly preparation of written documents, such as advance directives) involving the patient, the family, and the physician about treatment options, including the length and invasiveness of treatment, chance of success, overall prognosis, and the patient's quality of life during and after treatment.

Advance directives (or advance health care directives)  – instructions given by individuals specifying what actions should be taken for their health in the event that they are no longer able to make decisions due to illness or incapacity.

AHIC  – see American Health Information Community.

AIDS  – see Acquired Immune Deficiency Syndrome.

American Health Information Community &ndash; federal advisory body chartered to recommend to DHHS how to speed development and adoption of health information technology. AHIC Successor, Inc. (which subsequently renamed itself the National eHealth Coalition), was incorporated as a public-private corporation in late 2008.

AOA  – Administration on Aging.

AHRQ  – Agency for Healthcare Research and Quality.

ATSDR  – Agency for Toxic Substances and Disease Registry.

Appreciative inquiry  – Appreciative inquiry rejects the more traditional &lsquo;problem-focused&rsquo; approach and instead seeks to identify what is working well or opportunities for positive change.

Arrears/arrearages  – in ACF, past-due child support payments.

Balanced Budget Act of 1997 (BBA)  – designed to balance the federal budget by 2002. Among many other things, the Act contained major Medicare reforms.

Balanced Budget Refinement Act of 1999 (BBRA)  – moved to reinstate some of the funding that the Balanced Budget Act (BBA) of 1997 had cut. It provided financial relief to Medicare and Medicaid providers and State Children&rsquo;s Health Insurance Programs (SCHIPs).

Bayesian  – a statistical approach that assesses the probability of a hypothesis being correct (for example, whether an association is valid) by incorporating the prior probability of the hypothesis and the experimental data supporting the hypothesis (Named after the Reverend Thomas Bayes, 1702-1761).

Benchmark  – standard or point of reference (often some standard of best practices) against which program processes or outcomes can be compared.

Best practices  – program models or activities for which effectiveness in achieving specified goals or objectives has been demonstrated or suggested across a number of implementations and evaluations.

Benefits Improvement and Protection Act of 2000 (BIPA)  – provided $35 billion over a 5-year period to hospitals, nursing homes, managed care plans, home health agencies, hospices, and DME providers to reinstate some of the reimbursements that the Balanced Budget Act of 1997 cut.

Bias  – systematic distortion in a measurement instrument which results in data that tend to be either too high or too low in relation to the true value of a measure.

Biologic  – a virus, serum, toxin, antitoxin, vaccine, blood, blood component or derivative, allergenic product, or other similar product used to prevent, treat or cure disease or injury.

CAHPS  – see Consumer Assessment of Healthcare Providers and Systems.

Capitation  – the system of payment for each customer served, rather than by service performed.

Case studies  – methods of inquiry that focus on intensive data collection and analysis that investigates only a few units of analysis.

CCF  – see Compassion Capital Fund.

CDC  – Centers for Disease Control and Prevention.

Charge compression  – the tendency of hospitals to hospitals tend to markup high cost items less than they markup low cost items (the term is used by the Centers for Medicare & Medicaid Services).

Clinical efficacy  – In a medical context it indicates that the therapeutic effect of a given intervention (e.g. intake of a medicine, an operation, or a public health measure) is acceptable.

CMS  – Centers for Medicare & Medicaid Services.

Commissioned Corps  – a force of more than 6,000 well-trained, highly qualified public health professionals dedicated to protecting, promoting, and advancing the health and safety of the Nation. The Surgeon General implements Corps policy and manages operations of the Corps including training and assignment of officers, deployment of special response teams to public health emergencies, and allocation of officers to underserved communities and populations. Commissioned Corps officers serve throughout Office of Public Health and Science (OPHS), in assignments across the Department of Health and Human Services, as well as in other agencies and programs, providing essential public health leadership and fulfilling service roles.

Community-Level Social Indicator  – a social indicator that can be collected, reported and meaningfully interpreted for geo-political units such as neighborhoods, towns or cities, metropolitan areas, or regions (see also, social indicator).

Community-Level Social Indicator System &ndash; a compilation of community-level social indicators based on data from one or more sources such as archival, surveillance, or administrative data developed for other purposes and, in some cases, other information including data developed from surveys implemented specifically for tracking local indicators.

Comparison group, control group  – people who are not exposed to the program and who are compared to the program group.

Compassion Capital Fund (CCF)  – Federal grant opportunities — for the purpose of organizational capacity building — that are of interest to faith-based and community groups. Managed by the Administration for Children and Families (ACF).

Cost-benefit (or benefit-cost) ratio &ndash; the ratio of total discounted program benefits to total discounted program costs.

Consumer Assessment of Healthcare Providers and Systems  – public-private initiative to develop standardized assessments of patients&rsquo; experiences with ambulatory and facility-level care.

Cost-benefit analysis  – process of comparing values of all benefits less those of related costs when benefits can be valued in dollars the same way as costs. A cost-benefit analysis is performed in order to select the alternative that maximizes the benefits of the program.

Cost-effectiveness analysis  – comparison of the relative costs and benefits of two or more approaches to a problem.

Cost-utility analysis  – a form of cost-effectiveness analysis of alternative interventions in which costs are measured in monetary units and outcomes are measured in terms of their utility, usually to the patient, e.g., using quality adjusted life years (QALYs).

Data Use and Reciprocal Support Agreement &ndash; the agreement that provides the legal foundation enabling the secure exchange of health information between and among participants and their users through the Nationwide Health Information Network.

Debt compromise  – process whereby a State agrees to accept reduction or elimination of child support debt owed to the State by a non-custodial parent.

Diabetes  – The World Health Organization recognizes three main forms of diabetes mellitus: type 1, type 2, and gestational diabetes (occurring during pregnancy), which have different causes and population distributions. While, ultimately, all forms are due to the beta cells of the pancreas being unable to produce sufficient insulin to prevent hyperglycemia, the causes are different. Type 1 diabetes is usually due to autoimmune destruction of the pancreatic beta cells. Type 1 diabetes is usually diagnosed in children and young adults, and was previously known as juvenile diabetes. In type 1 diabetes, the body does not produce insulin. Insulin is a hormone that is needed to convert sugar (glucose), starches and other food into energy needed for daily life.

Type 2 diabetes is characterized by insulin resistance in target tissues. This causes a need for abnormally high amounts of insulin and diabetes develops when the beta cells cannot meet this demand. Gestational diabetes is similar to type 2 diabetes in that it involves insulin resistance; the hormones of pregnancy can cause insulin resistance in women genetically predisposed to developing this condition.

Diagnosis-Related Groups (DRG) &ndash; classification system that groups patients according to diagnosis, type of treatment, age, and other relevant criteria. Under the prospective payment system, hospitals are paid a set fee for treating patients in a single DRG category, regardless of the actual cost of care for the individual. (See also Medicare-severity DRG).

Discounted/discounting  – process of determining the net present value of a dollar amount of costs or benefits.

Discount rate  – rate of interest used in discounting costs and benefits, that is, converting all costs and benefits over the life of the policy, program, or project into net present values.

DMEPOS  – Medicare durable medical equipment, prosthetics, orthotics, and supply suppliers.

DRG  – see Diagnosis-Related Groups.

Drug risk-adjustment  – the process by which the Centers for Medicare & Medicaid Services (CMS) modifies medical insurance payments to drug insurance plans for prescription drugs to account for known differences in health status of the enrollees served — to take into account the higher costs of individuals who have medical conditions which require greater use of pharmaceuticals — in order to neutralize incentives drug plans have to enroll only healthy individuals with low drug utilization in order to maximize their profits.  

Dual eligibles  – persons entitled to Medicare (Part A and/or Part B) and who are also eligible for Medicaid.

DURSA  – see Data Use and Reciprocal Support Agreement.

Effectiveness  – extent to which the observed outcomes are consistent with the intended objectives.

Enrollment growth  – increases in the number of people eligible for and receiving benefits.

Environmental factors  – factors in the surroundings of a program that may have an effect on it and on the intended outcomes.

Episode treatment group (ETG)  – a collection of claims and health care encounter data, including prescribed medications, that allows the identification of clinically homogenous, risk-adjusted episodes of care, regardless of treatment location or duration of care.

Evaluability Assessment  – systematic process used to determine the feasibility of a program evaluation. It also helps determine whether conducting a program evaluation will provide useful information that will help improve the management of a program and its overall performance.

Evidence-based  – a philosophy that emphasizes the importance of using defensible evidence as the basis for actions/decisions (“ evidence-based decision making is sometimes associated with performance management).

Experimental/impact design  – a research design involving one or more treatment (program) and control groups, where program and control participants are randomly assigned to the groups, ensuring that the groups are equal except for the program itself.

FDA  – Food and Drug Administration.

Fee-For-Service  – system in health care by which particular services are paid for individually rather than provided as part of a comprehensive plan.

FFS  – see Fee-For-Service.

Focus group  – group of persons selected for their relevance for a particular evaluation question.

Formative evaluation -- an assessment of program efforts prior to their completion for the purpose of improving the efforts; examples include developmental and implementation evaluation.

Flores Agreement — requirements for proper treatment of alien children held or detained by the immigration service (such as not intermingling children and adults in detention centers); named after a child that became the center of a series of exchanges both in and outside the courts.

(Strategic) Goals  – 5-year, broad directions sought by the Department of Health and Human Services revised/updated at least every 3 years, by law. (Also See “ Objectives” ).

Grouper (software)  – software that permits the electronic creation and organization of episode treatment groups (see Episode treatment group for further discussion).

Health Plan Management System &ndash; administrative data collection and management system for the Medicare Advantage and and Part D programs; central e-government system for day-to-day operations of these programs.

HCBS  – see Home and Community-Based Services.

Health Information Exchange (HIE)  – a multi-stakeholder entity that enables the electronic movement of health-related data within state, regional or non-jurisdictional participant groups.

Health Information Technology (HIT) &ndash; software and infrastructure used in the clinical practice of medicine to support documentation, storage and exchange of patient data.

Health Maintenance Organization (HMO) &ndash; form of health insurance combining a range of coverages on a group basis. A group of doctors and other medical professionals offer care through the HMO for a flat monthly rate with no deductibles. However, only visits to professionals within the HMO network are covered by the policy. All visits, prescriptions and other care must be cleared by the HMO in order to be covered. A primary physician within the HMO handles referrals.

Health Plan Employer Data and Information Set  – a list of about 60 standardized performance measures developed and maintained by the National Committee for Quality Assurance (NCQA) in Washington, DC. Currently, most health plans report their results directly to NCQA as well as to their larger customers.

HEDIS  – see Health Plan Employer Data and Information Set.

HHS  – United States Department of Health and Human Services.

HIE  – see Health Information Exchange.

HIT  – see Health Information Technology.

HIV  – see Human Immunodeficiency Virus.

HMO  – see Health Maintenance Organization.

Home and Community Based Services (HCBS) &ndash; programs that offer different choices to some people with Medicaid; if a person qualifies, they receive care in their home and community so they can stay independent and close to family and friends. HCBS programs help the elderly and disabled, mentally retarded, developmentally disabled, and certain other disabled adults. These programs give quality and low-cost services.

Homebound restriction  – implicit statutory/regulatory limitation on the circumstances under which a patient may receive home health services; such services are available to patients who are confined to their home (or homebound). In general, a patient is considered homebound if they have a condition due to an illness or injury that restricts their ability to leave their place of residence except with the aid of supportive devices. The patient must be in need of intermittent skilled nursing care or physical or speech therapy.

HPMS  – see Health Plan Management System.

HRSA  – Health Resources and Services Administration.

IHS  – Indian Health Service.

Impact evaluation  – Focuses on the long-range results of the program or project, and changes or improvements as a result (e.g., long-term maintenance of desired behavior, reduced absenteeism from work, reduced morbidity and mortality).  Because such evaluations are the most comprehensive and focus on long-term results of the program and changes or improvements in health status, they are often seen as the most desirable.  However, they may also be difficult to conduct, costly, or involve extended commitment. Also challenging may be relating the results directly to the effects of the program, project, or activity due of the presence of other (external) influences on the target population.

Income  – infusion of cash/credit from both earned (typically from employment) and unearned (such as capital investments) sources.

Income recipiency -- income received as a result of eligibility under a public program; term often used in the context of welfare receipt.

Information Technology (IT)  – a general term encompassing the use of hardware, software and services to create, store, retrieve, transfer, process and present information. IT projects typically involve the introduction or enhancement of systems or technology to meet a particular business need.

Input Measure  – measure of what an agency or manager has available to carry out the program or activity to produce an output or outcome.

Insular Areas  – A United States territory that is neither a part of one of the 50 states nor the District of Columbia.

Interoperability  – ability of two or more systems or components to exchange information and to use the information that has been exchanged.

Interoperable health information exchange &ndash; the ability of two or more health information systems to exchange and use health-related information.

LIHEAP  – see Low Income Home Energy Assistance Program.

Logic model  – tool for planning, implementing, and evaluating programmatic efforts, by mapping out the theory or rationale that supports what is being done. Logic models typically tie together: long-term problem(s) to be addressed; factors that must be addressed that contribute to the problem(s); strategies and practices, and supporting resources, that can be mobilized to address the factors and the problems; and measurable impacts and outcomes that can be expected to result from implementing the strategies and practices  – as these relate to the long-term problem(s).

Low Income Home Energy Assistance Program &ndash; A block grand program that assists low income households, particularly those with the lowest incomes that pay a high proportion of household income for home energy, primarily in meeting their immediate home energy needs.

MA-PD  – see Medicare Advantage-Prescription Drug.

Medicaid  – joint Federal and State program that helps with medical costs for some people with low incomes and limited resources. Medicaid programs vary from state to state, but most health care costs are covered if a person qualifies for both Medicare and Medicaid.

Medicaid Information Technology Architecture (MITA)  – framework developed by CMS to help States modernize their Medicaid information systems.

Medicare  – Federal health insurance program for persons 65 years of age or older, the disabled, and those with end-stage renal disease.

Medicare Advantage (“ Part C” ) &ndash; program that gives a person more choices among health plans. Everyone who has Medicare Parts A and B is eligible, except those who have End-Stage Renal Disease (unless certain exceptions apply). Medicare Advantage Plans used to be called Medicare + Choice Plans.

Medicare Advantage-Prescription Drug &ndash; Medicare Advantage plan that includes a prescription drug payment benefit.

Medicare Economic Index (MEI)  – index often used in the calculation of the increases in the prevailing charge levels that help to determine allowed charges for physician services. In 1992 and later, this index is considered in connection with the update factor for the physician fee schedule.

Medicare+Choice (M+C) (now referred to as Medicare Advantage or Part C plans)  – public or private entity organized and licensed by a State as a risk-bearing entity (with the exception of provider sponsored organization receiving waivers) that is certified by CMS as meeting the M+C contract requirements.

Medicare physician payment update process &ndash; annual revisions to the amounts physicians are reimbursed, based on a statutory formula. It utilizes a comparison between target spending for Medicare physicians&rsquo; services and actual spending.

Medicare Prescription Drug, Improvement, and Modernization Act (MMA) of 2003 &ndash;statute that updated, revised portions of the Medicare program, allowed for increased benefits, and provided seniors with prescription drug benefits and more choices in health care.

Medicare private fee-for-service (PFFS) &ndash; type of Medicare Advantage Plan in which a person may go to any Medicare-approved doctor or hospital that accepts the plan&rsquo;s payment. The insurance plan, rather than the Medicare Program, decides how much it will pay and what the beneficiary pays for the services they get. The patient may pay more or less for Medicare-covered benefits and may receive extra benefits that the original Medicare plan doesn&rsquo;t cover.

Medicare severity DRG  – refinement of the diagnosis-related group (DRG) classification system to more fairly compensate hospitals for treating severely ill Medicare inpatients.

MedPAR  – Medicare Provider Analysis and Review file, a data file containing records for all Medicare beneficiaries who use hospital inpatient services. The records are stripped of most data elements that would permit identification of beneficiaries.

Mental Health Parity  – equivalence between mental health benefits and general health care benefits in health insurance plans.

Meta-analysis, meta-evaluation  – systematic analysis of a set of existing evaluations of similar programs in order to draw general conclusions, develop support for hypotheses, and/or produce an estimate of overall program effects.

MMA  – see Medicare Prescription Drug, Improvement, and Modernization Act of 2003.

MMA benefit stabilization funds  – a Medicare+Choice organization can request that an excess amount be withheld and reserved in a stabilization fund. This fund is used to stabilize and prevent undue fluctuations in additional benefits required.

MS-DRG  – see Medicare severity DRG.

National Health Plan Collaborative &ndash; in NIH, a project to bring together major health insurance companies, in partnership with organizations from the public and private sectors, to identify ways to improve the quality of health care for racially and ethnically diverse populations.

National Long Term Care Survey  – longitudinal survey, funded by the National Institute on Aging, NIH, designed to study changes in the health and functional status of older Americans (aged 65+).  It tracks health expenditures, Medicare service use, and the availability of personal, family, and community resources for caregiving.  The survey began in 1982; follow-up surveys were conducted in 1984, 1989, 1994, 1999, and 2004.

Nationwide Health Information Network &ndash; a &lsquo;network of networks,” intended to connect diverse entities that need to exchange health information, such as state and regional health information exchanges (HIEs), integrated delivery systems, health plans that provide care, personally controlled health records, Federal agencies, and other networks as well as the systems they, in turn, connect.

Needs assessment  – study that measures the nature and extent of the need for a program, either before a new program is developed or during its lifetime.

NIH  – National Institutes of Health.

NHIN  – see Nationwide Health Information Network.

Non-custodial parent  – a parent who does not have physical custody of a child and who is typically paying child support for the child, as directed by a court.

Non-experimental design  – only one group receiving the intervention is being observed or studied without the use of a comparison group to control for outside factors.  Such designs generally involve less data collection and are easier to plan and carry out. They typically involve observing and/or collecting all relevant data, including data on key performance measures, on participants at selected points in time during the project.  Examples of such design include, but are not limited to, case studies, structured interviews, surveys, pre-/post-tests, ethnographic studies, and document reviews. 

(Strategic) Objectives &ndash; statements derived from program goals that explain how the program goals will be accomplished; objectives are well-defined, specific, quantifiable statements of the program's desired results; they should include the target level of accomplishment, thereby further defining goals and providing the means to measure program performance.

Office of Child Support Enforcement (OCSE)  – located within ACF; seeks child care payments due from absent parents.

Office of Minority Health (OMH)  – located within OPHS, provides leadership for, and coordination of, racial and ethnic minority health and health disparities efforts within HHS.

Office of Inspector General  – although many federal agencies have an Inspector General, when used in this report, the Office of Inspector General or the Inspector General always refers to the Inspector General of the United States Department of Health and Human Services. Inspectors General have a dual reporting responsibility &ndash; their reports are not cleared by the Department in which they work and through which they receive their funding and staffing. This is different from all other components which must clear their work through senior departmental officers before being released to Congress or the public.

OPHS  – Office of Public Health and Science.

Opportunistic infection  – an infection caused by pathogens that usually do not cause disease in a healthy immune system; a compromised immune system, however, presents an opportunity for the pathogen to infect.

OSCAR  – Online Survey, Certification and Reporting data system, provides staffing data for all U.S. nursing homes that Medicare and/or Medicaid certifies. State survey and certification agencies collect the data, which are part of the annual nursing home certification and recertification process. Each facility completes a standardized form about the facility characteristics, e.g., number of beds, affiliation, etc., resident characteristics, e.g., limitations, chair bound, etc., and staffing levels. State surveyors review the form and enter the data into the OSCAR database. State surveyors also visit each facility and decide whether the facility meets each standard.

Outcome evaluation  – used to obtain descriptive data on a program or project and to document (typically) short- and intermediate-term results. Task-focused results are those that describe the output of the activity. Shorter-term results describe the immediate effects of the project on the target audience.  Information from such evaluation can show results such as knowledge and attitude changes, short-term or intermediate behavior shifts, and policies initiated or other institutional changes.

Outcome Measure  – measure of an event, occurrence, condition, or result of a program or project that indicates achievement of objectives and goal(s); this type of measure is used to measure the success of a program, project, or system (e.g., the percentage of people who do not get influenza).  

Output Measure  – measure of a product, service, or result of a particular activity (e.g., number of people vaccinated with the influenza vaccine, number of personnel trained; number of phone calls processed by the OMH Resource Center); this type of measure provides information about the activity, not the success in achieving the objectives and goals of the program/project.

Parity  – see Mental Health parity.

Part A  – Medicare hospital insurance coverage.

Part B  – Medicare medical insurance that helps pay for doctors' services, outpatient hospital care, durable medical equipment, and some medical services not covered by Part A.

Part C  – Medicare Advantage plans, approved by Medicare (like HMOs and PPOs) and offered by private companies; Medicare pays a fixed amount monthly to the company providing the care for the beneficiary.

Part D  – coverage for prescription drug benefits.

Part D sponsors  – an entity that offers or provides a prescription drug plan consistent with the provisions of the the Medicare statute.

PDP  – see Prescription Drug Plan.

Performance measurement  – process of designing and implementing quantitative and qualitative measures of program results, including outputs and outcomes.

Population  – group of people that may or may not be from the same geographic area, who receive services from public sector or nonprofit organizations.

PPO  – see Preferred Provider Organization.

Preferred Provider Organization  – plan that (a) has a network of providers that have agreed to a contractually specified reimbursement for covered benefits with the organization offering the plan, (b) provides for reimbursement for all covered benefits regardless of whether the benefits are provided with the network of providers, and (c) is offered by an organization that is not licensed or organized under State law as an HMO.

Prescription Drug Plan  – private prescription drug payment benefit provided for in Part D of the Medicare Program; may be separate (sometimes referred to as “ stand alone” drug plans) or combined with Medicare Advantage (HMO or PPO) plans (sometimes referred to as “ comprehensive health plans” ).

Probabilistic analysis  – a way to define statistical distributions for input parameters (e.g. joint orientation, shear strength, water level), to account for uncertainty in the values of input parameters. When the analysis is computed, this results in a safety factor distribution from which a probability of failure is calculated.

Process/implementation evaluation &ndash; examination of the tasks and procedures involved in implementing a program or activities, including the administrative and organizational aspects of, and delivery procedures involved in, the efforts.  Such evaluations enable monitoring to ensure feedback during the course of the program or project.

Process measure  – measure of the procedures, tasks, or processes involved in implementing program or project activities to produce an output or outcome.

Program  – group of individual (grantee) projects, unified by a set of features such as goals, health issues of focus, recommended types of activities, and eligible grant recipients.

Project  – an individual project (grantee), usually within an overall program, addressing one or more specific target populations or communities, and health issues.

Program components  – major clusters of activities in a program that are intended to drive the process of producing outcomes.

Program effectiveness  – extent to which the program achieves its intended outcomes.

Program evaluation  – systematic process for gathering and interpreting information intended to answer questions about a program.

Program outcomes  – what the program appears, through a process of measurement, to have achieved.

Proof of concept  – (especially in the context of developing advanced research or biomedical advances in NIH and FDA) short or incomplete realization/synopsis of a certain method or idea to demonstrate its feasibility, or a demonstration in principle, whose purpose is to verify that some concept or theory is probably capable of exploitation in a useful manner.

Public Health Service Commissioned Corps &ndash; see Commissioned Corps.

Public health surveillance  – the ongoing, systematic collection, analysis, interpretation, and dissemination of data regarding a health-related event for use in public health action to reduce morbidity and mortality and to improve health.

Purposive sampling  – sampling that targets a particular group of people. When the desired population for the study is rare or very difficult to locate and recruit for a study, purposive sampling may be the only option.

QALY  – see quality-adjusted life-years.

Quality-adjusted life-years  – a way of measuring both the quality and the quantity of life lived, as a means of quantifying in benefit of a medical intervention. It is a method of estimating utility that assigns a preference weight to each health state, determines the time spent in each state, and estimates life expectancy as the sum of the products of each preference weight and the time spent in each state.

Quasi-experimental  – research designs that do not involve random assignment to program and control groups but do include comparisons that make it easier to sort out the cause and effect linkages that are being tested.

Randomized controlled trials  – see experimental design, randomized experiments.

Randomized experiments  – research designs that involve randomly assigning units of analysis (usually people) to program and control groups.

Random sample  – sample that is selected at random from the population, where each member of the population has an equal or known chance of being selected, which enables the research results to be generalized to the whole population.

Reinsurance  – insurance that a primary insurer purchases to protect it against risk (in Medicare, typically purchased by some prescription drug payment plan providers). The primary insurer pays a premium to a reinsurer in exchange for protection against higher-than-expected claims.

Relevance  – extent to which the objectives of the program are connected to the assessed needs.

Reliability  – extent to which a measurement instrument produces consistent results over repeated applications.

Representative sample  – when the characteristics of a sample (demographic characteristics, for example) match those same characteristics for the population, the sample is said to be representative.

Resource and Patient Management System &ndash; an integrated data system for management of clinical, business practice and administrative information in Indian Health Care facilities.

RPMS  – see Resource and Patient Management System.

SAMHSA  – Substance Abuse and Mental Health Services Administration.

SCHIP  – State Children&rsquo;s Health Insurance Program.

Social Indicator  – a measure reflecting the status of the population (e.g., age range, income level, education attainment), and contextual influences (e.g., social, economic, ecological, and political influences) known to affect well-being at a particular time or over a period of time.

Social Security Administration (SSA) &ndash; Federal agency responsible for administering the Social Security program, which provides economic relief to citizens. The agency also plays important roles in public health, including disability determination and supporting electronic death registration systems.

Stakeholders  – all persons, agencies and organizations with an investment in the health and well-being of the community and the local public health and welfare systems.

Statistical significance &ndash; when the analysis of data results in statistical significance, it means that the result is not likely to have occurred by chance.  It confirms a relationship or difference between variables.

Strategic objectives  – see (Strategic) objectives, above.

Summative evaluation  – look at a combination of measures and conclusions for larger patterns and trends in performance, to assess, in summary, whether the program or project overall did what it was designed to do.  Summative evaluations are primarily retrospective, document evidence, and show results and achievement.  Examples of summative evaluations include outcome and impact evaluations, cost-effectiveness and cost-benefit analyses, and meta-analyses. (see also Formative evaluation).

Sustainable Growth Rate (SGR)  – system for establishing goals for the rate of growth in expenditures for physicians' services.

TANF  – see Temporary Assistance for Needy Families.

Temporary Assistance for Needy Families &ndash; welfare reform program enacted in 1994 that sought to encourage work rather than permanent welfare dependency by mandating training and movement toward employment and by setting a five-year maximum for the duration of welfare receipt.

Treatment groups  – persons who are provided with a program or some other intervention that is being evaluated.

Triangulation  – process of collecting data to answer an evaluation question from a variety of sources and/or using a variety of measurement procedures.

Type 1 (or Type 2 or gestational) diabetes  – see Diabetes.

Uniform Data Set ( UDS) -- systematic data reporting system recently developed for all OMH-funded activities that organizes data collection and reporting by type of activity conducted. The UDS is an Internet-based system. 

UAF  – see Update Adjustment Factor.

Update Adjustment Factor  – a payment revision method that includes the sustainable growth rate and squares up the actual Medicare expenditures with targeted Medicare expenditures for the year.

Utility  – in economics, utility is a measure of the relative satisfaction from or desirability of consumption of goods.

Utilization  – extent to which the program evaluation process and results (findings, conclusions, and recommendations) are deemed by stakeholders to be useful to them.

Validity  – extent to which a measuring instrument measures what it is intended to measure.

Variable  – an observable characteristic that we expect will be affected by one or more independent variables  – in most evaluations, the observed outcomes are dependent variables.

Appendix E - Studies by Agency

Studies by Agency

Appendix F - How To Obtain More Information

How To Obtain More Information

Here are 5 ways to can more information.:

1. Conduct a search of the full Policy Information Center (PIC) Database at http://aspe.hhs.gov/pic/performance/. The PIC database contains over 8,000 additional summaries, many with links to full evaluation reports. See below for more about this.

2. Call the Federal Contact identified for each entry in Chapter II. This individual, either the Project Officer responsible for the study or another individual from the office of agency that conducted the study, can provide copies of the report or other information.

3. Contact the Performer, identified for each entry, who did the work of the study. More than likely, they have a web site. They may also post copies of their final reports on the study even when the PIC Database does not identify it as available. A caution: since we can only provide the name of the performer, this route may be problematic although prominent organizations have well designed and up-to-date web sites.

4. Check the sponsoring agency&rsquo;s web site, found in Appendix C; some agencies make all work they carry out available this way (for example the HHS Office of Inspector General routinely posts all its reports).

5. Search the full HHS web site, the main gateway to HHS online, http://www.hhs.gov/.

MORE ABOUT SEARCHING THE PIC ONLINE DATABASE

Option 1  – Clicking the link in this report on line. Each summary in Chapter III of this report for which there is an online report has a hot link you can select.

Option 2  – To see all the annual reports in this series, go to the PIC database, at http://aspe.hhs.gov/pic/performance/ and click “ Performance Improvement Reports.”

Option 3  – To search the PIC database, go to http://aspe.hhs.gov/pic/performance/ and click “ Search Full Database.” Here there are three ways to find contents:

A.     If you know the unique 4-digit number (the “ PIC ID” ), enter it in the second field on the search page and press Enter to retrieve the individual record.

B.     Conduct a Full-Text Search (described below).

C.     Conduct a Specialized Search (described below).

Also, the ASTERISK ( * ) may be used. It functions as a “ wild card” (Example: “ immuniz*” will retrieve both immunize and immunization).

Proximity Designators  – Another option is to use the formula WORD<near/#>WORD. (Example: “ child<near/5>welfare” retrieves summaries in which the words “ child” and “ welfare” appear within 5 words of each other).

After entering your search, click SEARCH immediately to the right of the text entry box.

Editorial note: the full text search capability, outlined above, is the most powerful of all the search strategies and is, generally, the approach recommended when seeking all the studies on a particular area of interest. Of course, it will always be necessary to experiment with terms to use to obtain the most and clearly relevant retrievals.

Conduct Specialized Searches

There are 5 fields allowing the user to conduct specialized searches:

PIC ID (already described above)

SEARCH DESCRIPTORS

AGENCY

PROJECT OFFICER

CONTRACT PERFORMER

Both the PIC ID and the Search Descriptors of PIC database fields have dedicated SEARCH buttons. Only these buttons, next to the field, will result in a search of that field and that button can only be used for that one field:

1. PIC ID

If you know the unique number of the database item, enter its 4-digit number or number with suffix (e.g., 8546 or 8546.2) and click the SEARCH button to the right of the field.

This is the failsafe way to keep track of a particular record. Once a number is assigned, it is always the same, even when other records are deleted from the system.

2. Search Descriptors of PIC Database

(1)   Highlight individual or groups of terms (“ Control” and click allows you to select several individual terms; “ Shift” and click allows you to select several contiguous terms)

(2)   Click on the >> button between the boxes

(3)   Highlight the descriptors in the right-hand box

(4)   Click the SEARCH button that is immediately below the two boxes

Editorial note: contents of this field are selected by project officers; not all records for which the descriptors apply have been so labeled; there is no review or standardization by HHS about how the terms are used. A search using the whole text field will likely reveal more records in which the descriptors apply. On the other hand, some records, in which the specific term/descriptor is not used may still present information relevant to that policy area. You may wish to search separately using both descriptors and the whole text search features in order to assure the most inclusive record selection; this applies to the fields below as well.

3. The last set of 3 fields, described below, have a single SEARCH button serving them (at the bottom of the search page) and also a CLEAR button that serves only these last 3 fields. These fields can be searched individually or in combination with one another.

(1)   In one or more of the three fields, highlight individual or groups of terms (“ Control” and click allows you to select several individual terms; “ Shift” and click allows you to select several contiguous terms).

(2)   Click the SEARCH button that is immediately below the three boxes

These three fields contain an historical record of ALL the agencies, project officers, and contract performers who have EVER been associated with one or more records that were, or are, in the database even if they are no longer.

Editorial note: regarding search strategy, if for instance, you were interested in finding studies carried out by 5-6 different contract performers, you could either run separate searches or, if you wished a consolidated output, you could select and highlight all of the different performers. The search results would combine these into a single set. The downside is that they would not be grouped other than in the order the individual records are maintained in the database.

The search engine can search for groups of entities in more than one field simultaneously, for instance. The three areas that can be searched are:

A. Agency  – the Shift and Control features are particularly useful for this field where there are multiple entries for most agencies

B. Project Officer  – this field is valuable for an individual who wants to track or retrieve the final entries under their own, or a particular person&rsquo;s name.

4. Contract Performer  – this field, especially in conjunction with the Shift and Control features, is also useful in locating studies by an entity or group of related (or unrelated!) entities.

Example of a search strategy

How to find and look at all the records for a particular agency:

1. go to: http://aspe.hhs.gov/pic/performance/
2. click on the Search Full Database tab
3. after the full page loads, scroll down to the “ Agency” field (4^th one down)
4. click into the box, type the first letter of the agency in which you are interested
5. click on the first entry for a series of related agency listings so that it is highlighted
6. leaving the first item highlighted, use the right hand scroll bar to move to the end of the related series
7. holding the SHIFT key on your keyboard down, click on the last entry for the agency series --- at this point all the related agency items should be highlighted
8. click outside the box
9. scroll to the bottom of the page
10. click SEARCH

Brief entries for the write ups for a group of agency entries are shown, about 15 to a page. The agencies included in the search are listed across the top of the screen so you can check to see that they were all included in the search. Click on the red bolded titles of each entry and see the full database entry. Entries for which the reports are online will also have hot links to these reports.

Appendix G - What characterizes An Evaluation?

What characterizes An Evaluation?

For the purposes of deciding whether a project or study belongs in the Policy Information Center database of evaluations and hence in the annual Performance Improvement report to Congress, we encourage agency staff to cast a wide net. This Appendix provides a discussion to aid in this task.

Evaluation includes the process of determining the worth or value of something. It can be the analysis and comparison of actual progress versus prior plans, oriented toward improving plans for future implementation. The American Evaluation Association defines evaluation as assessing the strengths and weaknesses of programs, policies, personnel, products, and organizations to improve their effectiveness. Evaluation is the systematic collection and analysis of data needed to make decisions. Evaluation activities may include performance-related events:

• Pinpointing the services needed.
• Finding out what knowledge, skills, attitudes, or behaviors a program should address.
• Establishing clear, measurable, and realistic program objectives and deciding the particular evidence that will demonstrate that the objectives have been met.
• Developing or selecting from among alternative program approaches and determining which ones best achieve the goals.
• Tracking whether program objectives are achieved; setting up a system that shows who gets services, how much service is delivered, how participants rate the services they receive, and which approaches are most readily adopted.
• Trying out and assessing new program designs; determining the extent to which a particular approach is being implemented faithfully or the extent to which it attracts or retains participants.

Studies conducted or funded by an HHS agency or office would be included in the PIC evaluation database if they substantially met one or more of the following criteria:

• Consisted of systematically collected and assessed information concerning, and provided useful feedback about, a program, population, social environment, policy, technology, need, methodology, or activity;
• Generated information intended to inform policy decisions about or improve program effectiveness, advance the design, operation, or focus of a program or provided information regarding the context or clients served by a program;
• Answered agreed-upon questions and provided information on specific criteria;
• Included analysis of data and careful interpretation;
• Resulted in information derived from direct observations or a compilation of other primary data collections;
• Sought to provide findings that were action-focused and directed to users for whom the information would have practical value and influence thinking, policymaking or program design; or
• Assessed effectiveness of an ongoing program in achieving its objectives, relied on the standards of experimental design to distinguish a program's effects from those of other forces, and aimed at program improvement through a modification of current operations.

Study Types Entered in PIC Evaluation Database

Researchers use many near-synonymous terms to describe their work. Here is a partial presentation to parse out some of the distinctions found in the current report.

Process, Implementation or “ Formative” Evaluation

These terms tend to overlap in meaning. These evaluations focus on early stages in program implementation and operation, before formal outcomes are apparent. It identifies procedures undertaken and decisions made in developing a program. It describes how the program operates, the services it delivers, and the functions it carries out. Such evaluations addresses whether the program was implemented and is providing services as intended. However, by additionally documenting the program's development and operation, it allows an assessment of the reasons for successful or unsuccessful performance, and provides information for potential replication.

Formative evaluations are a type of process evaluation of new programs or services that focus on collecting data on program operations so that needed changes or modifications can be made to the program in the early stages. Formative evaluations are used to provide feedback to staff about the program components that are working and those that need to be changed. Such an evaluation may used by managers as an aid to decide which strategy a program should adopt in order to accomplish its goals and objectives at a minimum cost. In addition, the evaluation might include alternative specifications of the program design itself, detailing ideal milestone and flow networks, manpower specifications, progress objectives, and budget allocations.

Outcome or “ Summative” Evaluation

An evaluation used to identify the results of a program's effort. It seeks to answer the question, "What difference did the program make?" It provides management with a statement about the net effects of a program after a specified period of operation. This type of evaluation provides information on: (1) the extent to which the problems and needs that gave rise to the program still exist, (2) ways to ameliorate adverse impacts and enhance desirable impacts, and (3) program design adjustments that may be indicated for the future. Such an evaluation may contribute to performance evaluation by comparing actual performance with that planned in terms of both resource utilization and production. It may be used by management to redirect program efforts and resources and to redesign the program structure. Impact Evaluation and Cost-Benefit Analysis are forms of Outcome or Summative Evaluation.

Impact Evaluation and “ Interim” Impact Assessments

A type of outcome evaluation that focuses on the broad, long-term impacts or results of program activities. For example, an impact evaluation could show that improved grade-school performance was the direct result of local Head Start programs. An impact evaluation would typically include an experimental, random assignment design.

Cost-Benefit Analysis

An analysis that compares present values of all benefits less those of related costs when benefits can be valued in dollars the same way as costs. A cost-benefit analysis is performed in order to select the alternative that maximizes the benefits of a program.

Feasibility Study, Evaluability Assessment, Evaluation Protocol Development

Feasibility Study is a study of the applicability or practicability of a proposed action or plan. Evaluability Assessment generally involves determining specifically whether an evaluation is practical, possible, or desirable. Evaluation protocol development would be preliminary design of an evaluation and can also represent the final stages of an evaluability assessment, meant as an aid to management decision-making about whether to proceed with a full study.

Survey

The collection of information from a common group through interviews or the application of questionnaires to a representative sample of that group. The data collection techniques are designed to collect standard information from a large number of subjects. Surveys may include polls, mailed questionnaires, telephone interviews, or face-to-face interviews. Survey projects may not involve a statistically representative sample of respondents, but instead involve a group of respondents who are considered to be broadly typical of the sample universe.

Policy Analysis, Exploratory Study, Descriptive Overview

Policy Analysis is investigation or discussion intended to help managers understand the extent of a problem or need that exists and to set realistic goals and objectives in response to such problem or need. It may be used to compare actual program activities with the program's legally established purposes in order to ensure legal compliance. Exploratory Study may be policy analysis with more direct investigation or case study development. A Descriptive Overview may be, as the phrase implies, more descriptive than analytical, although, there is a blurring here too.

Program Analysis

An analysis of options in relation to goals and objectives, strategies, procedures, and resources by comparing alternatives for proposed and ongoing programs. It embraces the processes involved in program planning and program evaluation.

Performance Measurement, Performance Assessment

Performance Measurement is ongoing data collection to determine if a program is implementing activities and achieving objectives. It measures inputs, outputs, and outcomes over time. In general, pre-post comparisons are used to assess change. Performance Assessment is a term that emphasizes the analysis of data, over its mere collection.

Literature Review, Issue Brief, Research Brief

A Literature Review consists of a summary and interpretation of research findings reported in the literature. It may include unstructured qualitative reviews by single authors as well as various systematic and quantitative procedures such as meta-analysis. An issue brief may consist primarily of policy option discussions, either found in the literature, or not. A Research Brief may be another name for either of the foregoing, with varying connotations.

A Final Note to Evaluators

Good evaluation promotes good policy, good programs, and successful outcomes. To achieve this, we need unconstrained, thoughtful, and creative evaluators. Here are some notes from a recent presentation about global evaluation projects that may be relevant in the context of work of the Department of Health and Human Services and that could be helpful to legislative staff framing new or revised authorities under which the Department must act.

These are notes taken at a presentation by Rachel Glennerster, speaking on the subject, “ Using Scientific Evidence to Fight Poverty: The Role of Randomized Evaluations” at an event sponsored by the State Department&rsquo;s Foreign Affairs Evaluation Coordination Work Group, held in the Truman Building in Washington, DC, September 12, 2008.[14]

• Identify what are the truly important questions.
• Process evaluation is for accountability; impact evaluation seeks a deeper level of accountability.
• Impact evaluation can be done strategically; it doesn&rsquo;t need to be done everywhere or always.
• Most evaluations are merely review  – like project officer assessments.
• We need to focus on greater rigor of evidence because we don&rsquo;t know what is most cost-effective.
• Successful evaluation can yield surprising results (such as, that the most effective way to get children into school in Africa is through wide spread deworming).
• Some ways to design and carry out low cost, high impact, randomized studies include: randomizing around the margin target groups, first finding out what&rsquo;s already been done, learning from the rigorous impact evaluations of others, and working out the study kinks before conducting a major roll out.
• Identifying key questions to answer makes evaluation work more effective.
• Carry out representative studies, not gold plated versions of the program.
• Do things that you believe can be scaled up if they are found to work.
• Do beta testing of and learn from them.
• Be careful not to evaluate either too early or too late.
• Learn the fundamentals of what makes for good evaluation.
• Ask important questions in order to make sure that evaluation has real value.
• Look for small changes that can have big impacts.

Readers Evaluation

Performance Improvement 2009 Report User Comments

Help us make this report even better. Since this annual mandatory report to Congress is an evolving product, we welcome your comments, suggestions, and requests for ways to improve it.

1.                  Did this report meet your information needs regarding our evaluation activities? If not, what additional information would make this document more useful?

2.                  Were the contents clear? If not, how were they unclear?

3.                  Which sections did you find most helpful? How?

4.                  Which were the least useful? Why?

5.                  Have you used our online PIC database site?

6.                  Was it helpful? Please describe.

7.                  Do you wish to be contacted for a follow-up conversation about this feedback? If so, please provide name and contact information.

8.                  What is your organizational, work, or academic affiliation?

Please Email to:

PIC@hhs.gov with Subject line: “ Performance Improvement Report”

Or mail to:

Policy Information Center

Room 445F

200 Independence Avenue, SW

Washington, DC 20201

Report Preparation

This report was prepared by the Office of the Assistant Secretary for Planning and Evaluation. Andy Rock authored/edited it; he may be reached at andy.rock@hhs.gov. Readers should direct technical questions about specific studies and requests for reports not available on the web to the Federal Contact listed for the study. Policy Information Center staff provided systems and additional production support. Graphic images used in this report originate in the Department of Health and Human Services&rsquo; web site.

The unsung heroes/heroines of the annual evaluations are Project Officers throughout the Department; most identified as the Federal Contacts for each study in Chapter II. They have often conceived the study need, framed the questions, crafted the scopes of work, overseen the research, and drafted the summaries provided here. They, along with the creative thought and diligent effort of legions of contract colleagues (whose organizations are named for each entry in this report), make these evaluations possible. An important group of individuals, who contribute to creating this report, are the agency/office evaluation managers and Group Information Managers. They act as primary points of contact for the report and organize, coordinate, communicate, edit, and obtain agency report clearance.

(Evaluation matters because) if things occur that we don&rsquo;t know, it is almost as if they didn&rsquo;t happen. Anonymous

To obtain this report online and search the entire database, go to: http://aspe.hhs.gov/pic/performance

Footnotes

• [1] Funds are used for evaluation activities, and, as directed by subsequent appropriations acts, related activities, including, for example the full funding of the Agency for Healthcare Research and Quality and support for surveys carried out by the Substance Abuse and Mental Health Services Administration.
• [2] Many of the statutorily mandated demonstration projects carried out by CMS under Titles XVIII, XIX, and XXI of the Social Security Act include evaluation components that are reported here.
• [3] The Older Americans Act (OAA) specifies that $1.5 million from Title III and $1.5 million from Title IV are to be available from its annual appropriations to be used for the evaluation of OAA programs. Since 2000, the Administration on Aging (AoA) has used those funds for the Performance Outcome Measures Project and its annual national performance measurement surveys.  AoA initiated new evaluations of Title III-D Health Promotion and Disease Prevention, and Title III-B Supportive Services in FY 2004 and intends to continue evaluating all OAA titles on a rotating basis in the future.
• [4] FDA programs are principally authorized by the Food, Drug and Cosmetic Act. Appropriations are provided by the Appropriations Subcommittee on Agriculture, Rural Development, Food and Drug Administration and Related Agencies. IHS programs are principally authorized by the Indian Health Care Improvement Act and the Indian Self-Determination Act Appropriations are provided by the Appropriations Subcommittee on Interior and Related Agencies.
• [5] Set-aside funds are used to fund all of the activities of the Agency for Healthcare Research and Quality (AHRQ), much of the National Center for Health Statistics (NCHS), and much of the $28 million that the Health Resources and Services Administration (HRSA) spends on the Ryan White (AIDS) Special Projects of National Significance project., functions and activities some consider programmatic rather than evaluative. Similarly, some individuals consider the surveys supported by the Substance Abuse and Mental Health Services Administration (SAMHSA) with set-aside funds, at Congressional direction, not evaluative activities in the strictest sense. However, the information gathered through these means are essential to researchers and evaluators and fall under the category of “ basic” evaluation as described on page 17, above.
• [6] All Goals and Objectives are from the U.S. Department of Health and Human Services Strategic Plan, Fiscal Years 2007-2012; see Appendix B for complete list.
• [7] U.S. Department of Health and Human Services Strategic Plan, Fiscal Years 2007-2012, page 24.
• [8] Ibid, page 60.
• [9] Ibid, page 92.
• [10] Ibid, page 114.
• [11] Information on these HHS programs as well as all other Federal Government programs that administer federal domestic assistance is available through the Catalogue of Federal Domestic Assistance, http://www.cfda.gov/
• [12] From the OIG Website, http://www.oig.hhs.gov/organization/OEI/index.html.
• [13] Federal Register, Vol. 69, No. 127, Friday, July 2, 2004; and can be found at the OIG website, http://www.oig.hhs.gov/organization/oigorgstatement070204.pdf.
• [14] Rachel Glennerster, Executive Director of the Abdul Latif Jameel Poverty Action Lab at Massachusetts Institute of Technology earned her PhD in Economics from the University of London. Among many accomplishments, and activities in evaluating international aid programs, she coauthored “ Strong Medicine: Creating Incentives for Pharmaceutical Research on Neglected Diseases.”