Performance Improvement 2002

Foreword

Performance Improvement 2002: Evaluation Activities of the U.S. Department of Health and Human Services is the eighth annual report to Congress summarizing previous fiscal year evaluation efforts. The purpose of this report is to provide Congress with outcome-oriented evaluation findings for the Department’s programs, policies, and strategies. It contains brief summaries of evaluation results.

Evaluations summarized in this report contribute to performance improvement in one or more of the following ways:

• Program effectiveness studies provide a way to determine the impact of HHS programs on achieving intended goals and objectives.
• Performance measurement is the primary mechanism used to monitor annual progress in achieving departmental strategic and performance goals.
• Environmental assessment is the way we understand the forces of change in the health and human services environment that will influence the success of our programs and the achievement of our goals and objectives.
• Program management reflects the need of program managers to obtain information or data helpful for effectively designing and managing a program.

Performance Improvement 2002 includes HHS evaluation projects completed and in progress during fiscal year (FY) 2001. The report is organized into two chapters. Chapter I provides the “Results in Brief” of selected program effectiveness evaluations. The evaluations that were selected by an outside review panel as outstanding for their potential use by the larger health and human services community are highlighted, and are listed on the following page. Chapter II presents summaries of FY 2001 evaluations completed and in progress by the HHS agencies and the Office of the Secretary.

The appendices contain a description of the HHS evaluation program (Appendix A), acknowledgments of the HHS officials who contributed to the report (Appendix B), and the HHS Evaluation Review Panel, a group of external evaluation experts who recommended evaluations for highlighting and the criteria the panel used to review the 19 reports nominated for outstanding evaluations for FY 2001 (Appendix C).

We hope that you will find this report useful and informative.

ASPE 2001 Evaluation Review Panel Selections for Excellent Evaluations

ASPE 2001 Evaluation Review Panel
Selections for Excellent Evaluations

The following six evaluations were selected (out of 19 nominated reports) by an outside review panel as “outstanding” based on rigor of evaluation methodology, importance of findings, clarity of presentation, and other criteria (see appendix C for the full set of criteria).  The abstracts of these studies are included in Chapter 1 and are highlighted as outstanding evaluations due to their overall policy significance, methodological excellence, clarity of presentation, and other features.

• The Family Transition Program: Final Report on Florida's Initial Time-Limited Welfare Program (Administration for Children and Families)
• Head Start Children's Entry into Public School: A Report on the National Head Start/Public School Early Childhood Transition Demonstration Study (Administration for Children and Families)
• Early Head Start - Building their Futures: How Early Head Start Programs are Enhancing the Lives of Infants and Toddlers in Low-Income Families (Administration for Children and Families)
• Reforming Medicaid: The Experience of Five Pioneering States with Mandatory Managed Care and Eligibility Expansions (Centers for Medicare and Medicaid Services)
• The Final Evaluation Report on the National Home Health Prospective Payment Demonstration (Centers for Medicare and Medicaid Services)
• Nurse Staffing and Patient Outcomes in Hospitals (Health Resources and Services Administration)

Chapter I - Results in Brief: Completed Outcome-Oriented Program Evaluations

In this chapter of Performance Improvement 2002, the U.S. Department of Health and Human Services (HHS) presents the “results in brief” of FY 2001 completed evaluations that specifically address the effectiveness of a program or service. The evaluation may focus either on: (1) broad program strategies, which may encompass a number of authorized programs to address a particular need; (2) a single program that has it own congressional authorization and appropriation; or (3) a specific activity within a program that is essential to its overall effectiveness. These evaluations may provide important information for Congress in decisions about HHS programs. The evaluations are organized according to general HHS programs areas:

• Healthcare/Public Health
• Healthcare Financing
• Human Service Programs

The “results in brief” presentation for each evaluation provides the title of the final report, followed by the study purpose, major findings, and the recommendations or uses. The studies listed in Chapter I are not repeated in Chapter II.

Healthcare/Public Health

Outstanding Evaluation

Reforming Medicaid: The Experiences of Five Pioneering States with Mandatory Managed Care and Eligibility Expansions

This report summarizes findings of a six-year evaluation of Medicaid 1115 demonstrations (managed care) in five states - Hawaii, Maryland, Oklahoma, Rhode Island, and Tennessee - that were initiated in 1994-1997. Information was collected from various sources, including case studies of the demonstration states and household surveys of low-income insured people and people enrolled in demonstrations. Site visits, program documentation, and focus groups were used to assess implementation impacts on states, plans and providers. To assess impacts on beneficiaries, the researchers examined data on the Medicaid population before and after the demonstration period. Current population survey data and aggregate Medicaid cost data were also analyzed.

The evaluation found that:

• Each of the five states made major investments of time and resources to accomplish the sweeping changes they implemented in their Medicaid programs.
• By introducing managed care, states have enhanced the potential for improving the quality of care delivered to Medicaid beneficiaries.
• The states were successful in contracting with a relatively stable set of HMOs to provide services to Medicaid beneficiaries.
• Change in medical costs per beneficiary relative to the pre-demonstration period were fairly similar for the five demonstration states and the nation.
• Most Medicaid HMOs made money during the early years, but setting rates fairly required ongoing negotiations and adjustments.
• Comparison of managed care and fee-for-service Medicaid for Tennessee showed no differences in prenatal care or birth outcomes.
• Beneficiaries’ access to care and satisfaction varied widely across plans and plan characteristics.
• Rural beneficiaries in Hawaii, Tennessee, and Oklahoma fared nearly as well as did urban enrollees.
• Experiences of people with disabilities, especially those with mental illnesses, suggest room for improvement in managed care.
• Coverage expansion in Hawaii and Tennessee yielded much better experiences for the previously uninsured.

PIC ID: 6289.1; AGENCY SPONSOR: Centers for Medicare and Medicaid Services; CONTACT: Penny Pine (CMS), Jeff Buck (SAMHSA), Hunter McKay (ASPE); PERFORMER: Mathematica Policy Research Inc., Princeton, NJ

Outstanding Evaluation

Nurse Staffing and Patient Outcomes in Hospitals

In response to Congressional concern about the impact of staffing cuts on quality of care, HRSA, CMS, AHRQ, and NINR jointly funded this study, which was designed to identify outcomes of patient care sensitive to nursing and to analyze the relationship between these outcomes and nurse staffing in inpatient hospital settings.

The analyses were conducted on 799 hospitals from 11 states diversified with respect to size and geographic location. In addition, data taken from CMS discharge abstract files for a national sample of 3,357 hospitals was used to analyze these relationships for Medicare patients. A sensitivity analysis was also undertaken with a data set of 256 California hospitals that provided service-level data for nurse staffing (as opposed to data aggregated at the hospital level).

Results differed for medical and surgical patients. Staffing (the number and credentials of staff) was consistently linked to urinary tract infections, pneumonia, length of stay, upper gastrointestinal bleeding, and shock/cardiac arrest for medical patients, and to failure to rescue for surgery patients. Weaker relationships were observed for other outcomes. The investigators concluded that higher numbers of registered nurses (RN) were associated with a 3 to 12 percent reduction in rates of adverse health outcomes. Higher staffing levels of all types of nurses (RNs plus Licensed Practical Nurses (LPN) and aides) were associated with a decrease in these adverse outcomes, from 2 to 25 percent. Specifically, the study demonstrated that the strongest relationship overall between outcomes and staffing was associated with an increase in the number of RNs (i.e., LPN and aide staffing were not consistently related to the outcomes). The evaluators also pointed out the “catch 22”: higher staffing levels may lead to better charting of some adverse outcomes, thereby confounding the relationship between staffing levels and negative outcomes.

PIC ID: 6864; AGENCY SPONSOR: Health Resources and Services Administration; CONTACT: Carol Gassert; PERFORMER: Harvard School of Public Health, Boston MA

Do Food Labels Work? Gauging the Effectiveness of Food Labels Pre- and Post-NLEA

This chapter in the Handbook of Marketing and Society is a synthesis of findings on the use and apparent consequences of food labeling, with special attention to the possible changes associated with implementation of the National Labeling and Education Act (NLEA). NLEA had many features, including mandatory nutrition labeling for most packaged foods, uniform definitions for terms such as “light”, and limits on allowable health claims.

The chapter summarizes and integrates findings from investigations that used a variety of methods, including sample surveys (before and after NLEA implementation), experiments, focus groups, and analysis of food consumption trends. Populations used were generally not explicitly reported (but seemed apparent for sample surveys).

Findings were summarized for consumer effects (including awareness of labels, label use, changing food decisions based on label information, effects of label claims, and dietary impacts) and marketplace impacts (including prevalence of claims on labels, introduction of healthier brands, and market share). Key findings included increasing awareness of the new label over time and significant levels of reported label use to make product choices. The food label caused some consumers to stop buying products or to try new products. It also provided the types of information that consumers are most concerned about (e.g., dietary fats) and presented the information more clearly than previously, according to the authors.

PIC ID: 7735; AGENCY SPONSOR: Food and Drug Administration; CONTACT: Alan Levy; PERFORMER: Food and Drug Administration, Washington, DC

The Efficacy of Interventions to Modify Dietary Behavior Related to Cancer Risk, Vols. I and II

This review seeks to clarify the existing knowledge base on behavioral interventions to alter dietary behavior related to cancer risk and offers directions for future research. Interventions consisted of behavioral dietary interventions on fruit and vegetable intake and behavioral dietary interventions on decreases in fat intake.

In the studies that were reviewed, dietary interventions were consistently associated with an increase in fruit and vegetable consumption (with greater increase noted for fruit intake). More than three-quarters of the studies reviewed reported statistically significant increases in fruit and vegetable intake. Several dietary intervention components appear to be particularly promising in modifying dietary change favorable to cancer risk reduction: social support, goal setting, small groups, food-related activities, and the incorporation of family components.

Using the differences-in-deltas approach, the reviewers determined that the average increase in fruit and vegetable intake reported was 0.6 servings per day, and consistent decreases were noted in the intake of total fat and saturated fat. The mean change in total fat intake was estimated as 7.3 percent reduction in the percentage of calories from fat. Interventions appeared to be more successful at positively changing dietary behavior in populations at risk of (or diagnosed with) disease than in healthy populations.

PIC ID: 7691; AGENCY SPONSOR: Agency for Healthcare Research and Quality; CONTACT: Kathryn Rickard; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

5 A Day for Better Health Program Evaluation Project

The purpose of this project was to review and evaluate (1) the science underlying the 5 A Day program, (2) the implementation and accomplishments, and (3) the degree of achieved goals and objectives. Twelve scientific experts, as members of the Program Evaluation Committee, reviewed the scientific evidence and program infrastructure (materials, collaborations, partnerships, message delivery, implementation and process measures) and supported research grants.

During the first 10 years of the 5 A Day Program, public awareness on the importance of vegetable and fruit consumption and knowledge of the 5 A Day Program and its message has increased. The message has reached more women than men, and more whites than Latinos or African Americans. Evidence indicates that a diet rich in vegetables and fruits reduces the risk of cancer and other chronic diseases. For example, the evidence of a relationship between higher vegetable and fruit consumption and lower risk of several types of cancers has been strengthened.

It is recommended that the 5 A Day Program should continue as a multifaceted program to support research and applied public health efforts to promote increased vegetable and fruit consumption.
http://cancercontrol.cancer.gov/5ad_exec.html
PIC ID: 7635; AGENCY SPONSOR: National Institutes of Health; CONTACT: Kevin Callahan; PERFORMER: SCG: The Scientific Consulting Group, Inc., Gaithersburg, MD 20878-1409

The National Cross-Site Evaluation of High-Risk Youth Programs: Final Report

The purpose of this study was to determine the effectiveness of various programs designed to prevent substance abuse in high-risk youth.

This rigorous research involved more than 10,500 at-risk 9- to 18-year-olds in communities across the United States. Of 48 sites in this Cross-Site Evaluation, three were programs associated with universities or medical centers, three were higher education institutions, three were public school districts, and there were three state, county, or tribal agencies. The majority of grantees were small community-based programs not affiliated with larger institutions. The programs ranged in delivery method from primarily classroom-style, didactic programming in which youth receive substance abuse prevention-related information through lectures, videos, or other similar means, to interactive and experiential programming where youth are required to actively participate, reflect on the subject at hand, and act upon it. There were four areas of emphasis for programming: (1) programs that focused on developing life-skills (e.g., refusal skills, communication skills, anger management, and other social skills); (2) programs that primarily provided knowledge of substance abuse, and other attendant problems (e.g., teen pregnancy, AIDS, gangs, and violence); (3) programs focused on developing self-esteem, or “affective-dominant” programs; and (4) recreation-dominant programming.

Typically, the interactive type of programs included role play activities, team-building projects, outdoor wilderness experiences, or other activities requiring active engagement. These type programs were found to be more effective in reducing substance abuse than programs that were less interactive.

The research shows that “prevention works”--programs designed to prevent substance abuse produce lasting reductions in substance use and have important positive effects on the lives of youth at risk. The results represent accomplishments by programs operating in communities where youth face multiple risk factors. Even within the challenging circumstances of these programs, the entire pool of participants across all programs decreased their use of alcohol and marijuana relative to similar youth who did not participate in the programs. The study findings have important implications for emphases in prevention design and implementation. By contributing to understanding the factors that prevent substance abuse in youth, and producing science-based lessons about effective prevention strategies and program practices, this major research effort holds the promise of more effective prevention and more positive futures for America’s youth who are at high risk.
PIC ID: 6044; AGENCY SPONSOR: Substance Abuse and Mental Health Services Administration; CONTACT: Soledad Sambrano; PERFORMER: EMT Associates, St. Louis, MO

Effectiveness of Women’s Substance Abuse Treatment Programs: A Meta-Analysis

Recent data suggest that at least 4.5 million women in the United States are currently alcohol abusers, 3.1 million are regular users of illicit drugs, and 3.5 million misuse prescription drugs. The current demand for effective substance abuse treatment programs for women is, therefore, high. To date, however, no formal quantitative synthesis or meta-analysis of the women’s treatment literature has been conducted. In addition, compared to other social services fields, only a handful of meta-analyses have been conducted in the substance abuse treatment field.

This technical report represents an attempt to fill this key knowledge gap by describing a quantitative, meta-analytic synthesis of the literature on the effectiveness of substance abuse treatment for women. This study focused on the effectiveness of women-only treatment programs; women-only treatment programs compared to mixed-gender treatment programs; and, enhancements to women-only treatment programs. The meta-analysis consisted of defining criteria for study inclusion, a literature search, study screening and selection process, the identification of relevant and representative outcome domains, the coding of studies, and quantitative analysis of study outcomes.

The findings are: 1) different types of treatment were positive on average in all domains; 2) women-only programs provide additional treatment benefits beyond those resulting from standard, mixed-gender programs, i.e. women-only programs are best suited to the treatment needs of women; 3) enriching women’s treatment with additional components specifically oriented toward meeting women’s needs adds value above and beyond the expected effects of standard, women-only programs.
PIC ID: 5994.8; AGENCY SPONSOR: Substance Abuse and Mental Health Services Administration; CONTACT: Ron Smith; PERFORMER: Battelle Corporation, Arlington, VA

The Effectiveness of Substance Abuse Treatment in Reducing Violent Behavior

This analysis, using data from the National Treatment Improvement Evaluation Study (NTIES), addresses the important policy issue of the association between substance abuse and violent behavior and the manner in which effective substance abuse treatment can reduce the likelihood of interpersonal violence.

Reductions in violence were analyzed by gender, treatment modality--methadone treatment, non-methadone outpatient treatment, long-term residential treatment, short-term residential treatment, and facilities located in correctional institutions--and whether the client was a victim of violence in addition to being a perpetrator.

The findings indicate that treatment for substance abuse can be highly effective in reducing specific violent acts among persons with a history of committing violence (i.e., perpetrators). It was also found that these improvements are consistent across client subgroups such as gender, modalities of treatment, and past status as a victim of violence.
PIC ID: 5994.10; AGENCY SPONSOR: Substance Abuse and Mental Health Services Administration; CONTACT: Ron Smith; PERFORMER: Caliber Associates, Fairfax, VA

Assessing the Effectiveness of Hazard Analysis Critical Control Point (HAACP) Systems in the Seafood Industry

The Seafood Hazard Analysis Critical Control Point (HACCP) Program mandates the application of HACCP principles to the processing of seafood. HACCP is a preventive system of hazard control that can be used by processors to evaluate the kinds of hazards that could affect products, institute appropriate controls to keep the hazards from occurring or minimize their occurrence, and monitor performance of those controls. This program supplants the old FDA seafood inspection system.

This report provides an overview of the findings from a review of the relevant food safety literature and the advice from HACCP and evaluation experts regarding the feasibility of and alternative methods for evaluating the impacts of the seafood HACCP program.

The major finding is that although an analysis of the impact of HACCP on the number and severity of seafood-borne illnesses in the United States should be the area of highest priority for an evaluation, data limitations make it virtually impossible to assess. One reason is that attributing foodborne illness to a particular food is often not possible. Second, attributing reductions in foodborne illness to a specific regulation may not be possible, since it would be difficult to isolate the impact of the regulations from changes in other factors that may also affect the incidence of foodborne illness.

Recommendations are that an evaluation: (1) assess the impact of HACCP on the incidence and levels of physical, chemical, and biological hazards; (2) identify the set of factors affecting seafood safety; and (3) assess the impacts of those factors on seafood hazard levels, at least qualitatively.
PIC ID: 7343; AGENCY SPONSOR: Office of the Assistant Secretary for Planning and Evaluation; CONTACT: Laina Bush; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

Healthcare Financing

Evaluation of the Program of All-Inclusive Care for the Elderly Demonstration: Comparison of the Pace Capitation Rates to Projected Costs in the First Year of Enrollment

The Program of All-Inclusive Care for the Elderly (PACE) is designed to provide a complete range of integrated preventative, acute, and long term care services for the frail elderly by combining Medicaid and Medicare capitation in contracts with community providers of such services. This evaluation sought to estimate the economic costs and benefits of enrollment during the first 12 months of participation in PACE.

Eleven PACE programs operating under dual capitation were involved in the evaluation, which was based on a comparison group design. The treatment group included individuals who expressed interest in PACE, had a home visit, decided to enroll in PACE, and were accepted into the PACE program prior to the collection of site visit data. The comparison group consisted of individuals who had the same screening process and submitted initial applications, but decided not to enroll. During the study enrollment period, 3,009 individuals met the study criteria. Multivariate statistical procedures were used to control for selection bias and to adjust the impact measures. Regression models were used to analyze the relationships between background characteristics and subsequent Medicare and Medicaid reimbursement for comparison group members. Regression coefficients were then combined with information on the baseline characteristics of enrollees to estimate their projected Medicare and Medicaid costs.

Findings indicated that projected costs at the 11 PACE sites were about 10 percent lower than the combined Medicaid and Medicare capitation payments. When two high-cost sites were excluded from the analysis, estimated Medicare and Medicaid costs approximated the reimbursement rates. There was a differential impact on Medicare and Medicaid, with PACE representing savings for the Medicare program, but additional costs for Medicaid.
PIC ID: 6309; AGENCY SPONSOR: Centers for Medicare and Medicaid Services; CONTACT: Fred Thomas, 410-786-6675; PERFORMER: Abt Associates, Inc.

Outstanding Evaluation

The Final Evaluation Report on the National Home Health Prospective Payment Demonstration: Agencies Reduce Visits While Preserving Quality

This evaluation presents findings from Phase II of the National Home Health Prospective Payment Demonstration, which ran from 1995 to 1998 and was designed to test the effects of a predetermined per-episode payment rate. In that demonstration, 91 Medicare certified home health agencies from five states were randomly assigned to the control (i.e., cost reimbursement) and treatment (i.e., per-episode payment) groups.

The five studies reported here examined impacts on service delivery; patients’ use of Medicare services other than home health; costs per episode, per visit, and profit potential; and quality of care. Multiple data sources were used, including Medicare cost reports, Medicare claims data (including UB-92 bill records that include patient characteristics), CMS’s Enrollment Data Base, Area Resource File (for agency information), a quality assurance database constructed for the demonstration, a patient survey, and site visits.

The main finding was that agencies in the treatment group (per-episode payment) dramatically reduced their number of home health visits, primarily due to earlier discharge, without substantial adverse effects on quality of care. However, the evaluators noted that control agencies also reduced their visits, in response to various changes in the environment of home health. Other findings indicated that the overwhelming majority of patients in both groups were satisfied with their home health service. The evaluation also identified methods that agency staff found successful in reducing home health utilization, including more careful supervision of visiting staff and improved patient education. While treatment group agencies reduced costs per episode, their per-visit costs rose more for skilled nursing and aide visits than did the same costs for control group agencies. In other words, although the volume of services fell, the treatment agencies’ fixed costs could not adjust as rapidly. As well, some of the strategies used to reduce visits actually resulted in higher overhead costs.
PIC ID: 7738.4; AGENCY SPONSOR: Centers for Medicare and Medicaid Services; CONTACT: Ann Meadow; PERFORMER: Mathematica Policy Research, Inc., Princeton, NJ

Prospective Pymt (a) Prospective Payment Demonstration for Medicare Home Health: No Clear Signs that Quality of Care Suffered

This report represents a summary of the quality of home health care throughout the three years demonstration project noted in earlier reports (7738, 7738.1, 7738.2). It provides an update to the previous report which covered the first two years of the demonstration. The demonstration tests the extent to which a fixed, lump-sum prospective payment to home health agencies for the first 120 days of each episode of care provided to Medicare beneficiaries increases efficiency in service provision. By allowing agencies to retain most of any surplus payments over cost, this payment method gives agencies an incentive to provide home health care in a cost-efficient manner.

The main findings are that, despite prospectively paid agencies’ progressive reduction of services over time, there were no serious adverse effects on major outcomes of care. There were a few small negative effects in Year 3 on stabilization in functioning, as well as several large positive effects in all three years on improvement in symptoms. These could be viewed either as subtle signs of negative effects late in the demonstration or, just as easily, as the absence of any true impacts.
PIC ID: 7738.3; AGENCY SPONSOR: Centers for Medicare and Medicaid Services; CONTACT: Ann Meadow; PERFORMER: Mathematica Policy Research, Inc., Princeton, NJ

Prospective Pymt (b) Per-Episode Prospective Payment for Medicare Home Health Care Sharply Reduces Service Use

As part of its ongoing effort to study methods of providing more cost-effective care, the Centers for Medicare and Medicaid Services (CMS) implemented the Per-Episode Home Health Prospective Payment Demonstration. Under the demonstration, participating home health agencies are paid a fixed, lump-sum payment for the first 120 days of each episode of care provided to Medicare beneficiaries and a predetermined rate for each visit thereafter. By allowing agencies to retain most of any surplus payments over cost, this payment method gives agencies an incentive to provide home health care in a cost-efficient manner.

It was found that prospective payment reduced the average number of visits to a patient in the year following admission by 24 percent compared to their levels under cost-based reimbursement. Prospectively paid agencies achieved these reductions by shortening the overall length of service and by lowering the frequency of visits provided. These results strongly suggest that prospective payment is a highly successful method for controlling the costs of Medicare home health care; further analyses must evaluate the consequences of these declines on patient health and access, non-home health expenditures, and other outcomes before final recommendations can be made.
PIC ID: 7738; AGENCY SPONSOR: Centers for Medicare and Medicaid Services; CONTACT: Ann Meadow; PERFORMER: Mathematica Policy Research, Inc., Princeton, NJ

Prospective Pymt. (c) Prospective Payment for Medicare Home Health: A Promising System to Save Resources

(See Prospective Pymt (b) above for background.)

It was found that the typical agency in the demonstration was able to earn small profits under the prospective payment system. They were able to do so by reducing their cost per episode by 14 percent--a decrease achieved by reducing service use. However, efforts also to reduce service use resulted in increases in the agencies cost per visit. The outcome effectively reduced profits, as approximately half an average agency’s visits were paid according to the per-visit methodology, at a predetermined rate. Furthermore, the agencies’ Medicare revenues fell by 20 percent, largely because the number of visits rendered after 120 days of care dropped as well. Although the small number of some types of agencies in the sample makes it difficult to draw firm conclusions, it appears that almost all types of agencies experiences these financial changes. These results suggest that CMS could save resources relative to cost-reimbursement by implementing a prospective payment system.

PIC ID: 7738.1; AGENCY SPONSOR: Centers for Medicare and Medicaid Services; CONTACT: Ann Meadow; PERFORMER: Mathematica Policy Research, Inc., Princeton, NJ

Study of Pharmaceutical Benefit Management

The purpose of this study was to assist the Centers for Medicare and Medicaid Services in selecting a pharmacy benefit manager (PBM) as it implements a Medicare drug benefit. As the possibility of extending a Medicare benefit moves through legislation, Congress and the CMS will need to decide how to obtain discounted pricing, the benefits and services it will offer, and whether it will use the PBM’s formulary and clinical services or design its own.

PBMs have emerged as the national standard for the administration of prescription drug insurance in the United States. They manage drug benefits of approximately 70% of Americans, including 65% of the country’s seniors. This is also a very successful industry in that PBMs have the ability to reduce costs, provide national pharmacy access, and administer benefits that are customized to meet the needs of a wide range of clients in a highly automated environment.

The PBM industry, suggests the report, would not be overwhelmed by providing services to the Medicare population because the industry is relatively insensitive to volume increases. However, better rebates and administrative fees could result from the creation of a Medicare formulary, which could help control Medicare drug costs. A PBM could be contracted with for administrative services, such as claims processing, Medicare formulary, management, member services, and clinical control. This scenario would maximize price discounts and rebates.
PIC ID: 7591; AGENCY SPONSOR: Centers for Medicare and Medicaid Services; CONTACT: Peri Iz; PERFORMER: Price Waterhouse, Washington, DC

Discontinuous Coverage in Medicaid and the Implications of 12-Month Continuous Coverage for Children

This report is a policy analysis that examines the implication of 12-month Continuous Coverage for children under the Medicaid program by simulating enrollment based on 1994-95 State Medicaid Resource File (SMRF) data from California, Michigan, Missouri, and New Jersey.

The researchers modeled what the enrollment, payments, and ER use would have been had the states implemented this policy. Four types of yearly claims files were included: inpatient hospitalization, drugs, long-term care, and other services. Claims data from California were also used to examine impacts on quality of care as reflected in use and payments for ER care. Analyses included only children who could be eligible for Continuous Coverage; thus, medically needy, SSI, special refugee status, and other groups were excluded.

Findings indicated that a policy of 12-month Continuous Coverage would reduce, but not eliminate, the problem of discontinuous coverage; the policy does extend coverage to more than half of children who would otherwise lose it. In addition, results showed that Continuous Coverage would increase total Medicaid payments, but that over time, this policy could decrease overall health costs as acute episodes are presented. Continuous Coverage would also reduce estimated administrative costs related to enrollment and disenrollment as children move in and out of plans, but only by a small percentage. Findings were inconclusive with regard to analyses of costs. The researchers conclude that Continuous Coverage holds promise as a policy option that could improve continuity of care and costs for children under Medicaid. They state that future analyses should be conducted to examine the implications of these findings.
PIC ID: 7774; AGENCY SPONSOR: Health Resources and Services Administration; CONTACT: Jacob Tenenbaum; PERFORMER: Mathematica Policy Research, Inc., Princeton, NJ

The Medicaid DSH Program and Providing Health Care Services to the Uninsured: A Look at Five Programs

Perhaps one of the most important health policy issues facing the United States is how to care for the uninsured. Uninsured individuals often lack access to appropriate care, but they still use health services when they become ill and, in many cases, they do not have the financial resources to fully pay for their care. Through various public programs--local, state, and federal--many hospitals receive subsidies to help pay for the costs associated with uncompensated care.

One of the largest subsidy programs is the Medicaid disproportionate share hospital (DSH) program. This study provides some insights on the experience of five programs that have used Medicaid DSH funds to enhance care for the uninsured. Those programs were: Denver County (Denver), Colorado; Marion County (Indianapolis), Indiana; Ingham County (Lansing), Michigan; Wayne County (Detroit), Michigan; and Bexar County (San Antonio), Texas. These programs reflect diversity in target population, organization, financing, delivery system and services provided.

The findings indicate the Medicaid DSH funds were an important source of funding for all programs. The programs examined here highlight how DSH funds have been used in a positive way: To provide health care services to the uninsured. By emphasizing primary care services, all of these programs, as diverse as they are, aim to reduce hospitals’ uncompensated care burdens, one of the principal goals of the DSH program. The programs in this study provide models by which states and localities could provide primary care services using Medicaid DSH funding.
PIC ID: 6703; AGENCY SPONSOR: Office of the Assistant Secretary for Planning and Evaluation; CONTACT: George Greenberg; PERFORMER: Urban Institute, Washington, DC

Human Service Programs

Outstanding Evaluation

The Family Transition Program: Final Report on Florida’s Initial Time-limited Welfare Program

Florida’s Family Transition Program (FTP), initiated in 1994 in Escambia County, was designed to limit the number of months that welfare could be collected. This fifth and final report examines the implementation, implication, and cost benefits of FTP among 2,817 recipients randomly assigned to receive FTP or standard AFDC between 1994 and 1999. Excluded from the sample were families with disabled or chronically ill members and/or with children under seven months of age. At the time of enrollment, the FTP program included a time limit on cash assistance and an array of support services, whereas traditional AFDC had neither (although this was later changed).

The evaluation utilized information from a variety of sources, including baseline data collected prior to random assignment; administrative records (e.g., AFDC/TANF payments, food stamp benefits); and various stakeholder surveys, such as four-year client surveys in 1998 and 1999 that included questions on employment, household income, and child well-being.

Findings indicated that most families on AFDC left welfare during the study period; however, FTP substantially reduced long-term welfare receipt, with only 6 percent receiving welfare for more than 36 months versus 17 percent of those in the AFDC group. In addition, FTP families gained more in earnings than they lost in welfare, resulting in modestly higher incomes; only 17 percent of FTP families reached their time limits during the study period. While results indicated that FTP had few impacts, positive or negative, on the well being of elementary school children, adolescents in the FTP group performed somewhat worse on some of measures of school performance (e.g., expulsions). A cost-benefit analysis showed that program expenses were about $8,000 more per client over the four-year period when compared to typical welfare costs.
PIC ID: 6820; AGENCY SPONSOR: Administration for Children and Families; CONTACT: Alan Yaffe; PERFORMER: Manpower Demonstration Research Corporation

Head Start FACES (Family and Child Experiences Survey): Longitudinal Findings on Program Performance (Third Progress Report)

Head Start FACES is an ongoing national longitudinal study of the cognitive, social, emotional, and physical development of Head Start children; the characteristics, well-being, and accomplishments of their families; the observed quality of Head Start classrooms; and the characteristics and opinions of Head Start teachers and other program staff.

Using a nationally stratified random sample of 3,200 children and families in 40 Head Start programs, the evaluation addressed four study questions: 1) Does Head Start enhance children’s development and school readiness; 2) Does Head Start strengthen families as the primary nurturers of their children; 3) Does Head Start provide children with high quality educational, health, and nutritional services; and, 4) How is classroom quality related to child outcomes? Data components (child assessment, parent interview, teacher and staff interviews, and classroom observations) were collected at five different points in time, and 80 percent response rates were achieved. Multiple measurement instruments and scales were used. Quantitative information was enhanced with case studies of 120 families, also selected randomly (3 per site).

Based on Fall 1997 and Spring 1998 data, positive findings indicated that Head Start participation narrowed the gap between disadvantaged children and all children in vocabulary and writing skills, and led to improvements in areas such as social skills, word knowledge, and math skills. However, there were no improvements on literacy measures (e.g., letter-word identification), and behavior problems were largely unchanged. With regard to families, the parents of Head Start children reported extremely high levels of satisfaction (over 85 percent) with the program, and a greater sense of control over their lives after one year. However, participation by fathers in their children’s lives remained low, and family social support and mental health needs did not change.
PIC ID: 6331.2; AGENCY SPONSOR: Administration for Children and Families; CONTACT: Louisa B. Tarullo; PERFORMER: Westat, Ellsworth Associates, Abt Associates, Inc., and the CDM Group

Outstanding Evaluation

Head Start Children’s Entry into Public School: A Report on the National Head Start/Public School Early Childhood Transition Demonstration Study

This longitudinal study examined the scholastic functioning of 7,515 former Head Start children from 31 diverse sites across America, with approximately half the children receiving extended comprehensive services for grades K through 3. Modeled after Project Head Start, the National Transition Demonstration Project was passed by Congress in 1991 to provide extended developmentally appropriate and comprehensive school-based services to Head Start graduates.

The 12-chapter report covers a range of topics, beginning with a review of social policy behind the Head Start Transitional Program and the research questions to be addressed in the study, and ending with analyses of the main predictors of children’s school functioning. In this regard, growth curve analyses and hierarchical linear modeling (HLM) were utilized to identify influences that promote and detract from children’s achievement.

Overall, the most impressive finding was the rapid growth demonstrated in math, reading, and social development. Head Start children from both groups made rapid gains between kindergarten and third grade, often starting out a standard deviation below the mean in kindergarten and reaching the national mean (or above) by grade three. Results also highlighted the contribution that services and other aspects of the transition program make toward scholastic achievement and social development. Family gains were noted as well, including economic growth in transition families.
PIC ID: 4393; AGENCY SPONSOR: Administration for Children and Families; CONTACT: Mary Bruce Webb; PERFORMER: Civitan International Research Center, the University of Alabama at Birmingham, Birmingham, AL

Outstanding Evaluation

Building Their Futures: How Early Head Start Programs Are Enhancing the Lives of Infants and Toddlers in Low-income Families

Early Head Start provides services to low-income pregnant women and families with infants and toddlers in order to improve children’s development (cognitive, language, social-emotional, and health); to foster close supportive relationships between parents and their infants and toddlers; and to encourage the development of community partnerships.

This report summarizes the interim results of an evaluation initiated in 1995 that includes about 3,000 children and families across 17 sites selected to represent a variety of milieu (regional, rural, and urban), ethnic and racial compositions, and types of program approaches. At each site, children and families were randomly assigned to program and control groups; follow-up was carried out over three years of program participation through a child’s third birthday. Results described in this report reflect program impacts through the second birthday of study participants, with data collected from researcher observations, parent reports, and direct child assessment.

Findings indicate that at two years of age, Early Head Start children demonstrated higher positive impacts on infant and toddler cognitive development, vocabulary and sentence complexity, and aggressive behavior than did the control group (children not in Early Head Start). However, no differences were found between the two groups in terms of variables such as regulation of emotions, task-oriented behavior, or attention span (videotaped freeplay interaction with mothers). With regard to the parents of two year-olds, results showed that parents of Early Head Start group children learned more about early childhood development and were more likely to provide experiences and environments that supported development than did control group parents.
PIC ID: 3570; AGENCY SPONSOR: Administration for Children and Families; CONTACT: Rachel Chazan Cohen; PERFORMER: Mathematica Policy Research, Inc., Princeton, NJ

Chapter II - Compendium of HHS Agency FY 2001 Evaluations Completed and in Progress

The various agencies and offices of the Department of Health and Human Services (HHS) maintain evaluation programs which include evaluation planning and policy review, quality assurance through technical review, project coordination and management, dissemination of reports, and utilization of results. This chapter is a compendium of the evaluations completed and in progress during fiscal year (FY) 2001, organized by the following HHS components:

• Administration for Children and Families (ACF)
• Administration on Aging (AoA)
• Agency for Healthcare Research and Quality (AHRQ)
• Agency for Toxic Substances and Disease Registry (ATSDR)
• Centers for Disease Control and Prevention (CDC)
• Centers for Medicare and Medicaid Services (CMS)
• Food and Drug Administration (FDA)
• Health Resources and Services Administration (HRSA)
• Indian Health Service (IHS)
• National Institutes of Health (NIH)
• Office of the Assistant Secretary for Planning and Evaluation (ASPE)
• Office of Public Health and Science (OPHS)
• Substance Abuse and Mental Health Services Administration (SAMHSA)

Each HHS agency and office section begins with an overview of the relevant evaluation program, describing its philosophy, policies, and procedures. Next are descriptions of the major evaluations completed in FY 2001, grouped by major program areas (evaluations listed in Chapter I are not repeated in Chapter II). Lastly, the evaluations currently in progress are presented.

http://aspe.hhs.gov/pic/

Administration for Children and Families

Mission

To promote the economic and social well-being of families, children, individuals, and communities.

Evaluation Program

The Administration for Children and Families (ACF) administers a broad range of entitlement and discretionary programs, including income maintenance (Temporary Assistance for Needy Families); child support; children and family services (Head Start, Child Welfare, Family Preservation and Support, and youth programs); four block grants; and special programs for targeted populations, such as the developmentally disabled, immigrants, and Native Americans.

The objectives of ACF’s evaluations are to furnish information on designing and operating effective programs; to test new service delivery approaches capitalizing on the success of completed demonstrations; to apply evaluation data to policy development, legislative planning, budget decisions, program management, and strategic planning and performance measures development; and to disseminate findings of completed studies and promote application of results by state and local governments.

ACF actively engages with other federal agencies, state and local policy and program officials, national organizations, foundations, professional groups and practitioners, and consumers to stay current on emerging issues affecting its programs and to identify questions for evaluation studies. Systems changes and how they affect vulnerable populations, particularly children, are of primary concern. The movement toward devolving responsibility for health and human services to state and local organizations offer both tremendous opportunities and unprecedented challenges in redefining and implementing services for families.

Evaluation study designs are negotiated carefully with the states and other interest groups. Studies often are funded as joint ventures with the Office of the Assistant Secretary for Planning and Evaluation and other federal agencies and foundations. Such collaborations permit large-scale efforts that are better informed and more representative of varying perspectives. Proposals are reviewed by multidisciplinary experts. Work groups of various kinds are used to monitor the progress of projects and to advise on design refinements and the presentation of findings.

Completed Evaluations

National Evaluation of Family Support Programs Final Report: Vol. A: The Meta-Analysis

This work evaluating the effectiveness of family preservation and family support programs is the first of three volumes of the Final Report. They report on distinctly different aspects of the overall study of family support programs. This volume reports findings from a meta-analysis of existing research on programs that provide family support services. The results presented in this analysis asked the following questions: What are the effects of family support programs and services? What characteristics of programs or the families they serve account for differences in effects? A summary of the findings included in the meta-analysis, which represent a broad cross-section of programs that provide family support services, have small but statistically significant average effects in all nine outcome domains, e.g., children’s cognitive development, social and emotional development, parenting attitudes and knowledge, parenting behavior, and family functioning. The report notes, however, that in every outcome domain, a small group of programs accounted for the average effect. In child cognitive achievement, child social and emotional functioning, and parenting behavior the effects of family support programs are consistently meaningful, though small. The “meaningfulness” of effect size is not always clear, however, and must be considered in the context of evidence about their potential importance for a given population and the cost of obtaining the effect.
PIC ID: 5848; CONTACT: Mary Bruce-Webb, 202-205-8628; PERFORMER: Abt Associates Inc., Cambridge, MA

Delivering Employment Retention and Advancement Services

The state of Iowa made an early commitment to job retention and advancement services based on evaluation results of its welfare program, the Family Investment Program (FIP). These results indicated that FIP had increased employment and earnings among recipients but had not directly resulted in families leaving welfare. The state developed a model for post-employment services. The Post-Employment Pilot (PEP) Program tests that model in three sites across the state and lays the groundwork for future retention and advancement services in Iowa. This is the final report of the process study of the PEP program. It examines the decisions made by the three pilot sites about recruitment and referral processes, services, and staffing structures as well as the challenges and successes experienced by the sites during the first 10 months of program operation. A summary of the findings is: (1) The main challenge faced by all three pilot sites is low participation, mainly due to lack of time. (2) Clients participating in PEP have frequent contact with PEP staff, but most of those contacts do not occur in person, and the average length of time that clients spend in the program is relatively short (averaging 3.5 months). (3) While the core components of the PEP program are group activities, participants indicated they were more satisfied with one-on-one interactions with their caseworker. (4) The program failed to provide clients with services/resources addressing their most severe personal difficulties and challenges, such as transportation and child care. (See also PIC ID 6762)
PIC ID: 6761; CONTACT: Girley Wright, 202-401-5070; PERFORMER: Iowa Department of Human Services, Des Moines, IA

Enhancement to the Process and Impact Analysis of the Youth Employment and Training Initiative (YETI)

The purpose of this final report was to assess the impact of the Youth Employment Training Initiative (YETI) on key outcome measures and to suggest policy and programmatic implications of these findings. In addition to summarizing earlier analyses, new analyses reported here uses data drawn from several sources: (1) school-based records, (2) Illinois Department of Employment Security (IDES) wage files, and (3) Illinois Department of Human Services (IDHS) records of cash assistance from the TANF program. Hypotheses related to the impact of YETI on academic achievement, earnings, and dependence on public assistance were tested. This report shows, as did the previous reports, that the YETI program has had a modestly positive impact. The report concludes by offering several program and research implications generally derived from interviews with served and control group members. They highlight the value of the sustained presence of caring and knowledgeable adults for optimal adolescent transitions to adulthood and the benefits of the program’s sustained attention, guidance, and social skill-building components.
PIC ID: 6821; CONTACT: John Maniha, 202-401-5372; PERFORMER: Illinois Department of Human Services, Springfield IL

Process/Implementation Evaluation: Capitalizing the Bridge from Welfare to Work to Independence Demonstration Program

This project was designed to expand the employment and training services provided by Goodwill Industries by constructing new centers in four counties in southwest Florida and six parishes in southeast Louisiana. During the three-year evaluation period both regions met their placement quotas and are currently ahead of their goals. The report found that providing funds for construction up-front results in significant job placements after those facilities open for business, and for the foreseeable future. However, both Goodwill Acadiana and Goodwill Manasota are established organizations with proven track records and clearly defined goals. Whether or not this model will work in other settings as an initial start up operation is unclear.
PIC ID: 6755; CONTACT: Girley Wright, 202-401-5070; PERFORMER: Goodwill Industries of Manasota, Inc., Sarasota, FL and Goodwill, Industries of Acadiana, Inc., Scott, LA

Promoting Employment Retention Among TANF Recipients: Lessons from the GAPS Initiative

The Pittsburgh Foundation, in collaboration with the Allegheny County Assistance Office (ACAO) of the Pennsylvania Department of Public Welfare (DPW), developed the GAPS initiative, an employment retention program that consisted of case management and other support services for employed Allegheny County welfare recipients. The program was called “GAPS” because it aimed to help welfare recipients bridge the gap between dependence on welfare and self-sufficiency. This report is the second and final report on the GAPS initiative. Through site visits, service use, survey, and administrative records data, the following key findings about participants’ experiences and program operations emerged: (1) Most GAPS participants experienced steady economic progress during their first 18 months in the program.

(2) Substantial challenges remain in spite of the economic progress of participants, for example, affording health insurance and child care. (3) GAPS participants valued the supportive counseling, personal attention, and advice their case managers provided. (4) Supplementing case management with additional tangible services may help gain participants’ trust and, ultimately, improve their economic outcomes. (5) A greater emphasis on job advancement for newly employed welfare recipients may be needed. Many participants indicated on follow-up surveys that they were looking for another job, usually with higher wages or better benefits.
http://www.mathematica-mpr.com/pdfs/gapsreports.pdf
PIC ID: 6760.1; CONTACT: Nancye Campbell, 202-401-5760; PERFORMER: Mathematica Policy Research, Inc., Plainsboro, NJ

Second Assignments to Iowa’s Limited Benefit Plan

In 1993 Iowa implemented a comprehensive set of welfare reforms in the form of the Family Investment Program (FIP). A key component of FIP is the requirement that able-bodied adults participate in PROMISE JOBS, the state’s employment and training program for welfare recipients. If those clients do not comply with PROMISE JOBS requirement, their FIP cases are assigned to the Limited Benefit Plan (LBP), a short-term plan that restricts eligibility for FIP cash assistance. This study provides a comprehensive assessment of second LBP assignments--including how often they occur, who they affect, why they occur, and their policy implications. The study shows that about one-quarter of all FIP clients assigned to a first LBP ultimately enter a second LBP. LBP policy changes may reduce LBP recidivism through stricter rules that will require clients to comply with the employment and training activities before reapproval of a FIP application. Most clients that failed to keep a schedule appointment with PROMISE JOBS often cited personal and family circumstances, such as transportation problems, work and school schedule conflicts and child care problems. Finally, once in a second LBP clients were more likely to be employed after entering a second LBP than before, and average total household income was higher after entering a second LBP than before.
PIC ID: 6762; CONTACT: Girley Wright, 202-401-5070; PERFORMER: Iowa Department of Human Services, Des Moines, IA

Work-Based Strategies for Hard-To-Employ TANF Recipients: A Preliminary Assessment of Program Models and Dimensions

This study was designed to achieve two goals: (1) identify and provide detailed information about the design and structure of work-based programs that serve or that have the potential to serve hard-to-employ TANF recipients and (2) assess the feasibility of conducting a rigorous, large-scale evaluation of such programs. The project was intended to be exploratory in nature. Information was gathered on 65 programs based on written materials and conversations with a broad range of individuals. Then, information telephone conversations were conducted with 33 programs and site visits made to nine programs. Out of the assessment, four different program models were identified that are currently being used to help hard-to-employ individuals find and maintain employment: Model I) - paid work experience programs, (Model II) - supported transitional publicly funded jobs programs, (Model III) - supported transitional structured employment programs, and (Model IV) - supported competitive employment programs. Model I is targeted primarily to individuals who lack work experience although very little additional support, such as intensive case management or job coaching, is provided. Model II programs subsidize participant wages by using government funds, making them more costly than Model I programs. These programs provide temporary paid work experience. Model III programs provide transitional employment opportunities in a controlled setting such as a sheltered workshop, a social enterprise, or a group placement within a private company. While in transitional employment, participants receive substantial support. Model IV programs are almost always targeted to TANF clients with disabilities and provides significant support to promote success at the workplace.
PIC ID: 7016; CONTACT: Girley Wright, 202-401-5070; PERFORMER: Mathematica Policy Research, Inc., Plainsboro, NJ

In-Progress Evaluations

Head Start Child and Family Experiences Survey (FACES)

In order to provide longitudinal information on a periodic basis on Head Start program quality, family characteristics and experiences, and child outcomes in comprehensive domains of school readiness, the Head Start FACES study was launched in Fall 1997. The first cohort of 3200 children in 40 nationally representative programs was followed by a second cohort of 2800 children in 43 programs. Data are disseminated through regular progress reports, e.g., Head Start FACES: Longitudinal Findings on Program Performance (Third Progress Report, 2001). A final technical report on the 1997 cohort will be published in 2002.
PIC ID: 6331.3; EXPECTED COMPLETION: FY 2002; CONTACT: Louisa Tarullo, 202-205-8324; PERFORMER: Westat, Inc., Rockville, MD

Children of Color in the Child Welfare System

The objectives of this project are threefold: (1) to gain insights into how race, ethnicity, and culture are taken into account in the decision-making and service provision in the child welfare system; (2) to assess current efforts, and to identify promising practices that address disproportionalities or perceived inequities in service provision among racial and ethnic groups; and (3) to produce products that will provide guidance and assistance to the field in terms of policy and practice.
PIC ID: 7620; EXPECTED COMPLETION: FY 2002; CONTACT: Mary Bruce Webb, 202-205-8628; PERFORMER: Caliber Associates, Fairfax, VA

Descriptive Study of Families Served by Head Start

This three-year descriptive study will provide information on a nationally representative sample of families served by Head Start in forty programs across the country. Through a survey and more intensive case study methods, the study will chart families’ demographics, strengths, needs, expectations and experiences within Head Start programs, as well as programmatic efforts to join in partnership with families. (See PIC ID 6331.1)
PIC ID: 6331; EXPECTED COMPLETION: FY 2003; CONTACT: Louisa Tarullo, 202-205-8324; PERFORMER: Abt Associates Inc., Cambridge, MA

National Longitudinal Study of Children and Families in the Child Welfare System

(National Survey of Child and Adolescent Well-Being). The purpose of this study is to describe the outcomes experienced by children and families who come to the attention of the child welfare system, and to gain an understanding of the factors, including system-level and service factors, as well as child and family characteristics, that contribute to those outcomes. The study will select a nationally representative sample of 5,400 children upon entry into the child welfare system, and additional sample of 700 children who will have been in foster care for one year at the time they are selected. Information on these samples will be collected at baseline and at two follow-up interviews from the children and their caregivers, caseworkers, and other agency personnel and service providers. Public use data sets will be prepared following each wave of data collection.
PIC ID: 6748; EXPECTED COMPLETION: FY 2003; CONTACT: Mary Bruce-Webb, 202-205-8628; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

A Research Synthesis of the Effects of the Temporary Assistance for Needy Families (TANF) Program

This project will synthesize the most current research information available on the effect of TANF on income, earnings, receipt of government benefits, and family formation and structure, for individuals and families who were potentially affected by welfare reform, including both current and former TANF recipients, as well as others at risk of becoming dependent on cash assistance. It will also synthesize information available concerning the amount and duration of welfare and other benefits such as Medicaid, child care, child support, and food stamps. Finally, in addition to synthesizing research in subject areas that have been extensively studied, it will also document areas that have not been studied to identify possible ideas for future research.
PIC ID: 7540; EXPECTED COMPLETION: FY 2002; CONTACT: Alan Yaffe, 202-401-4537; PERFORMER: Rand Corporation, Santa Monica, CA

A Study of Infant Care Under Welfare Reform

Twenty-two states have used the new flexibility granted under the Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA) to require work of parents whose youngest child is less than one year old. The purposes of this study are to learn more about the policy and program challenges facing states that are encouraging welfare parents with infants to work or attend school, and to build the foundation for future research on programs, policies, and strategies that can successfully transition parents from welfare to work while promoting the health and development of infants. To achieve these objectives, the study will examine three central issues: (1) the policies, programs and strategies that states are currently using to transition parents with infants off of welfare and arrange child care for infants; (2) the ways in which parents who are transitioning from welfare to work meet the competing demands of work and family responsibilities and how they meet their infant care needs; and (3) the availability of infant care and its impact on parents’ ability to meet welfare program requirements.
PIC ID: 7113; EXPECTED COMPLETION: FY 2002; CONTACT: Richard Jakopic, 202-205-5930; PERFORMER: Mathematica Policy Research, Inc., Washington, DC

Achieving Change for Texans

This project continues an evaluation of the State of Texas’ original welfare reform demonstration. The demonstration consists of four major components: (1) a number of policies implemented Statewide addressing such recipient responsibilities as immunization for children, school attendance and adherence to a personal responsibility agreement; (2) a number of policies implemented in counties operating JOBS, the primary feature of which is differential benefit time limits based on consideration of work experience and the need for education; (3) a number of policy options implemented in four counties, providing for individual development accounts and fill-the-gap budgeting; and (4) a one-county pilot offering a check for $1,000 in lieu of regular Temporary Assistance for Needy Families (TANF), with no re-application for benefits for one year.
PIC ID: 6765; EXPECTED COMPLETION: FY 2003; CONTACT: John Maniha, 202-401-5372; PERFORMER: State of Texas, Department of Human Services, Austin, TX

Assessing Effective Welfare-to-Work Strategies for Domestic Violence Victims and Survivors in the Options/Opciones Project

This 5-year (1997-2002) research project studies the effective strategies in addressing the needs of abused women as they try to enter the labor market. This project documents the needs of battered girls and women on welfare, and will identify successful strategies employed to eliminate violence and exit welfare.
PIC ID: 6833; EXPECTED COMPLETION: FY 2002; CONTACT: Mary Ann MacKenzie, 202-401-5272; PERFORMER: Center for Impact Research, Chicago, IL

Assessing Enhanced Transitional Employment (ETE) Programs

This project will identify and describe employment-focused programs that help individuals who face significant employment challenges by providing transitional employment or work experience that builds the skills and capacity of participants through a supportive environment that may include close supervision, peer supports, and progressive performance expectations, and linkage or provision of needed services as well as other methods. The project will provide detailed descriptions of the programs identified, an assessment of program capacity and the feasibility of expanding existing programs to serve more participants, and, if not now served, TANF recipients with significant employment barriers; suggestions on how these programs might be replicated in other state/local settings; and an assessment of the feasibility of conducting a large-scale evaluation of such employment strategies, including an impact evaluation based on an experimental design.
PIC ID: 7539; EXPECTED COMPLETION: FY 2002; CONTACT: Girley Wright, 202-401-5070; PERFORMER: Mathematica Policy Research, Inc., Washington, DC

Assessing Medicaid and Food Stamps Access and Participation

This project, in cooperation with the Robert Wood Johnson Foundation, will help 12-14 states create and analyze performance data on how their Medicaid, SCHIP, and Food stamps enrollment and reapplications are functioning for families. It will identify root causes of problems in these processes, and develop specific implementation plans to solve identified problems and increase participation. In addition, 10 promising practice site visits will be conducted to help develop improvement implementation plans.
PIC ID: 7546; EXPECTED COMPLETION: FY 2002; CONTACT: Michael Dubinsky, 202-401-3442; PERFORMER: Mathematica Policy Research, Inc., Washington, DC

California Welfare Reform Impact Evaluation

This demonstration will attempt to estimate the impact of California’s Temporary Assistance for Needy Families (TANF) program. Generally, impacts will be assessed by comparing outcomes in the post-TANF period with the pre-TANF period. The impact of various county programs may also be examined by comparing outcomes among the counties. In developing models to estimate impacts, California will use a 10 percent statewide sample of welfare recipients. These samples date back to 1987, providing a rich database to use to test the models. The sample data consist of Department of Social Services information on Aid to Families with Dependent Children (AFDC), Supplemental Security Income (SSI), Medi-Cal and foster care. These data have been supplemented by data on unemployment and disability insurance, and Department of Health Services data on births. The major research questions concern the impact of states’ TANF program on self-sufficiency, family composition, and child well-being. To the extent possible, the impacts of TANF on different subgroups will be considered. Subgroups will include racial and ethnic subgroups, as well as urban and rural subdivisions. Attempts will also be made to determine the impacts of differing implementation of TANF among the counties.
PIC ID: 5548; EXPECTED COMPLETION: FY 2002; CONTACT: Leonard Sternbach, 415-437-7671; PERFORMER: California Department of Social Services, Sacramento, CA

Child Impact Studies

This project augments the existing welfare reform demonstration evaluations in five states (Connecticut, Florida, Indiana, Iowa, and Minnesota) to assess the effects of different welfare reform approaches on child well-being outcomes, including school achievement, behavioral problems, and health status. The effect of intervening mechanisms, such as the quality and regularity of the home environment, child care arrangements, and parental employment and income, will also be examined.
PIC ID: 6847; EXPECTED COMPLETION: FY 2002; CONTACT: Alan Yaffe, 202-401-4537; PERFORMER: Multi-Professional Services Contracts

Child Outcomes Synthesis Project

This project will synthesize the results from the Program on State-Level Child Outcomes, a series of demonstrations in five states that measure the impacts of welfare reform on the well-being of children.
PIC ID: 7527; EXPECTED COMPLETION: FY 2002; CONTACT: Alan Yaffe, 202-401-4537; PERFORMER: Child Trends, Inc., Washington, DC

Economic Analysis of the Prenatal and Early Childhood Nurse Home Visitation Program

This study will examine cost savings to government, resulting from an investment in a program of prenatal and early childhood home visitations that have been studied in a series of three randomized controlled experiments. The experiments will be conducted in three different settings (Elmira, NY, Memphis, TN, and Denver, CO) with different sample compositions.
PIC ID: 7547; EXPECTED COMPLETION: FY 2002; CONTACT: Girley Wright, 202-401-5070; PERFORMER: Kempe Prevention Research Center for Family & Child Health, Denver, CO

Economic Independence and Inclusion Model: A Person-Centered Approach

This project is intended to develop and test the feasibility of an effective local/community-based, long-term and holistic service delivery model for individuals with severe or multiple disabilities. The model will be person-centered and -controlled with easy access to a coordinated network of flexible services and supports designed to enhance the opportunity for these individuals to enter and/or remain in the job market under more satisfactory conditions. The methodology will consist of reviewing and analyzing research literature and pertinent federal and state policies and programs; conducting 22 case studies of past and present approaches and best practices; designing, field testing (in three sites) and finalizing the desired model; using an expert advisory panel to review all products. The feasibility of the draft/proposed model is currently being tested in three states.
PIC ID: 7544; EXPECTED COMPLETION: FY 2002; CONTACT: Hossein Faris, 202-205-4922; PERFORMER: McIver, Diana & Associates, Inc., Austin, TX

Employment and Retention and Advancement Project

In about 13 sites in nine (9) states, this project will conduct an evaluation of program techniques for enhancing employment retention and advancement of low income families. The methodology includes random assignment to experimental and control groups compared for impacts; process analyses in about 13 sites; and, cost effectiveness studies.

Evaluating the Survey of Income and Program Participation (SIPP) Welfare Benefits Using Matched Administrative Records from California

This project will complete a comparative analysis of patterns and trends in welfare participation and reporting in SIPP. It will also determine the accuracy of SIPP program data by comparing it against state administrative records.
PIC ID: 7536; EXPECTED COMPLETION: FY 2002; CONTACT: Leonard Sternbach, 415-437-7671; PERFORMER: University of California at Los Angeles, Los Angeles CA

Evaluation of Arizona Employing and Moving People Off Welfare and Encouraging Responsibility Program (EMPOWER)

Under this Aid to Families with Dependent Children (AFDC) and Temporary Assistance for Needy Families (TANF) demonstration, recipient families are encouraged to become self-sufficient by: (1) time-limiting cash assistance for adult recipients, (2) imposing a family cap, (3) extending transitional Medicaid and child care to 24 months, (4) eliminating the 100 hour rule for two parent families, (5) requiring unwed minor parents to live with a responsible adult, and (6) requiring 13 to 16 year old parents to participate in Job Opportunities and Basic Skills (JOBS).
PIC ID: 6818; EXPECTED COMPLETION: FY 2002; CONTACT: Leonard Sternbach, 415-437-7671; PERFORMER: Arizona State Department of Economic Security, Phoenix, AZ

Evaluation of Community-based Job Retention Programs

This project includes two components. In one, the project will provide an implementation analysis and outcome findings for current and former welfare recipients receiving employment retention and post-employment services through the GAPS program. The services will be provided to about 600 employed TANF recipients through five community-based organizations in the Pittsburgh area that received funding from The Pittsburgh Foundation. A common set of services will be provided but within the context of different existing service delivery systems among the five organizations. The project will describe the experiences of providers and participants, report on the economic success of participants over time, and present lessons for program administrators. The second component will include a descriptive analysis of the Community Solutions program through which the state Department of Human Services contracted for pre- and post-employment services from service providers using benchmark-based contracting and payments. Providers receive incremental payments when TANF recipient participants attain certain benchmarks including maintaining steady employment for 12 months. This component of the study focuses primarily on the experiences of service providers operating programs under the requirements and payment schedules of benchmark-based contracts.
PIC ID: 6760; EXPECTED COMPLETION: FY 2002; CONTACT: Nancye Campbell, 202-401-5760; PERFORMER: Pittsburgh Foundation, Pittsburgh, PA

Evaluation of the Employment First Program

This process evaluation will primarily focus on implementation and operation of the Nebraska Employment First Program and will conclude a special study to assess successful case management. In addition, a separate client barriers study oversampling rural areas will be conducted.
PIC ID: 6826; EXPECTED COMPLETION: FY 2002; CONTACT: Michael Dubinsky, 202-401-3442; PERFORMER: Nebraska State Department of Health and Human Services, Lincoln, NE

Experience of Tribal TANF Programs: Problems, Solutions, and Lessons Learned

This project will develop national-level research information on tribal TANF programs that will be responsive to the needs of Native American tribal governments in deciding to initiate or improve their own TANF programs, as well as the needs of policymakers at federal, state and local levels. The project will include a survey of all TANF tribes funded as of August 30, 2000 and a sample of non-TANF tribes, supplemented by in-depth on-site case studies of a sample of nine tribes. It is primarily an implementation study that will use a combination of qualitative and quantitative data and analytical methods.
PIC ID: 7542; EXPECTED COMPLETION: FY 2002; CONTACT: Hossein Faris, 202-205-4922; PERFORMER: Support Services International, Inc., Silver Spring MD

Fragile Families and Welfare Reform

This project will describe the conditions and capabilities of low-income unwed parents with a new child and examine the complex relationships between welfare reform legislation and families’ economic and non-economic well-being. The project will document how unwed mothers are packaging various forms of support and government programs, and how well families are doing as a result of individual efforts and social policies. The study utilizes the Fragile Families and Child Well-Being Survey that includes a random sample of new unmarried mothers and fathers in 20 large cities that are representative of large cities in the United States. The cities are stratified by policy regime and labor market strength so they represent an array of economic and policy conditions in the U.S. The study undertakes an analysis of the relationships between city variation (policies, economic conditions, etc.) and a number of outcomes including the types of assistance upon which unwed parents rely and individual and child well-being.
PIC ID: 7534; EXPECTED COMPLETION: FY 2003; CONTACT: Brendan Kelly, 202-401-5600; PERFORMER: Columbia University, New York, NY

Front-Line Management and Practice Study

This is a study of Temporary Assistance to Needy Families (TANF) implementation at the local level, and is embedded in the larger SUNY-Albany State Capacity study, which is a 20 state implementation analysis. The principal objective of the Front-Line Management and Practice Study is to evaluate whether front-line workers are implementing the welfare goals and policies established by the states. Locally-based researchers conducted in-depth observations in three local offices from each of four states. A criterion for site selection dictates that the state give primary emphasis to work and job preparation, thus enabling the researchers to make comparisons among management practices that aim to affect the same sort of policy outcomes. Some sites have been chosen because they use traditional rule-based management and others because they rely on performance-based management.
PIC ID: 6841; EXPECTED COMPLETION: FY 2002; CONTACT: Michael Dubinsky, 202-401-3442; PERFORMER: State University of New York at Albany, Albany, NY

Impact Study of the New Hampshire Employment Program

This demonstration is designed to determine the impact of New Hampshire’s entire Temporary Assistance for Needy Families (TANF) program. It will attempt to determine the effects of TANF on caseload and on employment and earnings, as well as the effects on areas such as foster care placements, child abuse and neglect, homelessness and child support collections.
PIC ID: 6828; EXPECTED COMPLETION: FY 2002; CONTACT: Leonard Sternbach, 415-437-7671; PERFORMER: New Hampshire DHHS, Concord, NH

Indiana Welfare Reform Evaluation Project

This project continues the evaluation of implementation and impacts of the “Indiana Manpower Placement and Comprehensive Training Program” (IMPACT) welfare reform waiver demonstration, now operating under Temporary Assistance for Needy Families (TANF).
PIC ID: 6869; EXPECTED COMPLETION: FY 2002; CONTACT: Alan Yaffe, 202-401-4537; PERFORMER: Indiana Family and Social Services Administration, Indianapolis, IN

Jobs-Plus: Community Revitalization Initiative for Public Housing Families

This is a seven and one-half year demonstration program aimed at dramatically increasing employment, earnings and job retention among the working-age residents of family housing developments, a large percentage of whom are on public welfare or at risk of dependency. The program supports the planning, development, implementation and evaluation of locally-based approaches to providing saturation-level employment opportunities, including a combination of training and supportive services, financial and other incentives (occasioned by the welfare reform and new public housing policies), and vigorous efforts to rebuild and strengthen the community in support of work. The bottom line question for the Jobs-Plus Demonstration is: Does the program achieve its primary goal of dramatically increasing employment and earnings among public housing residents? And, does a large increase in employment and earnings lead to a better quality of life for residents? Questions concerning the sites’ implementation strategies and experiences are also important and will be a major focus of the research. The evaluation design developed for the demonstration combines experimental and quasi-experimental methods in an unusually rigorous approach for studying the effectiveness of a place-based comprehensive social intervention. (It uses randomly selected sets of treatment and comparison sites.) The study will use administrative records data as well as survey data covering a wide variety of outcomes. It will examine whether the program’s impacts vary across sites and whether certain program strategies are likely to yield better results. The evaluation will include comprehensive cost and benefit-cost analyses.
PIC ID: 6835; EXPECTED COMPLETION: FY 2004; CONTACT: Hossein Faris, 202-205-4922; PERFORMER: Manpower Demonstration Research Corporation, New York, NY

Maryland Family Investment Program Evaluation

This is a Statewide process study to examine and document front-line assessment and allocation practices under TANF in Maryland’s 24 local jurisdictions. The objective of this analysis will be to help clarify the relationship between agency factors and county-level characteristics by describing county-level variation in front-line procedures.
PIC ID: 6823; EXPECTED COMPLETION: FY 2002; CONTACT: Lawrence Wolf, 202-401-5084; PERFORMER: University of Maryland, School of Social Work, Baltimore, MD

Minnesota WorkFirst Program (Track 2)

Using a quasi-experimental, pre-test/post-test comparison group design consisting of both an impact and process study, this study will compare the WorkFirst and MFIP-S programs. WorkFirst uses a rapid labor force attachment strategy versus MFIP-S, which represents a progressive labor force attachment model.
PIC ID: 6825; EXPECTED COMPLETION: FY 2002; CONTACT: Lawrence Wolf, 202-401-5084; PERFORMER: Minnesota Department of Human Services, St. Paul, MN

National Study of Child Care for Low Income Families

This project will study the low income child care market in 25 communities in 17 states with a sub-study to examine the family child care market in 5 neighborhoods drawn from these communities. In addition, the project will conduct a survey of 2,500 low-income families in the same 25 communities to determine how child care decisions are made and to study the relationship of child care subsidies to their choices. The information provided from the study will help inform the issues surrounding subsidized child care and its implementation by the states, with particular attention to the provisions in the Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA) over time, as well as how significant shifts in welfare policy and programs affect the child care market for welfare recipients and the working poor at the community level.
PIC ID: 6845; EXPECTED COMPLETION: FY 2003; CONTACT: Richard Jakopic, 202-205-5930; PERFORMER: Abt Associates Inc., Cambridge, MA

Neighbors, Services Providers, and Welfare Reform in Los Angeles County

This project will examine neighborhood variation in the availability of public and private social services throughout Los Angeles County. It will investigate how agencies are adapting to the current and anticipated changes in demand for their services as a result of welfare reform.
PIC ID: 6763; EXPECTED COMPLETION: FY 2005; CONTACT: Girley Wright, 202-401-5070; PERFORMER: Rand Corporation, Santa Monica, CA

New Hampshire Employment and Training Program Process and Outcome Study

This Process and Outcome Study goes hand-in-hand with the New Hampshire Impact Study. By using surveys of recipients, staff and employers, this process study will determine how Temporary Assistance for Needy Families (TANF) was planned, designed and implemented. There will also be several special studies concerning diversion from welfare, sanctions, child care and transitional case management.
PIC ID: 6827; EXPECTED COMPLETION: FY 2002; CONTACT: Leonard Sternbach, 415-437-7671; PERFORMER: New Hampshire DHHS, Concord, NH

New Jersey Substance Abuse Research Demonstration

This evaluation will provide information about the effectiveness of a type of evaluation several states are experimenting with to move substance abusing welfare clients toward self-sufficiency. The intervention New Jersey is implementing includes screening welfare recipients for substance abuse problems, treatment referral mechanisms with enhanced case management, and substance abuse treatment coordinated with employment and training or vocational services. The intervention being evaluated is intended to improve the post welfare prospects of TANF recipients with substance abuse problems. The evaluation is being conducted in two New Jersey counties, Essex County and Atlantic County.
PIC ID: 7528; EXPECTED COMPLETION: FY 2002; CONTACT: K.A. Jagannathan, 202-205-4829; PERFORMER: Mount Sinai School of Medicine, New York NY

Ohio Works First Evaluation

This demonstration will evaluate the Ohio Works First provisions and Temporary Assistance for Needy Families (TANF) policies which include: (1) completion of a self-sufficiency contract, (2) more generous income disregards, (3) a 36-out-of-any-60 month time limit, and (4) whole family sanctions. In Ohio, local jurisdictions have great flexibility in running their TANF programs, and this evaluation will attempt to determine the differential effects of the various local programs.
PIC ID: 6758; EXPECTED COMPLETION: FY 2002; CONTACT: Leonard Sternbach, 415-437-7671; PERFORMER: Ohio Department of Human Services, Columbus, OH

Partner and Father Involvement in the Lives of Low-Income First Time Mothers and Their Children

This project will investigate the role that fathers and partners play in improving the material, emotional, and developmental well-being of low-income women and children. It consists of a set of secondary analyses using data from three longitudinal experiments of a program of prenatal and infancy home visitation serving first time mothers from various ethnic and racial groups.
PIC ID: 6799; EXPECTED COMPLETION: FY 2002; CONTACT: Lawrence Wolf, 202-401-5084; PERFORMER: Children’s Hospital, Denver, CO

Prenatal and Early Childhood Nurse Home Visitation Program - Replication/Dissemination

The Administration for Children and Families (ACF) and the Department of Justice (DOJ), Office of Juvenile Justice and Delinquency Prevention are supporting this initiative. The project allows for data collection and evaluation of an initiative to replicate and disseminate the nurse home visiting model. The model has been tested through randomized trials and found to be effective in improving outcomes for mothers and their children on a number of important outcome measures, including: (1) educational attainment, (2) employment, (3) welfare dependency, (4) parenting attitudes, and (5) subsequent pregnancies. The DOJ funded a grant to support technical assistance and training to replicate the model in selected sites. This project supports the evaluation component of the initiative to test the effectiveness of the replication process within normal operating environments. The initiative will answer questions pertaining to: (1) whether programs are implemented with fidelity to the original program model; (2) whether the program is reaching the target population of at-risk, low-income pregnant women; (3) the aspects of the model that are most difficult to implement and maintain; (4) the factors explaining site variation in program fidelity; and (5) the outcomes for pregnant women and families enrolled in each site.
PIC ID: 6757; EXPECTED COMPLETION: FY 2002; CONTACT: Girley Wright, 202-401-5070; PERFORMER: University of Colorado Health Sciences Center, Denver, CO

Rural Welfare to Work Strategies Demonstration Evaluation Project

As part of its multifaceted national strategy for welfare reform research and evaluation, ACF is funding the Rural Welfare to Work Strategies Demonstration Evaluation Project to learn how best to help TANF and other low-income rural families move from welfare to work. The demonstration evaluation will lead to increased information on well-conceived rural welfare to work strategies and lessons about the operational challenges and methods to address them that can be used by state and local TANF agencies and others. It is expected that four states/sites will participate in the national demonstration evaluation. The project will address the following key questions: 1) What types and packages of services are provided under the RWtW project, and how do they compare with services already available under TANF or other funding? 2) What are the issues and challenges associated with implementing and operating the service packages and policy approaches studies? 3) What are the net impacts of selected approaches under the project on employment and on families’ well-being? 4) What are the net costs of the programs, and do the programs’ benefits outweigh the costs? 5) What strategies should policymakers and program managers consider in designing approaches to improve the efficacy of welfare to work strategies for families in rural areas? The evaluation plan includes three main components: 1) In-Depth Process and Implementation Study; 2) Impact Study; and 3) Cost-Benefit Study.
PIC ID: 7247; EXPECTED COMPLETION: FY 2005; CONTACT: James Dolson, 202-260-6165; PERFORMER: Mathematica Policy Research, Inc., Washington, DC

Sexual Abuse Experiences of Runaway Youth

This project responds to a legislative requirement to conduct a study that will address the extent to which sexual abuse exists in runaway youths, and provide information about the perpetrators of any such abuse. The project will review available literature and conduct secondary analyses to examine the sexual abuse experiences of runaway youth, particularly experiences that occur prior to the youths’ leaving home.
PIC ID: 7549; EXPECTED COMPLETION: FY 2002; CONTACT: Mary Bruce-Webb, 202-205-8628; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

South Carolina Project Independence

This project examines the state’s TANF implementation of two provisions: (1) relocation requirements and services affecting recipients in counties with higher unemployment rates; and (2) policies related to alcohol or drug abuse referral and treatment. In addition, Project Independence former and diverted clients will be surveyed, and the survey results will be linked to administrative records to determine the status and well-being of the families and their children.
PIC ID: 7550; EXPECTED COMPLETION: FY 2003; CONTACT: Michael Dubinsky, 202-401-3442; PERFORMER: South Carolina Department of Social Services, Columbia, SC

State Welfare Reform Evaluation Project (Connecticut’s Jobs First)

Using experimental methodology, this project continues an originally planned evaluation of the Jobs First demonstration, later incorporated into the State of Connecticut’s TANF plan. The evaluation includes: (1) a short (21 month) eligibility time limit for non-exempt recipients; (2) 24 months of transitional Medicaid; (3) liberalized disregards; (4) “Family Cap”, limited eligibility for children born while the family receives assistance; (5) strong job search emphasis, with employability assessment only if a search fails to yield a job; and (6) progressive sanctions for non-compliance.
PIC ID: 6819; EXPECTED COMPLETION: FY 2002; CONTACT: Lawrence Wolf, 202-401-5084; PERFORMER: Manpower Demonstration Research Corporation, New York, NY

TANF Time Limits

This project will report on what states are doing with clients that are approaching or that have reached TANF, federal and/or state time limits.
PIC ID: 7621; EXPECTED COMPLETION: FY 2002; CONTACT: Michael Dubinsky, 202-401-3442; PERFORMER: Manpower Demonstration Research Corporation, New York, NY

Testing Non-experimental Methodologies Using the National Evaluation of Welfare-to-Work Strategies (NEWWS) Data

The purpose of this demonstration is to test various non-experimental evaluation techniques. The emphasis of this demonstration will be to test whether an adequate control group (not randomly assigned) can be established using propensity score matching techniques as described in recent research literature. The data to be used will be from the NEWWS demonstrations that involved efforts to improve employment and earnings outcomes of welfare recipients at a number of sites. In the NEWWS demonstrations, evaluations were carried out using random assignment to establish treatment and control groups at each site. This demonstration will test modeling techniques by comparing outcomes from the randomly assigned control groups to the outcomes from control groups defined by non-experimental methodologies. This demonstration will deal with data from seven NEWWS sites.
PIC ID: 7541; EXPECTED COMPLETION: FY 2002; CONTACT: Leonard Sternbach, 415-437-7671; PERFORMER: Manpower Demonstration Research Corporation, New York, NY

The Fiscal Effects of Welfare Reform

Building on a four-state pilot project, this study will conduct case studies in 13 states to assess how social services spending and policies have changed after welfare reform was implemented. More specifically, the study will examine changes in spending in absolute dollars, in real dollars per poor person, and as a percentage of state budgets. This will provide insights into how state budgetary priorities have changed, to learn about trends and differences across states, and to target inquiries about underlying state policy changes that have led to spending changes.
PIC ID: 7545; EXPECTED COMPLETION: FY 2002; CONTACT: Michael Dubinsky, 202-401-3442; PERFORMER: State University of New York at Albany, Albany, NY

The Illinois Families Study

As families transition from welfare to work in the context of welfare reform policies, large variation in welfare use, income, employment and training, and health care coverage is inevitable. This project capitalizes on this variation to assess the relationship between each of these factors and multiple measures of child well-being within the TANF recipient population. The focus of the research is the first three (3) years of an ongoing panel study involving 1,500 families who were receiving TANF in late 1998. Annual survey interviews assess parental reports of each child’s academic performance, behaviors, and health, as well as key risk and protective factors. The survey data will be combined with administrative data from the public welfare system, the unemployment insurance system, children’s school records, medical chart reviews of a subset of children from the sample, and Medicaid claims data.
PIC ID: 7533; EXPECTED COMPLETION: FY 2002; CONTACT: Alan Yaffe, 202-401-4537; PERFORMER: Northwestern University, Evanston IL

Time Limits, Welfare Transitions, and the Age Distribution of Children Receiving Welfare

This demonstration will analyze the effects of time limits on welfare entry, welfare exits, and the age distribution of children receiving welfare. The analysis will use SIPP data from the 1985 through the 1996 panels covering September 1984 through February 1998. The analysis will be of welfare spells of female-headed households. The effects of time limits will be determined by considering the implementation dates of time limits that states implemented under waivers prior to the passage of PRWORA. Various demographic and economic state-level variables will be incorporated into the models to be used.
PIC ID: 7537; EXPECTED COMPLETION: FY 2002; CONTACT: Leonard Sternbach, 415-437-7671; PERFORMER: University of California at Los Angeles, Los Angeles CA

Virginia Independence Program

This project evaluates the State’s TANF implementation, which continues policies begun under its welfare reform demonstration, Virginia Independence Program (VIP). Among the policies to be studied under the state’s VIP are: (1) diversionary assistance, (2) a family cap, (3) time-limited assistance, (4) personal responsibility agreements, (5) expanded earned income disregards, (6) school attendance requirements, (7) paternity establishment rules, (8) requiring minor parents to live in adult settings, (9) child immunization requirements, and (10) incentives promoting individual savings accounts and Medicaid and child care benefits.
PIC ID: 6831; EXPECTED COMPLETION: FY 2003; CONTACT: John Maniha, 202-401-5372; PERFORMER: Virginia Commonwealth University, Richmond, VA

Welfare Restructuring Project (WRP)

This project, operating statewide in Vermont, continues the evaluation of a welfare reform demonstration project. One of the key features is a time limit that requires some adult recipients to participate in community work experience after fifteen months (for two-parent families) or thirty months (for single-parent families) of receiving cash assistance. It also tests more generous earned income disregards to promote work. The evaluation is designed with two treatment groups. One treatment group receives only the enhanced work incentives; the second receives the work incentives and is subject to work requirements. This will allow the evaluation to measure the distinct impact of work requirement policies.
PIC ID: 6764; EXPECTED COMPLETION: FY 2002; CONTACT: Girley Wright, 202-401-5070; PERFORMER: State of Vermont, Department of Social Welfare, Waterbury, VT

Welfare-to-Work: Monitoring the Impact of Welfare Reform on American Indian Families with Children

The overall purpose of this project is to monitor and document the implementation, and assess the impact of, welfare reform on American Indian families and reservations in Arizona caused by the evolving State and tribal responses to Temporary Assistance to Needy Families (TANF). Extensive demographic, contextual, socioeconomic and case-level data are to be compiled from a variety of sources, and analyzed in order to document the baseline conditions and changing patterns, and to identify short-term outcomes. The researchers will: (1) use existing national data (e.g., Census, CPS, and SIPP) on American Indians to provide a context for the study of Arizona, (2) collect quarterly case- or household-level data on the welfare recipients on Indian reservations in Arizona to examine their characteristics and monitor changes in their situations under TANF, and (3) track and document TANF implementation by tribal entities using content analysis of relevant documents, in-depth telephone interviews and site visits. The evaluation of the long-term impact of the reform program is to be undertaken during the last four years of the project.
PIC ID: 6832; EXPECTED COMPLETION: FY 2002; CONTACT: Hossein Faris, 202-205-4922; PERFORMER: Washington University, School of Social Work, St. Louis, MO

What Works Best for Whom: Effects of Welfare Reform Policies on Subgroups of Current and Former Welfare Recipients

This project investigates the effects of recent welfare policies on various subgroups of welfare recipients. It adds to earlier work on subgroups by adding more programs to the analysis, looking at more outcomes, looking at more subgroups, and including longer follow-up for some programs. The major goals of the project are to help program operators target precious resources toward groups most likely to benefit from the programs they are running, and to help policy makers understand which groups may need new approaches to help them move to work, stay at work, advance in their jobs, or help their children. The study will use data on nearly 100,000 individuals from 25 programs in 11 states, from MDRC experimental studies of welfare employment programs.
PIC ID: 7532; EXPECTED COMPLETION: FY 2002; CONTACT: Alan Yaffe, 202-401-4537; PERFORMER: Manpower Demonstration Research Corporation, New York, NY

Administration on Aging

Mission

To foster the development of services to help older persons maintain their independence.

Evaluation Program

The Administration on Aging (AoA) is the federal focal point and advocate agency for older persons and their concerns. The AoA administers key federal programs mandated under various titles of the Older Americans Act. These programs help vulnerable older persons remain in their own homes by providing supportive services. Other programs offer opportunities for older Americans to enhance their health and to be active contributors to their families, communities, and the Nation through employment and volunteer programs. The AoA works closely with its nationwide network of regional offices and State and Area Agencies on Aging to plan, coordinate, and develop community‑level systems of services that meet the unique needs of individual older persons and their caregivers. The AoA collaborates with federal agencies, national organizations, and representatives of business to ensure that, whenever possible, their programs and resources are targeted to the elderly and coordinated with those of the network on aging. As the responsibilities of this nationwide network of State and Area Agencies on Aging continue to grow, it is essential that they have the necessary information to meet these responsibilities.

The overall evaluation priorities of the AoA are to support studies that provide information on the following: (1) the success of existing program implementation in meeting the goals of the Older Americans Act; (2) the design and operation of effective programs; and (3) issues relevant to policy development, legislative planning, and program management.

In-Progress Evaluations

National Survey of Older Americans Act Participants Using Indicators Developed under the Performance Outcomes Measures Project

The purpose of this project is to conduct a statistically valid national survey of recipients of services under the Older Americans Act using survey questions designed in concert with states and Area agencies on aging. Questions have been field-tested by states under AoA’s current performance outcomes measures project. Telephone surveys will be conducted by Westat on a sample drawn with the cooperation of approximately 150 area agencies nationwide.
PIC ID: 7650; EXPECTED COMPLETION: FY 2002; CONTACT: David Bunoski, 202-260-0669; PERFORMER: Westat, Inc., Rockville, MD

Performance Outcome Measures Project - Field Testing Phase

The Performance Outcomes Measures project (POMP) has been sponsored by the Administration on Aging (AoA) to develop performance based measures for evaluating aging services. Another goal is to develop a prototype system for recording and reporting agency level information. POMP is a multi-agency collaboration involving state and local agencies on aging, AoA, contractors and consultants. The performance measures developed for this project are intended to cover a range of issues such as: documenting that services are targeted to persons with the greatest social and economic needs, as the Older Americans Act requires; identifying, exploring, or monitoring important aspects of the services provided on the communities and client populations served by programs, even where no clear link to services can be established and analyzing the accomplishments of programs or services for service recipients. AOA has recently made grants to thirteen states for a third year of field testing of the survey instruments developed during the first year of the project. Preliminary data from the first project year are now beginning to arrive and to be analyzed with an eye toward program improvements.
PIC ID: 6855; EXPECTED COMPLETION: FY 2002; CONTACT: David Bunoski, 202-260-0669; PERFORMER: Westat, Inc., Rockville, MD

Alzheimer’s Disease Demonstration Grants to States Program Evaluation

This contract provides for a statutorily mandated programmatic evaluation of the Alzheimer’s Disease Demonstration Grants to States (ADDGS) program, which was transferred to AoA from HRSA in November 1998. Included in this contract is data collection and management, and program evaluation of the 15 Demonstration sites nationwide. This contract is intended to support the continuation of data collection and monitoring for accuracy, completeness and quality, the collection of all program data from the 15 ADDGS grantees for a period of 20 months, and the dissemination of program data and information at national professional conferences. Contract includes the provision of a final report on the program.
PIC ID: 7296; EXPECTED COMPLETION: FY 2002; CONTACT: Melanie K. Starns, 202-401-4547; PERFORMER: University of Kansas, Lawrence, KS

Alzheimer’s Disease Demonstration Grants to States Program: Further Evaluation and Data Analysis

The purpose of this contract is to analyze the longitudinal data collected for the past seven years through the Alzheimer’s Disease Demonstration Grants to States program and to produce a written report of that analysis for use in reporting to Congress on some of the outcomes of the ADDGS Program.
PIC ID: 7297; EXPECTED COMPLETION: FY 2002; CONTACT: Melanie K. Starns, 202-401-4547; PERFORMER: University of Kansas, Lawrence, KS

Alzheimer’s Disease Demonstration Grants to States Program: Managed Care Initiative

The purpose of the ADDGS Managed Care Initiative, initiated by HRSA in 1997 (when they were administering this program) is to test the effect of community-based interventions for persons with Alzheimer’s disease and their families upon use of primary care physicians in a managed care setting. Final analysis will seek to determine if there is a decreased utilization of primary care physicians for resource and referral purposes and if there is any discernable difference between families participating in the program and those accessing information through routine channels.
PIC ID: 7298; EXPECTED COMPLETION: FY 2003; CONTACT: Melanie K. Starns, 202-401-4547; PERFORMER: University of Kansas, Lawrence, KS

Agency for Healthcare Research and Quality

Mission

To generate and disseminate information that improves the health care system.

Evaluation Program

The Evaluation Program within the Agency for Healthcare Research and Quality (AHRQ) provides information on the Agency’s effectiveness in meeting its long‑term goals and priorities, the efficiency of the Agency’s program performance through internal evaluations, and critical Agency and departmental concerns via “fast‑track” or quick‑turnaround assessments.

Evaluation components are built into virtually all major AHRQ programmatic activities and a wide variety of “freestanding” projects are undertaken as needs are identified. Among the evaluation mechanisms used by the Agency are targeted evaluation studies undertaken through contracts or grants; efforts to obtain feedback from “customers” on the usefulness of AHRQ research efforts, including such mechanisms as focus groups and surveys; and feedback from AHRQ’s User Liaison program (which provides information and technical assistance to state policymakers, health departments, and officials). Information gathered from AHRQ’s evaluation‑related activities are used in the annual performance plans developed in response to the Government Performance and Results Act.

Completed Evaluations

Evaluation of Performance of the Medical Expenditure Panel Survey: Final Report on Tasks 1 and 2

The purpose of this project was to evaluate key aspects of the Medical Expenditure Panel Survey (MEPS) in terms of data quality, cost, and timeliness. The evaluation focused on areas in which there have been programmatic concerns, especially those in which the data now exists for evaluation and which would be implemented in a timely way to inform quality enhancements for MEPS. In the current design of the MEPS, the Medical Provider Component (MPC) was not intended to support inference to the population of providers in the U.S. The research questions motivating MEPS have focused on household members as the unit of analysis and have concerned the health care received by this population. Data were obtained from medical providers only because the household members themselves are not reliable sources of data on health care use and expenditures. Where appropriate, the evaluation identified priorities for redesign, and outlined potential cost-effective alternatives.
PIC ID: 7401.1; CONTACT: Michael Hagan, 301-594-1354; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

Review of Existing Reports on Health Care Quality to Aid the Development of the National Quality Report

This review was designed to summarize for AHRQ what others have done in reporting on health care quality and, at the same time, to draw lessons from those works for the national quality report (NQR). The purpose was to review existing health care quality reports for content and style, to identify elements of reports that should be adopted and replicated in the NQR, and to determine approaches to be avoided. Included for review were reports developed by any organization, whether international organizations, foreign countries, state governments, purchasing coalitions, health plans, or other groups. Key insights were: (1) Audience: For the NQR, a primary audience should be defined early in the work. (2) Purpose: It will be important to articulate clearly the purpose of the report and its specific goals. (3) Content: These areas need to be considered for the NQR --populations covered, aggregations used, comparisons made, level of disclosure, and statistics presented. (4) Special Populations: The NQR may have to reanalyze existing data sources to obtain population subgroup comparisons and use static assessments of health care quality for special populations from special studies in the literature. (5) Data Sources: Existing reports provided few ready private sources of data for the NQR. (6) Media and Style: The NQR should be produced in both published and Internet versions, with the Internet version containing a full-text “PDF” format. (7) Evaluation: AHRQ may want to consider both Internet feedback and a scientific evaluation of the NQR after its release.
PIC ID: 7715; CONTACT: Diana Dodd, 301-594-1543; PERFORMER: Medstat Group, Washington, DC

In-Progress Evaluations

Development of Point of Service Assessments for the Office of Management

This evaluation will be undertaken to develop point-of-service assessments for the Office of Management in the areas of personnel, administrative support, information technology, telecommunications, and contract general support services.
PIC ID: 7485; EXPECTED COMPLETION: FY 2002; CONTACT: Barry Flaer, 301-594-1433; PERFORMER: Washington Consulting Group, Bethesda, MD

Evaluation of Accounting and Financial Management Services

The contractor is to provide on an as needed basis, consulting services for accounting, auditing, financial reporting review, financial management, analysis, planning, performance measurement, and other related services.
PIC ID: 7487; EXPECTED COMPLETION: FY 2002; CONTACT: Barry Flaer, 301-594-1433; PERFORMER: Barents Group, KPMG Peat Marwick LLP, Washington, DC

Evaluation of Centers for Education and Research on Therapeutics (CERTS)

This task is designed to evaluate the impact of the CERTs program and to determine if the program is moving toward its goals as designed in the RFA. The program goal is to develop free-standing continuing education centers which conduct research, educate health care providers and other decision makers, and improve the quality of care while reducing costs with a concomitant decrease in errors and adverse effects. There will also be a focus on the effectiveness and necessity of specific administrative/organization components of the CERTs program, namely the activities of the Coordinating Center and the Steering Committee. The products will include an evaluation of progress and products of the first year and one-half of the centers, the Coordinating Center and the Steering Committee, and will help identify accomplishments to date and new ideas for implementation in the CERTs II initiative.
PIC ID: 7686; EXPECTED COMPLETION: FY 2002; CONTACT: Lynn Bosco, 301-594-2416; PERFORMER: Washington Consulting Group, Bethesda, MD

Evaluation of Sources of Information Used to Track the Impact of AHRQ-Sponsored Research

The purpose of this task is to compare the effectiveness of five approaches to determining the impact of AHRQ funded research, tools and data.
PIC ID: 7685; EXPECTED COMPLETION: FY 2002; CONTACT: Kevin Murray, 301-594-6128; PERFORMER: Center for Health Policy Studies, Columbia, MD

Feasibility Study for an Applied Patient Safety Training & Outplacement Program (Patient Safety Improvement Corps)

This project is designed to determine the feasibility of establishing, operating, and maintaining an applied patient safety training and outplacement program.
PIC ID: 7683; EXPECTED COMPLETION: FY 2002; CONTACT: Marge Keyes, 301-594-1824; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

Implementation Planning Study for Integration of Medical Event Reporting Input and Data Structure for Reporting to AHRQ, CDC, CMS & FDA

The purposes of this project are: (1) to identify existing and developing data collection activities within the four participating DHHS partners, where they are located, how the information is stored (including specifications for the data storage), and similarities and redundancies in the current data infrastructures in the four Agencies; (2) to examine the conceptual and technological issues, complexity, and costs involved in development of a single interface for collecting data related to medical error and patient safety risks through the integrating existing and developing DHHS reporting systems; and, (3) to identify and evaluate advantages and disadvantages of options for the future development of a web-based integrated reporting interface among the existing and developing data collection systems in the four partnering agencies. AHRQ and its DHHS partners intend to apply the report of this implementation planning study task order to future procurement to continue the data and reporting integration development process.
PIC ID: 7681; EXPECTED COMPLETION: FY 2002; CONTACT: James Battles, 301-594-9892; PERFORMER: MEDSTAT Group, Cambridge, MA

Patient Safety Measures

This project will develop two baseline patient safety evaluation measures: 1) safety culture assessment and 2) a survey on medical event reporting systems. These instruments shall be used to determine context and antecedent conditions existing prior to initiation or at the beginning of implementation of AHRQ’s patient safety research initiative.
PIC ID: 7679; EXPECTED COMPLETION: FY 2002; CONTACT: James Battles, 301-594-9892; PERFORMER: Barents Group, KPMG Peat Marwick LLP, Washington, DC

Health Services Research Education: Assessing Customer Satisfaction and Program Needs

This is an evaluation of certain elements of a national training program in health services research funded by the Agency for Healthcare Research and Quality (AHRQ). The project will provide AHRQ with information about present and anticipated needs in health services research training and an assessment of the level of satisfaction that past trainees from the institutionally based training programs have had with their experiences during training, and with the adequacy of these training activities relative to trainees’ career expectations and requirements.
PIC ID: 7397; EXPECTED COMPLETION: FY 2002; CONTACT: Debbie Rothstein, 301-594-2918; PERFORMER: Battelle Corporation, Arlington, VA

An Evaluation of Quality Measures and Timely and Cost Effective Strategies to Implement the Design Enhancements to the Medical Expenditure Panel Survey (MEPS)

The purpose of this project is to evaluate quality measures that should be added to the MEPS and to evaluate timely and cost effective strategies that implement design enhancements to the MEPS supportive of the Agency’s national quality measurement initiative.
PIC ID: 7401.2; EXPECTED COMPLETION: FY 2002; CONTACT: Thomas W. Reilly, 410-786-2196; PERFORMER: Barents Group, KPMG Peat Marwick LLP, Washington, DC

Development and Analysis of HCUP - The Healthcare Cost and Utilization Project

The purpose of this project is to support the Agency by designing, collecting, developing and analyzing a system of information on hospitals and their patients suitable for national studies.
PIC ID: 7693; EXPECTED COMPLETION: FY 2005; CONTACT: Jenny Schnaier, 301-594-6827; PERFORMER: Medstat Group, Washington, DC

Evaluation of the HCUP Central Distributor & HCUP Net

The purpose of this evaluation is to assess two dissemination mechanisms of products produced as part of the HCUP. The first dissemination mechanism is the HCUP Central Distributor, a mechanism established to disseminate the State Inpatient Databases (SID), one in a family of databases produced as part of HCUP. The second dissemination mechanism is HCUPnet, an interactive web-based system designed to generate tables of outcomes by diagnoses and procedures based on queries from users. The purpose of this task is to provide information about how well the HCUP Central Distributor is satisfying its original objective and to provide recommendations for improvements.
PIC ID: 7684; EXPECTED COMPLETION: FY 2002; CONTACT: Herbert Wong, 301-594-6822; PERFORMER: Center for Health Policy Studies, Columbia, MD

Maintain and Expand the Healthcare Cost and Utilization Project (HCUP)

The objectives for this contract are: 1) to obtain encounter-level data from statewide information sources; 2) to maintain and expand a uniform, multi-state health care database for health services research and health policy analysis, and 3) to make these data available to a broad set of public and private users. This contract will extend the capabilities of the HCUP national data system to assist researchers and decision-makers at the national, state, and community levels by expanding HCUP to include more states and more settings of care, and to develop tools to make better use of administrative data.
PIC ID: 7694; EXPECTED COMPLETION: FY 2005; CONTACT: Jenny Schnaier, 301-594-6827; PERFORMER: Medstat Group, Washington, DC

Validating the HCUP Patient Safety Indicators

This proposal extends work that is currently being conducted on the first two modules of the next generation of HCUP Quality Indicators (QI) developed by Stanford University. The development of reliable and valid QI for HCUP is a high priority and this proposal aids in this effort. The current validation proposal was funded to evaluate approximately 45 measures developed by Stanford. The proposed project will validate approximately 30 additional measures that are currently being developed by Stanford focusing on complications or adverse medical events.
PIC ID: 7672; EXPECTED COMPLETION: FY 2003; CONTACT: Anne Elixhauser, 301-594-6815; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

Validating the HCUP Quality Indicators

This project will seek to validate the HCUP Quality Indicators (QIs) against information available from data sources other than hospital administrative data. The accuracy and validity of the HCUP QIs will be explored by comparing results obtained from the QI software with information from other data sources such as medical records, outpatient administrative data, and laboratory data.
PIC ID: 7671; EXPECTED COMPLETION: FY 2002; CONTACT: Anne Elixhauser, 301-594-6815; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

Innovative Approaches to Training Clinicians for Bioterrorist Attacks

This project will develop innovative approaches to training clinicians for bioterrorist attacks, and creative methods for evaluating their effectiveness. Such approaches must be applicable on a broad scale and must be adapted to the information needs and current challenges confronting busy clinicians.
PIC ID: 7680; EXPECTED COMPLETION: FY 2002; CONTACT: Carolyn M. Clancy, 301-594-1485; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

Empirical Analysis of a Guideline-developed Clinical Performance Measure-the Example of Pediatric Gastroenteritis Development

Assessing and categorizing whether actual care conforms to clinical practice guidelines is an increasingly common approach to measuring appropriateness and quality of health services. Often, however, the process of care that is to be categorized simply as conforming or not may involve several different components or dimensions. This project will analyze existing data concerning one example of such a summary measure, conformance with American Academy of Pediatrics recommendations regarding gastroenteritis.
PIC ID: 7037.1; EXPECTED COMPLETION: FY 2002; CONTACT: Steve Fox, 301-597-1485; PERFORMER: MEDSTAT Group, Cambridge, MA

Criteria for Determining Disability in Speech-Language Disorders

This task order is designed to develop an evidence report. The project title was nominated by the Social Security Administration (SSA), which will serve as a partner to the PC for this task order. The plan is to use the evidence report to assist the SSA in updating its Listing of Impairments for Speech-Language Disorders, and revising its disability policy, as may be appropriate. The SSA Office of Disability will represent the SSA on this task order.
PIC ID: 7688; EXPECTED COMPLETION: FY 2002; CONTACT: Kevin Murray, 301-594-6128; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

Evaluating AHRQ’s Low-Income Research Portfolio, 1989-2000

This task order is designed to identify AHRQ’s contribution to the field of health services research for low-income populations; to evaluate the impact this contribution has had on health care policy and practice; and, ultimately, outcomes for low-income health services research.
PIC ID: 7687; EXPECTED COMPLETION: FY 2002; CONTACT: Robin Weinick, 301-594-2010; PERFORMER: Center for Health Policy Studies, Columbia, MD

IDS Solutions for Transferring Medication Data Across Patient Care Settings

This proposal is important because it presents a reasonable methodology to examine and improve methods of data transfer between treatment settings. This is a vital concern to clinicians because information about the medications a patient is taking and which medications he/she is sensitive (or allergic) to must be accurately transferred to insure patient safety.
PIC ID: 7682; EXPECTED COMPLETION: FY 2002; CONTACT: Kelly Morgan, 301-594-1782; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

Agency for Toxic Substances and Disease Registry

Mission

To prevent exposure and adverse human health effects and diminished quality of life associated with exposure to hazardous substances from waste sites, unplanned releases, and other sources of pollution present in the environment.

Evaluation Program

The Agency for Toxic Substances and Disease Registry (ATSDR) was created as a federal agency by the Comprehensive Environmental Response, Compensation, and Liability Act (CERCLA), more commonly known as Superfund. ATSDR was created to implement the health‑related sections of CERCLA and other laws that protect the public from hazardous waste and environmental spills of hazardous substances. The ATSDR evaluation program is coordinated with the agency wide strategic planning process to implement requirements of the Government Performance and Results Act (GPRA). ATSDR’s strategic goals and its annual performance plan are the result of an interactive process that reflects a long‑term commitment by Agency staff to develop stronger relationships among external clients and stakeholders, to assess products and services using relevant data, and to improve our processes and systems for more efficient accomplishment of ATSDR’s mission.

In-Progress Evaluations

Hazardous Substances Emergency Events Surveillance (HSEES)

The Hazardous Substances Emergency Events Surveillance (HSEES) Evaluation Report, generated annually, discusses the evaluations conducted for the surveillance of hazardous events and system reporting. The evaluation is based on the ability to measure the sensitivity and reliability of the system. Sensitivity is measured by the ability of the system to receive notification of events involving hazardous substances meeting the HSEES case definition, evaluated by comparison to other national data collection systems (i.e., DOT, EPA). Reliability is measured by the ability of the system to receive uniformly recorded data from the 13 state health department partners. To address the reliability of HSEES, each quarter the ATSDR mails a report describing a case study involving the release or threatened release of a hazardous substance or substances to participating state health departments. State health department partners review the case study and completes a hard copy data collection form which is returned to the ATSDR. The ATSDR then compares the responses on these forms to each other and to the ATSDR’s preferred responses.
PIC ID: 6854.2; EXPECTED COMPLETION: FY 2005; CONTACT: Maureen Orr, 404-639-6391; PERFORMER: Agency for Toxic Substances and Disease Registry, Atlanta, GA

Centers for Disease Control and Prevention

Mission

To promote health and quality of life by preventing and controlling disease, injury, and disability.

Evaluation Program

The Centers for Disease Control and Prevention (CDC) conducts evaluation studies designed to provide information essential for CDC’s strategies to achieve its mission:

• Conduct public health research, including epidemiology, laboratory, behavior, and social sciences.
• Develop and implement ongoing evaluation of scientific research.
• Assure that scientific information is communicated effectively to the general public.
• Develop and implement public health information systems for monitoring and promoting the health of the Nation.
• Deploy multidisciplinary teams to detect and investigate health threats.
• Develop and implement a system for establishing CDC scientific and programmatic priorities.
• Routinely evaluate the effectiveness and cost of CDC programs.
• Demonstrate the value of investment in health prevention.
• Collaborate with diverse partners.
• Design, implement, and evaluate prevention programs based on community needs.
• Prepare the public health leaders (present and future) through training in management and public health science.

CDC places high priority on evaluations to answer policy, program, and strategic planning questions related to the goals and objectives of Healthy People 2010. Performance improvement studies, such as those focusing upon the development of indicators consistent with the Government Performance and Results Act (GPRA), are of particular interest and import to the Agency. With the support of 1‑percent evaluation funds, the GPRA planning process began at CDC in FY 1995 and is continuing.

Completed Evaluations

A Process Evaluation of the Vaccines for Children Program: Final Methodology Report

Vaccines for Children (VFC) is a state-operated, federal entitlement program that supplies private and public providers with federally-purchased vaccine. Children eligible for the program include American Indian/Alaskan Natives, Medicaid enrolled, uninsured, or children who are insured but whose insurance does not cover vaccinations. The study was designed to: (1) determine the VFC eligibility of children seen for immunization only services in Health Department clinics in South Carolina; and (2) to determine the factors associated with seeking immunization in this setting. To address these study objectives, the contractors drew a stratified sample (by number of immunization-only services during a 4-month period) of 24 health department clinics. Each of two interviewers was assigned 12 clinics from which to collect data. The interviewers visited each of the clinics at least four times during the data collection period to conduct in-person interviews with the chief caregiver of children 12 years of age and under. Respondents were sampled within clinics by time of visit, with the interviewers varying visit times. A total of 706 interviews were completed during the study period. Extensive information was provided in the report on sampling procedures, data entry and storage, and data verification. The study included a 10 percent verification call back to respondents. Since this was a methodology report, no data from the study were provided in the report.
PIC ID: 7197.1; CONTACT: Nancy Cheal, 404-639-7095; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

Evaluation of Adult Immunization Activities

The purpose of this evaluation was to clarify how adult immunization program activities were conducted by grantees and how those activities were related to immunization coverage. This study addressed an important public health issue of adult immunization coverage, particularly immunization for influenza, pneumococcal disease, and hepatitis B. Since 1997, 64 states, territories, and selected cities have been able to use CDC 317 grant funds to support vaccine and vaccination programs for adults. The evaluation was based on reviews of secondary information collected from the 64 grantees, including grant continuation applications and reports, and semi-structured interviews with grantee state immunization program managers or designees. Qualitative and quantitative data were analyzed. Only data from the 1998 applications and reports was included, as they were thought to be more complete and representative. Major findings include: (1) less than half (45%) of states have a separate coalition for adult immunization. (2) 73% and 39% of states reported all public health departments provided adults with influenza and pneumococcal vaccination services, respectively. (3) the top six groups that were specifically targeted for adult immunization were: those with diabetes, those at risk of hepatitis B, residents of nursing homes, African Americans, recent immigrants, and Hispanics. (4) 60% of states reported not collecting influenza and pneumococcal vaccination coverage for persons over age 65.
PIC ID: 7498; CONTACT: Susan Cleveland, 404-639-7042; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

Community Characteristics and Indicators: The Role and Measurement of Community Characteristics in Community-Level HIV/AIDS Interventions

As policy directives have changed to encourage and facilitate community-level interventions for HIV/AIDS prevention, CDC has undertaken several research projects and related activities to establish the efficacy and effectiveness of community-level interventions. One such activity was the one-year Community Indicators Project (1999) intended to identify measures or indicators that assess community-level changes. Initial results from the Community Indicators Project suggested that before we can identify relevant community indicators that measure change in community characteristics and their impact on behavior change and sustainability of prevention efforts, the following tasks should be completed:

1) clarify and refine working definitions of key terms such as community, culture, social structure and sustainability; 2) provide more detail on theoretical assumptions and hypotheses that explain causal pathways between community characteristics and risk behavior and prevention efforts; 3) review empirical evidence in HIV/AIDS and other health-related areas on the associations between literature and other sources (e.g., intervention protocols) for existing measures of community-level change.
PIC ID: 6701; CONTACT: Esther Sumartojo, 404-639-8300; PERFORMER: Macro International, Inc., Atlanta, GA

Evaluation of Medicaid Reporting Using SLAITS

With evaluation funds from the Centers for Disease Control and Prevention, the National Center for Health Statistics was able to use the State and Local Area Integrated Telephone Survey (SLAITS) to test the quality of study participants’ ability to accurately report Medicaid enrollment in one state. The Minnesota Department of Human Services worked with NCHS to provide contact information on resident children currently enrolled in either Medicaid or MinnesotaCare. From the administrative records, parents of children enrolled in 1999 were randomly selected for interviews using the SLAITS questionnaire module on Child Well-Being and Welfare. In addition to the information on health care coverage, this project provided substantial information on program participation and indicators of child well-being. These indicators include education, child care, family functioning, household stability, neighborhood characteristics, social development, public assistance program participation, and health insurance. Of the 504 completed interviews, 82.3% of parents/guardians correctly reported their child’s current Medicaid coverage. While the data indicated that substantial under reporting of Medicaid exists, they do not indicate the bias would lead to over reporting insurance coverage. Results of this study have been published, and a public use data file is available.
PIC ID: 7046; CONTACT: Marcie Cynamon, 301-436-7085; PERFORMER: Robert Wood Johnson Foundation, Princeton, NJ

Evaluation of the National Health Interview Survey

The purpose of this project was to determine the feasibility of releasing preliminary estimates for a selected set of health measures from the NHIS in a timely fashion. The project had several phases. The first was to identify candidate health measures to be considered for early release. A review was conducted of existing lists of key health measures and out of this grew a list of 15 health measures to evaluate. Data were analyzed from 1997 and 1998 for these potential measures to determine the need for editing, seasonality of estimates, and minimum amount of data needed to make reliable estimates. The project also developed a preliminary weighting scheme and compared results with final weights. The results indicate estimates could be released for most measures using data from 1-2 quarters. The first set of measures were released on the NCHS web site in April 2001 based on data from January - June 2000 (http://www.cdc.gov/nchs/nhis.htm). NCHS can now provide key data to policy makers and others on a very timely basis.
PIC ID: 7040; CONTACT: Ann Hardy, 301-436-7085; PERFORMER: TRW Government Information Services, Fairfax, VA

Vancomycin-Resistant Enterococci Colonization in Patients At Seven Outpatient Hemodialysis

Vascular access infections are a major cause of morbidity and mortality to hemodialysis patients, and the use of antimicrobials to treat such infections contributes to the emergence and spread of antimicrobial-resistant bacteria. This study was conducted to determine the incidence of and risk factors for vascular access infections. Hemodialysis patients were studied at seven outpatient dialysis centers (four in Richmond, VA and three in Baltimore, MD). Vascular access infections were defined as local signs at the vascular access site or a positive blood culture with no known source other than the vascular access; and hospitalization or receipt of an IV antimicrobial. A total of 796 patients were followed for 4,134 patient-months. Patients had a 3.5% risk of infection each month. The data show that vascular access infections are common in hemodialysis patients but that infection rates differ substantially among different centers. Catheter use should be minimized to reduce these infections. Additionally, it is possible, according to the report, that improved serum albumin and urea reduction ratio could reduce vascular access infections and should be evaluated.
PIC ID: 6703.1; CONTACT: Jerome Tokars, 404-639-6418; PERFORMER: Centers for Medicare and Medicaid Services, Washington, DC

A Framework for State TBI Surveillance Programs

Traumatic Brain Injury (TBI) is a leading cause of death and disability in the United States, especially among children and young people. Despite the prevalence and severity of TBI events, accurate surveillance data about TBI has not been consistently collected and analyzed. During the past decade, CDC’s National Center for Injury Prevention and Control (NCIPC) has developed and expanded a state-based surveillance system for TBI to help address this knowledge gap. The purpose of this study was to assess a sample of six currently funded programs in order to understand the evolution of state TBI surveillance programs, identify successes and common pitfalls, and craft a framework or logic model depicting the core activities of these types of programs. The study methods included: a review of background materials; a literature review; interviews with stakeholders; and site visits. This report represents key findings from the study’s data collection activities in three areas: (1) Measures of success - the responses included building the capacity for basic epidemiology, contributing to broader infrastructure goals, advancing the science base, affecting policy, and developing data systems. (2) Program Strengths - CDC’s state-based TBI surveillance programs have produced quality data on TBI, increased TBI’s visibility, influenced policy, shaped local programs, and advanced the science base. (3) CDC and State Activities - CDC support state-based TBI surveillance programs in its information sharing role, as a data disseminator, as a provider of technical assistance, particularly in the area of training, and, is a major funder for state TBI programs.
PIC ID: 7508; CONTACT: Doug Browne, 404-488-7098; PERFORMER: Management Assistance Corporation, Dulles, VA

In-Progress Evaluations

Evaluation of State-Based Data Dissemination Activities Via the Internet

Phase I of the project will evaluate current efforts by states to develop internet-based dissemination tools for integrated community-level data and develop criteria for usability of these data for community-based assessment activities. During subsequent phases, the study will evaluate the usefulness of a selected dissemination tool in structured focus groups and develop implementation strategies for states with different capacity or resources.
PIC ID: 7751; EXPECTED COMPLETION: FY 2002; CONTACT: Patricia Schumacher, 770-488-8375; PERFORMER: Macro International, Inc., Calverton, MD

Evaluation of a New Methodology for Community Participatory Research

This study will assess the participatory research methods of the CDC Urban Research Centers, located in Detroit, New York City and Seattle. These projects are in various phases of using participatory action research methods to determine “what works” to improve the quality of life of inner city, impoverished populations.
PIC ID: 7746; EXPECTED COMPLETION: FY 2002; CONTACT: Donna Higgins, 202-205-0932; PERFORMER: Macro International, Inc., Calverton, MD

Evaluation of the Public Health Prevention Service

In 1997 the Epidemiology Program Office established the Public Health Prevention Service (PHPS), a new three-year training program designed to improve the Nation’s public health practice by preparing entry-level public health professionals to conduct prevention programs. This evaluation project will establish the foundation for a monitoring and evaluation system designed to provide feedback to the PHPS program for continuous improvement.
PIC ID: 7038; EXPECTED COMPLETION: FY 2002; CONTACT: Dennis Jarvis, 404-639-4087; PERFORMER: Macro International, Inc., Atlanta, GA

Development of a Manual for Asthma Wellness in Schools

The purpose of this activity is to develop a manual for asthma wellness in schools that will 1) outline best practices and policies for an asthma wellness and education program in the context of a school health program, which includes education, physical education, health services, nutrition services, counseling, psychological and social services, health school environment, health promotion for staff, and family and community involvement. In addition, an appendix section of case studies will contain exemplary school-based asthma intervention programs, existing instruments used to assess and improve a school’s asthma related policies and activities, a reference section and a resource list of expert panel members and their institutional and organizational affiliation.
PIC ID: 7700; EXPECTED COMPLETION: FY 2002; CONTACT: Howell Wechsler, 770-488-6197; PERFORMER: Macro International, Inc., Calverton, MD

Evaluating National Dissemination Strategies for Effective HIV/Tobacco Prevention Programs for Youth

This project is an evaluation of dissemination strategies to increase the adoption and implementation of HIV prevention programs for youth in the United States. The goal of the evaluation is to improve national, state and local training programs that promote the adoption of effective health curricula identified by the ongoing CDC Research to Classroom Project (RTC).
PIC ID: 7750; EXPECTED COMPLETION: FY 2002; CONTACT: Steven Banspach, 770-488-3257; PERFORMER: Battelle Corporation, Arlington, VA

Evaluation of CDC Cardiovascular State Health

The purpose of this task order is to evaluate the Cardiovascular Disease (CVD) State Program. This will involve providing evaluation assistance to CDC in developing an evaluation framework for the CVH State Program; assessing state evaluation capacity and training needs; and providing technical assistance to states on implementation of the CDC evaluation framework.
PIC ID: 7701; EXPECTED COMPLETION: FY 2002; CONTACT: Dyann Matson Koffman, 770-488-8002; PERFORMER: Macro International, Inc., Calverton, MD

Evaluation of Cardiovascular Health National State-Based Program

The purpose of this project is to assist in evaluation activities related to the Cardiovascular Health (CVH) State Program. Specifically, the work will involve evaluation assistance in (1) providing technical assistance to CDC on developing an evaluation framework for the CVH State Program; (2) providing general technical assistance in the area of the evaluation that is consistent with the CDC Evaluation Framework; (3) providing technical assistance to states in understanding the CDC evaluation framework.
PIC ID: 7703; EXPECTED COMPLETION: FY 2002; CONTACT: Sheree Williams, 770-488-8007; PERFORMER: Macro International, Inc., Calverton, MD

Evaluation of Communication Activities of the National Diabetes Education Program (NDEA) National Network

The purpose of this evaluation is to contribute process measures to an evaluation of outcomes planned by the National Diabetes Education Program (NDEP). A “program operations” evaluation of the NDEP is proposed to ascertain the effectiveness of partnership network communication activities under the special population activities of NDEP, in the following areas: 1) design and development of culturally appropriate messages and products, 2) delivering the intervention, 3) reaching the intended audiences, and, 4) producing intended changes in awareness, attitudes and reported behavior. The research will be managed jointly by the Division of Diabetes Translation (NCCDPHP) and the Division of Health Communication, Office of Communications (OD).
PIC ID: 7430; EXPECTED COMPLETION: FY 2003; CONTACT: Phyllis McGuire, 770-488-5025; PERFORMER: Macro International, Inc., Calverton, MD

Evaluation of Public Health Care Providers Training, Screening and Referral Practices for Pregnancy-Related Violence

This project will conduct national population-based surveys of administrators and clinical providers in family planning programs funded by Titles X and XX, maternal and child health programs funded by Title V, and the federally-funded Primary and Migrant Health Centers. The information obtained from these surveys will be used to develop recommendations for the development of future clinical guidelines.
PIC ID: 6712; EXPECTED COMPLETION: FY 2002; CONTACT: Alison Spitz, 770-488-5260; PERFORMER: Battelle Corporation, Arlington, VA

Evaluation of the Coordinated Community Response

The purpose of this project is to assist CDC with management and oversight of intimate partner violence (IPV) inter-project communication and analysis of program data within the Coordinated Community Response (CCR) Projects.
PIC ID: 7705; EXPECTED COMPLETION: FY 2002; CONTACT: Annie Howerton, 770-488-4266; PERFORMER: Macro International, Inc., Calverton, MD

Review and Synthesis of Measures of Youth Physical Fitness and Physical Activity

The purpose of this project is to collect relevant information regarding the design, development and conduct of surveys of physical fitness and physical activity among children and adolescents, and to synthesize and summarize this information into a final report that can be used to design and develop a nationally-representative youth physical fitness and physical activity surveillance system.
PIC ID: 7702; EXPECTED COMPLETION: FY 2002; CONTACT: Janet Fulton, 404-488-5430; PERFORMER: Macro International, Inc., Calverton, MD

Evaluation of the National Folic Acid Campaign

The purpose of this study is to ascertain the effectiveness of current methods to (1) raise awareness about the benefits of folic acid for the prevention of neural tube defects (NTDs) and (2) to increase consumption of folic acid to levels sufficient to prevent the majority of NTDs.
PIC ID: 7748; EXPECTED COMPLETION: FY 2002; CONTACT: Katherine Lyon-Daniel, 770-488-7182; PERFORMER: Westat, Incorporated, Atlanta, GA

Folic Acid and the Prevention of Spina Bifida

This project proposes to evaluate the effectiveness of a free folic acid supplement distribution program in a family planning setting using blood folate determinates in conjunction with questionnaires of knowledge and supplement use behaviors.
PIC ID: 7042; EXPECTED COMPLETION: FY 2002; CONTACT: Margaret Watkins, 770-488-7187; PERFORMER: Battelle Corporation, Arlington, VA

Assessing the Evaluation Capacity of Health Departments

This study will assess and describe the current evaluation capacity (ability of state and local health departments to conduct evaluation activities outlined in the CDC Evaluation Guidance and Data System) for 65 HIV prevention fund grantees, to develop performance measures that will enable tracking of changes in evaluation capacity over time and for information evaluation training and technical assistance activities.
PIC ID: 7747; EXPECTED COMPLETION: FY 2002; CONTACT: Gary Uhl, 404-639-0950; PERFORMER: Macro International, Inc., Calverton, MD

Development of a National HIV/AIDS Taxonomy Prevention Intervention for Program Evaluation

This project will: (1) comprehensively and systematically identify and inventory taxonomy of interventions nationally that can be used for evaluation effectiveness purposes; (2) develop a systematic taxonomy of HIV/AIDS interventions that can be integrated into and used by local, state and national groups and organizations for program development, improvement and evaluation; and (3) adopt and develop an alphanumerically coded taxonomy for the HIV/AIDS interventions identified that complies with national and international metadata standards and methodologies through creation of intervention data dictionaries, thesauri, and a unique coding scheme for each of the various types of interventions, as well as a standard protocol to be followed when identifying or incorporating a particular type of HIV/AIDS intervention into an HIV/AIDS program.
PIC ID: 7707; EXPECTED COMPLETION: FY 2002; CONTACT: Timothy Akers, 404-639-0926; PERFORMER: Macro International, Inc., Calverton, MD

Evaluation of CDC Recommendations for the Prevention and Control of Tuberculosis in Jails

The purpose of this evaluation is to determine to what extent the CDC guidelines have been implemented in jails to identify barriers for implementation of the recommendations.
PIC ID: 7752; EXPECTED COMPLETION: FY 2002; CONTACT: Mark Lobato, 404-639-8131; PERFORMER: Abt Associates Inc., Cambridge, MA

Geo-Analysis of HIV Prevention

This project will build on the knowledge gained from the previous studies to provide a multi-level, geo-references review of CDC-funded, Community Based Organization (CBO) -provided HIV prevention services. The purposes of the project are: (1) to contribute to the construction of a national database of HIV prevention activities by developing a geo-coded database that identifies, locates and maps all CBOs directly and indirectly funded by CDC in the US and its territories, and (2) to evaluate the comprehensiveness of HIV Prevention services in seven geographic areas by piloting, in one state or Metropolitan Statistical Area (MSA), the use of GIS technology as the primary analytical tool. A telephone survey will be conducted.
PIC ID: 7699; EXPECTED COMPLETION: FY 2002; CONTACT: Aisha Gilliam, 404-639-0919; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

Health Care Adherence: TB Skin Testing

This study is being conducted to determine why health care workers comply or do not comply with recommended tuberculin skin testing and treatment for lateen tuberculosis infection policies and guidelines. It is expected that this study will provide a knowledge base for which products can be developed to improve adherence.
PIC ID: 7706; EXPECTED COMPLETION: FY 2002; CONTACT: Maureen Wilce, 404-639-8123; PERFORMER: Macro International, Inc., Calverton, MD

Know Now Campaign

This study will be used to evaluate the media components of the Serostatus Approach for Fighting the Epidemic (SAFE) project, to increase the number of people who know their HIV status and to promote the entry of those infected with HIV into a system of care. The proposed evaluation will contribute to the general understanding of how well targeting through marketing clusters can be implemented for prevention marketing campaigns and may lead to ways to improve CDC prevention marketing for the SAFE project and behavior-related diseases, such as HIV, in general.
PIC ID: 7433; EXPECTED COMPLETION: FY 2002; CONTACT: David Davis, 404-639-0938; PERFORMER: Aeffect, Inc., Chicago, IL

Survey of Syphilis and HIV Treatment, Reporting

The purpose of this project is to conduct a survey of a sample of physicians’ diagnosis, treatment, reporting and partner management practices for syphilis and HIV infection. The study will determine when, under what conditions, for which diseases, how and for which patients, practitioners notify and/or treat the sexual partners of patients who are diagnosed with syphilis and HIV infection.
PIC ID: 7054; EXPECTED COMPLETION: FY 2002; CONTACT: Janet St. Lawrence, 404-639-8376; PERFORMER: Battelle Corporation, Arlington, VA

Evaluation of Expanding Pharmaceutical Data- NHIS

The purpose of the project is to evaluate the collection of pharmaceutical data similar to data collected in the National Hospital Ambulatory Care Survey (NHMACS) for discharges sampled in the National Hospital Discharge Survey (NHDS). Activities will include evaluation of methods currently employed in NHAMCS; evaluation of potential sources and methods to collect pharmaceutical data for inpatients as part of the NHDS if NHAMCS methodology is not transportable; evaluate and describe similarities and differences between drug data collected in hospitals and in the NHAMCS; develop a plan for field testing two methods to collect pharmaceutical data as part of the NHDS; develop or modify NHDS forms, procedures and manuals for a field test to collect these pharmaceutical data and, develop training materials for the field test.
PIC ID: 7745; EXPECTED COMPLETION: FY 2002; CONTACT: Robert Pokras, 301-458-4439; PERFORMER: CODA, Inc.

Definition of Public Health Lab Core Capabilities

The purpose of this project is to begin the process of developing a consensus definition of core capacities for infectious disease surveillance starting with public health laboratories. A contractor will be hired to develop and pilot test a methodology for assessing necessary public health laboratory capacities and capabilities for select infectious diseases. For the purposes of this proposal, capacity refers to the output of testing or services accomplished over a defined time period; capability refers to a specific activity that will ensure the laboratory’s success in conducting infectious disease surveillance.
PIC ID: 7742; EXPECTED COMPLETION: FY 2002; CONTACT: Deborah Deppe, 404-639-4668; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

Evaluation of PICARD Software

The goal of this evaluation is to determine: usefulness and user friendliness; feasibility of using Preventable Injuries Costs and Related Deaths (PICARD) to generate state data on preventable injury and associated costs by states; if the use of PICARD contributes to the formation of new bicycle helmet promotion programs, policies and laws; and if NCIPC should develop additional PICARD modules for other priority injury prevention topics.
PIC ID: 7743; EXPECTED COMPLETION: FY 2002; CONTACT: Sarah Olson, 770-488-1302; PERFORMER: Battelle Corporation, Arlington, VA

Links Among Different Forms of Adolescent Violence

The purpose of this project is to prepare a literature review, protocol and data collection instruments for a subsequent survey on sexual, physical and non-physical aggression among same-sex and opposite-sex adolescent peers. This project will provide a description of a) prevalence, correlates and consequences of different types of aggressive behaviors used by adolescent males and females; b) the association between dating violence and other forms of peer violence; and c) how these guide efforts to develop prevention strategies that address peer and dating violence.
PIC ID: 7698; EXPECTED COMPLETION: FY 2002; CONTACT: Joyce McCurdy, 770-488-4266; PERFORMER: Battelle Corporation, Arlington, VA

Role of Power and Control in Intimate Partner Violence

The purpose of this project is to conduct an exhaustive literature review of theoretical and empirical work addressing the role of power and control in intimate partner violence (IPV) perpetration. IPV includes physical, sexual and psychological violence. The project seeks to identify how power and control is conceptualized, operationalized and measured. The project will also identify battering measurement instruments and batterer intervention programs, develop a relationship between power and control and their contribution to the perpetration of IPV. Findings from this study will guide prevention and intervention activities that address IPV perpetration. A telephone survey will be conducted.
PIC ID: 7697; EXPECTED COMPLETION: FY 2002; CONTACT: Annie Howerton, 770-488-4266; PERFORMER: Battelle Corporation, Arlington, VA

Second Injury Control and Risk Survey (ICARIS-2)

This project is a follow-up study to the first ICARIS (Injury Control and Risk Survey) conducted in 1994. By using data collected in ICARIS as a baseline, ICARIS-2 can measure changes and gauge the impact of injury prevention policies. It will also serve as the only readily available source of data to measure several of the Healthy People 2010 injury prevention objectives.
PIC ID: 7695; EXPECTED COMPLETION: FY 2002; CONTACT: John Horan, 770-488-1653; PERFORMER: Battelle Corporation, Arlington, VA

Evaluation of Provider Distribution of Flu Vaccine

The purpose of this project is to evaluate whether providers adhered to public health service recommendations regarding modification of vaccination practices and to assess the use of vaccine ordered from the nine million doses. More generally, the project will assess influenza vaccination practices of public and private health care providers in the U.S. in order to develop strategies to improve vaccination coverage of persons who are at risk for influenza-related medical complications and their contacts.
PIC ID: 7704; EXPECTED COMPLETION: FY 2002; CONTACT: Marika Iwane, 404-639-8769; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

Evaluation of Standing Orders in Nursing Homes

The purpose of this evaluation is to identify the costs of implementing standing orders programs and their components compared to other organized immunization programs in long term care facilities (LTC) and, subsequently, to determine the cost-effectiveness of such programs and their components. This evaluation will require two phases: Phase 1 will include identification of program costs and of effects of program on staff and resident flow at the LTC and, Phase 2 will include determination of cost effectiveness of the programs related to improvement in vaccination coverage rates in the LTC.
PIC ID: 7434; EXPECTED COMPLETION: FY 2002; CONTACT: Abigail Shefer, 404-639-8333; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

Evaluation of the Use of Data from Immunization Information Systems

Currently, at least 22 states have immunization information systems (IIS) functioning either state-wide or in some counties or local communities. At this point in the development cycle, it is important to evaluate how the data in the IIS are used. If it is understood how these data are used and structural and functional barriers to data use can be identified, recommendations can be made to promote the full use of these data resources.
PIC ID: 6713; EXPECTED COMPLETION: FY 2002; CONTACT: John Stevenson, 404-639-8730; PERFORMER: Advanced Technology Systems, Johnson City, TN

Centers for Medicare and Medicaid Services

Mission

We assure health security for beneficiaries.

Evaluation Program

The research arm of the Centers for Medicare and Medicaid Services (CMS), the Office of Research, Development and Information (ORDI), performs and supports research and evaluations of demonstrations (through intramural studies, contracts and grants) to develop and implement new health care financing policies and to provide information on the impact of CMS’ programs. The scope of ORDI’s activities embraces all areas of health care: costs, access, quality, service delivery models, and financing approaches. ORDI’s research responsibilities include evaluations of the ongoing Medicare and Medicaid programs and of demonstration projects testing new health care financing and delivery approaches.

Examples of research themes include state program flexibility, the future of Medicare, provider payment and delivery, and vulnerable populations and dual eligibles.

Completed Evaluations

The Impact of Home Health Prospective Payment on Medicare Service Use and Reimbursement

As part of its ongoing effort to study methods of providing more cost-effective care, the Centers for Medicare and Medicaid Services (CMS) implemented the Per-Episode Home Health Prospective Payment Demonstration. Ninety-one agencies in five states entered the three-year demonstration at the start of their 1996 fiscal years. Before the start of the demonstration, the participating agencies were randomly assigned to either the treatment or the control group. Agencies assigned to the treatment group were reimbursed under the demonstration’s prospective payment method, while those assigned to the control group continued to be reimbursed under cost-based reimbursement (the payment method Medicare used for all home health agencies when the demonstration began). This report examines data from the first two years of the demonstration to test hypotheses about the possible effects of prospective payment on the use of Medicare-covered services by agency patients and on reimbursement for those services. It was found that the prospective payment led to a 25 percent reduction in home health visits and episode length over the year following admission to a demonstration home health agency. However, these reductions did not appear to lead to an overall increase in the use of other Medicare services during that year. Other findings were: Emergency room use appeared to decline slightly; Use of skilled nursing facility and hospice services was not affected; Use of nondemonstration home health services increased somewhat but did not reflect adverse patient outcomes; and, Use of Part B services and overall reimbursement were not affected. Total Medicare spending for treatment agency patients was somewhat lower than spending for control agency patients.
PIC ID: 7738.2; CONTACT: Ann Meadow, 410-786-6602; PERFORMER: Mathematica Policy Research, Inc., Plainsboro, NJ

Economic and Cost-Effectiveness Studies from the U.S. Renal Disease Data

This interagency agreement (IAA) provided funds to the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) to cover the costs of having the coordinating center for the U.S. Renal Data System (USRDS) perform economic and cost-effectiveness studies. The NIDDK contracted with the University of Michigan to be the coordinating center for 5 years. Each year the coordinating center conducts cost or cost-effectiveness components for at least four existing data studies and for one special study focused on economic issues. This study produced economic chapters in the USRDS Annual Data Report and the economic components of a number of scientific publications in medical journals. The raw results are included in the many tables available at the USRDS web site: http://www.usrds.org/. Among the tables presented are: the incidence/prevalence of End-Stage Renal Disease (ESRD), patient characteristics at the start of ESRD, transplantation, preventive healthcare measures, provider characteristics, economic costs of ESRD, and international comparisons.
PIC ID: 7198; CONTACT: Joel Greer, 410-786-6695; PERFORMER: National Institute of Diabetes and Digestive and Kidney Diseases, Bethesda, MD

Educating New Members of Medicare+Choice Plans About Their Health Insurance Options: Does the National Medicare Education Program Make a Difference?

The National Medicare Education Program (NMEP) addresses one of the biggest challenges facing Medicare--educating beneficiaries about their insurance options. Data from a national survey of Medicare HMO enrollees and fee-for-service beneficiaries age 65 and over indicate that most of these beneficiaries are aware of at least one NMEP information source, such as the Medicare &You handbook. Recent Medicare HMO enrollees are more likely than fee-for-service beneficiaries to have searched for information about Medicare. About 44 percent of recent Medicare HMO enrollees recall using a NMEP source. Most beneficiaries who use NMEP sources find them helpful. About 40 percent of recent Medicare HMO enrollees and 67 percent of fee-for-service beneficiaries still do not understand key aspects of Medicare.
PIC ID: 7168.2; CONTACT: Peri Iz, 410-786-6589; PERFORMER: Mathematica Policy Research, Inc., Plainsboro, NJ

Evaluation of Oregon Medicaid Reform Demonstration

The Oregon Medicaid Reform Demonstration seeks to increase the number of individuals with access to affordable health care services and to contain State and Federal expenditures for health care. Under the demonstration, Medicaid coverage is made available to all State residents with family incomes less than, or equal to, the Federal poverty level (FPL) and who meet an assets test. This report presents selected analyses conducted as part of a CMS-funded evaluation of the Oregon Health Plan (OHP). While the report focuses largely on OHP's Phase 1 population (traditional Aid to Dependent Children (ADC) expansion eligibles), future reports also will examine the Phase 2 population: SSI disabled and dual Medicare eligibles. Higher than expected costs have meant that OHP has had to seek new ways to finance care. OHP garnered national attention for its use of a prioritized list of health care services to define the program's benefit package. However, the priority list has not served well as a tool for budgetary control. Restricting the list has been cumbersome because of approval requirements at the State and Federal levels. Like some other states, Oregon has turned to tobacco tax revenues to support its expanded Medicaid program. This will allow for the expansion of the program to cover pregnant women up to 170 percent of FPL; re-extend coverage to uninsured Pell Grant college students who had lost coverage as the result of an early OHP budget shortfall and create the Family Health Insurance Assistance Program (FHIAP) to subsidize private insurance premiums for low-income adults and children. The newest source of revenue for OHP will come from the State Children's Insurance Program (SCHIP).
PIC ID: 7705.1; CONTACT: Paul J. Boben, 410-786-6629; PERFORMER: Health Economics Research, Inc., Waltham, MA

Focus Group Results from the National Evaluation of Medicare & You 2000 Handbook: Non-Beneficiary Decision Helpers

In focus groups with friends and family members who help Medicare beneficiaries with their coverage decisions, this study found that the friends and family members who assist Medicare beneficiaries with their coverage decisions have some informational needs that overlap with beneficiaries’, while others are unique to their status. They understood the intent of the Handbook, and found it useful as a reference tool and as a learning resource. Helpers expressed more confusion and frustration than did beneficiaries in similar focus groups, particularly in attempting to collect consistent information dealing with what Medicare covers and to what extent. This disparity may be because they are less familiar with the complex Medicare system. The report suggests CMS should give consideration to whether the National Medical Education Program (NMEP) is intended to apply to non-beneficiaries who aid in decision-making as well as to beneficiaries. Informational material and the methods by which that material is disseminated may require some tailoring to the special needs of decision helpers who are not themselves beneficiaries.
PIC ID: 7363.2; CONTACT: Sherry Terrell, 410-786-6601; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

Focus Group Results from the National Evaluation of Medicare & You 2000: Beneficiaries

This report summarizes the findings from three focus groups with Medicare beneficiaries. The overall aim of this work was to contribute to efforts to evaluate the National Medicare Education Program (NMEP) by augmenting information collected in the national Medicare and You evaluation survey that RTI conducted between July 1999 and February 2000. Beneficiaries generally perceived that the purpose of the handbook was to provide general knowledge and information about the Medicare program. The handbook also increased participants’ awareness of CMS-sponsored information sources, although many were reluctant to use the Internet. Gains in participants’ understanding appeared to be associated with the perceived relevance of the issues addressed. These findings suggest that it will be challenging to develop effective methods to assess the impact of the NMEP on beneficiary knowledge, as interest is an important intervening variable between exposure and gains in knowledge. Most participants saw the handbook as a reference guide.
PIC ID: 7363.1; CONTACT: Sherry A. Terrell, 410-786-6601; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

In-Progress Evaluations

Evaluation of Group-Specific Volume Performance Standards Demonstration

The Physician Group Practice (PGP) demonstration tests a hybrid payment methodology that combines Medicare-fee-for-service payments with a bonus pool derived from savings achieved through improvements in practice efficiency and patient processes and outcomes by physician groups and affiliated organizations. The goals of the demonstration are to: (1) encourage coordination of Part A and Part B services, (2) promote efficiency via investment in administrative structure and care processes, and (3) reward physicians for improving health outcomes. The Benefits Improvement and Protection Act of 2000 mandated the PGP demonstration.
PIC ID: 7181; EXPECTED COMPLETION: FY 2002; CONTACT: John Pilotte, 410-786-6558; PERFORMER: Health Economics Research, Inc., Waltham, MA

Performance Assessment of Web Sites

This task order evaluates, sets up an ongoing system for feedback from consumers, and makes recommendations for future changes concerning two web sites sponsored by the Department of Health and Human Services. The project covers web sites: http://www.medicare.gov, which was developed by the Centers for Medicare and Medicaid Services (CMS), and www.healthfinder.gov, which was developed by the Office of Disease Prevention Health Promotion in collaboration with other agencies. Each focuses on different aspects of patient information rather than seeking to provide organizational information about the Department.
PIC ID: 7212; EXPECTED COMPLETION: FY 2002; CONTACT: Barbara Crawley, 410-786-6590; PERFORMER: Barents Group, KPMG Peat Marwick LLP, Washington, DC

Evaluating Alternative Methods to Assure and Enhance the Quality of Long-Term Care Services for Persons with Developmental Disabilities

This task order develops and validates a comprehensive set of performance measures and indicators of quality for institutional post-acute and long-term care settings. The post-acute settings involved are: SNF short-stay units, inpatient rehabilitation facilities (which include hospital-based rehabilitation units) and long-term care hospitals.
PIC ID: 6310; EXPECTED COMPLETION: FY 2002; CONTACT: David Greenberg, 410-786-2637; PERFORMER: Abt Associates Inc., Cambridge, MA

Evaluation of High Risk Pools

As a method of assuring availability of insurance in the individual market, the Health Insurance Portability and Accountability Act (HIPAA) allows the use of an acceptable state alternative mechanism in place of adopting precise HIPAA provisions. One of these acceptable mechanisms is to use a state’s High-Risk Pool for HIPAA eligibles. Since the statutory objective of this acceptable mechanism is to guarantee the availability of insurance to individuals, this project evaluates the best standards for a High Risk Pool, standards which allow it to be sustained and to remain an acceptable alternative mechanism for HIPAA eligibles. It summarizes and details the similarities and differences between risk pools in the context of the dynamics in individual insurance law for states which have accepted risk pools as alternative mechanisms.
PIC ID: 7422; EXPECTED COMPLETION: FY 2002; CONTACT: James Fuller, 410-786-3365; PERFORMER: Abt Associates Inc., Cambridge, MA

Assessment of Medicare & You Education Program

As part of the National Medicare Education Program, CMS must provide information to beneficiaries about the Medicare program and their Medicare+Choice (M+C) options. Performance assessment plays a critical part in CMS’s efforts to provide this information. This project provides assistance to CMS in assessing how well CMS is communicating with Medicare beneficiaries, caregivers and partners. The specific activities include: toll-free telephone services, including Mystery Shopping Monitoring and Assessment Activities; State Health Insurance Assistance Programs (SHIPs); Regional Education about Choices in Health (REACH), including National Training and Support for Information Givers, and Partnering Assessment; Print Materials including The Medicare & You Handbook.
PIC ID: 7666; EXPECTED COMPLETION: FY 2003; CONTACT: Lori Teichman, 410-786-6684; PERFORMER: Barents Group, KPMG Peat Marwick LLP, Washington, DC

Cost-Benefit of HIPPA

This is a multi-phase study that focuses on groups that have been in the forefront of interacting with the population affected by the Health Insurance Portability and Accountability Act of 1996 (HIPAA) and three HIPAA related provisions: MHPA (Mental Health Parity Act of 1996), NMPHA (Newborns and Mothers’ Health Protection Act of 1996) and WHCRA (Women’s Health and Cancer Rights Act of 1998). The groups, i.e., state agencies, consumer advocacy groups or individual researchers, are being contacted, and benefits, in terms of assisting individuals in obtaining coverage as guaranteed under HIPAA and related provisions, identified. Since strategy for implementation of HIPAA is technically based on state insurance regulatory models, the project will delineate and describe similar and/or differing effects as this model is applied at the federal level. Evaluation results to assist CMS in the planning of any future endeavors in private health insurance regulation.
PIC ID: 7420; EXPECTED COMPLETION: FY 2002; CONTACT: James Fuller, 410-786-3365; PERFORMER: Arthur Andersen and Company, Washington, DC

Evaluation of BBA Impacts on Medicare Delivery and Utilization of Inpatient and Outpatient Rehabilitation Therapy Services

This project studies the impact of the Balanced Budget Act of 1997 (BBA) on the delivery and utilization of inpatient and outpatient rehabilitation therapy services to Medicare beneficiaries. It is a continuation and extension of previous work under “Medicare Post-Acute Care: Evaluation of BBA Payment Policies and Related Changes” which covered the period 1996-1999. This project studies the period 2000-2003 and also tracks and analyzes (1) Medicare beneficiaries’ utilization patterns of rehabilitation therapies and other post-acute care from 2001-2003 and (2) the allocation of resources among post-acute care providers over the same period. These analyses will provide a general framework for understanding shifts in access and utilization of care among post-acute care settings. It will also explore in detail the changes specific to rehabilitation therapy services.
PIC ID: 7668; EXPECTED COMPLETION: FY 2005; CONTACT: Philip G. Cotterill, 410-786-6598; PERFORMER: Health Economics Research, Inc., Waltham, MA

Evaluation of Private Fee-for-Service Plans in the Medicare+Choice Program

The purpose of this project is to evaluate the new private fee-for-service (PFFS) option available under the Medicare+Choice (M+C) program. The evaluation uses a combination of primary and secondary data sources to evaluate the effects of the option on beneficiaries and program costs. The private fee-for-service plan option is one of the new types of organizations provided for under the M+C provisions. Primary data will be collected through site visits to participating plans and beneficiary surveys. The Sterling Plan is the first insurer approved to offer this option--it has been available to beneficiaries since July 2000 and by the end of January 2001, there were 10,098 beneficiaries enrolled. Analytic issues to be addressed in the evaluation can be grouped into three broad categories: impacts on beneficiaries, impacts on Medicare program expenditures, and impacts on participating plans and providers. The evaluation will report on the views of the Sterling Plan regarding issues of marketing and administering the PFFS policies, the reasons for the participation in the M+C program, how markets were selected to enter, and other pertinent issues relating to their participation in the M+C program. The evaluation will also report on provider impacts.
PIC ID: 7664; EXPECTED COMPLETION: FY 2004; CONTACT: Nancy Zhang, 410-786-9362; PERFORMER: Abt Associates Inc., Cambridge, MA

Evaluation of Programs of Coordinated Care and Disease Management

This project is an the evaluation of a group of Congressionally mandated demonstration programs and two CMS-initiated demonstration programs which test various methods of managing care in the fee-for-service Medicare environment. The demonstration programs to be studied as part of this evaluation will vary widely with respect to the demographics, medical and social situations of the target population, intensity of services offered, interventions under study, type(s) of health care professionals delivering the interventions, and other factors. Sites may be added to the demonstration as it progresses.
PIC ID: 7669; EXPECTED COMPLETION: FY 2005; CONTACT: Barbara Silverman, 410-786-8263; PERFORMER: Mathematica Policy Research, Inc., Plainsboro, NJ

Evaluation of the Impact on Beneficiaries of the Medicare+Choice Lock-in Provision

This project explores the impact on Medicare beneficiaries of the lock-in provision of the Balanced Budget Act of 1997 (BBA). Lock-in places limits on the frequency, timing and circumstances under which Medicare+Choice (M+C) enrollment elections can be made. These changes are being phased in over a two year period beginning January 1, 2002. The purpose of this project is to: Examine the pre-lock-in patterns of enrollment and disenrollment in M+C using existing CMS administrative data. Particular emphasis shall be on the types of actions (i.e. beneficiary enrollment choices) that will be impacted by the lock-in provision. This analysis will use existing administrative systems and data to provide the beneficiaries likely to be affected and characterize the resulting impact to beneficiaries.
PIC ID: 7665; EXPECTED COMPLETION: FY 2004; CONTACT: Mary Kapp, 410-786-0360; PERFORMER: Barents Group, KPMG Peat Marwick LLP, Washington, DC

Evaluation of the Program of All-Inclusive Care for the Elderly (PACE) as a Permanent Program and of a For-Profit Demonstration

PACE is an innovative model that seeks positive outcomes and cost savings by providing a range of integrated preventative, acute care, and long-term care services to manage the often complex medical, functional, and social needs of the frail elderly. The Balanced Budget Act (BBA) states this current study must cover the quality and cost of providing PACE program services under Medicare and Medicaid when it is operated as a permanent program. It also must compare the cost, quality, and access to services [provided] by entities that are private, for-profit entities operating under demonstration project waivers...with the costs, quality, and access to services of other PACE providers. This project is to expand on the foundations laid in the previous evaluations of PACE by predicting costs beyond the first year of enrollment, assessing the impact of higher end of life costs and long term nursing home care, and assessing the impact of local treatment practices.
PIC ID: 7667; EXPECTED COMPLETION: FY 2002; CONTACT: Fred Thomas, 410-786-6675; PERFORMER: Mathematica Policy Research, Inc., Plainsboro, NJ

Study of Pharmaceutical Benefit Management

This project is an extension of an earlier CMS ORDI research, completed in 1996. This earlier study remains valuable for its description of the industry functions and the origins. However, most information is no longer current as this industry has undergone major stages of evolution during the past five years. While the industry has grown impressively in size, there has been an increasing concentration in market power. The pharmacy benefit management (PBM) industry is becoming a dominant player in the administration of pharmaceutical benefits. It seems certain that the PBM sector will play a significant role in administering the Medicare program in case a drug benefit is added to Medicare. This study systematically examines the growing PBM industry.
PIC ID: 7591.2; EXPECTED COMPLETION: FY 2002; CONTACT: Peri Iz, 410-786-6589; PERFORMER: Price Waterhouse, Washington, DC

Evaluating the Use of Quality Indicators in the Long Term Care Survey Process

CMS’s goal is to move towards a regulatory monitoring system that allows for an appropriate use of indicators to evaluate the quality and appropriateness of care provided to residents, and to determine a facility’s compliance with the long-term care requirements. This study develops and tests (with volunteering state survey agencies) various options for using a variety of quality indicators to improve the effectiveness and efficiency of the CMS’s facility performance monitoring.
PIC ID: 7177; EXPECTED COMPLETION: FY 2003; CONTACT: Sue Nonemaker, 410-786-6825; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

Measurement, Indicators, and Improvement of the Quality of Life in Nursing Homes

This task order examines quality of life (QOL) issues for nursing home residents. It will focus on three topics: (1) measuring and developing indicators of QOL, (2) developing quality improvement programs for nursing home QOL, and (3) evaluating environmental design influences on QOL.
PIC ID: 7176; EXPECTED COMPLETION: FY 2002; CONTACT: Mary Pratt, 410-786-6867; PERFORMER: University of Minnesota, Minneapolis, MN

Normative Standards for Medicare Home Health Utilization

This task order develops a model that uses scientifically based, normative standards to determine thresholds for payment authorization within home health service categories. It will test the model to determine the extent of its validity and reliability. The project will also recommend an appropriate demonstration design to evaluate the use of the model by fiscal intermediaries prior to full implementation.
PIC ID: 7175; EXPECTED COMPLETION: FY 2002; CONTACT: Mary Wheeler, 410-786-6892; PERFORMER: Center for Health Policy Research, Denver, CO

Department of Defense Subvention Demonstration Evaluation

Under this demonstration, enrollment in the Department of Defense’s (DoD’s) Senior Prime plan is offered to military retirees over age 65 who live within 40 miles of the primary care facilities of one of the six sites, have recently used military health facility services and are enrolled in Medicare Part B. Medicare makes a capitation payment to the DoD for each enrollee, but the DoD must maintain a level of effort for health care services to all retirees who are also Medicare beneficiaries, whether or not they choose to enroll. The evaluation examines issues in four basic areas: (1) enrollment demand, (2) enrollee benefits, (3) cost of the program, and (4) impacts on other DoD and Medicare beneficiaries. See PIC ID 7171.1.
PIC ID: 7171; EXPECTED COMPLETION: FY 2002; CONTACT: Victor McVicker, 410-786-6681; PERFORMER: Rand Corporation, Santa Monica, CA

Evaluation System for Medicare+Choice

This task order designs and implements a strategy for tracking and evaluating the performance of managed health care organizations, both nationwide and within specific markets. Dimensions of performance being tracked include beneficiary access to managed care, and the cost and quality of services delivered to beneficiaries by managed care organizations.
PIC ID: 7169; EXPECTED COMPLETION: FY 2002; CONTACT: Brigid Goody, 410-786-6640; PERFORMER: Mathematica Policy Research, Inc., Washington, DC

Evaluation of CAHPS/Bulletin/Medicare and You in Kansas City MSA

A consortium of organizations in Kansas and Missouri agreed to participate in an Agency for Healthcare Research and Quality (AHRQ) test of a health plan quality assessment system--the Consumer Assessment of Health Plans Study (CAHPS) report. The report examined consumer quality ratings about local managed care plans’ performance. CMS joined AHRQ and the coalition to extend the evaluation from private plan enrollees and Medicaid enrollees to the Medicare population in Kansas City Metropolitan Statistical Area (MSA). This study will look at whether Medicare beneficiaries use comparative quality information to make health plan choices and whether the Medicare information program (print material) is effective.
PIC ID: 7168.1; EXPECTED COMPLETION: FY 2002; CONTACT: Sherry Terrell, 410-786-6601; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

Evaluation of Competitive Bidding Demonstration for DME and POS

This project tests the feasibility and effectiveness of establishing Medicare fees for durable medical equipment (DME) and Prosthetics, Prosthetic devices, Orthotics and supplies (POS) through a competitive bidding process. The evaluation examines competitive bidding impacts in terms of expenditures, quality, access and product diversity, as well as other impacts of the demonstration.
PIC ID: 7173; EXPECTED COMPLETION: FY 2003; CONTACT: Ann Meadow, 410-786-6602; PERFORMER: University of Wisconsin, Madison, WI

Evaluation of Phase II of the Home Health Agency Prospective Payment Demonstration

This demonstration tested two alternative methods of paying home health agencies (HHA) on a prospective basis for services furnished under the Medicare program: (1) per visit by type of HHA visit discipline (Phase I), and (2) per episode of Medicare-covered home health care (Phase II). The evaluation combined estimates of program impacts on cost, service use, access and quality, with detailed information on how agencies actually change their behavior to produce a full understanding of what would happen if prospective payment replaced the current cost-based reimbursement system nationally.
PIC ID: 7203; EXPECTED COMPLETION: FY 2002; CONTACT: Ann Meadow, 410-786-6602; PERFORMER: Mathematica Policy Research, Inc., Washington, DC

Evaluation of QMB and SLMB Programs

This project is designed to quantitatively and qualitatively evaluate the Qualified Medicare Beneficiary (QMB) and the Specified Low-Income Medicare Beneficiary (SLMB) Programs in the following areas: (1) the motivations and perceptions of enrollees and non enrollees, (2) reasons for state variation in enrollment patterns, (3) the impact of enrollment on Medicare and Medicaid costs and service use, and (4) the impact of enrollment on out-of-pocket costs of eligible individuals. Primary data collection activities include: a survey of a national sample of QMB and SLMB enrollees and of eligible non-enrollees, focus groups of enrollees and non-enrollees, a survey of state agencies, and case study interviews with officials from agencies and advocacy groups. Secondary data sources include: the Medicare Current Beneficiary Survey, the Medicare National Claims History file, the Medicaid Statistical Information System, Third party Buy-In file, and the Medicare Enrollment Database. Descriptive and multivariate analyses will be conducted with the primary and secondary data.
PIC ID: 7390; EXPECTED COMPLETION: FY 2003; CONTACT: Noemi Rudolph, 410-786-6662; PERFORMER: Health Economics Research, Inc., Waltham, MA

Evaluation of the Child Health Insurance Program

The State Children’s Health Insurance Program (SCHIP), established in 1997, is designed to provide medical coverage for children under age 19 who are not eligible for Medicaid and with family incomes below 200 percent of the federal poverty level or 50 percentage points above the current State Medicaid limit. States are required to examine and track the impact of SCHIP in reducing the numbers of low-income uninsured children. This project involves a summary and analysis of the state evaluations and an analysis of external SCHIP-related activities. It provides an analysis of the effect of SCHIP on enrollment expenditures and use of services in Medicaid and state health programs, and an evaluation of stand-alone and Medicaid expansion programs, including the effectiveness of their outreach activities and the quality of care.
PIC ID: 7380; EXPECTED COMPLETION: FY 2004; CONTACT: Rose Marie Hakim, 410-786-6698; PERFORMER: Mathematica Policy Research, Inc., Washington, DC

Evaluation of the Community Nursing Organization Demonstration

This demonstration tests a capitated, nurse-managed system of care. The two fundamental elements of the CNO are capitated payment and nurse case management. The evaluation tests the feasibility and effect on patient care of this capitated, nurse case-managed service delivery model. Both qualitative and quantitative components are included.
PIC ID: 6306.1; EXPECTED COMPLETION: FY 2002; CONTACT: James Hawthorne, 410-786-6689; PERFORMER: Abt Associates Inc., Cambridge, MA

Evaluation of the Diamond State Health Plan

The original purpose of this project was to evaluate the Delaware Health Care Partnership for Children, specifically the effectiveness of the demonstration in reaching its goal of improving access to, and the quality of, health care services delivered to Medicaid-eligible children in a cost-effective way. In May 1996, the project was modified to focus more generally on the impacts of the Diamond State Health Plan (DSHP) on children, including children with special health care needs (the original evaluation had been limited to the Nemours Children’s Clinics). The goal of the evaluation was broadened to assess whether this section 1115 demonstration’s objective of increased access to high-quality, cost-effective care for Medicaid children is being met.
PIC ID: 6288; EXPECTED COMPLETION: FY 2002; CONTACT: Penny Pine, 410-786-7718; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

Evaluation of the EverCare Demonstration Program

The EverCare demonstration attempts to reduce medical complications and dislocation trauma resulting from hospitalization, and to save the expense of hospital care when patients can be managed safely in nursing homes with expanded services. The EverCare evaluation combines data from site case studies, a network analysis of nurse practitioners, participant and caregiver surveys and participant utilization data to examine: (1) a comparison of enrollees and non-enrollees; (2) the process of implementation and operation of EverCare changes in the care process, as well as quality of care; (3) effects of the demonstration on enrollees’ health and health care utilization; (4) satisfaction of enrollees and their families; and (5) effects of the demonstration on the costs of care, as well as payment sources.
PIC ID: 7185; EXPECTED COMPLETION: FY 2002; CONTACT: John Robst, 410-786-1217; PERFORMER: University of Minnesota, Minneapolis, MN

Evaluation of the Home & Community-Based Services Waiver Program

The Home and Community-Based Services (HCBS) waiver program has been operating since 1981 and has experienced strong growth in recent years. The percent of Medicaid long-term care spending devoted to HCBS has increased from 10 percent to 19 percent (between the financial and beneficiary-level impacts of the program) in over a decade. The aim of this task order is to gain a better understanding of the broader HCBS waiver program and determine what programmatic mechanisms have been successful.
PIC ID: 7208; EXPECTED COMPLETION: FY 2002; CONTACT: Susan Radke, 410-786-4450; PERFORMER: The Lewin Group, Fairfax, VA

Evaluation of the Medical Savings Account Demonstration

This evaluation of the Medical Savings Account (MSA) demonstration compares the experiences of MSA enrollees with other Medicare beneficiaries. The evaluation will address access to care and determine if MSAs promote an inappropriately low use of services.
PIC ID: 7172; EXPECTED COMPLETION: FY 2003; CONTACT: Renee Mentnech, 410-786-6692; PERFORMER: Barents Group, KPMG Peat Marwick LLP, Washington, DC

Evaluation of the New York Medicare Graduate Medical Education Payment Demonstration and Related Provisions

Medicare’s annual graduate medical education (GME) spending reached $7 billion, of which nearly 20 percent was for New York teaching hospitals. This is a coordinated evaluation of a major demonstration which provided incentives for New York State teaching hospitals to reduce their residencies by 20 to 25 percent over a 5-year period, and several provisions of the Balanced Budget Act of 1997 (BBA) which were also aimed at reducing Medicare GME spending. The evaluation assessed the impacts of residency reduction on access to service delivery as well as the economic and workforce effects. The work is being performed in the manner described in the “Design for Evaluation of the New York Medicare GME Demonstration and Related Provisions in P.L. 105-330 (BBA): Recommended Design and Strategy for NY GME Demonstration and National BBA GME Provisions.” The project presents a series of reports.
PIC ID: 7379; EXPECTED COMPLETION: FY 2004; CONTACT: William Buczko, 410-786-6593; PERFORMER: Health Economics Research, Inc., Waltham, MA

Evaluation of the Ohio Behavioral Health Program

This project addresses: (1) a focused evaluation of the behavioral health component of OhioCare, and (2) a case study of the implementation of Ohio’s section 1115 State health reform demonstration, OhioCare. The case study will complement the focused evaluation by providing a context for findings and supplemental findings.
PIC ID: 7184; EXPECTED COMPLETION: FY 2002; CONTACT: Penny Pine, 410-786-7718; PERFORMER: Health Economics Research, Inc., Waltham, MA

Health Disparities: Longitudinal Study of Ischemic Heart Disease Among Aged Medicare Beneficiaries

This project assesses the use of Medicare covered services among Medicare beneficiaries with ischemic heart disease based on sociodemographic characteristics (e.g., race/ethnicity, sex, age, socioeconomic status). It is one part of a larger CMS and Department of Health and Human Services effort to address health disparities among Medicare beneficiaries. This is done using a longitudinal database that links Medicare enrollment and claims data with small-area geographic data on income (e.g., U.S. Census data). Due to recent change in the race/ethnic coding in the Medicare enrollment database (EDB), it is not possible to examine health care access, utilization, and outcomes among minority groups.
PIC ID: 7419; EXPECTED COMPLETION: FY 2002; CONTACT: Linda Greenberg, 410-786-0677; PERFORMER: Health Economics Research, Inc., Waltham, MA

Impact of Welfare Reform on Medicaid Populations

This project develops data and examines the impact of welfare reform on Medicaid eligibility, utilization and payments for various populations. It studies the effects of the following four changes: (1) de-linking Aid to Families with Dependent Children (AFDC) and Medicaid eligibility, (2) terminating access to Medicaid for some legal immigrants because of lost eligibility for Supplementary Security Income (SSI), (3) barring most future legal immigrants from Medicaid, and (4) narrowing Medicaid eligibility for selected disabled children and disabled alcohol and substance abuse populations.
PIC ID: 7183; EXPECTED COMPLETION: FY 2002; CONTACT: Penny Pine, 410-786-7718; PERFORMER: Mathematica Policy Research, Inc., Washington, DC

Maximizing the Cost Effectiveness of Home Health Care (HHC)

Rapid growth in home health use has occurred despite limited evidence about the necessary volume of HHC needed to achieve optimal patient outcomes, and whether or not it substitutes for more costly institutional care. The central hypotheses of this study are that: (1) volume-outcome relationships are present in HHC for common patient conditions, (2) upper and lower volume thresholds define the range of services most beneficial to patients, and (3) a strengthened physician role and better integration of HHC with other services during an episode of care can optimize patient outcomes while controlling costs.
PIC ID: 7179; EXPECTED COMPLETION: FY 2002; CONTACT: Ann Meadow, 410-786-6602; PERFORMER: Center for Health Policy Research, Denver, CO

Maximizing the Effective Use of Telemedicine: A Study of the Effects, Cost Effectiveness, and Utilization Patterns of Consultation via Telemedicine

This project is conducting an evaluation of the CMS’s Medicare payment demonstration. The evaluation examines the medical effectiveness, patient and provider acceptance, and costs associated with telemedicine services, as well as their impact on access to care in rural areas.
PIC ID: 6303; EXPECTED COMPLETION: FY 2002; CONTACT: Joel Greer, 410-786-6695; PERFORMER: Center for Health Policy Research, Denver, CO

Medicare Post-Acute Care: Evaluation of BBA Payment Policies and Related Changes

This project studies the impact of the Balanced Budget Act (BBA) and other policy changes on Medicare utilization and delivery patterns in post-acute care. Post-acute care is generally defined to include the Medicare covered services provided by skilled nursing facilities (SNFs), home health agencies, rehabilitation hospitals and distinct part units, long term care hospitals, and outpatient rehabilitation providers. Understanding the relationships among post-acute care delivery systems is critical to the development of policies that encourage appropriate and cost-effective use of the entire range of care settings. The results of this work may be useful in refining policies for individual types of post-acute care, as well as in developing a more coordinated approach across all settings.
PIC ID: 7417; EXPECTED COMPLETION: FY 2002; CONTACT: Philip G. Cotterill, 410-786-6598; PERFORMER: Medstat Group, Washington, DC

Multi-State Evaluation of Dual Eligible Demonstrations

This evaluation is designed to assess the impact of dual eligible demonstrations in the states of Minnesota, Colorado, Wisconsin and New York. Analyses are being conducted for each state and across states.
PIC ID: 7186; EXPECTED COMPLETION: FY 2005; CONTACT: Noemi Rudolph, 410-786-6662; PERFORMER: University of Minnesota, Minneapolis, MN

Racial Disparities in Health Services Among Medicaid Pregnant Women (Multi-State) Analysis

This is a study of associations between pregnancy-related care and outcomes, and the ethnic and racial characteristics of women who had a Medicaid covered delivery during calendar year 1995. This study is expected to identify and explain the patterns of disparities in prenatal and postpartum care and outcomes provided to Medicaid women. The project evaluates the use of health services from entry into prenatal care through the delivery and into the first three postpartum months. The CMS eligibility and utilization data contain information on racial and ethnic minority groups. These data include diagnoses, procedures, date and type of delivery, reimbursements, demographics, and geographic location. It examines the use of and Medicaid expenditures for health services from the initial prenatal care visit through the delivery and into the first three postnatal months. Specific prenatal care markets to be considered include delayed prenatal care, no prenatal care, and an insufficient total number of prenatal care visits for a full-term, normal pregnancy. For each of the health care utilization analysis, expenditures will also be analyzed.
PIC ID: 7416; EXPECTED COMPLETION: FY 2002; CONTACT: Beth Benedict, 410-786-7724; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

Survey of Medicare Beneficiaries Who Were Involuntarily Disenrolled from HMOs that Withdrew from Medicare

When HMOs withdraw from the Medicare program or reduce their service areas, thousands of Medicare beneficiaries become disenrolled involuntarily. There has been concern among policymakers about the impact of the recent HMO withdrawals on the beneficiary population. Additional withdrawals occurred in 2001 and may also occur in subsequent years. This project conducts a survey that asks about the experience of beneficiaries whose plans withdraw from Medicare or reduce their service areas in January 2001. The universe from which the survey sample will be drawn is the Medicare population enrolled in managed care plans that either terminated their risk contracts or reduced their service areas in January 2001. The survey is conducted by mail with telephone followup.
PIC ID: 7421; EXPECTED COMPLETION: FY 2002; CONTACT: Gerald Riley, 410-786-6699; PERFORMER: University of Wisconsin, Madison, WI

Food and Drug Administration

Mission

To protect and promote public health through food, drug, medical device, and cosmetic regulation.

Evaluation Program

The Food and Drug Administration’s FY2001 Evaluation Program ultimately reflected some of the goals established and promulgated by the Department of Health and Human Services (HHS) for which FDA has responsibility. HHS’s goals are products of its strategic performance planning process, and FDA uses its own strategic framework to accomplish these goals. This process also satisfies the implementation requirements of the Government Performance and Results Act (GPRA) and the Food and Drug Administration Modernization Act of 1997 (FDAMA). The strategic and performance process is an evolving set of program directions for FDA as changes occur in FDA’s dynamic environment. FDA’s challenges, now and in the future, will rest on its ability to leverage its efforts in that environment, which grows increasingly complex and more institutionally networked. The Agency will strive to maintain the scientific knowledge base necessary to achieve greater effectiveness in assuring the quality and availability of the products it regulates.

One goal area within FDA’s strategic framework is Pre-Market Review, where the objective is to make timely and cost-effective pre-market review decisions, while assuring product safety and efficacy. The results achieved and reported in the Prescription Drug User Fee Act (PDUFA) FY2000 Performance Report, are one indication of how well FDA is meeting its goal of making timely pre-market review decisions. The focus of performance goals under PDUFA is to expedite the entire drug development and review process, from research to approval, without compromising the safety or the quality expected from the Agency’s application review process.

Another goal area within FDA’s strategic framework is Post-Market Assurance, a goal that strives to strengthen the assurance that products on the market are safe. One strategy of accomplishing this goal is by targeting high-risk products. The Tobacco Program illustrated one of the initiatives in this area.

Completed Evaluations

An Assessment of Thimerosal Use in Childhood Vaccines

This report (published in PEDIATRICS, Vol. 107, No. 5, May 2001) provides an overview and summary assessment of the use of thimerosal, a mercury derivative, as a preservative in vaccines. The findings, made available in draft form to vaccine advisory committees, led to strong recommendations to vaccine manufacturers to remove thimerosal from childhood vaccines, recommendations that have now been implemented. This risk assessment consisted of hazard identification, dose-response assessment, exposure assessment, and risk characterization. The risk assessment focused on infants and young children because of their small body size, developing brain, and exposure to vaccines containing thimerosal. A literature review was conducted in order to identify the known toxicity of thimerosal and related organic mercury compounds, and to determine the doses at which toxicity occurs. Maximum potential exposure to mercury from vaccines was calculated for children 6 months and 2 years of age, under the U.S. childhood immunization schedule, and compared with mercury exposure limit standards. This assessment showed no evidence of harm caused by doses of thimerosal in vaccines, except for local hypersensitivity reactions. However, depending on the immunization schedule, vaccine formulation, and infant weight, cumulative exposure of infants to mercury from thimerosal during the first 6 months of life may exceed EPA guidelines. This article concludes that mercury in vaccines can be reduced or eliminated by using products formulated without thimerosal as a preservative.
PIC ID: 7736; CONTACT: Leslie K. Ball, 301-827-3070; PERFORMER: Food and Drug Administration, Washington, DC

U.S. Regulation of Pharmaceutical Outcomes Research

Outcome research often produces information that is particularly attractive for use in pharmaceutical product promotion. FDA is then faced with making a determination as to whether the information used in pharmaceutical product promotion is adequate to support labeling and advertising claims. A lack of consistency in the terminology used, and a lack of understanding and agreement on the evidentiary standards that should apply to the use of these data may hinder outcome evidence. This article presents categories of outcomes information (such as economic, productivity, compliance, and satisfaction) and discusses the unique set of scientific and regulatory issues associated with each. The article concludes that the future of outcomes research in drug development will be shaped by the consumers of outcome information and the perceived added value of these outcome measures. Furthermore, it is envisioned that development of this field of research will benefit by integration of outcomes researchers with the rest of the clinical development team to facilitate in the production of adequate evidence to meet developing regulatory hurdles. The final report appears in published form in: “Value in Health”, vol. 4, No. 2, 2001. Author: Laurie B. Burke, Chief, Evidence Review Branch, Food and Drug Administration (FDA).
PIC ID: 7733; CONTACT: Laurie Burke, 301-827-3910; PERFORMER: Food and Drug Administration, Consumer Research Staff, Washington, DC

Analysis of Hazard Analysis Critical Control Point (HACCP) Survey Data

The Food and Drug Administration (FDA) is placing increasing emphasis on the use of Hazard Analysis and Critical Control Point (HACCP) systems to ensure the safety of foods. Over a number of years, Research Triangle Institute under contract with FDA, conducted studies to estimate the costs of implementing HACCP in food plants under FDA jurisdiction. The project was conducted in three phases. The task of this report is to provide a final report on the outcome of the three phases of the overall project, and to provide a Microsoft Access tool that combines the data from the three phases of the overall project and that can be used to analyze the effects of various regulatory options related to HACCP. Section II of the report describes the results of the survey on HACCP practices in plants under FDA jurisdiction (Phase III) and includes a brief description of the sampling frame used to conduct the survey (from Phase I). Section 3 describes the results of the HACCP cost data collection process (from Phase II). Finally, Section 4 describes the interactive Microsoft Access tool that combines the data from the three phases of the project.
PIC ID: 6866; CONTACT: Clark Nardinelli, 202-205-8702; PERFORMER: Food and Drug Administration, Rockville MD

Food Testing Laboratory Industry Database: Final Report

In September 2000, FDA contracted with the Research Triangle Institute to collect information on the characteristics and capabilities of the 546 private laboratories that submit test packages to the FDA and to prepare a dataset containing this information. The primary objective of this study was to provide FDA with information that could be used to assess the quality and uniformity of results reported by private laboratories. Private laboratories performing analyses on regulated food products that may submit analytical data to FDA in order to demonstrate compliance with the Food, Drug, and Cosmetic Act were included (FDA’s 1997 definition of private laboratories.) Private laboratories were defined as independent providers of services that were not owned by the firms utilizing those services. Data on private laboratories were collected from FDA sources (such as OASIS) and publicly available sources on the range of companies offering food-testing services nationally and a survey of food testing laboratories in other countries. This information was entered into an Access database containing information on laboratory location, contact information, economic variables, test capabilities, and quality assurance programs.
PIC ID: 7731; CONTACT: Clark Nardinelli, 202-205-8702; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

Maternity Care Practices: Implications for Breastfeeding

The purpose of this study was to assess the impact of the type and number of Baby-Friendly practices experienced on breastfeeding. The Infant Feeding Practices Survey, a longitudinal study conducted by the FDA, was administered to women prenatally through 12 months postpartum. The study focused on 1,085 women with prenatal intentions to breastfeed for more than 2 months who initiated breastfeeding, using data from the prenatal and neonatal periods. Five predictor variables included indicators of the absence of specific Baby-Friendly practices (late breastfeeding initiation, introduction of supplements, no rooming-in, not breastfeeding on demand, use of pacifiers) and number of Baby-Friendly practices experienced. The main outcome measure was breastfeeding termination before 6 weeks. The findings: Only 7% of mothers experienced all 5 Baby-Friendly practices. The strongest risk factors for early breastfeeding termination was late breastfeeding initiation and supplementing the infant. Compared with mothers experiencing all 5 Baby-Friendly practices, mothers experiencing none were approximately eight times more likely to stop breastfeeding early. Additional practices decreased the risk for early termination. Based on the analysis presented in this study, it can be concluded that increased Baby-Friendly Hospital Initiative practices improve the chances of breastfeeding beyond 6 weeks. The need to work with hospitals to increase adoption of these practices is illustrated by the small proportion of mothers who experienced all 5 practices measured in this study. See final report in Birth, Vol. 28, #2, June 2001.
PIC ID: 7734; CONTACT: Sara Fein, 202-205-5349; PERFORMER: Food and Drug Administration, Washington, DC

Women’s Participation in Clinical Trials and Gender-Related Labeling

Through a retrospective review of clinical trial protocols and labeling for 185 new molecular entities approved by the FDA Center for Drug Evaluation and Research, this study assessed the extent to which women participated in clinical trials over a five-year period (1995-1999). In addition to looking at clinical trials for all products examined, the study’s scope included the percentage of women participating in the trials by year, phase (although 72 percent of the protocols did not specify phase), and product type (i.e., the division within the agency responsible for the product review). As well, the study looked at the frequency with which FDA medical officers’ evaluations of the products commented about whether the sponsors’ studies revealed gender differences. Differences in this frequency over time, by phase, and by product type were not examined. The study also examined the frequency with which package inserts contained some type of statement related to gender (e.g., whether or not there were similar effects of the drug in men and women) and whether this frequency differed by product type. The findings indicated that, overall, women appeared to participate in the clinical trials at nearly the same rate as men, and that labeling of two-thirds of the products contained some statement about gender (although only 22 percent described actual gender effects). Of the effects discussed, 90 percent were pharmacokinetic, 12 percent were safety, and 5 percent were efficacy.
PIC ID: 7737; CONTACT: Teresa Toigo, 301-827-4460; PERFORMER: FDA, Center for Drug Evaluation & Research, Rockville, MD

In-Progress Evaluations

Impact of the Mammography Quality Standards Act of 1992

The final report for this study has been drafted but not yet finalized or cleared. The study estimated the economic impact of regulations developed under the MQSA. It evaluated conditions of facilities after implementation of the regulations in order to evaluate whether patient access to quality screening services has been adversely affected. The Contractor examined agency data sets as well as other public sources to estimate the effect of regulations on facility closures and patient access. Field contacts were used to discuss impacts with industry experts. Preliminary findings are that the regulations have not adversely affected patient access to high quality mammography screens.
PIC ID: 6080.4; EXPECTED COMPLETION: FY 2003; CONTACT: Steve Tucker, 301-827-5339; PERFORMER: Eastern Research Group, Lexington, MA

Nationwide Evaluation of X-Ray Trends (NEXT)

The Nationwide Evaluation of X-Ray Trends (NEXT) survey program is a collaboration jointly administered by the Food and Drug Administration (FDA) and State radiation control program agencies. Each year a selected diagnostic x-ray is selected for survey, and nationally representative data are collected from US facilities on patient exposure levels and associated indicators of diagnostic quality. Surveys are repeated periodically to observe trends in the state of practice, and the results are published by the Conference of Radiation Control Program Directors (CRCPD), Frankfort, KY. Please refer to web site address.
PIC ID: 4984; EXPECTED COMPLETION: FY 2002; CONTACT: Orhan Suleiman, 301-594-3533; PERFORMER: Food and Drug Administration, Rockville MD

Drug Metabolism: Drug-Drug Interactions

Recent examples of studies that have provided information to specific review decisions include: (1) interactions and metabolism studies of anti-cancer drugs such as paclitaxel, (2) anti-abuse drugs such as LAAM, (3) anti-HIV drugs such as the protease inhibitors, (4) anti-epileptic drugs, (5) cardiovascular drugs, and (6) anti-malarial drugs. In addition to the general technical expertise brought to bear on these problems, results from the program have been incorporated into final product labeling for some drugs, including paclitaxel and saquinavir. Labeling changes for other drugs are under discussion. Also, cutting-edge issues regarding cases of intermediate potency for cytochrome P450 pathways are under investigation.
PIC ID: 7103; EXPECTED COMPLETION: FY 2004; CONTACT: Jerry Collins, 301-827-5471; PERFORMER: Food and Drug Administration, Rockville MD

Food Labeling and Package Survey (FLAPS)

This survey was implemented by the Food and Drug Administration (FDA) in 1977 with the purpose of producing a database containing label and package information obtained from a sample of processed food products. Food Labeling and Packaging (FLAPS) data enable FDA personnel to keep abreast of market responses to food labeling rules via changes in package labels. Data collection for the most recent survey (2000-2001) commenced on 5-1-2000 and should be completed by the end of FY 01. Data analysis will commence on 11-01-02 and end by 9-28-02. The end of FY02 schedules two reports for completion.
PIC ID: 5711.3; EXPECTED COMPLETION: FY 2002; CONTACT: Lori A. LeGault, 202-205-5269; PERFORMER: Information Resources Inc., Chicago, IL

Prescription Drug User Fee Act Reports

The Prescription Drug User Fee Act (PDUFA) of 1992 provided FDA with greater resources for the review of human drug and biologic applications. Fees that FDA collected from drug and biologics firms were used to reduce the time required to evaluate human drug applications without compromising review quality. FDA has submitted annual Performance and Financial Reports to Congress on progress in streamlining the drug review process and use of PDUFA fees. The original act expired September 30, 1997, but the FDA Modernization Act of 1997 amended and extended PDUFA through September 30, 2000. The amended act is now referred to as PDUFA II and its predecessor as PDUFA I.
PIC ID: 6079.6; EXPECTED COMPLETION: FY 2002; CONTACT: Dennis Hill, 301-827-5255; PERFORMER: Office of Policy, Planning, and Legislation, FDA, Rockville MD

Health Resources and Services Administration

Mission

To improve the Nation’s health by assuring equitable access to comprehensive, culturally competent, quality health care for all.

Evaluation Program

The Health Resources and Services Administration (HRSA) supports a wide array of very different programs and activities that promote access to needed health care for all, including primary health care centers, the National Health Service Corps, HIV/AIDS programs, maternal and child health activities, health professions training, rural health programs, organ donation and transplantation initiatives, and telehealth activities. To provide underpinning for these efforts, HRSA’s evaluation program is designed to enhance strategic planning, strengthen budget and legislative development, and improve program performance.

HRSA also supports activities to enhance the quality of evaluation agency-wide, such as funding short courses in evaluation for staff and expanding agency staff’s access to technical assistance on the conceptualization, design and implementation of evaluation activities. The broad dissemination of evaluation products and results is also an area of emphasis.

Completed Evaluations

Enabling Services and Perinatal Care: Final Report

A key strategy used by all Bureau of Primary Health Care grantees is ro provide enabling services which may include transportation, translation, case management, health education, nutrition counseling and outreach--are not typically reimbursed under managed care. The purpose of this study was to analyze the types and levels of enabling services provided by Community Migrant Health Centers (C/MHCs), how these services have changed over time, and whether enabling services improve outcomes and reduce costs. The study drew upon aggregate data from the BPHC Uniform Data System, the annual report submitted by all grantees. A total of 650 grantees filed reports in both 1996 and 1997. In 1997, enabling service personnel accounted for almost one-quarter of total direct care staff and sample grantees expended $211 million on these services (about 13% of total direct service cost). About 95% of grantees provided case management and health education services. At the other end of the spectrum, under 20% provided child care on-site or operated food banks and/or delivered meals. It was found that health center prenatal care users are twice as likely to be teenagers-- and twice as likely to be Hispanic -- than the comparable US prenatal population. About 72% of C/MHC grantees have specialized obstetrical staff; the average grantee with specialized staff has 1.3 OB-GYN/Certified Nurse Midwife full-time employees. The report concludes that the breadth of perinatal services, coupled with staffing that promotes continuity of care, contribute to appropriate and timely use of prenatal and after-delivery services by mothers and infants.
PIC ID: 7126; CONTACT: Fred Butler, 301-549-4281; PERFORMER: MDS Associates, Inc., Wheaton, MD

Professional Nurse Traineeship Grants: Who Gets Them and Where Do They Work After Graduation?

The Professional Nurse Traineeship Program (PNT) is one of the Health Resources and Services Administration (HRSA) programs intended to alleviate access to primary care provider problems in medically underserved communities. Nursing schools may apply for PNT funds to support the education of nurses in graduate programs of study. This report describes the findings from the first study of the PNT program. The study focuses on whether the PNT Program met the GPRA goal of placement of 40 percent of graduates in medically underserved communities. The overall findings: (1) The study showed that the PNT program appears to be meeting the standard specified for the Government Performance Reporting Act--40 percent placement of nurse graduates in medically underserved communities. In fact, 45 percent of the graduates worked in a medically underserved community at the job at which they had spent the most time since graduating. (2) The study did not show that schools receiving the preference had higher placement rates of their graduates in underserved communities than did the other schools.
PIC ID: 7130; CONTACT: Madeleine Turkeltaub, 301-443-6193; PERFORMER: Mathematica Policy Research, Inc., Washington, DC

America's Health Care Safety Net

This study examined the impact of Medicaid managed care and other changes in health care coverage on the integrity and viability of safety-net providers operating in primary care settings, including those funded by the Health Resources and Services Administration, including Community and Migrant Health Centers, Maternal and Child Health programs, Ryan White Comprehensive AIDS Resources Emergency (CARE) Act programs. The Institute of Medicine selected 14 experts to serve on a committee to conduct the study. The committee found convincing evidence that even within the context of insurance reform, segments of America's most disadvantaged populations will continue to rely on traditional safety net providers for their health care services, primarily because many of these providers are uniquely organized and oriented to the special needs of low-icome and uninsured populations. The IOM committee defines the “health care safety net” as follows: “Those providers that organize and deliver a significant level of health care and other related services to uninsured, Medicaid, and other vulnerable patients.”
PIC ID: 6815; CONTACT: Alexander Ross, 301-443-1512; PERFORMER: National Academy of Sciences, Institute of Medicine, Washington, DC

In-Progress Evaluations

Assessment of Domestic Violence Interventions and Staff Training Protocols in Community-Based Primary Health Care Settings

The purposes of this study are to examine a) the characteristics/components of the domestic violence protocols used in HRSA-funded community-based primary health care centers; b) the effect that these protocols have on the reporting of violence, clinical diagnoses, and referrals to local community services among female clients; and c) how other BPHC-funded programs can develop domestic violence interventions in their organizations and communities. Health providers often treat abused women without recognizing or addressing the underlying causes of their health condition. A 1995 survey of 10 BPHC-funded primary health care sites found that only half had a formal tool for assessing domestic violence. This study will analyze time series data to compare the effects of incorporating a protocol on domestic violence. Statistical analysis will control for sociodemographic characteristics and other known confounders. In-depth telephone interviews will be conducted with health care center staff (protocol developers, trainers, and key program coordinators). Documentation, screening, and reporting procedures will be reviewed. Training and referral strategies will be analyzed. Nine sites will be selected for study based on a comprehensive literature review of domestic violence interventions during the past five years. A Steering Committee for Family and Intimate Partner Prevention Violence, comprised of HRSA staff, will provide advice on methods.
PIC ID: 7284; EXPECTED COMPLETION: FY 2002; CONTACT: Kathleen Shannon, 301-594-3621; PERFORMER: North American Management Company, Alexandria, VA

Determinants of Change in Health Center Revenues, Service Capacity, and Payor Mix at the Marketplace Level

The purpose of this study is to examine the impact of changes in environmental and management factors in the health care system on the revenues, service capacity, and payor mix of Community Health Centers (CHCs). A previous study found that nationally, Medicaid users have decreased, uninsured users have increased, and Medicaid revenues per user have decreased somewhat. This study will explore CHC-specific data in selected areas of the country. Independent variables to be studied include the percentage of Medicaid recipients in managed care and the overall managed care penetration; the percentage and growth of uninsured; the number and nature of HMOs providing Medicaid services; the safety net provider capacity; Medicaid enrollment; patient access/satisfaction; participation in integrated delivery systems and networks; extent of CHC participation in managed care and; CHC performance (based on nine measures used in BPHC’s health center reinvestment process). Data will be drawn from the 12 areas in the RWJ Community tracking study, complemented by BPHC Market Place Analysis information for 4-5 areas. Relevant State Insurance Department and State Hospital Association data, and the Dartmouth Atlas will be reviewed. Data for CHCs will be drawn from BPHC’s Uniform Data System.
PIC ID: 6784; EXPECTED COMPLETION: FY 2003; CONTACT: Rhonda Hughes, 301-594-4280; PERFORMER: Ann Zuvekas, Consultant, Annandale, VA

Health Care Status Outcome Measures for the Bureau of Primary Health Care: The Assessment of Ambulatory Care Sensitive Conditions Through State Medicaid

This project will compare the relative risk of inpatient hospitalizations for Community Health Center (CHC) users with non-CHC users for ambulatory care sensitive conditions. Ambulatory care sensitive conditions are those considered preventable, treatable, or controllable in an outpatient setting. The project will design and test a methodology using selected areas in selected states from the newly available State Medicaid Research Files (SMRF). Major research questions for the study include: (1) the best methodological design to assess relative risk of inpatient hospitalizations for selected ambulatory conditions; (2) the relative risk of inpatient hospitalizations.
PIC ID: 6364; EXPECTED COMPLETION: FY 2002; CONTACT: Barbara Wells, 301-594-4463; PERFORMER: MDS Associates, Inc., Wheaton, MD

Health Status Outcome Measures for the Bureau of Primary Health Care: Examination of Episodes of Care for Diabetes, Hypertension, Asthma

A consensus conference in December 1995 recommended the use of Medicaid data to examine changes in utilization patterns for Community Health Center (CHC) patients diagnosed with ambulatory care sensitive conditions (ACSCs). These are conditions which frequently can be managed with timely and effective treatment in outpatient settings, thus preventing the need for hospitalization. The purpose of this study is to compare episodes of ambulatory care for CHC users to those of non-CHC users when both have been hospitalized with a primary diagnosis of diabetes, hypertension, asthma, or other ACSCs, as well as when neither has been hospitalized. A previous study (See PIC ID 6001) showed that Medicaid beneficiaries who received most of their care from CHCs had lower hospitalization rates for ACSCs than did non-CHC users. Findings of the current study will improve understanding as to why CHC users experience lower hospitalization rates for ACSCs, and may have different patterns of ambulatory care use. Findings should also identify the major strengths and limitations of the State Medicaid Research Files for examining episodes of care for a comparison of CHC users and non-CHC users.
PIC ID: 7127; EXPECTED COMPLETION: FY 2002; CONTACT: Barbara Wells, 301-594-4463; PERFORMER: MDS Associates, Inc., Wheaton, MD

Health Status Outcomes for the Bureau of Primary Health Care: A Pilot Study Assessing Physiologic Measures Through Medical Record Review

Through a review of medical records, this study will assess changes in health status among a sample of adult patients of Community Health Centers (CHCs). The conditions selected for evaluation are hypertension and diabetes mellitus. The review will address: (1) the definition of a CHC “user”; (2) confirmation of a diagnosis; (3) patient stratification by severity and/or onset of the condition; (4) expected attrition rates; (5) inclusion of insurance/payer status as a control variable; (6) the time frame in which the two conditions will be measured; (7) protocol for sampling medical records; (8) development of an index of co-morbidities; (9) preparation of a taxonomy of CHC site characteristics; and (10) the appropriate instrument for extracting pertinent data. This project continues the HRSA’s systematic effort to identify health status outcomes that may be used to measure the effectiveness of primary care programs.
PIC ID: 6802; EXPECTED COMPLETION: FY 2002; CONTACT: Barbara Wells, 301-594-4463; PERFORMER: The Lewin Group, Fairfax, VA

Impact of Publicly Funded Insurance Programs on Pediatric Safety-Net Providers

The purposes of this study are to a) describe the relationship between characteristics of publicly- funded programs and the survival/financial viability of pediatric safety-net providers; b) determine the differential effects of Medicaid Managed Care (MMC) and the implementation of the State Children’s Health Insurance Program (SCHIP) for pediatric safety-net hospitals relative to pediatric Federally Qualified Health Centers (FQHCs); c) investigate institutional and organizational factors among pediatric safety-net providers; and d) examine the success and failures that these providers have experienced in confronting changes in their community. Improved understanding of the impact of major policy changes on the viability of community pediatric safety-net health care providers can facilitate program strategies to lessen adverse consequences for vulnerable children. Case studies and interviews will be conducted to examine changes in the financial status of these institutions. A logistic regression model will be used to estimate the impact of hospital, market, and policy factors on closure of safety-net providers. Hospital cost, revenue, and profit equations will be estimated using a fixed effects regression model.
PIC ID: 6039; EXPECTED COMPLETION: FY 2002; CONTACT: Felicia L. Collins, 301-594-3732; PERFORMER: Northwestern University, Center for Health Services Policy, Evanston IL

The Impact of the State Children’s Health Insurance Program on Selected Community Health Centers and Maternal and Child Health Programs

This study is assessing: (1) the effect of the State Children’s Health Insurance Program (SCHIP) on the insurance status of children served by selected Community Health Centers (CHCs) and Maternal and Child Health (MCH) programs, and (2) the impact on the extent to which these children enter or remain in care at selected CHC and MCH sites. Issues to be addressed include: (1) the insurance history of children who have used or are new to the site; (2) the continuous nature and time span of the coverage; (3) insurance characteristics of, and source of care for children who are no longer users; and (4) the characteristics of sites relative to their ability to enroll and/or retain newly insured children. Previous analysis of CHC encounter files documented significant volatility in coverage, with patients going on and off coverage as many as five times in a given year. Study findings will provide a framework for future investigation, develop a transferable methodology for use by states and sites, and help to assess the extent to which SCHIP has affected the insurance coverage of children served by CHCs and MCH programs.
PIC ID: 7125; EXPECTED COMPLETION: FY 2002; CONTACT: Jean Yoon, 301-594-6460; PERFORMER: George Washington University, Washington, DC

Year 2000 Community Health Center and National Health Service Corps User/Visit Survey

The purpose of this study is to conduct: 1) a personal interview survey of users of Community Health Center/Maternal and Child Bureau health sites; 2) a separate record-based study of visits to these sites; and 3) a pilot test of the feasibility of a prospective sampling strategy for use in the Community Health Center and National Health Service Corps User/Visit Survey. The user and visit survey will provide in-depth information about the socio-demographic characteristics of users, their risk behaviors and health status, the reasons they seek care, most frequent diagnoses, satisfaction with care, monitoring of chronic conditions, and the services used in a medical encounter. Attention will be paid to whether the sites provide care that meets or exceeds the Healthy People 2000 and 2010 national objectives. A sample of 50-60 grantee health centers and 15 non-grantee, freestanding sites will be selected, and a sample of 40-50 clients per center/site from medical records. Sampling from the 48 contiguous states will involve urban /rural and the West, Midwest, Northeast, and South Census regions. Questions will be taken from the National Health Interview Survey to allow comparisons with the national population. A retrospective sample of visits will be drawn to obtain a profile of the kinds of conditions treated and services provided. PIC ID: 6811; EXPECTED COMPLETION: FY 2002; CONTACT: Jerrilynn Regan, 301-594-4283; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

HIV Service Utilization and Health Outcomes for PLWH with Comorbidities in RWCA-Funded Programs

The purpose of this study is to evaluate (a) the additional demands placed on the health care system by individuals living with HIV infection who have comorbid conditions and (b) the benefits and costs of integrating and coordinating treatment for these conditions. Two separate projects are underway. The Johns Hopkins University will describe and quantify the delivery of comorbidity services, compare this delivery with established guidelines and standards, and analyze the outcomes of the HIV infection and selected comorbidities (substance abuse, psychiatric illness, and hepatitis C). The Washington University School of Medicine will study comorbidities of adult and adolescent women with HIV. Conditions include chemical dependency, mental illness, TB, STD, cervical dysplasia and cancer, diabetes, hypertension, renal failure, and hepatitis B and C. Homelessness and encounters with the criminal justice system will also be explored. The project will analyze how many clients are receiving services for comorbid conditions and HIV, the standard of medical care for those with these conditions, and changes in this standard during 1996-1998. At least one publishable article is to discuss how grantees and planning bodies can use study findings to optimize the delivery of services under the Ryan White CARE Act.
PIC ID: 7267; EXPECTED COMPLETION: FY 2002; CONTACT: Moses B. Pounds, 301-443-2894; PERFORMER: Johns Hopkins University School of Medicine, Baltimore, MD

Impact of Increased Dental Medicaid Reimbursement Rates

The purpose of this study is to evaluate the impact of increased Medicaid dental fees on children’s utilization of dental services and access to dental care in South Carolina. The study will also include an analysis of the supply of dentists that accept Medicaid and the geographic distribution before and after the fee increase. In addition, it will assess changes in the utilization patterns of preventive and restorative care, and will examine the feasibility of developing performance measures that could be tracked using Medicaid data. The study will involve analysis of Medicaid and other extant databases, and secondary analysis of a survey by the South Carolina Dental Association.
PIC ID: 7196; EXPECTED COMPLETION: FY 2002; CONTACT: Raul A. Romaguera, 404-562-4180; PERFORMER: Medical University of South Carolina, Charleston, SC

Impact of RWCA Title I Funding on HIV Services Utilization and Health Outcomes in Las Vegas, Nevada and Norfolk, Virginia

The purpose of this collaborative project with CDC is to examine the impact of Ryan White CARE Act Title I funding on HIV service use and outcomes in newly eligible metropolitan areas (EMAs). Title I grants are expected to increase each EMA’s financial base for planning, developing, and expanding HIV-related health and support services. This project will determine how these funds affect the availability, accessibility, quality, continuity, and integration of care, and HIV-related morbidity (e.g., the incidence of opportunistic infection) and mortality among underserved and vulnerable populations with HIV/AIDS. A cross-sectional pre- and post-analysis of the effects of Title I funding will be conducted in two new EMAs--Las Vegas, Nevada and Norfolk- Newport News, Virginia. The analysis will provide additional information about the nature and magnitude of the impact of Ryan White program funding on health care services and outcomes for persons with HIV/AIDS.
PIC ID: 7215; EXPECTED COMPLETION: FY 2002; CONTACT: Faye Malitz, 301-443-3259; PERFORMER: The Learning Group Corporation, Rockville MD

Assessment of HRSA’s Distance Learning Program

This project is the initial phase of a longer-term effort to evaluate the effectiveness of HRSA’s many efforts to disseminate information and provide education and training through various distance learning techniques. To inform this eventual assessment, this project will: (1) provide a review of relevant literature on the relative effectiveness of different distance learning approaches; (2) catalogue HRSA’s dissemination and distance learning programmatic efforts to determine the methods and purposes for which they are used; (3) examine the data currently being collected on these efforts and the utility of those data; (4) determine additional data and analytical needs; and (5) recommend designs for an evaluative study.
PIC ID: 7114; EXPECTED COMPLETION: FY 2002; CONTACT: Jacob Tenenbaum, 301-443-9011; PERFORMER: The Lewin Group, Falls Church, VA

Hospital Organ Donation Best Practices: Study Design

The purpose of this project is to develop an approach to investigate factors associated with variations in hospital organ and tissue donation rates. Such factors include but are not limited to family request practices and hospital referral patterns for reporting deaths and imminent deaths to Organ Procurement Organizations (OPO). The goal is to be able to identify hospital and OPO practices that are associated with higher donor referral, consent, and recovery. Contingent upon the success of this design phase, a study will be implemented and information on best practices will be shared with the hospital and transplant communities to stimulate improvement in donation practices.
PIC ID: 7288; EXPECTED COMPLETION: FY 2002; CONTACT: Mary Ganikos, 301-443-7577; PERFORMER: The Lewin Group, Falls Church, VA

Analysis of Client-Level Data from the National Survey of Homeless Assistance Providers and Clients

The purpose of this study is to analyze the characteristics and health service use patterns of homeless people with special needs, and how the homeless population with alcohol, drug, and mental health problems compares to the general population. Issues to be addressed include: a) comparison of the rates of alcohol, drug, and mental health problems, and related treatment patterns with the general or low-income population; b) development of severity indexes on domains such as health, mental health, substance abuse, employability, and receipt of benefits; c) variation of service utilization patterns and their association with other significant variables (e.g., effect of Medicaid or other type of insurance on type and frequency of treatment); d) the relationship between severity indexes, service use patterns, and the history or nature of homelessness (e.g., relationship between treatment history and homelessness); and e) the factors associated with reported service needs and problems (e.g., reports of difficulty accessing primary/dental care). Core data (available in August 1999) will be drawn from the National Survey of Homeless Assistance Providers and Clients (NSHAPC), the first national-level and comprehensive survey of homeless clients since 1987. The NSHAPC has data on providers in 76 U.S. geographic regions, 52 urban and 24 small/rural cities, and a nationally representative sample of clients served by these providers. It was conducted by the Census Bureau on behalf of 12 federal sponsoring agencies.
PIC ID: 7250; EXPECTED COMPLETION: FY 2002; CONTACT: Lynette Araki, 301-443-6204; PERFORMER: Westat, Inc., Rockville, MD

Development of an Inventory of Health Indicators Comparing Large U.S. Cities

The purpose of this study is to develop a fourth edition of Big Cities Inventory, an inventory comparing mortality, natality, and morbidity data for the nation’s large cities. Local health departments require comparative data to determine the relative progress in their population’s health. Although county data are available, large city health departments often cannot distinguish their own performance from surrounding counties that may have very different socio-demographic and resource characteristics. Data needs will be identified in consultation with members of the National Association of County and City Health Officials (NACCHO) who are from big cities and the previous three editions of the Big Cities Inventory (Chicago Department of Public Health). Issues and barriers encountered in collecting and analyzing comparable data from national and local data sources will be documented. A plan for automating future editions of the Big Cities Inventory will be prepared, including approaches for using Web-based technology to disseminate the report.
PIC ID: 7241; EXPECTED COMPLETION: FY 2002; CONTACT: Michael Millman, 301-443-0368; PERFORMER: Chicago Center for Health Systems Development, Chicago, IL

Safety-Net Provider Capacity for Care to Low-Income Uninsured Patients

The purpose of this study is to evaluate the capacity of health departments, public hospitals, and other community-based, safety-net providers to serve the low-income uninsured. Evidence indicates that these safety-net providers are under increased financial pressure due to Medicaid managed care, reduced state funds for the direct delivery of health care services, and a continued rise in the number of uninsured and under-insured. The capacity of safety-net providers is to be measured in terms of change in revenues to serve the low-income uninsured (e.g., change in operating margins or limits on cash reserves), number and type of patient encounters, and the proportion of services provided. Secondary data analysis and site visits will be conducted in twenty communities, with both urban and rural locations. Secondary data will be drawn from AHA’s and National Association of Public Hospitals’ surveys, NACCHO’s survey of local health departments, HRSA’s data on community and migrant health centers, CDC’s proposed tracking of health departments, and RWJ’s Community Tracking Study of community-based providers. Time series analysis will include three years of the most recent available data. Site visits will collect information on state and local government policies that influence the safety net and the role of local provider organizations in the viability of the safety net. Focus groups and interviews with providers and local public health officials will be held in each of the sampled communities. With emphasis on HRSA-supported programs, this study will help policy makers identify the impact of shifts in health care financing on services provided to low-income uninsured patients. Resources within HRSA’s service delivery program can be reallocated to ensure that those most needing care are served.
PIC ID: 7240; EXPECTED COMPLETION: FY 2002; CONTACT: Alexander Ross, 301-443-1512; PERFORMER: Mathematica Policy Research, Inc., Washington, DC

Indian Health Service

Mission

The mission of the Indian Health Service, in partnership with American Indian and Alaska Native people, is to raise their physical, mental, social, and spiritual health to the highest level.

Evaluation Program

The goal of the Indian Health Service (IHS) is to assure that comprehensive, culturally acceptable, personal and public health services are available and accessible to American Indian and Alaska Native people. The importance of evaluation in supporting this goal has increased significantly in recent years and includes American Indians and Alaska Natives as the primary stakeholders in defining the purpose, design, and execution of evaluations. The stakeholders use the end product of the evaluations, and are typically the population or groups most likely to be affected by the evaluation findings. The IHS has formally adopted the principles of a responsive evaluation practice to address the needs and concerns of American Indians and Alaska Natives.

The evaluation program of the IHS is managed by the Office of Public Health, Staff Office of Planning, Evaluation, and Research, which provides national leadership and consultation for IHS and Area Offices on strategic and tactical planning, program evaluation and assessment, public health and medical services, research agendas, and special public health initiatives for the Agency.

The evaluation needs of the IHS service components are coordinated using two major types of short‑term studies: policy assessments and program evaluation studies. The IHS policy assessments contribute to decision making about budget, legislation, and program modifications and include background information to support the Agency’s initiatives. Evaluation studies are focused at the program level, or Area Offices, and focus on specific program needs.

Each year IHS selects high‑priority health care and management studies for funding through the submission of proposals to headquarters and Area Offices. These proposals are reviewed and rated by a panel of subject‑matter experts, evaluation experts, and IHS staff for concurrence with IHS strategic goals, objectives, and priority areas. The proposals are then prioritized and forwarded to the IHS Director, who reviews the projects that are recommended for funding and determines the respective funding levels.

Completed Evaluations

Hyperlipidemia in Native Americans: Evaluation of Lipid Management Through a Cardiovascular Risk Reduction Program

This evaluation utilized a retrospective chart review of all patients who received treatment at least two times through the Native American Cardiovascular Risk Reduction Program at the Santa Fe Indian Hospital from March 1997 through October 1999, and who had baseline and recent LDL-C values (N=74). The program serves the Native American population of eleven Rio Grande Pueblos and urban Indians with a comprehensive CVD risk reduction program for the treatment and management of dyslipidemias, obesity, inactivity, tobacco use, diabetes, and hypertension. The program team includes licensed pharmacist clinicians and registered dieticians. The goal of the program is for patients to reach National Cholesterol Education Program guidelines through individualized treatment plans that incorporate physical activity, dietary changes, and/or the use of Simvastatin. Based on initial lipid levels and the presence or absence of diabetes, patients were classified into three LDL-C treatment groups: low risk, high risk, and highest risk. Paired t-tests were used to detect significant changes in lipid levels between baseline and follow-up within and across the three risk groups. Significant changes in lipid levels were observed in the high and highest risk groups. Over one-half (55%) of the patients reached and/or maintained their LDLl-C goal for their group.
PIC ID: 7139; AGENCY SPONSOR: Indian Health Service; CONTACT: Debbie Melton, 301-443-4700; PERFORMER: Indian Health Service, Rockville MD

In-Progress Evaluations

Evaluation of the Behavioral Risk Factor Surveillance System’s Results and Their Applicability to the Native Population of Anchorage

This study will determine the relative accuracy, validity and reliability of the Behavioral Risk Factor Surveillance System (BRFSS) risk estimates of the Anchorage Native population, compared with data collected using other techniques that include: (1) door-to-door household surveys, (2) key informant surveys, and (3) intercept data collection from Natives seeking primary care services in Anchorage from the Alaska Native Medical Center and the Primary Care Center. The analyses are revealing those particular risk factors that appear to be most significantly misrepresented by the BRFSS. This factor, combined with a more clinical analysis of those factors that pose the greatest risks to the health and well-being of the Native community, will be used in determining which questions to include on the key informant survey. Due to the nature of the survey, a relatively concise instrument is required, and only those items holding a high priority on one or both of the selection criteria (i.e., representation problems with the BRFSS and high clinical/medical risk) will be included in the key informant survey instrument.
PIC ID: 7137; EXPECTED COMPLETION: FY 2002; CONTACT: Debbie Melton, 301-443-4700; PERFORMER: Indian Health Service, Rockville MD

Alaska Native Teen Tobacco

The purpose of the Alaska Native Teen Tobacco Cessation Project is to (1) help the youth who participate in the project to quit tobacco; (2) motivate the youth to become tobacco prevention and cessation advocates in their communities; (3) determine the effectiveness of cessation camp model in helping youth to quit tobacco. If successful, further application of this model would be promoted. This is a three year project.
PIC ID: 7128; EXPECTED COMPLETION: FY 2003; CONTACT: Debbie Melton, 301-443-4700; PERFORMER: Indian Health Service, Rockville MD

American Indian Family Caregivers and the Provision of Long-Term Care

This project is intended to help develop economically feasible and culturally acceptable long-term care modalities to support Native American families in delivering care to frail elders.
PIC ID: 5940; EXPECTED COMPLETION: FY 2003; CONTACT: Debbie Melton, 301-443-4700; PERFORMER: Indian Health Service, Office of Planning, Evaluation, and Legislation, Rockville MD

Concurrent Evaluation of the Elders Clinic, a Multidisciplinary Geriatric Assessment Clinic at the Zuni-Ramah Service Unit

The purpose of this study is to describe the epidemiologic surveillance capacity of the Indian Health Design Team Indian Health Service (IHS) Provider of Health Care/Tribal Provider of Health Care/Urban Indian Program Provider of Health Care (I/T/U) system. It will also make recommendations as to how the clinical capacity could be improved to better meet the needs of the IHS and its stakeholders. This study is of increasing importance as fewer federal resources will be available to conduct epidemiologic surveillance.
PIC ID: 6741; EXPECTED COMPLETION: FY 2003; CONTACT: Debbie Melton, 301-443-4700; PERFORMER: Indian Health Service, Rockville MD

Dental Caries in Preschool Children

This project will determine the existence of a Baby Bottle Tooth Decay (BBTD) pattern among preschool children clusters in families. It will also determine if dental nutrition education to families with one child with BBTD pattern caries will reduce the rate in subsequent children. Project data will be used to determine if the utility of providing family-directed dental and nutrition education can reduce BBTD clusters in families.
PIC ID: 5531; EXPECTED COMPLETION: FY 2002; CONTACT: Debbie Melton, 301-443-4700; PERFORMER: Indian Health Service, Rockville MD

Eastern Band of Cherokee Indian Health Care Delivery System Assessment

This study will provide the Eastern Band of Cherokees with relative data on their current health care service delivery system. This data is intended for use in a re-design of the system so that it can be more responsive to all client and tribal member needs.
PIC ID: 6408; EXPECTED COMPLETION: FY 2002; CONTACT: Debbie Melton, 301-443-4700; PERFORMER: Nashville Area Indian Health Service, Nashville TN

Effectiveness of Evaluation of Systematic Implementation of Clinical Prevention Protocols for Chronic Disease Risk Reduction

This project will address health prevention and the reduction of chronic disease complications through development of a tracking system for use by the physicians and patients. This tracking system will serve as a model for tribes with health contracts.
PIC ID: 5938; EXPECTED COMPLETION: FY 2002; CONTACT: Debbie Melton, 301-443-4700; PERFORMER: Nashville Area Indian Health Service, Nashville TN

Evaluating the Effects of Medical Nutrition Therapy on Patient Outcomes Among Native Americans with Newly Diagnosed Type II Diabetes Mellitus

This study will: (1) evaluate the effectiveness of medical nutrition therapy (MNT) on clinical/behavioral outcomes in American Indians/Alaska Natives (AI/AN) with newly diagnosed Type II diabetes, (2) create a uniform methodology for collecting nutrition-oriented outcome data that is consistent among participating tribes, service units and urban programs, (3) analyze the type and amount of MNT provided with regard to patient outcomes, and (4) link this information with current clinical data from the IHS Diabetes Program which identifies Continuous Quality Improvement performance measures.
PIC ID: 6746; EXPECTED COMPLETION: FY 2002; CONTACT: Debbie Melton, 301-443-4700; PERFORMER: Indian Health Service, Rockville MD

Evaluating the Impact of Primary Intervention Techniques on the Dental Caries Rate in Children Living in Southwest Alaska Native Villages

This project will attempt to identify the reason why some communities in Bristol Bay have significantly higher/lower caries rates in children than do other children in other Bristol Bay communities. Children ages six to eight have been selected for the project, and the results of the project will be used to identify the factors that create high-risk communities. A community model will be developed for use in allocating specific resources to address the oral health needs of the villages. The primary activities will continue to be directed towards continuing intervention, which includes dental advocate services, tooth brushing, topical flouride and chemotherapy programs, distribution of children’s books, and use of the medical model for dental treatment for one to eight year olds and their mothers.
PIC ID: 7138; EXPECTED COMPLETION: FY 2002; CONTACT: Debbie Melton, 301-443-4700; PERFORMER: Indian Health Service, Rockville MD

Impact of Breast-feeding Education Program on Rates in the Early Postpartum Period and on Duration of Breast-feeding Year 2

The purpose of this project is to study the effect of a specific, planned, consistent breast-feeding education program on rates of breast-feeding initiation, and on duration of breast-feeding in the patient population. The study population includes all live births at the Claremore Indian hospital for one year from the onset of the study. The control will be a historical control, and will consist of all live births in the year prior to the onset of the study, inclusive of March 1, 1997 to April 30, 1998. During this period there were 640 live births. The goal of the study will be to reach the Healthy People 2000 goal of having 75 percent of mothers initiating breast-feeding, and keeping 50 percent breast-feeding for five to six months.
PIC ID: 7141; EXPECTED COMPLETION: FY 2002; CONTACT: Debbie Melton, 301-443-4700; PERFORMER: Indian Health Service, Rockville MD

Impact of a Structured, Comprehensive, Multi-Disciplinary Patient Education Initiative--Year 2

This study will: (1) promote wellness and prevent disease by providing comprehensive, accurate and understandable education to each patient, (2) study the impact of this multidisciplinary, planned patient education program on the health and wellness knowledge base of the patient population, and (3) document positive lifestyle/behavioral changes as a result of the education provided. The study population encompasses all American Indian/Alaska Native (AI/AN) patients eligible for care at the Service Unit (predominantly Cherokees). The Service Unit covers 12 counties in northeastern Oklahoma with 149,939 outpatient visits, accounting for 32,105 patients. This study will be submitted for publication in current medical journals by the Indian Health Service (IHS) provider. Information generated from this study will be disseminated to all tribes in the service area and all IHS facilities.
PIC ID: 7140; EXPECTED COMPLETION: FY 2002; CONTACT: Debbie Melton, 301-443-4700; PERFORMER: Indian Health Service, Rockville MD

Mental Health Service Delivery Model for Urban Native Americans: An Evaluation of Utilization Rates and Mental Health Treatment Factors

This project will address the Indian Health Service (IHS) mental health objective calling for a mental health care delivery system in urban areas. Results of the project will identify a potential model of mental health delivery applicable to urban areas.
PIC ID: 5943; EXPECTED COMPLETION: FY 2002; CONTACT: Debbie Melton, 301-443-4700; PERFORMER: California Area Indian Health Service, Sacramento, CA

Program Assessment of Co-Morbidity of Chemical Dependency Mental Health and Diabetes

The purpose of this study is to determine the outcome of simultaneous treatment of co-occurring chemical dependency, mental health issues and diabetes.
PIC ID: 7029; EXPECTED COMPLETION: FY 2002; CONTACT: Debbie Melton, 301-443-4700; PERFORMER: Indian Health Service, Rockville MD

Resource Requirements Methodology Update

This project will update the existing Resource Requirements Methodology (RRM) necessary to reflect current and future program demands of the Indian Health Service (IHS). A complete revision and documentation of the RRM will involve an update of staffing criteria and modules, formulation of a needs assessment cost model, and needs assessment model training.
PIC ID: 6450; EXPECTED COMPLETION: FY 2002; CONTACT: Debbie Melton, 301-443-4700; PERFORMER: Information and Management Technologies, Inc., Silver Spring, MD

National Institutes of Health

Mission

To sponsor and conduct medical research that leads to better health for all Americans.

Evaluation Program

The National Institutes of Health (NIH) generates scientific knowledge that leads to improved health. This is done by conducting medical research in its intramural laboratories and by supporting research in universities, medical and health professional schools, and other health research organizations. NIH fosters the widespread dissemination of the results of medical research, facilitates the training of research investigators, and ensures the viability of the research infrastructure. The NIH Evaluation Program is an integral part of how NIH sponsors and conducts medical research.

Results based management is recognized as a basic principle for the sound and productive operation of government agencies and their programs. This is evidenced most notably by passage of the Government Performance and Results Act (GPRA) and by the considerable effort across the federal government to implement results based management mechanisms. With GPRA and other initiatives aimed at increasing public sector accountability (such as the Chief Financial Officers Act and the Government Management Reform Act), interest in the use of evaluation has increased steadily among NIH administrators and others, such as officials within the General Accounting Office, the Office of Management and Budget and the Department of Health and Human Services.

Philosophy and Priorities. The NIH Evaluation Program provides information to assist the NIH Director and the NIH Institute and Center (IC) Directors in determining whether NIH goals and objectives are being achieved and to help guide policy development and program direction. Evaluations are planned and conducted from two sources of funds: 1 percent evaluation set‑aside funds used to fund trans‑NIH projects, and IC program funds used for program evaluations for use by various committees, working groups, task forces, workshops, conferences, and symposia to assist the ICs in program management and development. This approach ensures that planning and priority setting specific to the mission of each IC are fully developed and implemented and that there is central leadership for developing crosscutting initiatives and promoting collaboration among the ICs.

NIH’s major evaluation priority areas fall within three broad program areas: basic research, research training and career development, and facilities. NIH conducts evaluations in these areas to assess strategies and goals, develop performance measures and improve operations.

Policies and Operations. A distinguishing feature of the NIH Evaluation Program is the utilization of a variety of evaluation strategies that include the use of national advisory councils, boards of scientific counselors, consensus development conferences, and ad hoc committees that help to chart scientific directions and select the most promising research to support.

A two‑tier system is used to review project requests that will use 1‑percent evaluation set‑aside funding. The first tier involves a review and recommendations by the NIH Technical Merit Review Committee (TMRC) on the technical aspects of project proposals and whether a project fits within HHS guidelines for use of the set‑aside fund. The second tier involves the NIH Evaluation Policy Oversight Committee, which considers TMRC recommendations, conducts policy level reviews, and makes final funding recommendations to the NIH Director or his designee.

Completed Evaluations

NIDCD Web Site Usability Review Report

In May 2001, Human Factors International (HFI) implemented a Usability Review of the NIDCD web site. The testing was designed to identify opportunities for user-centered improvements to the site. The Usability Review process began with a briefing from the NIDCD team outlining the Institute’s goals for the web site, sections of special importance and the anticipated user goals. Based on this input, HFI developed a task-based, empirical review tool to evaluate the user experience. Using these methods, the user experience was evaluated based on direct observation of new and experienced users navigating the NIDCD web site to accomplish tasks that real users of the site frequently complete. Overall, the site evaluated well: Participants located requested information 75% of the time. During the study, interviewers noted points of common confusion or frustration. By exploring these and other user-centered strategies, the NIDCD can significantly enhance the usability of the web site.
PIC ID: 7653; CONTACT: Patricia Blessing, 301-496-9497; PERFORMER: Human Factors International, Fairfield, IA

Full Scale Evaluation of the Regional Primate Research Centers Program

The two objectives of the evaluation of the Regional Primate Research Centers (RPRC) Program were: (1) to assess the Centers’ infrastructures, particularly in terms of efficiency, effectiveness, and the accessibility of RPRC-supported Center resources to outside researchers; and (2) to examine the research performance of the RPRCs (whether high quality scientific research is being performed) using a citation analysis. The final report contains detailed analyses of data collected from each of the seven original RPRCs. An Expert Panel, consisting of 11 members, developed a report based on site visits to the seven RPRCs, findings in the final report, and their own deliberations. The Expert Panel report contained the following recommendations: (1) survey the demand for nonhuman primate research resources: (2) meet needs for specially bred, well-characterized nonhuman primates; (3) change name to National Primate Research Centers (NPRC); (4) establish a standardized central data repository; (5) establish a Primate Resources Coordinating Center; (6) adopt a uniform pricing structure; (7) expand laboratory facilities to accommodate outside users; (8) enhance funding of venture-pilot studies to outside investigators; (9) increase the ceiling on R21 funding to $150,000 per year in direct costs; (10) communicate the improved availability and accessibility of these resources; (11) establish measures of productivity and performance; (12) standardize categories of data compilation across Centers; (13) promote unique Centers of Excellence; (14) strengthen host-RPRC affiliations; and (15) consider directions for future improvements.
PIC ID: 7640; CONTACT: Stephen G. Seidel, 301-435-0866; PERFORMER: James Bell Associates, Inc., Arlington, VA

Assessment of State Health Departments’ Vision Related Programs

The National Eye Institute assessed the current status of State Health Departments’ vision related policy and programmatic efforts. Specific goals for the project were to gather data on each state’s vision position; gather information about each state’s vision programs; and identify ways in which NEI can assist states. In-depth interviews were used to gain the necessary information. Key findings included: half of the health departments had goals, objectives, or laws related to eye health. Thirteen states plus the District of Columbia reported that HP2010 vision objectives will guide their states’ future eye health program. Diabetic eye disease programs are the most common. Recommendations include: 1) create an information exchange network to share eye health information and data with state health departments;

2) promote vision objectives in HP2010 and provide guidance to states in developing vision related activities; and 3) highlight state activities through the National Eye Health Education Program.
PIC ID: 7637; CONTACT: Rosemary Janiszewski, 301-496-5248; PERFORMER: ORC Macro, Calverton, MD 20705

Assessment of Vision-Related Programs and Services for American Indians and Alaska Natives

The National Eye Institute (NEI) assessed the current status of vision related programs and services for American Indians and Alaska Natives--a group that has been identified at high risk for eye diseases. Primary goals of the study were to gather information about the vision related programs and services, conduct a literature search, and identify gaps in eye health information, services, programs, and materials. Information will be used to develop and implement eye health awareness campaigns that target American Indians and Alaska Natives. In-depth interviews were used to collect the information. Major findings included: the Indian Health Service provides a variety of services in their optometry and diabetes programs, other federal agencies provide outreach to this target audience, and few non-governmental agencies provide vision-related services to this population. Recommendations include: develop culturally appropriate educational materials; involve Indian leadership at every level in program development; observe tribal protocols; work with community health representatives; and employ a multifaceted approach, including use of one-on-one patient education, print materials, videos, and translation services.; CONTACT: Rosemary Janiszewski, 301-496-5248; PIC ID: 7638; PERFORMER: ORC Macro, Calverton, MD

Blueprint for Change: Research on Child and Adolescent Mental Health

The National Advisory Mental Health Council established the Child Council Workgroup on Intervention Development and Deployment, with the charge: 1) assess status of NIMH portfolio and identify research opportunities in the development, testing, and deployment of treatment, service, and preventive interventions for children and adolescents in the context of families and communities; 2) assess human resource needs in recruiting, training, and retaining child mental health researchers; 3) make recommendations for strategically targeting research opportunities and infrastructure support to stimulate intervention development, testing, and deployment of research-based interventions. The methodology consisted of NIMH staff and consultant analysis of qualitative and quantitative data. Results of this study indicate that a child’s environment, both in and out of the womb, plays a large role in shaping brain development and subsequent behavior. Studies of the caregiving environment suggest that extreme environments (such as abuse and neglect) may affect brain cell survival, neuron density and neurochemical aspects of brain development, as well as behavioral reactivity to stress in childhood and adulthood. Methods to understand the more subtle effects of the environment on synapses and circuits are likely to become available in the near future.
http://www.nimh.nih.gov/child/blueprin.pdf
PIC ID: 7658; CONTACT: Kimberly Hoagwood, 301-443-3364; PERFORMER: National Institute of Mental Health, Rockville MD

Racial/Ethnic Diversity in Mental Health Research Careers: An Investment in America’s Future

This evaluation examined funding mechanisms used for training underrepresented minorities; the career life cycle (i.e., recruitment, retention, career mobility) of minority scientists; and the impact on minority research careers of undergraduate and university infrastructure development programs. The methodology consisted of an analysis of qualitative and quantitative data by a National Advisory Mental Health Council Workgroup. The findings indicate racial/ethnic minorities are underrepresented in the mental health research workforce. Although corrective measures have been introduced, these have not been in place long enough to assess their impact on (i.e., production of) established principal investigators. Need exists for enhanced tracking of the career status of NIMH-supported research trainees.
http://www.nimh.nih.gov/council/diversity.pdf
PIC ID: 7657; CONTACT: Robert Mays, 301-443-2847; PERFORMER: National Institute of Mental Health, Rockville MD

An Assessment of the Need for a National Clearinghouse on Fetal Alcohol Syndrome

Congress expressed interest in establishing a national clearinghouse to disseminate research-based information on fetal alcohol syndrome (FAS). A literature review identified a number of federal, education, and private nonprofit organizations that provide information about FAS. However, a central repository synthesizing the information contained in these disparate sources and making it available nationally to all races and ethnic groups does not exist. The National Institute on Alcohol Abuse and Alcoholism (NIAAA) commissioned a study to assess the need for a national clearinghouse on FAS. The contractor, IQ Solutions, Inc. conducted in-depth interviews and a telephone focus group with a diverse group of individuals representing organizations touched by FAS to determine whether a national clearinghouse on FAS and fetal alcohol effects (FAE) is warranted. The results of the qualitative analyses indicate clearly that a national clearinghouse on FAS would be valued and welcomed by the study participants. However, discussants also expressed the need for activities and functions that are more typically related to a national education initiative.
PIC ID: 7655; CONTACT: Diane Miller, 301-443-3860; PERFORMER: IQ Solutions, Inc., Rockville, MD

Focus Groups with Recipients of Clinical Research Career Awards/NIH K23 Program: Early Assessment

The NIH conducted a series of six focus groups with junior clinical faculty members who had either received or were eligible for NIH Mentored, Patient-Oriented Research Career Development Awards to examine the benefits of these awards to the career development of the recipients; identify barriers within their academic environment that may impede career development; and the features of the awards that might be modified to expand eligibility or the benefits for the recipients. The method of examination was focus groups. The findings indicate many recipients have difficulty setting aside the necessary time to focus on research. Surgeons find the salaries provided too small and the required commitment too onerous to participate in these awards. The recommendation is to discuss the features of these awards to see if changes need to be implemented.
PIC ID: 7663; CONTACT: Dr. Walter Schaffer, 301-435-2770; PERFORMER: Association of American Medical Colleges, Washington, DC

The Early Career Progress of NRSA Predoctoral Trainees and Fellows

This project was undertaken to examine the career outcomes of predoctoral NRSA trainees. The methodology consisted of a career outcome evaluation. The major findings are: (1) Recipients of nine or more months of National Research Service Award (NRSA) predoctoral training support are more likely to remain in research, to apply for and receive NIH and NSF research grants, than their colleagues that graduate at the same time without NRSA support. (2) They publish more papers and those papers are more highly cited than for individuals in the identified comparison groups.
http://grants.nih.gov/training/outcomes.htm
PIC ID: 6285; CONTACT: Dr. Walter Schaffer, 301-435-2770; PERFORMER: Vanderbilt University, Institute for Public Policy Studies, Nashville, TN

CSR Web Needs Assessment

This study had a two-fold purpose: (1) To examine current CSR Website design for both an intranet and public internet presence with a view to better facilitate communication, the flow of information, and, as appropriate, workgroup processes between and among Center for Scientific Review (CSR) and NIH staff; and (2) To develop recommendations for a redesign that promotes the enhanced productivity of CSR web development teams with an integrated security solution and connections to NIH legacy and prospective systems and technologies. Existing CSR Web sites were evaluated using heuristic evaluation methods, focus groups, and user surveys as inspection methods. The major findings are: The study reported several categories of usability problems with special emphasis on inconsistent page layout and design, the presence of lengthy text not optimized for Web viewing, and the failure to leverage use of accelerators, such as a persistent navigation tool or site search engine on both sites. The primary recommendations included: 1) restructuring site architecture to better group information together in a manner that is logical and intuitive; 2) automating periodic checks for expired links and new redirects; 3) providing a Web portal for users to personalize pages with features they use most frequently as their defaults; 4) improving site navigation and site appearance with the use of appropriate graphics; and 5) improving site accessibility (e.g., with the use of ALT tags).
PIC ID: 7608; CONTACT: Anne Phillips, 301-435-0601; PERFORMER: D. Appleton Company, Fairfax, VA

In-Progress Evaluations

Review of Fogarty Center’s International Cooperative Biodiversity Groups (ICBG) Grants Program

The Fogarty International Center (FIC) has planned a “focused process review” of the International Cooperative Biodiversity Groups (ICBG) grants. The review is part of the oversight FIC employs to assure that its grants continue to be productive over the long term and are sufficiently flexible to remain at the cutting edge of science in the field. The ICBG grants are intended to advance discovery and development of therapeutic agents in the biological environments in developing countries. Work conducted under these grants also addresses biodiversity conservation and the related economic, social, cultural, and legal issues that may either advance or inhibit the identification of new drugs for treatment of human disease and disability. FIC will convene a group of independent experts who will conduct a careful review of the grants program. The experts will be charged with providing observations and recommendations regarding the entire range and scope of the ICBG program grants. The experts recommendations will be put into a report to the FIC. This report will be a critical element in the deliberations on continuing management and direction of the ICBG grants program.
PIC ID: 7624; EXPECTED COMPLETION: FY 2002; CONTACT: Karen Hofman, 301-496-2571; PERFORMER: Dan C. VanderMeer, Chapel Hill, NC

NIDDK National Information Clearinghouse

The National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) launched an online survey in February 2001, seeking comments from the public on the products and services provided by its three information clearinghouses: the National Diabetes Information Clearinghouse (NDIC), the National Digestive Diseases Information Clearinghouse (NDDIC), and the National Kidney and Urologic Diseases Information Clearinghouse (NKUDIC). The following month the survey was also made available in print and by phone to anyone who calls, writes, or emails one of the clearinghouses for health information or other assistance with a health inquiry. Patients continue to make up the largest block of clearinghouse customers at 37.7 percent. A common comment on all clearinghouses, but especially NDIC, requests additional nutrition and diet information. Comments and suggestions received via the customer satisfaction survey will continue to be reviewed individually and implemented, if feasible.
PIC ID: 7636; EXPECTED COMPLETION: FY 2003; CONTACT: Kathy Kranzfelder, 301-496-3583; PERFORMER: Eagle Design & Management, Inc., Bethesda, MD

Cost of Clinical Trials Study

The Cost of Clinical Trials Study (CCTS) is a study of cancer patients throughout the U.S. being conducted by RAND, a private, non-profit research institution based in Santa Monica, California, with principal funding and scientific guidance from the National Cancer Institute (NCI). This study will estimate the incremental costs of medical treatment provided as part of NCI-sponsored protocols. Incremental costs refers to the costs of additional medical resources, if any, provided to patients on protocols above and beyond those that would have been received in the absence of trial participation. The results from this study should be of interest to policymakers, insurers and healthcare decision makers trying to determine appropriate reimbursement for clinical trials. As secondary endpoints, patient satisfaction and health outcomes of patients in trials will be compared with those not in trials.
PIC ID: 7116; EXPECTED COMPLETION: FY 2002; CONTACT: Mary S. McCabe, 301-496-6404; PERFORMER: Rand Corporation, Santa Monica, CA

Evaluation of Internet-based Tools to Improve Cancer Clinical Trials

This project is an outgrowth of a comprehensive review of NCI’s clinical trials program for cancer treatment. Based on this review, NCI has decided to restructure the way in which the Cooperative Groups, the largest single trials program sponsored by NCI, carry out large, randomized clinical trials. The restructuring will occur in a set-wise manner by focusing on a series of demonstration projects, all of which depend heavily on the use of automated applications distributed via the Internet. The evaluation project will carefully compare the new Internet-based methods with the previous methods using descriptive and inferential analyses.
PIC ID: 7482; EXPECTED COMPLETION: FY 2002; CONTACT: Jeffrey Abrams, 301-496-2522; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

Evaluation of Restructuring Efforts to Improve Cancer Clinical Trials System at NCI

This project is evaluating the ability of three pilot projects to re-structure phase III clinical treatment trials for cancer, and improve the quality of the science, the review of proposed trials and the efficiency with which the trials are performed. The projects are: 1) State of the Science Meetings, 2) Concept Evaluation Panels, 3) Cancer Trials Support Unit. These projects use internet based technologies to assist scientists in their tasks. The evaluation will focus on evaluating the utility of the Internet to aid clinicians in the conceptualization and conduct of clinical trials. The evaluation will use web-based surveys, mail surveys, and focus groups.
PIC ID: 7482.1; EXPECTED COMPLETION: FY 2003; CONTACT: Jeffrey Abrams, 301-496-2522; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

Full-Scale Evaluation of the Regional Primate Research Centers (RPRC) Program

The Regional Primate Research Center (RPRC) program represents a commitment by the National Institutes of Health (NIH) to the development and support of regional and national non-human primate research resources for biomedical and behavioral studies. This evaluation is intended to determine the optimum configuration of resources for non-human primate research programs.
PIC ID: 6045.1; EXPECTED COMPLETION: FY 2002; CONTACT: Barbara Perrone, 301-435-0871; PERFORMER: James Bell Associates, Inc., Arlington, VA

Survey of NIH Investigators Who Use Non-Human Primates

The contractor will develop and implement a web-based survey to generate information on barriers to access to Regional Primate Research Centers (RPRC) resources and unmet primate resource needs.
PIC ID: 7639; EXPECTED COMPLETION: FY 2002; CONTACT: Patricia Newman, 301-435-0866; PERFORMER: Humanitas, Inc., Silver Spring, MD

Survey of Scientific and Engineering Research Facilities at Colleges and Universities

Since 1986 and every two years subsequently, the National Science Foundation (NSF) and the National Institutes of Health (NIH) have conducted the Survey of Scientific and Engineering Research Facilities. The availability and condition of biomedical research space directly affects the scope and quality of the biomedical research conducted at the nation’s colleges, universities, medical schools, hospitals and other research organizations. To address the need for information on the amount and quality of S&E research space, Congress mandated that the NSF gather this information and report it to Congress.
PIC ID: 6863; EXPECTED COMPLETION: FY 2002; CONTACT: Fred W. Taylor, 301-435-0766; PERFORMER: National Science Foundation, Arlington, VA

Evaluating Quality Improvement Strategies on Pediatric Asthma Care

This study will compare the treatment and cost-effectiveness of two approaches to childhood asthma-- a disease management program and an office-based quality improvement strategy--in different practices in the same health care system. The investigators are using a disease management program for childhood asthma in a Boston-based integrated delivery system to simultaneously implement an office-based quality improvement strategy in different practices in the same system, and to assess via randomized design their relative effectiveness and cost-effectiveness in improving processes and outcomes of care. After each intervention has been in place for one year, they will offer the complementary intervention to the practice, to determine the marginal benefits and costs that each approach offers. Data will be derived from a parent telephone survey, a provider survey, and a computerized claims database for those asthmatic children in the integrated delivery system who are covered by one of three large managed care organizations.
PIC ID: 7646; EXPECTED COMPLETION: FY 2002; CONTACT: Dr. Virginia Taggart, 301-435-0202; PERFORMER: Children’s Hospital, Denver, CO

Evaluation of Modular Grants and Just-In-Time Experience of NIH Grant Recipients and NIH Personnel

The scope of work for this evaluation includes gathering, organizing, qualifying, and reporting comments, opinions and observations about the implementation of modular grants and Just-In-Time (JIT). Data collection, analysis, interpretation, and reporting will be accomplished through employment of traditional focus group techniques.
PIC ID: 7648; EXPECTED COMPLETION: FY 2002; CONTACT: Cheryl Howard, 301-496-4937; PERFORMER: Society of Research Administrators, Int’l., Arlington, VA

Evaluation of the African American Outreach Component of the Back to Sleep Campaign to Prevent Sudden Infant Death Syndrome (SIDS)

Since the NICHD-led “Back to Sleep” campaign was initiated in the early 1990s, the rate of Sudden Infant Death Syndrome (SIDS) has declined overall by approximately 40 percent. The decline, however, has been less in African American communities. In September 1999 and April 2000, the NICHD in collaboration with the National SIDS Alliance and the National Black Child Development Institute, hosted a meeting of experts to identify, discuss, and plan strategies for reaching African American communities with the “Back to Sleep” messages through a concerted public health campaign. NICHD has initiated an evaluation of the “Back to Sleep” campaign and its special African American outreach efforts. The evaluation findings will provide an opportunity to improve the outreach program and to measure its results.
PIC ID: 7628; EXPECTED COMPLETION: FY 2004; CONTACT: John McGrath, 301-496-5135; PERFORMER: National Institutes of Health, Bethesda, MD

Evaluation of the Extramural Associates Research Development Award Program

In 1994, the National Institutes of Health (NIH) established the Extramural Associates Research Development Award (EARDA) Program to provide grant support to minority and women educational institutions seeking to increase their involvement in biomedical and behavioral research and research training. Since then, the EARDA Program has awarded grants to 44 such institutions throughout the United States. The Office of Extramural Research, Office of the Director, NIH will evaluate the EARDA Program by determining if and how EARDA grants have enabled minority and women educational institutions to increase their engagement in research and research training. This evaluation will employ a multiple, or collective, case study approach in assessing the impact of EARDA funding and provide formative and summative information to help enhance future operations of the program.
PIC ID: 7627; EXPECTED COMPLETION: FY 2004; CONTACT: Mona Rowe, 301-496-1877; PERFORMER: National Institutes of Health, Bethesda, MD

Council for Chemical Research (CCR) Contributions of Chemical Research Study

The chemical research enterprise has long maintained that sustained or enhanced investment in research is critical to industry, academia, and government in helping them meet their overall objectives, including making chemistry one of the substantial contributors to the economic well-being of the nation. Although some activities have been undertaken over the years to assess the value and impact of this research investment, innovation successes are usually described via case studies. There have been few successful attempts to develop a quantitative assessment of the impact of chemical research on and through the chemical enterprise. In assessing the value of chemical research quantitatively, the objective of the commissioned study is to develop new methodologies, assessment tools, and approaches to aid NIH in measuring contributions of chemical research both retro- and prospectively.
PIC ID: 7659; EXPECTED COMPLETION: FY 2002; CONTACT: Dr. James Onken, 301-594-2764; PERFORMER: Council for Chemical Research, Washington DC

NINR Research Outcomes Evaluation

The overall purpose of this evaluation project is to improve the tracking, evaluation, and reporting of the findings that result from NINR-funded research through the creation of a database. The database will provide an organized, accurate, and easily accessible system to determine if the intended outcomes of building depth of science have been achieved. In this first phase of the project NINR is working with CIT at NIH to create the prototype interface through identifying the requirements for the project. The methodology includes meeting with content experts and potential end users to identify the required data fields and data sources. The technical requirements for the system are being defined by learning more about the data sources and their limitations. The user interface prototype will be the final outcome of this first phase. The second phase of the evaluation will be to create and populate the database and begin to query the data to evaluate the results of NINR-funded research. The proposed database is targeted for completion in 2003. It will be designed so it can be updated on an ongoing basis, used to make decisions regarding future NINR initiatives and to evaluate the impact of research on clinical practice.
PIC ID: 7644; EXPECTED COMPLETION: FY 2002; CONTACT: Ann Knebel, 301-402-6796; PERFORMER: Center for Information Technology, Bethesda, MD

Evaluation Feasibility Study for the National Institute on Aging’s Booklet/Videotape, Exercise: A Guide from the National Institute on Aging

The overall goal of this feasibility project is to determine if it is possible to devise an evaluation that can accurately speak to the validity of this assumption--that is, are there performance measures that will reveal whether or not the goal of influencing sustained exercise is being achieved. In addition, this project will consider if there is adequate justification to conduct a large-scale outcome evaluation and if so, what is the most appropriate approach to use. Finally, this project proposes to identify the best way to proceed to implement an outcome evaluation without imposing an excessive burden on program staff or on the public. This introductory study will systematically assess whether or not creating and implementing an evaluation of the exercise book/video is viable and cost-effective. The project will look at target populations, marketing, outcome assessments, and resources required to undertake a full-scale evaluation. It will collect and analyze relevant components from the scientific literature, consider data collection instruments, and pilot tests.
PIC ID: 7642; EXPECTED COMPLETION: FY 2002; CONTACT: Freddi Karp, 301-496-1752; PERFORMER: Academy for Educational Development, Washington, DC

Development of a Strategy to Estimate Expenditure Weights for the Extramural Activities Component of the Biomedical Research and Development Price Index (BRDPI)

This project’s purpose is to develop a plan to estimate expenditure weights for the extramural activities included in the Biomedical Research and Development Price Index (BRDPI). The estimation will require a survey of a representative sample of NIH awards or grantee and contract institutions to determine how expenditures are distributed over input categories such as salaries and wages, equipment, supplies, and patient care. The contractor must determine a survey design and methodology for gathering the data and for estimating the expenditure weights. An estimate of the cost of carrying out the study must also be provided.
PIC ID: 7629; EXPECTED COMPLETION: FY 2002; CONTACT: Jim Schuttinga, 301-496-2229; PERFORMER: Joel Popkin & Company, New York NY

Intramural Research Personnel Surveys-Part II-Postdoctoral Fellows Mentoring Survey

The purpose of this study is to determine the quality and quantity of the mentoring received by NIH intramural postdoctoral fellows. Issues that will be addressed are the source of primary mentoring; quality of scientific direction provided; type and extent of scientific evaluation provided; networking opportunities provided; quality and quantity of career development advice provided; expectations for mentoring experience. A questionnaire was developed in collaboration with an epidemiologist and statistician in NCI/DCEG. NCS Pearson printed and mailed the survey to a random statistically chosen sample of fellows and followed up several times.
PIC ID: 7660; EXPECTED COMPLETION: FY 2002; CONTACT: Joan Schwartz, 301-496-1248; PERFORMER: NCS Pearson, Irving, TX

Administration of Surveys to Neuroscience Study Section Applicants and Program Review Staff

Public Law 103-32 transferred most of the grant review responsibilities of former ADAMHA Institutes, including NIMH and NIDA, to CSR. To accommodate this change and improve the quality of neuroscience review, twenty-one (21) new CSR neuroscience study sections were created in 1998 to review neuroscience grant applications. To assess the impact of this transformation and help determine if additional modifications of the existing review structure are warranted, CSR is conducting a survey of neuroscience grant applicants and Institute/Center program staff. The surveys solicit opinions regarding the quality of review policies and the processing and review of neuroscience grant applications.
PIC ID: 7643; EXPECTED COMPLETION: FY 2002; CONTACT: Janet Healy, 301-496-9291; PERFORMER: Humanitas, Inc., Silver Spring, MD

Computing Frontiers: Prospects from Biology

The Computer Science and Telecommunications Board (CSTB) of the National Academy of Sciences’ Commission on Physical Sciences, Mathematics, and Applications is exploring the intersections of computer science and biomedical research, with the goal of finding interdisciplinary opportunities and new directions in which to extend information technology-related research. Through workshops and meetings of experts in the fields of computer science and biology, they will examine a wide range of research areas, describe the current research landscape (including current NIH-supported research), and provide recommendations for future planning.
PIC ID: 7626; EXPECTED COMPLETION: FY 2003; CONTACT: Dr. James Onken, 301-594-2764; PERFORMER: National Academy of Sciences, Washington, DC

Office of the Assistant Secretary for Planning and Evaluation

Mission

To provide analytical support and advice to the Secretary on policy development and assist the Secretary with the development and coordination of department wide program planning and evaluation activities.

Evaluation Program

The Assistant Secretary for Planning and Evaluation (ASPE) functions as a principal advisor to the Secretary on policy development and, in this capacity, conducts a variety of evaluation and policy research studies on issues of national importance. ASPE also is responsible for department wide coordination of legislative, planning, and evaluation activities. In its evaluation coordination role, ASPE has the following tasks:

• Provide annual guidance to all HHS agencies and staff offices regarding evaluation priorities, procedures, and review requirements.
• Review evaluation priorities proposed by HHS agencies, providing advice about the focus or method of proposed projects and identifying opportunities for collaboration and effective use of resources.
• Prepare planning and summary reports on evaluation activities as required by Congress.

Through the departmental evaluation planning process, ASPE has the capacity to identify crosscutting program or policy issues of particular concern to the Secretary and specific program and policy areas not covered by the HHS Agency evaluation plans. In these instances, ASPE initiates evaluations or collaborates with the agencies to conduct evaluations or policy assessments.

Another continuing evaluation objective of ASPE is to support and promote the development and improvement of databases that HHS agencies and ASPE use to evaluate health care programs and health trends. For example, ASPE has been the major initiator, in collaboration with the National Center for Health Statistics at the Centers for Disease Control and Prevention, of the first comprehensive survey of people with disabilities in the United States. The first component of these new data was completed in FY 1996, and national prevalence data on disability are now available.

The ASPE chairs the Research Coordination Council (RCC), which will evaluate Department-wide research priorities to ensure that efficiencies are realized and research funding priorities are consistent with the Administration’s priorities. The ASPE also co‑chairs and provides support to the HHS Data Council, which is charged with integrating key national surveys, such as linking health status indicators with indicators of well‑being.

Finally, ASPE uses evaluation funds to promote effective use of evaluation‑generated information in program management and policymaking. The latter is accomplished through the dissemination of evaluation findings and other activities, such as providing technical assistance to agencies in the development of performance measures.

Completed Evaluations

A Study of Screening and Assessment in TANF and Welfare to Work

This study, jointly funded with the Administration for Children and Families, highlights critical issues in the development and use of screening and assessment tools designed to identify TANF and/or welfare-to-work (WtW) recipients who experience barriers to employment. The barriers of specific interest for this study include substance abuse, mental health or illness, low basic skills, physical/developmental disabilities (including learning disabilities), and domestic violence. The project provided information about: (1) The screening and assessment tools used by experts in the field to identify each of the barriers to employment; 2) The screening and assessment process states and localities are using to identify and serve the target populations and how and why the states chose particular screening and assessment tools; and 3) The linkage between screening and assessment and referral to services, and the types of coordinated services that are being provided to TANF/WtW recipients. Question one notes the lack of a common terminology between TANF agencies and their partners. Question two provides an overview of incentives to screen or assess clients within TANF as well as a review of prevalence estimates for these barriers. Question three builds on this discussion, outlining key aspects of TANF policy that provide TANF agencies flexibility in how they meet the needs of TANF clients. Questions 4 ‑ 10 are as follows: (4) How can the case management process aid in identifying unobserved barriers to employment? (5) Are there tools that can be used to identify barriers to employment? (6) When should screening or assessment (S&A) occur? (7) Who should conduct the S & A? (8) What training issues are related to S&A? (9) What issues related to privacy and confidentiality should be considered? (10) What other questions should be asked? This paper is a first step in identifying barriers to employment. The second phase of the study will consist of conducting case studies to further explore how these issues are addressed in a select number of localities.
http://aspe.hhs.gov/daltcp/reports/scrnasmt.pdf
PIC ID: 7553; CONTACT: William Marton, 202-690-6443; PERFORMER: Urban Institute, Washington, DC

Indicators of Welfare Dependence: Annual Report to Congress 2001

The Welfare Indicators Act of 1994 requires the Department of Health and Human Services to prepare annual reports to Congress on indicators and predictors of welfare dependence. Each annual report on Welfare Indicators is developed with the advice and recommendations of the bipartisan Advisory Board on Welfare indicators, the assistance of the U.S. Department of Agriculture, the Social Security Administration and the U.S. Bureau of the Census. The purpose of this report, as stated in the law, is “to provide the public with generally accepted measures of welfare receipt so that it can track such receipt over time and determine whether progress is being made in reducing the rate at which and, to the extent feasible, the degree to which, families depend on income from welfare programs and the duration of welfare receipt.” Drawing on various data sources, including the Current Population Survey (CPS), this report provides a number of key indicators of welfare recipiency, dependence, and labor force attachment. Selected highlights from the many findings include: (1) In 1998, 3.8 percent of the total population was dependent in the sense of receiving more than half of total family income from TANF, down from 5.8 percent in 1993. (2) The drop in dependency parallels the more well-known drop in AFDC/TANF and food stamp caseloads. (3) In an average month in 1998, more than half (56 percent) of TANF recipients lived in families with at least one family member in the labor force. (4) Long-term dependency is relatively rare. Only 4 percent of those who were recipients in 1982 received more than 50 percent of their income from AFDC and food stamps in nine or more years over a ten-year period.
http://aspe.hhs.gov/hsp/indicators01/index.htm
PIC ID: 7281; CONTACT: Julia Isaacs, 202-690-6805; PERFORMER: Office of the Assistant Secretary for Planning and Evaluation

Supportive Health Services Needs of Children with Disabilities

The disability supplement to the National Health Interview Survey (NHIS-D) is the first comprehensive survey on disability in the United States. The survey is unique in that it focuses on several populations of persons with disabilities who are generally omitted or under-represented in national survey efforts: children, persons with mental retardation and other developmental disabilities and the working-age population. The goal of this overall project is to conduct a series of analyses and produce both short-term products and final reports that will inform ongoing Assistant Secretary for Planning and Evaluation (ASPE) departmental and administrative research and policy initiatives. This study focused on a subset of health-related services that are used almost exclusively by children with disabilities, namely, supportive services. The major finding is that policies focused on low-income children will reach the majority of children with disabilities who have unmet supportive service needs. These needs range widely across types of services, however. Reaching these children is dependent on how comprehensive the service coverage is. If policymakers wish to target the largest group of children with disabilities, they will do so by focusing on the nearly 90 percent of children with disabilities who have a learning limitation. Policies focusing on this group could potentially reach 3.5 million children, almost 13 percent of whose need for supportive services is unmet. Such policies would also potentially reach a proportion of children with other types of disabilities as well, since 30 percent of children with learning limitations have other functional limitations as well.
http://aspe.hhs.gov/daltcp/reports/suphsnd.htm
PIC ID: 7153; CONTACT: William Marton, 202-690-6443; PERFORMER: Urban Institute, Washington, DC

Clarifying the Definition of Homebound and Medical Necessity Using OASIS Data: Final Report

The purpose of this project was to develop and test a set of uniform and reliable indicators that can be systematically used to document and monitor two Medicare home health care coverage criteria: the “Homebound” and the “Medical Necessity” criteria. The indicators will be linked to items from the Outcome and Assessment Information Set (OASIS) in a decision tree algorithm (or a logic model). Two decision algorithms will be developed for the two criteria. The algorithms will represent a major step toward helping providers, CMS, and the intermediaries in administering the home health benefit. Findings from the study show that the two OASIS algorithms successfully identify patients highly likely to meet the homebound and medical necessity criteria for medicare home health care. Using OASIS data alone, almost 90% of the 600 patients in the sample were classified as meeting the medical necessity criterion. The figure for the homebound criterion is lower (48.5%). It should be noted, however, that individuals may be clearly eligible for home care benefits even though they are not captured by the OASIS algorithms. Nurses who were experts in chart review, in fact, concluded that over half of the patients who were not captured by the homebound algorithm did meet the requirement based on a careful review of patients’ medical records.
http://aspe.hhs.gov/daltcp/reports/OASISfr.htm
PIC ID: 7557; CONTACT: Kamal Hijjazi, 202-690-6443; PERFORMER: Visiting Nurse Service, New York, NY

Private Payers Serving Disabled Individuals and Research Synthesis on Managed Care for Persons with Disabilities

This study was undertaken in order to provide a better understanding of the role of private insurance plans in caring for people with chronic conditions and disabilities. It estimates the prevalence of chronic illness, analyzes the factors affecting the choice of indemnity plans versus managed care, and estimates the impact of managed care on service use and expenditures. The study also investigates the leading risk adjustment systems as a possible method for paying plans more appropriately in serving this population. The four major questions answered by this research are: (1) What is the prevalence of chronic illness and disability among the population enrolled in employer-based health insurance plans? (2) Are individuals with chronic illness more or less likely to choose managed care or indemnity plans, if given a choice? (3) What is the impact of health plan type on utilization and expenditures, taking into account differences in the populations that enroll in different types of insurance plans? (4) To what extent can risk-adjustment systems help employers and health plans predict expenditures of their chronically ill and disabled enrollees?
http://aspe.hhs.gov/daltcp/reports/privpay.htm
PIC ID: 6398; CONTACT: John Drabek, 202-690-6443; PERFORMER: MEDSTAT Group, Cambridge, MA

Screening and Assessment in TANF/Welfare-to-Work: Ten Important Questions TANF Agencies and Their Partners Should Consider

Changes to the welfare system brought about by the Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA), carries serious implications for TANF recipients with disabilities and barriers to employment. There is general agreement among TANF agencies that larger proportions of clients with difficulties transitioning from welfare to work are the “Hard-to-serve.” Given the employment focus and time-limited nature of TANF, there is increased interest in screening and assessment approaches that can be used to identify these barriers to employment. The Department of Health and Human Services asked The Urban Institute to explore the issues and challenges related to screening and assessment within the TANF context. This paper identifies ten of the important questions that should be considered by TANF agencies and their partners as they develop approaches to screening and assessing for barriers to employment, such as substance abuse, mental illness, low basic skills, developmental disabilities, and domestic violence. The second phase of this study will consist of conducting case studies to explore how these issues are addressed in a select number of localities. The ten questions include, for example: How can the case management process aid in identifying unobserved barriers to employment? Who should conduct screening or assessment and when should it occur? What issues related to privacy and confidentiality should be considered?
http://aspe.hhs.gov/daltcp/reports/scrnasmt.pdf
PIC ID: 7538; CONTACT: William Marton, 202-690-6443; PERFORMER: Urban Institute, Washington, DC

Understanding Medicaid Home and Community Services: A Primer

This Primer is designed to encourage use of the Medicaid program in a manner that minimizes reliance on institutions and maximizes community integration in a cost-effective manner. It describes the many options states have to use the Medicaid program to fund long-term care services and supports. Its intended audience is policymakers and others who wish to understand how Medicaid can be used--and is being used--to expand access to a broad range of home and community services and supports, and to promote consumer choice and control. In addition to comprehensive explanations of program features states can implement to achieve these goals, the Primer presents examples of state programs that have taken advantage of Medicaid’s flexibility to expand home and community services for people of all ages with disabilities. For every Medicaid service, states must answer two basic questions: (a) how to define medical necessity, and (b) how to manage overall utilization. Federal Medicaid law and policy, however, give states great latitude to offer individuals a wide range of home and community services through the states “regular” Medicaid program. States can offer a more comprehensive service range by operating one or several home and community based services waiver programs. The design of the Primer grew out of a series of discussions among federal officials, state policymakers, service providers, and advocates regarding how to make the document as useful as possible. Each chapter provides an annotated bibliography, with full information on how to obtain each publication. The report is designed to serve as a reference guide.
http://aspe.hhs.gov/daltcp/reports/primer.htm
PIC ID: 7762; CONTACT: Gavin Kennedy, 202-690-6443; PERFORMER: George Washington University, Center for Health Policy Research, Washington, DC

Teen Risk-Taking: A Statistical Portrait

This is the second of two projects that analyzed data from several major Departmental surveys of youth (the National Longitudinal Study on Adolescent Health, the Youth Risk Behavior Survey, and the National Survey of Adolescent Males) regarding the clustering or co-occurrence of youth risky behaviors. Both projects were conducted for ASPE by The Urban Institute. The first project produced three working papers that were synthesized into a single document entitled: “Teen Risk-Taking: A Statistical Portrait” which is available through the Urban Institute web site at www.urban.org (search by first author of the report - Laura Duberstein Lindberg). The second project produced four working papers that are available in the PIC. The objective of this research was to go beyond demographic data to identify vulnerability and protective factors related to profiles of risk which co-occur with health risk behaviors, such as alcohol use, binge drinking, cigarette use, sexual activity, marijuana use, other illicit drug use, fighting, and suicide. The sample includes 12, 578 students in grades 9 throught 12 from the National Longitudinal Study. Results indicate that profiles of risk are related to vulnerability and protective factors in the individual psychosocial adjustment, school, and family domains. Students in the lowest risk profiles reported consistently higher levels of protective factors and lower levels of vulnerability factors than students in any other risk profile. Students in profiles consisting of substance using, sexually active teens reported higher levels of protective factors and lower levels of vulnerability factors than both the highest risk profiles and the profiles distinguished by suicidal thoughts and behaviors.
PIC ID: 7349; CONTACT: Kevin Hennessy, 202-690-7272; PERFORMER: Urban Institute, Washington, DC

Assessing Core Capacity for Infectious Diseases Surveillance

This project was designed to develop an investment plan to improve the nation’s core capacity for infectious diseases surveillance in light of recent changes. These challenges include the changing nature of biological threats--including bioterrorism and emerging infections--the rapid pace of technological development, and changes in the health care delivery system. The investment plan that evolved is based on an analysis of recent literature, interviews with over 50 experts in the field, and validation through direct observation of the capacity currently in place for surveillance in three locations across the country. The researchers present a three pronged strategy for improving core capacity for infectious disease surveillance. Each of the three prongs has two major components along with a package of specific activities that relate to each component. The priority investment areas are: 1) Improve State and Local Preparedness to Identify and Respond to Infectious Disease Threats. (2) Engage the Health Care Delivery System and the Public as Partners in Infectious Diseases Surveillance. (3) Enhance Public and Private Laboratory Support for Infectious Diseases Surveillance.
PIC ID: 7161; CONTACT: Caroline Taplin, 202-690-7906; PERFORMER: The Lewin Group, Fairfax, VA

Assessing the Individual Health Insurance Market in the Post-HIPAA Era: A Review of the Literature

This paper reviews the available literature describing the individual health insurance market and the impact of state regulation on this market. By any measure, the individual health insurance market is much smaller than the group market. The number of people under age 65 who report individual coverage (about 16 million) is only about 10 percent as many as those who report employer group coverage (158 million). Despite its small size, the individual health insurance market plays an important role in a voluntary, private-public “system” of health insurance. For people without access to either employer coverage or public coverage, the individual market is their only source of insurance. Prices in this market are notoriously high. Administrative costs for individual coverage (measured as insurance loadings on medical benefits) are commonly a multiple of (according to one estimate, three times) those in the group market. The small size of the individual market allegedly makes it “fragile.” In light of the importance of the individual market to consumers who rely on it, its fragility is highly problematic. In addition, the cost of coverage in this market remains a principal problem. The study makes these recommendations: Potential areas for reform--guaranteed issue and some constraints on insurer products and pricing to ensure stability--may offer the most significant opportunities to assist consumers in the individual health insurance market. By guaranteeing access to coverage and making coverage more understandable, they would greatly improve consumers’ ability to compare insurance policies and prices.
PIC ID: 7758; CONTACT: Stephen Finan, 202-690-7387; PERFORMER: Mathematica Policy Research, Inc., Plainsboro, NJ

Assessment Approaches to Evaluating Telemedicine Programs and Projects

The purpose of the study was to extend the 1996 Institute of Medicine framework for telemedicine evaluation as it applies to teleconsultations. The purpose of this study was not to evaluate telemedicine, but rather to identify the different kinds of issues on which telemedicine evaluations can focus, and the kinds of information that such evaluations can yield. This report is intended to guide evaluators and policy makers in selecting the questions that they want to answer regarding the value of telemedicine programs, and in designing evaluations that will best serve their interests and purposes. There are a number of major findings, including: 1) A fundamental consideration in evaluating a telemedicine application is specifying a minimum set of elements such as purpose, target audience, and the scope or focus of the evaluation; 2) Patient satisfaction with telemedicine has consistently been demonstrated to be high. As such, resources for future evaluations may be better allocated to areas of higher priority. 3) Lack of reimbursement for telemedicine services has been a significant confounder in past evaluations of telemedicine. Future evaluation efforts should seek to establish comparable reimbursement environments for telemedicine and the usual care comparators whenever differences in reimbursement might affect study results.
PIC ID: 7340; CONTACT: Tom Hertz, 202-690-7779; PERFORMER: The Lewin Group, Falls Church, VA

Construction of an Integrated Measure of the Burden of Disease: Final Report

This study examined one of the main challenges in measuring the burden of disease: the fact that the burden of disease has multiple components (medical costs, reduced quality of life, reduced longevity, productivity loss, etc). The report includes a review of relevant literature, some discussion of various empirical approaches to measuring trade-offs between the various components of the burden of disease within health systems, and consideration is given to the problem of comparing performance across health systems when the outcome has multiple components. The report sums up as follows: It seems critical to recognize that the observed relationship between the multiple components of healthcare outcomes, including cost, are a product of the interplay between production and consumer preference factors when considering the burden of disease. The entire relationship is of relevance when comparing the effectiveness of health systems, not just individual outcomes. Yet our ability to understand how this relationship is determined, in theory, and to appropriately estimate it in practice is limited. Because costs are jointly determined with other outcomes, estimation of cost regressions is not the right approach. Approaches that treat cost and other outcomes symmetrically yield different answers, but there are many such approaches and no clear criteria to select among them.
PIC ID: 7142; CONTACT: Kevin Hennessy, 202-690-7272; PERFORMER: The Lewin Group, Fairfax, VA

Decliners of Employer Based Health Insurance: An Analysis Using the National Health Interview Survey

The purpose of this study was to examine the characteristics of workers who decline offers of health insurance coverage from their employers (“decliners”) and remain uninsured. Using the National Health Interview Survey (NHIS), this project focused on how decliners compare to other relevant employee groups (e.g., uninsured workers not offered health insurance coverage) and how the decliner population changed during the 1990’s. In addition to comparing the usual socioeconomic variables (such as age, sex, etc.), this analysis looks at decliners’ health status, health conditions and utilization of health care services. According to the NHIS, both employer-based insurance (ESI) offer and take-up rates fell between 1993 and 1997, which appears to imply that decliners of ESI have become, and will continue to be, more important in the pool of the uninsured. Length of spells of uninsurance seem to be decreasing for those decliners without coverage and their spell lengths are significantly shorter than those of workers who are uninsured and have no ESI offer. Still, in 1997, 57 percent of uninsured decliners had been without coverage of any kind for at least a year. Over time, decliners appear to be getting somewhat older, and less likely to be married or to have children. Oddly, they also seem to be getting more highly educated. In terms of health status, the decliners fell between ESI takers and those not offered in terms of the probability that at least 1 person in their family was in fair or poor health, although the probability of a sick family member was decreasing over the time period studies. Interestingly, decliners were less likely than those not offered to have had a recent doctor’s visit, and more likely to have not had one in 3 or more years. Finally, decliners were substantially more likely than ESI takers to report problems related to mental health conditions.
PIC ID: 7345; CONTACT: Bernadette Fernandez, 202-401-8398; PERFORMER: Urban Institute, Washington, DC

Evaluation of the Projects for Assistance in Transition from Homelessness (PATH) Grant Program

PATH is a formula grant program that provides funds to States and Territories for flexible, community-based services for persons with serious mental illnesses who are homeless or at imminent risk of becoming homeless. The findings are as follows: 1) Grantees demonstrated that PATH monies are used in a manner that is fully consistent with the authorizing federal legislation. Provider agencies use the PATH dollars, in combination with other state and local funds, to offer a wide array of appropriate services to members of the target population. 2) The evaluation confirmed the importance of PATH’s contribution to the overall service delivery system for people who are homeless and have serious mental illnesses. 3) Funding application material indicates that in FY 1999, 90 percent of the sample agencies offered outreach services with the support of PATH funds. 4) PATH-supported outreach contacted 115,217 individuals in FY 1999. 5) Annual report data indicate that 37 percent (42,441 clients) of the 115,217 individuals contacted became enrolled PATH clients.
PIC ID: 7210; CONTACT: Kevin Hennessy, 202-690-7272; PERFORMER: R.O.W. Sciences, Inc., Rockville, MD

Factors Influencing Effective Dissemination of Prevention Research Findings by DHHS: Final Report

Disease prevention and health promotion activities are receiving increased attention by the Department of Health and Human Services and its partners. This report addresses both primary and secondary prevention efforts. Primary prevention denotes interventions taken to prevent the development of a disease in individuals who do not have the disease in question. Secondary prevention denotes the identification of individuals who have developed a disease at an early stage in the disease’s natural history, followed by early intervention to prevent mortality or complications of the disease. Four major tasks and deliverables of this project, including a literature review, discussions with technical advisors (personnel from federal government agencies involved in prevention) and dissemination partners (representatives of professional groups, foundations, and communications media partnering with federal agencies), case study reports, and an expert panel meeting, served as inputs into this report. HHS roles in prevention activities are described based upon these inputs. The report then identifies factors critical to processes for the effective translation of research findings into clinical and public health practice and into individual behaviors that promote health.

This research effort determined that to develop and advance its prevention initiative, HHS must consider demographic, social, economic, and technological trends, and anticipate how these will affect priorities for prevention research translation, dissemination, and intervention. Further, HHS must account for the effects of these trends on the identity, needs, and means for reaching at-risk populations and other target audiences for these initiatives. In order to increase the likelihood of success, HHS must look for potential partners in the federal government, state governments, and the private sector who can provide market access, financial support, and other complementary advantages. Finally, ongoing evaluation of these functions will enable continued improvement of each prevention initiative and building a growing knowledge base of what works to improve the processes and health outcomes of preventive services.
PIC ID: 7347; CONTACT: Andrea Pernack, 202-690-7807; PERFORMER: The Lewin Group, Falls Church, VA

America’s Children: Key National Indicators of Well-Being 2001

This report was developed by the Federal Interagency Forum on Child and Family Statistics. It is the fifth annual synthesis of information on the status of the Nation’s most valuable resource, our children. This report presents 24 key indicators of the well-being of children. These indicators are monitored through official federal statistics covering children’s economic security, health, behavior and social environment, and education. The report also presents data on eight key contextual measures and includes two indicators as special features: asthma prevalence and youth employment. The 20 agencies of the Forum have also introduced improvements in the measurement of several of the indicators presented last year. The report is intended to stimulate discussions by policy-makers and the public, exchanges between the data and policy communities; and improvements in federal data on children and families.
http://www.childstats.gov/americaschildren/
PIC ID: 6170.5; CONTACT: Martha Moorehouse, 202-690-6939; PERFORMER: Westat, Inc., Rockville, MD

Children and Welfare Reform: A Guide to Evaluating the Effects of State Welfare Policies on Children

In the last 10 years, policymakers and researchers have expressed a growing interest in understanding how and to what extent welfare policies affect children’s health, cognition, school achievement, and social and emotional development. The purpose of this guidebook is to share the results of a unique project designed to assist States in measuring child outcomes in the context of welfare reform programs. The Project on State-Level Child Outcomes consisted of representatives of State welfare agencies, federal government representatives, and researchers working together to identify an important set of child outcomes and ways to measure them. The project proceeded in two phases: The Planning Phase - the primary goal of this phase was the selection of a common design and a common set of family and child well-being outcomes that could be measured in state welfare evaluations. Several meetings were organized and facilitated by Child Trends, Inc. Twelve States participated in this phase (California, Connecticut, Florida, Illinois, Indiana, Iowa, Michigan, Minnesota, Ohio, Oregon, Vermont, and Virginia). The second phase, The Operational Phase, began in the fall of 1997 and will continue for at least three years. Five States are participating in this phase (Connecticut, Florida, Indiana, Iowa, and Minnesota). They are diverse in their welfare policies and programs, recipient populations, geography, and political climate. This guidebook condenses and integrates the materials, discussions, and products from each phase of the project to provide the tools needed to initiate or augment a study of child well-being in the context of welfare reform.
PIC ID: 7590; CONTACT: Martha Moorehouse, 202-690-6939; PERFORMER: Child Trends, Inc., Washington, DC

Evaluating Two Approaches to Case Management: Implementation, Participation Patterns, Costs, and Three-Year Impacts of the Columbus Welfare-to-Work Programs

This is the latest in a series of evaluations of 11 welfare-to-work programs in seven sites across the nation. This report presents results from Columbus (Franklin County), Ohio. Two case management approaches (traditional with two case workers, one for income maintenance, the other for employment and training) and integrated case management (one case manager for both income maintenance and employment-related issues) were evaluated. The report provides information on how the integrated and traditional programs were implemented, how they affected participation in employment-related activities, and the costs of providing employment-related services in the two programs. To facilitate this assessment, from 1992 to 1994 over 7,000 single-parent welfare applicants and recipients were randomly assigned for the evaluation. The key findings include, but are not limited to, the following: 1) Integrated case managers provided more personalized attention than traditional case managers and more closely monitored participation in program activities; 2) The integrated program engaged more people in welfare-to-work activities than the traditional program; 3) Both Columbus approaches increased earnings by about $1,000, or 10 percent, relative to the control group average; 4) Neither program increased sample members’ average combined income from earnings, cash assistance, and Food Stamps; and, 5) For sample members who did not have a high school diploma or GED when they entered the study, the integrated program produced larger earnings gains and welfare reductions than the traditional program.
http://aspe.hhs.gov/hsp/NEWWS/columbus01/index.htm
PIC ID: 7089.6; CONTACT: Audrey Mirsky-Ashby, 202-401-6640; PERFORMER: Manpower Demonstration Research Corporation, New York, NY

Evaluation of Family Preservation and Reunification Programs: Interim Report

This is a report of an evaluation of programs intended to prevent the placement of children in foster care when it can be avoided. This report focuses on programs in three states, using a particular approach to family preservation, Homebuilders, thought by many to be the most promising approach. Homebuilders, a foster care placement prevention program developed in 1974 in Tacoma, Washington, calls for short-term, time-limited services provided to the entire family in the home. Key program characteristics include: contact with the family within 24 hours of the crisis, caseload sizes of one or two families per worker, service duration of four to six weeks, provision of both concrete services and counseling, and up to 20 hours of service per family per week. The programs evaluated were in Louisville, KY; seven counties in New Jersey; and Memphis, Tennessee. Did the study meet the goals of reducing foster care placement, maintaining the safety of children, and improving family functioning? No significant differences were found between the experimental and control groups on family level rates of placement, case closings, or subsequent maltreatment. It was also found that family preservation programs in two states resulted in higher assessments by clients of the extent to which goals have been accomplished and of overall improvement in their families' lives.
http://aspe.hhs.gov/hsp/hspyoung.htm
PIC ID: 5337.3; CONTACT: Martha Moorehouse, 202-690-6939; PERFORMER: Westat, Inc., Rockville, MD

Providing Mental Health Services to TANF Recipients: Program Design Choices and Implementation Challenges in Four States

This report profiles the efforts of four States (Florida, Oregon, Tennessee, and Utah) to address the mental health needs of welfare recipients. This study was designed (1) to identify and provide detailed information about the design and structure of mental health services developed by state and local welfare offices to address the mental health needs of welfare recipients, (2) to highlight service delivery options in designing and implementing these services, and (3) to discuss the key implementation challenges and lessons learned in providing mental health services to welfare recipients. The researchers visited one rural and one urban site in each state. The sites provided services in a variety of ways, and the research did not collect any evidence that suggests that one model for providing mental health services is better than any other. Rather, the project was designed to draw attention to the key design and implementation choices made by policymakers and managers in providing mental health services for welfare recipients.
http://aspe.hhs.gov/hsp/TANF-MH01/
PIC ID: 7618; CONTACT: Elizabeth Lower-Basch, 202-690-6808; PERFORMER: Mathematica Policy Research, Inc., Plainsboro, NJ

Trends in the Well-Being of America’s Children and Youth 2000

This report presents the fifth edition of a collection of national estimates of child and youth well-being. Trends are presented from the 1970s through the 1990s. The report presents the most recent and reliable estimates on more than 80 indicators which have been organized into these five broad areas: 1) Population, family, and neighborhood; 2) Economic security; 3) Health conditions and health care; 4) Social development, behavioral health, and teen fertility; and 5) Education and achievement. Some sections of this report, mostly sections one and two, have been significantly revised since the 1999 edition of this report. Some indicators have been combined while others have been rewritten or replaced with new sources of data. A new indicator has been added on the number of births in the US, while several have been removed. The indicators that have been removed are: children as a percentage of the dependent population; long-term childhood poverty; effect of government cash and near-cash transfer programs on poverty among persons living in families with children under 18; long-term welfare dependence; parental labor force detachment; secure labor force attachment; and lead exposure.
http://aspe.hhs.gov/hsp/00trends/index.htm
PIC ID: 6170.4; CONTACT: Martha Moorehouse, 202-690-6939; PERFORMER: Westat, Inc., Rockville, MD

EZ/EC Health Planning Capacity Survey

This study consisted of a survey of directors or acting directors in 141 Empowerment Zones and Enterprise Communities/EZ/ECs) during he summer of 2000. The survey asked about their interest in and capacity to engage in health improvement activities (defined as planned activities to address health issues) as well as their desires for technical assistance (TA) in this area. An 84% response rate was achieved. The survey findings are organized by five factors of EZ/EC health planning capacity and an additional section focusing on EZ/EC desires for technical assistance. The five factors are: 1) Interest and commitment to health - 88% reported interest in health issues, and 39% consider health issues among their top priorities. 2) Experience in various health improvement roles - Half of the respondents had participated in or had planned a health improvement activity during the past year. 3) Availability of expertise and data - Half reported they have access to regularly published information on local health statistics. (4) Existence and stability of local administrative structures and site advisory organizations - 89% of respondents have a general community advisory group and 5) Established relationships with community officials, business representatives, and health departments - 90% of respondents have business people they can call upon for advice, and 57% have conversed with business people about health issues. Regarding technical assistance, more than three-quarters of the respondents said they would use TA if offered for the participation of residents and community groups in EZ/EC-led activities. These responses show, according to the report, that many EZ/ECs already have the basic level of readiness to undertake and sustain successful health improvement efforts and to benefit from technical assistance.
http://www.phf.org/research.htm
PIC ID: 7341; CONTACT: Barbara Greenberg, 202-401-8279; PERFORMER: Public Health Foundation, Washington, DC

In-Progress Evaluations

1999 NLTCS/ICS: File Preparation and Preliminary Data Analysis

The purpose of this project is to prepare the 1999 NLTCS/ICS data file for analysis and to perform some preliminary descriptive analyses. This is a necessary prerequisite for more detailed analyses, which will be used to update the ASPE booklet, “Informal Caregiving: Compassion in Action” (published in 1988, based on 1995 NLTCS data).
PIC ID: 7761; EXPECTED COMPLETION: FY 2002; CONTACT: Pamela Doty, 202-690-6443; PERFORMER: Medstat Group, Washington, DC

A Comparative Study of the Outcomes and Costs Associated with Medicare Post-Acute Services in Skilled Nursing Facilities, Rehabilitation Hospitals/Unit

This project will study the cost-effectiveness of Medicare post-acute care services for Medicare beneficiaries with certain conditions, within and across post-acute care settings and over episodes of care. The study will examine: (1) demographic and health-related characteristics of Medicare beneficiaries who use post-acute services following a hospital stay to determine how much overlap there is in the patient populations treated by each provider type and across provider types; (2) patterns of service use and costs associated with the treatment of similar patients in each setting and across episodes of care; (3) outcomes for similar Medicare beneficiaries by each post-acute provider type and across episodes including those in which multiple providers are used; (4) relationship between outcomes for similar patients and differences in the mix and intensity of services provided, and level of reimbursement across post-acute care providers and episodes of care; and (5) core measures that are most useful to incorporate into on-going reporting requirements to monitor outcomes in each post-acute care setting and across episodes of care.
PIC ID: 6836; EXPECTED COMPLETION: FY 2004; CONTACT: Jennie Harvell, 202-690-6443; PERFORMER: Mathematica Policy Research, Inc., Plainsboro, NJ

Analyses of Changes in Elderly Disability Rates: Implications for Health Care Utilization and Costs

The purpose of this project is to conduct analyses using the 1984 to 1999 National Long-Term Care Survey (NLTCS) and the Medicare Current Beneficiary Survey (MCBS) to understand the nature of recent declines in elderly disability rates and their implications for health care utilization and costs. Specifically, researchers are (1) decomposing changes in elderly disability rates using the 1984 to 1999 NLTCS and exploring possible reasons for the decline, and (2) linking changes in elderly disability rates to the use of specific medical procedures (e.g., cataract surgery, coronary and joint replacement surgeries) and/or assistive technology. The MCBS is the primary data set for the latter analyses.
PIC ID: 7554; EXPECTED COMPLETION: FY 2002; CONTACT: William Marton, 202-690-6443; PERFORMER: Urban Institute, Washington, DC

Analysis of the Disability Supplement to the National Health Interview Survey

The disability supplement to the National Health Interview Survey (NHIS-D) is the first comprehensive survey on disability in the United States. The survey is unique in that it focuses on several populations of persons with disabilities who are generally omitted or under-represented in national survey efforts: children, persons with mental retardation and other developmental disabilities and the working-age population. The goal of this project is to conduct a series of analyses and produce both short-term products and final reports that will inform ongoing Assistant Secretary for Planning and Evaluation (ASPE) departmental and administrative research and policy initiatives. The analyses will address a specific set of questions in four topical areas using the NHIS-D: (1) welfare receipt and disability (including chronic illness); (2) disability and chronic illness among low income populations; (3) childhood disability and chronic illness; and (4) employment and disability.
PIC ID: 7153; EXPECTED COMPLETION: FY 2002; CONTACT: William Marton, 202-690-6443; PERFORMER: Urban Institute, Washington, DC

Analyses of Residential Transition of Older Americans

There are four main questions to be addressed in this project: (1) How do characteristics (both individual and environmental) of elderly persons residing in institutional settings differ from those residing in community-based settings? (2) How do these characteristics vary over time? (3) Are there differences in these characteristics between subgroups of institutionalized and non-institutionalized elderly? (4) What is the relationship between selected individual and environmental factors and the transition of the elderly between community and institutional residential settings? Data from six years of the Medicare Current Beneficiary Survey will be used to answer these questions. Understanding residential transitions will help staff in the Department improve surveys that monitor acute health and long-term care use in different settings (e.g., the Medical Expenditure Panel Survey) and address outstanding long-term care policy issues (e.g., allocation of resources between community and institutional settings).
PIC ID: 7555; EXPECTED COMPLETION: FY 2002; CONTACT: Hakan Aykan, 202-690-6443; PERFORMER: Urban Institute, Washington, DC

Case Studies and Technical Assistance for Medicaid Buy-Ins for People with Disabilities

In an effort to encourage more States to provide Medicaid to working individuals with disabilities, and to give States additional options for providing Medicaid to those individuals, Congress permitted States to expand their Medicaid programs (through a Medicaid “buy-in”) and allow people with disabilities to continue to receive Medicaid services even if they return to work and have earnings greater than 100% of the Federal Poverty Level. The goals of this project are to: (1) examine the early implementation experiences of States that have opted for a Medicaid buy-in for people with disabilities, and (2) offer technical assistance to States on developing cost models, design features of a buy-in, as well as implementation and administration considerations for a buy-in. The project represents a unique opportunity to compare the design and implementation experiences of different States, and use the information gathered to inform both State and federal policy makers about strategies for improving systems that support the employment of people with disabilities.
PIC ID: 7556; EXPECTED COMPLETION: FY 2002; CONTACT: Andreas Frank, 202-690-6443; PERFORMER: George Washington University, Washington, DC

Characteristics of Nursing Home Residents

Caring for persons with disabilities in the least restrictive setting is a major long-term policy objective. It is important to identify nursing home residents who could be discharged to the community if appropriate home and community-based services were available. This project will analyze data from the Minimum Data Set (MDS). The MDS consists of assessments which have been conducted on all nursing home residents in selected states as part of a CMS demonstration (and starting the summer of 1998, the data will be collected in all 50 States). Using this new data set will provide more information about the medical conditions, functional needs, and specific services used by nursing home residents than was possible with previous data sets. It will also be possible to study important subpopulations, especially the nonelderly.
PIC ID: 6275; EXPECTED COMPLETION: FY 2002; CONTACT: John Drabek, 202-690-6443; PERFORMER: Hebrew Rehabilitation Center for the Aged, Boston, MA

Design Options for Modeling Policy Impacts on Working Age Adults and Children with Disabilities

The Department currently has a limited ability to model the impacts of alternative health, long-term care and employment related policies on the program participation of younger individuals with disabilities and public and private costs, both in the short and long run. For example, none of the current models were capable of generating estimates about the cost and impact of the Kennedy-Jeffords proposal to extend Medicaid and Medicare coverage to SSI and/or SSDI disabled adults who return to work. The purpose of this project is to develop preliminary design options for constructing a microsimulation model of children, working age adults and their families. Such a model would age a sample of persons over time, keeping track of changes in their disability status, income, education, living arrangement, program participation, employment, health insurance status, health and long-term care service use, including personal assistance services/equipment.
PIC ID: 7764; EXPECTED COMPLETION: FY 2002; CONTACT: John Drabek, 202-690-6443; PERFORMER: Lewin Group Fairfax, VA and The Johns Hopkins University, Baltimore, MD

Developing Concise Measures of Child Disability

This project will support Network activities related to the development of a set of concise measures of childhood disability. The project will capitalize on recent conceptual and methodological developments in the demographic, social and biomedical study of disability. Specifically, the project will use data from the 1994 and 1995 disability supplements to the National Health Interview Survey (NHIS-D), the 1997 National Health Interview Survey (NHIS), and the 1992 and 1993 panels of the Survey of Income and Program Participation to develop concise measures of disabilities in children. The method of development will use two leading conceptual models of disability: that of the National Center for Medical Rehabilitation Research (NCMRR) and that of the World Health Organization (WHO). The intent of these concise measures is to provide reliable sets of indicators that are sensitive to subgroups of children to be used in population sample surveys and survey-based surveillance systems.
PIC ID: 7558; EXPECTED COMPLETION: FY 2002; CONTACT: John Drabek, 202-690-6443; PERFORMER: National Institute of Child Health and Human Development, Bethesda, MD

Disability Supplements to CMS Evaluation of Medicaid Managed Care in Tennessee

This project adds a disability component to an existing CMS evaluation of Medicaid 1115 waiver demonstrations in five states, with a focus on the Tennessee evaluation. The study will follow the experiences of SSI disabled children and adults (physically disabled, mentally ill, mentally retarded, or developmentally disabled) enrolled in the TennCare and TennPartners Programs. The project will conduct in-depth qualitative analyses of the State’s experience in enrolling individuals with disabilities into managed care systems, and will conduct quantitative analysis to examine cost and utilization data for these populations. The supplement will also include a survey of disabled consumers to examine issues of satisfaction, quality, health status, and functioning.
PIC ID: 6166.2; EXPECTED COMPLETION: FY 2002; CONTACT: Hunter McKay, 202-690-6443; PERFORMER: Mathematica Policy Research, Inc., Plainsboro, NJ

Evaluation of Practice in Care (EPIC)

From 1989 to 1992, there was a 210 percent increase in Medicare expenditures for home health services. This increase in utilization has generated policy interest in measures to control expenditures without compromising quality. Medicare home health has been the subject of considerable research, but the actual practice of home health care has not been extensively examined. This study will: (1) analyze “episodes” of care under the Medicare home health benefit, (2) assess the actual practice of care, (3) determine the extent to which there is variation in practice between acute and long-term care patients, and (4) uncover the factors accounting for that variation. The study will also examine decision-making processes between patients, providers and physicians. The events that take place during a visit and between visits as “actual practice” have never been measured. Furthermore, the function of decision-making by various parties has not been observed in “actual practice”. This effort to understand issues surrounding regional and practice variations of home health care delivery will aid the Department of Health and Human Services and the industry in combating fraud and abuse, as well as contribute valuable data to a future prospective payment system.
PIC ID: 5888; EXPECTED COMPLETION: FY 2002; CONTACT: Floyd Brown, 202-690-6443; PERFORMER: University of Colorado, Denver, CO

Long Term Care Microsimulation Model

The Long-Term Care Financing Model is a resource which has been extensively used by ASPE to project future long- term care utilization and expenditures and simulate various long-term care policy options. These include expansions of public financing, such as those proposed during health care reform, as well as changes to Medicaid and Medicare. The model has also been used extensively to study private sector policy options, such as the impact of further growth on private long-term care insurance and the impacts of changing trends in disability rates on long-term care use and expenditures. Recent data on disability rates, nursing home use, and home care use will be used to update existing portions of the model. In addition, the model will be expanded to include acute care use, thereby increasing the ability to simulate a wide range of policy options. ASPE will use this computer model for projections and a series of policy simulations which will be presented in reports. In addition to its previous long-term care policy uses, the revised model will enable ASPE to address acute care issues, such as the combined burden of acute and long-term care spending on the elderly. Since the model simulates the income and assets of the population, including pension and Social Security payments, the model will also be used to study other aging-related issues, such as the impact of changes in employer-sponsored health insurance for retirees.
PIC ID: 7143; EXPECTED COMPLETION: FY 2002; CONTACT: John Drabek, 202-690-6443; PERFORMER: The Lewin Group, Fairfax, VA

Longitudinal Study of Welfare Reform and Its Implications for Persons with Disabilities

This project will explore how welfare reform affects the well-being of persons with disabilities. Relevant questions to be addressed in this study include: How will welfare reform affect the well-being of adult TANF recipients with disabilities? What types of accommodations are being made to increase the work capacity of individuals with disabilities? Are persons with disabilities receiving appropriate job training that would enable them to move into the workforce? How will the well-being of children or adults with disabilities be affected if their primary caregiver has to work? How do families that contain persons with disabilities adapt to the new welfare environment? What types of changes in support systems occur to maintain the medical and other care needs of dependents with disabilities? What role do programs for persons with disabilities, such as early intervention programs and Supplemental Security Income (SSI), play following welfare reform?
PIC ID: 7362; EXPECTED COMPLETION: FY 2003; CONTACT: William Marton, 202-690-6443; PERFORMER: Johns Hopkins University, Baltimore, MD

Managed Delivery Systems for Medicare Beneficiaries with Disabilities and Chronic Illnesses

This project will study the experiences of elderly Medicare beneficiaries with disabilities and chronic illnesses in a managed delivery system (MDS) at four separate sites. An MDS is broadly defined as a health care system that integrates the financing and/or delivery of primary, acute and long-term care of persons for the purpose of controlling costs and improving access to, and coordination of, services across a continuum of providers. The specific goals of the study are to: (1) foster a better understanding of the unique characteristics and health/long-term care needs of elderly persons with disabilities and chronic illnesses among health plan administrators, providers and policymakers; (2) assess how managed care delivery systems meet the needs of this population and the factors that promote or impede success; and (3) describe the health care experiences of disabled elderly Medicare beneficiaries in MDS.
PIC ID: 6391; EXPECTED COMPLETION: FY 2002; CONTACT: Jennie Harvell, 202-690-6443; PERFORMER: Mathematica Policy Research, Inc., Plainsboro, NJ

Medicare Post-Acute Care: Quality Measurement

This project will specify performance measures that can be used to determine the cost-effectiveness of post-acute care (PAC) services. The specific measures and data elements will apply to patient conditions that are prevalent within and across PAC settings. In addition, this project will identify the data collection methods that could be used with respect to these elements. Finally, this project will identify issues that need to be considered in applying these measures, including whether there is the need to engage in validity and reliability testing, and, if so, how.
PIC ID: 7363; EXPECTED COMPLETION: FY 2002; CONTACT: Jennie Harvell, 202-690-6443; PERFORMER: Urban Institute, Washington, DC

Monitoring the Health Outcomes for Disabled Medicare Beneficiaries

The Balanced Budget Act (BBA) of 1997 mandated major changes in home health payment requiring the implementation of a Prospective Payment System (PPS) by October 1999 (later delayed until October 2000) and an Interim Payment System (IPS) prior to the implementation of PPS. It also contained changes in eligibility and coverage for home health services. These changes, while intended to reduce Medicare home health costs, run the risk of reducing beneficiaries’ access to appropriate care and adversely affecting health outcomes, especially for beneficiaries needing the most care. Disabled Medicare beneficiaries are most vulnerable. The purpose of this project is to study the impact of recent payment policy changes on disabled Medicare beneficiaries’ satisfaction and quality of life with a view toward formulating inferences that will inform national home health care policy for the disabled. The study will build on a research project recently funded by the Home Care Research Initiative of The Robert Wood Johnson Foundation that examines the direct and indirect effects of the BBA changes. The main focus is to examine BBA impacts on Medicare beneficiaries’ access to care, costs, satisfaction, and quality of care.
PIC ID: 7364; EXPECTED COMPLETION: FY 2002; CONTACT: Pamela Doty, 202-690-6443; PERFORMER: Laguna Research Associates, San Francisco CA

National Resource Center on Home and Community-Based Services

The purpose of this project is to develop and test the effectiveness of a National Consortium and Resource Center (NCRC) to improve access to consumer responsive home and community-based long-term care for people with disabilities of all ages. The long range purpose of such a center would be to foster long-term care policies and practices. Barring a fundamental shift in public financing that few expect, the Medicaid program will continue to be the principal purchaser of public long-term care services well into the next century. Several factors offer new opportunities and pose new dilemmas for long-term care policy and program development in the 21st century. Clearly, consumers, policy makers, and providers are experiencing some tensions in their relationships as a result of several of the above factors. The Department of Health and Human Services, along with several other federal agencies is committed to facilitating the development of a common agenda among the stakeholders in long-term care policy and program development at all levels, one that supports consumer decision making and choice.
PIC ID: 7767; EXPECTED COMPLETION: FY 2002; CONTACT: Pamela Doty, 202-690-6443; PERFORMER: MEDSTAT Group, Cambridge, MA

Personal Assistance Services “Cash and Counseling” Demonstration/Evaluation

This study will employ a classical experimental research design (i.e. random assignment of participants to treatment and control groups) to test the effects of “cashing out” Medicaid-funded personal assistance services for the disabled. The demonstration sites are located in Arkansas, New Jersey and Florida. Control group members will receive “traditional” benefits in the form of case-managed home and community-based services where payments for services are made to vendors. Treatment group members will receive a monthly cash payment in an amount roughly equal to the cash value of the services that they would have received under the traditional program. It is hypothesized that cash payments will foster greater client autonomy and that, as a result, consumer satisfaction (particularly among disabled persons under age 65) will be greater. It is also hypothesized that states will save Medicaid monies (mostly in administrative expenses) from cashing out benefits.
PIC ID: 6161; EXPECTED COMPLETION: FY 2004; CONTACT: Pamela Doty, 202-690-6443; PERFORMER: University of Maryland, Center on Aging, College Park MD

Preliminary Analysis of the 1988 MCBS Institutional Drug File

It is very important that ASPE be able to have a reliable source of drug utilization data for the elderly and disabled in long-term care facilities as well as in the community. The 1988 MCBS Cost and Use files are the most up-to-date, publicly available, person-level data available to address drug use and expense questions for community-dwelling Medicare beneficiaries. However, these files contain no drug use data for facility residents (despite the fact that MCBS surveyors have collected this information since 1997). The objective of this task is to create a research-ready file of 1998 drug use data for facility respondents and link it to the 1998 Cost and Use file data on facility residents. We have selected the 1998 facility drug file for this task order in order to facilitate linking with the latest available MCBS Cost and Use files. This linked file will serve as the primary data source to address policy questions of interest to ASPE.
PIC ID: 7768; EXPECTED COMPLETION: FY 2002; CONTACT: Andreas Frank, 202-690-6443; PERFORMER: University of Maryland, College Park, MD

Prescription Drug Coverage and the Medicare Disabled Population

This project proposes to complete a detailed descriptive study of prescription drug coverage among Medicare beneficiaries with a disability. Specifically, the contractor will focus on how disability in combination with income, race, gender and other factors may impact prescription drug coverage.
PIC ID: 7769; EXPECTED COMPLETION: FY 2002; CONTACT: Andreas Frank, 202-690-6443; PERFORMER: Pennsylvania State University, University Park, PA

Research on Employment Supports for People with Disabilities

Employment rates for people with disabilities have remained low despite advances in legislation (especially the Americans with Disabilities Act), the availability of technology which can assist with everyday activities and work-related functions, and an increase in positive societal attitudes toward the integration of people with disabilities into mainstream social institutions. Only about 25 percent of people with significant disabilities between the ages of 16 and 64 are employed. Most of the policy research on the employment of people with disabilities has relied on statistical data from national surveys. Detailed information on individuals’ experiences was needed to augment the survey data. This project will fill that need as it will: (1) gather descriptive data about public and private sector employment programs, (2) examine successful employment supports, and (3) investigate the factors that affect the ability of people with significant disabilities to work. Focus groups, examining the supports important to people with disabilities who are competitively employed, will be conducted in three cities. Policy papers, summarizing the results of the focus groups, will be completed by March 2001. The topics of the policy papers will include: transitions to employment, supports for employment, role of health insurance, role of employers, and supports used by individuals with mental disabilities.
PIC ID: 4917.2; EXPECTED COMPLETION: FY 2002; CONTACT: Andreas Frank, 202-690-6443; PERFORMER: The Lewin Group, Fairfax, VA

Study of Fair Hearing Adaptations

The purpose of this task order is to conduct a qualitative study that describes in detail selected aspects of four adaptations to the Medicaid Fair Hearing process for beneficiaries enrolled in managed care. The adaptations include evidentiary hearings, ombudsman programs, external review organizations and expedited review of appeals. Descriptive information will be gathered through case studies on the mechanics of each adaptation and the perceptions of state Medicaid staff, managed care organization administrators and advocacy organizations regarding issues related to their use by Medicaid beneficiaries. A policy meeting will be held to discuss the project’s findings and the potential for using the findings to improve the Medicaid Fair Hearing process.
PIC ID: 7559; EXPECTED COMPLETION: FY 2002; CONTACT: Hunter McKay, 202-690-6443; PERFORMER: Mathematica Policy Research, Inc., Plainsboro, NJ

Synthesis of Research on Disability and Managed Care

Since the mid-1990s, a number of research institutions initiated projects to assess how well disabled persons are cared for in managed care settings. The research focused on many different aspects of managed care and disability, and used a wide variety of research designs. The purpose of this delivery order contract is to: (1) synthesize recent research on the performance of managed care plans in caring for persons with disabilities, (2) summarize research on current trends in enrollment of disabled persons into Medicaid managed care programs, and, (3) recommend areas for future research.
PIC ID: 7560; EXPECTED COMPLETION: FY 2002; CONTACT: Hunter McKay, 202-690-6443; PERFORMER: Mathematica Policy Research, Inc., Plainsboro, NJ

The Contribution of Changes in Medication Use to Improvements in Functioning Among Older Adults

Drug treatment has become an increasingly important aspect of medical care for older Americans with approximately nine out of ten older Americans currently taking one or more prescription drugs daily. During the 1980s and early 1990s there have been major shifts in the classes of drugs prescribed for some of the more debilitating chronic conditions. For example, for arthritis, increased availability of disease modifying anti-rheumatic drugs, non-steroidal anti-inflammatory drugs, and, for women, estrogen replacement therapy may be associated with fewer debilitating effects. New classes of psychotropic agents have become available to treat depression and other psychiatric conditions, which have been identified as a major cause of premature disability among the elderly. Drug treatments for diabetes and hypertension have also expanded significantly during this period. This project supplements an existing National Institute on Aging grant to Vicki Freedman of the Philadelphia Geriatric Center. Under the grant Dr. Freedman is examining the role of changes in the use of medications in explaining aggregate changes in functioning in the U.S. population aged 51-61. The data sets for the analyses are the first (1992) and fourth (1998) waves of the Health and Retirement Survey (HRS), which provide nationally representative cross-sections of the noninstitutionalized population in this age range.
PIC ID: 7770; EXPECTED COMPLETION: FY 2002; CONTACT: William Marton, 202-690-6443; PERFORMER: Philadelphia Geriatric Center, Philadelphia, PA

Congressionally Mandated Evaluation of the State Children’s Health Insurance Program

ASPE is conducting a three-year, Congressionally mandated evaluation of the State Children’s Health Insurance Program (SCHIP). The evaluation will provide an opportunity to learn how effective the variety of state programs created under Title XXI have been in reducing the number of uninsured, and also help Congress, DHHS, and the States to take stock of the program’s first few years and provide information to guide future policies for children’s health insurance. Key topics to be examined through this evaluation include SCHIP’s impact on uninsurance rates; reasons why states designed their SCHIP programs as they did; enrollment and disenrollment trends; reasons for disenrollment and non-participation; barriers to enrollment; the impact of cost-sharing on enrollment, retention, and utilization of services; effectiveness of outreach approaches; SCHIP’s impact on access to care and enrollee satisfaction; and the relationships between SCHIP, Medicaid, and private coverage. The overall evaluation design includes both quantitative and qualitative research components, focusing on the experience of 10 States that represent a wide range of program designs and geographic regions.
PIC ID: 7730; EXPECTED COMPLETION: FY 2003; CONTACT: Stephen Finan, 202-690-7387; PERFORMER: Mathematica Policy Research, Inc., Plainsboro, NJ

Development and Validation of a Performance Measure Set for the Evaluation of Medicaid Services Rendered to People with Developmental Disabilities

Many public policies seek to influence the decisions employers make about whether and how to offer health benefits to their employees. The process of how employers make such decisions, however, is largely unknown. The Office of the Assistant Secretary for Planning and Evaluation convened two panels of senior corporate managers to discuss how firms make decisions about health benefits. The purpose was to help the office formulate a more comprehensive view of employer decision-making by posing common questions to a cross-section of knowledgeable people, and to identify issues that may be empirically measured. The long-term goal is to be able to better determine how public policies affect or influence the decisions made by employers and unions regarding health coverage. The first meeting focused on the “strategic” trends and issues affecting employer health coverage, addressing such questions as, “What is the role of the employer in shaping health care markets?” The second meeting focused on the internal and operational issues of how employers make health benefit decisions, addressing such questions as: “At what level of the corporate structure are health benefit decisions made?”
PIC ID: 7391; EXPECTED COMPLETION: FY 2002; CONTACT: Elizabeth Couchoud, 410-786-6722; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

Evaluating the Effects of Direct-to-Consumer Advertising on Consumers

The goal of this project is to improve our understanding of direct-to-consumer (DTC) advertising. The relation of this kind of advertising to aspects of public health will be examined through a survey of selected consumers and the Food and Drug Administration (FDA) Center for Drug Evaluation and Research’s (CDER) analysis of that survey. This analysis will yield an initial broad description of some possible associations between DTC advertising and consumer knowledge, attitudes and behaviors. The survey and CDER’s descriptive analysis will lay the foundation for the long- term task of determining what effects DTC advertising may have on overall consumer knowledge, attitudes and behaviors concerning prescription drugs and the overall effects of this practice on the public health.
PIC ID: 6343; EXPECTED COMPLETION: FY 2002; CONTACT: Susannah Bruns, 202-690-7996; PERFORMER: Market Facts, Inc., McLean VA

Impact of Medicare HMO Enrollment on Health Care Costs in California

The Office of the Assistant Secretary for Planning and Evaluation (OASPE) is supporting a time series study of the relationships between competition in hospital markets and the prices of hospital care, hospital revenues and hospital costs. The project began in the late 1980s and was extended in 1994 to investigate how health maintenance organization (HMO) enrollment affects changes in Medicare payments on behalf of beneficiaries. The project will collect time series data on California Hospitals (cost reports supplemented by a small survey on selective contracting by the hospital) to investigate whether competition continues to depress hospital prices, revenues and costs. In addition, several smaller projects are underway to investigate whether hospital mergers result in increased prices and a series of investigations (individual and county level) to determine if HMO enrollment leads to a decline in Medicare costs when other appropriate variables are held constant. California is particularly important in this study, because it tends to be a trend-setting State.
PIC ID: 6889; EXPECTED COMPLETION: FY 2002; CONTACT: George Greenberg, 202-690-7794; PERFORMER: Rand Corporation, Santa Monica, CA

Innovative Coverage of Health Promotion, Clinical Preventive Services, and Other Coverage Issues in the Private Sector

Despite the important role private sector employers play in shaping the current health care market, the views of this group concerning the provision of a variety of clinical preventive services and what their health plans currently provide have not been adequately described. In addition, the factors which influence their decisions to include or exclude such services have not been characterized. In collaboration with the Office of Disease Prevention and Health Promotion (ODPHP), and the Office of the Assistant Secretary for Planning and Evaluation (ASPE), researchers are forming a partnership with Partnership for Prevention to support the commissioning of a set of expert papers on the clinical preventive service benefits offered by employer-sponsored health plans. These papers are intended to meet two objectives: (1) to characterize the trends and decision processes occurring within the private sector; and (2) to evaluate the impact of government information on this issue in the private sector. As the government is investing its resources in producing and disseminating information designed to encourage the appropriate use of clinical preventive services, it is important to know how its customers in the private sector perceive and use such information.
PIC ID: 6347; EXPECTED COMPLETION: FY 2002; CONTACT: Caroline Taplin, 202-690-7906; PERFORMER: Partnership For Prevention, Washington, DC

Medicare Supplement Insurance: Structure, Change & Implications

The purpose of this research project is to determine whether or how the Medicare Supplement Insurance market has changed. Better information about how this market behaves and how it is changing over time can help the Department develop policies that will ensure stable health care financing for the elderly. This project is proposed to develop three research papers: (1) an examination of variation of the Med-supp market among states; (2) the effect of HMO penetration on Med-supp markets; and (3) the demand for prescription drug benefits in the Med-supp market. Collectively, these papers should provide a much better understanding of how this market has been operating, and thereby provide a better understanding of how new Medicare policies might affect it.
PIC ID: 7348; EXPECTED COMPLETION: FY 2002; CONTACT: Stephen Finan, 202-690-7387; PERFORMER: Alpha Center, Bethesda, MD

Safety Net Framework Project

The Nation’s health care provider safety net consists of: (1) hospitals, (2) primary care facilities, (3) specialized service providers such as family planning clinics, HIV service providers, community mental health centers, drug and alcohol treatment clinics, and (4) individual practitioners. These providers operate under both public and private auspices. As a group they are diverse, with varied funding sources including: (1) Medicaid and Medicare patient revenues and other payments, (2) federal grant support from multiple sources, (3) other grants, (4) state and local public funding, (5) limited private third party insurance, (6) patient fees (often sliding scale), and (7) private philanthropy. Federal policy affects these providers in several ways. Products from this project will include a series of working papers, a workshop summary and a final “framework” document. The project is intended to result in new ways of shaping the Department’s views concerning safety net providers. In particular, the Department plans to have a structure which considers how federal grant programs, payment policies and other federal policies affect these providers as they interact among each other at the community level.
PIC ID: 7156; EXPECTED COMPLETION: FY 2002; CONTACT: Caroline Taplin, 202-690-7906; PERFORMER: Northwestern University, Evanston IL

Safety Net Project: Exploring State Use of Medicaid Disproportionate Share Funds to Enhance Provision of Care to Uninsured

This project will study the cases in which States redirected or changed their use of Medicaid Disproportionate Share Funds (DSF) funding streams to strengthen safety net infrastructure in communities within their States; used DSF to increase the services available to uninsured populations; or provided insurance to previously uninsured individuals through DSF.
PIC ID: 7352; EXPECTED COMPLETION: FY 2002; CONTACT: George Greenberg, 202-690-7794; PERFORMER: Urban Institute, Washington, DC

State Activity Monitoring Medicaid Managed Care for Children with Special Health Care Needs

The purpose of this project is to investigate how States are monitoring and evaluating the experience of children with special health care needs enrolled in Medicaid Managed care. Research findings will be used to refine the standards CMS requires States to meet in order to enroll children with special health care needs into capitated Medicaid managed care plans, and to assess States’ ability to meet these standards. Policy makers are interested in addressing the issue of whether Medicaid plans and providers are meeting the complex needs of Supplemental Security Income (SSI) children, for example, but the lack of concrete data or information related to the experience of this population in Medicaid managed care hamper their efforts. In response, CMS has developed interim review criteria that will be applied to all upcoming 1915(b) waiver applications and renewals from States. The criteria establishes requirements regarding definition, identification, tracking, and access to care for children with special health care needs enrolled in capitated managed care arrangements. This study will help CMS and the States move forward with this important effort by providing critical information about current State activity and insights as to how these criteria can be refined, improved, and be put into operation.
PIC ID: 7350; EXPECTED COMPLETION: FY 2002; CONTACT: Jennifer Tolbert, 202-205-8678; PERFORMER: George Washington University Medical Center, Washington, DC

State Implementation of Immunization Recommendations and Guidance

This project in Phase 1 will conduct case studies in up to five States to learn how States (in public health, Medicaid and Medicare) are working on the immunization goals using the guidance and technical assistance provided by the federal government. This phase will highlight the status quo and understand the extent to which States are planning to incorporate CMS’s guidance into their activities. Phase 2 (FY 2000) the project will be in up to three of the States and will work with these States to take this activity to the next level, such as identifying and implementing appropriate outcome measures for immunization (e.g., vaccine-preventable disease rates). Both CDC/National Immunization Program and CMS would be involved in the project.
PIC ID: 7342; EXPECTED COMPLETION: FY 2003; CONTACT: Amy Nevel, 202-690-7795; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

The Changing Structure of Health Insurance Markets

This project will result in two research papers evaluating the following: (1) insurance market changes relating specifically to State laws complying with the Health Insurance Portability and Accountability Act of 1996, and (2) the relationship between health insurance market structure and health insurance coverage of the population. This study builds on an ongoing effort funded by the Robert Wood Johnson Foundation to compile an expanded and updated insurer database and to conduct several basic analyses of health insurance market structure, change and performance. State insurance markets are not well understood and have seldom been systematically studied in a manner that can inform public policy discussions and development. These products will contribute significantly to the public’s understanding of how insurance markets are changing.
PIC ID: 7160; EXPECTED COMPLETION: FY 2002; CONTACT: Stephen Finan, 202-690-7387; PERFORMER: Alpha Center, Washington, DC

Understanding Health Plan Accreditation and Its Potential to Promote the Consumer Bill of Rights and Responsibilities

The purpose of this project is (1) to provide policy and regulatory staff in the US Department of Health and Human Services (HHS) with an in-depth understanding of the aspects of the health plan accreditation process most closely related to the Consumer Bill of Rights and Responsibilities (CBRR); and (2) to determine the extent to which health plan accreditation fosters implementation of the CBRR. The study’s results will provide useful feedback to the individuals and entities involved in developing the CBRR--by illustrating difficulties that accrediting organizations and health plans may face in operationalizing CBRR provisions.
PIC ID: 7346; EXPECTED COMPLETION: FY 2002; CONTACT: Ellen Bayer, 202-690-7804; PERFORMER: Mathematica Policy Research, Inc., Washington, DC

A Research Synthesis of the Findings from ASPE’s “Leavers” Grants

This study will critically analyze and synthesize the findings of ASPE’s grants that have studied welfare outcomes for individuals and families who have left cash assistance under the Temporary Assistance for Needy Families (TANF) program. The project will result in an Initial Synthesis Report and a Final Report. The Office of the Assistant Secretary for Planning and Evaluation (ASPE) awarded grants to states and counties to study the outcomes of welfare reform on individuals and families who leave TANF, individuals and families entering the caseload, and TANF applicants and potential TANF applicants. All of the grantees are using a combination of linked administrative data and surveys to study the outcomes of the TANF “leavers.” The grantees include: Arizona, Cuyahoga County, District of Columbia, Florida, Georgia, Illinois, Los Angeles, Massachusetts, Missouri, New York, San Mateo County, Washington, and Wisconsin.
PIC ID: 7368; EXPECTED COMPLETION: FY 2002; CONTACT: Matthew Lyon, 202-401-3953; PERFORMER: Urban Institute, Washington, DC

Case Studies on Privatization of TANF Service Delivery and Performance-Based Contracting

The purpose of this project is to conduct case studies on the privatization of delivery of services funded under the Temporary Assistance to Needy Families (TANF) block grant in States and localities, including the implementation of performance-based contracts. The study will focus on those services that are the responsibility of and have historically been performed by the State or local human services agency, such as client intake and assessment, eligibility determination, and case monitoring. A final report, expected in December 2002, will synthesize the information gathered both during the case studies and from a literature review on the topic of privatization of social service delivery. The report will use the information gathered in the site visits to identify some of the challenges involved in privatization. It also will identify emerging positive trends and problem areas, in both the services delivered and the types of contracts that have been enacted. Ideally, this report will be instructive both to federal policymakers and to State and local human services agencies that are considering privatizing significant parts of their TANF programs.
PIC ID: 7755; EXPECTED COMPLETION: FY 2003; CONTACT: Matthew Lyon, 202-401-3953; PERFORMER: Mathematica Policy Research, Inc., Plainsboro, NJ

Devolution and Urban Change

This study presents a unique opportunity to learn about the implementation and impacts of welfare reform in four large urban areas--Cleveland, Philadelphia, Los Angeles, and Miami. From this study, we expect to learn about the challenges of welfare reform in a context of high levels of unemployment and concentrated poverty. The Urban Change project brings together data from a wide array of sources: longitudinal administrative data for all families receiving AFDC/TANF or Food Stamps dating back to 1992, survey data, an implementation study, neighborhood indicators, an institutional study focusing on local service providers, and an ethnographic study of a limited number of families. The contractor hopes to be able to assess the effects of welfare reform on recipient experiences based on the administrative data using a multiple cohort comparison design. The study seeks to answer a broad array of questions, including: How do welfare recipients and other low-income residents fare in the new welfare environment in terms of both economic and non-economic indicators of well-being? What policies do State and local officials formulate, and how are these carried out by line staff in the welfare offices? How do the social and economic conditions of neighborhoods change over the time period before, during, and after implementation of welfare reform?
PIC ID: 7754; EXPECTED COMPLETION: FY 2002; CONTACT: Elizabeth Lower-Basch, 202-690-6808; PERFORMER: Manpower Demonstration Research Corporation, New York, NY

Domestic Violence and Welfare: An Early Assessment

This project will study early implementation of the Temporary Assistance for Needy Families (TANF) Family Violence Option and child support enforcement program responses, including the use of good cause waivers for battered women. Evidence suggests that some battered women seek employment and child support while others need specialized assistance in addressing partner abuse. To address domestic violence, States may have policies regarding screening and other forms of identification, assessment of a battered woman’s needs, service provision, confidentiality, corroboration required, staffing arrangements, agreements with community resources and other considerations. The study will provide information about these policy issues, the status of program implementation and descriptions of useful models.
PIC ID: 6724; EXPECTED COMPLETION: FY 2002; CONTACT: Gerald Silverman, 202-690-5654; PERFORMER: Urban Institute, Washington, DC

Economic and Health Status of Immigrants, Their Communities, and the Organizations that Serve Them

This project will conduct research on the economic and health status of immigrants, their communities and the organizations serving them. The research will profile the status of immigrants with regard to health, employment and participation in programs administered by public and private agencies, with special attention to distinguishing categories of immigrants and drawing comparisons with the citizen population. The project will also explore the impacts of welfare reform on immigrants and organizations, with special attention to both individual and institutional adaptations. Since the loss of food stamps appears to be the most far-reaching and permanent change affecting current legal immigrants, its impact on immigrants and their families will also be a focus of the study. The project will supplement an examination of existing secondary data with intensive data collection in Los Angeles and New York, the two cities that together account for one-fourth of the immigrant population. Primary data collection in the two cities has been designed to develop profiles of immigrants in these cities and the agencies serving them. This effort has several components, including: (1) interviews with 1,650 immigrant households in each city; (2) telephone interviews with most of the non-profit agencies providing services to immigrants; (3) in-depth interviews with 100 immigrants who lose Food Stamps and with additional immigrants who retain food stamps in each city; and (4) in-depth interviews with administrators and staff in public and private agencies.
PIC ID: 6747; EXPECTED COMPLETION: FY 2002; CONTACT: David Nielsen, 202-401-6642; PERFORMER: Urban Institute, Washington, DC

Follow-up Work on the Minnesota Family Investment Program Evaluation

Recently released findings based upon a small sample from the Manpower Demonstration Research Corporation’s evaluation of the Minnesota Family Investment Program suggest that the program had significant positive impacts on the marital stability of two-parent families. This project will support MDRC’s follow-up study of all two-parent recipient families in the sample to determine if the robustness of their findings can be replicated for the larger group. This work is a necessary first step before a decision is made about conducting a longer-term follow-up analysis of well-being outcomes for these families. If the original findings hold up for the larger sample, work will begin on developing instruments for the measurement of longer-term well-being outcomes under MFIP. If the marital stability finding is not replicated, the balance of the project will focus on analyzing data already collected under the original MFIP evaluation to determine links between domestic violence and family formation and work patterns.
PIC ID: 7760; EXPECTED COMPLETION: FY 2003; CONTACT: Alana Landey, 202-401-6636; PERFORMER: Manpower Demonstration Research Corporation, New York, NY

Medical Child Support Cross-Program Coordination Descriptive Study

In the area of medical child support, States have identified coordination with SCHIP and private insurers as the number one issue for which they need federal assistance. Also, the HHS/DoL Medical Child Support Working Group recommended that further research be conducted on State’s efforts to coordinate health care coverage availability between child support, Medicaid, and SCHIP (Recommendation 70). This project will describe and analyze State efforts to coordinate between these three programs in order to secure appropriate health care coverage for eligible children; the effects of federal policy on this cross-program coordination; and barriers to these efforts. The project has two components. The first involves convening a panel of experts to discuss why State agencies collaborate; what the barriers to collaboration are perceived to be, and the effects of federal policy on the decision to coordinate. Second, a series of case studies of several sites where coordination between child support, Medicaid, and SCHIP is underway will document the nature of the coordination, identify innovative practices and barriers; and describe the effects of federal policies.
PIC ID: 7236; EXPECTED COMPLETION: FY 2003; CONTACT: Jennifer Tolbert, 202-205-8678; PERFORMER: Urban Institute, Washington, DC

National Evaluation of Welfare-to-Work Strategies (NEWWS)

The National Evaluation of Welfare-to-Work Strategies (NEWWS) evaluation is a study of the effectiveness of eleven mandatory welfare-to-work programs in seven locales: Atlanta, Georgia; Columbus, Ohio; Detroit and Grand Rapids, Michigan; Oklahoma City, Oklahoma; Portland, Oregon; and Riverside, California. Program impacts were evaluated by comparing outcomes for a randomly assigned experimental group subject to program requirements with outcomes for control groups. As part of the NEWWS, the effects of two approaches to preparing welfare recipients for employment were compared in three sites (Atlanta, Grand Rapids, and Riverside). In one approach, the human capital development approach, individuals were directed to avail themselves of education services and, to a lesser extent, occupational training before they sought work, under the theory that they would then be able to get better jobs and keep them longer. In the other approach, the labor force attachment approach, individuals were encouraged to gain quick entry into the labor market, even at low wages, under the theory that their work habits and skills would improve on the job and they would thereby be able to advance themselves. Sample members are being followed for five years from the time they entered the study.
PIC ID: 7089; EXPECTED COMPLETION: FY 2002; CONTACT: Audrey Mirsky-Ashby, 202-401-6640; PERFORMER: Manpower Demonstration Research Corporation, New York, NY

State Experience and Perspectives on Reducing Out-of-Wedlock Births

This project examines State efforts to reduce out-of-wedlock childbearing since the passage of PWRORA in 1996, and the issues States have encountered. The study will pay particular attention to the role that the Bonus to Reward Decrease in Illegitimacy ratio played in motivating or funding those efforts. The project will first gather and synthesize the results from several existing surveys of specific state efforts related to reducing nonmarital childbearing. Then the project will convene three panels representing a diverse sample of States. The panels will include both States that have and have not received the bonus, as well as those with greater and lesser publicized efforts to receive the bonus. The discussion will focus first on State experiences regarding their efforts to reduce nonmarital childbearing--what efforts they have undertaken and what issues have they faced. The discussion will then focus specifically on the extent to which the bonus played a role in their efforts (either their attempts to qualify for the bonus, or their actual receipt of the bonus).
PIC ID: 7757; EXPECTED COMPLETION: FY 2002; CONTACT: Kelleen Kaye, 202-401-6634; PERFORMER: Lewin Group Fairfax, VA and The Johns Hopkins University, Baltimore, MD

The National Evaluation of the Welfare to Work Grants Program

The Balanced Budget Act of 1997 authorized the Department of Labor to award $3 billion in welfare-to-work grants to States and local communities to promote job opportunities and employment preparation for the hardest-to-employ recipients of Temporary Assistance for Needy Families (TANF), and for non-custodial parents of children on TANF. This long-range project is evaluating the effectiveness of the DOL-sponsored Welfare-to-Work initiatives, including those undertaken by the Welfare-to-Work grantees and by Native American tribal organizations. HHS has worked with the Departments of Labor and Housing and Urban Development to design an evaluation to address five key questions: (1) What are the types and packages of services provided by Welfare-to-Work grantees? (2) What are the outcomes of the various program approaches on employment and other measures of family well-being? and (3) What challenges are confronted as grantees implement and operate the programs?
PIC ID: 7147; EXPECTED COMPLETION: FY 2004; CONTACT: Alana Landey, 202-401-6636; PERFORMER: Mathematica Policy Research, Inc., Washington, DC

Welfare Outcomes Panel Study

The Office of the Assistant Secretary for Planning and Evaluation (OASPE) is conducting a panel study with the National Academy of Sciences (NAS) to evaluate the design of current, proposed and future studies of the effects of the Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA) of 1996. This panel study will assist the department in: (1) reviewing existing research efforts on the effects of welfare reform; (2) examining the appropriate data sources, research designs and statistical methods for analyzing welfare reform outcomes; and (3) identifying research gaps and data needs for the continued study of welfare reform. Specific topics to be covered by the panel of experts include: (1) the appropriate outcome measures needed to effectively evaluate the outcomes of welfare reform; (2) the appropriate population of study; (3) the design and usefulness of existing national-level surveys (e.g., the Survey of Program Dynamics and the Survey of Income and Program Participation); (4) data needs for monitoring State and sub-state variation in outcomes; (5) potential uses and limitations of administrative data; and (6) integration of survey and administrative data. Over the course of the study, the NAS expert panel will address the above mentioned tasks through workshops, quarterly panel meetings and the publication of both an interim report and a final report. The interim report (see PIC # 7145.1) included early recommendations of the panel and the final report will document the comprehensive list of final recommendations to the department. The panel is also releasing an accompanying compilation of papers written for the panel study, entitled: “Studies of Welfare Populations: Data Collection and Research Issues.”
PIC ID: 7145; EXPECTED COMPLETION: FY 2002; CONTACT: Susan Hauan, 202-690-8698; PERFORMER: National Academy of Sciences, Washington, DC

State Telephone Survey Estimates for Evaluation and Monitoring

This project, funded by the Office of the Assistant Secretary for Planning and Evaluation (ASPE) and carried out by the National Center for Health Statistics (NCHS), develops and evaluates an integrated survey mechanism to collect broad State-based health and human services data. It also evaluates the quality and utility of State-level estimates developed from telephone surveys alone and in conjunction with, estimates obtained from the National Health Interview Survey (NHIS). A variety of health and human services data estimates for a selected number of States will be obtained through a telephone interview survey based on the NHIS questionnaire and selected supplemental data.
PIC ID: 6420; EXPECTED COMPLETION: FY 2002; CONTACT: James Scanlon, 202-690-7100; PERFORMER: National Center for Health Statistics, Hyattsville MD

The Importance of Place

There is a growing body of research documenting the synergistic effect that the configuration of community resources, community values, population demographic, geography, and other factors have on health and human service outcomes. The Office of the Assistant Secretary for Planning and Evaluation is funding the National Academy of Sciences (NAS) to commission a series of papers to synthesize the available research on the relationship between these place-based factors and outcomes in the areas of health, child and youth development, and welfare reform. The papers will identify available data to assess these interactions and make recommendations for a future research agenda. NAS also will convene a workshop of researchers and policy makers from federal, state and local government, academic institutions and private organizations to reflect on the papers and develop and refine an integrated research agenda. The commissioned papers as well as a workshop summary which will include the research agenda will be available for dissemination.
PIC ID: 7552; EXPECTED COMPLETION: FY 2002; CONTACT: Barbara Greenberg, 202-401-8279; PERFORMER: National Academy of Sciences, Washington, DC

Office of Public Health and Science

Mission

To provide advice on public health and science to the Secretary of Health and Human Services, to provide executive direction to program offices within the Office of Public Health and Science (OPHS), and, at the direction of the Secretary, to coordinate crosscutting public health and science initiatives in the Department.

Evaluation Program

The Office of Public Health and Science (OPHS) provides advice, policy and program coordination, and leadership in the implementation, management, and development of activities related to public health and science, as directed by the Secretary. OPHS helps HHS conduct broad‑based public health assessments to better define public health difficulties and solutions. It assists other components of HHS in anticipating future public health issues and helps ensure that HHS designs and implements appropriate approaches, interventions, and evaluations that will maintain, sustain, and improve the health of the Nation. OPHS provides leadership and policy recommendations on population‑based public health and science and, at the Secretary’s direction, leads or coordinates initiatives that cut across agencies and operating divisions. In addition, OPHS communicates and interacts, on behalf of the Secretary, with national and international professional and constituency organizations on matters of public health and science. Finally, OPHS’s unique role allows it to use its resources to link important HHS programs or fill gaps in areas needing better policy formulation and coordination.

OPHS’ evaluation strategy focuses on public health and science issues that cut across multiple interests of the operating divisions and requires a coordinated approach to achieve the most effective results. OPHS evaluations support the Assistant Secretary for Health in the respective roles as the Nation’s top doctor and senior advisor for public health and science to the Secretary. OPHS also conducts evaluations specific to the needs of the programs operated from the offices located within OPHS, such as emergency preparedness, women’s health, minority health, disease prevention and health promotion, and research integrity. Some evaluation funds are made available to the ten HHS Regional Health Administrators. In keeping with its role within the Department, OPHS does not undertake evaluations more appropriately and effectively undertaken by operating divisions of HHS or by the Assistant Secretary for Planning and Evaluation (ASPE).

Completed Evaluations

Data Analysis and Support for President’s Race Initiative

The President’s Advisory Board on Race submitted a report of its deliberations to the President in the summer of 1998. A “chart book” on the current state of race relations was prepared in conjunction with the Board’s activities. The White House also commissioned the National Academy of Sciences to prepare detailed summaries of the research literature related to race, including health. This project identified data sources and data elements for detailed univariate and multivariate analyses, evaluated the quality and error structure of the data, conducted analyses to understand the persistent health disparities for racial and ethnic minorities and disseminated the results through a variety of mechanisms.
PIC ID: 7014; CONTACT: Olivia Carter-Pokras, 301-443-9923; PERFORMER: National Center for Health Statistics, Rockville, MD

National Standards for Culturally and Linguistically Appropriate Services in Health Care

In 1997, the OMH undertook the development of national standards to provide a much-needed alternative to the current patchwork of independently developed definitions, practices, and requirements concerning culturally and linguistically appropriate services (CLAS). The purpose was to develop national CLAS standards that would support a more consistent and comprehensive approach to cultural and linguistic competence in health care. The first stage of the project involved a review and analysis of existing cultural and linguistic competence standards and measures, the development of draft standards, and revisions based on a review by a national advisory committee. The second stage focused on obtaining and incorporating input from organizations, agencies, and individuals that have a vital stake in the establishment of CLAS standards. In this final version, there are 14 CLAS standards organized by themes, such as culturally competent care, language access services, and organizational supports for cultural competence. Within this framework there are three types of standards of varying stringency: mandates, guidelines, and recommendations. The CLAS standards reflect input from a broad range of stakeholders, including hospitals, community-based clinics, managed care organizations, home health agencies, and other types of health care organizations; physicians, nurses, and other policymakers; purchasers of heath care; accreditation and credentialing agencies; educators; and patient advocates, advocacy groups, and consumers.
PIC ID: 7260; CONTACT: Guadalupe Pacheco, 301-443-5084; PERFORMER: IQ Solutions, Inc., Rockville, MD

In-Progress Evaluations

Role of Health Plans in Community-Level Health Improvement

The purposes of this project are to identify the potential for health plans’ participation in community-level health improvement activities, and to examine the incentives and barriers that will affect their decisions to take part in community-based health efforts.
PIC ID: 6492; EXPECTED COMPLETION: FY 2002; CONTACT: Matthew Guidry, 202-401-7780; PERFORMER: Partnership For Prevention, Washington, DC

Scientific Evaluation of Dietary Reference Intakes

The National Academy of Sciences, Institute of Medicine, Food and Nutrition Board is conducting a multi-phase scientific evaluation of human nutrient requirements that: (1) evaluates requirements and dietary and supplement intake data for nutrients and non-nutrient food components, (2) develops Dietary Reference Intakes (DRIs) where data are available, (3) provides guidance for how these values should be used in various applications of clinical and public health importance, and (4) identifies research needs. Multi-agency participation is facilitated by an OS/OPHS requirements contract and interagency steering committee. OPHS evaluation funds have supported completed reports on folate and other B vitamins (9/96 to 3/98); dietary antioxidants and related compounds (9/97 to 12/99); micronutrients (9/98 to 12/00); applications of DRIs in dietary assessment (9/99 to 7/00); on macronutrients (fat, carbohydrate, protein (9/99 to 6/02) and in electrolytes (sodium, potassium, chloride) and water (9/01 to 3/03).
PIC ID: 6323; EXPECTED COMPLETION: FY 2003; CONTACT: Kathryn McMurry, 202-401-0751; PERFORMER: National Academy of Sciences, Institute of Medicine, Washington, DC

Assessing the Impact of Language Barriers and Interpreter Services on Cost & Quality of Health Care

This project will study the impact of providing and not providing language assistance services to limited-English proficient (LEP) populations in the context of health care delivery. The project will include an examination of data from actual clinical encounters between physician/patients, design/development of a tool or instrument that can readily identify the nature and extent of medical errors due to language barriers (for LEP populations), initial creation of a model to estimate the impact of these errors on cost and quality of care, and comparisons of error frequency, cost and benefits, impacts, and outcomes between varying levels of interpreter services (no interpreters, untrained interpreters, and medically trained interpreters).
PIC ID: 7711; EXPECTED COMPLETION: FY 2003; CONTACT: Valerie Welsh, 301-443-9923; PERFORMER: Massachusetts General Hospital, Boston, MA

Assessment of State Laws, Regulations and Practices Affecting the Collection and Reporting of Racial and Ethnic Data by Health Insurers and Managed Care Plans

Not having racial and ethnic (r/e) data collected by health insurers and managed care organizations (MCOs) has led to a dearth of r/e data to assess the health status, access to and quality of health services for minorities, as well as the inability to monitor progress towards eliminating racial and ethnic disparities. This project includes a comprehensive review of: (1) the extent to which MCOs and health insurers can collect and report information on the applicants and enrolled members by race and ethnicity; (2) existing interpretation of State laws and regulations governing these entities; (3) parameters and overlap of civil rights versus insurance and managed care laws; and (4) practices of the state officials and these entities at the state level. A database search will be conducted in all 50 States and the District of Columbia to search for State laws and regulations which allow, require and/or prohibit the collection of r/e data. The findings developed in this project will clarify needed steps to develop a system to monitor progress towards the elimination of health disparities, one of the Nation’s health promotion and disease prevention goals for the next decade.
PIC ID: 7479; EXPECTED COMPLETION: FY 2003; CONTACT: Violet Ryo-Hwa Woo, 301-443-9923; PERFORMER: National Health Law Program, Washington, D.C.

Cultural and Linguistic Competence Research Agenda Project

This project would revise and promote an agenda of cost, quality, and outcome-related research on language and cultural competency, including research that would explore issues related to cost-effectiveness, clinical efficiency, and the validation of different methods and models of linguistic and culturally competent interventions, including medical interpretation, translation, and cross cultural provider and program assessment, training, and programming. Drawing on the expertise of researchers, policymakers, and providers already interested in linguistically and culturally competent care (complemented by experts in cost, quality, outcomes research), it would produce not just a onetime report, but sponsor and maintain an online research clearinghouse/networking site for regularly updated and synergistically developed information.
PIC ID: 7497; EXPECTED COMPLETION: FY 2002; CONTACT: Guadalupe Pacheco, 301-443-5084; PERFORMER: Center for the Advancement of Health, Washington, DC

Development and Testing of Cultural Competency Curriculum Modules

The purpose of this project is to develop and test curriculum modules that equip family practitioners with cultural and linguistic competencies. The curriculum modules will be anchored in the principles and concepts established in the National Standards on Culturally and Linguistically Appropriate Services (CLAS) in Health Care which provide the framework for developing the necessary and relevant competencies to increase the quality and effectiveness of health delivery systems and to ensure that health care is not impeded by cultural or linguistic barriers.
PIC ID: 7713; EXPECTED COMPLETION: FY 2003; CONTACT: Guadalupe Pacheco, 301-443-5084; PERFORMER: American Institutes for Research, Washington, DC

Development of Uniform Information/Data Set and Collateral Products for Assessing Impacts of OMH-funded Activities

The contractor will conduct a comprehensive analysis of past and current Office of Minority Health (OMH) funded activities including OMH grant programs, cooperative agreements, and personal services contracts to determine the current types of information/data collected and available from them. The contractor will then develop a uniform set of information/data elements that OMH grantees/partners could be encouraged to collect for OPHS and OMH GPRA performance measurement purposes. A project advisory group of federal and non-federal individuals will be utilized. This project will identify a uniform set of information/data that should be provided by the recipients of OMH funding in order for OMH to be able to demonstrate the “returns for the public’s investment.” The project is to include, but is not limited to, examinations of the findings of the recently completed evaluation of OMH’s Bilingual/Bicultural Services Demonstration Program, work being done within OMH to develop a statistical data progress report, and the related experiences and successes of other DHHS entities--building upon these efforts rather than “reinventing the wheel.” OMH anticipates that as a result of this contract, it will be able to implement a process for obtaining the information/data necessary to demonstrate that the various grant/funded programs supported by OMH make a difference that matters to and resonates with policy and budget decision makers. Collateral technical assistance documents related to the uniform data/information set will also be developed.
PIC ID: 7097; EXPECTED COMPLETION: FY 2002; CONTACT: Violet Ryo-Hwa Woo, 301-443-9923; PERFORMER: Development Services Group, Inc., Bethesda MD

Feasibility Study and Design to Assess Consumer Perceptions of and Experiences with the Provision of CLAS in Their Health Care Delivery System

The purpose of this project is to assess the feasibility of, and develop a plan for, conducting a study of the nature and extent of culturally and linguistically appropriate services (CLAS) from the perspective of the consumers/patients served in targeted delivery settings or communities across the country. The study instrument used for the CLAS in Managed Care Organizations (MCOs) study currently in progress will be used as a basis for the instrument to be used in the consumer/patient study. It is expected that, at a minimum, the instrument will need to be modified to address the perspectives and experiences specific to consumers/patients and will likely require translations of survey protocols and instruments, bilingual interviewers, and/or focus groups and pilot testing.
PIC ID: 7710; EXPECTED COMPLETION: FY 2002; CONTACT: Valerie Welsh, 301-443-9923; PERFORMER: Health Systems Research Associates, Chevy Chase, MD

Implementing Culturally and Linguistically Appropriate Services (CLAS) in Health Care

The purpose of this project is to implement a pilot project that will report on guidelines and processes for implementing the Culturally Linguistically and Appropriate Services (CLAS) standards among health care organizations. The project will document both enabling factors and potential barriers to the implementation of the CLAS standards. Additionally, the pilot project will measure the impact of implementation of the CLAS standards by a major managed care organization (MCO) on both its provider and patient populations. The data collected from the CLAS implementation experience through the pilot test and other processes will be analyzed and the trends, best practices, and lessons learned will be identified and presented in the final report.
PIC ID: 7714; EXPECTED COMPLETION: FY 2003; CONTACT: Guadalupe Pacheco, 301-443-5084; PERFORMER: Macro International, Inc., Calverton, MD

National Assessment of CLAS in Local Public Health Agencies Serving Racially/Ethnically Diverse and Vulnerable Populations

This project proposes to conduct a survey of a national sample of local public health agencies (LPHAs) serving communities or areas with high concentrations of vulnerable and/or racially and ethnically diverse populations to identify the nature and extent of culturally and linguistically appropriate services (CLAS) being provided by these agencies. This survey is intended to result in meaningful national baseline data on the nature and extent of CLAS being provided by LPHAs across the country, especially in communities in need.
PIC ID: 7712; EXPECTED COMPLETION: FY 2003; CONTACT: Betty Hawks, 301-443-5084; PERFORMER: COSMOS Corporation, Bethesda, MD

National Assessment of Culturally and Linguistically Appropriate Services in Managed Care Organizations

This project will conduct a survey of a national random sample of managed care organizations (MCOs) serving racially/ethnically diverse communities. The survey will identify the nature and extent of culturally and linguistically appropriate services (CLAS) being provided by these health plans. It is intended to inform and educate health service executives and managers about the nature and importance of CLAS, encourage policies and practices in support of CLAS for all Americans, and establish meaningful national baseline data on the nature and extent of CLAS being provided across the country. This is especially important in communities where language and cultural differences create access barriers and barriers to quality of care. It is expected that the end products will include an analysis and report of the range of such services, an identification of best practices and model approaches, and an explication of the community and organizational factors conducive to the provision or inhibition of CLAS.
PIC ID: 6674; EXPECTED COMPLETION: FY 2003; CONTACT: Betty Hawks, 301-443-5084; PERFORMER: Cosmos Corporation, Bethesda, MD 20824

National Youth Sports Program Evaluation

The purpose of this initiative is to evaluate the scope and design of the National Youth Sports Program (NYSP) as well as study potential program effects on the knowledge, attitudes, and behaviors of youth participating in the Year 2001 summer program. Activities include conducting a complete review of the NYSP; selecting five program sites to review as case studies; designing and pilot testing pre- and post-student survey instruments; and developing a report on the qualitative review of the NYSP and the pilot survey findings.
PIC ID: 7709; EXPECTED COMPLETION: FY 2002; CONTACT: Joyce Heinonen, 301-594-0769; PERFORMER: Inter-University Program for Latino Research, Notre Dame, IN

Substance Abuse and Mental Health Services Administration

Mission

To improve the quality and availability of prevention, treatment, and rehabilitation services in order to reduce illness, death, disability, and cost to society resulting from substance abuse and mental illnesses.

Evaluation Program

The Substance Abuse and Mental Health Services Administration (SAMHSA) is committed to evaluating its overall programs and individual grant projects to assess the effectiveness of prevention, treatment, and rehabilitation approaches and systems of care; the accountability of federal funds; and the achievement of SAMHSA’s programmatic and policy objectives.

To the greatest extent possible, SAMHSA encourages the use of comparable data elements and instruments across its evaluations in order to work toward a comprehensive evaluation system and to minimize respondent burden.

SAMHSA conducts grant programs under a variety of legislative authorities. These authorities have generally been grouped into two types: (1) services and (2) knowledge development activity (KDA). The evaluation required for a particular grant program is dependent on the type and purpose of the program. SAMHSA evaluates each of its service programs so as to provide information to program managers about the accountability of federal funds.

SAMHSA has an integrated model of evaluation and planning. Strategic planning identifies priorities, such as managed care, that drive the development of grant programs and evaluations. In compliance with the Government Performance and Results Act (GPRA), SAMHSA is improving performance management by identifying annual performance objectives and goals. The formulation of programmatic and evaluation priorities includes consultation with SAMHSA Center Advisory Councils, with other HHS agencies, and with other experts in the fields of evaluation and service delivery. Early and continuous coordination of program planning and evaluation design will result in the articulation of program objectives that may be evaluated. Evaluations will demonstrate the extent to which the grant programs have achieved their overall objectives, and SAMHSA will translate these results into information that can be used for program and policy development. The strategic planning and policy development processes will then use these results to refine SAMHSA’s priorities and performance objectives.

Completed Evaluations

Evaluation of the Community Action Grant Program Phase I - Round I Grantees

The Community Action Grant Program (CAG) facilitates the adoption and implementation of exemplary practices in mental health services for adults with serious mental illness and children with severe emotional disturbance. The grant program has two phases, each one year long. In Phase I, applicants are funded to convene stakeholders and build consensus around the adoption of a chosen practice, as well as identify a funding source to fund the practice and implementation. An evaluation of CAG grantees from the first year of the program found that all 17 grantees in the study reported achieving consensus around an exemplary mental health service practice. Grantees engaged a wide array of stakeholders and employed various tactics and consensus building models to overcome barriers in moving towards consensus. This preliminary evaluation serves as a stepping stone for other HIRI evaluation activities examining later rounds of the CAG program as well as an evaluation led by the Association for the Study and Development of Community, looking at trends in Phase I grantees since the program’s inception. Information from all three evaluations will be used by CMHS staff to guide the development of the CAG program through activities such as revising the program announcement and reporting requirements, as well as improving technical assistance.
PIC ID: 7716; CONTACT: David Morrissett, 301-443-3653; PERFORMER: Human Interaction Research Institute, Los Angeles, CA

The Impact of Prior Physical and Sexual Victimization on Substance Abuse Treatment Outcomes

This technical report presents a secondary analysis of the National Treatment Improvement Evaluation Study (NTIES) data to assess the effects of pre-treatment physical and sexual violence on substance abuse treatment outcomes. The relationship between treatment outcomes and past physical and sexual abuse are analyzed for males and females in five treatment modalities: methadone, non-methadone outpatient treatment, short-term residential treatment, long-term residential treatment, and treatment facilities located in correctional institutions. Results indicate that victimization by physical and/or sexual violence was a frequent event in the life histories of persons entering substance abuse treatment that was part of NTIES. Lifetime physical/sexual abuse was reported by three-fourths (73%) of the respondents in the outcome sample. Of the 60 estimated mixed-model regression equations, the impact of prior victimization on treatment outcomes was significant in the expected direction, namely, victimization was associated with poorer outcomes at follow-up.
PIC ID: 5994.11; CONTACT: Ron Smith, 301-443-7730; PERFORMER: Caliber Associates, Fairfax, VA

In-Progress Evaluations

An Evaluation of PASRR and Mental Health Services for Persons in Nursing Facilities

The process of screening and determining whether nursing facility services and specialized mental health care are needed by nursing facility applicants and residents is called the Preadmission Screening and Resident Review (PASRR) program. The PASRR program is a required component of each state’s Medicaid plan. More than a decade after Congress enacted the PASRR program to prevent the inappropriate admission and retention of people with mental disabilities in nursing facilities, many concerns regarding the efficacy and effectiveness of the legislation remain. Some national organizations say PASRR creates logistical barriers for persons requiring nursing facility placements and does not ensure access to appropriate (i.e., medically necessary) psychiatric services. The purposes of this study are to examine the: (1) intent, scope, and jurisdiction of PASRR; (2) implementation of PASRR at the state and nursing facility level; and (3) effect that PASRR has had on the identification of people with serious mental illness in nursing facilities.
PIC ID: 7729; EXPECTED COMPLETION: FY 2003; CONTACT: Judith Teich, 301-443-0588; PERFORMER: Lewin Group, San Francisco, CA

Consumer-Operated Services Program

The Consumer-Operated Services Program (COSP) is a four-year multisite evaluation of the effectiveness of consumer-run mental health services when provided as an adjunct to traditional mental health services. Begun in September 1998, seven projects are evaluating eight service programs in eight states, and a Coordinating Center is coordinating the multisite effort. Each participant enrolled in COSP is randomized to one of two conditions: (1) consumer-operated services plus traditional mental health services, or (2) traditional mental health services alone. Baseline enrollment ended on September 30, 2001 with over 1,900 participants enrolled. This project will assess the degree to which consumer-operated services and traditional mental health services work together through application of a systems integration measure to data collected from the sites during site visits to be made by the Coordinating Center.
PIC ID: 7724; EXPECTED COMPLETION: FY 2004; CONTACT: Betsy McDonel-Herr, 301-443-3653; PERFORMER: Missouri Institute of Mental Health, St. Louis, MO

Cooperative Agreements to Evaluate Housing Approaches for Persons with Serious Mental Illness-Phase II

The Housing Initiative, a multi-site program initially funded in 1997, was designed to evaluate the effectiveness of different housing approaches for persons with serious mental illness. The program was designed to be implemented in two phases. This project, Phase II, is designed to complete the analysis of the cross-site study and disseminate the findings through a combination of reports, conference presentations, and publications. The lack of available housing slots to place clients made it impossible for the individual sites to accrue their target samples within the time originally allocated. Consequently, the six sites were all given one-year extensions to complete their data collection activities and the analysis of the cross-site evaluation.
PIC ID: 7726; EXPECTED COMPLETION: FY 2003; CONTACT: Pam Fischer, 301-443-3706; PERFORMER: Vanderbilt University, Nashville, TN

Evaluation of the Community Action Grant Program--Phase I, Rounds 2 and 3, and Phase II, Round 1

The Community Action Grant Program (CAG) facilitates the adoption and implementation of exemplary practices in mental health services for adults with serious mental illness and children with severe emotional disturbance. The grant program has two phases, each one year long. In Phase I, applicants are funded to convene stakeholders (e.g., consumers, policy-makers, providers) and build consensus around the adoption of a chosen practice, as well as identify a funding source to fund the practice implementation. In Phase II, applicants who have successfully completed Phase I are given funding to maintain stakeholder support and aid in the implementation of the chosen practice (funds are not used for direct service delivery). This study will also evaluate consensus building processes, grantee reporting, and barriers to consensus building around the applicant’s chosen practice. The evaluation of Phase II grantees from Round 1 examines variables such as practice implementation, barriers to implementation, and commitments for the maintenance of exemplary practices.
PIC ID: 7716.1; EXPECTED COMPLETION: FY 2002; CONTACT: David Morrissett, 301-443-3653; PERFORMER: Human Interaction Research Institute, Los Angeles CA

Evaluation of the Comprehensive Community Mental Health Services for Children and their Families Program - Phase II

The Center for Mental Health Services (CMHS) is conducting an evaluation of 23 sites of the Comprehensive Community Mental Health Services for Children and their Families Program funded in fiscal years 1997 and 1998. Evaluators hired at each site currently gather descriptive data on such measures as demographics, functional status, diagnosis, and risk factors of enrolled children and their families. They are collecting outcome data in areas such as social functioning, emotional and behavioral strengths, substance abuse, school performance, and juvenile justice involvement in a representative sample of about 280 children per site. To obtain the outcome data, children are assessed at the time of entry (i.e., baseline) and every six months for up to 36 months. External evaluators visit the sites annually to study the development of systems of care according to system-of-care principles. In addition, two systems of care are being compared with two usual service delivery systems to study the effectiveness of systems of care.
PIC ID: 7269.1; EXPECTED COMPLETION: FY 2003; CONTACT: Rolando L. Santiago, 301-443-1333; PERFORMER: ORC Macro, Calverton, MD

Evaluation of the Comprehensive Community Mental Health Services for Children and their Families Program - Phase III

CMHS began a third phase of the evaluation with a six-year evaluation of 22 new grant communities funded in fiscal years 1999 and 2000. As of August 31, 2001, 2,333 children had enrolled in systems of care funded during this phase of the program. Evaluators hired at each site are gathering descriptive data on measures such as demographics, functional status in the home, school and community, diagnostic status, and participation in the Child Health Insurance Program of enrolled children and their families. Preliminary results indicate that with respect to diagnostic status, 31.8 percent of children are diagnosed with attention deficit hyperactivity disorders, 29.4 percent with mood disorders, 12.4 percent with oppositional defiant disorder, 4.8 percent with post-traumatic and acute stress disorder, 40 percent with conduct disorder, 3.8 percent with impulse control problems, and 14.0 percent with other disorders such as autistic and related, adjustment, psychosis, anxiety, among others.
PIC ID: 7269.2; EXPECTED COMPLETION: FY 2005; CONTACT: Rolando L. Santiago, 301-443-1333; PERFORMER: Macro International, Inc., Calverton, MD

Evaluation of the Effectiveness of Time-Limited Interventions for Homeless Families

The Center for Mental Health Services and the Center for Substance Abuse Treatment are collaborating in the funding and administration of a two-phase, five-year knowledge development initiative to document and evaluate the effectiveness of time-limited, intensive intervention strategies for providing mental health and/or substance abuse treatment, trauma recovery, housing, support, family preservation, and other wrap-around services to homeless mothers with psychiatric and/or substance use disorders who are caring for their dependent children. In the Fall of 1999, Phase I began a two-year process with 14 study sites to develop population descriptions, document their intervention models, collaborate in the development of a cross-site interview instrument, and formulate cross- and individual-site evaluation plan designs. In Phase II, which began in October 2001, study sites will conduct cross- and individual-site evaluations of their homeless families interventions, participate in developing the plan for analyzing the cross-site data, and begin collaboration on knowledge application and dissemination activities.
PIC ID: 7725; EXPECTED COMPLETION: FY 2004; CONTACT: Larry Ricards, 301-443-3706; PERFORMER: Policy Research Associates, Delmar, NY

Evaluation of the Effects of the California Mental Health Parity Act

In the past several years, federal and state legislation has been enacted to require parity benefits (equivalence between mental health benefits and general health care benefits in health insurance plans). These legislative efforts to address parity are often limited in terms of their scope or application. California enacted parity legislation in 1999, which became effective in 2000. It provides for equal coverage for severe mental illnesses, and explicitly covers children with one or more mental disorders. Further, unlike many other States, small businesses are not exempt from the law’s requirement. Due to the size and complexity of California’s economy and health care market, it is important for SAMHSA to evaluate the effects of this parity mandate. Consequently, the Center for Mental Health Services (CMHS) recently began a two-year study which will address such questions as: What were specific issues/problems in the implementation of the parity legislation? How have costs and utilization changed as a result of parity? What are consumer, employer, insurer, and provider opinions about the effects of the law?
PIC ID: 7727; EXPECTED COMPLETION: FY 2003; CONTACT: Judith Teich, 301-443-0588; PERFORMER: Mathematica Policy Research, Inc., Plainsboro, NJ

Evaluation of the Effects of the Vermont Parity Act

In 1997, Vermont enacted parity legislation. Effective January 1, 1998, or upon renewal, the law requires health plans to provide coverage for the treatment of “mental health conditions” and prohibits rates, terms, or conditions that impose a greater financial burden for access to treatment for mental illness than for physical illness. Unlike many other parity laws, Vermont’s statute includes substance abuse and does not restrict parity to specific diseases. Vermont’s parity law is more comprehensive than most others. It, therefore, offers a unique opportunity to learn about the actual effects of a full parity mandate. Some of the questions to be addressed in this evaluation include: What mandates governed MH/SA benefits prior to parity? What specifically does the law require? Were there identifiable public/private cost shifts? Has improved commercial reimbursement provided increased flexibility for use of funds from non-commercial sources (e.g., have dollars been redirected to prevention programming)? Do consumers report improved access and fewer barriers to MH/SA services as a result of the law?
PIC ID: 7728; EXPECTED COMPLETION: FY 2002; CONTACT: Judith Teich, 301-443-0588; PERFORMER: Mathematica Policy Research, Inc., Plainsboro, NJ

Evaluation of the HHS Access to Community Care and Effective Services and Supports (ACCESS) Program

The ACCESS Program is a five year demonstration program that provides grants to nine States to develop integrated systems of treatment and supportive services and housing for homeless persons with serious mental illnesses. The goal of the program is to identify strategies for developing integrated service systems and to evaluate their effectiveness in providing services to homeless persons with serious mental illnesses. The evaluation of the ACCESS Program has two major components: The system-level evaluation identifies the different systems integration approaches, documents how they are implemented, and measures their impact on system capacity and system barriers. The client-level evaluation determines whether systems integration efforts result in improved service delivery, improvements in mental health and health status, rehabilitation, quality of life and permanent exit from homelessness.
PIC ID: 4980.1; EXPECTED COMPLETION: FY 2003; CONTACT: Randolph Frances, 301-443-3706; PERFORMER: R.O.W. Sciences, Inc., Rockville, MD

National Evaluation of the Comprehensive Community Mental Health Services for Children and their Families Program - Phase IV

The Center for Mental Health Services (CMHS) will start a new phase of the evaluation of the Comprehensive Community Mental Health Services for Children and their Families Program with a six-year evaluation of about 20 new grant communities funded in fiscal year 2002. Evaluators hired at each site will gather descriptive data on measures such as demographics, functional status in the home, school, and community, diagnostic status, and resiliency and protective factors. Local evaluations will collect outcome data in areas such as behavioral and emotional strengths, school performance, and juvenile justice involvement in a representative sample of children. In addition, the national evaluation contractor will provide assistance in the development of a community-based management information system for monitoring service use and costs across child-serving sectors such as mental health, juvenile justice, child welfare, and education, especially those services and costs associated with an individualized care plan.
PIC ID: 7269.3; EXPECTED COMPLETION: FY 2008; CONTACT: Rolando L. Santiago, 301-443-1333; PERFORMER: ORC Macro, Calverton, MD

National Evaluation of the Safe Schools/Healthy Students

The Safe Schools/Healthy Students Interdepartmental Initiative (SS/HS) is a landmark effort supported by an unprecedented collaboration among the Departments of Education, Health and Human Services, and Justice. The goals are to: (1) help students develop the skills and emotional resilience necessary to promote positive mental health, engage in prosocial behavior, and prevent violent behavior and drug use; (2) ensure that all students learn in a safe, disciplined, and drug-free environment; and (3) develop an infrastructure that will institutionalize and sustain integrated services. This national evaluation of SS/HS is designed to document how coalitions and collaborations at the community level develop systemic change that results in beneficial outcomes for children, families, and schools.
PIC ID: 7717; EXPECTED COMPLETION: FY 2004; CONTACT: Office of Juvenile Justice, 202-514-4817; PERFORMER: Department of Justice, Juvenile Justice, Washington, DC

The Women, Co-Occurring Disorders, and Violence Study

SAMHSA is sponsoring a multi-site demonstration to assess the extent to which comprehensive, integrated, trauma-informed services can make a difference in the lives of women who have co-occurring mental health and substance abuse disorders and who have also experienced sexual and/or physical abuse. The evaluation involves a quasi-experimental design comparing study participants to comparison groups using a common interview protocol at baseline, six months, and twelve months to assess variations in outcomes across the involved nine sites. Study participants are currently being recruited and, thus, no outcome findings are available.
PIC ID: 7721; EXPECTED COMPLETION: FY 2003; CONTACT: Community Support Programs, 301-443-3653; PERFORMER: Policy Research Associates, Delmar, NY

Cross-site Evaluation of the Community Prevention Coalitions Demonstration Grant Program

This project will conduct a cross-site evaluation of the Community Prevention Coalition (CPC) grant program over a five-year period. The goals of the evaluation are to assess: (1) the extent to which coalitions of service organizations created by the program are able to implement comprehensive and coordinated alcohol, tobacco, and other drug (ATOD) prevention programs through an expanded and non-duplicative service delivery base; (2) the extent to which coalitions are able to achieve measurable reductions in the incidence and prevalence of ATOD use, and improvements in ATOD-related social, health and safety consequences among all age, ethnic and vulnerable groups within grantee communities; (3) the contributions of various prevention strategies to site-specific outcomes and to general prevention outcomes in those sites where measurable changes are made; and (4) those characteristics common to coalitions where measurable changes are achieved, as well as those that differentiate successful coalitions from unsuccessful ones. The contractor will be required to collect process and outcome data for analysis from coalitions. In addition, data collected locally by grantees and archived community-wide indicators will be used.
PIC ID: 6042; EXPECTED COMPLETION: FY 2002; CONTACT: Shakeh Kaftarian, 301-443-9136; PERFORMER: Cosmos Corporation, Bethesda, MD

Evaluation of Opioid Treatment Program Accreditation Project

The Center for Substance Abuse Treatment (CSAT) and other federal agencies are developing a regulatory framework for opioid substitution therapy that will change the current regulatory system for monitoring opioid treatment programs (OTPs) to an accreditation/regulatory approach. Over a five year period, CSAT will conduct and evaluate a phase-in program to obtain information that will aid in the full implementation of the new system. The goal of the evaluation is to obtain information that can guide the full national implementation of the new system for OTPs through systematic study of the processes, barriers, costs associated with a change from regulatory to an accreditation/regulatory process, and various other impacts.
PIC ID: 6739; EXPECTED COMPLETION: FY 2002; CONTACT: Herman Diesenhaus, 301-443-6575; PERFORMER: Research Triangle Institute, Research Triangle Park, NC

National Evaluation Data Services

The National Evaluation Data Services (NEDS) was established to increase the number of scientifically based analyses to answer vital questions in the substance abuse treatment field. NEDS represents part of an overall CSAT evaluation strategy that builds upon prior findings and seeks to mine existing data whose potential has not been fully explored. One objective of the NEDS project is to provide a wide array of data management and scientific support services across various program and evaluation activities, including an information system of available data related to the evaluation of substance abuse treatment. Another objective is to provide CSAT with a flexible analytical capability to use existing data to address policy-relevant questions about substance abuse treatment, and to clearly articulate the resulting analyses as they relate to policy through fact sheets, evaluation summaries, technical reports, methods development documents and technical guidance documents.
PIC ID: 5994.4; EXPECTED COMPLETION: FY 2002; CONTACT: Ron Smith, 301-443-7730; PERFORMER: Caliber Associates, Fairfax, VA

Persistent Effects of Treatment Studies (PETS)

The purpose of PETS is to evaluate the long-term effectiveness (up to thirty-six months) of substance abuse treatment services provided through a series of CSAT grants and cooperative agreements, and to conduct a number of special studies and policy analyses that address specific drugs of abuse, methods of treatment, populations, or policy issues. The primary follow-up studies will involve obtaining information at selected follow-up periods for persons who have completed the index treatment episode and for whom historical, intake and treatment exit data are available.
PIC ID: 6738; EXPECTED COMPLETION: FY 2002; CONTACT: Ron Smith, 301-443-7730; PERFORMER: Westat, Inc., Rockville, MD

Primary Care Research in Substance Abuse and Mental Health Services for the Elderly (PRISMe)

SAMHSA has developed a multisite study to compare the effectiveness of service delivery models that treat older adults with mental health and substance abuse (MH/SA) problems in primary care as opposed to specialty MH/SA settings. The study hopes to identify differences in clinical and cost outcomes between models referring consumers to specialty MH/SA services outside the primary care setting and those providing such services within the primary care settings. This four-year study plans to randomize close to 3,000 patients with MH/SA problems to either integrated or referral models of MH/SA care. This initiative includes the active collaboration of all three SAMHSA centers, with the Center for Mental Health Services (CMHS) as the lead, the Health Resource Services Administration’s Bureau of Primary Care, the Department of Veterans Affairs, and the Centers for Medicaid and Medicare Services.
PIC ID: 7722; EXPECTED COMPLETION: FY 2003; CONTACT: Betsy McDonel-Herr, 301-443-3653; PERFORMER: Harvard Medical School, Laboratory of Community Psychiatry, Boston, MA

State Substance Abuse Managed Care Evaluation Program

The State Substance Abuse Managed Care Evaluation Project will evaluate the effects of managed care on mental health and substance abuse services in five states. Each of these five state studies will inform future behavioral health care policy.
PIC ID: 6454; EXPECTED COMPLETION: FY 2002; CONTACT: Joan Dilonardo, 301-443-8555; PERFORMER: Brandeis University, Waltham, MA

Treatment Improvement Protocols Field Evaluation

More than 25 Treatment Improvement Protocols (TIPs) have been distributed to the public. The purpose of TIPs is to transfer protocols and guidelines for the treatment of alcohol and other drug (AOD) abuse from acknowledged clinical, evaluation and administrative experts to the nation’s AOD treatment field. Institution of a field evaluation is needed to: (1) assess the extent to which target audiences realize, read and implement TIPs; and (2) learn ways to strengthen the development, formatting, marketing, dissemination, use and evaluation of TIPs to ensure effective use of future federal resources devoted to TIPs. The overall objective of the proposed evaluation is to learn the impact of TIPs on the substance abuse treatment field. The study will examine methods of dissemination used by CSAT, the success of those methods in reaching the target audiences, users’ perceptions of the value of TIPs, decisions to implement TIPs, and the successes, correlates and barriers associated with implementation.
PIC ID: 6737; EXPECTED COMPLETION: FY 2002; CONTACT: Kevin Mulvey, 301-443-9472; PERFORMER: Johnson, Bassin and Shaw, Inc., Silver Spring, MD

Appendix A

Appendix A

Evaluation in the Department of Health and Human Services

Evaluation in the Department of Health and Human Services

Evaluation plays an integral role in carrying out the HHS mission. Assessing various aspects of agency program performance allows staff to identify means of improving that performance. The HHS evaluation function has three goals: (1) to provide information on HHS programs that helps government officials and members of the Congress make decisions related to programs, policies, budgets, and strategic planning; (2) to help HHS managers improve program operations and performance; and (3) to disseminate evaluation results and methodological tools useful to the larger health and human services community of state and local officials, researchers, advocates, and practitioners for improving the performance of their programs.

This appendix presents the organization and operation of evaluation at HHS. Described first is the definition and basic purposes of evaluation at HHS, followed by information on resources and funding mechanisms used to support evaluations. The last section of the appendix provides more details on various aspects of the management of HHS evaluation activities, such as the role played in departmental planning initiatives; how evaluation projects are managed; how HHS agencies ensure technical quality of evaluation projects, and promote effective dissemination and use of evaluation results.

Definition and Purpose of Evaluation at HHS

Definition and Purpose of Evaluation at HHS

HHS defines evaluation as the assessment of the performance (efficiency, effectiveness, and responsiveness) of HHS programs or strategies through the analysis of data or information collected systematically and ethically, and the effective use of resulting information in strategic planning, program or policy decision making and program improvement. As mentioned in the foreword of this report, evaluations serve one or more of the following purposes:

Program effectiveness provides a way to determine the impact of HHS programs on achieving intended goals and objectives.

Performance measurement is the primary mechanism used to monitor annual progress in achieving departmental strategic and performance goals. To support performance measurement, we are investing evaluation funds to develop and improve performance measurement systems and the quality of the data that supports those systems.

Environmental assessment is the way we understand the forces of change in the health and human services environment that will influence the success of our programs and the achievement of our goals and objectives. In turn, this understanding allows us to adjust our strategies and continue to deliver effective health and human services.

Program management reflects the need of program managers to obtain information or data helpful for effectively designing and managing a program. These evaluations generally focus on developmental or operational aspects of program activities and provide understanding of services delivered and populations served.

Evaluation Resources

Evaluation Resources

Evaluation activities of the various HHS agencies are largely supported through two funding mechanisms: direct use of program funds and use of special legislative set-aside authorities for evaluation. The first is a common mechanism by which programs managers have discretionary authority to use appropriated program funds to support contracts that will design, implement, and analyze evaluation data. In some cases, a program’s legislative authority calls for a specially mandated evaluation, and program funds are used directly to support the evaluation.

The second mechanism for evaluation funding is the legislative set-aside authority which permits the Secretary of HHS to use a proportion of overall program funds for evaluation purposes. The largest of such set-aside authorities at HHS is one established for evaluations conducted by several agencies of the U.S. Public Health Service (AHRQ, CDC, HRSA, NIH, and SAMHSA), ASPE, and the Office of Public Health and Science (OPHS) in the Office of the Secretary. The mechanism is called the PHS evaluation set-aside legislative authority, which is provided for in Section 241 of the Public Health Service (PHS) Act. This authority was established in 1970, when the Congress amended the Act to permit the HHS Secretary to use up to 1 percent of appropriated funds to evaluate authorized programs. Section 206 of the FY 2002 Labor, HHS, and Education Appropriations Act increases the amount the Secretary may use for evaluation to 1.25 percent. Section 241 limits the base from which 1.25 percent of appropriated funds can be reserved for evaluations of programs authorized by the PHS Act. Excluded are funds appropriated for FDA, IHS[1] and certain other programs that are managed by PHS agencies but not authorized by the Act (e.g., HRSA’s Maternal and Child Health Block Grant and CDC’s National Institute for Occupational Safety and Health).

Evaluation Management

Evaluation Management

The management of HHS evaluations, which are carried out on a regular basis by the HHS agencies and offices and coordinated by ASPE, involves four basic functions: (1) evaluation planning and coordination, (2) project management, (3) quality assurance, and (4) dissemination of evaluation reports. A description of each function in general terms follows.

Evaluation Planning and Coordination

The HHS agencies, ASPE, the Office of the Inspector General (OIG), and OPHS develop evaluation plans annually in concert with HHS’s program planning, legislative development, and budgeting cycles. Plan development is coordinated by ASPE. Each agency or office plan generally states the evaluation priorities, or projects under consideration for implementation. Typically, HHS evaluation priorities include: congressionally-mandated program evaluations, evaluations of Secretarial program or policy initiatives, assessments of new programs and ones that are candidates for reauthorization, and evaluations that support program performance management and accountability.

More specifically, HHS evaluation planning activities are coordinated with three department-wide planning initiatives. First, HHS evaluation activities support the Department’s strategic planning and performance management activities in several ways. Completed evaluation studies are used in shaping the specific HHS strategic goals and objectives. Evaluation findings provide an important source of information or evidence about the success of various HHS programs or policies that collectively make up the strategies to achieve the goals and objectives. The HHS Strategic Plan highlights evaluations that document efficacy or effectiveness of strategic programs or policies and lists future evaluations that will benefit strategic planning. HHS agencies are also using findings from their evaluations to support GPRA annual performance reporting to Congress and program budget justifications.

Secondly, Congress requests that HHS coordinate all of its research, demonstration, and evaluation (RD&E) programs to ensure that the results of these projects address HHS program goals and objectives. ASPE and the Assistant Secretary for Budget, Technology and Finance work together with HHS agencies to provide the Congress with a special annual research, demonstration, and evaluation budget plan that coincides with the preparation of the President’s fiscal year budget. The plan outlines HHS agency research, demonstration, and evaluation priorities as related to the Department’s strategic goals and objectives. ASPE plans to use the RD&E reporting mechanism as a research planning format.

The newly-formed Research Coordination Council (chaired by the ASPE, and containing representatives of HHS agencies), will foster greater interactions among its research programs. The Council will also begin to streamline research and evaluate Department-wide research priorities to ensure greater efficiencies in research, demonstration, and evaluation. The Council will strengthen HHS research coordination and planning around key Departmental priorities and themes.

Thirdly, those agencies and offices that use the PHS one-percent evaluation set-aside authority-AHRQ, CDC, HRSA, NIH, ASPE, OPHS, SAMHSA-submit a formal plan to ASPE, which coordinates and develops the individual plans into the HHS report to the Congress on the use of the PHS one-percent authority. This report must be submitted to the Congress before HHS can implement the plan.

Project Management

HHS agencies, OIG, and ASPE all are responsible for executing annual evaluation plans, developing evaluation contracts, and disseminating and applying evaluation results. All agencies and their subunits are encouraged to coordinate with each other on research/evaluation project planning and release of final reports that are likely to relate to work of other HHS agencies. Within agencies - while there is some oversight responsibility and execution capability in the Office of the Director or Administrator - the various subunits (centers, institutes, and bureaus) conduct much of the day-to-day evaluation activity.

The OIG performs independent evaluations through its Office of Evaluations and Inspections (OEI). The OEI’s mission is to improve HHS programs by conducting inspections that provide timely, useful, and reliable information and advice to decision makers. This information (findings of deficiencies or vulnerabilities and recommendations for corrective action) is usually disseminated through inspection reports issued by the Inspector General. A summary of individual inspection reports and other OIG reports can be viewed on the Internet at < http://oig.hhs.gov/oei/index.html>. OEI also provides technical assistance to HHS agencies in conducting their evaluations.

Quality Assurance and Improvement

Most evaluation projects are developed at the program level. The initial quality review is generally conducted by a committee of agency- or office-level policy and planning staff members. Before a project is approved, however, it is also reviewed for technical quality by a second committee with expertise in evaluation methodology. Technical review committees follow a set of criteria for quality evaluation practice established by each agency. Some HHS agencies also have external evaluation review committees composed of evaluation researchers and policy experts from universities and research centers.

Since HHS began reporting to Congress in 1995 on evaluations completed and in-progress through the Performance Improvement report series, the Department has focused more attention on improving the quality of evaluation studies performed every year. The HHS Evaluation Review Panel, convened yearly to recommend evaluations for highlighting in Chapter II of this report, has contributed insights to HHS evaluation officers on the strengths and challenges of ensuring quality evaluation studies. HHS evaluation officers have had opportunities to discuss these strengths and challenges with the Panel and to identify steps to improve agency evaluation capacity.

Based on these discussions, ASPE has initiated two evaluation capacity-building projects. The first project was to establish an evaluation consultant Blanket Purchase Agreement (BPA) contract mechanism to assist agency program managers in obtaining short term assistance in developing evaluation designs or monitoring the performance quality of evaluation contractors. The second project was to conduct a department-wide evaluation training needs assessment survey, which indicates training needs in various areas.

Dissemination of Evaluation Reports

Maintaining report libraries and distributing information on evaluation results is an important component of HHS evaluation management. The Department’s information and reports on major evaluations is available centrally through the website of the HHS Policy Information Center (PIC), located at the following address: <http://aspe.hhs.gov/pic/>. The PIC website offers users an opportunity to search (by key word or by selected program or policy topics) the departmental evaluation database and electronic report library maintained by ASPE. As an information database and library resource, the PIC contains over 8,000 completed and in-progress evaluation and policy research studies conducted by the Department, as well as key studies completed outside of HHS by the U.S. General Accounting Office (GAO) and private foundations.

Typically, the results of HHS evaluations are disseminated through targeted distribution of final reports, articles in refereed journals, and presentations at professional meetings and conferences. Although individual HHS agencies have primary responsibility for disseminating results, the ASPE will continue its Department-wide efforts to expand dissemination of evaluation results to the larger research and practice communities through centralized computer communications and publications like the PIC website and project information database.

[1] — FDA programs are principally authorized by legislation other than the PHS Act, specifically the authority of the Agriculture, Rural Development, Food and Drug Administration and Related Agencies Appropriations Act. IHS programs are authorized under the Indian Health Care Improvement Act and the Indian Self-Determination Act, and appropriated under the Department of the Interior and Related Agencies Appropriations.

Appendix B

Appendix B

Acknowledgments of HHS Officials

Acknowledgments of HHS Officials

Performance Improvement 2002: Evaluation Activities of the U.S. Department of Health and Human Services describes the continuous efforts of the various HHS agencies to examine service and research programs for the efficiency of their operations and their effectiveness in achieving objectives. The following persons from HHS agencies contributed to preparing information on evaluation projects:

Administration for Children and Families
Office of Planning, Research, and Evaluation
James V. Dolson
Mary Ann MacKenzie

Administration on Aging
Office of Planning and Evaluation
Frank Burns
David Bunoski

Agency for Healthcare Research and Quality
Office of the Administrator
Kathleen Kendrick
David Introcaso

Agency for Toxic Substances and Disease Registry
Office of Policy and External Affairs
Larry Young

Centers for Disease Control and Prevention
Office of Program Planning and Evaluation
Susan Cleveland

Centers for Medicare and Medicaid Services
Office of Strategic Planning
William Saunders
Sydney Galloway

Food and Drug Administration
Office of Policy, Planning, and Legislation
Theresa Mullin
Mary Bobolis

Health Resources and Services Administration
Office of Planning and Evaluation
Roger Straw
Willine Carr

Indian Health Service
Office of Public Health
Elizabeth Fowler
Debbie Melton

National Institutes of Health
Office of Evaluation
Joan Bailey
John Uzzell
Diane Buckley

Office of Public Health and Science
Omar Passons
Valerie Welsh

Substance Abuse and Mental Health Services Administration
Office of Policy and Program Coordination
George Fitzelle

Preparation of this report was managed by the Office of the Assistant Secretary for Planning and Evaluation, Office of Planning and Policy Support. Responsible officials include: Mike Herrell, Vijaya ChannahSorah, and Elsie Quinones. Annette Johnson provided clerical support. Additional information and support was provided by Elizabeth Lower-Basch, Martha Moorehouse, Roxanne Gibson, Floyd Brown, and Gavin Kennedy. Thanks also to Paul Johnson, formerly of ASPE, for consultative technical support.

Susan Belsinger and Marilyn Henry of Sanad Support Technologies, Inc. provided database management as well as support in the development and production of the report.

Cheryl J. McDonnell of James Bell Associates and Norma Gavin of Research Triangle Institute provided support and materials for the evaluation review panel process.

Appendix C

Appendix C

HHS Evaluation Review Panel

HHS Evaluation Review Panel

The following individuals served on the evaluation review panel that made recommendations on the reports highlighted in Chapter I of the Performance Improvement 2002: Evaluation Activities of the U.S. Department of Health and Human Services:

Douglas Barnett, Ph.D.
Department of Psychology
Wayne State University

Heather Becker, Ph.D.
University of Texas-Austin
School of Nursing
Austin, TX

Leslie J. Cooksy, Ph.D.
Center for Community Development & Family Policy
College of Human Services, Education, and Public Policy
University of Delaware
Newark, DE

Holly Korda, Ph.D.
Consulting Principal
Health Systems Research Associates
Chevy Chase, MD

John Kralewski, Ph.D.
University of Minnesota
School of Public Health
Minneapolis, MN

Kenneth McLeroy, Ph.D.
Texas A & M
Associate Dean for Academic Affairs
Bryan, TX

Marv Mandell, Ph.D.
Policy Sciences Graduate Program
University of Maryland, Baltimore County
Baltimore, MD

Melvin Mark, Ph.D.
Professor of Psychology
Penn State University
University Park, PA

Donna Mertens, Ph.D.
Gallaudet University
Washington, DC

Joy Quill
C. J. Quill & Associates, Inc.
Annapolis, MD

Contractors for Review Panel Process:

Cheryl J. McDonnell, Ph.D.
James Bell Associates
Arlington, VA 22201

Norma I. Gavin, Ph.D.
Research Triangle Institute
Research Triangle Park, NC

Project Manager: Vijaya ChannahSorah, Ph.D.
Office of the Assistant Secretary for Planning and Evaluation, HHS

Project Assistant: Elsie Quinones
Executive Potential Program
On Rotation with the Assistant Secretary for Planning and Evaluation, HHS

Program Evaluation Review Criteria

Program Evaluation Review Criteria

The following criteria were used by the HHS Evaluation Review Panel to select evaluation reports highlighted in Chapter I of Performance Improvement 2001:

Overall Significance

Overall Significance

• The study addresses a significant issue of policy relevance.
• Evaluation findings are likely to be useful.

Conceptual Criteria

Conceptual Criteria

Conceptual Foundations

• A literature review is included.
• The project is shown to be logically, based on previous findings; the report uses either theory, or models, or both.
• The program assumptions are stated.
• The evaluation draws from any previous evaluation.
• The report is linked with a program and describes the program.
• The report presents multiple perspectives.
• Multiple relevant stakeholders are consulted and involved.
• The timing is appropriate because the program is ready for evaluation.

Questions for Evaluation

• The aims of the evaluation are clear, well-specified, and testable.
• The questions are feasible, significant, linked to the program, appropriate for the resources and audience, and derive logically from the conceptual foundations.
• The questions show ingenuity and creativity.

  Findings and Interpretation

  • The conclusions are justified by the analyses.
  • The summary does not go beyond what the data will support.
  • The appropriate qualifiers are stated.
  • The conclusions fit the entire analysis.
  • Equivocal findings are handled appropriately.
  • The initial questions are answered.
  • The interpretation ties in with the conceptual foundation.
  • The report notes that the findings are either consistent with or deviate from the relevant literature.
  • The presentation is understandable.
  • The results have practical significance.
  • The extent of program implication is assessed.

  Recommendations

  • The recommendations follow from findings, are worth carrying out, and are affordable, timely, feasible, useful, and appropriate.
  • The recommendations are shown to be relevant to the questions asked.
  • The breadth of specificity of the recommendations is addressed.
  • Any recommendations for either future evaluations, or improvements, or both are clearly presented.

  Methods

  Evaluation Design

• Design considerations include overall appropriateness, soundness, feasibility, funding and time constraints, generalizability, applicability for cultural diversity, assessment of the extent of program delivery, validity, feasibility for data collection, reliability of selected measurements, use of multiple measures of key concepts, and appropriateness of the sample.
• Variables are clearly specified and fit with the questions and concepts.

The design permits measurement of the extent of program implementation and answering of the evaluation questions.

Data Collection

• Data are collected using appropriate units of measurement for analysis, controls for participant selection and assignment bias, and proper handling of missing data and attrition.
• Data collection is characterized by use of an appropriate comparison group of control; adequate sample size, response rate, and information about the sample; a data collection plan; data collection that is faithful to the plan; attention to and cooperation with the relevant community; project confidentiality; and consistency.
• The quality of the data (including the quality of any extant data sets used in the study) and the efficiency of sampling are addressed.
• The data collection is appropriate to evaluation questions.

Data Analysis

• The data analysis addresses the handling of attrition, the matching of the analysis to the design, the use of appropriate statistical controls, the use of methodology and levels of measurement appropriate to the type of data, and estimation of effect size.
• The analysis shows sensitivity to cultural categories.
• The analysis makes appropriate generalizability of inferences.
• The chosen analysis type is simple and efficient.

Cross-Cutting Factors

Cross-cutting factors that are likely to be important at all stages of a report include: clarity, presentation, operation at a state-of-the-art level, appropriateness, understandability, innovation, generalizability, efficiency of approach, logical relationships, and discussion of the report's limitations. The report should also address ethical issues, possible perceptual bias, cultural diversity, and any gaps in study execution.

Methods

Methods

Cross-Cutting Factors

Cross-Cutting Factors

Cross-cutting factors that are likely to be important at all stages of a report include: clarity, presentation, operation at a state-of-the-art level, appropriateness, understandability, innovation, generalizability, efficiency of approach, logical relationships, and discussion of the report's limitations. The report should also address ethical issues, possible perceptual bias, cultural diversity, and any gaps in study execution.