Each summary includes a brief abstract; a description of the study, including its purpose, background, methods, findings, and use of results; the names of any publications that resulted; and the name and phone number of the person to contact for additional information.
In Iowa, patch was viewed as a concept that, if appropriately replicated, could complement other approaches to system restructuring, including the Child Welfare Decategorization project that was developed to reshape the service system. Selected Iowa counties, including Linn County, were permitted to merge their share of various funding streams into one child welfare fund. From the outset, the Iowa Patch project was adopted as an initiative of the Linn County Decategorization project. The demographics of the 9,921 residents of the patch area included higher rates of poverty, unemployment, child placement, single-parent homes, and poor housing than those in the rest of Cedar Rapids or in the State. Many children and families had multiple needs, and human services agencies were heavily involved in the communities at the project's outset.
A logic model was developed for the project evaluation that included problems endemic to service delivery in the United States; substitutionism (formal services substitute for, rather than support, informal systems of care); turfism (agencies protect their service domains); fragmentation; unresponsiveness to local conditions and consumer needs; deficit orientation (a focus on weaknesses rather than strengths of families and neighborhoods); crisis orientation; and bureaucratic rigidity.
The evaluation focused on three aspects of the project:
The patch approach to practice and service delivery reduced problems of fragmentation, deficit- and crisis-orientation, bureaucratic rigidity, turfism, and substitution of formal for informal systems. Researchers observed the interweaving of formal and informal supports, innovative resource mixes, and consumer and citizen involvement. More emphasis was given to prevention, early intervention, and increasing collaboration among service workers. Putting a local team in the neighborhood made human service workers more accessible, more approachable, more able to develop informal relationships with consumers, and more able to establish and work with informal helping networks.
After the grant ended, the Patch project was integrated into an initiative to develop family resource centers. The project established a strong desire to maintain and further develop the patch approach and a commitment to support it financially and administratively.
The second component was a purposive survey of 600 youth who reported a runaway or homeless episode in the previous 12 months. These young people were found on streets or in other public places in 10 urban areas. A small substudy designed to validate self-reports of substance use also was undertaken.
The third component was a probability sample of youth in households surveyed through an interagency agreement between ACYF and the National Center for Health Statistics. Nine questions concerning runaway experiences were added to the Youth Risk Behavior Supplement of the National Health Interview Survey. In 1992 and 1993, 6,496 youth between the ages of 12 and 17 responded to in-home audiocassette interviews.
The fourth component was a mailed survey to directors of a nationally representative sample of youth shelters in 1992 and 1993. Surveys were returned by 160 shelter directors, an overall response rate of 97 percent from directors of eligible shelters.
Fifty-six percent of youths in the street component had never used a shelter. In the household component, 75 percent of youth with a runaway experience had not used either a youth or an adult shelter. An average of 10 beds were available in shelters serving only youth aged 12 to 17, while an average of 28 beds were available in shelters serving youth aged 12 to 21. On any given night, an average of 55 percent of the beds in youth shelters were occupied. Only 10 percent of youth in the shelter component and 14 percent of youth in the street component had ever been turned away from a shelter.
In the household component, lifetime substance use among youth with runaway and homeless experiences was high compared with other youth. Eighteen percent of substance users in the shelter component had been in treatment for an alcohol- or drug-related problem (24 percent in the street component). Of the young people who had ever received treatment, 44 percent of those in the shelter component (23 percent in the street component) were in treatment when data were collected.
Suicide attempts were reported by 26 percent of youth in the shelter component and 32 percent in the street component. More youth reported attempting suicide at home than away from home.
Many young people had been robbed (12 percent in the shelter component, 30 percent in the street component), assaulted (15 percent, shelter; 29 percent, street), or sexually victimized (8 percent, shelter; 14 percent, street). Sex was traded for money, food, shelter, or drugs by 7 percent of the shelter component and 21 percent of the street component. Reported crimes committed by youth in the study included theft-related activity (66 percent, shelter; 81 percent, street); assault-related activity (30 percent, shelter; 42 percent, street); and drug-related activities (32 percent, shelter; 50 percent, street). In the household component, youth with any runaway experience were three times as likely as other youth to have carried a weapon in the past 30 days and to have been in a physical fight in the past year.
In the household component, youth with any runaway experience were four times as likely as other youth to ever have had sexual intercourse. At the time of the interview, 12 percent of the females in the shelter and street components were pregnant; 41 percent of females in the shelter component (50 percent in the street component) reported having ever been pregnant.
A strong relationship was found between youth substance use and suicide attempts. Having at least one family member who got drunk or used marijuana or other drugs in the 30 days before a youth left home substantially increased the likelihood that the youth had ever attempted suicide.
Over the years, the role of the ombudsman has evolved to include both individual and systemic advocacy. Individual advocacy activities are designed to ensure that ombudsman services are accessible to all residents, systematically receive and respond to resident concerns about their long-term care services, and work cooperatively with a variety of agencies to resolve problems.
Systemic advocacy activities include evaluation of laws and regulations related to long-term care services for older adults, education of the public and facility staff, dissemination of data about the ombudsman programs, and promotion of the development of citizen organizations and resident and family councils. Both roles are important, but the ombudsman programs primarily identify, investigate, and resolve individual resident complaints.
Although there is no national database or source of standardized information, numerous sources of primary and secondary data were used. Two- to three-day site visits were made to programs in California, Colorado, Florida, Massachusetts, Minnesota, and Virginia. Seven papers were commissioned, and a thorough literature review was undertaken.
Structured, systematic contacts were made with directors of State units on aging, State and local long-term care ombudsman program offices, long-term care physicians, and consumer advocacy groups. A 1-day invitational symposium, a public hearing, and two "open-mike" sessions were conducted. Discussions were also initiated with four national associations of long-term care facility providers.
In some areas, the committee found that meaningful access to ombudsman services does not exist for all eligible nursing facility residents because of infrequent site visitation and insufficient information outreach efforts. In general, complaints receive a timely response. Program implementation in B&C homes is much more limited than in nursing facilities.
Considerable variation among States in the extent of program implementation in nursing facilities is noted. At the systemic level, the committee observed that too many States operate fragmented programs that only respond to individual complaints rather than functioning as unified, integrated programs. Many States lack access to adequate, independent legal advice and backup for ombudsmen and for residents who need assistance in courts and regulatory hearings.
The committee also found that current resources are inadequate to support full implementation of program mandates. It recommends that Federal funds for the program be increased from the current $25 million to $39.5 million by fiscal 1998. This increase would provide funding to achieve the committee's recommended staffing level of one ombudsman full-time equivalent for every 2,000 long-term care beds. (The current ratio is one to every 2,700 beds.) The committee also recommended that resources be apportioned among the States on the basis of the number of beds rather than the number of persons aged 60 years and over, as is currently done.
The committee recommended a strengthened leadership and oversight role for the Federal Government, including the implementation of an objective compliance review method for States. An improved data system is needed that includes information about program awareness, as well as the extent of complaint resolution, the extent of ombudsman input into systemic improvement activities, and the extent of ombudsman impact on the overall system of long-term care and services.
Concern was expressed that expanding the program to cover persons who do not reside in long-term care facilities may hinder the ability of ombudsmen to serve their current constituencies in nursing facilities and B&C homes. The committee recommended that expansion capacity, target populations, organizational placement, staffing needs, and funding receive careful consideration before any expansion efforts to non-institutional care settings are undertaken. The importance of developing and supporting research and demonstration projects in this arena was also noted, with attention to demographic and health care system trends affecting the long-term care system.
The report indicates that it was premature to evaluate the impact on residents of B&C homes because long-term care ombudsman programs have been only partially implemented there.
Current measurement techniques are plagued by a variety of flaws. A significant obstacle is that many types of data currently used as indicators of quality are not directly usable for comparisons of clinical performance. For example, utilization statistics (e.g., hospital admissions or rates of surgery) are not helpful unless applied to the individual patient. Measures of health status or patient outcomes are not useful without allowing for the probability that each case would experience a good outcome if good clinical care were provided. Measures of patient satisfaction are flawed because of subjectivity, although patient surveys are useful if patients are asked about the facts of their care.
Although many existing indicators of quality provide inaccurate comparisons of clinical performance, they can serve as an intermediary step toward better measures. For instance, measures of patient outcomes that cannot currently be used to compare clinical performance may become useful for that purpose as methods are developed to allow for patient differences in the likelihood of achieving a good outcome. Similarly, users who currently have inadequate data sources to construct precise clinical performance measures may find it essential to use crude indicators while working to improve data sources.
To be serviceable, measures must be useful internally within health care organizations and must have adequate levels of sensitivity, specificity, and predictive value. Measures must be reliable and valid for their intended purposes, as well as affordable. A reliable typology of performance measurements would allow potential users to select an approach--quality control, choice of health care plan, or accountability--that is appropriate for their own purpose.
Existing and evolving clinical performance measures were identified by two approaches: (1) a literature search conducted by using the Medical Analysis and Retrieval SystemŽ database of the National Library of Medicine; and (2) direct personal contact by phone with 112 individuals or agencies known to be involved with performance measurement research, use, or evaluation. The contacts yielded 40 sets of measures consisting of 1,287 clinical performance measures. Data concerning measure attributes were extracted, coded, and entered into six relational databases.
In addition to describing general characteristics about the type of measure set, the Typology framework classified measures with respect to their structure (factors such as data requirements, sampling, time window, scoring, risk adjustment, and interpretation) as well as their clinical content (e.g., whether a measure addresses health promotion, early detection, or treatment of a disease; whether it is a process or outcome measure, etc.).
The objective was to develop and test a prototype framework sufficiently flexible to encompass the structural and clinical characteristics of the wide variety of clinical performance measures currently used. The result was a series of interlinked databases containing information on measure sets, batches of measures (measures with similar structure or content), and clinical conditions and events that are associated with the measures.
The interrelated nature of the databases enables users to access data by measure or clinical event. Frequencies or percentages of measures in the various categories were computed and the results presented in graphs. A broad range of performance measures were included in the database (i.e., process and outcome measures, health care setting, demographics of patient population, data derived from clinician judgment and patient perception, and mental and physical health measures). The development of the prototype raised other issues to be considered (e.g., cost and sampling) when developing future databases for clinical performance measurement.
Performance measures were constructed from administrative data (e.g., enrollee files, claims data files, disease registers, pharmacy records, medical records), and from special data collections (e.g., patient or provider surveys).
The study also concluded that future work is needed to test the framework and prototype databases against the needs of users. To accomplish this objective, the Agency for Health Care Policy and Research (AHCPR) is using the typology framework as the basis for a follow-on project, CONQUEST 1.0, the COmputerized Needs-Oriented QUality Measurement Evaluation SysTem. This project builds on the typology framework in three ways. First, it evaluates the typology by verifying the content of the measure database with measure developers. The verification effort has resulted in enhancements to the typology structure and content. Second, CONQUEST 1.0 builds on the typology by creating a database of information on clinical conditions that can be used to steer the search for appropriate measures. The condition database contains information from AHCPR-supported clinical practice guidelines, clinical practice guidelines produced by other organizations, and medical effectiveness research findings. Third, CONQUEST 1.0 translates the typology into a more useful system by making it available on computer. The project develops a computerized system with a user-friendly interface to link measures to clinical information and guide the selection of measures.
Follow-on efforts are currently under way at AHCPR. One such related project involves evaluating this product by convening users to pilot test CONQUEST 1.0 and participate in focus groups about its usefulness. Also, AHCPR has issued a Request for Contract Proposals for a project called QM-Net, to use the typology and CONQUEST 1.0 as the basis for a national data source for information on clinical quality measures. Information on CONQUEST 1.0 can be obtained through AHCPR's web site (http://www.ahcpr.gov) or through the Agency's clearinghouse at (800) 358-9295.
In an effort to stem rising health care costs, many States have turned to managed care models to deliver personal health care services to low-income persons and to other vulnerable populations, including those eligible for Medicaid. At this time, all but eight States offer some type of managed care to Medicaid enrollees; approximately 25 percent of all Medicaid enrollees are part of a managed care arrangement, and most of these individuals are in full-risk capitation programs, which generally are responsible for all care rendered. The shift from the traditional indemnity insurance system to a managed care system is expected to alter markedly the delivery of direct and nondirect services. The type of managed care organization (MCO) models that predominate within a State will likely further affect the role of public health agencies.
At both the State and local levels, CLPP programs serve primarily the most vulnerable of populations--children of low-income families. Several mechanisms provide ongoing support for these programs. For example, the Centers for Disease Control and Prevention (CDC), which launched its CLPP initiative in 1989, funds 37 S/L PHDs in an effort to develop comprehensive programs to screen, identify, and provide adequate medical and environmental followup to children with elevated blood lead levels (EBLLs). The future of this and other programs is uncertain, however, because of changes in our health care delivery system. In October 1993, the Lead Poisoning Prevention Branch (LPPB) of CDC commissioned a study to examine the effects of managed care reforms on CLPP programs, with a focus on CDC-funded activities. The results of the CDC-initiated study are presented in this report.
Data on the first component--screening--indicated that the number of lead screenings in the surveyed States decreased between 1993 and 1994. In Tennessee, for example, approximately 40 percent fewer screens were reported in 1994 than were projected for that year, but decreases in lead screenings occurred studywide. Possible explanations for the decrease include inadequate reporting and communication between private and State laboratories; poor provider compliance (e.g., with Early and Periodic Screening, Diagnosis, and Treatment program [EPSDT]); and difficulty in obtaining approval for screening.
The report recommends the following strategies to help States ensure that lead screenings are conducted: increase provider-based education and advocacy; legislate a screening mandate; expand reporting requirements; add lead screening to the State's quality assurance indicators; enforce EPSDT; retain some public sector service delivery role, including joining MCO networks as primary care providers; and try to obtain blanket authorization for lead screening within the private sector.
Data on the second component studied--lab analysis--indicated that data collection in general is likely to be severely affected by managed care, particularly in States without a strong and comprehensive reporting law. Historically, CLPP programs have received the data on blood lead tests primarily from public laboratories, whether or not a State had a reporting mandate. Surveyed sites reported success using the following strategies: establish a reporting law; mandate the use of public laboratories for selected tests; establish price (and other) incentives; and focus on the importance that surveillance serves in maintaining and improving public health.
Case management services, including necessary followup treatment and retesting of children with EBLLs, are currently performed by CLPP staff either directly or through links with other public agencies. Data from site visits revealed that private providers and MCOs had a number of concerns about CLPP case management services and admitted to a lack of knowledge of comprehensive care and followup, a lack of ability, and a lack of incentive. The report suggests the following strategies to ensure appropriate and adequate case management: establish reporting laws; identify and separately fund core public health functions; require data collection at the individual person level; require that case management responsibilities be detailed in the MCO contract; and encourage mandated or contractual relationships between MCOs and S/L PHDs.
Finally, the report recommends that S/L PHDs seek to maintain public sector service delivery, build a legal infrastructure to influence the behavior of private providers, and fund core public health functions with monies derived from non-Medicaid sources.
As a result, LPPB was able to improve its guidance to State and local childhood lead poisoning prevention offices about the appropriate language for State Medicaid managed care contracts requiring lead poisoning prevention services. LPPB was also able to provide guidance on improving data systems to monitor managed care performance. Thus, some programs have quickened the pace of their efforts to complete monitoring systems, and others have formed ties with Medicaid agencies.
In addition, investigators studied the community perspective on the childhood lead poisoning prevention efforts of public health agencies. Representatives of two types of community groups were consulted: umbrella organizations, which generally have national or State-level structure and support, and community-based organizations, which are generally local in origin and support. Representatives participated in focus groups to discuss several topics, including their relations with public health agencies and their visions of how to improve these relationships. As a result, some programs have been able to change their tactics for collaborating with community-based organizations, and some have broadened the focus of their collaborative efforts.
The research demonstration was conducted from May 1991 through June 1996. HCFA ultimately selected seven hospitals to serve as demonstration sites during the term of the study. Selections were based on the completeness of the package of services each hospital was willing to provide Medicare beneficiaries under the demonstration, the quality of care provided by the hospital and its physicians, and the size of the discount the hospital and physicians were willing to accept for their services. The demonstration was implemented at four sites in May 1991: Saint Joseph Hospital of Atlanta, St. Joseph Mercy Hospital in Ann Arbor, the Ohio State University Hospital in Columbus, and Boston University Medical Center Hospital. In the spring of 1993, three more sites were added: St. Luke's Episcopal Hospital in Houston; St. Vincent Hospital and Medical Center in Portland, Oregon; and Methodist Hospital in Indianapolis.
Under the demonstration, participating hospitals and physicians received a global payment covering hospital and related physician services, including outliers, for each CABG procedure (DRGs 106 and 107). In each case, the negotiated rate represented a discount from what Medicare paid, on average, for these procedures. Depending on the package proposed by the individual hospital, the package of services also included pre- and postdischarge physician services and some cardiac rehabilitation services as well as readmissions within a specified time after discharge.
Selected sites received the designation of Medicare Participating Heart Bypass Centers and were encouraged to market their services to referring physicians and beneficiaries and to offer incentives to attract patients. Quality assurance was carefully monitored by HCFA as well as by project evaluators. Hospitals not participating in the demonstration continued to provide services under the traditional Medicare fee-for-service program, and Medicare beneficiaries were free to choose between demonstration and nondemonstration hospitals.
A two-phase evaluation strategy evolved. Phase I covers implementing the demonstration and evaluating the original four sites for a 2-year period while focusing on marketing, volume increases, quality of care, and developing and implementing the appropriateness model. In addition, the phase I evaluation gathered baseline data and first-year operational statistics for the three sites that began the demonstration in May 1993.
An interdisciplinary group of economists, physicians, and marketing experts assembled a variety of databases and conducted numerous onsite interviews with participants as part of an extensive quantitative and qualitative evaluation of the demonstration. Methodologies included two quasi-experimental designs, time-series studies, case studies, surveys, and interviews. Several major literature reviews also were undertaken, including one on efficacy and risks of CABG surgery. A separate appropriateness study was conducted, in which an expert technical advisory panel was convened to help formulate a model of appropriateness indicators for CABG surgery. Information from the literature reviews and the appropriateness study was used to draft materials to assess quality of care at the demonstration sites.
Medicare claims were used to document national trends in Medicare bypass volumes, patient demographics, lengths of stay, mortality rates, and costs. Physician costs were broken into three categories: 30 days prior to bypass surgery, inpatient, and 90 days post-discharge. Prices negotiated with each demonstration site were compared with predicted Medicare prospective payment rates and physician inpatient outlays to determine the immediate savings from the demonstration. Market share savings also were calculated.
Every demonstration hospital provided a set of clinical information on each patient, including discharge status, risk indicators, admission priority, age, gender, and previous bypass operations. Additional information was provided on disease anatomy. The seven demonstration hospitals were compared extensively by using logistic analyses.
In the first 2.5 years of the demonstration, total Medicare savings (Medicare program and beneficiaries and their insurers) were estimated at $17.6 million. Three of the four original hospitals were able to make major changes in physician practice patterns and in hospital operations that generated significant cost savings. Physicians in the three hospitals were able to quickly and dramatically reduce the length of inpatients' stay, substitute generic for brand drugs, and reduce unnecessary testing and other services. In the one hospital where surgeons resisted attempts to change practice patterns, costs continued to rise. The study found that alignment of physician and hospital incentives facilitated a closer working relationship between physicians and hospital staff.
One of the first four demonstration sites significantly reduced its patient mortality rate during the course of the demonstration. Most participating hospitals reduced intensive care unit stays by 1 full day and total hospital stays by 2 days. For the first 3 years of the demonstration, there was some evidence of a growing severity in case mix, including a higher percentage of patients over the age of 80 with comorbid conditions. Complication rates appeared to increase commensurately during this period. Nevertheless, these factors did not produce an upward trend in mortality.
The demonstration involved major changes in Medicare payment arrangements. According to providers, patients were satisfied with the single copayment amount. Hospitals, in general, were pleased with HCFA's prompt payment, which was received within 30 days by wire. Supplemental insurers responsible for paying patient deductibles and coinsurance amounts were uniformly displeased with the flat actuarial payment calculated by the government because it was incompatible with their computer systems, and patient policies differed in their deductibles, coinsurance amounts, and so forth. However, HCFA intentionally had calculated an artificial copayment amount that was comparatively low, intending to share the savings with the beneficiary--although in most cases, ironically, their supplemental insurers reaped the benefit.
The success of the demonstration has led to the inclusion of language in the President's 1997 proposed budget package calling for new legislation to implement negotiated bundled payment arrangements under the regular Medicare program. The experience gained from this demonstration has led HCFA to develop a new bundled payment demonstration that expands the concept to a group of orthopedic and a group of cardiovascular procedures. The Participating Centers of Excellence Demonstration for Orthopedic and Cardiovascular Services is expected to be implemented at selected hospitals beginning in mid-1997.
Year | Total | Medical Visits and Consultations | Surgery, Rad. Ther., Anesthesia, & Assistants | X-ray and Lab Tests | All Other |
|---|---|---|---|---|---|
| Allowed Charges (in millions) | |||||
| 1993 | $46,124 | $16,311 | $12,984 | $8,962 | $7,868 |
| 1992 | 43,942 | 14,926 | 13,301 | 8,785 | 6,930 |
| 1991 | 42,915 | 13,885 | 14,116 | 8,727 | 6,186 |
| Percent Changes | |||||
| 1992-93 | 5.0 | 9.3 | -2.4 | 2.0 | 13.5 |
| 1991-92 | 2.4 | 7.5 | -5.8 | 0.7 | 12.0 |
| Adjusted for Changed Population | |||||
| 1992-93 | 3.5 | 7.8 | -3.8 | 0.6 | 12.0 |
| 1991-92 | 0.7 | 5.7 | -7.3 | -1.0 | 10.2 |
Source: Medicare Part B Monitoring System: Allowed charges derived from Medicare National Claims History File. Population information from June 30 Medicare Part B enrollment files.
The report amplified these general trends with specific information about utilization rates for numerous procedures. From 1992 to 1993, there was a decline in rates of many procedures. For example, the rate of sigmoidoscopies (per 1,000 beneficiaries) declined by 8.9 percent; the rate of hip replacements declined by 1.8 percent; and the rate of hysterectomies declined by 5.4 percent.
Overall, the passage of the legislation was associated with improved patient access. There were general increases in the number of visits per beneficiary for outpatient, nursing home visits, and consultations. But disparities emerged when these data were analyzed for vulnerable populations within the overall Medicare population. Vulnerable populations are defined as beneficiaries who live in rural areas, poor areas, or areas with a shortage of health providers; who are African-American, Medicaid-eligible, disabled, or older than 85 years. These vulnerable populations still were found to face the barriers to care that they faced before MFS was in place. Yet MFS did not create any new barriers. The report concluded that greater understanding of barriers to care for vulnerable Medicare populations is needed.
Physicians' caseloads--the total number of different Medicare patients treated in a year--remained stable or increased during the period studied. Caseload is viewed as an indicator of physician willingness to deliver services to Medicare patients. From 1992 to 1993, the mean caseload for primary care and medical specialties tended to increase more (+4-5 percent) than that for surgical specialties (+2 percent). The trends in the mean-allowed physician charges revealed a somewhat similar pattern. Physicians in primary care and medical specialties tended to experience the greatest increases in caseload, while surgeons, radiologists, and anesthesiologists tended to experience little or no growth. Some groups of surgical specialists--ophthalmologists and otolaryngologists--may have witnessed decreases in mean-allowed charges.
According to a survey analyzed in the report, Medicare beneficiaries reported greater satisfaction with their care in the years after the introduction of MFS than before. They were more satisfied with the quality of care, its availability, the ease of getting to a doctor, and the cost. In addition, fewer beneficiaries reported a health problem for which they did not receive care.
The report also showed dramatic differences in income between White and African-American Medicare beneficiaries. Based on inferences from median household income by zip code, 73 percent of African-American, but only 19 percent of White, beneficiaries were classified in the lowest income quartile. In comparison with Whites, African-Americans experienced higher mortality rates, lower rates of ambulatory visits (see figure II-1), lower use of referral-sensitive procedures, and higher rates of hospitalization. These patterns suggest that African-Americans encountered greater barriers to comprehensive and continuous care.
Figure II-1. Ambulatory Visits per Person by Race and Income: Persons Age 65 Years and Older, 1993
Some of the racial differences in utilization were attributable to income. For example, the rate of ambulatory visits declined with income, almost irrespective of race. Whites in the lowest income quartile not only had far fewer visits than wealthier Whites, but their rate also was just as low as that for the poorest African-Americans (see figure II-1). Other analyses revealed that if incomes for Whites and African-Americans were equal, some racial differences in utilization would diminish. Race, however, continues to play a role in access and utilization.
Under GPRA, each Federal agency is required to report annually (beginning in March 2000) on actual program performance as it compares with program goals. A preliminary set of performance measures for program and spending is to be developed for fiscal 1997 budget submissions, with actual performance data obtained from fiscal 1995 to fiscal 1997. The intent of GPRA is to motivate Federal agencies to identify programs that are effective in achieving goals and to reallocate resources away from those that are ineffective. GPRA focuses on the quality of performance, quantity of products, timeliness of products, and program efficiency.
An interview protocol was developed and administered to each of the four bureaus and ORHP. Information was solicited about each program's objectives, inputs (e.g., dollars, staff), processes (e.g., training approaches), outputs (e.g., number of people trained), impacts (e.g., number of trainees who would not have found comparable work), outcomes (e.g., number of trainees who find and retain work), and current use of performance measures and indicators. In addition, a wide variety of program materials was collected and reviewed. Draft assessments were developed and shared with staff from the relevant bureaus and offices to ensure that the information was complete and accurate. Findings from the interviews and background research were synthesized, analyzed, and then compiled into assessments of the program clusters and individual programs or budget line items.
The study used an overall assessment approach and analytical framework to document the linkages and commonalities among various HRSA organizational components with respect to resources, processes, products, outcomes, and populations served. Application of a program logic model that illustrates the rationale, structure, and division of labor of each program was critical to the assessments. The program logic model uses a matrix to represent the interactions and relationships among a program's objectives, illustrating the flow of inputs, activities, outputs, outcomes, and impacts required to achieve them.
The study found that it is both feasible and desirable to use a standard approach to measuring program performance across HRSA programs. In addition, progress in measuring and collecting performance data is quite varied across HRSA programs and program clusters. Within most programs, there is inadequate linkage along the continuum of performance measurement, with a general emphasis on producing outputs rather than evaluating program performance or outcomes. The study also concludes that data and data sources are not treated as strategic HRSA resources and that approaches to specification, collection, and use of data vary widely. Furthermore, the budget does not provide resources specifically for the data collection and evaluation necessary to build and sustain the kind of performance measurement system recommended in the study.
Recommendations are made for both short- and long-term actions. Short-term recommendations include the appointment of a central coordinator responsible for performance management efforts; the establishment of intra-agency working groups to specify performance measures, data collection, support strategies, and involvement of service beneficiaries; the provision of resources; and the provision of technical assistance tailored to specific HRSA components for developing and implementing systems of performance measurement.
Long-term recommendations address the development of a performance management system infrastructure. First, the HRSA strategic planning process should be continued and refined. Second, the performance management process needs to be linked to the budget process, to evaluation, and to grants and contracts management. Third, mechanisms need to be developed to incorporate feedback and data from grantees and populations served by HRSA. Fourth, some further restructuring of HRSA programs will be needed to facilitate performance measurement in light of HRSA-wide strategic priorities. Finally, the commitment and ongoing support of senior management is needed if staff members are to undertake the changes necessary to implement and sustain the performance system.
The prevalence of diabetes in this population (6.9 percent) is almost three times higher than that in the U.S. population as a whole (2.4 percent). American Indians also suffer from disproportionately higher rates of diabetic complications. Complications include loss of vision or blindness, lower extremity amputations, and end-stage renal disease. Mortality rates among diabetic American Indians also are higher compared with those of other ethnic groups. Moreover, diabetics are intensive consumers of medical services, and the costs per patient are among the highest of any medical condition.
The underlying philosophy behind the creation of the model program was that a team of primary care professionals could effectively deliver patient education, outreach, and treatment. Patient education was designed to address diet, exercise, foot care, administration of medications, and monitoring of blood sugar. Equipped with more knowledge and skills, patients were thought to be in a better position to control their blood sugar levels. Achieving better control over blood sugar levels was expected to lower the rate of diabetic complications and other conditions that lead to hospitalization.
Focus groups with patients and interviews with IHS and tribal staff supplemented quantitative analyses. During four focus group meetings--two at model sites and two at control sites--patients were asked about problems they encountered in complying with medical recommendations and their views of the services they received. Interviews with medical personnel yielded information about the content and the perceived quality of diabetes services.
The study identified several elements of the program that were associated with good blood sugar control: consultations with a dietitian and a podiatrist in the first year after diagnosis and receipt of a hemoglobin A1c test one or more times in 2 years. This test for blood sugar levels is more reliable than others because it reflects average levels over the previous 3- to 4-month period. Similar programmatic elements helped prevent hospitalizations. As to the generalizability of all study findings to other IHS sites, the analysis of audit data could not provide a definitive answer because of undersampling at the control site.
Observations from patients and providers at focus groups and interviews, respectively, offered important insights about the model program. Patients and providers both claimed changes in diet to be the most difficult for patients to implement. Patients stressed the importance of good communication with their providers, and those receiving the usual care reported more communication problems. This finding was considered noteworthy because earlier published research revealed that the quality of the patient-physician relationship is predictive of diabetic patients' compliance with lifestyle changes and medication dosing.
The finding that patient education in the first year after diagnosis had important and long-lasting impacts led to a recommendation for early patient education that focused on difficult lifestyle changes, particularly dietary changes. Blood sugar evaluation by both standard tests and by the hemoglobin A1c test were requirements for good clinical management and were correlated with better sugar control and lower hospitalizations. Only a small percentage of patients at model sites, and even fewer at the control site, received both tests once a year.
The study recommended more time for direct patient contact with providers even in lieu of some educational services. A strong bond between patients and providers was viewed as essential for improving patient compliance. Finally, the study recommended training for providers in counseling patients who are especially stressed by the diagnosis of diabetes.
FAS and related disorders are characterized by a constellation of physical malformations, growth deficiencies, and neurodevelopmental abnormalities. The most obvious malformations are found on the face, often in the form of a flat upper lip and midface, but other malformations can be present in other parts of the body. Neuro-developmental defects include impairments in intelligence, motor skills, gait, hearing, and eye-hand coordination. Although not all FAS victims experience mental retardation, FAS is considered the most common totally preventable cause of mental retardation. Variability in the expression and permanence of these defects is dependent on numerous factors, including the amount and pattern of maternal drinking, the timing of drinking during pregnancy, and a host of biological and environmental factors before and after birth.
The incidence of FAS and related disorders appears to be 0.5 to 3 cases per 1,000 births, although the data are far from ideal, in part because of inaccuracies in diagnosis. While up to 14 percent of women drink heavily when they drink, only a small percentage of women, about 4 percent, are dependent on alcohol according to accepted clinical criteria. The rates of heavy drinking during pregnancy vary, but most studies find that less than 1 percent of pregnant women drink heavily. Although these percentages appear small in relative terms, the absolute number of women engaging in hazardous drinking is large. The annual costs of FAS and related disorders are estimated at $75 million to $9.7 billion. The great range of costs reflects uncertainty about the long-term consequences, such as the need for medical and educational interventions, as well as uncertainty about the actual incidence of FAS and related disorders.
Passive and active surveillance measures were compared. With passive surveillance, data are collected retrospectively through reviews of medical charts, although more expensive active surveillance in which data collection takes place prospectively through experimental protocols could identify more cases and provide superior data. The committee recognized that passive surveillance measures could be heightened but were inadequate to capture the incidences of FAS and related disorders.
Identified risk factors for maternal drinking were a major research gap. Understanding risk factors that predict drinking behavior during pregnancy was deemed vital for developing targeted prevention programs. The committee concluded that more study was needed of the psychological and social-environmental factors that lead to heavy drinking during pregnancy, including dependence on alcohol, depression, low self-esteem, family history of alcoholism, sexual abuse, other violent victimization, and dietary habits.
The committee constructed a conceptual framework for prevention that takes into account cultural, sociological, behavioral, public health, and medical disciplines. This new framework was seen as an important guide to the field, given the scarcity of prevention-related research on FAS and related disorders. The proposed prevention model is a continuum of interventions designed for targeting the general risks of all women who drink, to the specific risks incurred by some women who drink heavily during pregnancy. The committee endorsed the universal prevention message to all women that alcohol consumption should be avoided during pregnancy, although it noted a paucity of studies to determine the behavioral impact of this message.
The committee stressed that an important aspect of preventing FAS is treating the alcohol-dependent woman. There is a compelling need for studies that assess the effectiveness of specialized programs that treat pregnant women who are alcohol dependent. The committee recommended a high priority be placed on efforts to design, implement, and evaluate prevention programs that effectively guide pregnant women who drink heavily into alcohol treatment. The committee further recommended continued research on biological methods of detecting alcohol consumption by pregnant women and continued basic research in animals to elucidate mechanisms of alcohol-induced fetal abnormalities so that medications can be developed that ease the symptoms of FAS.
The availability of treatment for individuals affected by FAS and related disorders was viewed by the committee as inadequate. The committee was unable to identify specific treatment programs. Because studies have clearly shown the value of early intervention in ameliorating symptoms of other neurodevelopmental disorders, the committee stressed the importance of identifying children with FAS and related disorders as early as possible. The committee also concluded that there was inadequate and insufficient attention given to preventing secondary disabilities such as deteriorating mental health, criminal behavior, and substance abuse.
In terms of major policy recommendations, the committee embraced the Surgeon General's 1981 position of total abstinence from alcohol prior to conception and throughout pregnancy and recommended the establishment of clinical practice guidelines for treatment of children with FAS and related disorders. While recognizing that NIAAA has historically led research efforts on FAS, the committee noted that NIAAA is a small institute and FAS is a complicated and diverse problem. In the absence of a single organized discipline of health professionals with responsibility for FAS prevention and treatment, the committee recommended forming an interagency task force to coordinate national efforts. The task force, which would be spearheaded by NIAAA, would include appropriate Federal research, surveillance, and service agencies.
Current Federal Regulations (42 CFR 50 103) require that institutions develop policies and procedures to handle allegations of misconduct. These policies must include provisions for "undertaking diligent efforts to protect the positions and reputations of those persons who, in good faith, make allegations" (42 CFR 50 103 [D][13]). If there is an immediate need to protect the interests of the person(s) making the allegations, Federal policy requires institutions to notify HHS.
The National Institutes of Health Revitalization Act of 1993 created the Commission on Research Integrity, which was established in March 1994 to make recommendations to the Secretary of HHS and Congress on how the Public Health Service (PHS) should deal with research misconduct in research funded by the PHS Act. In an interim report released in January 1995, the Commission identified "retaliation against whistleblowers" as one of three problem areas affecting the responsible conduct of scientific research.
Congressional hearings revealed that some whistleblowers may have suffered retaliation by the accused or others in the institution, which led to mandated additional regulations to protect whistleblowers. Despite legislative and executive efforts, however, empirical evidence and anecdotal reports continue to suggest that some students and faculty engaged in scientific research are, or feel they are, vulnerable to retaliation if they report misconduct.
A second-phase questionnaire was mailed to individuals, and a followup telephone call was made to remind them to return the completed survey. Eighty-nine individuals responded to the questionnaire; of these, 68 whistleblowers were considered eligible and willing to participate. Ninety-one percent of respondents reported that they held doctoral degrees; 78 percent worked in an academic setting; 9 percent were postdoctoral or graduate students.
The majority of the negative consequences experienced by whistleblowers were due, they said, to the actions of institution officials, respondents, colleagues, and professional societies; the most serious consequences felt by whistleblowers were attributed to the actions of institutional officials while the institution was responding to their allegations (i.e., while the case was still open) and after the inquiry or investigation was completed. Consequences were most likely to involve pressure on the whistleblower to drop the allegations of misconduct.
Data revealed that whistleblowing was most likely to have adverse outcomes in situations in which fabrication of data was alleged, the case received publicity, the allegations were made to a senior administrative official or misconduct official of the institution or to the funding agency, the allegations were made both within and outside the institution, the allegations were made to many different types of individuals, and the allegations were subjected to an investigation without recourse to an initial inquiry.
About 62 percent of whistleblowers perceived their whistleblowing to have had a neutral impact on their careers, professional activities, and personal lives; 28 percent perceived a negative impact; and 10 percent reported a mixed impact. Although few whistleblowers perceived positive consequences of their actions, 68 percent reported a willingness to make another allegation; 12 percent said they probably would make another allegation; 10 percent were uncertain; and 10 percent said they would not.
Not every whistleblower suffers substantial negative consequences as a result of reporting misconduct, but most individuals who report allegations of misconduct frequently face the prospect of significant hardship because of their efforts.
These data suggest that ORI focus first on limiting adverse actions while the case is still active. To prevent the most serious consequences of whistleblowing, regulations and enforcement approaches will need to primarily target institutional officials. Finally, potential whistleblowers should be counseled about the likely harm they will suffer if they make their case a cause celebre by taking their concerns outside their institution or getting their case publicized by the media.