- Health Care Financing Administration (HCFA)
- The MEDSTAT Group
To improve the quality of programs administered under the Home and Community-Based Services (HCBS) Waiver Program.
Three waiver quality assurance/improvement projects are under implementation by the MEDSTAT Group: (1) Development of a HCFA Regional Office (RO) HCBS Waiver Monitoring Guide (The Protocol); (2) Development and Testing of Performance Measures for HCBS Waiver Programs; and (3) State Readiness to Use Data in Assessing HCBS Waiver Programs. Below, each project is described in more detail.
Development of a HCFA Regional Office HCBS Waiver Monitoring Guide
During CY 2000 MEDSTAT developed a monitoring protocol for HCFA RO staff to use in their periodic reviews of the quality of the HCBS waiver programs. This protocol was developed primarily for RO staff--to encourage consistency in and comprehensiveness of reviews. However, it was also developed with state use in mind in that it operationalizes the relatively non-specific assurances that states must provide to the Federal Government as a condition of waiver approval. In other words, it specifies what the states must do in order to comply with the assurances, and what evidence the RO will be looking for as verification that the assurances have been met. The Protocol also includes sections devoted to "Quality Enhancing" activities that the states may seek to implement in their quest for quality improvement (dubbed "Recommended But Not Required").
The development of The Protocol was a collaborative endeavor. Feedback on over 50 iterations was provided by a State-Federal Work Group (that included representatives from the advocacy community), two HCFA RO Work Groups--one focused on process and the other on content issues, as well as HCFA Central Office staff. HCFA also instituted a public comment period during last July and August, and based on comments received, certain sections of The Protocol were revised. HCFA sees The Protocol as a living document, and expects the content to evolve over time. That The Protocol is now in its third official version--Version 1.2--is evidence that this expectation is being borne out.
The Protocol went into effect on January 1, 2001. All reviews conducted by RO staff after this date must be performed using The Protocol. In July 2000, states and HCFA RO staff were invited to a one-day session where The Protocol was showcased. HCFA's Central Office has also provided additional training opportunities for HCFA RO staff conducting waiver reviews. In an effort to insure that The Protocol is implemented as intended, HCFA has assigned one of its RO waiver review staff, who is experienced both in waiver reviews and quality assurance, to accompany other RO staff on their initial reviews using The Protocol. HCFA is also currently developing a format for waiver review reports that will be consistent with The Protocol's format; adherence to this review format will be required and is viewed as yet another mechanism for insuring consistency and comprehensiveness of reviews. To date, at least five waiver reviews have been conducted using The Protocol. For all practical purposes, MEDSTAT's work on The Protocol has been completed.
Development and Testing of Performance Measures for HCBS Waiver Programs
This project requires MEDSTAT to develop and test one or more performance measures that states may use (at their discretion) in monitoring the quality of their waiver programs. Work on this project began in the Winter of 2000. The project's State-Federal Work Group (same Work Group that provided guidance on the development of The Protocol) was assigned the task of recommending the focus of the performance measures. This group strongly suggested a Consumer Experience measure that has evolved into a Consumer Experience Survey. Through a focus-group approach, this Work Group identified five priority areas, or domains, that the survey should address:
- Satisfaction with Services;
- Access to Care;
- Respect/Dignity; and
- Community Integration/Inclusion.
Initially, the Work Group recommended that one instrument be developed that would be appropriate for telephone administration, applicable to all HCBS waiver populations, and take no longer than 10 minutes to administer. Given these recommendations, it became clear that besides being relevant to the quality of care and services provided under the auspices of the waiver, the questions had to use language that would be as "universal" as possible and response patterns limited so as to maximize comprehension and ability of the interviewee to respond meaningfully. Additional feedback sessions were held with the Work Group during the Summer and Fall of 2000--to flesh out the questions to be asked, their wording and response patterns.
During December 2000 and January 2001, MEDSTAT conducted cognitive testing with waiver participants in two states. Cognitive testing took the form of a series of follow-up questions following each survey item intended to elicit responses that would indicate whether the respondent understood the intent of the question and response alternatives. Based on these face-to-face interviews, MEDSTAT concluded that the appropriateness of the survey items varied by waiver population, as well as by the type of services provided under the auspices of the waiver. MEDSTAT recommended that the survey be more tailored to the needs and experiences of the different populations. MEDSTAT also identified item wording/responses that required more refinement and simplification.
Following the findings of these initial pretests, the instrument was adapted to a modular approach so that it was more relevant to the waiver experiences of different types of persons with disability. In addition, MEDSTAT developed a companion instrument for proxy respondents. These new instruments--one for the frail elderly and non-elderly physically disabled, one for the MR/DD population, and an MR/DD proxy--were tested in three states during Spring 2001. A third round of cognitive testing, using a phone administration mode, will follow. In addition, MEDSTAT is planning additional testing in order to estimate administration time for the various waiver populations and surrogates. These testing phases should be accomplished by Summer 2001.
Under a new contract with HCFA, MEDSTAT will conduct additional psychometric testing of the Consumer Experience Survey, to evaluate reliability and validity. These tests will include inter-rater reliability testing, and a field test in one or more states. Information from the field test will be used to review frequencies of survey items, and to asses how discriminate response categories are. Also, field testing experience will be reflected in the guidance prepared for states. This additional testing is scheduled for Fall 2001 and Winter 2002.
The final stage of the project entails the development of a user's guide and related technical assistance materials that will provide guidance to the states in fielding the survey, interpreting results, and using the results within the context of a quality improvement strategy.
State Readiness to Use Data in Assessing HCBS Waiver Programs
The State Data Readiness initiative focuses on identifying exemplary quality improvement practices for HCBS waiver programs built around the collection of data or other information. For this initiative, MEDSTAT is in the process of identifying specific examples of the types of HCBS quality improvement activities that states are conducting (or plan to conduct), with an emphasis on the types of data and other information that is used to manage the development, implementation and monitoring of these activities. Examples of such activities could include, among others, the use of performance measures, outcomes data, systems for reporting and tracking complaints or special incidents, and surveys or focus groups to obtain waiver participant input on quality. Over the past few months, MEDSTAT has been identifying states that may be managing these activities through interviews with HCFA RO representatives, website solicitations, and letters to state units on aging. MEDSTAT is currently in the process of speaking with a number of states (Pennsylvania, Texas, South Carolina, Michigan, Oregon, Arizona, Florida, and North Carolina) about a variety of their activities. Once examples have been identified, in-depth case studies will be conducted of selected programs in order to understand how these activities have been developed and how they are managed on a day-to-day basis. It is expected that the case studies will be completed in CY 2001. Case study results will be shared broadly to assist other states in developing or implementing similar activities. The contract also calls for a series of technical assistance activities, to be determined, that would assist states in developing and using data to monitor and improve the quality of their waiver programs. Technical assistance activities are likely to occur in late CY 2001 and the first half of 2002.
Beth Jackson, Ph.D., The MEDSTAT Group, 125 CambridgePark Drive, Cambridge, MA 02140, (617)492-9326