Feasibility Study for the Evaluation of DHHS Programs That Are or May Be Operated Under Tribal Self-Governance. Detailed Description Of Illustrative Comprehensive Evaluation Model: Indian Health Service Programs Operated By Tribes Under Self-Governance

03/03/2004

  1. Objectives
    1. Conduct a comprehensive evaluation of the operations and outcomes of Indian Health Service health programs managed by Tribes under Self-Governance compacts.
  2. Assumptions
    1. Tribes to be included in the evaluation of health programs would be limited to those that have submitted data to RPMS.
    2. Participation of both compact and direct service Tribes in the evaluation would be voluntary.
    3. IHS would provide RPMS data for participating compact and direct service Tribes, for all years required.
    4. Site visits would be conducted to 15-25 compact and direct service Tribes that volunteer to participate in the in-depth evaluation.
  3. Research Questions to be Examined
    Note:  Specific research questions would be developed in consultation with the Tribes. Based on discussions conducted during the current study, the general research questions that are likely to be identified might include:
    1. Background Issues (retrospectively)—Health Programs
      1. What are the characteristics of Tribes that compact for health programs?  Are these characteristics different from those of Tribes that do not compact?
      2. What factors are reported by Tribes as influential in their decision to manage their health programs?  What was the most important factor in their decision?  What concerns were identified during the decision process?
      3. Was the community involved in the decision to manage health programs?  How was this accomplished?
    2. Process Issues (retrospectively)
      1. What changes in programs, staffing and organization have occurred since the Tribe began managing the health program?  What were the reasons for these changes?
      2. Were there changes in the goals/objectives for the health program/system after the initial implementation year?
      3. What oversight and monitoring of the health system does the Tribe conduct?  What is the process for addressing problems or issues that are identified?
      4. Is the community involved in decisions made about the health system?  How?
      5. Have the goals/objectives for the health system been met on an ongoing basis?
      6. Have program changes been made to achieve cost-savings and/or increase efficiencies?
      7. Have any health program funds been re-allocated to other priorities within the Tribe?  How was the decision made and who were the decision makers?
      8. What are the perceptions of Tribal leaders and others about the benefits of Tribal management of health programs?  Are there any perceived disadvantages?
    3. Quantitative Measures of Process and Outcome Issues – Health
      1. Do health programs managed by Tribes provide the same level and mix of services as would be available if the Tribes did not manage these programs?  If not, what are the reasons?
      2. Did the health program achieve at least two quantifiable goals, established by the Tribe, during the past three years?
      3. Does the Tribe employ directly all health providers that work at the health unit? If so, has this been the case since the Tribe began managing the health program?
      4. Has there been little or much staff turnover since the Tribe began managing the health program?
      5. Have there been changes in the allocation of program funds to personnel, space, materials, contract health services, administrative costs since the Tribe began managing the health program?
      6. Have program costs per person receiving services changed more/less than would be expected based on IHS funding levels?
      7. Are health program users more/less satisfied with services provided by the health program under Tribal management than are users who receive services through IHS direct service arrangements?
  4. Comparison Groups for Quantitative Measures
    1. Pre-post comparisons
    2. Self-Governance versus direct service comparisons
    3. National benchmark comparisons
  5. Data Necessary for the Evaluation
    Note:  Specific data needed would depend on the set of evaluation issues and research questions that would be developed through DHHS consultation with the Tribes. A set of possible data needs is described here.   
    1. RPMS data for pre-post compact period, for participating Tribes
    2. RPMS data for pre-post compact period, for participating direct service Tribes
    3. Detailed financial data on health programs managed by participating Tribes, baseline through evaluation year.
    4. Detailed data on staffing, staff-mix, salaries, credentials, and turnover of personnel for participating compact and direct service Tribes, since the beginning of health Self-Governance.
    5. Health care satisfaction survey of Tribal members (on/near Reservation), for participating Tribes managing health programs and for a sample of participating direct service Tribes.
    6. One round of site visits to participating Tribes managing health programs and participating direct service Tribes/IHS service units to collect qualitative data on operations, processes, goals/objectives, perceptions of advantages/disadvantages of Tribal management of health programs.
    7. Socioeconomic and demographic data for all health compact and direct service Tribes