Tribal Self-Governance Health Care and Social Services Delivery Effectiveness Evaluation Feasibility Study: Revised Draft of Background Materials

Publication Date

Feb 4, 2003

These revised drafts incorporate comments of the Technical Working Group."

Cover Memo

TO: Technical Working Group Members
Tribal Self-Governance Evaluation Feasibility Study

FROM: Kathy Langwell and Jo Ann Kauffman

DATE: March 7, 2003

SUBJECT: Revised Project Documents (attached)

Enclosed are revised project documents that you reviewed and discussed with us at the Technical Working Group meeting in Washington, DC on February 3-5. These revised documents are:

Revised Draft Literature Review
Revised Draft Tribal Matrix Report
Revised Draft Tribal Population Data Report
Revised Evaluation Issues, with additional background information

Note that the Revised Evaluation Issues document lays out the evaluation issues that would be the focus of the Evaluation Feasibility Study, based on our TWG discussion. Once these issues are finalized, specific research questions and data requirements will be developed.

In addition, this package includes:

Draft Summary of the First Technical Working Group Meeting
Draft Legislative History and Background on Tribal Self-Governance. (Note: This Report has not been revised, but a portion of the Preamble to the Title IV Regulations has been added as an Appendix. We are interested in receiving written comments and input from TWG members on other changes to this report that may be considered).

At the Technical Working Group meeting, it was agreed that TWG members would review these revised materials and give us comments by March 30, 2003. Please fax or email your comments to Kathy Langwell (langwell@mato.com or Fax: 605-584-2429) or email to Andy Rock (andy.rock@hhs.gov).

We would like to arrange a conference call with the TWG members during the week of April 1 to discuss your feedback on the revised project documents and to discuss and agree on the site selection criteria that will be used to recruit and select the six Tribes that will be visited. Please let us know any specific days that you would not be available for this call.

Thank you for taking the time to review these materials again. Your comments and suggestions are very helpful in ensuring that this project is successful. If you have any questions, please feel free to call us. (Andy at 202-260-0398; Jo Ann at 509-747-4994; Kathy at 605-584-2414).

Additional History Concerning Tribal Self-Governance Indian Self-Determination and Education Assistance Act Excerpt from the Part IV Final Rule; Applies to Department of Interior Published December 15, 2000, Volume 65, Number 242, Page 78687-78735

“The Indian Self-Determination Act Amendments of 1988 (Pub. L. 100-472), authorized the Tribal Self-Governance Demonstration Project for a 5-year period and directed the Secretary to select up to 20 Tribes to participate. The purpose of the demonstration project was to transfer to participating Tribes the control of, funding for, and decision making concerning certain Federal programs, services, functions and activities or portions thereof. In 1991, there were 7 annual funding agreements under the project, and this expanded to 17 in 1992. In 1991, the demonstration project was extended for an additional three years and the number of Tribes authorized to participate was increased to 30 (P.L. 102-184). The number of self-governance agreements increased to 19 in 1993 and 28 in 1994. The 28 agreements in 1994 represented participation in self-governance by 95 Tribes authorized to participate.

“After finding that the Demonstration Project had successfully furthered Tribal self-determination and self-governance, Congress enacted the “Tribal Self-Governance Act of 1994”, Public Law 103-413 that was signed by the President on October 25, 1994. The Tribal Self-Governance Act of 1994 made the Demonstration Project a permanent program and authorized the continuing participation of those Tribes already in the program.

“A key feature of the 1994 Act included the authorization of up to 20 Tribes per year in the program, based on their successfully completing a planning phase, being duly authorized by the Tribal government body and demonstrating financial stability and management capability. The Act was amended by Public Law 104-208 on September 30, 1996, to allow up to 50 Tribes annually to be selected from the applicant pool. In 1996, the Act was also amended by Public Law 104-109, “An Act to make certain technical corrections and law related to Native Americans”. Section 403 was amended to state:

“(1) INCORPORATE SELF-DETERMINATION PROVISIONS--At the option of a participating Tribe or Tribes, any or all provisions of title I of this Act shall be made part of an agreement entered into under title III of this Act or this title. The Secretary is obligated to include such provisions at the option of the participating Tribe or Tribes. If such provision is incorporated, it shall have the same force and effect as if set out in full in title III or this title.

“The number of annual funding agreements grew by one to 29 in 1995 and grew to 53 and 60 agreements in 1996 and 1997, respectively, to include 180 and 202 Tribes. Self-Governance has continued to grow. In 1999, there were 67 annual funding agreements with BIA covering 209 Federally recognized Tribes. Also in 1999, there were three annual funding agreements between Self-Governance Tribes and non-BIA bureaus.

“The Tribal Self-Governance Act of 1994, as amended, authorized the following:

The Director of the Office of Self-Governance may select up to 50 Tribes annually from the applicant pool to participate in Tribal Self-Governance.
To be a member of the applicant pool each Tribe must have: (a) Successfully completed a planning phase that includes budgetary research and internal Tribal government planning and organizational preparation; (b) have requested to participate in Self-Governance by resolution; and (c) have demonstrated financial stability and financial management capability for the previous 3 years as evidenced by the Tribe having no material audit exceptions in their required annual audits of Self-Determination contracts.
The Secretary is to negotiate and enter into annual written funding agreements with the governing body of each participating Tribe that will allow that Tribe to plan, conduct, consolidate and administer programs that were administered by the Bureau of Indian Affairs (BIA) without regard to agency or office within which such programs were administered. Subject to such terms of the agreement, the Tribes are also authorized to redesign or consolidate programs and reallocate funds.
The Secretary is to negotiate annual funding agreements with Tribes for programs administered by the Department other than through BIA that are otherwise available to Indian Tribes. Annual funding agreements may also include programs from non-BIA bureaus that have a special geographic, historic or cultural significance to the participating Tribe.
Tribes may retrocede all or a portion of the programs.
For construction projects, the parties may negotiate specific provisions of the Office of Federal Procurement and Policy Act and Federal Acquisition Regulations for inclusion in annual funding agreements. If not included, then such provisions do not apply.
Not later than 90 days before the effective date of the agreements, the agreements are to be sent to the Congress and to potentially affected Tribes.
Funding agreements shall provide for advance payments to the Tribes of amounts equal to what the Tribe would be eligible to receive under contracts and grants under this Act. This is to include direct program and contract support costs in addition to any funds that are specifically or functionally related to the provision of benefits and services by the Secretary to the Tribe or its members without regard to the organizational level within the Department where such functions are provided.
Except as otherwise provided by law, the Secretary shall interpret laws and regulations in a manner that will facilitate the inclusion of programs and the implementation of the agreements.
The Secretary has 60 days from the receipt of a Tribal request for a waiver of Departmental regulations in which to approve or deny such a request; denial can only be based upon a finding that such a waiver is prohibited by Federal law.
An annual report is to be submitted to the Congress regarding, among other things, the identification of the costs and benefits of Self-Governance and the independent views of the participating Tribes. The Secretary is to publish in the Federal Register, after consultation with the Tribes, a list of, and programmatic targets for, non-BIA programs eligible for inclusion in annual funding agreements.
Nothing in the Act shall be construed to limit or reduce in any way the services, contracts or funds that any other Indian Tribes or Tribal organizations are eligible to receive under any applicable Federal law or diminish the Secretary's trust responsibility to Indian Tribes, individual Indian or Indians with trust allotments.

“The Act also authorized the formation of a negotiated rulemaking committee if so requested by a majority of the Indian Tribes with Self-Governance agreements. Such a request was made to the Department of the Interior and a rule making committee was formed. Under section 407 of the Act, membership was restricted to Federal and Tribal government representatives, with a majority of the Tribal members representing Tribes with agreements under the Act. Eleven Tribal representatives joined the Committee. Seven Tribal representatives were from Tribes with Self-Governance agreements and four were from Tribes that were not in the Self-Governance Program. Formation of the Rulemaking Committee was announced in the Federal Register on February 15, 1995.

“The first meeting of the Joint Tribal/Federal Self-Governance Negotiated Rule Making Committee was held in Washington, DC on May 18,1995 prior to publication of the proposed rule, a total of 12 meetings of the full Committee were held in different locations throughout the country. Subsequently, several meetings were held to review public comments and to negotiate changes in the final rule. The last meeting was held in Washington, DC in March 1999. There were numerous workgroup meetings and teleconferences during this period that were used to develop draft material and exchange information in support of the full Committee meetings. At the first meeting of the Committee, protocols were developed. The main provisions of the protocols were:

The Committee meetings were open, and minutes kept. The Federal Advisory Committee Act did not apply under the Unfunded Mandates Reform Act of 1995.
A quorum consisted of 8 members, including 7 Tribal members and 1 Federal member. The Tribal and Federal representatives each selected co-chairs for the Committee and an alternate.
The Committee operated by consensus of the Federal and Tribal members and formed 5 working groups to address specific issues and make recommendations to the Committee.
The intended product of the negotiations is proposed regulations developed by the Committee on behalf of the Secretary and Tribal representatives. The Secretary agreed to use the preliminary report and the recommendations on the final regulations, developed by the Committee, as the basis for the Notice of Proposed Rulemaking.
The Committee has reviewed all comments received from the notice of the Proposed Rule making and has submitted a final report with recommendations to the Secretary for promulgation of a final rule.
The Federal Mediation and Conciliation Services was used to facilitate meetings.

“The proposed regulation was published in the Federal Register on February 12, 1998 with a 90-day public comment period. Topics on which the Negotiating Committee had not reached agreement were identified in a preamble section titled, “Key Areas of Disagreement.” The Federal Register notice specifically invited comments on these areas. Fifty-four comments were received from a wide variety of Tribal groups and individual Tribes, Federal entities, and other groups and individuals. Many comments presented positions on the areas of disagreement. Subpart S, which pertained to Property Donation in the preamble of the proposed rule, pertains to Conflicts of Interest in the final rule. Property Donation is now in subpart Q of the final rule.”

Technical Work Group Members

Name	Contact Information	IHS Area
W. Ron Allen Tribal Council Chair	Jamestown S’Klallam Tribe 1033 Old Blyn Highway Sequim, WA 98382 (360) 683-1109 Phone (360) 681-3405 Fax rallen@jamestowntribe.org	Portland
Merle Boyd Second Chief (Alternate)	Sac and Fox Nation Route 2, Box 246 Stroud, OK 74079 (918) 968-3526 Phone (918) 968-3887 Fax	Oklahoma
Julia Davis-Wheeler Tribal Council Member	Nez Perce Tribe P.O. Box 305 Lapwai, ID 83540 (208) 843-2253 Work (208) 843-2965 Home (208) 843-7354 Fax (208) 843-2965 Home Fax juliad@nezperce.org	Portland
Barbara Fabre Director of Tribal Child Care Services	White Earth Band of Chippewa P.O. Box 418 White Earth, MN 56591 (218) 983-3285 ext. 407 barbf@whiteearth.com	Bemidji
Dan Jordan Tribal Self Governance Coordinator	Hoopa Valley Indian Reservation P.O. Box 1348 Hoopa, CA 95546 (530) 625-4211 Phone (530) 625-4594 Fax djordan@pcweb.net	California
Violet Mitchell-Enos Human Services Department Director	Salt River Pima Maricopa Indian Community 10005 Osborne Road Scottsdale, AZ 85256 (480) 850-8410 Phone (480) 850-7373 Fax violet.mitchell-enos@srpmic-nsn.org	Phoenix
Myra Munson Attorney	Sonosky, Chambers, Sachse, Miller and Munson 318 4^th Street Juneau, AK 99801 (907) 586-5880 Phone (907) 586-5883 Fax myra@sonosky.com	Alaska
Mickey Peercy Deputy Director of Health	Choctaw Nation Drawer 1210 Durant, OK 74702 (580) 924-8280 Phone (580) 924-3393 Fax mpeercy@choctawnation.com	Oklahoma
Gerrie Small President	Northern Cheyenne Tribe PO Box 182 Lame Deer, MT 59043 (406) 477-6284 work (406) 477-6210 fax ncexecassistant@ncheyenne.net	Billings
Anna Whiting Sorrell Director of Support Services	Salish & Kootenai Confederated Tribes P.O. Box 278 Pablo, MT 59855 (406) 675-2700 Phone (406) 675-2806 Fax annas@cskt.org	Billings
Alvin Windy Boy Chairman	Chippewa-Cree Tribe of Rocky Boy Rocky Boy Route, Box 544 Box Elder, MT 59521 (406) 395-4210 Work (406) 395-4497 Fax (406) 868-2134 Cell alvin@cct.rockyboy.org	Billings
Tim Martin Executive Director (Alternate)	United South and Eastern Tribes (USET) 711 Stewarts Ferry Pike, Suite 100 Nashville, TN 37214 (615) 872-7900 Phone (615) 872-7417 Fax jtmartin@usetinc.org	Nashville

Legislative History and Development of Tribal Self-Governance and Contracting Revised Report

Draft: Tribal Population Data Report

Technical Work Group Meeting February 3, 4 and 5, 2003: Meeting Minutes

Attendance:

Paul Alexander, ABW&W
Ron Allen, Jamestown S'Klallam Tribe
Ray Apodaca, Division of Tribal TANF
Lena Belcourt, Rocky Boy Health Board
Eric Broderick, HHS/OSIGA
Sheila Cooper, ANA
Kelly Eagleman, RBHB Chippewa-Cree Tribe of Rocky Boy
Sherm Edwards, Westat, Inc.
Barbara Fabre, White Earth Nation
Ginny Gorman, AHHS/ACF/CCB
Linda Greenberg, CMS
Peg Halpern, HHS/ASPE
Cynthia Helba, Westat, Inc.
Thomas Hertz, HHS/ASPE
Cyndi Holmes, Jamestown S'Klallam Tribe
Jessica Imotichey, HHS/OSIGA
Yvonne Jackson, HHS/AoA
Dennis Jones, Kimaiw Medical Center
Daniel Jordan, Hoopa Tribe
Jo Ann Kauffman, Kauffman and Associates, Inc.
Melanie Knight, Cherokee Nation
Rita LaFrance, HHS/OSIGA
Alana Landey, HHS/ASPE
Pam Langer, HHS/OSIG

Kathy Langwell, Project HOPE Center for Health Affairs
Tim Martin, USET
Violet Mitchell-Enos, Salt River Pima Miracopa Indian Reservation
Matthew Morgan, Chickasaw Nation
Myra Munson, Sonosky, Chambers, Sachse, Miller and Munson
Robin Patch, HHS/ASPE
Victor Paternoster, Kauffman and Associates, Inc.
Mickey Peercy, Choctaw Nation of Oklahoma
Juliet Pittman, SENSE, Inc.
Andrew Rock, HHS/ASPE
Benjamin Smith, Choctaw Nation IHS/OTSG
Anna Whiting Sorrell, Confederated Salish and Kootenai Tribes
Georgeline Sparks, Indian Head Start Bureau
Chris Stearns, Hobbs, Straus, Dean and Walker
Geoff Strommer, Hobbs, Straus, Dean and Walker
Eugenia Tyner Dawson, HHS/OSIGA
Richard Weston, HHS/ATSDR
Cliff Wiggins, Indian Health Service
Paula Williams, IHS/HHS/OTSG
Alvin Windy Boy Sr., RBHB Chippewa-Cree Tribe of Rocky Boy

Monday, February 3, 2003

Welcome and Introductions
After a brief welcome, the facilitator requested that all participants introduce themselves.
Project Overview
- Andy Rock thanked everyone for their attendance and mentioned that the combined efforts of Westat, Inc., Project HOPE and Kauffman and Associates, Inc. brings with it a wealth of experience to the study. He stated that the Technical Work Group (TWG) will participate throughout the study and provide advice when necessary. He said the study will provide information needed on a possible future evaluation. He stressed that this is a feasibility study and not an evaluation. Designing and conducting an evaluation are not part of this immediate process. Rather, he stated that what the TWG is engaged in is preliminary work that will lead up to the possible design and conduct of an evaluation in the future. Also, he said the primary focus of this process is on health services programs and programs that the demonstration study report (regarding non-IHS programs) recommended be included in a project extending Self Governance. He mentioned that there are 11 of these programs. He stated that the Office of Intergovernmental Affairs will be conducting consultations with the Tribes and he introduced Gina Tyner-Dawson of that office who is responsible for these consultations. (She mentioned that these will be the first consultations since 1999 and the forum will be open for budget issues and that information regarding the Evaluation Feasibility Study will be included.) Andy Rock then stated that role of the TWG is to advise the contractor on all aspects of the study. He stressed that this will be a productive activity and should not be viewed as critical of Self Governance, but rather it is seeking constructive ways of advancing it.
- Jo Ann Kauffman explained her role in facilitating the TWG process and recording information to documentation the Work Group's deliberations and advice. She also provided her tribal affiliation, background on KAI, and role as a subcontractor. Jo Ann provided a brief overview of her past projects and advocacy activities as they relate to SG, as well as her philosophy on SG.
- Sherm Edwards of Westat, Inc. explained that Westat is the "prime contractor" and this project was being conducted as one of its Task Order projects under its broad contract with the Office of the Assistant Secretary for Planning and Evaluation. Westat will be looking at some of the data and the logistical details. He mentioned that Westat has an administrative role and final review of documents before they are submitted to the Task Order Monitors (Andy Rock and Tom Hertz). In response to the question from a member of the TWG of whether Westat, Inc. has a Native American on its staff, familiar with Self Governance, Sherm responded affirmatively and indicated that this staff member would be working on the project.
- Kathy Langwell, with Project HOPE a subcontractor, mentioned that she has directed and participated in many quantitative research projects and has been designing programs and short term and multi-year projects. She mentioned that in the last two years, she has been working on a CMS project for eligibility of NAs in rural areas and conducting site-visits in tribal communities. She has several grants relating to NA issues.
- Alvin Windy Boy, Sr. expressed concern that he said he has spent 9 1/2 months in a process similar to this one with no results. He stressed that Tribes just want the ability to provide a service. He said that the trust responsibility has broken down and that Indian Country is no further along in 10 years and has not increased funding since 1955. Research is not a strange word in Indian Country. The research has not benefited tribes. If this (evaluation feasibility study) process results in things he can utilize to benefit Indian Country, he is behind it. But there needs to be a plan.
- Dan Jordan mentioned that studies often do not produce the desired results. He stressed that Self Governance has allowed Tribes to become empowered to address problems on reservations and that Tribes generally respond to studies in a constructive manner, but this information is not always reflected in studies. He suggested thinking about Self Governance just like we do democracy. He stated that Self Governance makes the economy work and creates jobs and this needs to be captured in any evaluation study. He stressed that a tribe's ability to control its environment is the fundamental purpose of Self Governance and program issues are secondary. He mentioned that we would be looking at analysis of increase in services and funding. Services have increased but not funding. Has SG changed the lives of those living on reservations? Have these services improved the lives of Tribal members? Does SG (????) the study actually empower the Tribes comparable to the rest of society?
- Tim Martin mentioned that Alabama is mentioned in the Data Matrix Report, Page 3. He pointed out that state-recognized tribes are listed, but not federally recognized tribes in the state. Self Governance only applies to federally-recognized tribes. He indicated that we should purge this data. Kathy Langwell responded that this table was taken from the Federal Register listing federally recognized tribes and there shouldn't be any State Tribes in this report. She will double check and make the corrections.
- Mickey Peercy stressed the importance of staying focused on our purpose and that the ultimate questions are whether there is or is not data available to do a study. He suggested also making a list of areas the group does not want to address in order to stay focused as it will be easy to move down different directions.
TWG Ground Rules
A discussion was conducted to seek agreement about procedural matters.
1. TWG Agreements on the Ground Rules:
  1. Stay focused on:
    1. Purpose of the study
    2. Feasibility of an evaluation process
    3. Is data available to conduct the study
  2. Written charge from the HHS to the TWG is to be developed
  3. TWG is advisory to the contractor
  4. Formal Tribal Consultation is a separate process
  5. Protocol will be developed and reviewed (This was completed before the 3-day meeting adjourned.)
Legislative History & Literature Review
1. TWG Recommendations about the Legislative History Report:
  1. Allow written comments
  2. The History paper needs to capture why Tribes became involved in Self Governance
    1. Complexity and nuances have been lost in this report. It's too sanitized. It doesn't appear to fully take into account the Tribal perspective.
    2. Should reflect failure of non-BIA DOI SG effort.
    3. It should reflect the frustration of Tribes over 1988 regulatory process.
    4. Should indicate also that Tribes may exercise their SG even when electing to keep HIS direct services arrangements.
    5. Title IV preamble of proposed and final rules describes this process very well and it is a federal document that has already been published. It should be used to convey this history.
    6. There are gaps in what is currently available. Perhaps will need an oral history of tribal leaders that were involved to capture some of this history.
    7. The difference between negotiated rule making and the previous process for establishing regulations should be reflected.
    8. The Lummi Demonstration Project is a good source for information.
    9. Early SG reports are available and there is a "Blue Print for Self Governance" which is a good source.
    10. Paula Williams of IHS/HHS/OTSG mentioned that there is a case study with Valerie Davidson that is coming out and she can distribute to the TWG when it comes out.
  3. There needs to be clarification that Self Governance does not stem from PL 93-638, but from a different process; a government-to-government process.
  4. The reasons why some Tribes opt for PL 93-638 contracting and not SG compacting should be addressed.
2. TWG Recommendations for the Draft Literature Review:
  1. Obtain DHHS evaluation on EPSDT Medicaid Expansion at Yukon Kuskokwim through HRSA.
  2. Need section describing quantitative and qualitative methods, and the history of each approach in regards to SG. The study seems to imply that only quantitative methods are good. The paper should identify what needs to be examined and how both qualitative and quantitative methods could be used to achieve answers to these questions.
  3. Need to ask IHS for existing quantitative evaluative studies on Self Governance and Non-Self Governance and make effective use of RPMS data system.
  4. Move emphasis on limitations of the study and indicate it may be difficult to conduct evaluation because:
    1. Small tribal populations;
    2. Dispersed populations through out country;
    3. Move this limitations section to the front;
    4. Constraints in longitudinal approach;
    5. Causal relationship to health status versus relationship to health care; and
    6. Define what would be statistically significant and what this research would cost.
  5. Need to note that Indian programs are starting from a point of under-funding, for example LNF. (level of need funding????)
  6. Page 4 regulation statement needs to be reworded, it says compacting is not subject to regulation and that is not true.
  7. Reference relationship of poverty to health status, and provide more emphasis on the economic stimulus that SG can have on communities.
  8. Be careful in wording, so that phrases cannot be lifted out of context to the unfair detriment of tribes.
  9. Better transition from IHS to non-IHS programs needed.
  10. Is there a risk of documenting successes? i.e. :
    1. Federal cutbacks?
    2. Unfair comparisons?
  11. Conclusion needs rewording.
  12. Add 477 successes experienced by tribes combining jobs, vocational rehab and child care resources.
  13. Better explanation and application of "user population" is needed as this will impact quantitative data.
  14. Discuss importance of Tribal buy-in or support for research/evaluation.
  15. Note Tribal investments; e.g., Contract health services, Staff, Construction, etc.
  16. Obtain report on Tribal shares [94-95 JAMAW (???) report through Mr. Lincoln or Mickey Peercy].
    
    Additional Project Team Comments:
    - The paper reflects what research has been conducted to date.
    - More literature was found than anticipated.
    - Most of the existing research is qualitative with focus on process, perceptions, documentations, and descriptions. For example, NIHB study did excellent job conducting a survey and seeking perceptions of SG tribes.
    - It is not surprising that there is little quantitative data.
    - What we were looking for in the literature review is a comparison of existing literature.
    - There is quantitative data and studies being done on Indian programs, ie there is a Mathematica Quantitative Data Report regarding TTANF; and Westat, Inc. is doing an evaluation of Head Start projects.
    - This is an important document in laying groundwork of where the project and data might go.
    - It may not be worthwhile for us to dig into everyone's data sets to tease out information. What is the relative standard of comparison is an issue.
    - Pre and post comparisons are a good way of looking at changes.
    - Cross-sectional comparison is another method.
    - We are exploring IHS data.
    - In the end, politics and "telling the story" and not quantitative data is important to Congressional staff when considering legislative changes.
Methodology
1. TWG Recommendations for Evaluation Methodology:
  1. Feasibility of quantitative evaluations
    1. a. IHS and non-IHS
      1. What data is there? Is it accessible? Is it comparable?
    2. Six (6) Tribal site visits
      1. What data is there on site?
      2. Prepare Tribes to address non-IHS questions
    3. What is the baseline to compare the progress of Self Governance?
      1. IHS activity pre-Self Governance?
      2. Treaty obligations?
    4. Describe the process of success
  2. No single measure will apply to all Tribes. This must be flexible.
  3. Given literature supporting local control of health care, can we then document process of increased Tribal leader and community involvement?
    1. Consumers are more involved in priority setting under SG
    2. Increased role of Tribal leaders in health resources and priority setting
  4. Determine feasibility to evaluate IHS Self Governance to better inform potential non-IHS Self Governance expansion and evaluation considerations
  5. Track the time needed to produce necessary data for this study
  6. Questions of tribes engaged in gaming should not be an issue for this study; a focus on tribes' efforts to maximize their resources could be viewed negatively by some policy-makers.
  7. Consider the geographic and demographic differences of Tribes across the country
  8. This process should be carried out so as to move us closer to implementing a Title VI Demonstration
  9. How can we measure "Empowerment" under SG?
    1. Participation of community via quantitative measures
    2. Probably qualitative would be most descriptive method
  10. Health outcome measures probably not realistic for this purpose.
    1. Is this realistic for small populations and time with Self Governance?
    2. Is it realistic to assume health status changes due to SG given the short time frame?
    3. How do you make causal link between health status changes and SG?
  11. Recommendations for evaluation comparisons that should be made:
    1. Pre/Post Self Governance of the same tribe;
    2. Program processes; and
    3. Measures for reorganization, administration or governance.
  12. Elements of qualitative measurement that focuses on SG could include:
    1. How much program redesign was done;
    2. Whether financial systems modified/improved;
    3. What changes were made to personnel and administrative systems;
    4. Differences under local decision making; and
    5. Whether services are more comprehensive under SG.
      
      Project Team responses to comments:
      - The feasibility study seeks to answer whether it is feasible to do an evaluation. We are not actually doing the evaluation now.
      - We started with a broad range of topics.
      - If funding and patient utilization data aren't acceptable measures, what quantifiable data would you like to see that would be of interest to all stakeholders?
      - We seek to define the broader issues and then go to IHS and do a review of all the data and then recruit Tribes to participate in the site visits (Tribes, program managers, etc.).
      - If this is structured to address the impact of Self Governance on key outcomes, etc., then we need to know what was there before (IHS data) and then go out to the Self Governance Tribes. We're going to have to rely upon individual Tribes who are willing to participate.
      - We hope to recruit Tribes to participate in the feasibility study and who would also be willing to participate in an evaluation that may follow. Interested Tribes represented on the TWG are encouraged to consider such participation.
      - We are looking at this (TWG) group to provide advice on the parameters of the study. We might end up with recommendations for an evaluation emphasizing qualitative measures if that is the only feasible way of collecting data.
      - The Tribal Self Governance Demonstration report shows a consensus and willingness to proceed on the part of the Department. The decision of when and how to proceed with a demonstration project would need Congressional legislative action. We would expect an evaluation would be part of any mandated demonstration project enacted.

Tuesday, February 4, 2003

Major Study Areas & Questions
1. TWG Recommendations for Study Area Parameters:
  1. Focus on Governance - Should answer the question: "What is the impact on Tribes"
    1. Administration or Governance changes
    2. Structural changes
    3. Finance changes
  2. Health Indicators (utilization, systems, status)
  3. What did we do and how did we do it to fill the shoes of the federal government?
    1. FTCA
    2. Budget
    3. Staffing
    4. JCAHO Ratings
  4. Proposed "Guiding Principles" for this feasibility study:
    1. Evaluation must start with consultation and negotiation with specific Tribes involved in the evaluation process;
    2. What are the do's and don'ts for SG?
    3. Identify Tribal specific measures
    4. Identify where/if data might be available
      
      Technical Work Group Comments:
      - Tribal leaders and tribal representatives caucused. Discussion resumed.
      - Discussion regarding qualitative and quantitative means to answering the same questions. The TWG recommended that we go through the questions and that there be both a quantitative piece relating to Self Governance as well as qualitative. Questions drafted to date are health indicator questions and we want management level information as well.
      - Some Tribes were doing these things prior to Self Governance via 638.
      - Check with the government for existing records and reports, as it is disrespectful to the Tribal administrator and staff, for federal contractors to come to the community and ask for data that has already been submitted to federal agencies.
      - There is a copy of the Health Indicators Baseline Measures. However, none of the health indicators are mandated and usually it is not used.
      - It's critical that you speak to Tribal Chairman and leaders and get their perspectives.
      - You need to talk about more programs than just health care. We cannot just look at one program by itself.
      - What is the array of providers and the relationships in place?
      - All the people at the table are willing to help regarding coming up with some questions. We have a question about your contractor's understanding of what the leaders are saying and what the contract is about.
      - What is the change in staffing and what is the change in programs of similar size in 94 are two controls worth looking at.
      - We assume that IHS hasn't been stagnant either; that it has also changed services, provided greater access, etc.; and that they have the data.
      - Because programs are so different, Tribes must you pick and choose what is applicable for them and an evaluation needs to take into account these choices.
      - An evaluation would need feedback on the quality of data. The data cannot just be taken by itself; it must be explained and the context provided.
      - You have to ask how many providers you have access to, etc. to broaden the questions and not get hung up on the words as this will be redrafted.
      - Will we be asking about socio-demographic data?
      - Open ended questions will allow for more qualitative data. The qualitative questions will allow for additional information. There needs to be both qualitative and quantitative components.
      - Part of what we want to focus on is the importance of Self Governance that allowed for getting a doctor or other opportunities. An increase or decrease may not be bad. Having no doctors isn't necessarily bad since the tribe may have contracted for these services with a local clinic.
      - To measure success, you have to look at how an individual tribe achieves its own goals rather than universal goals. They control the resources more now than before Self Governance.
      - The tribe can provide a certain matter of added value. This study doesn't indicate that there are study rules up front. Tribes could use this as an option. There should be health and non health criteria/standards out there. There have been a tremendous amount of studies conducted.
      - The source of funding is not critical to the evaluation of the program.
      Project Team responses to comments:
      - We are primarily looking at quantitative data when doing site visits.
      - We are looking at both qualitative and quantitative data. Whether information is available is the main focus. We are not getting the data for purposes of evaluation. Information availability is not a criterion for site visit selection.
      - Can we get the information from the site visits is the question. We might find there are ways of getting the data, but it could be cost prohibitive. We need to answer these questions. Another question could be: can you get historical information today?
2. TWG Recommendations for Study Questions - The questions should be reworked and consider the following issues:
  1. Need to Recognize Phases in the SG Process. Not all features are fully in place from the outset; they occur over time and SG tribes are learning organizations that change.
  1. 1. Stabilize/trust
    2. Management changes
    3. Growth/expansion
  2. Governance
    1. Formal connection to the community
    2. Organizational structure (lines of communication)
  3. Management Issues
    1. Process and changes, etc.
    2. How did Self Governance help in addressing management issues?
3. TWG Recommendations for Non-IHS Study Questions --- the non-IHS study questions should be reworked and consider the following issues:
  1. Describe organization/governance
  1. 1. Advisory Boards
  2. Inter program coordination already underway
    1. 477
    2. TANF
  3. Historical documentation
    1. Original Annual Funding Agreements (AFA), (Congressional Appropriations)
    2. Tribal Plans
    3. 638 Transition history
    4. Program Budgets
    5. Inter-Tribal Coordination
4. TWG Recommendations for Program and Benefits Section:
  1. Expand and supplement…..
  2. Reorganize
  3. Annual Report?
  4. Annual ACF 700 report and other Title VI reports/regulations
5. TWG Recommendations for Accreditation Section:
  1. List
  2. Years
6. TWG Recommendations for Facility Section:
  1. Numbers
  2. Accreditation
7. TWG Recommendations for Quality/Satisfaction Section:
  1. Background
  2. Compare to National/Data?
  3. Identify Existing Standards/Data Collected by IHS and Non-IHS
8. TWG Recommendations for Management Challenges Section:
  1. Add "and Opportunities" to read Management Challenges & Opportunities
  2. Add: What changes do you contemplate would result in improved services to Tribal members?
  3. Add question: How does operating as a SG Tribe allow you to address problems differently?
    
    Project Team Comments:
    - There may be times when Self Governance is relevant and times when it isn't. What are the management issues you were trying to tackle before and after Self Governance?
9. TWG Recommendations for Availability of Services/Access to Care Questions (Page 2):
  1. Add: What changes did you make and why?
  2. Add: What problems still exist and why?
  3. Add: What could fix the problems?
  4. Add: What factors are most important in the decision to bring services in-house versus Contract Health Services?
  5. Discuss with community issues considered in order to decide how services are delivered (direct, CHS?) relative to "in house" question
  6. What is the CHS Risk Management under Self Governance
  7. Did you first do a feasibility study to take over CHS?
  8. Management opportunities under Self Governance and CHS
  1. 1. Clear up old debts?
    2. Negotiate Better hospital/provider rates?
    3. Hired direct provider?
  2. Waiting Time
    1. What has a Tribe done to reduce waiting times?
      
      Technical Work Group Comments:
      - Services are different everywhere.
      - Questions need to be refined.
      - 2001 User Population is the most current total tribal client data (per Cliff Wiggins).
      - How do you manage the risk with contract health?
      - It is possible to bankrupt a system when becoming involved in CHS.
      - What was the reasoning behind doing Contract Health?
      - CHS under IHS is a difficult system with long delays in payment.
      - What did the Tribes do to manage Contract Health after taking it over from the IHS?
10. TWG Recommendations for Quality of Care/Process Questions (Page 3):
  1. IHS/Non-IHS
  2. Accreditation as a measure
  3. Licensure as a measure (or at least to describe what is followed)
  4. National Standards as a measure
  5. Did you change application requirements with Self Governance?
    
    Technical Work Group Comments:
    - Look at Alaska reports to outline problems with data collection in all areas not just AI/AN data collection.
    - Remember to look into questions about IHS not being adequate or labeled as insurance.
    - Immunization gap between fed and non-fed programs.
    - How to differentiate that the data collected will go along with regional differences.
    - NIH Research into diabetes, but remember diabetes funding is not included under Self Governance.
    - Preliminary qualitative diabetes data rather than quantitative.
11. TWG Recommendations for Quality of Care/ Health Outcomes Questions (Page 4):
  1. Eliminate all original questions
  2. IHS GPRA Seminal Health Indicators
  3. Baseline Measures (95-96)
    
    Technical Work Group:
    - All of these questions should be eliminated.
    - Mickey Peercy will come up with some additional health indicators. He indicated that there are 110 Baseline Measures in the Report, but these are not required of tribes.
    - Cliff Wiggins mentioned that IHS is going to use seminal indicators that are part of the overall budget, which is more measurable than the overall assessment.
12. TWG Recommendations for Quality of Care/Patient Satisfaction Questions (Page 5):
  1. Eliminate: What proportion of the population eligible for services uses the SU annually.
  2. Add: Have you opened a new facility?
  3. Add: Is there data on change patterns?
  4. Add: What percent of potential users utilize your facility?
  5. JCAHO Customer Satisfaction Survey
    
    Technical Work Group Comments:
    - There is no common denominator to who is eligible. Census could be used, but it is self reporting.
    - Standards of eligibility varies from tribe to tribe.
    - The information you are seeking is not reliable enough to make the links you are trying to make.
    - Every facility has some measure of patient satisfaction. Ask interviewee if you can quantify this data and what changes have taken place. Is there a pattern and do they have the data.
13. TWG Recommendations for Financial Performance Questions (Page 6):
  1. Eliminate the last four questions
  2. Add qualitative questions and identify as such
  3. Regarding question 2, units of service, need to first ask whether there have been changes in the types of care rendered.
  4. It is unclear what the value of the numbers is; the data here may not answer the question of why they are higher or lower; the interpretation of the data is the essential key; merely collecting it could be misleading or worse.
  5. Add question relating to the pre/post collection of dollars
  6. Regarding the question concerning collection from third parties; a more important issue may be the efficiency of the business office.
  7. Add question relating to financial incentives for MD's, etc.?
  8. Add question relating to traditional practitioners
  9. Add question relating to negotiated rates
  10. There may be other proxies rather than dollars for the effectiveness of financial performance; such as audit information.
  11. Add question relating to providers per square footage of facility
  12. Add question relating to the efficiency of the business office
  13. Add question relating to 3^rd party billing system reserves devoted to tribal priorities
  14. Add question relating to the availability of a reserve fund
    
    Technical Work Group Comments:
    - The questions need to start with numbers rather than percentages.
    - You have to measure the level of effort on the part of the tribe. You can get this information of 1 and 2 through RPMS.
    - You have to put all the answers in some sort of a context. Numbers always mean something to the reader and they can be read in many ways, thus, it is important that they are put in some sort of a context. The data is feasible to collect, but does it make any sense.
    - Why is the break even and surplus question important? You can look at the proxies and the audits.
    - If at the end of the report, you mention that there is a lot of data available from the Federal government, this would be helpful to Congress.
    - Remove the last four questions.

Wednesday, February 5, 2003

Small Group Discussions
1. TWG Recommendations for the Discussion Groups --- Contractor will rework the Discussion Group format based upon TWG input:
  1. Focused on Self Governance Tribes
1. 1. 1. Evaluation Feasibility
    2. Title VI Expansion evaluation issues
    3. Target and invite Tribes that are running 11 programs
    4. National Tribal 477 programs
    5. Originally planned for early in the project to help inform site visits and focus of inquiry.
    6. TWG proposes moving to end of study to test draft findings
      
      Technical Work Group Comments:
      - Schedule small group discussions after the site visits instead of before like a Focus Group testing what you have found out.
      - You can do the Focus Groups with specific questions.
      - Are the six sites representative of the whole?
      - Wait until you have the preliminary findings in hand and then ask if these are consistent with the experiences of the 6 discussion groups.
      - If this was an evaluation of a project, which it is not, then one would have to go out in advance and ask for comments on the evaluation protocol. This study is seeking the feasibility of an evaluative study. The critical stage of this is the preliminary findings.
      - There is always an update at conferences. The discussion groups make sense at the end. A combination of Tribes considering Self Governance, TANF, Head Start, etc.
      - NCAI would be a good forum for Discussion Groups.
      - Tribes running 11 programs should be invited. We should use 477 programs as a good resource.
      - August through October is a difficult time for good conferences.
      Project Team Comments:
      - We will rework a draft schedule and format for the discussion groups looking at an Aug through Oct window.
      - This will likely require an extension of the contract.
      - May not be possible to cover agenda with 10 people in two hours unless the agenda was focused/limited.
      - We've scheduled into the contract 6 groups at 2 separate locations for 2-3 hours each. We also talked about doing this process in two different segments. Pre and Post.
Tribal Matrix
1. TWG Recommendations for the Tribal Matrix:
  1. Project HOPE will correct and re-send to the TWG
1. 1. 1. Delete State Tribes
    2. Add Federally Recognized Tribes that are missing (i.e., Havasupai, Tonto Apache, etc.)
  2. IHS User population in addition to census data
  3. Use footnotes to address limitations of data sources
  4. Seek volunteers for participation in the site visits and group discussions at the April SG Conference
    
    Technical Work Group Comments:
    - Use user population data from HIS; it has already been reviewed by tribes.
    - Cliff Wiggins mentioned that the TWG can obtain user population with little effort for health programs.
    - Census data that is self reported doesn't mean anything in the context of who a tribe provides services to.
    - User population has limitations too.
    - Census data however might be used for other non-IHS programs;
    - We deal with our service areas differently.
    - FDI/LNF data is useful also and should be reviewed.
    Project Team Comments:
    - This data is used not as a benchmark, but more relative than absolute numbers.
Site Selection
1. TWG Recommendations for Site Selection Criteria - the TWG was not willing to identify site selection criteria until it is clear what the purpose of the site visits will be and what the study questions will be for the site visits --- Issue addressed included:
  1. Cross section of tribes, sizes, demographics, locations, etc
  2. Recruitment process
  3. Governance profile
  4. FHDI/LNF data
  5. Regional disparity
  6. Visit those with the most experience
  7. Tribes must volunteer
  8. Self Governance Tribes with experience in the 11 Title VI programs will narrow the focus
  9. Size stratification
  10. Service Area demographics
  11. TWG needs to review study methodology first
  12. TWG will draft the site selection criteria for review
    
    TWG Comments:
    - During site visits, we are looking at data. We need to define the characteristics of selected sites.
    - We need to know what we're testing and why first.
    - Criteria for site selection should include volunteer sites, geographic distribution, multiple HHS programs operated, size/demographics of service areas, land based status, governance status.
    - Need to use IHS rather than Census user population data.
    - Questions need to be broader than health care as there are many different programs.
    - Need to indicate what the data limitations are and how the matrix data will be used.
    - Are the sites all going to be Self Governance?
    - We need to start with the questions before selecting sites.
    - Questions need to reflect 638 and what it brings to Tribes.
    - Tribes are very sensitive. They may feel that this study seeks to evaluate Self Governance.
    - This is about advancing Self Governance and not comparing them. This is not comparing one tribe to another. Rather, does Self Governance allow Tribes to be more successful or not and how do these Tribes do business.
    - We don't want this to be an impact on non-Self Governance.
    - Assessing data on services needs to be the focus, not comparisons.
    - The difference between why Tribes are doing what they are doing is more the question.
    - Mandated reallocation of funds by Tribes is not the issue, rather showing how Tribes match and exceed the dollars that are received is the issue.
    - There is an opportunity to move forward the Title VI efforts.
    - How will the money flow smoothly through the system is a key issue.
    - A good assessment can help us with legislation.
    - Need to emphasize levels of need and federal health disparity index.
Redrafting Communication Protocol
1. TWG Recommendations for Redrafting the Protocol --- The redrafted protocol was eventually agreed to and signed between the Title VI Advisory Committee and the DHHS --- There were discussion within the TWG regarding this process:
  
  Technical Work Group Comments:
  - We need to define the relationship between Title VI Advisory Committee and TWG. Not all the tribes represented on the TWG are involved with the Title VI Advisory Committee, and it is not clear to them what this relationship is.
  - Why is the TWG approving or reviewing an agreement between the Title VI Advisory Committee and ASPE?
  - There is a responsibility of HHS to provide information/communication to Title VI Group.
  - There are members of the TWG on the Title VI Group and thus there is a linkage.
  - There is a responsibility to communicate with all tribes, not just those on the Title VI Advisory Committee. Will the Advisory Committee take on that additional role?
  - The Title VI Advisory Committee needs funding to continue the link and communication between the two groups. Protocol defines communication, but this will require funding.
  - The Title VI Steering Committee provided the start to this evaluation feasibility study. This protocol provides support to the TWG and the Title VI Steering Committee. There is a need to cover the costs to Sense, Inc., for facilitating the tribal communication process.
  - Those members of the Title VI Steering Committee have not been communicating properly with the TWG.
  - This document was developed before the TWG. It sets protocol for the study and not for the TWG. This is the Title VI Steering Committee telling the TWG what to do rather than vice versa.
  - This background information would have been helpful earlier. We had on the agenda Ground Rules and why we are here. Some on TWG originally viewed this as there was not support for the Title VI Steering Committee by the Department and that the Title VI Steering Committee was trying to get their support through the TWG.
  - The TWG should have a list of those on the Title VI Steering Committee and a list of their meetings.
  - The TWG requests to see a breakdown of the prime and sub-contracts and who does what and the percentage of the total used for communications. The TWG requests to see a breakdown of the dollars being used on this project.
  - This isn't about nitpicking, but rather dialogue. We on the Title VI Steering Committee don't have much control, but we do have an interest.
  - We wish we had more input into the contract negotiations to ensure success and efficiency.
  - TWG expressed concern about what is the role of Westat, Inc.. and concern about contractors who don't know anything about Indian Country doing Indian projects.
  - Two meetings of the TWG are not going to get this job done.
  Project Team Comments:
  - Contractors will be in communication with TWG. We also have tribal leader representatives from the NIHB and NCAI who are not at this meeting, but may want input on the protocol.
  - In principle we should support the cost of communication, but we have to know what we are talking about before we seek more funding.
  - Exclusively quantitative studies aren't appropriate. Management changes that are quantitative, not just health indicators. You can measure things like daycare, etc. over time for turnover.
  - We have to be careful not to deviate to far from the scope of work that has already been agreed to and signed between the contractors and the government.
  - The Steering Committee works at a higher policy level with DHHS. They will continue after the TWG ends. The TWG is advisory to the contractors on this particular study.
  - What can be accomplished is based on the workload and tasks. There has been much work that went into the project based on last year's budget.
  - The timing wasn't in DHHS' and the project expanded from its original proposal.
Next Steps
TWG Recommendations for Next Steps:
1. Myra Munson redrafted the protocol (See attachment A), which was signed and is posted on the ASPE web site.
2. Contractors will redraft all materials to go out to the TWG by March 8^th
3. TWG (tribal only) teleconference to be conducted to review redrafted materials.
4. TWG will propose site selection criteria and comments on the redrafted materials by March 30^th
5. Follow-up with ASPE, contractors and leadership to identify possible sources of funding to support Sense, Inc., tribal communication efforts.
6. Review budget
7. Formal TWG (all participants) teleconference to be conducted in April (to be scheduled).
8. Six sites need to be identified after agreement on site selection criteria.
9. Six site reviews in June, July and first half of August
10. Next (not budgeted) TWG Meeting will be May 2^nd at Salt River, if possible. This meeting will be held to coincide with Spring Self Governance Conference at the end of April.
11. 2^nd Budgeted Meeting will be tentatively targeted for September.
12. Identify best opportunities to conduct small Discussion Groups at two locations (3 discussion groups in each location), ie., NIHB Consumer Conference, NCAI Convention, national TANF or Head Start gatherings, etc.,

Draft: Evaluation Issues, Questions, and Data Requirements

Revised Draft Literature Review

Revised Draft Tribal Matrix Report

Topics

Service Delivery & Capacity Building | Healthcare Reform

Populations

American Indian & Alaska Native People (AI-AN)

Location- & Geography-Based Data

Tribal Communities