Feasibility Study for the Evaluation of DHHS Programs That Are or May Be Operated Under Tribal Self-Governance. Tribal Support for a Specific Evaluation Approach

03/03/2004

The extent to which Tribes may support an evaluation is a key issue for this study. Both the Technical Working Group and most of the Tribal staff who participated in site visits stated that Tribes would, in general, be supportive and positive about an evaluation of DHHS programs that may be operated by Tribes under a new Self-Governance demonstration.  Designing an appropriate evaluation approach that could produce useful findings for the Tribes, as well as for DHHS, is desirable and should take into account the unique issues of Tribes and the principles of Self-Governance.

Many Tribal representatives who contributed to this project emphasized that any evaluation should be structured as an evaluation of DHHS programs managed by Tribes under a Self-Governance demonstration, rather than as an evaluation of Self-Governance.  There is concern that an evaluation of Self-Governance could be construed and/or the findings used to alter the terms or basis of Self-Governance.  To allay those concerns and encourage Tribes to participate in an evaluation, it would be very important to be clear in the stated evaluation objectives that DHHS programs are to be evaluated, rather than Self-Governance.

Discussions with the Technical Working Group and others also stressed that it would be inappropriate to design an evaluation that used a standard set of outcomes to examine DHHS programs operated under Self-Governance.  A principle of Self-Governance is that Tribes should have flexibility to set objectives and design programs to meet each Tribe’s priorities, which may be different than priorities set for Federal programs, generally.  Tribes might be less likely to support an evaluation that set a standard set of outcomes and more likely to support an evaluation that permitted Tribes to set specific and unique program goals that then were examined to determine whether and what extent these goals were achieved.

In addition, it is probable that Tribes might be more willing to participate if:  1) there is a perceived benefit to Tribes from an evaluation, 2) there is extensive consultation on the evaluation objectives, issues, and data that will be collected, and 3) the costs of data collection and reporting are minor or are the responsibility of the Federal government.  Tribes might be more willing to support an evaluation, also, if there were clear and detailed agreements in place that indicate that evaluation data collection/reporting would be limited to the evaluation period and would not continue after that period.  In addition, an evaluation that was structured to report findings across all participating Tribes or large subsets of Tribes would be more likely to encourage participation than an evaluation that would report on individual Tribes.

Technical Working Group members also said that the total costs to DHHS and the Tribes of an evaluation approach, relative to the potential value of the results, would be a consideration for many Tribes.