Congress, DHHS, and Tribes may have quite different views on the usefulness, need for, and objectives of evaluations of programs.
When Congress authorizes a demonstration program, it is generally with an understanding that it will be designed and implemented for a limited period of time to determine whether the new approach can effectively and efficiently meet specific goals. Evaluation of demonstration programs provides information and evidence on the process through which the programs are implemented, operational issues, and on impacts and outcomes of the demonstration programs, relative to the goals of the programs. Evaluation findings may also provide information that can be used to refine and improve the demonstration program as it transitions to permanent status. Because evaluation is an accepted tool for assessing new programs, Congress often requires that an evaluation be conducted of new demonstration programs that it authorizes.
Agencies within the Department of Health and Human Services may have an interest in evaluating new programs that provide services to target populations. Management and program staff that have responsibility for specific programs have often worked in their fields for many years to develop effective programs that are designed to provide services and meet defined needs of the population they serve. When a new program is undertaken, there may be concerns about whether the program objectives will continue to be achieved and whether the target population will be as well served as it was under DHHS program management. Evaluations of how the newly structured programs operate and meet the needs of the target population may allay concerns and provide increased support for the new program.
Tribes and Tribal organizations may perceive evaluations as having less value for them and as posing some risk that there may be potential negative consequences associated with evaluation findings. This may be particularly the case for evaluation of DHHS programs managed by Tribes under self-governance, since the underlying principle of self-governance is to permit Tribes to develop and administer programs that are more responsive to Tribal priorities and needs than are Federally-run programs. Any evaluation of Federal programs operated by Tribes under self-governance may be perceived as an attempt by the Federal government to find problems with Tribal management of programs. A traditional evaluation approach that identifies a standard set of objectives to be measured across all self-governance Tribes also may be perceived as inappropriate, since each Tribe may have unique objectives for its programs. The resistance of Tribes to participate in evaluations of Federal programs operated by Tribes is a critical challenge for any potential evaluation. Tribal cooperation and agreements to participate in an evaluation is necessary for the conduct of any rigorous and useful study.
Potential Benefits of an Evaluation
If an evaluation of DHHS programs operated under Tribal self-governance were to be conducted, it would potentially have several benefits both to DHHS and to Tribes and Tribal organizations. DHHS program managers and staff who have had responsibility for administering programs that the Tribes would manage under a demonstration, if authorized by Congress, would perceive an evaluation as consistent with the normal approach to assessing the effectiveness and success of any new program arrangements undertaken within the Department. Evaluation could provide evidence that the needs of clients are being met under the new management structure, even though some aspects of the program may be different than under direct Federal program management. In addition, results of evaluation of DHHS programs managed by Tribes could provide information that would increase understanding of Tribal issues and goals among DHHS program managers and staff and the benefits and successes of Tribal self-governance in better meeting the unique needs of Tribal members.
Evaluation of DHHS programs managed under Tribal self-governance could provide useful information to Tribes and Tribal organizations, as well. Results could provide information on “best practices” and innovative programs that could be used by other Tribes to improve services and performance in program management. There is also the potential for the findings to demonstrate that Tribal self-governance is an effective method for improving services and meeting needs of individual Tribes and Tribal members that would provide support for further expansion of self-governance to additional DHHS and other agencies’ programs.
Potential Disadvantages of an Evaluation
There are, however, some potential disadvantages to DHHS and to Tribes of conducting an evaluation of DHHS programs managed under Tribal self-governance. It would be necessary for DHHS to initiate and engage in what might be a lengthy consultation process with the Tribes in order to define the extent and range of any evaluation that might be conducted. In addition, individual negotiations with each Tribe to obtain and work out the terms of an agreement to participate in the evaluation would likely be necessary. Finally, the costs to DHHS of conducting an evaluation of the DHHS programs managed by Tribes would likely be significant, necessitating trade-offs between the comprehensiveness and rigor of an evaluation and the associated costs.
There are also potential disadvantages of an evaluation for Tribes. If some evaluation findings suggested that Tribal management of DHHS programs was less than successful, or that some Tribes were less effective than others in managing these programs, those results could have negative consequences for the advancement of Tribal self-governance. There would also likely be costs to the Tribes associated with data collection and reporting for the evaluation, even if DHHS provided uniform data collection systems and training in support of the evaluation.