Feasibility Study for the Evaluation of DHHS Programs
Operated under Tribal Self-Governance

U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES
OFFICE OF THE ASSISTANT SECRETARY FOR
PLANNING AND EVALUATION
November 30, 2003

Draft Final Report

Delivery Order 27
Under Contract No. HHS-100-97-0017

Authors:
Kathy Langwell
Cynthia Helba
Jo Ann Kauffman

Prepared for:
Andrew Rock, Task Order Mgr.
Office of Planning & Evaluation
Dept. of Health & Human Svs.
200 Independence Ave., SW
Washington, DC   20201

Prepared by:
WESTAT
Rockville, Maryland
1650 Research Blvd.
Rockville, MD   20850

ACKNOWLEDGEMENTS

Cynthia Helba and Kathy Langwell, Westat, and Jo Ann Kauffman, Kauffman and Associates, Inc., prepared this Draft Final Report.  W. Sherman Edwards and Brian Colhoff, Westat, Victor Paternoster, Kauffman and Associates, Inc., Janet Sutton, Project HOPE Center for Health Affairs, and Frank Ryan, project consultant, made substantial contributions to the study.  We would like to thank the members of the Technical Working Group for their assistance and thorough reviews of draft reports throughout the study.  In addition, we are grateful to the Tribes/Tribal organizations that volunteered to participate in site visits that were conducted for the study, including:  Bois Forte Band of Chippewa, Choctaw Nation, Hopi, Little Traverse Bay Band of Odawa Indians, Port Gamble S’Klallam Tribe, and the Yukon-Kuskokwim Health Corporation.  Several individuals at the Indian Health Service also contributed generously of their time and provided assistance with data review activities.  Finally, we would like to thank Andy Rock and Tom Hertz, OASPE/DHHS Task Order Managers for the project, who provided valuable guidance and insights throughout the project.

TABLE OF CONTENTS

• Chapter 1 - SUMMARY OF FINDINGS, BackGround, AND OBJECTIVES
  • 1.1 - Summary of Findings
  • 1.2 - Background of the Study
  • 1.3 - Objectives of the Study
  • 1.4 - Organization of this Draft Final Report
• Chapter 2 - ISSUES AND CONSIDERATIONS FOR EVALUATION FEASIBILITY
  • 2.1 - Overview
  • 2.2 - Perspectives on Evaluation Research:  Congress, DHHS, and Tribes
  • 2.3 - Feasibility Considerations
• Chapter 3 - STUDY METHODOLOGY
  • 3.1 - Approach to Assessing Feasibility of an Evaluation
  • 3.2 - Overview of Background Information Collection
  • 3.3 - Methods for Obtaining Input from Technical Working Group and Others
  • 3.4 - Methodology for Site Visits and Tribal Data Review
  • 3.5 - Methodology for Small Group Discussions
• Chapter 4 - SITE VISIT FINDINGS
  • 4.1 - Overview
  • 4.2 - Availability, Accessibility, and Quality of Data on History of Self Governance and Program Management
  • 4.3 - Availability, Accessibility, and Quality of Data on Health Programs
  • 4.4 - Availability, Accessibility, and Quality of Data on Non-Health DHHS Programs
  • 4.5 - Summary of Availability of Data
• Chapter 5 - DISCUSSION GROUP FINDINGS
• Chapter 6 - EVALUATION FEASIBILITY
  • 6.1 - Overview and Approach
  • 6.2 - Tribal Consultation in Development of Any Future Evaluation
  • 6.3 - Description of Four Illustrative Evaluation Models for Assessment of Feasibility
  • 6.4 - Comprehensive Evaluation Model – Non-Health DHHS Programs
  • 6.5 - Comprehensive Evaluation Model – Health Programs
  • 6.6 - Limited Evaluation Model
  • 6.7 - Evaluation Model Using Aggregate Monitoring and Reporting Data
  • 6.8 - Comparison of Feasibility of the Three Illustrative Models
• Chapter 7 - SUMMARY AND DISCUSSION

• APPENDIX A: Technical Working Group Members
• APPENDIX B: Site Visit Data Collection Forms
  • Qualitative Data Collection Form
  • Quantitative Data Collection Form
• APPENDIX C: Matrix of Issues and Concerns Raised at Discussion Groups at NIHB and DHHS/DOI Self-Governance Conferences
• APPENDIX D: Description of Illustrative Evaluation Models
  1. COMPREHENSIVE EVALUATION MODEL: NEW DHHS PROGRAMS
     1. Objectives
     2. Assumptions
     3. Research Questions to be Examined
     4. Comparison Groups for Quantitative Measures Questions 
     5. Data Necessary for the Evaluation
  2. Comprehensive Evaluation Model: DHHS Health Programs
     1. Objectives
     2. Assumptions
     3. Research Questions to be Examined
     4. Comparison Groups for Quantitative Measures
     5. Data Necessary for the Evaluation
  3. Limited Evaluation Model
     1. Objectives
     2. Assumptions
     3. Research Questions
     4. Comparison Groups for Quantitative Measures
     5. Data Necessary for the Evaluation
  4. Evaluation Model Using Only Monitoring and Reporting Data
     1. Objectives
     2. Assumptions
     3. Research Questions to be Examined
     4. Comparison Groups for Quantitative Measures
     5. Data Necessary for the Evaluation

   

Printable (PDF) Version

Tribal Home Page

Return to the Office of Planning and Policy Support Home Page

[ SEARCH CONTENTS | HHS Home Page | ASPE Homepage | Privacy Policy ]

Website comments? Contact the Webmaster