Feasibility Study for the Evaluation of DHHS Programs Operated under Tribal Self-Governance. 3.2 - Overview of Background Information Collection


During the initial months of this project, a substantial amount of information was assembled by the project team, as background for understanding and laying the groundwork for  the Evaluation Feasibility Study.  Each of these activities and the associated reports that were prepared are described in this section.

Literature Review

The Literature Review was conducted to provide a foundation for the development of the evaluation issues and related data requirements that guided the design of the feasibility study.  The objectives of the literature review included:

  • Identification of data limitations and other factors that pose barriers to conducting comprehensive evaluations of self-governance and Tribal management of health and social service programs.
  • Identification of existing studies and evaluations of Tribal self-governance and/or Tribal management of health and social service programs;
  • Review of the methodologies and data sources used in previous studies, in order to assess both analytic rigor and generalizability of their findings;
  • Synthesis of the available evidence and findings from existing studies; and
  • Assessment of the implications of these findings for the Tribal Self-Governance Evaluation Feasibility Study.

Most of the studies employ qualitative techniques, such as key informant interviews, which relied on stakeholders’ perceptions to reach conclusions about program effectiveness.  These qualitative studies provide insight into how various Tribes structure their health and social service programs, the characteristics of Tribal residents participating in these programs, and successes encountered in program implementation.  Further, these studies – particularly those that focused on the Tribal Temporary Assistance to Needy Families program – effectively highlight how social and economic conditions on Reservations, such as the high rate of poverty, high unemployment rates, and the lack of an economic base, may pose substantial barriers to achieving the intended goals of these programs.  These studies do not, however, provide reliable quantitative evidence on the extent to which and how Tribally-managed health and social service programs have operated to better meet the needs of their members. Because most studies did not incorporate a comparison group in their design, it is not possible to determine how persons participating in Tribal programs fare compared to how they would have fared if control over these programs were still vested with the Federal or State government.

Previous research on process, structure, and impacts of Tribal management of health and social services is limited in major ways:  1) many of the programs that are currently managed by Tribes have not been in existence for a sufficient time to permit an assessment of the longer-term effects and effectiveness of Tribal management; 2) Tribes are unique in cultural, socioeconomic, and geographic circumstances and, as a result, successful program structures and effectiveness may also be unique and not generalizable; and 3) adequate and comparable data across Tribally-managed programs and between Tribally-managed programs and federal and State managed programs are not available.

Given these findings, a primary focus of the Evaluation Feasibility Study was to review and identify potential sources of data that would be adequate to permit a quantitative evaluation of relevant issues. 

Legislative History and Development of Tribal Self-Governance

In addition to the literature review, a summary of the legislative history and development of Tribal self-governance was prepared to provide background and context for understanding the context within which self-governance has evolved and the underlying principles on which the development of Tribal self-governance has been based.

Tribal Matrix of Programs Managed Under Contracts and Compacts

As background for the Evaluation Feasibility Study, OASPE was interested in determining the extent to which Tribes are currently managing DHHS or other federally-funded programs under compacts, contracts, and grants.  These other programs include programs of the Department of the Interior, Bureau of Indian Affairs that relate to certain of the programs that are recommended for inclusion in a DHHS non-IHS demonstration project and programs carried out under the  “477” program (P.L. 102-477).[3] The information on Tribes that are currently managing programs also provided background information for recruiting six Tribes to participate in the site visit component of the Evaluation Feasibility Study.

The construction of the Tribal Matrix and identification of programs that are managed by each Tribe required: 1) identification of each federally-recognized Tribe (including those  Tribes in Alaska that have authorized a tribal organization to carry out programs on their behalf ); and 2) identification of data sources and individuals in the federal government that could provide information on Tribal management of the specific DHHS programs of interest.

A complete list of all federally-recognized Tribes was obtained from the Federal Register[4]. This list was then cross-referenced with Indian Health Service information to match Tribes in Alaska to the tribal organizations they may have authorized to carry out programs on their behalf. 

Project staff searched each DHHS program area web site, as an initial step, to determine whether the program maintained a list of Tribes and Tribal Organizations that hold contracts, grants, or compacts to manage specific programs.  Then, direct telephone contacts were made with staff associated with each program area to verify the accuracy of information obtained from the web site or to request information on Tribal management of programs. For several programs (5), the information required was maintained on the federal agencies’ web sites; information was provided by program staff for the remaining seven DHHS programs.  Data were also obtained from BIA staff on Tribal management of BIA programs under Title I self-determination contracts and under Title IV self-governance compacts, as well as self-governance compacts under P.L. 102- 477 provisions.

Tribal Population Characteristics and Related Data

To provide addition background information for the study, data were compiled from relevant data sources that provided Tribal/Tribal organization-specific information on population, age and gender distribution, socioeconomic characteristics, and any other variables that might be useful for describing and comparing Tribes.  This data compilation was intended to provide information for selecting and describing Tribes for the site visit component of the study, as well as information that could be useful to OASPE if an evaluation was conducted at some future time.

The Data Report provides information on a range of demographic and socioeconomic data that are useful for characterizing Tribes that manage both health and social service programs. Because of this broad interest, the Data Report does not emphasize health data, but instead is a compilation of information on population size, age distribution, economic characteristics, and other general data that may be relevant to Tribal management of many programs. 

Data sources that were used to develop the population and other characteristics, by Tribe, included the 2000 Census, the Bureau of Indian Affairs, and the Indian Health Service. There are serious limitations of these data sources and no good solutions are available to ensure that complete, accurate, and comparable data can be assembled for each federally-recognized Tribe.  Despite these limitations, the Data Report provides some useful information on socio-economic and demographic characteristics of specific Tribes.  The data, however, should be viewed as providing relative indications of differences among Tribes, rather than absolute and accurate data on each Tribe’s characteristics.

Indian Health Service Data Review

            Several IHS staff were interviewed about data available through the Resource and Patient Management System (RPMS).  There were two primary foci in these discussions:  patient-level data and administrative and personnel-related data.  The RPMS is an integrated software system for management of clinical and administrative data in IHS and tribally operated healthcare facilities.  It is composed of several different data collection components.  The Patient Care Component (PCC) comprises data collected at the patient level regarding all care received through the service units and includes a number of client characteristics.  Among the data elements that were investigated and found to be available in some form from the RPMS, and particularly the PCC, from 1998 forward are those listed in the table below.

 Table 1:  Selected data available through the Patient Care Component of the RPMS

Unit of Measurement

Data Available

At the Service Unit level

Number of patients provided services in SU, by age and gender


Number and type of Contract Health Services provided by quarter of the fiscal year


Percent with Medicare


Percent with Medicaid


Percent with SCHIP


Percent with Private Health Insurance


Number of hospital admissions


Number of hospital days


Number of primary care visits


Number of specialist physician visits


Number of dental visits


Number of prescriptions filled




Percent children under age 5 immunized


Percent aged 50+ receiving influenza immunizations


Percent of women over 18 with annual Pap smears


Percent pregnant women obtaining prenatal care in first trimester


Percent of adults screened for diabetes


Percent diagnosed with breast cancer surviving 5 years


Percent diagnosed with cervical cancer surviving 5 years


Percent of births that are low-weight or premature


Percent of births that are high-weight


Percent of deaths attributable to diabetes


For each Service Unit, for all patients with diabetes, three years:

Percent seeing physician at least once in 3 months


Percent receiving HbA1c testing once in 3 months


Percent receiving dilated eye exam annually


Percent of people with diabetes who have diabetic retinopathy


Percent of people with diabetes who have had amputation

The availability of administrative and personnel-related data elements was also investigated.  These elements would include staffing information, information on staff credentials, staff turnover, pharmacy information, and payment information.  Some of this information are likely available through other components of the RPMS (e.g., accounts payable, contract health, staff credentials), if these components are in use by the Tribal entities of interest.  

Review of Reporting Requirements for Other DHHS Programs

Current reporting requirements for each of the DHHS programs were also investigated by talking with several program staff and reviewing the documents provided to us by the Tribes visited.  The current reporting requirements are described below.

Tribal Temporary Assistance for Needy Families (TTANF).     Currently, Tribes managing this program are required to provide the standard Federal financial reporting form SF269[5] and electronic submission (preferred) of family-level and individual-level data elements for families receiving TTANF. (Some Tribes may qualify to sample the caseloads on which they report these data.) For the family, these data elements include funding stream, number of family members, type of family for work participation, receiving subsidized housing, receiving medial assistance, receiving food stamps and amount, receiving subsidized child care and amount, child support, and family cash resources. At the individual level, Tribes are required to submit characteristics such as adult and minor child head-of-household characteristic such as date for birth, ethnicity, gender, receipt of disability benefits, marital status, relationship to head of household, parent with minor child in the family, needs of pregnant women, educational level, citizenship, cooperation with child support, employment status, and work participation status. The child characteristics submitted by TTANF grantees include family affiliation, race/ethnicity, gender, receiving disability benefits, relationship to head of household, educational level, amount of unearned income,

Low Income Home Energy Assistance Program.     Currently, Tribes managing this program are required to provide the Household Service Report—Short Format or a letter containing similar information.  This information includes number of household receiving the following types of assistance: heating, cooling, winter/year round crisis, summer crisis, or weatherization.  Tribes are also required to file the SF269.

Community Services Block Grant.     Currently, there is no specific Federal required reporting form beyond the SF269.

Child Care and Development Fund.     Standard Child Care and Development Fund Annual reporting requires the following information: number of families and children receiving services, age breakdown for children receiving services, reasons for needing childcare (e.g., working, in school), number of hours services provided, amount of CCDF subsidy, amount of parent co-payment, poverty status of families receiving services, and financial reporting (SF269).

Native Employment Works.     Current Federal reporting requirements include the SF269 and a Program Report that includes a narrative section that compares achievements for the year to their plan for the year.  It also summarizes significant barriers to implementation, provides explanations for variances with the plan, and describes actions taken.  Grantees must also summarize plans for unobligated funds.  The Program Report also includes a statistical report that provides the following information: number of clients served characteristics of clients served (e.g., age, sex, TANF recipients), number of clients participating in types of NEW activities and services (e.g., classroom training, on-the-job training, counseling), and number of clients with selected outcomes (e.g., GED, unsubsidized employment).

Head Start.     The standard Head Start reports includes information in the broad categories of children enrolled by demographics, staff information by demographics, information on classes/ groups/ centers, volunteer information, and services provided. Head Start also currently has a requirement for extensive outcome measurement. 

Child Welfare Services.     As reported to us, there are no specific reporting requirements.  Each grantee must report how they are progressing toward their 5-year plan.  Tribes are required to file the SF269 also.

Promoting Safe and Stable Families.     Like other Child Welfare programs, we are aware of no specific reporting requirements beyond reporting concerning progress toward planned activities and the SF269.

Family Violence Prevention: Grants for Battered Women’s Shelters.     Narrative or summary reports generally list the number of clients served and the services provided. Current Federal reporting requirements also include the SF269.

Administration on Aging Grants for Native Americans.    The standard report for AoA includes information on full-time/part-time staff; program resources and expenditures, including sources of income other than grants; unduplicated numbers of people that receive support services, congregate meals, home-delivered meals; total numbers of congregate and home-delivered meals; units of supportive services, legal services, at-home services, ombudsmen services, and others.  In addition, grantees must submit the SF269.

SAMHSA Targeted Capacity Expansion Grants.    Current Federal reporting requirements include the SF269 and a quarterly report and specified GPRA measures.  The quarterly and GPRA reports include the following information: grantee information; staffing information; data including number of new clients, services provided, and individual-level information on the clients as required by GPRA[6]; and narrative information about the project such as challenges and successes over the past quarter.

View full report


"draftreport.pdf" (pdf, 169Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®