Core Performance Indicators for Homeless-Serving Programs Administered by the U.S. Department of Health and Human Services. Types of Performance Data Collected and Prospects for Including Additional Data About Homeless Individuals in Mainstream Programs

09/01/2003

All four of the mainstream programs have well-established data systems which are used to collect data on characteristics of those served, services received, and results of service delivery. These data  along with other special surveys and data sources  are used by the federal agencies overseeing each of these programs for GPRA reporting and to generally monitor program performance. As is discussed below, the four mainstream programs collect minimal (if any) data on the homeless status of program participants and, for the most part, do not place a high priority on collecting additional data concerning the homeless individuals they serve (especially in light of budgetary constraints and many competing demands that agencies face for generating performance data on their programs).

Two of the four mainstream programs  Head Start and the Health Centers Cluster  track numbers of homeless served; Medicaid and SAPT do not track number of homeless served. As shown in Exhibit 5-2, Head Start grantees are required to submit an annual report (known as the Program Information Report, PIR) to the Head Start Bureau that includes data on enrollment levels, child/family characteristics, center staffing and program services, and participant outcomes.(19) As part of the PIR, each center reports the total numbers of children and families served, as well as aggregate number of homeless children and families served during the enrollment year.(20) The other data item submitted (as part of the PIR) pertaining to homelessness is the number of homeless families that acquired housing during the enrollment year.

 

Exhibit 5-2:
Background on Colleciton of Performance Information for Mainstream Programs and
Feasibility of Enhancing Tracking of Homeless Individuals Served
Program Characteristics Head Start Medicaid Substance Abuse Prevention and Treatment Block Grant BPHCs Health Centers Cluster
Data System(s) Used to Collect Performance Data -Program Information Report (PIR) Medicaid Management Information System (MMIS) Treatment Episode Data Set (TEDS) Uniform Data System (UDS)
Currently Tally Number or Percentage of Homeless Served? Yes (homeless estimated at about 2 percent of children enrolled) No (though some states may track homelessness or living situation on their own) No (though some states collect data on living situation at intake through TEDS supplement) Yes  Grantees report actual or estimated number of users known to be homeless through UDS (however, no current estimate available)
Definition of Homeless
  • Though definition is generally left up to local grantees, Bureau has given following guidance: "Homeless Families" include those that live temporarily in shelters, motels, or vehicles and families that move frequently between the homes of relatives or friends. Include all families that had any period of homelessness during the enrollment year.
  • Federal government does not provide a definition for homelessness.
  • States are free to develop their own definition of homelessness and data items collected with regard to homelessness or living situation.
  • The TEDS supplemental data set (not required) includes variable living arrangement, which includes 3 choices: (1) homeless (clients with no fixed address; includes shelters); (2) dependent living (clients living in supervised setting such as a residential institution, halfway house or group home; (3) independent living (clients living alone or with others without supervision)
  • UDS defines homeless individuals as follows: individuals who lack housing, including individuals whose primary residence during the night is a supervised public or private facility that provides temporary living accommodations, and individuals who reside in transitional housing.
  • HCH (only) provides additional breakdown of homeless program users by type of shelter arrangement
Potential Ability to Count Number/Types of Homeless Served No problem (already tracking) Very difficult; would involve costly redesign of data systems at state/local levels Difficult  federal government in negotiations with states on improving data collection No problem (already tracking); though difficult to go beyond basic counts of homeless individuals
Specific Data Collected on Homeless Individuals
  • Number of homeless families served during the enrollment year
  • Number of homeless children served during the enrollment year
  • Number of homeless families that acquired housing during the enrollment year
  • None reported to federal government
  • Some states may include homelessness or living situation in their data systems; but federal government does not track which states collect such data
  • None reported to the federal government
  • As part of the intake process, some states/local health facilities may use TEDS supplemental data set, which asks clients his/her current living arrangement (one option is homeless)
  • Health Centers  Grantees report # of users known to be homeless at some time during reporting period
  • HCH-only  provide counts of homeless users in (1) homeless shelters, (2) transitional, (3) doubling up, (4) street, (5) other, and (6) unknown living situation
  • HCH-only  provide counts of homeless individuals by selected diagnosis
Data Currently Being Collected on Suggested Homeless Core Performance Measures for Homeless-Serving Programs
  • Data collected on one core measure: # of Homeless Individuals Enrolled
  • A second measure collected as part of the PIR  # of homeless families who acquired housing during the enrollment year  is similar to following core measure: #/% of homeless individuals enrolled whose housing condition is upgraded.
  • None
  • States that do include homelessness in their data system may be able to provide # of homeless beneficiaries
  • None
  • States are in various stages in enhancing data systems some are already collecting and reporting on living status, while others are in the process of implementing new or revised information systems to report on living situation.
  • Data collected on one core measure: # of Homeless Individuals Enrolled
Assessment of Feasibility of Collecting Data on Core Measures on Homeless Served by the Program
  • Extremely difficult  except for overall count of number of homeless individuals served, which is already collected. PIR extensively revised in 2001-02 (3 data items related to homeless-ness were added); Head Start official observed, it would be a difficult row to hoe and extremely difficult to go beyond what is currently collected with regard to homelessness
  •  
  • Extremely difficult  homeless are small percentage of those served; very costly to make data system changes (even for slight changes/additions to MMIS).
  • Given current fiscal constraints faced by many states it would be difficult to impose any new reporting burden on states/providers
  • Very Difficult  though would be possible to track numbers of homeless served if states/localities utilize TEDS supplemental data elements; would be difficult to go beyond counts to collect data on other core measures for homeless individuals, though there might be some possibilities for measuring substance abuse use/changes for homeless individuals
  • Very difficult and costly to make change to UDS to capture more than counts of homeless (which the system already captures)
Comment
  • Estimates of homeless served already available
  • Aggregate data submitted by centers limits possibilities for further analysis/breakdowns of characteristics and outcomes for homeless individuals
  • FACES survey may offer opportunity to conduct periodic, focused studies of characteristics, services received and outcomes for homeless children and families served under program.
  • Because of its extreme size, Medicaid is special case
  • Desirable to include indicator of homelessness on individual records  but will be difficult to implement
  • Best option in short term may be to include homeless indicator and additional questions on special surveys and studies periodically conducted on Medicaid.
  • SAMHSA is working collaboratively with states to broker agreements to upgrade information provided on individuals served under the SAPT block grant. Emphasis is on collaboration (not mandating) and negotiation of improved collection of data on individuals receiving treatment funded under SAPT block grant. States are being pushed to collect data on all clients receiving treatment. There is interest in how the clients situation may have changed over time from the point of enrollment to various other points  at discharge, 6 months after discharge, and 12 months after discharge.
  • Health Centers Cluster conducts special studies and surveys, which may provide an opportunity for more in-depth data collection on numbers/types of homeless served, as well as services received by and outcomes for homeless participants

The other mainstream program that collects data on the number of homeless individuals served is the Health Centers Cluster. As part of the Uniform Data System (UDS), all grantees (primarily CHCs) are required to report annually on the number of users known to be homeless at some time during the reporting period. Grantees submit aggregate counts and are permitted to submit estimates of homeless users. In addition, Health Center Cluster grantees with HCH sites (one of the Center Cluster programs) are also required to provide separate annual counts of homeless users in (1) homeless shelters, (2) transitional, (3) doubling up, (4) street, (5) other, and (6) unknown living situation.

There are no separate breakouts or counts of homeless individuals served submitted to the federal government as part of the Medicaid or the SAPT block grant programs data systems. Under Medicaid, states may elect to include homelessness or living situation in their state Medicaid data systems. However, the federal government does not track which states collect data on number of homeless individuals served and the MMIS (the reporting system by which Medicaid providers report on services to Medicaid beneficiaries to the federal government) does not include data elements that would enable providers to report on the number of homeless served.(21)

SAPT grantees use the Treatment Episode Data Set (TEDS) to collect client-level data, including the following core data elements: client identifiers, client characteristics (date of birth, sex, race, etc.), date of admission, types of services client received, source of referral, employment status, substance abuse problem, and frequency of use). While the required portion of the TEDS does not include a data item to track homelessness, CSAT has made available a TEDS supplemental data set (which grantees can elect to use) that does include a variable designed to capture living arrangement at the time of intake. Grantees using the supplemental data set, are provided with three living arrangement choices to capture at the time of enrollment: (1) homeless (clients with no fixed address; includes shelters); (2) dependent living (clients living in supervised setting such as a residential institution, halfway house or group home; and (3) independent living (clients living alone or with others without supervision).

Estimates of the number of homeless served are available for one of the four mainstream programs. In our interviews and review of data, we could obtain a firm estimate of the number of homeless served only from the Head Start program  estimated at about 2 percent of all those children enrolled in Head Start. The data system utilized by BPHCs Health Centers Cluster grantees  the Uniform Data System  includes a field for grantees to report either actual or estimates of the number of homeless individuals served. Although this data field should enable users to estimate roughly the total percentage of users of Health Center Cluster services that are homeless, we were unable to obtain such a current estimate. Estimates of the percentage of homeless served are not available for the Medicaid or SAPT program (i.e., the federal government does not collect this data from states, though such estimates may be available for individual states.(22)

Three of the four mainstream programs (all except Medicaid) provide guidance on the definition of homeless. Explicit guidance on how to define homeless individuals served is provided as part of the automated reporting systems for the Head Start and Health Centers Clusters programs. The annual Performance Information Report (PIR) submitted by Head Start grantees to the federal government provides the following guidance on how to define homelessness: Homeless families include those that live temporarily in shelters, motels, or vehicles and families that move frequently between the homes of relatives or friends. Include all families that had any period of homelessness during the enrollment year. Similarly, the Uniform Data System (UDS) report (submitted annually by each Health Center Cluster grantees) provides guidance on what constitutes a homeless individual: individuals who lack housing, including individuals whose primary residence during the night is a supervised public or private facility that provides temporary living accommodations, and individuals who reside in transitional housing. The TEDS supplemental data set  which may be used by SAPT block grant providers, but is not required  includes living arrangement as a variable that may be collected at the time of intake to SAPT services. As noted earlier, three living arrangement choices are provided (homeless, dependent living, and independent living) and homeless is defined as clients with no fixed address; includes shelters.

With the possible exception of counts of homeless individuals served, the mainstream programs do not collect sufficient information to address the suggested core performance measures (identified in Chapter 4, Exhibit 4-1). Data are being currently collected at the federal level by only two of the mainstream programs  Head Start and the Health Centers Cluster  on any of the suggested core performance measures identified in Chapter 4. Both of these programs can generate an overall count of homeless individuals served that addresses the first suggested performance measure (i.e., number of homeless individuals enrolled). The Health Centers Cluster also collects a second measure as part of the annual Performance Information Report submitted by grantees  the number of homeless families who acquired housing during the enrollment year  that is somewhat similar to another suggested core outcome measures: number/percent of homeless individuals enrolled whose housing condition is upgraded during the past month [or quarter]. The other two mainstream programs  Medicaid and SAPT  do not collect data on homeless status at the federal level  and hence, it is not possible to generate the data needed to address any of the suggested core measures.

Mainstream program GPRA measures are combination of process- and outcome-oriented measures and are not closely aligned with suggested core performance measures for homeless-serving programs. Exhibit 5-3 displays the GPRA measures for the four mainstream programs, along with several of the sources used to collect client level data and report on GPRA measures. Appendix D provides additional details about the GPRA measures and specific targets for key measures. Each of the programs has a set of measures that are tailored to specific goals of the program. All four of the programs include both process and outcome measures, though the Head Start program generally places more emphasis on outcomes for individuals served, while the other three mainstream programs tend to place somewhat more emphasis on process measures. For example, Head Starts includes GPRA measures and targets for improved cognitive skills, improved gross and fine motor skills, improved emergent literacy, numeracy, and language skills, etc.(23) Perhaps of greatest relevance to this study  none of the mainstream programs have GPRA measures that refer to serving homeless individuals or to improving outcomes for homeless individuals. Overall, as they are currently structured, the GPRA measures for the four mainstream programs are not aligned with the suggested performance measures for the homeless-serving programs (shown earlier in Chapter 4).

 

Exhibit 5-3:
Overview of FY 2004 GPRA Measures and Data Sources
Used to Collect Participant and Performance Measurement Data
Program Overview of GPRA Measures Methods Used to Collect Participant and Performance Data
Head Start FY 2004 GRPR performance goals/measures organized as follows:

Goal #1: Enhance childrens growth and development 

  • Children demonstrate improved emergent literacy, numeracy and language skills.
  • Children demonstrate improved general cognitive skills.
  • Children demonstrate improved gross and fine motor skills
  • Children demonstrate improved positive attitudes toward learning.
  • Children demonstrate improved social behavior and emotional well-being.
  • Children demonstrate improved physical health
  • Head Start parents demonstrate improved parenting skills
  • Parents improve their self-concept and emotional well-being.
  • Parents make progress toward their education, literacy and employment goals

Goal #2: Children receive educational services 

  • Programs provide developmentally appropriate educational environments
  • Staff interact with children in a skilled and sensitive manner

Goal #3: Children in Head Start receive health and nutritional services 

  • Children in Head Start receive needed medical, dental and mental health services
  • All Head Start programs submit annual Program Information Report (PIR). PIR includes data about the center, enrollment levels, participant and family characteristics, and limited outcome data on the children and families. PIR data are at aggregate level by center.
  • Family and Child Experiences Survey (FACES) is periodic (in 3-year cycles), longitudinal survey of nationally representative sample of Head Start children and families. FY 1999 baseline FACES survey conducted with 3,200 children and families in 40 programs. Survey gathers individual-level data on cognitive, social, emotional and physical development of Head Start children, as well as well-being of families and quality/ characteristics of Head Start classrooms.
  • PIR, FACES, and other sources used to generate data for GPRA reporting.
Medicaid CMS FY 2004 GPRA performance goals/measures organized as follows:
  • Increase the percentage of Medicaid 2-year old children who are fully immunized (supports DHHS Strategic Goal #1: Reduce the major threats to the health and well-being of Americans and Goal #7: Improve the stability and healthy development of our Nations children and youth)
  • Assist states in conducting Medicaid payment accuracy studies for the purpose of measuring and ultimately reducing Medicaid payment error rates (supports DHHS Strategic Goal #8: Achieve excellence in management practices)
  • Improve health care quality across Medicaid and the State Childrens Health Insurance Program (SCHIP) (supports DHHS Strategic Goal #5: Improve the quality of health care services)
  • Decrease the number of uninsured children by working with states to implement SCHIP and by enrolling children in Medicaid (supports DHHS Strategic Goal #3: Increase the percentage of the Nations children and adults who have access to regular health care services and expand consumer choices)
  • Medicaid Management Information System (MMIS) is main data system states are required to have for collection of beneficiary data.
  • Variety of data sources used for reporting on GPRA measures. For example, Health Plan Employer Data Information Set (HEDIS), the Clinical Assessment and Software Application (CASA), and immunization registries provide standardized measurement of childhood immunization (for the first goal).
SAPT FY 2004 GPRA performance goals/measures organized as follows:
  • Number of clients served (annual targets set)
  • Increase # states/territories voluntarily reporting measures in SAPT Block Grant application
  • Increase % states that express satisfaction with TA provided
  • Increase % of TA events that result in systems, program, or practice changes
  • Increase % of Block Grant applications that include needs assessment data
  • Increase % of states that indicate satisfaction with CSAT customer services throughout the entire Block Grant process
  • Increase % of states reporting satisfaction with CSATs responsiveness to state suggestions on services.
  • SAPT grantees use Treatment Episode Data Set (TEDS) to collect client- level data, including following core data elements: client identifiers, client characteristics (DOB, sex, race, etc.), date of admission, types of services client received, source of referral, employment status, substance abuse problem, and frequency of use.
  • TEDS supplemental data elements include: living arrangement, DSM criteria, sources of income, and health insurance
  • TEDS Discharge Data Set collects types of services at discharge, date of discharge, date of last contact, and reasons for discharge
Health Centers Cluster FY 2004 GPRA performance goals/measures are organized as follows:

Goal #1: Eliminate barriers to care:

  1. Increase utilization  (1) By 2006, establish additional 1,200 new or expanded sites; serve 6.1 million more patients; (2) Increase # of uninsured and underserved persons serve by Health Centers, with emphasis on areas with high % of uninsured children
  2. Increase access point  (1) Increase infrastructure of Health Center program to support an increase in utilization, via new starts, new satellite sites, & expanded sites; (2) Among new grant applicants, increase # of faith-based or CBOs
  3. Focus on target pop.  (1) By 2006, assure access to most vulnerable by serving 16% of nations pop. at or below 200% of poverty; (2) Continue to assure access to preventive and primary care for (a) minority and (b) uninsured individuals

Goal #2: Eliminate health disparities:

  1. Utilization of services  (1) Increase % of diabetic users with up-to-date testing of glycohemoglobin and who have annual dilated eye exam; (2) By 2006, reduce low birth weight rates to 6.53%; (3) Decrease % of births to prenatal care Health Center users to 2500 grams (LBW); (4) Increase % of Health Center women receiving age-appropriate screening for cervical and breast cancer, (5) Increase % of Health Center adults with hypertension who report blood pressure under control

Goal#3: Assure Quality of Care:

  1. Decrease % of Health Center users hospitalized for potentially avoidable conditions
  • Each grantee submits standardized set of tables annually using Uniform Data System (UDS)
  • Grantees submit 9 UDS tables reporting aggregate (not individual-level) data, such as: users of services by age category, race/ethnicity, income category, type of 3rd party insurance source, staffing levels at facilities by type of personnel, and numbers of encounters
  • Grantees aggregate data on most UDS tables for all Cluster programs, except Tables 3 and 4 (demographics), and Table 6 (users and encounters by diagnostic category)
  • BPHC also conducts User Visits Survey of a representative sample of health center users and provider visits  which provides data on individuals served and the care they receive.

Mainstream programs face substantial constraints to making changes to existing data systems to increase tracking of homeless individuals. Our discussions with program administrators suggest that the prospects for going much beyond tracking whether an individual is homeless are not promising  and implementation of additional performance measures such as those suggested in Chapter 4 is likely to be a non-starter with mainstream programs. There are two main hurdles that would need to be overcome to expand tracking of homeless individuals by mainstream programs. First, as large mainstream programs, the homeless typically represent a relatively small proportion of total individuals served  and are not typically a population of primary interest. For example, as noted earlier, in a program such as Head Start, homeless account for no more than about two percent of the total number children enrolled at centers. Capturing additional data on homeless individuals serves competes poorly with other critical information needs faced by these mainstream programs and hence, are likely to be viewed well down on the list of must have data elements that such programs are seeking to obtain. Second, adding new data items to existing systems  especially programs such as Medicaid and Head Start which have with well-established data system  is a costly proposition for federal and state agencies, as well as burdensome for service providers and participants served by these programs. The federal government will likely need to negotiate with state agencies and/or grantees on planned changes to such data systems and may encounter resistance or requests for additional funding even when agreement is reached over the addition of new data or reporting elements. For example, a Medicaid program official we interviewed underscored the difficulties involved  noting that at both the federal and state level, it would be a huge problem to mandate tracking of homeless individuals served. This would mean a redesign of the MMIS, which would be very time consuming and expensive. Given the current budgetary environment (with states/localities facing financial difficulties/crises), this Medicaid official observed that it would be very difficult to impose new reporting requirements on states. A Head Start official noted that the program had conducted a major redesign of its grantee performance reporting system just two years ago (adding three new homeless measures, among many other changes). He noted that this recent revision required considerable time and effort at all levels  and that making changes to the data system used by Head Start so soon after it had been extensively restructured would be a difficult row to hoe.

View full report

Preview
Download

"report.pdf" (pdf, 1.01Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®