Core Performance Indicators for Homeless-Serving Programs Administered by the U.S. Department of Health and Human Services. Considerations and Constraints on Developing a Common Set of Performance Measures

09/01/2003

Using the material and analyses conducted under earlier study tasks, the focus of this chapter is on offering a set of suggested performance measures that could be common and useful across homeless-serving programs of DHHS. In our presentation and analysis of these measures, we have attempted to differentiate between performance measures that are possible from current reporting approaches and those derived from HADS operations, operations needed to collect and aggregate the data, and quality and uses of the data. In developing these measures, we took into consideration the following important factors:

  • Extent currently collected. Items that are already collected by more programs have the advantages of already being highly regarded and contributing the least resistance for inclusion in a uniform system.
  • Ease of collection. For items not universally collected, the ease at which an item can be collected is of interest. We are concerned with initial costs to establish the collection system as well as ongoing costs.
  • Relationship to outcome and process measures of interest. In some instances, proxy measures for the measures of interest must be used because the proxies are preferable on criteria such as ease of collection and extent currently used.

In proposing a set of core performance measures for the four homeless-serving programs that are the focus of this study, the findings from our earlier review of each program and its current performance measurement system catalogue constraints for development of a common set of performance measures that cut across the programs. Perhaps most important, our earlier analysis of the four homeless-serving programs indicated that there are substantial cross-program differences that complicate efforts to develop similar performance measures and systems for collecting data (see Chapter 2 for more detailed discussion of cross-program differences and for a chart comparing the four homeless-serving programs). For example:

  • Programs target different subpopulations of homeless individuals. For example, the RHY programs target youth (both runaway and homeless), while the other three programs target services primarily on adult populations (though other family members are often also served). While the HCH program funds initiatives that serve a broad range of homeless individuals (especially those unable to secure medical care by other means), the PATH program focuses on homeless individuals with serious mental illness; and the Treatment for Homeless Persons program targets homeless persons who have a substance abuse disorder, or both a diagnosable substance abuse disorder and co-occurring mental illness or emotional impairment.
  • The definition of enrollment and termination in the programs and duration of involvement in services all vary considerably by program. For example, in a program such as PATH  which is considered to be a funding stream at the local operational level  it is often difficult to identify a point at which someone is enrolled or terminates from PATH. In a program such as HCH, a homeless individual becomes a participant when he/she receives clinical services at a HCH site. Length of participation in HCH is highly variable  it could range from a single visit to years of involvement. Finally, of the four programs, enrollment in the Treatment and Homeless Persons Program appears to be most clearly defined. Homeless individuals are considered participants when the intake form (part of the Core Client Outcomes form) is completed on the individual (though there is no standardized time or point at which this form is to be completed at sites).
  • Numbers of homeless individuals served are quite different across the four programs. While actual numbers of individuals served or participating are difficult (if not impossible) to compare because of varying definitions across programs, the sizes of programs appear quite different. For example, HCH (with 142 grantees nationwide) reports that about 500,000 persons were seen in CY 2000. This compares with the RHY program estimates that it helps 80,000 runaway and homeless youth each year and estimates that PATH served (in FY 2000) about 64,000 homeless individuals with serious mental illness.
  • Types of program services vary considerably across programs. Common themes cutting across the programs include emphases on flexibility, providing community-based services, creating linkages across various types of homeless-serving agencies, tailoring services to individuals needs (through assessment and case management), and providing a continuum of care to help break the cycle of homelessness. However, the specific services provided are quite different. For example, the Treatment for Homeless Persons Program emphasizes linkages between substance abuse treatment, mental health, primary health, and housing assistance; HCH emphasizes a multidisciplinary approach to delivering care to homeless persons, combining aggressive street outreach, with integrated systems of primary care, mental health and substance abuse services, case management, and client advocacy. Of the four programs, the RHY program (in part, because it targets youth) provides perhaps the most unique mix of program services  and even within RHY, each program component provides a very distinctive blend of services (e.g., street outreach [the Street Outreach Program] versus emergency residential care [Basic Center Program] versus up to 18 months of residential living [Transitional Living Program]).

Given the variation in the structure of these four programs, it is not surprising that the four homeless-serving programs have adopted quite different approaches to information collection, performance measurement, and evaluation (see Exhibit 2-2 earlier for specific measures used by each program). With respect to GPRA measures, three of the four programs have explicit measures; there are no GPRA measures specific to HCH. GPRA measures apply to the BPHCs Health Centers Cluster of programs as a whole, of which HCH program is part.(15) The measures used for the three other programs include both process and outcome measures. The Treatment for Homeless Persons Program has outcome-oriented GPRA measures, as well as a data collection methodology (featuring intake and follow-up client surveys) designed to provide participant-level data necessary to produce the outcome data needed to meet reporting requirements. RHY and PATH employ mostly process-oriented GRPA measures.

Reliability and quality of data collected and submitted to federal offices varies by program. In addition, intensity and duration of participant involvement in the four homeless-serving programs ranges considerably across and within programs, with implications for performance measurement: short or episodic involvement (such as the involvement in some participants in RHY and HCH programs) limit opportunities for collection of data from participants.

In Chapter 2, we concluded that it would be both a difficult and delicate task to develop a common set of performance measures across the four homeless-serving programs. We noted that the willingness and ability of programs to undertake change (e.g., incorporate new outcome-oriented GPRA measures) is uncertain and the changes in how programs collect data and report on performance would require substantial efforts on the part of agency officials and programs. Hence, in specifying performance measures, it is important to be sensitive to the substantial cross program differences and the constraints that program administrators (at the federal, state, and local levels) face in making changes to how they collect and report on program performance.

View full report

Preview
Download

"report.pdf" (pdf, 1.01Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®