Core Performance Indicators for Homeless-Serving Programs Administered by the U.S. Department of Health and Human Services. Main Findings from Interviews with Agency Officials and Document Reviews

09/01/2003

Overall, our interviews with agency officials at the four homeless-serving programs provided: (1) background information about each program, including information about key program components and client flow; (2) principal performance measures; (3) methods used to collect performance data; and (4) program officials views on potential measures that might be incorporated to enhance performance monitoring. In synthesizing the results from our discussions and review of background documentation on each project, we have attempted to the extent possible to provide cross-program comparisons at a fairly detailed level of key program features and, particularly, with respect to performance measurement used by each of the programs. Exhibit 2-1 provides a comparative analysis of some of the key programmatic features of the four homeless-serving programs. Below, we highlight several key findings that emerge from this program comparison.

Program funding, allocation, role of the federal/state governments, and number and types of agencies providing services vary substantially across programs. While all four of the programs serve homeless individuals as their target population, there are substantial cross-program differences that complicate efforts to develop and implement common measures of performance and systems for collecting data across the four programs. Some underlying programmatic differences are highlighted in the exhibit:

  • Authorizing Legislation: Authorizing legislation for two of the four programs comes from the McKinney Act (PATH and HCH). Authorization for RHY dates back nearly three decades to Juvenile Justice and Delinquency Act of 1974, while authorization came only about two years ago (in 2001 by Congressional directive) for the Treatment for Homeless Persons Program.
  • Budget, Funds Allocation, and Matching Requirements: FY 2002 funding runs from $9 million (Treatment for Homeless Persons) to in excess of $100 million ($116 million for HCH). Three of the four programs allocate funds competitively; one of the programs allocates funds to states by formula (PATH). However, even within the three programs using a competitive process to select grantees, the methods for allocating funds and selecting grantees are quite different and quite complex. For example, under RHYs Basic Center Program, 90 percent of funds are allocated to states based on the state population under age 18 in proportion to the national total. Regardless of the size of its youth population, the minimum allocation for a state is $100,000 ($45,000 for territories). Despite this initial allocation to states, the federal government runs a competitive grant process in which service providers (mostly local nonprofits and county agencies, though state agencies may also compete) submit grant applications. The applications are peer reviewed and awarded based on scores. If all the funding is not awarded within a state (i.e., all grant awards within a state do not add up to a states allocation), funds are re-allocated from that state to other states within the states region to fund other grant awards. By comparison, SAMHSA/CSAT issued Guidance for Applicants (GFA) for the Treatment for Homeless Persons Program for the first two rounds of grant awards. Nonprofit agencies submitted proposals and CSAT rated proposals and made awards to agencies based on how their proposals stacked up with others submitted. Finally, across the four programs, requirements for providing matching funds ranged from no match to providing $1 in non-federal funds for every $10 of federal funds to providing $1 in non-federal funds to every $3 of federal funds.
  • Role of Federal versus State Governments: The federal government plays a significant role in all four of the programs  distributing funds to states (PATH) or competing grants and selecting grantees (in the case of the other three programs); providing oversight and collecting performance information; and providing technical assistance. In terms of state involvement, only under the PATH program among the four programs does the state play a significant role  distributing funds to local areas, providing technical assistance to local grantees, monitoring subgrantee performance, and submitting annual performance reports to the federal government.
  • Number and Types of Grantees/Subgrantees: The number of grantees ranges from 50 grantees selected under the Treatment for Homeless Persons Program to about 640 under the three RHY programs. There is some overlap across programs in the types of local agencies receiving grant funds and providing direct services. All four of the programs rely to some extent upon nonprofit community-based organizations to recruit and deliver services (e.g., two-thirds of HCH grantees and about one-fourth of HCH grantees are CBOs). Community mental health centers account for nearly two-thirds of PATH grantees; CHCs represent about half of current HCH grantees; and RHY Basic Center Program grantees include a broad network of (about 400) youth shelters operated by public and nonprofit entities.
Exhibit 2-1:
Key Programmatic Features of Four Homeless-Serving Programs
Program Characteristics Projects for Assistance in Transition from Homelessness (PATH) Health Care for the Homeless (HCH) Program Treatment for Homeless Persons Program Runaway and Homeless Youth (RHY) Program
Authorizing Legislation McKinney Act (Section 521 of PHS) McKinney Act (Section 340 of PHS) Congressional Directive in FY 2001 Appropriations Report Runaway and Homeless Youth Act (part of the Juvenile Justice and Delinquency Act of 1974)
Year Program Started 1991 1988 2001 1974
Budget (FY 2002) $40 Million $116 million $11 million $84 million
How Funds Are Allocated By formula  states receive annual grants of $300,000 to $3 million On competitive basis  every 5 years grantees submit comprehensive application; grant periods are for up to 5 years, though funds allocated annually Funds distributed on competitive basis  via a GFA, providing funds under 3-year cooperative agreements, primarily with CBOs On competitive basis  local organization submit grant applications to federal government
Matching Requirements $1 for every $3 of federal funds No match (but significant state/local contributions  federal funds about 30% of local project funding) No match requirement $1 for every $10 of federal funds
DHHS Administering Agency Homeless Programs Branch (HPB), Center for Mental Health Services (CMHS), Substance Abuse and Mental Health Services Administration (SAMHSA) Health Centers Cluster at Bureau of Primary Health Care (BPHC), Health Resources and Services Administration (HRSA) Center for Substance Abuse Treatment (CSAT), Substance Abuse and Mental Health Services Administration (SAMHSA) Family and Youth Services Bureau (FYSB), Administration for Children and Families (ACF)
Federal Role
  • Distributes funds to states/territories
  • Reviews annual state applications
  • Monitors & evaluates state performance
  • Provides TA to states
  • Competes grants and selects local grantees
  • Provides oversight and technical assistance, and collects data
  • Federal regional offices provide oversight; conduct periodic site visits; and resolve compliance issues
  • Policy Research Associates contracted to provide TA
  • Issues GFAs and selects agencies under cooperative agreements
  • Monitors work under the agreements (including review of progress reports), and provides technical assistance
  • ROW Sciences contracted to provide TA
  • Allocates funds (by formula) to states and territories; administers the competitive grantee process
  • Tracks and monitors grantee performance (e.g., regional offices conduct compliance visits every three years to grantees)
State Role
  • States play significant role
  • Distribute PATH funds to local areas and providers
  • Provide TA to local providers and convene conferences
  • Monitor subgrantee performance; prepare annual report and grant application
  • No formal role for states
  • States are eligible to apply as grantees, but none have to date
  • No formal role for states
  • During 1st round states could apply (Connecticut was one of 17 grantees 1st round grantees selected); after 1st round states no longer eligible to apply
  • No formal role for the states  funding goes directly from the federal government to local grantees
  • States may serve as grantees (e.g., under Basic Center Program, Utah and South Carolina are grantees)
Number and Type of Grantees/ Subgrantees Providing Services
  • 376 agencies received PATH funding through states and territories
  • Subgrantees include community mental health centers (61%), other mental health organizations (11%), social service providers (9%), shelter/temporary housing (5%), HCH programs (3%), and a range of other local agencies (e.g. county agencies)
  • 142 current grantees
  • Grantees include CHCs (50%), public health dept. (18%), hospitals (7%), and other CBOs (25%).
  • Generally, one CHC grantee serves a city/local community (except NYC), but grantees encouraged to partner with other local agencies (~300-400 agencies involved in service delivery)
  • 50 grantees selected through first 3 rounds; receive 3 year grants (1st round, 17; 2nd round, 19; 3rd round, 14 grantees)
  • Grantees include CBOs, county, and city agencies (as well as one state agency selected during 1st round)
~640 grantees across three RHY programs:
  • Basic Center Program (BCP): Funding provided to network of ~400 youth shelters
  • Transitional Living Program (TLP): 114 public and private agencies
  • Street Outreach Program (SOP): 140 private, nonprofit organizations
Target Population Persons with serious mental illnesses who are homeless or at imminent risk of becoming homeless Homeless individuals, especially those unable to obtain medical care and treatment for substance abuse problems Homeless persons with a substance abuse disorder, or co-occurring substance abuse disorder and mental illness/emotional impairment Homeless and runaway youth (some variation across programs): BCP services limited to youth (under 18); TLP services limited to youth (ages 16 to 21)
Key Program Goals
  • Engage homeless participants in mental health services, housing services, and other appropriate services
  • Improve health care status of homeless individuals, with the goal of contributing to housing stability
  • Link substance abuse services with housing programs and other services for homeless persons
  • -Secure and maintain housing for homeless persons with substance abuse problems
Goals vary somewhat across 3 RHY programs:
  • BCP: reunite children (where appropriate) with their families and encourage resolution of interfamily problems; strengthen family relationships and encourage stable living conditions for youth; help youth decide upon future course of action
  • TLP: address the long-term needs of street youth and promote self-sufficiency
  • SOP: identify street youth and bring them in for assessment and services
Main Program Services
  • PATH provides funds for flexible, community-based services, including outreach, screening and diagnostic treatment, community mental health services, case management, alcohol or drug treatment, habilitation and rehabilitation, supportive and supervisory services in residential settings, and referral to other needed services
  • HCH emphasizes multi-disciplinary approach to delivering care to homeless persons, combining aggressive street outreach with integrated systems of primary care, mental health and substance abuse services, case management, and client advocacy
  • Emphasis placed on coordinating efforts with other community health providers and social service agencies
  • Program emphasizes linkages between substance abuse treatment, mental health, primary health, and housing assistance
  • Program services include: services that meet basic needs for food, shelter, and safety; substance abuse treatment; mental health services; street outreach; primary health care; case management; community-based educational & preventive efforts; school-based activities; health education and risk reduction information; access and referrals to STD/TB testing; and linkages with justice system
Services vary by program component:
  • BCP: outreach to bring youth to facility; emergency residential care for up to15 days; help with family reunification; and referrals to other agencies
  • TLP services: residential care for up to 18 months; other services as needed, including counseling; basic skills building; referral to education and training programs; and referrals for mental health, substance abuse, and other medical services
  • SOP: grants to local agencies to conduct street-based outreach

While there is a similar focus on homeless individuals across the four programs, there are differences in terms of the number and types of individuals served, definitions of enrollment, and duration of involvement in services. Not surprisingly, all four programs serve homeless individuals  though programs target different subpopulations of the homeless. The RHY programs target youth (both runaway and homeless), while the other three programs target services primarily on adult populations (though other family members are often also served). The HCH program funds initiatives that serve a broad range of homeless individuals (especially those unable to secure medical care by other means). The PATH and Treatment for Homeless Persons Programs serve a somewhat narrower subgroup of the homeless population than the other programs: the PATH program focuses on homeless individuals with serious mental illness; and the Treatment for Homeless Persons program targets homeless persons who have a substance abuse disorder, or both a diagnosable substance abuse disorder and co-occurring mental illness or emotional impairment.

The definition of enrollment and termination in the programs and duration of involvement in services all vary considerably by program. In a program such as PATH  which is considered to be a funding stream at the local (operational) level  it is often difficult to identify a point at which someone is enrolled or terminates from PATH. In a program such as HCH, a homeless individual becomes a participant when he/she receives clinical services at an HCH site. Length of participation in HCH is highly variable  it could range from a single visit to years of involvement. HCH program grantees would like to become the medical home for each individual until a point at which they are no longer homeless and can connect with another health care provider (to serve as the medical home). Much like any other private practice doctor, there is not generally a point in time in which an individual is terminated  rather, a case file is set up on the individual and at some point they simply do not show up (or may come in only sporadically for services).

Even within a program like RHY  which is composed of three program components  there is considerable variation in what constitutes enrollment and duration of involvement. For example, in RHYs Street Outreach Program (SOP)  a program designed to get youth off the street and into a safe situation (and linked to needed services)  involvement is very brief (often a single contact) and presents little opportunity for collecting information about the individual. In contrast, RHYs Transitional Living Program (TLP) provides residential care for up to 18 months under the program and a broad range of other services to move homeless youth toward self-sufficiency and independent living. RHYs third program component  Basic Center Program (BCP)  offers up to 15 days of emergency residential care, help with family reunification, and other services. Hence, BCPs involvement with homeless youth is longer and more intensive than SOP, but much shorter and less intensive than TLP.

Finally, of the four programs, enrollment in the Treatment for Homeless Persons Program appears to be most clearly defined. Homeless individuals are considered participants when the intake form (part of the Core Client Outcomes form) is completed on the individual (though there is no standardized time or point at which this form is to be completed by program sites). Involvement in the program is extended over a year or longer  with follow-up surveys being conducted with participants at six and 12 months after intake into the program.

While actual numbers of individuals served or participating are difficult (if not impossible) to compare because of varying definitions across programs, the sizes of programs appear quite different. For example, HCH (with 142 grantees nationwide) reports that about 500,000 persons were seen in CY 2000. Under its BCP program component (with a network of about 400 youth shelters nationwide providing services), the RHY program estimates that it helps 80,000 runaway and homeless youth each year. According to figures reported annually by states, the number of homeless individuals with serious mental illness who were PATH clients in FY 2000 was about 64,000 (though as noted earlier, because PATH is regarded as a funding stream rather than a distinct program, it is often difficult to isolate an individual as a participant or being served by PATH). Finally, through the first two rounds of funding, the 36 grantees funded under the Treatment for Homeless Persons Program anticipate serving about 7,700 individuals (over the three-year grant period).

Wide range of program services offered through the four programs. As shown earlier in Exhibit 2-1, despite their many differences, there is a fair degree of convergence in the goals of the four DHHS homeless-serving programs. All four of the programs are aimed at improving prospects for long-term self-sufficiency, promoting housing stability, and reducing the chances that participants will become chronically homeless. Each program has more specific goals that relate to the populations served and the original program intent for example, RHYs BCP component has as one of its goals family reunification (when appropriate); HCH aims to improve health care status of homeless individuals; PATH aims to engage participants in mental health care services and improve mental health status; and the Treatment for Homeless Persons Program aims at engaging participants in substance abuse treatment and reducing/eliminating substance abuse dependency.

Exhibit 2-1 (shown earlier) provides an overview of services delivered through the four programs. Common themes cutting across the programs include emphases on flexibility, providing community-based services, creating linkages across various types of homeless-serving agencies, tailoring services to each individuals needs (through assessment and case management), and providing a continuum of care to help break the cycle of homelessness. For example, the Treatment for Homeless Persons Program emphasizes linkages between substance abuse treatment, mental health, primary health, and housing assistance; HCH emphasizes a multidisciplinary approach to delivering care to homeless persons, combining aggressive street outreach, with integrated systems of primary care, mental health and substance abuse services, case management, and client advocacy. Of the four programs, the RHY program (in part, because it targets youth) provides perhaps the most unique mix of program services  and even within RHY, each program component provides a very distinctive blend of services (e.g., street outreach [the Street Outreach Program] versus emergency residential care [Basic Center Program] versus up to 18 months of residential living [Transitional Living Program]).

The four homeless programs feature substantially different approaches to performance measurement, collection of data, and evaluation. Given the variation in the structure of these four programs, it perhaps comes as no surprise that their approaches to information collection, performance measurement, and evaluation are quite different (see Exhibit 2-2). With respect to GPRA measures, three of the four programs have explicit measures; there are no GPRA measures specific to HCH. GPRA measures apply to the BPHCs Health Centers Cluster of programs as a whole, of which HCH program is part.(10) The measures used for the three other programs range from process to outcome measures. The Treatment for Homeless Persons Program has outcome-oriented GPRA measures, as well as a data collection methodology (featuring intake and follow-up client surveys) designed to provide participant-level data necessary to produce the outcome data needed to meet reporting requirements. For example, the GPRA measures for adults(11) served by the Treatment for Homeless Persons Programs are the percent of service recipients who  (1) have no past month substance abuse; (2) have no or reduced alcohol or illegal drug consequences; (3) are permanently housed in the community; (4) are employed; (5) have no or reduced involvement with the criminal justice system; and (6) have good or improved health and mental health status. In contrast, the measures employed by PATH are process measures (rather than outcome-oriented): (1) percentage of agencies funded providing outreach services; (2) number of persons contacted, (3) of those contacted, percent enrolled in PATH. Of the three main GPRA measures used in the RHY program, just the first one is outcome-oriented: (1) maintain the proportion of youth living in safe and appropriate settings after exiting ACF-funded services; (2) increase the proportion of BCP and TLP youth receiving peer counseling through program services; and (3) increase the proportion of ACF-supported youth programs that are using community networking and outreach activities to strengthen services.

Exhibit 2-2:
Overview of Key GPRA Measures and Methods for Collecting Performance Data of Four Homeless-Serving Programsc
Program Characteristics Projects for Assistance in Transition from Homelessness (PATH) Health Care for the Homeless (HCH) Program Treatment for Homeless Persons Program Runaway and Homeless Youth (RHY) Program
Key GPRA Measures
  • 3 GPRA measures: (1) percentage of agencies funded providing outreach services; (2) number of persons contacted, (3) of those contacted, percent enrolled in PATH
  • No GPRA measures specific to HCH. Measures used are for the Health Center Cluster as a whole
  • HCH is clustered with several programs, including Community Health Centers [CHCs] (accounting for 75 percent of the Clusters budget), Migrant Health Centers, Health Services for Residents of Public Housing, and other community-based health programs
  • GPRA measures for adults: % of service recipients who:  (1) have no past month substance abuse; (2) have no or reduced alcohol or illegal drug consequences; (3) are permanently housed in com-munity; (4) are employed; (5) have no or reduced involvement with criminal justice system; & (6) have good or improved health and mental health status
  • GPRA measures for youth (17 & under): % of service recipients or children of adult service recipients who:  (1) have no past month use of alcohol or illegal drugs; (2) have no or reduced alcohol or illegal drug consequences; (3) are in stable living environments; (4) are attending school; (5) have no or reduced involvement in juvenile justice system; and (6) have good or improved health and mental health status
  • RHY uses combined measures across the BCP and TLC programs
  • 3 GPRA measures: (1) maintain the proportion of youth living in safe and appropriate settings after exiting ACF-funded services; (2) increase proportion of BCP and TLP youth receiving peer counseling through program services; and (3) increase proportion of ACF-supported youth programs that are using community networking and outreach activities to strengthen services
How Participant/ Performance Data Are Collected by the Program
  • States submit 16 tables annually, including  (1) federal PATH funds allocated to states, (2) total FTEs providing PATH supported services, (3) PATH providers by type of organization, (4) state/local matching funds, (5) PATH portion of local provider budgets, (6) PATH clients as a percentage of homeless clients in all services, (7) number of organizations providing PATH services by type of service and funding, (8) number and percent of PATH outreach contacts that eventually become enrolled in services, (9) number and percent of PATH clients by: age, gender, race, principal diagnosis, dual diagnosis, veteran status, clients housing status, and length of time homeless
  • HCH grantees submit 9 tables each year. Data submitted are aggregate (not individual-level)  e.g., users of services by age category, race/ethnicity, income category, type of 3rd party insurance source, staffing levels at facilities by type of personnel, and numbers of encounters
  • Grantees aggregate and report on HCH participants along with all other Cluster programs in all but 3 of the 9 tables
  • Separate breakout for HCH participants provided for Tables 3 and 4 (demographics) and Table 6 (users and encounters by diagnostic category)
  • Grantees are required to complete the Core Client Outcomes form on each participant at intake, 6 months follow-up and one-year follow-up. Grantees are expected to collect 6- and 12-month data on a minimum of 80 percent of all clients in the intake sample
  • CSAT staff generate aggregate data on GPRA measures across all grantees for reporting purposes
  • Grantees report data quarterly using Runaway and Homeless Youth Management Information System (RHYMIS)
  • Because of incomplete/ unreliable data being reported in RHYMIS, program is developing revised streamlined data system (RHYMIS-LITE); revised reporting requirements and MIS are focused on GPRA reporting requirements; data system expected to be operational and provide reliable data in FY 2002
  • Agencies range from large, multi-service agencies with fairly sophisticated data collection capabilities to small single-service agencies just beginning to use MIS technology to track service delivery
Automated Data System Capabilities States send tables annually (with totals for the state) via the Internet (using web-based system); federal office then use some data from tables to generate data on 3 GPRA measure HCH grantees submit standardized set of tables annually using Uniform Data System (UDS) CSAT began using new web-based system in January 2003 that enables grantees to easily submit client-level data directly to CSAT and for CSAT staff to generate data needed on GPRA measures Grantees submit semi-annual (prior to FY 2000 submission was quarterly) reports using RHYMIS; revised, streamlined MIS (dubbed RHYMIS-LITE) is under development
Issues Surrounding Performance Measures
  • GPRA measures are limited to process measures (no outcome measures)
  • PATH is regarded as a funding stream at the local level (often combined with other funds to cover part of staff or service delivery costs)  so, at local level, the program is not always well defined as a separate program and participants may not be formally enrolled in a PATH program
  • Current (voluntary) pilot effort underway by PATH to improve data collection
  • PATH officials are not interested in changing GPRA measures, but are interested in generating additional outcome data
  • Tables do not provide data on whether individuals actually receive treatment or on outcomes
  • Under existing reporting system, grantees aggregate data on participants of HCH with participants in other Cluster-funded programs on most data tables, so it is not possible to report separately on HCH
  • BPHC gathers data and reports for purposes of GPRA on the cluster of programs, rather than on HCH  to introduce separate data collection and reporting for HCH on GPRA measures would represent a substantial change
  • Program already has standardized client intake and follow-up surveys that are directly linked to generating data needed for reporting on GPRA measures
  • Program has well developed and explicit outcome oriented GPRA measures  of the four programs it has the closest link between data collection and GPRA measurement
  • Program has available participant-level data
  • Limited opportunity to collect data and track youth involved in the SOP and BCP components because of short duration of participant involvement in services
  • Recent report indicates RHYMIS data are unreliable because of chronic low levels of grantee reporting (less than 50% of grantees submitted reports for all 4 quarters in FY 99); also many youth served by centers are not counted as admitted to services in RHYMIS because services are funded by non-federal sources.
  • RHY is in the midst of revising program measures & RHYMIS
Program Evaluation Efforts and Links Between Evaluation and GPR Measures
  • SAMHSA required to evaluate PATH every 3 years (most recent is 1999 process evaluation by Westat/ROW Sciences)
  • Evaluation report is geared to address GPRA measurement, but difficult to generate reliable data to address 2 of 3 GPRA measures
  • Last formal evaluation of HCH completed in 1995 (process study by UCLA)
  • BPHC conducts many other studies across Cluster programs, which sometimes include analysis of HCH
  • No linkage between the HCH evaluation and GPRA reporting
  • Grantees must conduct a local evaluation (evaluations not required to use an experimental design  though one grantee is using one)
  • Because the program was initiated only recently (in 2001), no evaluations yet available
  • The federal office funds external evaluations from time to time
  • Evaluations have been process and outcome studies (i.e., no random assignment net impact studies)

As displayed in Exhibit 2-2, how performance data are collected and the quality of the data collected also varies across the four programs. Three of the four programs have states (PATH) or grantees (HCH and RHY) submit aggregate data tables either annually or semi-annually. For example, under the PATH program, states submit 16 tables annually, including  (1) federal PATH funds allocated to the state, (2) total FTEs providing PATH supported services, (3) PATH providers by type of organization, (4) state/local matching funds, (5) PATH portion of local provider budgets, (6) PATH clients as a percentage of homeless clients in all services, (7) number of organizations providing PATH services by type of service and funding, (8) number and percent of PATH outreach contacts that eventually become enrolled in services, and (9) number and percent of PATH clients by: age, gender, race, principal diagnosis, dual diagnosis, veteran status, clients housing status, and length of time homeless. States send these tables via the Internet to the federal program office (HPB/CMHS), which abstracts data from each state to generate figures needed on each of the three GPRA measures. Data provided is aggregate (rather than at the participant-level) for PATH (as is the case for HCH and RHY).

The Treatment for Homeless Program (the newest of the four programs) takes an entirely different approach to collection of data than the other three programs. Each of the grantees collects participant-level data at three points of the clients involvement in the program (using standardized data collection form across all sites, referred to as the Core Client Outcomes form)  at intake, 6 months after intake, and 12 months after intake. This generates the data needed by the program to address the outcome-oriented GPRA measurement (e.g., percent of participants who have no past month substance abuse). In addition, it is possible on an individual participant basis to make comparisons on the various outcome measures between the time of intake and follow-up to determine pre/post change for each participant.

All four of the programs use (or are in the process of developing and implementing) some type of automated database for transmission of performance data to their federal administering agencies. In the case of PATH, HCH, and RHY standardized data tables are produced by each state (PATH) or grantee (HCH and RHY) and submitted via the Internet. CSAT has recently implemented a web-based application, which enables Treatment for Homeless Persons Program grantees to submit participant-level records via the Internet.

There are several other issues with regard to the collection of performance data that affect their appropriateness for GPRA reporting and evaluating program performance:

  • The reliability and quality of the data collected and submitted to federal offices varies by program. For example, RHY is substantially revising and streamlining its data system (RHYMIS) in response to past problems with data completeness and quality. A recent report noted RHYMIS data are unreliable because of chronic low levels of grantee reporting (less than 50% of grantees submitted reports for all four quarters in FY 1999); also, many youth served by RHY centers are not counted as admitted to services in RHYMIS because services are funded by non-federal sources.
  • The HCH program is embedded in the Health Cluster of programs at BPHC (and is a relatively small program despite its in excess of $100 million funding, when compared to the Community Health Centers funding). Annual reports submitted by grantees receiving more than one Health Cluster source of funds combine HCH participants with participants of other Health Cluster programs and, hence, it is not possible (except on several of the tables provided by grantees) to produce disaggregated counts for HCH participants.
  • Sophistication with data collection and reporting varies considerably across and within programs. For example, RHYs BCP program funds about 400 youth shelters. A recent RHY report indicated that funded agencies range from large, multi-service agencies with fairly sophisticated data collection capabilities to small single-service agencies just beginning to use MIS technology to track service delivery.
  • Intensity and duration of participant involvement in programs ranges considerable across and within programs. As noted earlier, short or episodic involvement of participants in programs (such as that of some participants of the RHY and HCH programs) limit opportunities for in-depth collection of data from participants.

Program Use Data for a Variety of Purposes. Data collected by the four programs are used for a broad set of purposes, particularly reporting to Congress and others about the program, budgeting purposes, deciding on how to allocate funds to grantees, and to support evaluations of the programs and technical assistance efforts. Programs do not use the data at this time for performance rewards  though in some cases, the data has an effect on which grantees are funded in future rounds. As described by agency officials, the following are the main ways in which data currently collects are being used:

  • HCH: Data are used mainly to report to the Department and Congress on whom the program serves and the types of services provided. No rewards or sanctions result directly from data collected  though technical assistance may be provided for those with poor performance. Some funding decisions are based, in part, on trends in users over several years time.
  • PATH: The data collected during the grant application process and through the annual reporting system are used to generate cross-state data tables and U.S. totals to analyze program and participant characteristics. These data are used to report to Congress on the program and to generate data on the three GRPA measures. The federal office carefully reviews annual reports and grant applications  providing comments where appropriate to states.
  • Treatment for Homeless Persons Program: Though the program just recently started, data collected will ultimately be used to report to Congress on the performance of the program. No performance bonuses are planned.
  • RHY: Data collected from grantees is used by the Family and Youth Services Bureau (FYSB) to report annually on the program. Data are not used to distribute performance awards; data collected may be used to guide decisions in issuing future grants to existing or past grantees.

View full report

Preview
Download

"report.pdf" (pdf, 1.01Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®