A new, periodic survey focused specifically on disability offers significant potential because, in theory, it could be designed in any manner to address disability data limitations and not be hampered by the structure of existing data collection efforts. Attendees at the FAEP and TAG meetings supported the idea of such a survey but also expressed doubts as to its feasibility. There were also different ideas about how such a survey might be structured and what its content should be. In the sections that follow, we discuss a variety of potential features of a national disability survey. These features can be combined in various ways to yield hundreds of different options for conducting a national disability survey. It is not our intent to judge which options or combinations of options would be best for such a survey, but rather to discuss the issues and highlight potential advantages and disadvantages of particular survey features.
There are many different data limitations a national disability survey could be designed to address and the primary purpose of the survey will dictate many of the specific design features of the survey. Potential purposes include:
- Provide a broad range of detailed information about disability
- Collect disability-related information not captured in existing surveys
- Act as the gold standard for general population prevalence and disability measurement
- Ensure periodic and consistent collection of key information on people with disabilities
- Focus on a subgroup of people with disabilities neglected in other surveys
We describe three examples of potential purposes of a national disability survey to illustrate some basic issues that might be encountered in developing the survey. These examples include: a small supplement to capture information not collected in other surveys; a longitudinal survey focused on working-age people with disabilities; and a large, ongoing survey that collects information on a wide variety of topics, similar in some respects to the NHIS-D.
Supplemental survey to capture nonexistent information. A small supplemental survey could be designed to focus on a narrow set of issues for which information is currently not collected in other surveys (for example, on issues related to barriers and supports to community living, disability-related service use, needs, and adequacy of existing programs). The survey could use one of the large national surveys (such as the NHIS or SIPP) for its sampling frame, and then be administered as a supplement separately from the core survey. The primary issues associated with this approach would be deciding on the focus of the survey, determining the parent survey and how the supplement would be incorporated, and obtaining funding for the effort. Of the three examples we discuss, this is likely the most feasible and least costly form of a disability survey.4
Longitudinal survey of working-age people with disabilities. Longitudinal information about working-age people with disabilities was identified as a key area where existing information is lacking. A longitudinal survey effort, similar in nature to the HRS, could be designed to focus on working-age individuals. Developing such a survey would involve considerable effort and require substantial resources to implement. In addition to determining the specific content of the survey, decisions would need to be made regarding whether the sample should include people without disabilities, follow-up intervals and methods to minimize and address attrition, duration of follow-up, and whether the survey could be administered as a supplement to an existing effort (like the HRS) to leverage the existing surveys infrastructure. Longitudinal survey efforts are generally much more complex and costly to implement than cross-sectional ones. Despite the complexity and potential costs associated with this survey, it might still garner support because it addresses a significant gap in existing disability data.
A large, ongoing survey on a wide variety of topics. Another potential purpose of a national disability survey would be to act as the primary and ongoing resource for disability-related information on the United States population. This survey would have a large sample, covering all ages and to the extent possible, all residential settings. It would cover a wide range of disability topics and be conducted on a periodic basis, with question modules being added or modified as disability-related information needs change. The survey would collect information on disability-related topics not addressed in other surveys, but also collect other general information available in existing surveys. The reason for doing the latter would be to ensure that the information is collected consistently over time for a sample defined in a uniform manner to which the survey is administered in a uniform fashion. An effort to develop such a survey would encounter similar issues as those noted for the previous two examples related to content, sampling, incorporation with a parent survey, and funding. In addition, obtaining support for such a survey might be more difficult if it is perceived as duplicating information that is already available from existing data collection efforts. While it might be more efficient to include certain types of disability-related content on a proposed national disability survey, existing efforts might be resistant to modifying their instruments for purposes of reducing duplication or otherwise enhancing the rationale for a new national disability survey.
2. Stand-alone versus Supplement
One of the most basic design decisions to be made with respect to conducting a national disability survey is identifying a target population. The target population for a stand-alone would have its own sampling frame and administration methods. Alternatively, a supplemental survey could build off an existing national data collection effort, such as efforts (for example, topical surveys) outlined in Chapter III.
Stand-alone Survey. Developing a stand-alone survey provides the opportunity to design all aspects of the sampling and administration methods. This is advantageous for a disability survey because some people with disabilities can be left out of existing data collection efforts because survey administration procedures fail to locate sample members with disabilities, adequately accommodate their participation in interviews, or too readily involve the use of proxy respondents when respondents with disabilities are encountered (Ballou and Markesich 2009). Administration methods for a stand-alone disability survey could be tailored to better capture people with disabilities and their experiences as reported by themselves, rather than by proxy interviewees.
A stand-alone survey offers the opportunity to create more extensive screening criteria if there are concerns about adding a significant number of questions to an existing survey for purposes of a supplement, or if the sampling frame of the existing survey is unlikely to identify sufficient numbers of individuals with rare types of disabilities. In a stand-alone survey, a set of disability screener questions could be developed to identify people with a variety of types and levels of disabilities for inclusion. Many surveys on which disability statistics are based contain only a limited set of questions to identify people with disabilities. However, screening potential sample members in a stand-alone survey may be burdensome as a large number of people without disabilities would have to be interviewed and screened out. Hence, the screener imposes some burden on a much larger number of potential respondents and would be costly to use.
To facilitate comparisons and place the findings in context, a sample of people without disabilities could also be included in the survey. If designed to have a longitudinal component (discussed further below), including a sample of people without disabilities would also allow the survey to track people who eventually experience disability or have a condition that is currently in relapse, giving the full perspective on the dynamics of disability.
Supplemental Survey. Adding a topical supplement or survey to an existing national survey might be a less burdensome and costly approach, but also poses significant constraints on the survey design relative to a stand-alone survey. Designing the national disability survey to be a supplement allows the effort to take advantage of the existing sampling frame and administration methods of the parent survey. While efficient, for these reasons this can impose some limitations. Like a stand-alone survey, the supplement would require a set of screener questions to identify a sample of people with disabilities. As noted in Chapter III, the addition of the six-question disability series to more federal surveys provides additional opportunities to use these questions as the screener questions to potential disability supplements. A significant advantage of a supplement over a stand-alone survey is that comparison data for a sample of individuals without disabilities is collected in the parent survey at no additional cost to the disability supplement.
Two data collection efforts serve as examples of national disability surveys; both were designed as supplements to existing national surveys. The NHIS-D, the only national disability survey ever conducted in the United States, was designed as a supplement to the NHIS, deriving its sample through screening questions administered in the NHIS. The NHIS-D has only been conducted once (fielded in 1994 and 1995), and collected cross-sectional information on over 32,000 people with disabilities. Depending on the age of the respondent, the NHIS-D collected information on disabling conditions, functional limitations, impacts of the disability on the family, development, employment, use of services and benefits, transportation and personal assistance needs, housing characteristics, environmental barriers, and participation in social activities.
Another example of this model is the Participation and Activity Limitation Survey (PALS), a national disability survey conducted in Canada. Like the NHIS-D, PALS is not a stand-alone survey as it screens potential participants based on an affirmative response to one of two questions on activity limitations in the Canadian Census.5 An analogous survey in the United States would use ACS respondents with disabilities as its sampling frame. Unlike the one-time NHIS-D effort, PALS is conducted every five years. The most recent version was conducted in 2006 and surveyed approximately 47,500 respondents (Statistics Canada 2007). PALS is a cross-sectional survey and collects detailed information on limitation type and severity, specialized equipment or aids, health care and social services, costs, unmet needs, barriers and accommodation to employment, education, retirement, housing, transportation, leisure, social interaction, discrimination, satisfaction, and stress.
Using an existing survey as a mechanism for identifying the population of interest is a more practical way to conduct a new survey of people with disabilities relative to conducting a stand-alone survey. Two potential candidates from which people with disabilities could be identified are the NHIS and ACS. The NHIS collects detailed information on health and will also include the standard six-question disability series. The NHIS identifies additional health conditions including but not limited to general health status, presence of ADL and IADL limitations, specific health conditions, and use of assistive equipment or personal assistants. Thus, the NHIS identifies more people with disabilities. The NHIS also has the advantage that it may be linked to several administrative data sources including the NDI, Medicare enrollment and claims data, and SSA administrative data.
The ACS has the advantage of size. The 2009 ACS conducted interviews with over 2 million Americans, a size large enough to support state-level estimates as well as estimates for smaller areas such as counties and Census tracts. The NHIS typically has about 87,500 individuals and does not support estimates below the state level. The ACS also samples individuals living in households, noninstitutional group quarters, and institutions, whereas the NHIS does not include those living in institutions. Disadvantages of the ACS relative to the NHIS are that it contains a limited number of disability questions that could be used as screeners, no other information about health and health care, and is not currently linked to administrative data sources. Another challenge is that to our knowledge, the ACS has not been used previously as a sample frame for a supplemental survey and there might be practical or legal impediments to doing so.
3. Population Coverage and Level of Estimates
If a national disability survey were to be developed, a number of decisions would need to be made regarding the specific population(s) to be sampled, which would dictate the population for which the estimates are representative. Below, we describe some of the factors that might be considered in deciding whether to develop a supplement or a stand-alone survey, and different options that could be pursued.
Residential Settings. To be fully nationally representative, a national disability survey should sample all Americans, including people who reside in the community, in group quarters, and in institutions, as well as those who are homeless. Many surveys fail to include people living in institutions, which may omit a large population of people with disabilities and thereby reduce their attractiveness as a potential parent for a supplemental disability survey. Based on ACS data, people living in institutions are 3.7 times more likely to have a disability as people living in noninstitutionalsettings (Brault 2009). For some demographic groups, the percentage of those with disabilities residing in institutions is extremely large. For instance, Stapleton et al. (forthcoming) report that over 23 percent of Black males with disabilities between the ages of 25 and 39 reside in group quarters, including 19 percent in institutions. Studies that have compared estimates of long-term care populations in various settings have found estimates to vary substantially across national survey sources due to a variety of sampling, administration, and methodological issues (Spillmanand Black 2005; 2006). Rates of disability are also high among people who are homeless; 26 percent of homeless people are estimated to have a severe mental illness (SAMHSA 2011). Surveys dedicated specifically to these populations exist (such as surveys of inmates or people in nursing homes), but provide an uneven picture of these populations as the disability measures and topical questions vary across surveys. If consistent information is desired across residential settings, a national disability survey would need to rely either on the ACS sampling frame (the broadest available) or, alternatively, on a stand-alone survey to capture additional groups not covered by the ACS and other surveys, such as homeless people.
Individuals or Families. Another design decision relevant to population coverage relates to whether the unit of observation should be individuals, families, or both. Many of the existing surveys that focus on disability subpopulations typically include limited information about family circumstances or other family members. The large national surveys typically include more extensive information about all or most family members. A national disability survey attached to one of the large national surveys (like the SIPP, NHIS, or ACS) would contain substantial information about both individuals and families. If a stand-alone disability survey were to be designed, a decision would need to be made regarding how much information about the family members of individuals with disabilities would be collected. Such information can be very time and resource-intensive to collect, but might be important to understanding the support system available to respondents with disabilities.
Age Groups. Coverage of all Americans and interest in disability across the life cycle implies that people of all ages could be included in a national disability survey or certain groups could be oversampled, such as the working-age population. FAEP participants identified the current state of disability data as deficient in providing a lifetime perspective of disability. Age and population-specific surveys often provide detailed information, but only for a window of time. In addition, many surveys lack disability measures appropriate for children. Accordingly, it is difficult to find useful data on disability onset and changes in functioning over time. The inclusion of all age groups, and of disability questions that are appropriate for different ages, would be important to a national disability survey intended to provide information on disability across the life cycle.
Specific Subpopulations. Aside from specific age groups (children, working-age, and elderly individuals), many existing disability surveys are fielded to cover specific subpopulations and help to provide information on people for whom little or no data are collected in larger and broader national surveys, primarily due to small sample sizes but also because of other factors. Surveys of subpopulations are an important tool for collecting data on populations that are overlooked or undercounted in other surveys. A national disability survey could be designed to oversample disability subpopulations neglected in other surveys. One group of interest might be people with disabilities who are making certain kinds of transitions. As noted previously, federal and state agency staff believed there is an information gap surrounding key transitions experienced by people with disabilities (for example, from education to employment, changes in residential settings, and entries to and exits from public assistance programs). A broader, longitudinal national disability survey focused on transitions could potentially capture people before and after they belong to a given subpopulation. FAEP respondents also noted a lack of disability-related information for racial and ethnic minorities, people with rare disabling conditions, and employment subgroups, all of which could be oversampled in the design of a national disability survey with appropriate screening questions.
State-Level Estimates. Small sample sizes of people with disabilities in existing national surveys limit the ability to study people with specific disabilities or to produce state or local-level estimates. The inability of many federal surveys to produce state-level estimates was noted as an important limitation of existing data at the TAG meeting. Without state-level data, it is difficult for states to get basic prevalence rates, assess access to and unmet need for services, or understand the impacts of state-level policies and programmatic changes. The BRFSS was noted as a possible vehicle for a national disability survey to support state-level analysis. As noted previously, the ACS is another option for this purpose because of its large sample sizes.
Communities. Thus far, the options discussed have focused on individuals with disabilities as the unit of observation. Another option for a national disability survey would be to have communities be the unit of observation in a manner similar to what was done for the Community Tracking Study conducted by MathematicasCenter for Studying Health Systems Change for the Robert Wood Johnson Foundation. As part of this study, data were collected from 12 metropolitan communities selected randomly from a stratified sample of national communities to be representative of the nation. Data were collected via site visits and interviews with staff of various types of health care institutions and providers, along with formal surveys of both consumer households and physicians. The goal of the project was to track changes in the health care system over time. Because little is known about the array of disability-related supports and barriers at the community level, and because it is problematic to collect such information via a consumer survey alone, a focus on communities, rather than only on individual experiences and perspectives, might be valuable for purposes of gathering information about the community environment, including supports for people with disabilities and how the environment affects outcomes for individual consumers.
4. Frequency of Administration and Longitudinal Data
The frequency of administration and the extent of any longitudinal data collection are key considerations that interact with each other.
Frequency. The NHIS-D was conducted as a onetime disability survey to address unanswered disability-related questions at the time (1994 and 1995). Members of the FAEP and TAG agreed that waiting over 17 years between surveys (the minimum length of time between the NHIS-D and any subsequent national disability survey that might be conducted) is too long. Ideally, a national disability survey would be conducted more regularly (for example every 2-5 years). Regular administration of a core set of questions would permit analyses of changes in the status and experiences of people over time.
As a variant, a scaled-down version of a national disability survey could be administered on a more frequent basis (for example, every three years) with a more comprehensive disability survey administered less frequently (for example, every nine years). The comprehensive survey could take the form of the options described above (that is, have boosted sample sizes to support state or local-level estimates, contain a longer and more detailed battery of questions, and/or include oversamples of subgroups for which little or no data are typically collected, such as those residing in institutional settings or people with specific health conditions).
Longitudinal Data. The inclusion of longitudinal information is a key aspect of a national disability survey and the decision to include longitudinal data can affect decisions regarding survey frequency. FAEP and TAG members were in favor of collecting longitudinal data and thought it necessary to capture the dynamics of disability. FAEP and TAG members were also cognizant of the costs and other barriers to collecting longitudinal information. Although potentially less-expensive mechanisms can be used to capture a longitudinal perspective (for example, linkages with administrative data and the use of retrospective questions), the most direct way is to conduct a longitudinal survey. Panel length and survey frequency must be balanced against financial costs and the risk of sample attrition. The formats of the PSID and HRS, which follow sample members biennially from sampling until death, would be ideal if resources were not an issue. A long-running panel might be conducted with survey administration occurring relatively infrequently (for example, every five years). A less ambitious effort might involve a shorter-term panel, perhaps running for a period of five years with interviews conducted annually, to capture the shorter-term dynamics of disability.
Even in the context of a cross-sectional national disability survey, historical information can be collected through the use of retrospective questions. Basic information such as age, marital status, and employment status at disability onset could easily be collected through the use of such questions. Changes in disability status and associated outcomes occurring between onset and the survey, however, would be more difficult to capture. A sophisticated form of retrospective questions, such as that planned for use in the re-designed SIPP (described in Chapter III) might capture more detailed information.
5. Disability Measures
One of the key concerns mentioned by FAEP and TAG participants was the inconsistency and quality of measures of disability. A national disability survey could ideally address this limitation. The survey should minimally include the standard measure of disability that is common across national surveys. At this point in time, that standard appears to be the six-question series to identify disability in the ACS, AHS, and CPS, which seems likely to be promulgated in future federal surveys. Including the six-question series in addition to a battery of more detailed questions to measure onset, severity, and underlying causes of disability would provide an opportunity to analyze and better understand which individuals with disabilities are being identified--and which are missed--by the standard questions. Given the limitations of the six questions, development of a better understanding of disability measurement could be an important function and contribution of a national disability survey. The findings of such analyses might be used to develop better measures, or to develop cross-walks or weighting mechanisms for use with estimates based on data from surveys that include only the six-question series.
With respect to candidates for disability measures beyond the six-question series, FAEP and TAG participants expressed interest in including questions to identify behavioral, emotional, intellectual, and cognitive health conditions, measures of functioning, difficulty with specific activities, and work-limitations. Questions to identify specific conditions, the role of the environment in determining disability, use of accommodations and supports, and disability onset would also enhance current knowledge on disability. Short general health scales, such as the SF-12 and more detailed SF-36, might also be included to assess how these scales operate in the context of disability, and the extent to which they might be used to measure changes over time if longitudinal data on health and disability were to be collected.
6. Topical Content
Because disability can affect many aspects of life, a national disability survey could potentially cover a wide variety of topics, including health, health care, education, employment, programparticipation, participation an array of social activities, housing, disability services, supports, and quality of life. Many of these topics are covered in existing surveys, some in great detail. However, information is fragmented, with most surveys focused only on certain subgroups or certain topics. For example, the HRS provides detailed information on accommodations, but only for those age 51 or older. A national disability survey provides the opportunity to collect in-depth information on the many topic areas relevant to people with disabilities.
There remain topics for which little or no information is available for people with disabilities. The extensive list of unanswered questions and topic areas where federal and state staff considered information to be inadequate (provided in Chapter II) offers potential areas of focus for a national disability survey. As discussed previously, current surveys do an incomplete job of covering people with disabilities as they make transitions. However, in written responses and during the advisory panel proceedings, very few questions related to transitions surrounding retirement were raised. This suggests that existing surveys, such as the HRS, do an adequate job of capturing these types of transitions. The HRS may serve as a model for surveys capturing other populations during other types of disability-related transitions.
Survey information on the role of the surrounding environment is especially limited. Information related to the type, amount, effectiveness, and cost of supports and modifications at home, at work, and in the community are needed, and might be topics addressed in a national disability survey. Similarly, more detailed information related to environmental barriers to participation in all life activities is needed.
The reasons behind many disability-related outcomes are unknown, while a deeper level of understanding is desired. Federal and state agency respondents cited unanswered questions related to incentives affecting individuals decision to apply or not apply for public assistance programs and successful versus unsuccessful attempts to gain and retain employment. Understanding the motivation behind such decisions may help programs become more effective and efficient.
7. Links to Administrative Data
A national disability survey, regardless of its other features, would significantly benefit from links with administrative data. People with disabilities participate in public programs at high rates; creating linkages with administrative data from federal programs would provide accurate historical information on program participation and usage.
Administrative data from SSA and CMS programs seem the most likely candidates for linkages with national disability survey data, both because people with disabilities participate in SSI, SSDI, Medicare, and Medicaid, and because linkages between survey data and these sources are currently undertaken. If a national disability survey were conducted as a supplement to the NHIS, these linkages, along with linkage to the NDI, would occur essentially automatically through the existing efforts of the NCHS restricted data center, assuming those efforts continue in the future. Linkages with other types of administrative data, including earnings data and data from program administered by the states, would also be useful. Past Census Bureau surveys (the CPS and SIPP) have been linked to IRS earnings data; that might be a reason to consider using a Census survey as the sampling frame for a national disability survey.